The document discusses developing measurable competencies and skills through videogames or real-world training. It argues that videogames and simulations can specify the goals of education by defining outcomes through gameplay in everyday situations. Traditional instruction and on-the-job experience often aim to retain skills and enable learning transfer, but alternatives like simulations may better support deliberate practice and objective performance measurement.

1. A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training K. Anders Ericsson Department of Psychology Florida State University by

2. One of the crucial issues of training interventions is to be able to specify their goals so one can measure and evaluate objective outcomes. I believe that videogames and simulators can provide us with the tools to specify the goal and targets of education for everyday and professional life.

3. Getting away from paper-and-pencil or multiple-choice tests of the type of Scholastic Aptitude Tests (SAT) Define outcomes with video games and simulations of everyday situations and tasks

4. Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization”

5. Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization” Alternatives , such as Simulations Videogames

7. I. If the desired competency concerned a domain, like Chess, How is Expert Performance Attained How did chess players attain their performance? Benjamin Franklin learned to play chess from an acquaintance in 1733 He was one of the first chess players in America. Although he was an avid player he reached a modest level of chess skill

8. Our Knowledge about Learning and Skilled Performance is Limited How good is a chess player or an individual in a domain of expertise? -- Self ratings

9. Professors Drivers Doctors (94% above average) (80% above average) (Davies et al., 2006) Self-Assessments on Relative Performance How would you rate your own performance? bottom 10% below average average above average top 10%

10. Professors Drivers Doctors (94% above average) (80% above average) (Davies et al., 2006) Self-Assessments on Relative Performance How would you rate your own performance? bottom 10% below average average above average top 10%

11. How could a chess player in the chess club be able to keep improving? Playing games Watching better players play Reading books Improving One’s Chess Performance or Performance in Tennis, Golf or SCRABBLE

12. Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation

13. Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation

14. Effects and Dose-Response Relations to Practice Activities Professional teachers and coaches * Monitor students’ development * design training tasks for individual students Expert Performance

15. Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation

16. Simulated Play Against World Class Players Study published games by chess masters Make predictions for each next move Check if your prediction was correct, if not, study the chess position until you understand why the correct move was played Black on move Position B

18. On-the-job supervised experience Classroom Knowledge Acquisition Measurable Outcomes Part II. Objective Measurement of Performance Independent ability to increase patients’ health Kirkpatrick’s Four Level Model of Training Evaluation

19. The most respected experts (“expensive” stockbrokers) are not markedly better in picking stocks on New York Stock Exchange than a random process, such as a monkey throwing darts (c.f. Wall Street Journal study)

20. Superior Performance Chess winning chess games 50 games ( 100-250 hours ) for a rating Psychotherapy superior ability to increase mental patients’ health (post-pre) 50-100 patients (400-800 hours) Teaching superior ability to increase students’ performance (post - pre) 500-1000 contact hours

21. Minimal Effects of Traditional Indicators of Expertise Long Education Extended experience as a professional Teaching Psychotherapy

22. Identify challenging and difficult situations, where experts are supposed to excel. Adriaan de Groot (1914–2006)

23. Recreate the situation and task in laboratory Actions and thoughts of novices and experts can be directly compared

24. Select the best chess move for this position Type as much of the presented text as possible within one minute Play the same piece of music twice in same manner Chess Typing Music Domain Presented Information Task Selection Speed Control

25. Reproducible Soccer Performance

26. From cover story in Time Magazine (March 10, 2008) on The Science of Experience Response to Critical Unexpected Problems

27. Salchow Part III. Mechanisms, Skills, and their Acquisition through Deliberate Practice

28. The problem with learning during work and play Andy Murray (Scotland) in 2010

29. The problem with learning during work and play

30. Immediacy of Informative Feedback Sport training  performance (1-2 years) Investing  value (10-20 years) Medicine  patient outcomes (12-36 months)

31. Increase in Complexity and Control as a Function of Years of Piano Training Years of piano training

32. “ individualized training activities especially designed by a coach or teacher to improve specific aspects of an individual's performance through repetition and successive refinement. To receive maximal benefit from feedback, individuals have to monitor their training with full concentration, which is effortful and limits the duration of daily training”. (Ericson & Lehmann, 1996, pp. 278-279) Deliberate Practice

33. Design and Sequencing of Training Activities Professional teachers and coaches * Monitor students’ development * design training tasks for individual students Expert Performance

34. Accumulated Amount of Practice Alone During the Development of Amateurs and Expert Musicians

35. The Body’s Resilience to Change The Protection of Equilibrium

36. Increase of Fitness

37. Deliberate Practice Optimal

38. How many consecutive push-ups can someone perform without pause? (H. H. Hart, 1974) Range of modifiability with extended Deliberate Practice

39. Charles Linster Non-stop push-up records http://www.recordholders.org/en/list/ulysses.html 6,006 Charles Linster (USA) 05-Oct-1965 … 10,507 Minoru Yoshida (JAP) Oct-1980 Fu Bingli, a kung fu master from China 12 press ups with right finger http://www.telegraph.co.uk/news/newstopics/howaboutthat/6385592/Man-proves-he-has-worlds-strongest-fingers.html# Accessed on October 23 rd , 2009

40. Part IV. Acquisition of Control and Complex Mental Representations—Beyond Automaticity

41. The Development of Increased Control Actively Avoiding Automating Control of Critical Aspects of Performance

42. Reproducibly Superior Performance is not automatic but is associated with informative verbalizations of thoughts

43. Black on move Position B * What aspect was overlooked * When could this aspect have been discovered * How to avoid similar mistakes in the future * Develop new skills by deliberate practice Learning from Making an Incorrect Move

44. Alekhine beat most of the 30 skilled players while playing them simultaneously under blindfold conditions Ref P16 A http://images.google.com/imgres?imgurl=http://www.chessbase.com/news/2007/bilbao04.jpg&imgrefurl=http://www.chessbase.com/newsdetail.asp%3Fnewsid%3D4189&h=318&w=480&sz=36&hl = en&start=1&usg=__JwOiqXRmMEJcHCssYGF1N_5BqU4=&tbnid=DC9VzEbhAUofSM:&tbnh=85&tbnw=129&prev=/images%3Fq%3Dblindfold%2Bchess%26gbv%3D2%26hl%3Den%26sa%3DG Accessed on August 26, 2008 Blindfold Chess Exceptional abilities

45. The Importance of Acquired Mental Representations

46. Expert Performers are Better Able to Represent and Analyze Situations Slowing Down (Moulton et al.,2010) (Ward & Williams, 2003) (Tuffiash, Roring, & Ericsson, 2007) (Kesting, et al., 2010)

47. V. Training and the Fidelity Assumption It is frequently assumed that the best learning of skills occurs in the natural environment . Hence, simulation should therefore reproduce the experience in the natural environment What do we know about natural learning? How optimal is that type of natural learning?

48. Perceptual Performance on Cardiac Auscultation Instruction and Experience 0-9 years 10-20 years Over 20 years General Practitioners Student Based on Butterworth & Reppert (1960) Reviews (Choudhry, Fletcher, & Soumerai, 2005; Ericsson, 2004; Ericsson, Whyte, & Ward, 2007) Performance No Correlations between Amount of Experience and Performance after the First Year

49. Benefits of Provided Experience on Performance Use of Cockpits with Visual and Kinesthetic Simulation Modest transfer of performance— savings of training in real airplanes (except new procedures) (Allerton, 2000; Roessingh, 2005; Rantanen & Talleur, 2005) Focus on Fidelity and Safety A airline captain needs 3,500 hours of flying plus 60 hours of simulation (Parker, Johns, & Hellige, 2007)

50. Identifying the Crucial Elements of Effective Simulator Training - Deliberate Practice A review of 109 studies of high-fidelity medical simulation showed that the characteristics of Deliberate Practice accounted for improvements (Issenberg et al., 2005) A review of 31 of those studies that examined repetitive practice, and “ found a strong association ( η2 = .46 ) between hours of practice on high-fidelity medical simulators and standardized learning outcomes .” (Issenberg et al., 2006, p. 792)

51. The Use of Simulators for Deliberate Practice with Performers at Different Levels of Expertise Allows exposure to challenging situations without danger Allows exposure to difficult situations under conditions optimal to learning and performance - Individuals being focused and ready - Immediate feedback (and opportunity for repetition) - Presentation of related cases to facilitate discrimination Allows the presentation of rare emergency situations

52. Deliberate Practice with Simulators Authentic test conditions with actors (Kneebone et al., 2005) Reinstating actual scenarios in anesthesiology (Liu et al, 2009)

53. Measurement and Training Capturing the essence of expert performance Measuring it with representative tasks Measuring Identification current level of  of optimal performance training tasks

54. Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization” Alternatives , such as Simulations Videogames

55. Recommendations Identify skills and analyze domains of activity where performance can be measured with objective methods that capture on-the-job performance with large individual differences in attained performance where increases in performance motivate major investment Develop libraries of representative situations with appropriate feed-back about correct/appropriate actions with scaled difficulty to maximize optimal training effects Develop cognitive structures to support Deliberate Practice assessment of representations for thinking, planning, and evaluations in Long-term working-memory (LTWM) essential for “Retain[ing] skills and knowledge” “ learning transfer” “ generalization”

56. The Complex Process of Acquisition of of Expert Performance