Keynote Presentation at SIGGRAPH Asia Symposium on Education
Kobe, Japan November 1st 2015
Abstract: Gamification has been explored recently as a way to promote content delivery in education, yielding promising results. However, little is known regarding how it helps different students experience learning and acquire knowledge.
In this talk I discussed our experiences with gamified engineering courses and the reactions of students to the gamified experience.
By examining student performance and attitude data collected from several years we identified distinct student types.
I described the different student types, according to behavior and explained how gamification can provide for smarter learning by catering to students with different profiles.
2. Joaquim Jorge
Instituto Superior Técnico
Universidade de Lisboa
Visualization and Multimodal Interfaces
@ INESC-ID Lisboa
http://web.ist.utl.pt/jorgej
Research Interests:
Calligraphic Interaction,
Multimodal Interfaces,
Graphical Modeling
3. About ACM
ACM, the Association for Computing Machinery is the world’s
largest educational and scientific computing society, uniting
educators, researchers and professionals to inspire dialogue, share
resources and address the field’s challenges.
ACM strengthens the computing profession’s collective voice
through strong leadership, promotion of the highest standards,
and recognition of technical excellence.
ACM supports the professional growth of its members by
providing opportunities for life-long learning, career development,
and professional networking.
With over 100,000 members from over 100 countries, ACM works
to advance computing as a science and a profession. www.acm.org
4. The Distinguished Speakers Program
is made possible by
For additional information, please visit http://dsp.acm.org/
115. International Conference Papers
1. Barata, G., Gama, S., Jorge, J. A., and Gonçalves, D. J. (2014b). Relating gaming
habits with student performance in a gamified learning experience. In Proceedings
of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play, CHI
PLAY ’14, pages 17–25, New York, NY, USA. ACM.
2. Barata, G., Gama, S., Fonseca, M. J., and Gonçalves, D. (2013a). Improving student
creativity with gamification and virtual worlds. In Proceedings of the First International
Conference on Gameful Design, Research, and Applications, Gamification ’13, pages 95–98,
New York, NY, USA. ACM.
3. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2013e). Improving participation
and learning with gamification. In Proceedings of the First International Conference on
Gameful Design, Research, and Applications, Gamification ’13, pages 10–17, New York,
NY, USA. ACM.
4. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2013d). So fun it hurts - gamifying
an engineering course. In Proceedings of the 15th International Conference on Human-
Computer Interaction, HCII 2013, pages 639–648, Las Vegas, Nevada, USA.
5. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2013b). Engaging engineering
students with gamification. In Proceedings of 5th International Conference on Games and
Virtual Worlds for Serious Applications (VS-GAMES), 2013, pages 1–8.
116. International Conference Papers
1. Barata, G., Gama, S., Jorge, J. A., and Gonçalves, D. J. (2014b). Relating gaming
habits with student performance in a gamified learning experience. In Proceedings
of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play, CHI
PLAY ’14, pages 17–25, New York, NY, USA. ACM.
2. Barata, G., Gama, S., Fonseca, M. J., and Gonçalves, D. (2013a). Improving student
creativity with gamification and virtual worlds. In Proceedings of the First International
Conference on Gameful Design, Research, and Applications, Gamification ’13, pages 95–98,
New York, NY, USA. ACM.
3. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2013e). Improving participation
and learning with gamification. In Proceedings of the First International Conference on
Gameful Design, Research, and Applications, Gamification ’13, pages 10–17, New York,
NY, USA. ACM.
4. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2013d). So fun it hurts - gamifying
an engineering course. In Proceedings of the 15th International Conference on Human-
Computer Interaction, HCII 2013, pages 639–648, Las Vegas, Nevada, USA.
5. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2013b). Engaging engineering
students with gamification. In Proceedings of 5th International Conference on Games and
Virtual Worlds for Serious Applications (VS-GAMES), 2013, pages 1–8.
117. Journal Papers
1. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2015b).
Predicting student profile with performance and gaming
preferences. IEEE Transactions on Learning Technologies,
(submitted) .
2. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2015c).
Studying student differentiation in gamified education: A
long-term study. Computers in Human Behavior, (accepted).
3. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2015a).
Gamification for smarter learning: tales from the
trenches. Smart Learning Environments, 2(1):10.
4. Barata, G., Gama, S., Jorge, J., and Gonçalves, D. (2014a).
Identifying student types in a gamified learning
experience. International Journal of Game-Based Learning
(IJGBL), 4(4):19–36.
Hinweis der Redaktion
Modern Universities started with Bologna in 1088
In Europe we still follow the Prussian Model from the XIX Century (Pictured: University of Heidelberg )
However Education models have not changed much in the last two centuries
Even though people tend to think about MOOCS as the latest advance, THEY FOLLOW the conventional classroom paradigm
All students get the same materials and Educational Content
Except that classroom size is now 10 of 1000s of students (instead on 100s)
This is because for all the adoption of technology the classroom of the XXI century still follows the same approach as those of the XIX century: everybody is learning the same content in lockstep
Indeed University (and then some!) Education follows the Industrial Revolution symbolized by the Ford Assembly Line. You can have the Model T in Any Color as Long as it is Black!
So MOOCS may very well follow the fate of other e-learning initiatives of the past decades
For lack of a clear business model and for their XIX century approach MOOCs will likely join other e-learning experiences – the graveyard of technology-enhanced learning
THIS TALK is not about SOFTWARE TOOLS! Rather we focus on THE learning experience!
And Indeed there is still a LONG WAY TO GO. Because we have been focusing on the wrong things
So in My Talk I will discuss Gamification as a tool to increase student participation
And in the second part of the talk, I will discuss our results to enable adaptation of content delivery
…and become more familiar with ubiquitous technology and especially with games
Education is a great concern of modern society and many efforts have been done to make it more effective and available for everybody. Traditional courses and teaching techniques rely on blackboards, oral lectures, books and written exercises as the main vehicle to transmit knowledge.
We face difficult problems: lack of student motivation and BOREDOM
Which leads to classrooms full of bored or distracted students
This is tough for students and professors alike
The pioneering experiences by Sheldon in 2010 transformed the classroom into a game
HOWEVER, Gamification is NOT (playing games in class) students already do that better
THOSE are Serious Games!
Just enumerate a few: XP, Badges, Rewards, Achievements, Skill Trees…
Gaming is about motivation!
One of the most recent models is Self-determination Theory by Deci and Ryan. Or SDT for short.
SDT is a comprehensive Theory on Human Motivation which answers the question on what characteristics should embody an activity to motivate us to perform it
It builds on three foundations:
Related to our ability at making choices and taking control of our lives and destiny
Comes from the universal need of human beings to socialize and belonging to groups (or tribes) – which makes football so popular
Competence, or feeling empowered to
Perform tasks,
Achieving goals
while become proficient
MCP is a course of the Master program in Information Systems and Computer Engineering at Instituto Superior Técnico
. It is part of the Multimedia Systems specialization area. Instituto Superior Técnico has two campii, Alameda and Taguspark, where this area is a Minor or a Major, respectively. The MCP Course, as part of either one, runs simultaneously at both campii. Pictured is TagusPark outside Lisbon
And this is the other campus, in downtown Lisbon.
We have been running the gamified experience since 2010
We combine classroom (traditional) lectures with online experience
Based on several successful social games, like Foursquare or FarmVille, we decided to embody our course into such a game, where players would have the chance to progress in the game experience as they do on the course.
We picked five of the most used game elements to drive student behavior towards reverting boredom and distractedness: points, levels, leaderboards, challenges and badges.
GOAL: To increase IMMERSION driven by novelty
And ENGAGEMENT by participation in course activities
To engage students in course activities, we reward them with experience points (XP) for almost every action they perform, as a form of instant gratification, which has been previously used with success to motivate college students. Examples include posting something on the forums, reading class slides, attending classes on time, reporting bugs in class material, etc.
Portuguese Grading system is structured from 0-20, where 10 is a passing grade and 20 is max!
The final grade was represented by a value between 0 and 20.
In 2010-2011, instead of grade points, the students engaged in a game-like experience and were awarded Experience Points (XP). The more XP users had, the more they progressed in the gaming experience, advancing into new levels, reached at 1000 point intervals.
Each level corresponds to a single grade point (1 out of 20), and students needed 20000 points to reach the maximum grade.
ALMOST EVERYTHING EXCEPT SNEEZING (yet) IS GRADED
The first motivational factors to be explored by us were peer pressure and competition.
A key component to spur competitiveness is for players to and compare themselves with others, to ascertain who is winning and how many points apart they are from other players.
Our game has a leaderboard that displays enrolled students by row, sorted in descending order by XP. Each row portrays the player’s rank, photo and name, campus, XP, level and # achievements they were awarded by completing challenges.
Each student has an individual page where they can see their progress (see charts on the right)
And how they compare to the outher students (see the XP and Badge histograms Bottom-Left)
Most of our challenges are multi-level.
For each level there is a badge,
players have to perform iterations of the same task to reach the maximum level.
Higher levels have increased difficulty. A an example of such challenges we have “Bookworm” where the user has to read 50% of the lectures’ slides for level 1, 75% for level 2 and 100% for level 3.
Another good example is the “Proficient Tool User” challenge, in which students had to compose one creative artifact using multimedia tools (e.g. Gimp, Audacity) for level 1, two for level 2 and 3 for level 3. Multi-level challenges not only make things more interesting to students but might also provide them with a sense of autonomy.
Multi-level challenges not only make things more interesting to students but also provide them with a sense of autonomy.
IMMERSION and ENGAGEMENT
67 Badges Overall
32 will get you up to Level 20
35 will get you beyond! (EXTRA GRADE!)
different ways to reach max bonus
Choose your preferred path!
We also rewarded cooperation (to increase connectedness)
TREASURE HUNT in teams
15 different levels
Students need to cooperate to achieve the last level
We used this to combat post-midterm break disconectedness
Examples of competition include the above
This was an interesting experience. For students who asked intelligent questions or remarks in class.
Lurkers hated them though! and complained that 10% of the class got 90% of these badges
Skill-tree is an example of self-determination: students can complete tasks at their pace and pick up XP
There are four levels and multiple paths to complete the tree,
Tiles become active when pre-requisites are achieved
When a task is completed, its tile is highlighted in blue aura
Drawing on the design implications from the previous study, posts started to be graded based on a rating from 0 to 4 assigned by faculty, with the aim of promoting quality over quantity. Therefore, a student making poor quality posts would have to make four times more posts, to have the same grade as a student making quality posts.
Students are represented by an avatar that they can use to explore the world. Avatars can be customized with equipment items and handheld objects, which can be unlocked by acquiring certain course badges. Students can also create custom content for the game, such as buildings and equipment items, using tools and techniques introduced in class. Submissions were made via posts and graded by faculty, based on their creativity and technical correctness. The main goal here was to appeal to the students’ needs for self-expression and social recognition (Richter et al., 2015). We wanted to allow them to develop a sense of identity (Crumlish and Malone, 2009), by enabling them to customize their learning experience and to be creative.
Usually, getting started is the harder part. We added a few challenges of least difficulty so that, right at the,
students could collect a few achievements and become more.
For multi-level challenges, we give greater score to level#1, to encourage 1st steps.
For example, “Bookworm” level 1 is worth 100XP, level 2 is 70XP & level 3 is 40XP.
Achievement awards make it as transparent as possible: students know when them or others received 1 achievement and why.
By clicking a student in the leaderboard, we can see the history of all earned achievements, including date, the XP, the challenge and the respective badge.
This allows students to get acquainted with the game, understand what others did it and strive to compete and perform better.
Our system is based on Moodle, a Virtual Learning Environment
We use moodle, since it is free and provides basic functionalities such as support fora, quizzes ad exams as well as learning materials
MANY game-related activities evolve around posts. BUT NOT ALL.
Since Moodle does not support gamified environments (YET)
We developed an application
To extract content from google database and to
Collect MOST activity information (forum posts, quizzes, intermediate grades – teacher assigned – etc.)
Grading is done manually by course instructors – heavier load than a traditional course.
INSTANT GRATIFICATION: Students start to complain if grades are not assigned after a few hours
EVEN THOUGH we told them, at the beginning
that grading would be done by humans
We use googledocs intensively to record activities outside moodle and database-driven applications to collect student activity
The scripts generate the leaderboard and individual pages (HTML5) for each student
As well as badges and computing statistics. We use PHP, Python, HTML5 and Javascript
We deployed the gamified version of the MCP course on the second semester of the 2010-2011 academic year. Data were collected regarding many aspects of the students’ performance and satisfaction, which we compared with those from the year before (2009-2010), in order to assess the impact of gamification.
In 2010-2011, there were still significant gains in the number of downloads of the course’s reference material, of the number of Moodle posts and the number of attended lectures. We will address these changes in the next subsections, along with the feedback we got from students, based on their answers to a satisfaction survey at the beginning, middle and at the end of the course.
COMPARED TO PREVIOUS YEARS Our data show that students do not necessarily perform better, but they perform more first and reply posts on most components, which shows that they participate more and are more proactive.
We got valuable information via questionnaires issued three times during the semester.
Students considered our course to be easier to learn from as compared to other courses.
Students consider that our course allows them to be creative.
A comparison between the two academic years (gamified vs non-gamified) reveals that there was a larger gap between the amounts of posts in the first two months than in any other period. We believe that one of the reasons behind this were the challenges that were issued in lab classes.
The results not only suggest that creative challenges are a powerful weapon to boost student engagement and motivation, but also that they should occur more sparsely over time.
Miho, PLEASE do not read this: These required students to come up with interesting ideas and become authors themselves, by creating funny multimedia contents, such as montages, audio remixes and video parodies. The fact that as few as three challenges, issued during three weeks, managed to get 19% of the total amount of posts, indicates that they were indeed a major contribution to student participation. We also had informal feedback from students complimenting the creativity of the challenges. Another plausible reason seems to be the theoretical challenges, which were responsible for 23% (336 out of 1439) of the total amount of posts. These, in turn, occurred during the first two months.
Students consider that our course allows them to do more of what they like
And that it teaches them useful stuff for the future.
Besides the key findings described above, we also found that students considered our course more motivating and interesting than other non-gamified courses, and that they considered the gamified experiences to have gone well and that they would like to see it applied to other courses.
We studied student activity model and patterns of grade acquisition.
These support a (stable) cluster structure observed in the 3rd and 4th gamified iterations of the course
We use the findings and data collected since the first gamified course to develop a model of student performance and participation in a gamified environment.
This model offers an overview of what we consider to be the four archetypes of our gamified course, with the main goal of providing a clearer understanding of what distinguishes the students of gamified MCP.
We managed to cluster students following grade acquisition patterns and leaderboard rank
This tracks evolution of different students throughout the semester (numbers are days from start of experience)
Colors tag different student types or clusters
Of the 76 students that took the course last year, we could identify 4 types according to grade acquisition patterns
Achievers are go-getters
Regular students tend to budget their efforts towards a desired grade
Half-Hearted start like achievers but lose steam as the semester progresses
Underachievers show the least level of engagement
Data from radar charts was computed by normalizing each measurement to a value between 1 and 5, where 1 represented the lowest performance value and 5 the highest. These values were then averaged on all years in which the cluster was represented. The further away it is from the center of the radar, the better a cluster performed on average in comparison to others, in all years it was observed.
For the performance measures, only the XP earned from the Skill Tree and AvatarWorld (a game within a game) were consistently discriminant over time. However, we kept the other features as they are important evaluation components in the context of our course. It provides a subjective yet organized view on our data.
Here we can see that each cluster shows typical (different) performance profiles
Most of the participation features included in the model were consistent between the two years. The exceptions were the number of badges earned from the Artist achievement and the number of posts made on AvatarWorld, which were only significant in one of the years, but still provide valuable insights.
Were we can also see that levels of activity highlight different levels of participation for each cluster
Self-reported engagement values typical of students for each cluster
We used classification algorithms to train statistical models with data from one instance and predict the type of a student population data from another year, as early as of midterm.
predicting a student type early in the course is key to create smart gamified learning environments that can adapt to different students and cater to different needs,
predicting student types from one year with data from another, with high accuracy, would further support the consistency of our cluster structure.
THIS IS OUR MAIN RESULT SO FAR: By using data mining and pattern classification techniques we were able to predict current year’s performance using the previous year data to train classifiers and a gamer-type questionnaire to match students to clusters.
Initial predictions are not very good, BUT later ones become very effective: After 1/3 semester we can predict >70% accuracy the cluster a student belongs to.
ALSO We can see that the grade accumulation patterns are consistent across two different years for each cluster
Another way to differentiate students comes through self-reported student satisfaction. This was done on the non-gamified course, but findings seem applicable here too.
Here we used data from three questionnaires collected at beggining, mid-term ad end-of semester and could identify three different clusters according to perceived usefulness, ease of use (moodle interface) and perceived quality of contents
In this graph we highlight different (student) cluster matches according to different delivery types (the 4 quadrants in CSCW)
Duffirent students focus on different issues:
And student profiles were repeatable across different years too
We found student profiles across different clusters consistent across different years too
Students must consider gamified tasks and assignments meaningful. If they do not seen any value in them, no game elements can help you.
Provide students with rich feedback via points, badges and progress bars about goals and tasks that are meaningful to the student. This will improve their sense of mastery and competence.
Provide several ways for students to draw their own learning path.
We already mentioned the importance of creating different types of assignments to provide diverse paths for students to choose from. However, it is important that these paths cover varied types of subjects, catering to different needs and preferences. This will provide a more flexible learning environment, more likely to create the conditions for new behaviors to emerge, thus reaching out to more students and making the gamified experience easier to learn from. This may allow different types of students to achieve similar levels of performance through different routes.
Badges, titles and levels provide social recognition which can be motivating.
Provide ways for students to manage their pace
Need to balance competition vs cooperation. HOWEVER, our findings indicate that students FOCUS on competition while they keep asking for more cooperation!
Our findings support this idea. This is where we are going to focus for the next editions of the course
We believe that it is possible to support better gaming experiences through personalization made possible by data-mining and pattern classification, using even more student activity data.
Our research brings us towards adaptive gamified learning environments, which can promptly adapt to students with different preferences and needs.
From Top to Bottom, Left to Right:
Daniel Gonçalves
Gabirle Barata (who is about to defend his PhD thesis on gamification)
Joaquim Jorge
Sandra Gama (she was TA and co-author of several papers on the gamified environment)
Manuel João Fonseca
João Amaral (who worked on the scripts)
We will have an Educational Program in Eurographics 2016 TOO!
Please submit your work and come to EG 2016