Embed Size (px)
Citation preview
Using Learning Analytics to support a more scientific approach to
Serious Games: Three ExamplesBaltasar Fernandez-Manjon
[email protected] , @BaltaFM
e-UCM Research Group , www.e-ucm.es
Jornadas eMadrid, 2017, 04/07/2017
Realising an Applied Gaming Eco-System
Can Learning analytics help?- Do games actually works?
- Usually, no full formal evaluation has been carried out- Limited number of users
- Formal evaluation could be as expensive as creatingthe game (or even more expensive)
- Difficult to deploy games in the classroom- Teachers have very little info about what is happening
when a game is being used
xAPI Serious Games application profile
New standard interactions model developed and implemented in Experience API (xAPI) by UCM with ADL (Ángel Serrano et al, 2017).
The model allows tracking of all in-game interactions as xAPI traces(e.g. level started or completed, interactions with NPC or game items, options selected, score increased)
https://www.adlnet.gov/serious-games-cop
Analytics and Game Learning Analytics
Game Learning Analytics (GLA) for Serious Games: - collect, analyze and visualize data from learners’ interactions
Can GLA be systematized?
Realising an Applied Gaming Eco-System
Systematization of Analytics Dashboards
As long as traces follow xAPI format, these analysis do not require further configuration!Also possible to configure game-dependent analysis and visualizations for specific games and game characteristics.
First Aid - CPR validated game• Collaboration with Centro de Tecnologias Educativas de Aragon, Spain
• Identify a cardiac arrest and teach how to do a cardiopulmonary resuscitation to middle and high school students
• Validated game, in 2011, 4 schools with 340 students
Marchiori EJ, Ferrer G, Fernández-Manjón B, Povar Marco J, Suberviola González JF, Giménez Valverde A. Video-game instruction in basic life support maneuvers. Emergencias. 2012;24:433-7.
Available at http://first-aid-game.e-ucm.es
BEACONING GLA Pilot: Experiment description
• 227 students• 1, 2, 3 and 4 year of ESO and 1 year of
BACHILLERATO• From 12 to 17 years old• Only 4 morning sessions
• Game rebuilt with uAdventure
• Included analytics using RAGE trackerbased on xAPI specification
Experimental design
3 Steps:
• Pre-Test
• GameplayxAPI LA
• Post-Test
Learning compared to original experiment
• Original experiment with the game
• Original experiment, control group
• Current experiment(from 8 to 9.8 out of 15)
Lower learning but still significative!
Replicability of Results
Predicting post test score
1.With pre test information + game traces
Greater importance of:- score in pre test- game habits- interactions with game elements
Predicting post test score
2.Only with game traces
Greater importance of:- interactions with game elements- scores in game levels
Long-term goal: predict score solely with in-game actions and, therefore, avoid the pre test.
Case study: Downtown• Serious Game designed and develop to
teach young people with Down Syndrome to move around the city usingthe subway
• Evaluated with 45 people with cognitivedissabilities
• Full Analysis of xAPI GLA info under way
• Audience: People between 15 and 40 y/o withDown syndrom
Case Study: Downtown
• From user requirements to a gamedesign and its observables
• Know more about how and what islearn by people with Down Syndrome
13
Next steps: Cyberbullying
Desing based on research studies
Seminario eMadrid sobre Serious games 2017-02-24 15
Social Networks Risks
Seminario eMadrid sobre Serious games 2017-02-24 16
Implications of the social networks
Experimental design
3 Steps:
• Pre-Test
• GameplayxAPI LA
• Post-Test
The experiment: initial validation
With students from 3 institutes (Madrid, Zaragoza, Teruel)
223 pre-post and gameplays (121 males, 102 females)
First results
Age average
Pre-test valueaverage
Post-test value average14,2
6,385,72
Once validated Ministry of Education is interested in creating a NOOC for teachers training where the game is used
Integrating xAPI LA in games authoring
Previous game engine eAdventure (in Java)• Helps to create educational
point & click adventure games
Platform updated to uAdventure (in Unity)
Full integration of game learning analyticsinto uAdventure authoring tool
No extra effort required to integrate default analytics into uAdventure games!
Game Learning analytics can help us to: create better games and to (formally) validate games
• Moving from pre-post to Learning Analytics based evaluation
• To use games as assessments
Conclusions
22
Thank You!
Gracias!
¿Questions?• Mail: [email protected]
• Twitter: @BaltaFM
• GScholar: https://scholar.google.es/citations?user=eNJxjcwAAAAJ&hl=en&oi=ao
• ResearchGate: www.researchgate.net/profile/Baltasar_Fernandez-Manjon
• Slideshare: http://www.slideshare.net/BaltasarFernandezManjon
