Upload
carol-warren
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
California Educational Research Association (CERA)
San Francisco, CA – November 19, 2009
Gregory K.W.K. Chung
Research Issues in Developing Games for Learning and Assessment
2 / ∞
Overview
• Project overview
• Why study games for learning?
• Tensions along the way
• Some design variables
• Study results
• Conclusion and next steps
2
3 / ∞
Project Overview
• Center for Advanced Technology in Schools (CATS)
• USC Game Innovation Lab
• R&D focused on games and simulations for learning and assessment
• Content focus is pre-algebra (rational numbers, solving equations, functions)
• Target population is underprepared students
• Systematic testing of features (instructional variations, game-based) before full-scale implementation
4 / ∞
Why Study Games for Learning?
• If you build it, they will play (and learn) ...
• Given: Students choose to spend hours playing games
• Idea: Let’s put academic content in games
• Magic: Students will play the game, be engaged in the game, and will learn the stuff
• fait accompli
• Recall scantron (1950s), word processors (1980s), calculators (1980s), OPAC (1980s), Web (1990s) ...
• It’s going to happen with or without R&D, so let’s figure out ways to shape the process
5 / ∞
Why Study Games for Learning?
• Help determine the relationship among:
• Different instructional design variables AND
• Different game design variables AND
• Different types of learning outcomes AND
• Different types of students AND
• Different types of game outcomes
6 / ∞
Tensions: Games for Learning Math
• game <–--> learning
• fun <–--> math
• play time <–--> efficiency
• choose to play <–--> have to play
• “pure” math <–--> “applied” math
• basic skills <---> 21st century skills
• simple tasks <–--> complex tasks
• unobtrusive measures (embedded) <---> obtrusive measures (external)
6
7 / ∞
The R&D Challenge
7
Math outcomes
• Skills
• Conceptual understanding
Game outcomes
• Game level
• Gaminess
Instruction
• Tutorial
• Feedback
Core mechanics
• Must use math
Motivational elements
• Bling
?
8 / ∞
Game Design Variables
• Feedback
• Type
• Timing
• Precision
• Impasse-driven
• In-game Assessment
• Scoring
• Performance sensing
8
• Instruction
• Game mechanics
• Conceptual
• Procedural
• Core mechanics
• Part of game
• Motivation
• Bling
9 / ∞
Outcome Variables
• Math outcomes
• Skills
• Conceptual understanding
• Game outcomes
• Student perception of “gaminess”• Flow
• Game level
9
Prototype Gamelet
11 / ∞
Game Design Requirements
• The Outcome
• Conceptual and computational fluency with rational numbers (fractions)
• The Math
• Idea of “unit” and fractional parts
• Additive operations
• Denominator no. of pieces in 1 unit
• Numerator no. of pieces
• Equivalence
• The Challenge: How to do math without killing the game
12 / ∞
Prototype Game Design
• Genre
• Puzzle—need to figure out how to navigate from start to end points
• Game and Learning Mechanics
• Jumping/bouncing from point to point
• Adding coils to go from point to point
• Only allowed to add pieces of the same fractional size (i.e., common denominator)
• Need to convert among equivalent units (2/2 = 3/3 = 4/4)
13 / ∞
14 / ∞
15 / ∞
Study
17 / ∞
Research Study
• Research Question
• To what extent do different kinds of feedback affect understanding of fractions (i.e., unit), game performance, and perception of game play?
• Design
• 2 conditions that varied feedback
• Gamey: Minimal math instruction
• Mathy: Emphasized math concepts related to unit
18 / ∞
Sample
• Sample
• N = 137
• 9th (30%); 10th (18%), 11th (31%), 12th (15%)
• Amount of weekly game play
• 0hr (21%); 1-2hr (40%); 3-6hr (19%); > 6hr (23%)
• Math achievement
• Self-reported grades: A’s and B’s (55%), C’s (31%), D’s and F’s (13%)
• Math pretest: M = 6.34, SD = 3.39, Min. = 0, Max. = 11
19 / ∞
Measures
• Math outcome
• Pretest, posttest
• Game outcome
• Last level reached, perception of game
• Game process measures
• Time, correct fraction additions, incorrect fraction additions
• Background
20 / ∞
Results
• Did we build a game?
• Did students learn from the game?
• Was there an effect of type of feedback on:
• Learning?
• Game performance?
• Game perception?
Did we build a game?
Yes
23 / ∞
Results
24 / ∞
Results
25 / ∞
Results
Did students learn from the game?
It depends
28 / ∞
Did students learn from the game?
• No overall effects of game play on math posttest scores
• Not surprising—sample was composed of high and low performers
• However, our target group—low math performers—appeared to profit from game play
• Low performers’ posttest scores (M = 3.08, SD = 2.04) were significantly higher than their pretest scores (M = 2.55, SD = 1.22). t (48) = 2.0, p = .05, d = 0.32.
Was there an effect oftype of feedback on learning?
No
Was there an effect oftype of feedback on game
performance?
Yes
33 / ∞
Was there an effect oftype of feedback on game performance?
• Students in the mathy condition (vs. the gamey condition):
• Appear to have gone further in the game (p = .08, d = 0.31)
• Committed more correct additions (p = .003, d = 0.49)
• Committed fewer incorrect additions (p = .007, d = 0.48)
Was there an effect oftype of feedback on game
perception?
Probably
36 / ∞
Was there an effect oftype of feedback on game performance?
• Students in the mathy condition (vs. the gamey condition):
• Perceived the game as more game-like (p = .08)
• Were more willing to use the game as part of school work (p = .06)
• Agreed more with the statement that the game helped them understand math (p = .003, d = 0.54)
37 / ∞
Summary
• Did we build a game? (YES)
• Did students learn from the game? (ONLY LOW PERFORMERS)
• Was there an effect of type of feedback on:
• Learning? (NO)
• Game performance? (YES)
• Game perception? (PROBABLY)
38 / ∞
Conclusion and Next Steps
• Beginning to understand conditions under which “mathification” may not hurt game play
• Speculate that math instruction helped students progress in game
• Impasse-driven instruction
• Results promising for the development of a game that includes math content while preserving game aspect
• Need stronger instructional intervention
• Building tutorial, just-in-time feedback
Backup