Two Justifications forGame-Based

Assessement:An Introduction to GBA

Thomas E HeinzenWilliam Paterson University

ASSESS 2014 KDD 2014Bloomberg, NYC

August, 2014

This presentation is like speed dating; I use images and bold print so you remember only

what I want you to remember.

Two justifications for GBA in higher eduation that have been hiding in plain

sight:

1. People love to be assessed.We develop our sense of self by comparing what we have to what others have: Social comparisons.

2. College is already a game.Points = G.P.A.

Badges = Diploma

Leaderboard = Relative feedback

Positive emotions (joy) are denied until graduation day.

Two Observations 1. GBA requires us to design more difficult courses. 2. GBA uses chance as motivation (not error).

6

Four Critical Definitions

1. Fun is any intrinsically motivated, engaging activity.

2. Play is the exuberant expenditure of energy in aimless activities.

3. A game is a situation in which participants voluntarily try to overcome unnecessary obstacles (i.e. rules and goals).

4. Gamification is the application of game-based principles to non-game situations. (Players may not know that they are in a game.)

Elements of Game-Based Assessment

1. The assessment must be fun (including “hard fun” and “serious games”) so that we can

a. measure peak performance;

b. conduct comprehensive assessments.

2. The experience of failure must be motivating.

3. The assessment data must be psychometrically sound (reliable, valid, and actionable).

Games are ancient.

Game-play has been adaptive (are based in evolution).

Games are inclusive, cross-cultural, social activities.

Notice the level of spectator engagement in this game of hop scotch.

11

Games are a big part of our present…

In 2011, Cityville went from 0 to 100 million users in 41 days.

Video gaming is a 66 billion dollar industry (double Hollywood); online sales will surpass retail sales this coming year (in 2013). China is the biggest online market.

$8 billion was spent on supportive goods such as cyber seeds for Farmville.

Games are NOT just about boys blowing stuff up:

Sandbox (Minecraft) Building (Civilization, Sim City)

Social Building (Farmville) MMOGs (World of Warcraft)

Puzzles (Portal 2, Angry Birds) Vocabulary (Words With Friends)

12

Games also are a big part of our future because…

“We’re witnessing what amounts to no less than a mass exodus to virtual worlds and online game environments” (Economist Edward Castronova).

The 10,000 Hour Principle has been at work for several years (Ericsson, Krampe, & Tesch-Romer, 1993).

There are 500 million gamers who are “extraordinarily good at something” (Jane McGonigal’s TED talk).

13

Why Now?

• We have a growing base of students for whom playing games is the social norm.

• Games are already connecting people through social media.

• Game design is already being applied in non-game settings.

Drivers who obey the speedenter a lottery funded by drivers who did not.

The scientific foundation is growing.

Ifenthaler, D., Eseryel, D., & Ge X. (2012). Assessment in game-based learning. New York: Springer.Kaya, T. (Nov. 7, 2010). A ‘Stealth Assessment’ Turns to Video Games to Measure Thinking Skills, Chronicle of Higher Education, https://chronicle.com/article/A-Stealth-Assessment-Turns/125276/ Shute, V. & Ventura, M. (2010). Stealth assessment: Measuring and supporting learning in video games. Cambridge, MA: MIT Press.

Game-based research in psychology appears to be

accelerating. Game-related terms: “game,” “game-based,” “gamer,”

“gamification,” but not “gaming” or “game theory.”

1975 2012 Increase factor

Total # articles in PsychInfo

29,538 173,184 5.9

# articles with game-related terms

127 1527 12.3

Increase factor in PsychInfo entries between 1975 and 2012:

(Total: 29,538 to 173,184 vs. Game-Related: 127 to 1,527)

Total Entries Game-Related Entries

5.9

12.3

# References with Game-Related Terms

in Two Databases (1975-2012)

ERIC PsychInfo

516

591

127

1567

1975

2012

More DVs: The TAPAS Guide to Automated Assessments

TAPAS

Time to respond.

Accuracy of responses.

Points earned (weighted based on data).

Attempts (to measure persistence in some games)

Send it all to a spread sheet (automatically)

Item Analysis

Individual Analysis

Group Analysis

A more informative leaderboard strengthens the feedback loop and can

provide a rich information trail.

100 150 200 250 300 350 400 450 5000

10

20

30

40

50

60

70

80

Alexa

Janita

Rosa

Tomas

Henri

Keishan

Ernie

Becco,

Rosario

Bert

Ann

Gus

Leaderboard as of Week 3

Total Points Earned So Far

Total HoursSpent in the

Digital Textbook

Keishan (Week 2)

Keishan (Week 3)

Keishan’s Information Trail

Keishan (Week 1)

Converting to GBA will be like switching from a manual to an automated inventory assessment

(less work; more analysis).

Some candid student assessments after a gamified statistics course through

“fat” points, leveling up, and allowing choice.

(I probably should have been offended.)

Trial and error were good. There was no real help along the way, but there were no consequences.

I like that we got to leave early if we learned all we needed to know for that week.

I liked having someone other than the professor teach me; it was much better. (!)

When, if ever, did you experience something like an “epic win”?

When I began to be able to do the material and solve problems alone.

Whenever I finished a new level I felt that I accomplished it.

Completing the practice tests and doing them correctly.

I inadvertently did several things well because ancillary benefits seem to flow out of good design.

Ancillary benefits flow out of good design.

Allowing high achieving others to teach other students

Rewarding immediate achievement with immediate rewards (leaving early)

Giving them self-paced practiceKeeping their attention for four hoursBecoming more confident in the fairness of the

tests, including when some students still deserved to fail

The assessment process can be natural.

Fun can be an effective assessment tool.

Help Donny!(Testing Beliefs in Myths

About Psychology)

Designed by Sam Richman (User Experience Strategist)

FairLawn, NJ

As Donny’s therapist, indicate what you believe is the best science-based answer.

Note the inappropriate use of theFreud couch by the game designer.

“I’m getting such low grades that I don’t know how I made it this far. My father says I need to work on my self-esteem. Will building my self-esteem help me earn higher grades?”

A) Sorry Donny. Building self-esteem has little to do with improving academic performance; study more.

B) Your father is right, Donny. Raising your self-esteem will usually improve academic achievement.

A B

“While I’m taking tests sometimes I’m unsure of the answers. Should I go with my gut or would it be better to go back over the answers?”

A) Stick with your first hunch; you can almost always trust your intuition.

B) Go over the questions you are worried about and change them if you think another answer might be right.

A B

“Could I be doing so badly in my math classes because I’m left handed? If I am right brained then it would make math courses more difficult, right?”

A) It’s true, people who are right brained usually lean more towards the arts, while you may not be great at math you are probably much more creative than other people if you are right brained.

B) Actually, the brain works in an integrated fashion and which hemisphere is dominant mostly doesn’t matter. You probably just need to study a little more.

A B

“I am sooooo angry, I just want to explode! I hate it. What should I do to get rid of this anger?”

A) Try punching a pillow. It will help you blow off steam and redirect your anger so that you are less likely to actually harm anyone.

B) Try listening to music or going for a walk. Distraction and self-control are the best ways to manage your anger.

A B

Profile 1: Donny is not doing well. His Quality of Life Score = 0.

Profile 2: Donny is doing better.His Quality of Life Score = 5.

Two possible profiles of Donny based on the player’s advice:

Stinkin’ Thinkin’A Critical Thinking Game

1 of 5 Story Boards

The Job Interview(One of seven critical thinking assessments using

everyday examples and classic experiments)Story Board 1.

Stinkin’ Thinkin’A newly graduated psychology major was running late for a job interview.

Stinkin’ Thinkin’So he rushed out to his brand new car,

Stinkin’ Thinkin’…turned the key and guess what?

Stinkin’ Thinkin’The car did not start.

Stinkin’ Thinkin’He felt discouraged...

Stinkin’ Thinkin’…so he started to think.

Stinkin’ Thinkin’How can I get my car to start?

Stinkin’ Thinkin’Before trying to start the car again, he asked a friend to use her mighty psychic powers…

Stinkin’ Thinkin’He also tried slamming the hood.

Stinkin’ Thinkin’…and putting the key into the ignition very, very slowly…

Stinkin’ Thinkin’After doing all these things, he turned the key again, and guess what…

Stinkin’ Thinkin’The car started!

Stinkin’ Thinkin’He felt like he would nail the interview!

Stinkin’ Thinkin’But he still had one burning question:

Stinkin’ Thinkin’What had caused the car to start?

Stinkin’ Thinkin’When you are ready, click on the stop watch and drag each box into the appropriate column on the next screen until you have earned 100%!

Screaming Using psychic powers

Hitting the steering wheel

Slamming the hood

Ways to get the car to

start

The car starts or does not

start

One of these influences caused the car to start

None of these influences caused the car to start

One of these influences may have

caused the car to start.

Being worried about the interview

Having a dead cell phone

We cannot logically

determine why the car

started

Putting the key in very

slowly

START

Screaming Using psychic powers

Hitting the steering wheel

Slamming the hood

Ways to get the car to

start

The car starts or does not

start

One of these influences caused the car to start

None of these influences caused the car to start

One of these influences may have

caused the car to start

Being worried

about the interview

Having a dead cell

phone

We cannot logically

determine why the car

started

Putting the key in very

slowly

Independent Variable(s)

Dependent Variable(s)

Alternative Explanation(s)

Extraneous influence(s)

A Logical Conclusion

An Illogical Conclusion

A more informative leaderboard strengthens the feedback loop and can

provide a rich information trail.

100 150 200 250 300 350 400 450 5000

10

20

30

40

50

60

70

80

Alexa

Janita

Rosa

Tomas

Henri

Keishan

Ernie

Becco,

Rosario

Bert

Ann

Gus

Leaderboard as of Week 3

Total Points Earned So Far

Total HoursSpent in the

Digital Textbook

Keishan (Week 2)

Keishan (Week 3)

Keishan’s Information Trail

Keishan (Week 1)

ReferencesDaniels, V. (1967). Communication, Incentive, and structural variables in interpersonal exchange and negotiation. Journal of Experimental Social Psychology, 3, 47-74.

Day, J. (1838). An inquiry into the self-determining power of the will; or, contingent volition. New Haven, CT: Herrick & Noyes.

Everett, C. C. (1891). Ethics for young people. Boston: Ginn & Company.

Hard, R. (1995). The discourses of Epictetus. ed. With introduction and notes by Christopher Gill. London: Everyman/Dent.

de Holstein, S. (1798). A treatise on the influence of the passions unpon th happiness of individuals and of nations: Illustrated by striking references t the principal events and characters that have distinguished the French revolution. London: George Cawthorne.

Ifenthaler, D., Eseryel, D., & Ge X. (2012). Assessment in game-based learning. New York: Springer.

Ifenthaler, D., Eseryel, D., & Ge X. (2012). Assessment in game-based learning. New York: Springer.

Kennedy, J. L. (1962). The system approach: Organizational Development, 4, 25-52.

Lesley, J. P. (1881). Lecture XII. The destiny of man. In Man’s origin and destiny, sketched from the platform of the physical sciences. London: Trübner & Co.

Lubbock, J., (1894). The use of life. London: Macmillan and Co.

Parkhurst, J. L. (1832). Elements of moral philosophy (2nd ed., rev. and improved). Boston, MA: Perkins & Marvin.

Seddon, K. H. (2005). Epictetus (55-135 CE). Internet Encyclopedia of Philosophy. Retrieved on October 24, 2013, from http://www.iep.utm.edu/epictetu/.

Riis, J. (1890). How the other half lives: Studies among the tenements of New York. New York: Charles Scribner’s Sons.

Schell, J. (2008). The art of game design. Boca Raton, FL: CRC Press.

Thank you.

Questions and comments please…