CS/INFO 4154 · 2018-09-10 · Nobody Reads and Nobody Listens. 9/12. Discoverability. 9/19. Alpha Testing 1. 9/14. ... The science of video game tutorials 4. Techniques for learnability

CS/INFO 4154:Analytics-driven Game Design

Nobody Reads and Nobody Listens

Class 7:

Box!

Mon Wed Fri

Wed 9/19 @ 10:10am: Alpha Prototype

9/21Alpha Testing 2

9/10Nobody Reads and

Nobody Listens

9/12Discoverability

9/19Alpha Testing 1

9/14Learning Progressions

9/17Playtesting and Bias

Assignment 5: Alpha Prototype one functioning level core game mechanics for this level minimal art/UX integration a list of input keys

Challenge: teach player how to playFor most teams, this is the primary challenge to overcome

The Elder Scrolls IV: Oblivion (2006)

Aground (2017)

Aground (2017)

Aground (2017)

Aground (2017)

Outline1. Setting the stage2. A brief history of tutorial design3. The science of video game tutorials4. Techniques for learnability


5 minutes: pair activity Pick your favorite game (quickly) Discuss: How did you learn how to play this game?

How long did it take to learn how to play? How long did it take to get good at it? Did you use any resources? Did you ignore any resources?


In the beginning, there were arcades

Pacman (1980)

Then there were consoles…

Then there were consoles…

Super Mario Bros. 3 (1998)

Instruction booklets!

Problems with this?

Manuals also common in PC games

Doom (1994)

The rise of in-game tutorials

Legend of Zelda: Ocarina of Time (1998)

Deus Ex: Training mode

Deus Ex (2000)

Plants vs. Zombies: Stencils

Plants vs. Zombies (2009)

What is the trend?

No help A lot of hand-holding

Doom Manual

Doom (1994)

“If Doom was done today” (2011)

Link: https://www.youtube.com/watch?v=C4yIxUOWrtwAuthor: Chubzdoomer







Question #1: Do tutorials matter at all?

Question #2: What is the impact of teaching in context?

Question #3: What is the impact of limiting player freedom?

Question #4: What is the impact of providing help on demand?

5 minute pair debate1. Do tutorials matter at all?2. What is the impact of teaching in context?3. What is the impact of limiting player freedom?4. What is the impact of providing help on demand?

Discussion: Question #1 Do tutorials matter at all?

Discussion: Question #2 What is the impact of teaching in context?

Discussion: Question #3 What is the impact of limiting player freedom?

Discussion: Question #4 What is the impact of providing help on demand?


How to design an experiment?1. Do tutorials matter at all?2. What is the impact of teaching in context?3. What is the impact of limiting player freedom?4. What is the impact of providing help on demand?

Evaluating Tutorial Effectiveness

Refraction Hello Worlds Foldit

Andersen et al. CHI 2012

Refraction

Refraction: Gathering data

13,159 players

Hello Worlds!

Hello Worlds: Gathering data

22,157 players

Foldit

Foldit: Gathering data

9,754 players

Question #1: Tutorial presence

How did adding tutorials compare to no tutorials?





no change no changeno change

Performance (levels completed)

p < 0.001 p > 0.05 p > 0.05

Refraction Hello WorldsFoldit

012345678

0

5

10

15

20

0

2

4

6

8

10

12

Engagement (time played, seconds)

p < 0.001 p > 0.05 p > 0.05


0100200300400500600700

0

200

400

600

800

1000

1200

0

200

400

600

800


no change no change

Question #2: Context sensitivity

Context-sensitive Context-insensitive


How did context-sensitive tutorials compare to context-insensitive tutorials?






Performance (levels completed)

p < 0.001 p > 0.05 p > 0.05


012345678

0

5

10

15

20

0

2

4

6

8

10

12

Engagement (time played, seconds)

p <= 0.014 p > 0.05 p > 0.05


0100200300400500600700

0

200

400

600

800

1000

1200

0

200

400

600

800


no change no change

Question #3: Freedom

Context-SensitiveStenciled


How did freedom-restricting tutorials compare to freedom-granting tutorials?






This is the correct answer.

Question #4: Providing help

No helpHelp button






0

100

200

300

400

500

600Time Played (sec)

0

1

2

3

4

5Levels Completed

Nothing Help Button Nothing Help Button

p = 0.036 p = 0.001

Providing help in Foldit

Providing help in Hello Worlds

0

2

4

6

8

10

12

Levels Completed

0

100

200

300

400

500

600

700

800Time Played (sec)

Nothing Help Button Nothing Help Button

p = 0.434 p = 0.190

Providing help in Refraction

0

5

10

15

20

Levels Completed

Nothing Help Button0

200

400

600

800

1000

1200

Time Played (sec)

Nothing Help Button

p = 0.031 p = 0.013


no change

Question #5: provide hints?

O’Rourke et al. L@S 2014

“We present results from a study of 50,000 students showing that all four hint systems negatively impacted performance compared to a baseline

condition with no hints”

Key Lesson of this Class #2

Nobody reads and nobody listens


Nobody reads and nobody listens*

*unless they absolutely have to


Generating, sharing, and testingmultiple ideas leads to better outcomes

Source: Wikipedia

Counterexample: Minecraft

Minecraft (2011)

Counterexample: Minecraft

Minecraft (2011)


Learnability

the capability of a software product to enable the user to learn how to use it

Mario, revisited

World 1-1

Vox interview: Shigeru Miyamoto

Author: Christophe Haubursinand Joe Posner Link: https://youtu.be/K-NBcP0YUQI?t=141

https://youtu.be/K-NBcP0YUQI?t=141

https://youtu.be/K-NBcP0YUQI?t=141

Advice: design for discoverability

Braid (2008)

Advice: design for discoverability

Braid (2008)

Advice: progress through mechanics

The Company of Myself (2009)

Documents

CS/INFO 4154 · 2018-09-10 · Nobody Reads and Nobody Listens. 9/12. Discoverability. 9/19. Alpha Testing 1. 9/14. ... The science of video game tutorials 4. Techniques for learnability