Researching Spore: A New Approach to Player Testing

A NEW APPROACH TOPLAYER TESTING

RESEARCHING

• “Impact of digital technology on society” at UCSD

• Co-founded B|P in 2001

• 7 people. UX research and Design

• San Francisco

• 2,272 Participants. 188 Research Studies.

photo by andy price visualpause.com

Why we care about games

2007 was like this

shit.

Who cares?

We started the spore research with

one question.

Why are video games designed in this environment…

but researched here?

no idea.

“Focus groups were invented to focus the research, not be the research”

- Marty Neumeier | “The Brand Gap”

So what can we do different?

“Guys... I put my pants on just like the rest of you - one leg at a time. Except, once my pants are on, I make gold records.”

#1 native environment is critical…

But timing is everything

One year. 59 Participants

393 Hours of Gameplay.3 Hour Sessions with 2 Nights Each

who cares about fun?

who doesn’t.

DISCUSSION

here are five things we changed.

1. simulated native environment.

2. multi-user voice chat(AKA “talking” and Think Aloud Protocol)

3. touch screens.

4. metrics(PENS model)

5. delivery

Simulated Native Environment

Live observation room

Multi-user Voice Chat

Gamers love TeamSpeak and keyboard shortcuts,

researchers can too.

Talking to six people is crazy

1

2

3

4 6

7

5

#3. Touch screens

Survey Touchscreen

Game Screen(w/ webcam)

#4. The PENS ModelPlayer Experience of Need Satisfaction

Intrinsic motivational needs?

Doing things for their inherent satisfactions (e.g. games, sports). Play through punishment and boredom.

PENS measures three specific motivational needs:

what sucked

•Bordering on chaos at first

•Restricted by game genre

•Lots of technical problems

•Maintaining think aloud protocol

•Difficult to respond to questions

•Recordings take weeks to render

welcome, now play like

Event Time

Welcome / Consent 4:30pm

Stage 1 start 4:45pm

Survey 1 5:15pm

Survey 2 5:45 pm

Dinner 6:00 pm

Stage 2 start 6:15 pm

Survey 3 6:45 pm

Survey 4 7:15 pm

Review 7:20 pm

3 Hours

(according to our precise time limits)

We Learned… Not Much

But then, something changed.

No more timing structure.

Game Level Test Phase Time Action

Welcome / Consent At 0:0

NDA check, equipment check, Paul intro, think aloud

Stage X Begin Phase At :5

Smiley Survey 1 At :15

Smiley Survey 2 At 1:00

End&Survey At 1:30

Announce: Mouse over + Press A to go to Stage Y

Stage Y At 1:40

Smiley Survey 1 At 1:50

Smiley Survey 2 At 2:25

End&Survey At 2:45

Review Survey At 2:55

3 Hours

…fun went way up

Stage Y

#5. Delivery

Technology

•Video Recording•ZDSoft Game Recorder

•Live Observation•Six Webcams using VNC •Long VGA cables

•Audio •Teamspeak (free)

•ELO Systems Touch Screens

$79

free

$39

free

Summary

1. Native environment is critical, but timing is everything

2. You can do this for dirt cheap3. Touch screens are fancy4. The Dow Jones is scary

the future is remote game research

Nov 7th, in San Francisco

userresearchfriday.com

Questions

1. You can do this for dirt cheap

2. Go to San Francisco for URF

boltpeters.com nate@boltpeters.com