Waddington g4h 2012 final

June 2012 www.gamesforhealth.org

When “Real” Users Strike Back Post-mortem/Process/Evaluation

Bob Waddington SimQuest, LLC.

[email protected]

@waddingtondc

June 12-14, 2013 Boston, MA

mailto:[email protected]



What I’m Playing…


Sim-Game Based Training Systems for Scene and Patient Management Following Blast Injury from Explosives

Including Improvised Explosive Device (IED)

Supported by the US Army Medical Research and Materiel Command Award No. W81XWH-09-C-0060.

Tom Reeves, PhD, UGA

http://www.amazon.com/gp/reader/041599800X/ref=sib_dp_pt/181-9995832-2490250

http://virtualheroes.com/

http://www.humansim.com/

http://www.ara.com/


When “Real” Users Strike Back

“The feedback in the program is poor.”

“Terrible mouse movement.”

“… the ‘hand’…”

“Improve instructions on how to use it.”

“More carnage. More range of injuries.”


the Challenge

Common errors made by responders • Enter scene too early or get too close with

ambulance or personnel before scene is cleared • Gain access with no egress (blocked exits) • Don’t triage; just treat and transport 1st person • Risk casualties by moving them to unsafe

triage/treatment areas • Improperly assess and/or treat casualties


the Challenge

Blast events introduce other challenges • Scene safety, triage and treatment require

different approaches to scene and patient management

• Additional risks to responding personnel from secondary explosion or shooters

• Different injury patterns • Visual triage is not enough


the Challenge

Training • Large scales exercises are expensive • Any sized exercise is logistically challenging • Current training is mostly delivered through

lecture and power point – some work on manikins


the Solution

Utilizing 1st person gaming technology • Develop self-contained blast-specific modules

that augment existing combat medic tactical combat casualty care (TCCC)

• Provide lessons learned from bomb attacks in Iraq, Afghanistan, Madrid, London, etc.

• Use real-world blast scene injury data

• Provide safe individual/independent training


Project Description

A first-person blast response game that allows

users to respond to a terrorist bombing outside a

busy train station.

Users are assessed on their ability to

• quickly and accurately assess victims’ injuries

and tag them for treatment

• assess the scene and identify additional risks prior to declaring the scene safe.




Evaluation Strategy

Evaluate Overall

• game design

• usability

• content

• user choices

• user path through scenario


Formative Evaluation

• Multidisciplinary Reviews

• End-user Reviews

• Players Feedback

• Formal Study (IRB)

• Implement findings into design and game


Iterative reviews


Formal Study

Primary methods

User tracking data

Evaluation questionnaires

Individual interviews

Locations

North Carolina

Virginia (EMT-B Class)

Maryland


Study Participants

• 42 first responders • 22 males • 20 females • 22 EMS, • 3 firefighter, • 17 police/security

personnel • age range: 17 to 56 • average age: 34


Open Answer Questions What is the strongest aspect of the BLAST program? The most frequent responses included realism, its interactive nature, and the fact that it provides the opportunity to practice triage skills. What is the weakest aspect of the BLAST program? Responses focused on usability issues, especially problems with using the mouse to control movements and decisions on screen., especially the lag between mouse movements and correlating screen movements . If you could improve anything in the BLAST program, what would it be? The most salient request for improvements concerned the clarity of the feedback, especially with the need to put a “face” on the victims to improve the context of the scoring and feedback . Other desired improvements included making the mouse more responsive with less lag time and increasing the noise and confusion presented in the scenario to make it more realistic. What other feedback can you provide concerning the BLAST program? Most of the study participants encouraged further development of additional scenarios for the BLAST program. Most of the concluding comments were very complimentary of the program.


When “Real” Users Strike Back What is the weakest aspect of the BLAST program?


When “Real” Users Strike Back What is the strongest aspect of the BLAST program?


When “Real” Users Agree

88% Similar programs should be developed

79% Program provided an engaging learning

opportunity


When “Real” Users Agree

69% State-of-the-art for (educational games)

76% Should be used by most first responders

65% Program is very realistic


When “Real” Users Strike Back

“I love the fact that it is visual, it has a real life sense to it.”

“…playing this and seeing different scenarios will help me be able to manage triage casualties.”

“…games such as this give a realistic scenario while taking (away) some of the stress a real situation would create.”


Formal Recommendations

Derived from the analysis of evaluation findings:

1. Develop more scenarios if funding is available.

2. Enhance the feedback in such a way that the learner understands the specific victim for whom feedback is being given. This could involve putting a unique “face” on each victim to identity of the character whose “triage” was incorrect.


Recommendations

Derived from the analysis of evaluation findings:

3. Reduce the lag between mouse movements and screen actions should be reduced. 4. Consider increasing ambient noise and other distractions in the scenario to make it more dynamic and engaging. 5. The program developers should scrutinize the detailed recommendations from the online survey transcripts to fix any specific problems identified by the evaluation participants.


Unexpected Findings

Group settings • fosters discussion • peers can assist with controls

Provides “lab” to a lecture

Pre/post larger scales exercises


Conclusions

“I’ve never seen the students so engaged” Chauncey Bowers CSP, ARM, Central Piedmont Community College

Games need to involve more than the game play itself; delivery medium, end-user play environment, and the game’s integration into the greater curriculum need to be factored for effective game design.

Real users can help get you there.

Thank You!

Games for Health Conference

Bob Waddington SimQuest, LLC

[email protected]

@waddingtondc

November 5-6, 2012 www.gamesforhealtheurope.org

June 12-14, 2013 Boston, MA



Technology

Waddington g4h 2012 final