Mixed Reality Game Prototypes for Upper Body Exercise and Rehabilitation

Marientina Gotsis* University of Southern California

Vangelis Lympouridis University of Edinburgh

David Turpin University of Southern California

Amanda Tasse University of Southern California

Irina C. Poulos Choreo Theatro Company, Inc.

Diane Tucker University of Southern California

Maximilian Swider University of Southern California

Alasdair G. Thin Heriot-Watt University

Maryalice Jordan-Marsh University of Southern California

ABSTRACT

This research demonstration consists of an integrated hardware and software platform developed for rapid prototyping of virtual reality-based games for upper body exercise and rehabilitation. The exercise protocol has been adopted from an evidence-based shoulder exercise program for individuals with spinal cord injury. The hardware consists of a custom metal rig that holds a standard wheelchair, six Gametraks attached to elastic exercise bands, a Microsoft Kinect, a laptop and a large screen. A total of 21 prototypes were built using drivers for Kinect, MaxMSP and Unity Pro 3 in order to evaluate game ideas based on deconstruction of the exercise protocol. Future directions include validation of our heuristic design and evaluation model and the development of an exercise suite of point-of-care VR games. KEYWORDS: virtual reality, games, interaction design, rehabilitation, exercise. INDEX TERMS: K.4.9 [Software/Software Engineering]: Design—Rapid Prototyping; K.9.6 [Computing Methodologies]: Simulation, Modeling, and Visualization—Gaming; K.10.3 [Computer Applications]: Life and Medical Sciences—Health.

1 BACKGROUND

This demonstration consists of an integrated hardware and software platform developed for rapid prototyping of VR-based games for upper body exercise and rehabilitation. The exercise protocol has been adopted from an evidence-based shoulder exercise program for individuals with spinal cord injury [1].

2 COMPONENTS

We developed a platform for rapid prototyping of concepts using off-the-shelf hardware and popular software. The hardware consists of a custom metal rig that holds a standard wheelchair, six disassembled Gametraks attached to elastic exercise bands to enable exercises that require resistance, a Microsoft Kinect for upper body tracking, a laptop and a large screen (Fig. 1). We developed 21 game concept prototypes using MaxMSP, which receives and processes real-time data from the Kinect and Gametraks and sends it to Unity Pro 3 to enable interaction, graphics rendering, animation and sound processing (Fig. 2).

3 FUTURE DIRECTIONS

Future directions include validation of our heuristic design and evaluation model and the development of an exercise suite of point-of-care VR games.

Figure 1. Custom rig with one Kinect, six Gametraks attached to elastic exercise bands and screen

4 PROJECT ACKNOWLEDGEMENTS

This research is funded by the U.S. Department of Education National Institute on Disability and Rehabilitation Research (NIDRR) in collaboration with Rancho Los Amigos National Rehabilitation Center and the Institute for Creative Technologies.

* e-mail: mgotsis@cinema.usc.edu

181

IEEE Virtual Reality 20124-8 March, Orange County, CA, USA978-1-4673-1246-2/12/$31.00 ©2012 IEEE

Table 1. Heuristic Design & Evaluation Model

Ideals Checklist

COGNITIVE CHALLENGE

is it sufficiently challenging? is it adaptive to individuals’ skills? is there enough depth of play?

AFFECT REGULATION

does it feel fun? can it adapt to mood state changes? does it provide a healthy range of affect?

DIALECTICAL ENGAGEMENT

what does it promote? is it persuasive? does it stimulate internal dialogue?

SOMATIC GRATIFICATION

do the physical interactions feel natural? is the body connected to the brain and mind? is the physical interaction rewarding?

SOCIOECOLOGICAL VALIDITY

are social interactions fostered? are social settings respected? where and how will this be used?

SEMIOTICINTEGRITY

is the experience meaningful? are the interactions purposeful? what is this perceived to be?

5 CREATIVE MEDIA & BEHAVIORAL HEALTH CENTER

The Creative Media & Behavioral Health Center (CM&BHC) is an organized research unit between the School of Cinematic Arts and the Keck School of Medicine. CM&BHC builds on USC’s existing academic strengths and unique reputation within the creative and technology industries to develop the preeminent center for innovation in the use of entertainment applications at the intersection of behavioral science, medicine and public health. CM&BHC’s mission is to increase public awareness of critical issues in mental health and behavioral science, to revitalize formal medical education and to improve treatments through innovative clinical, research and commercial applications using transmedia storytelling and emerging technologies.

6 INITIATIVES

The center pursues projects through the following initiatives:

6.1 Sports & Exercise for Wellness

This initiative aims to investigate novel assessment and treatment techniques using entertainment & technology for improvement of health outcomes related to obesity, nutrition and physical activity.

6.2 Healthy Brain Architecture

This initiative aims to develop and disseminate innovative storytelling products toward promotion of healthy behavior for brain development and affect regulation during the entire human lifespan.

6.3 Social & Sensorimotor Play

This initiative aims to investigate novel assessment and treatment techniques using interactive play for improving social skills, visuospatial navigation & motor coordination in physical space.

Figure 2. Game concepts screen captures

6.4 Games for Health

This initiative was established in 2007 as a pre-cursor to the Center in an effort to develop and assess innovative games with behavioral health applications [2].

7 METHODOLOGY

CM&BHC is comprised of multidisciplinary teams of artists, designers, researchers and technologists that uses an eclectic design, testing and evaluation methodology that includes: paper prototyping, physical prototyping, software prototyping, personal interviews, informal focus groups, observational studies and longitudinal playtesting.

8 THEORETICAL FRAMEWORK

CM&BHC has been developing a transtheoretical checklist for design heuristics for ideation and evaluation of prototypes. The checklist combines concepts from neuroscience, public health, psychotherapy, psychology, neurobiology/psychophysiology, game design, game rhetoric and art (Table 1).

9 FUNDING

CM&BHC has received support from the USC Zumberge Fund, the National Institutes of Health (NIH), the U.S. Department of Education National Institute on Disability and Rehabilitation Research (NIDRR), the U.S Army Telemedicine and Advanced Technology Research Center (TATRC), the Robert Wood Johnson Foundation, the Norlien Foundation, Humana, Inc., the Shafallah Center for Special Needs Children and Kohl’s.

REFERENCES

[1] S. J. Mulroy, L. Thompson, B. Kemp, P. P. Hatchett, C. J. Newsam, D. G. Lupold, L. L. Haubert, V. Eberly, T. T. Ge and S. P. Azen. Strengthening and optimal movements for painful shoulders (STOMPS) in chronic spinal cord injury: A randomized controlled trial. Phys. Ther. 91(3), pp. 305-324. 2011.

[2] M. Gotsis, "Games, Virtual Reality, and The Pursuit of Happiness," IEEE Comput. Graphics Appl., vol. 30, pp. 6-11, September/October 2009, 2009.

182