Playful Stimulation against Parkinson’s Disease - a tentative exploration

Final presentation, May 29, 2015Adrian Hänni and Padideh Pezeshki University of Fribourg

Outline● Introduction● Dandelion

o Game Rules● Types of Modalities used● MYO● Sphinx 4● Interfaces● Commands● CASE / CARE● Architecture● Video● Conclusion● Questions

Introduction

● Parkinson’s disease (PD) o Around 6.3 million people worldwideo Caused by the lack or reduced levels of dopamine in braino First affect one side, then develop to both sides of body, dominant on primary sideo Primary sign of PD is tremor, mostly on hands and fingerso Current treatments are medication, physiotherapy, and surgery

● Exergameso Developing exergames for health care is currently a hot topic in the field of virtual

rehabilitation (VRH)o We aimed to develop one for VRH of PD patients using multimodal interfaces

Dandelion

● Exergame prototypeo The character is a dandelion hovering in the spaceo Collect blue flowers / avoid thorn flowers

● Game ruleso Play time 60so Collecting blue flowers

Adds 10 points Increases play time by 2s An additional bonus score is added and increases by 1

o Thorn flower Subtracts 10 points Decreases play time by 5s Resets additional bonus score to 0

Types of Modalities used

● Gesture

o Use of gestures intends to animate patients suffering PD to perform physiotherapy exercises

o Hardware: MYO gesture recognition device

o Software: MYO Connect

● Speech

o Voice recognition as supportive alternative when certain movements cannot be performed

o Microphone

o Software: Sphinx 4 running on JAVA 1.7 VM

MYOFeatures

● Electromyographic (EMG) sensors

● Accelerometer (Vector of Units of g)

● Gyroscope (Vector of Degrees / Sec)

● Orientation (Roll / Pitch / Yaw as Quaternion)

o Requires calibration of world orientationDrawbacks

● Warm-up time (2min)● Requires every time new calibration profile for best results

Sphinx 4Pure Java speech recognition library for speech recognition, to identify speakers, adapt models, and transcriptionDictionary

● Grapheme to phoneme (G2P) conversionexit EH K Z IH Texit(2) EH G Z IH Texit(3) EH K S AH TAcoustic model

● Provided by Sphinx data package● Set to “en-us”

Grammar● Written manually in JSpeech Grammar Format (JSGF) format

public <WORDS> = ( up | down | left | right | score | help | exit | etc. ) ;

Interfaces

● Interface A

o MYO: Direction

o Voice: Speed Level

● Interface B

o MYO: Speed Level

o Voice: Direction

Both interfaces use Voice and/or electromyographic gestures for the game’s menu

Commands

Speech● Menu Screens

o START / HELP / SCORE / EXIT / NEXT● Game Screen

o FASTER / SLOWER / UP / DOWN / LEFT / RIGHTo SPEED <LEVEL>o <LEVEL> = FAST / MEDIUM / SLOW

Gesture● Menu Screens

o WAVE IN / WAVE OUT / FIST / FINGER SPREAD / DOUBLE TAP● Game Screen

o <Arm direction> = UP / DOWN / LEFT / RIGHT / REST POSITION

o <Arm direction> = Q<transition> = Q<rotation> * Q<rest position>^-1

CASE / CARE

CASE

● Exclusive

o Commands are exclusive and happen one after another in the order performed

CARE● Assignment

o Only FINGER SPREAD pose can set the MYO rest positiono Character control

Depending on interface type, speed and direction can be performed only by one modality

● Equivalenceo Choice between Speech and Gesture in the Menu screens

Architecture

Video

Conclusion● Absence of modality fusion is intended

o Game is aimed to be used by patients suffering Parkinson diseaseo Different health conditions increase the importance of alternatives, respectively

equivalenceo Focus on providing equivalent inputs of different modalities, as provided by the interfaces

can be considered to be beneficial

● Evaluation is pending

Questions