Thursday, October 25, 2012 Technical Session #7 Maysam Ghovanloo, PhD – ON Semiconductor Junior Professor, Georgia Tech; Director, GT Bionics Lab; Associate Editor, IEEE Trans. Circuits & Systems II
Citation preview
1. Dual-Mode Tongue Drive System Using Speech and Tongue Motion
to Improve Computer Access for People with Disabilities Xueliang
Huo Hangue Park Maysam Ghovanloo, Ph.D. GT-Bionics Lab School of
Electrical and Computer Engineering 2012 Maysam Ghovanloo
1www.GTBionics.org
2. Some Events and Their Consequences 2012 Maysam Ghovanloo
2www.GTBionics.org
4. 54M Americans (~20%) 11,000 cases of severe SCI add every
living with disabilities year to a total population of 55% 250,000
of the SCI victims are They need lifelong 16~30 special care
Financial, emotional, and productivity years old cost to the
families and the society 2012 Maysam Ghovanloo
4www.GTBionics.org
5. How to Improve their Quality of Life? 2012 Maysam Ghovanloo
5www.GTBionics.org
6. Why Using Tongue? Along with mouth occupies the amount of
sensory and motor cortex that rivals fingers and hand:
sophisticated motor control capability evident in speech and
ingestion Fast movement with many degrees of freedom (DoF). Very
flexible Connected to brain by a cranial nerve: escapes even high
level spinal cord injuries Noninvasive access to tongue is
possible. Motor Homunculus Not afflicted by repetitive motion
disorders Does not fatigue easily. Very low rate of perceived
exertion Cosmetic advantage and privacy. It is all inside the mouth
Not influenced by the position of the rest of the body Hypoglossal
Unlike BCIs does not need concentration. Nerve 2012 Maysam
Ghovanloo Ghovanloo, IEEE EMBC 2007 6www.GTBionics.org
7. Dual-Mode Tongue Drive System (dTDS) 3-axial magnetic
sensors + microphones on headset Wireless data Processing in
smartphone Wireless commands Target devices 2012 Maysam Ghovanloo
US Patents 8044766, 8242880, Other patents pending
7www.GTBionics.org
8. Magnetic Tracer Attachment Temporary attachment: By tissue
adhesives to test-drive the TDS Semi-permanent attachment: By
magnetic tongue piercing to use the TDS on a regular basis 2012
Maysam Ghovanloo 8www.GTBionics.org
9. Command Definition and Training Current version: Six tongue
commands plus the tongue resting position as neutral 2012 Maysam
Ghovanloo 9www.GTBionics.org
10. TDS Clinical Trials Computer access and wheelchair
navigation using TDS 2012 Maysam Ghovanloo 10www.GTBionics.org
12. dTDS Specifications Specification Value Magnetic Sensors
Type Honeywell HMC1043 AMR sensor Dimensions 3.0 3.0 1.5 mm3
Sensitivity / range 1 mV/V/Gauss / 600 T Microphone Type SiSonic
SPM0408HE5H Dimensions 4.7 3.8 1.1 mm3 Sensitivity / SNR -22 dB /
59 dB Control Unit Microcontroller TI CC2510 SoC Wireless frequency
/ data rate 2.42 GHz / 500 kbps Sampling rate 50 sample/s/sensor
Number of sensors /duty 4 / 8% cycle Audio codec / interface
TLV320AIC3204 / I2S Audio sampling rate / 8 ksps / 16 bits / -Law
resolution / compression 3 V / 35 mA (audio on) Operating voltage /
current 6 mA (audio off) PCB Dimensions 36 16 mm2 Wireless data
transfer to Headset Material Object VeroGray resin a PC running
Dragon Total weight 90 g (including battery) Naturally Speaking
2012 Maysam Ghovanloo 12www.GTBionics.org
14. Experimental Design 14 able-bodied subjects (age: 21 30
years old, 9 males and 5 females) from the Georgia Tech graduate
and undergraduate student population 7 subjects had prior
experience with TDS, and 7 were naive 7 subjects were native
English speakers and 7 were non-native A within-subject model with
each subject repeating the same tasks using three devices: TDS,
Dragon, and dTDS Two sessions, ~3h each, maximum of one week apart:
1) Instructional session: Subjects learned to use TDS, Dragon, and
dTDS 2) Experimental session: Quantitative and qualitative
measurements Two tasks: 1) Text transcription: subjects transcribed
two short paragraphs from a hard copy onto a word document to test
acoustic input 2) Maze navigation: subjects navigated the mouse
cursor through an on- screen maze, clicked on designated areas, and
typed words/numbers Questionnaire: At the end subjects rated their
experience with each device 2012 Maysam Ghovanloo
14www.GTBionics.org
15. Experimental Methods R L R R R L R L L R L Subjects were
asked to go through the maze as quickly and accurately as possible,
issue right/left click on yellow/green boxes, followed by typing A
minimum of 12 cursor movements, 11 clicks (excluding those for
typing), and typing 36 characters (on average) in each round of the
trial. Subjects were also asked to perform both tasks with a
combination of standard mouse and keyboard to generate a reference
point. Performance Measures: Recognition accuracy, total completion
time, typing time, navigation time, typing error, and navigation
error (deviation) 2012 Maysam Ghovanloo 15www.GTBionics.org
16. Experimental Results Total completion time: Significant
effect of device on all three measures Recognition accuracy:
Microphone type Native Speakers Non-native Speakers Significant
(7)* (7)* effect of Commercial 94.0% 1.9% 78.6% 5.9% accent dTDS
91.5% 2.3% 75.7% 5.1% 2012 Maysam Ghovanloo
16www.GTBionics.org
17. Qualitative Results Recognition accuracy: Novice subjects
only (7) A higher number represents more positive perception 2012
Maysam Ghovanloo 17www.GTBionics.org
18. Intraoral Tongue Drive System (iTDS) 2012 Maysam Ghovanloo
Park and Ghovanloo, ISSCC 12 18www.GTBionics.org
20. Conclusions Tongue Drive System (TDS) is a wireless,
wearable, and minimally invasive brain-tongue-computer interface
(BTCI) that enables individuals with severe disabilities to access
and control with their voluntary tongue motion. The latest
dual-mode TDS (dTDS) prototype appears as a wireless headphone with
both tongue motion and speech recognition (SR) capabilities for
navigation and typing, respectively. The subjects performance with
the dTDS was significantly better than unimodal TDS and SR in a
task that involved both navigation and typing (e.g. web surfing).
Subjects preferred dTDS over either TDS or SR in terms of speed,
ease of use and overall satisfaction. We are now in the process of
developing a multimodal Tongue Drive System (mTDS) as well as an
intraoral Tongue Drive System (iTDS) that is completely
inconspicuous. 2012 Maysam Ghovanloo 20www.GTBionics.org
22. Acknowledgements Funding provided by: National Science
Foundation Christopher and Dana Reeve Foundation National
Institutes of Health Army Research Office (ARO) ON Semiconductor
Collaborators: Dr. Michael Jones, Shepherd Center, Atlanta, GA Dr.
Ann Laumann, Northwestern University, Chicago, IL Dr. Joseph Manns,
Emory University, Atlanta, GA Dr. Elliot Roth, Rehab. Institute of
Chicago, Chicago, IL Dr. Elizabeth Bailey, University of Arizona,
Tucson, AZ Dr. Karim Oweiss, Michigan State University, Lansing, MI
Dr. Kimberly Wilson, Emory Hospital, Atlanta, GA Dr. Stephen
Sprigle, Georgia Tech, Atlanta, GA 2012 Maysam Ghovanloo
22www.GTBionics.org
