Upload
marc-pous
View
1.635
Download
4
Tags:
Embed Size (px)
DESCRIPTION
Multimodal Interaction in Distributed and Ubiquitous Computing presentation
Citation preview
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc
Multimodal Interaction in Distributed and Ubiquitous Computing
Marc Pous and Luigi Ceccaronibdigital
ICIW 2010 - The Fifth International Conference on Internet and Web Applications and Services
May 9 - 15, 2010 - Barcelona, Spain
• Why multimodality?
• Multimodal distributed services deployment
• User tests and conclusions
SUMMARY
In our cities, urban services practically have not changed in a century
1: Location-based services
MOTIVATIONS
http://www.flickr.com/photos/nnova/4512346900/
http://picasaweb.google.com/marc.pous/Xina2007#5102695479944027890
http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415374502175450370
http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415371534810675298
http://www.flickr.com/photos/loquat73/3385335980/
2: Interactions and multimodality
MOTIVATIONS
http://www.flickr.com/photos/nnova/4423756547/in/set-72157623598200316/
http://www.flickr.com/photos/nnova/4424521014/in/set-72157623598200316/
3: Accessibility
MOTIVATIONS
http://www.dynamiclanguageblog.com/2009/12/technology-grows-for-hard-of-hearing.html
MOTIVATIONS
What is an ICD?(Interactive Community Display)
INREDIS
3G = ACCESSIBILITY + UBIQUITY + INTEROPERABILITY
http://www.inredis.es
INREDIS ICD
Management of distributed services that offer the capability of processing and synthesizing multiple modalities of interaction
DISTRIBUTED INTERACTION SERVICES
• Voice services– Voice recognition
• Via microphone, real-time streaming converting the voice signal into text
– Voice synthesis
• Sign language service– Spanish sign language recognition
• Via webcam, real-time streaming converting the image into text
• Emotion service– Emotion recognition (voice + image)
• Via webcam and microphone, real-time streaming
• Avatar service– Emotional avatar– Sign-language speaker avatar
DISTRIBUTED SERVICES
SIGN LANGUAGE SERVICES
DISTRIBUTED SERVICES
VOICE SERVICES
How to consume multimodal distributed services offered by an ICD?
W3C MMI Architecture
• W3C MMI Architecture
http://www.w3.org/TR/mmi-arch/
W3C MMI Framework
http://www.w3.org/TR/mmi-framework/
Proposed multimodal interaction framework
• ICD Architecture
ICD ARCHITECTURE
(1) Interaction identification
(2) Main page- Map or avatar- Command buttons- Content- Scenario options
(3) Contextualized map with services
(4) Avatar with map and Street View service
ICD DESIGN and INTERFACES
USER–CENTERED DESIGN
• User profiles (Personas)– Blind users– Deaf users
• Unable to read– People living or visiting a city
• Scenarios– Context-aware informative scenario– Emergency scenario
IMPLEMENTATION
USABILITY TESTING
• PLI (People Led Innovation) methodology
• Preliminar usability tests with a limited number of users
• But the ICD implementation received good feedback!
CONCLUSIONS and FUTURE WORK
• Implementation based on the W3C MMI Architecture idea• Platform able to integrate multimodal interactive distributed
services and offer interaction in real-time• Web-based applications and device-independent
implementation• Improvement of the user experience and accessibility of
people with special needs
What are we working on now?– Real-time interaction: delay reduction between interactions– Browser-based apps not being able to easily access device hardware– Synchronous interaction vs. Asynchronous technologies– Improvement of the distributed services orchestration– Enhancement of the mobile usability
MOBILE PHONES?
http://www.slideshare.net/nokiaconversations/younghee-jung-and-jan-chipchase-presentation-for-nokia-design-london-studio-event-london-29th-april-2008
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc
Thank you!
Moltes gràcies!
¡Muchas gracias!
Marc Pous and Luigi [email protected] and [email protected]