Embed Size (px)
DESCRIPTION
Several interaction techniques have been proposed to enable transfer of information between different displays in heterogeneous multi-display environments. However, it is not clear whether subjective user preference for these different techniques depends on the nature of the displays between which information is transferred. We explore subjective usability of speech, touch and gesture for moving information between various displays in a heterogeneous multi-display environment, consisting of a multi-touch table, a wall-mounted display and a smartphone. We find that subjective user evaluation of the various interaction techniques depends on the combination of displays being used. This implies that the type of display combination should be taken into consideration when designing interaction techniques for the transfer of items between displays in a heterogeneous multi-display environment. Also, gesture based interactions were judged more acceptable when they involved holding a mobile phone, probably since this provided a cue explaining the action.
Citation preview
Subjective Usability of Speech, Touch and Gesture
in a Heterogeneous Multi-Display Environment
Arnoud de Jong Susanne Tak Alexander Toet Sven Schultz
Jan-Pieter Wijbenga Jan van Erp
TNO, The Netherlands
Overview
‒ Introduction‒ Related work‒ User Study‒ Tasks & Results‒ Conclusions‒ Future Work
Introduction:Facilitating cross-display interaction
Multi-touch Table
Interactive small groupsessions
Wall Screen
Large grouppresentations
Mobile Devices
Personal contentviewing
Introduction:Using different interaction techniques
Speech- Spoken commands- Direct hands-free interaction at close to medium range
Gesture- In-air body movements- Interaction over longer ranges
Touch - Hand contact - Physical interaction at close range
Related Work
− New techniques have been proposed to move objects between heterogeneous devices.
− Typically keyboard, touch, gesture, speech based.
− Single-user single-display paradigm still dominates.
− Available techniques complement each other:
Related Work
− New techniques have been proposed to move objects between heterogeneous devices.
− Typically keyboard, touch, gesture, speech based.
− Single-user single-display paradigm still dominates.
− Available techniques complement each other:
− Direct touch is fast - but only feasible at close range.
Related Work
− New techniques have been proposed to move objects between heterogeneous devices.
− Typically keyboard, touch, gesture, speech based.
− Single-user single-display paradigm still dominates.
− Available techniques complement each other:
− Direct touch is fast - but only feasible at close range.
− Speech suitable for descriptive techniques
and in conditions with occlusion.
Related Work
− New techniques have been proposed to move objects between heterogeneous devices.
− Typically keyboard, touch, gesture, speech based.
− Single-user single-display paradigm still dominates.
− Available techniques complement each other:
− Direct touch is fast - but only feasible at close range.
− Speech suitable for descriptive techniques and in conditions with occlusion.
− Gestural interaction enables remote manipulation of objects.
Related Work
− New techniques have been proposed to move objects between heterogeneous devices.
− Typically keyboard, touch, gesture, speech based.
− Single-user single-display paradigm still dominates.
− Available techniques complement each other:
− Direct touch is fast - but only feasible at close range.
− Speech suitable for descriptive techniques and in conditions with occlusion.
− Gestural interaction enables remote manipulation of objects.
− Users prefer combination of speech and gestural interaction
when handling graphics.
User Study: Usability and Preference
Display Pairs:
Table Screen Table MobileScreen MobileMobile Screen
Participants:
21 participants12 male 9 female27 average age
Usability Questionnaire:
− without thinking − intuitive− unnatural− tiring− responsive− complex− error-prone
Interaction Modes:
− Speech− Touch− Gesture
BlablaBlabla
Interaction Techniques
Table to screenSpeech: Select photo and say “send to screen”.Touch: Drag photo to a window entitled ‘Screen’.Gesture: Select photo and point at the screen.
Table to mobile
Speech: Select photo and say “send to Harry”.Touch: Drag photo to a window entitled ‘Harry’.Tangible: Place mobile on table and drag photo to it.
Screen to mobile
Speech: Start voice command by saying “screen” then say “send to Harry”.Gesture: Hold phone as if taking a photo of the screen.
Mobile to screen
Speech: Start voice command by dragging finger downwards over screen and say “send to screen”.Touch: Press send button below photo and select the ‘Screen’ menu item.Gesture: Point phone at the screen.
Tasks: Table to Screen
Results
• Most participants preferred touch technique
• 5 participants thought speakingcommands out loud was awkward
SpeechSelect photo and say “send to screen”.
TouchDrag photo to window entitled ‘Screen’.
GestureSelect photo and point at screen.
Tasks: Table to Screen
SpeechSelect photo and say “send to screen”.
TouchDrag photo to window entitled ‘Screen’.
GestureSelect photo and point at screen.
Preference Usability
Tasks: Table to Mobile
Results
• Most participants preferred the tangible technique
• The tangible technique was very well received,
with 10 participants calling it “fun” or “cool”.
SpeechSelect photo and say “send to Harry”.
TouchDrag photo to window entitled ‘Harry’.
TangiblePlace mobile on table and drag photo to it.
Tasks: Table to Mobile
SpeechSelect photo and say “send to Harry”.
TouchDrag photo to window entitled ‘Harry’.
TangiblePlace mobile on table and drag photo to it.
Preference Usability
Tasks: Screen to Mobile
Results
• No significant preference for either of the techniques
• 12 participants preferred speech the most, 9 participants preferred gesture the most
SpeechSay “screen” followed by “send to Harry”.
GestureHold phone as if taking photo of screen.
Tasks: Screen to Mobile
SpeechSay “screen” followed by “send to Harry”.
GestureHold phone as if taking photo of screen.
Preference Usability
Tasks Mobile to Screen
Results
• Most participants preferred the gesture technique
SpeechDrag finger and say “send to screen”.
TouchSelect ‘Screen’ menu item and press ‘Send’.
GesturePoint phone at screen.
Tasks Mobile to Screen
SpeechDrag finger and say “send to screen”.
TouchSelect ‘Screen’ menu item and press ‘Send’.
GesturePoint phone at screen.
Preference Usability
Conclusions
‒ User preference for interaction techniques depends on the nature of the displays.
‒ Speech technique is generally disliked: people feel embarrassed when speaking commands out loud.
‒ Holding an object that provides a clear visual cue indicating the user’s actions makes gesture-based interaction more acceptable.
Future Work
‒ Explicitly address embarrassment: test whether participants are primarily spatial, verbal or object oriented
‒ Register objective performance measures (e.g. time required to perform different actions).
‒ Investigate multiple users settings and/or more complex display combinations.
Questions ?
