Upload
brittany-graves
View
53
Download
2
Embed Size (px)
DESCRIPTION
blindSight: Eyes-free mobile phone interaction. Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research. blindSight. “How ab out Monday morning?”. calendar. “Monday 9am”. preview. “tic, tic, sssssh”. - PowerPoint PPT Presentation
Citation preview
blindSight:Eyes-free mobile phone interaction
Kevin Li, University of California, San DiegoPatrick Baudisch, Microsoft ResearchKen Hinckley, Microsoft Research
calendar calendar
preview preview
“Monday 9am”
“tic, tic, sssssh”
“How about Monday morning?”
“Yeah, looks likeI’m free after 10”
blindSight
blindSightis an application running on Microsoft Windows Smartphone
is launched when user places or receive a call. It then replaces the in-call menu
unlike the in-call menu, blindSight uses auditory feedback
why?
PCs…PC screens have the users’ undivided attention
design for the visual channel
eyePhone
environment
visual impairment
screen-less device
can’t seescreen
Lots of information is stored on mobile phones…
… the interfaces are visual
0123456789
# of
Peo
ple
Would Like
Very Important
# o
f pa
rtic
ipa
nts
“I need to access as part of a phone conversation:”
survey
Ok, so let’s just translate all text from visual to auditory
– “Menu: Press 1 to search contacts; press 2 to add a contact; press 3 to access your calendar…”
Wait, that sounds familiar
Please listen carefully as ouroptions have changed…
related work
User’s should be able to “dial ahead” [Perugini et al.,CHI 2007]
Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994]
Use visual channel to inform users about options [Yin and Zhai, CHI 2006]
interactive voice response
Time compress audio[Dietz and Yerazunis, UIST 2001]
Integrate speech commands into the conversation [Lyons et al., CHI 2004]
phone interactionmid-conversation
blindSight’sauditory feedback
audio is heard only by the user, not by the person at the other end
rationale
people can recover from audio interruptionsas long as interruption is short
human-human conversation contains redundancy
can we use this redundancy to injectauditory feedback from the device?
how do we make sure device feedback fits into these time windows of low information content?
rules
1. feedback only on-demand
hearvoice note
mute speaker phone
heartask list
addcontact
recordvoice
findcontact
calendar
hearemails
hear textmessage
home
rules
2. brevity
21 3
8
delete
4
97
65
abc def
tuv
ghi
wxyzpqrs
mnojkl
nextplay
find contact
type 6“200 hits”
type 2“12 hits”
type 7“Marion”
rules
3. non-speech previews of composites
week week
day day
3 hours 3 hours
½ hour ½ hourblock½h
previewday
+–––
+
_
whereAmI go today
preview3 hours
++
calendar
(what if the content is a long list,such as appointments for a day?)
rules
4. decomposition
week week
day day
3 hours 3 hours
½ hour ½ hournext
previewday
+–––
+
_
whereAmI go today
preview3 hours
(what if the content is a long list,such as appointments for a day?)
rules
5. interruptability
user interface runs as a separate thread
1 32
64 5
97 8
actiondelete save
rules
6. minimize modes
1 32
64 5
97 8
actiondelete save
mon wedtue
satthu fri
sun
actiondelete save
pick day start time end time
…modes
rules
6. minimize modes (avoid wizards)
week week
day day
3 hours 3 hours
½ hour ½ hourblock½h
previewday
+–––
+
_
whereAmI go today
preview3 hours
++
++
hearvoice note
mute speaker phone
heartask list
addcontact
recordvoice
findcontact
calendar
hearemails
hear textmessage
home
add contact
21 3
8
save
4
97
6
0
5
delete
iterator
1 32
64 5
97 8
action
action
delete save
menu
patterns
calendar
week week
day day
3 hours 3 hours
½ hour ½ hourblock½h
previewday
+––
–
+
_
whereAmI go today
preview3 hours
++
+
+
(shows fast usage by an experienced user)
demo video
hardware
space
epoxy dots
enlarged spaces
0%
2%
4%
6%
8%
10%
12%
14%
16%
1 2 3
Erro
r Rat
e (%
)
Flip
EarThumb
Visual
Flip
Ear
Visual
error
13 2
6 45
9 78
#*
0
.
0%
2%
4%
6%
8%
10%
12%
14%
16%
1 2 3
Erro
r Rat
e (%
)
Flip
EarThumb
Visual
Flip
Ear
Visual
error
blindSightevaluation
interfaces
Smartphone 2003 (sighted)BlindSight (eyes-free) vs.vs.
task
while “driving”idle
(1) schedule appointments and (2) add contacts
0 1 2 3 4 5 6 7 8
Was not missing information
Knew position in the menu
Knew what day/time I was at
Felt in control of the conversation
Better for setting meeting times
Prefer if driving and talking
Prefer Overall
blindSight Smartphone
Overall preference
results
1. brevity is good, but use in moderationclarification of navigation overrides brevity
2. predictable/modeless user interface is key
3. auditory feedback goes a long way even during phone call(disclaimer: need to study how it interferes with activities… driving)
lessons
environment
visual impairment
can’t seescreen
screen-lessdevice
next:
eyePhone
eyePhone
eyesFreePhone
?
blindSight:Eyes-free mobile phone interaction
Kevin Li, University of California, San DiegoPatrick Baudisch, Microsoft ResearchKen Hinckley, Microsoft Research
extra slides
0 1 2 3 4 5 6 7
Could hear partner despite …
Could hear blindSight despite …
Easy to listen and hit buttons
Day preview was useful
3 hour preview was useful
Useful to hear content without …
Tactile dots on keypad were useful
1. built a system
2. a set of eyes-free design rules
3. keypad modifications enabling eyes-free
4. user study comparing with a product (Smartphone 2003)
contributions
1. feedback only on-demand
2. brevity
3. non-speech previews of composites
4. decomposition
5. interruptability
6. minimize modes
rules
iterator
1 32
64 5
97 8
action
action
delete save
menu
patterns
hearvoice note
mute speaker phone
heartask list
addcontact
recordvoice
findcontact
calendar
hearemails
hear textmessage
home
add contact
21 3
8
save
4
97
6
0
5
delete
iterator
1 32
64 5
97 8
action
action
delete save
menu
patterns
calendar
week week
day day
3 hours 3 hours
½ hour ½ hourblock½h
previewday
+––
–
+
_
whereAmI go today
preview3 hours
++
+
+
hearvoice note
mute speaker phone
heartask list
addcontact
recordvoice
findcontact
calendar
hearemails
hear textmessage
21 3
8
save
4
97
6
0
5
delete
week week
day day
3 hours 3 hours
½ hour ½ hourblock½h
previewday
+–––
+
_
whereAmI go today
preview3 hours
++
21 3
8
delete
4
97
65
abc def
tuv
ghi
wxyzpqrs
mnojkl
nextplay
add contact find contact
calendar
home
type type
folder folder
n items n items
item itemplay
+–––
+
_ +preview
+
email, tasks, voice, SMS home
help
hold bottom left for
hold bottom right for
menu
…is a phenomenon in which people who are perceptually blind in a certain area of their visual field demonstrate some visual awareness, without any qualitative experience
blindSight...
[wikipedia]
don’t mode me in
blind sight
10 design rulesto allow eyes-free use and flow
tactile features
…are in in a mobile situation
If they requires visual attention,users will fail at their current activity
interference with social activitiesdrive off the road…
phones…
vs.vs.baseline
interfaces