Interactivity & Design
David KirshDept of Cognitive Science
UCSD
Question
Designing for experience
is it similar to
Designing for efficiency
Topics
• Physical interactivity - an example
• Efficiency
• Fundamental Design Principle
• Metrics
• Experience
• Conclusion
Physical Interactivity: an example
• Complex folding sequence– Easy to forget
• Continuous interaction
• Difficult for one person
How would you make this process easier?
Think: context aware, digital elements in around
cardboard.
VideoQuickTime™ and a
YUV420 codec decompressorare needed to see this picture.
Design it better!
• Embed instructions (digital or physical)
– Where, when, how
• Improve manipulability
– Where, when, how
Fundamental Design Principle
Design Principle for Efficiency
right
Info
Tools
Cues
Constraints
Affordances
right Form, Place, Time, Pace
For a given task, design an environment so that it provides the:
to support efficient work activity.
Right for what?
• Pragmatic efficiency (getting job done quickly, few errors)
• Cognitive Efficiency
• Optimizes certain performance metrics
• Enhances experience of working/acting?
right
Info
Tool
Cue
Constraint
Affordance
right Form, Place, Time, Pace
Right(Info, Tools, Cues, Constraints, Affordances)
Constraint: Can’t fold in certain ways, forces compliance
Information: words, ‘fold in’+ instructions
Cues: arrows, colors, symbols of folding, fold marks, next fold lights up.
Tools: jig that holds and re-orients, actuators
Affordances: surfaces made for grasping
All revealed and concealed to manage attention
For cardboard task:
Right Info
• What is the right information to spatialize?
– Recipe study• problems with modularizing info
– Origami study• What activities should you support?
Spatially Distributed Recipe
ParticipantCOOKING TOOLS
INGREDIENTS
POTS & PANS
2/2 OF RECIPE
1/2 OF RECIPE Cooking phase
Assembling phase
RECIPE STIMULI: Spatially Distributed - improve localitySpatially distributed
After learning from pilot
Spatially Distributed Activity Map: Preparation
ParticipantCOOKING TOOLS
INGREDIENTS
POTS & PANS
2/2 OF RECIPE
1/2 OF RECIPE
Laying out ingredients, chopping,washing, measuring
Results
• Participants made better use of space– Used more surface, stabilized better, prepared
better
• BUT: – took much longer, – looked at recipe more
• Wanted to look ahead!
• Implication: we don’t always know what information is needed by users
What do subjects do?Origami
• More than fold– flip over, inflate, rotate, register, point, try out in gesture form …
• Pragmatic actions: needed to complete structure
– on sub-goal trajectory
– fold and some non-folding actions - flatten, flip over, inflate
• But we saw other actions that were not pragmatic but which seemed important for the subject
• Only a fraction of the actions performed are represented explicitly in origami instructions
To support activity we need to know what users do - their routines etc. ..the task structure.
Non-pragmatic Hand Actions
Registration
Verification
Gestural Thought
Focusing Attention
Trying out - exploratory
QuickTime™ and aH.263 decompressor
are needed to see this picture.
Upshot
• Choosing the right information to spatialize is hard– Even when workflow is known– What should be shown, when and where?– But often we have
little idea what thereal workflow involves
Right Form
Cognitive efficiency
– Faster processing
Given some information content, cue, constraint or tool how should it be displayed to support:
That is something up with which I will not put.
I won’t put up with that.
Police police police police police.
Police whom policemen police also police other police.
Topology vs. algebra
• Topological constraints are more natural
Right Form: Modality
Right modality
– Visual decisions are visual
– Audio statements free
visual search
Given some information content, cue, constraint or tool how should it be displayed to be the:
Interface 2
Interface 3
Rely on recognition rather than memory. Show, don’t just tell
Right Form
Visibility
- cue stands out
Given some information content, cue, constraint or tool how should it be displayed to have:
Right Form: visibility
R R R RR RR
R
R
RR R R
RRR P
Is there a non-R?
Right Form: visibility
R R R RR RR
R
R
RR R R
RRR P
Upshot
• Each step or phase in a routine or activity requires information or cues to be in the right form
• The right form may vary with step and task
• There are some general principles
Right Place
• The information or cue should be placed where you need it - given your resources and workflow
Starbuck’s cup
Right Place
On radio - logically
Volume, ChannelButtons
On steering wheel - workflow
Right Place
Interface 2
Goes together semantically goes together visually
Options are where they should be
Right place for cue when there are distracters
reduce descriptive complexityreduce visual complexity
Better place
Upshot• Spatializing information correctly depends
not just on workflow and resources but on:
– Showing semantic or work significant relations between information or cues - what goes with what
– A theory of attention• Should we use P or RRR P?
– Visibility or Place
Right Time
• See what I need to at this stage of activity– Eg. Jigsaw puzzle might show perimeter pieces first
• Show instructions, cues, tools just when needed in workflow – Hide tools in Illustrator™ that cannot be used in current
context
• Encourage right sequence - soft constraints– Travel: If I choose departure time first then calendar for
return time can be autoset– Show horizon of relevant options
Right Time: relevant optionsGiven structure of task - order of sub-goals - show only relevant actions
D H M
A B C E F G I J K L
orand
D H or M P
P
A
B
C
Upshot
• Time, place and form interact
• Get them right and users have what they need to make an informed choice right at their fingertips
Right Pace• Game coming at you too fast • Activity has a natural frequency• Slides in a presentation - comprehension rate
StressedBored
Cognitive Load
AttentivenessLoses focus
Disrupted
optimal
Right Pace: blind to change if slow
Change blindness
Before After
Fast enough and you see it.
Upshot
• Pace is the overall speed users find comfortable when performing their tasks
• Pace can change with mood and other user states
Bottom Line
• To give the right information at the right time … is equivalent to creating a– Dynamic keyhole - cognitive sweet spot
QuickTime™ and aAnimation decompressor
are needed to see this picture.
Metrics
Speed Accuracy & Design How fast can you fold the cardboard house?
Learnability & Design How long did it take to master the folding routine?
Complexity & Design How much more complex structures can you fold?
Error Recovery & DesignIf you make an error how long does it take to recover?
Variance & Design How much variance is there in your performance?
Bean Counters & DesignHow often do you damage or ruin the cardboard?
Interim Summary
• Designing for efficiency is hard• Requires developing a science of:
– Workflow analysis– Cue effectiveness, task informativeness– Cognitive complexity of forms
• E.g. visual simplicity, representational efficiency
– Impact of timing on routine efficiency– Impact of pace on attention, routine efficiency and
comprehension
Experience
Feeling versus Efficiency
Fast, few errors, easy, functional
Efficiency
Beautiful - clean, harmonious,
Kinetic - bodily movement
Tactile - texture, smooth
Emotional - erotic, rage, affectionate, love
Stimulating - intellectual,
Feeling
What feeling is there in state space environments?
Timeless •do things in sequence but state
transitions have no function beside moving to next state
•Qualitative feel of transition is irrelevant
States and Actions
At each node there are:• Possible actions
Goal is to reach end stateby selecting actions
Start
End End
Reflection One
• Experience is essentially in time– People are still experiencing even when system
is stopped– Experience is continuous never discrete
• It’s about the ‘time between’ the state transitions– About the process more than the outcomes
• Longer beautiful route rather than shortest uglier
Coverflow
QuickTime™ and aAnimation decompressor
are needed to see this picture.
What matters is the feel of the transition
Smooth, nice reflections - non-pragmatic properties
Design Principle
• For state space environments:– E.g. Forms, wisards, most websites, database
queries, dialogue based interfaces, most tool based interfaces (photoshop) …
– Design principles needs addition
Right(…) right (….) that makes user feel good
right
Info
Tool
Cue
Constraint
Affordance
right Form, Place, Time, Pace
Reflection Two
• What matters is:
– How agent engages an object or environment • He was hungry and attacked his food with gusto
–How agent feels when acting or watching While patting his dog he felt love and devotion
Continuous Control environmentsWorld acts back
continuously
Part of a system
Actions are continuous so not like state spaceoperators - everything is a transition
Continuous control
Theremin the first musical instrument designed to be played without being touched.
Highly interactive gesture based
Engagement and Feeling
• Depends on inner state: hormones, appetite, conditioning
• Agent can learn to engage or to feel
Reflection ThreeTwo forms of experience:
– Consciousness - qualia– Nature of sensation
– Cut a tomato - sharp knife
– Visceral feeling
– Content - interpreting as, seeing as, categorizing
Ecstasy of Fruit Loops
Qualia
• Varies between individuals• Sensitive to inner state: hormones,
emotions,• Some qualia - visceral feeling are the
result of mirror neurons - physical empathy
• If affected by stage setting, anticipation, scripting then not purely qualia
Interpretation needs anchoring
• Cue effectiveness– Image completion
– Word fragment completion
– Intentional frameworkcausation vs. self-propelling, desires
Rational gestures– Behaving ‘like a friend’
• Criteria for making an ‘inference-rich attribution’
wrd frgmnt
Upshot
• Study of projection of meaning - what is recognized and why - is necessary to determine:– Right (cue) right (Form, time, place, pace)
to engender an interpretation
Further reflections• Pleasure and delight is also derived from self-expression• Thought has an experiential component - intellectual
delight is content related• Absorption in environments often requires commitment
(this work is important) or the willing suspension of disbelief - that avatar is Josh
• Flow states are desirable but mostly ill-understood
Conclusion
• Designing for experience in state space environments is about the ‘time between’
• Designing for experience in continuous environments is about delight, absorption, engagement, hormones, emotions
• Both efficiency and experience design requires developing a science of:
right
Info
Tool
Cue
Constraint
Affordance
right Form, Place, Time, Pace
The End