Embed Size (px)
DESCRIPTION
Much of our work in UX research focuses on usability – evaluating products and interfaces to ensure they are easy-to-use. However, in today’s digital world, they are no longer enough. Consumers also have come to expect entertaining and engaging experiences. Web and mobile applications need to be usable, useful and engaging. So, how do we evaluate web interfaces to determine how useful and engaging they are? Desirability has been evaluated in recent years by the use of the Product Reaction Card technique, originated by folks at Microsoft. However, there are many other techniques used in market and industrial design research that we can borrow to complement this technique. Likewise, we can use standard usability testing techniques with lines of questioning with a slightly different focus to evaluate the relative usefulness of different solutions for a particular user group. In this talk, I discuss several techniques that I have used in recent months to evaluate the usefulness and desirability of interfaces The best techniques I have discovered to evaluate usefulness involve open-ended interview questions regarding current processes and pain points, followed by a usability evaluation of the interface and then a reflective interview discussing the benefits and drawbacks of that solution to their personal situation. To evaluate desirability, I will discuss the product reaction card technique and variations using more defined vocabularies for emotional responses and product personalities. In addition I will show results from techniques borrowed from psychology and marketing research - sentence completion, collaging, and the use of dyad rating scales. These techniques offer a variety of both qualitative and quantitative data that can be used to compare different interface options.
Citation preview
Beyond Just Usability: Desirability & Usefulness Testing
Susan Mercer, Senior Experience Researcher @susanamercer
Usability Testing
Usability = (Efficiency + Effectiveness) In a given
context of use
Usable
Cute
Acerbic
Usability
Usefulness Desirability
Usability Consistent
Organized
Understandable
Simple
Efficient
Usefulness
Effec=ve
Essen=al
Relevant
Helpful
Suppor=ng
Desirability
Exci=ng
Fresh
Friendly
Encouraging
Meaningful
Usability
Usefulness Desirability
DESIRABILITY TESTING METHODS
Open-‐Ended Ques,oning • What do you think of…?
• What is your reac,on to…?
• How would you describe…?
Sentence Comple,on • This app makes me feel…
• If I used this all day, I would… • Reading on this website is… • Compared to other apps, this app is…
• I want to use this because… • I do not want to use this because…
Product Reac,on Cards • 118 words
• 58% posi,ve • 36% nega,ve • 6% unclear
• 21% synonyms with 1+ other words • Mostly posi,ve words
Product Reac,on Cards – Alternate List
• Removed synonyms
• Added opposites to create Posi,ve / Nega,ve Pairs • Compared against two other emo,onal word vocabularies for Product Appeal and Emo,ons
• Marc Hassenzhahl, Darmstadt University of Technology, Germany
• Geneva Emo,on Wheel
Annoying Costly Expected Interes=ng Poor quality Trivial Apathe=c Crea=ve Familiar Irrelevant Predictable UnaJrac=ve AJrac=ve Desirable Fast Managable Professional Uncontrollable Boring Difficult Flexible Meaningful Relaxing Understandable Busy Difficult Fragile Mo=va=ng Relevant Undesirable Calm Disappointed Fresh Nondescript Reliable Unexci=ng Cheap Discouaraging Friendly Not Secure Rigid Unimagina=ve Clean Disorganized Frustra=ng Not Valuable Sa=sfying Unimportant Clear Dull Fun Novel Secure Unnecessary CluJered Easy to use Hard to Use Obstruc=ng Simple Unpredictable Common Effec=ve High quality Old Slow Unprofessional Compelling Efficient Impersonal Op=mis=c Sophis=cated Unrefined Complex Effortless Impressive Ordinary Stable Unreliable Comprehensive Empowering Incomplete Organized Standard Unusable Confident Encouraging Incomprehensible Original Strange Unwelcoming Confusing Enthusias=c Inconsistent Overwhelming Stressful Usable Conserva=ve Essen=al Ineffec=ve Patronizing Suppor=ng Useful Consistent Excep=onal Inefficient Personal Tedious Useless Controllable Exci=ng Innova=ve Pessimis=c Time-‐consuming Valuable Convenient Exclusive Inspiring Pleasant Time-‐Saving Weak
AJrac=ve UnaJrac=ve Excep=onal Common Organized Disorganized Calm Busy Exci=ng Dull Original Ordinary Clean CluJered Exclusive Standard Personal Impersonal Clear Confusing Familiar Strange Pleasant Annoying Compelling Unimportant Fast Slow Predictable Unpredictable Comprehensive Incomplete Flexible Rigid Professional Unprofessional Confident Weak Fresh Old Relaxing Stressful Consistent Inconsistent Friendly Unwelcoming Relevant Irrelevant Controllable Uncontrollable Frustra=ng Encouraging Reliable Unreliable Convenient Difficult Fun Tedious Sa=sfying Disappointed Costly Cheap High quality Poor quality Secure Not Secure Crea=ve Unimagina=ve Impressive Nondescript Simple Complex Desirable Undesirable Innova=ve Conserva=ve Sophis=cated Unrefined Easy to use Hard to use Inspiring Unexci=ng Stable Fragile Effec=ve Ineffec=ve Interes=ng Boring Suppor=ng Obstruc=ng Efficient Inefficient Manageable Overwhelming Time-‐saving Time-‐consuming Effortless Difficult Meaningful Trivial Understandable Incomprehensible Empowering Patronizing Mo=va=ng Discouraging Usable Unusable Enthusias=c Apathe=c Novel Expected Useful Useless Essen=al Unnecessary Op=mis=c Pessimis=c Valuable Not Valuable
Reliable Accessible Appealing Confusing
Comfortable Organized Professional Busy
Confident Straight Forward Approachable Complex
Effec=ve Understandable Efficient Dull
Friendly Easy to Use Flexible Hard to Use
Invi=ng Clear Helpful Impersonal
Time-‐saving Convenient Overwhelming
Trustworthy Clean Boring
Useful Familiar Frustra=ng
Simplis=c In=mida=ng
Valuable Stressful
Usable Time-‐consuming
Customers Customers & Non-‐Customers Non-‐Customers
68 67
62
42
20 22
17
31
0
10
20
30
40
50
60
70
Exis1ng Design A Design B Design C
Coun
t
Number of Words
Posi=ve
Nega=ve
Exis1ng Design A Design B Design C
Appealing Appealing Appealing Appealing
Clean Clean Clean Clean
Efficient Efficient Efficient Efficient
Engaging Engaging Engaging
Intui=ve
Professional Professional Professional
Useful Useful Useful Useful
Total 9 20 16 9
Product Reac,on Dyads Rate on a 5-‐point scale from Posi,ve – Nega,ve word pairs
AJrac=ve ¤ ¢ ¢ ¢ ¢ UnaJrac=ve
Calm ¢ ¤ ¢ ¢ ¢ Busy
Clean ¢ ¤ ¢ ¢ ¢ CluJered
Clear ¤ ¢ ¢ ¢ ¢ Confusing
Comprehensive ¢ ¢ ¢ ¤ ¢ Incomplete
-‐3.5 -‐2.5 -‐1.5 -‐0.5 0.5 1.5 2.5 3.5
Discouraging
Unrefined
Useless
Complex
Irrelevant
Boring
Expected
Unprofessional
Mo=va=ng
Sophis=cated
Useful
Simple
Relevant
Interes=ng
Novel
Professional
Photo Selec,on
LEM Tool (Susa Group)
Open-‐Ended Ques,oning
Sentence Comple,on
Product Reac,on Cards
Product Reac,on Dyads
Photo Selec,on
LEM Tool
Usability
Desirability Usefulness
USEFULNESS TESTING
Pre-‐Usability Interview • How do you reconcile invoices today…? • Who needs to approve these and why…?
• Where do you track which claims have been processed…?
• When a new pa,ent arrives, what do you do?
During Usability Tes,ng Watch for comments about usefulness
• “Wow. This would really make it easier to reorder the same things.”
• “I don’t like this. It would take me too long to do this part.”
Watch for neutral reac,ons
• “Eh. This is OK.” • Probe: “How would this help, or not help, you do your job?”
Post-‐Usability Interview
Show screens again and ask:
• “How would this work for you in your job?” • “What about this works for you? What doesn’t work?”
TESTING METHODS
Desirability Tes,ng: • Open-‐Ended Ques,oning
• Sentence Comple,on • Product Reac,on Cards
• Product Reac,on Dyads
• LEM Tool
Usefulness Tes,ng: • Pre & Post-‐Interview Ques,ons
• Listening & Asking Probing Ques,ons
