Upload
jennifer-romano-bergstrom
View
194
Download
0
Embed Size (px)
DESCRIPTION
a workshop for DCWW Code(Her) on integrating usability testing in development. Includes a link to a live usability test we conducted in class!
Citation preview
Launch w/ Confidence: Integrating Usability Testing Methods in Development
Jennifer Romano BergstromMarch 8, 2014
DCWW Code(Her) Workshop| Washington, DC@romanocog
2
About you
@forsmarshgroup @romanocog
1. What is your name?2. What do you do?3. Why are you here?4. What does usability mean to you?
3
@forsmarshgroup @romanocog
Measuring the UX
• How does it work for the end user?
• What does the user expect?
• How does it make the user feel?
• What is the user’s story and habits?
• What are the user’s needs?
“the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11
+ emotions
4
Usability vs. User Experience (UX)?
The 5 Es to Understanding Users (W. Quesenbery): http://www.wqusability.com/articles/getting-started.htmlUser Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php
@forsmarshgroup @romanocog
Whitney’s 5 Es of Usability
Peter’s User Experience Honeycomb
5Krug, S. Don’t Make Me Think
@forsmarshgroup @romanocog
What People do on the Web
6
Why is testing important?
@forsmarshgroup @romanocog
• Put it in the hands of the end user.• Things may seem straightforward to you but
maybe not to your users.
7
Why is testing important?
@forsmarshgroup @romanocog
• Put it in the hands of the end user.• Things may seem straightforward to you but
maybe not to your users.
8
Why is testing important?
@forsmarshgroup @romanocog
9
Why is testing important?
@forsmarshgroup @romanocog
• Put it in the hands of the end user.• Things may seem straightforward to you but
maybe not to your users.• You might have overlooked something big!
10
When to test
@forsmarshgroup @romanocog
11
@forsmarshgroup @romanocog
Where to test• Controlled environment
• All participants have the same experience
• Record and communicate from control room
• Observers watch from control room and provide additional probes (via moderator) in real time
• Incorporate physiological measures (e.g., eye tracking, EDA)
• No travel costs
LABORATORY REMOTE IN THE FIELD
• Participants tend to be more comfortable in their natural environments
• Recruit hard-to-reach populations (e.g., children, doctors)
• Moderator travels to various locations
• Bring equipment (e.g., eye tracker)
• Natural observations
• Participants in their natural environments (e.g., home, work)
• Use video chat (moderated sessions) or online programs (unmoderated)
• Conduct many sessions quickly
• Recruit participants in many locations (e.g., states, countries)
12
@forsmarshgroup @romanocog
How to test• In-depth feedback
from each participant
• No group think
• Can allow participants to take their own route and explore freely
• No interference
• Remote in participant’s environment
• Flexible scheduling
• Qualitative and Quantitative
ONE-ON-ONE SESSIONS
FOCUS GROUPS SURVEYS• Representative
• Large sample sizes
• Collect a lot of data quickly
• No interviewer bias
• No scheduling sessions
• Quantitative analysis
• Participants may be more comfortable with others
• Interview many people quickly
• Opinions collide
• Peer review
• Qualitative
13
@forsmarshgroup @romanocog
What to measure
OBSERVATIONAL+ Ethnography
+ Time to complete task+ Reaction time
+ Selection/click behavior+ Ability to complete
tasks+ Accuracy
IMPLICIT+ Facial expression analysis
+ Eye tracking+ Electrodermal activity (EDA)
+ Behavioral analysis+ Linguistic analysis of
verbalizations+ Implicit associations
+ Pupil dilation
EXPLICIT+ Post-task satisfaction
questionnaires+ In-session difficulty
ratings+ Verbal responses
+ Moderator follow up+ Real-time +/- dial
14
Eye tracking
• Observing and recording eye movements as a participant interacts with a product– Deeper insight into how users
perform tasks– Objective behavioral data
@forsmarshgroup @romanocog
15
Qualitative heat maps
@forsmarshgroup @romanocog
16
Qualitative gaze plots
@forsmarshgroup @romanocog
17
Qualitative gaze plots
@forsmarshgroup @romanocog
18
Qualitative gaze plots
@forsmarshgroup @romanocog
19
Qualitative gaze plots and comments
@forsmarshgroup @romanocog
M
“Man, this is a long paragraph.”
“There's a lot of information, it'd be a lot better in list form. Ideally, you want to get your information quick without reading through all this.”
20
What can be tested?
@forsmarshgroup @romanocog
• Concept• Existing websites/surveys• Low-fidelity prototypes• Paper mockups or mockups on computer• Basic idea is there• May lack functionality or graphical look
• High-fidelity prototypes• As close as possible to final interface in look
and feel
21
Focus group to test a concept
@forsmarshgroup @romanocog
• video
22
Usability test of a low-fidelity prototype
@forsmarshgroup @romanocog
• video
23
Usability test of a high-fidelity prototype
@forsmarshgroup @romanocog
• video
24
@forsmarshgroup @romanocog
* Couper, M.P., Baker, R., & Mechling, J. (2011). Placement and Design of Navigation Buttons in Web Surveys. Survey Practice, 4(1).
• “Where and how you click is a bit counter-intuitive. [It’s] not super obvious which button to click to get to next sections.”
• “I feel like the ‘Next’ should be at the bottom and not the top.”
Intuitive ‘Next’ button location
Non-intuitive ‘Next’ button location
Usability test across devices
• video
25
Combining qualitative and quantitative data
@forsmarshgroup @romanocog
• “I’m not expecting them to email or call me. I don’t expect any person to notify me.”
• “I’m not sure when I’d get an answer. It’s not like Yahoo! Answers where it’s immediate.”
Participants had different expectations about what would happen next. One expected an immediate response, two said they would call for assistance, five expected to hear back via email, and one expected to see the answer posted somewhere on the site.
• video
26
The older participants (ages 58, 64, and 65) tended to struggle more with the website. Specifically, they seemed to have more difficulties with the tags and with learning how to navigate to the answer pages (see videos below). We recommend testing a future iteration of the site with more older participants to determine if the site is usable for this population.
Usability testing with diverse users
@forsmarshgroup @romanocog
Both participants click on the tags rather than the question text, and both were confused when the tags did not work as expected.
• video• video
27
https://www.youtube.com/watch?v=aOyTG9MeKDw
Live usability test during the workshop!
@forsmarshgroup @romanocog
Thank you!• Twitter: @forsmarshgroup• LinkedIn: http://www.linkedin.com/company/fors-marsh-
group • Blog: www.forsmarshgroup.com/index.php/blog
Jennifer Romano Bergstrom@romanocog
DCWW Code(Her) Workshop| Washington, DC