Launch With Confidence! Integrate UX Research Throughout Development

Launch w/ Confidence: Integrating Usability Testing Methods in Development

Jennifer Romano BergstromMarch 8, 2014

DCWW Code(Her) Workshop| Washington, DC@romanocog

2

About you

@forsmarshgroup @romanocog

1. What is your name?2. What do you do?3. Why are you here?4. What does usability mean to you?

3

@forsmarshgroup @romanocog

Measuring the UX

• How does it work for the end user?

• What does the user expect?

• How does it make the user feel?

• What is the user’s story and habits?

• What are the user’s needs?

“the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11

+ emotions

4

Usability vs. User Experience (UX)?

The 5 Es to Understanding Users (W. Quesenbery): http://www.wqusability.com/articles/getting-started.htmlUser Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php

@forsmarshgroup @romanocog

Whitney’s 5 Es of Usability

Peter’s User Experience Honeycomb

5Krug, S. Don’t Make Me Think

@forsmarshgroup @romanocog

What People do on the Web

6

Why is testing important?

@forsmarshgroup @romanocog

• Put it in the hands of the end user.• Things may seem straightforward to you but

maybe not to your users.

7

Why is testing important?

@forsmarshgroup @romanocog

• Put it in the hands of the end user.• Things may seem straightforward to you but

maybe not to your users.

8

Why is testing important?

@forsmarshgroup @romanocog

9

Why is testing important?

@forsmarshgroup @romanocog

• Put it in the hands of the end user.• Things may seem straightforward to you but

maybe not to your users.• You might have overlooked something big!

10

When to test

@forsmarshgroup @romanocog

11

@forsmarshgroup @romanocog

Where to test• Controlled environment

• All participants have the same experience

• Record and communicate from control room

• Observers watch from control room and provide additional probes (via moderator) in real time

• Incorporate physiological measures (e.g., eye tracking, EDA)

• No travel costs

LABORATORY REMOTE IN THE FIELD

• Participants tend to be more comfortable in their natural environments

• Recruit hard-to-reach populations (e.g., children, doctors)

• Moderator travels to various locations

• Bring equipment (e.g., eye tracker)

• Natural observations

• Participants in their natural environments (e.g., home, work)

• Use video chat (moderated sessions) or online programs (unmoderated)

• Conduct many sessions quickly

• Recruit participants in many locations (e.g., states, countries)

12

@forsmarshgroup @romanocog

How to test• In-depth feedback

from each participant

• No group think

• Can allow participants to take their own route and explore freely

• No interference

• Remote in participant’s environment

• Flexible scheduling

• Qualitative and Quantitative

ONE-ON-ONE SESSIONS

FOCUS GROUPS SURVEYS• Representative

• Large sample sizes

• Collect a lot of data quickly

• No interviewer bias

• No scheduling sessions

• Quantitative analysis

• Participants may be more comfortable with others

• Interview many people quickly

• Opinions collide

• Peer review

• Qualitative

13

@forsmarshgroup @romanocog

What to measure

OBSERVATIONAL+ Ethnography

+ Time to complete task+ Reaction time

+ Selection/click behavior+ Ability to complete

tasks+ Accuracy

IMPLICIT+ Facial expression analysis

+ Eye tracking+ Electrodermal activity (EDA)

+ Behavioral analysis+ Linguistic analysis of

verbalizations+ Implicit associations

+ Pupil dilation

EXPLICIT+ Post-task satisfaction

questionnaires+ In-session difficulty

ratings+ Verbal responses

+ Moderator follow up+ Real-time +/- dial

14

Eye tracking

• Observing and recording eye movements as a participant interacts with a product– Deeper insight into how users

perform tasks– Objective behavioral data

@forsmarshgroup @romanocog

15

Qualitative heat maps

@forsmarshgroup @romanocog

16

Qualitative gaze plots

@forsmarshgroup @romanocog

17

Qualitative gaze plots

@forsmarshgroup @romanocog

18

Qualitative gaze plots

@forsmarshgroup @romanocog

19

Qualitative gaze plots and comments

@forsmarshgroup @romanocog

M

“Man, this is a long paragraph.”

“There's a lot of information, it'd be a lot better in list form. Ideally, you want to get your information quick without reading through all this.”

20

What can be tested?

@forsmarshgroup @romanocog

• Concept• Existing websites/surveys• Low-fidelity prototypes• Paper mockups or mockups on computer• Basic idea is there• May lack functionality or graphical look

• High-fidelity prototypes• As close as possible to final interface in look

and feel

21

Focus group to test a concept

@forsmarshgroup @romanocog

• video

22

Usability test of a low-fidelity prototype

@forsmarshgroup @romanocog

• video

23

Usability test of a high-fidelity prototype

@forsmarshgroup @romanocog

• video

24

@forsmarshgroup @romanocog

* Couper, M.P., Baker, R., & Mechling, J. (2011). Placement and Design of Navigation Buttons in Web Surveys. Survey Practice, 4(1).

• “Where and how you click is a bit counter-intuitive. [It’s] not super obvious which button to click to get to next sections.”

• “I feel like the ‘Next’ should be at the bottom and not the top.”

Intuitive ‘Next’ button location

Non-intuitive ‘Next’ button location

Usability test across devices

• video

25

Combining qualitative and quantitative data

@forsmarshgroup @romanocog

• “I’m not expecting them to email or call me. I don’t expect any person to notify me.”

• “I’m not sure when I’d get an answer. It’s not like Yahoo! Answers where it’s immediate.”

Participants had different expectations about what would happen next. One expected an immediate response, two said they would call for assistance, five expected to hear back via email, and one expected to see the answer posted somewhere on the site.

• video

26

The older participants (ages 58, 64, and 65) tended to struggle more with the website. Specifically, they seemed to have more difficulties with the tags and with learning how to navigate to the answer pages (see videos below). We recommend testing a future iteration of the site with more older participants to determine if the site is usable for this population.

Usability testing with diverse users

@forsmarshgroup @romanocog

Both participants click on the tags rather than the question text, and both were confused when the tags did not work as expected.

• video• video

27

https://www.youtube.com/watch?v=aOyTG9MeKDw 

Live usability test during the workshop!

@forsmarshgroup @romanocog

Thank you!• Twitter: @forsmarshgroup• LinkedIn: http://www.linkedin.com/company/fors-marsh-

group • Blog: www.forsmarshgroup.com/index.php/blog

Jennifer Romano Bergstrom@romanocog

jbergstrom@forsmarshgroup.com

DCWW Code(Her) Workshop| Washington, DC