Measuring UXMichael Le

Why bother? You did rapid prototyping using Balsamiq You used Agile during development You are feature and code complete You delivered the application on time

Consider the following… Are people using your application? Are people using your application the way you intended? Are people using your application differently? Do they like it?

Goal Continuous improvement

How? Observing Listening Analyzing

Observing

The user is not like me The moment you accept that the easier you can observe

Be humble

Task Observations Preset scenarios that involve the user going through one

story or key action

Thinking out loud is encouraged Observer should refrain from helping the user in the actions

Video recording is recommended On screen User

Example Task 1 – Marking points on the EUR price curve Task 2 – Marking points on the GBP price curve

Based on rating scale of (1 – 5, hard to easy)

Same task but different score

Users mentioned that they found the second stage harder because they couldn’t navigate back to the main selection area

Eye heatmaps Using sophisticated tools such as eye tracking cameras you

can create heatmaps of what people are looking at in an application

Facebook: Attention is on areas with pictures

CNN: People avoid looking at ads generally

Listening

System usability scale Simple list of likert scaled questions

Surveys General questions about the user and how they interact with

the system

1. I think that I would like to use this system frequently.

2. I found the system unnecessarily complex.

3. I thought the system was easy to use.

4. I think that I would need the support of a technical person to be able to use this system.

5. I found the various functions in this system were well integrated.

6. I thought there was too much inconsistency in this system.

7. I would imagine that most people would learn to use this system very quickly.

8. I found the system very cumbersome to use.

9. I felt very confident using the system.

10. I needed to learn a lot of things before I could get going with this system.

Graphing the responses

Watch out for “bad” questions Leading questions Questions with yes/no responses

“Would you use this application?”

For web applications to general public this is a good question

For internal application for regulatory purposes, maybe not Answer would be YES

What is a better question?

“What would make you stop using this application?”

May find out information about what is important to users

People were generally worried about the calculations accuracy

People were worried about who did what (auditing)

Example: Walmart Trying to compete with Target in the area of store layout

Clean and tidy (Target) vs. Packed aisles (Walmart)

Walmart surveyed their customers “Would you like Walmart aisles to be less cluttered?”

People said Yes that’s a great idea, less clutter in the aisles is good!

Walmart reacted 15% of inventory was removed from the aisles

Removed pallets of items like juice boxes in the centers of aisles.

Reduced displays at the ends of aisles Shortened shelves

What happened1.85 billion dollars loss in sales for Walmart

Cause of error Walmart came up with the answer first, then asked

customers to agree to it.

You should react to what customers do rather than what they say

Ie. How often do you work out?

Analyzing

Logs What people are searching for? How often?

Uncover relationships in the data searched for and your application

Add hooks to your applications so that you can track uses when different views are invoked

Taking the ideas of tracking web clicks to the desktop application world

Heuristic evaluation This can be done during the rapid prototyping stage as well

Using a few evaluators who are familiar with heuristic evaluation to go through an rate an application on certain criteria

Considered cheaper and as effective as user testing

This is not a graphical design evaluation

Nielsen’s Heuristics Jakob Nielsen’s list of heuristics is one of the most used set

for evaluating user interfaces

There are 10 heuristics

Each heuristic is measured on a numeric scale with 1 – Low 10 – High priority, must fix

Visibility of system status Essentially feedback Users should be informed of system state within a

reasonable time

Nielsen’s Heuristics (1/10)

Match between system and real world System should use users’s language with words, phrases

and concepts “Item” versus “songs” in an album

Nielsen’s Heuristics (2/10)

User control and freedom Making errors is a good way for users to discover features of

the application

There should be affordances to let users fail without fear: undo, redo, exit without saving

Nielsen’s Heuristics (3/10)

Consistency and standards Follow platform conventions Users should not have to worry if words mean different things

in different situations (ie. “close” vs. “exit”)

Nielsen’s Heuristics (4/10)

Error prevention Best error handling is error prevention

Nielsen’s Heuristics (5/10)

Recognition rather than recall Minimize user’s memory load Make objects, actions and options visible “Preview” “Intellisense”

Nielsen’s Heuristics (6/10)

Flexibility and efficiency of use Allow users to tailor frequent actions “Accelerators”

Nielsen’s Heuristics (7/10)

Aesthetic and minimalist design Key information should be clearly presented

Nielsen’s Heuristics (8/10)

Help users recover Error messages that are user friendly Suggest how to solve the issue

Nielsen’s Heuristics (9/10)

Help and documentation Provide help and documentation to simplify tasks Video guides Tool tips

Nielsen’s Heuristics (10/10)

Graphed as a matrix

Rows are the different evaluators Columns are the heuristic measures

Data sorted to isolate easy problems together to highlight hard problems

Result of Measuring UX Knowing your users better Continuous feedback and improvement cycles Driving change that is measurable

Leading businesses to make decisions on key areas rather than just speculation

Next steps? If there is interest, gather as a group and go through design

exercises Similar set up as the Human Computer Interaction course

from Coursera

Skills Rapid prototyping (Balsamiq) Heuristic evaluation Presentation

References http://whitneyhess.com/blog/2012/05/04/the-user-is-not-like-

me/

http://dailyartifacts.com/walmarts-185-billon-dollar-mistake http://www.businessinsider.com/eye-tracking-heatmaps-2012

-5?op=1

http://www.useit.com/papers/heuristic/heuristic_evaluation.html