1

The Benefits of Using Eye Tracking in Usability Testing

Jennifer C. RomanoUsability Laboratory

Statistical Research DivisionU.S. Census Bureau

2

Overview

• What is usability?• Introduction to eye tracking• Using eye tracking in usability testing

3

What is usability?

• Defined as “the effectiveness, efficiency, and satisfaction with which a specified set of users can achieve a specified set of tasks in a particular environment (ISO/TR 16982:2002)”

4

What is usability testing?

• Usability testing is concerned with assessing how usable a product is.– Can users interact with the product in an effective,

efficient, and satisfying way?– Evaluation at any stage of development– Iterative testing

5

Why do usability testing?

• Design often does not take the users’ mental models into account.

• Users have their own cognitive limitations and capabilities.

• Test the product on the end user.

6

Usability Evaluation Benefits

• Satisfied users• Improved data quality• Less measurement error• Reduced training time

7

Usability Testing Measures

• Effectiveness– Accuracy

• Efficiency– Time to complete a task/survey

• Satisfaction– Self-rated satisfaction

8

Introduction to Eye Tracking

Photo from: www.smith.umd.edu/behaviorlab/eyetracker.aspx

9

Introduction to Eye Tracking

• Captures where people look as they navigate through a Web site

• Shows an individual’s pathway • Accumulates data and shows common

pathways

10

Eye-Tracking Measures

• Fixations – Time to first look at an area of interest (AOI)– How often people look at AOI

• Hot spots– Areas that get most attention

• Gaze plots– Pattern of search

11

Benefits of Eye Tracking

• Identify areas that attract users’ attention or areas that are difficult to understand

• Identify how long it takes people to look at something

• Identify where people look– Where people look– Where people never look

12

Questions Eye Tracking Can Answer

• Why does it take users a long time to complete the task/survey?

• Why do users have low accuracy?• Why aren’t the users satisfied?• Are people looking at…something?• Terminology or…something else?

13

Usability Study

14

Task 1

• Task: You want to use data from two different data sources, but are unsure if they are comparable. Find out if this information exists on the site.

• Solution: Get Data or User Guide

15

Task 1: Usability Findings

– Users seemed unclear on where to find the information. Five participants (56%) selected Get Data from the center of the screen, and three (33%) selected User Guide from the top navigation.

– Of the five participants who selected Get Data, two successfully completed the task. Of the three participants who selected User Guide, only one successfully completed the task.

– Accuracy: 33%– Efficiency (correct only): 3m12s

16

17

Task 1: Hot Spot Findings

18

Task 1: Hot Spot Findings

19

Task 1: Hot Spot Findings

20

Task 2

• Task: You are unsure of the difference between ‘Places’ and ‘Metropolitan Areas.’ Where would you find this information on this site?

• Solution: Definition Geography

21

Task 2: Usability Findings

• Most participants initially scrolled through the definitions list, expecting to find the meanings of ‘Metropolitan Areas’ and ‘Places’ (which were not in the listing but rather located on the Geography page).

• Two participants never used the Geography link and instead navigated to the Site Map to find the target information.

22

Task 2: Usability Findings

• Users said that they expected the format to be the same as the format of the Definitions page.

• Accuracy: 78%• Efficiency (correct only): 4m32s

23

24

Task 2: Gaze Plot Findings

25

Task 2: Gaze Plot Findings

26

Fixation FindingsAreas of Interest (AOIs)

Participant Top nav Left nav Center Right nav

1 49s 5s 3s 7s

2 2s 3s 2s 4s

3 45s 34s 6s 20s

4 7m48s, task 3 27s 2s 27s

5 2m2s, task 2 3s 2s 1m20s

6 18s, task 2 15s, task 2 2s 16s, task 2

7 2m2s 20s 2s 2m23s

8 49s 1m50s 8s 9s

Mean 1m43s 24s 3s 34s

27

Conclusion

• Obtain measures that are not obtainable from traditional usability measures.– What people (do not) look at– How often they look there– Time into the study to look there– Quantitative data supports usability findings

28

Conclusion

• Without obtaining this type of data, it is impossible to determine where people look on the screen and how often they look there.

• It’s simply not possible to get data on where people looked, how quickly they looked, and how often they looked using only conventional experimental methods.

29

Conclusion

• Future usability testing should aim to include eye tracking as a method of collecting data.

30

Thank you!

Jennifer C. RomanoUsability LaboratoryStatistical Research DivisionU.S. Census Bureau(301)763-3577Email:Jennifer.romano@census.govLinkedIn: http://www.linkedin.com/in/jenniferromano