Chapter 15 · 2013. 11. 25. · Fitts’ Law (Fitts, 1954) •Fitts’ Law predicts that the time to point at an object using a device is a function of the distance from the target

©2011 1 www.id-book.com

Analytical evaluation

Chapter 15


Aims:

• Describe the key concepts associated with inspection methods.

• Explain how to do heuristic evaluation and walkthroughs.

• Explain the role of analytics in evaluation.

• Describe how to perform two types of predictive methods, GOMS and Fitts’ Law.


Inspections

• Several kinds.

• Experts use their knowledge of users & technology to review software usability.

• Expert critiques (crits) can be formal or informal reports.

• Heuristic evaluation is a review guided by a set of heuristics.

• Walkthroughs involve stepping through a pre-planned scenario noting potential problems.


Heuristic evaluation

• Developed Jacob Nielsen in the early 1990s.

• Based on heuristics distilled from an empirical analysis of 249 usability problems.

• These heuristics have been revised for current technology.

• Heuristics being developed for mobile devices, wearables, virtual worlds, etc.

• Design guidelines form a basis for developing heuristics.


Nielsen’s original heuristics

• Visibility of system status.

• Match between system and real world.

• User control and freedom.

• Consistency and standards.

• Error prevention.

• Recognition rather than recall.

• Flexibility and efficiency of use.

• Aesthetic and minimalist design.

• Help users recognize, diagnose, recover from errors.

• Help and documentation.


Discount evaluation

• Heuristic evaluation is referred to as discount evaluation when 5 evaluators are used.

• Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability problems.


No. of evaluators & problems


3 stages for doing heuristic evaluation

• Briefing session to tell experts what to do.

• Evaluation period of 1-2 hours in which:

– Each expert works separately;

– Take one pass to get a feel for the product;

– Take a second pass to focus on specific features.

• Debriefing session in which experts work together to prioritize problems.


Advantages and problems

• Few ethical & practical issues to consider because users not involved.

• Can be difficult & expensive to find experts.

• Best experts have knowledge of application domain & users.

• Biggest problems: – Important problems may get missed;

– Many trivial problems are often identified;

– Experts have biases.


Heuristics for websites focus on key criteria (Budd, 2007)

• Clarity

• Minimize unnecessary complexity & cognitive load

• Provide users with context

• Promote positive & pleasurable user experience


Cognitive walkthroughs

• Focus on ease of learning.

• Designer presents an aspect of the design & usage scenarios.

• Expert is told the assumptions about user population, context of use, task details.

• One or more experts walk through the design prototype with the scenario.

• Experts are guided by 3 questions.


The 3 questions

• Will the correct action be sufficiently evident to the user?

• Will the user notice that the correct action is available?

• Will the user associate and interpret the response from the action correctly? As the experts work through the scenario they note problems.


Pluralistic walkthrough

• Variation on the cognitive walkthrough theme.

• Performed by a carefully managed team.

• The panel of experts begins by working separately.

• Then there is managed discussion that leads to agreed decisions.

• The approach lends itself well to participatory design.


A project for you …

http://www.id-book.com/catherb/

• provides heuristics and a template so that you can evaluate different kinds of systems.

• More information about this is provided in the interactivities section of the id-book.com website.


Analytics

• A method for evaluating user traffic through a system or part of a system

• Many examples including Google Analytics, Visistat (shown below)

• Times of day & visitor IP addresses


Social action analysis (Perer & Shneiderman, 2008)


Predictive models

• Provide a way of evaluating products or designs without directly involving users.

• Less expensive than user testing.

• Usefulness limited to systems with predictable tasks - e.g., telephone answering systems, mobiles, cell phones, etc.

• Based on expert error-free behavior.


GOMS • Goals – what the user wants to achieve

eg. find a website.

• Operators - the cognitive processes & physical actions needed to attain goals, eg. decide which search engine to use.

• Methods - the procedures to accomplish the goals, eg. drag mouse over field, type in keywords, press the go button.

• Selection rules - decide which method to select when there is more than one.


Keystroke level model

• GOMS has also been developed to provide a quantitative model - the keystroke level model.

• The keystroke model allows predictions to be made about how long it takes an expert user to perform a task.


Response times for keystroke level operators (Card et al., 1983)

Operator Description Time (sec)

K Pressing a single key or buttonAverage skilled typist (55 wpm)Average non-skilled typist (40 wpm)Pressing shift or control keyTypist unfamiliar with the keyboard

0.220.280.081.20

P

P1

Pointing with a mouse or other device on adisplay to select an object.This value is derived from Fitts’ Law which isdiscussed below.Clicking the mouse or similar device

0.40

0.20

H Bring ‘home’ hands on the keyboard or otherdevice

0.40

M Mentally prepare/respond 1.35

R(t) The response time is counted only if it causesthe user to wait.

t


Summing together


Using KLM to calculate time to change gaze (Holleis et al., 2007)


Fitts’ Law (Fitts, 1954)

• Fitts’ Law predicts that the time to point at an object using a device is a function of the distance from the target object & the object’s size.

• The further away & the smaller the object, the longer the time to locate it & point to it.

• Fitts’ Law is useful for evaluating systems for which the time to locate an object is important, e.g., a cell phone, a handheld devices.


A project for you …

• Use the web & other resources to research claims that heuristic evaluation often identifies problems that are not serious & may not even be problems.

• Decide whether you agree or disagree.

• Write a brief statement arguing your position.

• Provide practical evidence & evidence from the literature to support your position.


A Project for you …Fitts’ Law

Visit Tog’s website and do Tog’s quiz, designed to give you fitts!

http://www.asktog.com/columns/022DesignedToGiveFitts.html


Key points

• Inspections can be used to evaluate requirements, mockups, functional prototypes, or systems.

• User testing & heuristic evaluation may reveal different usability problems.

• Walkthroughs are focused so are suitable for evaluating small parts of a product.

• Analytics involves collecting data about users activity on a website or product

• The GOMS and KLM models and Fitts’ Law can be used to predict expert, error-free performance for certain kinds of tasks.

Documents

Chapter 15 · 2013. 11. 25. · Fitts’ Law (Fitts, 1954) •Fitts’ Law predicts that the time to point at an object using a device is a function of the distance from the target