Usability and Accessibility Lecture 14 – 09/04/10

© Simeon Keates 2010

Usability and AccessibilityLecture 14 – 09/04/10Dr. Simeon Keates


Exercise – Part 1

Last week you were asked to prepare your user trial protocols Today – put them into practice

Perform a pilot study of the usability of your web-site with at least 1 user

Remember – the principal aim is to “test the test” • (or “trial the trial” or “evaluate the evaluation”…)

Page 2


Exercise – Part 2

Prepare a progress presentation for the board for Friday Show that good progress is being made

Summarise:• The tasks performed • The data collected• Whether the user liked the site• Whether the user could use the site (e.g. complete the tasks)• What you think is working well in the design• What you think needs to be looked at more closely in the design• Any changes you would like to make to the site and protocol

Page 3


Exercise - Practicalities

Remember to print out copies of your protocol

Allow plenty of blank space for adding observation notes

Allocate one person to do the pre-session briefing and debrief

Allocate one person to be the facilitator (the person who directs the user)

The remaining members act as observers

Page 4


Cognitive models

Page 5


The Power Law of Practice

Tn = T1 n-α

α = 0.4, T1 = 60s, T2 = 45.5s (24% faster), T10 = 23.9s (60%faster)

Page 6


Motor skills – Fitts’ Law

A person wishes to hit this target:

We know that a correction cycle takes:

τp + τc + τm≈ 240 ms

And so n corrections takes n * 240 ms

Page 7

Startx0 x1 x2

S

D


Fitts’ Law

Now let xi be the remaining distance after the i-th correction

And let x0 (= D) be the starting point

We will assume that the relative accuracy of movement is constant, i.e.:

Where ε < 1 and is the constant error

On 1st cycle: x1 = ε x0 = ε D

On 2nd cycle: x2 = ε x1 = ε (ε D) = ε2 D

On n-th cycle: xn = εn D

Process stops when: εn D ≤ ½ S

Solving for n gives:

Page 8

€

x ix i−1

= ε

€

n = −log2(2D /S)

log2 ε


Fitts’ Law

From:

Total movement time, Tpos is given by:

This can be re-written as:

Where:

ε has been found to be ~ 0.7

Thus IM ≈ -240 / log2(0.7) = 63 ms/bit [27~122 ms/bit]

Page 9

€

n = −log2(2D /S)

log2 ε

€

Tpos = n(τ p + τ c + τ m )

€

Tpos = IM log2(2D /S)

€

IM =−(τ p + τ c + τ m )

log2 ε

Fitts’ Law


Fitts’ Law corrections

There are several modifications to Fitts’ Law

Fitt’s Law becomes less accurate for low values of log2(2D / S)

i.e. where the target is quite big compared with the distance

An example correction by Welford (1968):

Page 10

€

Tpos = IM log2(D S + 0.5)


Fitts’ Law – Implications for web-site design

Long, thin targets are not good• Small S value => longer acquisition times

Example of long, thin target:• Text-only hyperlinks• e.g. Heinz tomato ketchup

Better to include something large• e.g. an image of a ketchup bottle…

Page 11


Merging the models

One basic merged model is the Keystroke Level Model (KLM):

Texecute = TK + TP + TH + TD + TM + TR

Where TK = total time spent keystroking = nk tk (# * time per stroke)

• Time per stroke determined experimentally

TP = total time spent pointing (from Fitts’ Law)

• Assume, say, 1.1 s per pointing action

TH = total time spent homing (moving hands between devices)

• Assume 0.4 s per homing

TD = total time spent drawing = tD (nD, lD) (i.e. f(#, total length))

• Example: 0.9nD + 0.16lD

TM = total time to mentally prepare

• Assume 1.35 s per preparation

TR = total system response timePage 12


Using the KLM

[Note: M = mental prep, K = keyboard, P = pointing] Rule 0: Insert Ms in front of all Ks that are not part of argument strings

proper. Place Ms in front of all Ps that select commands Rule 1: If an operator following an M is fully anticipated in an operator

just previous to M, then delete the M (e.g. PMK -> PK) Rule 2: If a string of MKs belongs to a cognitive unit (e.g. name of a

command), then delete all Ms but the first one Rule 3: If a K is a redundant terminator (e.g. terminates a command

immediately following the terminator of its argument), then delete the M in front of it

Rule 4: If a K terminates a constant string (e.g. a command name), then delete the M in front of it, but if the K terminates a variable string (e.g. an argument string) then keep the M in front of it

Page 13


An more generic approach - GOMS

The user’s cognitive structure consists of: A set of Goals A set of Operators A set of Methods A set of Selection rules

Page 14


GOMS – a quick breakdown

Goals: Symbolic structures that define a state of affairs to be achieved• Examples: GOAL: EDIT-MANUSCRIPT or GOAL: MODIFY-TEXT• Goals can comprise sub-goals

Operators: Elementary perceptual, motor or cognitive acts whose execution is

necessary to change any aspect of the user’s mental state or to affect the task environment• Examples: GET-NEXT-PAGE or GET-NEXT-TASK

Page 15


GOMS – a quick breakdown

Methods: Procedures for accomplishing a goal – must be pre-learned at

performance time (i.e. user already knows them)• Contain sets of Operators

Selection rules: Rules for helping the user decide which method to use to accomplish the

goal• Example: if_such_and_such_is_true_then_use_method_M1_else_use_M2

To summarise: Several Operators make up a Method, and Selection rules are used to determine the best Method to reach the Goal

Page 16


Using models of interaction

Fundamentally, you need to perform a comprehensive task analysis

The models indicate suggested performance for each sub-task

Those models help you to predict the performance of the interface

This can be used:• In design: Estimate performance using standard parameters to optimise your

design• In usability trials: Estimate the performance and compare with actual

observed data – investigate significant discrepancies

Page 17


Extending to Universal Access applications – The MHP

Page 18


Experiment I – Testing the MHP for motor-impaired users

Page 19

User Description

PJ3 Tetraplegia (from head injury)

PJ4 Muscular Dystrophy

PJ5 Spastic Quadriplegia Cerebral Palsy

PJ6 Athetoid Cerebral Palsy

PJ7 Friedrich’s Ataxia

PJ8 Athetoid Cerebral Palsy


Perceptual response times

Page 20

User Group τp Delay(ms)

τp Smooth(ms)

Card, Moran & Newell: Able-bodied

100[50~200]

100[50~200]

Cambridge:Able-bodied

80[70~100]

80[70~90]

Cambridge:Motion-impaired

100[70~120]

100[70~120]


Cognitive cycle times

Page 21

User Group τc (ms)


70[25~170]


90[90~100]


110[100~130]


Motor function times

Page 22

User Group τm (ms)


70[30~100]


70[60~80]


210[100~310]


Reaction times

Page 23

User Group Predicted(ms)

Observed(ms)


310[100+70+(2*70)]

-


310[80+90+(2*70)]

320


630[100+110+(2*210)]

620


Explaining the observed motor times (100-310 ms)

Theoretical interaction is:• Press the button (motor function)• Release button (motor function)

Consequently, either • very slow motor function times

or• extra steps being inserted

Page 24


Identifying the delays

c & p calculated as for Experiment I

m button-down and button-up times separated

Motor function and reaction time tasks performed Range of input devices used• mouse• touchpad button• space bar• EasyBall

Page 25


The MS EasyBall

Page 26


User descriptions

Page 27

User Description τp (ms)

τc (ms)

PI3 Athetoid Cerebral Palsy

100 120


100 100


90 110

PI7 Friedrich’s Ataxia 90 110


Results

Page 28

User Input Device

Button down Button up

Mean time (ms)

SD Mean time (ms)

SD

PI3 Mouse 230 35 210 34

PI5 MouseSpace barTrackpad

1708090

425

14

150115100

262219

PI6 MouseSpace barTrackpad

1008080

281417

90110110

231320

PI7 MouseSpace barTrackpadEasyBall

210120200220

71394367

180240230310

43865880


Results

Page 29


Results

Page 30

τm

?


PI7 results – Motor function task

Page 31

0%

5%

10%

15%

20%

25%

30%0-

25

50-7

5

100-

125

150-

175

200-

225

250-

275

300-

325

350-

375

400-

425

450-

475

Time (ms)

Fre

qu

en

cy o

f o

ccu

ren

ce (

%) Button-

down

Button-up

© Simeon Keates 2010BackgroundPage 32

The MHP results

0

50

100

150

200

250

300

350

Mo

tor

resp

on

se t

ime

(m

s)

A B C D E PI4 PI6 PI3 PI7 PI5 PI8

User

Able-bodied

Motion-impaired

~c


Conclusions

Extra cognitive cycles are being inserted Interaction process is:

• Decide to press button (cognitive) - OPTIONAL• Press the button (motor) - REQUIRED• Decide to release button (cognitive) - OPTIONAL• Release button (motor) - REQUIRED

Page 33


Sources of extra cognitive steps

Users always in learning mode?

Users being overly careful?

Extra cognitive load from impairment?

Page 34


Implications for use of user models

Individual components were comparable

However• method of combination was not

Therefore• need to verify user model assumptions before use

Page 35


Implications for design

Users “add” own extra cognitive load

Need to support users by:• Minimising user uncertainty• Minimising cognitive load from program• Maximising interface intuitiveness• Maximising useful feedback

Page 36


Extending to Universal Access applications – Cursor control

Page 37


Symptoms associated with ageing and Parkinson’s

Symptoms: Essential tremor Restricted motion Reduced strength Poor hand-eye co-ordination Fatigue

Page 38


Examples of motor-impaired cursor control

0

100

200

300

400

500

600

700

0 200 400 600 800 1000

Page 39


Cursor movement theories

Fitts’ Law• Relates target distance and width to time

Movement Optimization Model• Initial, pre-planned ballistic move• (Optional) Secondary corrective submovements• Submovements based on visual feedback

Analysis of movement paths• Describes effect of changes in distance, width and height of target• Longer distances => higher peak velocity• Smaller target => longer deceleration phase

Initial studies [Hwang et al., 2004] suggest NOT universally applicable

Page 40


Cursor measures(MacKenzie et al - CHI 2001)

Target Re-Entry (TRE)

Task Axis Crossing (TAC)

Movement Direction Change (MDC)

Orthogonal Direction Change (ODC)

Page 41


Cursor measures (cont.)

Movement Offset (MO)• mean deviation of points from task axis ( y )• signed

Movement Error (ME)• average deviation of points from task axis• unsigned

Movement Variability (MV)• standard deviation of points from task axis

Missed Click (MCL)

Path Length / Task Axis Length (PL/TA)

Additional measures

Page 42


Cursor measures (cont.)

Can distinguish between motor impaired and able-bodied users• As “groups”• Keates et al. ASSETS 2002

Can they do more?

Designed to explain why differences exist

Page 43


User trials - The users

4 groups of users

• IBM interns (Y) – mean age 23, SD = 2.0• IBM regulars (A) – mean age 47, SD= 9.4• Older adults (OA) – mean age 79, SD = 4.5• APDA members (P) – mean age 57, SD = 5.2

6 users per group

Page 44


User trials - The experimental methodology

Fitts’ Law type task• 3 target sizes• 3 target distances• 36 target acquisitions per target session

• 4 of each size/distance combination

• Random angle of approach to target

4 target sessions per user session (144 target acquisitions)

Interviews between each target session

Post-session debrief

Page 45


User trials – Qualitative results

21 difficulties reported with mouse use, e.g.:• Keeping hand steady when navigating• Slipping off menus• Losing the cursor• Moving in the desired direction• Running out of room on the mouse pad• Mouse ball getting stuck (and/or dirty)

12 compensatory strategies, e.g.:• Avoid use of menus• Switch hands• Consciously go slower• Pause before clicking

Page 46

© Simeon Keates 2010Page 47

User trials – Quantitative results

0

0.5

1

1.5

2

2.5

3

3.5

Group OA Group P Group Y Group A

Target activation times

0

20

40

60

80

100

120


No. of incorrect clicks

Peak velocities

0

1

2

3

4

5



User trials – Cursor measures (cont.)

Cursor measure Group OA Group P Group Y Group A p()

Path length / task axis length (PL/TA)

1.79 1.21 1.38 1.41 0.036

Missed clicks (MCL) 0.13 0.04 0.06 0.02 0.010

Task axis crossings (TAC) 3.32 2.55 2.71 2.49 0.018

Target re-entries (TRE) 0.88 0.46 0.58 0.59 0.913

Movement direction changes (MDC)

11.94 10.80 8.40 7.48 0.267

Orthogonal direction changes (ODC)

9.34 4.61 4.36 4.46 0.006

Movement error (ME) 33.00 21.40 24.61 28.78 0.152

Movement offset (MO) 26.74 18.22 20.76 24.34 0.258

Movement variability (MV) 30.53 18.93 23.31 27.45 0.200

Page 48


User trials – Nature of movement observed

Differences in peak velocity do not explain all of target activation time differences

Theory: Target user movements are like able-bodied movements only more of them needed to complete the task

00.5

11.5

22.5

33.5

44.5

5


No. of pauses >100 msec

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6


No. of pauses >250 msec


User trials – Nature of movement observed

Submovements can distinguish between user groups (p<0.01)• Submovements are significantly related to:

• Path length / task axis length (PL/TA)• Missed/incorrect clicks (MCL)• Task axis crossings (TAC)• Target re-entries (TRE)• Movement direction changes (MDC)• Orthogonal direction changes (ODC)

• Submovement not significantly related to:• Movement error (ME)• Movement offset (MO)• Movemenet variability (MV)

Cumulative measures

Normalised measures

Page 50


User trials – Pauses and cursor measures

Cursor measure # of 100 msec pauses

# of 250 msec pauses

Path length / task axis length (PL/TA) 1% 5%

Missed clicks (MCL) 1% 5%

Task axis crossings (TAC) 1% n.s.

Target re-entries (TRE) 1% 1%

Movement direction changes (MDC) 1% 1%

Orthogonal direction changes (ODC) 1% 1%

Movement error (ME) n.s. n.s.

Movement offset (MO) n.s. n.s.

Movement variability (MV) n.s. n.s.

Page 51


User Trials – Where the pauses occur

0

50

100

150

200

250

300

3505

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

10

0

% of movement

No

. of

pa

us

es

Young

Adult

Senior

PD

Page 52


User trials – Verification pauses

0

50

100

150

200

250

300

475 425 375 325 275 225 175 125

msec before mouse down

No

. of

pa

us

es

Young

Adult

Senior

PD

Page 53


User trials – Peak velocities

Peak velocity & Distance to target – strong +ve correlation Peak velocity & Target size – weak –ve correlation Peak velocity & User group:

0

1

2

3

4

5


Page 54


User trials – Incorrect click results

No. of incorrect clicks as target session progresses No. of incorrect clicks as target size No. of incorrect clicks is not related to target distance No. of incorrect clicks is related to user group:

0

20

40

60

80

100

120


Page 55


User trials – Incorrect click results (cont.)

Question – Can we tell the difference between correct and incorrect clicks?

No significant difference in:• Pause before• Duration of press

Significant difference in:• Distance moved while button pressed • Number of events while button pressed

Note: these are across ALL incorrect clicks• Across user groups• Across types of incorrect clicks

Page 56


User trials – Observed types of incorrect clicks

210 incorrect clicks observed Near misses – within 50% of target radius (110) Not-so-near misses – >50% and <100% of target radius (35) Slipped clicks (32) Accidental clicks – >200% of target radius (9) Two button press (5)• User presses left button• While button down, user also presses right button• User releases right button, then left button• Not predicted!

Middle button press (2) Unclear (17)

Page 57


Conclusions – Movement behaviour of older adults

Lower peak velocities• Do not explain all of longer target acquisition times

Increased # of pauses• Lack of confidence? Lack of expertise?

Most likely – difficulty getting on to target• High correlation of pauses with TRE, MDC and ODC• # of pauses towards (but not only at) end of movement

Do not follow single large ballistic / single homing phase Many (smaller) submovements towards target

Page 58


Conclusions – Movement behaviour of users with PD

Many measures consistent with age of group• Which is more dominant – effects of age or PD?

Initiating movement can be difficult• “Bump” at start of pauses graph• Shared feature with older adults

Tend to be very deliberate• Lowest peak velocity• Second lowest error rate

Slight movement of mouse when pressing button• Multiple pauses in last half second of button press• Increased verification pause time

Page 59


Summarising the differences

Younger adults (IBM interns)• Shortest (1), fastest (1), more errors (3) - slapdash

• “I can fix it”

• Games culture?

Adults (IBM regulars)• Shorter (2), faster (2), fewest errors (1)

• Best compromise between speed and accuracy?

Parkinson’s users• Longer (3), slowest (4), fewer errors (2)

• Slow, but sure

Older adults• Longest (4), slower (3), most errors (4)

• Lack of expertise

• Difficulty acquiring target


Exercise

Page 61


Exercise

On Tuesday(-ish) you performed a pilot study

Today, make any changes you identified to your usability protocol

Also, make any changes to your web-site based on the feedback that you obtained

Please mail your finalised protocols to Susanne and me

Page 62

Documents

Usability and Accessibility Lecture 14 – 09/04/10