Descriptive Model Examples• Politics• Groupware• Keyboards• Two-handed input• Graphical input

25

Graphical Input• Considerable research on GUIs followed the successful introduction of

the Apple Macintosh in 1984– Common interactive techniques (tasks):

• pointing, dragging, selecting, inking, rubber-banding, texting– Common technologies (devices):

• mouse, trackball, touch panel, joystick, stylus, finger

• How can the tasks and devices be reconciled and understood to promote better designs?

• Buxton commented on…– “…the lack of a vocabulary that is capable of capturing salient features of

interactive techniques and technologies in such a way as to afford finding better matches between the two”1

• To address this, Buxton presented a three-state model of graphical input (next slide)

261 Buxton, W. (1990). A three-state model of graphical input. Proceedings of INTERACT '90, 449-456, Amsterdam: Elsevier.

Buxton’s Three State Model of Graphical Input

• A descriptive model• Expresses GUI interaction in terms of three states• Mouse example: (different for other devices)

– State 0 out of range– State 1 tracking– State 2 dragging

27

Newman’s State Model• Seems the idea first surfaced about 20 years earlier:

28

1 Newman, W. M., A system for interactive graphical programming, Proceedings of the Spring Joint Computer Conference of the American Federation of Information Processing - AFIP '68, (New York: ACM, 1968), 47-54.

Application of Buxton’s Model• In 1994 Apple introduced the Trackpad

on its Powerbook 500 notebook computer

• Soon after, the Trackpad (usuallycalled a touchpad) became the standard pointing device on notebook computers

• Besides physical buttons to mimic mouse buttons, a touchpad includes “lift and tap” to implement button down/up actions using touch

• But, lift-and-tap actions are error prone during a tap, if the finger moves before lifting, a dragging action is sometimes invoked, instead of a click

29

Analysis

• Let’s analyse and compare common touchpad and mouse interactions, guided by Buxton’s three-state model (next slide)

30

31

0

1

2

0

1

2

0

1

2

Sta

te

Time

Sta

te

Time

Sta

te

Time

Mouse

Click

DoubleClick

Drag

0

1

2

0

1

2

0

1

2

Sta

te

Time

Sta

te

Time

Sta

te

Time

Touchpad

32

0

1

2

0

1

2

0

1

2

0

1

2

0

1

2

0

1

2

Sta

te

Time

Sta

te

Time

Sta

te

Time

Sta

te

Time

Sta

te

Time

Sta

te

Time

Mouse Touchpad

Click

DoubleClick

Drag

Extra State Transitions

Aha… Moment• But… touchpads are capable of sensing finger pressure

(like the pressure of a finger on a mouse button)• Descriptive models aha... moments• Touchpad protocols

– Mouse emulation mode

– Native mode

33

L R X Ybuttons displacement

L R X Y Zbuttons position pressure

The Tactile Touchpad1

• Uses native x-y-z mode (z = finger pressure)• Implements button down (state 2) by “pressing harder”• Button click feedback provided by relay below touchpad• Design guided by Buxton’s three-state model

341 MacKenzie, I. S., & Oniszczak, A. (1997). The tactile touchpad. Extended Abstracts of the ACM SIGCHI Conference on Human Factors in Computing Systems - CHI '97, 309-310, New York: ACM.

Demo

35http://www.youtube.com/watch?v=fxfu-Yo6yEk

Three-state Touch Products

36

See the link below for press describing these product’s use of three-state interaction, with reference to the tactile touchpad

http://www.touchusability.org/2008/10/LINK BROKEN

Blackberry Storm (2008) Apple Macbook (2008)

[click]

Post Script• Buxton’s three-state model remains widely used in human-

computer interaction• Google Scholar returns 101 citations since 2014 to

Buxton’s 1990 paper (“A three state model of…”)• Contemporary applications include

– Models for preview and undo– Puck and stylus input for two-handed interaction– Docking tasks for tabletop displays– Camera control for navigating animated scenes– Modeling multi-touch on touchscreens– Modeling panning and zooming on touchscreens– Modeling selection of moving targets– Modeling the rotation mode of a 3 DOF mouse

37

See HCI:ERPfor citations

See HCI:ERPfor citations

Frame Model of Visual Attention1,2

38

Point frame … requires the greatest demand in visual attention . requires sharp central vision … demand on visual attention is high … Examples: selecting a thin line or very pixel target

Target frame … selecting targets such as icons, toolbar buttons, or keys on a soft keyboard ... Visual attention needed, but with less demand … slightly less precision and attention are needed

Surface frame … flicks, pinches, most forms of gestural input … user needs a general spatial sense of the surface ... visual demand is minimal; peripheral vision sufficient

Environment frame … includes user surroundings ... frame of reference: the user, the device, the environment. ... peripheral vision only … interactions involving accelerometer or camera

1 MacKenzie, I. S., & Castellucci, S. J. (2012). Reducing visual demand for gestural text input on touchscreen devices. CHI 2012, pp. 2585-2590. New York: ACM.2 MacKenzie, I. S., & Castellucci, S. J. (2013). Eye on the message: Reducing attention demand for touch-based text entry. Int J Virtual Worlds and HCI, 1, 1-9.

Critiquing the Model (Insight)

39

MoreVisual Attention

LessVisual Attention

Circumplex Model of Emotion1

40

ACTIVATION

DEACTIVATION

PLEASANTUNPLEASANT

tensenervous

stressed

upset

sad

depressed

lethargic

fatigued

alertexcited

elated

happy

contented

serene

relaxed

calm

1Kim, J., and Andre, E. (2009) Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 2067-2083.

Predictive Models

41

Predictive Models• A predictive model is an equation• Predicts the outcome on a criterion variable (aka

dependent variable or human response) based on the value of one or more predictor variables (aka independent variables)

• Note: the predictor variables must be ratio-scale attributes (See HCI:ERP for discussion)

• Predictive models, like descriptive models, allow a problem space to be explored

• However, predictive models deal with numbers, not concepts

42

Why Use Predictive Models• Card et al. presented perhaps the first predictive model in

HCI.1 In many respects, their work was straight-forward experimental research; but they went further:– “While these empirical results are of direct use in selecting a pointing

device, it would obviously be of greater benefit if a theoretical account of the results could be made. For one thing, the need for some experiments might be obviated; for another, ways of improving pointing performance might be suggested.”

• This is a call for the use of predictive models in HCI• They went on to present predictive models using Fitts’ law

(which we meet shortly)

431 Card, S. K., English, W. K., & Burr, B. J. (1978). Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics, 21, 601-613.

Predictive Model Examples• Linear prediction equation• Fitts’ law• Choice reaction time• Keystroke-level model (KLM)• Skill acquisition• More than one predictor

44

Linear Prediction Equation• The basic prediction equation expresses a linear

relationship between a predictor variable (x) and a criterion variable (y):

45

Linear Regression• A linear prediction equation is built using a statistical

procedure know as linear regression• Goal:

– Given a set of x-y sample points, find the coefficients m and b(previous slide) for the line that minimizes the squared distances (least squares) of the points from the line

• The result is a prediction equation that gives the best estimate of y in terms of x

• The assumption, of course, is that the relationship is linear• Want the details? Just enter “linear regression” or “least

squares” into Google or Wikipedia

46

Example• A research project investigated text entry on soft

keyboards1

• The research also asked…– Can stylus tapping entry speed be predicted from touch typing

entry speed?

• Touch typing speed is the predictor variable (x - measured in a pre-test)

• Stylus typing speed is the criterion variable (y - measured experimentally)

• Data and scatter plot

471 MacKenzie, I. S., & Zhang, S. X. (2001). An empirical investigation of the novice experience with soft keyboards. Behaviour & Information Technology, 20, 411-418.

Data and Scatter Plot

48

There seems to be a relationship: Faster touch typists seem to be faster at stylus tapping.

Questions:What is the prediction equation?How strong is the relationship?

Prediction Equation

49

Prediction equationPrediction equation

Squared correlationSquared correlation

Note:The prediction equation explains 27% of the variation in the data – a modest predictor, at best.

Note:The prediction equation explains 27% of the variation in the data – a modest predictor, at best.

Best-fitting lineBest-fitting line

95% confidence interval95% confidence interval