Interaction Models I Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development March 9, 1999

Interaction Models I

Marti Hearst (UCB SIMS)SIMS 213, UI Design &

DevelopmentMarch 9, 1999

Last Time: Heuristic Evaluation

A “discount” usability testing method UI experts inspect an interface

– prototype initially– later, the full system

Check the design against a list of design guidelines or heuristics

Present a list of problems to the UI designers and developers, ranked by severity

See homework assignment (Due Mar 18)

Outline

Windowing systems Events overview Event dispatching and handling For more information, see Olsen 98, Developing User

Interfaces

Windowing Systems

Provide infrastructure to support common UI-related services

Provide functionality for– Input/Output device handling– Which window gets which

input/output Window manager

»create and organize windows» implement interaction in those windows

Adapted from slide by James Landay

Windows Top level windows known as root

windows– provide an abstraction to separate

applications– windowing system arbitrates resources

among these Each root window belongs to an app.

– all descendant windows belong to same application

– not used by OLE (ActiveX)


Networked Windowing Systems

XWindows designed to allow applications to run on remote machines

Uses client-server model– (X reverses standand usage of client/server terminology)

X Serverstd system software

Clientapp software

Network

User


XWindows Note backwards terminology

– User is on “server” not “client” X Server

– interprets X commands and can send events– determines which window receives events

and forwards over network to proper client X Client

– software interface to X (Xlib)– assembles the output from Xlib routines into

packets for transmission to server


Events An event is something “interesting” that

happens in the system– Mouse button goes down– Item is being dragged– Keyboard button was pressed


Window Events User interacts with input device

– action translated into software events– must be distributed to the

appropriate window Events contain information on

– type– mouse position or character key– the window the event is directed to


Input Events

Mouse button events– depress/release mouse key– modifier (shift keys, etc.)– double click (X doesn’t have this fakes it)

Mouse movement events– implement painting with mouse– mouse drag

Mouse enter and exit events– e.g. if you entered / exited a button region


Button Events

Button

Button

mouseenter

mouseexit

(But using mouse move events within a buttonis unnecessary, due to the semantics of buttons.)


Events (cont.)

Keyboard events– translate raw “scan codes” into ASCII


Events (cont.) Windowing events on window

– creation / destruction– opening / closing– iconifying / deiconifying– selection / deselection– resize– redraw

» (window manager tells the application to redraw within a certain region; the application does the actual redrawing)


Widgets that can register specific kinds of eventsclose box

title bar

folder

scroll bar

size control


Major Issues

How to decompose the UI into interactive objects (widgets)?

How to distribute inputs to the interactive objects?

Contrast– Sequential (standard) program flow– Event-driven program flow

Sequential Programs

Program takes control, prompts for input

Examples include – command-line prompts (DOS, UNIX)– LISP interpreter

The user waits on the program– Program tells user it’s ready for more

input– User enters more input


Sequential Programs (cont.) The user waits on the program

– 1. Program tells user it’s ready for more input

– 2. User enters more input

– 3. Program responds and goes back to 1. But how to model the many

actions a user can take?– For example, a word processor?– Need to do editing, inserting, etc.


Event-Driven Programming Instead of the user waiting on program,

have the program wait on the user All communication from user to

computer is done via “events” An event is something “interesting” that

happens in the system– Mouse button goes down– Item is being dragged– Keyboard button was pressed


Interactor Tree Decompose interactive objects into a tree

– based on screen geometry of objects– use nested rectangles

Used for dispatching events– Events are dispatched (sent) based on code

associated with the widget– The code responds appropriately to the event

Variety of methods for dispatching events– Return to this later


Interactor Tree

Display Screen

“F:\cs160\Public” window Inner Window title bar

horizontal scroll bar contents area

“CDJukebox” folder“Home Ent…” folder…

size control …

“Web Newspaper” window…


Interactor Tree

Display Screen

Outer Win [white]

7 8 9

4 5 6

0 + -

1 2 3

=

93.54

ENT

?????


Interactor Tree

Display Screen

Outer Win [white]

Result Win [tan]Result String

Inner Win [green]

Keypad [grey]

- button+ button0 button

= button

7 8 9

4 5 6

0 + -

1 2 3

=

93.54

ENT


Main Event Loop

Main event loopInitialization

While (not time to quit) {

Get next event E

Dispatch event E

}

The meat of the program is in the code that handles the “dispatch”


Event Queues

Input events are placed in a queue– Ensures events are processed in order

Main event loop removes them from the queue (get-next-event) & dispatches for processing

Mouse move (22, 33)Mouse move (40, 30)Mouse down left (45, 34)Mouse up left (46, 35)


Event Queues (cont.)

Can use ignore unwanted events– e.g., ignore mouse moves in a forms-

based program» just get enter/exit events

– however, do want to track mouse moves in a drawing program


Event Dispatch

Dispatch (event E) { switch (E.window) { ... case FIVE-KEY:

if (E.type == left-down){ cur = 5 + 10*cur; display (cur); last-op = NUMBER; } ...

7 8 9

4 5 6

0 + -

1 2 3

=

0

ENT

Hit the ‘5’ key


Event Dispatch

Dispatch (event E) { switch (E.window) { ... case TWO-KEY:

if (E.type == left-down) { cur = 2 + 10*cur; display (cur); last-op = NUMBER; } ...

7 8 9

4 5 6

0 + -

1 2 3

=

5

ENTHit the ‘2’ key


Event Dispatch

Dispatch (event E) { switch (E.window) { ... case ENTER-KEY: if (E.type == left-down){

push (cur); cur = 0; last-op = COM; }

...

7 8 9

4 5 6

0 + -

1 2 3

=

52

ENT

Hit the ‘enter’ key


Event Dispatch

Dispatch (event E) { switch (E.window) { ... case SIX-KEY:

if (E.type == left-down) { cur = 6 + 10*cur; display (cur); last-op = NUMBER; } ...

7 8 9

4 5 6

0 + -

1 2 3

=

0

ENT

Hit the ‘6’ key


Event Dispatch

Dispatch (event E) { switch (E.window) {

...case PLUS-WIN: if (E.type == left-down){

if (last-op == NUMBER)push (cur);

result = pop() + pop();push (result);

display (result);cur = 0;last-op = COM;

}

7 8 9

4 5 6

0 + -

1 2 3

=

6

ENTHit the ‘+’ key


Dispatching Events

If user scrolls the text, the software must:– direct the mouse events to the scroll bar– update the scroll bar display during the drag– notify the text editing window it needs to

scroll itself so that the text appears to have moved


Dispatching Events (cont.) Algorithm selects the bottom-most, front-

most region in the interactor tree Called bottom-first event dispatching

– scroll bar or contents over outerwin (bottom-most)– scroll bar over contents (front-most)– Advantage:

» each window need only consider its own events» simple

– disadvantage:» difficult to impose a high level of control» inflexible


Dispatching Events (cont.) Top-down event dispatching

– events passed to top-most, front-most window

– it dispatches to one or more of its children


Event Focus Where should keyboard events go?

– mouse-based»attach mouse position to all key events

and dispatch events in the same way as mouse events

– click-to-type »send all keyboard events to last window

where mouse-down occurred


Event Focus Mouse focus

– scrollbar example: retain the focus on the scrollbar widget until the mouse button is released

– compensates for difficulty of keeping the mouse within the narrow scrollbar


Event Handling Event tables (in the early days…)

– indexed by event types (integer from 0 - 255)– hold pointers to functions that handle each event– one table per / window– lots of things to maintain when attached to a

widget that you want to make reusable Callbacks

– each kind of widget defines a set of named callbacks which will be run when the widget receives an appropriate event


Callback Example How do we notify text window to

scroll when the scroll bar is moved?– create a vertical scroll bar widget– write a callback procedure which has code

to notify text windows of their new position– register the callback as being the program

to invoke when the scroll bar is moved– register the text window as the data for the

callback» so the system knows which window to scroll

Summary

Windowing systems– software to manage and arrange windows

» keeps track of current focus

– software to support handling of user-created events

» complex GUIS are built up of hierarchically nested windows

Events– associated with various types of input

devices and actions– are handled one by one from a queue

Documents

Interaction Models I Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development March 9, 1999