Visual Thinking and Visual Thinking Tools:

Space, Time and Simple Cognitive Models to Support Design

Colin WareData Visualization Research Lab, CCOM,

University of New Hampshire

Architecture for visual thinking

Central Problem: How do we perceivethe world in all its rich detail?

The Nature of Visual Space

Capacity of visual working memory (Vogal, Woodman, Luck, 2001) Task – change detection Can see 3.3 objects Each object can be complex

1 second

Sequential comparison task

Solution “The world is its own memory” O’Regan Task-related active vision “What you see is what you need”

Treish et al. (2003)

Seeing is a process that helps us solve problems

Task-related eye movementsHayhoe and Ballard, 2005

Example 2: How to get focus and context?

Zooming (Bedersen) Linked windows

(Fowler & Ware) Fisheye (Furnas,

Carpendale)

Zooming Vs Multiple Windows (Matt Plumlee)

Problem: When do we need extra windows?

Comparing parts of a visual scene.

2 solutions: Zooming, multiple windows

Task: searching for target patterns that match

Zooming vsWindows + eye movements

1,3,5,7 items per cluster

Conditons:

Cognitive Model (grossly simplified)

Time = setup cost +

number of “visits” x time per visit

Number of visits is a function of number of objects to be compared and visual working memory capacity.

Visits = n/M

Prediction Results

As targets (and visual working memory load) increases, multiple

Windows become more attractive.

Design heuristic

When we need to compare more two or three simple pattern components add windows.

Example 2

Tools for finding new underwater behaviors from humpback whale tag data

(Why turning time into space is a good idea)

BigEyes

Antenna

The gear

DTAG Mark Johnson

Dave Wiley

Task: find new behaviors

= stereotyped patterns

Cognitive Algorithm repeat

Review behavior sequence looking for patterns. Remember patterns.

Look for more instances.

until no new patterns

The old way

Solution 1. GeoZui 4D

Cognitive process for finding new behaviors

stereotyped patternsCognitive Algorithm repeat

Review behavior sequence looking for patterns by playback. Remember patterns using space-time notes.

Look for more instances. May involve reviewing all other whale tracks.

Until no new patterns Cost k*playback time.

Solution 2: trackplot

Foraging patterns

Traversing

04

06 06

07

2006Mostly

Process for finding new behaviors

stereotyped patternsCognitive algorithm Get to a good viewpoint repeat

Review behavior sequence looking for patterns eye movements. Remember patterns using visual working memory.

Look for more instances. May involve reviewing all other whale tracks. Can be posted on the wall

until no new patterns Cost Nav + Eye Movement time *pattern

matching.

Gain in efficiency – from playback tool to pattern finding tool

Many hours (with playback) A few minutes (with patterns)

Approximately a factor of 100

Design heuristic

Whenever possible: Turn time into a spatial pattern – one that converts critical events into shapes or patterns

Try to make natural mappings – proper use of texture color, etc.

How to get perceptual and cognitive principles into the designer’s head?

OverviewZoom&filterDetails on Demand

ACT-R

Shneiderman

Anderson

ME GraphConstellation

Activities at all levels

Example 1: Network diagram

Where are we going?

Simple cognitive process models involving Perceptual and cognitive operations Interaction methods

For Design

AcknowlegementsNSF ARDA NOAA ONR

David Wiley, Rusty Bobrow, Matt Plumlee, Anne Gilman

Where are we going?

DESIGN of visual thinking tools

We need to understand perception and the cognitive process

The machinery

DisplayFeatures

Proto-objects andPatterns

VisualWorkingMemory

GIST

VisualQuery

VerbalWorkingMemory

Egocentric object andPattern map

OBJECTFILES“Nexus”

a

AB

C

D

Change Blindness

Simons and Levin

Follow up study with eye trackerMulti-window condition only.

Similar result in Gajewski and Henderson, 2005

One eye movement (from cluster to cluster) per visual object

Sketching ideas:Many people have noted that we are cognitive cyborgs: Edwin Hutchins, Herbert Simon, Don Norman.. Etc.

The query process

Tuning the visual system