Exploratory Visual Analysis in Large High-Resolution Displays

Exploratory Visual Analysis in Large High-Resolution Display Environments

Khairi Reda

• Society is generating data at ever escalating rates

• The rise of the supercomputer

• Advances in data collection instruments and techniques (e.g., high-throughput genome sequencing)

• We can now afford to capture and store larger quantities of information, in the hope that we can learn something unexpected from it

!2background

background!3[Tukey and Wilk 66, Tukey 77, Tukey 80]

• Confirm / disconfirm a particular hypothesis

!• Typically involves statistical

tests and computation modeling

!• Perfect for computers

!!!

Confirmatory analysis

• Look for unexpected patterns and outliers

!• Ask new questions!

!• Not a fit task for computers

!!!!

Exploratory analysis

Hypothesis-driven inquiry

Data-driven inquiry






!!!





!!!!



Data-driven inquiry






!!!





!!!!



Data-driven inquiry

New perceptual aids are needed to facilitate the exploration of “big data” sources

• Cost of digital displays is decreasing rapidly

• Large high-resolution displays are becoming more affordable

• Deployed in real-world scientific settings

• Collaboration between co-located teams

• Visual analysis and exploration of large-scale datasets

Nanoscale materials science

Earth sciences

Ecology and behavioral

biologybackground[Reda et al. 13b, Febretti et al. 13, Reda et al 13c] !4

Could these display improve the rate and quality of insight during exploratory visual analysis? And how/why??

!5

information space

visual exploration

!5

information space

visual exploration

!5

information space

visual exploration

!5

information space

+ + =?

visual exploration

!6visual exploration

temporal separation


temporal separation spatial separation (i.e., juxtaposition)



?Aha!



1. What is the effect of increasing the physical size and resolution of the display on user behavior, during visual exploration?

2. And on insight acquisition?

3. Are there new design patterns for exploratory multi-view visualizations on large high-resolution displays?

?Aha!

related work

!8

Low-level visualization tasks (e.g., find the state with the highest population growth between 1960 and 1990)

Task completion

time

Data sizeStr

ictly

linea

r sca

ling

User performance on a

big display

[Yost and North 06, Yost et al. 07, Ball et al. 07]related work

!9related work [Andrews et al. 10, Andrews and North 13]

!9related work [Andrews et al. 10, Andrews and North 13]

theory !10

theory !10

View1

display

View2

View3

View4

theory !10

View1

display

View2

View3

View4time

sequentialaccess to

information

theory !10

View1

display

View2

View3

View4

View1 View2

View3 View4

display

time

sequentialaccess to

information

theory !10

View1

display

View2

View3

View4

View1 View2

View3 View4

display

time

sequentialaccess to

information

embodied, non-sequential

access to information

theory !10

View1

display

View2

View3

View4

View1 View2

View3 View4

display

time

sequentialaccess to

information



Cost (Temporal-separation) Cost (Spatial-separation)

theory !10

View1

display

View2

View3

View4

View1 View2

View3 View4

display

time

sequentialaccess to

information



Cost (Temporal-separation) Cost (Spatial-separation)=>

<

!11[Norman 02]

seven stages of

action

Goal

System

Intention

Action

Execution Perception

Interpretation

Evaluation

Cost ~ Usability: the ease and learnability of a human-made object

theory

!12[Lam 08]

Cost (temporal-separation)

theory

Form goal!

Visualization tool

Form physical sequence!

Execute physical sequence

Form system operators!

Perceive state

framework of interaction

costs in visualizations

Evaluate interpretation

Interpret perception

!12[Lam 08]


theory





!12[Lam 08]


theory




Interpret perceptionTemporal-view association

!12[Lam 08]


theory




Interpret perceptionTemporal-view association

Mental map (re)-building[Purchase et al. 07]

Cognitive integration[Ratwani et al. 08, Plumlee & Ware 06 ]

}

!12[Lam 08]


theory

Form goal!

Visualization tool




Perceive state





!12[Lam 08]


theory “cognitive resistance”

Form goal!

Visualization tool




Perceive state





!13[Lam 08]

Perceive stateExecute physical sequence

Visualization tool


Form goal!




Cost (spatial-separation)

theory

!13[Lam 08]


Visualization tool


theory

!13[Lam 08]


Visualization tool


theory

Potentially strenuous physical navigation

(head turns, walking, leaning)

!13[Lam 08]


Visualization tool


theory

Managing attention in a more complex environment

Visually resolve a more cluttered visualization

Potentially strenuous physical navigation

(head turns, walking, leaning)

!14

• Study I: exploratory case study

• Comparative visual analysis of ensemble datasets on large displays

• Real-world application + domain expert (behavioral ecology)

!

• Study II: comparative study

• Investigate the effects of increasing the size/resolution of the display on user behavior and insight acquisition

• Simulated analysis task on a real-world dataset

!15

to nestto food

off-trail navigation?

Understanding the navigational strategies of Seed harvester ants

case study [Offord et al. 13]

!15

to nestto food

off-trail navigation?

Understanding the navigational strategies of Seed harvester ants

case study [Offord et al. 13]

!16case study

!16case study

!16case study

On-trail ants West side ants East North south

!16case study

On-trail ants West side ants East North south

design pattern: query-by-example brush

!17

Findings: user strategy

case study

• Think-aloud protocol

• Verbal protocol analysis

• Coding scheme: Observations, Hypotheses, and Decisions

• Interactions: Query by example, Workspace management

Workspacemanagement

Query by example

Hypothesis formulation

Decision making

Observing outliers

layout-preserving interactions

temporal separation of views

!18

Study II: Effects of increasing the display size and resolution on user analytic behavior and insight acquisition

• Goals

• Measure effects of increasing the display size and resolution on quantity and quality of discoveries made during visual exploration

• Understand variations in user behavior and analytic strategy induced as a result of using a larger display with more pixels

• Study design

• Two display conditions (Small, Large)

• Between-subject design


• Scenario

• Open-ended visual exploration task: analysis of Chicago crime patterns between 2006 - 2012 (approximately 2.8 million crime incidents)

experiment

experiment !19

Visualization interface

[Cockburn et al. 08]

201220092006

TheftBurglaryNarcotics

overview map

experiment !19



design pattern: “seed and grow”

201220092006


overview map

experiment !19




201220092006


overview map

!20

participantexperimenter

4 m

eter

s

wide-angle video camera

keyboard & mouse

papernotepad

CAVE2

experiment [Reda et al. 13b]

!21

Experimental conditions

small !

3 x 4 panels 12 Megapixels 40 degree FOV

large !

13 x 4 panels 54 Megapixels

190 degree FOV

experiment

!21

Experimental conditions

small !

3 x 4 panels 12 Megapixels 40 degree FOV

large !

13 x 4 panels 54 Megapixels

190 degree FOV

experiment

!22experiment

• Participants

• 10 unpaid volunteers (4 female) recruited from EVL

• Distributed evenly under the two experimental conditions

• Procedure

• 15-minutes training

• 2.5 hours of open-ended exploration


• Semi-structured debriefing interview

• Verbal protocol analysis

• Coding scheme: Observation, hypothesis, question, goal, comment.

• 5-points quality score: lower-score refer to isolated insights. Higher scores imply broader, more integrative insights

!23

Results: exploration time

experiment

!23


experiment

0!20!40!60!80!

100!120!140!

large! small!

min

utes!

Average length of exploratory activity!

!23


experiment

0!20!40!60!80!

100!120!140!

large! small!

min

utes!

Average length of exploratory activity!

t(8) = 33.5, p < .01

!24

Results: observations

experiment

!24


experiment

0!

40!

80!

120!

160!

200!

O1! O2! O3! O4! O5! All!

Average number of observations!

large!small!

* * * *

!24


experiment

0!

40!

80!

120!

160!

200!

O1! O2! O3! O4! O5! All!


large!small!

* * * *

𝝌2(4, N=1327) = 263.3, p < .001

!24


experiment

0!

40!

80!

120!

160!

200!

O1! O2! O3! O4! O5! All!


large!small!

* * * *0!

0.2!

0.4!

0.6!

0.8!

1!

1.2!

1.4!

1.6!

O1! O2! O3! O4! O5! All!

obse

rvat

ion

/ min

ute

of a

nalys

is!

Observation rate!

large!small!

* * *

𝝌2(4, N=1327) = 263.3, p < .001

!25

Results: hypotheses

experiment

!25

Results: hypotheses

experiment

0!

5!

10!

15!

20!

25!

H1! H2! H3! H4! H5! All!

Average number of hypotheses!

large!small!

!25

Results: hypotheses

experiment

0!

5!

10!

15!

20!

25!

H1! H2! H3! H4! H5! All!


large!small!

𝝌2(4, N=145) = 67.3, p < .001

!25

Results: hypotheses

experiment

0!

5!

10!

15!

20!

25!

H1! H2! H3! H4! H5! All!


large!small!

0!

0.05!

0.1!

0.15!

0.2!

0.25!

H1! H2! H3! H4! H5! All!

hypo

thes

is /

min

ute

of a

naly

sis!

Hypothesis formulation rate!

large!small!

𝝌2(4, N=145) = 67.3, p < .001

!26

Results: quantity and rate of insights

experiment

• Comparable rate of insight between the two display conditions

• Participants chose to spend 35 minutes extra time on average exploring the dataset with the large display

• This may have caused participants to make more observations with the large display

!26

Results: quantity and rate of insights

experiment

• Comparable rate of insight between the two display conditions

• Participants chose to spend 35 minutes extra time on average exploring the dataset with the large display

• This may have caused participants to make more observations with the large display

minutes into activity

com

mut

ativ

e in

sigh

ts

!27experiment

Results: quality of insights

!27experiment


!27experiment

insight “breadth”

prob

abilit

y

• The large display is more likely to elicit broader, more integrative insights.


!28

Results: user behavior

experiment

!28


experiment

participant S1 (small)

!28


experiment

participant S1 (small)

participant L5 (large)

!29experiment

p(large) - p(small) Results: user behavior

!29experiment

p(large) - p(small)

• Significant tendency to “integrate information / continue to make observations” with the large display

• More frequent transitions to “form goal” (36% increase, non-significant)

• Suggests the formulation and pursuit of more exploratory goals with the large display


!30

Discussion: human limits

experiment

• Some complaints suggest information overload situations with the large display • “It’s hard to look over so much! It’s so hard to compare so many things.”

• Difficulty in integrating information across spatially disparate views • “By the time I finished turning my head to the other side I would forget [the contents of

the previous view]”

• Participants on the large display seem to limit themselves to 6-7 columns (87-102 degrees of visual angle)

!30

Discussion: human limits

experiment

• Some complaints suggest information overload situations with the large display • “It’s hard to look over so much! It’s so hard to compare so many things.”

• Difficulty in integrating information across spatially disparate views • “By the time I finished turning my head to the other side I would forget [the contents of

the previous view]”

• Participants on the large display seem to limit themselves to 6-7 columns (87-102 degrees of visual angle)

!31

Discussion: summary• RQ1 - What is the effect of increasing the display size/resolution on user behavior?

• Significant increase in the length of the exploratory activity.

• Suggests a tendency to pursue more ambitious exploratory goals with the large display

• Small display participants seem to resist exploration when they did not have a priori intuition: “I only bothered to look at the years when I knew something about an area– like Cabrini Green and the Taylor area” !

• Significant tendency to integrate observations / continue to derive new observations!

!• RQ2 - What is the effect of increasing the display size/resolution on insight acquisition?

• Significant increase in the number of observations reported

• Slight increase in the number of hypotheses formulated (not significant)

• Comparable rates of insight (for observations and hypotheses)

• Significant tendency to generate higher-level, more integrative insights.

!• RQ3 - Are there new design patterns for multi-view based visualization on large displays?

• “Seed and grow” design patter

• Loose view coordination model to support multiple exploratory threads

• Lens metaphor to facilitate the exploration of disparate parts of the information space

!32

Scientific reasoning through dual search

outlook [Klahr and Dunbar 88]

!33

Scientific reasoning through dual search: experimenters

outlook

Hypothesis space

experiment / Information

space

[Klahr and Dunbar 88]

!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


!33


outlook

Hypothesis space


space


observations

!33


outlook

Hypothesis space


space


observations

!33


outlook

Hypothesis space


space


generalize

!34

Scientific reasoning through dual search: theorists

outlook

Hypothesis space



space

!34


outlook

Hypothesis space



space

!34


outlook

Hypothesis space


predict


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

!34


outlook

Hypothesis space


predict

collect evidence


space

evidence

!34


outlook

Hypothesis space


predict

collect evidence


space

evidence

!34


outlook

Hypothesis space


predict

collect evidence

evaluate evidence


space

evidence

!34


outlook

Hypothesis space


predict

collect evidence

evaluate evidence

re-frame


space

!35contributions

!35contributions

theory[c2]

!35contributions

theory[c2]

!35contributions

theory[c2]

design patterns

[c3]

!35contributions

apply

theory[c2]

design patterns

[c3]

!35contributions

apply

theory[c2]

design patterns

[c3]

case study + experiment

[c4-5]

!35contributions

apply instantiate

theory[c2]

design patterns

[c3]


[c4-5]

!35contributions

apply instantiate

validate

theory[c2]

design patterns

[c3]


[c4-5]

!35contributions

apply instantiate

validate

theory[c2]

design patterns

[c3]


[c4-5]

!35contributions

apply instantiate

validate

theory[c2]

design patterns

[c3]


[c4-5]

scientific reasoning

hypothesis

observations

[c6]

!35contributions

apply instantiate

validate reflect

theory[c2]

design patterns

[c3]


[c4-5]

scientific reasoning

hypothesis

observations

[c6]

!36future work

• New design patterns?

• Desktop has “overview, zoom, detail-on-demand” [Shneiderman, 96]

• “Spread, eye-ball, coalesce”?

• Design space for multi-view-based visualizations on large displays

• Collaborative visual analytics

• This is where the cost investment makes more sense.

• Applications and real-world case studies

thank you!

Extras

!39

background

Why visualize data?

!40

visualization statistics

I II III IVx mean 9 9 9 9y mean 7.50 7.50 7.50 7.50

x variance 10 10 10 10

y variance 3.75 3.75 3.75 3.75

x/y correlation 0.81 0.81 0.81 0.81

[Anscombe’s quartet]

background

Why visualize data?

!40

visualization statistics

I II III IVx mean 9 9 9 9y mean 7.50 7.50 7.50 7.50

x variance 10 10 10 10

y variance 3.75 3.75 3.75 3.75

x/y correlation 0.81 0.81 0.81 0.81

[Anscombe’s quartet]

Contained within the data of any investigation is information that can yield conclusions to questions not even originally asked. That is, there can be surprises in the data… To regularly miss surprises by failing to probe thoroughly with visualization tools is terribly inefficient because the cost of intensive data analysis is typically very small compared with the cost of data collection.

-William Cleveland, the Elements of Graphing Data

!41

intuition, prior hypotheses,

experience

!!

Visualization

spontaneous observations,

new hypotheses

observations: unit of knowledge acquired from the visualization !hypothesis: conjecture that cannot be directly inferred from the visualization

visual exploration

{insight

form exploratory goals

interpret visual patterns

[Liu and Stasko 10, Treisman 86]

!42

Alternate projections of the same information

Multiple visualization states (i.e., exploratory

threads)

Different subsets of information

(aka, small-multiples)

visual exploration

http://flowingdata.com/tag/small-multiples/

[Wang Baldonado et al. 2000]

Vis!workspace 1

Vis!workspace 2

Vis!workspace 3

http://flowingdata.com/tag/small-multiples/

!43

Visualization

• User performance kept pace with increasing data and screen size [Yost et al. 07] • Increased physical navigation over virtual navigation [Ball and North 05, Ball et al. 07] • Mixed results in map-related tasks; some interfaces are poorly-suited to large displays (e.g. context + focus lenses) [Jakobsen and Hornbæk 11]

Intelligence analysis

(text)

• Memory externalization, semantic spatial arrangement (users attach meaning to space), Schematization [Andrews et al. 10] • Automatic inferring of semantics from interactions [Endert et al. 12b]

Office environment

• Improved spatial cognition + embodied memory [Tan 04] • Reduced gender performance disparities in virtual navigation [Tan 03b] • Reduced window switching, and reduced user frustration [Ball and North 05] • Increased productivity in cognitively demanding office work [Czerwinski et al. 03, Bi et al. 09]

related work

!44

High-res / reduced FOV

[Ball and North 08]related work

Low-res / wide FOV

• High-resolution + physical navigation is more important to improving user performance

!44

High-res / reduced FOV

[Ball and North 08]related work

Low-res / wide FOV

• High-resolution + physical navigation is more important to improving user performance

context + focus displays [Baudisch et al. 02]

!45

action

perception

visual exploration

!46

action

perception

data mining, filtering,

transformation, re-projection

induction, deduction, analogical thinking

T1

T2

information space insights

visual exploration [Sedig et al. 12]

!46

action

perception

data mining, filtering,

transformation, re-projection

induction, deduction, analogical thinking

T1

T2

information space insights

Temporal separation

visual exploration [Sedig et al. 12]

!47theory

!!

!!

!!

!!

!!

View1 V2 V3 V5V4

[Maxcey-Richard and Hollingworth 13]

unexpected discoveries

Cost (temporal-separation): bottom-up costs

time

!47theory

!!

!!

!!

!!

!!

View1 V2 V3 V5V4

[Maxcey-Richard and Hollingworth 13]

unexpected discoveries

Cost (temporal-separation): bottom-up costs

timeP(View1)

!48theory

• Top-down costs

• Discourage the formation of new exploratory goals

• `Tunnel vision’ phenomenon where exploration is focused on isolated subsets in the information space

!

• Bottom-up cost

• Overuse of visual working memory to retain visual patterns of interest

• Reduction in the probability of making spontaneous inferences relating to patterns deposited in temporally-separated views

Cost (temporal-separation): summary

!48theory

• Top-down costs

• Discourage the formation of new exploratory goals

• `Tunnel vision’ phenomenon where exploration is focused on isolated subsets in the information space

!

• Bottom-up cost

• Overuse of visual working memory to retain visual patterns of interest

• Reduction in the probability of making spontaneous inferences relating to patterns deposited in temporally-separated views

Cost (temporal-separation): summary

?

!49case study

!49

Screen surface

Time

2D movement

Trajec

tory

case study

experiment !50



201220092006


overview map

experiment !50




201220092006


overview map

experiment !50



Density of crime incidence

Yearly, weekly, daily crime trends


201220092006


overview map

experiment !50



Density of crime incidence

Yearly, weekly, daily crime trends


201220092006


overview map

!51experiment

structure layoutfreeform layout

large

small

!52experiment

• 3 “mental states”

• Make observation

• Form goal

• Formulate hypothesis

• 2 types of interaction

• Layout-preserving (brush-and-link / pan map)

• Layout-changing (create new view, changing view contents)


!53

Results: user behavior / strategy

experiment

average (small)

average (large)

!54

0!

1!

2!

3!

4!

5!

large!

small!

usability utility

usability / utility

experiment

!55experiment


Do higher-level insights occur at a later time in the activity?

!55experiment


minutes into activity

leve

l of i

nsig

ht

Do higher-level insights occur at a later time in the activity?

!56

Form goal!

Visualization tool



Execute physical sequence Perceive state



!56

Form goal!

Visualization tool






!56

Form goal!

Visualization tool






!56

Form goal!

Visualization tool






!57contributions

1. Theory of how interaction costs affect user behavior in exploratory visual analysis

• `Tunnel vision’ phenomenon in situations involving large amounts of data

• Costs are elevated due to excessive temporal-separation of views

2. Effects of increasing the size and resolution of the visualization interface on user behavior and insights acquisition

• Increased user investment / exploration

• More observations

• Formation of higher-level, more integrative insights

3. Design patterns for large high-resolution displays

• Query-by-example brush for the analysis of ensemble data

• “Seed and grow” pattern: loose view coordination model with lens metaphor

Science

Exploratory Visual Analysis in Large High-Resolution Displays