Upload
samantha-bishop
View
217
Download
0
Embed Size (px)
Citation preview
Understanding Visualization through Spatial Ability Differences
Maria C. Velez, Deborah Silver and Marilyn Tremaine
Rutgers University2005
2
What this Talk is About
Different people have been shown to have a lot of trouble with 3D visualizations
To investigate this issue further, we ran an experiment comparing measured spatial skills to abilities to understand visualizations
The results suggested key problems individuals were having and ways in which we can make the visualizations understandable by a larger audience
3
Motivation
Issues in Visualization Understanding
Classic 3D visualizations (2D projection and slices) have been found to be suboptimal for tasks like understanding shape and 3D space layout.
Examples of conventional visualization displays used in medical and weather imaging
4
Previous Solutions to Visualization Difficulties
Improvements have been proposed:
Exovis Cube Corner
Training methods have been developed:
“Pool of water” metaphor used in cutting planes training
Improvements are ad hoc rather than theory-based
Mental Rotation Test used to study spatial comprehension
5
Focus – What Makes a Visualization Difficult?
We use differences in human spatial abilities to understand the problems that affect people’s understanding of a visualization• Controlling for human variability makes effects detectable• Looking at extremes helps us understand normal behavior
Our questions• Does everyone solve problems similarly?• Do they make the same error?• How does diversity in the population affect performance in the
visualization?• Do the solutions proposed help everyone equally?
6
Research Approach
I. Select a set of cognitive skills that are likely to play a role in visualization understanding
II. Measure these cognitive skills with standardized tests using a group of subjects selected for their variability
III. Measure the level of visualization understanding of the subjects (via one type of “prototypical” visualization test)
IV. Match the visualization performance results to the standardized test results
V. Examine the properties of the visualization for both successful and unsuccessful comprehensions for each spatial ability subgroup
VI. Examine the error distributions of the wrong answers for each spatial ability subgroup
7
Human Spatial Abilities
What are spatial abilities?
• Skills involving the retrieval, retention and transformation of visual information in a spatial context.
Are there other relevant cognitive factors ?
Spatial Orientation
Spatial Location Memory
Targeting
Spatial Visualization
Disembedding
Spatial Perception
Visual Memory
Perceptual Speed
IDetermine which cognitive skills that might play a role in visualization understanding
8
Standardized Tests
Measuring spatial abilities• We measured spatial abilities through the Kit of • Factor-Referenced Cognitive Tests available at ETS.
IIMeasure a subset of those cognitive skills with standardized tests using a group of subjects selected for their variability
• Spatial Orientation: Cube Comparison Test
• Spatial Visualization: Paper Folding Test
• Disembedding: Hidden Patterns Test
• Visual Memory: Shape Memory Test
• Perceptual Speed: Identical Figures Test
9
The Visualization Test IIIMeasure the level of visualization understanding of the subjects (via a “prototypical” visualization test)
Goal: Examine the comprehension of a “prototypical” visualization: orthogonal projection• Basic visualization without bells and whistles• Easy to learn by untrained experiment participants• Use geometrical (geon-like and compounded) and
common realistic objects
Geon-like Compounded Realistic
10
The Visualization Test – Screen 1
Mentally form an
image of the object and its
alignment
11
The Visualization Test – Screen 2
Select the object that represents the object
creating the projections
This should be your answer
12
Experiment Design
Measures of performance• Accuracy: percentage of correct answers• Analysis Time : Time spent analyzing the object’s
projections (seconds)• Selection Time : Time spent selecting the answer
(seconds)Analysis Time and Selection Time are measured
independently
Experiment Participants (selected for variability)• 56 paid participants, 50% percent female• Average age: 21 years (range: 18 to 31 years) • 84% undergraduate, 16% graduate students
13
Experimental Method
Experiment Procedure
• Five paper-based cognitive factor tests• General instructions and five practice questions• Computer-based visualization test : 60 minutes
to complete 38 questions • Debriefing explaining the purpose of the
experiment
14
Road Map to Analysis
Spatial test resultsStandardized cognitive test
Performance
Analysis of subject’s errors(Case by case analysis)
Correlation
Visualization properties results
Spatial ability groups results
Correlation
Differences
15
Expected Relation Between Spatial Ability and Visualization Performance
Shape Memory
Cube Comparison
Paper Folding
Pattern Matching
Identical Figures
Accuracy Low Positive /
No correlation
+ High + High + Low No correlation
Time of Analysis
Low Positive /
No correlation
+ High + High + Low - High
Time of Selection
Low Positive /
No correlation
+ High + High + Low - High
+ Positive correlation- Negative correlation
16
Spatial Test Results
Analysis• Pearson correlation analysis between performance and scores in
standardized tests
IVMatch the visualization performance results to the standardized test results
Results
Shape Memory
Cube Comparison
Paper Folding
Pattern Matching
Identical Figures
Accuracy + Low + High + High No Corr. - Low
Time of Analysis No Corr. No Corr. No Corr. No Corr. - Low
Time of Selection No Corr. No Corr. No Corr. - Medium - Low
Implications• Visualization comprehension on diverse populations affected by
spatial ability diversity• Paper tests were time constrained which may have affected the
time correlations
Not ExpectedExpected
17
Visualization Properties Results
For the object’s properties (i.e. surfaces, edges and vertices) we calculated:• Total count in the original 3D object• Distinct properties that would be visible in a wireframe
rendering of the projection.• Visible properties in a uniformly shaded object
e1
e2
e3
e4
e5
e6e7 e1
e2
e3
e4
e5
e6e7e1
e24
e35
e6
Total count of edges: 12
Distinct edges: 7 Visible edges: 4
18
Visualization Properties Results
Analysis• Pearson correlation analysis between performance and property
counts and ratios
Implications• The hidden geometric properties make visualization understanding
cognitively harder and thus, more time consuming• Rotation of objects and animation will help users’ comprehension• Complex objects require slower animations to give viewer time to extract
information
CountsRatios Visualized / Distinct
Edges Vertices Surfaces
Accuracy No Corr. + High + Medium + Medium
Time of Analysis + Medium - High - High - High
Time of Selection No Corr. No Corr. No Corr. No Corr.
Results
19
Other Visualization Properties Results
A learning curve was not detected (see figure)
HGFEDCBA
35.00
30.00
25.00
20.00
15.00
10.00
Tim
e (s
eco
nd
s)
Time of answerselection
Time of projectionanalysis
Analysis time vs. Selection time
Repeated object in order of appearance
73% 52% 46% 73% 63% 70% 70% 46% Accuracy
Accuracy was affected by choices that differed from the correct answer by small differences in orientation
No significant performance differences were found between geometric and realistic objects
20
Spatial Ability Groups Results
No significant property differences between:• All questions with high percentage of correct answers• All questions with high percentage of incorrect answers
VExamine the properties of the visualization for both successful and unsuccessful comprehensions for each spatial ability subgroup
BUMMER!
Our Next Step is to Look at the Data in More Detail
Divide-up participants 3 groups and selected the High Spatial (HS) and Low Spatial (LS) ability participants (based on Paper Folding Test).• Knowing a source of variability and looking at the extremes
helps to make the effect visible
21
Spatial Ability Groups Results
Analysis• Are there properties that only high spatial people use?• Compare Properties of Questions Answered Correctly by high
spatial participants to Properties of all Questions
Results • Total Number of Edges and Total Number of Vertices were found
significantly higher in questions which the high spatial participants answered correctly
• The Ratio of Distinct to Visualized Surfaces was found significantly higher in questions answered correctly by high spatial participants
Implications• High spatial participants understand more complex objects and can
process a higher number of hidden properties
22
Analysis of Subject’s Errors VI Examine the error distributions of the wrong
answers for each spatial ability subgroup
Analysis• Create a bar graph showing distribution of answers for each
question• Analyze the questions where distributions clearly not evenly
distributed
Interesting results for further analysis (possible strategies)
Frequency of answers
High spatial ability 6
Low spatial ability 8
High spatial ability
Low spatial ability
23
Issues
Experiment trials organized according to what was believed to be trial difficulty. This organization was wrong.
There was ambiguity in the answers that participants had to choose between, in particular because participants were allowed to rotate the answers, they were not able to see the differences in orientation between two possible answers
Only projection visualization was studied and thus, the results cannot be readily extrapolated to many other 3D visualizations
The object properties manipulated in the questions were horizontal and vertical alignment. Future studies will include properties such as size, shape (sides), aspect ratio.
24
Summary of Results
Spatial abilities are related to 3D visualization comprehension
Problem solution time was not found to be related to visualization accuracy
Counts of geometric properties affected visualization accuracy for low spatial subjects, and time of analysis for everyone
The “hidden” geometric properties in the visualization affect visualization accuracy for low spatial subjects
Small rotation differences are difficult to detect in a visualization
A case by case analysis suggests that high spatial and low spatial ability participants use different strategies
25
Larger Implications of Research
Visualization designers can use measures of cognitive ability to help understand what makes visualizations hard/easy to comprehend
Using interactive rotation and animations is likely to help users better understand visualizations
Visualization difficulty may be highly variable for a diverse population
There exist educated people who cannot understand simple 3D visualizations
26
Acknowledgments
Thanks to our reviewers for their comments
This research is supported by the National Science Foundation through the SGER grant # 0503680
27