Upload
arianna-norris
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Saliency-guided Enhancement for Volume Visualization
Youngmin Kim and Amitabh Varshney
Department of Computer Science
University of Maryland at College Park
2
Motivation
The volume datasets have grown in complexity• Visible Human Project
• 13GB ~ 60GB
• National Library of Medicine (NIH)
• Richtmyer-Meshkov Instability Simulation• 2 TB (= 7.5GB * 273 time steps)
• Lawrence Livermore National Laboratory
Human visual capabilities remain fixed The need to draw visual attention to appropriate
regions in their visualization
3
Motivation
We can draw viewer attention in several ways Obtrusive methods like arrows or flashing pixels
• Distracts the viewer from exploring other regions Principles of visual perception used by artists and
illustrators• Gently guide to regions that they wished to emphasize
4
Contributions
A new saliency-based enhancement operator• Guides visual attention in volume visualization without
sacrificing local context
• Considers the influence of each voxel at multiple scales
Augments the existing visualization pipeline• Enhances regional visual saliency
Validation by eye-tracking-based user study• Our method elicits greater visual attention
5
Related Work - Saliency
Computation and Evaluation• Computational models for image [Itti et al. PAMI 98]
and mesh [Lee et al. SIGGRAPH 05]• Evaluation by predicting eye movements
[Parkhurst et al. 02], [Privitera and Stark PAMI 00]
Use of eye movements• Volume composition [Lu et al. EuroVis 06]• Abstractions of photographs [DeCarlo and Santella SIGGRAPH 02,
NPAR 04] Use of Saliency
• Progressive visualization [Machiraju et al., 01]• Importance-based enhancement [Rheingans and Ebert TVCG 01]• Interior and exterior visualization [Viola et al. TVCG 05]• Generalizing focus+context [Hauser Dagstuhl 03]
Saliency has not been used for guiding visual attention
Mesh Saliency
6
Related Work – Transfer Functions
Transfer Functions map the physical appearance to the local geometric attributes such as:• Gradient magnitude [Levoy CG&A 88]
• First and second derivatives [Kindlmann and Durkin Volume Rendering 98]
• Multi-dimensional transfer functions [Kindlmann et al. Vis 03], [Kniss et al. TVCG 02], [Kniss et al. Vis 03], [Machiraju et al. 01]
Have played a crucial role in informative Visualization Difficult to emphasize (or deemphasize) regions specified
exclusively by locations in a volume
7
Overview Saliency Field Enhancement Operators Emphasis Field Saliency Enhancement Saliency-enhanced Volume Rendering Validation by eye-tracking based user study
Transfer Functions
Saliency Field by User Input
Emphasis Field Computed
Enhancement Operators
Saliency-enhanced Volume Rendering
Validation by eye-tracking device
SaliencyEnhancement
8
S (v) = |G(C, v, σ) – G(C, v, 2σ)|
Basic idea from Saliency Computation
Saliency map is:
Mesh saliency based on curvature values Image saliency based on intensity and color In general, saliency may be defined on a given
scalar field
C : Mean curvature
9
Emphasis Field Computation
Mesh Saliency: S (v) = G(C, v, σ) – G(C, v, 2σ) We introduce the concept of an Emphasis Field
E to define a Saliency Field S in a volume
S (v) = G(E, v, σ) – G(E, v, 2σ)
Given a saliency field, can we design some scalar field that will generate it?
KnownUnknown
Known Unknown
10
Emphasis Field Computation
Expressible as simultaneous linear equations
Saliency Enhancement Operator (C-1)• CE =S , which implies E = C-1S• Given a saliency field S , the enhancement operator C-1 will
generate the emphasis field E
where cij is the difference between two Gaussian weights at scale σ and at scale 2σ for a voxel vj from the center voxel vi
=
11
Emphasis Field Computation
We like to use enhancement operators at multiple scales σi
• Let E i be the emphasis field at scale σi
• Compute this by applying the enhancement operator Ci-1 on the
saliency field S
• Final emphasis field is computed as the summation of E i
12
Emphasis Field in Practice
A system of simultaneous linear equations in n variables• Generally, can handle arbitrary saliency regions and values• Computationally expensive: O(kn2) or O(n3)
Alleviate this by solving a 1D system of equations• Given a saliency field• Solve 1D system of equations at
multiple scales and sum them up• Approximate results using
piecewise polynomial radial functions [Wendland 1995]
Interpret results to be along the radial dimension• Assume spherical regions of interest (ROI)
13
Visualization Enhancement
Emphasis Fields can alter visualization parameters in several ways• Various rendering stylizations and effects possible
We outline a couple of possibilities
• Brightness• Widely used to elicit visual attention by artists • Modulate the Value parameter in the HSV model as follows:
– Vnew(v) = V(v)•(1+E (v)), where –λ- ≤E (v) ≤ λ+
– Used 0.4 ≤ λ+ ≤ 0.6 and 0.15 ≤ λ- ≤ 0.35
• Saturation• Can modulate Saturation instead of Value if the latter is not
effective (for instance, in regions already very bright)
14
Gaussian-based vs. Saliency-guided Enhancement
Previous Gaussian-based Enhancement of a Volume• Volume Illustration [Rheingans and Ebert TVCG 01]• Importance-based regional enhancement
We use a Gaussian fall-off from the boundary of ROI
15
Visualization Enhancement - Brightness
Traditional Volume Rendering
Gaussian-based Enhancement
Saliency-guided Enhancement
Traditional Volume Rendering
Gaussian-based Enhancement
Saliency-guided Enhancement
16
Visualization Enhancement - Saturation
Traditional Volume Rendering
Saliency-guided Enhancement
Increasing brightness diminishes the appearance of blood vessels at the center of the Sheep Heart model
17
User Study
Validated results by an eye-tracking-based user study
Hypotheses: The eye fixations increase over the region of interest (ROI) in a volume by the saliency-guided enhancement compared to• the traditional volume visualization (Hypothesis H1)• the Gaussian-based enhancement (Hypothesis H2)
18
User Study – Experimental Design
Eye-tracker and General Settings• ISCAN ETL-500
• Records eye movements at 60Hz• 17-inch LCD monitor
• With a resolution of 1280x1024• Placed at a distance of 50cm (19.7’’) from the subjects
Eye-tracker Calibration• Desired accuracy of 30 pixels• Two-step calibration process
• Standard calibration with 5 points• Look and click on 13 points
– Triangulation and interpolationwith 4 corner points
• Accuracy test on 16 random points
19
User Study – Experimental Design
Extracting fixations from raw points Raw points: all points from the eye-tracker Saccade Removal
• Velocity > 15°/sec Fixation combining
• Filter out the points which stay less than 100ms within 15 pixels
• Average eye locations within 15 pixels and 100ms
20
User Study – Experimental Design
Image Ordering• 10 users (who passed the accuracy tests)• Total of 20 images: 4 models * (1 original + 2 regions * 2
different enhancement methods (Gaussian, Saliency))• Each user saw 12 images out of these 20 images
• 4 models * (1 original + 2 altered))• Enhanced different regions with different methods
• Placed similar images far apart to alleviate differential carryover effects
• Randomized the order of regions and the order of enhancement types (Gaussian and saliency-based) to counterbalance overall effects
Duration• 12 trials (images), each of which takes 5 seconds
21
User Study – Result I
Traditional Volume RenderingTraditional Volume Rendering With Fixation Points
Saliency FieldGaussian-based EnhancementGaussian-based Enhancement With Fixation Points
Saliency-guided Enhancement With Fixation Points
Saliency-guided Enhancement
22
User Study – Result II
Traditional Volume RenderingTraditional Volume Rendering With Fixation Points
Saliency FieldGaussian-based EnhancementGaussian-based Enhancement With Fixation Points
Saliency-guided Enhancement With Fixation Points
Saliency-guided Enhancement
23
Data Analysis I
The percentage of fixations on the ROI for the original, Gaussian-enhanced, and Saliency-enhanced visualizations
24
Data Analysis II
A two-way ANOVA on the percentage of fixations for two conditions, regions and enhancement methods for each volume
For regions, no statistically significant results as expected• F(1,34) = 0.2827 ~ 3.3336, p > 0.05
For enhancement methods, statistically significant results• F(2,34) = 7.2668 ~ 31.479, p ≤ 0.01
25
Data Analysis III
Carried out a pairwise t-test on the percentage of fixations before and after we applied enhancement techniques for each model
Found a statistically significant difference in the percentage of fixations with saliency-guided enhancement for all the models
H1
H2
H1
H1
H1
H2
H2
H2
Hypothesis H1: More fixations than the traditionalHypothesis H2: More fixations than the Gaussian
26
Conclusions
Introduced the concept of the Emphasis Field for selective visual emphasis (or de-emphasis)
Developed the computational framework to generate the Emphasis Field from a given Saliency Field
Illustrated the use of the Emphasis Field in Visualization Validated its ability to successfully guide visual attention
to desired regions Saliency-guided Enhancement provides a powerful tool
to help scientists, engineers, and medical researchers explore large visual datasets
27
Future Work
Measure comprehensibility of the volume rendered images
Explore other appearance attributes such as opacity and texture detail
Generalize to handle time-varying datasets with multiple superposed scalar and vector fields
Identify the relative importance of various scales
28
Acknowledgments
Datasets: Stefan Roettger (University of Erlangen) and Dirk Bartz (University of Tuebingen)
Discussions: David Jacobs, François Guimbretière, Derek Juba, and Robert Patro (University of Maryland)
Eye-tracker: François Guimbretière
The Anonymous Referees
Supported by NSF grants: CCF 05-41120, CCF 04-29753, CNS 04-03313, and IIS 04-14699
29
Questions ??
www.cs.umd.edu/gvil
www.cs.umd.edu/gvil/projects/sevv.shtmlSupplemental material in the DVD-ROM
Lab:
Project:Images: