CAVERNUS - Advanced CAVE Workshop Series Examples of the CAVE working for Scientists Rachael Brady Beckman Institute, UIUC August 15, 2000

CAVERNUS - Advanced CAVE Workshop Series

Examples of the CAVE working for Scientists

Rachael Brady

Beckman Institute, UIUC

August 15, 2000


Outline

• Crumbs Overview– Motivation– Rendering– Toolset– Audio

• Formative Evaluation

• Uses by Scientists


Writing a VR Application for Scientists (Biologists)

• Navigation– Driving/Flying paradigms

only work with landmarks and horizon

• Interaction– Object control

– Parameter setting

– Measuring

• Hardcopy


Illustration or Exploration• Illustration

– High rendering quality– Annotation– Accurate measurements

• Exploration– Fast, interactive rendering– Multiple viewpoints– Multiple viewing tools– Measurement/Analytic tools


48 Hour Old Chicken EmbryoJo Ann Eurell, Janet Sinn-HanlonVeterinary Biosciences, UIUC

Illustration

Exploration

C:\WINDOWS\PLAY32.EXE d:\My Documents\ppt slides\movies\48hour.Janet.mov


Crumbs

General viewing tool for regularly gridded 3D scalar data in the CAVE. Developed to trace and measure 1D fibers


Outline





Motivation of Crumbs (Feb ’94)• Problem

– Trace and model fibrous structures in dense, volumetric datasets from imaging modalities such as MRI, TEM and Optical Microscopy

• Will a virtual environment help?


Volume Rendering• Ray Traced • Texture Mapped


First Attempts• Navigation

– Flying

– Outside/Inside + Map

– Windshield

– Physical Object you pick up & resize

• Rendering– Show entire data set at once

– Viewing window

– Several viewing windows

– Slices


Current Features

• Parameter Settings– Menus– Colormap & Opacity Control– Voice Command

• Crumbs Placement– Intelligent use of space– Spline Plane– Sonification


What Crumbs Looks Like in Use

Data volume is a horse fetlock (ankle). Source: Mark Martinelli


Outline





Crumbs Environment

Color Map

Clear BoxWidth (pixels)Depth (slices)

Size (feet)Size (pixels)

Slice Plane

Volumetric Dataset

Spline Plane

Crumbs

Color Map Editor

Wand Types

Trash Can


Color Map Adjustment

Color Map

Color Map Editor


Opacity Adjustment


Voice CommandNecessary for Instrument Control


crumbs Manipulation

Insert New Crumb Into

Spline Segment

Reposition Existing Crumb

Reverse Spline

Node List and Extend Spline by Adding Another Crumb

Extend Spline by Adding Another

Node to the List


Spline Plane


Outline





Audio• Ambient Sounds (mood music)• Event Feedback• Localization• Sonification

– Can sound help in Crumbs placement?• Must be fatigue resistant• Must convey data value• Must synchronize with graphical rendering & task


First Attempt• Four Melodies

• Orchestration mapped to data value

• Implemented by Robin Bargar & Insook Choi


Current Implementation

• No MIDI device needed• Implementation by Carlos Ricci


Current Implementation


Outline





Formative Evaluation

• Kent Swartz & Deborah Hix– Virginia Tech, Dept. of Computer Science

• Umesh Thakkar– University of IL, NCSA


Formative Evaluation - Tasks

• Crumbs has “feature-itis”. Decided to focus on core task of placing crumbs along a 1D feature of interest.– Two data sets chosen.

• Fertilized fruit fly embryo, place crumbs along sperm tail within egg nucleus

• Human Spine, place crumbs on 3D centroid of each vertebrae


Formative Evaluation - Users• User #1, pilot study

– Wide VR experience, demos Crumbs, not user

• User #2– Wide VR experience, runs Crumbs for demos, used once for own

data

• User #3– User, no VR experience

• User #4– Developer, no VR experience

• User #5– User, no VR experience


Formative Evaluation - Survey Results

Overall Satisfaction

Usability of Menu Sonification Opacity Editor

Colormap Editor

User #1 Satisfied Hard to Use Effective Easy EasyUser #2 Satisfied Very Easy Effective Very Difficult VeryDifficultUser #3 Satsfied Easy Ineffective Easy DifficultUser #4 Satisfied Easy Very Ineffective Difficult Very DifficultUser #5 Satisfied Easy Ineffective Easy Difficult

Sword Pointer

Grouped Objects Immersed?

Get disoriented?

User #1 Easy Easy Yes NeverUser #2 Easy Hard No NeverUser #3 Easy So-So Somewhat So-SoUser #4 Easy No Answer Somewhat SometimesUser #5 Very Easy Very Easy Somewhat Sometimes


Formative Evaluation - Results• Removal of objects from the environment

– Delete vs. Hide, not a problem in general. One person had trouble with this.

• Some aural feedback choices “bad”– “Ouch” caused people to release button and cringe– “Good Luck” made people pause and wonder why

• “Grab and Drag” button changes for thumbslider on Opacity, Colormap & Contrast tools– True problem. Lots of confusion from this

• Suggestion: Grey out unavailable options on the menu


Outline





Uses by Scientists• Fruit Fly Fertilization• Protozoa• Spinal Stenosis• Human Pharynx• Egyptian Mummy• Chicken Development• Oil Exploration• Plasma Physics• Optical Tomography• Architecture


Drosophila Sperm Tails

Tim Karr

University of Chicago

Beckman Institute UIUC


Structure of fertilized Drosophilla Egg


Evolution of sperm study • What is length of tail? 1994 - 1995

– Initially used CAVE to trace twisting, knotty tail in order to determine length. Topological accuracy was not important.


Evolution of sperm study• What is shape of tail? 1996-1998

– Why do different species have different shapes? – What is statistical variation of shape within a species?


Evolution of sperm study• How does sample preparation effect measurements

(1999-present)

• Results so far:– Same prep, same gestation, same species yield

measurement repeatability within 5%– Compression of sample from confocal imaging results in

longer tail measurements


Sperm’s effect on Crumbs

• Added capability to view many tracings simultaneously

• Added seim-automatic tracing capability

• Branching capability out of knots

Published in Nature (1995) and Science (1996)


Protozoa: Vorticella convallaria

Howard E. Buhse Jr.

Biological Sciences

University of Illinois at Chicago

Aboral Region

PeristomeInfundibulum

Trochal Band

Oral Region


Structure of Vorticella convallaria


Discovering the OF-1 Antibody in the Myonemes

• Use of the confocal microscope forced the scientist to prepare his organism with a different stain

• The proteins carrying the stain appeared in an unexpected location in the organism


Spinal Stenosis

Budinger, Schoenem, Maltz, LBL


Spinal Stenosis

• Want to find point of tightest constriction along vertebrae of spine• Added capability to select locations for 2D slices, and save

images to disk• Results: no significant differences discovered from classical

methods


Human Pharynx

David Kuehn & Sandy Ettema

Speech and Hearing Science, UIUC


Mummy

Sarah Wisseman

Ancient Technologies

UIUC


Mummy• CAT scan performed at CARLE clinic

• Mummy is boy, around 9 years old– Teeth– Pelvis

• Body is on wood (pine?) board

• Organs not removed prior to mummification


48 Hour Old Chicken Embryo

Jo Ann Eurell

Janet Sinn-Hanlon

Veterinary Biosciences, UIUC


Chicken Embryo• Optical serial sections

• 24-hour and 48-hour embryos available

• Used as field trip for developmental biology course– Heart development– Nervous system development

• Now illustration available (video)


Oil Exploration• Seismic data is very dense and difficult to see

structures• Slice planes most popular rendering method• Incorporated ability to render predefined meshes so

that land marks could be seen in seismic volume• Jan 99: oil drill hit calcium deposit

– decided to stop drilling in that location– used Crumbs to explore area around well– data showed oil deposit behind deposit– continued drilling: found deposit worth $15M


Militzer/Ceperly version of Crumbs

Path-integral Monte Carlosimulations of hydrogen.The calculation is used tostudy hydrogen transitionfrom the plasma state withfree protons and electrons to a molecular gas, and finally to molecular metallic liquid.


Militzer version of Crumbs• Volume rendering is used to show electron density

• Proton locations are denoted by spheres, yellow line connecting them indicates a molecule

• Altered Crumbs path to be electron path– Blue and Red denote opposite spin states– Permuted electrons are shown in yellow


Militzer version of Crumbs


Optical Tomography

David BradyElectrical and Computer Engineering

UIUC


Argus


Argus• 64 cameras in 14’ diameter, 8’ tall cylinder• 32 node dual processor Beowulf Linux cluster• Cameras acquire 64 images in parallel at 60 Hz,

synchronized and calibrated• Computed data includes

– 3D computed tomography data volumes– Stereo pairs computed for any viewpoint in volume

• Storing images on disk allows control over time dimension


Argus in Action


Argus Goals• How do you calculate and convey information from

dense sensor arrays?

• What can the networking support?

• What kinds of compression are effective?

• How can a person interact with and control the system?

• IGRID demonstration in Japan, July 2000.


Architect, Joy Malnar• NCSA faculty fellow• Uses slices and curve-drawing of Crumbs to create

spaces• Not interested in “walk throughs”• Wants lots of interation and control• Teaching course, fall ‘2000