114

VisIt is a richly featured, turnkey application

• VisIt is an open source, end user visualization and analysis tool for simulated and experimental data

– Used by: physicists, engineers, code developers, vis experts

– >100K downloads on web

• R&D 100 award in 2005• Used “heavily to exclusively” on 8

of world’s top 12 supercomputers

217 pin reactor coolingsimulation. Run on ¼ of Argonne BG/P.

1 billion grid points

22

Terribly Named!! Intended for more than just visualization!

Data Exploration

VisualDebugging Quantitative Analysis

Presentations

=?

Comparative Analysis

33

VisIt has a rich feature set that can impact many science areas.

Meshes: rectilinear, curvilinear, unstructured, point, AMR Data: scalar, vector, tensor, material, species Dimension: 1D, 2D, 3D, time varying Rendering (~15): pseudocolor, volume rendering,

hedgehogs, glyphs, mesh lines, etc… Data manipulation (~40): slicing, contouring, clipping,

thresholding, restrict to box, reflect, project, revolve, … File formats (~85) Derived quantities: >100 interoperable building blocks

+,-,*,/, gradient, mesh quality, if-then-else, and, or, not Many general features: position lights, make movie, etc Queries (~50): ways to pull out quantitative information,

debugging, comparative analysis

44

VisIt employs a parallelized client-server architecture.

Client-server observations:• Good for remote

visualization• Leverages available

resources• Scales well• No need to move data

Additional design considerations:• Plugins• Multiple UIs: GUI (Qt), CLI

(Python), more…

remote machine

Parallel vis resources

Userdata

localhost – Linux, Windows, Mac

Graphics Hardware

55

The VisIt team focuses on making a robust, usable product for end users.

Manuals• 300 page user manual• 200 page command line interface manual• “Getting your data into VisIt” manual

Wiki for users (and developers) Revision control, nightly regression testing, etc Executables for all major platforms Day long class, complete with exercises

Slides from the VisIt class

66

VisIt is a vibrant project with many participants.

Over 50 person-years of effort Over one million lines of code Partnership between: Department of Energy’s Office of Nuclear

Energy, Office of Science, and National Nuclear Security Agency, and among others

2004-6

User communitygrows, includingAWE & ASC Alliance schools

Fall ‘06

VACET is funded

Spring ‘08

AWE enters repo

2003

LLNL user communitytransitioned to VisIt

2005

2005 R&D100

2007

SciDAC Outreach Center enablesPublic SW repo

2007

Saudi Aramcofunds LLNL to support VisIt

Spring ‘07

GNEP funds LLNL to support GNEP codes at Argonne

Summer‘07

Developers from LLNL, LBL, & ORNLStart dev in repo

‘07-’08

UC Davis & UUtah research done in VisIt repo

2000

Project started

‘07-’08

Partnership withCEA is developed

2008

Institutional supportleverages effort from many labs

Spring ‘09

More developersEntering repo allthe time

7

VisIt: What’s the Big Deal?

Everything works at scale Robust, usable tool Vis to code development to scientific insight

7

8

● Weak scaling study: ~62.5M cells/core

8

#coresProblem Size

ModelMachine

8K0.5TIBM P5Purple

16K1TSunRanger

16K1TX86_64Juno

32K2TCray XT5JaguarPF

64K4TBG/PDawn

16K, 32K1T, 2TCray XT4Franklin

VisIt was recently demonstrated to show good performance at unprecedented scales.

9

Three Ways To Get Data Into VisIt

(1) Write to a known output format (2) Write a plugin file format reader (3) Integrate VisIt “in situ”

• “lib-VisIt” is linked into simulation code (Note: Memory footprint issues with

implementation!)• Use model:

simulation code advances at some time interval (e.g. end of cycle), hands

control to lib-VisIt. lib-VisIt performs vis & analysis tasks, then hands

control back to simulation code repeat 9

10

Tutorial schedule

10 minute overview (done) 6 part tutorial on using VisIt

• Introduction• Data analysis / expressions• Scripting• Moviemaking• Comparisons• Alternate data representations

Data model overview Database plugin development example Operator plugin development example Presentation on VisIt architecture / parallel algorithms

10

11

Before we begin…

Tutorial:• Hank Childs (hchilds@lbl.gov), Sean Ahern (ahern@ornl.gov)• Script: http://www.visitusers.org/index.php?Title=Short_Tutorial

• Another tutorial upcoming SC09 User resources:

• Main website: http://www.llnl.gov/visit• Wiki: http://www.visitusers.org• Email: visitusers@ornl.gov

Development resources:• Email: visit-developers@ornl.gov• SVN: http://portal.nersc.gov/svn/visit

11

12

AFTER THIS POINT IS THE SLIDES TO CLOSE THE TUTORIAL

12

Outline

Parallelism• Contracts• Volume rendering• Streamlines

Data model• AMR• Subset Inclusion Lattices• CSG

Miscellaneous• More on steering• Synthetic Diagnostics

13

14

Contracts are an extension to the standard data flow network design.

Work is performed by a pipeline

A pipeline consists of data objects and components (sources, filters, and sinks)

File Reader(Source)

Slice Filter

ContourFilter

Renderer(Sink)

V0

V1

V2

Exec

Up

dat

e

Execu

te

Data Flow Networks “101”:

Extension:• Contracts are coupled with

the Update phase

• Pipeline execution begins with a “pull”, which starts Update phase

• Data flows from component to component during the Execute phase

15

Operating on Optimal Subset of Data

…

Slice Filter

V(I)

…

V(I+1)

Filter(base class)

(inheritance)

Slice FilterHundreds of others

Contract *SliceFilter::ModifyContract (Contract *V(I)){ ... return V(I+1);}

1) Get meta-data2) Determine

domains that intersect slice

3) Restrict list of domains to process in V(I+1)

16

Operating on Optimal Subset of Data

…

Slice Filter

V(I)

…

V(I+1) 1) Get meta-data2) Determine

domains that intersect slice

3) Restrict list of domains to process in V(I+1)

D3D2D1D0

D3D2D1D0

17

The contract-based system provides high flexibility for this optimization.

…

SphericalSlice Filter

V(I)

…

V(I+1)

A new, plugin filter can use this optimization

without any modification to base data flow network

architecture

…

Slice Filter

V(J)

…

V(J+1)

…

Contour Filter

V(I+1)

V(I)

…

…

Multiple filters can use the same optimizations

Visualization of massive data requires sophisticated algorithms.

• “Image Space” Partition• Redundant I/O• Each frame requires

new partition

• “World Space” Partition• Camera inside data

set causes huge imbalance.

A hybrid approach that acts as a world space partition, but identifies regions of imbalance and handles those regions using an image space partition.

This analysis depended on improvements in parallel particle advection

Two extremes:• Partition data over processors

and pass particles amongst processors Parallel inefficiency!

• Partition seed points over processors and process necessary data for advection Redundant I/O!

Notional streamlineexampleNotional streamlineexample

P0 P0 P0 P0 P0

P1 P1 P1 P1 P1

P2 P2 P2 P2 P2

P3 P3 P3 P3 P3

P4 P4 P4 P4 P4

P0

P1P2

P3P4

Hybrid solution:• Master-slave approach that

adapts between parallel inefficiencies and redundant I/O

P0 P0

P1

P1P2

P2 P3P4

Iteration Action

0 P0 reads B0,P3 reads B1

1 P1 passes points to P0,P4 passes points to P3,P2 reads B0

0: Read

0: Read

Notional streamlineexampleNotional streamlineexample

1: Pass

1: Pass1: Read

- Decision of when to pass or read is simplified here and based on load of processors owning data. - Heuristic weights against I/O, but allows for redundant I/O- All coordination done by slave masters.

This analysis depended on improvements in parallel particle advection

21

AMR is handled natively in VisIt

VisIt understands:• Nesting of patches / Boundaries between patches

Strategy:• Each patch is processed separately• After reading data, subsequent passes to:

Identify coarse elements that are refined and mark them as “ghost”

Create ghost layers around outer boundaries (needs work)

UI: • Color by patch, color by level• Remove / show patches, levels • Have appropriate info returned for picks

22

CSG Visualization

Adaptively creates a volumetric representation

Supported shapes• Sphere• Cone• Cylinder• Plane• Quadric• Tetrahedron• Pyramid• Prism• Hexahedron

23

Users need to reference their data “as they know it”.

Many different decompositions of data:• By simulation processor ID• By material• By part in an assembly• By level/patch for AMR• Plus key subsets, nodes/elements representing:

Boundary conditions Slide surfaces ProbesWe need to allow users to cull away

unneeded subsets of the data.

24

Representing data relationships

Relationships between many decompositions are complicated.

Need sophisticated machinery to store this knowledge. Machinery must be customizable to give right “look and

feel”. Idea: incorporate Subset Inclusion Lattices (SIL)

25

Subset Inclusion Lattices (SILs)

SIL: directed, acyclic graph

Bipartite: one set of nodes for subsets, the other for categories

Encodes relationships between sets (i.e. subset relations)• Also encodes

“category” of that relation (i.e. material, processor ID, etc)

26

VisIt has full built-in support for SILs VisIt supports over fifty file formats.

• Now each can present data subsets to users in an appropriate way.

VisIt can build GUI windows, etc, based on a SIL construction. VisIt creates SILs based on basic information (material

names, # of processors, etc)• Only have to modify a SIL for advanced cases.

27

● Simple steering– Scalar inputs to the

simulation– Specified through a GUI

● Dashboard of status

● Built by extending our simulation monitoring

● Simulation code can specify a custom GUI window

– Loaded through a Qt UI file built with Qt Designer

– State is synchronized between VisIt and simulation

27

Simulation name

Simulation valuepairs

Simulation statusEngine controls

Commands control

Component name

Text value

Simulation steering

28

Quantitative analysis means different things to different people.

1) Techniques that span scientific domains (e.g. integration, volumes, surface areas, fluxes, etc.)

2) Techniques specific to scientific domains (e.g. hohlraum flux at AGEX)

Detectorat AGEX

Detectorprovided by VisIt

(synthetic diagnostic)

29

… and we’re done.

Tutorial:• Hank Childs (hchilds@lbl.gov), Sean Ahern (ahern@ornl.gov)• Script: http://www.visitusers.org/index.php?Title=Short_Tutorial

• Another tutorial upcoming SC09 User resources:

• Main website: http://www.llnl.gov/visit• Wiki: http://www.visitusers.org• Email: visitusers@ornl.gov

Development resources:• Email: visit-developers@ornl.gov• SVN: http://portal.nersc.gov/svn/visit

29