Upload
derick-treadaway
View
220
Download
1
Tags:
Embed Size (px)
Citation preview
Ultra-Scale Visualization with
Open-Source SoftwareBerk GeveciKitware Inc.
Background
Kitware Is• A software company• creating open-source collaboration
platforms• which are used globally for
• research• teaching• commercial application.
• This software is created by• internationally recognized experts• in extended communities• using a rigorous, quality-inducing software
development process.
Technical Portfolio
Heart and Soul: Software Quality Process
In The Beginning There Was VTK
•From Ohloh: Very large, active development team: Over the
past twelve months, 66 developers contributed new code to VTK. This is one of the largest open-source teams in the world, and is in the top 2% of all project teams on Ohloh.
VTK Development Team
and many others...
Then Came ParaView
ParaView Architecture
Support for Large Displays
ParaView is VTK•ParaView is developed by the VTK team
•The ParaView team develops in the VTK repository : all development benefits the whole community
•ParaView leverages all features of VTK
Short Demo
Problem Statement
The Good
Ability to Compute
Ability to Communicate
Remote Access
The Bad
Ability toCompute>>Ability to
Transfer>> Ability toStore/Read
State of Large Data Vis
Parallel Visualization
Visualization Resources
•LANL Vis Cluster (2001): 128 dual-core nodes with NVIDIA Quadro FX 540
•Sandia Red Rose (2005): 264 dual-core nodes with NVIDIA Quadro FX 3400
•TACC Longhorn (2009): 256 8 core nodes with 2 NVIDIA Quadro FX 5800s
General Purpose Tools
• EnSight• ParaView• VisIt• FieldView• …
Specialized Tools
•VAPOR
•ViSUS
•...
1 billion cell asteroid detonation simulation (AMR)
½ billion element weather simulation (rectilinear)
150 million elementfire simulation (unstructured)
Where to Next?
Large Data, Small Bandwidth and
Small Vis Resources
• Vis on the supercomputer• Batch• Interactive
• Co-Processing• Multi-Resolution
Streaming
Ability to Compute
Vis on Supercomputer
Vis on Supercomputer
• Compiling• Parallel IO• Scalability• Software
Rendering• Compositing• Client/server• Resource allocation
ParaView on Supercomputers
•Blue Gene L and P
•Cray Xt 3, 4 and 5
•AIX
•Linux
Visualization of VPIC resultson Kraken (image courtesy of
Bill Daughton, LANL)
Co-Processing
1111
Co-Processing
Run-Time Visualization and
Setup
Library Architecture
• Based on VTK/ParaView• C, Fortran and Python bindings• Built-in Python interpreter (optional)• Full demand-driven pipeline• Distributed computing with MPI• Extract generation• Rendering (off-screen with Mesa)
Connection with the Simulation
simulation_initialize()
coprocessor_initialize()
for t in time_steps:
do_compute()
if coprocess_needed():
do_coprocess()
coprocessor_finalize()
simulation_finalize()
Simulations – the More the Merrier
• Overflow (overset curvilinear)• Phasta (unstructured)• Acusolve (unstructured)• Helios (unstructured - AMR hybrid)• CTH (AMR)• NPIC (structured)• ...
QuickTime™ and aPNG decompressor
are needed to see this picture.
Multi-Resolution Streaming
Data-Parallel Processing
Streaming
Prioritized Streaming
Adaptive Streaming
Collaboration and Web Visualization
Remote Access
Collaboration
Web Visualization
Architecture
Comparative Visualization
180
18
01
80
18
01
80
18
01
80
18
0
Data and Meta-Data Management
Loading Meta-Data in ParaView
Analysis and Subsetting of
Meta-Data
Aperture for Data-Analysis
Conclusions
Ability to Compute and Communicate
Bottlenecks
We need multiple solutions!
Interactive on Supercomputer
Co-Processing
Multi-Resolution Streaming
It’s About Collaboration
We need frameworks to
build specialized tools
General Purpose Tools
• EnSight• ParaView• VisIt• FieldView• …
Specialized Tools
•VAPOR
•ViSUS
•...
The End