Creating Animations with ParaView Visualization Technology Workshop SuperComputing 20014 November 17, 2014 Bill Sherman shermanw@iu.edu Advanced Visualization Lab Indiana University

The Visualization Process

The Visualization Process

Visualization dichotomy

•  Batch vs. Interactive

•  Interactive: •  Explorative •  Immediate feedback •  Performance constraints

•  Batch: •  Pre-determined •  Big-data / big-processing •  HPC-ready

bigred2%  qsub  render-­‐data.pbs  

vs.  

Visualization dichotomy Visualization process •  Batch vs. Interactive •  Interactive è Batch

•  Interactive: •  Explore •  Reduced data

•  Batch: •  Guided by exploration •  Full data / high-res

bigred2%  qsub  render-­‐data.pbs  

then  

What is ParaView?

•  An open-source, scalable, multi-platform visualization application.

•  Support for distributed computation models to process large data sets.

•  An open, flexible, and intuitive user interface. •  An extensible, modular architecture based on open

standards. •  A flexible BSD-2 license. •  Commercial maintenance and support.

Data Types

Uniform Rectilinear (vtkImageData)

Non-Uniform Rectilinear (vtkRectilinearGrid)

Curvilinear (vtkStructuredGrid)

Polygonal (vtkPolyData)

Unstructured Grid (vtkUnstructuredGrid)

Multi-block Hierarchical Adaptive

Mesh Refinement (AMR)

Hierarchical Uniform AMR

Octree

Supported Data Types •  ParaView  Data  (.pvd)  •  VTK  (.vtp,  .vtu,  .v>,  .vts,  .vtr)  •  VTK  Legacy  (.vtk)  •  VTK  Mul>  Block  

(.vtm,.vtmb,.vtmg,.vthd,.vthb)  •  Par>>oned  VTK  

(.pvtu,  .pv>,  .pvts,  .pvtr)  •  ADAPT  (.nc,  .cdf,  .elev,  .ncd)  •  ANALYZE  (.img,  .hdr)  •  ANSYS  (.inp)  •  AVS  UCD  (.inp)  •  BOV  (.bov)  •  BYU  (.g)  •  CCSM  MTSD  (.nc,  .cdf,  .elev,  .ncd)  •  CCSM  STSD  (.nc,  .cdf,  .elev,  .ncd)  •  CEAucd  (.ucd,  .inp)  •  CMAT  (.cmat)  •  CTRL  (.ctrl)  •  Chombo  (.hdf5,  .h5)  •  Claw  (.claw)  •  Comma  Separated  Values  (.csv)  •  Cosmology  Files  

(.cosmo,  .gadget2)  •  Curve2D  (.curve,  .ultra,  .ult,  .u)  •  DDCMD  (.ddcmd)  •  Digital  Eleva>on  Map  (.dem)  

•  Dyna3D(.dyn)  •  EnSight  (.case,  .sos)  •  Enzo  boundary  and  hierarchy  •  ExodusII  (.g,  .e,  .exe,  .ex2,  .ex2v..,  etc)  •  ExtrudedVol  (.exvol)  •  FVCOM  (MTMD,  MTSD,  Par>cle,  

STSD)    •  Facet  Polygonal  Data  •  Flash  mul>block  files  •  Fluent  Case  Files  (.cas)  •  GGCM  (.3df,  .mer)  •  GTC  (.h5)  •  GULP  (.trg)  •  Gadget  (.gadget)  •  Gaussian  Cube  File  (.cube)  •  JPG  Image  (.jpg,  .jpeg)  •  LAMPPS  Dump  (.dump)  •  LAMPPS  Structure  Files  •  LODI  (.nc,  .cdf,  .elev,  .ncd)  •  LODI  Par>cle  (.nc,  .cdf,  .elev,  .ncd)  •  LS-­‐DYNA  (.k,  .lsdyna,  .d3plot,  d3plot)  •  M3DCl  (.h5)  •  MFIX  Unstructred  Grid  (.RES)  •  MM5  (.mm5)  •  MPAS  NetCDF  (.nc)  •  Meta  Image  (.mhd,  .mha)  

•  Miranda  (.mir,  .raw)  •  Mul>level  3d  Plasma  (.m3d,  .h5)  •  NASTRAN  (.nas,  .f06)  •  Nek5000  Files    •  Nrrd  Raw  Image  (.nrrd,  .nhdr)  •  OpenFOAM  Files  (.foam)  •  PATRAN  (.neu)  •  PFLOTRAN  (.h5)  •  PLOT2D  (.p2d)  •  PLOT3D  (.xyz,  .q,  .x,  .vp3d)  •  PLY  Polygonal  File  Format  •  PNG  Image  Files  •  POP  Ocean  Files  •  ParaDIS  Files  •  Phasta  Files  (.pht)  •  Pixie  Files  (.h5)  •  ProSTAR  (.cel,  .vrt)  •  Protein  Data  Bank  (.pdb,  .ent,  .pdb)  •  Raw  Image  Files  •  Raw  NRRD  image  files  (.nrrd)  •  SAMRAI  (.samrai)    •  SAR  (.SAR,  .sar)    •  SAS  (.sasgeom,  .sas,  .sasdata)    •  SESAME  Tables  •  SLAC  netCDF  mesh  and  mode  data    

•  SLAC  netCDF  par>cle  data  •  Silo  (.silo,  .pdb)  •  Spheral  (.spheral,  .sv)  •  SpyPlot  CTH  •  Spy  Plot  (.case)  •  Stereo  Lithography  (.stl)  •  TFT  Files  •  TIFF  Image  Files  •  TSurf  Files  •  Tecplot  ASCII  (.tec,  .tp)  •  Tecplot  Binary  (.plt)  •  Tetrad  (.hdf5,  .h5)  •  UNIC  (.h5)    •  VASP  CHGCA  (.CHG)  •  VASP  OUT  (.OUT)    •  VASP  POSTCAR  (.POS)    •  VPIC  (.vpc)  •  VRML  (.wrl)  •  Velodyne  (.vld,  .rst)  •  VizSchema  (.h5,  .vsh5)  •  Wavefront  Polygonal  Data  (.obj)  •  WindBlade  (.wind)  •  XDMF  and  hdf5  (.xmf,  .xdmf)  •  XMol  Molecule  

Basic User Interface Menu Bar

Toolbars

Pipeline Browser

Object Inspector

3D View

Geometry Representations

Points Wireframe Surface Surface with Edges

Volume

Calculator  

Contour  

Clip  

Slice  

Threshold  

Extract  Subset  

Common Filters

Glyph  

Stream  Tracer  

Warp  (vector)  

Group  Datasets  

Extract  Level  

Filters Menu

Multiple simultaneous data views:

ParaView usage at Indiana University

•  Large complement of features addressing time-varying computational data

•  Can be learned and used directly by researchers •  Runs on desktop and HPC

•  the same HPC systems researchers already using •  Collaborations with:

•  Astronomy •  Geology •  Nuclear Physics

Animation with ParaView

•  capable of reading time-varying data •  sequence of VTK files •  ParaView format (XML pointing to a sequence of files)

•  “temporal interpolator” filter to in-between the data •  Control pane for basic animation operations •  choice of animation mode:

•  sequence •  real time •  snap to timesteps

Animation Toolbar control pane

First Frame

Previous Frame Play Next

Frame Last

Frame Loop

Animation Current Time

Current Time Step

What can be animated?

•  Time (of course) •  Camera •  Annotation

•  special “time” annotation source

•  Object properties •  sphere resolution •  visibility •  …

•  Python script!

Animated Python script “Cue”

•  Three methods: •  start_cue(self) •  tick(self) •  end_cue(self)

Animated Python script “Cue”

def start_cue(self): print 'Animation starting' print 'NumberOfFrames = ' + str(scene.NumberOfFrames) def tick(self): i = int(scene.TimeKeeper.Time) print "time is " + str(i) view.ViewTime = i scene.AnimationTime = i Render() def end_cue(self): print 'Animation ending'

Batch Animations with Python script

•  ParaView State can be saved in a “StateFile” (.pvsm) •  Statefiles can be loaded into interactive shell or batch python:

•  pvpython •  pvbatch

•  Once a statefile is loaded, animation parameters can be set and run from paraview.simple import * servermanager.LoadState(”myvis.pvsm") SetActiveView(GetRenderView()) view = GetActiveView() scene = GetAnimationScene()

Batch Animations with Python script (cont) Python Cue

PythonAnimationCue1 = PythonAnimationCue() PythonAnimationCue1.Script= """ def start_cue(self): print 'Animation starting' print 'NumberOfFrames = ' + str(scene.NumberOfFrames) def tick(self): i = int(scene.TimeKeeper.Time) print "time is " + str(i) view.ViewTime = i scene.AnimationTime = i Render() def end_cue(self): print 'Animation ending' ""” scene.Cues.append(PythonAnimationCue1)

Batch Animations with Python script (cont) Tweak and Render

# Make some tweaks to the visualization sphere=FindSource('Glyph5').GlyphType sphere.PhiResolution=9 sphere.ThetaResolution=9 # Setup the rendering parameters scene.NumberOfFrames=1000 view.ViewSize = [1920,1080] view.StereoRender = 1 view.StereoType = "Left" view.UseOffscreenRendering = 1 WriteAnimation("animationC_L.png") view.StereoType = "Right" view.UseOffscreenRendering = 1 WriteAnimation("animationC_R.png")

ParaView & VTK Example (batch animation):

•  Write a Python animation script: •  aprun% cat 3Drender.py

from paraview.simple import * servermanager.LoadState("v2D_4-1cam-br.pvsm") SetActiveView(GetRenderView()) view = GetActiveView() scene = GetAnimationScene()

[…] # Setup the rendering parameters view.ViewSize = [1920,1080] view.StereoRender = 1 view.StereoType = "Left" view.UseOffscreenRendering = 1 WriteAnimation("initialV2D_L.png") view.StereoType = "Right" view.UseOffscreenRendering = 1 WriteAnimation("initialV2D_R.png")

What else can I add to my Python batch rendering script?

•  Time consumed rendering previous frame •  Running frame-render-time average •  Memory consumption (absolute and percentage) •  System page swaps in previous frame •  How many polygons created in last frame

Some of the problems

•  2D vs. 3D •  Camera control •  Normalized time •  Start at the beginning •  Rendering speed

2D vs. 3D

•  2D & 3D shouldn’t be opponents •  Textual annotations generally work •  No clean mechanism to cleanly blend 2D information over the 3D view

•  “narrative information” •  renderings of histograms, plots, etc. •  non-static text •  shapes representing a scalar value (e.g. time as “progress bar”)

Camera Control

•  ParaView offers 3 types of camera movement: •  “Orbit” •  “Follow Path” •  “Interpolate Camera Locations”

•  ParaView GUI client has no way to turn off computation to allow for quick rendering to evaluate path

•  Camera visualization only available when editing the move

•  Crude solution …

Camera Control: rendering camera data

•  Python source allows one to add “data” to scene •  (including polygonal data)

•  “Showcams.py”

Normalized Time

•  GUI interface shows time in seconds •  Internally this is stored as a fraction of the length

•  Good when just want to extend the length of the animation and have all the times expand

•  Bad (very bad) when trying to add new narrative moves at the end of an animation

•  Bill’s workaround •  vim & vimscript

Start at the beginning

•  ONLY!

•  Multi-hour batch rendering job on a queued HPC system may be terminated in the middle – no recovery! •  Justification

•  some filters (path generators) build up geometry over multiple time steps

•  practitioner could alleviate this by starting N-steps earlier

Rendering Speed

•  ParaView is not a rendering speed deamon •  Not clear how to best take advantage of “para” when rendering batch jobs

•  Breaking into small jobs would be an easy method •  If jobs could be started in the middle !!!

•  Perhaps made worse when using “rich-man’s anti-aliasing” •  (render big and scale down)

Other minor things

•  Several other little fiddly issues •  programmable source isn’t rendered each frame

•  (can use a programmable filter with “time” as input) •  Unclear how to adjust stereoscopic rendering via python script •  GUI numeric output issues

•  Animation timesteps shown in scientific notation – how many frames do they expect me to render?

•  Numbers with lots of “precision” show the least-important digits •  GUI bugs that don’t show all the cue names

Overall impressions

•  Good to be able to batch render from a GUI exploration •  Good that the rendering can be easily moved from desktop to HPC •  Most of the difficulties can be worked around / suffered through •  Python tools can ease the suffering

•  Really, really, really need to be able to start rendering in the middle

Ques>ons?  

(or  perhaps  a  video)