paraview infos

1

Advanced ParaView VisualisationTips and tricks beyond GMV-like operations

Graduate Class Numerical Simulation / IANUS

Technische Universität Dortmund

Sven Buijssen [email protected]

November, 5th 2009

mailto:[email protected]

2

Outline

1 Choosing suitable ParaView module2 Client/Server setup3 Exporting vector graphics (PS, EPS, SVG)4 Converting data to fulfill filter’s input requirement5 Emulate gmvdiff in ParaView6 Emulate gmvpt for steady state data set / Stream Tracer7 Emulate gmvpt for transient data / Particle Tracer8 Equivalent of gmvmpeg9 Examples Programmable filter10 Custom filters11 Lookmarks12 Live Data13 Stability of ParaView14 Learn more about ParaView

3

Outline


4

Choosing a suitable module

ParaView is a big package and can be configured and compiled againstnumerous third-party libraries, e.g.:

without FFMPEG: create sequence of PNGs, TIFs or JPGs from GUIwith FFmpeg:additionally create AVIs directly from GUIwith Python:possible to write small filters in GUI, no need to code C++without MPI:Only 1 ParaView data server possible.with MPI:Arbitrary number of ParaView data/render servers possible (clusterswith/without GPUs)with OpenGL:Use GPU on compute server (if not careful: GPU on display host!)with Mesa:Use software rendering for hosts without GPUs

5

Why compiling ParaView yourself?

Kitware Inc. provides ready-to-use binaries for Linux, Mac OS andWindows. Why bother compiling ParaView yourself?ParaView is a big package and as such contains numerous minor andmajor bugs. New releases are issued every 4–6 months. Find a bug,report it and a fix may be available in CVS/Git long before a new binaryrelease is available.

Both developing plugins and compile-time changes (e.g. Postscriptexport for rendered objects) require self-compilation.Provided binaries are without MPI support. Because MPI isimplemented with different (internal) APIs by different vendors.Kitware Inc. would have to create a different binary for crossproduct of operating system and MPI implementation.

6

Choosing a suitable module at TU Dortmund (1)

The workstations & servers at Institute of Applied Mathematics andon LiDOng clusters have multiple ParaView installations:

Official ParaView releases (version 3.4.0 and 3.6.1)Multiple CVS builds. Main difference there: with OpenGL or withMesa.

Why care? Because running an unsuitable version may result inslower performance than possible(Mesa means software rendering, Open GL implies hardware rendering)

segfault on startup ⇒ You’re lucky

segfault (because of invalid GL command) any time when building apipeline in GUI ⇒ You loose a pipeline and have wasted your time.(Segfault may happen for ’gmvgl’, too, when run in context without OpenGLsupport.)

7


Choices at one of the workstations & serversat Institute of Applied Mathematics:

Linux 32 bit workstations (most of them have GPUs):User sits in front of the workstation

⇒ feasible and recommended to use OpenGL version, e.g.% module load \

paraview/3.7.0-gl-cvs2009-10-28-postscriptexport-gmvimport

User is connected to workstation via ssh (e.g. in a NX session) oruses one of the 32 bit compute servers without a GPU

⇒ use Mesa version!! E.g.% module load \

paraview/3.7.0-mesa-cvs2009-10-28-postscriptexport-gmvimport

Linux 64 bit workstation and servers (only a few have GPUs,but those are reserved for GPU computing, not visualisation):

⇒ use Mesa version, e.g.% module load \


8


Choices at one of the workstations & serversat Institute of Applied Mathematics:

Solaris SPARC servers, i.e. SunRay users:Sun Solaris 10’s OpenGL has a bug that causes PV to segfault andtypically crashes the Xserver, too (⇒ immediate logout!)



64 bit Itanium server:no ParaView installedWindows terminal server:no ParaView installed

9


Choices on LiDOng cluster:Linux 64 bit gateway servers and compute nodes(neither one has GPU(s)):



10

Outline


11

Choosing a suitable module for client/server setup (1)

ParaView supports separating the tasks of data display, reading ofdata and rendering of data.By default, ParaView uses ’builtin’ mode:reading, rendering and display on one machineVia ’Connect’ method ParaView client can connect to local orremote ParaView server processes

In case of a remote server log in via ssh and create a tunnel forParaView connection. Example:% ssh lidong1.hrz.tu-dortmund.de \

-L<client port>:localhost:<server port>

Caution: ensure that ports are not already in use!Check this beforehand e.g. with% netstat -an | grep <client-side port number>on host where ’ssh’ is invoked and% netstat -an | grep <server-side port number>on host you are connecting to.In case of a local multi-core machine (that noone else uses)skip this step and use default ports 11111 in the following

12


Example (continued):2 Load suitable module

% module load paraview/3.7.0-cvs2009-10-283 Start ParaView data+render server processes

Start 1 process% pvserver \

–server-port=<server port> –use-offscreen-renderingStart multiple processes% mpirun -np 16 pvserver \

–server-port=<server port> –use-offscreen-rendering

4 Run paraview client (GUI) locally and choose ’Connect’ tolocal port <client port>% paraview

File - ConnectAdd server.Name: LiDOng via ssh tunnelServer Type: Client/ServerHost: localhostPort: <client port>OkSelect entry and press Connect

13


Example (continued):5 Test with e.g. Sources - Cone - Apply6 Distribute data to parallel processes (aka partition) by applying ’D3’

filter7 Visualise partitioning by applying ’Process Id Scalars’ filter

14


Caution:When running ParaView in client/server mode, do not mix versions!

Never use e.g. ParaView 3.6.1 in combination with ParaView 3.7In general do not mix ParaView 3.7 <some date> withParaView 3.7 <some later date>, either.

15

Outline


16

Exporting vector graphics

By default, ParaView supports screenshots in file formats PNG,BMP, TIFF, PPM, JPG and PDF. All are bitmap based formats.True vector graphics (PS, EPS and SVG) export is only available inself-compiled ParaView, not in official binary releases(Not sure whether it is possible to convert the required changes into a plugin)

Compile time requirements: my Postscript patch plus old versions ofclass vtkPVScalarBarActor (pre-November 2008)Run time requirement: Disable ’Interpolate Colors’ checkbox inObject Inspector, Display tab.Otherwise object will be exported with greyscale colormap!Works only for 3D view (i.e. not for 2D view nor bar char/xyplot/spreadsheet view...)Works only in built-in mode, not in server modeDoes support all types of source objects(sole exception: Mandelbrot source) and mesh typesLive demonstration

17

Outline


18

Converting data to fulfill filter’s input requirement

Most filters require certain types of mesh as inputTo convert your data set to required format apply one of

Clean to Grid filterDelaunay 2D filterDelaunay 3D filterExtract Surface filterTriangulate filterTetrahedralize filterTesselate filter

Live demonstration:2D data set from level 7, Extract Surface, Triangulate, Decimate⇒ ≈ data set on level 2.Use case: reduce memory imprint of large file series, createanimation scene with reduced dataset (→ faster prototyping), savestate file, edit it to use the original file series, finally use ParaViewGUI or pvpython script to create high-resolution animation

19

Outline


20

Emulate gmvdiff in ParaView

Jens Acker has written ’gmvdiff’, a command line tool that imports two (2D/3D) gmvfiles (possibly using different meshes) and creates a new gmv file containing dataarrays being the difference of data arrays in the input files.Use case: How did velocity and pressure change between T=3.0 and T=3.5?

Realisation in ParaView:Load both data setsApply ’Calculator’ filter on each of them to give field of interest aunique name (e.g. ’velocity’ → ’vel1’ / ’vel2’)Apply ’Append Attributes’ filter to both Calculator objectsApply ’Calculator’ and compute difference between the two fieldusing their unique names (operation: ’vel1 - vel2’)In case of different meshes, apply a ’Resample with dataset’ filterbefore using ’Append Attributes’ filter.

21

Outline


22

Emulate gmvpt in ParaView (1)

Jens Acker has written ’gmvpt’, a command line tool that imports a sequence of(2D/3D) gmv files providing velocity data arrays, performs particle tracing based on aASCII configuration file and creates a sequence of new gmv files containing tracers.

Realisation in ParaView for steady state data set:Load data setApply ’Stream Tracer’ filterPosition ’Point Source’ or ’Line Source’ somewhere inside of thegeometry

23

Outline


24

Emulate gmvpt in ParaView (2)

Jens Acker has written ’gmvpt’, a command line tool that imports a sequence of(2D/3D) gmv files providing velocity data arrays, performs particle tracing based on aASCII configuration file and creates a sequence of new gmv files containing tracers.

Realisation in ParaView for transient data:Load in a time-dependent file seriesIn case of GMV files possibly convert them to ExodusII file format first:

1 Loading of ExodusII files faster due to smarter file format (no need to parse all filesfor time information, see previous tutorial on GMV reader plugin)

2 ParticleTracer filter will cause GMV reader plugin to raise (harmless) warnings of kind“vtkFileSeriesReader: vtkTemporalDataSet not fully supported.”

Apply ’Temporal Cache’ filter first (mandatory!),set to cache 2 time stepsCreate ’Point Source’ somewhere inside of the geometryApply ’ParticleTracer’ filter (Input: TemporalCache object, Source: PointSource)

Toggle visibility of TemporalCache object in Pipeline browserOptional: Apply ’Pathlines’ filter

25

Outline


26

Equivalent of gmvmpeg3 / gmvmpeg4

gmvmpeg is a script of mine that automates the process of applying a given attributefile to a gmv file series, creating snapshots and converting them into a movie.

ParaView supports loading and saving of state files (equivalent toGMV’s attribute files), from GUI and in plain Python scriptsManipulation of state files from within Python possible, but tricky:Following scenario as replacement for gmvmpeg4 in principlepossible, but not mastered yet:

Load a single file or a small file series, set up a pipeline, save stateLoad state file from Python script and let script replace file seriesfrom state file with a different (complete) file (series)Let Python script generate snapshot/animation

Easier: Edit state file (plain XML file) and alter file name references.Automate this in a custom shell script.Additionally: ParaView supports generation of AVIs as well as savingsequences of PNGs, TIFs and JPGs directly from GUI.⇒ No replacement for my gmvmpeg scripts available nor planned

Bitmap file series can be converted to movie by means of, e.g., Unix command linemencoder "mf://animation*.png" -mf type=png:w=1280:h=444:fps=25 -nosound \

-ovc lavc -lavcopts vcodec=wmv2:vbitrate=1200 -o wmv2.avi

27

Outline


28

Examples Programmable filter

Problem: Export data as CSV file, but only some of the data, not all.No ready-to-use solution in ParaViewPossible with the following pipeline:

Apply ’Extract Block’ and ’Merge Blocks’ filter for multi-block datasetApply Programmable Filter with the following Python source code:

inp = self.GetInput()filename = "foo.csv"

FILE = open(filename, "w")FILE.write("x,y,z, x-velocity\n")numPoints = inp.GetNumberOfPoints()vel = inp.GetPointData().GetArray("velocity")for i in xrange(0, numPoints):

coords = inp.GetPoint(i)[:3]FILE.write(str(coords[0]) + ",")FILE.write(str(coords[1]) + ",")FILE.write(str(coords[2]) + ",")FILE.write(str(vel.GetValue(i * 3)) + "\n")

FILE.close()

29

Examples Programmable filter (cont.)

Problem: Node and cell numbers should start at 1, not 0.VTK always starts counting at 0, so does ParaView.

No ready-to-use solution in ParaViewPossible with the Programmable Filter(apply ’Extract Block’ + ’Merge Blocks’ filter for multi-block data sets first)

inp = self.GetInput()out = self.GetOutput()

# Provide all input data sets ourselvesout.ShallowCopy(inp)

offset = +1

# Shift point number by 1numPoints = inp.GetNumberOfPoints()

newData = vtk.vtkIdTypeArray()newData.SetName(’Point IDs (shifted by 1)’)newData.SetNumberOfComponents(1)for i in xrange(0, numPoints):

newData.InsertNextValue(i+offset)out.GetPointData().AddArray(newData)

# (continued in next column)

# (continued from left column)

# Shift cell number by 1numCells = inp.GetNumberOfCells()data = inp.GetCellData().GetArray("IDs")

newData = vtk.vtkIdTypeArray()newData.SetName(’Cell IDs (shifted by 1)’)newData.SetNumberOfComponents(1)for i in xrange(0, numCells):

newData.InsertNextValue(i+offset)out.GetCellData().AddArray(newData)

30

Examples Programmable filter (cont.)

Problem: Plot number of nodes and cell numbers over time.No ready-to-use solution in ParaViewPossible with the Programmable Filter(apply ’Extract Block’ + ’Merge Blocks’ filter for multi-block data sets first)

inp = self.GetInput()out = self.GetPolyDataOutput()

newPoint = vtk.vtkPoints()newPoint.InsertPoint(0,0,0,0)out.SetPoints(newPoint)

numPoints = inp.GetNumberOfPoints()numCells = inp.GetNumberOfCells()

newData = vtk.vtkUnsignedLongArray()newData.SetName("numPoints")newData.SetNumberOfComponents(1)newData.InsertNextValue(numPoints)out.GetPointData().AddArray(newData)# (continued in next column)

# (continued from left column)

newData1 = vtk.vtkUnsignedLongArray()newData1.SetName("numCells")newData1.SetNumberOfComponents(1)newData1.InsertNextValue(numCells)out.GetPointData().AddArray(newData1)

Select newly created pointApply ’Plot Selection over Time’ filter

31

Outline


32

Custom filters

Provide a means of grouping and saving a set of filters (asub-pipeline) for future use as a new filter.Can be exported and imported easily (shareable, XML-based format)Live demonstration (Save last ’Programmable Filter’ example asCustom filter, store it as ’Demo - Plot Number of Points & Cellsover Time’)

33

Outline


34

Lookmarks

Are (more or less) custom filters plus stored ’color by’ settings and(optionally) stored camera positionsFacilitate reproduction of pipeline for another source (e.g. readersource)Are created via Tools - Create LookmarksPersist through sessions if savedA lookmark’s camera settings are restored if the corresponding checkbox is enabled in the Lookmarks Inspector (accessible via View menuor by selecting edit in a lookmark’s context menu)Live demonstration

35

Outline


36

Live Data

GMV supports auto-read of new files (through busy-wait⇒ 100% cpu load ⇒ discouraged use case)Difficult to realise in VTK and ParaView and not available fordefault ParaView readers

Reason:No means in general to guarantee consistency of new file, i.e. has itbeen written completely yet by external application such that it issafe to read the file?

Has been discussed frequently in the past (see e.g. discussionhttp://markmail.org/message/734wfelphy3uepjr) and realised atleast once (by Takuya Oshima for his parallel OpenFOAM readerplugin, accepting the fact of the mentioned race conditions). Noplans to incorporate this into the GMV reader plugin.

37

Outline


38

Stability of ParaView

2,000,000 million lines of code (non-blanks, non-comments)(see http://www.ohloh.net/p/3249/analyses/latest)rule of thumb in computer science: 1 critical error per 1,000 lines ofcode ⇒ many bugs in ParaView!Public bug tracker: http://www.paraview.org/Bug/(Note: Just don’t upload any GMV files as GMV reader plugin is neither yetknown nor available outside our department. Try converting to one of VTK fileformats if necessary to reproduce bug.)

Evaluation of StabilityVery stable for standard GMV-like operationsExpect frequent segmentation faults when trying to create high glossanimations. Save state files often!Better: Make sure to activate "Crash Recovery" checkbox in Edit -Settings - General.Official releases more stable than CVS HEAD versions. The latterhave nice new features, though. (Official release cycle: 4–6 months)

39

Outline


40

Learn more about ParaView

http://www.paraview.org/Wiki/The_ParaView_Tutorialhttp://www.paraview.org/Wiki/IEEE_Vis08_ParaView_Tutorial(mirror: http://vgtc.org/wpmu/vis08/2008/11/03/advanced-paraview-visualization/)http://www.paraview.org/Wiki/SNL_ParaView_3_Tutorialshttp://www.paraview.org/Wiki/ParaViewParaView mailing list

Thank you for your attention.

Documents

paraview infos