AMCS / CS 247 – Scientific Visualization Lecture 12 ... · GLSL 1.50 (OpenGL 3.2) • Vertex...

AMCS / CS 247 – Scientific VisualizationLecture 12: Volume Visualization, Pt. 2

Markus Hadwiger, KAUST

2

Reading Assignment #6 (until Oct. 22)

Read (required):• Real-Time Volume Graphics, Chapter 1

(Theoretical Background and Basic Approaches),from beginning to 1.4.4 (inclusive)

• Real-Time Volume Graphics, Chapter 3.2.3 (Compositing)

• Data Visualization book, Chapter 5 until 5.2 (inclusive)

3

Reading Assignment #7 (until Oct. 29)

Read (required):• Real-Time Volume Graphics, Chapters 5.3, 5.4, 5.5, 5.6

• Real-Time Volume Graphics, Chapter 7(GPU-Based Ray Casting)

• Real-Time Volume Graphics, Chapter 4 (Transfer Functions)until Sec. 4.4 (inclusive)

4

Programming Assignments Schedule (updated)

Assignment 3:• Volume ray-casting due: Oct 30

Assignment 4:• Flow vis 1 (hedgehog plots, streamlines, pathlines) due: Nov 20

Assignment 5:• Flow vis 2 (LIC with color coding) due: Dec 4

GPU‐Based Ray‐Casting

5

• “Natural” volume rendering method

• Image-order approach– Most common CPU approach– Well-supported by current GPUs!

• Standard optimizations– Early ray termination– Empty space skipping

(empty space leaping)

• Many possibilities– Adaptive sampling– Realistic lighting

6

Ray-Casting

Why Ray-Casting on GPUs?

Most GPU rendering is object-order (rasterization)

Image-order is more “CPU-like”• Recent fragment shader advances

• Simpler to implement

• Very flexible (e.g., adaptive sampling)

• Correct perspective projection

• Can be implementedin a single rendering pass!

• Native 32-bit float compositing

7

Recent GPU Ray-Casting Approaches

Regular grids• [Krüger and Westermann, 2003], [Röttger et al., 2003]

• [Green, 2004] (in NVIDIA SDK)

• [Stegmaier et al., 2005]

• [Scharsach et al., 2006]

• [Gobbetti et al., 2008]

Unstructured (tetrahedral, ...) grids• [Weiler et al., 2002, 2003, 2004]

• [Bernardon et al., 2004]

• [Callahan et al., 2006]

• [Muigg et al., 2007]

8

Single-Pass Ray Casting

• Enabled by conditional loops in fragment shaders(Shader Model 3.0 and higher / NVIDIA CUDA)

• Substitute multiple passes and early-z testing by single loop and early loop exit

• Volume rendering examplein NVIDIA CUDA SDK

9

Implementation

Ray setup

Loop over ray

Resample scalar value

Classification

Shading

Compositing

Markus Hadwiger, KAUST 10

Implementation

Ray setup

Loop over ray

Resample scalar value

Classification

Shading

Compositing

Markus Hadwiger, KAUST 11

Ray Setup

Two main approaches:• Procedural ray/box intersection

[Röttger et al., 2003], [Green, 2004]

• Rasterize bounding box[Krüger and Westermann, 2003]

Some possibilities• Ray start position and exit check

• Ray start position and exit position

• Ray start position and direction vector

12

Procedural Ray Setup/Termination

• Everything handled in the fragment shader / CUDA kernel

• Procedural ray / bounding box intersection

• Ray is given by camera positionand volume entry position

• Exit criterion needed

• Pro: simple and self-contained

• Con: full computational loadper-pixel/fragment

13

Rasterization-Based Ray Setup

• Fragment == ray

• Need ray start pos, direction vector

• Rasterize bounding box

• Identical for orthogonal and perspective projection!

- =

14

Implementation

Ray setup

Loop over ray

Resample scalar value

Classification

Shading

Compositing

Markus Hadwiger, KAUST 15

Classification – Transfer Functions

During Classification the user defines the “look“ of the data.• Which parts are transparent?

• Which parts have what color?

Classification – Transfer Functions

During Classification the user defines the “look“ of the data.• Which parts are transparent?

• Which parts have what color?

The user defines a transfer function.

Emission RGB

Absorption Ascalar S Transfer

Function

Transfer Function Application

texture = scalar field

transferfunction texture = [Emission RGB, Absorption A]

scalar value S

S

RGBA

T(S)resampling

Applying Transfer Function: Cg Example

// Cg fragment program for post-classification

// using 3D textures

float4 main (float3 texUV : TEXCOORD0,

uniform sampler3D volume_texture,

uniform sampler1D transfer_function) : COLOR

{

float index = tex3D(volume_texture, texUV);

float4 result = tex1D(transfer_function, index);

return result;

}

Windowing Transfer Function

Map input scalar range to output intensity range• Select scalar range of interest

• Adjust contrast

Markus Hadwiger, KAUST 20

Implementation

Ray setup

Loop over ray

Resample scalar value

Classification

Shading

Compositing

Markus Hadwiger, KAUST 21

Illumination Model

E.g. Phong: Sum of

V … View vector

L … Light vector (e.g., headlight model L=V)

R … Reflection vector of L at the surface

N … Surface normal

22

xy y

x

Convolution with Continuous Filters

Mixture of function and derivative reconstruction kernels

Apply three filter kernels for three gradient components

2D example with Gaussian kernels

Gradient Reconstruction

Central differences• Cheap and quality often sufficient (2+2+2 neighbors in 3D)

Discrete convolution filters on grid• Image processing filters; e.g. Sobel (3x3x3 neighbors)

Continuous convolution filters• Derived continuous reconstruction filters

• E.g., the cubic B-spline and its derivatives(4x4x4 neighbors)

On-the-fly Gradient Estimation

Pre-compute gradients at grid points with any method

Store normalized gradient directions in RGB texture

Sample gradient texture in fragment shader: interpolation

Re-normalize after fetch!

nX

nY

nZ

RGB

Pre-Computed Gradients

RGB gradient texture lerp of texture filter renormalize!

On-The-Fly Gradients

Reduce texture memory consumption!

Central differences before and after linear interpolationof values at grid points yield the same results

Caveat: texture filter precision

Filter kernel methods are expensive, but:

Tri-cubic B-spline kernels can be used in real-time(e.g., GPU Gems 2 Chapter “Fast Third-Order Filtering”)

Implementation

Ray setup

Loop over ray

Resample scalar value

Classification

Shading

Compositing

Markus Hadwiger, KAUST 28

Fragment Shader

• Rasterize front facesof volume bounding box

• Texcoords are volumeposition in [0,1]

• Subtract camera position

• Repeatedly check forexit of bounding box

CUDA Kernel

• Image-based raysetup

– Ray start image– Direction image

• Ray-cast loop– Sample volume– Accumulate

color and opacity

• Terminate

• Store output

30

GPU PROGRAMMING

Markus Hadwiger, KAUST 31

32

What‘s in a GPU?

Lots of floating point processing power• Stream processing cores

different names:stream processors,CUDA cores, ...

• Was vector processing, now scalar cores!

Still lots of fixed graphics functionality• Attribute interpolation (per-vertex -> per-fragment)

• Rasterization (turning triangles into fragments/pixels)

• Texture sampling and filtering

• Depth buffering (per-pixel visibility)

• Blending/compositing (semi-transparent geometry, ...)

• Frame buffers

What can the hardware do?

RasterizationDecomposition into fragmentsInterpolation of colorTexturing

Interpolation/Filtering Fragment Shading

Fragment OperationsDepth Test (Z-Test)Alpha Blending (Compositing)

Pixels

Graphics Pipeline

Vertices Primitives Fragments

GeometryProcessing

FragmentOperations

Scene Description Raster Image

Rasterization

Geometry Processing

Per-VertexLightingClipping,

Perspect.DividePrimitiveAssemblyTransformation

Multiplication withModelview and

Projection Matrix

Per-VertexLocal Illumination

(Blinn/Phong)

GeometricPrimitives

(Points, LinesTriangles)

Primitives

Clip SpaceTo

Screen Space

Vertices

GeometryProcessing Rasterization Fragment

Operations

GeometryProcessing Rasterization Fragment

OperationsFragment

Operations

TextureFetchTexture

ApplicationPolygon

Rasterization

PrimitiveVertices

Decompositionof primitives

into fragments

Interpolation oftexture coordinates

Filtering oftexture color

Primitives Fragments

Rasterization

Combination ofprimary color with

texture color

Combination ofprimary color with

texture color

Fragment Operations

StencilTestAlpha

BlendingDepthTest

AlphaTest

Discard allfragments within

a certain alpha range

Discard afragment ifthe stencil buffer is set

Discard alloccludedfragments

GeometryProcessing Rasterization Fragment

Operations

Pixels

Graphics Hardware

Vertices Primitives Fragments

GeometryProcessing

FragmentOperations

Scene Description Raster Image

RasterizationVertexShader

FragmentShader

Programmable Pipeline

39

Direct3D 10 Pipeline

New geometry shader stage:• Vertex -> geometry -> pixel shaders

• Stream output after geometry shader

Courtesy David Blythe, Microsoft

Programmable Processing Stages

GLSL 1.50 (OpenGL 3.2)• Vertex shaders (run on vertex processors)

• Geometry shaders (run on geometry processors)

• Fragment shaders (run on fragment processors)

From the language / API perspective it is useful to consider separate types of processors, even when all of these shaders are in reality executed on identical processing cores on current GPUs!

40

Vertex Shader (1)

Process vertices and their attributes• Position

• Color, texture coordinate(s), ...

41

GLSL 1.20

Vertex Shader (2)

“Pass-through“ example• Pass through per-vertex color

• Transform vertex position with OpenGL Model-View-Projection matrix

42

Geometry Shader (1)

Process whole primitives• Emit vertices

• Emit primitive(s)

43GL_EXT_geometry_shader4

Geometry Shader (2)

“Pass-through“ example• Pass through (emit) all vertices

• Pass through (emit) one primitive

44

Fragment Shader (1)

Process fragments• Write one or more output fragments

• Use input color, texture coordinates, ...

• Compute shading, sample textures, ...

• Optionally discard fragment

• MRT: multiple render targets

45

GLSL 1.20

Fragment Shader (2)

“Pass-through“ example• Pass through interpolated color as fragment color

46

Thank you.

Thanks for material• Helwig Hauser

• Eduard Gröller

• Daniel Weiskopf

• Torsten Möller

• Ronny Peikert

• Philipp Muigg

• Christof Rezk-Salama