Upload
monte
View
41
Download
0
Embed Size (px)
DESCRIPTION
Road to Real-Time Order-Independent Transparency. Marco Salvi. Talk Outline. Motivation Compositing Equation Recursive Solvers Visibility Based Solvers State of the Art and Future W ork Q&A. Why Compositing?. - PowerPoint PPT Presentation
Citation preview
Beyond Programmable Shading Course, ACM SIGGRAPH 20111/66
Beyond Programmable Shading Course, ACM SIGGRAPH 20112/66
Road to Real-Time Order-Independent Transparency
Marco Salvi
Beyond Programmable Shading Course, ACM SIGGRAPH 20113/66
• Motivation• Compositing Equation
– Recursive Solvers
– Visibility Based Solvers
• State of the Art and Future Work• Q&A
Talk Outline
Beyond Programmable Shading Course, ACM SIGGRAPH 20114/66
“Reliance on a single program for rendering an entire scene is a poor strategy..”
“Separating the image into elements which can be independently rendered saves enormous time..”
Why Compositing?
“Compositing Digital Images”, @SIGGRAPH 1984Thomas Porter & Tom Duff
Beyond Programmable Shading Course, ACM SIGGRAPH 20115/66
Motivation
“Order-dependent transparency has always been a big limitation for content creators & developers
– Restrictive art pipeline: no glass houses
– Even windows on cars & buildings can be painful
– Restrictive interaction between objects”
“Order-independent transparency is must going forward– Big challenge! Gradual process”
“Five Major Challenges in Interactive Rendering”, @SIGGRAPH 2010Johan Andersson – DICE/EA
Beyond Programmable Shading Course, ACM SIGGRAPH 20116/66
Motivation
Alpha-Blending
Model courtesy of Cem Yuksel.
Beyond Programmable Shading Course, ACM SIGGRAPH 20117/66
Motivation
Alpha-Blending
Model courtesy of Cem Yuksel.
Order-Independent Transparency
Beyond Programmable Shading Course, ACM SIGGRAPH 20118/66
Motivation
Beyond Programmable Shading Course, ACM SIGGRAPH 20119/66
Motivation
Beyond Programmable Shading Course, ACM SIGGRAPH 201110/66
Motivation
Beyond Programmable Shading Course, ACM SIGGRAPH 201111/66
Motivation
Scene courtesy of Valve Corporation.
Order-Independent Transparency
Alpha-Test
Beyond Programmable Shading Course, ACM SIGGRAPH 201112/66
Motivation
Order-Independent Transparency
Alpha-Test
Scene courtesy of Valve Corporation.
Beyond Programmable Shading Course, ACM SIGGRAPH 201113/66
Compositing Equation
0
1
Distance from viewer (depth)
Tran
smitt
ance
pixel color =
Beyond Programmable Shading Course, ACM SIGGRAPH 201114/66
Compositing Equation
pixel color = …
Beyond Programmable Shading Course, ACM SIGGRAPH 201115/66
Ideal Real-Time OIT Method• High image quality
– Accurately evaluate compositing equation
– No major spatial and temporal artifacts
• High and stable performance– Low variability. Performance mostly independent from fragments ordering
• Bounded memory usage– No variable length data structures
Beyond Programmable Shading Course, ACM SIGGRAPH 201116/66
• Solve compositing equation recursively– Composite fragment with result of previous composite operation
• Solve compositing equation by computing and evaluating (compressed) visibility functions
OIT Algorithms Classification
Beyond Programmable Shading Course, ACM SIGGRAPH 201117/66
• Alpha Blending / Compositing• Depth Peeling• A-buffer• Z³ algorithm
Recursive Solvers
Beyond Programmable Shading Course, ACM SIGGRAPH 201118/66
Alpha Compositing [Porter and Duff 1984]
Beyond Programmable Shading Course, ACM SIGGRAPH 201119/66
Alpha Compositing [Porter and Duff 1984]
Beyond Programmable Shading Course, ACM SIGGRAPH 201120/66
Alpha Compositing [Porter and Duff 1984]
Beyond Programmable Shading Course, ACM SIGGRAPH 201121/66
Alpha Compositing [Porter and Duff 1984]
Beyond Programmable Shading Course, ACM SIGGRAPH 201122/66
Alpha Compositing [Porter and Duff 1984]
Beyond Programmable Shading Course, ACM SIGGRAPH 201123/66
Alpha Compositing [Porter and Duff 1984]
• Fast and stable/predictable performance
• No additional storage required
• But order-dependent..
Beyond Programmable Shading Course, ACM SIGGRAPH 201124/66
• Peel layers and composite in depth-sorted order
• Number of passes proportional to max image depth complexity
Depth Peeling [Everitt 2001][Liu et al. 2009]
EVERITT, C. 2001. Interactive order-independent transparency. Tech. rep.
Beyond Programmable Shading Course, ACM SIGGRAPH 201125/66
1) Render fragments color and depth in per-pixel lists
A-buffer [Carpenter 1984] [Yang et al. 2009]
Beyond Programmable Shading Course, ACM SIGGRAPH 201126/66
1) Render fragments color and depth in per-pixel lists
2) Per-pixel sort and composite fragments
A-buffer [Carpenter 1984] [Yang et al. 2009]
Beyond Programmable Shading Course, ACM SIGGRAPH 201127/66
1) Render fragments color and depth in per-pixel lists
2) Per-pixel sort and composite fragments
A-buffer [Carpenter 1984] [Yang et al. 2009]
to the frame buffer
Beyond Programmable Shading Course, ACM SIGGRAPH 201128/66
A-buffer Limitations
• Poor & unstable performance, tipically memory bandwidth limited
• Unbounded memory requirements
Scene courtesy of Valve Corporation.
Beyond Programmable Shading Course, ACM SIGGRAPH 201129/66
• Bounded A-buffer– Up to N fragments per pixel (sorted)
• Merge fragments to keep pixel memory footprint constant– Distance & coverage based compression
metric
The Z³ algorithm [Jouppi and Chang 1999]
JOUPPI, N. P., AND CHANG, C.-F. 1999. Z3: an economical hard- ware technique for high-quality antialiasing and transparency.
Beyond Programmable Shading Course, ACM SIGGRAPH 201130/66
Visibility Function
• Models light absorption
• Product of step functions (thin blockers)
0
1
Distance from viewer (depth)
Tran
smit-
tanc
e
Beyond Programmable Shading Course, ACM SIGGRAPH 201131/66
• Fragment Parallel Compositing• Occupancy Maps• Opacity Shadow Methods• Stochastic Transparency• Adaptive Transparency
Visibility Based Solvers
Beyond Programmable Shading Course, ACM SIGGRAPH 201132/66
• Compute visibility via parallel segmented scan– Load-balance across irregular number of fragments per
pixel
• Evaluate compositing equation via parallel reduction
Frag. Parallel Compositing [Patney et al. 2010]
pixel color = …
Beyond Programmable Shading Course, ACM SIGGRAPH 201133/66
• Assume all transp. geom. has the same alpha– Not applicable to many objects (smoke, foliage, etc.)
• Bit-field represents step functions of same height and spaced at regular depth– A slab map allows to re-modulate steps height on a given depth region
• Use visibility representation for OIT & shadows
Occupancy Maps [Sintorn et al. 2009]
Beyond Programmable Shading Course, ACM SIGGRAPH 201134/66
• Replace visibility with opacity:– Less well-behaved function (no monotonicity, no [0,1] bounds)
• Basis function– Piece-wise constant intervals: Opacity Shadow Maps [Kim et al. 2001]
• Warp OSMs first layer: Deep Opacity Shadow Maps [Yuksel et al. 2008]
– Trigonometric: Fourier Opacity Mapping [Jansen et al. 2010]
Opacity Mapping Methods
Beyond Programmable Shading Course, ACM SIGGRAPH 201135/66
• Fixed length visibility representation– Regular steps at irregular locations
– A2C & z-test on MSAA samples• Compression by removing random step
• Efficient use GPU fixed function HW
• Fast but “noisy” with complex objs– Can require many samples per-pixel
Stochastic Transparency [Enderton et al. 2010]
ENDERTON, E., SINTORN, E., SHIRLEY, P., AND LUEBKE, D. 2010. Stochastic transparency.
Beyond Programmable Shading Course, ACM SIGGRAPH 201136/66
• Derived from volumetric shadow method [Salvi et al. 2010]
– Optimized for thin objs and OIT
• Store up to N steps per pixel– Arbitrary location and height
– Area based compression metric
Adaptive Transparency [Salvi et al. 2011]
Salvi M., Montgomery J., Lefohn A. 2001. Adaptive Transparency
Beyond Programmable Shading Course, ACM SIGGRAPH 201137/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000001
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201138/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000001
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201139/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000001
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201140/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201141/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201142/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1smallest area variation
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201143/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201144/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201145/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201146/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201147/66
• To compress visibility AT removes the node that generates the smallest area variation
0.700000000000002
1smallest area variation
0.700000000000002
1
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201148/66
Adaptive Transparency [Salvi et al. 2011]
FOREST scene8 ms - 6.0 MFragmentMax fragment per pixel: 457x faster than A-buffer2x faster than Stoc. Transp.
SMOKE scene21 ms - 10.6 MFragmentMax fragment per pixel: 31230x faster than A-buffer2.5x faster than Stoc. Transp.
HAIR scene48 ms - 15.0 MFragment
Max fragment per pixel: 66340x faster than A-buffer
2x faster than Stoc. Transp.
Scene courtesy of Valve Corporation.
Model courtesy of Cem Yuksel.
Scene courtesy of Valve Corporation.
Beyond Programmable Shading Course, ACM SIGGRAPH 201149/66
• Higher quality than other “lossy” OIT methods– Works on any transparent object type (foliage, smoke, glass, etc.)
• Designed to run in fixed memory– Prototype uses variable memory data struct due to 3D APIs limitations
• High and scalable performance– Easy to trade-off IQ for performance and storage by tuning per-pixel
step/node count
Adaptive Transparency [Salvi et al. 2011]
Beyond Programmable Shading Course, ACM SIGGRAPH 201150/66
• AT & ST are good examples of new algorithms close to ideal OIT method for real-time apps– ST main limitation is noise / lower quality per vis. rep. storage
– AT main limitation is current variable memory implementation
• We are rapidly converging towards entirely practical and robust OIT methods for real-time rendering!
What’s Next?
Beyond Programmable Shading Course, ACM SIGGRAPH 201151/66
• Deferred shading and OIT methods– Many methods are trivial to extend to DSers..
– ..but awfully inefficient..
• Some interesting work left to do in this area
Open Problems
Beyond Programmable Shading Course, ACM SIGGRAPH 201152/66
Acknowledgements
• Special thanks to Jefferson Montgomery and Aaron Lefohn
• Craig Kolb, Matt Pharr, Charles Lingle and Elliot Garbus for supporting this work.
• Jason Mitchell and Wade Schinn at Valve Software for the assets from Left-for-Dead-2
• Cem Yuksel (Cornell University) for the hair model
Beyond Programmable Shading Course, ACM SIGGRAPH 201153/66
Q&A
“Adaptive Transparency” pre-print: http://intel.ly/at_hpg11
“Adaptive Transparency” source code: http://intel.ly/aoit_gdc
twitter: @marcosalvi
e-mail: [email protected]
blog: http://pixelstoomany.wordpress.com
Beyond Programmable Shading Course, ACM SIGGRAPH 201154/66
Backup
Beyond Programmable Shading Course, ACM SIGGRAPH 201155/66
Bibliography• PORTER, T., AND DUFF, T. 1984. “Compositing digital images.” SIGGRAPH Comput. Graph. 18, 3, 253–
259.
• CARPENTER, L. 1984. “The A -buffer, an antialiased hidden surface method.” SIGGRAPH Comput. Graph. 18, 3, 103–108.
• SINTORN, E., AND ASSARSON, U. 2009. “Hair self shadowing and transparency depth ordering using occupancy maps.” In I3D ’09: Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, 67–74
• PATNEY, A., TZENG, S., AND OWENS, J. D. 2010. “Fragment parallel composite and filter.” Computer Graphics Forum (Proceedings of EGSR 2010) 29, 4 (June), 1251–1258.
• YANG, J., HENSLEY, J., GR¨U 547 N, H., AND THIBIEROZ, N. 2010. “Real-time concurrent linked list construction on the gpu.” Computer Graphics Forum (Proceedings of EGSR 2010) 29, 4 (June),
• SALVI, M., VIDIMCE, K., LAURITZEN, A., AND LEFOHN, A. 2010. “Adaptive volumetric shadow maps.” Computer Graphics Forum (Proceedings of EGSR 2010) 29, 4 (June), 1289–1296.
Beyond Programmable Shading Course, ACM SIGGRAPH 201156/66
• Idea: Save bandwidth by working with an approximate visibility function
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce 200+ steps
Beyond Programmable Shading Course, ACM SIGGRAPH 201157/66
• Idea: Save bandwidth by working with an approximate visibility function
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce
200+ steps
Beyond Programmable Shading Course, ACM SIGGRAPH 201158/66
• Idea: Save bandwidth by working with an approximate visibility function
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce 32 steps
200+ steps
Beyond Programmable Shading Course, ACM SIGGRAPH 201159/66
• Idea: Save bandwidth by working with an approximate visibility function
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce 32 steps
200+ steps
Beyond Programmable Shading Course, ACM SIGGRAPH 201160/66
• Idea: Save bandwidth by working with an approximate visibility function
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce
0
1
Distance from viewer (depth)
Tran
smit
-ta
nce 32 steps
200+ steps
Beyond Programmable Shading Course, ACM SIGGRAPH 201161/66
AT GPU Implementation
• Store visibility in the frame buffer?– Data update cannot be mapped to DX11 blend modes
– No RMW operations on the frame buffer
• Store visibility in a Read/Write buffer (UAV)?– Cause data races
Beyond Programmable Shading Course, ACM SIGGRAPH 201162/66
AT Proof-of-Concept Implementation
1) Render transparent fragments to per-pixel lists– Same as A-buffer implementation
2) For each pixel: build an approximate visibility function and use it to composite all transparent fragments
– Full-screen pass guarantees atomicity
Beyond Programmable Shading Course, ACM SIGGRAPH 201163/66
Bandwidth Requirements
Beyond Programmable Shading Course, ACM SIGGRAPH 201164/66
AT Future Work
• Investigate bounded memory implementations– Per-pixel locks? New frame-buffer format?
• Better visibility data compression– Reduce MSAA impact on memory requirements
Beyond Programmable Shading Course, ACM SIGGRAPH 201165/66
Scene courtesy of Valve Corporation.
Beyond Programmable Shading Course, ACM SIGGRAPH 201166/66
• Capture fragments in FIFO– Enable efficient multi-pass shading
• Require global sort– A-buffer requires local sort
• FIFO can overflow– Similar to A-buffer
F-Buffer [Mark and Proudfoot 2001]
MARK, W. R., AND PROUDFOOT, K. 2001. The F-buffer: A rasterization-order FIFO buffer for multi-pass rendering