Xianyou Hou, Li-Yi Wei, Heung-Yeung Shum, Baining Guo

Real-time Multi-perspective Rendering on Graphics Hardware

Teaser

Real-time near object reflection off curved surface

Environment map Ray tracing Our technique

Camera Projection Model

Single-perspective

Projection directions merge at a single point

Pin-hole, orthographic, oblique, etc

Supported on graphics hardware

Multi-perspective

Arbitrary projection directions

General linear camera [Yu and McMillan 2004 - 2005]

Not directly supported on graphics hardware

Why Multi-perspective Projection?

Model many natural phenomena

curved reflection refraction caustics

may not intersect at one point!

Goal and Previous Work

curved reflection

near geometry

fully dynamic speed

Ray tracing О О О Х

Environment map О Х О О

Image/depth sprite

О О Х О

Mirror or gem stone Х О О О

[Ofek et al 1998] concavity? О О depend on

dicing

Our method О О О

polygon rasterizatio

n

Our Methodology

Multi-perspective projection on feed-forward pipeline

Similar to beam tracing

Handles only planlar reflection

Curved reflection via polygon rasterization

(as opposed to ray tracing on GPU)

Custom vertex and fragment program

Non-linear rasterization

reflector

reflector

reflector

Beam

eye

Algorithm Overview

Build coarse triangle mesh

Determine projection directions at mesh vertices

Render each camera triangle via polygon rasterization

Multi-perspective projection

Custom vertex/fragment program

Parameterization

Barycentric interpolationC0 continuity

p

v1

d1

d

d3

d2

v3

v2

d = w1d1 + w2d2 + w3d3

w1

w2

w3

Rasterize one scene △ in one camera △Vertex program: bounding triangle estimation

Fragment program: ray-triangle intersection

d1

d2d3

v1

v2v3 p11 p13

p12

pij = projection of vj under dikilled

shaded

Limitations and Acceleration

Very brute forceRasterize every scene △ in every camera △Bounding △ over-estimationNo Zcull

Acceleration

object culling

bounding △ culling

camera tessellation level

Camera Tessellation Level

Ray tracing 26 (51 fps)

26x4 (63 fps) 26x4x4 (5 fps)

Smaller camera △ provides better bounding △ estimation

faster

Performance

# scene △ # camera △fps

no acceleration

fps

+ acceleration

800 100 4.5 20.11200 100 3.6 14.61600 100 2.1 9.2400 256 1.5 6800 256 0.5 2.1

1200 256 0.35 1.8400 512 0.8 2.5800 512 0.3 1.0

Applications

reflection refraction

caustics visualization

Refraction

Ray tracing Our result

Only C0 continuity

Caustics

1. Multi-perspective rendering into light point

2. Photon splatting

light

photon location map

reflector

caustic receiver

Visualization

Future Work

Speed improvement

Tighter bounding △ estimation

Multiple reflections/refractions (i.e. beam tracing)

Ck continuity with k > 0

Ride with polygon rasterization!

GPU is not designed for ray tracing [Purcell et al 2002]

CPU ray tracing [Wald et al 2006] ?

Thank you!