Computer Graphics CMU 15-462/15-662, Fall 2016

The Rendering Equation

CMU 15-462/662, Fall 2015

Review: What is radiance?▪ Radiance at point p in direction N is radiant energy (“#hits”)

per unit time, per solid angle, per unit area perpendicular to N.

— radiant energy— solid angle— projected area

CMU 15-462/662, Fall 2015

Intuition: Radiance and Irradiance

irradiance radiance in direction ω

angle between ω and normal

CMU 15-462/662, Fall 2015

Incident vs. Exitant RadianceINCIDENT EXITANT

In both cases: intensity of illumination is highly dependent on direction (not just location in space or moment in time).

CMU 15-462/662, Fall 2015

The Rendering Equation▪ Core functionality of photorealistic renderer is to estimate

radiance at a given point p, in a given direction ωo

▪ Summed up by the rendering equation (Kajiya):

outgoing/observed radiance emitted radiance (e.g., light source)

incoming radiance

all directions in hemisphere

angle between incoming direction and normal

incoming direction

direction of interestpoint of interest

scattering function

Key challenge: to evaluate incoming radiance, we have to compute yet another integral. I.e., rendering equation is recursive.

CMU 15-462/662, Fall 2015

Recursive Raytracing▪ Basic strategy: recursively evaluate rendering equation!

CMU 15-462/662, Fall 2015

Renderer measures radiance along a ray

Pinholeo,d

o,d x

y

At each “bounce,” want to measure radiance traveling in the direction opposite the ray direction.

CMU 15-462/662, Fall 2015

Renderer measures radiance along a ray

Pinholeo,d

o,d x

y

Radiance entering camera in direction d = light from scene light sources that is reflected off surface in direction d.

CMU 15-462/662, Fall 2015

How does reflection of light affect the outgoing radiance?

CMU 15-462/662, Fall 2015

Reflection models▪ Reflection is the process by which light incident on a surface

interacts with the surface such that it leaves on the incident (same) side without change in frequency

▪ Choice of reflection function determines surface appearance

CMU 15-462/662, Fall 2015

Categories of reflection functions▪ Ideal specular

▪ Ideal diffuse

▪ Glossy specular

▪ Retro-reflective

Diagrams illustrate how incoming light energy from given direction is reflected in various directions.

Perfect mirror

Uniform reflection in all directions

Reflects light back toward source

Majority of light distributed in reflection direction

CMU 15-462/662, Fall 2015

Materials: diffuse

CMU 15-462/662, Fall 2015

Materials: plastic

CMU 15-462/662, Fall 2015

Materials: red semi-gloss paint

CMU 15-462/662, Fall 2015

Materials: Ford mystic lacquer paint

CMU 15-462/662, Fall 2015

Materials: mirror

CMU 15-462/662, Fall 2015

Materials: gold

CMU 15-462/662, Fall 2015

Materials

CMU 15-462/662, Fall 2015

Models of Scattering▪ How can we model “scattering” of light?

▪ Many different things that could happen to a photon: - bounces off surface - transmitted through surface - bounces around inside surface - absorbed & re-emitted - …

▪ What goes in must come out! (Total energy must be conserved)

▪ In general, can talk about “probability*” a particle arriving from a given direction is scattered in another direction

*Somewhat more complicated than this, because some light is absorbed!

CMU 15-462/662, Fall 2015

Hemispherical incident radiance

Consider view of hemisphere from this point

At any point on any surface in the scene, there’s an incident radiance field that gives the directional distribution of illumination at the point

CMU 15-462/662, Fall 2015

Diffuse reflection

Incident radiance Exitant radiance

Exitant radiance is the same in all directions

CMU 15-462/662, Fall 2015

Ideal specular reflection

Incident radiance Exitant radiance

Incident radiance is “flipped around normal” to get exitant radiance

CMU 15-462/662, Fall 2015

Plastic

Incident radiance Exitant radiance

Incident radiance gets “flipped and blurred”

CMU 15-462/662, Fall 2015

Copper

Incident radiance Exitant radiance

More blurring, plus coloration (nonuniform absorption across frequencies)

CMU 15-462/662, Fall 2015

Scattering off a surface: the BRDF▪ “Bidirectional reflectance distribution function”

▪ Encodes behavior of light that “bounces off” surface

▪ Given incoming direction ωi, how much light gets scattered in any given outgoing direction ωo?

▪ Describe as distribution fr(ωi→ωo)

bv (Szymon Rusinkiewicz)

why less than or equal?

“Helmholtz reciprocity”

Q: Why should Helmholtz reciprocity hold? Think about little mirrors…

where did the rest of the energy go?!

CMU 15-462/662, Fall 2015

Radiometric description of BRDF

“For a given change in the incident irradiance, how much does the exitant radiance change?”

CMU 15-462/662, Fall 2015

Example: Lambertian reflectionAssume light is equally likely to be reflected in each output direction

fr = c

Lo(!o) =

Z

H2

fr Li(!i) cos ✓i d!i

=fr

Z

H2

Li(!i) cos ✓i d!i

=frE

fr =⇢

⇡

CMU 15-462/662, Fall 2015

Example: perfect specular reflection

[Zátonyi Sándor]

CMU 15-462/662, Fall 2015

Geometry of specular reflection

~n

!i !o

✓i ✓o

�i �o

!o

+ !i

= 2 cos ✓ ~n = 2(!i

· ~n)~n

!o

= �!i

+ 2(!i

· ~n)~n

✓ = ✓o

= ✓i

Top-down view (looking down on surface)

CMU 15-462/662, Fall 2015

Specular reflection BRDF

✓i ✓o

Li(✓i,�i) Lo

(✓o

,�o

)

Lo

(✓o

,�o

) = Li

(✓o

,�o

± ⇡)

fr

(✓i

,�i

; ✓o

,�o

) =

�(cos ✓i

� cos ✓o

)

cos ✓i

�(�i

� �o

± ⇡)

▪ Strictly speaking, fr is a distribution, not a function

▪ In practice, no hope of finding reflected direction via random sampling; simply pick the reflected direction!

Dirac delta

CMU 15-462/662, Fall 2015

TransmissionIn addition to reflecting off surface, light may be transmitted through surface.

Light refracts when it enters a new medium.

CMU 15-462/662, Fall 2015

Snell’s Law

~n!i

!t

⌘i sin ✓i = ⌘t sin ✓t

Transmitted angle depends on index of refraction of medium incident ray is in and index of refraction of medium light is entering.

Vacuum Air (sea level) Water (20°C) Glass Diamond

1.0 1.00029 1.333 1.5-1.6 2.42

⌘Medium *

* index of refraction is wavelength dependent (these are averages)

CMU 15-462/662, Fall 2015

Law of refraction

~n!i

!t

Total internal reflection: When light is moving from a more optically dense medium to a less optically dense medium:

Light incident on boundary from large enough angle will not exit medium.

⌘i sin ✓i = ⌘t sin ✓t

1�✓⌘i⌘t

◆2

(1� cos

2 ✓i) < 0

⌘i⌘t

> 1

cos ✓t =q1� sin

2 ✓t

=

s

1�✓⌘i⌘t

◆2

sin

2 ✓i

=

s

1�✓⌘i⌘t

◆2

(1� cos

2 ✓i)

CMU 15-462/662, Fall 2015

Optical manholeOnly small “cone” visible, due to total internal reflection (TIR)

CMU 15-462/662, Fall 2015

Fresnel reflectionMany real materials: reflectance increases w/ viewing angle

[Lafortune et al. 1997]

CMU 15-462/662, Fall 2015

Snell + Fresnel: Example

Refraction (Snell)

Reflection (Fresnel)

CMU 15-462/662, Fall 2015

Without Fresnel (fixed reflectance/transmission)

CMU 15-462/662, Fall 2015

Glass with Fresnel reflection/transmission

CMU 15-462/662, Fall 2015

Anisotropic reflectionReflection depends on azimuthal angle �

Results from oriented microstructure of surface e.g., brushed metal

CMU 15-462/662, Fall 2015

Subsurface scattering▪ Visual characteristics of many

surfaces caused by light entering at different points than it exits - Violates a fundamental

assumption of the BRDF[Jensen et al 2001]

[Donner et al 2008]

CMU 15-462/662, Fall 2015

Scattering functions▪ Generalization of BRDF; describes exitant radiance at one

point due to incident differential irradiance at another point:S(x

i

,!

i

, x

o

,!

o

)

▪ Generalization of reflection equation integrates over all points on the surface and all directions(!)

L(xo,!o) =

Z

A

Z

H2

S(xi,!i, xo,!o)Li(xi,!i) cos ✓i d!i dA

CMU 15-462/662, Fall 2015

Translucent materials: Jade

CMU 15-462/662, Fall 2015

Translucent materials: skin

CMU 15-462/662, Fall 2015

Translucent materials: leaves

CMU 15-462/662, Fall 2015

BRDF

CMU 15-462/662, Fall 2015

BSSRDF

CMU 15-462/662, Fall 2015

Ok, so scattering is complicated!

What’s a (relatively simple) algorithm that can capture all this behavior?

CMU 15-462/662, Fall 2015

The reflection equation

Lr(p,!r) =

Z

H2

fr(p,!i ! !r)Li(p,!i) cos ✓i d!i

dLr(!r) = fr(!i ! !r) dLi(!i) cos ✓i

CMU 15-462/662, Fall 2015

The reflection equation▪ Key piece of overall rendering equation:

▪ Approximate integral via Monte Carlo integration

▪ Generate directions sampled from some distribution

▪ Compute the estimator

▪ To reduce variance should match BRDF or incident radiance function

Lr(p,!r) =

Z

H2

fr(p,!i ! !r)Li(p,!i) cos ✓i d!i

!j p(!)

1

N

NX

j=1

fr(p,!j ! !r)Li(p,!j) cos ✓jp(!j)

p(!)

CMU 15-462/662, Fall 2015

Estimating reflected light

// Assume:

// Ray ray hits surface at point hit_p

// Normal of surface at hit point is hit_n

Vector3D wr = -ray.d; // outgoing direction

Spectrum Lr = 0.;

for (int i = 0; i < N; ++i) {

Vector3D wi; // sample incident light from this direction

float pdf; // p(wi)

generate_sample(brdf, &wi, &pdf); // generate sample according to brdf

Spectrum f = brdf->f(wr, wi);

Spectrum Li = trace_ray(Ray(hit_p, wi)); // compute incoming Li

Lr += f * Li * fabs(dot(wi, hit_n)) / pdf;

}

return Lr / N;

CMU 15-462/662, Fall 2015

The rendering equation

Now that we know how to handle reflection, how do we solve the full rendering equation? Have to determine incident radiance…

Pinhole x

y

p!o

!i

Lo

(p,!o

)

Lo(p,!o) = Le(p,!o) +

Z

H2

fr(p,!i ! !o)Li(p,!i) cos ✓i d!i

CMU 15-462/662, Fall 2015

Key idea in (efficient) rendering: take advantage of special knowledge to break up integration into “easier” components.

CMU 15-462/662, Fall 2015

Path tracing: overview▪ Partition the rendering equation into direct and indirect

illumination

▪ Use Monte Carlo to estimate each partition separately - One sample for each - Assumption: 100s of samples per pixel

▪ Terminate paths with Russian roulette

CMU 15-462/662, Fall 2015

Direct illumination + reflection + transparencyImage credit: Henrik Wann Jensen

CMU 15-462/662, Fall 2015

Global illumination solutionImage credit: Henrik Wann Jensen

CMU 15-462/662, Fall 2015

▪ Definite integral

▪ Random variables

▪ Expectation of f

▪ Estimator

is the value of a random sample drawn from the distribution is also a random variable.

Review: Monte Carlo integration

Xi ⇠ p(x)

Yi = f(Xi)Xi ⇠ p(x)

Xi ⇠ p(x)Yi = f(Xi)

Z b

af(x)dx

E[Yi] = E[f(Xi)] =

Z b

af(x) p(x) dx

FN =b� a

N

NX

i=1

YiMonte Carlo estimate of

Assuming samples drawn from uniform pdf. I will provide estimator for arbitrary PDFs later in lecture.

f(x)�Xi

Z b

af(x)dx

What we seek to estimate

CMU 15-462/662, Fall 2015

Basic Monte Carlo estimator

Assume uniform probability density (for now)

Xi ⇠ U(a, b)

p(x) =1

b� a

CMU 15-462/662, Fall 2015

Näive direct lighting estimate

✓

d!

dA

E(p) =

ZL(p,!) cos ✓ d!

=2⇡

ZL(p,!) cos ✓

1

2⇡d!

=2⇡

ZL(p,!) cos ✓ p(!) d!

Estimator:L(p,!)

p(!) =1

2⇡

Uniformly-sample hemisphere of directions with respect to solid angle

Xi ⇠ p(!)

Yi = f(Xi)

Yi = L(p,!i)cos ✓i

FN =2⇡

N

NX

i=1

Yi

CMU 15-462/662, Fall 2015

Näive direct lighting estimate

E(p) =

ZL(p,!) cos ✓ d!

=2⇡

ZL(p,!) cos ✓

1

2⇡d!

=2⇡

ZL(p,!) cos ✓ p(!) d!

Given surface point p

For each of N samples:

Generate random direction:

Compute incoming radiance arriving at p from direction:

Compute incident irradiance due to ray:

Accumulate into estimator

Uniformly-sample hemisphere of directions with respect to solid angle

2⇡

NdEi

dEi = Licos ✓i

Li

!i

!i

CMU 15-462/662, Fall 2015

Uniform hemisphere sampling

0.2 0.4 0.6 0.8 1.0

0.2

0.4

0.6

0.8

1.0

Generate random direction on hemisphere (all directions equally likely)

p(!) =1

2⇡

Direction computed from uniformly distributed point on 2D plane:

Try at home: derive from the inversion method [Arvo]

(⇠1, ⇠2) = (

q1� ⇠21 cos(2⇡⇠2),

q1� ⇠21 sin(2⇡⇠2), ⇠1)

CMU 15-462/662, Fall 2015

Hemispherical solid angle sampling, 100 sample rays

(random directions drawn uniformly from hemisphere)

Light source

Occluder (blocks light)

CMU 15-462/662, Fall 2015

Why is the image in the previous slide “noisy”?

CMU 15-462/662, Fall 2015

Incident lighting estimator uses different random directions in each pixel. Some of those directions point towards the light, others do not.

(Estimator is a random variable)

CMU 15-462/662, Fall 2015

Idea: don’t need to integrate over entire hemisphere of directions (incoming radiance is 0 from most directions)

Only integrate over the area of the light (directions where incoming radiance is non-zero)

CMU 15-462/662, Fall 2015

Direct lighting: area integralE(p) =

ZL(p,!) cos ✓ d!

=2⇡

ZL(p,!) cos ✓

1

2⇡d!

=2⇡

ZL(p,!) cos ✓ p(!) d!

Integral over directions

Change of variables to integral over area of light

E(p) =

Z

A

0Lo

(p

0,!0)V (p, p0)

cos ✓ cos ✓0

|p� p

0|2 dA0

Outgoing radiance from light point p, in direction w’ towards p

Binary visibility function: 1 if p’ is visible from p, 0 otherwise (accounts for light occlusion)

dw =dA

|p0 � p|2 =dA

0cos ✓

|p0 � p|2A0

p0

p

✓

✓0

!0 = p� p0

! = p0 � p

CMU 15-462/662, Fall 2015

Light source area sampling, 100 sample rays

If no occlusion is present, all directions chosen in computing estimate “hit” the light source.(Choice of direction only matters if portion of light is occluded from surface point p.)

CMU 15-462/662, Fall 2015

Hemispherical solid angle sampling, 100 sample rays

(random directions drawn uniformly from hemisphere)

CMU 15-462/662, Fall 2015

Light source area sampling, 100 sample rays

CMU 15-462/662, Fall 2015

Estimating indirect lighting

CMU 15-462/662, Fall 2015

The reflection equation

Need to know incident radiance. So far, have only computed incoming radiance from scene light sources.

Pinhole x

y

p!o

!i

Lo

(p,!o

)

Lo(p,!o) = Le(p,!o) +

Z

H2

fr(p,!i ! !o)Li(p,!i) cos ✓i d!i

CMU 15-462/662, Fall 2015

Accounting for indirect illumination

Pinhole x

y

p!o

!i

Lo

(p,!o

)

Incoming light energy from direction may be due to light reflected off another surface in the scene (not an emitter)

!i

Lo(p,!o) = Le(p,!o) +

Z

H2

fr(p,!i ! !o)Li(p,!i) cos ✓i d!i

CMU 15-462/662, Fall 2015

Path tracing: indirect illumination

▪ Sample incoming direction from some distribution(e.g. proportional to BRDF):

▪ Recursively call path tracing function to compute incident indirect radiance

▪ Monte Carlo estimator:

!i ⇠ p(!)

Z

H2

fr(!i ! !o)Lo,i(tr(p,!i),�!i) cos ✓i d!i

fr

(!i

! !o

)Lo,i

(tr(p,!i

),�!i

) cos ✓i

p(!i

)

CMU 15-462/662, Fall 2015

Direct illumination

p

CMU 15-462/662, Fall 2015

One-bounce global illumination

p

CMU 15-462/662, Fall 2015

Two-bounce global illumination

p

CMU 15-462/662, Fall 2015

Four-bounce global illumination

p

CMU 15-462/662, Fall 2015

Eight-bounce global illumination

p

CMU 15-462/662, Fall 2015

Sixteen-bounce global illumination

p

CMU 15-462/662, Fall 2015

Wait a minute… When do we stop?!

Stanford CS348b, Spring 2014

Russian roulette▪ Idea: want to avoid spending time evaluating function for

samples that make a small contribution to the final result

▪ Consider a low-contribution sample of the form:

V (p, p0)

L =

fr

(!i

! !o

)Li

(!i

)V (p, p0) cos ✓i

p(!i

)

Stanford CS348b, Spring 2014

Russian roulette

▪ If tentative contribution (in brackets) is small, total contribution to the image will be small regardless of

▪ Ignoring low-contribution samples introduces systemic error - No longer an unbiased estimator

▪ Instead, randomly discard low-contribution samples in a way that leaves estimator unbiased

V (p, p0)

L =

fr

(!i

! !o

)Li

(!i

) cos ✓i

p(!i

)

�V (p, p0)

L =

fr

(!i

! !o

)Li

(!i

)V (p, p0) cos ✓i

p(!i

)

Stanford CS348b, Spring 2014

Russian roulette▪ New estimator: evaluate original estimator with probability

, reweight. Otherwise ignore.

▪ Same expected value as original estimator:

prr

prrE

X

prr

�+ E[(1� prr)0] = E[X]

Stanford CS348b, Spring 2014

No Russian roulette: 6.4 seconds

Stanford CS348b, Spring 2014

Russian roulette: terminate 50% of all contributions withluminance less than 0.25: 5.1 seconds

Stanford CS348b, Spring 2014

Russian roulette: terminate 50% of all contributions withluminance less than 0.5: 4.9 seconds

Stanford CS348b, Spring 2014

Russian roulette: terminate 90% of all contributions withluminance less than 0.125: 4.8 seconds

Stanford CS348b, Spring 2014

Russian roulette: terminate 90% of all contributions withluminance less than 1: 3.6 seconds

CMU 15-462/662, Fall 2015

Next time:▪ Variance reduction—how do we get the most out of our samples?