Yizhou Yu

Recovering BRDF Models for Recovering BRDF Models for Architectural ScenesArchitectural Scenes

Recovering BRDF Models for Recovering BRDF Models for Architectural ScenesArchitectural Scenes

Yizhou Yu

Computer Science Division

University of California at Berkeley

Yizhou Yu

Computer Science Division

University of California at Berkeley

SIGGRAPH 2000 Course onImage-Based Surface Details

Yizhou Yu

Image-based Rendering versus Image-based Rendering versus Traditional Graphics ( circa 1997 )Traditional Graphics ( circa 1997 )Image-based Rendering versus Image-based Rendering versus Traditional Graphics ( circa 1997 )Traditional Graphics ( circa 1997 )

+ Improved photorealism

- Static scene configuration

- Fixed lighting condition

Yizhou Yu

Image-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and Rendering

• Vary lighting– Recover reflectance properties for multiple objects in a

mutual illumination environment

• Vary lighting– Recover reflectance properties for multiple objects in a

mutual illumination environment

5:00am 6:00am 7:00am 10:00am

Yizhou Yu

The ProblemThe ProblemThe ProblemThe Problem

• Forward Problem: Global Illumination– Couple lighting and reflectance to generate images

• Backward Problem: Inverse Global Illumination– Factorize images into lighting and reflectance

• Forward Problem: Global Illumination– Couple lighting and reflectance to generate images

• Backward Problem: Inverse Global Illumination– Factorize images into lighting and reflectance

Illumination Reflected Light

Reflectance

Yizhou Yu

Global IlluminationGlobal IlluminationGlobal IlluminationGlobal Illumination

Reflectance Properties Images

Geometry Light Sources

LightTransport

Yizhou Yu

Inverse Global IlluminationInverse Global IlluminationInverse Global IlluminationInverse Global Illumination

Reflectance Properties Images

Geometry Light Sources

Yizhou Yu

Input ImagesInput ImagesInput ImagesInput Images

Every surface should be covered by at least one photographA specular highlight should be captured for every specular surface

Yizhou Yu

Camera Radiance Response CurveCamera Radiance Response CurveCamera Radiance Response CurveCamera Radiance Response Curve

• Pixel brightness value is a nonlinear function of radiance.– Debevec & Malik[Siggraph’97]

gives a method to recover this nonlinear mapping.

• Pixel brightness value is a nonlinear function of radiance.– Debevec & Malik[Siggraph’97]

gives a method to recover this nonlinear mapping.

Radiance

IntensitySaturation

Yizhou Yu

In Detail ... In Detail ... In Detail ... In Detail ...

Yizhou Yu

Recovered Geometry and Camera PoseRecovered Geometry and Camera PoseRecovered Geometry and Camera PoseRecovered Geometry and Camera Pose

Yizhou Yu

Light SourcesLight SourcesLight SourcesLight Sources

Spherical light sources are easier to modelLight source intensity can be calibrated from dynamic range images

Yizhou Yu

Synthesized ImagesSynthesized ImagesSynthesized ImagesSynthesized Images

Original Lighting Novel Lighting

Yizhou Yu

A ComparisonA ComparisonA ComparisonA Comparison

Hand-crafted Recovered

Yizhou Yu

OutlineOutlineOutlineOutline

• Diffuse surfaces under mutual illumination

• Non-diffuse surfaces under direct illumination

• Non-diffuse surfaces under mutual illumination

• Diffuse surfaces under mutual illumination

• Non-diffuse surfaces under direct illumination

• Non-diffuse surfaces under mutual illumination

Yizhou Yu

Lambertian Surfaces under Lambertian Surfaces under Mutual IlluminationMutual IlluminationLambertian Surfaces under Lambertian Surfaces under Mutual IlluminationMutual Illumination

j

ijjiii FBEB j

ijjiii FBEB

• Bi, Bj, Ei measured

• Form-factor Fij known

• Solve for diffuse albedo

• Bi, Bj, Ei measured

• Form-factor Fij known

• Solve for diffuse albedo i

iB

jBijF

Source

Target

Yizhou Yu

Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]

Isotropic Kernel

Anisotropic Kernel

NHi

r

),(

Ksd

2

22

4

]/tan[exp

coscos

1),(

ri

K

yx

yx

ri

K

4

)]/sin/cos(tan[exp

coscos

1),(

22222

( 3 parameters)

( 5 parameters)

Yizhou Yu

Non-diffuse Surfaces underNon-diffuse Surfaces underDirect IlluminationDirect IlluminationNon-diffuse Surfaces underNon-diffuse Surfaces underDirect IlluminationDirect Illumination

2

,,)),(( min arg iisi

i

di IKIL

sd

2

,,)),(( min arg iisi

i

di IKIL

sd

NH

iisid

i IKIL )),((

iisid

i IKIL )),((

P1P2

P1

P2

Yizhou Yu

Non-diffuse Surfaces under Non-diffuse Surfaces under Mutual IlluminationMutual IlluminationNon-diffuse Surfaces under Non-diffuse Surfaces under Mutual IlluminationMutual Illumination

• Problem: LPiAj is not known. ( unlike diffuse case, where LPiAj = LCkAj )

• Solution: iterative estimation

• Problem: LPiAj is not known. ( unlike diffuse case, where LPiAj = LCkAj )

• Solution: iterative estimation

Cv

Ck

Aj

Pi

LPiAj

LCkAj

LCvPi

Source

Target

Yizhou Yu

Estimation of Specular Difference SEstimation of Specular Difference SEstimation of Specular Difference SEstimation of Specular Difference S

• Estimate specular component of by Monte Carlo ray-tracing using current guess of reflectance parameters.

• Similarly for

• Difference gives S

• Estimate specular component of by Monte Carlo ray-tracing using current guess of reflectance parameters.

• Similarly for

• Difference gives S Cv

Ck

Aj

Pi

LPiAj

LCkAj

LCvPi

LPiAj

LCkAj

Yizhou Yu

Recovering Diffuse Albedo MapsRecovering Diffuse Albedo MapsRecovering Diffuse Albedo MapsRecovering Diffuse Albedo Maps

• Specular properties assumed uniform across each surface, but diffuse albedo allowed to vary.

• Subtract specular component

• Recover pointwise diffuse albedo

• Specular properties assumed uniform across each surface, but diffuse albedo allowed to vary.

• Subtract specular component

• Recover pointwise diffuse albedo

Yizhou Yu

ResultsResultsResultsResults

• A simulated cubical room• A simulated cubical room

Yizhou Yu

Results for the Simulated CaseResults for the Simulated CaseResults for the Simulated CaseResults for the Simulated Case

Diffuse Albedo

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

1 2 3 4 5 60

0.05

0.1

0.15

0.2

0.25

0.3

0.35

1 2 3 4 5 6

Specular Roughness

Yizhou Yu

ResultsResultsResultsResults

• A real conference room• A real conference room

Yizhou Yu

Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting

Real

Synthetic

Yizhou Yu

Diffuse Albedo Maps of Identical Diffuse Albedo Maps of Identical Posters in Different PositionsPosters in Different PositionsDiffuse Albedo Maps of Identical Diffuse Albedo Maps of Identical Posters in Different PositionsPosters in Different Positions

Poster A Poster B Poster C

Yizhou Yu

Inverting Color BleedInverting Color BleedInverting Color BleedInverting Color Bleed

Input Photograph Output Albedo Map

Yizhou Yu

Real vs. Synthetic for Novel LightingReal vs. Synthetic for Novel LightingReal vs. Synthetic for Novel LightingReal vs. Synthetic for Novel Lighting

Real

Synthetic

Yizhou Yu

Modeling Outdoor IlluminationModeling Outdoor IlluminationModeling Outdoor IlluminationModeling Outdoor Illumination

• The sun– Diameter 31.8’ seen from the earth.

• The sky– A hemispherical area light source.

• The surrounding environment– May contribute more light than the

sky on shaded side.

• The sun– Diameter 31.8’ seen from the earth.

• The sky– A hemispherical area light source.

• The surrounding environment– May contribute more light than the

sky on shaded side.

Yizhou Yu

A Recovered Sky Radiance ModelA Recovered Sky Radiance ModelA Recovered Sky Radiance ModelA Recovered Sky Radiance Model

) cos ) exp( 1 ))( /cos exp( 1 Lvz( 2 edcba f ) cos ) exp( 1 ))( /cos exp( 1 Lvz( 2 edcba f

Yizhou Yu

Coarse-grain Environment Radiance MapsCoarse-grain Environment Radiance MapsCoarse-grain Environment Radiance MapsCoarse-grain Environment Radiance Maps

• Partition the lower hemisphere into small regions

• Project pixels into regions and obtain the average radiance

• Partition the lower hemisphere into small regions

• Project pixels into regions and obtain the average radiance

Yizhou Yu

Comparison with Real PhotographsComparison with Real PhotographsComparison with Real PhotographsComparison with Real Photographs

Synthetic Real

Yizhou Yu

Inverse Global IlluminationInverse Global IlluminationInverse Global IlluminationInverse Global Illumination

• Detect specular highlights on the surfaces.• Choose sample points inside and around highlights.• Build links between sample points and facets in the

environment• Assign to each facet one photograph and one average

radiance value • Assign zero to Delta_S at each link.• For iter = 1 to n

– For each link, use its Delta_S to update its radiance value.– For each surface having highlights, optimize its BRDF parameters.– For each link, estimate its Delta_S with the new BRDF parameters.

• End

• Detect specular highlights on the surfaces.• Choose sample points inside and around highlights.• Build links between sample points and facets in the

environment• Assign to each facet one photograph and one average

radiance value • Assign zero to Delta_S at each link.• For iter = 1 to n

– For each link, use its Delta_S to update its radiance value.– For each surface having highlights, optimize its BRDF parameters.– For each link, estimate its Delta_S with the new BRDF parameters.

• End