Approaches for Retinex and Their Approaches for Retinex and Their RelationsRelations

Yu DuYu Du

March 14, 2002March 14, 2002

2

Presentation OutlinePresentation Outline

Introductions to retinexIntroductions to retinex

Approaches for retinexApproaches for retinex

The variational frameworkThe variational framework

Relation of these approachesRelation of these approaches

ConclusionsConclusions

3

What Is RetinexWhat Is Retinex

Lightness and retinex theoryLightness and retinex theoryE. H. Land 1971E. H. Land 1971

Visual system of humanVisual system of humanRetinaRetina: the sensory membrane lining the eye that receives the : the sensory membrane lining the eye that receives the

image formed by the lens (Webster)image formed by the lens (Webster)

Reflectance and illuminationReflectance and illumination

Edges and independent color senstionEdges and independent color senstion

4

Model of retinex (1)Model of retinex (1)

),(),(),( yxLyxRyxS

The given imageThe given image

The reflectance partThe reflectance part

The illumination partThe illumination part

5

Model of retinex (2)Model of retinex (2)

),(),(),( yxlyxryxs

Input ImageInput Image LogLog

Estimate the Estimate the

IlluminationIllumination

ExpExp++S s

l̂

r̂ R̂

6

Three Types of Previous ApproachesThree Types of Previous Approaches

Random walk algorithmsRandom walk algorithmsE. H. Land (1971)E. H. Land (1971)

Homomorphic filteringHomomorphic filteringE. H. Land (1986), D. J. Jobson (1997)E. H. Land (1986), D. J. Jobson (1997)

Solving Poisson equationSolving Poisson equationB. K. P. Horn (1974)B. K. P. Horn (1974)

7

Random Walk Algorithms (1)Random Walk Algorithms (1)

First retinex algorithmFirst retinex algorithm

A series of random pathsA series of random pathsStarting pixel Starting pixel

Randomly select a neighbor pixel as next pixel on pathRandomly select a neighbor pixel as next pixel on path

Accumulator and counterAccumulator and counter

1x

))(log())(log()()( 1xfxfxAxA iii 1)()( ii xNxN

8

Random Walk Algorithms (2)Random Walk Algorithms (2)

Adequate number of random pathsAdequate number of random pathsCover the whole imageCover the whole image

Small varianceSmall variance

Length of pathsLength of paths>200 for 10x10 image (D. H. Brainard)>200 for 10x10 image (D. H. Brainard)

9

Special Smoothness of Random WalkSpecial Smoothness of Random Walk

The value in the accumulatorThe value in the accumulator

The illumination partThe illumination part

pixel passed

thatpaths

))(log())(log()(

x

ixfxfxA

NNxfxfxG

xGxl

)()()(

))(log()(

1

10

Homomorphic FilteringHomomorphic Filtering

Assume illumination part to be smoothAssume illumination part to be smooth

Apply low pass filterApply low pass filter

LD

vuDc

LH evuH

)1)((),(20

2 ),(

11

Poisson Equation Solution (1)Poisson Equation Solution (1)

Derivative of illumination part close to zeroDerivative of illumination part close to zero

Reflectance part to be piece-wise constantReflectance part to be piece-wise constant

Get the illumination partGet the illumination partTake the derivative of the imageTake the derivative of the image

Clip out the high derivative peaksClip out the high derivative peaks

12

Poisson Equation Solution (2)Poisson Equation Solution (2)

Solve Poisson equationSolve Poisson equation

Iterative methodIterative method

Apply low-pass filter (invert Laplacian operator)Apply low-pass filter (invert Laplacian operator)

other wise0

)(Tss

s

)(ˆ sl

13

Comments on Above ApproachesComments on Above Approaches

Random walk algorithmRandom walk algorithm

Too slowToo slow

Homomorphic filteringHomomorphic filtering

Low-pass filtering first or Low-pass filtering first or loglog first? first?

More work needed to be done on Poisson equation More work needed to be done on Poisson equation

solvingsolving

14

Variational FrameworkVariational Framework

Presented by R. Kimmel etc.Presented by R. Kimmel etc.

From assumptions to penalty functionFrom assumptions to penalty function

From penalty function to algorithmFrom penalty function to algorithm

15

Assumptions On Illumination ImageAssumptions On Illumination Image

Spatial smoothness of illuminationSpatial smoothness of illumination

Reflectance is not pure whiteReflectance is not pure white

Illumination close to intensity imageIllumination close to intensity image

Spatial smoothness of reflectanceSpatial smoothness of reflectance

Continues smoothly beyond boundariesContinues smoothly beyond boundaries

16

Penalty Function and RestrictionsPenalty Function and Restrictions

Goal to minimize:Goal to minimize:

Subject to:Subject to:

And onAnd on

dxdyslslllF ))()()(222

sl

0, nl

17

Solve the Penalty Function (1)Solve the Penalty Function (1)

Euler-Lagrange equationsEuler-Lagrange equations

And And

)()(0)(

slslll

lF

sl

18

Solve the Penalty Function (2)Solve the Penalty Function (2)

Projected normalized steepest descent (PNSD)Projected normalized steepest descent (PNSD)

Iteratively to get Iteratively to get illumination partillumination part

},min{ 1 sGll NSDjj

))(( 11 sllG jj

))1((22

2

GG

GNSD

19

Multi-resolutionMulti-resolution

Make PNSD algorithm converges fasterMake PNSD algorithm converges faster

Illumination part is smoothIllumination part is smooth

Coarse resolution image firstCoarse resolution image first

Upscale coarse illumination as initial of finer resolution Upscale coarse illumination as initial of finer resolution

layerlayer

Not multi-scale techniqueNot multi-scale technique

20

Relationship of Different Approaches (1)Relationship of Different Approaches (1)

Random walk and Homomorphic filteringRandom walk and Homomorphic filtering

R. Kimmel’s words on Homomorphic filteringR. Kimmel’s words on Homomorphic filtering

and remove constraint and remove constraint

0sl

21

Relationship of Different Approaches (2)Relationship of Different Approaches (2)

Apply appropriate scaling on images, Apply appropriate scaling on images,

Homomorphic filtering satisfies constrainHomomorphic filtering satisfies constrain

and and

Poisson equation approach:Poisson equation approach:

sl 0

)(),( syx

22

ConclusionsConclusions

Retinex is trying to simulate human vision processRetinex is trying to simulate human vision process

Different approaches are from same assumptionsDifferent approaches are from same assumptions

Implementation details are important for resultsImplementation details are important for results

Thank YouThank You

March 14, 2002March 14, 2002