Light, Surface and Feature

in Color Images

Lilong ShiPostdoc at Caltech

Computational Vision Lab, Simon Fraser University

Topics

Color Constancy

Surface Reflectance Model

Feature Analysis

Color Formation

reflectance spectral

Illum. power distribution BGRkdSEp kk ,,)()()(

camera response

sensor sensitivity

Light

Surface

Feature

Color Constancy

Light

Surface

Feature

Automatic White Balance

AWB

Light

Surface

Feature

Canonical

Color Constancy Methods

Retinex Theory (McCann64)

MaxRGB/White-Patch (Land77): max(R)

Gray-World (Buchsbaum80): mean(R)

Shades-of-Gray (Finlayson04): [mean(Rp)]1/p

Gray-Edge Hypothesis (Weijer07): mean(edge(R))

Non-Negative Matrix Factorization (Shi07)

Light

Surface

Feature

=

Color Constancy Methods

Gamut Mapping (Forsyth90)

Color by Correlation (Finlayson01)

Neural Network (Cardei02)

Support Vector Regression (Xiong06)

Thin Plate Spline (Shi11)

Light

Surface

Feature

Color Constancy Methods

Classification-based (Bianco09)

Scene-based (Gijsenij11)

Light

Surface

Feature

Color Constancy Evaluation

MethodInpu

tTrain speed Para.

Relative Performance

Assumptions

Max-RGB img no very fast none poor white surface

Gray-World img no very fast none poor average gray

Shades-of-Gray img no moderate one moderate/good average gray

Edge-based Hyp. img no fast none moderate average gray

Color-by-Corre. hist yes fast a few moderate candidate illums

Neural-Network hist yes moderate some good none

Sup. Vector Reg. hist yes dep. train some moderate/good none

Thin-Plate-Spline thum yes dep. train a few good none

Light

Surface

Feature

Blackbody Radiator Lights

Tungsten lamps, sunrise/sunset, sky light

Planckian locus

Narrowband sensors

Light

Surface

Feature

T

c

eIcTE2

51),(

2

51( ) ( ) ( ) , { , , }i

c

Ti i i i ip S E S Ic e i R G B

Surface Reflectance Model

LIS Coordinate (Finlayson 01)

Light

Surface

Feature

Achromatic Surface

Detection in LIS

Light

Surface

Feature

GraySurface

Skin Color Model

Skin: melanin + hemoglobin

Skin Reflectance (Hiraoka et al 93)

Under blackbody illumination

Light

Surface

Feature

)]()()()()(exp[)( hhhmmm llS

pigment density

absorbance length in epidermis/dermi

s

absorbance of other material

c1ωσσΠ

bτb

SEp

hhmmhm ),,,(

Skin Color Locus

Linear model

m is melanin basis,

h is hemoglobin basis,

is blackbody radiator basis,

c is a constant vector

Light

Surface

Feature

Skin Tone Correction

Even simpler model:

Tone correction

Preserve melanin

16 different illum + camera

calibrations

( , )m m m τ Π σ ω

Light

Surface

Feature

Features

Textures, edges, corner, blobs, etc..

Colors

Integrated by Quaternion

Light

Surface

Feature

Quaternion

Real, complex, quaternion (q = a + bi + cj + dk)

Non-commutative (pq ≠ qp)

Convolution, Correlation, Fourier, Wavelet, etc

SVD, EVD, PCA

Light

Surface

Feature

Texture Feature Extraction

Light

Surface

Feature

QPCA

Image-specific quaternion texture

basisSampled sub-windows

Texture Feature Extraction

Light

Surface

Feature

Single quaternion

A texture patch

1st QPCA Basisma

gn

itu

de

re

al la

ye

rre

d la

ye

rg

re

en

la

ye

rb

lu

e la

ye

r

T

Texture Feature 1st Feature

Light

Surface

Feature

magnitude

real layer i layer j layer k layer

Segmentation

Quaternion Hoang(05)

Light

Surface

Feature

Segmentation

Light

Surface

Feature

Color Curvature

Light

Surface

Feature

Iso-luminance

Color -> Gray

Cancellation in combining +/- derivatives

Light

Surface

Feature

Hessian Descriptor

2nd order local shape

Principle Curvature

Light

Surface

Feature

2

22

2

2

2

),(

y

I

xy

Iyx

I

x

I

yxH eigenvectors: (e1, e2 )

eigenvalues: |1|<|2|

e1

e2

1

λ2

e1

e2 λ2

1

Curvature

Tubular, vessel-like structures [Frangi98]

With eigen-values blobness: backgroundness: vesselness: R and S

Gray image, 2 λ’s; RGB image, 6λ’s

Light

Surface

Feature

Color Curvature

Quaternion-valued Hessian

QSVD on H 2 real singular values

Light

Surface

Feature

k

y

B

xy

Byx

B

x

B

j

y

G

xy

Gyx

G

x

G

i

y

R

xy

Ryx

R

x

R

HQ

2

22

2

2

2

2

22

2

2

2

2

22

2

2

2

QT

Q UVHQ

Curvature Detection

Light

Surface

Feature

Frangi Quaternion

Future Works

Content-based color constancy

Color blob/points detection

Possibilities …

Light

Surface

Feature