Color Img at Prisma Network meeting 2009

Departamento de Departamento de ÓÓptica, Facultad de Ciencias, ptica, Facultad de Ciencias, Universidad de Granada, 18071Universidad de Granada, 18071--Granada (SPAIN)Granada (SPAIN)

Photometric -stereo estimation from spectral systemsJuan Luis Nieves et al.

Colour Imaging Lab (www.ugr.es/local/colorimg)

3rd Thematic Network MeetingFebruary 5, 2009, Toledo (Spain)

Colour Imaging Lab

Universidad de Granadawww.ugr.es/~colorimg

Javier Romero

Juan Luis Nieves

Eva M. Valero

Clara PlataJavier Hernández

Andrés

Raul Luzón

At present our research revolves around the development of techniques for acquiring and processing multispectral images and applying them to…

� Photometric-stereo techniques,� Design of optimum sensors in the identification and measurement of the

spectral composition of daylight,� Color constancy in the development of color-descriptors invariant to

changes in illumination applicable to color imaging,� … etc.

The problem

Reflected light from objects depends on reflectance(albedo) and illumination…

Scene400 500 600 7000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Wavelength (nm)

Spectral image acquisition

� Better resolution� Easy and cheaper

The problem

� Different approaches

Liquid Crystal Tunable Filtre (LCTF)

A CCD camera with a narrow -band filtre set

coupled

The problemSpectral image acquisition

Filtre wheel

n colour filtres

� Different approaches

A CCD camera throughbroad-band colour filtres

some “a priori” information+

The problemSpectral image acquisition

Spectral approach vs. colorimetric approach• Not limited to the human visual range: advantages to include

the infrared (IR) and UV (ultraviolet);• Metamerism is avoided;• Illuminant changes can be simulated; • Applications from remote sensing, astronomy, medicine, art

restoration, cosmetics, printing, computer graphics, biology, agriculture, etc.

The problem

The problem

… and on surface relief.

Reflected light from objects depends on reflectance (albedo) and illumination…

��PhotometricPhotometric--stereo as a way of obtaining stereo as a way of obtaining 3D and albedo from surfaces.3D and albedo from surfaces.

��An RGB digital camera to simultaneously An RGB digital camera to simultaneously recover spectral reflectance and surface recover spectral reflectance and surface relief from images.relief from images.

Outline

Directmethods

�A technique in computer vision for estimating the surface normals of objects by observing that object under different lighting conditions.

Photometric Stereo

�A technique in computer vision for estimating the surface normals of objects by observing that object under different lighting conditions.

Photometric Stereo

Indirect methods

ρNL1 L2

L3

Photometric Stereo

z y

x

e

g

i

N

R LN

( )yxSz ,=

( )x

Syxp

∂∂=, ( )

y

Syxq

∂∂=,

( )Tqp

qp

1,,1

1

22

−++

=N

Indirect methods

ρN

L1

L2

L3

Photometric Stereo

( )TIII321

,,=I

[ ] ( )TL321

,, LLL=

[ ]NI Lρ=

( )NL ·kk

I ρ=

[ ] NI-1 ρ=L

4-source photometric stereo

� It’s faster than laser scanning;

� It can be made surface rotation invariant.

� The colour maps of 3D surface textures may be separated and analysedindependently.

Why albedo is important in computer graphics?

4-source photometric stereoFigura con el ilumiante descontado

utilizando la media de las normales recuperadas

Figura con el iluminante descontadoutilizando la normal recuperada para cada canal

� It’s faster than laser scanning;

� It can be made surface rotation invariant.

� The colour maps of 3D surface textures may be separated and analysedindependently.

Why albedo is important in computer graphics?

Spectral analysis and synthesis

sensors’ responses EC=ρ illuminant spectrum

sensors’ spectral sensitivities

kx1 Nx1kxN

Pseudoinverse: ( ) TT ρρρρ 1−+ =Training set:

+= ρSD

S ρ

Retiga 1300

+

Test set:1

ρ11

ρDS =⇒

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5

400 700

0

0.1

0.2

0.3

0.4

0.5


Retiga 1300

+

400 450 500 550 600 650 7000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Wavelength (nm)

Re

lativ

e S

PD

Illuminant: TRILITEGFC= 0.9268Colour Difference= 6.0RMSE= 0.049IIE= 0.39

400 450 500 550 600 650 7000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Wavelength (nm)

Re

lativ

e S

PD

Illuminant: TRILITEGFC= 0.8361Colour Difference= 9.8RMSE= 0.073IIE= 0.501

Results for a 6-band camera

Outline

Spectral imageacquisition

Photometric - stereo+

400 450 500 550 600 650 700

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Wavelenght (nm) N

Original, 0º

Original, 90º

Original, 180º

Original, 270º Albedo


( )NL·ρ=IρNL1 L2

L3

400 450 500 550 600 650 7000

0.2

0.4

0.6

0.8

1

Wavelength (nm)

Nor

mal

ized

SP

D

Digilite 600IncandescentTri-Lite

( ) ( ) ( )∑= λλλρkk

QSE

Mul

tidim

ensi

onal

pro

blem

Mul

tidim

ensi

onal

pro

blem

� Root-Mean-Square-Error (RMSE)

Spectral and colorimetric evaluation metrics

�Goodness-of-Fit-Coefficient (GFC): colorimetric accurate fit >0.995good spectral fit >0.999almost exact fit >0.9999

[ ] [ ]21

2

21

2

)()(

)()(

GFC

∑∑

∑=

j

jR

j

j

j

jRj

EE

EE

λλ

λλ

� CIELab colour difference: colorimetricallyacceptable ~3 CIELab( ) ( ) ( )[ ] 2/12*2*2**

baLEab

∆+∆+∆=∆

� RGB-error: ( )222

3

1xx

BGRRGBerrorxx

∆+∆+∆=

( )1cos

j o eAE

−= ⋅ρ ρ� Angular error:

Results : color reproduction

SAME illuminant for captured and recovered images;DIFFERENT geometry;

1.520.750.75AE

6.02.63.1CIELAB

1718082RGBerror

Averaged results

1.530.750.76AE

6.12.83.2CIELAB

1778987RGBerror

Test set of samples

1.510.740.74AE

5.82.32.9CIELAB

1657078RGBerror

Training set of samples

90 prctile

Std DevMean

�The results show good color recovery (around2% of total error);

20 40 60 80 100

20

40

60

80

100

50

100

150

200

250

300

0 50 100 150 200 250 3000

500

1000

1500

RGBE differences

Mean: 73Std Dev: 3190 perc: 113


Original Simulated

0 5 10 15 20 25 30 35 40 45 50 55 600

200

400

600

800

1000

RMS Diferences

Gris 1390

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

1000

2000

3000

4000

5000

6000Gris 1390

SCIELab differences


6.206.204.004.001.021.022.072.070.710.7112.5812.58SCIELabSCIELab333321217788006060RMSRMS

PercPerc 9999PercPerc 9595StdStdDevDevMeanMeanMinMinMaxMaxGris Gris

13901390

270º 270º

Original Simulated0 10 20 30 40 50

0

500

1000

1500

2000

2500

3000

SCIELab differences

Rosa Valencia 1518


PercPerc 9999PercPerc 9595StdStdDevDevMeanMeanMinMinMaxMaxRosa Rosa

ValenciaValencia

0 50 100 150 200 2500

200

400

600

800

1000

1200

RMS differences

Rosa Valencia 1518


0º 0º


Original Simulated


PercPerc 9999PercPerc 9595StdStdDevDevMeanMeanMinMinMaxMaxAlabastro Alabastro

13841384

0 20 40 600

500

1000

1500

2000

2500

SCIELab differences

Alabastro 1384, 90º, Digilite

0 50 100 150 2000

200

400

600

800

1000

1200

RMS differences

Alabastro 1384, 90º, Digilite

Results : color image synthesis


0º 0º

Original Simulated


PercPerc 9999PercPerc 9595StdStdDevDevMeanMeanMinMinMaxMaxAlabastro Alabastro

13841384

0 500 1000 1500 20000

200

400

600

800

1000

1200

RMS differences

Alabastro 1384, 90º, Incandescent

0 10 20 30 40 50 600

500

1000

1500

2000

2500

3000

SCIELab differences

Alabastro 1384, 0º, Incand


0º 0º

DIFFERENT illuminant for captured and recovered images;DIFFERENT geometry;

Original Simulated


PercPerc 9999PercPerc 9595StdStd DevDevMeanMeanMinMinMaxMaxHueso Hueso 13821382

0 20 40 60 80 100 1200

500

1000

1500

2000

SCIELab differences

Hueso 1382, 90º, Tri-Lite

0 100 200 300 4000

100

200

300

400

500

600

RMS differences

Albero 1391, 180º, Digilite


DIFFERENT illuminant for captured and recovered images;DIFFERENT geometry;

90º 90º

• A digital RGB camera and a 4-source photometric-stereo can be used to model surface properties under Lambertian assumption.

• A spectral-based photometric-stereo algorithm have been developed to synthesize real-world textured objects under different illumination geometries.

• It is possible to use albedo in spectral analysis to recover spectral reflectances from linear pseudo-inverse.

Conclusions

• What are the deviations from Lambertian behaviors in real-world textured surfaces?

• …and what do other computer graphics and vision models predict?

Future

Thanks!Departamento de Departamento de ÓÓptica, Facultad de Ciencias, ptica, Facultad de Ciencias, Universidad de Granada, 18071Universidad de Granada, 18071--Granada (SPAIN)Granada (SPAIN)

MASTER Erasmus Mundus

"Color in Informatics and MEdia

Technology“ (CIMET)

University of Granada, SpainUniversity of Joensuu, Finland

University College of Gjovik, NorwayUniversity of Saint-Etienne, France

www.master-erasmusmundus-color.eu

120 ECTS (two years, in English)

The study programme of this Master course is broadly interdisciplinary, encompassing photonics, computer vision and imaging science, computer science and multimedia technology as a mix of relevant theoretical and practical knowledge.

Official fees:Non-EU students: 10.060 € per year (with 18 grants of 21.000 € to the best non-EU students)

EU students: 3.350€ per year (with 8 grants to reduce the fees to 350€ per year to the best EU students). Deadline for application: 22nd of May 2009 .

www.master-erasmusmundus-color.eu

Documents

Color Img at Prisma Network meeting 2009