Upload
katherine-watson
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Computational Colour Vision
Stephen WestlandCentre for Colour Design Technology
University of [email protected]
June 2005
http://www.colourtech.org
Oxford Brookes University
Computational Colour Vision
Introduce some basic concepts - the physical basis of colour
Computational approaches to how colour vision works
Phenomenology of colour perception (the problem)
Computational and psychophysical studies of transparency perception
The Physical Basis of Colour
C() = E()P()
The colour signal C() is the product at each wavelength of the power in the light source and the reflectance of the object
E()
P()
E()P()
Cone spectral sensitivity
LM
S L = E()P()L()
M = E()P()M()
S = E()P()S()
∫
∫
∫
Cone Responses
L = E()P()L()
M = E()P()M()
S = E()P()S()
∫
∫
∫Each cone produces a univariant response
LMS
Colour perception stems from the comparative responses of the three cone responses
Colour is a perception – ‘the rays are not coloured’
Colour Constancy
Objects tend to retain their approximate daylight appearance when viewed under a wide range of different light sources
P
0.01
0.99
Indoors (100 cd/m2)
1
99
Outdoors (10,000 cd/m2)
100
9900
The visual system is able to discount changes in the intensity or spectral composition of the illumination
WHY? / HOW?
noonnoon sunsetsunset
X
X
Computational Explanation
L1 = E1()P()L()
M1 = E1()P()M()
S1 = E1()P()S()
∫
∫
∫
L2 = E2()P()L()
M2 = E2()P()M()
S2 = E2()P()S()
∫
∫
∫
L1 / L1W = L2 / L2W
M1 / M1W = M2 / M2W
S1 / S1W = S2 / S2W
e1 = De2
D = L1W/L2W
M1W/M2W
S1W/S2W
000
000
e1 = L1
M1
S1
e2 = L2
M2
S2
Practical Use – Colour Correction
Camera RGB values vary for a scene depending upon the light source
colour correction
In order to correct the images we need an estimate of the light source under which the original image was taken
brightest pixel is white grey-world hypothesis
Colour Constancy
Adaptation is too slow to explain colour constancy
“Any visual system that achieves colour constancy is making use of the constraints in the statistics of surfaces and lights” – Maloney (1986)
Is it possible for the visual system to recover the spectral reflectance factors of the surfaces in scenes from the cone responses?
L = E()P()L()
M = E()P()M()
S = E()P()S()
∫
∫
∫
P wiBi()
Using a process such as SVD or PCA we can compute a set of basis functions Bi() such that each reflectancespectrum may be represented by a linear sum of basis functions - a linear model of low dimensionality.
If we use n basis functions then each spectrum can be represented by just n scalars or weights.
Basis Functions
1 Basis Function
Original 1 BF
P() = w1B1()
400 450 500 550 600 650 700-0.4
-0.2
0
0.2
0.4
0.6
0.8
Wavelength
Val
ue
400 450 500 550 600 650 7000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Ref
lect
ance
val
ue
Wavelength
2 Basis Functions
Original 1 BF 2 BF
P() = w1B1() + w2B2()
400 450 500 550 600 650 700-0.4
-0.2
0
0.2
0.4
0.6
0.8
Wavelength
Val
ue
400 450 500 550 600 650 7000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Ref
lect
ance
val
ue
Wavelength
3 Basis Functions
400 450 500 550 600 650 700-0.4
-0.2
0
0.2
0.4
0.6
0.8
Wavelength
Val
ue
400 450 500 550 600 650 7000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Ref
lect
ance
val
ue
Wavelength
Original 1 BF 2 BF 3 BF
P() = w1B1() + w2B2() + w3B3()
PC Variance Total Variance
1 76.77 76.77 2 15.83 92.60 3 5.96 98.56
4 0.76 99.32
5 0.37 99.68
6 0.12 99.80
7 0.09 99.89
8 0.04 99.93
About 99% of the variancecan be accounted for by a 3-D model (Maloney & Wandell, 1986)
But what proportion of the variance do we need to account for?
How many Basis Functions are Required?
6-9 basis functions are required
Simultaneous Contrast
original original covered by filter
original with small filter
Colour Constancy - spatial comparisons
“For the qualities of lights and colours are perceived by theeye only by comparing them with one another” (Alhazen, 1025)
“… object colour depends upon the ratios of light reflected from the various parts of the visual field rather than on the absolute amount of light reflected” (Marr)
ei,1/ei,2 = e'i,1/e'i,2 (Foster)
i = {L, M, S} Ratio under first light source
Ratio under second light source
Spatial Comparison of Cone Excitations
Retinex – Land and McCann (1971)
Foster and Nascimento (1994)
L1
L2=k
L’1
L’2
=k
Transparency Perception
(Ripamonti and Westland, 2001)
e’1 e’2
e1 e2
e1/e2 = e’1/e’2
What is transparency?
An object is (physically) transparent if some proportion of the incident radiation that falls upon the object is able to pass through the object.
What is perceptual transparency?
Perceptual transparency is the process ‘of seeing one objectthrough another’ (Helmholz, 1867)
Physical transparency is neither a necessary or sufficient condition for perceptual transparency (Metelli, 1974)
Even in the complete absence of any physical transparencyit is possible to experience perceptual transparency
Perceptual transparency
Research Questions
What mechanisms could drive perceptual transparency?
What are the chromatic conditions that cause transparency?
Could transparency and colour constancy be linked?
Perceptual transparency
Transparency and Spatial Ratios
ei,1/ei,2 = e'i,1/e'i,2
ei,1 ei,2
T()
e'i,1e'i,2
Experimental
Computational analysis to investigate whether for physical transparency the cone ratios are preserved
Psychophysical study to investigate whether the invariance of spatial ratios can predict chromatic conditions for perceptual transparency
Psychophysical study to compare the performance of the ratio-invariance model when the number of surfaces is varied
opaque surface P
T
(1-b)bP2T4(1-b)PT2
(1-b)b2P3T6
Physical Model of Transparency
b
P'() = P()[T()(1-b)2]2(Wyszecki & Stiles, 1982)
Monte Carlo Simulation
4. Steps 1-3 repeated 1000 times
1. A pair of surfaces P1() and P2() were randomly selected
2. A filter was randomly selected (defined by a gaussiandistribution)
m
3. The cone excitations were computed for the surfaces viewed directly (under D65) and through the filter
ei,1/ei,2 e'i,1/e'i,2
P1() P2()
ei,1/ei,2
e' i,
1/e
' i,2
i,2e
'i,1e
i,2'e
i,1eMonte Carlo Results
i,2e
'i,1e
i,2'e
i,1eMonte Carlo Results
The ratios are approximately invariant
Invariance is slightly better for the S cones
Invariance decreases as the spectral transmittancedecreases
filter S cones M cones L cones = 10nm 0.9988 (0.9968) 0.9964 (0.9951) 0.9996 (0.9955) = 50nm 0.9978 (0.9231) 1.0037 (0.8666) 0.9788 (0.8599) = 200nm 0.9231 (0.8032) 0.8885 (0.7154) 0.9144 (0.7284)
Da Pos, 1989, D’Zmura et al., 1997
xB
xP xQ
g
xA
xP
xB
xQ
xP = xA + (1-) gxQ = xB +(1- ) g
xA
Convergence
(a) convergent(deviation
0)
(b) invariant
(deviation = 0)
deviationi = 1 - [ei,1/ei,2]/[ e'i,1/e'i,2]
Psychophysical Stimuli I
d'<0 indicates subjects' preference for convergent filter; d'=0 no preference; d'>0 indicates subjects' preference for invariant filter
-3
-1
1
3
5
0 0.1 0.2 0.3 0.4 0.5 0.6
LMS deviations
d'
L M S Log. (L) Log. (S) Log. (M)
Psychophysical Results I
(a)realfilter
number of surfaces
2 4 6 8 12
(b)filter
with noise
Psychophysical Stimuli II
y = 1.1 Ln (x + 2.23)
-1
0
1
2
3
4
5
0 2 4 6 8 10 12 14
number of surfaces
d'L
M
LMS
Psychophysical Results II
Computational and pyschophysical studies show that the invariance of cone-excitation ratiosmay be a useful cue driving transparency perception
Conclusions
Colour constancy and transparency perceptionmay be related. Could they result from similar mechanisms, perhaps even similar groups of neurones?
There are still a mass of unsolved problems in computational colour vision including the relationship between cone excitations and the actual sensation of colour
xB
xP xQ
g
xA
xP
xB
xQ
xA
xP = xA + (1-) gxQ = xB +(1- ) g
xP = xA xQ = xB
Cone excitations are transformed by a a diagonal matrix whose diagonal elements are all equal
xB
xP xQ
xA xP = xA
xQ = xB
Cone excitations are transformed by a a diagonal matrix whose diagonal elements are not necessarily all equal
The two models can be made to be the same if the convergence model has no additive component and if the invariance model has equal cone scaling