Upload
hortense-bridges
View
215
Download
1
Embed Size (px)
Citation preview
Human Visual Sensor Array
physical sensor response
How the physical sensors respond to light
… actually a measure of pigment’s ability to absorb photons
virtual sensor responseimplied sensor response based on perceptual studies
…derived from colour matching studies
… using 3 primaries (700nm, 546nm, 436nm)
R=175G=200B=25
What is a colour matching study?
Subject is asked to adjust primaries until the colour of the 2 regions appears identical? to match one region with the other
Visual field divided into 2 regionsregion 1 illuminated by monochromatic lightregion 2 illuminated by primaries
R=200G=200B=50
R=0G=25B=25y=175
What is being proposed?
Subject is asked to adjust primaries until the colour of the 2 regions appears identical? to match one region with the other
4 primaries rather than 3:RGB + yellow
R=0G=0B=50y=200
… and using modern LCD technology
monochrome LCD
with modified backlighting
• one region lit by single spectrum source
• the second region lit by 4 primaries
Why?(the simple answer)
… to resolve the problem of negative primaries
ie. areas where colour matching with RGB fails
Amount of red needed to add to monochromatic stimuli to get a match
… but really, because the human brain is wired with 4 sensors in mind – organized into 2
opponent channels
How would the brain like to see its visual sensor input?
Colour information is packed into 2 ‘opponent channels’ (2 signed numbers).Driven by 4 sensors ideally, but otherwise what is available is used.
Senso
r V
alu
e
Wavelength(λ, in nm)400300 430 460 490 520 550 580 610 640 670 700
0.8
0.6
0.2
0.0
1.0
0.4
Why is this interesting?
July 2006. “What birds see”. Scientific American.
how a bird sees colour“… is difficult – impossible in fact – for humans to know”
370 nm 445 nm 508 nm 565 nm
700 nm330 nm 400 nm 500 nm 600 nm
1.0
0.5
0.0
Background
to colour
sensor array of natural visual systems
arrangement is random
note:very few blue sensors, none in the centre
Sensors we buildX
Y
The naïve approach:
Just measure
R
G
B
Opposite of what natural visual system do
http://www.cvl.iis.u-tokyo.ac.jp/~zhao/database.html
Human Perceputal Responseto luminance
350 400 450 500 550 600 650 7000
10
20
30
40
50
60
70
80
90
100
Wavelength (nm)
Abso
rpti
on
(%
)
RGB
Luminance Sensor IdealizedSe
nsor
Val
ue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
note linear response in relation to wavelength
-
What does a light stimulus look like?Se
nsor
Val
ue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
The sensor response is simple integration (summation across spectral range)
-
How do we code stimuli?Se
nsor
Val
ue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
When spectral composition is approximately equal sensor response = luminous intensity
we assume intensity is equal throughout spectrum
-
Spatial Opponency
A Peculiarity of natural visual systems:Luminance is always measured by taking the difference between two sensor values.Produces: contrast value
Sens
or V
alue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
Sens
or V
alue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
Moving from Luminance to Colour
• Primitive visual systems were luminance only
• Night-vision remains luminance only
• Evolutionary Path– Monochromacy– Dichromacy (most mammals – eg. the dog)– Tetrachromacy (birds, apes, some monkeys)
• Vital for evolution: backward compatibility
Electro-Magnetic Spectrum
Visible SpectrumVisual system must represent light stimuli within this zone.
Low resolution – equal distribution is okHigh resolution – not!
Sens
or V
alue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
spectral distribution is more complexsimple luminous intensity fails to describe stimuli correctly
-
Given a light stimulus within the visible range:
Sens
or V
alue
Wavelength(λ)λ
0.8
0.6
0.2
0.0
1.0
0.4
λ+δλ−
What information do we need to describe the stimulus fully?
1.
Lu
min
ou
s In
ten
sity
2. WavelengthIf we had a reference luminance we could calculate wavelength (by halves).
-
modify one sensor pair – shifting spectral sensitivity reference sensor:
Sen
sor
Valu
e
Wavelength
0.8
0.6
0.2
0.0
1.0
0.4
λ-δ λ λ+δ
RG
Roughly speaking wavelength is:
λ + ( R – G )One sensor can be used as a reference to measure intensity and the second to measure spectral position
the ideal light stimulusS
en
sor
Valu
e
Wavelength
0.8
0.6
0.2
0.0
1.0
0.4
λ-δ λ λ+δ
RGMonochromatic Light
Allows wavelength to be measured relative to a reference.
Problem:natural stimuli are often not ideal
Sen
sor
Valu
e
Wavelength
0.8
0.6
0.2
0.0
1.0
0.4
λ-δ λ λ+δ
RG
• Light stimulus might not activate reference sensor fully.
• Light stimulus might not be fully monochromatic.
Sens
or V
alue
Wavelength(λ, in nm)
400300 430 460 490 520 550 580 610 640 670 700
0.8
0.6
0.2
0.0
1.0
0.4
Solution:
A 3rd sensor is used to measure equiluminance.
Which is subtracted.
Then reference sensor can be normalized
This means a 3rd piece of information: 3. Equiluminance
Coding colour
With the assumption that a stimuli is monochromaticAny light stimulus (within the spectral range) can be represented exactly by 3 values:
• luminous intensity• wavelength• equiluminance
Wavelength is coded by taking a difference (or opponent) value of 2 sensors – simplest solution.
a 4 sensor opponent designS
en
sor
Valu
e
Wavelength(λ, in nm)
400300 430 460 490 520 550 580 610 640 670 700
0.8
0.6
0.2
0.0
1.0
0.4
2 opponent pairs• only 1 of each pair can be active• min sensor is equiluminance
,R G y B
350 400 450 500 550 600 650 7000
10
20
30
40
50
60
70
80
90
100
Wavelength (nm)
Abso
rpti
on
(%
)RGB
Pigment Absorption Data of human cone sensors
350 400 450 500 550 600 650 7000
10
20
30
40
50
60
70
80
90
100
Wavelength (nm)
Abso
rpti
on
(%
)
RGB
Red > Green
human colour representation is circular!
Which is not a new idea, but not currently in fashion.
540nm
620nm
480nm
Dual Opponency with Circularity
an ideal model using 2 sensor pairs
Senso
r V
alu
e
Wavelength(λ, in nm)400300 430 460 490 520 550 580 610 640 670 700
0.8
0.6
0.2
0.0
1.0
0.4
yellow - blueThe Primaries:
red - green
Defining Coloura working hypothesis
‘Colour’ means a stimulus is represented by 3 values
• luminous intensity• distance between 2 primaries• equiluminance
The primaries are fixed locations on the spectrum. The distance between primaries is measured.
Deliverables for 3 year Research Proposal
a 4-colour image standard
• luminous intensity value• dual opponent value• equiluminance value
Senso
r V
alu
e
Wavelength(λ, in nm)400300 430 460 490 520 550 580 610 640 670 700
0.8
0.6
0.2
0.0
1.0
0.4
method to produce 4 primary colours from 3 sensors
technique to create images in 4-colour format
3 sensor raw output of conventional technology may be used
algorithm is specific to device used
need to be able to translate sensor values to wavelength
Canon 400D
examine possibility of translating historical image archive to 4-colour
format
• photography• film
4-colour display prototype
adapt existing technology
if there is no direct hardware access – use pixels as sub-pixelsas long as each pixel is addressible
a diagnostic test to determine colour primaries in humans
• opponency means colours can be ‘tweaked’ by opposing complement
• human colour perception varies in the individual
• individuals with variation outside normal bounds are called ‘colour blind’
• ‘colour blindness’ can be ‘cured’ by ‘tweaking’ the primaries
a colour matching study to confirm approach
Does the 4 primary approach solve the ‘negative’ primary problem?
primariesR = 650nmG = 530nmB = 460nm
primariesR = 700nmG = 546.1nmB = 435.8nm
…using conventional lcd display technology for colour matching
Light Source
http://www.ccs.neu.edu/home/bchafy/monitor/crtlcd.html
… with modified backlighting – 2 light sources
light from 4 primaries
mono-chromatic light
(1) monochromatic light source
User selectable
(2) 4 primaries (red, green, blue, yellow)
high quality white light is already often produced by 4 primaries
each primary individually adjustable
Apparatus• monochrome LCD
display• spectrophotometer• monochromator• full spectrum light
source• 4-primary light
source
Further work:
• colour arithmetic• Transparency• implied objects
Why is understanding colour correctly important?
Colours are computed, not measured!Very important that colour information is in correct form!Starts with sensor information!
Colour is very useful for transparency
What is the colour?
Why do we need transparency?
otherwise we might have trouble with windows
… and difficulties with these kinds of tasks
Colour is very helpful in deciphering the layers
Aim: to reconstruct scenes with transparency
visual systems with 4 sensors
• Birds• Reptiles• Dinosaurs• Therapsids (our
dinosaur-like ancestor)
about 60nm between sensors
evenly spaced frequencies narrowed
370 nm 445 nm 508 nm 565 nm
700 nm330 nm 400 nm 500 nm 600 nm
1.0
0.5
0.0
The Ideal Sensor
• Equally spaced on spectrum
• Overlap with linear transition
0.8
0.6
0.2
0.0
−5 −3 1 3 5
1.0
−1 0 2 4−2−4
y
0.8
0.6
0.2
0.0400 460 580 640
1.0
520 550 610 670490430
λ
0.4
0.8
0.6
0.2
0.0400 460 580 640
1.0
520 550 610 670490430
λ
0.4
0.8
0.6
0.2
0.0400 460 580 640
1.0
520 550 610 670490430
λ
0.4
colour channel 1: R - Gcolour channel 2: yellow - B
• No overlap of opponent pairs
spectrum is shifted toward more even spacing
445
555600
020406080100380 420 460 500 540 580 620 660 700
LMS
Absorption
424 530 560
Actual Sensor Response
Sensor Response calculated from CIE perceptual data
460 530 640
CRT RGB Phosphorsspectrum is shifted more towards even spacing
HVS Sensor
+ yellow almost equal distribution
a yellow sensor + a few tweaksmakes human vision equivalent to bird vision
• even spacing• 60nm between
primary colours• response
narrowed• intermediary
colours at half-way points
0.8
0.6
0.2
0.0
1.0
y
0.4
400 460 580 640520 550 610 670490430
λrequires more processing, is less accurate, but is equivalent
Summary
• Colour is based on contrast• HVS has a circular model of spectrum• Colour is a code for where on spectrum• 2 colour channels, bi-polar 4 primary
colours• 2 channels 2-d colour space• Simple transform to circular representation• Single variable represents all colours• Purpose is to allow systematic colour
transforms colour computation
References
Poynton, C. A. (1995). “Poynton’s Color FAQ”, electronic preprint.http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html
Bangert, Thomas (2008). “TriangleVision: A Toy Visual System”, ICANN 2008.
Goldsmith, Timothy H. (July 2006). “What birds see”. Scientific American: 69–75.
Neitz, Jay; Neitz, Maureen. (August 2008). “Colour Vision: The Wonder of Hue”. Current Biology 18(16): R700-r702.
Questions?
Samples of simple colour
transforms
Blue-Yellow set to 0
Red-Green inverted
Blue-Yellow
inverted
playing with colour
is easy
these are simple
transforms
not touched by
hand