View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Mobile Robotics:6. Vision 1
Dr. B
rian Mac N
amee (w
ww
.comp.dit.ie/bm
acnamee)
2of25
2of34 Acknowledgments
These notes are based (heavily) on those provided by the authors to accompany “Introduction to Autonomous Mobile Robots” by Roland Siegwart and Illah R. Nourbakhsh
More information about the book is available at:http://autonomousmobilerobots.epfl.ch/
The book can be bought at:The MIT Press and Amazon.com
3of25
3of34 Today’s LectureWhy is vision hard?
Brief historical overview– From early cameras to digital cameras
Low-level robot vision– Camera as sensor– Color representation– Object detection
4of25
4of34 Vision In GeneralVision is our most powerful sense providing us with an enormous amount of information about our environment and enables us to interact intelligently with the environmentIt is therefore not surprising that an enormous amount of effort has occurred to give machines a sense of visionVision is also our most complicated sense
– Whilst we can reconstruct views with high resolution on photographic paper, understanding how the brain processes the information from our eyes is still in its infancy
5of25
5of34 Vision In General (cont…)When an image is recorded through a camera, a 3-D scene is projected onto a 2-D planeIn order to try and recover some “useful information” from the scene, usually edge detectors are used to find the contours of the objectsFrom these edges or edge fragments, much research time has to been spent attempting to produce fool proof algorithms which can provide all the necessary information required to reconstruct the 3-D scene which produced the 2-D imageThe interpretation of 3-D scenes from 2-D images is not a trivial task
6of25
6of34 Vision Is Hard! (Segmentation)
7of25
7of34 Vision Is Hard! (Classification)
8of25
8of34 Vision Is Hard! (Perspectives)
9of25
9of34
Vision Is Hard! (Brightness Adaptation)
For more great illusion examples take a look at: http://web.mit.edu/persci/gaz/
10of25
10of34 Vision Is Hard! (Illusions)
Our visual systems play lots of interesting tricks on us
11of25
11of34 Vision Is Hard! (Illusions)
12of25
12of34 Vision Is Hard! (Illusions)
Stare at the cross in the middle of the image and think circles
13of25
13of34 Camera ObscuraMo Ti, Chinese philosopher, 5th Century B.C.
– Described linear light paths, pinhole image formation
Leonardo da Vinci (1452-1519)– Demonstrated camera obscura (lens added later)
Frisius (1544)
Portmerion Village, North Wales
Photograph of camera obscura interior:
14of25
14of34 Toward PhotographyPeople sought a way to “fix” the images at the back of the camera obscura
Pursued decades of experimentation with light-sensitive salts, acids, etc.
First photograph produced when?
15of25
15of34
Harry Ransom Center Kodak (reproduction)
First Photograph
Joseph Nicéphore Niépce “View from the Window at Le Gras”, c. 1826
Aluminum plate coated with light-sensitive materialMore information on the first photograph is available at: http://
www.hrc.utexas.edu/exhibitions/permanent/wfp/
Joseph Nicéphore Niépce
16of25
16of34 First Digital CamerasPhotoelectric effect (Hertz 1887; Einstein 1905)Charge-coupled devices as storage (late 1960’s)Light sensing, pixel row readout (early 1970’s)First electronic CCD still- image camera (1975):
– Fairchild CCD element – Resolution: 100 x 100 b&w– Image capture time: 23 sec.,
mostly writing cassette tape– Total weight: 8½ pounds
Kodak, c. 1975
17of25
17of34 Modern Digital CamerasToday, fifty Euro buys a camera with:
– 640 x 480 pixel resolution at 30Hz– 1280 x 960 still image resolution– 24-bit RGB pixels (8 bits per channel)– Automatic gain control, color balancing– On-chip lossy compression algorithms– Uncompressed images if desired– Integrated microphone, USB interface– Limitations
• Narrow dynamic range• Narrow FOV, with fixed spatial resolution• No motion / active vision capabilities
18of25
18of34 Vision-Based Sensors: HardwareCCD (light-sensitive, discharging capacitors of 5 to 25 micron)
2048 x 2048 CCD array
Cannon IXUS 300
Sony DFW-X700
Orangemicro iBOT Firewire
CMOS (Complementary Metal Oxide Semiconductor) technology
19of25
19of34 What Is A Digital Image?A digital image is a 2-D representation of a 3-D scene as a finite set of digital values, called picture elements or pixels
20of25
20of34 What Is A Digital Image? (cont…)Pixel values typically represent gray levels, colours, heights, opacities etc
Remember digitization implies that a digital image is an approximation of a real scene
1 pixel
21of25
21of34 Digital Image ContentsWhy are pixels represented as “RGB”?
– Is world made of red, green, and blue “stuff”?
… Answer requires a digression (or two)about human vision, cameras as sensors
Image coordinates (pixels)
u
v
IO width
height
22of25
22of34 Visible Light Spectrum
Freedman & Kaufmann, Universe
Solar (ECI, Oxford)
Incandescent (Wikipedia)
23of25
23of34 Image As MeasurementWhat does eye/camera actually observe?
… the product of illumination spectrum
with absorption or reflection spectrum!
= (at each image point)
X
Illumination spectrum IJVSReflection spectrum
24of25
24of34 Eye AnatomySpectrum incident on light-sensitive retina
Incident spectraldistribution
After Isaka (2004)
(View of R eye from above) Rods and cones
25of25
25of34 Blind-Spot ExperimentDraw an image similar to that below on a piece of paper (the dot and cross are about 6 inches apart)
Close your right eye and focus on the cross with your left eye
Hold the image about 20 inches away from your face and move it slowly towards you
The dot should disappear!
26of25
26of34
27of25
27of34 Cone SensitivitiesThree cone types (S, M, and L) are roughly blue, green, and red sensors, respectively
Their peak sensitivities occur at ~430nm, ~560nm, and ~610nm for the “average” human
Rods & cones, ~1.35 mm fromcenter of fovea
Rods & cones, ~8 mm fromcenter of fovea
Cone sensitivities as afunction of wavelength
4 m
28of25
28of34 Color PerceptionThe cones form a spectral “basis” for visible light; incident spectral distribution differentially excites S,M,L cones, leading to color vision
= (at each cone site)
X
IJVS
X
29of25
29of34 Origin Of RGB CCD SensorsSo, in a concrete sense, CCD chips are designed as RGB sensors in order to emulate the human visual system
(Vaytek)
30of25
30of34 RGB Colour ModelThink of R, G, B as color orthobasis
(0,1,0) – pure green
(0,0,1) – pure blue
(1,0,0) pure red
(1,1,1) - white
(0,0,0) - black (hidden)
31of25
31of34 HSV Colour Model
More robust against illumination changes
Still must confront noise, specularity etc.
32of25
32of34 Object DetectionSuppose we want to detect an object (e.g., a colored ball) in the field of view
We simply need to identify the pixels of some desired colour in the image … right?
Image coordinates (pixels)
u
v
IO width
height
33of25
33of34 Real-World Images
Occluded light source
Specularhighlights
Mixedpixels
Complexsurfacegeometry(self-shadowing)
Noise!
34of25
34of34 SummaryVision is our most useful sense, but is also the most difficult to replicate
Digital cameras have evolved to be powerful, cheap and accurate
Artificial vision systems tend to be modelled after the human eye
How do we do it?
35of25
35of34 Questions?
?