Upload
randolf-johns
View
228
Download
1
Tags:
Embed Size (px)
Citation preview
CS 523 (CS 423/EE 533)
Computer Vision
Lecture 1INTRODUCTION TO COMPUTER VISION
About the Course
2
http://vvgl.ozyegin.edu.tr
Objective
Introduction to the theory, tools, and algorithms of 3D computer vision
Instructor
Assist. Prof. M. Furkan Kıraç
E-mail: [email protected]
Room: 219
Hours
Wednesdays, 10:40-13:30, Room: 241
Grading
Projects: 6x10%
Final Exam: 40%
Syllabus
3
Short Projects:Late submissions are not accepted. Copying answers from others’ work is not permitted.
Final Exam:At least 3 of the 6 Short Projects must be turned in by the due date in order to qualify for the Final Exam. No make-up will be given for the Final Exam. Students can take the Bütünleme exam if they miss the Final Exam.
Grading
4
Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010.
Computer Vision: A Modern Approach, David A. Forsyth and Jean Ponce, Prentice-Hall, 2002.
Introductory Techniques for 3D Computer
Vision, Emanuele Trucco and Alessandro Verri, Prentice-Hall 1998.
Recommended Books
5
Learning OpenCV, Gary Bradski and Adrian Kaehler, O'Reilly, 2008.
OpenCV 2 Computer Vision Application Programming Cookbook, Robert Laganiere, Packt Publishing, 2011.
Mastering OpenCV with Practical Computer Vision Projects, Daniel Lelis Baggio, et al., Packt Publishing, 2012.
OpenCV Resources
6
Week Lectures
24 September 2014 Lecture 1
1 October 2014 Lecture 2
8 October 2014 Lecture 3
15 October 2014 Lecture 4
22 October 2014 Lecture 5
29 October 2014 Lecture 6
5 November 2014 Lecture 7
12 November 2014 Lecture 8
19 November 2014 Lecture 9
26 November 2014 Lecture 10
3 December 2014 Lecture 11
10 December 2014 Lecture 12
17 December 2014 Lecture 13
24 December 2014 Lecture 14
31 December 2014 Lecture 15 ?
Applications of Computer Vision
8
Image Stitching
Image Matching
Object Recognition
3D Reconstruction
Interior Modeling
13
3D Augmented Reality
14
3D Camera Tracking
15
16
Stereo Conversion for 3DTV
Depth Estimation and View Interpolation for 3DTV
17
Human Tracking
18
License Plate Recognition
19
Human Pose Estimation
20
Course Outline
21
3D geometry fundamentals Transformations and projections Camera calibration Feature detection and matching Image stitching Single view geometry Two view geometry Multiple view geometry Stereo vision and depth estimation 3D structure from motion 3D camera tracking
Topics to be covered
22
Relation to Other Fields
23
Computer Vision
24
Figure from "Computer Vision: Algorithms and Applications,” Richard Szeliski, Springer, 2010.
Lights and materials Shading Texture mapping Environment effects Animation 3D scene modeling 3D character modeling (OpenGL)
Computer Graphics
25
Computer Graphics
26
Resampling Enhancement Noise filtering Restoration Reconstruction Segmentation Image compression (MATLAB and OpenCV)
Image Processing Topics
27
Image Processing
28
Spatio-temporal sampling Motion estimation Frame-rate conversion Multi-frame noise filtering Multi-frame restoration Super-resolution Video compression (MATLAB & OpenCV)
Video Processing Topics
29
Video acquisition-display chain
30
Capture Representation Coding
Transmission Decoding Rendering
Human vs. Computer
31
Optical illusions
Actual vs. Perceived Intensity (Mach band effect)
33
Brightness Adaptation of the Eye
34
Optical illusions
Optical illusions
Why is Computer Vision Difficult?
Human perception
Human perception
Human Visual System
41
Human Eye
Photoreceptors: Rods & Cones
Rods vs. Cones
RodsPerceive brightness onlyNight vision
ConesPerceive colorDay visionRed, green, and blue cones
Cone Distribution
64%
32%
2%
Blue is less-focused
Visual Threshold drop during Dark Adaptation
Spatial Resolution of the Human Eye Photopic (bright-light) vision:
Approximately 7 million cones Concentrated around fovea
Scotopic (dim-light) vision Approximately 75-150 million rods Distributed over retina
(HDTV: 1920x1080 = 2 million pixels)
50
Frequency Responses of Cones
Same amount of energy produces different sensations of brightness at different wavelengths
Green wavelength contributes most to the perceived brightness.
51
Trichromatic Color Mixing
Any color can be obtained by mixing three primary colors Red, Green, Blue (RGB) with the right proportion
valuessTristimulu :,3,2,1
kk
kk TCTC
Image Formation
54
Human Eye vs. Camera
Camera components Eye components
Lens Lens, cornea
Shutter Iris, pupil
Film Retina
Cable to transfer images Optic nerve to send the incident light information to the brain
Human Vision
Image formation
Pin-Hole Camera Model
Point Spread Effect
Out-of-Focus Blur
Shrinking the Aperture
Converging Lens
Correction with a Converging Lens
Perfectly In-Focus for a Certain Distance Only
“circle of confusion”
Depth-of-Field
Depth-of-Field
“Sharp Image” within Depth-of-Field due to Finite Sensor Size
NZFZ
Focal Length (F) and Depth (Z)
Z
YFy
F
y
Z
Y
Z
XFx
Aperture Size Affects Depth-Of-Field
f / 5.6
f / 32
Aperture
2dA
Camera f-number
d
Ff
2
f
FA
Exposure Time
Motion Blur Effect due to Finite Exposure Time
Decrease in aperture implies…
Increase in depth-of-field Decrease in motion blur Decrease in exposure
2D Image Representation
76
77
Image Capture
(Courtesy Gonzalez & Woods)
Digital Image Capture
Digital Image Capture
Light sensitive diodes convert photons to electrons
Color Image Capture: Single vs. Three CCD Arrays
RGB splitter(three separate imaging sensors, higher resolution)
Bayer filter(cheaper but introduces spatial resolution loss)
Digital Camera Issues
Noise caused by low light
Color color fringing (chromatic aberration) artifacts from Bayer patterns
Blooming charge overflowing into neighboring pixels
In-camera processing over-sharpening can produce halos
Compression creates blocking artefacts
Digitization: Sampling and Quantization
Over Sampling
Over Quantization
84
85
Images as Matrices of Integers
126 127 126
125 126 127
123 126 125
128 127 124
123 120 144
121 128 155
126 123 127
120 122 124
119 121 123
122 142 162
130 157 161
145 162 164
158
163
160
164
166
165
m
n
(0,0)
0 ≤ s(m,n) ≤ 255 } quantization
0 ≤ m ≤ M-1
0 ≤ n ≤ N-1
MxN 8-bit gray-scale (intensity, luminance) image
sampling
0 → black, 255 → white
Images as Functions
We can think of an image as a function, f, from R2 to R: f( x, y ) gives the intensity at position ( x, y ) Realistically, we expect the image only to be defined
over a rectangle, with a finite range:• f: [a,b]x[c,d] [0,1]
A color image is just three functions pasted together. We can write this as a “vector-valued” function:
( , )
( , ) ( , )
( , )
r x y
f x y g x y
b x y
RGB Color Bands (Channels)
Red
Green Blue
YUV Bands
Also called Y Cb Cr Y : Luma
Cb : Chrominance_blueCr : Chrominance_red
Y
U (Cb)
V(Cr)
Color
YUV-RGB Conversion
Summary
90
Human visual system
Pin-hole camera model
Image representation
Summary
91
How to find camera parameters? Where is the camera, where is it directed at? What is the movement of the camera?
Where are the objects located in 3D? What are the dimensions of objects in 3D? What is the 3D structure of a scene?
How to process stereo video? How to detect and match image features? How to stitch images?
Problems to be Addressed
92