Section 4 -
Jinchao Ye
CS231A Section 4 Camera Models (Problem Set 2)
10/19/2012
Jinchao Ye
1 10/19/2012
Section 4 -
Jinchao Ye
Topics
• Homogeneous Coordinates
• Transformations
• Solving a Homogeneous Linear Equation System • SVD
• Camera Calibration
2 10/19/2012
Section 4 -
Jinchao Ye
Homogeneous Coordinates
• Why?
• Without homogeneous
• With homogeneous
3
Convert to homogeneous
Linear projection
Convert from homogenous
Non-linear operation
3D World 2D Image
3D World 2D Image
3D 4D
10/19/2012
Section 4 -
Jinchao Ye
Homogeneous Coordinates
• Convert to homogeneous
𝑥, 𝑦 ⇒𝑥𝑦1
(image) 𝑥, 𝑦, 𝑧 ⇒
𝑥𝑦𝑧1
(scene)
• Convert from homogeneous
𝑥𝑦𝑤
⇒ 𝑥/𝑤, 𝑦/𝑤
𝑥𝑦𝑧𝑤
⇒ 𝑥/𝑤, 𝑦/𝑤, 𝑧/𝑤
• Keep track of dimensions – Usually, 3 (world) to 2 (image) – Homogeneous: 4(world) to 3(image) – X’ = MX
• **Hint 𝑥𝑦𝑧𝑤
⇒ 𝑥/𝑤, 𝑦/𝑤, 𝑧/𝑤
𝑐𝑥𝑐𝑦𝑐𝑧𝑐𝑤
⇒ 𝑥/𝑤, 𝑦/𝑤, 𝑧/𝑤
10/21/2011 4
Section 4 -
Jinchao Ye
Homogeneous Coordinates of Lines
5 10/19/2012
• The homogeneous coordinate of a line is • The dot product of the homogeneous coordinates of a line and a point on it is zero
• Cross product of two lines gives the homogeneous coordinate of their point of intersection
• Points at infinity (ideal points): • Line at infinity (ideal line): set of ideal points,
Section 4 -
Jinchao Ye
Rotation Matrices
• Examples
𝑅𝑧 =𝑐𝑜𝑠𝜃𝑧 −𝑠𝑖𝑛𝜃𝑧 0𝑠𝑖𝑛𝜃𝑧 𝑐𝑜𝑠𝜃𝑧 00 0 1
𝑅𝑦 =
𝑐𝑜𝑠𝜃𝑦 0 −𝑠𝑖𝑛𝜃𝑦0 1 0
𝑠𝑖𝑛𝜃𝑦 0 𝑐𝑜𝑠𝜃𝑦
𝑅𝑥 =1 0 00 𝑐𝑜𝑠𝜃𝑥 −𝑠𝑖𝑛𝜃𝑥0 𝑠𝑖𝑛𝜃𝑥 𝑐𝑜𝑠𝜃𝑥
𝑅 = 𝑅𝑥𝑅𝑦𝑅𝑧
• Important Properties – 𝑅 = 1
– 𝑅𝑇𝑅 = 𝐼
– 𝑅𝑇 = 𝑅−1
6 10/19/2012
Section 4 -
Jinchao Ye
Homography/Projective/Perspective
• Parallel lines intersect at vanishing points
• World->image
𝑥𝑦𝑤
=
𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎23𝑎31 𝑎32 𝑎33
𝑏1𝑏2𝑏3
𝑋𝑌𝑍1
• Image->image
𝑥𝑦𝑤
=
𝑎11 𝑎12 𝑏1𝑎21 𝑎22 𝑏2𝑎31 𝑎32 𝑏3
𝑋𝑌1
7 10/19/2012
Section 4 -
Jinchao Ye
A Quick Question…
• Do these 2 homography matrices yield the same points in the image space 𝑥 and 𝑥’?
8
𝑥 = 𝐻𝑋 𝑥′ = 𝑐𝐻𝑋
10/19/2012
Section 4 -
Jinchao Ye
Affine Transform
• Special case: weak perspective – simpler math (less computations) • Parallel lines remain parallel • World->image
𝑥𝑦𝑤
=𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎230 0 0
𝑏1𝑏21
𝑋𝑌𝑍1
• Image->image 𝑥𝑦𝑤
=𝑎11 𝑎12 𝑏1𝑎21 𝑎22 𝑏20 0 1
𝑋𝑌1
• |𝐻| ≠ 0 • w = 1 (no division required!)
9 10/19/2012
Section 4 -
Jinchao Ye
Solving For an Affine Transform Matrix
𝑥1𝑦11
𝑥2 𝑦2 1
𝑥3𝑦31…
=𝑎11 𝑎12 𝑎13 𝑏1𝑎21 𝑎22 𝑎23 𝑏20 0 0 1
𝑋1
𝑌1𝑍1
𝑋2
𝑌2 𝑍2
𝑋3
𝑌3 …𝑍3
1 1 1
10
unknowns
10/19/2012
Section 4 -
Jinchao Ye
Solving For an Affine Transform Matrix
𝑥1𝑦11
𝑥2 𝑦2 1
𝑥3𝑦31
=𝑎11 𝑎12 𝑏1𝑎21 𝑎22 𝑏20 0 1
𝑋1
𝑌11
𝑋2
𝑌21
𝑋3
𝑌31
11
unknowns
10/19/2012
Section 4 -
Jinchao Ye
Solving a Homogeneous Linear Equation System
• A standard problem in computer vision is to solve a set of homogeneous linear equations.
• 𝐴𝑥 = 0 – Where x is the vector of 𝑁 unknowns, and 𝐴 is the
matrix of (𝑀 × 𝑁) coefficients. We only consider the situation of 𝑀 ≥ 𝑁, so that we have enough equations to solve for the unknowns.
• 𝒙 = 0 is a trivial solution. – Constraint 𝒙 2 = 1. This is reasonable because 𝑥 is up
to scale.
10/21/2011 12
Section 4 -
Jinchao Ye
Solving a Homogeneous Linear Equation System
• 𝑀 = 𝑁
• 𝑀 > 𝑁
10/21/2011 13
𝑨𝒙 = 0 𝑠. 𝑡. 𝒙 2 = 1
𝑚𝑖𝑛 𝑨𝒙 𝟐 𝑠. 𝑡. 𝒙 2 = 1
Section 4 -
Jinchao Ye
Singular Value Decomposition
• The best numerical way to solve the homogeneous linear equation system is to perform singular value decomposition on matrix 𝑨.
10/21/2011 14
𝐴 = 𝑈Σ𝑉∗ 𝑠𝑖𝑧𝑒 𝐴 = 𝑚, 𝑛 𝑠𝑖𝑧𝑒 𝑈 = 𝑚,𝑚 𝑠𝑖𝑧𝑒 Σ = 𝑚, 𝑛 𝑠𝑖𝑧𝑒 𝑉∗ = 𝑛, 𝑛
𝑈𝑈∗ = 𝐼 𝑉𝑉∗ = 𝐼 Σ: 𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒 𝑑𝑖𝑎𝑔𝑜𝑛𝑎𝑙 𝑚𝑎𝑡𝑟𝑖𝑥 𝑤𝑖𝑡 𝑛𝑜𝑛𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒 𝑟𝑒𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟𝑠
𝑙𝑒𝑓𝑡 𝑠𝑖𝑛𝑔𝑢𝑙𝑎𝑟 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝐴 𝑎𝑟𝑒 𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟𝑠 𝑜𝑓 𝐴𝐴∗ right 𝑠𝑖𝑛𝑔𝑢𝑙𝑎𝑟 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝐴 𝑎𝑟𝑒 𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟𝑠 𝑜𝑓 𝐴𝐴∗
non − zero 𝑠𝑖𝑛𝑔𝑢𝑙𝑎𝑟 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝐴 𝑎𝑟𝑒 𝑡𝑒 𝑠𝑞𝑢𝑎𝑟𝑒 𝑟𝑜𝑜𝑡𝑠 𝑜𝑓 𝑡𝑒 𝑛𝑜𝑛− 𝑧𝑒𝑟𝑜𝑠 𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝑏𝑜𝑡 𝐴𝐴∗ 𝑎𝑛𝑑 𝐴∗𝐴
Section 4 -
Jinchao Ye
Singular Value Decomposition
• The least square solution is given by the column of 𝑽, which corresponds to the smallest diagonal entry of 𝚺.
– 𝐴𝑇𝐴𝑥 = 𝜆2𝑥 && 𝑥 2 = 1
–⇒ 𝐴𝑥 2 = 𝑥𝑇𝐴𝑇𝐴𝑥 = 𝑥𝑇𝜆2𝑥 = 𝜆2 𝑥 2 = 𝜆2
• How to calculate using Matlab
– [U,D, V] = svd(A,0);
– x = V(:, end);
10/21/2011 15
Section 4 -
Jinchao Ye
Camera Calibration
• Maps 3D world scene onto 2D image in homogeneous coordinates.
• M= 𝐾 𝑅 𝑇
– [R T]: rigid transformation
• From world to camera reference
• Extrinsic parameters
– K: camera calibration matrix
• From camera reference to sensor
• intrinsic parameters
16 10/19/2012
Section 4 -
Jinchao Ye
Camera Calibration
10/21/2011 17
Section 4 -
Jinchao Ye
Camera Calibration
10/21/2011 18
Section 4 -
Jinchao Ye
Camera Calibration
10/21/2011 19
Section 4 -
Jinchao Ye
Camera Calibration
• 𝑷𝒎 = 0
• 𝑷: 2n × 12;𝒎: 12 × 1
• Over-determined solution, minimize algebraic
error: 𝑷𝒎
– min 𝑷𝒎 𝑠. 𝑡. 𝒎 = 1
• Every singular value is non-negative.
– The right-singular vector corresponds to the smallest singular value is the solution to the optimization problem.
10/21/2011 20
Section 4 -
Jinchao Ye
Camera Calibration
10/21/2011 21
• Last column of 𝑽 gives 𝒎.
• 𝒎 = 𝒎𝟏𝑻,𝒎𝟐
𝑻,𝒎𝟑𝑻 𝑻
• 𝑴 = [𝒎𝟏;𝒎𝟐;𝒎𝟑]
• 𝑴𝑷𝒊 = 𝒑𝒊
Section 4 -
Jinchao Ye
Camera Calibration
• The above method is called Direct Linear Transformation (DLT).
• In practice, to prevent unexpected numerical error, normalization is necessary. – Use a similarity transformation 𝑇 to normalize the image
points, s.t. centroid of the points is at origin, and their RMS distance from the origin is 2. 𝑝𝑖 = 𝑇𝑝𝑖
– Use a similarity transformation 𝑈 to normalize the space points, s.t. centroid of the points is at origin, and their RMS distance from the origin is 3. 𝑃𝑖 = 𝑈𝑃𝑖
– Use DLT to minimize algebraic error. Get 𝑀
– 𝑀 = 𝑇−1𝑀 𝑈
10/21/2011 22
Section 4 -
Jinchao Ye
Camera Calibration
10/21/2011 23
Section 4 -
Jinchao Ye
Some Notation…
24 10/19/2012
Section 4 -
Jinchao Ye
Epipolar Geometry
25 10/19/2012
Section 4 -
Jinchao Ye
Essential and Fundamental Matrix
26 10/19/2012
Section 4 -
Jinchao Ye
A Simple “Trick”
• The fundamental matrix corresponding to a camera pair, M = 𝐼 0 and M′ = 𝐴 𝑎 is equal to 𝑎 𝑥𝐴
27 10/19/2012