Upload
mq
View
213
Download
1
Embed Size (px)
Citation preview
A new easy calibration algorithm for para-catadioptric cameras
F.Q. Duanl, R.Liu2, M.Q. Zhoul
l. College oflnformation Science and Technology, Beijing Normal University, Beijing, 100875, P.R China. 2. Base Department, Beijing Institute of Clothing Technology. Beijing, 100029, P.R China
Email: [email protected]
Abstract
In this paper, a new easy calibration method for para-catadioptric cameras is proposed. We derive a nonlinear constraint on all camera intrinsic parameters from the projections of any three collinear space points on the viewing sphere. With the principal point known, the constraint becomes linear on all other intrinsic parameters which are the effective focal length, the aspect ratio and the skew, so the three parameters can be estimated linearly by using SVD. The principal point can be well determined from the bounding ellipse of the catadioptric image in real applications. Unlike approaches using lines in literature, the proposed method needs no conic fitting, which is hard to accomplish and highly affects the accuracy of the calibration. Experiments on simulated and real data show this method is robust and effective.
Keywords: central catadioptric camera, camera calibration
1 Introduction
Many computer vision applications such as robot navigation, surveillance, 3D modelling of large environment, and virtual reality etc, expect the imaging system could have a large field of view (FOV). However, a conventional camera has a very limited FOV. One effective way to enhance the FOV is to combine mirrors with conventional cameras, which is called a catadioptric imaging system [ 1-3]. In catadioptric systems, a single effective viewpoint is highly desirable due to its superior and useful geometric properties [3, 4]. A catadioptric system with a unique viewpoint is called a central catadioptric system. The complete class of central catadioptric systems is presented by Baker and Nayar [3]. They introduce that a central catadioptric system can be built by setting a parabolic mirror in front of an orthographic camera, or a hyperbolic, elliptical or planar mirror in front of a perspective camera, where the single viewpoint constraint can be fulfilled via a careful alignment that the camera is located at the mirror focus. Camera calibration is to estimate imaging parameters from 3D to 2D, and it is required in those application fields. Since the distortion of catadioptric images is very large, the catadioptric camera calibration is more difficult than traditional cameras. Now the calibration of catadioptric cameras has been an active research field [5- 12].
Lines are common geometric entities in man-made scenes, and are usually used in camera calibration. Geyer and Daniilidis [6, 7] use a single view of two sets of parallel lines or a single view of three lines to
978-1-4244-9630-3/10/$26.00 ©2010 IEEE
calibrate a parabolic catadioptric camera. They also propose a unified sphere model [4] for describing central catadioptric cameras, under which some algorithms [8- 12] are proposed for calibrating central catadioptric cameras. Ying and Hu [8] apply some geometric invariants of lines or spheres to calibrate central catadioptric cameras. Barreto and Arau 'jo [5] study the projective invariant properties of catadioptric images of space lines and show that any central catadioptric camera can be fully calibrated from an image of three or more space lines. Wu and Duan et al [ 10] present a group of linear constraints on the catadioptric parameters from the catadioptric projections of spatial lines. Ying and Zha [ 12] present some identical projective geometric properties of central catadioptric images of lines and spheres, and apply these properties to calibration. Since a space line is projected to a conic segment in a central catadioptric image, nearly all these approaches need conic fitting of line images, and the accuracy of the calibration highly depends on the accuracy of the conic fitting. In general, only a small segment of the conic is visible in the catadioptric image due to the partial occlusion. This makes the conic estimation hard to accomplish. Wu et al [ 14] present a calibration method of no conic estimation for para-catadioptric cameras, which uses the constraint [ 15] that straight lines have to be straight. Our work is similar to it, but different in the following aspects: Firstly, the constraint in this paper is deduced from the projection of a space line on the viewing sphere, while the one in [ 14] is from a rectified perspective image. Secondly, the constraint in this paper has a more simple expression.
The remainder of this paper is organized as follows: Section 2 reviews the unified imaging model given by Geyer and Daniilidis. Section 3 describes the proposed algorithm. Experimental results are reported in Section 4, and followed are some conclusions in Section 5.
2 Central catadioptric camera The imaging mode proposed by Geyer and Daniilidis [4] is widely used in central catadioptric camera calibration due to its simplicity and generality. They show that the central catadioptric imaging process is equivalent to the following two-step mapping by a view sphere (see Fig. 1):
1 ).Under the viewing sphere coordinate system 0-
xyz, a 3D space point X = [x, y, Z]T is projected to a point XS on the unit sphere centered at the viewpoint O byXS =[x/r,y/r,z/r]T,r=IIXII·
2).The point XS on the viewing sphere is projected to a point m on the catadioptric image plane IT by a pinhole camera through the perspective centerOc .
Figure.1 Central catadioptric camera
Tn this camera system, the optical axes of the pinhole camera is the line going through the viewpoints 0 and Oc . Hence, the image plane is perpendicular to the line OCO , and its principal point is the intersection,p=[uo,vo,I]T, of the line OcO with the image plane IT . The distance � from point o to OC is called the mirror parameter, which determines the mirror type used in a central catadioptric camera. The mirror is a paraboloid if � = 1 , an ellipsoid or a hyperboloid if 0 < � < 1 ,
and a plane if � = o. The details can be found in [4].
Let the intrinsic matrix of the pinhole camera be
K = Ir� ; �: l,
lo 0 d (1)
where f is the effective focal length; r is the aspect ratio; p = [uo, vo' l]T is the principal point and s the parameter describing the skew of the two image axes. Then the catadioptric image of a 3D space point X is
m = A K[I, �e] [ X/�IXlll = A K(X/IIXII + �e) (2)
where .A is a scalar, I is an identical matrix, e = [0,0, If , and�e is the coordinate vector of the sphere center 0 under the pinhole coordinate system OC_xc/zc. In this paper, we consider the case of a paraboloid mirror, i.e. � = 1. There are totally five parameters [[,r,s,uo ,vo] to be determined in camera calibration.
3 Calibrating algorithm
3.1 A new constraint
Tn [10], we have derived the projection of a 3D space point as follows:
Proposition 1: Let m be the catadioptric image of a space point X. Then, under the pinhole coordinate system OC_xc/zc , the projection of the point X on the viewing sphere can be expressed as:
(3)
In the case of a paraboloid mirror, i.e. � = 1 , we have
XS =3..K- 1m . r; Where T =( 1_�2 )/S2 r;=mTrom
(4)
andro = K-T K-1 is image of the absolute conic (lAC) of the pinhole camera . Based on the proposition 1, we have the following constraint on the para-catadioptric parameters:
Proposition 2: Let {mj: j = 1,2, 3} be the catadioptric image points of three collinear space points{Xj : j = 1,2, 3}. Then,
¢(ro) �
det[2m1-r;IP, 2m2 -r;2P, 2m3 -r;3P]=O. (5)
Where 17j =mjTromj,j=1,2,3 IS the algebraic
distance from the image pointmj to lAC.
Proof:
According to the unified sphere model, the projections of the three collinear space points on the viewing sphere, {X' : j = 1, 2,3} , should be on a great
I circle of the sphere. That is, the three unit vectors
ox; ,OX;, OX� are coplanar. Thus we have
det [X' -0 X' -0 X' -OJ =o 1 ' 2 ' 3
By equation (4), we have
X, 0 2
K-1 - =- m.-e
J
17j
-I ( 2 J J =K 17;
mi - Ke
K-I =- (2m -17P)
17
J J J Since detK-1 *- 0, we obtain
(6)
(7)
det [ X� -0, X; -0, X; -OJ detKI
=---det [2ml -17IP, 2m2 -172P, 2m3 -173P] =0 171172173
C/J det [2ml -7JIP, 2m2 -7J2P, 2m3 -7J3P] = o.
Hence, the constraint holds.
(8)
D
Apparently, equation (5) gives a nonlinear constraint on all five camera parameters. Based on this constraint, we can calibrate all five camera parameters using catadioptric images of several space lines by a nonlinear optimization. Usually the principal point can be well determined from the bounding ellipse of the catadioptric image [15] or by quadric equations [9]. In the following, we give a linear constraint on the other three parameters with assumption that the principal point is known.
Let
o
1
o
s
f
o
then m = Tp m is a transfonnation translating the
origin of the image plane to principal point p.
From equation (5), we have
det (Tp ) det [2ml -17IP, 2m2 -17zP, 2m3 -173P] = det [ Tp (2ml -17IP, 2m2 -172P, 2m3 -173P) ] = det [2ml -17le, 2m2 -172e, 2m3 -173e]
mIx mzx =8det mly m2y
1- ){ 1-17j{ =0.
i = 1,2,3
A K' -T K' -I [� l l roll
ro = = ro12 ro2 2 o 0
Let
a = det [� IX
. m ly
b = det [� IX
Iy [m2X c = det
m 2y
m3x m3y
1-17j{
m3X ] m '.
3y
By expanding equation (9), we get
( em�x - bm;x + amix ) rol l +
( A 2 b A 2 A 2 ) A emly - m2y + am3y (022 +
2 ( emlXmly - bm2xm2y + am3xm3y ) ro21
=e-b+a
3.2 Linear Algorithm
(9)
(10)
Since one catadioptric line image can provide three constraints about the intrinsic parameter [4,12], three triples of image points on one line image is enough to solve the three unknowns, However, using more triples and more line images can improve the calculation stability. Assume there are N catadioptric line images. The implementation of the algorithm is as follows:
Step 1. On each line image, randomly choose M
(M � 3 ) groups of three image points, and establish
a linear equation (10) using each group of three image points.
Step 2. Solve the obtained equation system by
SVD method for the three unknowns roll' ro22, ro21 . Step 3. Compute the effective focal length, scale
factors and skew factor from ro -1 , e.g. using
Choleskey decomposition.
Remark: For improving the calculation stability, the selected three image points in each group should be scattered on the line image. Here, we use distance
measure to reject the bad triples.
4 Experimental results Tn this section, we use simulated and real data to evaluate the perfonnance of the proposed algorithm.
4.1 Simulated data
The intrinsic parameters of the simulated camera
are (r,f,s, uo, vo) = (1.25,400,1,512,384) We
assume the principal point is known. Six catadioptric line images are randomly generated by choosing six unit nonnal vectors corresponding to six great circles on the viewing sphere. To simulate actual conditions, we choose 100 points on a one-third portion of the entire circle, and project these points to the catadioptric image plane. The generated line images
are shown in Fig.2. Gaussian noise with zero mean and standard deviation (Y is added to each image point
and the principal point. The noise level (Y is varied
from 0 to S pixels with a step of one pixel. For each noise level, we perform 100 independent trials. Tn each trial, SO triples of image points are randomly chosen for each line image. Means and standard deviations of the estimated parameters with respect to different noise levels are shown in Fig.3a, Fig.3b and Fig.3c for the effective focal length, the aspect ratio and the skew respectively. We can see from the figures that the mean values are very close to the ground truth in each noise level, and the standard deviations are increased gradually with respect to the noise level. It shows that the proposed method is effective and robust to image noise.
To analyze the algorithm's sensitivity to variations of
the principal point, we add Gaussian noise to the principal point. The noise level (Y is also varied from
o to S pixels with a step of one pixel. Tn addition, we fix noise level 1 pixel to the image points. Means and standard deviations of the estimated parameters with respect to different noise levels are shown in FigAa, FigAb and FigAc respectively. We can see the proposed algorithm is stable to the deviations of the principal point.
4.2 Real data
Here, the principal point is determined from the bounding ellipse of the catadioptric image. SO triples
of image points are randomly chosen for each line image.
Fig.Sa shows an image with a resolution of 1024x768 downloaded from http://maiLisr.uc.pt/�carloss/software/ so ftware.htm, which is acquired by an uncalibrated paracatadioptric camera. Five line images are selected and shown in the figure. The used points in the line images and the boundary ellipse are manually selected using the software presented in this website. Tn order to evaluate the calibration result, we use the estimated
parameters to rectifY the catadioptric image, and the rectified image is shown in Fig.Sb. We can clearly see the rectified lines are very straight. For comparison, we also perform the algorithm proposed in [S], which needs to fit line images to conics, hereafter called CPo Since the approach in [16] for conic fitting of line images in paracatadioptric systems is effective, it is used here in performing CPo The rectified image using the result of CP is shown in Fig.Sc. We can see several rectified lines in Fig.Sc are not straight.
Figure.6a shows another catadioptric image with a resolution of 2048x IS36. The image is captured by a catadioptric system consisting of a perspective camera with a hyperbolic mirror. The mirror is designed by the Center for Machine Perception, Czech Technical University, and the eccentricity of the hyperbolic mirror is l.302, corresponding to � = 0.966, which is
close to 1. It can be considered as a quasiparacatadioptric camera. The rectified image using the estimated parameters is shown in Fig.6b. The rectified lines are also very straight. On the contrary, calibration by CP is not solvable due to bad conic fitting of the line images.
The rectified lines in the figures show that the proposed method is efficient.
5 Conclusion Central catadioptric cameras are widely used in robot navigation, surveillance and virtual reality, and camera calibration is necessary for these applications. Nearly all calibration approaches using lines need conic fitting of line images, which is hard to accomplish and highly affects the accuracy of the calibration. Tn this paper, we propose a new easy calibration method for para-catadioptric cameras. We derive a nonlinear constraint on all camera intrinsic parameters from the projections of any three collinear space points on the viewing sphere. The constraint becomes linear on all other intrinsic parameters with known principal point. The principal point can be well determined from the bounding ellipse of the catadioptric image in real applications. The proposed method needs no conic fitting and is linear, so it is easy to implement. Experiments on simulated and real data show this method is robust and effective.
Figure.2. the simulated line images
+-- ---t----------
39�1;-----;;--;-----;;-' -7, --;------;------; �Lise level (pix�)
(a) For the effective focal length
1 1 1--/ - ---- r 1,23_1;-----;;--;-----;;-2 -73 --;------;------;
r1Jisele'.<ll(pix'"i
(b) For the aspect ratio
--+f-2 3
Noise le",,1 (pix,")
(c) For the skew
Figure.3. Means and standard deviations of the estimated parameters vs. noise to image points and the principal point
�---.------ ----
399.,c--cc----c--cc-, --:-3 --'---�_____c Noise level (pix,,)
(a) For the effective focal length
1,2�.,L -;------;----:-2 --:-3 --';-�_____c r,oiselevel(pixel)
(b) For the aspect ratio
_' _<,L -;------;----:-, --;-, --';-�_____c
r,oiselevel(pixel)
(c) For the skew Figure.4. Means and standard deviations of the estimated parameters vs. noise to the principal point
(a) The used image
.. "" I .. �
-
'11--' ,,- -:lflr/i'*' :: �:�II- '"· �.·I
.. .. .. -.. �
,
.. .. .. :11 .. III
(b) The rectified image by the proposed method
(c) The rectified image by CP
Figure.5. Paracatadioptric camera
a) The used image
(b) The rectified image by the proposed method
Figure.6. Quasi-paracatadioptric camera
Acknowledgements
This work was supported by the National Natural
Science Foundation of China (Grant No.60872127).
References [1] Vagi Y & Yachida M, Real-time
omnidirectional image sensors. lJCV, 2004,58(3), 173-207.
[2] Swaminathan R, Grossberg M D, & Nayar S, Non-single viewpoint catadioptric cameras: geometry and analysis. lJCV, 2006, 66(3), 211-229
[3] Baker S and Nayer S, A theory of single viewpoint catadioptric image formation, lJCV, 1999, 35(2),175-196.
[4] Geyer C and Daniilidis K, Catadioptric projective geometry, lJCV, 2001, 45(3), 223-243
[5] Barreto J P and Araujo H, Geometric properties of central catadioptric line images and their application in calibration, IEEE Trans on PAMI, 2005, 27(8),1327-1333
[6] Geyer C and Daniilidis K, Catadioptric camera calibration, Proc.7th ICCV, 1999, YoU, 398-404
[7] Geyer C and Daniilidis K, Paracatadioptric camera calibration, IEEE Transactions on Pattern Analysis and Machine intelligence, 2002, 24(5), 687-695.
[8] Ying X.H and Hu Z.Y, Catadioptric camera calibration using geometric invariants, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(10), 1260-1271.
[9] Wu Y H and Hu Z Y, Geometric Invariants and Applications under Catadioptric Camera Model. ICCV, 2005, VolTT, 1547-1554
[10] Wu F C, Duan F Q, Hu Z Y, et aI, A new linear algorithm for calibrating central catadioptric cameras, Pattern Recognition, 2008, 41(3), 3166-3172
[11] Deng X.M, Wu F.C, Duan F.Q, et aI, Catadioptric camera calibration with onedimensional objects, Chinese J. Comput. 2007, 30(5), 737-747
[12] Ying X H and Zha H B, Identical projective geometric properties of central catadioptric line images and sphere images with applications to calibration, lJCV, 2008, 78, 89-105,
[13] Kang S, Catadioptric self calibration. CVPR, 2000, VoLl, 201-207
[14] Wu Y, Li Y, and Hu Z, Easy calibration for para-catadioptric-like camera, IEEE International Conference on Intelligent Robots and Systems, 2006, 5719-5724,
[15] Devernay. F, Faugeras.O, Straight lines have to be straight, automatic calibration and removal of distortion from scenes of structured environments, Machine Vision and Applications 2001, 13,14-24
[16] J.P Barreto, H. Araujo, Fitting conics to paracatadioptric projection of lines. Computer Vision and Image Understanding, 2006,101(3), 151-165