A new easy calibration algorithm for para-catadioptric cameras

F.Q. Duanl, R.Liu2, M.Q. Zhoul

l. College oflnformation Science and Technology, Beijing Normal University, Beijing, 100875, P.R China. 2. Base Department, Beijing Institute of Clothing Technology. Beijing, 100029, P.R China

Email: fqduan@gmail.com

Abstract

In this paper, a new easy calibration method for para-catadioptric cameras is proposed. We derive a nonlinear constraint on all camera intrinsic parameters from the projections of any three collinear space points on the viewing sphere. With the principal point known, the constraint becomes linear on all other intrinsic parameters which are the effective focal length, the aspect ratio and the skew, so the three parameters can be estimated linearly by using SVD. The principal point can be well determined from the bounding ellipse of the catadioptric image in real applications. Unlike approaches using lines in literature, the proposed method needs no conic fitting, which is hard to accomplish and highly affects the accuracy of the calibration. Experiments on simulated and real data show this method is robust and effective.

Keywords: central catadioptric camera, camera calibration

1 Introduction

Many computer vision applications such as robot navigation, surveillance, 3D modelling of large environment, and virtual reality etc, expect the imaging system could have a large field of view (FOV). However, a conventional camera has a very limited FOV. One effective way to enhance the FOV is to combine mirrors with conventional cameras, which is called a catadioptric imaging system [ 1-3]. In catadioptric systems, a single effective viewpoint is highly desirable due to its superior and useful geometric properties [3, 4]. A catadioptric system with a unique viewpoint is called a central catadioptric system. The complete class of central catadioptric systems is presented by Baker and Nayar [3]. They introduce that a central catadioptric system can be built by setting a parabolic mirror in front of an orthographic camera, or a hyperbolic, elliptical or planar mirror in front of a perspective camera, where the single viewpoint constraint can be fulfilled via a careful alignment that the camera is located at the mirror focus. Camera calibration is to estimate imaging parameters from 3D to 2D, and it is required in those application fields. Since the distortion of catadioptric images is very large, the catadioptric camera calibration is more difficult than traditional cameras. Now the calibration of catadioptric cameras has been an active research field [5- 12].

Lines are common geometric entities in man-made scenes, and are usually used in camera calibration. Geyer and Daniilidis [6, 7] use a single view of two sets of parallel lines or a single view of three lines to

978-1-4244-9630-3/10/$26.00 ©2010 IEEE

calibrate a parabolic catadioptric camera. They also propose a unified sphere model [4] for describing central catadioptric cameras, under which some algorithms [8- 12] are proposed for calibrating central catadioptric cameras. Ying and Hu [8] apply some geometric invariants of lines or spheres to calibrate central catadioptric cameras. Barreto and Arau 'jo [5] study the projective invariant properties of catadioptric images of space lines and show that any central catadioptric camera can be fully calibrated from an image of three or more space lines. Wu and Duan et al [ 10] present a group of linear constraints on the catadioptric parameters from the catadioptric projections of spatial lines. Ying and Zha [ 12] present some identical projective geometric properties of central catadioptric images of lines and spheres, and apply these properties to calibration. Since a space line is projected to a conic segment in a central catadioptric image, nearly all these approaches need conic fitting of line images, and the accuracy of the calibration highly depends on the accuracy of the conic fitting. In general, only a small segment of the conic is visible in the catadioptric image due to the partial occlusion. This makes the conic estimation hard to accomplish. Wu et al [ 14] present a calibration method of no conic estimation for para-catadioptric cameras, which uses the constraint [ 15] that straight lines have to be straight. Our work is similar to it, but different in the following aspects: Firstly, the constraint in this paper is deduced from the projection of a space line on the viewing sphere, while the one in [ 14] is from a rectified perspective image. Secondly, the constraint in this paper has a more simple expression.

The remainder of this paper is organized as follows: Section 2 reviews the unified imaging model given by Geyer and Daniilidis. Section 3 describes the proposed algorithm. Experimental results are reported in Section 4, and followed are some conclusions in Section 5.

2 Central catadioptric camera The imaging mode proposed by Geyer and Daniilidis [4] is widely used in central catadioptric camera calibration due to its simplicity and generality. They show that the central catadioptric imaging process is equivalent to the following two-step mapping by a view sphere (see Fig. 1):

1 ).Under the viewing sphere coordinate system 0-

xyz, a 3D space point X = [x, y, Z]T is projected to a point XS on the unit sphere centered at the viewpoint O byXS =[x/r,y/r,z/r]T,r=IIXII·

2).The point XS on the viewing sphere is projected to a point m on the catadioptric image plane IT by a pinhole camera through the perspective centerOc .

Figure.1 Central catadioptric camera

Tn this camera system, the optical axes of the pinhole camera is the line going through the viewpoints 0 and Oc . Hence, the image plane is perpendicular to the line OCO , and its principal point is the intersection,p=[uo,vo,I]T, of the line OcO with the image plane IT . The distance � from point o to OC is called the mirror parameter, which determines the mirror type used in a central catadioptric camera. The mirror is a paraboloid if � = 1 , an ellipsoid or a hyperboloid if 0 < � < 1 ,

and a plane if � = o. The details can be found in [4].

Let the intrinsic matrix of the pinhole camera be

K = Ir� ; �: l,

lo 0 d (1)

where f is the effective focal length; r is the aspect ratio; p = [uo, vo' l]T is the principal point and s the parameter describing the skew of the two image axes. Then the catadioptric image of a 3D space point X is

m = A K[I, �e] [ X/�IXlll = A K(X/IIXII + �e) (2)

where .A is a scalar, I is an identical matrix, e = [0,0, If , and�e is the coordinate vector of the sphere center 0 under the pinhole coordinate system OC_xc/zc. In this paper, we consider the case of a paraboloid mirror, i.e. � = 1. There are totally five parameters [[,r,s,uo ,vo] to be determined in camera calibration.

3 Calibrating algorithm

3.1 A new constraint

Tn [10], we have derived the projection of a 3D space point as follows:

Proposition 1: Let m be the catadioptric image of a space point X. Then, under the pinhole coordinate system OC_xc/zc , the projection of the point X on the viewing sphere can be expressed as:

(3)

In the case of a paraboloid mirror, i.e. � = 1 , we have

XS =3..K- 1m . r; Where T =( 1_�2 )/S2 r;=mTrom

(4)

andro = K-T K-1 is image of the absolute conic (lAC) of the pinhole camera . Based on the proposition 1, we have the following constraint on the para-catadioptric parameters:

Proposition 2: Let {mj: j = 1,2, 3} be the catadioptric image points of three collinear space points{Xj : j = 1,2, 3}. Then,

¢(ro) �

det[2m1-r;IP, 2m2 -r;2P, 2m3 -r;3P]=O. (5)

Where 17j =mjTromj,j=1,2,3 IS the algebraic

distance from the image pointmj to lAC.

Proof:

According to the unified sphere model, the projections of the three collinear space points on the viewing sphere, {X' : j = 1, 2,3} , should be on a great

I circle of the sphere. That is, the three unit vectors

ox; ,OX;, OX� are coplanar. Thus we have

det [X' -0 X' -0 X' -OJ =o 1 ' 2 ' 3

By equation (4), we have

X, 0 2

K-1 - =- m.-e

J

17j

-I ( 2 J J =K 17;

mi - Ke

K-I =- (2m -17P)

17

J J J Since detK-1 *- 0, we obtain

(6)

(7)

det [ X� -0, X; -0, X; -OJ detKI

=---det [2ml -17IP, 2m2 -172P, 2m3 -173P] =0 171172173

C/J det [2ml -7JIP, 2m2 -7J2P, 2m3 -7J3P] = o.

Hence, the constraint holds.

(8)

D

Apparently, equation (5) gives a nonlinear constraint on all five camera parameters. Based on this constraint, we can calibrate all five camera parameters using catadioptric images of several space lines by a nonlinear optimization. Usually the principal point can be well determined from the bounding ellipse of the catadioptric image [15] or by quadric equations [9]. In the following, we give a linear constraint on the other three parameters with assumption that the principal point is known.

Let

o

1

o

s

f

o

then m = Tp m is a transfonnation translating the

origin of the image plane to principal point p.

From equation (5), we have

det (Tp ) det [2ml -17IP, 2m2 -17zP, 2m3 -173P] = det [ Tp (2ml -17IP, 2m2 -172P, 2m3 -173P) ] = det [2ml -17le, 2m2 -172e, 2m3 -173e]

mIx mzx =8det mly m2y

1- ){ 1-17j{ =0.

i = 1,2,3

A K' -T K' -I [� l l roll

ro = = ro12 ro2 2 o 0

Let

a = det [� IX

. m ly

b = det [� IX

Iy [m2X c = det

m 2y

m3x m3y

1-17j{

m3X ] m '.

3y

By expanding equation (9), we get

( em�x - bm;x + amix ) rol l +

( A 2 b A 2 A 2 ) A emly - m2y + am3y (022 +

2 ( emlXmly - bm2xm2y + am3xm3y ) ro21

=e-b+a

3.2 Linear Algorithm

(9)

(10)

Since one catadioptric line image can provide three constraints about the intrinsic parameter [4,12], three triples of image points on one line image is enough to solve the three unknowns, However, using more triples and more line images can improve the calculation stability. Assume there are N catadioptric line images. The implementation of the algorithm is as follows:

Step 1. On each line image, randomly choose M

(M � 3 ) groups of three image points, and establish

a linear equation (10) using each group of three image points.

Step 2. Solve the obtained equation system by

SVD method for the three unknowns roll' ro22, ro21 . Step 3. Compute the effective focal length, scale

factors and skew factor from ro -1 , e.g. using

Choleskey decomposition.

Remark: For improving the calculation stability, the selected three image points in each group should be scattered on the line image. Here, we use distance

measure to reject the bad triples.

4 Experimental results Tn this section, we use simulated and real data to evaluate the perfonnance of the proposed algorithm.

4.1 Simulated data

The intrinsic parameters of the simulated camera

are (r,f,s, uo, vo) = (1.25,400,1,512,384) We

assume the principal point is known. Six catadioptric line images are randomly generated by choosing six unit nonnal vectors corresponding to six great circles on the viewing sphere. To simulate actual conditions, we choose 100 points on a one-third portion of the entire circle, and project these points to the catadioptric image plane. The generated line images

are shown in Fig.2. Gaussian noise with zero mean and standard deviation (Y is added to each image point

and the principal point. The noise level (Y is varied

from 0 to S pixels with a step of one pixel. For each noise level, we perform 100 independent trials. Tn each trial, SO triples of image points are randomly chosen for each line image. Means and standard deviations of the estimated parameters with respect to different noise levels are shown in Fig.3a, Fig.3b and Fig.3c for the effective focal length, the aspect ratio and the skew respectively. We can see from the figures that the mean values are very close to the ground truth in each noise level, and the standard deviations are increased gradually with respect to the noise level. It shows that the proposed method is effective and robust to image noise.

To analyze the algorithm's sensitivity to variations of

the principal point, we add Gaussian noise to the principal point. The noise level (Y is also varied from

o to S pixels with a step of one pixel. Tn addition, we fix noise level 1 pixel to the image points. Means and standard deviations of the estimated parameters with respect to different noise levels are shown in FigAa, FigAb and FigAc respectively. We can see the proposed algorithm is stable to the deviations of the principal point.

4.2 Real data

Here, the principal point is determined from the bounding ellipse of the catadioptric image. SO triples

of image points are randomly chosen for each line image.

Fig.Sa shows an image with a resolution of 1024x768 downloaded from http://maiLisr.uc.pt/�carloss/software/ so ftware.htm, which is acquired by an uncalibrated paracatadioptric camera. Five line images are selected and shown in the figure. The used points in the line images and the boundary ellipse are manually selected using the software presented in this website. Tn order to evaluate the calibration result, we use the estimated

parameters to rectifY the catadioptric image, and the rectified image is shown in Fig.Sb. We can clearly see the rectified lines are very straight. For comparison, we also perform the algorithm proposed in [S], which needs to fit line images to conics, hereafter called CPo Since the approach in [16] for conic fitting of line images in paracatadioptric systems is effective, it is used here in performing CPo The rectified image using the result of CP is shown in Fig.Sc. We can see several rectified lines in Fig.Sc are not straight.

Figure.6a shows another catadioptric image with a resolution of 2048x IS36. The image is captured by a catadioptric system consisting of a perspective camera with a hyperbolic mirror. The mirror is designed by the Center for Machine Perception, Czech Technical University, and the eccentricity of the hyperbolic mirror is l.302, corresponding to � = 0.966, which is

close to 1. It can be considered as a quasi­paracatadioptric camera. The rectified image using the estimated parameters is shown in Fig.6b. The rectified lines are also very straight. On the contrary, calibration by CP is not solvable due to bad conic fitting of the line images.

The rectified lines in the figures show that the proposed method is efficient.

5 Conclusion Central catadioptric cameras are widely used in robot navigation, surveillance and virtual reality, and camera calibration is necessary for these applications. Nearly all calibration approaches using lines need conic fitting of line images, which is hard to accomplish and highly affects the accuracy of the calibration. Tn this paper, we propose a new easy calibration method for para-catadioptric cameras. We derive a nonlinear constraint on all camera intrinsic parameters from the projections of any three collinear space points on the viewing sphere. The constraint becomes linear on all other intrinsic parameters with known principal point. The principal point can be well determined from the bounding ellipse of the catadioptric image in real applications. The proposed method needs no conic fitting and is linear, so it is easy to implement. Experiments on simulated and real data show this method is robust and effective.

Figure.2. the simulated line images

+-- ---t----------

39�1;-----;;--;-----;;-' -7, --;------;------; �Lise level (pix�)

(a) For the effective focal length

1 1 1--/ - ---- r 1,23_1;-----;;--;-----;;-2 -73 --;------;------;

r1Jisele'.<ll(pix'"i

(b) For the aspect ratio

--+f-2 3

Noise le",,1 (pix,")

(c) For the skew

Figure.3. Means and standard deviations of the estimated parameters vs. noise to image points and the principal point

�---.------ ----

399.,c--cc----c--cc-, --:-3 --'---�_____c Noise level (pix,,)

(a) For the effective focal length

1,2�.,L -;------;----:-2 --:-3 --';-�_____c r,oiselevel(pixel)

(b) For the aspect ratio

_' _<,L -;------;----:-, --;-, --';-�_____c

r,oiselevel(pixel)

(c) For the skew Figure.4. Means and standard deviations of the estimated parameters vs. noise to the principal point

(a) The used image

.. "" I .. �

-

'11--' ,,- -:lflr/i'*' :: �:�II- '"· �.·I

.. .. .. -.. �

,

.. .. .. :11 .. III

(b) The rectified image by the proposed method

(c) The rectified image by CP

Figure.5. Paracatadioptric camera

a) The used image

(b) The rectified image by the proposed method

Figure.6. Quasi-paracatadioptric camera

Acknowledgements

This work was supported by the National Natural

Science Foundation of China (Grant No.60872127).

References [1] Vagi Y & Yachida M, Real-time

omnidirectional image sensors. lJCV, 2004,58(3), 173-207.

[2] Swaminathan R, Grossberg M D, & Nayar S, Non-single viewpoint catadioptric cameras: geometry and analysis. lJCV, 2006, 66(3), 211-229

[3] Baker S and Nayer S, A theory of single viewpoint catadioptric image formation, lJCV, 1999, 35(2),175-196.

[4] Geyer C and Daniilidis K, Catadioptric projective geometry, lJCV, 2001, 45(3), 223-243

[5] Barreto J P and Araujo H, Geometric properties of central catadioptric line images and their application in calibration, IEEE Trans on PAMI, 2005, 27(8),1327-1333

[6] Geyer C and Daniilidis K, Catadioptric camera calibration, Proc.7th ICCV, 1999, YoU, 398-404

[7] Geyer C and Daniilidis K, Paracatadioptric camera calibration, IEEE Transactions on Pattern Analysis and Machine intelligence, 2002, 24(5), 687-695.

[8] Ying X.H and Hu Z.Y, Catadioptric camera calibration using geometric invariants, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(10), 1260-1271.

[9] Wu Y H and Hu Z Y, Geometric Invariants and Applications under Catadioptric Camera Model. ICCV, 2005, VolTT, 1547-1554

[10] Wu F C, Duan F Q, Hu Z Y, et aI, A new linear algorithm for calibrating central catadioptric cameras, Pattern Recognition, 2008, 41(3), 3166-3172

[11] Deng X.M, Wu F.C, Duan F.Q, et aI, Catadioptric camera calibration with one­dimensional objects, Chinese J. Comput. 2007, 30(5), 737-747

[12] Ying X H and Zha H B, Identical projective geometric properties of central catadioptric line images and sphere images with applications to calibration, lJCV, 2008, 78, 89-105,

[13] Kang S, Catadioptric self calibration. CVPR, 2000, VoLl, 201-207

[14] Wu Y, Li Y, and Hu Z, Easy calibration for para-catadioptric-like camera, IEEE International Conference on Intelligent Robots and Systems, 2006, 5719-5724,

[15] Devernay. F, Faugeras.O, Straight lines have to be straight, automatic calibration and removal of distortion from scenes of structured environments, Machine Vision and Applications 2001, 13,14-24

[16] J.P Barreto, H. Araujo, Fitting conics to paracatadioptric projection of lines. Computer Vision and Image Understanding, 2006,101(3), 151-165