PTZ Camera Network Calibration from Moving People in Sports Broadcasts

Jens PuweinETH Zurich

puweinj@student.ethz.ch

Remo ZieglerLiberoVision

http://www.liberovision.com

Luca Ballan, Marc PollefeysETH Zurich

{luca.ballan,marc.pollefeys}@inf.ethz.ch

Abstract

In sports broadcasts, networks consisting of pan-tilt-zoom (PTZ) cameras usually exhibit very wide baselines,making standard matching techniques for camera calibra-tion very hard to apply. If, additionally, there is a lack oftexture, finding corresponding image regions becomes al-most impossible. However, such networks are often set up toobserve dynamic scenes on a ground plane. Correspondingimage trajectories produced by moving objects need to ful-fill specific geometric constraints, which can be leveragedfor camera calibration.

We present a method which combines image trajectorymatching with the self-calibration of rotating and zoomingcameras, effectively reducing the remaining degrees of free-dom in the matching stage to a 2D similarity transforma-tion. Additionally, lines on the ground plane are used to im-prove the calibration. In the end, all extrinsic and intrinsiccamera parameters are refined in a final bundle adjustment.The proposed algorithm was evaluated both qualitativelyand quantitatively on four different soccer sequences.

1. IntroductionPan-tilt-zoom (PTZ) camera networks are widely used

in sports broadcasts. In order to analyze and understand thecaptured events, free-viewpoint video and augmented real-ity have proven to be valuable tools. The calibration of PTZcamera networks plays a crucial role in these applications,and it is a requirement for creating novel synthetic views ofthe recorded scenes. Even a small error in the alignment oftwo images can result in visible artifacts during scene re-rendering from a different point of view. This requires ajoint calibration of the cameras and priority should be givento the minimization of this alignment error.

Typically, the calibration can only be performed basedon the acquired footage since the access to the recording fa-

cilities is constrained by restrictive rules. Estimating theintrinsic and the extrinsic parameters of each camera ateach time instant from the recorded videos is a challeng-ing problem. The wide baselines and the absence of tex-tured regions, typical for the footage recorded during socceror rugby games, e.g., make standard calibration techniqueshard to apply. While appearance based matching techniquessucceed in relating contiguous images of a video capturedby the same camera, in general, no correspondences can befound between images captured by different cameras. Onthe other hand, methods solely relying on the detection offield model lines work only if a model is available and aminimum number of lines is visible and detectable in eachimage.

While appearance based matching techniques are not ap-plicable to find correspondences, player motions providevery strong cues. Just like feature correspondences, corre-sponding player tracks from different cameras need to fulfillspecific geometric constraints.

In this paper, we present an approach to multi-view PTZcamera calibration of sports broadcasts. Since the cameranetwork is set up to capture players moving on a groundplane, the player trajectories are leveraged for the cameracalibration. If present, field lines are used to further improvethe calibration. The resulting calibration technique shows tobe robust and allows the calibration of challenging footagelike the one obtained from soccer matches.

2. Related workIn sports broadcasts, it is common to use a 3D line model

of a standardized playing field for calibration. Relating im-age lines and model lines leads to a set of 2D-to-3D linecorrespondences. Correspondences are hypothesized andthe resulting camera parameters are verified by comparingthe projected model lines with the image lines [8]. Oncethe center of projection of the camera is known, pan, tiltand zoom can be determined more robustly by exhaustively

25

comparing image lines with a database of projected modellines for different values of pan, tilt and zoom [23]. How-ever, such approaches depend on a minimum number of vis-ible lines and fail if this requirement is not met. If team lo-gos, advertisements or other distinctive features are presenton the playing field, calibration is still possible [20]. Insports like soccer or rugby, however, no logos or advertise-ments are available on the playing field.

For static camera networks, object trajectories have al-ready been exploited for calibration. For instance, Steintreated the case of unsynchronized cameras with known in-trinsic parameters [22]. The extrinsic parameters are foundby estimating the temporal offset and the homography relat-ing the trajectories of objects moving on a common groundplane. The approach proposed by Jaynes additionally dealswith an unknown focal length per camera, but it requiresthe user to specify two bundles of coplanar parallel linesfor each camera [13]. Assuming a planar scene and syn-chronized cameras, image trajectories are then warped onto3D planes and the extrinsics are found by estimating a 3Dsimilarity transformation between these planes. The workof Meingast et al. does not require objects to move on aground plane [19]. Track correspondences are hypothe-sized and verified through geometric constraints to deter-mine the correct essential matrices and temporal offsets be-tween views. In the case of single static cameras, objectswith known properties provide constraints for camera cali-bration. A common approach is to treat people as verticalpoles of constant height [18, 14, 6]. A solution based onobjects moving on non-parallel lines at constant speeds waspresented by Bose and Grimson [3].

However, all these approaches do not address the prob-lem of cameras which pan, tilt and zoom during the record-ing. Moreover, in sports broadcasts, assumptions like thevertical pole and the constant speed of the players cannotbe used.

3. Algorithm OverviewThe input to our problem consists of a number of syn-

chronized video sequences captured by PTZ cameras. LetIci denote the image captured by camera c at time i. Thedesired output is the full calibration of the PTZ camera net-work, i.e. the intrinsic parameters Kci , the distortion pa-rameters kci , the rotation matrices R

ci , and the centers of

projection Cc for all the cameras at each time instant. LetPci denote the projection matrix of camera c at frame i, de-fined as Pci = K

ci [R

ci | RciCc]. The center of projection

(COP) of each camera is assumed to be constant over time,but unknown. In addition, the pan axis is assumed to bealways perpendicular to the playing field. To simplify theequations in this paper, the world coordinate system is cho-sen such that the xy-plane coincides with the ground plane.Hence, the pan axis is parallel to the z-axis, and the tilt axis

Figure 1. Typical camera arrangement with respect to the playingfield. The pan axis (red) is orthogonal to the ground plane, whilethe tilt axis (blue) is parallel to it.

is parallel to the xy-plane and orthogonal to the pan axis(see Figure 1). The pan angles, tilt angles and focal lengthsof each camera change with every frame.

Ideally, the intrinsic parameters of the cameras would bepre-calibrated in a controlled environment, but access to therecording facilities and the cameras is typically restricted.We assume that the principal point lies in the center of theimage. Since there is a correlation between the principalpoint and the remaining camera parameters, the error that ismade by fixing the principal point in the middle of the imagecan be partially compensated by slightly changing the focallength, the COP and the orientation. Skew is assumed to bezero and the aspect ratios are assumed to be known. Underthese assumptions, each intrinsic matrix Kci can be writtenas diag(f ci , f

ci , 1), where f

ci denotes the focal length. Ra-

dial distortion is modeled with a single parameter kci vary-ing over time, such that[

xdyd

]= (1 + kci (x

2n + y

2n))[xnyn

],

where (xn, yn)T are the normalized image coordinates, and(xd, yd)T are the distorted ones.

First, each camera is calibrated independently. A localcoordinate system is chosen for each camera, and the panand tilt angles, the focal lengths and the distortion coeffi-cients are determined with respect to this local coordinatesystem. Once these parameters are estimated, a metric rec-tification of the ground plane can be computed for eachframe. A top-down view resulting from such a rectificationis shown in Figure 2.

Subsequently, players are tracked in each video sequenceindependently and the tracks are warped onto the groundplane. The COPs and the rotation angles with respect to acommon coordinate system are determined by the 2D sim-ilarity transformations relating the warped player tracks inall the video sequences. Camera parameters are refined in afinal bundle adjustment.

26

Figure 2. Top-down views of the playing field for two cameras.

4. Single Camera CalibrationIn the first phase, each video sequence is treated inde-

pendently. To improve readability, camera indices are re-moved in this section. SIFT features are extracted every tenframes and matched to SIFT features extracted ten framesbefore and after [16]. Additionally, Harris corners are ex-tracted and tracked using a GPU implementation of the KLTtracker [17, 21]. Corresponding points in images Ii and Ijare related by the following homography [12]:

Hij = KjRjRTi Ki1 = KjRijKi1

where Ri and Rj are the rotation matrices defined in the lo-cal coordinate system of the camera (denoted by the tilde),at time i and j, respectively. Rij is the relative rotation be-tween the two views, which is independent of the coordinatesystem. A point xj in image Ij is related to its correspond-ing point xi in Ii as

xj Hijxiwhere xi and xj are given in homogenous coordinates and denotes equality up to scale.

From an image sequence produced by a rotating andzooming camera, it is in general possible to determine theintrinsic parameters of the camera as well as its relative rota-tions [11]. If only the focal lengths and the rotation anglesare needed, there exists a minimal solution using 3 pointcorrespondences between two frames [4]. We use this min-imal solution to remove outliers in a RANSAC step and toget initial values for the focal lengths and the rotation an-gles [9]. The estimation of the focal length is unstable forsmall rotations, but the work of Agapito et al. suggests thatthe relative change of focal lengths can still be recovered[1]. This means that an absolute value has to be determinedonce from a pair of images separated by a sufficiently largerotation. Keeping the focal length fixed for those images,the remaining absolute values are given through the relativechanges between frames. Given the initial absolute valuesfor the focal lenghts, the initial rotations are determined.

The initial rotation angles and focal lengths are refinedin a bundle adjustment using the sparseLM library together

with the Cholmod solver (the same libraries are also usedfor all subsequent nonlinear least squares problems) [24, 15,7]. The reprojection errors are minimized by optimizingfor the feature positions, the rotation angles and the focallengths. Under the assumptions stated in Section 3, depictedin Figure 1, the rotation matrix Ri is given by two rotationangles and can be rewritten as

Ri = RiRi

where

Ri =

1 0 00 cosi sini0 sini cosi

and

Ri =

cos i sin i 0sin i cos i 00 0 1

.While the tilt angle i is given with respect to a coordinatesystem common to all the cameras, the pan angle i is givenin the local coordinate system of the respective camera, i.e.,up to an additive constant c . More precisely, i = i+ c .The relative rotation between frame i and frame j can bewritten as

Rij = Rj Rj RTiRi

T = RjRijRiT

which corresponds to cosij sinijcosi sinijsinicosjsinij sinjsinij

,where ij is the pan angle between frames i and j. j canbe obtained as atan2(Rij(3, 1),Rij(2, 1)). Similarly, i isobtained from Rij(1, 2) and Rij(1, 3). Care has to be takenwhen ij is zero, i.e., when there is no panning motion. Inthis case, the tilt angles cannot be recovered. However, thisis not an issue since normally for every frame in the se-quence there exists at least another one with a different panangle. Using the initial values of the absolute tilt angles iand the relative pan angles ij , another bundle adjustmentis performed, also including radial distortion coefficients ki.Rotation matrices are now parametrized with pan and tiltangles instead of three Euler angles.

To further improve the results, image lines are includedin the calibration process. After undistorting the input im-ages, lines are extracted and matched between neighboringframes. For all sets of matching image lines, we try to findthe camera parameters and the line positions on the groundplane which best explain the image lines.

In order to add lines to the calibration process, we intro-duce a mapping between points on the ground plane and im-age points. Let some frame r be a reference frame. At this

27

point, the projection matrix Pr is known up to the pan angleand the center of projection. The constraint of the groundplane being the xy-plane of the world coordinate system isnot violated by arbitrary rotations around the z-axis, trans-lations in x- and y-direction or scalings of the scene. There-fore, the world coordinate system can be chosen such thatPr = Kr[Rr | Rr (0, 0,1)T ]. Since points on theground plane are of the form (x, y, 0), the homographymapping the ground plane to image Ii is HriKrRr .

The coordinates of a line on the ground plane are givenby a point on the 3D unit sphere, parameterized by two ro-tation angles and . Each image line is represented bypoint samples, where each point sample x corresponds to apoint x on a line l(, ) on the ground plane. To specifythe coordinates of x lying on the line given by and , weintroduce an additional parameter for each point sample,such that x is a function p of , and :

x HriKrRr x = HriKrRrp(, , )

To extract lines, a Canny edge detector is used in the undis-torted images (see Figure 3) [5]. E.g., white field lines ongreen background generate two lines. Such lines are veryclose and are merged into a single line. Correspondencesbetween image lines are then determined by using the cur-rent estimates of the Hijs. If a line in image Ii is suffi-ciently close to a line in Ij after being transformed usingHij , the two lines are considered a match. If, however, athird line is close to either one of those two lines, the matchis simply discarded. For each set of matching lines, a line onthe ground plane is initialized by calculating a least squaresfit to the line samples projected onto the ground plane.

In addition to the reprojection errors obtained from theKLT tracks and the SIFT feature points, the reprojection er-rors of the line samples are added to the bundle adjustment.Additionally, e.g., on soccer fields, lines are either paral-lel or orthogonal. The deviation from known angles can beadded as well. In a third bundle adjustment, we optimizefor all pan angles, tilt angles, focal lengths, distortion coef-ficients and all the s, s and s.

5. Camera Network Calibration

5.1. Fixing a World Coordinate System

So far, cameras were treated completely independently,and the projection matrices Pci were computed up to an un-known COP and up to an unknown offset of the pan angles.

The world coordinate system is chosen with respect toan arbitrary reference camera a and an arbitrary referenceframe r, such that Ca = (0, 0,1)T and ar = 0, i.e.,Par = K

ar [R

ar |Rar (0, 0,1)T ]. For any additional cam-

era b, we get Pbr = Kbr[R

br| RbrCb]. The relative pan and

translation with respect to camera a needs to be determined

Figure 3. Lines detected in the undistorted image.

in order to find br and Cb. By applying the transformation

Tar with

Tar =[Rar

T (0, 0,1)T0 1

]to Par and P

br we get P

ar = K

ar [I|0] and Pbr = Kbr [R|t],

where R = RbrRarT and t = Rbr((0, 0,1)T Cb). Since

points in image Ici can be warped into a reference image Icr

by using Hcir, the following formulas are derived for ref-erence frame r and the index denoting the frame numberis dropped to improve readability. Pc, Kc, Rc, Rc andRc denote P

cr, K

cr, R

cr, R

cr and R

cr . For camera matri-

ces Pa = Ka [I|0] and Pb = Kb [R|t], the homography Hinduced by the plane with normal n located at a distance dfrom the origin is given as [12]:

H = Kb(R 1dtnT )K1a .

n is the normal of the ground plane, i.e., the xy-plane. Thusthe associated normal vector is (0, 0,1)T . However, sinceTar was applied to the projection matrices, this normal needsto be transformed accordingly and we get

n = Ra

001

.Because Ka, Kb and the tilt angles are known, the imagecoordinates and H can be transformed accordingly, leavingfour unknowns b, tx, ty and tz:

H = RTbK1b HKaRa

= RTbK1b Kb(RbRbR

Ta

1dtnT )K1a KaRa

= Rb RTb1dt

001

T =cos(b) sin(b) txsin(b) cos(b) ty

0 0 tz

Matrix H is defined up to scale, therefore a linear solu-

tion can be obtained from 2 correspondences of points lyingon the ground plane, i.e., 4 equations.

28

One might think that 3 equations are enough to recoverone angle and a translation up to scale. However, 1d t is un-affected by scale changes of the scene, hence it is not up toscale. Determining H can also be seen as finding the simi-larity transformation - scale, 2D translation and one rotationangle - between two top down views.

If the points are not lying on the ground plane, 3 pointcorrespondences are still enough to determine pan andtranslation [10]. A method to extract correspondences be-tween points on the ground plane is presented in the follow-ing subsection.

5.2. Correspondences From Player Trajectories

To generate player trajectories, we used a simple blobtracker. Foot positions are extracted from the player seg-mentations. A Gaussian mixture color model is created forboth foreground and background colors by having the userselect a few pixels with a few strokes [2].

The segmentation results in a few blobs. Associationsbetween the blobs of different frames are made based onspatial proximity in a very conservative way. Wheneverthe trajectories of two blobs get too close, the tracking ofthese blobs stops. We use the strategy of Lv et al. to extractthe foot positions of the players by using the principal axesof the blobs [18]. An example of foot tracks is shown inFigure 4. If a good people tracker is available, image tra-jectories can also be created from the centers of boundingboxes. However, the trajectories do not lie on the groundplane anymore, which has to be taken into account [10].

For each camera, image trajectories of the feet are thenwarped onto the ground plane given in the local coordinatesystem of the respective camera, as explained in Section 4.We define the set of candidate matches for each pair of cam-eras as all the possible pairs of extracted trajectories with atemporal overlap of at least 10 frames. Since the soughtafter transformation, a similarity transformation, is deter-mined by two point correspondences, one correct trajectorymatch is sufficient to find the remaining unknowns in theabsence of errors. This holds even if the tracks are perfectlystraight. The only condition which has to be fulfilled is thatthe trajectory itself is not a single point, i.e., it is not relatedto a static object.

In order to find the correct association between playertracks, a similarity transformation S is generated for everycandidate match and verified for all remaining candidates.The transformation leading to the largest number of inliersis assumed to be correct and the inliers represent matchingtrajectories. For a trajectory let i denote the position atframe i. Let S(i) denote the result of applying transforma-tion S to point i. A candidate match (, ) is consideredan inlier of transformation S, if S(i) i2 < tdist for alloverlapping frames i.

However, depending on the input footage, the accuracy

of the self-calibration varies and the metric rectification canbe inaccurate. Therefore, in some cases, trajectories are notaligned accurately enough to correctly determine the inliers.In order to compensate for such inaccuracies, we determinea homography from two candidate matches. A homographycan explain any projective transformation between planes.Instead of testing all possible combinations of two candi-date matches, the similarity transformations obtained be-fore are used to determine suitable combinations. When-ever a second match is compatible with the similarity trans-formation obtained from the first match, the homographyis calculated and verified. A candidate match is consideredcompatible with a transformation, if the shape and size ofthe transformed track is similar to the one of its match. Letvij() denote the vector from the position at frame i to theposition at frame j of trajectory . Formally, for a candi-date match (, ) and a transformation S, this means thatthe following two conditions need to be fulfilled:

(vbe(S()), vbe()) < tanglemax(

vbe(S())2vbe()2 ,

vbe()2vbe(S())2 ) < tlength,

where b denotes the first frame and e the last frame which and have in common. A valid set of correspondencesis a 1-to-1 matching of image trajectories. If one track ismatched to more than one other track, only the match withthe lowest mean distance between corresponding points isused. Given the matching image trajectories, the homogra-phy induced by the ground plane can be calculated for everypair of images Iai and I

bj , as explained in the previous sub-

section. To improve the calibration, additional features canbe matched using these homographies. In soccer, the linesare the obvious choice. Detected line segments are matchedif they are close enough but not too close to other segments.As for the self-calibration, image line samples are allowedto slide along the line on the ground plane. This time, a lineon the ground plane is the same for all cameras and is givenin the common world coordinate frame.

In a final bundle adjustment, all correspondences avail-able are used to determine the full calibration of the cameranetwork. This includes the camera intrinsics and extrinsics,one distortion coefficient per camera and image, SIFT andKLT feature positions extracted in the self-calibration stage,world positions of feet and heads as well as world positionsof lines on the ground plane. The errors to be minimizedare the reprojection errors and the deviations of the anglesbetween lines from the known angles.

6. EvaluationThe proposed algorithm was evaluated on four different

soccer game sequences. Each sequence was captured withtwo synchronized SD cameras working at a resolution of

29

Figure 4. Foot tracks extracted from a soccer sequence.

720x576 pixels and undergoing a large range of rotationsand zooms. The length of the sequences varies between 10and 17 seconds, i.e., between 250 and 425 frames. The ob-tained results were evaluated both quantitatively and quali-tatively.

Quantitative comparison In order to generate convinc-ing synthetic views, a consistent calibration with respect tothe different PTZ cameras is very important. To evaluatethis property numerically, we used the root mean square ofthe symmetric epipolar transfer distances defined as the dis-tance between a point in the first image and the epipolarline of its corresponding point in the second image and viceversa. More precisely, we evaluated

eepipolar =

12N

Ni=1

(d(xi,Fxi)2 + d(xi,FTxi)2),

where F denotes the fundamental matrix, and d(xi,Fxi) de-notes the epipolar transfer distance defined as the Euclideandistance between point xi and line Fxi. Salient correspond-ing points were selected manually over the course of thewhole sequence. This includes heads, feet and joints ofplayers, intersections of field lines, the corners of the goalsand banners at the sidelines.

The symmetric epipolar transfer distance was evaluatedon all the tested sequences. The calibration obtained by ourmethod, once ignoring image lines and once accounting forimage lines, was compared with a calibration obtained bymanually selecting field model points. More precisely, thislatter calibration was obtained by hand-clicking between 4and 15 model points of the playing field every tenth frame.This includes all the corners and intersections formed byfield lines, as well as the penalty points. Given the selected3D-2D correspondences, the pan and tilt angles, the focallengths and the COP were determined for each camera sep-arately by minimizing the reprojection errors. Due to thelimited number of hand-clicked points, radial distortion wasnot taken into account in the manual calibration.

Table 1 summarizes the root mean square of the symmet-ric epipolar transfer distances obtained in the different se-quences for both the manual calibration and our approach.Additionally, the table reports the median of all the com-puted epipolar transfer distances d(xi,Fxi). The secondcolumn indicates the number of hand-clicked correspon-dences used to calculate eepipolar for the manual calibration.Adding correspondences between field lines improved theresults and in three out of four sequences, our approachleads to a smaller eepipolar since it takes both cameras intoaccount for the estimation. In sequence 4, the error obtainedby our approach is a bit higher than the one obtained usingmanual calibration. This is due to the fact that in this se-quence only few players are moving and the tracks of neigh-boring players got confused.

Figure 6 shows the distribution of the epipolar transferdistances d(xi,Fxi) for all the points and sequences. It isvisible that by using our method the distances concentratemore around zero. An additional comparison is shown inFigure 7. Here, tilt angles and focal lengths obtained aftersingle camera calibration (Section 4), network calibration(Section 5), and by manual calibration for both cameras ofsequence 1 are illustrated. The values for the manual cali-bration are not available every tenth frame because in someparts of the sequence not enough model points were visi-ble. The shapes of the curves are very similar, suggestingthat the estimations of the relative changes between framesare very similar. The additional constraints added by opti-mizing for two cameras simultaneously lead to a shift of thecurves estimated from a single camera towards the ones es-timated using manual calibration. Rotation angles and focallengths obtained from single rotating cameras are correlatedand, to some extent, increased focal lengths can be compen-sated by decreasing the rotation angles and vice versa [1].

Qualitative comparison Using the homography in-duced by the ground plane, the input image of one camerawas warped and blended with the corresponding input im-age of the other camera. A result obtained accounting forlines and a result obtained without using any lines are shownin Figure 5. Non-overlapping lines and players cause visibleartifacts when generating novel synthetic views using view-dependent textures. Hence it makes sense to minimize sucherrors.

7. ConclusionIn this paper, we presented an approach to multi-view

PTZ camera calibration for sports broadcasts. Each cam-era is first calibrated in its own local coordinate systemand then the player trajectories are leveraged to establisha common world coordinate system. Matching field linesfurther improves the calibration. Differently from previousapproaches, the proposed calibration method does not relyon a 3D line model of the playing field or on detectable fea-

30

(a) (b)

Figure 5. Blend of two images aquired by two cameras observing the same scene. The first image was warped according to the homographyinduced by the ground plane and superimposed to the second one. Figure (a) shows the result obtained ignoring lines. Figure (b) shows theresult obtained accounting for lines.

manual calibration our approach, no lines our approach, incl. lines#points RMS error [px] median [px] RMS error [px] median [px] RMS error [px] median [px]

sequence 1 108 2.9979 1.7343 2.8901 1.9629 1.6159 0.8144sequence 2 147 6.2398 1.8590 2.6000 1.5034 1.8539 1.0943sequence 3 55 2.8181 2.1985 4.8823 3.9737 1.3922 0.7909sequence 4 82 2.5651 1.4776 8.5352 6.4020 4.5386 3.2121

Table 1. Epipolar transfer distance computed for the tested sequences.

tures on the field. This makes our method more robust inthe absence of visible field lines and textured regions. Theresults presented in this papers show that the proposed tech-nique can handle challenging sequences recorded by PTZcameras separated by very wide baselines.

Limitations and Future Work Problems can be en-countered in sequences where a camera constantly exhibitslarge focal lengths because the projection becomes almostorthographic and no perspective information can be in-ferred. In this case, self-calibration is difficult and typi-cally inaccurate. However, as seen in Figure 7, if the camerazooms out at some point, the calibration can still be recov-ered.

In zoomed-in cameras, due to the narrow field-of-view,only a few players are visible, which means that there areonly a few trajectories and hence trajectory matching be-comes more difficult. Since a zoomed-in camera followingthe action needs to move fast, motion blur leads to inac-curate tracking results which, again, complicates trajectorymatching. We would like to address these issues to be ableto deal with constantly zoomed-in cameras and blurry sub-sequences.

If only a few of the trajectories extracted in the differentcameras stem from common objects, the number of inliersobtained using the correct matching can be lower than thenumber of inliers obtained by a wrong matching. Additional

cues like player colors can help in reducing the number ofoutliers.

8. AcknowledgmentsThe data is courtesy of Teleclub and LiberoVision. This

project is supported by a grant of CTI Switzerland, the4DVideo ERC Starting Grant Nr. 210806 and the SNFRecording Studio Grant.

References[1] L. Agapito, E. Hayman, and I. Reid. Self-calibration of ro-

tating and zooming cameras. Int. J. Comput. Vision, 2001.[2] J. Bilmes. A gentle tutorial of the em algorithm and its ap-

plication to parameter estimation for gaussian mixture andhidden markov models. Technical report, 1998.

[3] B. Bose and E. Grimson. Ground plane rectification by track-ing moving objects. In IEEE International Workshop on Vi-sual Surveillance and PETS, 2004.

[4] M. Brown, R. Hartley, and D. Nister. Minimal solutions forpanoramic stitching. In Proc. CVPR, 2007.

[5] J. Canny. A computational approach to edge detection. IEEETrans. PAMI, 1986.

[6] T. Chen, A. Del Bimbo, F. Pernici, and G. Serra. Accurateself-calibration of two cameras by observations of a movingperson on a ground plane. In IEEE Conference on AdvancedVideo and Signal Based Surveillance (AVSS)., 2007.

31

0 10 20 30 400

50

100

150

200

epipolar transfer distance [px]0 10 20 30 40

0

20

40

60

80

100

120

140

160

180

epipolar transfer distance [px]0 10 20 30 40

0

50

100

150

200

250

300

epipolar transfer distance [px]

(a) (b) (c)

Figure 6. Distribution of the epipolar transfer distances for all sequences obtained by manual calibration (a), automatic calibration notaccounting for lines (b) and automatic calibration including lines (c).

0 100 200 300 40076

77

78

79

80

81

frame number

tilt a

ng

le [

de

gre

e]

0 100 200 300 400

78

80

82

84

frame number

tilt a

ng

le [

de

gre

e]

0 100 200 300 400

103.1

103.2

103.3

frame number

foca

l le

ng

th [

px]

0 100 200 300 40010

3

frame number

foca

l le

ng

th [

px]

manual

single camera

multiview

104

(a) (b) (c) (d)

Figure 7. (a) and (b): tilt angles computed in each frame of sequence 1 for both cameras. (c) and (d): focal lengths computed in each frameof sequence 1 for the same cameras, shown at a logarithmic scale.

[7] Y. Chen, T. A. Davis, W. W. Hager, and S. Rajamanickam.Algorithm 887: Cholmod, supernodal sparse cholesky fac-torization and update/downdate. ACM Trans. Math. Softw.,2008.

[8] D. Farin, S. Krabbe, W. Effelsberg, and P. H. N. de With. Ro-bust camera calibration for sport videos using court models.In SPIE Storage and Retrieval Methods and Applications forMultimedia, 2004.

[9] M. Fischler and R. Bolles. Random sample consensus: Aparadigm for model fitting with applications to image analy-sis and automated cartography. Commun. ACM, 1981.

[10] F. Fraundorfer, P. Tanskanen, and M. Pollefeys. A minimalcase solution to the calibrated relative pose problem for thecase of two known orientation angles. In Proc. ECCV, 2010.

[11] R. I. Hartley. Self-calibration of stationary cameras. Int. J.Comput. Vision, 1997.

[12] R. I. Hartley and A. Zisserman. Multiple View Geometry inComputer Vision. Cambridge University Press, 2000.

[13] C. Jaynes. Multi-view calibration from planar motion forvideo surveillance. In Second IEEE Workshop on VisualSurveillance (VS99), 1999.

[14] N. Krahnstoever and P. Mendonca. Bayesian autocalibrationfor surveillance. In Proc. ICCV, 2005.

[15] M. I. Lourakis. Sparse non-linear least squares optimizationfor geometric vision. In Proc. ECCV, 2010.

[16] D. Lowe. Distinctive image features from scale-invariantkeypoints. Int. J. Comput. Vision, 2004.

[17] B. D. Lucas and T. Kanade. An iterative image registrationtechnique with an application to stereo vision. In IJCAI81:Proceedings of the 7th International Joint Conference on Ar-tificial Intelligence, 1981.

[18] F. Lv, T. Zhao, and R. Nevatia. Camera calibration fromvideo of a walking human. PAMI, 2006.

[19] M. Meingast, S. Oh, and S. Sastry. Automatic camera net-work localization using object image tracks. In Proc. ICCV,2007.

[20] J. Puwein, R. Ziegler, J. Vogel, and M. Pollefeys. Robustmulti-view camera calibration for wide-baseline camera net-works. In IEEE Workshop on Applications of Computer Vi-sion (WACV), 2011.

[21] J. Shi and C. Tomasi. Good features to track. In Proc. CVPR,1994.

[22] G. P. Stein. Tracking from multiple view points: Self-calibration of space and time. Proc. CVPR, 1999.

[23] G. Thomas. Real-time camera tracking using sports pitchmarkings. Journal of Real-Time Image Processing, 2007.

[24] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon.Bundle adjustment: A modern synthesis. In ICCV 99:Proceedings of the International Workshop on Vision Algo-rithms, 2000.

32