Flexible Camera Calibration Flexible Camera Calibration by Viewing a Plane from by Viewing a Plane from Unknown OrientationsUnknown Orientations

Zhengyou ZhangZhengyou Zhang

Vision Technology GroupVision Technology Group

Microsoft ResearchMicrosoft Research

Problem Statement

Determine the characteristics of a camera Determine the characteristics of a camera (focal length, aspect ratio, principal point) (focal length, aspect ratio, principal point) from visual information (images)from visual information (images)

Motivations

Recovery of 3D Euclidean structure from Recovery of 3D Euclidean structure from images is essential for many applications.images is essential for many applications.

This requires camera calibration.This requires camera calibration. Look for a Look for a flexibleflexible and and robustrobust technique, technique,

suitable for desktop vision systems.suitable for desktop vision systems.(such that it can be used by the general public)(such that it can be used by the general public)

Classical ApproachClassical Approach(Photogrammetry)(Photogrammetry)

Use precisely known Use precisely known 3D3D points points

Shortcomings:Shortcomings: Not flexibleNot flexible– very expensive to make such a calibration very expensive to make such a calibration

apparatus.apparatus.

Known displacement

Known displacement

Futuristic ApproachFuturistic Approach(Self-calibration)(Self-calibration)

Shortcoming:Shortcoming: Not always reliableNot always reliable– too many parameters to estimatetoo many parameters to estimate

Move the camera in a static environmentMove the camera in a static environment– match feature points across images match feature points across images – make use of rigidity constraintmake use of rigidity constraint

Realistic Approach(my new method)

Use only one planeUse only one plane– Print a pattern on a paperPrint a pattern on a paper– Attach the paper on a planar surfaceAttach the paper on a planar surface– Show the plane Show the plane freelyfreely a few times to the camera a few times to the camera

Advantages:Advantages:– Flexible!Flexible!– Robust?Robust? Yes.Yes. See RESULTS See RESULTS

Camera Model

M

tR

Am

trrr

~

3210

0

~ 1100

0

1

z

y

x

v

u

v

u

s

CC

),( 00 vu

zyx

M

mm

),( tR

y

xM

0z

CC

uv

),( 00 vu

Plane projection

The relation between image points and The relation between image points and model points is then given by:model points is then given by:

MHm ~~ s withwith trrAH 21

mm

For convenience, assume the plane at For convenience, assume the plane at z = z = 00..

What do we get from one image? We can obtain two equations in 6 We can obtain two equations in 6

intermediate homogeneous parameters.intermediate homogeneous parameters.

trrAhhh 21321

Given Given HH, which is defined up to a scale factor,, which is defined up to a scale factor, 321 hhhH And letAnd let , we have, we have

This yieldsThis yields

021

1

21

211

1

hAAh

hAAhhAAh

TT

TTTT

Geometric interpretation

r3

01r

02r

Plane at infinity

021 rr i

a

)( 21 hh ia

Absolute conic 0xxT

01

mAAm TT

C0)()( 21

121 hhAAhh T ii T

Linear Equations

LetLet

DefineDefine

up to a scale factorup to a scale factor RewriteRewrite

as linear equations: as linear equations:

333231

232221

1312111

BBB

BBB

BBBT AAB

021

1

21

211

1

hAAh

hAAhhAAh

TT

TTTT

332313221211 BBBBBBb

0Mb

symmetric

What do we get from 2 images?

If we impose If we impose = = 0, which is usually the 0, which is usually the case with modern cameras, we can solve all case with modern cameras, we can solve all the other camera intrinsic parameters.the other camera intrinsic parameters.

How about more images?Better!Better! More constraints than unknowns. More constraints than unknowns.

Solution Show the plane under Show the plane under nn different orientations ( different orientations (nn > >

1) 1) Estimate the Estimate the n n homography matriceshomography matrices

((analytic solution followed by MLEanalytic solution followed by MLE))

Solve analytically the 6 intermediate parameters Solve analytically the 6 intermediate parameters ((defined up to a scale factordefined up to a scale factor))

Extract the five intrinsic parametersExtract the five intrinsic parameters Compute the extrinsic parametersCompute the extrinsic parameters Refine all parameters with MLERefine all parameters with MLE

Experimental results

Extracted corner points

Result (1)

Result (2)

Correction of Radial Distortion

Corrected imageOriginal image

Errors vs. Noise Levels in data

Errors vs. Number of Planes

Errors vs. Angle of the plane

Errors vs. Noise in model points

Errors vs. Spherical non-planarity

Errors vs. Cylindrical non-planarity

Application to object modeling

Reconstructed VRML Model

Conclusion We have developed a flexible and robust We have developed a flexible and robust

technique for camera calibration.technique for camera calibration. Analytical solution exists.Analytical solution exists. MLE improves the analytical solution.MLE improves the analytical solution. We need at least two images if c = 0.We need at least two images if c = 0. We can use as many images of the plane as We can use as many images of the plane as

possible to improve the accuracy.possible to improve the accuracy.

It really works!

Currently used routinely in both Vision and Currently used routinely in both Vision and Graphics Groups.Graphics Groups.

Binary executable will be distributed on the Binary executable will be distributed on the Web to the public soon.Web to the public soon.

Source code will also be made available.Source code will also be made available.