# Parameter estimation class 6 Multiple View Geometry Comp 290-089 Marc Pollefeys

• View
215

• Category

## Documents

Embed Size (px)

### Text of Parameter estimation class 6 Multiple View Geometry Comp 290-089 Marc Pollefeys

• Parameter estimationclass 6Multiple View GeometryComp 290-089Marc Pollefeys

• ContentBackground: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation.Single View: Camera model, Calibration, Single View Geometry.Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies.Three Views: Trifocal Tensor, Computing T.More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality

• Multiple View Geometry course schedule(subject to change)

• Parameter estimation2D homographyGiven a set of (xi,xi), compute H (xi=Hxi)3D to 2D camera projectionGiven a set of (Xi,xi), compute P (xi=PXi)Fundamental matrixGiven a set of (xi,xi), compute F (xiTFxi=0)Trifocal tensor Given a set of (xi,xi,xi), compute T

• DLT algorithmObjectiveGiven n4 2D to 2D point correspondences {xixi}, determine the 2D homography matrix H such that xi=HxiAlgorithmFor each correspondence xi xi compute Ai. Usually only two first rows needed.Assemble n 2x9 matrices Ai into a single 2nx9 matrix AObtain SVD of A. Solution for h is last column of VDetermine H from h

• Geometric distancee.g. calibration patternd(.,.) Euclidean distance (in image)Reprojection error

• Geometric interpretation of reprojection errorEstimating homography~fit surface to points X=(x,y,x,y)T in 4

• Statistical cost function and Maximum Likelihood EstimationOptimal cost function related to noise modelAssume zero-mean isotropic Gaussian noise (assume outliers removed)

Error in one image

• Statistical cost function and Maximum Likelihood EstimationOptimal cost function related to noise modelAssume zero-mean isotropic Gaussian noise (assume outliers removed)

Error in both images

• Mahalanobis distanceGeneral Gaussian caseMeasurement X with covariance matrix

• Invariance to transforms ?will result change? for which algorithms? for which transformations?

• Non-invariance of DLTGiven and H computed by DLT, and

Does the DLT algorithm applied to yield ?

• Effect of change of coordinates on algebraic errorso

• Non-invariance of DLTGiven and H computed by DLT, and

Does the DLT algorithm applied to yield ?

• Invariance of geometric errorGiven and H, andAssume T is a similarity transformations

• Normalizing transformationsSince DLT is not invariant,what is a good choice of coordinates?e.g.Translate centroid to originScale to a average distance to the originIndependently on both images

• Importance of normalization~102~102~102~102~104~104~10211orders of magnitude difference!

• Normalized DLT algorithmObjectiveGiven n4 2D to 2D point correspondences {xixi}, determine the 2D homography matrix H such that xi=HxiAlgorithmNormalize points Apply DLT algorithm to Denormalize solution

• Iterative minimization metodsRequired to minimize geometric error Often slower than DLTRequire initializationNo guaranteed convergence, local minimaStopping criterion requiredTherefore, careful implementation required:Cost functionParameterization (minimal or not)Cost function ( parameters )InitializationIterations

• ParameterizationParameters should cover complete space and allow efficient estimation of cost

Minimal or over-parameterized? e.g. 8 or 9 (minimal often more complex, also cost surface) (good algorithms can deal with over-parameterization)(sometimes also local parameterization)Parametrization can also be used to restrict transformation to particular class, e.g. affine

• Function specificationsMeasurement vector XN with covariance Set of parameters represented by vector P NMapping f : M N. Range of mapping is surface S representing allowable measurementsCost function: squared Mahalanobis distance

Goal is to achieve , or get as close as possible in terms of Mahalanobis distance

• Error in one imageReprojection error

• InitializationTypically, use linear solutionIf outliers, use robust algorithm

Alternative, sample parameter space

• Iteration methodsMany algorithms existNewtons methodLevenberg-Marquardt

Powells methodSimplex method

• Gold Standard algorithmObjectiveGiven n4 2D to 2D point correspondences {xixi}, determine the Maximum Likelyhood Estimation of H(this also implies computing optimal xi=Hxi)AlgorithmInitialization: compute an initial estimate using normalized DLT or RANSAC Geometric minimization of -Either Sampson error: Minimize the Sampson error Minimize using Levenberg-Marquardt over 9 entries of hor Gold Standard error: compute initial estimate for optimal {xi} minimize cost over {H,x1,x2,,xn} if many points, use sparse method

• Robust estimationWhat if set of matches contains gross outliers?

• RANSACObjectiveRobust fit of model to data set S which contains outliersAlgorithmRandomly select a sample of s data points from S and instantiate the model from this subset.Determine the set of data points Si which are within a distance threshold t of the model. The set Si is the consensus set of samples and defines the inliers of S.If the subset of Si is greater than some threshold T, re-estimate the model using all the points in Si and terminateIf the size of Si is less than T, select a new subset and repeat the above.After N trials the largest consensus set Si is selected, and the model is re-estimated using all the points in the subset Si

• Distance thresholdChoose t so probability for inlier is (e.g. 0.95) Often empiricallyZero-mean Gaussian noise then follows distribution with m=codimension of model (dimension+codimension=dimension space)

CodimensionModelt 21l,F3.8422H,P5.9923T7.812

• How many samples?Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99

proportion of outliers es5%10%20%25%30%40%50%22356711173347911193543591317347254612172657146647162437972937482033541635888592644782721177

• Acceptable consensus set?Typically, terminate when inlier ratio reaches expected ratio of inliers

• Adaptively determining the number of samplese is often unknown a priori, so pick worst case, e.g. 50%, and adapt if more inliers are found, e.g. 80% would yield e=0.2

N=, sample_count =0While N >sample_count repeatChoose a sample and count the number of inliersSet e=1-(number of inliers)/(total number of points)Recompute N from eIncrement the sample_count by 1Terminate

• Robust Maximum Likelyhood EstimationPrevious MLE algorithm considers fixed set of inliers

Better, robust cost function (reclassifies)

• Other robust algorithmsRANSAC maximizes number of inliersLMedS minimizes median error

Not recommended: case deletion, iterative least-squares, etc.

• Automatic computation of HObjectiveCompute homography between two imagesAlgorithmInterest points: Compute interest points in each imagePutative correspondences: Compute a set of interest point matches based on some similarity measureRANSAC robust estimation: Repeat for N samples(a) Select 4 correspondences and compute H(b) Calculate the distance d for each putative match(c) Compute the number of inliers consistent with H (d
• Determine putative correspondencesCompare interest pointsSimilarity measure:SAD, SSD, ZNCC on small neighborhood

If motion is limited, only consider interest points with similar coordinates

More advanced approaches exist, based on invariance

• Example: robust computationInterest points(500/image)

Putative correspondences (268)

Outliers (117)Inliers (151)

Final inliers (262)

• AssignmentTake two or more photographs taken from a single viewpointCompute panoramaUse different measures DLT, MLE

Use Matlab Due Feb. 13

• Next class: Algorithm evaluation and error analysisBounds on performanceCovariance propagationMonte Carlo covariance estimation

Nonlinear, except for line fitting = affine homographies (no quadratic terms in this case!) (x,x,y and x,y,y) => linear solution

Recommended ##### Camera Calibration class 9 Multiple View Geometry Comp 290-089 Marc Pollefeys
Documents ##### Parameter estimation class 5 Multiple View Geometry Comp 290-089 Marc Pollefeys
Documents ##### More on single-view geometry class 10 Multiple View Geometry Comp 290-089 Marc Pollefeys
Documents #####  -Linearities and Multiple View Tensors Class 19 Multiple View Geometry Comp 290-089 Marc Pollefeys
Documents Documents