IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 1

Interacting Geometric Priors For RobustMulti-Model Fitting

Trung T. Pham, Student Member, IEEE, Tat-Jun Chin, Member, IEEE,Konrad Schindler, Senior Member, IEEE and David Suter

Abstract—Recent works on multi-model fitting are often for-mulated as an energy minimization task, where the energyfunction includes fitting error and regularization terms, such aslow-level spatial smoothness and model complexity. In this paper,we introduce a novel energy with high-level geometric priors thatconsider interactions between geometric models, such that certainpreferred model configurations may be induced. We argue that inmany applications, such prior geometric properties are availableand should be fruitfully exploited. For example, in surface fittingto point clouds, the building walls are usually either orthogonalor parallel to each other. Our proposed energy function isuseful in dealing with unknown distributions of data errors andoutliers, which are often the factors leading to biased estimation.Furthermore, the energy can be efficiently minimized by usingthe expansion move method. We evaluate the performance onseveral vision applications using real datasets. Experimentalresults show that our method outperforms the state-of-the-artmethods without significant increase in computation.

Index Terms—Robust Statistics, Multi-Model Fitting, ModelSelection, Global Constraints, Geometric priors.

I. INTRODUCTION

THIS paper addresses the challenging problem of fittingmultiple models onto noisy and outlier-prone data. Such a

capability underpins computer vision applications such as two-view multi-homography estimation, vanishing point detection,and motion segmentation from feature tracks. Given the data,the task is typically accomplished by first sampling a largenumber of hypotheses, then selecting a small subset of thehypotheses that best explains the data.

Depending on the complexity of the data and the model ofinterest, numerous strategies have been proposed for modelselection. For example, if the data contains a single structure,RANSAC [1] selects the putative model with the largest num-ber of supporting inliers. When there are a variable numberof structures, one can either apply RANSAC sequentially orresort to global optimisation techniques [2]–[5] to select allthe models jointly. The latter strategies have been shown tooutperform sequential RANSAC.

Typically, the optimisation based methods find the optimalfitting by minimising an energy function, which includes fittingerror and regularization terms. The most popular regularization

T.T. Pham, T.-J. Chin and D. Suter are with the Australian Centrefor Visual Technologies (ACVT), and School of Computer Science, TheUniversity of Adelaide, South Australia. E-mail: {trung.pham, tat-jun.chin,david.suter}@adelaide.edu.au

K. Schindler is with the Photogrammetry and Remote Sensing Group,ETH Zurich, Wolfgang-Pauli-Str. 15, 8093 Zurich, Switzerland. E-mail:konrad.schindler@geod.baug.ethz.ch

(a) Plannar surface detection. (b) Vanishing point detection.

Fig. 1. Some examples of computer vision applications where geometric pri-ors should be effectively exploited to produce plausible results. (a) Detectingplanar homographies using two views of an indoor scene (the first view isshown). In the indoor environments, the planar surfaces (e.g., ground floors,walls) are usually either parallel or orthogonal to each other. (b) Estimatingvanishing points in a single image. In a Manhattan world [6], the threevanishing directions (coloured in red, green and blue) are mutually orthogonal.

is model complexity, used to avoid over-fitting, i.e., favour-ing the solutions with fewer structures. Recent work [5],[7] popularize the idea of encouraging spatial smoothness,i.e., neighbouring points should belong to the same model.Although such priors improve fitting accuracy, recovering theunknown model parameters from data remains a formidableproblem, especially when the data is largely contaminated bynon-normal noise and non-uniform outliers.

In this paper, we show that incorporating higher levelpriors, which consider interactions between geometric models,can further improve fitting accuracy and yield more plausi-ble results. Such high-level priors capture knowledge (e.g.,geometric shape) of the scenes or objects. For example, indetecting planes from indoor images (see Fig. 1(a)), the roomwalls are usually either orthogonal or parallel to each other. Ina vanishing point detection problem (see Fig. 1(b)), the threedominant vanishing directions are mutually orthogonal underthe Manhattan world assumption [6].

Fundamentally, using high-level geometric priors helpscombat against undesirable effects from assuming an unre-alistic noise and outliers models, e.g., i.i.d. normal noise forthe inliers, uniform distribution for outliers. Such assumptions,typically encoded in the goodness-of-fit term, may break downunpredictably. These issues often happen in practice, and couldbe minimised with geometric priors.

To achieve our goals, we construct an energy function thatencapsulates goodness-of-fit, spatial smoothness, model com-plexity, and adherence to the high-level geometric priors. (Thespatial regularization can be omitted when the smoothnessassumption is not valid.) Although such an energy function

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 2

formulation has been acknowledged in vision (e.g., in imagesegmentation [8]), it has not been proposed previously ingeometric model fitting problems in computer vision. Tooptimise our energy, we first approximate the continuousmodel parameter space by a discrete set of model hypotheses,then resort to graph-cut based inference technique [8] to findthe optimal fitting.

In summary, our main contributions in this paper are:• We propose a general-purpose energy-based multi-model

fitting framework which seamlessly integrates geometricconstraints into model selection/evaluation. A specificimplementation of our fitting framework requires vari-ous components that can be varied depending on theapplications. The hypothesis generation, for example, canbe constructed by a RANSAC-style random samplingor a guided sampling technique Muli-GS [9] for moreefficiency. The geometric compatibilities between modelscan be easily customised for different geometric models.

• We show that high-level geometric knowledge is a power-ful clue that can effectively alleviate the effects of noiseand outliers. Consequently, the assumptions of normalnoise and uniform outliers can be relaxed.

• We implement several important computer vision appli-cations including multi-homography estimation, surfacereconstruction from 3D point clouds, circular object de-tection and vanishing point detection; to demonstrate thesignificant practical improvements enabled by our fittingapproach over the state-of-the-art methods.

The rest of the paper is organized as follows. We reviewrelated work in the next section. In Sec. III we describe ourfitting framework including the energy formulation, energynormalisation, and the optimisation method. The practicaladvantages of our geometric model fitting framework aredemonstrated in Sec. IV. We draw conclusions in Sec. V.

II. PREVIOUS WORK

Given a set of N data points X = {xn}Nn=1 and a class ofmodelsM (e.g., homography matrices), the task of geometricfitting is to estimate a set of model parameters Θ = {θk}Kk=1

that best explain the data, where θk denotes the parameters ofan instance of the modelM. Depending on the complexity ofΘ and M, a wide range of methods have been proposed.

For the data with a single structure (e.g., K = 1), the mostwell-known solution is Least Square (LS). The LS methodminimises the sum of squares of the fitting errors. AlthoughLS has a closed-form solution, LS is not robust to outliers.Instead, RANSAC [1] maximises the number of inliers (e.g.,points that “belong” to the model within a certain threshold).Despite being robust to outliers, RANSAC is very sensitive tothe choices of inlier thresholds.

In the multiple-structure case, fitting is more challeng-ing. One can sequentially apply a particular single-structuremethod (e.g., RANSAC) and remove the inliers until nomore structures are found. Such a strategy is suboptimal [10]because the inaccuracies in estimating the first structures willheavily affect the estimates of the remaining structures. Othermethods such as the Hough transform (HT) and [11] detect

either peaks or clusters in the sampled parameter space, whereeach peak/cluster indicates a model instance. Alternatives [12],[13] cluster the data into distinct groups, then fit the modelparameters into each group separately using standard fittingmethods (e.g., LS). Obviously, the imperfection of the cluster-ing step strongly affects the fitting accuracy. Moreover, thesemethods are unable to impose prior information into the fitting.

Another class of methods for multi-model fitting followan energy minimisation approach. In [14], Torr proposes anenergy function including fitting error and model complexity,which is optimised using a greedy searching method. The fit-ting error term typically measures the goodness-of-fit while themodel complexity term accounts for the number of structures.Intuitively, among the solutions with equal fitting errors theone with fewer structures is preferred. This approach is furtherextended by using more advanced optimisation tools such asMCMC [15], graph-cuts [4], quadratic programming [3] toimprove the optimality and efficiency. Although these methodshave been tested successfully in some datasets, they havedifficulty in cases where the structures are imbalanced andnot well separated. The methods are also not reliable whenthe data is contaminated by non-normal noise and non-uniformoutliers.

Prior information is usually helpful to improve the per-formance of many vision applications, including geometricmodel fitting. In [4] the authors introduce an energy-basedfitting method, namely PEARL, in which the energy containsa spatial smoothness prior beside the standard fitting error andmodel complexity. Such a smoothness prior has proved toincrease the fitting accuracy. Unfortunately, this “low-level”constraint is still insufficient to deal with difficult data (e.g.,noisy data which contains “clustered” outliers). Our workimproves upon PEARL to enable “high-level” geometric priorsby introducing interactions between models into the energyfunction.

There have been various works that consider high-level con-straints between models for multi-model fitting. The majorityof these fall under the stochastic geometry framework [16]–[18], where the distribution of objects in a spatial coordinatespace is modelled. Given a set of measurements, the mostlikely object configuration under the posterior is inferred. Aprior is introduced to encourage favoured object configura-tions. This framework is, however, unable to handle moregeneral “objects” not lying in a spatial space such as motions,affine transformations. Moreover, the framework has not con-sidered the low-level pairwise potentials over the data points.Therefore, the methods cannot simultaneously consider (low-level) spatial smoothness and (high-level) geometric priors,unlike ours. Other application-specific multi-model fitting (us-ing geometric priors) include RGB-D image segmentation [19]and vanishing point detection [20]–[22].

Recently, Olsson and Boykov [23] have introduced the cur-vature based regularization for fitting surfaces onto 3D pointclouds. Their method assigns a tangent plane to each noisydata point such that the estimated tangent planes conforma smooth surface. To impose the smoothness constraint, themethod models interactions between tangent planes at pairwiseneighbouring points. Though their simple energy function can

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 3

be approximately minimised using combinatorial optimisa-tion techniques (e.g., TRW-S [24]), making the inter-modelrelations depend on the spatial neighbourhood system limitstheir potential application in the general geometric fitting. Incontrast, the interplays between models in our formulationare independent of the underlying neighbourhood structure.Moreover, their method does not consider model selection (i.e.,estimate the quantity of models) as each point is assigned aunique label. On the other hand, our fitting method only returnsa small number of labels conforming with the geometric priors.

Though with a different application and emphasis, the workmost related to ours is [8]. In [8], the problem of semantic im-age segmentation is modelled using CRF with co-occurrencestatistics. Typically, the co-occurrence terms can help preventimplausible combinations of semantic labels (e.g, cow, lion)appearing in a single image. We extend the co-occurrence idea(and their inference algorithm) to a general geometric multi-model fitting problem, where consistent geometric modelsshould be recovered from the data. Unlike the image segmen-tation task where the co-occurrence potentials possibly helpovercome the weaknesses of the object detectors/classifiers,our geometric consistency is specifically used to alleviate theinfluences of noise and outliers in the data. (Noise and outliersare the main roots of biased parameter estimation.) Moreover,the inference method in [8] is for a discrete set of labels,to deal with the continuous geometric model parameter, wediscretize the parameter space by randomly sampling a largecollection of model hypotheses. Accordingly, the geometricconsistency between models are computed directly from themodel parameters rather than learning from the training data,as in [8].

III. MULTI-MODEL DATA FITTING

Following [3]–[5], we formulate the multi-model fitting asan energy minimisation task, i.e.,

Θ∗ = argminΘ⊂P

E(Θ), (1)

where E is some energy function, and P is a continuousspace of parameters (e.g., plane parameters). A commonsimplification is to generate a large set of model hypothesesH = {θm}Mm=1, which samples the parameter space P . Thetask then reduces to finding the best subset Θ ⊆ H thatminimises the energy E(Θ). Differing from the previous meth-ods, in our work the optimal Θ should include geometricallyconsistent structures. Fig. 2 provides the conceptual illustrationof our fitting framework.

For generating the hypothesis set H, there are variousstrategies, e.g., a hypothesis can be generated by fitting themodel onto a randomly sampled subset of data. In our fittingframework, the hypothesis sampling step varies with applica-tions. We will demonstrate different sampling techniques forparticular applications in Sec. IV.

Next, we derive our fitting energy formulation, which jointlyconsiders geometric fitting error, (optional) low-level spatialsmoothness priors, model complexity and high-level geomet-ric constraints. The energy approximation algorithm adoptedfrom [8] is also described in detail.

Energy minimization

Energy function

Geometric priors

Model selectionHypothesis samplingInput data Output models

Geometric multi-model fitting

Fig. 2. A conceptual diagram of the proposed multi-model fitting framework.Given the input data, our fitting framework returns a set of geometricallyconsistent models.

A. Energy with low-level spatial smoothness priorsThe spatial smoothness priors are often imposed at the point

level, e.g., neighboring points should be assigned the samelabels. In the context of geometric fitting, labels are putativegeometric objects (e.g., lines, circles, homographies). Given aset of labels H, define a labelling f , where for each xn ∈ X ,fn assigns xn to model θ ∈ H if fn = θ, or designates xnas an outlier if fn = null — note that to ease exposition wemake no distinction between a model θ in H, and its indexwhich identifies it within H. A particular labeling f then hasa corresponding set of fitted models

Θ(f) = {θ : θ ∈ H,∃xn ∈ X s.t. fn = θ}, (2)

and we can achieve multi-model fitting by performing infer-ence over f . To this end, Isack and Boykov [4] introduced thefollowing energy function for geometric model fitting

E(f) =N∑n=1

Dxn(fn) + w1

∑{p,q}∈N

V (fp, fq) + w2|Θ(f)|, (3)

where D and V functions respectively encode the fitting errorand spatial (ir)regularity; w1 and w2 are weighting parameters;N represents a sparse neighbourhood structure defined on X ,i.e., N contains a set of edges that connect nearby points. Forcompleteness we define the terms in (3) as follows.

The unary term Dxn(fn) evaluates the fitting error on xn

Dxn(fn) =

{r(xn,fn)2

σ2 if fn ∈ H

1 if fn = null,(4)

where r(xn, fn) gives the absolute residual of xn from themodel/label fn, σ refers to the noise level.

The pairwise term V (fp, fq) is defined over the labels ofneighboring points p, q ∈ X , where {p, q} ∈ N . We have

V (fp, fq) =

{1 if fp 6= fq

0 if fp = fq,(5)

which implements the spatial smoothness priors, since itencourages nearby points to be assigned to the same model.

The energy (3) can then be approximately minimised bythe graph-cut based algorithm [7]. Though the spatial regu-larisation is useful for geometric fitting, in many cases thesolution obtained by optimising the energy (3) might be verydifferent from the expected outcome due to the presence ofnoise and outliers in the data. Moreover, spatial smoothness isnot always a valid assumption. Thus it is crucial to considerhigh-level geometric constraints to resolve the data ambiguity.In our fitting framework, the smoothness term is optional andcan be discarded without any issues.

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 4

B. Energy with high-level geometric priors

In this work, our fitting energy has the following form:

E(f) =N∑n=1

Dxn(fn) + w1

∑{p,q}∈N

V (fp, fq) + w2G(Θ(f)), (6)

where G(Θ(f)) encodes the high-level geometric constraintson Θ(f). In many practical cases, the function G(Θ(f)) canbe defined as the sum over the subsets of models, i.e.,

G(Θ(f)) =∑S⊂Θ(f)

C(S), (7)

where S is any subset of Θ(f), and C(S) ≥ 0 measures the“compatibility” between the models in S. When |S| = 1, C(S)simply computes the model complexity.

The size of S and the compatibility measure C(S) varywith applications. For example, in detecting planar surfaces(from images) of the man-made environments (e.g., buildings),it is natural to enforce every pair of planes (|S| = 2) to beeither orthogonal or parallel. It is also possible to impose morecomplicated constraints, e.g., the three planes (|S| = 3) thatintersect at the same corner should be mutually orthogonal.Another application is estimating vanishing points in images.Under a Manhattan world assumption, the three vanishingdirections are mutually orthogonal (see Fig. 1(b)). Thereforewe could place an orthogonality constraint on every triplet(|S| = 3) of vanishing points. We also could enforce everypair (|S| = 2) of vanishing points to be orthogonal.

For simplicity and reducing the energy approximation error(see Sec. III-D), in this work, we consider at most pairwiseinteractions between the geometric models. That is

G(Θ(f)) =∑

θ∈Θ(f)

C(θ) +∑

β,γ∈Θ(f)

C(β,γ). (8)

C(θ) > 0 is a per-model cost function; C(β,γ) ≥ 0 is apair-model cost function imposing the geometric consistency.For example, C(β,γ) can be defined as:

C(β,γ) =

{0 if β ∼ γ

l if β � γ(9)

where ∼ (�) implies the consistency (inconsistency) betweentwo models, l is a positive penalty. The actual definition ofgeometric consistency depends on the models and problems.(See Sec. IV for real examples.)

C. Energy normalisation

In order for the minimum of (6) to produce plausible fittingresults, the terms in (6) should be balanced—none of the termsin (6) dominates the other ones. This can be achieved bysimply tuning the parameter w1 and w2. However manuallyselecting the best values of w1 and w2 for every single datasetis impractical and tedious.

Here we suggest a strategy to choose w1 and w2 suchthat the energy scales well with different datasets. Notice thatin (6), the first fitting error term linearly increases with N ,and the second pairwise smoothness cost linearly grows withthe number of edges in N , which is only a few times bigger

than N (assuming sparsely connected graphs). Thus, they areapproximately in the same scale. Our empirical study showsthat a value of w1 within a range [0.1, 0.5] would work well.

Selecting a good value of w2 is, however, more challenging.In part, it is because the model consistency term G(Θ(f))depends on the size of Θ(f), which is unknown and has to beestimated. In this work, we propose w2 = λklog(N)/Kmax,where λ > 0 is a free parameter, k is the number ofparameters to define the geometric model (e.g., k = 3 for lines,k = 4 for planes), Kmax is the maximum expected numberof structures in the data. The amount klog(N) fundamentallyhas the same functionality as classical model selection criteriasuch as BIC [25] widely used in the statistical estimation.Kmax works as a normalizing constant, which can be choseneasily depending on the specific problems. Finally λ is anactual tuning parameter. We will show exact settings of theseparameters for each particular application in Sec. IV.

D. Energy minimisation

The fitting energy (6) with only pairwise interactions be-tween geometric models can be rewritten as:

E(f) =

N∑n=1

Dxn(fn) + w1

∑{p,q}∈N

V (fp, fq)

+ w2

∑θ∈H

Cθδθ(f) +∑

β,γ∈H

Cβ,γδβ(f)δγ(f)

,(10)

where Cθ = C(θ), Cβ,γ = C(β,γ), δ(.) is an indicatorfunction, i.e.,

δθ(f) =

{1 if ∃xn : fn = θ

0 otherwise.(11)

To optimise the energy (10), we follow the extended ex-pansion move algorithm proposed in [8]. The idea of theexpansion move method is to break a multi-label problem(e.g., (10)) into a sequence of binary-label subproblems, whereeach binary energy can be solved very efficiently using graph-cuts.

More specifically, in the expansion move framework, start-ing from the current labeling f , for some label α ∈ H, themethod estimates a new labeling fα such that a label variableeither changes to α or retains its old value from f . This “ex-pansion” step is iterated for all other labels until convergence.The new labelling fα can be obtained via a transformationfunction T over binary variables t = [t1, t2, . . . , tN ], i.e.,

fαn = Tα(fn, tn) =

{α if tn = 0

fn if tn = 1.(12)

The task reduces to an inference over the binary variables t.Consequently we need to derive an expansion move energyEα(t) so that Eα(t) can be solved using graph-cuts. Werewrite the energy (10) in terms of the binary variables t

Eα(t) = Eα1 (t) + w2E

α2 (t), (13)

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 5

where Eα1 (t) encodes the unary and local pairwise potentials,

and Eα2 (t) encodes the model complexity and model pairwise

interactions in (10). The expansion move algorithm was orig-inally proposed by Boykov et al. [26] to tackle the standardunary and pairwise energy such as Eα

1 (t). We refer readersto [26] for the detailed derivation of Eα

1 (t) and the associategraph construction. Here we focus on the second term Eα

2 (t).Let Θ be a set of models corresponding to the current

labelling f , and assume that α /∈ Θ. Let us define thefollowing terms.

Xθ = {xn ∈ X |fn = θ} ∀θ ∈ Θ. (14)

δα(t) =

{1 if ∃xn ∈ X s.t. tn = 0

0 otherwise.(15)

∀θ ∈ Θ, δθ(t) =

{1 if ∃xn ∈ Xθ s.t. tn = 1

0 otherwise.(16)

Xθ is a set of points currently assigned to label θ. δα(t) (orδθ(t)) is set to 1 if model α (or θ) is present in the solutionafter the expansion move and 0 otherwise. Now the binaryenergy Eα

2 (t) can be derived as:

Eα2 (t) =

∑θ∈Θ

Cθδθ(t) +∑

β,γ∈Θ

Cβ,γδβ(t)δγ(t)

+Cαδα(t) +∑θ∈Θ

Cθ,αδθ(t)δα(t). (17)

Since either α or θ (∀θ ∈ Θ) must be present in Θ after anyexpansion move, the following statement holds

δθ(t)δα(t) = δθ(t) + δα(t)− 1. (18)

After replacing the term δθ(t)δα(t) and disregarding theconstants, the energy (17) becomes

Eα2 (t) =

∑β,γ∈Θ

Cβ,γδβ(t)δγ(t) + [Cα +∑θ∈Θ

Cθ,α]δα(t)

+∑θ∈Θ

[Cθ,α + Cθ]δθ(t). (19)

The energy (19) still can not be readily minimised using graph-cuts since (19) contains a higher-order component, i.e., the firstterm. Since Cβ,γ stays non-negative for all β and γ, the firstterm in (19) can be over-estimated as:

Cβ,γδβ(t)δγ(t) ≤ 0.5Cβ,γ [δβ(t) + δγ(t)] , (20)

⇒∑

β,γ∈Θ

Cβ,γδβ(t)δγ(t) ≤∑β∈Θ

δβ(t)∑

γ∈Θ\β

0.5Cβ,γ . (21)

See Sec. I in the supplemental material for detailed derivation.After simplified, the over-estimated energy of (19) becomes:

Eα2 (t) = cαδα(t) +

∑β∈Θ

cβδβ(t), (22)

where

cα = Cα +∑θ∈Θ

Cθ,α ≥ 0, (23)

cβ = Cβ + Cβ,α +∑

γ∈Θ\β

0.5Cβ,γ ≥ 0. (24)

Note that the over-estimation strategy as in Eq. (20) is notrecommended for larger subsets S. In the general form, theover-estimation is

C(S)∏β∈S

δβ(t) ≤ C(S)∑β∈S

1

|S|δβ(t). (25)

Therefore the moves removing one of the labels from S willbe excessively over-estimated.

Lemma III.1. The over-estimated expansion move energyEα

2 (t) in (22) can be represented as a binary pairwise energy

Eα2 (t, z) = cα(1− zα) +

∑β∈Θ

cβzβ +∑xi∈X

cα(1− ti)zα

+∑β∈Θ

∑xi∈Xβ

cβti(1− zβ), (26)

where z is a vector of |Θ|+ 1 binary auxiliary variables.

A proof of the Lemma III.1 can be found in [8]. The binaryenergy (26) is pairwise and submodular [26], therefore (26)can be minimised using graph-cuts. In the case α ∈ Θ, we cansimply discard the term cαδα(t) from (22), and re-computethe energy Eα(t∗) with the optimised t∗. If the new energydecreases, the move is accepted, otherwise rejected.

E. Comparison with the category costs

Delong et al. [7], [27] have introduced an energy functionwith the “category costs”, which is somewhat related to ourgeometric consistency cost G(Θ). Their category cost has aform: ∑

H⊂HL(H)δH(f), (27)

where

δH(f) =

{1 if ∃xn ∈ X : fn ∈ H

0 otherwise.(28)

Note that the energy (27) pays a category cost L(H) assoon as a single label from the subset H appears in f (seeEq. (28)). Clearly this kind of category costs is different fromour geometric consistency costs, where a penalty C(β,γ) ispaid only if both labels β and γ are used.

IV. EXPERIMENTS AND APPLICATIONS

In this section, we aim to test the capability of our fittingapproach on various vision applications including two-viewmulti-homography estimation, 3D planar surface reconstruc-tion, circular object detection and vanishing point extraction.We compare our approach, denoted as MFIGP (Model-Fitting-with-Interacting-Geometric-Priors), against state-of-the-art fit-ting methods such as: Jlinkage1 [12], PEARL2 [4] as wellas other application-specific algorithms. Due to space limita-tions, here we only report the results on the two-view multi-homography estimation and vanishing point detection. Otherresults are provided in the supplementary material.

1Matlab source: http://www.diegm.uniud.it/fusiello/demo/jlk/JLinkage.rar2C++ source code: http://vision.csd.uwo.ca/code/gco-v3.0.zip

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 6

Libr

ary

Obj

ect

EE

CS

Bui

ldin

g

Extracted features Jlinkage PEARL MFIGPFig. 3. Qualitative comparisons on two-view multi-homography estimation on a subset of datasets. (Best viewed in color.) Note that only the first views areshown. The first column shows different images overlaid with extracted features, and the next three columns display the segmentation results of Jlinkage,PEARL and MFIGP respectively. Different colors indicate different structures; white crosses are detected outliers. Observe that several segments obtained byJlinkage and PEARL do not correspond to the actual planes. In contrast, MFIGP reasonably segments the data into the ground floor, left wall, right wall andthe ceiling, and detects more inliers than Jlinkage and PEARL. Notice the MFIGP’s results in the right most column that although points on the front walltend to be assigned to the other walls, the estimated homographies (e.g., of the left wall) are not arbitrarily biased. This is because points on the font wallcan be seen as distant points on the other walls. The top-right figure best demonstrates this situation.

A. Two-view multi-homography estimation

We first apply MFIGP to estimate multiple homographiesbetween two views of planar scenes. Specifically, consider twoviews of a static scene which contains a number of planes.Given a collection of point correspondences X = {xn}Nn=1

between the two views, where xn = (x1n,x

2n), a homography

H ∈ R3×3 constraints the positions of matches lying on thesame plane by x2

n ∼ Hx1n; x is x in homogeneous coordinates,

and ∼ denotes equality up to scale. The target is to recoverthe homographies, classify the point matches according to thehomographies as well as detect the wrongful matches. Thenumber of planes/homographies is unknown and must alsobe estimated. H can be instantiated from at least 4 pointmatches using the DLT method [28]. The residual r(xn,H)is computed as the Sampson distance [28].

We use the Michigan Indoor Corridor 2011 VideoDataset [29]. The video were obtained by a camera placed ona wheeled vehicle. The camera was moving with zero tilt androll angle with respect to the ground floor. We apply the KLTmethod [30] to track the features across the video sequences,then choose the matches from the first and 30th frames forestimating the homographies. We only choose the sequenceswhich contain sufficient features on the scene planes, andsatisfy the “box” assumption (explained next), namely Library,Library2, Locker Room, Object and EECS Building.

In indoor environments, the planes (e.g., walls) usuallyconform the “box” assumption, i.e., the planes are eithermutually orthogonal or parallel. The “box” constraint can beimposed by the orthogonality and parallelism between pairs ofhomographies. In particular, given two homographies Hi and

Hj , our method defines the pair-model cost function as

C(Hi,Hj) = min

(e

90−A(ni,nj)

2 , eA(ni,nj)

2

)− 1, (29)

where ni and nj are the normal vectors extracted from thehomographies Hi and Hj , A(., .) measures the angle betweentwo vectors. Although for each homography matrix Hi or Hj ,there are at most two physically possible sets of parameters(i.e., translation, rotation and normal) decomposed from Hi

or Hj [31], we can select the true normals ni and nj basedon the fact that Hi and Hj share the same (true) rotation andtranslation parameters.

Moreover, since the camera is aligned parallel to the groundfloor, the normal vector of the ground plane is known, i.e.,ng = [0, 1, 0]. To take advantage of this prior knowledge, wedefine the per-model cost function as

C(Hi) = min(

e90−A(ni,ng)

2 , eA(ni,ng)

2

), (30)

to penalize the planes/homographies inconsistent with theground. Note that the label cost function (30) is also appliedfor PEARL method.

Parameter settings. Given a set of point matches X , theneighbourhood structure N is constructed by a Delaunaytriangulation, which is used by both MFIGP and PEARL. ForMFIGP, we fix σ = 0.008, λ = 5 and Kmax = 10 for all theimage pairs. For PEARL and Jlinkage, the same inlier noisescale σ is used; we tune the remaining required parameters sothat exactly 4 homographies are returned. These homographiesare expectedly corresponding to the ground floor, left wall,right wall and the ceiling.

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 7

Hypothesis sampling. As discussed in Sec. III, the geomet-ric model fitting can be achieved by optimising the energy (6)over a set of model hypotheses H, under the assumption thatH sufficiently covers all the true models. Traditionally, His constructed by sampling minimal subsets as in RANSAC.However, since the model is high-order (homography has 8free parameters) and the data is noisy, using minimal subsetsmight not produce accurate hypotheses. Inspired by [5], wegenerate homography candidates using larger-than-minimalsubsets. Accordingly, we propose an iterative algorithm whichalternately samples hypotheses and optimises the energy. Al-gorithm 1 outlines the idea. Our algorithm firstly generates1000 hypotheses using minimal subsets, and optimises theenergy (6) to select the best subsets of models Θ and labelingf . Based on the labeling f , we partition the data into anumber of clusters using the RCM method proposed in [5],then generate a hypothesis set H by fitting the model ontoeach cluster of data. (Only clusters whose sizes are larger thanminimal are used.) These hypotheses are concatenated with thepreviously estimated models Θ, and input to the optimisationto select the best Θ and labeling f . The process is repeateduntil convergence3. In our experiments, the convergence istypically reached after dozens of iterations. To make a faircomparison, PEARL is also adapted to this algorithm.

Algorithm 1 Iterative hypothesis sampling and model selec-tion for homography estimation.

1: Sample a set of hypothesis H as in RANSAC.2: Optimise (6) over H to obtain labeling f and Θ.3: Partition the data into clusters using f (see [5] for details).4: Generate new hypotheses H using the sampled clusters.5: H = H ∪Θ.6: Optimise (6) over H to obtain new labeling f and Θ.7: Goto Step 3.

Qualitative comparisons. Fig. 3 depicts the results on theLibrary, Object and EECS Building datasets. It is evident thatJlinkage and PEARL are unable to satisfactorily segment thedata into the groups corresponding to the actual scene planes.The flaws of Jlinkage and PEARL could be explained by thefact that the changes (the camera motions) between two viewsare relatively small. In such cases, all the valid homographiesare not much different. On the contrary, MFIGP method, byimposing the geometric constraints, better classifies the datainto the ground, left wall, right wall and the ceiling.

Quantitative comparisons. To quantify the estimation ac-curacy, we use the point transfer error. Given a true matchx = (x1,x2) and a homography H, the point transfer error isdefined as d(x2,Hx1), where d(., .) is a Euclidean distancemeasure. More specifically, for each image pair, we extractthree sets of true matches from the ground floor, left walland right wall separately by running RANSAC on each planeindividually. These sets of true matches are denoted as Xg ,Xl and Xr. The ceiling and front wall are not consideredeither because they do not contain sufficient tracked features

3By setting the initial labeling for the expansion move algorithm (seeSec. III-D) to the previously estimated labeling, the energy (6) will be non-increasing. Therefore the algorithm 1 is guaranteed to converge.

or they are not always correctly detected by all the methods(see Fig. 3). The total error of an estimate Θ is computed as∑

X∈{Xg,Xl,Xr}

minH∈Θ

∑x∈X

d(x2,Hx1). (31)

Table I reports the mean and median errors over 20 repetitionson the 5 tested image pairs. It is clear that in most of the casesMFIGP outperforms its competitors. Table I also depicts theaverage computation time, where MFIPG is comparable withPEARL and faster than Jlinkage.

Datasets Jlinkage PEARL MFIGP

LibraryMean error 629.55 550.64 442.55Median error 539.21 579.40 430.82Run time (seconds) 14.71 8.85 9.39

Library2Mean error 571.44 513.43 385.13Median error 556.01 541.54 366.04Run time (seconds) 13.59 8.35 9.19

Locker RoomMean error 201.23 120.28 122.46Median error 173.86 120.43 120.37Run time (seconds) 12.82 8.64 9.13

ObjectMean error 1028.19 715.02 624.99Median error 1001.94 783.73 548.72Run time (seconds) 11.16 8.46 9.31

EECS BuildingMean error 1095.43 840.83 663.39Median error 1142.83 748.67 574.44Run time (seconds) 11.29 8.19 8.77

TABLE IQUANTITATIVE COMPARISON RESULTS ON TWO-VIEW MULTIPLE

HOMOGRAPHY ESTIMATION. THE LOWEST ERROR AND COMPUTATIONTIME ARE BOLDFACED.

B. Vanishing point detection in a Manhattan world.

In this section, we demonstrate the performance of ourfitting method on detecting vanishing points (VP) from a singleimage. The task can be accomplished by first extracting edgesfrom the image, then searching for the most likely vanishingpoints that the extracted edges pass through. This problem isclearly a multi-model fitting problem where the models arevanishing points and the data are line segments.

In a Manhattan world [6], the 3D lines are considered tobelong to the three mutually orthogonal directions. Thereforethe orthogonality constraints must be imposed to ensure thedetection accuracy.

We apply MFIGP and its main competitor, PEARL, tosolve the VP detection task; then compare the performance.We also compare against the recent vanishing point detectorssuch as Tardif’s method [21] and RNS [22]. Tardif’s methodis based on a multi-model fitting method Jlinkage [12]. Themethod first clusters the edges into groups (i.e., pencils oflines), then estimates the vanishing points separately on eachgroup. Finally, a triplet of most orthogonal vanishing pointsis heuristically selected. RNS assumes that the three dominantdirections in images correspond to the Manhattan directions,then formulates the problem as a single-model fitting where amodel is a triplet of orthogonal vanishing points. A RANSAC-like algorithm is used to search for the best model.

We use the York Urban Dataset [32], which consists of102 calibrated images of man-made environments. For eacheach image, the database provides the ground truth orthogonal

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 8

triplet of vanishing points, and groups of manually extractedline segments corresponding to the vanishing points. Thecamera internal parameters are also given.

Assume that we have extracted a set of edges X = {xn}Nn=1

from the image. Each edge is represented by two end pointsand one middle point, i.e., xn = [e1

n, en, e2n]. The task is

to estimate a set of vanishing points Θ = {θk}Kk=1. Pointsare in the homogeneous coordinate, i.e., e = [e1, e2, 1], θ =[θ1, θ2, 1]. Following the work [21], the residual from an edgexn to a vanishing point θk is computed as

r(xn,θk) =|l>e1

n|√l21 + l22

with l = [l1, l2, l3] = en × θk. (32)

l is the line passing through the middle point en and thevanishing point θk. The geometric consistency between twovanishing points is defined as:

C(β,γ) =

{0 if A(β,γ) ∈ [89, 91]

10 otherwise(33)

where A(β,γ) measures the angle between two directionscorresponding to two vanishing points β and γ. SpecificallyA(β,γ) is defined as [28]

A(β,γ) = acos

[β>wγ√

β>wβ√γ>wγ

]180

π, (34)

where w is the image of absolute conic given by the camerainternal matrix K (i.e., w = K−>K−1 [28]). A constant per-model cost function is used, i.e., C(θ) = 1 ∀θ.

In this application, the spatial smoothness assumption is notsensible — the vertical and horizontal edges within a real planebelong to two structures (vanishing directions) although theyare spatially close. Thus we remove the spatial smoothnesscosts from both MFIGP and PEARL (i.e., by setting w1

in (6) to 0). Our method uses the following parametersσ = 1, λ = 5,Kmax = 10. PEARL also uses the samenoise level of 1 pixel, and is manually tuned to return the bestresults. Moreover, we randomly sample 1000 VP hypothesesfor MFIGP and PEARL using minimal subsets of 2 edges.

To evaluate the performance, we use two different measures.The first one is the accuracy of the detected vanishing points.Let Θ and Θ be the estimated and ground truth vanishingpoints respectively, the estimation accuracy is computed asthe angular deviation between Θ and Θ. That is

AD(Θ) = minΓ

1

3

3∑i=1

A(θΓi , θi) (35)

where Γ is a permutation on Θ and the function A is definedas in (34). Since some images contain only one or twovanishing directions, a post-processing step is carried outbefore evaluating (35). If Θ contains only two vanishing points(i.e., Θ = {θ1,θ2}), the third one can be computed as:

θ3 = Null(w(θ1,θ2)), (36)

where Null(M) returns the left nullspace of M. θ3 will beorthogonal with θ1 and θ2. If Θ consists of only one vanishingpoint (i.e., Θ = {θ1}), θ2 and θ3 are set to [0, 0, 1]. If |Θ| > 3,

the top three largest (in terms of the number of supportinginliers) vanishing points are selected.

The second error measure is the consistency between theestimated vanishing points and the ground truth edges. That is

CE(Θ) =1

|X |∑x∈X

minθ∈Θ

r(x,θ), (37)

where X is a set of ground truth line segments. The medianangular deviation and consistency errors over 50 repetitionsare recorded for evaluation.

Using the supplied edges. We first test the performance ofall the methods using the supplied edges given by YorkUrbanDatabase. Fig. 4 shows the quantitative results. Expectedly, allthe methods are comparable since the data contains no outliers.

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

Angular deviation (degree)

MFIGP

RNS

Tardif

PEARL

(a) Cumulative angular deviation.

0 1 2 3 4 50

0.2

0.4

0.6

0.8

1

Consistency error (pixel)

MFIGP

RNS

Tardif

PEARL

(b) Cumulative consistency error.

Fig. 4. Vanishing point detection results using the manually extractedlines. The graphs show that all the methods are relatively comparable. Theperformance of all the methods is not much different because the data is notcontaminated by outliers.

Using extracted edges. We perform the same experimentagain, but using the edges automatically extracted by themethod in [21]. The quantitative comparison results are shownin Fig. 5. It is evident that MFIGP significantly outperformsPEARL in both error measures, which proves the robustnessof our fitting approach. Compared against the specialised VPdetectors, MFIGP is considerably better than Tardif’s methodand comparable with RNS. Our improvements over Tardif’ssuggest that it is generally better to impose the geometricconstraints during the model parameter estimation rather thanafter the estimation as in Tardif’s method.

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

Angular deviation (degree)

MFIGP

RNS

Tardif

PEARL

(a) Cumulative angular deviation.

0 1 2 3 4 50

0.2

0.4

0.6

0.8

1

Consistency error (pixel)

MFIGP

RNS

Tardif

PEARL

(b) Cumulative consistency error.

Fig. 5. Vanishing point detection results using the automatically extractedlines. Observe that our method (MFIGP) performs significantly better than itscompetitor (PEARL). The low performance of PEARL is mainly due to itsincapability to handle data with strong noise and outliers. In contrast, MFIGPwith the use of geometric constraints is highly robust. Also MFIGP is stillcompetitive with RNS [22] and considerably better than Tardif’s method [21].

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 9

Ext

ract

eded

ges

PEA

RL

MFI

GP

Fig. 6. Qualitative comparisons on detecting vanishing points (Best viewed on screen). The top row shows the images with the automatically extracted lines.The second and bottom rows show the line clustering results according to the vanishing points detected by PEARL and MFIGP separately. In the last tworows, the red, green and blue lines show the three estimated directions while the black lines are detected outliers. Notice the first two images (from left toright), the PEARL method fails to recover the three Manhattan directions due to the presences of non-Manhattan directions. In contrast, MFIGP exactly returnsthree orthogonal directions. The last three columns demonstrate another advantage of MFIGP over PEARL. Observe that the images contain uneven-structures(i.e., planes with different number of edges). In such cases, PEARL tends to split the big structures into smaller ones, resulting in wrong detections. HoweverMFIGP can tackle this problem easily since inconsistent models (e.g., two similar models) are discouraged to jointly appear in the solution.

Fig. 6 qualitatively demonstrates the edge clustering resultsof MFIGP and PEARL on some difficult datasets. The resultsof Tardif and RNS are visually similar to ours, so we donot show those here. It can be seen that PEARL incorrectlyestimates the three Manhattan directions when there existnon-Manhattan directions in the images (see the first twocolumns of Fig. 6). Moreover, the last two columns showanother weakness of PEARL, where the big structures (e.g.,the vertical directions) are split into smaller similar structures.On the contrary, these issues can be tackled effectively inour fitting approach (see the last row of Fig. 6). Indeed,as opposed to PEARL, our method would unlikely returntwo approximately identical vanishing points since they aregeometrically inconsistent.

C. Sensitivity to the choice of tuning parameters.

It is worth noting that our framework has used a single setof tuning parameters for different datasets in each specific ap-plication, and there has been a relatively small adjustment fordifferent applications. The results suggest that our frameworkis not very sensitive to the choice of tuning parameters. Wesuggest the parameter λ should not be smaller than 1 since itwould cause over-fitting. If we are working with images, anamount of a few pixels for σ is sufficiently good.

V. CONCLUSIONS

The importance of geometric priors has been acknowledgedin various specific settings. In this work, we proposed a generalgeometric fitting framework, which allows the integration ofhigh-level geometric priors into the model hypothesis eval-uation. Besides the essential hypotheses evaluation using an

energy minimisation, our framework can work with variouscustomisable routines, such as hypothesis generation, geomet-ric constraint specifications for different types of models.

Empirically, our method outperforms the existing modelfitting methods on a variety of computer vision applicationssuch as homography estimation, vanishing point detection,planar surface reconstruction and circular object detection. Weshowed that geometric constraints are useful to eliminate theinfluences of (non-normal) noise and (non-uniform) outliers.We believe that our fitting framework will prove to be usefulto many other multi-model fitting applications, where priorknowledge of the scenes/objects is available and can betranslated into interactions between structures.

In future work we plan to investigate further the issue of tun-ing weighting parameters in the energy function. Although ourframework works quite well with the suggested parameters,an automatic tuning mechanism is certainly more preferable.Furthermore, we plan to study the role of interacting geometricpriors in higher-dimension models such as projective trans-formations (i.e., fundamental matrices), spline curves, whichunderpin many interesting applications: multi-body structurefrom motion and multiple object tracking.

ACKNOWLEDGEMENT

The authors would like to thank the anonymous reviewersfor their insightful and constructive comments. We appreciateProfessor Hanzi Wang for providing the implementation of theASKWH method [11]. We also thank Dr. Anton Milan for hishelpful discussion of the graph-cuts implementation.

IEEE TRANSACTIONS ON IMAGE PROCESSING — ACCEPTED MANUSCRIPT 10

REFERENCES

[1] M. A. Fischler and R. C. Bolles, “Random sample consensus: Aparadigm for model fitting with applications to image analysis andautomated cartography,” Comm. of the ACM, vol. 24, pp. 381–395, 1981.

[2] K. Schindler and D. Suter, “Two-view multibody structure-and-motionwith outliers through model selection,” IEEE Trans. on Pattern Analysisand Machine Intelligence (PAMI), vol. 28, no. 6, pp. 983–995, 2006.

[3] J. Yu, T.-J. Chin, and D. Suter, “A global optimization approach to robustmulti-model fitting,” in CVPR, 2011.

[4] H. Isack and Y. Boykov, “Energy-based geometric multi-model fitting,”International Journal of Computer Vision, vol. 97, no. 2, pp. 123–147,2012.

[5] T. T. Pham, T.-J. Chin, J. Yu, and D. Suter, “The random cluster modelfor robust geometric fitting,” in CVPR, 2012.

[6] J. Coughlan and A. L. Yuille, “Manhattan world: compass direction froma single image by bayesian inference,” in ICCV, vol. 2, 1999.

[7] A. Delong, A. Osokin, H. Isack, and Y. Boykov, “Fast approximateenergy minimization with label costs,” in CVPR, 2010.

[8] L. Ladicky, C. Russell, P. Kohli, and P. H. S. Torr, “Graph cut basedinference with co-occurrence statistics,” in ECCV, 2010.

[9] T.-J. Chin, J. Yu, and D. Suter, “Accelerated hypothesis generation formulti-structure data via preference analysis,” IEEE Trans. on PatternAnalysis and Machine Intelligence (PAMI), vol. 34, no. 4, pp. 625–638,2012.

[10] M. Zuliani, C. S. Kenney, and B. S. Manjunath, “The multiRANSACalgorithm and its application to detect planar homographies,” in ICIP,2005.

[11] H. Wang, T.-J. Chin, and D. Suter, “Simultaneously fitting and segment-ing multiple-structure data with outliers,” Pattern Analysis and MachineIntelligence, IEEE Transactions on, vol. 34, no. 6, pp. 1177–1192, 2012.

[12] R. Toldo and A. Fusiello, “Robust multiple structures estimation withJ-Linkage,” in ECCV, 2008.

[13] T.-J. Chin, H. Wang, and D. Suter, “Robust fitting of multiple structures:The statistical learning approach,” in ICCV, 2009.

[14] P. H. S. Torr, “Geometric motion segmentation and model selection,”Phil. Trans. Royal Society of London A, vol. 356, pp. 1321–1340, 1998.

[15] T. T. Pham, T.-J. Chin, J. Yu, and D. Suter, “Simultaneous sampling andmulti-structure fitting with adaptive reversible jump MCMC,” in NIPS,2011.

[16] A. J. Baddeley and M. N. M. van Lieshout, “Stochastic geometry modelsin high-level vision,” Advances in Applied Statistics, 1993.

[17] M. Ortner, X. Descombes, and J. Zerubia, “A marked point processof rectangles and segments for automatic analysis of digital elevationmodels,” IEEE TPAMI, vol. 30, no. 1, 2008.

[18] F. Lafarge, G. Gimel’farb, and X. Descombes, “Geometric featureextraction by a multimarked point process,” IEEE Transactions onPattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1597–1609, 2010.

[19] Z. Jia, A. Gallagher, A. Saxena, and T. Chen, “3d-based reasoning withblocks, support, and stability,” in CVPR, 2013.

[20] G. Schindler and F. Dellaert, “Atlanta world: an expectation maximiza-tion framework for simultaneous low-level edge grouping and cameracalibration in complex man-made environments,” in CVPR, vol. 1, 2004.

[21] J.-P. Tardif, “Non-iterative approach for fast and accurate vanishing pointdetection,” in ICCV, 2009.

[22] F. Mirzaei and S. Roumeliotis, “Optimal estimation of vanishing pointsin a manhattan world,” in ICCV, Nov 2011.

[23] C. Olsson and Y. Boykov, “Curvature-based regularization for surfaceapproximation,” in CVPR, 2012.

[24] V. Kolmogorov, “Convergent tree-reweighted message passing for en-ergy minimization,” Pattern Analysis and Machine Intelligence, IEEETransactions on, vol. 28, no. 10, pp. 1568–1583, Oct 2006.

[25] G. Schwarz, “Estimating the dimension of a model,” The Annals ofStatistics, vol. 6, no. 2, pp. 461–464, 1978.

[26] Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy mini-mization via graph cuts,” IEEE Trans. on Pattern Analysis and MachineIntelligence (PAMI), vol. 23, no. 11, pp. 1222–1239, 2001.

[27] A. Delong, L. Gorelick, O. Veksler, and Y. Boykov, “Minimizingenergies with hierarchical costs,” Int. J. Comput. Vision, vol. 100, no. 1,pp. 38–58, Oct. 2012.

[28] R. Hartley and A. Zisserman, Multiple View Geometry. CambridgeUniversity Press, 2004.

[29] G. Tsai, C. Xu, J. Liu, and B. Kuipers, “Real-time indoor sceneunderstanding using bayesian filtering with motion cues,” in ICCV, 2011.

[30] J. Shi and C. Tomasi, “Good features to track,” in CVPR, 1994.

[31] Y. Ma, S. Soatto, J. Koeck, and S. Sastry, “Reconstruction from twocalibrated views,” in An Invitation to 3-D Vision, 2004, vol. 26, pp.109–170.

[32] P. Denis, J. H. Elder, and F. J. Estrada, “Efficient edge-based methodsfor estimating manhattan frames in urban imagery,” in ECCV, 2008.

Trung T. Pham received a B.Sc. in Mathematicsand Computer Science from Ho Chi Minh CityUniversity of Science, Vietnam in 2007, M.Eng. inElectronics and Computer Engineering from Chon-nam National University, South Korea in 2010, andsubsequently a Ph.D. in Computer Science from TheUniversity of Adelaide, South Australia in 2014.His main research interests include robust parameterestimation for computer vision applications, machinelearning and discrete/continuous optimisation.

Tat-Jun Chin received a B.Eng. in Mechatron-ics Engineering from Universiti Teknologi Malaysia(UTM) in 2003, and subsequently in 2007 a Ph.D.in Computer Systems Engineering from MonashUniversity, Victoria, Australia. He was a ResearchFellow at the Institute for Infocomm Research (I2R)in Singapore from 2007–2008. Since 2008 he isa Lecturer at The University of Adelaide, SouthAustralia. His research interests include robust es-timation and statistical learning.

Konrad Schindler received a Diplomingenieur(M.tech) degree in photogrammetry from ViennaUniversity of Technology, Austria in 1999, and aPhD from Graz University of Technology, Austria, in2003. He has worked as a photogrammetric engineerin the private industry, and held researcher positionsin the Computer Graphics and Vision Department ofGraz University of Technology, the Digital Percep-tion Lab of Monash University, and the ComputerVision Lab of ETH Zurich. He became assistantprofessor of Image Understanding at TU Darmstadt

in 2009, and since 2010 has been a tenured professor of Photogrammetryand Remote Sensing at ETH Zurich. His research interests lie in the eldof computer vision, photogrammetry, and remote sensing, with a focus onimage understanding and 3d reconstruction. He currently serves as head ofthe Institute of Geodesy and Photogrammetry, and as associate editor for theISPRS Journal of Photogrammetry and Remote Sensing, and for the Imageand Vision Computing Journal.

David Suter received a B.Sc.degree in appliedmathematics and physics (The Flinders University ofSouth Australia 1977), a Grad. Dip. Comp. (RoyalMelbourne Institute of Technology 1984)), and aPh.D. in computer science (La Trobe University,1991). He was a Lecturer at La Trobe from 1988 to1991; and a Senior Lecturer (1992), Associate Pro-fessor (2001), and Professor (2006-2008) at MonashUniversity, Melbourne, Australia. Since 2008 he hasbeen a professor in the school of Computer Science,The University of Adelaide. He is head of the

School of Computer Science. He served on the Australian Research Council(ARC) College of Experts from 2008-2010. He is on the editorial boards ofInternational Journal of Computer Vision. He has previously served on theeditorial boards of Machine Vision and Applications and the InternationalJournal of Image and Graphics. He was General co-Chair or the AsianConference on Computer Vision (Melbourne 2002) and co-Chair of the IEEEInternational Conference on Image Processing (ICIP2013).