A New Image Registration Scheme Based on Curvature Scale

The Visual Computer manuscript No.(will be inserted by the editor)

Cui Ming; Peter Wonka; Anshuman Razdan; Jiuxiang Hu

A New Image Registration Scheme Based on CurvatureScale Space Curve Matching

Abstract We propose a new image registration schemefor remote sensing images. This scheme includes threesteps in sequence. First, a segmentation process is per-formed on the input image pair. Then the boundaries ofthe segmented regions in two images are extracted andmatched. These matched regions are called confidenceregions. At last, a non-linear optimization is performedonly in the matched regions to get a global set of trans-form parameters. Experiments show that this scheme ismore robust and converges faster than registration ofthe original image pair. We also develop a new curve-matching algorithm based on curvature scale space tofacilitate the second step.

Keywords image registration · curve matching ·curvature scale space

1 Introduction

Image registration is a fundamental problem in imageprocessing and often serves as an important preprocess-ing procedure for many applications such as image fu-sion, change detection, super resolution, and 3D recon-struction. While we can draw from a large number ofsolutions [4,46], most existing methods are specificallytailored to a particular application. For example, video-tracking focuses on non-rigid transformations with smalldisplacements and medical image registration can takeadvantages of the uniform background and absence ofocclusion. In this paper, we work with remote sensingimage pairs that have been geometrically and photomet-rically rectified to some extent and differ by a globalsimilarity transform. The output of the computation isthe set of global parameters of the transform.

Ming. CuiPrism, ASU Pobox 878609 Tempe AZ 85287-8609E-mail: [email protected]

S. Authorsecond address

In the scope of remote-sensed image registration ex-isting methods fall into two categories: area-based meth-ods and feature-based methods [46]. Area-based meth-ods [26,45] directly use the whole image or some pre-defined windows without attempting to find salient fea-tures. First, a function is defined, such as correlation ormutual information, to evaluate the similarity betweenthe image pairs under a given transformation. The sec-ond step is an optimization procedure to find the pa-rameter set that maximizes the function. The drawbackof this approach is twofold: searching for a global opti-mum is computationally expensive and the search spaceis not smooth due to non-overlapping areas and occlusionin the input images. Therefore, the search process willprobably stop at a local optimum instead of a global opti-mum. In contrast, feature-based methods determine thetransformation by matching salient features extractedfrom the original image pairs. The difficulty of these al-gorithms is to find corresponding features robustly. Iffeatures are matched incorrectly, they become outliers.As features only draw from a smaller local region, thealgorithms might create a large number of such outliersand the elimination of outliers is very time consumingand unstable.

We propose a new registration scheme that is a com-bination of these two approaches: The first step is tosegment the input images and to extract closed bound-ary curves. The second step is to match the extractedcurves by an improved curvature scale-space algorithm.In the third step, we try to register the two images us-ing only the regions enclosed by the matched boundarieswith an area-based method. This registration frameworkcompares favorably to using feature-based registrationor area-based registration alone. Because we use curvesas geometric features, our feature-based matching algo-rithm is more robust than relying on point features. Ad-ditionally, the curve based feature-matching algorithmidentifies potentially matched regions. This helps the area-based optimization algorithm in the following ways: a)the curve matching gives multiple initial guesses to starta local optimization algorithm b) the search space is

2 Cui Ming; Peter Wonka; Anshuman Razdan; Jiuxiang Hu

smoother compared to the search space induced by theoriginal image pair and therefore there are less chances ofgetting stuck in a local optimum. c) The computationaloverhead is greatly reduced. We will demonstrate theseadvantages through experiments on selected data sets.

The paper is organized as follows: Section 3 will givean overview of the registration scheme. Section 4 detailsthe curve extraction and curve matching steps of theframework. Section 5 explains the area-based registra-tion step. Section 6 gives results of our implementationon selected test data sets. Section 7 provides concludingremarks and outlines ideas for future work.

2 Related Work

2.1 Image Registration

Image registration techniques can be categorized accord-ing to their application area [13,29,5]. The most promi-nent areas are medical imaging, remote sensing, and com-puter vision applications such as video registration. Formedical images, the objects under registration often goesthrough non-rigid deformation, therefore non-rigid regis-tration methods [30] are very popular such as diffusion-based methods [36] and level set methods [40]. In [14]the state of art of video registration is demonstrated.In remote sensing image registration, the methods de-scribed in [27,25] are representative. In this area bothfeature-based methods and area-based methods are use-ful. In the methods described in [43,16,40] features in-variant to a selected transform are first extracted andmatched. Remote sensing images often contain many dis-tinctive details that are relatively easy to detect (pointfeatures, line features, region features). Wavelet decom-position and multi-resolution [20] techniques are ofteninvolved to make the procedure more effective and ef-ficient. Methods based on template matching [12] canhelp with images that are not radiometrically rectified.Methods using mutual information [42,9,34,21] are alsointroduced to deal with multi-modal images registration.

2.2 Curve Matching

We use a curve(shape) matching algorithm based on cur-vature scale space descriptor(CSSD) [33,31] to facilitatethe registration process. CSSD is one of the shape rep-resentation and description techniques aimed to effec-tively capture important shape features based on eitherboundary information or the interior content informa-tion of the target shapes. Other shape description meth-ods include shape signature such as chain code [19] andmoments [28], fourier descriptor [7], medial axis trans-form [35], and shape matrix [18]. Researchers have foundthat shape description and matching techniques are veryuseful for remote sensing image registration. Thus dif-ferent shape description methods have been adopted to

Image 1

Segmetation

Image 2

Segmetation

Curve Matching

Matched Curves

Registration in Confidence Regions

Transform Parameters

Fig. 1 This figure shows an overview of our registration al-gorithm consisting of three major parts: the first part is asegmentation algorithm that extracts closed curves from thetwo input images. The second part is a curve matching algo-rithm that computes a set of matched curve pairs. The thirdpart is an area-based registration algorithm that computesthe final result by iterative optimization of a similarity met-ric in selected confidence regions.

aid registration such as improved chain code [43], mo-ments [17] and shape matrix [3]. It has been generallyrecognized that CSSD is a good shape description methodbecause it can provide a multi-scale representation forthe shape and the extracted features have good per-ceptual significance. However, the CSSD matching algo-rithm is very complex and sensitive to noise and partialocclusion. It also includes several ad-hoc parameters toadjust for different inputs, which makes it not robust. Inthis article, we suggest a new matching algorithm whichis more robust and has no ad-hoc parameters to alleviatethese shortcomings. Furthermore, we are the first to usecurvature scale space for image registration.

3 Overview

The new registration scheme consists of three buildingblocks, as shown in figure 1. The details of these buildingblocks will be discussed in subsequent sections. In thisoverview section we will mainly focus on the input andoutput of each block, and how they cooperate to makethe whole scheme work:

Segmentation: The first building block is segmen-tation. Segmentation requires a gray scale image as in-put and computes a set of non-intersecting closed re-gions. The boundaries of these regions are closed curves.These closed curves are the output of this building block.Ideally, the segmentation finds some salient regions that

A New Image Registration Scheme Based on Curvature Scale Space Curve Matching 3

correspond to meaningful physical objects in the remotesensing images, such as lakes, forests, or regions sepa-rated by road networks or rivers. The requirement forthe segmentation algorithm is that it is able to pick upseveral of these outlines. Currently, we use a modifiedwatershed transform for the segmentation algorithm.

Curve Matching: the input of the curve matchingalgorithm is a set of curves extracted from image 1 andanother set of curves extracted from image 2. The as-sumption of the curve matching algorithm is that thesame physical objects will have very similar outlines ifthey appear in both input images. The curve-matchingstep tries to match outlines that correspond to the samephysical objects. The algorithm computed the similaritybetween each possible pair of curves. A valid pair con-sists of one curve stemming from image one and anothercurves stemming from image 2. The output of the curvematching is a set of N curve pairs with the highest simi-larity. The number N is decided by the user or some sim-ilarity criterion. We call the regions inside the matchedcurves potentially matched or confidence regions. Alongwith each matched pair, a gross estimation of the trans-form is also included in the output. The algorithm weuse is an improvement of the curvature scale-space algo-rithm.

Area-based Registration: the input to the area-based registration step are: a) the two original input im-ages, b) the potentially matched regions, including c)the estimates of the transform parameters. The outputof the registration step is the optimal set of transformparameters obtained by a non-linear optimization pro-cess. The optimization process consists of an algorithmto search for a local optimum near a given point in thesearch space. This algorithm is invoked several times us-ing the initial guesses obtained by the curve-matchingstep. How many local optimizations are performed canbe decided by a user-defined number M. Alternatively,all initial guesses can be used. The registration algorithmconducts the optimization only in the confidence regionsto improve performance.

4 Feature Matching

4.1 Segmentation and Curve Extraction

The segmentation step plays a significant role in theresulting registration. Currently, we work with existingsegmentation methods. We employ a modified watershedtransform [10]. As observed by other researchers in thisfield, current segmentation methods have certain weak-nesses: a) almost all segmentation algorithms have pa-rameters that need to be adjusted for different inputimages. b) Segmentation is sensitive to noise, shadows,occlusion, etc. The second problem is hard to address ingeneral. However, since we are dealing with aerial imagesthat are already rectified and differ only by a similarity

transform this problem is partially alleviated. While ourimplementation shows that using current segmentationmethods allows for strong registration results, we need acurve-matching algorithm that is able to overcome someof the shortcomings of the segmentation. Therefore, ourcurve-matching scheme is designed in such a way, thatit tolerates noise and partial deterioration of curves. Ad-ditionally, the complete registration framework will stillbe functional if some curves are matched incorrectly. Thesegmentation algorithm gives a decomposition of the im-age into labeled regions. We extract boundary curves inthe form of lists of two-dimensional points [28] and passthese point lists to the curve-matching algorithm.

4.2 Curve Matching

The core idea behind the matching algorithm is that twocurves representing the same physical object will differonly by a similarity transform, which includes uniformscaling, rotation and translation. Therefore, we need touse a description of a curve that is invariant to the sim-ilarity transform. Further, we need a robust algorithmthat compensates for noise in the segmentation process.Our algorithm uses the curvature scale space descriptorto represent and match boundary curves. This represen-tation is invariant to the similarity transform and pro-vides the basis for a robust matching algorithm. We willfirst explain the curvature scale space descriptor. Thenwe will give the details of our new matching algorithmto estimate the similarity between two curves and finallydescribe how this curve-matching algorithm can be usedto match two sets of curves.

The curvature scale space image (CSSI) [33] is a bi-nary two-dimensional image that records the position ofinflection points of the curve convoluted by different-sized Gaussian filters. A planar curve r is given as apolyline defined by a set of discrete points. This curvecan be represented by the parametric vector equationwith parameter u, where x and y describe the coordi-nate values:

r(u) = (x(u), y(u)) (1)

From the original curve, a set of curves is createdby low-pass Gaussian filtering with different kernel sizeparameter σ:

Γσ = (X(u, σ), Y (u, σ))|u ∈ [0, 1] (2)

Where (X(u, σ) = x(u)∗g(u, σ) and Y (u, σ) = y(u)∗g(u, σ). Convolution is denoted by ∗ and

g(u, σ) =1

σ√

2πe−u2

2σ2 (3)

Each curve in the set will create one row of the scalespace image by the following method: the parameter u


Fig. 2 Left: a closed curve depicting the outline of a fish.Inflection points are shown in red Right: the scale space imageof the curve to the left. Each row of the scale space imagecorresponds to a curve of scale σ that is parameterized byu ∈ [0, 1]. The black entries record the inflection point for onescale determined by the smoothing parameter of the Gaussianfilter σ.

is discretized and for each value of u the curvature k canbe estimated by the following equation:

k(u) =x(u)y(u)− y(u)x(u)

(x(u)2 + y(u)2)32

(4)

If the curve with smoothing parameter has a zerocrossing in curvature at location u we set CSSI(σ, u) = 1and CSSI(σ, u) = 0 otherwise. A zero crossing in curva-ture is an indicator for a significant feature in the curve.This indicator is invariant to affine transformations. Seefigure 2 left for an example of a curve and its curvaturescale space image.

Given two curves the respective CSSIs are great rep-resentations for curve matching. The conventional CSSImatching algorithm [31] first tries to align the CSSIs tocompensate for the difference in orientation. Such analignment can be done by computing the appropriatecircular horizontal shift of the image. The original algo-rithm is based on an analysis of the highest and secondhighest contour maximum. However, this algorithm hassome problems [22,44] that we try to overcome:

– The algorithm relies on the extraction of contourmaxima. In practice, the CSSI will often be non-continous and have several close-by maxima. As aresult the maximum extraction is unstable, time con-suming, and requires the specification of ad-hoc pa-rameters. In constrast, the idea of our algorithm isto avoid the detection of contour maxima to improvestability.

– To increase stability of maximum extraction, the orig-inal algorithm used small increments for the supportof the Gaussian filter σ. This results in CSSIs witha large number of rows. Our algorithm works withincrements that are one order of magnitude larger sothat we only need to compare relatively small CSSIs.

– To compute the circular shift parameter between twocurves would require an exhaustive search that isoverly time consuming for most applications. Theoriginal algorithm proposes an acceleration that only

Fig. 3 This figure shows a CSSI (scale space curvature im-age). Each line of the CSSI is a binary string.

Fig. 4 This figure shows how we can compare two lines of aCSS image.

examines four distinct shift parameters. For curvesthat have partial deterioration this acceleration algo-rithm is very likely to fail. Our algorithm makes useof dynamic programming, so that we can find theglobal optimum of our similarity metric.

We propose a new CSS matching algorithm that hastwo new features: first, the whole CSSIs is compared lineby line in contrast to the conventional algorithm; second,dynamic programming is used to guarantee a global op-timum.

First, we define a cost metric for comparing a line iin CSSI image one with a line j in CSSI image two. Asshown in figure 3 left, each line of the CSSI is a binarystring with values of either 0 or 1. We warp this line intoa circle and consider the position where values take 0 asempty and 1 as occupied by a small ball with unit mass.Then the centroid of all these small balls is calculated.We can estimate four parameters (two of them are shownin figure 4): the angle α spanned by the connecting lineand a reference horizontal line, the distance d betweenthe centroid and the center of the circle, the mass m com-puted by the number of 1′s in the line, and the standarddeviation std. The main reason we chose these featuresis that they are a good description of the distribution ofthe inflection points and that they are invariant to cir-cular rotation (circular shift). Additionally, higher ordermoments could be used, but that was not stable in ourexperiments. We use three features to establish a costmetric c that compares a line i in CSSI one with line jin CSSI two. We use only three of the features in thisformula. The use of the angle feature and the parame-


Fix image Move image

Interpolate Transform

Metric Optimization

Stop?

False

True

Output Transform Parameters

Fig. 5 an overview of the complete registration framework.

ter values for λi will be explained later in the dynamicprogramming step.

c(i, j) = λ1|mi −mj |+ λ2|di − dj |+ λ3|stdi − stdj | (5)

Using this metric c to compare two single lines i andj, a dynamic programming [41] process is used to find thetotal cost TC between two curves’ CSSIs. The difficultyarises from the fact that we cannot simply compare allpairs of lines with the same index i. The intrinsic prop-erties of CSSI images suggest that we have to considernon-linear scaling along the scale axis of the CSSIs inthe total cost TC [1]. Dynamic programming [2] is aperfect match for this task, as it allows comparison oflines across different scales. The first step of dynamicprogramming is to build a cost table. Each row of thetable corresponds to a line of CSSI one and each col-umn corresponds to a line of CSSI two. A cell in thecost table contains the cost c(i, j) described above (seefigure 5). The second step is to build an aggregated costtable as follows: ac(1, 1) = c(1, 1) and

ac(i, j) = c(i, j)+

min

ac(i− 1, j) + λ4||αi−1 − αj | − |αi − αj ||ac(i, j − 1) + λ4||αi − αj−1| − |αi − αj ||ac(i− 1, j − 1)) + λ4||αi−1 − αj−1| − |αi − αj ||

The aggregate cost function of every cell is composedof two parts: 1) the distance of matching two rows (scales)of CSSIs that intersect at this cell and 2) an angle depen-dent cost related to the absolute difference between theangle at this cell and the previous cell. The second partof the cost function means that we do not restrict thecomparison to finding one global horizontal shift, but weallow small variations between neighboring cells. How-ever, we penalize large differences in angle to only allowminor differences. The angle dependent cost will thenbe multiplied by a weight factor λ4. The total cost ofmatching the two curves can then be found in the bot-tom right cell TC = ag(imax, jmax). Because we do notuse contour maximum detection as the originals algo-rithm our technique is stable even with large increments

of the Gaussian filter support σ. Therefore, the table sizeis fairly small and we did not optimize the search processfor the global optimum. We obtained table sizes of 10 x10 up to 20 x 20 in our results. The table size is implic-itly determined by the complexity of the curve. Throughexperiments we found λ1 = 0, λ2 = 1, and λ3 = 0.3, andλ4 = 1 to provide the best results. Alternatively, we ex-perimented with a version that had higher weight λ4 atlower resolution. The idea is that higher resolutions aremore likely to be subject to noise and we want to decreasethe effect of noise in the overall similarity metric. Whilethis idea results in minor changes in curve matching, itdid not influence the outcome of the registration for anyimage pair. Even though the CSSI matching algorithmcompares complete curves, it can also match curves withpartial correspondence [32]. This is shown in figure 7.

The output of the dynamic programming algorithm isa similarity between two curves. Using this algorithm asa building block, our next task is to find the best matchesbetween two sets of curves. Suppose m and n curves areextracted from two images respectively, then the costs ofeach possible match is evaluated. These costs form a ma-trix. If l = min(m,n), then we want to find l curve pairsthat will minimize the overall cost. We further restrictthe matching to allow each curve to appear only in onecurve pair. This is an example of an assignment problem.We solve this problem with the Hungarian method [24].

4.3 Complexity Analysis

The complexity of the the original curve matching al-gorithm and our new algorithm is similar. Suppose theinput curves have M discrete points and the Gaussianfilter has N levels. The size of the CSSI will be M ∗N .In the original algorithm, the whole CSSI has to bescanned once to find all the contour maxima. This step isO(MN). The contour maxima will be stored as a featurelist. We assume the feature list will have the size log(M).Then a circular shift will be performed to align the high-est maxima in the two CSSIs. For each feature in the firstlist, the second list will be scanned to find the nearestfeature. Then a circular shift will be performed to alignthe highest maxima in the first CSSI to the second high-est maxima and repeat the following operations. Overall,the complexity of the original curve-matching algorithmwill be O(MN + log(M)3). In our algorithm, the CSSIwill also be scanned once to decide the shift angle anddistance for each row. Then the dynamic programmingwill have a constant complexity of N ∗ N . The overallcomplexity will be O(MN + N2). Since N is usually farless then M , the complexity is comparable. Our resultsin section 6 also show comparable running speeds of theimplementation.


Curve 2

Curve 1

Fig. 6 An overview of dynamic programming for comparingtwo curves.

5 Area-based Registration

We adopt a general parametric registration frameworkthat is illustrated in figure 6. We have two input images:one is selected as the reference image and the other oneis the moving image. We estimate initial transform pa-rameters for each matched curve pair and then run thelocal optimization process. The best result is returned asthe global optimal set of parameter.

5.1 Local Optimization

We use an iterative optimization process to find the bestlocal transform parameters. At each step of the optimiza-tion, the moving image is transformed by the currenttransform parameters and resampled on the referenceimage’s grid. We implemented two different resamplingschemes: nearest neighbor interpolation and bilinear in-terpolation (bilinear interpolation is used in the resultssection). After the two images are aligned we computethe set of pixel locations Ω that are inside confidence re-gions in both input images. A similarity value can thenbe computed using either a mean square metric, a mu-tual information metric, or normalized cross correlation.The mean square metric,

MS(R,M) =1N

∑

i∈Ω

(Ri −Mi)2 (6)

calculates the L2 norm of the intensity differences. Themutual information metric,

MI(R, M) =∫

R

∫

M

p(x, y)logp(x, y)

p(x)p(y)dxdy (7)

compares the probability density function of the sepa-rate and joint distribution for each intensity pair. Thenormalized cross-correlation

NC(R, M) =∑

i∈Ω(RiMi)√∑i∈Ω(R2

i )∑

i∈Ω(M2i )

(8)

is another similarity metric that is similar to the meansquare metric, but it is insensitive to scaling the inten-sity of one or both images with a multiplicative factor.We use a gradient descent method for the optimizationprocess, because we already have an initial estimate. Ourimplementation follows the description in the book In-troduction to Optimization [15], pages 115− 122 . Morecomplicated global search strategies exist, such as geneticalgorithms [6], simulated annealing [23] and stochasticgradient algorithms [8]. These strategies are used in casethe initial guess is far from the global optimum. In con-trast, our registration scheme computes a good initial es-timate through geometric curve matching and smoothesthe search space through the use of confidence regionsto make a local search sufficient. Given a matched curvepair, we can estimate an initial guess for the transformas follows: 1) The translation parameter is estimated bythe difference between the centroids of the two regions.2) The scaling parameter can be estimated by the differ-ence between the areas of the matched regions. 3) Therotation parameter is estimated by the circular horizon-tal shift between the two matched curve’s CSSI.

6 Results

In this section we evaluate the curve matching algorithmand the complete image registration framework on se-lected test cases.

To evaluate the curve matching algorithm we are us-ing eight test cases. The first two test cases are selectedcurves from the Kimia shape database [39]. The other sixtest cases are sets of curves extracted from aerial images.We compare our curve matching algorithm to three otheralgorithms: the original CSS algorithm [31], the originalfourier descriptor methods [7], and a recent improvedfourier descriptor method [11]. The results are shown infigure 10. We can see that our method outperforms allother methods in terms of matching performance with allmethods having similar running times. As we will showbelow we can improve the performance even more byincreasing the sampling resolutions of the curves. It isimportant to note that both fourier descriptor methodscannot improve the performance with more expensivesettings. To give a visual impression of our test cases andresults, we show three examples: curves from the Kimashape database used in test1 (see figure 7) and curvesextracted from aerial images used in test3 (see figure 8)and test4 (see figure 9).

We conducted other tests and varied the two mostimportant parameters of the curve matching algorithm:


Fig. 7 The results of the curve matching algorithm on theKimia shape database. In the top row we show a curve match-ing result for the standard CSS algorithm and in the secondrow we show our results. Each row contains the same im-age pair with nine shapes per image. The label assignment iscomputed by the curve matching algorithms. Note that bothalgorithms miss classify two shapes.

Fig. 8 Results of the curve-matching algorithm for real aerialimages. The first row is the original image pair. The secondrow is the result of our curve matching algorithm. We showthe extracted curves for both images. The computed pairingis indicated by the numbers next to the curves. The third rowis the result of the original curve matching algorithm.

Fig. 9 see figure 8

Fig. 10 This table compares our improved CSSI based curvematching algorithm with three other published algorithms.FD1 is the original fourier descriptor method and FD2 is arecent improvement of fourier descriptors. OCSS denotes theoriginal scale space algorithm and NCSS is our new methoddescribed in this paper.

Fig. 11 This figure visualizes the search space spannedby the rotation and scaling parameters. Left: search spacespanned by confidence regions. Right: Search space of theoriginal image. Note that the confidence regions result in asmoother search space.


Fig. 12 This figure visualizes the search space spanned bythe translation parameters. Left: search space spanned byconfidence regions. Right: Search space of the original image.Note that the confidence regions result in a smoother searchspace.

the sampling resolution and the number of features used.We evaluated the influence of the sampling resolution sralong the arc-length of a curve by comparing the settings64, 128, 256, and 512 (setting 128 is used for all othertests in this paper). We also changed the number of fea-tures we used to describe a line in the CSSI. The mainmethod uses three features: angle, first moment, and sec-ond moment. This setting corresponds to the columnswhere F = 3. We compare this setting to F = 4 wherewe add the mass (zeroth moment) and F = 2 wherewe drop the second moment. The features are describedin section 4. The results of the comparison is given infigure 13. Please note that our method can improve thematching performance with a higher sampling resolutionof 256 and that the algorithm performs best with threefeatures.

To evaluate the complete registration framework, wecompare our registration scheme to four other methods:Sift–probably the most popular feature based method [37],UCSB [38]–registration with fit assessment, a differentvariation of feature based matching, Area MS [5]–anarea based method using the mean squares as metric,and Area MI [9,42]–an area based method using mu-tual information. We also show results for two settingsof our new algorithm. The main method MM uses meansquares as a matching function and NC uses normalizedcross correlation. We also tested the mutual informationmetric, but this version of our algorithm sometimes hasnot enough sample points inside the confidence regionsto work well. We created a test suite of eleven imagepairs. Figures 8, 9, and 14 show image pairs 1, 2, and3 respectively. The image pairs 4, 5, 6, and 7 are shownin figure 16. The image pairs 8,9,10,and 11 are shownin figure 17. The most important evaluation is a visualcomparison of the matching result. Typically, all of themethods provide either excellent results, or results thatare totally off which makes visual matching easy (seefigure 14 for an example). In the tests we recorded if amethod was able to find the right transformation andhow much time the computation took. The results aregiven in figure 15. Please note that we cannot reportmeaningful times for the UCSB method, because it wasused over a web interface. The results show that feature

Fig. 14 First row: input images. Second row: left, result ofan area-based method; right, result of our method. To showthe visual result for the registration, the second input imageis laid on the first one through the transform we got fromthe corresponding algorithm. Then we calculate the intensitydifference pixel by pixel. If the difference is equal to zero, weset the corresponding pixel in the result image to 125, whichis gray. If the difference is 255, we set it to 255. If the differ-ence is -255, we set it to 0. Other values can be interpolatedlinearly. If the two images are perfectly registered, then theoverlapping part result image should be uniformly gray.

based methods are much stronger then the area-basedones. However, we can see that the feature based meth-ods are not able to cope with blurred images, while ournew algorithm is still successful. The downside of our ap-proach is that it does not work well with gradient images,such as the one selected in our test case.

Compared to the area based methods, we have amuch stronger initial estimate to start a local search. Ad-ditionally, the restriction to confidence regions results ina much smoother search space. We visualize the searchspace of the image pair in figure 8 in figures 11 and 12.Since high dimension space is hard to illustrate, we onlyshow the space spanned by two parameters at one time.In figure 11 the space spanned by the rotation param-eter and scaling parameter for fixed translation and infigure 12 the space spanned by the translation parameterin X and Y direction.


Fig. 13 This table compares various parameter settings for the curve matching algorithm. We compare combinations of fourdifferent sampling resolutions 64, 128, 256, and 512, with different combinations of used features. F = 2 only uses distanceand angle, F = 3 uses the second moment as additional feature, and F = 4 additionally uses mass. Note that the moreexpensive resolution 256 clearly outperforms fourier descriptors (see figure 10)

Fig. 15 This figure gives a comparison of multiple regis-tration techniques. We compare two feature based methods(Sift, UCSB), two area based methods (AREA MS, AREAMI) with two versions of our new algorithm (MM, NC). Thealgorithms are described in the text. We report results on 11image pairs by describing if the registration was successful (yor n) and by reporting the computation time in seconds.

7 Conclusions and future work

In this paper, the problem of remote sensing image regis-tration is attacked. This paper makes two contributionsto the state of the art in image registration. First, weintroduce a new hybrid image registration scheme thatcombines the advantages of both area-based methodsand feature-based methods. We extract curves from the

Fig. 16 Several image pairs from our test suite not shownin other figures.


Fig. 17 Several image pairs from our test suite not shownin other figures.

two input images through image segmentation and per-form feature-based image registration: we compare andmatch these curves to obtain several initial parametersfor an area-based registration method. Then we use ex-isting area-based methods in confidence regions to obtaina final result. This scheme allows us to combine feature-based and area-based registration in a robust and effi-cient manner. The second contribution of this paper is anew curve-matching algorithm based on curvature scalespace to facilitate the registration scheme. We improvethe matching algorithm of two scale space images andthereby the robustness and accuracy of the algorithm.

Currently our curve-matching algorithm works on theassumption that the same physical object will have thesame shape contour in two input images. This is truefor input images got from the same sensor and samespectrum. However, in multi-spectral images the samephysical object may have different shape contours in dif-ferent spectral images. Our future work focuses on howto extend our registration scheme to multi-spectral im-age registration.

References

1. Sadegh Abbasi, Farzin Mokhtarian, and Josef Kittler.Enhancing css-based shape retrieval for objects with shal-low concavities. Image Vision Comput., 18(3):199–211,2000.

2. A. A. Amini, T. E. Weymouth, and R. C. Jain. Us-ing dynamic programming for solving variational prob-lems in vision. IEEE Trans. Pattern Anal. Mach. Intell.,12(9):855–867, 1990.

3. Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin. Anautomatic image registration for applications in remotesensing. Geoscience and Remote Sensing, IEEE Trans-actions on, 43:2127 – 2137, 2005.

4. L. G. Brown. A survey of image registration techniques.ACMComput. Surv., 24:325–376, December 1992.

5. Lisa Gottesfeld Brown. A survey of image registrationtechniques. ACM Computing Surveys, 24(4):325–376,1992.

6. Prachya Chalermwat. High-performance automatic im-age registration for remote sensing. PhD thesis, GeorgeMason University, 1999.

7. R. Chellappa and R. Bagdazian. Fourier coding of imageboundaries. IEEE Trans. Pattern Anal. Mach. Intell,6:102C105, 1984.

8. K.L.; LeMoigne J.; Zavorin I. Cole-Rhodes, A.A.; John-son. Multiresolution registration of remote sensing im-agery by optimization of mutual information using astochastic gradient. Image Processing, IEEE Transac-tions on, 12:1495 – 1511, December 2003.

9. Maes F. Delaere D. Vandermeulen D. Suetens P. Col-lignon, A. and G. Marchal. Automated multimodalityimage registration using information theory, 1995.

10. Pan Yun-he Cui Ming, Sun Shou-qian. An image re-gion merging algorithm based on modified fast watershedtransform. Journal of Computer-Aided Design & Com-puter Graphics, China, 17:546–552, March 2005.

11. Guojun Lu Dengsheng Zhang. A comparative study offourier descriptors for shape representation and retrieval.In Proc. of 5th Asian Conference on Computer Vision(ACCV), Melbourne, Australia, 2002.

12. L. Ding, T. C. Kularatna, A. Goshtasby, and M. Satter.Volumetric image registration by template matching. InK. M. Hanson, editor, Proc. SPIE Vol. 3979, p. 1235-1246, Medical Imaging 2000: Image Processing, KennethM. Hanson; Ed., volume 3979 of Presented at the Soci-ety of Photo-Optical Instrumentation Engineers (SPIE)Conference, pages 1235–1246, June 2000.

13. M. Holden D.J. Hawkes D.L.G. Hill, P.G. Batchelor.Medical image registration. Physics in Medicine and Bi-ology, 46:1–45, 2001.

14. Rakesh Kumar. edited by Mubarak Shah. Video Regis-tration. Boston : Kluwer Academic Publishers, Boston,2003.

15. Stanislaw H. Zak Edwin Kah Pin Chong. An Introductionto Optimization. John Wiley & Sons, Inc., 2001.

16. J Flusser and T Suk. A moment-based approach toregistration of images with affine geometric distortion.Geoscience and Remote Sensing, IEEE Transactions on,32:382–387, March 1994.

17. Chen Fulong, Zhang Hong, and Wang Chao. A novelfeature matching method in airborne sar image regis-tration. In Geoscience and Remote Sensing Symposium,2005. IGARSS’05. Proceedings, pages 4722– 4724, 2005.

18. A. Goshtasby. Description and discrimination of pla-nar shapes using shape matrices. IEEE Trans. PatternAnal.Mach. Intell., 7:738–743, 1985.

19. A. Saghri H. Freeman. Generalized chain codes for planarcurves. In Proceedings of the Fourth International JointConference on Pattern Recognition, pages 701–703, Ky-oto, Japan, 1978.


20. M. McGuire H.S. Stone, J. le Moigne. The translationsensitivity of wavelet-based registration. IEEE Trans-actions on Pattern Analysis and Machine Intelligence,21:1074–1081, 1999.

21. M.K. Hua-Mei Chen Varshney, P.K. Arora. Performanceof mutual information similarity measure for registrationof multitemporal remote sensing images. IEEE Trans.Pattern Analysis Machine Intell., 24:603–619, May 2002.

22. Chin-Chen Chang; I-Yen Chen; Yea-Shuan Huang. Pull-in time dynamics as a measure of absolute pressure.In Pattern Recognition, 2002. Proceedings. 16th Interna-tional Conference on, pages 386 – 389, August 2002.

23. Maruc Johansson. Master of Science Thesis, Stockholm,Sweden, 2006.

24. Harold W. Kuhn. The hungarian method for the as-signment problem. Naval Research Logistic Quarterly 2,pages 83–97, 1955.

25. J. le Moigne. First evaluation of automatic image reg-istration methods. In Proceedings of the InternationalGeoscience and Remote Sensing Symposium IGARSS98,pages 315–317, Seattle, Washington, 1998.

26. H. Li, B. S. Manjunath, and S. K. Mitra. A contour-basedapproach to multisensor image registration. IEEE Trans-actions on Image Processing, 4:320–334, March 1995.

27. B.S. Manjunath L.M.G. Fonseca. Registration techniquesfor multisensor remotely sensed imagery. Photogram-metric Engineering and Remote Sensing, 62:1049–1056,1996.

28. R. Boyle M. Sonka, V. Hlavac. Image Processing, Analy-sis and Machine Vision. Chapman & Hall, London, UK,NJ,.

29. J. Maintz and M. Viergever. A survey of medical imageregistration. Medical Image Analysis, 2(1):1–36, 1998.

30. T. McInerney and D. Terzopoulos. Deformable modelsin medical images analysis: a survey, 1996.

31. F Mokhtarian. Silhouette-based isolated object recogni-tion through curvature scale space. IEEE Transactionson Pattern Analysis and Machine Intelligence, 17:539–544, May 1995.

32. Farzin Mokhtarian and Miroslav Bober. Curvature ScaleSpace Representation: Theory, Applications, and MPEG-7 Standardization. Kluwer Academic Publishers, Norwell,MA, USA, 2003.

33. Mackworth A Mokhtarian F. A theory of multi-scale,curvature-based shape representation for planar curves.IEEE Trans. Pattern Anal. Machine Intell., 14:789–805,August 1992.

34. M. Unser P. The venaz, U.E. Ruttimann. Iterative multi-scale registration without landmarks. In PProceedings ofthe IEEE International Confernece on Image ProcessingICIP95, pages 228–231, Washington DC, 1995.

35. Stephen M. Pizer, P. Thomas Fletcher, Sarang Joshi, An-drew Thall, James Z. Chen, Yonatan Fridman, Daniel S.Fritsch, A. Graham Gash, John M. Glotzer, Michael R.Jiroutek, Conglin Lu, Keith E. Muller, Gregg Tracton,Paul Yushkevich, and Edward L. Chaney. Deformablem-reps for 3d medical image segmentation. Int. J. Com-put. Vision, 55(2-3):85–106, 2003.

36. J.-P. Thirion. Image matching as a diffusion process: ananalogy with maxwell’s demons. Medical Image Analysis,2:1–20, 1998.

37. UCLA. Sift implementation in matlab/c.http://vision.ucla.edu/ vedaldi/code/sift/sift.html.

38. UCSB. Feature based registration platform.http://nayana.ece.ucsb.edu/registration/.

39. Brown university. Kimia shape database.http://www.lems.brown.edu/vision/software/index.html.

40. B. C. Vemuri, J. Ye, Y. Chen, and C. M. Leonard. Imageregistration via level-set motion: applications to atlas-based segmentation. Med Image Anal, 7(1):1–20, March2003.

41. David B. Wagner. Dynamic programming.http://citeseer.ist.psu.edu/268391.html.

42. W. Wells, P. Viola, H. Atsumi, S. Nakajima, and R. Kiki-nis. Multi-modal volume registration by maximization ofmutual information, 1996.

43. S. Xiaolong Dai; Khorram. A feature-based image regis-tration algorithm using improved chain-code representa-tion combined with invariant moments. Geoscience andRemote Sensing, IEEE Transactions on, 37:2351 – 2362,September 1999.

44. D. Zhang and G. Lu. Review of shape representationand description technique. Pattern Recognition, 37:1–19,January 2004.

45. Q. Zheng and R. Chellappa. A computational vision ap-proach to image registration. IEEE Transactions on Im-age Processing, 2:311–326, July 1993.

46. B. Zitova and J. Flusser. Image registration methods: Asurvey. Image Vis. Comput., 21:977–1000, April 2003.

Ming. Cui Ming Cui works atArizona State University.

Documents

A New Image Registration Scheme Based on Curvature Scale