7/27/2019 05945566

1/6

Fast block matching algorithms using frequency

domainDriss Aboutajdine and Fedwa Essannouni

LRIT, Laboratoire associe au CNRST, Faculte des Sciences,

Universite Mohamed V-Agdal, B.P. 1014 Rabat, Morocco.aboutaj@fsr.ac.ma, efedwa@yahoo.fr

Abstract Translational template matching addresses theregistration problem and has long been a problem of interest insuch areas as video compression, robot vision, and biomedicalengineering. In this paper, we will present new fast algorithmsdeveloped in our group for template and block matching purposesusing either second or higher order statistics. Fast Fouriertransforms (FFTs) have been called one of the ten most importantalgorithms of the twentieth century. Using some substitutions andcomplex arithmetic, computation of the sum square differences

sum absolute difference and sum fourth order moment arederived to be correlation functions of substituting functions.The former can be computed using the fast Fourier transform(FFT) approach, which is greatly less computationally expensivethan the direct computation. The performance of the proposedmethods, as well as some illustrative comparisons with othermatching algorithms in the literature, are verified throughsimulations. The algorithm based on the higher order momentis seen to have better performance in terms of fastness androbustness even in low SNR meanwhile the extra computationalcost is negligible.

I. INTRODUCTION

Template matching is a common tool in many applications,including object recognition, video compression, stereo

matching, and feature tracking. It has been used for

video applications other than sequence coding such as

motion compensated spatial temporal interpolation motion-

compensated enhancement. Many applications of image

matching use the sum-of-square differences (SSD) measure

to determine the best match. The SSD is the most commonly

used metric. Indeed, the correlation can be used to evaluate

the SSD criterion effortlessly and the straightforward way to

compute the correlation is to use the fast Fourier transform

(FFT) [1][3] using many correlation. In [4], we have solved

the problem of non cyclic SSD matching using only one

correlation operation. The SAD (sum of absolute differences)

is another commonly used similarity metric. However, unlike

the SSD metric, the algorithms which have been developed

to speed up the process of sum of absolute difference (SAD)

matching are designed exclusively in the spatial domain.

Although there are many approaches whose objective is to

speed up the process of SAD matching, they can only give

the position of the SAD minimum. When the distance SAD

should be calculated for every location in the image, the

direct SAD computation requires more much time. In [5], we

have proposed to expedite this naive approach using the FFT

transform while exploiting the Fourier correlation theorem.

It is usually supposed in template and block matching that

the regions of interest do not change their appearances very

much. Hence, the sum of absolute difference (SAD) or the sum

of squared difference (SSD), are the metrics commonly used

in template and block matching. Using higher than second-

order statistics is appropriate in case the image sequence or

the signal is severely corrupted by additive Gaussian noise

[6][9]. Recently, cumulants have gained popularity in multi-

dimensional theory and applications. Motivated by the noise

insensitivity of cumulant-based time delay estimators, in [10],

the authors have introduced a fourth-order cost function of

the displaced frame difference (DFD) capable of estimating

motion even for small regions or blocks. Later a fourth-order

cost function of the displaced frame difference (DFD) capable

of estimating motion even for small regions or blocks has

been introduced in [11]. In [12], we have proposed a new

algorithm based on the fourth order moment. The use of

matching criteria based on the fourth order moment is suitable

for image and video sequences under an additive Gaussian

noise and is better suited to the analysis of complex noisy

image sequences contaminated by correlated Gaussian noise(CGN).

This paper is organized as follows: Section II introduces the

problem statement. Section III is concerned with the frequency

method for SAD template matching computation. Section IV

describes our method for SSD computation using only one

correlation. The fourth order moment computation using FFT

algorithms is described in section V. Section VI discuses the

computational cost of different frequency matching algorithms

presented in this paper. Some simulations results are shown in

section VII. Section VIII concludes the paper.

II . PROBLEM STATEMENT

Assume a 2D m n pattern or block, g(x, y), is to bematched within an image or search area f(x, y) of size N2,where (N > m and N > n).For each pixel location (x, y) in the image, the block distortionmeasure (BDM) is calculated as follows:

BDM(x, y) =n1l=0

m1k=0

(dx,y(k, l)) , (1)

where dx,y(k, l) = f(x + k, y + l) g(k, l), the values f(k, l)and g(k, l) denote either luminance or other real features.

978-1-61284-732-0/11/$26.00 2010 IEEE

7/27/2019 05945566

2/6

When the distance BDM should be calculated for every

location in the image, the function must be applied (Nm + 1)(N n + 1) times which is very expensive especiallyfor large images.

The motion vectors can be estimated by using a measure of

matching error. The matching error between the block g and

the candidates blocks in a search area f, is usually defined as

the sum of absolute difference (SAD) or the sum of absoluteof difference (SSD):

SAD(x, y) =n1l=0

m1k=0

|dx,y(k, l)| , (2)

SS D(x, y) =n1l=0

m1k=0

|dx,y(k, l)|2

. (3)

III. SAD COMPUTATION USING FF TS

In this section we are concerned with the absolute kernel

function: (x) = |x|Using the Fourier cosine series, we can approximate the

absolute kernel function defined only on the interval [, ].

L(x) =

2

4

Lp=1

1

(2p 1)2cos((2p 1)x) . (4)

The main idea to get the approximation of the SAD surface

matching using Fourier cosine series is to normalize the values

of the luminance or other features to the interval [0, ] andthen use the Fourier cosine series of the absolute function.

Taking this into consideration and using the approximation of

the absolute function given by (4) to the value dx,y(k, l) weobtain:

|dx,y(k, l)|

2

4

Lp=1 1(2p 1)2 cos((2p 1)dx,y(k, l)) .(5)

Finally, the approximation of the SAD metric (1) at L level

using Fourier cosine series can be given as follows:

SADL(x, y) =

2mn

4

L

p=1

1

(2p 1)2n1l=0

m1k=0

cos((2p 1)dx,y(k, l))

.(6)

If we note:

SC DL(x, y) = L

p=11

(2p 1)2

n1

l=0m1

k=0 cos ((2p 1)dx,y(k, l)) .(7)

Then using (6) the approximation to SAD at L level can be

written as follows:

SADL(x, y) =

2mn

4

SC DL(x, y). (8)

From this formula, we can see that minimizing SADL corre-

sponds to the maximization of SC DL. On the other hand, the

SC DL function can be viewed as a series of cross correlation

operations. Indeed, if we note that:

gp(x, y) = exp(j(2p 1)g(x, y)), (9)

fp(x, y) = exp(j(2p 1)f(x, y)), (10)

where j is the square root of1 and exp(jx) is the complexexponential, using the Eulers identity cos(x) = (exp(jx)),the SC DL can be written as:

SC DL(x, y) =

Lp=1

1

(2p 1)2n1

l=0m1

k=0g

p(k, l)fp(k + x, l + y)

.

(11)

From the Fourier correlation theorem, the SC DL surface (11)

can be computed using FFT algorithms as follows:

SC DL(x, y) =

I F F T L

p=1

1

(2p 1)2Fp(u, v)G

p(u, v) .

(12)

where Gp is the F F T of gp and Fp is the F F T of fp, denotes the real part of a complex number and the asterisk

denotes complex conjugation.

Note that fp and gp are correlated with FFTs by zero padding

the size ofgp to the size offp prior to taking the forward FFTs.There are two different reasons for zero padding. Firstly, in

order that the elements multiplication could occur, they must

be of the same size. Secondly to avoid cyclic correlation.

The last m 1 rows and n 1 columns of the result willcontain wrap-around data that should be discarded. Finally the

resulting position (mvx, mvy) of the pattern g in the image fis measured from the position of the maximum in (12).

The computation of the SAD using the frequency domain

is given as follows:

SADL =

2

mn4

I F F T

L

p=11

(2p 1)2Fp(u, v)Gp(u, v) .(13)

IV. SSD COMPUTATION USING ONE CORRELATION

In [4], we have proposed a new correlation with an exact

performance as the direct full search under SSD metric and

without any approximation.

This correlation is based on a simple remark. Indeed if we

note:

fc(x, y) = f2(x, y) + j f(x, y), (14)

gc(x, y) = 1 2j g(x, y) (15)

where j is the square root of1

From the definition of fc and gc, we can simply remark that:

(fc(x, y) g

c (x, y)) = f2(x, y) 2g(x, y)f(x, y). (16)

Thus the SSD metric can be rewritten as:

SS D(x, y) = (n1l=0

m1k=0

fc(k + x, l + y) g

c (k, l))

+n1

l=0m1

k=0g2(k, l). (17)

7/27/2019 05945566

3/6

Fig. 1. A simple diagram for the proposed fourth moment computation.

Using the Fourier complex cross correlation theorem, the

surface presented in (17) can be also computed using FFT

algorithms as follows:

SS D(x, y) = (n1l=0

m1k=0

gc (k, l) fc(k + x, l + y))(18)

+n1

l=0

m1

k=0

g2(k, l). (19)

where Gc is the F F T of gc and Fc is the F F T of fc.The resulting motion vector (mvx, mvy) of the block g in thesearch area f is measured from the position of the maximum

in (19).

V. THE FOURTH CENTRAL MOMENT COMPUTATION USING

FREQUENCY DOMAIN

In probability theory and statistics, the kth moment about

the mean (or kth central moment) of a real-valued random

variable X is the quantity:

Mk = E[(X E[X])k],

where E is the expectation operator.

The first central moment is zero. The second moment about

the mean is called the variance, and is usually denoted 2,

where represents the standard deviation. The third and fourth

moments about the mean are used to define the standardized

moments which are used to define skewness and kurtosis,

respectively.

The kurtosis K is defined to be the normalized fourth central

moment minus 3. Some authors do not subtract three, but itis usually more convenient to have the normal distribution at

the origin of coordinates. If a distribution has a peak at the

mean and long tails, the fourth moment will be high and the

kurtosis positive; and conversely; thus, bounded distributions

tend to have low kurtosis.

The fourth central moment is a measure of whether the

distribution is tall and skinny or short and squat, compared

to the normal distribution of the same variance. Since it is

the expectation of a fourth power, the fourth central moment,

where defined, is always positive (except for a point distribu-

tion), the fourth central moment of a normal distribution

is 3 4.

In matching purposes, HOS-based cost functions can be

built from different criteria. In [11], the authors have also makeuse of this criterion for motion estimation and block matching

purposes. Their approach is based on a fourth-order statistics

cost function that utilizes the kurtosis of the dx,y(k, l), whichis asymptotically unaffected by correlated Gaussian noise.

In order to take into account the presence of noise, we have

introduced in [12], the cost function based on the fourth central

moment given by:

M4c(x, y) = M(dx,y(k, l)), (20)

where the fourth central order moment is defined, as follows:

M(dx,y(k, l)) = E

(dx,y(k, l) E{dx,y(k, l)})4

. (21)

Since the fourth central moment of a normal distribution is

34, M4c is insensitive to Gaussian noise whos variance isconstant.

If we consider the cost function based on the fourth central

moment, for each pixel location (x, y) in the image, thematching surface (21) can be expressed as follows:

M4c(x, y) =1

mn

n1

l=0m1

k=0 (f(x + k, y + l) g(k, l))

E{dx,y(k, l)} 4

.

(22)

7/27/2019 05945566

4/6

The correct displacement is found by minimizing M4c(x, y).To omit the term E{dx,y(k, l) from this equation, the searcharea f and the block g must become zero mean as follwing:

f(x, y) = f(x, y)

1

mn

n1l=0

m1k=0

f(x + k, y + l) (23)

g(x, y) = g(x, y) 1mnn1l=0m1k=0

g(k, l). (24)

The termn1l=0

m1k=0

f(x+k, y+l) can be efficiently calculated

on the whole image beforehand. They can be calculated

through recursive algorithms very efficiently [13].

Let f and g be respectively the search area f and the pattern

g with zero mean as described in (23).

Let fc1, fc2 and gc1, gc2 be four complex signals obtained as

follows:

fc1(x, y) = f4(x, y) + 2j f3(x, y), (25)

fc2(x, y) = f2(x, y) + j f(x, y), (26)

and

gc1(x, y) = 1 2j g(x, y),

gc2(x, y) = 6g2(x, y) 4jg3(x, y), (27)

where the pixel values f(x, y) and g(x, y) denote eitherluminance or other real features.

On the other hand, from the definition of fc1 and gc1 and

from the definition of fc2 and gc2 we also obtain:

fc1(x + k, y + l)gc1(k, l) = f4(x + k, y + l)4f3(x + k, y + l)g(k, l),

fc2(x + k, y + l)

gc2(k, l)

= 6f2(x + k, y + l)g2(k, l)

4f(x + k, y + l)g3(k, l).

The fourth moment of dx,y(k, l) can be rewritten as:

M4c(x, y) =1

mn

n1l=0

m1k=0

(fc1(x + k, y + l) g

c1(k, l))

+1

mn

n1

l=0m1

k=0 (fc2(x + k, y + l) g

c2(k, l))

+1

mn

n1l=0

m1k=0

g4(k, l). (28)

This latter can be viewed as the summation of two spatial

complex correlation between fc1, gc1 and fc2 , gc2. By

using again the Fourier complex cross correlation theorem,

the surface presented in (28) can be computed using FFT

algorithms as follows:

M4c(x, y) =1

mn

I F F T

Fc1(u, v)G

c1(u, v)+Fc2(u, v)G

c2(u, v)

+n1

l=0m1

k=0g4(k, l)

(29)

where Gc1, Gc2, Fc1, Fc2 are the F F T of gc1, gc2, fc1, fc2 ,

respectively.

Figure 1 show simple flow chart of the computation of these

two correlation methods for the computation of the fourth

moment.

VI. COMPUTATIONAL COST

Much of the computational cost of the frequency methods

is with the Fourier transform. The frequency SAD algorithm

requires 2L forward and one backward complex FFTs of sizeN2

besides the multiplications involved by equations (9)

and (10) and by the normalization of the image intensities to

the range [0 : ]. These operations, however, may be avoidedusing lookup tables.

On the other hand, the SSD computation using FFTs

requires 2 forward FFTs and one backward FFT of sizeN2, meanwhile the algorithm of the fourth order moment

computation requires 4 forward and one backward FFTs of

size N2 .Using the recent algorithm published in [14], which is

a simple recursive modification of the split-radix algorithm

that computes the DFT with asymptotically about 6% feweroperations, the arithmetic operations of the computation of the

fourth order moment is given in table I.

From this table, it can be seen that as the pattern size increases,

the advantage of the Fourier approach using each of the metrics

becomes more prominent.

VII. SIMULATION RESULTS

The experiments are performed on a Pentium IV processor

3,06 GHz and for FFT computation, the library FFTW3 isused [15]. We compare between the methods discussed above:

(SSD, SAD and fourth order moment M4) . We have to note

that in these simulations we distinguish between the motion

estimation based on the fourth order moment with a zero mean

d(x, y)(M4C) and with no zero mean (M4).In this experiment, patterns of size 32 x 32 are extracted

from the frames in the football sequence. Search areas of

size 64x64 which are likely to contain the selected patterns

are chosen from the successive frame with a step of 2. The

patterns are tracked using both of the direct SAD search and

the proposed frequency fourth order moment method and the

frequency SSD search. In this experiment three aspects are

compared 1) Time ratio with respect to the full search (ES),

2) Average distance from the true motion vector per pattern,

and 3) Probability of finding the true motion vector. Note that

the true motion vectors are regarded as those found in ES

under the distance SSD considered in the absence of the noise.

These aspects show how are the vectors far from the true ones

and also the percentage of finding the true motion vectors.

These measures permit to know the influence of the noise on

the motion vectors and so on the metric used.

Figures 2, 3 show the results over 10 frames from the

Football sequence by adding a Gaussian white noise of mean

0 and variance v as parameter of experiment to the search

area.

7/27/2019 05945566

5/6

TABLE I

FLOP COUNTS (REAL ADDITIONS + MULTIPLICATIONS ) OF THE PROPOSED FREQUENCY METHODS AND THE DIRECT SEARCH ( FULL SEARCH).

Pattern size Pattern size=32x32 Pattern size=64x64

Search area size 64x64 128x128 256x256 128x128 256x256 512x512

(SSDcorr/SSD FS)% 11.22% 6.15% 5.29% 3,43% 1,80% 1,51%SSD SSD Full Search 4460544 38539264 207360000 69222400 610287610 3303030780

SSD correlation 500574 2373798 10980849 2373798 10980849 49865529

(SADcorr/SAD FS)% 24,94% 13,69% 11,77% 7,62% 4,00% 3,35%

SAD (L=2) SAD Full Search 33454 08 289044 48 15552 0000 519 16800 45 77157 10 24772 73080SAD correlation (L=2) 83429 0 39563 30 18301 415 39 56330 1 830141 5 8310 9215

(M4corr/L4FS)% 12,47% 6,84% 5,88% 3,81% 2,00% 1,68%M4 M4 Full Search 6690816 57808896 311040000 103833600 915431420 4954546170

M4 correlation 834292 3956334 18301416 3956330 18301415 83109215

The method based on the central fourth order moment gives

almost higher probability of finding the same vectors, than the

exhaustive SAD search and the SSD one. Also distance found

by the central fourth order moment based method is almost

less than the ones found by the other methods.

VIII. CONCLUSION

The motion estimation based on the fourth central moment

is more robust than the classical SSD and SAD matching

errors. Using only the complexity of two complex correla-

tions, we can get the same results as the direct fourth order

moment matching based method while highly decreasing the

computational cost especially for large images. The methods

presented in this paper can be used for any image and

signal processing problem which requires the translational

displacement estimation.

REFERENCES

[1] C.-K. Chen, J. Duluk, and J. F., System and method for crosscorrelation with application to video motion vector estimator,no. U. S. Patent 5,535,288, July 1996. [Online]. Available:http://www.freepatentsonline.com/5535288.html

[2] F. Essannouni, R. O. H. Thami, A. Salam, and D. Aboutajdine, A newfast full search block matching algorithm using frequency domain, in

ISSPA05 (Proceedings o f the Eighth International Symposium on Signal

Processing and Its Applications), vol. 2, Sydeny, Australia, 2005, pp.559 562.

[3] Kilthau, S. Drew, and T. M.S. Moller, Full search content independentblock matching based on the fast fourier transform, in ICIP, 2002, pp.I669 I672.

[4] F. Essannouni, R. Oulad Haj Thami, D. Aboutajdine, and A. Salam,Simple noncircular correlation method for exhaustive ssd matching,Optical Engineering Journal, vol. 46, Octobre 2007.

[5] , Adjustable sad matching algorithm using frequency domain,Journal of Real-Time Image Processing, vol. 1, no. 4, pp. 257265,July 2007.

[6] M. Bakrim and D. Aboutajdine, Higher-order statistics based blindestimation of non-gaussian bidimensional moving average models,Signal Process., vol. 86, no. 10, pp. 30313042, 2006.

[7] M. El Hassouni, H. Cherifi, and D. Aboutajdine, Hos-based imagesequence noise removal, IEEE Transactions on Image Processing,vol. 15, no. 3, pp. 572581, March 2006.

[8] S. Thil, H. Garnier, and M. Gilson, Third-order cumulants basedmethods for continuous-time errors-in-variables model identification,

Automatica, vol. 44, no. 3, pp. 647658, 2008.[9] D. Aboutajdine, A. Adib, and A. Meziane, Fast adaptive algorithms

for ar parameters estimation using higher order statistics, IEEE Trans.Sinal Processing, vol. 44, pp. 19982009, Aug 1996.

[10] J. Anderson and G. Giannakis, Image motion estimation algorithmsusing cumulants, IEEE Transactions on Image Processing, vol. 4, no. 3,

pp. 346357, March 1995.

[11] E. Sayrol, A. Gasull, and J. Fonollosa, Motion estimation using higher-order statistics, IEEE Transactions on Image Processing, vol. 5, no. 6,pp. 10771084, June 1996.

[12] F. Essannouni and D. Aboutajdine, Fast frequency template matchingusing higher order statistics. IEEE Transactions on Image Processing,pp. 826830, 2010.

[13] W. Li and E. Salari, Successive elimination algorithm for motionestimation, IEEE Transactions on Image Processing, vol. 4, no. 1, pp.

105107, January 1995.[14] S. G. Johnson and M. Frigo, A modified split-radix fft with fewer

arithmetic operations, IEEE Trans. Signal Processing, vol. 55, no. 1,pp. 111119, 2007.

[15] M. Frigo and S. G. Johnson, The design and implementation ofFFTW3, Proceedings of the IEEE, vol. 93, no. 2, pp. 216231, 2005,special issue on Program Generation, Optimization, and PlatformAdaptation.

7/27/2019 05945566

6/6

1 2 3 4 5 6 7 8 9 100

2

4

6

8

10

12

14

FRAMES

DISTANCE

SSD

SAD

M4c

M4

(a) variance of Gaussian noise=0.3

1 2 3 4 5 6 7 8 9 100

2

4

6

8

10

12

14

FRAMES

DISTANCE

SSD

SAD

M4c

M4

(b) variance of Gaussian noise=0.4

Fig. 2. The distance of the found vectors from the true ones using each of SAD, SSD and fourth order moment based method under an additive Gaussiannoise of mean 0 and variance v.

1 2 3 4 5 6 7 8 9 1025

30

35

40

45

50

55

60

65

FRAMES

PROBABILITY

SSD

SAD

M4c

M4

(a) variance of Gaussian noise=0.1

1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

70

FRAMES

PROBABILITY

SSD

SAD

M4c

M4

(b) variance of Gaussian noise=0.2

1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

70

FRAMES

PROBABILITY

SSD

SAD

M4c

M4

(c) variance of Gaussian noise=0.3

1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

FRAMES

PROBABILITY

SSD

SAD

M4c

M4

(d) variance of Gaussian noise=0.4

Fig. 3. Probability in % of finding the same motions vectors as in the absence of noise using each of SAD, SSD and fourth order moment based method.