Published on

14-Jul-2016View

212Download

0

Embed Size (px)

Transcript

aetw

Giuseppe Coppini a,*, Stefano Diciotti b, Guido Valli b

noise indicate considerable robustness against statistical variability. Applications to clinical images are presented.

preattentive processes to attentive tasks. A wide of paramount importance in modern medical

imaging. In the latter case, it is necessary to buildcomprehensive functional and anatomical repre-

sentations of the observed biological structures by

utilizing the partial views oered by basic imaging

procedures such as MRI, PET, CT (van den Elsen

qPartially founded by the Italian Ministry for Education,

University and Research (MIUR).*

Pattern Recognition Letters 25Corresponding author. Tel.: +39-503-153-480; fax: +39-

503-152-166. 2003 Elsevier B.V. All rights reserved.

Keywords: Image matching; Medical imaging; SOM neural network; Gabor wavelets

1. Introduction

Finding correspondences between images is a

central problem in many vision activities, from

category of image matching problems arises from

the need to integrate images generated by dierent

modalities. The importance of integration tasks

has rapidly grown in recent years and has becomea Consiglio Nazionale della Ricerche (CNR), Institute of Clinical Physiology, Via Moruzzi 1, 56124 Pisa, Italyb Department of Electronics and Telecommunications, University of Florence, Italy

Received 11 March 2003; received in revised form 21 October 2003

Abstract

A general approach to the problem of image matching which exploits a multi-scale representation of local image

structure and the principles of self-organizing neural networks is introduced. The problem considered is relevant in

many imaging applications and has been largely investigated in medical imagery, especially as regards the integration of

dierent imaging procedures.

A given pair of images to be matched, named target and stimulus respectively, are represented by Gabor Wavelets.

Correspondence is computed by exploiting the learning procedure of a neural network derived from Kohonens SOM.The SOM units coincide with the pixels of the target image and their weight are pointers to those of the stimulus images.

The standard SOM rule is modied so as to account for image features. The properties of our method are tested by

experiments performed on synthetic images. The considered implementation has shown that is able to recover a wide

range of transformations including global ane transformations and local distortions. Tests in the presence of additiveMatching of medical imneural nE-mail addresses: coppini@ifc.cnr.it (G. Coppini), diciotti@

asp.det.uni.it (S. Diciotti), valli@det.uni.it (G. Valli).

0167-8655/$ - see front matter 2003 Elsevier B.V. All rights reservdoi:10.1016/j.patrec.2003.10.012ges by self-organizingorks q

(2004) 341352

www.elsevier.com/locate/patrecet al., 1993). Several correspondence problems

have been faced which include mono-modal or

ed.

subject variability, nevertheless the observed

structures keep some general properties such asconnectedness and proximity among sub-struc-

tures. In general, it seems reasonable to search for

a correspondence law which is unique from T toS, 1 and which maps features in the T plane tosimilar features in the S plane by matching spa-tially contiguous features in T into spatially con-tiguous features in S. These conditions can berestated more concisely by saying that the desiredmapping is a feature- and topology-preserving

transformation from T into S. The importance oftopology preservation in image matching has been

ognition Letters 25 (2004) 341352multi-modal image correspondence and intra- or

inter-subjects matching. In addition, matching of

3D images has become an important issue along

with the mapping of imaged structures to reference

representations such as anatomical atlases. Much

eort has been devoted to nding global defor-mations, which has led to ecient algorithms

being able to recover linear transformations. A

popular method is based on the maximization of

the mutual information of the two images (Wells

III et al., 1996). More recently, the use of deform-

able models has been widely investigated so as to

recover local non-linear deformations. A number

of references on the subject can be found in(Maintz and Viergever, 1998). Thermodynamical

analogy with Maxwell demons has been also

investigated (Thirion, 1998).

In general, specic solutions have been pro-

posed for many matching problems. The related

algorithms usually exploit some form of prior

knowledge, and are often based on strong

assumptions. On the other hand many corre-spondence nding problems share important

common facets, such as the statement of image-

similarity criteria or the denition of adequate

computational schemes. In this view, the impor-

tance of general solutions for image matching

problems has been considered by several

researchers (Haralick and Shapiro, 1993). In par-

ticular, we believe that a self-organizing process isa natural and very promising setting. As reported

by Bellando and Kotari (1996), the computation of

topology preserving maps using Kohonen neural

networks (Kohonen, 2001) can provide valid

solutions to establishing image correspondence.

Wurtz et al. (1999) compare the behavior of Ko-

honens SOM with the Dynamic Link Architecture(Konen et al., 1994) paying particular attentionto the computational eciency of self-organizing

processes. It is worth mentioning that the appli-

cation of a SOM network to image registration is

described by Huwer et al. (1996).

In this work we will take the following general

problem into account. Let Itr and Isr0, twoimages (target and stimulus image, respectively)

with r x; y, and r0 x0; y0 co-ordinate vectorsdened in proper subsets T and S, respectively

342 G. Coppini et al. / Pattern Rec(image planes) of R2 space. We assume that afeature vector ftr ff it rg describing relevantproperties of Itr can be computed for each pointr in T . Similarly, fsr0 ff is r0g will indicate thefeature vector of Isr0 for each r0 2 S. We searchfor a correspondence rule M : T 7!S that mapspoints in T to points in S which have similarmorphological properties, as described by the

feature vectors (see Fig. 1). This is an ill-posed

problem, and further constraints must be consid-

ered to compute a useful solution, when stated so.We believe that powerful constraints can be de-

rived from the regularity of the observed world

even if the use of specic knowledge available for

each matching problem considered can lead to

adequate solutions. For example, images of nor-

mal anatomical structures exhibit a large inter-

tI (r)

sI (r)

r

r

T

Sstimulus image

target image

Fig. 1. A general matching procedure is expected to preserve

pictorial features and their neighborhood relationship.1 One-to-one mapping can be desirable in some cases.

given in the image planes I fT ; Sg and the fea-

ble image description to compute a useful set of

pictorial features f ff ig. This led us to investi-gate a self-organizing neural network coupled with

ognition Letters 25 (2004) 341352 343ture space F , respectively. Let us assume that forany given r1; r2 2 T , r01; r02 2 S are the correspond-ing transformed points, i.e. r01 Mr1, r02 Mr2. We will say that the transformation Mpreserves image topology if the following condi-

tions are met:

(1) Preservation of image plane topology:

limdI r1;r2!0

dIr01; r02 0

(2) Feature preservation:

limdI r1;r01!0

dF ftr1; fsr01 0

Though further constraints can be taken into

account (such as the high order smoothness of

the mapping), our aim is the investigation of the

preservation of the neighborhood relationship.

To this end we consider the computation of

image matching by using a topology preserving

neural network derived from Kohonens SOM.The resulting computation is typically data-drivenand no specic prior model is expected to con-

strain the obtained solution. In this sense our

approach is typically non-parametric. In order to

investigate the related capabilities, we outline the

basic scheme of the adopted computational para-

digm in the next section. Subsequently we present

and analyze the performances observed when a

set of known mathematical transformations wasapplied to phantom images in the presence of

additive noise. Applications to computing corre-

spondence between biomedical images are also

described.

2. Computational paradigm

Following the previous considerations, a gen-

eral framework of image matching problems re-

quires: (a) a proper computational architecture todiscussed by Musse et al. (2001) who propose the

use of hierarchical parametric models. A formal

denition of the meaning of topology preserving

transformation is useful for the following. To this

end we assume that proper metrics dI and dF are

G. Coppini et al. / Pattern Recimplement a topology preserving map, (b) a exi-image features computed by the Gabor Wavelet

Transform.

2.1. Matching through self-organization

The computation of topology preserving maps

between vector spaces can be carried out by using

Kohonen neural networks (Kohonen, 2001).

Basically, they include a grid of units, typically 2D,

each of which receives the same input vector

(stimulus) and operates on a winner-takes-allbasis. For a given input, the network weights are

changed according to the rule: (i) nd a winner

unit, and (ii) change the weights of the units in a

neighborhood of the winner unit, the size of the

neighborhood (dening the extent of lateral plas-

ticity) is a decreasing function of time. The relax-

ation process obtained by iteratively applying the

above rule for a sequence of input stimuli is thebasis of the training of ordinary Kohonen net-

works. In this work, the same process, properly

adapted, is exploited to compute the desired cor-

respondenceM. In our computational scheme, thepixels of one of the two images Itr (called thetarget image) correspond to the units of the Ko-

honen network. In general we assume the image to

be sub-sampled by using a N N grid of pixels,which provides a N N SOM. The weights of eachSOM unit can be interpreted as pointers to the

second image Isr0 (called stimulus image) whosecoordinates are the input of the network (Fig. 2).

In this way, the weight vectors directly give the

displacement needed to match point r with the cor-

responding point r0. For this reason, in the fol-lowing the network weight of the ith unit 2 isindicated byMri, while r0 is the input (or stimu-lus) vector. The standard Kohonens algorithmthat governs the updating of network weights in

response to a stimulus r0 presented at time t is asfollows:

2 Without loss of generality we will assume that each unit isidentied by a predened index.

ognitii(i) locate the winner unit c by:

c argminkkr0 Mrkk

(ii) change the network weights according to theequation:

Mt1i ri Mtiri athictr0 Mtiri

The term at is the learning rate which is oftengiven by:

at a0 tmax ttmaxbeing a06 1 and tmax the (prexed) number ofiterations of the relaxation process. The function

Fig. 2. Image matching by a 2D self-organizing network: SOM

units are superimposed to the pixels of the target image and the

network is made to relax under the action of the pixel of the

stimulus image.

344 G. Coppini et al. / Pattern Rechic accounts for lateral plasticity: its value is onefor i c and falls o with distance kri rck fromthe winner unit. Gaussian is a widely used neigh-

borhood function:

hict exp" kri rck

2

2rt2#

The parameter rt determines the width of theneighborhood and controls the topology preser-

vation process. As in the case of the learning rate,

rt is decreased as the relaxation progresses.Unfortunately, the previous self-adaptation

equation is too simple to produce the desired re-

sults. Only condition 1 of our denition (i.e.

preservation of image plane topology) is satisedby the obtained mapping. On the other hand,

image features are not taken into account and aunitary mapping is produced independently of the

image content. In order to achieve a meaningful

behavior, i.e. to ensure that feature preservation is

satised (condition 2), we modied the standard

SOM rule so as to include image features. To be

more precise, we dened feature similarity S by:

Sfsr0; ftr 11 kfsr0 ftrk

where k k is the ordinary Euclidean metric. ThefunctionS assumes a unit value when fsr0 ftrand drops to zero as kfsr0 ftrk increases.Having dened S, we replaced the SOM weightupdate rule with the equation:

Mt1i ri Mtiri athict

r0 MtiriSfsr0; f tr 1

With this choice, maximal weight changes are al-

lowed when similarity between corresponding

features is high. On the other hand, weight changes

are weakened when dierent image features arecoupled. Otherwise stated, image features inu-

ence the computed map by simply modulating the

entity of weight update. One can have an idea of

the eect of such a process by considering the ex-

treme case of zero-valued similarity: this prevents

any weight change despite the competition mech-

anism that has no actual eect. In brief, competi-

tion among network units involves spatialcoordinates only, while changes to mapping are

controlled by similarity of image-feature.

2.2. Image representation by Gabor Wavelets

In general, the matching process can be based

on pictorial features which are optimized with re-

spect to the problem to be faced. Nevertheless,especially in the absence of explicit knowledge,

several low-level descriptions of images are known

(such as derivative of Gaussians, and several

classes of wavelets) which are well suited to cap-

ture image structures. In this view, we took Gabor

Wavelets into account. They provide a multi-scale

image representation which can be rigorously

framed in the theory of wavelet transform (Lee,1996) while keeping an intriguing biological

inspiration (Daugman, 1988). Gabor Wavelets for

on Letters 25 (2004) 341352an image Ir can be obtained by a set of lters:

Ir Wr; kwith:

Wr; k 4k2 exp2k2r2

expi2pk r

exp p

2

2

where k kx; ky is the spatial frequency vectorwhich is usually more conveniently expressed in a

polar reference k k;u where the modulus kacts as a scale-selection parameter. The gaborian

kernel, for a xed k vector, is a Gaussian which is

modulated by a complex sinusoid. Such a kernelcan be decomposed into a pair of quadrature l-

dierent scales, image textures with dierent

coarseness are enhanced. In addition, Gabor ker-

nels are well suited to model the receptive elds of

simple cells in biological vision systems. Moreover,

the modulus mr; k of the output of a pair ofGabor lters:

mr; k;u mr; k

cr; k Ir2 sr; k Ir2

qis usually considered a good approximation of the

output of complex cells (Spitzer and Hochstein,

1985). The non-linearly ltered images obtained by

mr; k jIr Wr; kj are used in this work.

G. Coppini et al. / Pattern Recognition Letters 25 (2004) 341352 345ters:

cx; y; k;u 4k2 exp2k2x2 y2

cos2pkx cosu y sinu expp2=2

sx; y; k;u 4k2 exp2k2x2 y2

sin2pkx cosu y sinu

These are two Gaussians with a standard deviation12k, each Gaussian being modulated by a sinusoid

with a frequ...