Matching of medical images by self-organizing neural networks

Embed Size (px)

Text of Matching of medical images by self-organizing neural networks

  • aetw

    Giuseppe Coppini a,*, Stefano Diciotti b, Guido Valli b

    noise indicate considerable robustness against statistical variability. Applications to clinical images are presented.

    preattentive processes to attentive tasks. A wide of paramount importance in modern medical

    imaging. In the latter case, it is necessary to buildcomprehensive functional and anatomical repre-

    sentations of the observed biological structures by

    utilizing the partial views oered by basic imaging

    procedures such as MRI, PET, CT (van den Elsen

    qPartially founded by the Italian Ministry for Education,

    University and Research (MIUR).*

    Pattern Recognition Letters 25Corresponding author. Tel.: +39-503-153-480; fax: +39-

    503-152-166. 2003 Elsevier B.V. All rights reserved.

    Keywords: Image matching; Medical imaging; SOM neural network; Gabor wavelets

    1. Introduction

    Finding correspondences between images is a

    central problem in many vision activities, from

    category of image matching problems arises from

    the need to integrate images generated by dierent

    modalities. The importance of integration tasks

    has rapidly grown in recent years and has becomea Consiglio Nazionale della Ricerche (CNR), Institute of Clinical Physiology, Via Moruzzi 1, 56124 Pisa, Italyb Department of Electronics and Telecommunications, University of Florence, Italy

    Received 11 March 2003; received in revised form 21 October 2003


    A general approach to the problem of image matching which exploits a multi-scale representation of local image

    structure and the principles of self-organizing neural networks is introduced. The problem considered is relevant in

    many imaging applications and has been largely investigated in medical imagery, especially as regards the integration of

    dierent imaging procedures.

    A given pair of images to be matched, named target and stimulus respectively, are represented by Gabor Wavelets.

    Correspondence is computed by exploiting the learning procedure of a neural network derived from Kohonens SOM.The SOM units coincide with the pixels of the target image and their weight are pointers to those of the stimulus images.

    The standard SOM rule is modied so as to account for image features. The properties of our method are tested by

    experiments performed on synthetic images. The considered implementation has shown that is able to recover a wide

    range of transformations including global ane transformations and local distortions. Tests in the presence of additiveMatching of medical imneural nE-mail addresses: (G. Coppini), diciotti@ (S. Diciotti), (G. Valli).

    0167-8655/$ - see front matter 2003 Elsevier B.V. All rights reservdoi:10.1016/j.patrec.2003.10.012ges by self-organizingorks q

    (2004) 341352 al., 1993). Several correspondence problems

    have been faced which include mono-modal or


  • subject variability, nevertheless the observed

    structures keep some general properties such asconnectedness and proximity among sub-struc-

    tures. In general, it seems reasonable to search for

    a correspondence law which is unique from T toS, 1 and which maps features in the T plane tosimilar features in the S plane by matching spa-tially contiguous features in T into spatially con-tiguous features in S. These conditions can berestated more concisely by saying that the desiredmapping is a feature- and topology-preserving

    transformation from T into S. The importance oftopology preservation in image matching has been

    ognition Letters 25 (2004) 341352multi-modal image correspondence and intra- or

    inter-subjects matching. In addition, matching of

    3D images has become an important issue along

    with the mapping of imaged structures to reference

    representations such as anatomical atlases. Much

    eort has been devoted to nding global defor-mations, which has led to ecient algorithms

    being able to recover linear transformations. A

    popular method is based on the maximization of

    the mutual information of the two images (Wells

    III et al., 1996). More recently, the use of deform-

    able models has been widely investigated so as to

    recover local non-linear deformations. A number

    of references on the subject can be found in(Maintz and Viergever, 1998). Thermodynamical

    analogy with Maxwell demons has been also

    investigated (Thirion, 1998).

    In general, specic solutions have been pro-

    posed for many matching problems. The related

    algorithms usually exploit some form of prior

    knowledge, and are often based on strong

    assumptions. On the other hand many corre-spondence nding problems share important

    common facets, such as the statement of image-

    similarity criteria or the denition of adequate

    computational schemes. In this view, the impor-

    tance of general solutions for image matching

    problems has been considered by several

    researchers (Haralick and Shapiro, 1993). In par-

    ticular, we believe that a self-organizing process isa natural and very promising setting. As reported

    by Bellando and Kotari (1996), the computation of

    topology preserving maps using Kohonen neural

    networks (Kohonen, 2001) can provide valid

    solutions to establishing image correspondence.

    Wurtz et al. (1999) compare the behavior of Ko-

    honens SOM with the Dynamic Link Architecture(Konen et al., 1994) paying particular attentionto the computational eciency of self-organizing

    processes. It is worth mentioning that the appli-

    cation of a SOM network to image registration is

    described by Huwer et al. (1996).

    In this work we will take the following general

    problem into account. Let Itr and Isr0, twoimages (target and stimulus image, respectively)

    with r x; y, and r0 x0; y0 co-ordinate vectorsdened in proper subsets T and S, respectively

    342 G. Coppini et al. / Pattern Rec(image planes) of R2 space. We assume that afeature vector ftr ff it rg describing relevantproperties of Itr can be computed for each pointr in T . Similarly, fsr0 ff is r0g will indicate thefeature vector of Isr0 for each r0 2 S. We searchfor a correspondence rule M : T 7!S that mapspoints in T to points in S which have similarmorphological properties, as described by the

    feature vectors (see Fig. 1). This is an ill-posed

    problem, and further constraints must be consid-

    ered to compute a useful solution, when stated so.We believe that powerful constraints can be de-

    rived from the regularity of the observed world

    even if the use of specic knowledge available for

    each matching problem considered can lead to

    adequate solutions. For example, images of nor-

    mal anatomical structures exhibit a large inter-

    tI (r)

    sI (r)




    Sstimulus image

    target image

    Fig. 1. A general matching procedure is expected to preserve

    pictorial features and their neighborhood relationship.1 One-to-one mapping can be desirable in some cases.

  • given in the image planes I fT ; Sg and the fea-

    ble image description to compute a useful set of

    pictorial features f ff ig. This led us to investi-gate a self-organizing neural network coupled with

    ognition Letters 25 (2004) 341352 343ture space F , respectively. Let us assume that forany given r1; r2 2 T , r01; r02 2 S are the correspond-ing transformed points, i.e. r01 Mr1, r02 Mr2. We will say that the transformation Mpreserves image topology if the following condi-

    tions are met:

    (1) Preservation of image plane topology:

    limdI r1;r2!0

    dIr01; r02 0

    (2) Feature preservation:

    limdI r1;r01!0

    dF ftr1; fsr01 0

    Though further constraints can be taken into

    account (such as the high order smoothness of

    the mapping), our aim is the investigation of the

    preservation of the neighborhood relationship.

    To this end we consider the computation of

    image matching by using a topology preserving

    neural network derived from Kohonens SOM.The resulting computation is typically data-drivenand no specic prior model is expected to con-

    strain the obtained solution. In this sense our

    approach is typically non-parametric. In order to

    investigate the related capabilities, we outline the

    basic scheme of the adopted computational para-

    digm in the next section. Subsequently we present

    and analyze the performances observed when a

    set of known mathematical transformations wasapplied to phantom images in the presence of

    additive noise. Applications to computing corre-

    spondence between biomedical images are also


    2. Computational paradigm

    Following the previous considerations, a gen-

    eral framework of image matching problems re-

    quires: (a) a proper computational architecture todiscussed by Musse et al. (2001) who propose the

    use of hierarchical parametric models. A formal

    denition of the meaning of topology preserving

    transformation is useful for the following. To this

    end we assume that proper metrics dI and dF are

    G. Coppini et al. / Pattern Recimplement a topology preserving map, (b) a exi-image features computed by the Gabor Wavelet


    2.1. Matching through self-organization

    The computation of topology preserving maps

    between vector spaces can be carried out by using

    Kohonen neural networks (Kohonen, 2001).

    Basically, they include a grid of units, typically 2D,

    each of which receives the same input ve