Edge detection in multispectral images using the self-organizing map

  • Published on

  • View

  • Download

Embed Size (px)


  • muor

    P.J. Toivanen a,*, J. Ansamaaki b, J.P.S. Parkkinen c, J. Mielikaainen a

    determines the amount of found edges. If the SOM is taught using a large color vector database, the same SOM can be

    1996) a new edge detection method is presented,

    which is derived from an adaptive 2-D edge model.

    in this paper, multi- and hyperspectral images are

    used, and all operations are performed on the

    spectral vectors.

    One approach in gray-level images is to see the

    pixel values as an ordering of pixels. From this

    image, in which an ordering scalar is associated

    Pattern Recognition Letters 24 (20* Corresponding author. Tel.: +358-5-621-2812; fax: +358-5-utilized for numerous images.

    2003 Elsevier B.V. All rights reserved.

    Keywords: Multispectral image edge detection; Ordering of multivariate data; Self-organizing maps; Feature extraction; Pattern

    recognition; Machine vision

    1. Introduction

    There exists a large number of methods for edge

    detection in digital images. In (Qian and Huang,

    It is optimal in terms of signal-to-noise ratio

    (SNR) and edge localization accuracy (ELA). A

    new edge detection method for 3-component color

    images is presented in (Fan et al., 2001). However,a Laboratory of Information Processing, Department of Information Technology, Lappeenranta University of Technology,

    P.O. Box 20, FIN-53851 Lappeenranta, Finlandb Kouvola Business Department, Kymenlaakso Polytechnic, Salpausselaantie 57, FIN-45100 Kouvola, Finland

    c Department of Computer Science, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland

    Received 26 June 2002; received in revised form 12 May 2003


    In this paper, two new methods for edge detection in multispectral images are presented. They are based on the use

    of the self-organizing map (SOM) and a grayscale edge detector. With the 2-dimensional SOM the ordering of pixel

    vectors is obtained by applying the Peano scan, whereas this can be omitted using the 1-dimensional SOM. It is shown

    that using the R-ordering based methods some parts of the edges may be missed. However, they can be found using the

    proposed methods. Using them it is also possible to nd edges in images which consist of metameric colors. Finally, it is

    shown that the proposed methods nd the edges properly from real multispectral airplane images. The size of the SOMEdge detection inusing the self-621-2899.

    E-mail address: pekka.toivanen@lut. (P.J. Toivanen).

    0167-8655/$ - see front matter 2003 Elsevier B.V. All rights reservdoi:10.1016/S0167-8655(03)00159-4ltispectral imagesganizing map

    03) 29872994

    www.elsevier.com/locate/patrecwith every pixel vector, the edges are found using

    an edge detection operator.


  • dj kxj xkk; 1

    ognition Letters 24 (2003) 29872994It is not possible to dene uniquely the ordering

    of multivariate data. A number of ways have been

    proposed to perform multivariate data ordering.

    They are usually classied into the following cate-

    gories: marginal ordering (M-ordering), reduced

    or aggregate ordering (R-ordering), partial order-ing (P-ordering), and conditional ordering (C-

    ordering) (Barnett, 1976). Of these ordering

    methods, the R-ordering is the most used in edge

    detection and ltering of multispectral images

    (Trahanias and Venetsanopoulos, 1993). It gives a

    natural denition of the vector median as the rst

    sample in the sorted vectors, and large values of

    the aggregate distance give an accurate descriptionof the vector outliers (Astola et al., 1990). Fur-

    thermore, the other ordering methods suer from

    certain drawbacks in the case of color image pro-

    cessing. M-ordering corresponds actually to a

    componentwise processing and P-ordering implies

    the construction of convex hulls which is very

    dicult in 3 and higher dimensions. C-ordering is

    simply an ordering according to a specic compo-nent and it does not utilize the information con-

    tent of the other signal components. A thorough

    discussion of the ordering method is given in

    (Barnett, 1976).

    Conventionally, edge detection methods of mul-

    tispectral images are based on gradient methods

    (Cumani, 1991) or ordering the spectral vectors

    rst using a suitable ordering method, e.g. R-ordering (Trahanias and Venetsanopoulos, 1993).

    Unfortunately the gradient approach is unsatis-

    factory in cases where the image gradients show

    the same strength but in opposite directions. Then,

    the vector sum of the gradients would provide a

    null gradient (Zenzo, 1986).

    This paper is organized as follows. Section 2

    presents the R-ordering and the proposed newordering methods of multispectral image pixels for

    edge detection purposes. It is shown in Section 3

    that the R-ordering based methods may miss some

    parts of the edges, because R-ordering gives the

    same scalar value to some pixels which lie in dif-

    ferent areas. These edges can be found using the

    proposed methods. Also, the obtained results

    using real multispectral airplane images are shownin Section 3. Section 4 presents a discussion on the

    2988 P.J. Toivanen et al. / Pattern Recissue.k1

    where k k represents an appropriate vector norm.An arrangement of the djs in ascending order,d16 d26 6 dn, associates the same ordering tothe multivariate xjs, x16x26 6 xn. x1 is thevector median of the data samples (Astola et al.,

    1990). As a result of the R-ordering, the original

    multispectral image is transformed to a scalar


    2.2. The self-organizing map and Peano scan

    The basic idea of the self-organizing map

    (SOM) (Kohonen, 1989) assumes a sequence of

    input vectors fxj; j 1; 2; . . . ; ng, where n is thenumber of the vectors. The set of representative

    neuron vectors which form the SOM at the iter-

    ation phase j is denoted by fmji ; i 1; 2; . . . ; kg.The number of vectors in the SOM is denotedby k. Every mji is a p-dimensional vector.

    In the learning phase, it is assumed that the m0ihave been initialized in some proper way; random

    selection will often do. Every input xj is comparedto all the mji . The input signal vector x

    j, the rep-

    resentative neuron vectors in the SOM mji , andbest matching unit c are related by Eq. (2),

    j j j j2. Edge detection by ordering pixels

    2.1. R-ordering

    In this paper, a multispectral image is viewed asa vector eld, represented by a discrete vector-

    valued function g : Z2 ! Zp, where Z representsthe set of integers and p is an integer.

    Let x represent a p-dimensional vector x x1;x2; . . . ; xpT, where xl, l 1; 2; . . . ; p, are the spec-tral components of a pixel and let xj, j 1; 2; . . . ;n, be the pixel j in the image g. n is the number ofpixels in the image g. Each xj is a p-dimensionalvector xj xj1; xj2; . . . ; xjpT. In R-ordering, eachvector xj is reduced to a scalar value dj in thefollowing way:

    Xnkx mck mini fkx mikg; 2

  • vector of the original image. An input vector x,the Peano vectors in the Peano matrix P p1;

    ognitwhere k k represents an appropriate vector norm.In this paper, the Euclidean norm is used (Koho-

    nen, 1989).

    Updating the SOM in the learning phase is

    done according to Eqs. (3) and (4),

    mj1i mji ajxj mji 8i 2 Njc ; 3

    mj1i mji 8i 62 Njc : 4Njc is a topological neighborhood which is centeredaround that representative neuron vector for

    which the best match with input xj is found. Theradius of Njc is shrinking monotonically with time.aj is a scalar parameter that decreases monotoni-cally during the course of the process, 0 < a < 1(Kohonen, 1989).

    When a 2-dimensional SOM is used, afterteaching every vector in the SOM is traversed

    using the Peano scan. As a result, we get a 2-

    dimensional matrix P which orders the vectors insuch a way that a scalar can be given to every

    column vector. The Peano curve used in this paper

    is quantized to match the size of the SOM. The

    Peano curve is one of the family of fractal curves

    discussed in more detail by Mandelbrot (1977).Patrick et al. (1968) showed how similar curves

    could be used to map multidimensional data onto

    a line for dierent applications. A general mathe-

    matical approach to the algorithms for generating

    these curves is reported by Butz (1971). The rele-

    vant properties of the Peano scan for this paper

    can be found in (Stevens et al., 1993). Fig. 2 shows

    the ordering process with the 2-dimensional SOM.P is dened by

    P p1; p2; . . . ; pk; 5where k is the number of neurons in the SOM.In this paper, k 32 32 1024 or k 6464 4096. Then, each Peano vector pi is denedby

    pi pi1; pi2; . . . ; piM T; 6where M 61 in the articial images or M 25 inthe real-world image is the number of components

    of the vectors. Furthermore, Peano vectors pi

    which are near each other in the Peano matrix Pare also quite near each other according Euclidean

    P.J. Toivanen et al. / Pattern Recdistance in the 61-dimensional or 25-dimensionalp2; . . . ; pN, and the best matching unit c arerelated as follows:

    kx pck minifkx pikg: 7

    The scalar value of this best matching unit c isinserted into a new image f to replace the vector ofthe same location in g:

    f x; y c; 8where f f x; y denotes the new order image,which is a gray-level image. The edges in f are theneasy to nd using any grayscale edge detector. In

    this paper, the Laplace and Canny operators are


    In the case of 1-dimensional SOM the neigh-

    borhood Nc must be redened. Let M m1;m2; . . . ;mk be the 1-dimensional SOM with kvectors. In the teaching phase, every pixel vector

    of the original image gx; y is taught to the 1-dimensional SOM with the neighborhood Ncdened as follows:

    Nc fmax1; c l; c;mink; c lg; 9where l is a suitable positive integer. During thelearning phase l decreases. After the teaching iscompleted, an input vector x of g is inserted intothe SOM to nd out the best matching unit ac-

    cording to Eq. (2). The scalar value of this best

    matching unit c is inserted into a new image f toreplace the vector of the same location in g ac-cording to Eq. (8). Then, the edges in f are easy tond using any grayscale edge detector. In thispaper, the Laplace and Canny operators are used.

    3. Results

    In multispectral image edge detection, R-

    ordering is usually used inside a mask. A suitable

    mask, e.g. 5 5, is conveyed pixel by pixel alongvector space. After this stage, every vector in the

    original image is matched to the Peano matrix

    column vectors pi by nding the best matching unitc 2 f1; 2; . . . ; kg, where k is the number of vectorsin P, i.e., nding the vector which is nearest the

    ion Letters 24 (2003) 29872994 2989the image calculating the sum of distances from

  • ognit2990 P.J. Toivanen et al. / Pattern Recthe center point of the mask to all the other mask

    points. After this, usually a conventional edge

    detector is applied to this R-ordered image. For

    instance, Trahanias and Venetsanopoulos (1993)

    applied vector dispersion and gradient based edgedetectors are to the R-ordered scalar image.

    Usually the R-ordering based methods work well.

    However, R-ordering sometimes orders two dif-

    ferent spectra into the same scalar value. Fig. 1(a)

    shows an image plane which is divided into three

    dierent areas, marked with as, bs, and cs. Eacharea has a distinct scalar value. Scalar values are

    used here for simplicity. A 5 5 mask used in the

    Fig. 2. (a) Schematic drawing of the ordering process; (b) an

    Fig. 1. (a) The three areas ac in which the image plane is divideddenoted by a; (c) the mask in the second position and the center poion Letters 24 (2003) 29872994R-ordering calculations is shown in Fig. 1(b), the

    center point being in area a. It is assumed thatthe mask proceeds downwards. Calculating the

    R-ordering value according to Eq. (1) yields

    d1 11a a 5a b 9a c. Calculatingthe corresponding value d2 for Fig. 1(c) givesd2 6c a 7c b 12c c. Now theproblem is to nd a combination values a c suchthat d1 d2 holds. By setting d1 d2 we get10a b 11c 0. This equation is true for in-stance with values a 4, b 26, c 6. Thismeans that R-ordering gives the same scalar values

    for these two pixels, the rst in area a, and the

    RGB presentation of the original multispectral image.

    ; (b) the 5 5 mask in the rst position and the center point isint is denoted by c.

  • second in area c. Therefore, no edge between themcan be found. According to the tests made, this

    phenomenon occurs when the image plane is

    divided into more than two areas.

    It should be noted that this example with sca-

    lar pixel values can easily be extended to vector-valued images. Several tests were made with

    images with more than two areas. In all of them,

    with a certain combination of pixel values the R-

    ordering gave same values for some of the pixels

    lying in two distinct areas. This results in errone-

    ous edges, i.e., bended edges, very wide edges, or

    often no edge at all. Using the methods presented

    in this paper, this problem can be overcome, i.e.,

    followed by the Peano scan and the Laplace op-

    erator is depicted in Fig. 3(a). Fig. 3(b) shows an

    image which contains metameric colors. This

    means that all the pixel vectors of it map into the

    same point in RGB space. Therefore, a metameric

    RGB image cannot be used for edge detection.Fig. 4(a) shows the SOM which has been taught

    using the pixel vectors of the original image, Fig.

    4(a), as input. Fig. 4(b) depicts the edge image

    when the Canny edge detector (Canny, 1986) is

    applied to the obtained gray-level image after the

    ordering process.

    Fig. 5(a) shows a multispectral real-world image

    taken from an airplane. In this image, every

    or a 2

    P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 29872994 2991the missing edge can be found, and it is in its

    correct location and is of normal thickness. This is

    due to fact that the proposed methods give the

    same scalar value to every vector of the same kind.

    If the SOM used is large enough, the proposed

    methods produce dierent scalar values for dier-

    ent areas, and this concerns every single pixel in

    them. The edges are then easy to nd by applyingan edge detector operator to the scalar image.

    Fig. 2(b) shows an RGB presentation of the

    original multispectral image, in which every pixel

    is a 61-dimensional vector. Areas 3, 4, 5, 7, and 8

    hold dierent vectors. Areas 3 and 4 are green,

    areas 5 and 7 blue, and area 8 is red in RGB space.

    The whole Munsell book (Munsell, 1976) was

    taught to a 1- and 2-dimensional SOM. The re-sulting edge image using a 1-dimensional SOM

    and the Laplace operator or a 2-dimensional SOM

    Fig. 3. (a) The edge image of Fig. 3 using a 1-dimensional SOM,operator; (b) the original multispectral image converted to an RGB ipixel consists of 25 uniformly distributed spectral

    channels measured between 649.9 and 747.4 nm.

    The resolution is 140 140 pixels. Fig. 5(b) depictsthe 2-dimensional SOM which is taught using the

    pixel vectors of the image in Fig. 5(a). After this,

    the SOM image is scanned using the Peano scan to

    get an ordering for the image vectors. The ob-

    tained edge image using the 2-dimensional SOM,Peano scan and Canny operator is shown in Fig.

    6(a). In this image, a 32 32 SOM is used. Fig. 6(b)shows a corresponding image with a 64 64 SOM.It can be seen that the size of the SOM has a great

    inuence on...


View more >