Download pdf - Edge detection in multispectral images using the self-organizing map

Pattern Recognition Letters 24 (2003) 2987–2994

www.elsevier.com/locate/patrec

Edge detection in multispectral imagesusing the self-organizing map

P.J. Toivanen a,*, J. Ansam€aaki b, J.P.S. Parkkinen c, J. Mielik€aainen a

a Laboratory of Information Processing, Department of Information Technology, Lappeenranta University of Technology,

P.O. Box 20, FIN-53851 Lappeenranta, Finlandb Kouvola Business Department, Kymenlaakso Polytechnic, Salpaussel€aantie 57, FIN-45100 Kouvola, Finland

c Department of Computer Science, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland

Received 26 June 2002; received in revised form 12 May 2003

Abstract

In this paper, two new methods for edge detection in multispectral images are presented. They are based on the use

of the self-organizing map (SOM) and a grayscale edge detector. With the 2-dimensional SOM the ordering of pixel

vectors is obtained by applying the Peano scan, whereas this can be omitted using the 1-dimensional SOM. It is shown

that using the R-ordering based methods some parts of the edges may be missed. However, they can be found using the

proposed methods. Using them it is also possible to find edges in images which consist of metameric colors. Finally, it is

shown that the proposed methods find the edges properly from real multispectral airplane images. The size of the SOM

determines the amount of found edges. If the SOM is taught using a large color vector database, the same SOM can be

utilized for numerous images.

� 2003 Elsevier B.V. All rights reserved.

Keywords: Multispectral image edge detection; Ordering of multivariate data; Self-organizing maps; Feature extraction; Pattern

recognition; Machine vision

1. Introduction

There exists a large number of methods for edge

detection in digital images. In (Qian and Huang,1996) a new edge detection method is presented,

which is derived from an adaptive 2-D edge model.

* Corresponding author. Tel.: +358-5-621-2812; fax: +358-5-

621-2899.

E-mail address: [email protected] (P.J. Toivanen).

0167-8655/$ - see front matter � 2003 Elsevier B.V. All rights reserv

doi:10.1016/S0167-8655(03)00159-4

It is optimal in terms of signal-to-noise ratio

(SNR) and edge localization accuracy (ELA). A

new edge detection method for 3-component color

images is presented in (Fan et al., 2001). However,in this paper, multi- and hyperspectral images are

used, and all operations are performed on the

spectral vectors.

One approach in gray-level images is to see the

pixel values as an ordering of pixels. From this

image, in which an ordering scalar is associated

with every pixel vector, the edges are found using

an edge detection operator.

ed.

mail to: [email protected]

2988 P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994

It is not possible to define uniquely the ordering

of multivariate data. A number of ways have been

proposed to perform multivariate data ordering.

They are usually classified into the following cate-

gories: marginal ordering (M-ordering), reduced

or aggregate ordering (R-ordering), partial order-ing (P-ordering), and conditional ordering (C-

ordering) (Barnett, 1976). Of these ordering

methods, the R-ordering is the most used in edge

detection and filtering of multispectral images

(Trahanias and Venetsanopoulos, 1993). It gives a

natural definition of the vector median as the first

sample in the sorted vectors, and large values of

the aggregate distance give an accurate descriptionof the vector outliers (Astola et al., 1990). Fur-

thermore, the other ordering methods suffer from

certain drawbacks in the case of color image pro-

cessing. M-ordering corresponds actually to a

componentwise processing and P-ordering implies

the construction of convex hulls which is very

difficult in 3 and higher dimensions. C-ordering is

simply an ordering according to a specific compo-nent and it does not utilize the information con-

tent of the other signal components. A thorough

discussion of the ordering method is given in

(Barnett, 1976).

Conventionally, edge detection methods of mul-

tispectral images are based on gradient methods

(Cumani, 1991) or ordering the spectral vectors

first using a suitable ordering method, e.g. R-ordering (Trahanias and Venetsanopoulos, 1993).

Unfortunately the gradient approach is unsatis-

factory in cases where the image gradients show

the same strength but in opposite directions. Then,

the vector sum of the gradients would provide a

null gradient (Zenzo, 1986).

This paper is organized as follows. Section 2

presents the R-ordering and the proposed newordering methods of multispectral image pixels for

edge detection purposes. It is shown in Section 3

that the R-ordering based methods may miss some

parts of the edges, because R-ordering gives the

same scalar value to some pixels which lie in dif-

ferent areas. These edges can be found using the

proposed methods. Also, the obtained results

using real multispectral airplane images are shownin Section 3. Section 4 presents a discussion on the

issue.

2. Edge detection by ordering pixels

2.1. R-ordering

In this paper, a multispectral image is viewed asa vector field, represented by a discrete vector-

valued function g : Z2 ! Zp, where Z represents

the set of integers and p is an integer.

Let x represent a p-dimensional vector x ¼ ½x1;x2; . . . ; xp�T, where xl, l ¼ 1; 2; . . . ; p, are the spec-

tral components of a pixel and let xj, j ¼ 1; 2; . . . ;n, be the pixel j in the image g. n is the number of

pixels in the image g. Each xj is a p-dimensionalvector xj ¼ ½xj1; x

j2; . . . ; x

jp�T. In R-ordering, each

vector xj is reduced to a scalar value dj in the

following way:

dj ¼Xn

k¼1kxj � xkk; ð1Þ

where k � k represents an appropriate vector norm.

An arrangement of the dj�s in ascending order,d1 6 d2 6 � � � 6 dn, associates the same ordering to

the multivariate xj�s, x16x2

6 � � � 6 xn. x1 is the

vector median of the data samples (Astola et al.,

1990). As a result of the R-ordering, the original

multispectral image is transformed to a scalar

image.

2.2. The self-organizing map and Peano scan

The basic idea of the self-organizing map

(SOM) (Kohonen, 1989) assumes a sequence of

input vectors fxj; j ¼ 1; 2; . . . ; ng, where n is the

number of the vectors. The set of representative

neuron vectors which form the SOM at the iter-

ation phase j is denoted by fmji ; i ¼ 1; 2; . . . ; kg.

The number of vectors in the SOM is denotedby k. Every mj

i is a p-dimensional vector.

In the learning phase, it is assumed that the m0i

have been initialized in some proper way; random

selection will often do. Every input xj is compared

to all the mji . The input signal vector xj, the rep-

resentative neuron vectors in the SOM mji , and

best matching unit c are related by Eq. (2),

kxj �mjck ¼ min

ifkxj �mj

ikg; ð2Þ

P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994 2989

where k � k represents an appropriate vector norm.

In this paper, the Euclidean norm is used (Koho-

nen, 1989).

Updating the SOM in the learning phase is

done according to Eqs. (3) and (4),

mjþ1i ¼ mj

i þ aj½xj �mji � 8i 2 Nj

c ; ð3Þ

mjþ1i ¼ mj

i 8i 62 Njc : ð4Þ

Njc is a topological neighborhood which is centered

around that representative neuron vector for

which the best match with input xj is found. The

radius of Njc is shrinking monotonically with time.

aj is a scalar parameter that decreases monotoni-

cally during the course of the process, 0 < a < 1

(Kohonen, 1989).

When a 2-dimensional SOM is used, afterteaching every vector in the SOM is traversed

using the Peano scan. As a result, we get a 2-

dimensional matrix P which orders the vectors in

such a way that a scalar can be given to every

column vector. The Peano curve used in this paper

is quantized to match the size of the SOM. The

Peano curve is one of the family of fractal curves

discussed in more detail by Mandelbrot (1977).Patrick et al. (1968) showed how similar curves

could be used to map multidimensional data onto

a line for different applications. A general mathe-

matical approach to the algorithms for generating

these curves is reported by Butz (1971). The rele-

vant properties of the Peano scan for this paper

can be found in (Stevens et al., 1993). Fig. 2 shows

the ordering process with the 2-dimensional SOM.P is defined by

P ¼ ½p1; p2; . . . ; pk�; ð5Þwhere k is the number of neurons in the SOM.

In this paper, k ¼ 32� 32 ¼ 1024 or k ¼ 64�64 ¼ 4096. Then, each Peano vector pi is defined

by

pi ¼ ½pi1; pi2; . . . ; piM �T; ð6Þ

where M ¼ 61 in the artificial images or M ¼ 25 in

the real-world image is the number of components

of the vectors. Furthermore, Peano vectors pi

which are near each other in the Peano matrix Pare also quite near each other according Euclidean

distance in the 61-dimensional or 25-dimensional

vector space. After this stage, every vector in the

original image is matched to the Peano matrix

column vectors pi by finding the best matching unit

c 2 f1; 2; . . . ; kg, where k is the number of vectors

in P, i.e., finding the vector which is nearest thevector of the original image. An input vector x,the Peano vectors in the Peano matrix P ¼ ½pð1Þ;pð2Þ; . . . ; pðNÞ�, and the best matching unit c are

related as follows:

kx� pck ¼ minifkx� pikg: ð7Þ

The scalar value of this best matching unit c is

inserted into a new image f to replace the vector of

the same location in g:

f ðx; yÞ c; ð8Þwhere f ¼ f ðx; yÞ denotes the new order image,

which is a gray-level image. The edges in f are then

easy to find using any grayscale edge detector. In

this paper, the Laplace and Canny operators are

used.

In the case of 1-dimensional SOM the neigh-

borhood Nc must be redefined. Let M ¼ ½m1;m2; . . . ;mk� be the 1-dimensional SOM with kvectors. In the teaching phase, every pixel vector

of the original image gðx; yÞ is taught to the 1-

dimensional SOM with the neighborhood Nc

defined as follows:

Nc ¼ fmaxð1; c� lÞ; c;minðk; cþ lÞg; ð9Þwhere l is a suitable positive integer. During thelearning phase l decreases. After the teaching is

completed, an input vector x of g is inserted into

the SOM to find out the best matching unit ac-

cording to Eq. (2). The scalar value of this best

matching unit c is inserted into a new image f to

replace the vector of the same location in g ac-

cording to Eq. (8). Then, the edges in f are easy to

find using any grayscale edge detector. In thispaper, the Laplace and Canny operators are used.

3. Results

In multispectral image edge detection, R-

ordering is usually used inside a mask. A suitable

mask, e.g. 5 · 5, is conveyed pixel by pixel along

the image calculating the sum of distances from

Fig. 2. (a) Schematic drawing of the ordering process; (b) an RGB presentation of the original multispectral image.

Fig. 1. (a) The three areas ða–cÞ in which the image plane is divided; (b) the 5· 5 mask in the first position and the center point is

denoted by ðaÞ; (c) the mask in the second position and the center point is denoted by ðcÞ.


the center point of the mask to all the other mask

points. After this, usually a conventional edge

detector is applied to this R-ordered image. For

instance, Trahanias and Venetsanopoulos (1993)

applied vector dispersion and gradient based edgedetectors are to the R-ordered scalar image.

Usually the R-ordering based methods work well.

However, R-ordering sometimes orders two dif-

ferent spectra into the same scalar value. Fig. 1(a)

shows an image plane which is divided into three

different areas, marked with a�s, b�s, and c�s. Eacharea has a distinct scalar value. Scalar values are

used here for simplicity. A 5 · 5 mask used in the

R-ordering calculations is shown in Fig. 1(b), the

center point being in area a. It is assumed that

the mask proceeds downwards. Calculating the

R-ordering value according to Eq. (1) yields

d1 ¼ 11ða� aÞ þ 5ða� bÞ þ 9ða� cÞ. Calculatingthe corresponding value d2 for Fig. 1(c) gives

d2 ¼ 6ðc� aÞ þ 7ðc� bÞ þ 12ðc� cÞ. Now the

problem is to find a combination values a� c suchthat d1 ¼ d2 holds. By setting d1 ¼ d2 we get

10aþ b� 11c ¼ 0. This equation is true for in-

stance with values a ¼ 4, b ¼ 26, c ¼ 6. This

means that R-ordering gives the same scalar values

for these two pixels, the first in area a, and the


second in area c. Therefore, no edge between them

can be found. According to the tests made, this

phenomenon occurs when the image plane is

divided into more than two areas.

It should be noted that this example with sca-

lar pixel values can easily be extended to vector-valued images. Several tests were made with

images with more than two areas. In all of them,

with a certain combination of pixel values the R-

ordering gave same values for some of the pixels

lying in two distinct areas. This results in errone-

ous edges, i.e., bended edges, very wide edges, or

often no edge at all. Using the methods presented

in this paper, this problem can be overcome, i.e.,the missing edge can be found, and it is in its

correct location and is of normal thickness. This is

due to fact that the proposed methods give the

same scalar value to every vector of the same kind.

If the SOM used is large enough, the proposed

methods produce different scalar values for differ-

ent areas, and this concerns every single pixel in

them. The edges are then easy to find by applyingan edge detector operator to the scalar image.

Fig. 2(b) shows an RGB presentation of the

original multispectral image, in which every pixel

is a 61-dimensional vector. Areas 3, 4, 5, 7, and 8

hold different vectors. Areas 3 and 4 are green,

areas 5 and 7 blue, and area 8 is red in RGB space.

The whole Munsell book (Munsell, 1976) was

taught to a 1- and 2-dimensional SOM. The re-sulting edge image using a 1-dimensional SOM

and the Laplace operator or a 2-dimensional SOM

Fig. 3. (a) The edge image of Fig. 3 using a 1-dimensional SOM, or a 2

operator; (b) the original multispectral image converted to an RGB i

followed by the Peano scan and the Laplace op-

erator is depicted in Fig. 3(a). Fig. 3(b) shows an

image which contains metameric colors. This

means that all the pixel vectors of it map into the

same point in RGB space. Therefore, a metameric

RGB image cannot be used for edge detection.Fig. 4(a) shows the SOM which has been taught

using the pixel vectors of the original image, Fig.

4(a), as input. Fig. 4(b) depicts the edge image

when the Canny edge detector (Canny, 1986) is

applied to the obtained gray-level image after the

ordering process.

Fig. 5(a) shows a multispectral real-world image

taken from an airplane. In this image, everypixel consists of 25 uniformly distributed spectral

channels measured between 649.9 and 747.4 nm.

The resolution is 140 · 140 pixels. Fig. 5(b) depicts

the 2-dimensional SOM which is taught using the

pixel vectors of the image in Fig. 5(a). After this,

the SOM image is scanned using the Peano scan to

get an ordering for the image vectors. The ob-

tained edge image using the 2-dimensional SOM,Peano scan and Canny operator is shown in Fig.

6(a). In this image, a 32 · 32 SOM is used. Fig. 6(b)

shows a corresponding image with a 64 · 64 SOM.

It can be seen that the size of the SOM has a great

influence on the outcome. Increasing the size of the

SOM increases the number of edges. Therefore,

the size of the SOM is heavily application depen-

dent. Fig. 7 shows an edge image which is obtainedby calculating the R-ordering inside a 5 · 5 mask

over the original image.

-dimensional SOM and the Peano scan, followed by the Laplace

mage.

Fig. 4. (a) The image after the SOM and Peano scan; (b) the edge image after the Canny operator is applied to the image of Fig. 6.

Fig. 5. (a) The original real-world image of a landscape; (b) the SOM taught using the pixel vectors of the image of Fig. 8.

Fig. 6. (a) The edge image using the 2-D SOM of size 32 · 32, the Peano scan, and the Canny operator; (b) the edge image using the

2-D SOM of size 64 · 64, the Peano scan, and the Canny operator.


Fig. 7. The edge image using the R-ordering inside a 5 · 5mask.


Fig. 8 shows the original hyperspectral AVIRIS

image (MultiSpec, 2003). The spatial size of the

image is 145 · 145 and the number of bands is 220.

Fig. 9(a) depicts the edge image obtained with a

1-D SOM of size 8. Finally, Fig. 9(b) shows the

Fig. 8. AVIRIS hyperspectral image from (MultiSpec, 2003).

Fig. 9. (a) The edge image using the 1-D SOM of size 8 and the

Canny operator; (b) the edge image using the 1-D SOM of size

64 and the Canny operator.

edge image with a 1-D SOM of size 64. It can be

clearly seen that also with AVIRIS images the

number of edges increases with the size of the SOM.

The dimension of the SOM does not play a

crucial role in detecting edges in a real-world

multispectral image. According to the tests made,the same edges can be located also by using a 1-

dimensional SOM. The size of the SOM and its

learning parameters have a greater influence on the

outcome than the dimension of the SOM.

The edge strength is visible in the final edge

image of the proposed methods. The more the gray

value of the edge pixel deviates from the back-

ground, the stronger the edge is, and the fartherapart the original spectral vectors are on both sides

of the edge. This gives a natural and easy way to

define edge strength in future work.

4. Discussion

In this paper, two new methods for edge de-tection in multispectral images are presented. They

are based on the use of the SOM, Peano scan, and

a grayscale edge detector.

The vectors of the original multispectral image

are ordered using both a 1-dimensional and a 2-

dimensional SOM. The former gives directly a 1-

dimensional ordering for the spectral vectors, and

no Peano scan is needed. To the latter SOM thePeano scan is applied to give a 1-dimensional

ordering. A recursive curve is used to produce the

topological closeness of similar and nearly similar

SOM vectors. It is shown that using the R-order-

ing based methods some parts of the edges may be

missed or they will be erroneous. Using the pro-

posed methods it is easy to find those edges

properly.The ordering is achieved by letting the SOM

learn the whole vector space under consideration

or the vector space available in the original image.

In the former case, the same SOM can be applied

for edge detection of several images. In the latter

case, a new SOM must be taught for every image.

By increasing the size of the SOM it is possible

to improve the ability of the new method to findedges between vectors which are very near each

other according to a chosen similarity measure. If


the size of the SOM is decreased, the edge detector

will become more robust by clustering more vec-

tors to the same ordering value in the Peano vec-

tor, which will diminish its ability to find edges

between vectors with only small variations. It is

shown in this paper that this phenomenon is validboth for multispectral and hyperspectral images.

It is also shown that the new methods are ca-

pable of finding edges in images which contain

metameric colors. In a metameric image all the

pixel vectors map into the same point in RGB

space, and therefore it is not possible to find edges

in such an RGB image with conventional methods

which use an RGB image as input.The performance of the new methods is also

tested using a real-world multispectral airplane

image in which every pixel is a 25-component

vector. It is shown that the proposed methods

produce the edges properly. By altering the size of

the 1- or 2-dimensional SOM and changing the

number of iteration rounds in SOM teaching it is

possible to govern the number of edges. The for-mer is, of course, crucial, and the latter only has a

minor effect on the outcome. If the size of the

SOM is increased, the method produces more and

more weak edges. Also test are made using a real-

world hyperspectral image in which every pixel is a

220-component vector. It is shown that the pro-

posed method produces the edges properly, simi-

larly as with multispectral images. Also in this casethe size of the SOM determines the number of

found edges.

The exact topologies of the 220-, 61- and 25-

dimensional color spaces used in this paper are

unknown. Therefore, the selection of norm for the

SOM is not obvious, but the Euclidean norm is a

natural choice for the spectra of real colors. Any-

way, the selection of norm requires further re-search on the color space. It has been shown by

Parkkinen and J€aa€aaskel€aainen (1987) that a spectral

image vector of arbitrary dimension can be re-

duced to a vector whose dimensionality is 4–7.

This can be done without losing information, i.e.

the original spectral vector can be recovered. This

means that theoretically the 1-dimensional SOM is

not good enough to learn the structure of the orig-inal multispectral image, i.e., for the ordering of

arbitrary spectral vectors. A 2-dimensional SOM

is closer to the theoretical basis. However, in trans-

forming the original image to a gray-level image

via the SOM, the problem of traversing the SOM

nodes rises if the 2-dimensional SOM is applied. In

the test images presented in this paper, no detect-

able difference can be found between the use of 1-and 2-dimensional SOMs. According to the tests

made, it seems that in practice there is no differ-

ence between the 1- and 2-dimensional SOM in

their ability to order the spectral vectors for edge

detection purposes.

References

Astola, J., Haavisto, P., Neuvo, Y., 1990. Vector Median

Filters. Proceedings of the IEEE 78, 678–689.

Barnett, V., 1976. The ordering of multivariate data. J. Roy.

Statist. Soc. Ser. A 139 (Part 3), 318–355.

Butz, A.R., 1971. Alternative algorithm for Hilbert�s space-

filling curve. IEEE Trans. Comput. C-20, 424–426.

Canny, J., 1986. A computational approach to edge detection.

IEEE Trans. Pattern Anal. Machine Intell. PAMI-8 (6).

Cumani, A., 1991. Edge detection in multispectral images.

CVGIP: Graphical Models Image Process. 53 (1), 40–51.

Documentation for MultiSpec. Available at http://www.ece.

purdue.edu/~biehl/MultiSpec/documentation.html 5 May

2003.

Fan, J., Aref, W.G., Hacid, M.-S., Elmagarmid, A.K., 2001. An

improved automatic isotropic color edge detection tech-

nique. Pattern Recognition Lett. 22 (13), 1419–1429.

Kohonen, T., 1989. Self-Organization and Associative

Memory, third ed. Springer-Verlag.

Mandelbrot, B., 1977. Fractals––Form, Chance and Dimen-

sion. W.H. Freeman.

Munsell Book of Color-Matte Finish Collection, 1976. Munsell

Color, Baltimore, MD, USA.

Parkkinen, J., J€aa€aaskel€aainen, T., 1987. Color representation

using statistical pattern recognition. Appl. Opt. 26 (19),

4240–4245.

Patrick, E.A., Anderson, D.R., Bechtel, F., 1968. Mapping

multidimensional space to one dimension for computer

output display. IEEE Trans. Comput., 949–953.

Qian, R.J., Huang, T.S., 1996. Optimal edge detection in two-

dimensional images. IEEE Trans. Image Process. 5 (7),

1215–1220.

Stevens, R.J., Lehar, A.F., Preston, F.H., 1993. Manipulation

and presentation of multidimensional image data using the

Peano scan. IEEE Trans. Pattern Anal. Machine Intell.

PAMI-5 (5), 520–526.

Trahanias, P.E., Venetsanopoulos, A.N., 1993. Color edge

detection using vector order statistics. IEEE Trans. Image

Process. 2 (2), 259–264.

Zenzo, S., 1986. A note on the gradient of a multi-image.

Comput. Vision Graphics Image Process. 33, 116–125.

http://www.ece.purdue.edu/~biehl/MultiSpec/documentation.html

http://www.ece.purdue.edu/~biehl/MultiSpec/documentation.html