Upload
pj-toivanen
View
221
Download
0
Embed Size (px)
Citation preview
Pattern Recognition Letters 24 (2003) 2987–2994
www.elsevier.com/locate/patrec
Edge detection in multispectral imagesusing the self-organizing map
P.J. Toivanen a,*, J. Ansam€aaki b, J.P.S. Parkkinen c, J. Mielik€aainen a
a Laboratory of Information Processing, Department of Information Technology, Lappeenranta University of Technology,
P.O. Box 20, FIN-53851 Lappeenranta, Finlandb Kouvola Business Department, Kymenlaakso Polytechnic, Salpaussel€aantie 57, FIN-45100 Kouvola, Finland
c Department of Computer Science, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland
Received 26 June 2002; received in revised form 12 May 2003
Abstract
In this paper, two new methods for edge detection in multispectral images are presented. They are based on the use
of the self-organizing map (SOM) and a grayscale edge detector. With the 2-dimensional SOM the ordering of pixel
vectors is obtained by applying the Peano scan, whereas this can be omitted using the 1-dimensional SOM. It is shown
that using the R-ordering based methods some parts of the edges may be missed. However, they can be found using the
proposed methods. Using them it is also possible to find edges in images which consist of metameric colors. Finally, it is
shown that the proposed methods find the edges properly from real multispectral airplane images. The size of the SOM
determines the amount of found edges. If the SOM is taught using a large color vector database, the same SOM can be
utilized for numerous images.
� 2003 Elsevier B.V. All rights reserved.
Keywords: Multispectral image edge detection; Ordering of multivariate data; Self-organizing maps; Feature extraction; Pattern
recognition; Machine vision
1. Introduction
There exists a large number of methods for edge
detection in digital images. In (Qian and Huang,1996) a new edge detection method is presented,
which is derived from an adaptive 2-D edge model.
* Corresponding author. Tel.: +358-5-621-2812; fax: +358-5-
621-2899.
E-mail address: [email protected] (P.J. Toivanen).
0167-8655/$ - see front matter � 2003 Elsevier B.V. All rights reserv
doi:10.1016/S0167-8655(03)00159-4
It is optimal in terms of signal-to-noise ratio
(SNR) and edge localization accuracy (ELA). A
new edge detection method for 3-component color
images is presented in (Fan et al., 2001). However,in this paper, multi- and hyperspectral images are
used, and all operations are performed on the
spectral vectors.
One approach in gray-level images is to see the
pixel values as an ordering of pixels. From this
image, in which an ordering scalar is associated
with every pixel vector, the edges are found using
an edge detection operator.
ed.
2988 P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994
It is not possible to define uniquely the ordering
of multivariate data. A number of ways have been
proposed to perform multivariate data ordering.
They are usually classified into the following cate-
gories: marginal ordering (M-ordering), reduced
or aggregate ordering (R-ordering), partial order-ing (P-ordering), and conditional ordering (C-
ordering) (Barnett, 1976). Of these ordering
methods, the R-ordering is the most used in edge
detection and filtering of multispectral images
(Trahanias and Venetsanopoulos, 1993). It gives a
natural definition of the vector median as the first
sample in the sorted vectors, and large values of
the aggregate distance give an accurate descriptionof the vector outliers (Astola et al., 1990). Fur-
thermore, the other ordering methods suffer from
certain drawbacks in the case of color image pro-
cessing. M-ordering corresponds actually to a
componentwise processing and P-ordering implies
the construction of convex hulls which is very
difficult in 3 and higher dimensions. C-ordering is
simply an ordering according to a specific compo-nent and it does not utilize the information con-
tent of the other signal components. A thorough
discussion of the ordering method is given in
(Barnett, 1976).
Conventionally, edge detection methods of mul-
tispectral images are based on gradient methods
(Cumani, 1991) or ordering the spectral vectors
first using a suitable ordering method, e.g. R-ordering (Trahanias and Venetsanopoulos, 1993).
Unfortunately the gradient approach is unsatis-
factory in cases where the image gradients show
the same strength but in opposite directions. Then,
the vector sum of the gradients would provide a
null gradient (Zenzo, 1986).
This paper is organized as follows. Section 2
presents the R-ordering and the proposed newordering methods of multispectral image pixels for
edge detection purposes. It is shown in Section 3
that the R-ordering based methods may miss some
parts of the edges, because R-ordering gives the
same scalar value to some pixels which lie in dif-
ferent areas. These edges can be found using the
proposed methods. Also, the obtained results
using real multispectral airplane images are shownin Section 3. Section 4 presents a discussion on the
issue.
2. Edge detection by ordering pixels
2.1. R-ordering
In this paper, a multispectral image is viewed asa vector field, represented by a discrete vector-
valued function g : Z2 ! Zp, where Z represents
the set of integers and p is an integer.
Let x represent a p-dimensional vector x ¼ ½x1;x2; . . . ; xp�T, where xl, l ¼ 1; 2; . . . ; p, are the spec-
tral components of a pixel and let xj, j ¼ 1; 2; . . . ;n, be the pixel j in the image g. n is the number of
pixels in the image g. Each xj is a p-dimensionalvector xj ¼ ½xj1; x
j2; . . . ; x
jp�T. In R-ordering, each
vector xj is reduced to a scalar value dj in the
following way:
dj ¼Xn
k¼1kxj � xkk; ð1Þ
where k � k represents an appropriate vector norm.
An arrangement of the dj�s in ascending order,d1 6 d2 6 � � � 6 dn, associates the same ordering to
the multivariate xj�s, x16x2
6 � � � 6 xn. x1 is the
vector median of the data samples (Astola et al.,
1990). As a result of the R-ordering, the original
multispectral image is transformed to a scalar
image.
2.2. The self-organizing map and Peano scan
The basic idea of the self-organizing map
(SOM) (Kohonen, 1989) assumes a sequence of
input vectors fxj; j ¼ 1; 2; . . . ; ng, where n is the
number of the vectors. The set of representative
neuron vectors which form the SOM at the iter-
ation phase j is denoted by fmji ; i ¼ 1; 2; . . . ; kg.
The number of vectors in the SOM is denotedby k. Every mj
i is a p-dimensional vector.
In the learning phase, it is assumed that the m0i
have been initialized in some proper way; random
selection will often do. Every input xj is compared
to all the mji . The input signal vector xj, the rep-
resentative neuron vectors in the SOM mji , and
best matching unit c are related by Eq. (2),
kxj �mjck ¼ min
ifkxj �mj
ikg; ð2Þ
P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994 2989
where k � k represents an appropriate vector norm.
In this paper, the Euclidean norm is used (Koho-
nen, 1989).
Updating the SOM in the learning phase is
done according to Eqs. (3) and (4),
mjþ1i ¼ mj
i þ aj½xj �mji � 8i 2 Nj
c ; ð3Þ
mjþ1i ¼ mj
i 8i 62 Njc : ð4Þ
Njc is a topological neighborhood which is centered
around that representative neuron vector for
which the best match with input xj is found. The
radius of Njc is shrinking monotonically with time.
aj is a scalar parameter that decreases monotoni-
cally during the course of the process, 0 < a < 1
(Kohonen, 1989).
When a 2-dimensional SOM is used, afterteaching every vector in the SOM is traversed
using the Peano scan. As a result, we get a 2-
dimensional matrix P which orders the vectors in
such a way that a scalar can be given to every
column vector. The Peano curve used in this paper
is quantized to match the size of the SOM. The
Peano curve is one of the family of fractal curves
discussed in more detail by Mandelbrot (1977).Patrick et al. (1968) showed how similar curves
could be used to map multidimensional data onto
a line for different applications. A general mathe-
matical approach to the algorithms for generating
these curves is reported by Butz (1971). The rele-
vant properties of the Peano scan for this paper
can be found in (Stevens et al., 1993). Fig. 2 shows
the ordering process with the 2-dimensional SOM.P is defined by
P ¼ ½p1; p2; . . . ; pk�; ð5Þwhere k is the number of neurons in the SOM.
In this paper, k ¼ 32� 32 ¼ 1024 or k ¼ 64�64 ¼ 4096. Then, each Peano vector pi is defined
by
pi ¼ ½pi1; pi2; . . . ; piM �T; ð6Þ
where M ¼ 61 in the artificial images or M ¼ 25 in
the real-world image is the number of components
of the vectors. Furthermore, Peano vectors pi
which are near each other in the Peano matrix Pare also quite near each other according Euclidean
distance in the 61-dimensional or 25-dimensional
vector space. After this stage, every vector in the
original image is matched to the Peano matrix
column vectors pi by finding the best matching unit
c 2 f1; 2; . . . ; kg, where k is the number of vectors
in P, i.e., finding the vector which is nearest thevector of the original image. An input vector x,the Peano vectors in the Peano matrix P ¼ ½pð1Þ;pð2Þ; . . . ; pðNÞ�, and the best matching unit c are
related as follows:
kx� pck ¼ minifkx� pikg: ð7Þ
The scalar value of this best matching unit c is
inserted into a new image f to replace the vector of
the same location in g:
f ðx; yÞ c; ð8Þwhere f ¼ f ðx; yÞ denotes the new order image,
which is a gray-level image. The edges in f are then
easy to find using any grayscale edge detector. In
this paper, the Laplace and Canny operators are
used.
In the case of 1-dimensional SOM the neigh-
borhood Nc must be redefined. Let M ¼ ½m1;m2; . . . ;mk� be the 1-dimensional SOM with kvectors. In the teaching phase, every pixel vector
of the original image gðx; yÞ is taught to the 1-
dimensional SOM with the neighborhood Nc
defined as follows:
Nc ¼ fmaxð1; c� lÞ; c;minðk; cþ lÞg; ð9Þwhere l is a suitable positive integer. During thelearning phase l decreases. After the teaching is
completed, an input vector x of g is inserted into
the SOM to find out the best matching unit ac-
cording to Eq. (2). The scalar value of this best
matching unit c is inserted into a new image f to
replace the vector of the same location in g ac-
cording to Eq. (8). Then, the edges in f are easy to
find using any grayscale edge detector. In thispaper, the Laplace and Canny operators are used.
3. Results
In multispectral image edge detection, R-
ordering is usually used inside a mask. A suitable
mask, e.g. 5 · 5, is conveyed pixel by pixel along
the image calculating the sum of distances from
Fig. 2. (a) Schematic drawing of the ordering process; (b) an RGB presentation of the original multispectral image.
Fig. 1. (a) The three areas ða–cÞ in which the image plane is divided; (b) the 5· 5 mask in the first position and the center point is
denoted by ðaÞ; (c) the mask in the second position and the center point is denoted by ðcÞ.
2990 P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994
the center point of the mask to all the other mask
points. After this, usually a conventional edge
detector is applied to this R-ordered image. For
instance, Trahanias and Venetsanopoulos (1993)
applied vector dispersion and gradient based edgedetectors are to the R-ordered scalar image.
Usually the R-ordering based methods work well.
However, R-ordering sometimes orders two dif-
ferent spectra into the same scalar value. Fig. 1(a)
shows an image plane which is divided into three
different areas, marked with a�s, b�s, and c�s. Eacharea has a distinct scalar value. Scalar values are
used here for simplicity. A 5 · 5 mask used in the
R-ordering calculations is shown in Fig. 1(b), the
center point being in area a. It is assumed that
the mask proceeds downwards. Calculating the
R-ordering value according to Eq. (1) yields
d1 ¼ 11ða� aÞ þ 5ða� bÞ þ 9ða� cÞ. Calculatingthe corresponding value d2 for Fig. 1(c) gives
d2 ¼ 6ðc� aÞ þ 7ðc� bÞ þ 12ðc� cÞ. Now the
problem is to find a combination values a� c suchthat d1 ¼ d2 holds. By setting d1 ¼ d2 we get
10aþ b� 11c ¼ 0. This equation is true for in-
stance with values a ¼ 4, b ¼ 26, c ¼ 6. This
means that R-ordering gives the same scalar values
for these two pixels, the first in area a, and the
P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994 2991
second in area c. Therefore, no edge between them
can be found. According to the tests made, this
phenomenon occurs when the image plane is
divided into more than two areas.
It should be noted that this example with sca-
lar pixel values can easily be extended to vector-valued images. Several tests were made with
images with more than two areas. In all of them,
with a certain combination of pixel values the R-
ordering gave same values for some of the pixels
lying in two distinct areas. This results in errone-
ous edges, i.e., bended edges, very wide edges, or
often no edge at all. Using the methods presented
in this paper, this problem can be overcome, i.e.,the missing edge can be found, and it is in its
correct location and is of normal thickness. This is
due to fact that the proposed methods give the
same scalar value to every vector of the same kind.
If the SOM used is large enough, the proposed
methods produce different scalar values for differ-
ent areas, and this concerns every single pixel in
them. The edges are then easy to find by applyingan edge detector operator to the scalar image.
Fig. 2(b) shows an RGB presentation of the
original multispectral image, in which every pixel
is a 61-dimensional vector. Areas 3, 4, 5, 7, and 8
hold different vectors. Areas 3 and 4 are green,
areas 5 and 7 blue, and area 8 is red in RGB space.
The whole Munsell book (Munsell, 1976) was
taught to a 1- and 2-dimensional SOM. The re-sulting edge image using a 1-dimensional SOM
and the Laplace operator or a 2-dimensional SOM
Fig. 3. (a) The edge image of Fig. 3 using a 1-dimensional SOM, or a 2
operator; (b) the original multispectral image converted to an RGB i
followed by the Peano scan and the Laplace op-
erator is depicted in Fig. 3(a). Fig. 3(b) shows an
image which contains metameric colors. This
means that all the pixel vectors of it map into the
same point in RGB space. Therefore, a metameric
RGB image cannot be used for edge detection.Fig. 4(a) shows the SOM which has been taught
using the pixel vectors of the original image, Fig.
4(a), as input. Fig. 4(b) depicts the edge image
when the Canny edge detector (Canny, 1986) is
applied to the obtained gray-level image after the
ordering process.
Fig. 5(a) shows a multispectral real-world image
taken from an airplane. In this image, everypixel consists of 25 uniformly distributed spectral
channels measured between 649.9 and 747.4 nm.
The resolution is 140 · 140 pixels. Fig. 5(b) depicts
the 2-dimensional SOM which is taught using the
pixel vectors of the image in Fig. 5(a). After this,
the SOM image is scanned using the Peano scan to
get an ordering for the image vectors. The ob-
tained edge image using the 2-dimensional SOM,Peano scan and Canny operator is shown in Fig.
6(a). In this image, a 32 · 32 SOM is used. Fig. 6(b)
shows a corresponding image with a 64 · 64 SOM.
It can be seen that the size of the SOM has a great
influence on the outcome. Increasing the size of the
SOM increases the number of edges. Therefore,
the size of the SOM is heavily application depen-
dent. Fig. 7 shows an edge image which is obtainedby calculating the R-ordering inside a 5 · 5 mask
over the original image.
-dimensional SOM and the Peano scan, followed by the Laplace
mage.
Fig. 4. (a) The image after the SOM and Peano scan; (b) the edge image after the Canny operator is applied to the image of Fig. 6.
Fig. 5. (a) The original real-world image of a landscape; (b) the SOM taught using the pixel vectors of the image of Fig. 8.
Fig. 6. (a) The edge image using the 2-D SOM of size 32 · 32, the Peano scan, and the Canny operator; (b) the edge image using the
2-D SOM of size 64 · 64, the Peano scan, and the Canny operator.
2992 P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994
Fig. 7. The edge image using the R-ordering inside a 5 · 5mask.
P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994 2993
Fig. 8 shows the original hyperspectral AVIRIS
image (MultiSpec, 2003). The spatial size of the
image is 145 · 145 and the number of bands is 220.
Fig. 9(a) depicts the edge image obtained with a
1-D SOM of size 8. Finally, Fig. 9(b) shows the
Fig. 8. AVIRIS hyperspectral image from (MultiSpec, 2003).
Fig. 9. (a) The edge image using the 1-D SOM of size 8 and the
Canny operator; (b) the edge image using the 1-D SOM of size
64 and the Canny operator.
edge image with a 1-D SOM of size 64. It can be
clearly seen that also with AVIRIS images the
number of edges increases with the size of the SOM.
The dimension of the SOM does not play a
crucial role in detecting edges in a real-world
multispectral image. According to the tests made,the same edges can be located also by using a 1-
dimensional SOM. The size of the SOM and its
learning parameters have a greater influence on the
outcome than the dimension of the SOM.
The edge strength is visible in the final edge
image of the proposed methods. The more the gray
value of the edge pixel deviates from the back-
ground, the stronger the edge is, and the fartherapart the original spectral vectors are on both sides
of the edge. This gives a natural and easy way to
define edge strength in future work.
4. Discussion
In this paper, two new methods for edge de-tection in multispectral images are presented. They
are based on the use of the SOM, Peano scan, and
a grayscale edge detector.
The vectors of the original multispectral image
are ordered using both a 1-dimensional and a 2-
dimensional SOM. The former gives directly a 1-
dimensional ordering for the spectral vectors, and
no Peano scan is needed. To the latter SOM thePeano scan is applied to give a 1-dimensional
ordering. A recursive curve is used to produce the
topological closeness of similar and nearly similar
SOM vectors. It is shown that using the R-order-
ing based methods some parts of the edges may be
missed or they will be erroneous. Using the pro-
posed methods it is easy to find those edges
properly.The ordering is achieved by letting the SOM
learn the whole vector space under consideration
or the vector space available in the original image.
In the former case, the same SOM can be applied
for edge detection of several images. In the latter
case, a new SOM must be taught for every image.
By increasing the size of the SOM it is possible
to improve the ability of the new method to findedges between vectors which are very near each
other according to a chosen similarity measure. If
2994 P.J. Toivanen et al. / Pattern Recognition Letters 24 (2003) 2987–2994
the size of the SOM is decreased, the edge detector
will become more robust by clustering more vec-
tors to the same ordering value in the Peano vec-
tor, which will diminish its ability to find edges
between vectors with only small variations. It is
shown in this paper that this phenomenon is validboth for multispectral and hyperspectral images.
It is also shown that the new methods are ca-
pable of finding edges in images which contain
metameric colors. In a metameric image all the
pixel vectors map into the same point in RGB
space, and therefore it is not possible to find edges
in such an RGB image with conventional methods
which use an RGB image as input.The performance of the new methods is also
tested using a real-world multispectral airplane
image in which every pixel is a 25-component
vector. It is shown that the proposed methods
produce the edges properly. By altering the size of
the 1- or 2-dimensional SOM and changing the
number of iteration rounds in SOM teaching it is
possible to govern the number of edges. The for-mer is, of course, crucial, and the latter only has a
minor effect on the outcome. If the size of the
SOM is increased, the method produces more and
more weak edges. Also test are made using a real-
world hyperspectral image in which every pixel is a
220-component vector. It is shown that the pro-
posed method produces the edges properly, simi-
larly as with multispectral images. Also in this casethe size of the SOM determines the number of
found edges.
The exact topologies of the 220-, 61- and 25-
dimensional color spaces used in this paper are
unknown. Therefore, the selection of norm for the
SOM is not obvious, but the Euclidean norm is a
natural choice for the spectra of real colors. Any-
way, the selection of norm requires further re-search on the color space. It has been shown by
Parkkinen and J€aa€aaskel€aainen (1987) that a spectral
image vector of arbitrary dimension can be re-
duced to a vector whose dimensionality is 4–7.
This can be done without losing information, i.e.
the original spectral vector can be recovered. This
means that theoretically the 1-dimensional SOM is
not good enough to learn the structure of the orig-inal multispectral image, i.e., for the ordering of
arbitrary spectral vectors. A 2-dimensional SOM
is closer to the theoretical basis. However, in trans-
forming the original image to a gray-level image
via the SOM, the problem of traversing the SOM
nodes rises if the 2-dimensional SOM is applied. In
the test images presented in this paper, no detect-
able difference can be found between the use of 1-and 2-dimensional SOMs. According to the tests
made, it seems that in practice there is no differ-
ence between the 1- and 2-dimensional SOM in
their ability to order the spectral vectors for edge
detection purposes.
References
Astola, J., Haavisto, P., Neuvo, Y., 1990. Vector Median
Filters. Proceedings of the IEEE 78, 678–689.
Barnett, V., 1976. The ordering of multivariate data. J. Roy.
Statist. Soc. Ser. A 139 (Part 3), 318–355.
Butz, A.R., 1971. Alternative algorithm for Hilbert�s space-
filling curve. IEEE Trans. Comput. C-20, 424–426.
Canny, J., 1986. A computational approach to edge detection.
IEEE Trans. Pattern Anal. Machine Intell. PAMI-8 (6).
Cumani, A., 1991. Edge detection in multispectral images.
CVGIP: Graphical Models Image Process. 53 (1), 40–51.
Documentation for MultiSpec. Available at http://www.ece.
purdue.edu/~biehl/MultiSpec/documentation.html 5 May
2003.
Fan, J., Aref, W.G., Hacid, M.-S., Elmagarmid, A.K., 2001. An
improved automatic isotropic color edge detection tech-
nique. Pattern Recognition Lett. 22 (13), 1419–1429.
Kohonen, T., 1989. Self-Organization and Associative
Memory, third ed. Springer-Verlag.
Mandelbrot, B., 1977. Fractals––Form, Chance and Dimen-
sion. W.H. Freeman.
Munsell Book of Color-Matte Finish Collection, 1976. Munsell
Color, Baltimore, MD, USA.
Parkkinen, J., J€aa€aaskel€aainen, T., 1987. Color representation
using statistical pattern recognition. Appl. Opt. 26 (19),
4240–4245.
Patrick, E.A., Anderson, D.R., Bechtel, F., 1968. Mapping
multidimensional space to one dimension for computer
output display. IEEE Trans. Comput., 949–953.
Qian, R.J., Huang, T.S., 1996. Optimal edge detection in two-
dimensional images. IEEE Trans. Image Process. 5 (7),
1215–1220.
Stevens, R.J., Lehar, A.F., Preston, F.H., 1993. Manipulation
and presentation of multidimensional image data using the
Peano scan. IEEE Trans. Pattern Anal. Machine Intell.
PAMI-5 (5), 520–526.
Trahanias, P.E., Venetsanopoulos, A.N., 1993. Color edge
detection using vector order statistics. IEEE Trans. Image
Process. 2 (2), 259–264.
Zenzo, S., 1986. A note on the gradient of a multi-image.
Comput. Vision Graphics Image Process. 33, 116–125.