Adaptive morphological operators, fast algorithms and their applications

Pattern Recognition 33 (2000) 917}933

Adaptive morphological operators, fast algorithmsand their applications

F. Cheng!, A.N. Venetsanopoulos",*

!Electronic Systems R and D, Zenith Electronics Corporation, Glenview, IL 60025, USA"Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada M5S 3G4

Received 15 April 1998; accepted 2 May 1999

Abstract

In this paper, adaptive morphological operators are further developed, extending those of Ref. [1], to allow morefreedom in forming their operational windows that can adapt their shapes according to the local features of the processedimages. The properties of the adaptive operators are investigated. These properties lead to an interesting way to handleimages on the basis of the geometrical structure of images, and lead to the development of fast algorithms for the practicalapplication of the adaptive operators. The e$ciency of adaptive operators in image processing is demonstrated withexamples. ( 2000 Published by Elsevier Science Ltd. All rights reserved.

Keywords: Nonlinear operator; Adaptive morphological operator; Generalized structuring element; Image geometrical structures;Tip and bottom regions; Geometrical performance; Optimization on geometry

1. Introduction

In recent years, a number of nonlinear operators suchas the median "lter [2] and the morphological "lter [3,4]have attracted a great deal of research interest and havefound numerous applications in the areas of image pro-cessing and analysis. The early types of those nonlinearoperators utilized one operational window with "xedshape and size. In the case of image processing, thosenonlinear operators have been reported to have draw-backs such as creating arti"cial patterns and removingsigni"cant details [5,6], because of the "xed operationalwindow.

Many approaches have been considered to deal withthose problems. A well-accepted approach is based onthe combination of a family of operational windows.Each window in the family is designed to preservea special type of detail. The combination of all the win-dows in the family results in better performance than that

*Corresponding author. Tel.: #1-416-978-8670.E-mail addresses: [email protected] (F. Cheng), anv@

dsp.toronto.edu (A.N. Venetsanopoulos)

with one "xed operational window [4,6]. The problem ofthat approach is that in practical cases, the images to beprocessed may contain too many patterns of signi"cantdetails. Thus, it may be di$cult to combine enoughoperational windows to preserve many possible patternsof signi"cant details, while keeping the computationalcomplexity practical. Nonlinear operators that adapttheir operational windows according to the local statis-tics of images were also reported with improved perfor-mance [7]. But in some cases, those adaptive nonlinearoperators may have two basic di$culties. One is that thecomputational burden of these may be too heavy forpractical applications. Another is that the local statisticsof images may not be a good description of the geometri-cal features of images.

To deal with the problems of these existing techniques,a new type of adaptive morphological operators is pro-posed in Ref. [1]. The operational window of the oper-ators can adapt their shapes according to the geometricalfeatures of images and can take any connected shape ofa given size. The work of Cheng and Venetsanopoulos[1] suggested a new way to develop an image processingapproach based on the geometrical structures in imagesand showed through application examples that the

0031-3203/00/$20.00 ( 2000 Published by Elsevier Science Ltd. All rights reserved.PII: S 0 0 3 1 - 3 2 0 3 ( 9 9 ) 0 0 1 5 5 - 7

Fig. 1. (a) Position of the basic element. (b) Neighboring rela-tions of the basic elements.

Fig. 2. An example of the general structuring element, sizeN"4, actual size"22.

distinct way of image processing is promising. But theadaptive morphological operators of Ref. [1] are still intheir simplest forms and their properties are largely un-known. In order to consider more application areas,these adaptive operators need to be extended to moregeneral forms. Their properties need to be systematicallyinvestigated for the further development in both theoryand applications. Meanwhile, fast algorithms have to bedesigned to make the adaptive operators attractive forpractical applications. These problems are addressed inthis paper.

In Section 2, we introduce a general structure of theoperational window for the adaptive morphological op-erators proposed. In Section 3, we de"ne the adaptivemorphological operators that utilize the operational win-dow of a general structure and describe their properties.These properties lead to an interesting way to handleimages on the basis of the geometrical structure of im-ages, and lead to the proof of a number of propositionsdescribed in Section 4. Fast algorithms are designed onthe basis of those propositions. In Section 5, applicationexamples are described. Section 6 summarizes the mainconclusions.

2. The basic element and the related structuring element

The operational window of morphological operators iscalled structuring element. In Ref. [1], the structuringelement is formed by connected pixels. In this paper, weintroduce a more general approach. We form a structur-ing element by connecting basic elements. A basic ele-ment is considered as any connected shape. Generally,the shape depends on a speci"c image processing prob-lem. The advantage of this approach will be shownthrough examples in Section 5. Before giving a formalde"nition of the structuring element, we "rst de"ne theneighboring relation and the connectivity of the basicelement.

2.1. Dexnitions related to the basic element

Throughout this paper, only the discrete case is con-sidered. That is, let y(i, j) denote an image, the domain setis Mi, jNLZ2 and the range set is MyNLZ, where Z is theset of integers. Let d denote a basic element of anyconnected shape. The basic element d can be described byits support domain, since only #at structuring elementsare considered in this paper.

De5nition 1. A reference pixel of a basic element is de-"ned as a pixel selected in the domain of the basicelement. The position of the reference pixel of a basicelement is de"ned as the position of the basic element.

In Fig. 1(a), the shaded pixel is chosen as the referencepixel of the basic element. The neighboring relation of

basic elements can be de"ned in many ways. Here, weonly consider one case.

De5nition 2. The basic elements located at (i#k, j#s)are de"ned as the neighbors of the basic element locatedat (i, j), where (k, s)3M!1, 0, 1N, and k and s cannot bezero simultaneously.

The neighboring relation de"ned is shown in Fig. 1(b).According to De"nition 2, neighboring basic elementsmay overlap with each other depending on their shape.The connectivity of the basic elements is de"ned on thebasis of the neighboring relations of basic elements.

De5nition 3. Two neighboring basic elements are saidconnected to each other.

2.2. The generalized structuring element

Based on De"nitions 1}3, we can de"ne a new type ofgeneralized structuring element.

De5nition 4. The structuring element is formed by con-necting N basic elements. The size of the structuringelement is de"ned as N. The actual size of the structuringelement is de"ned as the number of pixels in the domainof the structuring element.

918 F. Cheng, A.N. Venetsanopoulos / Pattern Recognition 33 (2000) 917}933

Fig. 3. An example of the search of N basic elements.

In Section 3, we will further de"ne the adaptive mor-phological operators which utilize the generalized struc-turing element. On the basis of the de"nition, it will beshown that the shape of the structuring element is ableto adapt to the local features of images. Fig. 2 shows anexample of a structuring element formed by connectingfour basic elements of 3]3 pixels. In this paper, we onlyconsider "xed N. We also have the limitation of theconnectivity of both basic element and structuring ele-ment. In future work, we shall extend N to be adaptiveand drop the limitation of connectivity.

3. The adaptive morphological operators and theirproperties

In the appendix of this paper, we give a brief descrip-tion of the morphological operators with one structur-ing element and of the morphological operators with acombination of a family of structuring elements. Thoseoperators are the basis for the development of the adap-tive morphological operators in this section.

The adaptive morphological operators were originallydeveloped on the basis of Eqs. (A.7) and (A.8) in theappendix, and were called the NOP and NCP (a newtype of opening and closing operators) in Ref. [1]. Inthis paper, we still use the names NOP and NCPfor adaptive morphological operators for convenience.One of our further research goals is to systematicallydevelop a geometrical way for video processing on thebasis of the adaptive morphological operators. Improvednames may be considered at that time according tothe new understanding of the adaptive morphologicaloperators.

3.1. The NOP and NCP

Although the results in this subsection look similar tothat of their counterparts of Ref. [1], it should be men-tioned that introducing the basic element d has madethe NOP and NCP in this paper quite di!erent from theNOP and NCP of Ref. [1]. The di!erence will be shownthrough the development of fast algorithms and throughapplication examples in Sections 4 and 5.

Let $Nd

denote the set of the structuring elements of allthe shapes formed by connecting N basic elements d. Theproposed NOP (x " $N

d) is de"ned as

De5nition 5

(x " $Nd)(i, j)"max

B(k)|$N

d

[(x"B(k))(i, j)] (1)

Generally, it is impossible to compute Eq. (1) by directcomputation because of two reasons. One is that $N

dusually contains too many elements. Another is that

introducing d makes the determination of the domain ofeach structuring element in $N

dnot a simple matter any-

more, since d can be in any connected shape. In order tomake the de"nition of the generalized NOP meaningful,it is necessary to develop a practical approach for thecomputation of (x " $N

d). For that purpose, we consider

what is really performed by (x " $Nd) in Eq. (1). Combining

with Eq. (A.5), Eq. (1) can be expressed as

(x " $Nd)(i, j)"max

B(k)|$N

dC max((s1 ,s2 )>(i,j)|B(k)

s1,s2 )C min(t1 ,t2 )|B(k)

s1,s2

(x(t1, t

2))DD

(2)

In Eq. (2), the minimum of x is computed in the domainof every translation B(k)

s1 ,s2, which contains (i, j), of every

structuring element B(k) in $Nd. Then the maximum is

computed over all the minima obtained. In other words,let DN,d

i,jdenote the set of all the domains containing (i, j)

and formed by N connected basic elements. (x " $Nd)(i, j) is

assigned the minimum of x in such a domain Si,j3DN,d

i,jthat, for any domain S(k)

i,j3DN,d

i,j!S

i,j,

(x " $Nd)(i, j)" min

(t1,t2 )|Si,j

(x(t1, t

2))* min

(s1 ,s2 )|S(k)i,j

(x(s1, s

2))

holds. Those results are summarized as Proposition 1.

Proposition 1. An equivalence of the NOP (x " $Nd) dexned

by Eq. (1) can be expressed as

1. Search for a domain containing (i, j) and formed byN connected basic elements, in which the minimum of x isnot smaller than the minimum of x in any other domaincontaining (i, j) and formed by N connected basic ele-ments.

2. Assign the minimum to (x " $Nd)(i, j).

A simple example of the domain searched in step 1 ofProposition 1 is given in Fig. 3. In Proposition 1, thedomain searched is in fact the structuring element of theopening in Eq. (1), which gives the maximum value. Incontrast with the method requiring the computation ofall the openings in Eq. (1) before taking the maximum,Proposition 1 gives the relation between the structuringelement satisfying (1) at (i, j) and the local geometricalstructures of the image x at (i, j) and its neighboringpixels. It enables us to directly deal with only one open-ing. All the other openings in Eq. (1) do not have to becomputed. In that way, Proposition 1 o!ers a great

F. Cheng, A.N. Venetsanopoulos / Pattern Recognition 33 (2000) 917}933 919

potential in the development of fast algorithms to com-pute (x " $N

d).

Proposition 1 requires to search for the domain ofa structuring element among all possible structuring ele-ments. We have mentioned that to determine the domainof a structuring element is still a troublesome task, sincethe basic element d can be any shape. In Section 4, weshall prove a proposition which eliminates the search forthe whole domain, thus further facilitating the computa-tion.

By using the duality between opening and closing, thecase of the NCP can be described in a similar way. Weomit the details and only brie#y mention the results. TheNCP (x z $N

d) is de"ned as

De5nition 6

(x z $Nd)(i, j)"min

B(k)|$N

d

[(x z B(k))(i, j)]. (3)

Proposition 2 gives an equivalent description ofEq. (3).

Proposition 2. An equivalence of the NCP (x z $Nd) dexned

by Eq. (3) can be expressed as

1. Search for a domain containing (i, j) and formed byN connected basic elements, in which the maximum ofx is not larger than the maximum of x in any otherdomain containing (i, j) and formed by N connected basicelements.

2. Assign the maximum to (x z $Nd)(i, j).

We have mentioned that the domains searched inPropositions 1 and 2 are the structuring elements satisfy-ing Eqs. (1) and (3), respectively. On the other hand,we may consider the connected maximum pixels on animage surface as bright geometrical structures, and theconnected minimum pixels as dark geometrical struc-tures. Propositions 1 and 2 show the relation between thestructuring elements of the NOP and NCP and the localgeometrical structures of images. Thus, they show howthe structuring elements adapt their shapes according tothe local geometrical structures of images. In Section 3.2,we will give rigorous proofs to such geometrical perfor-mance of the NOP and NCP.

3.2. The properties of NOP and NCP

So far the properties of the NOP and NCP are largelyunknown. In this paper, nine properties and seven prop-ositions of the NOP and NCP are proved. These resultsmay allow a deeper understanding of the theoreticalaspects of the NOP and NCP and may enable us todevelop fast algorithms and to open new applicationareas. In this section we investigate a special case, wherethe basic element d is a single pixel. In Section 4, theextension of the results in this subsection to the general

case of the basic element is discussed. Some properties ofthe NOP and NCP are obvious extensions of the proper-ties of the conventional morphological operators. Thoseproperties are mentioned in the appendix.

Our work on the NOP and NCP has revealed thepossibility to develop an image processing approachbased on the geometrical structures in images. In thissection, we try to use a geometrical language, rather thana morphological language, to make de"nitions and todescribe and explain the properties of the NOP andNCP. We hope that this may provide a new beginningtowards a geometrical approach in image processing.

Consider an image as a surface, the local maxima asthe tips on the surface, the local minima as the bottoms,and the other parts on the surface as the slopes. A tip isusually characterized by a rising area on the image sur-face and a bottom by a falling area. Those rising andfalling areas related to the tips and bottoms form geomet-rical structures in images. Those structures are often themost interesting parts in image processing. De"nition7 gives a description of the rising areas.

Denote the basic element of a single pixel by d0, and

denote x1)x

2if for every (i, j) in the domains of x

1and

x2,x

1(i, j))x

2(i, j) holds. Let Ts1

denote the set of all thedomains ¹iq1 that contain only one tip q

1. Let Siq1 denote

the set of the boundary pixels of ¹iq1 , and Ss1 denote theset of all Siq1 . Let X(Siq1 ) denote the set of all the values ofthe image x on Siq1 , and x6 (Siq1 ) the maximum value inX(Si

s1 ).

x6 (Siq1)" max(s,t)|Siq

1

x(s, t).

De5nition 7. The tip region ¹q1 is de"ned as the min-imum domain satisfying

x6 (Sq1)" min+Siq1|Ss1

[x6 (Siq1 )]. (4)

In Eq. (4), Sq1 is the set of the boundary pixels of thetip region ¹q1 . According to De"nition 7, the tip region¹q1 is the domain which has the minimum x6 (Sq1 ) (themaximum boundary value) among all the domains¹iq1 containing only one tip q

1, and which is the smallest

among all the domains with the same maximum bound-ary value as x6 (Sq1) in Eq. (4). The tip region can beroughly considered as the domain corresponding to the#at area left after horizontally cutting o! the tip q

1at the

height of x6 (Sq1). One of the advantages of de"ning the tipregion in this way is that it shows the maximum domainto characterize the rising area leading to the tip q

1, which

does not overlap (except for the boundary pixels) withother tip regions. Fig. 4(a) shows one example of thetip region. We have mentioned that the tip structuresare often the most interesting parts of image process-ing. With De"nition 7, we may consider these structuresas objects characterized by the tip regions. Imageprocessing according to these objects allows for the full


Fig. 4. (a) De"nitions related to a tip q1. The shaded area is at

the level of x(Sq1 ) and is corresponding to the tip region. (b) The1-D geometric description of the NOP. The dots show theoriginal image surface. (c) The 2-D description of the NOP.

utilization of the spatial correlation of image features.Such an advantage is more bene"cial in the case ofthree and four dimensional images x(i, j, k), x(i, j, t) andx(i, j, k, t). For example, a moving spot in image x(i, j) hasnot much di!erence from a noise spot. But in imagex(i, j, t), a moving spot becomes an object of a long curve,and is quite di!erent from a noise spot.

Now let us turn the image surface upside down. Thebottoms become tips. Then, the bottom regions can be

de"ned in the same way as that of tip regions. Before wegive a description of the performance of the NOP, wede"ne a few more geometrical characters of an imagesurface. Let ;q1 denote the #at area of the tip q

1, D;q1 D

denote the number of the pixels contained in ;q1 . Let/q1 denote the set of all the pixels (i, j) satisfying(i, j)3M¹q1!Sq1 N, and (i

k, j

k), k"1,2, D/q1 D denote the

pixels in /q1 so that for any 1)k(t)D/q1 D,x(i

k, j

k)*x(i

t, jt).

Property 1. The geometric performance of the NOP(x " $N

d0) can be described in four ways.

1. When D;q1 D*N, the tip q1

will not be changed by theNOP.

2. When D;q1 D(N, D/q1 D*N,q1

will be yattened to a tipq2

satisfying

(a) D;q2 D*N;

(b) (x " $Nd0

)(i, j)"x(iN, j

N), for any (i, j)3;q2 .

(c) /q2 "/q1 , Sq2"Sq1 , and ¹q2"¹q1 .

(d) ;q2 can be in any connected shape. (5)

3. When D/q1 D"N0(N, (x " $N1

d0), where N

1"N

0#1,

can be used to cut q1,

(x " $N1d0

)(i, j)"x6 (Sq1 ); for (i, j)3/q1 .

Then ¹q1 becomes a part of a new tip, of a slope or a partof a bottom. (1) } (4) can be used again to describe thechange of this new tip, slope or bottom.

4. The other parts of the image surface, such as bottoms andslopes, will not be changed by the NOP.

The proof of Property 1 is given in the appendix.A simple 1-D example and a 2-D example of the geomet-ric performance of the NOP are illustrated in Fig. 4(b)and (c).

Let ¹b1,;b1

, Sb1and x (Sb1

) denote the bottom regionof a bottom b

1, the #at area of b

1, the set of the boundary

pixels of ¹b1, and the minimum value of x on Sb1

,respectively. Let /b1

denote the set of all the pixels (i, j)satisfying that (i, j)3M¹b1

!Sb1N, and (i

k, j

k), k"1,2,

D/b1D denote the pixels in /b1

so that for any 1)k(t)D/b1

D,

x(ik, j

k))x(i

t, j

t).

Then, the geometric performance of the NCP (x z $Nd0

)can be described and proved in a similar way as that inthe case of the NOP, with q

1, ¹q1 ,;q1 , Sq1 , x6 (Sq1 ), /q1

and tip replaced by b1, ¹b1

,;b1, Sb1

, x(Sb1), /b1

andbottom, respectively.

Those properties show that the NOP #attens the tipsand the NCP "lls the bottoms according to the localgeometrical structures in images. That is, the change ismade along the geometrical features. In contrast, mostof the existing linear or nonlinear image processing


Fig. 5. An example of a less biased approximation.

approaches change images according to the shapes oftheir operational windows. It has long been reported thatthese shapes may not well represent the local features ofimages in many cases. The unique geometrical perfor-mance of the NOP and NCP reveals a possible way ofimage processing based on the geometrical structuresrather than the statistical characterization of images.

The following properties of the NOP and NCP can beproved on the basis of Property 1, as well as PropertiesA.1}A.3 in the appendix.

Property 2

[[(x " $Nd0

) z $Nd0

] " $N~id0

] z $N~id0

"(x " $Nd0

) z $Nd0

(6)

where i"0,2, N!1.

Property 3

[[(x " $N~id0

) z $N~id0

] " $Nd0

] z $Nd0*(x " $N

d0) z $N

d0, (7)

[[(x z $N~id0

) " $N~id0

] z $Nd0

] " $Nd0)(x z $N

d0) " $N

d0, (8)

where i"1,2, N!2.

Generally, the left-hand side of Eq. (7) gives a lessbiased approximation of x than that given by the right-hand side. The same conclusion holds for Eq. (8).Fig. 5 shows a 1-D example of a less biased approxima-tion. In Fig. 5, x is an image surface, y

1and y

2are the

outputs of morphological operators,

y1"[[(x " B

1) z B

1] " B

2] z B

2(9)

y2"[x " B

2] z B

2(10)

where B1

and B2

are structuring elements, and B2

islarger than B

1. Fig. 5 shows y

1resembles the image

closer with the details removed.

Property 4. Among all the openings (closings) with struc-turing elements of size N or larger, the NOP(x " $N

d0)(NCP(x z $N

d0)) causes the minimum change of the

processed image.

Property 4 can be proved on the basis of Eqs. (1), (3)and of Property A.2 in the appendix. Consider the case ofopening. The opening operation always cuts down theimage surface. When a structuring element of the min-imum size N is required for an image processing task, Eq.(1) shows that the NOP (x " $N

d0) always cuts down the

image surface the least, thus causes the minimum changeof the processed image. The meaning of Property 4 to theimage processing can be explained as a type of optimiza-tion di!erent from the classical one. In the classical way,optimization is usually based on the statistics of signals.In the case of image processing, it has long been knownthat a result optimized on the basis of the statistics does

not mean a perceptually optimized result, since the statis-tics of images are not good descriptors of the geometricstructures, especially local geometric structures, of im-ages. By contrast, here we consider a type of optimizationbased on geometry. We assume that noise and signalpatterns di!er only by their sizes. A geometric patternsmaller than a given size is considered as noise. We alsoassume that the shapes of the geometric patterns of noiseand signal objects are not speci"ed. Our task is to removenoise. In such a case, the optimal approach can be sum-marized as an attempt to remove all the noise patternsand to change the signal patterns as little as possible.This is what is implied by Property 4. Although thegeometric interpretation of the optimization given here isfar from rigorous and systematic, it can serve as anexample to show that it is desirable and possible todevelop such a way for image processing.

Property 5. For two arbitrary points (i, j) and (s, t), we willobtain the same result of (x " $N

d0) by either xrst computing

(x " $Nd0

)(i, j), assigning the result to x(i, j), then computing(x " $N

d0)(s, t), assigning the result to x(s, t), or performing

the computation in a reverse order. The same property holdsin the case of the NCP (x z $N

d0).

Proof. The proof of Property 1 shows that the result ofthe NOP and NCP described by Property 1 does notdepend on the order of the NOP and NCP operationsperformed at the points (i, j) and (s, t). h

Based on Property 5, we can compute (x " $Nd0

)(i, j) or(x z $N

d0)(i, j), assign the result to x(i, j), then to compute

the NOP or NCP at the next point. That way enables usnot only to reduce the memory used in the computation,but also to develop fast algorithms for the implementa-tion of the NOP and NCP, as will be shown in the next


section. Property 5 also allows us to use multiprocessorsfor parallel processing of image.

4. The algorithms for the computation of the NOP andNCP

In this section, we "rst deal with the case of the NOP,then extend the results to the case of the NCP. In Section4.1, we prove several properties of the NOP in relationwith a basic algorithm given in the appendix. Fast algo-rithms are developed on the basis of those properties. InSection 4.2, several propositions are proved to show thatthe general NOP proposed in this paper can be decom-posed into three stages. The decomposition allows us tocompute the general NOP by the fast algorithms ofSection 4.1. The decomposition also allows us to extendthe properties proved in Section 3 to the case of thegeneral NOP and NCP operators. The computationalcomplexity of the algorithms is discussed in Section 4.3.

4.1. Propositions related to fast algorithms

A basic algorithm for the computation of the NOP andNCP was published in Ref. [1]. The propositions and thefast algorithm developed on these propositions in thissection are based on the basic algorithm. Because thesymbols de"ned in the basic algorithm are extensivelyused in the description of the propositions in this section,the basic algorithm is described in the appendix to makethe reference to those symbols easier. In Ref. [1], weproved two propositions for the development of fastalgorithms. Later we found that Proposition 4 in Ref. [1]did not work well in many programs. The propositionhas been dropped from all our programs. In this section,two new propositions are proved, which have proven tobe very e!ective in reducing the computational cost.

Proposition 3. When we compute (x " $Nd0

)(i, j) at pixel (i, j)in Step 1 of the basic algorithm, if x(c

i))x(bbN ), then,

(x " $Nd0

)(ci)"x(c

i) (11)

can be determined by the computation at (i, j), where ci

andbbN are dexned in the basic algorithm of the appendix.

Proposition 3 indicates that, under the conditionx(c

i))x(bbN ), the result of the NOP (x " $N

d0) at a num-

ber of neighboring pixels of Si,j

, that is de"ned in Section3.1, can be determined by the computation at (i, j). Nocomputation is necessary at those pixels later. The proofis brie#y described as follows. According to the conditionof the proposition, the de"nition of c

kand the de"nition

of bk, there are at least N pixels (s, k) connected to

ck

satisfying x(s, k)*x(ck). The proof is complete by

considering the de"nition of the NOP. Proposition 3 can

be easily combined into the basic algorithm, since wehave to compare x (c

i) with x(b

i) when we search for Md

iN

from MciN and Mb

iN in Step 1 of the basic algorithm in the

appendix. Here MbiN, Mc

iN and M d

iN are de"ned in the

basic algorithm in the appendix.Suppose the current computational position is at (i, j).

In the basic algorithm, we start from (i, j) to search forother N!1 connected pixels of S

i,j. Denote the N pixels

of Si,j

by z1,2, z

N. The order k of z

kcorresponds to the

order in which zk

is found in the search. We havez1"(i, j).

Proposition 4. Assume the minimum of x(z1),2, x(z

N) is at

the pixels zk1

,2, zkt, where 1)k

1)2)k

t)N and

1)t)N. Then we can assign

(x " $Nd0

)(zk)"x(z

k1), 1)k)k

1, k"k

i, i"2,2, t;

(12)

and the value of (x " $Nd0

)(zk); 1)k)k

1, and k"k

i, i"2,

2, t will not be changed in the later NOP computation.

Proof. By assumption,

x(zkt)" min

(s,t)|Si,j

(x(s, t)). (13)

In the following, we prove that we cannot "nd a domainD

zkof N connected pixels, which contains z

k, where

k)k1

or k"ki, i"2,2, t, so that

min(s1 ,t1 )|Dzk

(x(s1, t

1))' min

(s,t)|Si,j

(x(s, t)). (14)

At the pixels zki, i"1,2, t, Eq. (14) is obviously untrue.

Suppose k"k1. Then, the existence of the domain Dz

ksatisfying Eq. (14) means that z

k1NS

i,j, since the basic

algorithm always searches for the pixels corresponding tothe N largest values of x. When the search for S

i,jreaches

zk, it will continue through the path of Dz

k, rather than

the path to go to zk1

. The result contradicts the assump-tion z

k13S

i,j. Hence, the maximum of the left-hand side

of Eq. (14) over all possible Dzkis equal to the right-hand

side of Eq. (14). Thus, according to Proposition 1, we canchoose S

i,jas the domain searched in the computation of

(x " $Nd0

) at zk, 1)k)k

1, and k"k

i, i"2,2, t. That is

(x " $Nd0

)(zk)"x(z

k1) at those pixels. h

Proposition 4 shows that the result of the NOP com-puted at one pixel (i, j) can be used to determine the valueof (x " $N

d0) at a set of pixels in S

i,j. Thus, the NOP does

not have to be performed at those pixels again in the latercomputation. To combine Proposition 4 into the basicalgorithm, we can make the following two modi"cations.Suppose the current computation position is at (i, j).

(a) In the search for Si,j

starting at (i, j), we check whetherthe value of (x " $N

d0) at a pixel has been determined

before we include the pixel into Si,j

. By doing the


check, we obtain two bene"ts:

1. If (x " $Nd0

)(i, j) has been determined by the compu-tation at other pixels, then we can bypass (i, j) tocompute the NOP at the next pixel.

2. If (x " $Nd0

)(s, t), (s, t)O(i, j) has been determined,x(s, t)"(x " $N

d0)(s, t). Then on the basis of Prop-

erty 3, at pixel (s, t), we can always "nd connectedN pixels including (s, t), so that the values of x atthose pixels are not smaller than x(s, t). This meansthat the search for S

i,jcan be ended at (s, t). In this

way, the search can be expedited.

(b) In the search for Si,j

, we check whether the value of(x " $N

d0) at the pixels included in S

i,jcan be deter-

mined.

Although the propositions proved in this section areonly the initial results of our studies, they have resulted ina vast reduction of computational complexity as will beshown in Section 4.3. Our study has revealed more inter-esting properties of the NOP and NCP, which may beused to further reduce the computational complexity ofthe NOP and NCP.

4.2. The extension to the general NOP and NCP

The properties in Section 3.2 and the algorithms inSection 4.1 apply only to the case where the basic elementis a single pixel. In this section, we extend the results tothe general case, where the basic element d can be anyconnected shape depending on the requirement of a spe-ci"ed image processing task. Let d

i,jdenote the domain

of the basic element d located at (i, j). Let b(i, j) denote theminimum value of x in d

i,j. According to Eq. (A.1), b(i, j)

is the erosion of the input image x by the structuringelement d.

b(i, j)"(x>ds)(i, j) (15)

where ds is the symmetrical domain of d.

Proposition 5. The minimum of an image x in the domainof N connected basic elements is equal to the minimum of bin the domain of N connected pixels located at the samepositions as the corresponding basic elements.

Proof. According to De"nition 4, the connectivity of theN pixels corresponding to b is guaranteed by the con-nectivity of the N basic elements. The rest of the proof isobvious. h

Let g denote a domain formed by N connected basicelements d, among which one basic element is located at(i, j), and BN,d

i,jdenote the set of all possible g. The next

proposition enables us to use the algorithms developed inSection 4.1 for the computation of the general NOP.

Proposition 6

(b " $Nd0

)(i, j)" maxg(n)|BN,d

i,j C min(s,t)|g(n)

[x(s, t)]D. (16)

Proof. The proof is based on Proposition 5 andEq. (2). h

Let DN,di,j

denote the set of the domains considered inProposition 1. By de"nition, BN,d

i,jis a subset of DN,d

i,j. The

relation is

DN,di,j

"[XBN,ds,t

D(i, j)3ds,t

] (17)

Hence, according to Proposition 1,

(x " $Nd)(i, j)" max

(r1 ,r2 )@(i,j)|dr1,r2 C maxg(n)|BN,d

r1,r2 C min(t1 ,t2 )|g(n)

(x(t1, t

2))DD

" max(r1 ,r2 )@(i,j)|dr1,r2

[(b " $Nd) (r

1, r

2)]. (18)

According to Eq. (A.2), the maximum operation in thelast part of Eq. (18) is a dilation of (b " $N

d)(r

1, r

2) by ds.

Combining Eqs. (16), (17) and (18), we obtain Proposition 7.

Proposition 7

(x"$Nd)(i, j)"[[(x>ds)"$N

d0]=d](i, j) (19)

Proposition 7 shows that in general case, the NOP(x " $N

d) can be computed in three steps. The "rst step is

an erosion with the basic element d as the structuringelement. The second step is an NOP (b " $N

d0) with the

size of the structuring element being N. The third step isa dilation with the symmetrical set ds of the basic elementd as the structuring element. The "rst and the third stepscan be computed by conventional morphological algo-rithms. Since d is usually small, the computation is fast.The second step can be computed by the fast algorithmsdeveloped in the Section 4.1. The block diagram of thecomputational procedure of the algorithm is shown inFig. 6(a). An example of the relation between the struc-turing element of the NOP (x " $N

d) and the structuring

element utilized in each computational step is shown inFig. 6(b). The computational structure shown in Fig. 6 issimilar to that of the opening of one structuring elementwith the structuring element being decomposed into sev-eral smaller structuring elements [9]. But the computa-tional structure shown in Fig. 6 cannot be obtained on thebasis of the theory of the structuring element decomposi-tion, since the NOP with d

0in step 2 cannot be decom-

posed in the form of an erosion followed by a dilation.On the basis of the computational structure of the

general NOP and NCP shown by Proposition 7 and inFig. 6, the properties in Section 3.2 can be extended to thecase of the general NOP and NCP. In this section, wedo not go through the details of all those extensions. Weonly show the proof of the translation invariant propertyof the general NOP, as an example.


Fig. 6. (a) The computational structure of the general NOP.(b) An example of the relation between the structuring elementof the general NOP and the structuring elements used in eachcomputation step.

Fig. 7. The original images of (a) `Lenaa and (b) `Toysa.

Let d denote a general basic element and d0

denotethe basic element of a single pixel. Let ( f )

s,tdenote the

translation of the function f in bracket by (s, t). Basedon the translation invariant property of erosion and

dilation, Property A.4 in appendix and Fig. 6, we have

(x " $Nd)s,t"[[(x>ds) " $N

d0]=d]

s,t

"[(x>ds) " $Nd0

]s,t

=d

"[(x>ds)s,t

" $Nd0

]=d

"[(xs,t

>ds)"$Nd0

]=d

"(xs,t

"$Nd). (20)

By the duality between closing and opening, the resultsobtained in Section 4.1 and in this section can be easilyextended to the case of the NCP.

4.3. The computational complexity of the algorithms

The computational complexity of the algorithms heav-ily depends on the complexity of the geometrical struc-tures of images. Since there is no suitable model torepresent a natural image on a geometrical basis, gener-ally there is no way to have a theoretical analysis of thecomputational complexity of the algorithms. In this pa-per, the computational complexity is measured throughexperimental results. The results given in this section areonly those of the algorithms developed in Section 4.1,where the basic element is a single pixel. Based on thecomputational structure of the general NOP and NCP,we consider that this section gives a complete pictureof the computational complexity of the general NOPand NCP, since the "rst and the third steps of the compu-tation are usually very simple. Two test images `Lenaaand `Toysa are used to show the dependency of the


Fig. 8. (a) The computation time. (b) The numbers of compari-sons/per pixel.

computational complexity on the complexity of theimages. `Toysa shown in Fig. 7(b) is simpler than Lenashown in Fig. 7(a). The size of the images is 256]256.

Fig. 8(a) gives the computation time versus the size ofthe structuring elements. The results are measured ona SUN-3 workstation. In measuring the complexity of

the arithmetic operations, we only consider the computa-tion of comparison since it is the main computation ofthe NOP and NCP. In Fig. 8(b), the numbers of compari-sons/per pixel versus the size of the structuring elementsare given. Fig. 8(a) and (b) show that when the size of thestructuring elements becomes larger, the algorithms be-come faster. The fact can be explained by Proposition 3and 4. When the size of the structuring elements becomeslarger, in the computation at one pixel (i, j), the algorithmcan compute the NOP and NCP values for more pixelswhich are in S

i,jor which are the neighbors of S

i,j.

5. Application examples

Some application examples of the NOP and NCPhave already been published in other papers [1,11], InRef. [1], the adaptive "lters based on the NOP and NCPare compared with other "lters in removing impulsivenoise from monochrome images. In Ref. [11], a detailedstudy is given on the e!ect of adaptive "ltering to thecolor appearance of nature color images. Di!erent typesof noise and images are used in Ref. [11]. The resultsshowed the advantage of the adaptive "lters based on theNOP and NCP over many other well-known "lters. TheNOP and NCP used in Refs. [1,11] are the early versionswhose basic element is a single pixel. The early NOP andNCP work well in noise "ltering. But as shown in thissection, they may fail in some other areas of imageprocessing. This section still gives two examples of theearly NOP and NCP to show the detail preserving per-formance and the robustness of the operators over thesize change of the structuring elements. Then we presentan example that requires the basic element to be chosenaccording to the speci"ed image processing task, not justto be a single pixel.

5.1. Performance on synthetic image

The example given in this section is to show the detailpreserving performance of the NOP and NCP on a syn-thetic image. The basic element in this section is onepixel. The synthetic image is introduced in Ref. [10], andis used to evaluate a number of detail-preserving ran-ked-order "lters in Ref. [6]. The synthetic image shownin Fig. 9(a) is sampled from function b(r) de"ned by

a(r)"

GA cos[u

0r2/R]#128, r)R/2,

A cos[u0(R2!(r!R)2)/R]#128, R/2(r)3R/2,

b(r)"G250, a(r)*250,

a(r), 0(a(r)(250,

0, a(r))0,

(21)


Fig. 9. (a) The original synthetic image. (b) Error image by the max of openings and the min of closings. (c) Error image by the NOP andNCP.

where r is the radius from the center, u0"3.135,

R"160 and a large A"103 is used to reduce the e!ectof the discontinuity of the circles in the image caused bythe MoireH patterns.

Fig. 9(b) shows the reversed absolute di!erence be-tween the synthetic image and the image processed by themaximum of four openings and the minimum of fourclosings. The structuring elements of the four openings


Fig. 10. The e!ect of the size change of the structuring elements.

(closings) are of size N"4, and are oriented at the anglesof 0, 45, 90 and 1353. The resulting MSE is 8.08]10~2.Fig. 9(c) shows the reversed absolute di!erence betweenthe synthetic image and the image processed by theNOP and NCP. The size of the structuring element of theNOP and NCP were 7 pixels. The resulting MSE is3.68]10~4. The two sizes of the structuring elements areexperimentally shown to be adequate for the correspond-ing operators to remove 10% impulsive noise. In Fig. 9,we observe that the NOP and NCP only caused error atthe four corners, and that the details in all other partswere completely preserved.

5.2. Ewect of the size change of structuring element

The example in this section is to show the robustnessof the NOP and NCP over the size change of structuringelement. The basic element in this section is one pixel. Inpractice, the suitable size of the structuring element ofa morphological "lter is often chosen subjectively ac-cording to the type of noise and images. In this section,the sensitivities of the MSE changes with respect to thesize changes of the structuring elements of the two typesof morphological operators mentioned in Section 5.1 areinvestigated. The results shown in Fig. 10 are based on`Lenaa contaminated by 10% impulsive noise. The errorbefore the minimum point in Fig. 10 is mainly caused bythe remaining noise and that after the minimum point ismainly by the loss of details in the processed image. Fig. 10shows that after the minimum point, the wrong size of thestructuring elements cause much less performance de-terioration in the case of the NOP and NCP compared tothat of the maximum of openings and the minimum ofclosings. Such a property of the NOP and NCP may easethe demand for an optimal size of the structuringelement, and thus may make it easier to develop theNOP and NCP with an adaptive size of the structuringelement.

5.3. Performance on image decomposition

The application considered in this section is to extractthe contours of large objects in an image. The require-ment is that the extracted contour image should containas few details, such as hair or "ne grass, as possible, andthat the extracted contours should match the originallarge objects, including the detailed parts of the objects,such as the sharp corners. This type of image processinghas been used in Ref. [8] to achieve image decompositionfor coding. It may also "nd applications in pattern recog-nition and in other areas.

According to the requirement, the "ne details such ashair have to be removed before extracting the requiredcontours, since most edge-extracting approaches alsopick up "ne details. This section compares the NOP andNCP with other two morphological approaches used forremoving details. Linear approaches are not consideredsince they are known to cause large distortion of theedges and the detailed parts of the large objects in images.After the details are removed, the Sobel Operator is usedin all the cases to extract the contours for comparison.

The original image Lena is shown in Fig. 7(a). Thedetails in the image are de"ned as the objects smallerthan 30 connected 2]2 basic elements, and the lines withwidth less than two pixels. Fig. 11(a) gives the edge imageof `Lenaa without removing the details.

Fig. 11(b) gives the contour image with the detailsremoved by the NOP and NCP,

[[[(x " $Nd0

) z $Nd0

] " $Nd] z $N

d](i, j), (22)

where N"30, d0

is a pixel and d is a basic element of2]2 pixels. Two decomposition steps are used to obtaina smoother result. Fig. 11(c) shows the contour imagewith the details removed by opening-closing with onestructuring element. The result is obtained with twodecomposition steps. The structuring element in the "rststep is a square of 3]3 pixels. That in the second step isshown in Fig. 12(a), whose size is 21 pixels. Fig. 11(d)shows the contour image with the details removed by theopening}closing with a combination of four structuringelements. The decomposition has also two steps. Thestructuring elements of the "rst step are the compositionsof a 2]2 structuring element and four 1-D structuringelements of 3 pixels. The de"nition of the composition ofthe structuring elements can be found in Ref. [9]. Thestructuring elements of the second step are the composi-tion of a small structuring element and four 1-D structur-ing elements of 5 pixels shown in Fig. 12(b). The sizes ofthe structuring elements in all the cases are chosen on thebasis that the resulting images have about the sameentropy [8].

Comparing the four images in Fig. 11 shows that theNOP and NCP work well in both removing the "nedetails and preserving the detailed parts of large objects.


Fig. 11. (a) The edge image of `Lenaa without removing details. (b) The edge image with details removed by the NOP and NCP. (c) Theedge image with details removed by the opening}closing of one structuring element. (d) The edge image with details removed by theopening}closing with a combination of four structuring elements.

The contours in (b) match those in (a) very well while (b)contains no "ne details. In (c) there is obvious distortionof the contours of the detailed parts of large objects. Ascan be observed in the place of the eyes, the shapes aredistorted according to the shapes of the structuring ele-ments. (d) shows that the problem with one structuring

element is not signi"cantly alleviated by using four struc-turing elements. The reason is that the shapes of thefour structuring elements are only a very small part of allthe possible shapes of the size. Thus, the ability for theoperator to preserve the shapes of the detailed parts oflarge objects is very limited. In fact, the problem will be


Fig. 12. (a) One structuring element. (b) Composition of fourstructuring elements.

shared by many other operators with a "xed operationalwindow or with limited ability to change the shapes oftheir operational windows. In contrast, the NOP andNCP can adapt the shapes of their structuring elementsto all the possible shapes.

6. Conclusions

In this paper, we proposed the NOP and NCP withgeneralized operational windows. Quite a few interestingresults are obtained in the study of the properties of theNOP and NCP. We would like specially to mentionProperty 4 that shows the necessity and possibility tosystematically develop a geometric approach for imageprocessing. On the basis of the properties obtained, fastalgorithms are developed for the computation of theNOP and NCP. Our work has brought the computationtime of a fully adaptive operator in the range of seconds,and still shows a large room for further improvement.The distinctive performance of the NOP and NCPis demonstrated through several examples. The resultsshowed that, due to the ability to handle the imagefeatures as objects, the NOP and NCP not only areattractive for noise "ltering, but also have a great poten-tial in the areas such as coding and pattern recognition.We believe that our work in this area will not onlyo!er useful tools, but also produce innovated ideas for

image processing based on the geometric structures ofimages.

7. Summary

A new type of adaptive morphological operators wereproposed in Ref. [1]. The operational window of thoseoperators can adapt their shapes according to the geo-metrical features of images and can take any connectedshape of a given size. The adaptive morphological oper-ators in Ref. [1] were proposed to be as simple as pos-sible and their properties were not investigated. In thispaper, adaptive morphological operators are further ex-tended to allow more freedom in forming their opera-tional windows that can adapt their shapes according tothe local features of the processed images. The propertiesof these adaptive operators are also investigated. Theseproperties lead to an interesting way to handle imagesbased on the geometrical structure of images, and showthe necessity and possibility to systematically developa geometric approach for image processing. These prop-erties also lead to the development of fast algorithms forthe practical application of the adaptive operators. Ourwork has reduced computation time of a fully adaptiveoperator to be in the range of seconds, and still showspromise of further improvement. The distinctive perfor-mance of the NOP and NCP operators is demonstratedthrough several examples. The results show that, due tothe ability to handle the image features as objects, theNOP and NCP operators not only are attractive fornoise "ltering, but also have a great potential in the areassuch as coding and pattern recognition.

Appendix A. Morphological operators with one or severalstructuring elements

For reference, we give a brief description of the mor-phological operators with one structuring element and ofthe morphological operators with a combination of afamily of structuring elements. These operators are thebasis for the development of the adaptive morphologicaloperators in Section 3.

Let B denote a structuring element. Since we onlyconsider #at structuring elements, B can be expressed byits support domain BLZ2. Denote Bs"M!b : b3BN asthe symmetric set of B, and B

t1 ,t2as the translation of

B by (t1, t

2), where Mt

1, t

2NLZ2. Denote the input image

by x(i, j). The erosion x>Bs and dilation x=Bs can beexpressed as [2]

(x>Bs)(i, j)" min(t1,t2 )|Bi,j

(x(t1, t

2)), (A.1)

(x=Bs)(i, j)" max(t1,t2 )|Bi,j

(x(t1, t

2)), (A.2)


where the reference pixel of the structuring element B canbe any pixel in the domain of B. Opening x " B andclosing x z B are de"ned as [2]

(x " B)"(x>Bs)=B, (A.3)

(x z B)"(x=Bs)>B. (A.4)

On the basis of Eqs. (A.1)}(A.4), opening and closing canalso be expressed as follows:

(x " B)(i, j)" max((s1 ,s2 )>(i,j)|Bs1,s2 ) C min

(t1 ,t2 )|Bs1,s2

x(t1, t

2)D, (A.5)

(x z B)(i, j)" min((s1 ,s2 )>(i,j)|Bs1,s2 )

C max(t1,t2 )|Bs1,s2

x(t1, t

2)D. (A.6)

Let GNd

denote a family of structuring elements formedby N connected basic elements d. The morphologicaloperators combining GN

dare de"ned as the max of open-

ings and the min of closings, whose structuring elementsare all the elements in GN

d[4]. Let x"GN

dand x zGN

ddenote

the max of openings and the min of closings, whosestructuring elements are all the elements in GN

d, respec-

tively, we have

(x " GNd)(i, j)"max

B(k)|GN

d

[(x " B(k))(i, j)], (A.7)

(x z GNd)(i, j)" min

B(k)|GN

d

[(x z B(k))(i, j)]. (A.8)

The existing way to compute Eq. (A.7) (or Eq. (A.8)) is"rst to compute each opening (closing) on the right-handside of Eq. (A.7) (or Eq. (A.8)), then to take the maximum(minimum). In this way, the number of the structuringelements in GN

dis greatly limited by considerations of

computational complexity. In practice, GNd

usually con-tains only a few structuring elements. Hence, the abilityof such operators to preserve details and suppress arti"-cial patterns is limited, since an image may contain farmore than just a few patterns of signi"cant details. Infact, such a problem is shared by many other nonlinear"lters, which combine a number of "xed windows [6].

Property A.1 (Increasing).

x1)x

2N(x

1" $N

d0))(x

2" $N

d0) (A.9)

x1)x

2N(x

1z $N

d0))(x

2z $N

d0) (A.10)

Proof. The property can be proved on the basis of Prop-ositions 1 and 2.

Property A.2 (Ordering). Suppose 0(N1)N

2. Then,

x*(x " $N1d0

)*(x " $N2d0

), (A.11)

x)(x z $N1d0

))(x z $N2d0

) (A.12)

Proof. According to Proposition 1, the "rst inequality ofEq. (A.11) is obvious. To prove the second inequalityof Eq. (A.11), denote S

i,jas the domain searched in step 1

of Proposition 1 in the computation of the NOP(x " $N2

d0)(i, j) performed at (i, j). Then by Proposition 1,

(x " $N2d0

)(i, j)" min(s,t)|Si,j

x(s, t). (A.13)

Choose a subset ¹i,j

of connected N1

pixels in Si,j

,which contains (i, j). Then,

(x " $N1d0

)(i, j)"maxB(k)|$N1

d0C max((s1 ,s2 )>(i,j)|B(k)

s1,s2 )C min(t1 ,t2 )|B(k)

s1,s2

(x(t1, t

2))DD

.

(A.14)

Then the second inequality of Eq. (A.11) can be provedon the basis of Eq. (A.13) and (A.14). Eq. (A.12) can beproved in a similar way.

Property A.3 (Idempotent).

(x " $Nd0

) " $N~id0

)"(x " $N~id0

) " $Nd0

)

"(x " $Nd0

) (A.15)

(x z $Nd0

) z $N~id0

)"(x z $N~id0

) z $Nd0

)

"(x z $Nd0

), (A.16)

where i"0,2, N!1.

Proof. We only give the proof of the second equality ofEq. (A.15), the rest of the property can be proved ina similar way. On the basis of Properties A.1 and A.2, wehave

(x " $N~id0

) " $Nd0)(x " $N

d0). (A.17)

In the following, we prove that the left-hand side ofEq. (A.17) is not smaller than the right-hand side. LetSi,j

denote the domain searched in step 1 of Proposition1 in the computation of (x " $N

d0)(i, j) at (i, j). Then by

de"nition,

(x " $Nd0

)(i, j)" min(s,t)|Si,j

x(s, t). (A.18)

At an arbitrary pixel (s, t)3Si,j

, choose a connected N!ipixels subset ¹

s,tLS

i,j, which contains (s, t). Then, in the

same way as that in Eq. (A.14),

(x " $N~id0

) (s, t)* min(s1 ,t1 )|Ts,t

x(s1, t

1)* min

(s2 ,t2 )|Si,j

x(s2, t

2).

(A.19)

Based on Eq. (A.19) and Proposition 1,

(x " $N~id0

) " $Nd0

)(i, j)* min(s,t)|Si,j

(x"$N~id0

)(s, t)

* min(s2 ,t2 )|Si,j

x(s2, t

2). (A.20)


Eqs. (A.18) and (A.20) show that the left-hand side ofEq. (A.17) is not smaller than the right-hand side. Thus,the second equality of Eq. (A.15) is proved.

Property A.4 (Translation invariant). Let xs,t

denote thetranslation of x by (s, t). Then

(x " $Nd0

)(i!s, j!t)"(xs,t

" $Nd0

)(i, j), (A.21)

(x z $Nd0

)(i!s, j!t)"(xs,t

z $Nd0

)(i, j). (A.22)

Proof of Property 1. Let M(ik, jk)Dk"1,2, NN denote the

set of the N pixels in /q1 (/q1 is de"ned in the sectionbefore Property 1) on which the image x has the largestvalues. The proof is given corresponding to the four ways.

1. According to Proposition 1, it is obvious.2. When D;q1 D(N, D/q1 D*N (;q1 is de"ned in the sec-

tion before Property (1), we consider the problem intwo parts.

1. First, we consider the computation of the NOP(x " $N

d0)(i, j) at (i, j)3M(i

k, jk) D k"1,2, NN. Pixels in

M(ik, jk) D k"1,2, NN are connected since ¹q1 (¹q1

is de"ned by Eq. (4)) contains only one tip. Hence,M(i

k, jk) D k"1,2, NN can be taken as the domain

searched in step 1 of Proposition 1. Thus, for(i, j)3M(i

k, jk) D k"1,2 NN,

(x " $Nd0

)(i, j)" min(s,t)|M(ik ,jk )@k/1,2, NN

[x(s, t)]. (A.23)

Secondly, we consider all the other pixels (s, t)3M¹q1!M(i

k, j

k) D k"1,2, NNN. Based on the de"ni-

tion of ¹q1 , we can always "nd a path from (s, t) toM(i

k, j

k) D k"1,2, NN, on which the values of x are

not smaller than x(s, t). Thus, according to Proposi-tion 1, at those pixels,

(x " $Nd0

)(s, t)"x(s, t). (A.24)

The proof of (2) is complete.3. The proof can be based on Property A.3 and the

proofs of (1) and (2).4. The proof is similar to the second part of the proof

of (2).

The basic algorithm

The basic algorithm is for the calculation of the simpleNOP described in Ref. [1]. De"ne S

i,jas the domain

described in the Step 1 of Proposition 1 with the basicelement as a single pixel. The basic algorithm has threesteps. Step 0 is an initialization. Step 1 is a search at theneighbors of the pixels included in S

i,jfor the candidate

pixels considered in Step 2. In Step 2, we determine whichpixels picked up in Step 1 can be included in S

i,jaccord-

ing to the conditions in Proposition 1. If the number ofthe pixels included in S

i,jis still less than the given size of

Si,j

, the calculation goes back to Step 1 again. Steps 1 and2 constitute a search cycle of the algorithm.

Before describing the search procedure, we de"ne anumber of bu!ers and the corresponding counters.

(1) Si,j

contains the pixels included into the operationalwindow so far. Counter M indicates how many pixelsare included in S

i,j. x

0denotes the minimum value of

image x at the pixels included in Si,j

so far.(2) Bu!er Ma

nN contains the pixels included in S

i,jduring

the previous search cycle. The corresponding pixelcounter is AN.

(3) Bu!er MbnN contains the pixels, which are candidates

but are not included in Si,j

in the previous searchcycle. The corresponding pixel counter is BN.

(4) Bu!er McnN contains the neighboring pixels of Ma

nN.

The corresponding pixel counter is CN.(5) Bu!er Md

nN contains N!M pixels chosen from Mb

nN

and McnN, which correspond to the N!M largest

values of image x at the pixels in MbnN and Mc

nN, where

N is the given size of the operational window. Pixels inMd

nN are candidates in the current search cycle. The

corresponding pixel counter is DN.

The current position of the calculation is at pixel (i, j)in image x. The following is the search procedure.

Step 0: Pixel (i, j) is included.Assign the initial values:Si,j

contains 1 pixel. Thus assign M"1 andx0"x(i, j).

Assign pixel (i, j) to a1. Ma

iN contains 1 pixel, thus

assign AN"1.Mb

iN, Mc

iN, and Md

iN are empty. Thus assign

BN"CN"DN"0.Step 1: Assign the neighboring pixels of a

i, i"1,2, AN

to ci. Use a check-board to record which pixel

has been searched to avoid to pick up the pixelspreviously searched in the computation at (i, j).Assign the number of the pixels in set Mc

iN to

counter CN.Choose the N!M largest values from x(b

i)

and x(cj), i"1,2, BN, j"1,2, CN. Order the

N!M values and assign the correspondingN!M pixels to d

iin the order that

x(di)*x(b

i`1). Assign DN"N!M.

Step 2: Comparing x(di) with x

0, we have three cases:

1. If x(d1)(x

0, then assign x

0"x(d

1),

a1"d

1, AN"1; b

i"d

i`1, for i"1,2,

DN!1;1. BN"DN!1 and M"M#1.1. If M"N, assign (x"B

N)(i, j)"x

0, quit.

1. If M(N, goto step 1.


About the Author*FULIN CHENG received the B.E. degree in radio engineering from the South China University of Technology,China in 1982, and the M.E. and Ph.D. degrees in electrical engineering from Kyushu University, Japan in 1986 and 1989, respectively.He worked in the University of Toronto, Canada as a research assistant from 1989 to 1992. He is now with Zenith Electronics Crop.,USA. His research interests include multi-channel and multi-dimensional system and signal processing, image and video processing,nonlinear adaptive "ltering, as well as server and network for video on demand.

About the Author*ANASTASIOS N. VENETSANOPOULOS (SM'79}F'88) received the Dipl. Eng. degree from the NationalTechnical University of Athens (NTU), Greece, in 1965, and the M.S., M.Phil., and Ph.D. degrees in electrical engineering from YaleUniversity, New Haven, CT, in 1966, 1968 and 1969, respectively.

He joined the University of Toronto, Toronto, Ont., Canada, in September 1968, were he has been a Professor in the Department ofElectrical and Computer Engineering since 1981. He has served as Chairman of the Communication Group and Associate Chairman ofthe Department Electrical Engineering. He was on research leave at the Federal University of Rio de Janerio, Brazil, the ImperialCollege of Science and Technology, London, U.K., the National Technical University of Athens, Swiss Federal Institute of Technology,Lausanne, Swizterland, and the University of Florence, Italy, and was Adjunct professor at Concordia University, Montreal, P.Q.,Canada. He has served as Lecturer in 130 short courses to industry and continuing education programs, and as Consultant to severalorganizations. His general research interests include liner M-D and nonlinear "lters, processing of multispectral (color) image and imagesequences, telecommunications, and image compression. In particular, he is interested in the development of e$cient techniques formultispectral image transmission, restoration, "ltering, and analysis. He is a contributor to 24 books, and is co-author of NonlinearFilters in Image Processing: Principles and applications (Boston: Kluwer) and Arti"cial Neural Networks: Learning Algorithms,Performance Evaluation and applications (Boston: Kluwer), and has published over 500 papers on digital signal and image processingand digital communications.

Dr. Venetsanopoulos has served as Chairman on numerous boards, councils, and technical conference committees including IEEEcommittees, such as the Toronto Section (1977}1979) and the IEEE Central Canada Council (1980}1982). He was president of theCanadian Society for Electrical Engineering and Vice President of the Engineering Institute of Canada (EIC) (1983}1986). He has beena Guest Editor or Associate Editor for several IEEE journals, and Editor of the Canadian Electrical Engineering Journal (1981}1983).He is a member of the IEEE Communications, Circuits and Systems, Computer, and Signal Processing Societies, as well as a member ofSigma Xi, the Technical Chamber of Greece, the European Association of Signal Processing, the Association of Professional Engineersof Ontario (APEO) and Greece. He was elected as a Fellow of the IEEE `for contributions to digital signal and image processinga, isa Fellow of EIC, and was awarded an Honorary Doctorate from the National Technical University of Athens for his `contribution toengineeringa in October 1994.

2. If x(dDN

)*x0, then assign (x"BN)(i, j)"x

0,

quit.3. If x(d

k)*x

0, x(d

k`1)(x

0, 1(k(DN, as

sign ai"d

i, for i"1,2, k; AN"k;

bi"d

i`k, for i"1,2, DN!k;

BN"DN!k and M"M#k, goto Step 1.

References

[1] F. Cheng, A.N. Venetsanopoulos, Adaptive morphological"lters for image processing, IEEE Trans. Image Process.1 (4) 1992.

[2] J. Serra, Image Analysis and Mathematical Morphology,Academic press, New York, 1982.

[3] P. Maragos, R.W. Schafer, Morphological systems formultidimensional signal processing, IEEE Proc. 78 (4)(1990) 690}709.

[4] R.L. Stevenson, G.R. Arce, Morphological "lters: statisticsand further syntactical properties, IEEE Trans. CAS 34(1987).

[5] I. Pitas, A.N. Venetsanopoulos, Nonlinear Digital Filters,Kluwer Academic Publishers, Dordrecht, 1990.

[6] G.R. Arce, R.E. Foster, Detail-preserving ranked-orderbased "lters for image processing, IEEE Trans. ASSP37 (1) (1989) 83}98.

[7] E.R. Dougherty, Minimal search for the optimalmean-square digital gray-scale morphological "lter, in:M. Kunt (Ed.), Proceedings of the Visual Commun-ications and Image Processing'90, SPIE Vol. 1360, 1990,pp. 214}225.

[8] F. Cheng, A.N. Venetsanopoulos, fast, adaptive mor-phological decomposition for image compression, in: Pro-ceedings of the Conference on Information Science andSystems, Baltimore, USA, March 1991.

[9] X. Zhuang, R.M. Haralick, Morphological structuring ele-ment decomposition, Computer Vision, Graphics ImageProcess. 35 (1986) 370}382.

[10] T. Thong, Digital image processing test patterns, IEEETrans. ASSP (1983) 31.

[11] P. Deng-Wong, F. Cheng, A.N. Venetsanopoulos, Adap-tive morphological "lters for color image enhancement, J.Intelligent Robotic Systems (15) (1996) 181}207.


Documents

Adaptive morphological operators, fast algorithms and their applications