21
Fuzzy Connectedness and Image Segmentation JAYARAM K. UDUPA, SENIOR MEMBER, IEEE, AND PUNAM K. SAHA, MEMBER, IEEE Invited Paper Image segmentation—the process of defining objects in im- ages—remains the most challenging problem in image processing despite decades of research. Many general methodologies have been proposed to date to tackle this problem. An emerging frame- work that has shown considerable promise recently is that of fuzzy connectedness. Images are by nature fuzzy. Object regions manifest themselves in images with a heterogeneity of image intensities owing to the inherent object material heterogeneity, and artifacts such as blurring, noise and background variation introduced by the imaging device. In spite of this gradation of intensities, knowl- edgeable observers can perceive object regions as a gestalt. The fuzzy connectedness framework aims at capturing this notion via a fuzzy topological notion called fuzzy connectedness which defines how the image elements hang together spatially in spite of their gradation of intensities. In defining objects in a given image, the strength of connectedness between every pair of image elements is considered, which in turn is determined by considering all possible connecting paths between the pair. In spite of a high combinatorial complexity, theoretical advances in fuzzy connectedness have made it possible to delineate objects via dynamic programming at close to interactive speeds on modern PCs. This paper gives a tutorial review of the fuzzy connectedness framework delineating the various advances that have been made. These are illustrated with several medical applications in the areas of Multiple Sclerosis of the brain, magnetic resonance (MR) and computer tomographic (CT) angiography, brain tumor, mammography, upper airway disorders in children, and colonography. Keywords—Digital topology, fuzzy connectedness, image seg- mentation, medical imaging. I. INTRODUCTION A. Background Two-dimensional (2-D) digital picture processing activi- ties started in the early 1960s [1]. Very early on, Rosenfeld [2] recognized the need and laid the foundation for digital counterparts of key topological and geometric concepts that have long been available for the continuous space (point sets) such as neighborhood, connectedness, curves, boundaries, Manuscript received July 15, 2002; revised March 17, 2003. This work was supported by a DHHS grant NS37172. The authors are with the Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/JPROC.2003.817883 surroundness, distance, perimeter, and area. In particular, the concept of connectedness he developed found widespread use in image analysis such as in separating an already seg- mented object from other irrelevant structures, and in sepa- rating the component parts of a disconnected object that has also been segmented already. This concept was defined in a hard (crisp) fashion in the sense that a set of pixels was con- sidered to be either connected or not connected. Similar con- cepts were also developed for the three-dimensional (3-D) space when 3-D image processing efforts commenced in the 1970s [3]–[5]. We shall see that when this definition is gen- eralized to fuzzy subsets so as to consider a strength of con- nectedness value, it has implications far beyond just the fuzzy generalization of the concept of connectedness. Why fuzzy? Images produced by any imaging device are inherently fuzzy. This fuzziness comes from several sources. Because of spatial and temporal resolution limitations, imaging devices introduce blur. They also introduce noise and other artifacts such as background intensity variation. In addition to these factors, objects usually have material heterogeneity, and this introduces heterogeneity in the images. This is especially true in the biomedical context of the internal organs of living beings. The combined effect of all these factors is that object regions manifest themselves with a heterogeneity of intensity values in acquired images. The main rationale for taking fuzzy approaches in image analysis is to try to address and handle the uncertainties and intensity gradations that invariably exist in images as realistically as possible. It appears to us that the fuzzy treatment of the geometric and topological concepts can be made in two distinct man- ners in image analysis. The first approach is to carry out a fuzzy segmentation of the given image first so that we have a fuzzy subset of the image domain wherein every image ele- ment has a fuzzy object membership assigned to it and then to define the geometric and topological concepts on this fuzzy subset. The second approach is to develop these concepts directly on the given image, which implies that these con- cepts will have to be integrated somehow with the process of segmentation. Taking the first approach, Rosenfeld did some early work [6], [7] in developing fuzzy digital topological 0018-9219/03$17.00 © 2003 IEEE PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003 1649

Fuzzy Connectedness and Image Segmentationkcies/Jay/UdupaSaha2003.pdf · Fuzzy Connectedness and Image Segmentation JAYARAM K. UDUPA, SENIOR MEMBER, IEEE, AND PUNAM K. SAHA, MEMBER,

Embed Size (px)

Citation preview

Fuzzy Connectedness and Image Segmentation

JAYARAM K. UDUPA, SENIOR MEMBER, IEEE,AND PUNAM K. SAHA, MEMBER, IEEE

Invited Paper

Image segmentation—the process of defining objects in im-ages—remains the most challenging problem in image processingdespite decades of research. Many general methodologies havebeen proposed to date to tackle this problem. An emerging frame-work that has shown considerable promise recently is that of fuzzyconnectedness. Images are by nature fuzzy. Object regions manifestthemselves in images with a heterogeneity of image intensitiesowing to the inherent object material heterogeneity, and artifactssuch as blurring, noise and background variation introduced bythe imaging device. In spite of this gradation of intensities, knowl-edgeable observers can perceive object regions as a gestalt. Thefuzzy connectedness framework aims at capturing this notion via afuzzy topological notion called fuzzy connectedness which defineshow the image elements hang together spatially in spite of theirgradation of intensities. In defining objects in a given image, thestrength of connectedness between every pair of image elements isconsidered, which in turn is determined by considering all possibleconnecting paths between the pair. In spite of a high combinatorialcomplexity, theoretical advances in fuzzy connectedness havemade it possible to delineate objects via dynamic programmingat close to interactive speeds on modern PCs. This paper gives atutorial review of the fuzzy connectedness framework delineatingthe various advances that have been made. These are illustratedwith several medical applications in the areas of Multiple Sclerosisof the brain, magnetic resonance (MR) and computer tomographic(CT) angiography, brain tumor, mammography, upper airwaydisorders in children, and colonography.

Keywords—Digital topology, fuzzy connectedness, image seg-mentation, medical imaging.

I. INTRODUCTION

A. Background

Two-dimensional (2-D) digital picture processing activi-ties started in the early 1960s [1]. Very early on, Rosenfeld[2] recognized the need and laid the foundation for digitalcounterparts of key topological and geometric concepts thathave long been available for the continuous space (point sets)such as neighborhood, connectedness, curves, boundaries,

Manuscript received July 15, 2002; revised March 17, 2003. This workwas supported by a DHHS grant NS37172.

The authors are with the Medical Image Processing Group, Departmentof Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA(e-mail: [email protected]).

Digital Object Identifier 10.1109/JPROC.2003.817883

surroundness, distance, perimeter, and area. In particular, theconcept of connectedness he developed found widespreaduse in image analysis such as in separating an already seg-mented object from other irrelevant structures, and in sepa-rating the component parts of a disconnected object that hasalso been segmented already. This concept was defined in ahard (crisp) fashion in the sense that a set of pixels was con-sidered to be either connected or not connected. Similar con-cepts were also developed for the three-dimensional (3-D)space when 3-D image processing efforts commenced in the1970s [3]–[5]. We shall see that when this definition is gen-eralized to fuzzy subsets so as to consider a strength of con-nectedness value, it has implications far beyond just the fuzzygeneralization of the concept of connectedness.

Why fuzzy? Images produced by any imaging device areinherently fuzzy. This fuzziness comes from several sources.Because of spatial and temporal resolution limitations,imaging devices introduce blur. They also introduce noiseand other artifacts such as background intensity variation.In addition to these factors, objects usually have materialheterogeneity, and this introduces heterogeneity in theimages. This is especially true in the biomedical context ofthe internal organs of living beings. The combined effect ofall these factors is that object regions manifest themselveswith a heterogeneity of intensity values in acquired images.The main rationale for taking fuzzy approaches in imageanalysis is to try to address and handle the uncertaintiesand intensity gradations that invariably exist in images asrealistically as possible.

It appears to us that the fuzzy treatment of the geometricand topological concepts can be made in two distinct man-ners in image analysis. The first approach is to carry out afuzzy segmentation of the given image first so that we have afuzzy subset of the image domain wherein every image ele-ment has a fuzzy object membership assigned to it and then todefine the geometric and topological concepts on this fuzzysubset. The second approach is to develop these conceptsdirectly on the given image, which implies that these con-cepts will have to be integrated somehow with the process ofsegmentation. Taking the first approach, Rosenfeld did someearly work [6], [7] in developing fuzzy digital topological

0018-9219/03$17.00 © 2003 IEEE

PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003 1649

and geometric concepts. The second approach pursued by usand others has been mainly for developing the fuzzy connect-edness concepts. Rosenfeld [6] defined fuzzy connectednessby using a min-max construct on fuzzy subsets of 2-D picturedomains: the strength of connectedness of any path betweenany two pixels and is the smallest fuzzy pixel membershipalong the path and the strength of connectedness betweenand is the strength of the strongest of all paths betweenand . The process of obtaining the fuzzy subset in the firstplace is indeed the act of (fuzzy) segmentation. Through theintroduction of a fundamental concept — a local fuzzy re-lation on image elements calledaffinity [8] — it is possibleto develop the concept of fuzzy connectedness directly onthe given image and integrate it with, and actually utilize itfor facilitating, image segmentation. As we already pointedout, image elements have a gradation of intensities within thesame object region. In spite of this graded composition, theyseem to hang together (form a gestalt) within the same object.We argue that this “hanging-togetherness” should be treatedin a fuzzy manner, and an appropriate mathematical vehicleto realize this concept is fuzzy connectedness. If hard (crisp)connectedness were to be used, then it would require a pre-segmentation. In other words, it cannot influence the result ofsegmentation, except in separating a (hard) connected com-ponent. Fuzzy connectedness when utilized directly on thegiven image (via affinity), on the other hand, can influencethe segmentation result by the spatio-topological considera-tion of how the image elements hang together, rather than bystrategies of feature clustering that have used fuzzy clusteringtechniques [9] but have ignored the important topologicalproperty of hanging-togetherness. It is widely acknowledgedthat segmentation is the most crucial and difficult problem inimage processing. Any fundamental advance that empowersthe process of segmentation will take us closer to having agrip on this problem. We believe that fuzzy connectedness isone such.

B. Outline of the Paper

We have tried to make the paper as self contained as pos-sible, leaving out a few details that are irrelevant for the maindiscussion. These details may be obtained from the cited ref-erences. By nature, this paper is mathematical. The mathe-matical treatment may discourage some readers who are in-terested in only the main ideas embodied in fuzzy connect-edness. Furthermore, in a casual reading, the mathematicaldetails may submerge the central ideas that motivated themathematics. For these reasons, in the rest of this section,we delineate the key ideas involved in fuzzy connectednessin an intuitive fashion utilizing Figs. 1 and 2.

There are five key ideas underlying fuzzy connectedness— those related to affinity, scale, generalized fuzzy connect-edness, relative fuzzy connectedness, and iterative relativefuzzy connectedness. The latter three will be abbreviatedGFC, RFC, and IRFC, respectively, and fuzzy connectednesswill be referred to as FC for ease of reference from now on.

Affinity is intended to be alocal relation between every twoimage elements and . That is, if and are far apart, thestrength of this relationship is intended to be zero. If the ele-

Fig. 1. Illustration of the ideas underlying generalized fuzzyconnectedness and relative fuzzy connectedness.

Fig. 2. Illustration of the ideas underlying iterative relative fuzzyconnectedness.

ments are nearby, the strength of their affinity, lying between0 and 1, depends on the distance between them, on the homo-geneity of the intensities surrounding them, and on the close-ness of their intensity-based features to the feature values ex-pected for the object. In determining the homogeneity com-ponent and the features atand , thescalevalues at bothand are utilized. Thescaleat any element is the radius ofthe largest hyperball centered at that element within whichthe intensities are homogeneous under some criterion of ho-mogeneity. In determining the affinity betweenand , allelements in the scale regions of bothand are consulted,the reason being to make affinity (and subsequently, FC) lesssusceptible to noise and to intensity heterogeneity within thesame object region.

Contrary to affinity, FC is intended to be a global fuzzyrelation. Its strength between any two elements, such asand in Fig. 1, is the strength of the strongest of all possiblepaths between and . A path such as is a sequence ofnearby elements, and the strength of a path is the smallestaffinity of pairwise elements along the path. Imagine Fig. 1to be a gray-level image with three object regions, , and

1650 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

, and a background . With respect to , if an affinityrelation is chosen appropriately, we expect pathto have ahigher strength than any path, such as, that treads acrossmultiple object regions.

In GFC, a fuzzy connected object of a certain strengthand containing an element is the largest set of image elementscontaining that element such that the strength of connected-ness between any two elements in that set is at least. Thetheoretical framework demonstrates that this problem can besolved via dynamic programming. In RFC, all co-objects ofimportance (in Fig. 1, these are , , , ) are let tocompete for claiming membership of image elements via FC.Again, referring to Fig. 1, let , , , be seed elementsin respective objects. An element such asis considered tobelong to object if the strength of connectedness ofwithrespect to ’s seed is greater than the strength of con-nectedness of with respect to each of the other seeds. Thethreshold required in GFC is thus eliminated inRFC. InIRFC, the idea is to iteratively refine the competition rules forRFC for different objects depending upon the results of theprevious iteration. Referring to Fig. 2, for an element suchas , its strength of FC from and from are likely to besimilar due to the blurring that takes place whereandcome close together. In this case, the strongest path fromto

is therefore likely to pass through the “core” of whichis indicated by the dotted curve in the figure. This core canbe detected first and then excluded from consideration in asubsequent iteration for any path fromto to pass through.Then, we can substantially weaken the strongest path fromto compared to the strongest path fromto which is stillallowed to pass through the core. This leads us to an iterativestrategy to separate and via RFC.

The organization of this paper is as follows. In Section II,we describe some basic terms and the notations usedthroughout the paper. In Section III, the theory of fuzzyconnectedness is outlined quite independent of its connec-tion to image segmentation. In Section IV, its implicationsin image segmentation are described. In Section V, a varietyof large applications that have utilized fuzzy connectednessfor image segmentation are described, and in Section VI,our conclusions are stated with some pointers for futurework. We have tried to make this paper as self contained aspossible, leaving out only some details that may be irrelevantfor the main discussion. These details can be obtained fromthe cited references.

II. BASIC NOTATIONS AND DEFINITIONS

Let be any reference set. Afuzzy subset[10] ofis a set of ordered pairs

where . is called themembershipfunctionof in . The fuzzy subset is callednonemptyif there exists an such that ; otherwiseit is called theempty fuzzy subsetof . We use todenote an empty fuzzy subset andto denote an emptyhard set. Thefuzzy intersectionand fuzzy unionbetweentwo fuzzy subsets and of are defined as follows:

where

,where .

A 2-ary fuzzy relation in is a fuzzy subset of, where

. Since we are not interested in fuzzy-ary relations for , we drop the qualifier “2-ary”

for simplicity. We use subscripted by the fuzzy subsetunder consideration to denote the membership functionof the fuzzy subset. For hard subsets,will denote theircharacteristic function. Let be any fuzzy relation in . issaid to bereflexive, if, , ; symmetric, if,

, ; transitive, if, ,, . A fuzzy

relation is called asimilitude relationin if it is reflexive,symmetric and transitive. A binary (hard) relation in

is a subset of ; similar to a fuzzy relation, issaid to bereflexive, if, , ; symmetric,if, , ; transitive, if,

, . A binaryrelation that is reflexive, symmetric, and transitive is calledan equivalencerelation. Similitudity and equivalence areanalogous concepts in fuzzy and hard subsets.

For any , let -dimensional Euclidean space besubdivided into hypercuboids bymutually orthogonal fam-ilies of parallel hyperplanes. Assume, with no loss of gen-erality, that the hyperplanes in each family have equal unitspacing so that the hypercuboids are unit hypercubes, and weshall choose coordinates so that the center of each hypercubehas integer coordinates. The hypercubes will be calledspels(an abbreviation for “space elements”). When , spelsare calledpixels, and when they are calledvoxels.The coordinates of the center of a spel are represented by an

-tuple of integers, defining a point in . itself will bethought of as the set of all spels in with the above inter-pretation of spels, and the concepts of spels and points inwill be used interchangeably.

A fuzzy relation in is said to be afuzzy adjacencyif it is both reflexive and symmetric. It is desirable thatbe such that is a non increasing function of the dis-tance between and . It is not difficult to see that thehard adjacency relations commonly used in digital topology[11] are special cases of fuzzy adjacencies. We call the pair( ), where is a fuzzy adjacency, afuzzy digital space.Fuzzy digital space is a concept that characterizes the under-lying digital grid system independent of any image-relatedconcepts. We shall eventually tie this with image related con-cepts to arrive at fuzzy object related notions.

A scene overa fuzzy digital space ( ) is a pairwhere for some ;

is the set of -tuples of positive integers;, calledsceneintensity, is a function whose domain is, called thescenedomain, and whose range [ ] is a set of numbers (usuallyintegers). is abinary scene over( ) if the range ofis {0,1}. Binary scenes usually result from a hard segmen-tation of a given scene. When the result of segmentation isa fuzzy subset of the scene domain, this fuzzy subset can beequivalently represented by a scene wherein the scene inten-sity represents the fuzzy membership value in [0, 1]. We call

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1651

such scenesmembership scenesover ( ). Unless spec-ified otherwise, whenever we refer to a scene in this paper,it encompasses all three types — gray-level scene, member-ship scene, and binary scene. We assume in this paper that

.A nonemptypath in from to is a sequence

of spels in ; is called thelengthof the path. Note that the successive spels in a path need notbe “adjacent” in the sense hard adjacency is usually definedin digital topology [11]. When they are indeed adjacent, weshall refer to this as acontiguous path. The set of all spelsin the path is denoted by . An empty path in ,denoted , is a sequence of no spels. Paths of length 2 willbe referred to aslinks. The set of all paths in from to

is denoted by . (Note that and are not necessarilydistinct.) The set of all paths in, defined as ,is denoted by . We define a binaryjoin to operation on

, denoted “ ” as follows. For every two nonempty pathsand

[note that ],

(2.1)

(2.2)

(2.3)

(2.4)

Note that the join to operation between and isundefined if . It is shown in [8] (Proposition 2.1)that for any scene over any fuzzy digital space( ), the following relation holds for any two spels

:

(2.5)We define a binary relationgreater thanon , de-

noted “ ”, as follows. Let andbe any paths in . We say that

if and only if we can find a mapping from the set of spelsin to the set of spels in that satisfies all of the followingconditions:

1) only if .2) There exists some , for which .3) For all , whenever ,

for some and .Some examples follow:

; .Trivially, every non empty path in is greater than theempty path in .

III. T HEORY OFFUZZY CONNECTEDNESS

A. Early Works and History

The notion of the “degree of connectedness” of two spelswas first introduced by Rosenfeld [6], [7] in the contextof studying the topology and geometry of fuzzy subsets of

. He considered contiguous paths in and defined thestrength of connectednessof a contiguous path fromto in a fuzzy subset of as the smallest membership

value along the path, and thedegree of connectedness, de-noted by (“ ” to denote Rosenfeld’s approach andto distinguish it from more general approaches describedlater), as the strength of the strongest path between the twospels. Therefore

(3.1)

He used the notion of the degree of connectedness to de-fine certain topological and geometrical entities which hadbeen previously defined for hard sets of spels in. Twospels are said to beconnectedif

. He showed that this binary (hard) rela-tion of connectedness in a fuzzy subset, denoted by ,is reflexive and symmetric, but not transitive, and conse-quently, is not an equivalence relation. Therefore, thecomponents defined by may not partition , i.e., theymay have nonempty overlap. In the same paper, he definedthe number of components and genus in a fuzzy subset andthe membership value of a spel in a component using the no-tions of plateaus, tops, and bottoms and associating three dif-ferent connected sets with each top. Also, he introduced thenotions of separation and surroundedness in a fuzzy subsetand showed that surroundedness describes a weak partial or-dering, i.e., the hard binary relation defined by surrounded-ness is reflexive, antisymmetric, and transitive.

Rosenfeld’s “degree of connectedness” [6] was furtherstudied to understand the topological, geometrical, and mor-phological properties of fuzzy subsets [7]. However, therewas no indication in these works as to how this concept couldbe exploited to facilitate image segmentation. Dellepianeand Fontana [12], [13] and Udupa and Samarasekera [8],[14] were the first to suggest this use. Dellepianeet al.utilized Rosenfeld’s “degree of connectedness” to arrive at asegmentation method (to be discussed in Section IV). Udupaet al. [8] simultaneously introduced a different framework,bringing in a key concept of a local fuzzy relation calledaffinity on spels to capture local hanging-togetherness ofspels. They showed how affinity can incorporate variousimage features in defining fuzzy connectedness, presented ageneral framework for the theory of fuzzy connectedness,and demonstrated how dynamic programming can be uti-lized to bring the otherwise seemingly intractable notion offuzzy connectedness into segmentation. They also advancedthe theory of fuzzy connectedness considerably [15]–[19],bringing in notions of relative fuzzy connectedness [17],[19] which was further extended by Herman and De Car-valha [20], iteratively defined relative fuzzy connectedness[18], [17], and addressing the virtues of the basic min–maxconstruct used in fuzzy connectedness [16]. Affinity formsthe link between fuzzy connectedness and segmentation.Sahaet al. [15] studied the issue of how to construct ef-fective affinities and the use of local scale for this purpose.Aspects related to the computational efficiency of fuzzyconnectedness algorithms have also been studied [21], [22].

The general framework of fuzzy connectedness based onaffinity has been utilized in conjunction with other methodsof segmentation [23]–[28], particularly with deformable

1652 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

models [23], [24], a Voronoi diagram based method [25], andlevel set methods [26]. The fuzzy connectedness methodshave been utilized for image segmentation extensively inseveral applications, including Multiple Sclerosis lesionquantification [29]–[36], late life depression [37], [38], MRangiography [39], [40], CT angiography [41], [42], braintumor assessment [43], [44], breast density quantificationvia mammograms [45], craniofacial surgery [46], and CTcolonography [47].

In the rest of this section, we shall delineate the generalfuzzy connectedness theory based on affinity and its exten-sions to relative and iterative relative fuzzy connectedness.We will present all pertinent details to understand the struc-ture of the theory but only state the main results as theorems.For details including proofs, please refer to [8], [15]–[19].

B. Generalized Fuzzy Connectedness

Fuzzy Affinity: Let be any scene over ( ).Any fuzzy relation in is said to be afuzzy spel affinity(or, affinity for short)in if it is reflexive and symmetric. Inpractice, for to lead to meaningful segmentation, it shouldbe such that, for any , is a function of 1) thefuzzy adjacency betweenand ; 2) the homogeneity of thespel intensities at and ; 3) the closeness of the spel inten-sities and of the intensity-based features ofand to someexpected intensity and feature values for the object. Further,

may depend on the actual location ofand (i.e.,is shift variant). A detailed description of a generic func-

tional form for and how to select the right affinity for aspecific application are presented in Section IV. Throughoutthis paper, with appropriate subscripts and superscripts willbe used to denote fuzzy spel affinities.

Note that affinity is intended to be a “local” relation. Thisis reflected in the dependence of on the degree ofadjacency of and . If and are far apart, willbe zero. How the extent of the “local” nature ofis to bedetermined will be addressed in Section IV. We note thatthe concept of affinity is applicable to membership and bi-nary scenes also. If the scene is a membership scene, thenaffinity should depend perhaps only on adjacency and ho-mogeneity of spel intensities. Affinity has meaning even inbinary scenes, wherein it should depend only on fuzzy adja-cency for like-valued spels.

Path Strength and Fuzzy Connectedness:Our intent isthat fuzzy connectedness is a global fuzzy relation which as-signs a strength to every pair ( ) of spels in . It makessense to consider this strength of connectedness to be thelargest of the strengths assigned to all paths betweenandas originally suggested by Rosenfeld. (The physical analogyone may consider is to think ofand as being connected bymany strings, each with its own strength. Whenand arepulled apart, the strongest string will break at the end, whichshould be the determining factor for the strength of connect-edness betweenand .) However, it is not so obvious asto how the strength of each path should be defined. Severalmeasures based on the affinities of spel pairs along the path,including their sum, product, and minimum, all seem plau-sible. Saha and Udupa [16] have shown that the minimum of

affinities is the only valid choice for path strength (under aset of reasonable assumptions stated in Axioms1 – 4 below)so as to arrive at important results that allow us to computefuzzy connectedness via dynamic programming. We shall ex-amine some details pertaining to this tenet now.

Let be a scene over a fuzzy digital space( ) and let be a fuzzy affinity in . A fuzzy -netin is a fuzzy subset of with its membership function

. assigns a strength to every path of. For any spels , is called astrongest

path from to if . The ideaof -net is that it represents a network of all possible pathsbetween all possible pairs of spels inwith a strength as-signed to each path.

Axiom 1: For any scene over ( ), for any affinityand -net in , for any two spels , the strength

of the link from to is the affinity between them; i.e.,.

Axiom 2: For any scene over( ), for any affinityand -net in , for any two paths ,implies that .

Axiom 3: For any scene over( ), for any affinityand -net in , fuzzy -connectedness in is a fuzzyrelation in defined by the following membership function.For any

(3.2)

For fuzzy connectedness, we shall always use the upper caseform of the symbol used to represent the corresponding fuzzyaffinity.

Axiom 4: For any scene over( ), for any affinityand -net in , fuzzy -connectedness inis a symmetricand transitive relation.

Axiom 1 says that, a link is a basic unit in any path, andthat the strength of a link (which will be utilized in definingpath strength) should be simply the affinity between the twocomponent spels of the link. This is the fundamental way inwhich affinity is brought into the definition of path strength.Note that, in a link in a path (unless it is a con-tiguous path), and may not always be adjacent (inthe sense “adjacency” is usually considered in hard digitaltopology) — that is, and may be far apart differingin some of their coordinates by more than 1. In such cases,Axiom 1 guarantees that the strength of is deter-mined by and not by “tighter” paths of the form

wherein the successivespels are indeed adjacent. Sinceis by definition reflexiveand symmetric, this axiom guarantees that link strength isalso a reflexive and symmetric relation in. Axiom 2 guar-antees that the strength of any path changes in a non in-creasing manner along the path. This property is sensible andbecomes essential in casting fuzzy connected object tracking(whether in a given membership scene or in a given scene) asa dynamic programming problem. Axiom 3 says essentiallythat the strength of connectedness betweenand should be

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1653

the strength of the strongest path between them. Its reason-ableness has already been discussed. Finally, Axiom 4 guar-antees that fuzzy connectedness is a similitude relation in.This property is essential in devising a dynamic program-ming solution to the otherwise seemingly prohibitive com-binatorial optimization problem of determining a fuzzy con-nected object. Using the above axioms, the following maintheorem was proved in [16].

Theorem 1: For any scene over( ), andfor any affinity and -net in , fuzzy -connectedness

in is a similitude relation in if and only if

where is the path .Following the spirit of the above theorem, path strength

from now on will be defined as follows:

(3.3)

where is the path . For the remainder of thispaper, we shall assume the above definition of path strength.

Fuzzy Connected Objects [8], [16]:Let beany scene over ( ), let be any affinity in , and letbe a fixed number in [0,1]. Let be any subset of . Weshall refer to as the set ofreference spels orseed spels andassume throughout that . A fuzzy -object of

containing a seed spel of is a fuzzy subset ofwhose membership function is

ifotherwise.

(3.4)

In this expression, is an objectness functionwhose do-main is the range of and whose range is [0,1]. It maps im-aged scene intensity values into objectness values. For mostsegmentation purposes,may be chosen to be a Gaussianwhose mean and standard deviation correspond to the inten-sity value expected for the object region and its standard de-viation (or some multiple thereof). The choice ofshoulddepend on the particular imaging modality that generatedand the actual physical object under consideration. (When ahard segmentation is desired, (defined bellow) willconstitute the (hard) set of spels that represents the extent ofthe physical object and will simply be the characteristicfunction of .) is a subset of satisfying allof the following conditions:

(3.5)

(3.6)

(3.7)

is called thesupport of , i.e., a maximalsubset of such that it includes and the strength of con-nectedness between any two of its spels is at least.

A fuzzy -object of containing a set of seedspels of is a fuzzy subset of whose membership functionis

ifotherwise.

(3.8)is thesupport of , i.e., the union of the sup-

ports of the fuzzy connected objects containing the individualseed spels of .

Several important properties of fuzzy connected objectshave been established [8], [16]. The following theoremgives a guidance as to how a fuzzy-object ofshould be computed. It is not practical to use the definitiondirectly for this purpose because of the combinatorialcomplexity. The following theorem provides a practical wayof computing .

Theorem 2: For any scene over ( ), forany affinity in , for any , for any objectnessfunction , and for any non empty set , the support

of the fuzzy -object of containing equals

(3.9)

The above theorem says that, instead of utilizing(3.5)–(3.8) that define the support of the fuzzy

-object containing and having to check the strengthof connectedness of every pair ( ) of spels for eachseed , we need to check the maximum strength ofconnectedness of every spelwith respect to the seedspels. This is a vast simplification in the combinatorialcomplexity. as shown in (3.9) can be computed viadynamic programming, given, , , and , as describedin Section IV.

The following theorem characterizes the robustness ofspecifying fuzzy -objects through sets of seed spels.

Theorem 3: For any scene over ( ), forany affinity in , for any , for any objectness func-tion , and for any non empty sets ,

if and only if and .The above theorem has important consequences in the

practical utilization of the fuzzy connectedness algorithmsfor image segmentation. It states that the seeds must beselected from the same physical object and at least one seedmust be selected from each physically connected region.High precision (reproducibility) of any segmentation algo-rithm with regard to subjective operator actions (and withregard to automatic operations minimizing these actions),such as specification of seeds, is essential for their practicalutility. Generally, it is easy for human operators to specifyspels within a region in the scene corresponding to the samephysical object in a repeatable fashion. Theorem 3 guaran-tees that, even though the sets of spels specified in repeatedtrials (by the same or other human operators) may not be thesame, as long as these sets are within the region of the samephysical object in the scene, the resulting segmentations willbe identical. It is important to point out that the intensitybased connectedness method as proposed by Dellepianeetal. [12], [13] (see Section IV) fails to satisfy this robustness

1654 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

Fig. 3. Illustration used for defining relative fuzzy connectedness.

property. Many other region growing algorithms [48] whichadaptively change the spel inclusion criteria during thegrowth process cannot guarantee this robustness property.

Other interesting properties of fuzzy connectedness andof fuzzy connected objects have been studied in [8], [16]. Ithas been shown that the support of the fuzzy con-nected object monotonically decreases in size withthe strength of connectedness thresholdwhile monotoni-cally increases with the set of seed spels. It has also beenshown that is connected (in the hard sense) if andonly if the strength of connectedness between every two seedspels is at least.

C. Relative Fuzzy Connectedness [19], [17]

In the original fuzzy connectedness theory [8] as describedin Section III-B, an object is defined on its own based onthe strength of connectedness utilizing a threshold. Thekey idea of relative fuzzy connectedness is to consider allco-objects of importance that are present in the scene andto let them to compete among themselves in having spelsas their members. Consider a 2-D scene composed of mul-tiple regions corresponding to multiple objects as illustratedin Fig. 1. If is the object of interest, then the rest of theobjects , , and may be thought of as constituting thebackground as far as is concerned. With such a thinking,Fig. 3 shows just two objects , the object of interest, and

, the background, equivalent to the scene in Fig. 1. Al-though the theory for RFC can be developed for simultane-ously considering multiple objects [49], for simplicity, weshall describe here the two-object case, but keeping in mindthe treatment of object grouping mentioned above.

Suppose that the path shown in Fig. 3 represents thestrongest path betweenand . The basic idea in relativefuzzy connectedness is to first select reference spelsand, one in each object, and then to determine to which ob-

ject any given spel belongs based on its relative strength ofconnectedness with the reference spels. A spel, for ex-ample, would belong to since its strength of connected-ness with is likely to be greater than that with. This rela-tive strength of connectedness offers a natural mechanism forpartitioning spels into regions based on how the spels hang

together among themselves relative to others. A spel suchas in the boundary between and will be grabbed bythat object with whom hangs together most strongly. Thismechanism not only eliminates the need for a threshold re-quired in the GFC method but also offers potentially morepowerful segmentation strategies for two reasons. First, it al-lows more direct utilization of the information about all ob-jects in the scene in determining the segmentation of a givenobject. Second, considering from a point of view of thresh-olding the strength of connectedness, it allows adaptivelychanging the threshold depending on the strength of connect-edness of objects that surround the object of interest. Theformal definition of relative fuzzy connected objects alongwith their important properties are presented in this section.

For any spels in , define

(3.10)

The idea here is that and are typical spels specified in“object” and “background”, respectively. Note that

, if .A fuzzy -object of a scene containing a spelrelative to a background containing a spelis the fuzzy

subset of defined by the following membership function.For any ,

ifotherwise.

(3.11)

For short, we will refer to as simply arelative fuzzy -ob-ject of . The following proposition states the cohesivenessof spels within the same relative fuzzy connected object.

Proposition 4: For any scene over ( ),for any affinity in , and for any spels and insuch that ,

(3.12)

if, and only if, .Note that the above result is not valid if “” is used in

(3.10) instead of “ ”.This proposition asserts the robustness of the relative fuzzy

-object to the reference spel selected in the object region.However, constancy of the-object with respect to referencespels specified in the background requires more constraints,as indicated by the following theorem.

Theorem 5: For any scene over ( ), forany affinity in , and for any spels and in suchthat

(3.13)

Note that the condition in (3.13) is sufficient (for), but not necessary. The necessary and sufficient condi-

tion is expressed in the following theorem.Theorem 6: For any scene over ( ), for

any affinity in , and for any spels and in suchthat ,

(3.14)

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1655

The above two theorems have different implications in thepractical computation of relative fuzzy-objects in a givenscene in a repeatable, consistent manner. Although less spe-cific, and therefore more restrictive, Theorem 5 offers practi-cally a more relevant guidance than Theorem 6 for selectingspels in the object and background so that the relative fuzzy

-object defined is independent of the reference spels.An algorithm for determining relative fuzzy connected ob-

jects follows directly from algorithms for objects based ongeneralized fuzzy connectedness as described in Section IV.

D. Iterative Relative Fuzzy Connectedness [18], [17]

The principle behind this strategy is to iteratively refinethe competition rules for different objects depending uponthe results of the previous iteration. Consider the situation il-lustrated in Fig. 2 which demonstrates three objects,and . It is very likely that, for a spel such as,

because of the blurring that takes place in thoseparts where and come close together. In this case, thestrongest path from to is therefore likely to pass throughthe “core” of which is indicated by the dotted curve inthe figure. This core (which is roughly defined in Sec-tion III-C) can be detected first and then excluded from con-sideration in a subsequent iteration for any path fromto

to pass through. Then, we can substantially weaken thestrongest path fromto compared to the strongest path from

to which is still allowed to pass through the core. Thisleads us to an iterative strategy to grow from(and so com-plementarily from ) to more accurately capture (and )than if a single-shot relative connectedness strategy is used.The phenomenon illustrated in Fig. 2 is general and may becharacterized in the following way. Most objects have a corepart, which is relatively easy to segment after specifying theseed spel, and other diffused, subtle and fine parts that spreadoff from the core, which pose segmentation challenges. Al-though the latter seem to hang together fairly strongly withthe core from a visual perceptual point of view, because of theubiquitous noise and blurring, it is difficult to devise compu-tational means to capture them as part of the same object byusing a single-shot strategy. The iterative strategy capturesthese loose parts in an incremental and reinforcing manner.This formulation is described in this section.

For any fuzzy affinity and any two spels , define

(3.15)

(3.16)

Note that is exactly the same as , defined in (3.10).Assuming that and are defined for any positiveinteger , and are defined as follows. For any

ifif orotherwise

(3.17)

(3.18)

An iteratively defined fuzzy -object of a scenecontaining a spel relative to a background con-

taining a spel is a fuzzy subset of defined by the mem-bership function

ifotherwise.

(3.19)

For short, we will refer to simply as aniterative relativefuzzy -object of . Below, we state a few important prop-erties of iterative relative fuzzy connectedness.

Proposition 7: For any scene over ( ),for any affinity in , for any spels and in , and for anynon negative integers , .

The above proposition states essentially that is noncontracting as iterations continue. The following propositionstates that and maintain their disjoincy at everyiteration for any and .

Proposition 8: For any scene over ( ),for any affinity in , for any spels in , and for anynon-negative integers , .

The following theorem states an important property of it-erative relative fuzzy -objects and shows their robustnesswith respect to reference seed spels. It is analogous to The-orem 5 on non iterative relative fuzzy-objects.

Theorem 9: For any scene over ( ), forany affinity in , for any spels , and in such that

, and for any non negative integer, if.

Algorithms for determining iterative relative fuzzy objectsare more complicated. See Section IV for a further discussionon this topic.

IV. M ETHODS AND ALGORITHMS

A. Affinity Definition

As pointed out earlier, affinity is the link between thetopological concept of fuzzy connectedness and its utilityin image segmentation. As such, its definition determinesthe effectiveness of segmentation. A detailed formulation ofaffinity definition was presented in [15]. In the same paper,it was first explicitly pointed out that, besides adjacency —a purely spatial concept — affinity should consist of twoadditional and distinct components — homogeneity-basedand object feature-based components. This formulationcan also be identified in the original affinity-based fuzzyconnectedness work [8] although this was not explicitlystated. These two components are quite independent andthere exist certain dichotomies between them. The objectfeature-based component does not capture the notion ofpath homogeneity. To clarify this, consider two spelsand

that are in the same object region but that are far apart.Assume that there is a slow varying background intensitycomponent such as that often found in MR images due tomagnetic field inhomogeneities. Spelsand are likely tohave very different scene intensities although they belongto the same object. Nevertheless, one can find a contiguouspath from to in the scene domain such that the intensitiesof each pair ( ) of successive spels in the path are very

1656 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

similar. That is, and have a strong homogeneity-basedaffinity. Conversely, the homogeneity-based componentalone cannot adequately capture the notion of a globalagreement to known object intensity characteristics. Againconsider a path from to along which spel intensitychanges very slowly. The path may pass through differentobject regions. Without an object feature-based component,this path will indicate a strong connectedness between

and and therefore will merge the different objects itpasses through into one object. The following is a generalfunctional form for as proposed in [8], [15]

(4.20)

where and represent the homogeneity-based andobject-feature-based components of affinity, respectively.and may themselves be considered as fuzzy relations in

. The strength of relation indicates the degree of localhanging togetherness of spels because of their similarities ofintensities. The strength of relationindicates the degree oflocal hanging togetherness of spels because of the similarityof their feature value to some (specified) object feature.General constraints on the functional form ofwere de-scribed in [15] and the following examples were presented:

(4.21)

(4.22)

(4.23)

(4.24)

(4.25)

A fundamental question that arises in defining affinity(both and ) is how to determine the extent of the neigh-borhood of spels. (This question also arises in all imageprocessing operations that entail a local process.) If theintensities, or features derived from them, at the level ofthe individual spels are considered for definingand ,this definition will be sensitive to noise. The idea then is toconsult a neighborhood around each ofand in defining

and such that the neighborhood representsa region within the same object as in whichor is situated.

“Scale” is a fundamental, well-established concept inimage processing [50]–[52]. The premise behind thisconcept is to consider the local size of the object as theneighborhood size in carrying out whatever local operationsthat are done on the image. Sahaet al. [15] introduced anew notion of location-specific object scale. Roughly,objectscale in at any spel is the radius of the largesthyperball centered atwhich lies entirely in the same objectregion. Ironically, all this is done exactly for the purposeof defining the object in the first place and it appears thatobject definition is needed first to define scale. A simple andeffective algorithm is presented in [15] to estimate objectscale at each spel without explicit object definition butbased on the continuity of intensity homogeneity alone. Toestimate scale at a spel, a fraction of object on the

periphery of a digital ball with center at and radiusis computed as follows:

(4.26)

where is a homogeneity function. As demonstrated in[15], a zero-mean unnormalized Gaussian is generally a goodchoice for . The standard deviation parameter de-pends on the overall noise level in the image. An automaticmethod to determine this parameter for a specific image is de-scribed in [15]. The scale computation algorithm iterativelyincreases the ball radiusby 1, starting from 1, and checksfor the fraction of the object containing that is con-tained in the ball. The first time when this fraction falls belowa threshold level, the ball is considered to have entered intoa significantly different region. See [15] for details.

In the rest of this section, we describe how scale is utilizedin defining and . Let denote the digital ball de-fined by

(4.27)

First, to define , consider any two spels suchthat . Consider any spels and

such that they represent the corresponding spelswithin and , that is . We willdefine two weighted sums and of differ-ences of intensities between the two balls as follows:

ifotherwise

(4.28)ifotherwise

(4.29)

and

(4.30)

(4.31)

and are window functions (such as a Gaussian). Wenote that the parameters of depend on and .

The connection of the above equations to the homo-geneity-based affinity is as follows. There are two typesof intensity variations surroundingand — intra- and in-terobject variations. The intraobject component is generallyrandom, and therefore, is likely to be near 0 overall. Theinterobject component, however, has a direction. It eitherincreases or decreases along the direction given by ,and is likely to be larger than the intraobject variation. It is

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1657

reasonable, therefore, to assume that the smaller ofand represents the intraobject component and theother represents the combined effect of the two components.(Note that when the values of (respectively, ) aresmall, (respectively, ) also becomessmall.) Note that, if there is a slow background componentof variation, within the small neighborhood considered,this component is unlikely to cause a variation comparableto the interobject component. This strategy leads us to thefollowing functional form for :

(4.32)

Note that represents the degree oflocal inhomogeneity of the regions containingand . Itsvalue is low when both and are inside an (homogeneous)object region. Its value is high whenand are in the vicinityof (or across) a boundary. The denominator in (4.32) is anormalization factor.

In the formulation of , instead of considering directly thespel intensities , we will consider a filtered version of itthat takes into account the ball at defined by

(4.33)

The filtered value at any is given by

(4.34)

where (another window function) and its parameters de-pend on . The functional form of is given by (4.35),shown at the bottom of the page, where and are in-tensity distribution functions for the object and background,respectively. See [15] for details on how the parameters ofthese and other functions can be estimated for a given appli-cation.

B. Algorithms

Dellepianeet al. [12], [13] used the formulation of fuzzyconnectedness proposed by Rosenfeld [6], [7] for image seg-mentation. Briefly, their method works as follows. Let

be a given scene and let be the maximum intensityover the scene. A membership scene is firstcreated, where, for all , . Let de-note a seed spel; another membership sceneis computed from and , where, for all

Finally, the intensity-based connectedness ofwith respectto the seed is expressed as another membership scene

, where, for all

A hard segmentation is obtained by thresholding . Theauthors presented a non iterative algorithm using a tree ex-pansion mechanism for computing given . Note thatthe above intensity based connectedness is sensitive to seedselection as the seed intensity is used for computing, andsubsequently for the connectedness .

The affinity-based approach of Udupa and Samarasekera[8], [14] not only overcomes this difficulty of dependence onseed points (Theorems 3, 5, 6, and 9) but also made it pos-sible to extend the fuzzy connectedness concept to relativeand iterative relative fuzzy connectedness (with much rele-vance in practical image segmentation) and to demonstrate(Theorem 2) that the well-developed computational frame-work of dynamic programming [53] can be utilized to carryout fuzzy connectedness computations efficiently.

In this section we describe two algorithms [8] for ex-tracting a fuzzy -object containing a set of spels ina given scene for a given affinity in , both based ondynamic programming, and another for computing iterativerelative fuzzy connected object. The first algorithm, named

( -fuzzy object extraction for multiple seeds),extracts a fuzzy -object of of strength generated bythe set of reference spels. In this algorithm, the value of

is assumed to be given as input and the algorithm usesthis knowledge to achieve an efficient determination of the

-object. The second algorithm, named , out-puts what we call a -connectivity sceneof generated bythe set of reference spels defined by

.Algorithm terminates faster than

for two reasons. First, produces the hard setbased on (3.9). Therefore, for any spel , once we find apath of strength or greater from any of the reference spels to, we do not need to search for a better path upto, and hence,

can avoid further processing for. This allows us to reducecomputation. Second, certain computations are avoided forthose spels for which .

Unlike , computes the best pathfrom the reference spels ofto every spel in . Therefore,every time the algorithm finds a better path upto, it mod-ifies the connectivity value at and subsequently processes

if andifotherwise

(4.35)

1658 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

other spels which are affected by this modification. The algo-rithm generates a connectivity scene of .Although, terminates slower, it has a practicaladvantage. After the algorithm terminates, one can interac-tively specify and thereby examine various-objects andinteractively select the best. The connectivity scene has in-teresting properties relevant to classification and in shell ren-dering and manipulation [54] of -objects.

AlgorithmInput: , , , , and as defined inSections II & IV .

Output: as defined in Section IV .Auxiliary Data Structures: An -Darray representing a temporary scene

such that corresponds tothe characteristic function of ,and a queue of spels. We refer tothe array itself by for the purposeof the algorithm.

begin1. set all elements of to 0 exceptthose spels which are set to 1;2. push all spels such that forsome to ;3. while is not empty do4. remove a spel from ;5. if then6. set ;7. push all spels such that

to ;endif ;

endwhile ;8. create and output by as-signing the value

to all for which and 0 tothe rest of the spels;

end

AlgorithmInput: , , and as defined Sec-tions II & IV .

Output: A scene representingthe -connectivity scene of gen-erated by .

Auxiliary Data Structures: An -D arrayrepresenting the connectivity scene

and a queue of spels. Werefer to the array itself by for thepurpose of the algorithm.

begin1. set all elements of to 0 exceptthose spels which are set to 1;2. push all spels such that, forsome , to ;3. while is not empty do4. remove a spel from ;

5. find ;6. if then7. set ;8. push all spels such that

to ;endif ;

endwhile ;9. output the connectivity scene ;

end

Both algorithms work in an iterative fashion, and in eachiteration in thewhile-doloop, they remove exactly one spelfrom the queue and check if the strength of connectednessat that spel needs to be changed. In case a change is needed,the change is passed on to the rest of the scene through theneighbors by entering them into. It may be noted that bothalgorithms start after initialization of strengths of connect-edness at the seed spels. Also, note that the strength of con-nectedness at any spel never reduces during runtime of thesealgorithms. It was shown [8], [16] that both algorithms ter-minate in a finite number of steps, and when they do so, theyproduce the expected results. Recently, studies [21], [22] onthe speed up of these algorithms have been reported. A pre-liminary effort was made by Carvalhoet al. [21] by usingDijkstra’s algorithm [55] in place of the dynamic program-ming algorithm suggested in [8]. Based on 2-D phantomsand two 2-D MR slice scenes, they suggested that a seven-to-eightfold speed-up in fuzzy connectedness object computa-tion can be achieved. Nyul and Udupa [22] made an exten-sive study on the speed of a set of 18 algorithms utilizinga variety of 3-D scenes. They presented two groups of al-gorithms — label-correcting and label-setting — and a va-riety of strategies of implementation under each group. Theytested these algorithms in several real medical applicationsand concluded that there is no “always optimal” algorithm.The optimality of an algorithm depends on the input data aswell as on the type of affinity relations used. In general, itwas found that, for blob-like objects with a spread-out distri-bution of their intensities or other properties (such as braintissues in MRI), label setting algorithms with affinity-basedhashing outperform others, while, for more sparse objects(vessels, bone) with high contrast (MRA, CTA), label settingalgorithms with geometric hashing are more speed-efficient.Also, they demonstrated that using the right algorithms onfast hardware (1.5 GHz Pentium PC), interactive speed (lessthan one second per 3-D scene) of fuzzy object segmentationis achievable.

Given , an algorithm for relative fuzzy con-nectedness is easy to realize. First, the connectivity scenes

and are computed usingfor seeds and , respectively. Subsequently,

a spel is included in the support of the fuzzyconnected object of relative to the background containing

if . Iterative relative fuzzy connected-ness is an iterative method; the connectivity scene

is computed once and in each iteration, the connec-tivity scenes is computed for seed spel

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1659

and affinity as defined in (3.17) in each iteration. Thealgorithm for computing iterative relative fuzzy connected-ness [17], [18] is presented below.

AlgorithmInput: , , spels and , andas defined in Section II .

Output: Iteratively defined fuzzy -ob-ject containing relative to abackground containing .

Auxiliary Data Structures: The -connec-tivity scene of in ; the

-connectivity scene ofin .

begin0. compute using with theinput as , , and ;1. set and ;2. set , and ;3. while is true do4. set , , and

;5. compute using with the

input as , , and ;6. set ;7. for all do8. if then9. set ;10. increment by 1;11. for all do12. set ;

endfor ;13. else set ;

endif ;endfor ;

14 if then set;

15. increment by 1;endwhile ;15. output ;

end

V. APPLICATIONS

In this section, we will describe seven applicationsin which the fuzzy connectedness algorithms have beenutilized and tested extensively. It is one thing to have apowerful segmentation framework and quite another matterto put it to practical use, such as in clinical radiology, forroutinely processing 100s and 1000s of scenes. The latterusually involves considerable work in 1) engineering theproper utilization of the framework, and 2) evaluating theefficacy of the method in each application. For the fuzzyconnectedness framework, the first task entails the determi-nation of the affinity relation appropriate for the application,choosing one among generalized, relative, and iterativerelative fuzzy connectedness strategies, and devising otherauxiliary processing and visualization methods needed

under the application. The second task consists of assessingthe precision, accuracy, and efficiency of the completemethod in the application [56]. Precision here refers to thereliability (reproducibility) of the complete method takinginto consideration all subjective actions that enter into theentire process including possible variations due to repeatacquisition of the scenes for the same subject on the sameand different scanners, and any operator help and inputrequired in segmentation. Accuracy relates to the agreementof the segmented result to truth. Efficiency refers to thepractical viability of the segmentation method. This hasto consider pure computational time and the time requiredin producing any operator help needed in segmentation.The above two tasks have been accomplished under eachapplication (except evaluative studies for the last applicationwhich are very different in their requirement from otherapplications). For reasons of space, we shall not presenthere details of either the engineering or the evaluationstudies, but refer the reader to references given under eachapplication. In the rest of this section, we give a cursorydescription of each of the seven application areas outliningthe segmentation need and methodology.

A. Multiple Sclerosis (MS)

Multiple Sclerosis (MS) is an acquired disease of thecentral nervous system characterized primarily by multifocalinflammation and destruction of myelin [57]. Inflamma-tion and edema are accompanied by different degrees ofdemyelination and destruction of oligodendrocytes, andmay be followed by remyelination, axonal loss and/orgliosis. The highest frequency of MS occurs in northernand central Europe, Canada, the United States, and NewZealand and South of Australia [58]. In the US, it affectsapproximately 350 000 adults and stands out as the mostfrequent cause of nontraumatic neurologic disability inyoung and middle-aged adults. In its advanced state, thedisease may severely impair the ambulatory ability andmay even cause death. The most commonly used scaleto clinically measure disease progression in MS is theexpanded disability status scale (EDSS) introduced byKurtzke [59]. The clinical quantification of disease severityis subjective and sometimes equivocal. The development ofnew treatments demands objective outcome measures forrelatively short trials. Therefore, MR imaging has becomeone of the most important paraclinical tests for diagnosingMS and in monitoring disease progression in MS.

Various MRI protocol have been used in studying MS [34].At present they include dual-echo T2-weighted imaging,T1-weighted imaging with and without Gadoliniumadministration, magnetization transfer imaging, diffu-sion-weighted imaging, and MR spectroscopy. The differentprotocols convey information about different aspects ofthe disease. Current MRI-based research on MS is focusedon 1) characterizing the natural course of the disease andto distinguish among disease subgroups through images,and 2) characterizing treatment effects to assess differenttreatment strategies. In achieving these goals, several imageprocessing operations are utilized, most crucial among them

1660 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

Fig. 4. Top row: A slice of a T2- and proton-weighted scene of an MS patient’s head and thesegmented brain parenchymal region. Below row: the segmented CSF region and the brain lesions(which appear hyper intense in both T2 and PD scenes).

being segmentation of MR brain images in the different pro-tocols into different tissue regions and lesions. Our efforts inthis direction utilizing fuzzy connectedness and the ensuingclinical results are summarized in [34]. Fig. 4 shows anexample of a T2- and proton-density-weighted scene of anMS patient’s head and the segmented tissue regions.

A somewhat similar problem from the image processingperspective arises in the study of late-life depression [37],[38]. The aim here is to understand the disease and its treat-ment effects in the case of white matter lesions occurring inelderly subjects due to age and the associated effects on thefunction of their brain. In MS (and other neurological ap-plications), a variety of MRI protocols are utilized. The ac-tual imaging parameters used in these protocols vary amonginstitutions. In spite of numerous brain MR image segmen-tation methods developed during the past 15 years, none ofthem is capable of handling variations within the same pro-tocol, and much less, the variation among protocols. What weneed is a segmentation “workshop” wherein a protocol-spe-cific segmentation method can be quickly fabricated. In [60],one such workshop is described utilizing the fuzzy connect-edness framework.

B. MR Angiography

MR angiography (MRA) plays an increasingly importantrole in the diagnosis and treatment of vascular diseases.Many methods are available to acquire MRA scenes; themost common is perhaps acquisition with an intravascularcontrast agent such as Gadolinium. An image processing

challenge in this application is how to optimally visualizethe imaged vascular tree from the acquired scene. In themaximum intensity projection (MIP) method commonlyused for this purpose, the maximum voxel intensity encoun-tered along each projection ray is assigned to a pixel in theviewing plane to create a rendition as illustrated in Fig. 5.Although, the MIP method does not require prior segmen-tation of the scene, it obscures the display with clutterdue to high intensities coming from various artifacts asshown in Fig. 5. By segmenting those aspects of the vesselscontaining bright intensities via fuzzy connectedness, theclutter can be mostly eliminated as illustrated in Fig. 5 [39].Since the intensity of the image varies over the scene (dueto various reasons including magnetic field inhomogeneitiesand different strengths of the contrast agent in differentparts of the vessels), simple thresholding and connectivityanalysis fail to achieve a proper segmentation of the vessels.

Recently, new imaging techniques with contrast agentsthat stay in the blood much longer that cause far less extrav-essation than Gadolinium have been devised [61]. Although,these agents stay in the blood longer and hence provide amore uniform enhancement of even significantly thinner ves-sels, their side effect is that they enter into venous circula-tion also because of the long decay time. Consequently, boththe arteries and veins enhance in the acquired scenes and itbecomes very difficult to mentally separate the arterial treefrom the venous tree in a display such as the one created bya MIP method (see Fig. 6) [40]. Iterative relative fuzzy con-nectedness can be effectively utilized in this situation [40] toseparate the arterial tree from the venous from a scene that is

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1661

(a)

(b)

Fig. 5. A slice of an MRA scene acquired with Gadoliniumadministration, a MIP rendition of this scene which shows muchclutter, and a MIP rendition of the region segmented by using fuzzyconnectedness.

created by segmenting the entire vascular tree first by usinggeneralized fuzzy connectedness. This is illustrated in Fig. 6.

C. CT Angiography

Currently, in the study of vascular diseases, CT angiog-raphy [42] is a serious contender with MRA to eventually re-place conventional X-ray angiography. A major drawback ofCTA, however, has been the obscuration of vessels by bonein the 3-D display of CTA images. The problem is caused byblood as well as bone assuming high and comparable inten-sity in the CTA scene and often by vessels running close toosseous structures. Partial volume and other artifacts make itreally difficult to suppress bone in the latter situation. Thisscenario is particularly true for cerebral vessels since bonesurrounds the brain and vessels come close to the inner sur-face of the calvarium at many sites. Interactive clipping ofbone in three dimensions is not always feasible or wouldproduce unsatisfactory results and slice-by-slice outlining is

(a)

(b)

(c)

Fig. 6. (a) An MIP rendition of an MRA scene acquired withthe blood-pool contrast agent MS325. (b) The entire vasculartree segmented via generalized fuzzy connectedness and volumerendered. (c) The arteries and veins separated by using iterativerelative fuzzy connectedness. See Fig. 12(a) for color illustration.

simply not practical since most studies involve 100s of slicescomprising a CTA scene.

A method employing fuzzy connectedness has been pro-posed in [41] for the suppression of osseous structures, andits application in neurosurgery, particularly for managingcerebral aneurysms, is described in [42]. The method usesboth generalized and iterative relative fuzzy connectednessalong with some morphological operations to separate thevessels from osseous structures. Particularly challenging inthis context are the suppression of thin bones and of bonesjuxtaposed close to vessels. Fig. 7 shows an example of

1662 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

(a)

(b)

Fig. 7. An MIP rendition of CTA of the head of a subject, andan MIP rendition after bone has been suppressed via relative fuzzyconnectedness.

a MIP rendition of a cerebral CTA scene before and aftersuppression of bones by using the method.

D. Brain Tumor

The management of patients with brain tumor (particularlygliomas) is at present difficult in the clinic since no objective

information is available as to the morphometric measures ofthe tumor. Multiprotocol MR images are utilized in the radi-ologic clinic to determine the presence of tumor and to sub-jectively discern the extent of the tumor [43]. Both subjectiveand objective quantification of brain tumor are difficult dueto several factors. First, most tumors appear highly diffusedand fuzzy with visually indeterminable borders. Second, thetumor pathology has different meaning in images obtained byusing different MRI protocols. Finally, a proper definition ofwhat is meant by “tumor” in terms of a specific pathology anda determination of what definition is appropriate and usefulfor the task on hand are yet to be determined. In the pres-ence of these difficulties, we arrived at the following oper-ational definition for the purpose of quantification. We seektwo measures to characterize the size of the tumor, the firstbeing the volume of the edema and the second being the ac-tive part of the tumor. Edema is defined to be constituted bythe regions of high signal intensity in scenes acquired viathe FLAIR sequence. Active aspects of the tumor are con-sidered to be formed of regions that have high signal inten-sity in T1-weighted scenes acquired with Gadolinium ad-ministration but not in T1-weighted scenes acquired withoutGadolinium.

Our approach to derive these measures from the MRscenes is described in [44]. Briefly, it consists of applyingthe generalized fuzzy connectedness method to the FLAIRscene to segment the hyper-intense regions and estimatingthe volume. Further, the two T1 scenes are registered, thenone scene is subtracted from the other, and the hyper-intenseregion in the subtracted scene is segmented and its volumeis computed. The reason for subtracting the scenes is toremove any postsurgical scar/blood appearing as hyper-in-tense regions in the T1 scenes (without Gadolinium). Fig. 8shows an example of this application.

E. Breast Density

Breast density as measured from the volume of densetissue in the breast is considered to indicate a risk factorfor breast cancer [62]. X-ray mammography is one of themost commonly used imaging modalities for breast cancerscreening. Estimation of breast density from mammogramsis therefore a useful exercise in managing patients at highrisk for breast cancer. It may also be useful in assessing howthis risk factor is affected by various treatment proceduressuch as hormone therapy.

In the Wolfe classification scheme [63], [64], a mammo-grapher assigns a grade of density based on a visual inspec-tion of the mammogram. This method, besides being veryrough and subjective, may have considerable inter and intrareader variations and variation from one projective image toanother of the same breast. We have developed a methodutilizing generalized fuzzy connectedness to segment a dig-itized mammogram into dense fibroglandular and fatty re-gions and to provide several measures of density [49]. Thefuzzy connectedness framework seems to be especially suit-able in this application since the dense regions appear fuzzywith a wide range of scene intensities. Fig. 9 shows an ex-ample.

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1663

(a) (b) (c)

(d) (e) (f)

Fig. 8. (a)–(c) A slice each of a FLAIR scene, a T1-weighted scene, and a T1 with Gadoliniumscene. (d)–(f) A surface rendition of brain plus edema segmented from the FLAIR scene; brain plusedema and enhancing aspect of tumor segmented from the two T1 scenes; edema plus enhancingtumor volume rendered. See Fig. 12(b)–(d) for color illustrations.

F. Upper Airway in Children

MRI has been used to visualize the upper airway of adulthumans. This enables imaging of the air column as well asof the surrounding soft tissue structures including lateralpharyngeal wall musculature, fat pads, tongue and softpalate [65]. With detailed images of the upper airway and itsnearby structures, researchers can now use image processingmethods to efficiently segment, visualize, and measure thesestructures. Measurements of upper airway geometry and thearchitecture of the surrounding structures can be used toinvestigate the reasons for obstructive sleep apnea (OSA),and to assess the efficacy of medical treatment [65].

Similarly, MRI has recently been introduced to study theupper airway structure of children during growth and devel-opment and to assess the anatomic role in pathological con-ditions such as OSA [66]. MRI is particularly useful in chil-dren since it can delineate the lymphoid tissues surroundingthe upper airway very accurately. These tissues, the tonsilsand adenoids, figure predominately in childhood OSA andare a common cause for pediatric medical visits and surgery.

Our methodology [67] uses axial T2 images. We have de-veloped a PC-based system that runs under the Linux op-erating system (which also runs on other Unix platforms)for upper airway segmentation and measurement. General-ized fuzzy connectedness is used for segmenting the upperairway and adenoide and palatine tonsils. The system per-forms all tasks from transforming data to giving a final result

report. To achieve accurate quantification and assure quality,the system allows clinicians to interact with critical data pro-cessing steps as minimally as possible. Fig. 10 shows an ex-ample.

G. CT Colonoscopy

An established imaging method for detecting coloncancer has been X-ray projection imaging. A morefool-proof method has been the endoscopic examinationthrough a colonoscopy that is commonly employed inthe clinic for screening process. Recently, in view of theinvasiveness and attendant patient discomfort of this latterprocedure, methods utilizing tomographic images havebeen investigated [68]. Dubbed virtual colonoscopy or CTcolonoscopy, the basic premise of these methods is to utilizethe 3-D information available in a CT scene of the colon,acquired upon careful cleaning and preparation of the colon,to produce 3-D displays depicting views that would beobtained by an endoscope. Since full 3-D information isavailable, displays depicting colons that are cut or split openor unraveled in other forms can also be created.

In the system we have developed [47], generalized fuzzyconnectedness is used for segmenting the colon. A 3-Ddisplay of the entire colon is then created and the userselects several points along the colon on the outer surfaceso displayed to help the system create a central line runningthrough the colon. Subsequently, along this central line view

1664 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

(a) (b) (c)

Fig. 9. A digitized mammogram; the fuzzy connectivity scene of the dense region; and thesegmented binary region.

points are selected at uniform intervals, and at each suchpoint, three renditions are created of the inner wall. Theserenditions represent the views of the inner wall of the colonthat would be obtained by viewing the wall surface enface inthree directions that are 120apart. The speed of our surfacerendering method is high enough to produce in real-time theset of three views that will be obtained as the view pointmoves along the central line automatically. Fig. 11 shows anexample.

VI. CONCLUDING REMARKS

Although the idea of fuzzy connectedness existed in the lit-erature since the late 1970s [6], its full-fledged developmentand application to image segmentation started in the 1990s[14]. Apart from generalizing the well known concept of hardconnectedness to fuzzy subsets, fuzzy connectedness allowscapturing the spatio-topological concept of hanging-togeth-erness of image elements even in the presence of a grada-tion of their intensities stemming from natural material het-erogeneities, blurring and other imaging-phenomenonrelatedartifacts. In this paper, we have examined in a tutorial fashionthe different approaches that exist for defining fuzzy connect-edness in fuzzy subsets of the discrete space as well as di-rectly in given scenes. The latter, in particular, allow us to tiethe notion of fuzzy connectedness with scene segmentation,leading to an immediate practical utilization of fuzzy con-nectedness in scene analysis. Starting from the fundamental

notion of fuzzy connectedness, we have demonstrated howmore advanced concepts such as relative and iterative rela-tive fuzzy connectedness can be developed which have prac-tical consequences in scene segmentation. We have also ex-amined seven major medical application areas to which thefuzzy connectedness principles have been applied for delin-eating objects.

Fuzzy connectedness has several interesting character-istics that enhance its effectiveness in scene segmentation.The scale-based versions of fuzzy connectedness makeit less sensitive to noise. It is also possible to considertexture in the definition of affinity to arrive at connectednessand segmentation of regions in terms of texture. We notethat fuzzy connectedness is to some extent immune to aslow varying component of background scene intensity asdemonstrated in [14]. This is not difficult to explain. Suchslow varying components do not affect affinities much,as such the strengths of connectedness of paths are notsignificantly affected, and therefore, so also the overallfuzzy connectedness.

Many methods exist for fuzzy clustering of feature vec-tors in a feature space [69], [70]. It would be worth investi-gating if the added hanging-togetherness character of fuzzyconnectedness can lead to improved clustering strategies inthis area. Another direction for future research is to use fuzzyconnectedness, relative and iterative relative fuzzy connect-edness on binary scenes and on membership scenes for sep-

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1665

(a) (b) (c)

(d) (e) (f)

(g)

Fig. 10. A slice of a T2 scene of the upper airway region of a child, the corresponding segmentedairway and adenoid and palatine tonsils, and a 3-D rendition of the three structures. See Fig. 12(f) forcolor illustration.

arating components and for doing morphological operationson them. The aspect of the size of the object at various spelscan be captured via scale values and separation or connected-ness of binary objects can be analyzed via fuzzy connected-ness of scale scenes. It should be feasible to directly obtainmedial line and skeletal representations of objects from bi-nary scenes by using fuzzy connectedness on scale scenes.

The iterative relative fuzzy connectedness theory and algo-rithms currently available [17], [18] are for only two objects.Considering the fact that most scenes contain more than twoobjects, for these algorithms to be useful, they and the asso-ciated theory should be extended to multiple objects.

One of the current limitations (stemming from a theoret-ical requirement) in the design of affinities [49] is that theirfunctional form should be shift-invariant over the wholescene domain, although there are ways of tailoring affinities

to variations in the characteristics of different object regions(see [49]). It will be practically useful to investigate ways toallow different functional forms for different object regionsand their theoretical and practical consequences.

A fundamental question arises in relative fuzzy connect-edness as to how to do object groupings. Even when multipleobjects are considered as in Fig. 1, each object region mayconsist of a group of objects. From the consideration ofthe effectiveness of segmentation, certain groupings maybe better than others or even better than the considerationof all object regions that may exist in the scene. Shouldthe object groupings be considered based on separability?That is, the scene domain is first conceptually divided intoa set of most separable groups of objects. These groupsare then segmented via RFC (or IRFC). Next, within eachgroup, further conceptual groupings are made based on

1666 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

Fig. 11. A surface rendition of the whole colon with the central line shown. Three enfaceviews of the colon wall obtained from a view point situated on the central line with the viewingdirections that are 120apart.

Fig. 12. (a)–(e) Color illustrations of Figs. 6(c), 8(d)–(f) and 10(g), respectively.

separability, and each such sub group is segmented viaRFC. This process is continued until the desired objectsare reached and segmented. Such a hierarchical RFC (orIRFC) has interesting theoretical, computational, and appli-cation-specific questions to be studied.

Fuzzy connectedness in its present form attempts to ex-tract as much information as possible entirely from the givenscene. It, however, does not attempt to use in its frameworkprior shape and appearance knowledge about object bound-aries to be segmented. These entities have been shown to be

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1667

useful in scene segmentation [71], [72]. When information ismissing in certain segments of the boundary, fuzzy connect-edness often goes astray establishing connection with otherobjects surrounding the object of interest. Ways to incorpo-rate prior boundary shape appearance knowledge into thefuzzy connectedness framework may mitigate this problem.This is a challenging and worthwhile direction to pursue. Theconsideration of multiple objects in the model and utilizationof such models in conjunction with multiple object relativefuzzy connectedness opens many new avenues for segmen-tation research.

ACKNOWLEDGMENT

The authors are grateful to Dr. T. Lei and J. Liu for Figs. 6,8, and 10 and to Dr. H. Eisenberg for data pertaining toFig. 11.

REFERENCES

[1] R. L. Grimsdale, F. H. Summer, C. J. Tunis, and T. Kilburn, “Asystem for automatic recognition of patterns,”Proc. Inst. Elect. Eng.,vol. 106B, pp. 210–221, 1959.

[2] A. Rosenfeld, “Connectivity in digital pictures,”J. Assoc. Comput.Machinery, vol. 17, pp. 146–160, 1970.

[3] E. Artzy, G. Frieder, and G. T. Herman, “The theory, design, imple-mentation, and evaluation of a three-dimensional surface detectionalgorithm,” Comput. Graph. Image Processing, vol. 15, pp. 1–24,1981.

[4] A. Rosenfeld, “Three dimensional digital topology,”Inform. Contr.,vol. 50, pp. 119–127, 1981.

[5] J. K. Udupa, S. N. Srihari, and G. T. Herman, “Boundary detectionin multidimensions,”IEEE Trans. Pattern Anal. Machine Intell., vol.4, pp. 41–50, 1982.

[6] A. Rosenfeld, “Fuzzy digital topology,”Inform. Contr., vol. 40, no.1, pp. 76–87, 1979.

[7] , “The fuzzy geometry of image subsets,”Pattern Recogn. Lett.,vol. 2, pp. 311–317, 1991.

[8] J. K. Udupa and S. Samarasekera, “Fuzzy connectedness and objectdefinition: Theory, algorithms, and applications in image segmenta-tion,” Graph. Models Image Processing, vol. 58, pp. 246–261, 1996.

[9] D. L. Pham, “Spatial models for fuzzy clustering,”Comput. VisionImage Understanding, vol. 84, pp. 285–297, 2001.

[10] A. Kaufmann,Introduction to the Theory of Fuzzy Subsets. NewYork: Academic, 1975, vol. 1.

[11] T. Y. Kong and A. Rosenfeld, “Digital topology: Introduction andsurvey,” Comput. Vision, Graph. Image Processing, vol. 48, pp.357–393, 1989.

[12] S. Dellepiane and F. Fontana, “Extraction of intensity connectednessfor image processing,”Pattern Recogn. Lett., vol. 16, pp. 313–324,1995.

[13] S. Dellepiane, F. Fontana, and G. L. Vernazza, “Nonlinear imagelabeling for multivalued segmentation,”IEEE Trans. Image Pro-cessing, vol. 5, pp. 429–446, 1996.

[14] J. K. Udupa and S. Samarasekera, “Fuzzy connectedness and objectdefinition,” in Proc. SPIE: Med. Imag., vol. 2431, 1995, pp. 2–11.

[15] P. K. Saha, J. K. Udupa, and D. Odhner, “Scale-based fuzzy con-nected image segmentation: Theory, algorithms, and validation,”Comput. Vision Image Understanding, vol. 77, pp. 145–174, 2000.

[16] P. K. Saha and J. K. Udupa, “Fuzzy connected object delineation:Axiomatic path strength definition and the case of multiple seeds,”Comput. Vision Image Understanding, vol. 83, pp. 275–295, 2001.

[17] J. K. Udupa, P. K. Saha, and R. A. Lotufo, “Relative fuzzy connect-edness and object definition: Theory, algorithms, and applicationsin image segmentation,”IEEE Trans. Pattern Anal. Machine Intell.,vol. 24, pp. 1485–1500, 2002.

[18] P. K. Saha and J. K. Udupa, “Iterative relative fuzzy connectednessand object definition: Theory, algorithms, and applications in imagesegmentation,”Proc. IEEE Workshop on Mathematical Methods inBiomedical Image Analysis, pp. 28–35, 2000.

[19] J. K. Udupa, P. K. Saha, and R. A. Lotufo, “Fuzzy connected objectdefinition in images with respect to co-objects,” inProc. SPIE: Med.Imag., vol. 3661, 1999, pp. 236–245.

[20] G. T. Herman and B. M. De Carvalho, “Multiseeded segmentationusing fuzzy connectedness,”IEEE Trans. Pattern Anal. Machine In-tell., vol. 23, pp. 460–474, 2001.

[21] B. M. Carvalho, C. J. Gau, G. T. Herman, and T. Y. Kong, “Algo-rithms for fuzzy segmentation,”Patt. Anal. Appl., vol. 2, pp. 73–81,1999.

[22] L. G. Nyul, A. X. Falcao, and J. K. Udupa, “Fuzzy-connected 3Dimage segmentation at interactive speeds,” Graph. Models, vol. 64,pp. 259–281, 2002, to be published.

[23] T. N. Jones and D. N. Metaxas, “Automated 3D segmentation usingdeformable models and fuzzy affinity,” inProc. Inform. Processingin Med. Imag., 1997, pp. 113–126.

[24] T. N. Jones, “Image-Based Ventricular Blood Flow Analysis,” Ph.D.dissertation, University of Pennsylvania, 1998.

[25] E. D. Angelini, C. Imielinska, Y. Jin, and A. Laine, “A improving sta-tistics for hybrid segmentation of high-resolution multichannel im-ages,” inProc. SPIE: Med. Imag., vol. 4684, 2002, pp. 401–411.

[26] Y. Jin, A. Laine, and C. Imielinska, “An adaptive speed term basedon homogeneity for level-set segmentation,” inProc. SPIE: Med.Imag., vol. 4684, 2002, pp. 383–400.

[27] N. Bonnet and S. Maegherman, “Algorithme de Segmentation Semi-Antomatique par la Ligne de Partage des Eaux,” (in French), LERI,University of Reims, 1998.

[28] J. Cutrona and N. Bonnet, “Two methods for semi-automatic imagesegmentation based on fuzzy connectedness and watershed,” inFrance-Iberic Mecroscopy Congress, Barcelona, Spain, 2001, pp.23–24.

[29] J. K. Udupa, L. Wei, S. Samarasekera, Y. Miki, M. A. van Buchem,and R. I. Grossman, “Multiple sclerosis lesion quantification usingfuzzy connectedness principles,”IEEE Trans. Med. Imag., vol. 16,pp. 598–609, 1997.

[30] S. Samarasekera, J. K. Udupa, Y. Miki, and R. I. Grossman, “Anew computer-assisted method for enhancing lesion quantificationin multiple sclerosis,”J. Comput. Assisted Tomogr., vol. 21, pp.145–151, 1997.

[31] Y. Miki, R. I. Grossman, J. K. Udupa, M. A. van Buchem, L. Wei,M. D. Philips, U. Patel, J. C. McGown, and D. L. Kolson, “Differ-ences between relapsing remitting and chronic progressive multiplesclerosis as determined with quantitative MR imaging,”Radiology,vol. 210, pp. 769–774, 1999.

[32] Y. Miki, R. I. Grossman, J. K. Udupa, L. Wei, M. Polansky, L.Mannon, and D. L. Kolson, “Relapsing-remitting multiple sclerosis:Longitudinal analysis of MR images – Lack of correlation betweenchanges in T2 lesion volume and clinical findings,”Radiology, vol.213, pp. 395–399, 1999.

[33] Y. Ge, R. Grossman, J. K. Udupa, L. Wei, L. Mannon, M. Polansky,and D. L. Kolson, “Brain atropy in relapsing-remitting multiplesclerosis and secondary progressive multiple sclerosis: Longitudinalquantitative analysis,”Radiology, vol. 213, pp. 665–670, 2000.

[34] J. K. Udupa, L. G. Nyul, Y. Ge, and R. I. Grossman, “MultiprotocolMR image segmentation in multiple sclerosis: Experience with over1000 studies,”Acad. Radiol., vol. 8, pp. 1116–1126, 2001.

[35] T. Alderliesten, W. Niessen, K. Vincken, J. Maintz, F. Janson, O. vanNieuwenhuizen, and M. Viewgever, “Objective and reproduciblesegmentation and quantification of tuberous sclerosis lesions inFLAIR brain MR images,” inProc. SPIE: Med. Imag., vol. 4322,2001, pp. 1509–1518.

[36] R. He and P. A. Narayana, “Automatic delineation of Gd enhance-ments on magnetic resonance images in multiple sclerosis,”Med.Phys., vol. 29, pp. 1536–1546, 2002.

[37] A. Kumar, W. Bilker, Z. Jin, J. K. Udupa, and G. Gottlieb, “Age ofonset of depression and quantitative neuroanatomic measures: Ab-sence of specific correlations,”Psych. Res. Neuroimag., vol. 91, pp.101–110, 1999.

[38] A. Kumar, W. Bilker, Z. Jin, and J. K. Udupa, “Atropy and highintensity lesions: Complementary neurobiological mechanisms inlate-life major depression,”Neuropsychopharmacology, vol. 22, pp.264–274, 2000.

[39] B. L. Rice Jr. and J. K. Udupa, “Clutter-free volume rendering formagnetic resonance angiography using fuzzy connectedness,”Int. J.Imag. Syst. Technol., vol. 11, pp. 62–70, 2000.

[40] T. Lei, J. K. Udupa, P. K. Saha, and D. Odhner, “Artery-vein separa-tion via MRA – An image processing approach,”IEEE Trans. Med.Imag., vol. 20, pp. 689–703, 2001.

1668 PROCEEDINGS OF THE IEEE, VOL. 91, NO. 10, OCTOBER 2003

[41] P. K. Saha, J. K. Udupa, and J. M. Abrahams, “Automatic bone-freerendering of cerebral aneurysms via 3D-CTA,” inProc. SPIE: Med.Imag., vol. 4322, San Diego, CA, 2001, pp. 1264–1272.

[42] J. M. Abrahams, P. K. Saha, R. W. Hurst, P. D. LeRoux, and J. K.Udupa, “Three-dimensional bone-free rendering of the cerebral cir-culation using computed tomographic angiography and fuzzy con-nectedness,”Neurosurgery, vol. 51, pp. 264–269, 2002.

[43] J. Liu, J. K. Udupa, D. C. Hackney, and G. Moonis, “Estimationof tumor volume with fuzzy connectedness segmentation of MRI,”in Proc. SPIE: Med. Imag., vol. 4322, San Diego, CA, 2001, pp.1455–1465.

[44] G. Moonis, J. Liu, J. K. Udupa, and D. Hackney, “Estimation oftumor volume using fuzzy connectedness segmentation of MR im-ages,”Amer. J. Neuroradiol., vol. 23, pp. 356–363, 2002.

[45] P. K. Saha, J. K. Udupa, E. F. Conant, D. P. Chakraborty, and D.Sullivan, “Breast tissue density quantification via digitized mammo-grams,”IEEE Trans. Med. Imag., vol. 20, pp. 792–803, 2001.

[46] J. K. Udupa, J. Tian, D. Hemmy, and P. Tessier, “A pentiumPC-based craniofacial 3D imaging and analysis system,”J. Cranio-facial Surg., vol. 8, pp. 333–339, 1997.

[47] J. K. Udupa, D. Odhner, and H. C. Eisenberg, “New automatic modeof visualizing the colon via CT,” inProc. SPIE: Medical Imaging,vol. 4319, San Diego, CA, 2001, pp. 237–243.

[48] N. R. Pal and S. K. Pal, “A review of image segmentation tech-niques,”Pattern Recogn., vol. 26, pp. 1277–1294, 1993.

[49] P. K. Saha and J. K. Udupa, “Relative fuzzy connectedness amongmultiple objects: Theory, algorithms, and applications in imagesegmentation,”Comput. Vision Image Understanding, vol. 82, pp.42–56, 2001.

[50] T. Lindeberg,Scale-Space Theory in Computer Vision. Boston,MA: Kluwer, 1994.

[51] S. M. Pizer, D. Eberly, D. S. Fritsch, and B. S. Morse, “Zoom-in-variant vision of figural shape: The mathematics of cores,”Comput.Vision Image Understanding, vol. 69, no. 1, pp. 55–71, 1998.

[52] M. Tabb and N. Ahuja, “Multiscale image segmentation by inte-grated edge and region detection,”IEEE Trans. Image Processing,vol. 6, no. 5, pp. 642–655, 1997.

[53] T. Cormen, C. Leiserson, and R. Rivest,Introduction to Algo-rithms. New York: McGraw-Hill, 1991.

[54] J. K. Udupa and D. Odhner, “Shell rendering,”IEEE Comput.Graph. Appl., vol. 13, no. 6, pp. 58–67, 1993.

[55] E. Dijkstra, “A note on two problems in connection with graphs,”Num. Math., vol. 1, pp. 269–271, 1959.

[56] J. K. Udupa, V. R. LaBlanc, H. Schmidt, C. Imielinska, P. K. Saha, Y.Zhuge, P. Molholt, and Y. Jin, “Methodology for evaluating imagesegmentation algorithms,” inProce. SPIE: Medical Imaging, vol.4684, San Diego, CA, 2002, pp. 266–277.

[57] J. M. Charcot, Lectures on the Diseases of the NervousSystem. London: The New Sydenham Society, 1877.

[58] J. F. Kurtzke, “Geographic distribution of multiple sclerosis: An up-date with special reference to Europe and the mediterranean region,”Acta Neurologica Scandinavica, vol. 62, pp. 65–80, 1980.

[59] , “Rating neurologic imapairment in multiple sclerosis: Anexpanded disability status scale (EDSS),”Neurology, vol. 33, pp.1444–1452, 1983.

[60] L. G. Nyul and J. K. Udupa, “Protocol-independent brain MRI seg-mentation method,” inProc. SPIE: Medical Imaging, vol. 4684, SanDiego, CA, 2002, pp. 1588–1599.

[61] R. B. Lauffer, D. J. Parmelee, S. U. Dunham, H. S. Ouellet, R. P.Dolan, S. Witte, T. J. McMurry, and R. C. Walovitch, “MS-325: Al-bumin-targeted contrast agent for MR angiography,”Radiology, vol.207, pp. 529–538, 1998.

[62] E. Warneret al., “The risk of breast-cancer associated with mam-mographic parenchymal patterns: A meta-analysis of the publishedliterature to examine the effect of method of classification,”CancerDetection and Prevention, vol. 16, pp. 67–72, 1992.

[63] J. N. Wolfe, “Breast pattern as an index of risk for developing breastcancer,”AJR, vol. 126, pp. 1130–1139, 1976.

[64] , “Breast parenchymal patterns and their changes with age,”Ra-diol., vol. 121, pp. 545–552, 1976.

[65] R. J. Schwab, K. B. Gupta, W. B. Gefter, E. A. Hoffman, andA. I. Pack, “Upper airway soft tissue anatomy in normals andpatients with sleep disordered breathing: Significance of the lateralpharyngeal walls,”Amer. J. Respiratory Crit. Care Med., vol. 152,pp. 1673–1689, 1995.

[66] R. Arens, J. M. McDonough, A. M. Corbin, E. M. Hernandez, G.Maislin, J. R. Schwab, and A. I. Pack, “Linear dimensions of theupper airway structure during development: Assessment by mag-netic resonance imaging,”Amer. J. Respiratory Crit. Care Med., vol.165, pp. 117–122, 2002.

[67] J. G. Liu, J. K. Udupa, D. Dewey, J. M. McDonough, and R. Arens,“A system for upper airway segmentation and measurement viaMR imaging and fuzzy connectedness,”Acad. Radiol., vol. 10, pp.13–24, 2003.

[68] D. J. Vining, “Virtual colonoscopy,”Gastrointest. Endosc. Clin.North America, vol. 7, pp. 285–291, 1997.

[69] J. C. Bezdek, “Fuzzy Mathematics in Pattern Classification,” Ph.D.dissertation, Cornell University, Ithaca, NY, 1973.

[70] E. Backer and A. C. Jain, “A clustering performance measure basedon fuzzy set decomposition,”IEEE Trans. Pattern Anal. MachineIntell., vol. 3, pp. 66–74, 1981.

[71] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, “Activeshape models-their training and application,”Comput. Vision ImageUnderstanding, vol. 61, pp. 38–59, 1995.

[72] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearancemodels,” IEEE Trans. Pattern Anal. Machine Intell., vol. 23, pp.681–685, 2001.

Jayaram K. Udupa (Senior Member, IEEE)received the Ph.D. degree in computer sciencefrom the Indian Institute of Science, Bangalore,in 1976 with a medal for best research.

During the past 25 years, he worked in theareas of biomedical image and signal processing,pattern recognition, biomedical computergraphics, 3-D imaging, visualization, and theirbiomedical applications. He has published over120 journal papers in these areas, over 130 fullconference papers, edited two books on medical

3-D imaging, written over 20 book chapters and editorials, given over120 invited talks (since 1986), offered consultancy to several industries,organized conferences, seminars, and workshops, and codeveloped thefirst-ever software package for medical 3-D imaging and developed andwidely distributed large software systems for 3-D imaging. Presently, he isthe Chief of the Medical Imaging Section and Professor of RadiologicalSciences in the Department of Radiology at the University of Pennsylvania,Philadelphia.

Dr. Udupa is a Fellow of the American Institute of Medical and BiologicalEngineering.

Punam K. Saha (Member, IEEE) received theBachelor’s and Master’s degrees in computerscience and engineering from Jadavpur Uni-versity, India, in 1987 and 1989, respectively.In 1997, he received the Ph.D. degree from theIndian Statistical Institute, which he joined as afaculty member in 1993.

In 1997, he joined the University of Penn-sylvania, Department of Radiology, MedicalImaging Section, as a Postdoctoral Fellow,where he is currently an Assistant Professor. His

present research interests include biomedical imaging problems and theapplication of their solutions, scale, segmentation, filtering, image registra-tion, inhomogeneity correction, digital topology and their applications, andestimation of trabecular bone strength from MR images. He has publishedover 40 papers in international journals.

Dr. Saha received a Young Scientist award from the Indian Science Con-gress Association in 1996. He is a member of the International Associationfor Pattern Recognition and a member of the Governing body of the IndianUnit for Pattern Recognition and Artificial Intelligence.

UDUPA AND SAHA: FUZZY CONNECTEDNESS AND IMAGE SEGMENTATION 1669