Morphological Attribute Proﬁles for the Analysis of Very ...€¦ · Recently, the standard morphological operators of opening, closing, white, and black top hat, along with opening

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 10, OCTOBER 2010 3747

Morphological Attribute Profiles for the Analysisof Very High Resolution Images

Mauro Dalla Mura, Student Member, IEEE, Jón Atli Benediktsson, Fellow, IEEE,Björn Waske, Member, IEEE, and Lorenzo Bruzzone, Fellow, IEEE

Abstract—Morphological attribute profiles (APs) are defined asa generalization of the recently proposed morphological profiles(MPs). APs provide a multilevel characterization of an imagecreated by the sequential application of morphological attributefilters that can be used to model different kinds of the structuralinformation. According to the type of the attributes considered inthe morphological attribute transformation, different parametricfeatures can be modeled. The generation of APs, thanks to anefficient implementation, strongly reduces the computational loadrequired for the computation of conventional MPs. Moreover, thecharacterization of the image with different attributes leads to amore complete description of the scene and to a more accuratemodeling of the spatial information than with the use of con-ventional morphological filters based on a predefined structuringelement. Here, the features extracted by the proposed opera-tors were used for the classification of two very high resolutionpanchromatic images acquired by Quickbird on the city of Trento,Italy. The experimental analysis proved the usefulness of APsin modeling the spatial information present in the images. Theclassification maps obtained by considering different APs resultin a better description of the scene (both in terms of thematic andgeometric accuracy) than those obtained with an MP.

Index Terms—Classification, mathematical morphology, mor-phological attribute profiles (APs), morphological profiles (MPs),object detection, remote sensing, very high resolution (VHR)images.

I. INTRODUCTION

H IGH spatial resolution in the latest generation ofoptical sensors such as Ikonos, QuickBird, Spot-5, and

Worldview (up to 0.5 m) has increased the range of applicationswhere remote sensing (RS) data can be usefully employed. Thegreat amount of thematic information contained in very highresolution (VHR) images can be exploited in tasks addressingthe analysis of land cover/use and object extraction. In partic-ular, VHR imagery is very useful for investigating urban envi-ronments (e.g., in urban growth planning and monitoring, road

Manuscript received October 3, 2009; revised December 3, 2009. Date ofpublication June 7, 2010; date of current version September 24, 2010.

M. Dalla Mura is with the Department of Information Engineering andComputer Science, University of Trento, 38123 Trento, Italy and also withthe Faculty of Electrical and Computer Engineering, University of Iceland,101 Reykjavik, Iceland (e-mail: [email protected]).

J. A. Benediktsson is with the Faculty of Electrical and Computer Engineer-ing, University of Iceland, 101 Reykjavik, Iceland (e-mail: [email protected]).

B. Waske is with the Institute of Geodesy and Geoinformation, Depart-ment of Photogrammetry, University of Bonn, 53115 Bonn, Germany (e-mail:[email protected]).

L. Bruzzone is with the Department of Information Engineering andComputer Science, University of Trento, 38123 Povo, Italy (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TGRS.2010.2048116

network map-updating, discovering building abuse, etc.) wherea detailed representation of the scene can significantly improvethe results of the analysis with respect to low-resolution data(like those acquired by Landsat Thematic Mapper and En-hanced Thematic Mapper Plus). For example, fine representa-tion of details in a scene can be exploited in object detectiontasks, where the characterization of the geometrical features ofobjects is of fundamental importance.

The technical features of VHR data require the developmentof specific methods for data analysis. For example, the con-textual spectral similarity of connected pixels is a fundamentalproperty of VHR data, whereas, it is less relevant in medium-resolution images. Furthermore, the fine representation ofgeospatial objects and the great amount of details improve therepresentation of the surveyed scene but, at the same time,significantly increase the complexity of VHR images, leadingto a substantial difficulty in extracting the relevant informativecomponents according to automatic analysis procedures. Thegreat heterogeneity of the imaged scene (e.g., the same thematicobjects might appear either as homogeneous or highly texturedregions in the image), due to the high resolution of the sensor,cannot be handled by general image processing techniquesdeveloped for medium/high-resolution sensors. Moreover, theincreased geometrical resolution leads on the one hand to afine representation of the scene, whereas, to the other handto a decreased resolution in the spectral domain which furtherincreases the spectral ambiguity of different land-cover types.This results in a reduction of the effectiveness of conventionalclassification methodologies based on the analysis of spectralfeatures [1]. Thus, features that can be used effectively to modelthe spatial information of the pixels by exploiting contextualrelations must be included in the analysis of VHR data.

Several techniques specifically developed for incorporatinginformation extracted by modeling the spatial properties inanalysis of VHR images have been presented in the literature.Usually, information extracted from the spatial characteristicsof the image is combined with the available spectral fea-tures in the analysis. The spatial information can be extractedthrough the application of filters by performing a contextualimage transformation, i.e., an image mapping that transformsa pixel as a function of the values of a set of pixels (usuallyits neighbors). In most cases, the transformation reduces, insome ways, the complexity of the scene by attenuating somedetails. The outcome of the transformation depends on howthe structures that are present in the image interact with theneighborhood of the filter. Since, particularly in VHR images,the shapes and contours of the regions are perceptually very

0196-2892/$26.00 © 2010 IEEE

3748 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 10, OCTOBER 2010

significant, the filtering technique should simultaneously at-tenuate the unimportant details and preserve the geometricalcharacteristics of the other regions. This property is elegantlyachieved by morphological connected filters, such as filtersby reconstruction [2]. For instance, openings and closings byreconstruction can suppress brighter and darker regions (withrespect to the graylevel of the adjacent regions), respectively,that are smaller than the moving window used in the trans-formation [which is called structuring element (SE)]. On thecontrary, the structures that are larger than the SE are com-pletely preserved, leaving their geometry unaffected. The SE,which specifies the neighborhood considered for each pixel andthe morphological operator, defines the amount of contextualrelations included in the analysis. Pesaresi and Benediktsson[3] introduced the application of this family of filters to VHRimages. They performed a multiscale analysis by computing ananti-granulometry and a granulometry, (i.e., a sequence of clos-ings and openings of increasing size), appended in a commondata structure called morphological profile [3]. The derivativeof the morphological profile (DMP), which shows the residuesof two successive filtering operations (i.e., two adjacent levelsin the profile), was exploited for classifying VHR panchromaticimages and for the definition of a novel segmentation tech-nique. The obtained segmentation map, called morphologicalcharacteristic, is generated by associating each pixel to the levelwhere the maximum of the DMP (evaluated at the given pixel)occurs. Since their definition, MPs and DMPs have been widelyused for the analysis of remote sensing images. In [4], the MPgenerated by standard opening and closing was computed on aQuickbird panchromatic image acquired on an area hit by the2003 Bam earthquake. The spatial features extracted by the MPwere used for assessing the damages caused by the earthquake.Recently, the standard morphological operators of opening,closing, white, and black top hat, along with opening and clos-ing by reconstruction, were used together with support vectormachines for the classification of a Quickbird panchromaticimage, [5]. An automatic hierarchical segmentation techniquebased on the analysis of the DMP was proposed in [6]. Thesegmentation process is performed according to a criterionbased on the spectral homogeneity and spatial connectednesscomputed on the segments extracted by the DMP at each level.The DMP was also analyzed in [7], by extracting a fuzzy mea-sure of the characteristic scale and contrast of each structurein the image. The computed measures were compared withthe possibility distribution predefined for each thematic class,generating a value of membership degree for each class used forclassification. In [8], feature extraction techniques were appliedto the DMP in order to reduce the dimensionality of the featuresconsidered by a neural network classifier. In [9], the analysisbased on MPs was successfully extended to the processing ofhyperspectral high-resolution images, by computing the MPson the principal components of the data [which were calledextended morphological profiles (EMPs)]. Since the EMP donot fully exploit the spectral information, in [10], they wereconsidered along with the original hyperspectral data by asupport vector machines for classification.

As can be observed from the aforementioned literature, thecomputation of a multiscale processing (e.g., by MPs, DMPs,

EMPs) has proven to be effective in extracting informativespatial features from the analyzed images. For example, MPscomputed with a compact SE (e.g., square, disk, etc.) can beused for modeling the size of the objects in the image (e.g.,in [10] this information was exploited for discriminating smallbuildings from large ones). Recently, the computation of twoMPs was proposed for modeling both the length and the widthof the structures [11]. In greater detail, one MP is built by disk-shaped SEs for extracting the smallest size of the structures,while the other employs linear SEs (which generate directionalprofiles [12]) for characterizing the objects maximum size(along the orientation of the SE). This is useful for defining theminimal and maximal length but, as all the possible lengths andorientations cannot be practically investigated, such analysis iscomputationally intensive. Nevertheless, as proven in [13], fil-ters by reconstruction are suitable for handling the geometricalinformation of the scene. This was observed by applying filtersby reconstruction in order to reduce the image complexity forchange detection on VHR images. In [14], we introduced theuse of morphological attribute filters for VHR remote sensingimages, as an extension of the common morphological filters byreconstruction based on SEs. These operators are morphologi-cal connected filters. Thus, they process the image without dis-torting or inserting new edges but only by merging existing flatregions [2]. Attribute filters were employed for modeling thestructural information of the scene for classification and build-ing extraction in [14] and [15], respectively, where they provedto be suitable for the modeling of structural information in VHRimages. Attribute filters include in their definition, the morpho-logical operators based on geodesic reconstruction [16]. More-over, they are a flexible tool since they can perform a processingbased on many different types of attributes. In fact, the attributescan be of any type. For example, they can be purely geometric,or related to the spectral values of the pixels, or on differentcharacteristics. Furthermore, in [15], the problem of the tuningof the parameters of the filter was addressed by proposing anautomatic selection procedure based on a genetic algorithm.

In this paper, we propose to characterize the spatial informa-tion of VHR data by using a multilevel, multi-attribute approachbased on morphological attribute filters. In particular, this paperaims at extending the works in [14] and [15] by presenting aformal definition of morphological attribute profiles and differ-ential attribute profiles. These are proposed to be an extensionof the morphological profiles and of their derivative concepts,which are conventionally defined for openings and closingsby reconstruction. Thus, the proposed theoretical frameworkpermits the definition of a more general set of profiles basedon the morphological attribute operators. The profiles built bymorphological attribute filters permit a more flexible investiga-tion of the scene, leading to a better modeling of the spatialinformation. Moreover, thanks to an efficient implementation,their application becomes computationally less demanding thanconventional profiles built with operators by reconstruction.

The paper is organized in six sections. The next sectionrecalls and discusses the concept of morphological profiles.Section III introduces morphological attribute filters theory. InSection IV, attribute profiles are formally defined. Section Vpresents the results of the experimental analysis carried out for

DALLA MURA et al.: MORPHOLOGICAL ATTRIBUTE PROFILES 3749

assessing the effectiveness of the proposed operators in mod-eling the spatial information. Finally, conclusions are drawn inSection VI.

II. BACKGROUND ON MORPHOLOGICAL PROFILES

In this section, we introduce the concepts of morphologicalprofiles and differential morphological profiles following thepresentation given in [3]. This leads us to investigate howthe profiles are computed in order to point out the relatedlimitations. In the following, we recall some useful definitionsfor the next discussion.

A. Definitions

A binary image F is a mapping of the subset E, of the imagedomain R

n or Zn (usually n = 2, i.e., 2-D images) into the

couple {0, 1}. A grayscale image f (with single tone values)is a mapping from E to R or Z. Here, we follow the arbitraryconvention of assigning capital letters to binary variables andbinary transformations, whereas small caps refer to grayscaleimages and mappings.

A connected component X of a binary image (in [17] alsocalled a grain) is a set of pixels in which each pair of pixels isconnected. Two pixels are connected according to a connectiv-ity rule. The iso-intensity connected components of a grayscaleimage are called flat zones. Common connectivity rules are thefour- and eight-connected, where a pixel is said to be adjacentto four or eight of its neighboring pixels, respectively. Theconnectivity can be extended by more general criteria defininga connectivity class [18].

A criterion T is a mapping of a generic set S to the couple ofBooleans {false, true}.

An image transformation ψ is a mapping from E to E withψ(f) → R or Z in the grayscale case. We recall some propertiesof an image transformation ψ:

• Increasingness: ψ(f) ≤ ψ(g) if f ≤ g ∀f, g ∈ E• Anti-extensivity: ψ(f) ≤ f• Extensivity: ψ(f) ≥ f• Idempotence: ψψ = ψ• Absorption property: ψiψj = ψiψj = ψmax(i,j).

B. Morphological Profiles

Let us first consider an opening by reconstruction, γiR(f),

applied to an image f with an SE of size i. The opening by

reconstruction can be computed as a sequence of an erosionwith the SE followed by a reconstruction by dilation [2]. Byduality, a closing by reconstruction, φi

R(f), is defined as thedilation of the original image with SE of size i, followedby a geodesic reconstruction by erosion [2]. The geodesicreconstruction, either by dilation or by erosion, is an iterativeprocedure that is performed until idempotence is reached.

When opening by reconstruction is computed on the imagewith an SE of increasing size, we obtain a morphologicalopening profile which can be formalized as

ΠγR(f) =

{Πγλ : Πγλ = γλ

R(f),∀λ ∈ [0, . . . , n]}. (1)

This leads to perform a multiscale analysis of the image.According to its definition, the opening profile is a granu-

lometry built by openings by reconstruction, which is definedin the mathematical morphology framework as a family ofidempotent, anti-extensive and increasing transformations (i.e.,openings) that fulfill the absorption property [2]. Analogously,a morphological closing profile is defined as

ΠφR(f) =

{Πφλ : Πφλ = φλ

R(f),∀λ ∈ [0, . . . , n]}. (2)

The closing profile is an anti-granulometry generated byclosings by reconstruction.

When the size, λ, of the SE is zero, then∏

γ0(f) =∏

φ0(f)holds, corresponding to the original image f .

A morphological profile, generated by geodesic operators, issimply the concatenation of closing and opening profiles, asshown in (3) at the bottom of the page. The resulting MP isa stack of 2n+ 1 images (n images from the closing profile,the original image and n images from the opening profile). Bycomputing the derivative of a MP, a differential morphologicalprofile is generated, as shown in (4) at the bottom of the page.Above the differential opening profile, Δγ , and the differentialclosing profile, Δφ, are respectively defined as

ΔγR(f)=

{Δγλ : Δγλ =Πγ(λ−1) − Πγλ,∀λ ∈ [1, n]

}(5)

ΔφR(f)=

{Δφλ : Δφλ =Πφλ − Πφ(λ−1),∀λ ∈ [1, n]

}. (6)

As seen from (4), the DMP stores the residuals of the sub-sequent increasing transformations applied to the image. Thismight be more practical with respect to the MPs for analyzingthe output of the multiscale analysis since the most importantcomponents of the profiles are more evident in the DMP.

Π(f) ={

Πi :⟨

Πi = Πφλ, with λ = (n− i+ 1), ∀i ∈ [1, n]Πi = Πγλ, with λ = (i− n− 1), ∀i ∈ [n+ 1, 2n+ 1]

}(3)

Δ(f) ={

Δi :⟨

Δi = Δφλ, with λ = (n− i+ 1), ∀i ∈ [1, n]Δi = Δγλ, with λ = (i− n), ∀i ∈ [n+ 1, 2n]

}(4)


C. Limitations of Morphological Profiles

The main limitation of MPs lies in the partial analysis that isperformed with the computation of the profile. In greater detail,MPs attempt to model the spatial information within the sceneby analyzing the interaction of a set of SEs of fixed shape andincreasing size with the objects in the image. Although this is apowerful tool for performing an investigation on the scale of thestructures (thanks to the suitability of the SEs for modeling thesize of the objects), it leads only to a partial characterization ofthe objects in the scene. In fact, one could aim at a descriptionof the image based on other features (e.g., shape, texture, etc.)rather than the size in order to increase the discriminativepower of the analysis. From a theoretical viewpoint, filtersby reconstruction based on SEs could be used to model othergeometrical features, e.g., to represent the information on theshape of the regions by analyzing a set of MPs generated bySEs of different shapes. Nonetheless, the generation of profilesfor different shapes would be computationally unfeasible. Infact, in order to perform an analysis aimed at modeling theshape characteristic, the range of the possible sizes assumedby all the components in the image should be investigated byeach profile in order to remove the dependence of the results tothe scale.

Another important limitation is the strong constraint givenby the use of a SE for modeling the concepts of differentcharacteristics of the spatial information (e.g., size, shape,homogeneity, etc.). This limitation is particularly evident whenfeatures more complex than the geometrical primitives of sizeand shape are required (e.g., shape factor, length of the skeletonof a region, etc.). Moreover, SEs are intrinsically unsuitable todescribe features related to the gray-level characteristics of theregions (e.g., spectral homogeneity, contrast, etc.).

A final limitation of MPs is the computational complexityassociated with their generation. The original image has tobe completely processed for each level of the profile, whichrequires two complete processing of the image; one performedby a closing and the other by an opening transformation. Thus,the complexity increases linearly with the number of levelsincluded in the profile.

III. MORPHOLOGICAL ATTRIBUTE FILTERS

Morphological attribute openings and attribute thinnings(called attribute filters) are morphological adaptive filters intro-duced by Breen and Jones [16]. For simplicity, we introducethese operators for the binary case and later, we extend theconcepts to the grayscale. The discussion will be focusedfirst on opening and thinning. Then, the results are reportedanalogously for closing and thickening.

A. Binary Attribute Operators

Binary attribute openings operate on connected componentsof a binary image according to an increasing criterion. Thetransformation removes all those connected components forwhich the criterion is not satisfied, leaving the others unaf-fected. In order to introduce their formal definition, the bi-

nary connected opening and binary trivial opening have to bedefined.

Binary connected opening, Γx, transforms a binary imagef given a pixel x, by keeping the connected component thatcontains x and removing all the others. Binary trivial openingΓT operates on a given connected componentX according to anincreasing criterion T applied to the connected component. Ifthe criterion is satisfied, the connected component is preserved,otherwise it is removed according to

ΓT (X) ={X, if T (X) = true0, if T (X) = false.

(7)

In general, one or more features of the connected component,on which the filter is applied, are compared to a given thresholddefined by the rule. If a criterion T is increasing, then, if it issatisfied for a connected component X , it will be also satisfiedby all the regions that enclose X (i.e., the superset of X).Formally for X ⊆ Y , T (X) ⊆ T (Y ). Examples of increasingcriteria are the comparison to a reference value (λ) of theattributes computed on a region such as the area, the volume(sum of the graylevels of all the pixels belonging to the region),the size of the bounding box, etc. It is straightforward to provethat every region enclosing the one on which these criteria arecomputed will have attribute values that are greater or equalthan the computed ones.

Binary attribute opening ΓT , given an increasing criterionT , is defined as a binary trivial opening applied on all the con-nected components of F . This can be formally represented as

ΓT (F ) =⋃x∈F

ΓT [Γx(F )] . (8)

If the criterion evaluated in the transformation is not increasing,e.g., when the computed attribute is not dependent itself on thescale of the regions (e.g., shape factor, orientation, homogene-ity, etc.), the transformation also becomes not increasing. Evenif the increasingness property is not fulfilled, the filter remainsidempotent and anti-extensive. For this reason, the transfor-mation based on a non-increasing criterion is not an opening,but a thinning. Then, analogously to the definition of attributeopenings, binary attribute thinning Γ̃T can be defined as

Γ̃T (F ) =⋃x∈F

Γ̃T [Γx(F )] (9)

where Γ̃T denotes a trivial thinning.All the definitions above can be extended to the respective

dual transformations. Binary attribute closing ΦT .1 Thus, de-fined as the binary union of the connected components thatfulfill the criterion T

ΦT (F ) =⋃x∈F

ΦT [Φx(F )] . (10)

This is based on the operators of ΦT and ΦX , respectively,binary trivial closing and binary connected closing.

1In contrast to [16], here Φ denotes a closing transformation.


Non-increasing attribute T , leads to a binary attribute thick-ening Φ̃T given by

Φ̃T (F ) =⋃x∈F

Φ̃T [Φx(F )] (11)

with Φ̃T the binary trivial thickening.

B. Grayscale Attribute Operators

Attribute openings and thinnings introduced for the binarycase can be extended to grayscale images employing the thresh-old decomposition principle [2]. A grayscale image can berepresented by a stack of binary images obtained by thresh-olding the original image at each of its graylevels. Then, thebinary attribute opening can be applied to each binary imageand the grayscale attribute opening is given by the maximumgraylevel of the results of the filtering for each pixel and can bemathematically presented as

γT (f)(x) = max{k : x ∈ ΓT [Thk(f)]

}(12)

where Thk(f) is the binary image obtained by thresholding fat graylevel k (with k ranging on the graylevels of f ).

The extension to grayscale of the binary attribute thinningis not straightforward and not unique. For example, a possibledefinition of a grayscale attribute thinning can be given analo-gously to (12) as:

γ̃T (f)(x) = max{k : x ∈ Γ̃T [Thk(f)]

}. (13)

However, other definitions are possible, according to the filter-ing rule considered in the analysis. A list of possible filter-ing rules is presented in Section III-C. If the criterion T isincreasing, the result of the thinning is actually an openingtransformation. In this case, (13) is equal to (12). Although, thisapproach does not lead to the fastest implementation of suchoperators, it permits a more direct link to the correspondentbinary operators.

Attribute openings are a family of operators that also includesopenings by reconstruction [16]. If we consider a binary imagewith a connected componentX and the increasing criterion “thesize of the largest square enclosed by X must be greater thanλ,” the result of the attribute opening is the same as applying anopening by reconstruction with a squared SE of size λ. The ex-tension of this example to the grayscale case is straightforward.Given this correspondence between the criterion and the SE, wepoint out that any opening by reconstruction can be denoted andperformed as attribute opening.

The concepts presented above are straightforwardly extendedto closing/thickening leading to the definition of grayscaleattribute closing, i.e.,

φT (f)(x) = min{k : x ∈ ΦT [Thk(f)]

}(14)

and grayscale attribute thickening:

φ̃T (f)(x) = min{k : x ∈ Φ̃T [Thk(f)]

}. (15)

As for attribute thinning, other definitions are available accord-ing to the selected filtering rule. For an example of the effectsof attribute filtering, the reader can refer to Fig. 2 (syntheticimage) and Fig. 6 (real remote sensing image).

C. Max-Tree

The Max-tree data representation, introduced bySalembier et al. [19], is of particular interest because itincreases the efficiency of the filtering by splitting the trans-formation process into three separate phases: 1) tree creation;2) filtering; and 3) image restitution. These phases are presentedin detail below.

1) Max-tree creation. For simplicity, let us initially considera binary image F . It can be represented in a rootedtree structure with a depth of two. This is composed bya single root node, which represents the pixels of thebackground, and children nodes connected to the root,where each of them refers to a connected component(Ci) in F . The extension of the tree built for the binarycase to the grayscale can be easily explained by thethreshold decomposition of the image. The levels in thedepth of the tree represent the graylevels of the imageand at each level, the number of nodes corresponds to thenumber of connected components present in the binaryimage obtained by thresholding the current graylevel.The tree starts to grow from the root. The connectedcomponent of the root is given by thresholding the imageat its lowest graylevel, thus representing the entire imagedomain, E. By increasing the value of the graylevel,the thresholded image will show separated connectedcomponents, represented by nodes at the level in the treecorresponding to the threshold. Those nodes are thenlinked to their parent nodes at the closest inferior levelin the tree. In the image, each parent node correspondsto a connected component which is a superset of the con-nected component represented by the children node in thetree. The procedure is iterated until the threshold reachesthe maximum graylevel of the image, which defines theleaves of the tree (the absolute maxima of the image).

2) Filtering. Once the tree is defined, the criterion associatedwith the transformation is evaluated at each node (i.e.,the attribute is checked against a reference value λ). Sub-sequently, the tree is pruned by removing the nodes thatdo not satisfy the criterion according to a filtering rule.There are two typologies of filtering rules: 1) pruningstrategies, which remove or preserve a node together withits descendants; and 2) nonpruning strategies, where ifa node is removed, its children are linked to the par-ent of the removed node. Below we briefly discuss afew strategies. The Min, Max and the Viterbi decisionrules [19] are pruning strategies, while the Direct andthe Subtractive rules [20] are nonpruning strategies. Inparticular, the Subtractive rule proved to be particularuseful when associated to attributes for describing theshapes of objects [21].

• Min: If a node does not satisfy the criterion, then itis removed together with all its descendants;


Fig. 1. Example of the Max-tree representation. (a) Input image and (b) Max-tree structure representing the connected components of the image. Each nodereports also the values of three attributes: area (A), moment of inertia (I) andstandard deviation (S).

• Max: A node that does not satisfy the criterion issuppressed, only if the criterion is not satisfied byall of its descendants.

• Direct: If a node does not satisfy the criterion, thenit is removed by leaving unaffected all of its descen-dants that satisfy it.

• Viterbi: The nodes are removed by evaluating thecosts associated with the decision and by taking thesolution with the minimum cost.

• Subtractive: If a node does not satisfy the criterion,then it is removed and all its descendants are loweredby its gray level. A formal definition of the grayscaleattribute thinning operator employing this filteringrule can be found in [21].

Again, if the criterion is increasing (i.e., the transfor-mation is an opening), all the strategies lead to the sameresult.

3) Image restitution. The final phase of the transformationaims at converting back the pruned tree to an image.This is done by assigning to each pixel the graylevelcorrespondent to the highest level of the tree having anode whose correspondent connected component in theimage encloses the pixel.

In Fig. 1(b), the Max-tree of a sample image of 1000 × 1000pixels [Fig. 1(a)] is presented. On each connected component ofthe image (correspondent to a node in the tree), three attributesare computed. Fig. 2 shows the Max-tree of the image prunedaccording to different criteria and the correspondent filteredimages. The attributes selected are: 1) area (related the sizeof the regions); 2) first moment invariant of Hu [22], alsoreferred as “moment of inertia” (which models the elongationof the regions); and 3) standard deviation (which measures thehomogeneity of the pixels enclosed by the regions). The firstmoment invariant of Hu can be associated to the moment ofinertia in kinematics because it measures the spread of a regionwith respect to its center of mass [22]. The moment of inertiaattribute is a measure of the noncompactness of the objects,since it has small values for compact regions, while rapidlyincreases for the elongated ones [23].

The flexibility of the attribute selection makes these filtersappealing with respect to traditional opening and closing byreconstruction based on SEs, since any attribute that can becomputed on the regions of the image can be selected for theanalysis. As an example, some attributes that can be interestingfor the analysis of remote sensing images are area, volume(sum of the intensities of the pixels belonging to each region),length of the diagonal of the box bounding each region, mo-ment of inertia, shape factor, simplicity and complexity of theregions [19], homogeneity, standard deviation, and entropy ofthe grayscale values of the pixels. Other examples of attributesused by attribute filters can be found in [16] and other regionaldescriptors that can be used as attributes can be found in [23].

The separation of the computation of the attributes and thefiltering phase results in another advantage of this filteringarchitecture over the application of conventional operators byreconstruction. In fact, since the value of the attributes iscomputed on all the regions before the filtering phase, it ispossible to avoid defining λ, the threshold value checked bythe criterion, for values that are not significant (e.g., out of therange of the real values of the attributes).

IV. MORPHOLOGICAL ATTRIBUTE PROFILES2

In this section, we introduce the concepts of attribute profiles(APs) and differential attribute profiles (DAPs). These mul-tilevel filters are based on morphological attribute operatorsand they are a generalization of the conventional MPs andDMPs discussed in Section II-B. For simplicity, we first discussin detail the case of anti-extensive idempotent operators (i.e.,openings and thinnings). Then, we extend the obtained resultsto the extensive counterpart (i.e., closings and thickenings).

A. Attribute Profiles

The definition of an attribute opening profile is quite straight-forward since a sequence of attribute openings with a familyof increasing criteria T = {Tλ : λ = 0, . . . , n}, with T0 = true∀X ⊆ E, leads to a granulometry. Thus, attribute openingprofiles can be mathematically defined as

ΠγT (f) ={ΠγTλ : ΠγTλ = γTλ(f),∀λ ∈ [0, . . . , n]

}. (16)

As for an MP, when λ = 0, ΠγT0 (f) = γT0(f) = f . We pointout that this definition of attribute opening profile includesalso the morphological opening profile by reconstruction, sinceopenings by reconstruction are a particular set of attributeopenings. By comparing attribute profiles to conventional MPs,it can be noticed that both perform multiscale analysis of theimage, since the SE/criterion, driven by the increasing scalarλ, progressively erases from the image larger structures. More-over, attribute opening profiles provide the same capabilitiesin processing the image as for openings by reconstruction, butadding more flexibility in the definition of the filtering criterion.For example, if we consider a compact SE (e.g., square-,

2A MATLAB application that implements the proposed attribute profiles isavailable on request.


Fig. 2. Examples of attribute filtering on the image of Fig. 1(a). The first row reports the obtained filtered images, the second row the correspondent prunedMax-tree. Criterion: (a) “area > 20 000”; b) “moment of inertia > 0.25” and (c) “standard deviation > 0.”

disk-shaped), the structures are removed from the scene if theSE does not fit in them. Thus, the image is processed accordingto the smallest size of the regions. If we consider instead thelength of the diagonal of the box bounding each region as anattribute, then, the structures are filtered according to a measureof their global extension, which is still related to the conceptof scale, but in a different way with respect to considering thesmallest size of the objects. Moreover, if we take into accountthe area of the regions, a different measure of the size ofthe objects is provided. Thus, by selecting different type ofattributes, even if they are all increasing measures, differentcharacterizations of the scale of the structures are generated.

If we consider other types of attributes not constrained by theincreasingness property, a different behavior is achieved by thefilters. For instance, it is possible to assess how the image reactsto a filtering done on multiple levels with an attribute invariantto changes in scale. This would permit to characterize the imageby extracting information related to the shape of the structuresthrough a measure which is independent of their size [21]. Thus,the application of attribute thinning in a multilevel approachleads to attribute thinning profiles. However, their definition isnot direct as for attribute opening profiles. In fact, since attributethinnings are not increasing, the absorption law might not besatisfied in the profile. This can result in sequential elements ofthe profile that are not ordered. For example, regions erased ata certain level of the profile might appear again in subsequentlevels associated to more relaxed criteria. This is an undesirableeffect particularly if a derivative of the profile needs to becomputed. In order to build a consistent profile on attributethinnings, it is necessary that the absorption law is fulfilled bythe filtered images, leading the AP to be a set of cumulativefunctions. This can be obtained by constraining the criteria usedin the filtering. The family of non-increasing criteriaU = {Uλ :λ = 0, . . . , n} considered for computing the profile has to bean ordered set. Moreover, the criteria have to be consistentlyeither in the form of Uλ = a(X) > τλ or Uλ = a(X) < τλ forall the connected component X ⊆ E, and τi ≤ τj for i ≤ j,

with a denoting a generic non-increasing attribute computedon the component X , and τλ being the scalar value taken asthe threshold at the level λ of the profile. If the criteria areordered and defined as mentioned above, then the following ruleholds: If a connected set X ⊆ E does not satisfy the criterionUi (i.e.,Ui(X) = false), then alsoUj(X) = false, with i ≤ jand Ui, Uj ∈ U . Thus, for binary trivial thinning, it holdsthat if Γ̃Ui

(X) = ∅ ⇒ Γ̃Uj(X) = ∅ and this leads Γ̃Ui(F ) ⊆

Γ̃Uj (F ) for binary attribute thinning. In the grayscale case, itbecomes γ̃Ui(f) ≤ γ̃Uj (f). The latter property corresponds tothe absorption law that can be expresses also as γ̃Ui γ̃Uj (f) =γ̃Umax(i,j)(f). Thus, by selecting these criteria, the profile isbehaving like a granulometry.

Consequently, it is possible to define an attribute thinningprofile, based on a set of ordered criteria U = {Uλ : λ =0, . . . , n}, with U0 = true ∀X ⊆ E as

Πγ̃U (f) ={Πγ̃Uλ : Πγ̃Uλ = γ̃Uλ(f),∀λ ∈ [0, n]

}. (17)

Actually Πγ̃U includes also ΠγT in its definition since theattribute thinning profile produces the same results as for theattribute openings if the criteria U fulfill the more restrictiveproperty of increasingness. By duality, the attribute closingprofile can be defined as

Πφ̃U (f) ={

Πφ̃Uλ : Πφ̃Uλ = φ̃Uλ(f),∀λ ∈ [0, n]}

(18)

and analogously to (3), we can define an attribute profile as(19), shown at the bottom of the next page. Attribute thinningprofiles permit us to perform a multilevel analysis of the imagebased on attributes (represented by ordered criteria) not nec-essarily related to the scale of the structures of the image. Infact, the choice of attributes like the shape factor, the spatialmoments, etc., results in an AP that represents a multilevel (notmultiscale) decomposition of the image according only to theshape of the regions. Furthermore, the attribute can also be a


Fig. 3. Examples of four differential profiles computed on four samples belonging to different thematic classes (Vegetation, Road, Building, and Shadow) froma panchromatic Quickbird image of Trento (Italy). The values of the shown profiles are normalized in the range [0,1]. The horizontal axis reports the levels of theprofiles. In the legend, DMP refers to the conventional DMP built by a squared-SE, DAPa, DAPi, DAPs denote the differential attribute profiles built on the area,moment of inertia, and standard deviation attribute, respectively. The subtractive rule was considered for the non-increasing criteria. (a) Vegetation. (b) Building.(c) Road. (d) Shadow.

measure which is not related to the geometry of the regionsbut to the graylevels of their pixels. For example, the scene canbe simplified by removing structures according to homogeneityinstead of their scale or shape.

As for MPs, the residuals of the progressive filtering canbe important. Thus we can extend (4) by introducing thedifferential attribute profile for the set of non-increasing criteriaU as (20), shown at the bottom of the page, where Δφ̃Uλ

and Δγ̃Uλ represent the differential thickening and thinningprofiles, respectively, whose definition is straightforward andthus not reported.

Examples of a DMP and three DAPs computed on a panchro-matic Quickbird image of the city of Trento (Italy) for fourdifferent thematic classes are presented in Fig. 3. The attributesselected for the three DAPs are: 1) area; 2) moment of inertia;and 3) standard deviation. By analyzing the differential profiles,

Π(f) ={

Πi :⟨

Πi = Πφ̃Uλ , with λ = (n− i+ 1), ∀i ∈ [1, n]Πi = Πγ̃Uλ , with λ = (i− n− 1), ∀i ∈ [n+ 1, 2n+ 1]

}(19)

Δ(f) ={

Δi :⟨

Δi = Δφ̃Uλ , with λ = (n− i+ 1), ∀i ∈ [1, n]Δi = Δγ̃Uλ , with λ = (i− n), ∀i ∈ [n+ 1, 2n]

}(20)


Fig. 4. Data set 1—(a) Panchromatic image of 400 × 400 pixels, (b) map of the test areas, and (c) map of the objects selected for the assessment of thegeometrical accuracy.

as expected, it can be noticed that the DMP shows a similarbehavior to the DAP with the area attribute, since both processshows the image according to the scale of the objects. TheDAPs built on the moment of inertia and the standard deviationhave a different behavior from the scale attributes. However,given a region, regardless of the type of attribute considered,the active responses of the pixels belonging to the region inthe profile are all located either in the opening or closing partof the profile. In fact, dark objects are detected in the closingprofile and bright ones on the opening side. The diversityshown by considering the DAPs built on different types ofattributes results in features that potentially can increase in theseparability of the information classes.

B. Analysis of the Complexity

The main advantage, in terms of computational complexity,of the approach based on the Max-tree, with respect to the use ofoperators by reconstruction for performing multilevel filtering,relies on the fact that the image has not to be completelyprocessed at each level of the profile. In fact, the tree structureis built only once from the original image and after the attributeis computed on the components of the image, the same datastructure is pruned by a set of thresholds λ, generating thefiltered images at the different levels. Moreover, we pointout that, if an attribute can be computed incrementally (e.g.,area, volume, standard deviation, etc.), the computation of theattribute can be embedded in the creation of the tree, thusavoiding visiting all the nodes further. If a multilevel, multi-attribute analysis is performed, the processing can further takeadvantage from the architecture based on the Max-tree. In fact,the tree is still created only once, and the investigated attributescan be computed on the nodes, if possible, directly during thecreation of the tree. However, even if the attributes need to becomputed off-line after the creation of the tree, they can becalculated simultaneously at the visiting of each node, requiringa single scan of the tree. Moreover, during the computation ofthe attributes, their dependences can be exploited. For example,if the standard deviation and the area attributes need to becomputed, the former requires in its definition the computationof the area, which can be directly exploited from the secondattribute. Obviously, this further optimizes the analysis. Finally,the evaluation stage simply checks the criteria against theattributes values of the nodes in the tree. This is the onlyoperation in the entire analysis that linearly depends on thenumber of levels and attributes considered.

If we quantitatively analyze the computational complexityof the implementation of the different operators, the conven-tional opening by reconstruction based on the iterative geodesicreconstruction [2] has a worst case time complexity with anupper bound of O(N2), where N is the number of pixels inthe image. When computing a granulometry by reconstructioncomposed by L levels, the computational complexity has anorder of 2LN2 in the worst case. Vincent [24] proposed anefficient algorithm based on first-input-first-output queue andtwo raster scans of the image which is an order of magnitudefaster than the conventional technique and, thus, can reducethe load of computing a profile. Nevertheless, the image hasto be entirely processed 2L times, regardless the algorithmconsidered. Instead, when considering an approach based onthe Max-tree, the computational complexity of the analysiscan be reduced. The most demanding stage of an attributefiltering based on the Max-tree is the creation of the tree thatrelies on a flood-filling algorithm. This algorithm is linear withrespect to both the number of pixels and the connectivity [25].The pruning of the tree and the image restitution are bothO(N) operations. Thus, the computational cost of a profileis O(NG+ 4LN), being G the number of graylevels in theimage. On parallel machines, the Max-tree computation is fur-ther speeded up according to a slightly varying implementationbased on the union-find algorithm [26]. More considerations onthe memory use of Max-trees according to their implementationcan be found in [25] and [26].

V. EXPERIMENTAL ANALYSIS

A. Data Set Description

The experimental analysis was carried out by classifyingtwo portions taken from a large VHR panchromatic imageacquired by the Quickbird sensor on July 2006 with geometricresolution of 0.6 m. We did not consider the multispectralimages acquired by the Quickbird scanner in order to focus theanalysis only on the capabilities of different APs to model thegeometrical/spatial information. This choice is also reasonablefor some operational conditions when satellites that acquireonly the panchromatic band (e.g., WorldView 1) are used.

The two considered images are made up by 400 × 400[Fig. 4(a)] and 900 × 900 [Fig. 5(a)] pixels, respectively. Boththe images represent two complex urban areas belonging to thecity of Trento, Italy. Most of the surveyed buildings are residen-tial with heterogeneous size and shape. Some large industrialbuildings are also present in the scene. The presence of shadows


Fig. 5. Data set 2—(a) Panchromatic image of 900 × 900 pixels, (b) map of the test areas, and (c) map of the objects selected for the assessment of thegeometrical accuracy.

TABLE INUMBER OF SAMPLES PER CLASS FOR THE TRAINING AND TEST SET FOR THE TWO DATA SETS

can be observed particularly in proximities of buildings. Allthese factors contribute to the complexity of the consideredscene.

The pixels of the two images were grouped into four infor-mative classes: 1) Road; 2) Building; 3) Shadow; and 4) Veg-etation. For both images, a training set, composed by samplesrandomly selected from labeled areas not included in the testsets, was considered and two independent test sets were definedby photo-interpretation in order to evaluate the performancesof the classification. One test set is devoted to the evaluationof the thematic accuracy, while the other checks the geometricprecision of the classification map on a set of selected objects inthe scene according to the protocol proposed in [27] and [28].The geometrical accuracy is evaluated by a set of five indexesmodeling: 1) oversegmentation (OS); 2) undersegmentation(US); 3) fragmentation (FG); 4) shape factor (SH); and 5) errorson the objects borders (ED). The index modeling the OS gives ameasure of the overlap between the region which mostly coversa reference objects in the classification map and the area ofreference objects. The US error computes how much the regionscorrespondending to the reference objects are larger than thereference objects. The FR index refers to a descriptor of howthe areas of the reference objects are fragmented in differentregions in the classification map. Finally, the SH and the EDmeasures indicate how the shapes and the edges, respectively,of the reference objects differ to those of the correspondentregions in the reference map. All the error indexes range fromzero to one (in the tables, the values are given in percentages),with zero representing a perfect match and one the greatestdivergence between the reference objects and the correspondentregions in the classification map. For further information onthe geometric error indexes, the reader can refer to [28]. Thetwo test sets are reported in Fig. 4(b) and (c) and Fig. 5(b) and(c) for data set 1 and 2, respectively. The number of samplesselected for training and testing the two data sets are reported inTable I.

B. Results

For both the images, a 17-D morphological profile wasgenerated using a squared-SE with size increasing in eight steps(7, 13, 19, 25, 31, 37, 43, and 49). These values were arbitrarilychosen and since they range from 4.2 to 29.4 m, they are ableto model the size of the heterogeneous objects in the scene.Three attribute profiles with the same dimensionality of theMP were also created following the approach based on theMax-tree data structure. All the filtering transformations wereperformed on the already constructed tree in order to reduce thecomputational burden. For all the APs, the considered criterionwas “the attribute must be greater than λ.”

Three different attributes were considered for the construc-tion of the AP: 1) the area; 2) first moment of Hu; and 3) thestandard deviation. The AP with the area attribute describes thescale of the structures in the scene; it is the only increasingattribute among the three selected. In order to create the profilewith the area attribute, the following values of λ were selected:49, 169, 361, 625, 961, 1369, 1849, and 2401. Although, thesevalues correspond to the square of the SE sizes used for creatingthe MP, the multiscale analysis obtained models; the scale ofthe objects in the scene with a different criterion with respectto the MP. The second attribute considered is the momentof inertia. The original image was filtered by progressivelysuppressing from the scene those regions with attribute smallerthan the following increasing thresholds: 0.2, 0.3, 0.4, 0.5, 0.6,0.7, 0.8, and 0.9. The AP based on the standard deviationattribute performs a multilevel decomposition of the objectsin the scene that is not related to the geometry of the regionsbut models the homogeneity of the graylevels of the pixels inthe regions. The profile was built according to the followingreference values of the standard deviation: 10, 20, 30, 40, 50,60, 70, and 80. As for the definition of the SE sizes in theMP, the threshold values of λ were arbitrarily selected in orderto cover the significant range of variation of the attribute forall the connected components of the image. Different analyses


Fig. 6. Extracts of differential profiles built on the first data set. (a) DMP created by a SE with a square shape, DAPs with (b) area attribute, (c) moment of inertiaattribute, and (d) standard deviation attribute. For all the profiles the levels one, three, five, and seven are reported from left to right. All the images are stretchedfor visual purposes.

were carried out on the data. At first, each AP was consideredseparately and then, all the features extracted by the APs weretaken into account simultaneously.

In order to compare the behavior of the different profiles,we chose to present the derivatives of the constructed profiles(i.e., DMP and DAPs) because the differences among them areperceptually more visible than by analyzing the correspondentmorphological/attribute profiles. The DMP [Fig. 6(a)] is visu-ally similar to the DAP built by evaluating the area attribute[Fig. 6(b)]. Many regions, which are suppressed at a certainlevel in the DMP, are present at the same level in the DAP.However, some other objects are not revealed at the same levelin the two profiles but in adjacent levels. For example, the thinand elongated region in the middle of the scene that is presentin the second and third image from the left (levels three andfive, respectively) in the DMP, in the DAP results in the thirdand fourth (respectively five and seven). These differences inthe two differential profiles are mainly due to the different

modeling of the concept of scale and to the choice done forthe step size of the SEs and of the values of the thresholdsλ, for the area attribute. In particular, the filters based on thearea attribute remove the structures from the image according totheir cardinality, whereas, the operators by reconstruction witha square SE interact to the smallest size of each region. Thus,this different behavior is particularly evident when consideringelongated regions.

Different conclusions can be drawn by comparing the DMPto the DAPs generated by the moment of inertia and thestandard deviation. At first, it is evident that at higher levelsof the profiles (i.e., related to large values of λ), also regionsthat are spatially smaller than some others, appeared in previouslevels, are present. This is due to the non-increasingness of theselected criterion.

In order to quantitatively compare the capabilities of theproposed profiles in modeling the spatial characteristics ofthe scene, we classified the original image using each profile.


TABLE IIERRORS OBTAINED BY CLASSIFYING THE PANCHROMATIC IMAGE ALONG WITH MORPHOLOGICAL/ATTRIBUTE PROFILES FOR DATA SET 1

TABLE IIIERRORS OBTAINED BY CLASSIFYING THE PANCHROMATIC IMAGE ALONG WITH MORPHOLOGICAL/ATTRIBUTE PROFILES FOR DATA SET 2

TABLE IVCLASS SPECIFIC PRODUCER ACCURACY (PA) AND USER ACCURACY (UA) OBTAINED BY CLASSIFYING THE PANCHROMATIC IMAGE ALONG

WITH MORPHOLOGICAL/ATTRIBUTE PROFILES FOR DATA SET 1. THE BEST ACCURACIES OBTAINED ARE MARKED IN BOLD

A random forest technique with 200 trees was used for theclassification [29]. The random forest classifier is formed by anensemble of decision tree classifiers. We chose to use this non-parametric classifier because of the high redundancy shownby the profiles that can be critical for the estimation of thestatistics in classical parametric classifiers. The classificationis achieved by selecting the output of the ensembles of thetree classifiers according to a majority voting. The featuresconsidered by the classifier were the panchromatic band andthe generated profiles. For the definition of the split on eachnode in the random forest, the number of considered variableswas correspondent to the square root of the number of inputfeatures. The aim of this analysis was to investigate how theaccuracy (both thematic and geometric) varies when includingin the analysis the knowledge gathered on the spatial domainby the profiles. In particular, the results obtained by consideringthe panchromatic band and a conventional MP were comparedto those obtained by different APs.

Tables II and III show the thematic error index, in termsof percentage overall error and the kappa error (computed as

1-kappa coefficient in percentage) on the test set, and the fivegeometric error indexes. Furthermore, the accuracies obtainedby each class are shown in Tables IV and V. In particular, theproducer and user accuracy are reported. We recall that the PAis computed, for each class, as the total number of the patternscorrectly classified divided by the total number of the patternsbelonging to the considered class in the reference map. The PAmeasures how many reference patterns are correctly classifiedby each class. The UA is obtained by dividing the total numberof correctly classified patterns for each class by the total numberof patterns classified to the same class. The UA indicates howmany samples associated to a class are actually belonging tothat class in the reference. More information on PA and UA canbe found in [30].

By analyzing the thematic accuracies reported in Table IIfor the original panchromatic band, one can observe that aclear increase of the accuracy is obtained by using jointlythe features that model the spatial information. The accuracyachieved by considering the MP is comparable to the oneobtained by the single APs with moment of inertia and standard


TABLE VCLASS SPECIFIC PRODUCER ACCURACY (PA) AND USER ACCURACY (UA) OBTAINED BY CLASSIFYING THE PANCHROMATIC IMAGE ALONG

WITH MORPHOLOGICAL/ATTRIBUTE PROFILES FOR DATA SET 2. THE BEST ACCURACIES OBTAINED ARE MARKED IN BOLD

Fig. 7. Data set 1. Classification maps obtained by (a) panchromatic image only, (b) MP, (c) AP area, (d) AP inertia, (e) AP std, (f) AP inertia + AP std, and(g) AP all.

deviation attributes. Instead, the AP constructed on the areaattribute produced the highest overall error and kappa erroramong the profiles. This is due to the selected thresholds usedfor computing the filtering, which might not properly modelthe great variety in the scale of the objects for the consideredscene. The best results, according to the thematic accuracy, areobtained by the joint use of the AP with moment of inertiaand the AP with standard deviation attribute, which reduced theoverall classification error by about 24% and the kappa error of32%, with respect to the use of the only original panchromaticimage. The improvement was about 9% in overall error and14% in kappa error, with respect to the conventional MP. Evenif the global accuracies are in general quite small, making morecomplex the visual interpretation of the maps, by evaluatingthe geometric indexes, one can see that the classification ofthe panchromatic image shows a large oversegmentation error(thus, a small undersegmentation error) with respect to the mapsobtained by considering the profiles. This behavior is also con-firmed by a visual inspection of the classification maps shownin Fig. 7. In fact, it is possible to observe that the classificationmap obtained with the panchromatic image [Fig. 7(a)] is highlyfragmented, whereas, the other maps are more homogeneous.As best case, when considering the map obtained by the APwith moment of inertia, a reduction by about 29% and 7% inthe oversegmentation and fragmentation error, respectively, is

achieved. This effect can be noticed in the row of buildingsat the top of the image. Nevertheless, the AP inertia shows ahigh US error which can be due to the missed recognition ofthe buildings on the bottom of the image and the generationof broad areas. The lowest US error among the profiles andthe overall lowest ED error are achieved, by considering theAP with moment of inertia and the AP with standard deviationattribute together.

Table III shows the error rates on the test set obtained byanalyzing the data set 2. As for the previous data set, thethematic errors decrease when considering the spatial infor-mation provided by the profiles. Also, in this case, the resultsobtained by considering a single AP are similar to those gen-erated by the MP (this is also clear from the classificationmaps in Fig. 8). In this experiment, the AP built on the areaattribute results in a thematic error only slightly smaller thanthe one of the original panchromatic image (about 4% inboth overall and kappa errors). However, as confirmed by themap, the geometrical errors are similar with those obtainedby considering the other profiles. Again, the highest thematicaccuracy is obtained when considering the APs with momentof inertia and standard deviation attributes. The thematic errorsare reduced by about 28% in the overall error and 38% inkappa errors with respect to the original panchromatic imageand by about 12% and 17% (overall and kappa errors) against


Fig. 8. Data set 2. (a) Panchromatic band; classification maps obtained by (b) MP, (c) AP area, (d) AP inertia, (e) AP std, (f) AP inertia + AP std, and(g) AP all.

the conventional MP. As for the first data set, better accuraciesare obtained by considering only the AP with moment of inertiaand the AP with standard deviation attribute than consideringtogether also the AP with area attribute. This can be due tothe increase in the dimensionality of the feature space givenby considering also the AP with area attribute, which makesthe analysis more complex (i.e., Hughes phenomenon), withoutproviding enough additional independent information than tothe other APs. However, one should observe that the selectionof the threshold values λs affects the capability of the computedprofile in modeling the spatial features of the objects. Thus, anAP with different threshold values for the area attribute mightprovide features which are more discriminant.

In addition, also the geometry of the reference objects isglobally more precisely preserved by the AP with momentof inertia and AP with standard deviation attribute consideredtogether in comparison to the other single profiles. In particular,the oversegmentation error in the map obtained by consideringall the APs decreases of about 34% and 18%, compared to thatof the map generated by the only panchromatic image and bythe MP, respectively. This can be observed as a more uniformclassification of the vegetated areas in the middle of the imageand of some roads. As for data set 1, the AP inertia shows smallUS and FG errors but high US error, which can be due to thepresence of large areas associated to the Vegetation class.

By considering Tables IV and V, it is possible to make adetailed class-by-class analysis by considering the produceraccuracy (PA) and user accuracy (UA) obtained. The two resultsfor both the data sets are analyzed together in order to observetrends in the obtained results. Focusing the attention on thespecific thematic classes, we can underline as, with respect tothe other attributes, the AP with moment of inertia performedwell in identifying the roads, in particular for data set 2.However, for both data sets, the best results were obtained byconsidering all the APs. When considering the Building class,the conventional MP, the AP with standard deviation attribute,

and the AP with all the attributes performed the best and gavecomparable results. For this particular class, good results werealso obtained by the AP with moment of inertia but only indata set 1. The class Shadow was globally well-classified by allthe profiles and no particular trend in the results was noticed.Finally, the vegetated areas were extracted well by the AP withthe standard deviation attribute particualrly in data set 1. TheMP and the AP with moment of inertia and the one with all theattributes also reached similar results.

VI. CONCLUSION

In this paper, attribute profiles have been introduced forclassification of very high resolution remote sensing images anddifferential attribute profiles have been proposed and formallydefined. The motivation of this work relies on the need to im-prove the flexibility, the capability of modeling different kind ofobjects, and the computational load associated with the widelyused conventional morphological profiles and their derivative.

Attribute profiles can be used for extracting informationfrom the spatial domain by reducing the limitations of themorphological profiles. This approach allows one to analyzethe original image in a multilevel fashion by the applicationof a sequence of morphological attribute operators. Theseoperators are adaptive morphological connected filters, whichinclude in their general definition also opening and closing byreconstruction. Attribute filters are flexible tools that enableto analyze an image, not only on the basis of the scale ofthe structures (as for operators by reconstruction), but alsoaccording to other measures/attributes computed on the regions.Thus, it is possible to perform a multilevel analysis of the sceneby exploiting measures related to many different geometricprimitives (e.g., shape), the graylevel of the pixels, or any otherparameter that can be computed on the regions.

In the paper, we propose to compute the attribute profiles ac-cording to an effective implementation based on the Max-tree,


i.e., an efficient representation of the data, which leads to areduction of the computational load of about one order ofmagnitude with respect to morphological profiles.

The proposed technique was applied to two very high res-olution panchromatic images acquired by Quickbird satelliteon the city of Trento, Italy. Three attribute profiles, based ondifferent attributes, were extracted from the panchromatic band.We considered 1) the area (which is related to the MP createdwith a squared SE); 2) the moment of inertia (which is adescriptor of the geometry of a region invariant to the scale);and 3) the standard deviation of the graylevels of the pixels(which measures the homogeneity of the regions). The datawere classified by a random forest classifier. The obtainedmaps were evaluated by checking their thematic accuracy andthe geometric precision in representing some reference ob-jects in the scene. The results pointed out the effectiveness ofthe proposed APs, which involved a sharply higher thematicand geometric accuracy with respect to considering the onlypanchromatic band. Moreover, the profiles built on differentattributes led to similar results in terms of accuracy, but alsoconveyed different and complementary information into theclassification process. In fact, the joint use of the three attributeprofiles in the classification tasks resulted in an decrease of theclassification kappa errors up to 38% and 17% with respectto the only panchromatic image and to the MP, respectively.The obtained classification maps are also more precise in therepresentation of the geometry of the regions as proven by thegeometrical error indexes.

As future developments, we plan to investigate in depththe capabilities of APs and DAPs in the analysis of veryhigh resolution images particularly for applications where theextraction of the spatial information has a fundamental role,such as the extraction and characterization of the objects inthe scene. Moreover, it would be very interesting to analyzehow the modeling of many different geometrical features canimprove the analysis of multitemporal image series (e.g., forchange detection task) particularly for urban areas.

ACKNOWLEDGMENT

This work was supported in part by the Research Fund of theUniversity of Iceland and of the University of Trento.

REFERENCES

[1] F. Dell’Acqua, P. Gamba, A. Ferari, J. A. Palmason, J. A. Benediktsson,and K. Arnason, “Exploiting spectral and spatial information in hyper-spectral urban data with high resolution,” IEEE Geosci. Remote Sens.Lett., vol. 1, no. 4, pp. 322–326, Oct. 2004.

[2] P. Soille, Morphological Image Analysis, Principles and Applications,2nd ed. Berlin, Germany: Springer-Verlag, 2003.

[3] M. Pesaresi and J. A. Benediktsson, “A new approach for the morpho-logical segmentation of high-resolution satellite imagery,” IEEE Trans.Geosci. Remote Sens., vol. 39, no. 2, pp. 309–320, Feb. 2001.

[4] M. Chini, N. Pierdicca, and W. Emery, “Exploiting SAR and VHR op-tical images to quantify damage caused by the 2003 Bam earthquake,”IEEE Trans. Geosci. Remote Sens., vol. 47, no. 1, pp. 145–152,Jan. 2009.

[5] D. Tuia, F. Pacifici, M. Kanevski, and W. Emery, “Classification ofvery high spatial resolution imagery using mathematical morphologyand support vector machines,” IEEE Trans. Geosci. Remote Sens., vol. 47,no. 11, pp. 3866–3879, Nov. 2009.

[6] H. Akcay and S. Aksoy, “Automatic detection of geospatial objects usingmultiple hierarchical segmentations,” IEEE Trans. Geosci. Remote Sens.,vol. 46, no. 7, pp. 2097–2111, Jul. 2008.

[7] J. Chanussot, J. Benediktsson, and M. Fauvel, “Classification of remotesensing images from urban areas using a fuzzy possibilistic model,” IEEEGeosci. Remote Sens. Lett., vol. 3, no. 1, pp. 40–44, Jan. 2006.

[8] J. A. Benediktsson, M. Pesaresi, and K. Arnason, “Classification andfeature extraction for remote sensing images from urban areas basedon morphological transformations,” IEEE Trans. Geosci. Remote Sens.,vol. 41, no. 9, pp. 1940–1949, Sep. 2003.

[9] J. A. Benediktsson, J. A. Palmason, and J. R. Sveinsson, “Classificationof hyperspectral data from urban areas based on extended morphologicalprofiles,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 3, pp. 480–491,Mar. 2005.

[10] M. Fauvel, J. Benediktsson, J. Chanussot, and J. Sveinsson, “Spectraland spatial classification of hyperspectral data using SVMs and mor-phological profiles,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 11,pp. 3804–3814, Nov. 2008.

[11] R. Bellens, S. Gautama, L. Martinez-Fonte, W. Philips, J. C.-W. Chan, andF. Canters, “Improved classification of VHR images of urban areas usingdirectional morphological profiles,” IEEE Trans. Geosci. Remote Sens.,vol. 46, no. 10, pp. 2803–2813, Oct. 2008.

[12] P. Soille and M. Pesaresi, “Advances in mathematical morphology appliedto geoscience and remote sensing,” IEEE Trans. Geosci. Remote Sens.,vol. 40, no. 9, pp. 2042–2055, Sep. 2002.

[13] M. Dalla Mura, J. A. Benediktsson, F. Bovolo, and L. Bruzzone, “An un-supervised technique based on morphological filters for change detectionin very high resolution images,” IEEE Geosci. Remote Sens. Lett., vol. 5,no. 3, pp. 433–437, Jul. 2008.

[14] M. Dalla Mura, J. A. Benediktsson, B. Waske, and L. Bruzzone,“Morphological attribute filters for the analysis of very high resolu-tion images,” in Proc. IEEE IGARSS, Cape Town, South Africa, Jul.12–19, 2009, vol. III, pp. 97–100.

[15] M. Dalla Mura, J. A. Benediktsson, and L. Bruzzone, “Modeling struc-tural information for building extraction with morphological attributefilters,” in Proc. SPIE Eur. Remote Sens., Berlin, Germany, Aug. 31–Sep. 3, 2009, p. 747 703.

[16] E. J. Breen and R. Jones, “Attribute openings, thinnings and granulome-tries,” Comput.Vis. Image Underst., vol. 64, no. 3, pp. 377–389, Nov. 1996.

[17] G. Matheron, Random Sets and Integral Geometry. Hoboken, NJ: Wiley,1975.

[18] J. Serra, “Connectivity on complete lattices,” J. Math. Imaging Vis., vol. 9,no. 3, pp. 231–251, Nov. 1998.

[19] P. Salembier, A. Oliveras, and L. Garrido, “Anti-extensive connectedoperators for image and sequence processing,” IEEE Trans. ImageProcess., vol. 7, no. 4, pp. 555–570, Apr. 1998.

[20] E. R. Urbach and M. H. F. Wilkinson, “Shape-only granulometries andgrey-scale shape filters,” in Proc. ISMM, 2002, pp. 305–314.

[21] E. R. Urbach, J. B. T. M. Roerdink, and M. H. F. Wilkinson, “Connectedshape-size pattern spectra for rotation and scale-invariant classificationof gray-scale images,” IEEE Trans. Pattern. Anal. Mach. Intell., vol. 29,no. 2, pp. 272–285, Feb. 2007.

[22] M. K. Hu, “Visual pattern recognition by moment invariants,” IRE Trans.Inf. Theory, vol. IT-8, no. 2, pp. 179–187, Feb. 1962.

[23] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed.Englewood Cliffs, NJ: Prentice-Hall, 2008.

[24] L. Vincent, “Morphological grayscale reconstruction in image analysis:Applications and efficient algorithms,” IEEE Trans. Image Process.,vol. 2, no. 2, pp. 176–201, Apr. 1993.

[25] A. Meijster and M. H. F. Wilkinson, “A comparison of algorithms forconnected set openings and closings,” IEEE Trans. Pattern Anal. Mach.Intell., vol. 24, no. 4, pp. 484–494, Apr. 2002.

[26] M. H. F. Wilkinson, H. Gao, W. H. Hesselink, J. E. Jonker, andA. Meijster, “Concurrent computation of attribute filters on sharedmemory parallel machines,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 30, no. 10, pp. 1800–1813, Oct. 2008.

[27] L. Bruzzone and C. Persello, “A novel protocol for accuracy assessmentin classification of very high resolution multispectral and SAR images,”in Proc. IGARSS, 2008, vol. 2, pp. 265–268.

[28] C. Persello and L. Bruzzone, “A novel protocol for accuracy assessmentin classification of very high resolution images,” IEEE Trans. Geosci.Remote Sens., vol. 48, no. 3, pp. 1232–1244, Mar. 2010.

[29] L. Breiman, “Random forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32,Oct. 2001.

[30] R. G. Congalton and K. Green, Assessing the Accuracy of RemotelySensed Data: Principles and Practices, 2nd ed. Boca Raton, FL: CRCPress, 2008.


Mauro Dalla Mura (S’08) received the B.S.(Laurea) and M.S. (Laurea Specialistica) degrees intelecommunication engineering from the Universityof Trento, Trento, Italy, in 2005 and 2007, respec-tively. He is currently working toward the Ph.D.jointly between the University of Trento and theUniversity of Iceland, Reykjavik, Iceland.

Since 2007, he has been with the Remote SensingGroup at the Department of Information Engineeringand Computer Science, University of Trento andwith the Faculty of Electrical and Computer Engi-

neering, University of Iceland, Reykjavik, Iceland. His main research activitiesare in the fields of remote sensing, image processing, and pattern recognition.In particular, his interests include mathematical morphology, classification, andchange detection.

Mr. Dalla Mura is a Reviewer for the IEEE TRANSACTIONS ON GEO-SCIENCE AND REMOTE SENSING and the Canadian Journal of RemoteSensing.

Jón Atli Benediktsson (S’84–M’90–SM’99–F’04)received the Cand.Sci. degree in electrical engi-neering from the University of Iceland, Reykjavik,Iceland, in 1984, and the M.S.E.E. and Ph.D. degreesfrom Purdue University, West Lafayette, IN, in 1987and 1990, respectively.

He is currently Pro Rector for academic affairs anda Professor of electrical and computer engineeringwith the University of Iceland. He has held visitingpositions with the Department of Information andCommunication Technology, University of Trento,

Trento, Italy, since 2002; the School of Computing and Information Systems,Kingston University, Kingston upon Thames, U.K., during 1999 to 2004; theJoint Research Centre of the European Commission, Ispra, Italy, in 1998;the Technical University of Denmark, Lyngby, Denmark, in 1998; and theSchool of Electrical and Computer Engineering, Purdue University, in 1995.In August 1997, he was a Fellow with the Australian Defence Force Academy,Canberra, A.C.T., Australia. From 1999 to 2004, he was the Chairman of theenergy company Metan Ltd. He was also the Chairman of the University ofIceland’s Science and Research Committee from 1999 to 2005. Since 2006,he has been the Chairman of the University of Iceland’s Quality AssuranceCommittee. He coedited (with Prof. D. A. Landgrebe) a Special Issue onData Fusion of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE

SENSING (TGRS) (May 1999) and (with P. Gamba and G. G. Wilkinson)a Special Issue on Urban Remote Sensing from Satellite (October 2003).His research interests include remote sensing, pattern recognition, machinelearning, image processing, biomedical engineering, and signal processing, andhe has published extensively in those fields.

Dr. Benediktsson is a member of Societas Scinetiarum Islandica and TauBeta Pi. He was the Past Editor and is currently Associate Editor for TGRSand Associate Editor for the IEEE GEOSCIENCE AND REMOTE SENSING

LETTERS. From 1999 to 2002, prior to being Editor, he was Associate Editorfor TGRS. From 1996 to 1999, he was the Chairman of the IEEE Geoscienceand Remote Sensing Society (GRSS) Technical Committee on Data Fusion andwas elected to the Administrative Committee of the GRSS for the terms 2000 to2002 and 2008 to 2010. He is currently, the Executive Vice President of GRSS.In 2002, he was appointed as Vice President of Technical Activities of GRSS,and in 2008, he was appointed as Vice President of Professional Activities ofGRSS. He was the founding Chairman of the IEEE Iceland Section and servedas its Chairman from 2000 to 2003. He was a member of a NATO AdvisoryPanel of the Physical and Engineering Science and Technology Subprogram(2002–2003). He was a member of Iceland’s Science and Technology Council(2003–2006) and a member of the Nordic Research Policy Council (2004). Hereceived the Stevan J. Kristof Award from Purdue University, in 1991, as anoutstanding graduate student in remote sensing. In 1997, he was the recipientof the Icelandic Research Council’s Outstanding Young Researcher Award. In2000, he was granted the IEEE Third Millennium Medal. In 2004, he was acorecipient of the University of Iceland’s Technology Innovation Award. In2006, he received the yearly research award from the Engineering ResearchInstitute, University of Iceland. In 2007, he received the Outstanding ServiceAward from the IEEE GRSS.

Björn Waske (S’06–M’08) received the M.Sc. de-gree in applied environmental sciences with a majorin remote sensing from the University of Trier, Trier,Germany, in 2002 and the Ph.D. degree from theUniversity of Bonn, Bonn, Germany, in 2007.

From 2004 to 2007, he was with the Center ofRemote Sensing of Land Surfaces (ZFL), Universityof Bonn and visited the Department of Electricaland Computer Engineering, University of Iceland,Reykjavik, Iceland for three months in 2006. Untilmid-2004, he was a Research Assistant with the

Department of Geosciences, Munich University, Munich, Germany, where heworked on the use of remote sensing data for flood forecast modeling. SinceSeptember 2009, he has been an Assistant Professor at the Institute of Geodesyand Geoinformation, Faculty of Agriculture, University of Bonn. His currentresearch interests include advanced concepts for image classification and datafusion, focusing on multisensor applications.

Dr. Waske is a Reviewer for the IEEE TRANSACTIONS ON GEOSCIENCE

AND REMOTE SENSING and the IEEE GEOSCIENCE AND REMOTE SENSING

LETTERS.

Lorenzo Bruzzone (S’95–M’98–SM’03–F’10) re-ceived the M.S. (Laurea) degree in electronic engi-neering (summa cum laude) and the Ph.D. degree intelecommunications from the University of Genoa,Genoa, Italy, in 1993 and 1998, respectively.

He his currently a Full Professor of telecommu-nications at the University of Trento, Trento, Italy,where he teaches remote sensing, pattern recogni-tion, radar, and electrical communications. He is theHead of the Remote Sensing Laboratory in the De-partment of Information Engineering and Computer

Science, University of Trento. His current research interests are in the areasof remote sensing, signal processing, and pattern recognition (analysis of mul-titemporal images; feature extraction and selection; classification, regression,and estimation; data fusion; and machine learning). In 2008, was appointed asa member of the joint NASA/ESA Science Definition Team for Outer PlanetFlagship Missions. He is the author (or coauthor) of 90 scientific publicationsin referred international journals (61 in IEEE journals), more than 140 papersin conference proceedings, and 13 book chapters. He is the Editor/Co-editorof 10 books/conference proceedings. He is a Referee for many internationaljournals and has served on the Scientific Committees of several internationalconferences. He conducts and supervises research on these topics within theframeworks of several national and international projects.

Dr. Bruzzone is a member of the Managing Committee of the Italian Inter-University Consortium on Telecommunications and a member of the ScientificCommittee of the India–Italy Center for Advanced Research. He is also a mem-ber of the International Association for Pattern Recognition and of the ItalianAssociation for Remote Sensing (AIT). Since 2009, he has been a member ofthe Administrative Committee of the IEEE Geoscience and Remote SensingSociety. He was the General Chair and Co-chair of the First and Second IEEEInternational Workshop on the Analysis of Multi-temporal Remote-SensingImages (MultiTemp), and is currently a member of the Permanent SteeringCommittee of this series of workshops. Since 2003, he has been the Chair of theSPIE Conference on Image and Signal Processing for Remote Sensing. From2004 to 2006, he served as an Associated Editor of the IEEE GEOSCIENCE

AND REMOTE SENSING LETTERS, and currently is an Associate Editor forthe IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (TGRS)and the Canadian Journal of Remote Sensing. He ranked first place in theStudent Prize Paper Competition of the 1998 IEEE International Geoscienceand Remote Sensing Symposium (Seattle, July 1998). He was a recipient of theRecognition of IEEE TGRS Best Reviewers in 1999 and was a Guest Co-Editorof different Special Issues of the IEEE TGRS.

Documents

Morphological Attribute Proﬁles for the Analysis of Very ...€¦ · Recently, the standard morphological operators of opening, closing, white, and black top hat, along with opening