IME: an image management environment with content-based access

Andrea F. Abate* , Michele Nappi, Genny Tortora, Maurizio Tucci

Dipartimento di Matematica e Informatica, University of Salerno, via S. Allende-I-84081 Baronissi, Salerno, Italy

Received 26 March 1998; received in revised form 17 November 1998; accepted 26 November 1998

Abstract

The article describes an experimental visual environment to handle digital images by contents. A suitable spatial index is used to organizethe images in a spatial access structure for efficient storage and retrieval. An image is indexed according to both the spatial arrangement of itsobjects and the morphological and geometrical measures of these objects. Therefore, in the database population phase a user identifies theobjects that characterize the visual content of each image by a user-friendly interface. In order to let the system retrieve images based on thepresence of given patterns, it is necessary to define similarity matching criteria between a query and an image. To efficiently perform such amatch, each image is stored together with a collection of metadata that are a very compact representation of the visual contents of the image.These metadata form the index of the image. The system implements a Spatial Access Method based on k-d-trees to achieve a significantspeedup over sequential search.

We prove the effectiveness and the efficiency of the system by performing standard tests on a database containing a large number ofmedical images, namely lung CT scans.q 1999 Elsevier Science B.V. All rights reserved.

Keywords:Spatial index; Content based retrieval; Spatial access method

1. Introduction

In many database applications the usability and under-standability of a system is considerably enhanced if theinformation is represented in multimedia forms, such asimages, audio and video annotations. This is especiallytrue for applications where multimedia objects naturallyplay a central role. In particular, the usability of a largeimage database system can be considerably improved ifnon-expert users are provided with an intuitive and simplevisual interface that allows them to manipulate the imageseasily. Typical handling of images include image processing(e.g. edge detection and segmentation), image storing andretrieval. Moreover, several applications of image databasesrequire content-based access, but a raster image is toocomplex and too large to be suitably described by traditionalindexing methods, based on user-supplied alphanumericdescriptors. Indeed, Picture Archiving and CommunicationSystems (PACS) have become a basic research topic incomputer science. In fact, to properly support advancedapplications, new techniques still need to be developed toprovide timely delivery and easy access to images, video,text and associated information, using appropriate represen-tations for the intended tasks [1–5].

To index an image we can associate it with two kinds ofdescriptors: explicit information about its contents (intextual form), and implicit information related to the spatialarrangement of its pictorial elements and their morphologi-cal and geometrical measures. Then picture icons can beused as picture indexes, following an iconic indexing meth-odology [6–10]. With this Query-by-Pictorial-Exampleapproach [2,3], the index of an image is an iconic imageitself, which represents the visual information contained inthe image in a suitable form for different levels of abstrac-tion and management. An image database system shouldthen integrate an image management technique to accessinformation uniformly and consistently according to apowerful database model, and a spatial image indexingmethodology that allows efficient content-based access. Itshould also offer a user-friendly visual interface to helpusers in storing and retrieving images by means of a visualbrowser that easily navigates the database.

This article illustrates an experimental system, the ImageManagement Environment (IME), for handling images inlarge databases with approximate search by contents. IMEconsists of an integrated visual environment for loadingdigitized images, processing them to extract suitable spatialindexes, and storing and retrieving such images easilythanks to a user-friendly visual interface. The systemapplies an image indexing methodology, based onvirtualimages [9], that provides content-based access and

Image and Vision Computing 17 (1999) 967–980

0262-8856/99/$ - see front matterq 1999 Elsevier Science B.V. All rights reserved.PII: S0262-8856(98)00186-3

* Corresponding author.E-mail address:andaba@dia.unisa.it (A.F. Abate)

www.elsevier.com/locate/imavis

similarity retrieval in a very efficient and effective way. Asynthetic review of virtual images is given in Section 4.

To make the management of an image database systemeffective and efficient, the IME system includes four funda-mental issues in an integrated graphical environment:

• image feature extraction by image processing (e.g. edgedetection and segmentation);

• image content representation and indexing;• storage and retrieval methodologies for efficient spatial

access to images by exact and/or inexact match;• user-friendly interface (graphical environments, visual

query languages, etc.).

It is worth noting that most of the existing (prototype andcommercial) systems [11–15] address only a subset of theprevious requirements, as discussed later. A user-friendlyvisual interface lets users issue store–retrieve commandsby simply editing real and iconic images by means of apalette of image processing tools and a palette of graphicalobjects. To achieve an efficient access to images, it is impor-tant to have a Spatial Access Method (SAM) that organizesa large collection of image descriptions in a data structurethat allows faster searching than sequential. The data struc-ture used in IME to manage the index for images is based onk-d trees [1,16]. In Section 2 several related content-basedindexing approaches are discussed.

The article also describes the basic techniques used toimplement the IME system, and an experimental applicationof IME to a real-world domain, namely medical images.This is a typical homogeneous image database system,since all the diagnostic images of a given human bodydistrict have approximately the same contents. As a casestudy, we have implemented a diagnostic database ofComputed Tomography (CT) images and evaluated itsperformances in terms of efficiency and effectiveness. Inour tests, the efficiency is evaluated in terms of mediumresponse time for retrieval, while effectiveness considersthe quantity of false alarms and false dismissals.

2. Related work on image database systems

The existing systems that were proposed so far can begrouped into two classes:

1. The first class includes all the systems in which imagecontent description is given in terms of the properties(e.g. area, perimeter, roundness, symmetry, etc.) of theobjects contained in the images (these objects are identi-fied using on line or off line classical image segmentationtechniques) and in terms of the spatial relationshipsbetween such objects. These methods are usuallyexploited for the design of homogeneous IDBs (e.g.medical Image Databases), relying on the assumptionthat a fixed number of objects are common to all images(e.g. lung, spine, aorta, etc.) in addition to a variablenumber – usually smaller than the previous one – of

objects (e.g. a nodular suspect cancer) that characterizeeach image in terms of spatial relationships with respectto other objects. Query by example or by sketch aretypically supported (find all CT images that have anodule in left lung, vertically included between spineand aorta).

2. The second class includes methods in which imagecontent is given using global descriptions, such as colors,shapes and textures. These methods are adopted, usually,for designing heterogeneous IDBs, for which theprevious assumption might not be true. A user canquery the system by examples, clicking for instanceone of the displayed images to ask for similar ones orspecifying the color distribution (e.g. 20% red, 43% blueand 37% green), one or more textures and one or moreshapes. Obviously for homogeneous IDBs the latterproposals are totally inadequate. For example medicalimages, such as CT images of lungs district, haveapproximately all the same textures and colordistribution.

As already stated, the design and implementation of aneffective and efficient image database system requires in anycase to address four fundamental issues: image featureextraction, image content representation and indexing, fastsearch and retrieval (exact or inexact match), user interfacedesign. In the following we briefly discuss the main featuresof some of the most relevant content-based retrievalsystems.

Addressing the design and development of a standardframework for user-friendly interface requires an integra-tion between database and human computer interactionresearches. The interest of the researchers in the area ofdatabase systems has recently focused on developinggraphical user interfaces. This interest is especially impor-tant from an IDB perspective, for the obvious reason that theuse of graphical tools is an intrinsic property of these kindsof systems. However, advanced visual interfaces are oftenused for displaying information rather than for providing afriendly browsing [17].

Among the proposals that provide a friendly browsing, in[11] an interesting technique was introduced using a seman-tic color language. The user can perform visual queries andvisual refinements to choose the images to be discarded andthe ones to keep. Besides, in [18] an extension of 2-D stringwas proposed to deal with 3D image scenes.

Finally, in [19] an experimental multiparadigmatic visualinterface was introduced, called Visual Query and ResultHypercube (VQRH), which supports progressive queries.Users are gradually introduced to the database content, thenavigation technique and the visual reasoning strategy, aswell as the query process. The VQRH system was tested onmedical databases and library databases.

Although the systems described earlier provide the userwith a friendly graphical interface, they do not solve searchand retrieval in an efficient way, often suffering of a high

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980968

complexity, and do not offer a SAM to manage imageindexes.

2.1. Homogeneous IDB systems

In [15,20] an indexing methodology is proposed, basedon an extension of 2-D strings [6] to treat CT scans andmagnetic resonance imaging. Indexing and retrieval ofimages are based on spatial relationships and morphologi-cal/geometrical properties of objects. Access to images hasa low complexity, but it is limited to an exact match. Thissystem does not issue the design of a user friendly interface.It should be noted that 2-D strings were also used for inexactmatch retrieval, although the proposed techniques have aless efficient computing time [8]. In addition, the useof 2-D strings often leads to many false alarms and falsedismissals.

In [21] an indexing methodology is designed and imple-mented, based on a representation of image content byAttributed Relation Graphs (ARGs). The system firstextracts the objects from images by a segmentation process,and then organizes them in ARGs. Finally, the ARGs arerepresented by an R-tree structure. The problem of retriev-ing images is treated as a subgraph isomorphism problem,whose solution has a high time complexity (e.g. not poly-nomial). In contrast, the R-tree approach is very efficientwhen the query and the images have the same number ofobjects. Also this system does not issue the design of a userfriendly interface. A similar indexing method was devel-oped in [22]. The representation of image contents isbased on spatial relations, which are encoded in graph struc-tures and then stored in a hash table. This hash table is usedto perform similarity retrieval efficiently. However, an inte-grated environment implementing the latter technique is notcurrently available.

In [23] an indexing technique based on elastic matchingof sketched templates is illustrated. The method providesscale invariance and takes into account spatial relationshipsbetween objects in multi-object queries. The experimentalresults have demonstrated the capability to minimize falsealarms and false dismissals. It does not adopt a SAM tospeed up the search and does not provide a user friendlyinterface.

In [24] a technique based on global signature describingthe texture, shape and color content is presented. Thegeneral idea is that the system first computes the occur-rences of the above low level features of an image, andthen make a histogram. A normalized distance betweenprobability density functions of histograms is adopted tomatch signatures. The Comparison Algorithm for Navigat-ing Digital Image Database (CANDID) system was experi-mented on a database containing a total of 152 lung images.The obtained results are interesting, even if the technique issimilar to the QBIC approach and the similarity measureseems to suffer of computational inefficiency. Like thepreviously discussed systems, CANDID does not adopt a

SAM to speed up the search and does not provide a user-friendly interface.

A different approach applied to a multimedia medicaldistributed database system, called MEd, is proposed in[25]. The authors developed a semantic, spatial, evolution-ary data model to allow images to be retrieved by featuresand contents. The whole process of storing and querying isknowledge based. MEd is supported by a predefined PACSformat, in which all the image and textual information iscontained.

2.2. Heterogeneous IDB systems

In [13] an indexing methodology is proposed, which usesthe image content, such as color, texture, and shape, as thebasis of the queries. It introduces a SAM based on the R*-tree and does not focus on the design of a user-friendlyinterface. These articles present a part of the Query byImage Content (QBICe) project [14], whose techniquesare used in some commercial products (IBM’s MultimediaManager, IBM’s Digital Library, and the DB2 series ofproducts). The same technology for content-based retrievalwas adopted for the commercial product ILLUSTRAe.

The indexing method presented in [26] is based on thefractal framework of the Iterated Function Systems (IFS)[27]. The image feature it uses is the contractive functionfrequency, adopting the query-by-sketch and query-by-example paradigms. Each contractive function frequencyrepresents one component in ann-dimensional space.Therefore, each image is associated with ann-dimensionalpoint. The images are then organized in the R*-tree, whichallows the system to perform fast access. The system doesnot issue the design of a user-friendly interface.

3. The image management environment

The Image Management Environment (IME) system is anexperimental environment for the management of imageswith content-based access. It supports any applicationdomain where the spatial relationships among objects arethe primary keys to index the images. The current prototypeimplementation of the IME system deals with medicalimages. To achieve user-friendliness, it incorporates avisual interface that merges the query-by-pictorial-exampleand query-by-sketch paradigms. The user interface wasconceived as a typical Windows application. A paletteincludes a set of anatomic objects – e.g. right lung, leftlung, spine, aorta, nodule, etc. – for each specific humanbody district. Each object in the palette is taken from astandard CT scan and is iconically represented by meansof its contour.

In imaging diagnosis, segmentation and edge detectionare very helpful for physicians [28,29], since some featuressuch as spatial locations, opacities, shapes and geometricalmeasures can be automatically produced. In order to obtainsuch features, many segmentation and edge detection

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980 969

techniques are available [28,29]. In contrast, several appli-cations require manual annotation or at least an expertcontrol over the automatic feature extraction. For example,in clinical radiology, a physician is faced with the task ofdetermining the presence or absence of disorders in imagesfrom different modalities, such as CT scan, when the correctanswer is unknown. In particular, for a given image a physi-cian has to identify abnormalities, consider their spatialrelationships with certain organs, and evaluate theirmorphological and geometrical features like opacity,shape, symmetry, roundness, area, etc. In order words, aphysician has to evaluate the semantic contents of theimage based on his/her personal knowledge to formulate adiagnosis and a treatment plan for the patient. The personalknowledge of a physician is typically based on rememberingsimilar features of previously examined patients. In order toformulate a diagnosis, the physician has to perform threecomplex operations:

1. identify the hot spots;2. retrieve related images by similarity from his/her

mnemonic database (experience) or from a very largearchive of analogic films;

3. formulate a diagnosis.

Therefore, it would be helpful for him/her to have moresupport in accessing images and related data of patientswith similar abnormalities.

As an example, consider Fig. 1, in which a lung CT scanis illustrated. The circle focuses on a suspect cancer. Such apathology was diagnosed by the radiologist, considering its

morphological and geometrical features and its positionwith respect to other objects, e.g. lungs, spine, etc.

Unlike alphanumeric text, medical images usually requirea long time to be accessed, processed, stored, retrieved anddistributed, because of their size and internal complexity.Thus, an electronic database should meet the requirement ofeffectively handling the above time-consuming operations[30]. The image processing tools integrated in the IMEsystem allow hot spots and canonical objects extraction bymeans of an entropy based method for segmentation andedge detection [28]. Applying a such method to the lungCT scan of Fig. 1, the image of Fig. 2 is produced, whichincludes some canonical organs (left and right lungs, aortaand thorax) and a hot spot, pointed by an arrow.

Before storing the image in the database, it must beprocessed to extract the key features to be used for indexing.In this phase the visual interaction with the physician isparticularly useful to assign the correct meaning to thepatterns in the image and to select the significant ones.Contrarily, the physician can invoke several processingtools, in order to outline the features of interest in the origi-nal image. Fig. 3 shows a typical IME window where the CTscan of Fig. 1, namely theANC10.BMP subwindow, isloaded in the left-up corner. The results of its segmentationis shown in theANC10.SEG subwindow. Usually, thephysician is supposed to choose a subset of the edges tobe the representative ones. He/she has to add more informa-tion to each of them, such as which organs they represent,whether they are normal or not, which abnormalities he/shehas detected, and so on. Besides these descriptors, a virtualrepresentation of the image is automatically computed todescribe the spatial contents of the real image and toallow content-based similarity retrieval. All this collectedinformation is in fact theindex of the image. Fig. 3 alsoshows an IME subwindow, namely theQuery window,where a user is composing a query starting from a segmen-ted image and using the iconic palette of the environment.The latter contains some thorax CT scan iconic features,such as spine, left and right lung, etc. As previously stated,a record of alphanumeric descriptors are automaticallyextracted from the image, such as spatial relationships,morphological and geometrical measures of each hot spot.When considering a diagnostic image, a physician may wantto retrieve some similar images from the database andcompare them to the first one in order to be supported informulating his/her diagnosis. Therefore, he/she first asksfor edge detection from the image, and then selects theobjects by which he/she wants to formulate a query. In theexample, the query consists of spine and right lung with astar-shaped hot spot inside the aorta. All the images in thedatabase with a similar abnormality approximately in thesame position will be retrieved.

Fig. 4 shows the IME window that displays the results ofthe previous query, that is a set of retrieved images, and theirassociated similarity degrees with respect to the query.As expected, each image has a similar abnormality

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980970

Fig. 1. A lung CT scan: the circle focuses on a suspect cancer.

Fig. 2. Edge detection of the lung CT scan illustrated in Fig. 1.

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980 971

Fig. 3. A query by example.

Fig. 4. The set of images resulting from the query of Fig. 3.

approximately in the same location as the image of Fig. 1, sothat the similarity degrees are all greater than 90%. It isworth noting that no false alarms were retrieved, i.e., imagesthat do not show any abnormalities or images with abnor-malities in different positions (e.g. within the left lung).Besides comparing their contents with those of the originalone in order to formulate a diagnosis, the retrieved imagescan in turn be used to formulate new queries.

In the next section we will concentrate our attention onthe relationships between the visual contents of a query (itssemantics) and the iconic index of the retrieved images.

4. Iconic indexing of images

One of the most important problems to be considered inthe design of image database systems is how images arestored in the image database to model and access pictorialdata [3,31]. An image can be associated with two kinds ofdescriptors: explicit information about its content (in textualform), and implicit information related to the shape and tothe spatial arrangement of its pictorial elements. To make animage database flexible, the spatial knowledge embedded inimages should be preserved by the data structures used tostore them [7]. The use of picture icons as picture indexes isthe basic issue of the iconic indexing methodologies devel-oped [10]. Chang et al. [3,30,32] observe that the spatialcontents of an image can be suitably represented bymeans of the 13 types of spatial relationships shown inFig. 5.

The iconic index we associate with images is a virtual

description calledvirtual image, as defined in [9]. We give ashort description of virtual images in this section. The readeris referred to the cited reference for further details.

The virtual image of a real image consists of a set ofobjects and a set of binary spatial relations over the objects.To represent the 13 different types of spatial relations ineach dimension, we use the set of spatial operators {‘, ’,‘ u’, ‘ � ’, ‘[’,‘]’, ‘%’, ‘/’} defined in Table 1 [8] in terms ofthe beginning and ending points of the objects involved.

The spatial information embedded in a real image isrepresented in a relational format and the spatial relationsamong its objects are explicitly expressed. Similarapproaches can be found in [13,22], where spatial relationsare represented through graph structures and then indexedon R*-trees or hash tables. The interest of using virtualimages as an iconic index is essentially motivated by thepossibility of exploiting traditional database techniques forthe solution of pictorial queries. Moreover, virtual imagesare well-suited for defining a similarity measure betweenimages, used for the similarity retrieval with inexact match-ing, that is flexible and easy to compute efficiently.Although in principle every kind of binary relation can beused, in the following we restrict ourselves to the set ofspatial relations defined in Table 1.

Definition 1. Given a real imageim, thevirtual image imvi

associated withim is a pair (Ob, Rel) where:

Ob � {ob1,…, obn} is a set of objects ofimRel � (Relx, Rely) is a pair of sets of binary spatial

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980972

Fig. 5. The 13 types of spatial relations in one dimension (horizontal projection).

Table 1The definition of 2D G-string spatial operators

Notation Meaning Condition

A , B A disjoin B End(A) , begin (B)A � B A is the same as B Begin(A)� begin(B); end(A)� end(B)AuB A is edge to edge with B End(A)� begin(B)A%B A and B have not the same bounds and A contains B Begin(A), begin(B); end(A). end(B)A[B A and B have the same begin bound and A contains B Begin(A)� begin(B); end(A). end(B)A]B A and B have the same end bound and A contains B Begin(A), begin(B); end(A)� end(B)A/B A is partly overlapping with B Begin(A), begin(B), end(A), end(B)

relations over Ob where Relx (resp. Rely) containsthe mutually disjoint subsets of Ob× Ob thatexpress the relationships ‘, ’, ‘ u’, ‘ � ’, ‘[’,‘]’, ‘/’, ‘%’ holding between pairs of objects ofim alongthe x-projection (resp.y-projection).

For simplicity, we use the notation obigobw to indicatethat the pair (obi, obw) belongs to the relationg , where obi,obw [ Ob andg [ { , , u, � , [, ], /, %}. A triple likeobigobw is called anatomic relationin the following. Wealso say that the atomic relation obigobw belongsto Relx(resp. Rely) if the spatial relation holding between obi andobw along thex-projection (resp.y-projection) isg . Then wecan regard both Relx and Rely simply as sets of atomic rela-tions.

Example 1. Let us consider the picture of Fig. 6. Itsvirtual image is the pair (Ob, Rel), where

Ob � {bull, butterfly, cat};Rel � (Relx, Rely);Relx � {bull [butterfly,bull , butterfly,bull ,

cat,butterfly, cat,butterfly, butterfly};Rely � {bull[cat,bull , butterfly,cat, butterfly,butter-

fly[butterfly}.

Virtual images are also efficient means for retrieval andbrowsing. As a matter of fact, a query can be simply speci-

fied by drawing an iconic sketch from which a virtual imageis easily obtained. Given a query, the set of images satisfy-ing the condition expressed in the query is selected bycomputing the similarity degree between each image andthe query by means of their corresponding virtual images.Then, the query is solved by matching its virtual imageagainst those of the images in the database. In contrast,the virtual image associated with an image of the databasecan be easily transformed into an iconic sketch, which canbe efficiently used for a visual interface in place of the realimage for browsing.

Generally speaking, our own experience is that usersdiscriminate the images according to some basic features:the objects they contain, the shape of the objects, their colorand their spatial arrangement. In contrast, the accuracy of animage retrieval algorithm has to be based on both howsuccessful the system is in satisfying the user expectationsand how independent of the human interpretation the retrie-val method is.

To capture the fuzzy nature of spatial arrangements ofobjects, for the similarity degree between a queryQ and avirtual imageimvi we take into account the fact that pairs ofobjects ofQ possibly have different spatial relations inimvi,althoughQ and imvi have similar visual contents. Then, weassume that a value of similaritysim(g1,g2) [ [0,1] isdefined between each pair (g1, g2) of spatial relations(these values can be appropriately tuned for specific appli-cations). In our examples, thesim function is defined as inTable 2 [9]. The similarity degree between a queryQ and avirtual image imvi, denoted by Sim_deg(Q, imvi), iscomputed according to a formula that considers howmany objects ofQ are contained inimvi and how similarspatial relations are found inimvi with respect to those ofQ,and returns a value in the range [0,1].

Definition 2. A query on spatial relationships Qis a 4-tuple (F, G, Rel, t) where ({F < G}, Rel) is a virtual imageand t [ [0,1] is a similarity degree threshold.

In Definition 2, F is the set ofmandatoryobjects: animage im of the database satisfies the queryQ only if itsvirtual image imvi contains all the objects ofF, with thesame spatial relations as inQ. G is the set ofoptionalobjects: intuitively, the similarity degree betweenQ andim increases if some of the objects inG are contained inimvi. The last component,t, is a parameter that indicates theminimum required similarity degree betweenQ and im inorder forim to satisfyQ. When the candidate image does notsatisfy user’s requirements in terms of mandatory objectsand minimum similarity threshold, it is not retrieved; other-wise it belongs to the answer set with the correspondingsimilarity degree.

As an example, let us consider Fig. 7, which shows aquery where the user has specified the objects in the setF

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980 973

Fig. 6. A sample image containing four objects.

Table 2A possible definition for the functionsim

SIM , / [ ] % u �

, 1 0.4 0 0.5 0 0.7 0/ 0.4 1 0 0.5 0 0 0[ 0 0 1 0.7 0.8 0 0.5] 0.5 0.5 0.7 1 0.8 0 0.5% 0 0 0.8 0.8 1 0 0.6u 0.7 0 0 0 0 1 0� 0 0 0.5 0.5 0.6 0 1

(the bull and the leftmost butterfly) and the ones in the setG(the cat and the rightmost butterfly). Then, he/she specifiesthe minimum required similarity degreet (for example, lett� 0.75). The resulting query isQ� (F, G, Rel, t), with F�{bull, butterfly}, G� {cat, butterfly} andRel is the same asin Example 1.

Definition 3. Let Q � (F, G, Rel, t) be a query withF �{ q1,…, qn}, G � { qn11,…,qn1m} and Rel� (Relx, Rely) andlet imvi � (Obim, Relim) be the virtual image of an imageim.By RelxF

(resp., RelyF) we denote the subset ofRelx (resp.,

Rely) consisting of all the spatial relations between pairs ofobjects in F along thex-projection (y-projection, resp.).Then, the setsX and Y indicate the highest value of thesimilarity between each pair of atomic relations asfollows:

X � { �qgq0; s�uqgq0 [ �Relx 2 RelxF� and

s� Maxqg 0q0[Relimx

{sim�g;g 0�}} ;

Y � { �qgq0; s�uqgq0 [ �Rely 2 RelyF� and

s� Maxqg 0q0[Relimy

{sim�g;g 0�}} :

The setsX andY are computed to take into account thefact that the objects required in a queryQ could appear morethan once in a virtual imageimvi, possibly with differentspatial relationships. Then, the highest value of the similar-ity degree is chosen.

Example 2. Let us compute the setsX andY for the queryQ of Fig. 7 and the virtual imageimvi � (Obim, Relim)

corresponding to the image of Fig. 8, where:

Obim � {bull, butterfly, cat};Relim � (Relimx

, Relimy);

Relimx� {bull % butterfly, butterfly, bull, cat , bull,

cat, butterfly, butterfly, cat, butterfly, butter-fly};

Relimy� {bull % cat, bull , butterfly, cat, butterfly,

butterfly [butterfly}.

According to Definition 3, the RelxF, RelyF

, X, andY setsare:

RelxF� {bull [ butterfly};

RelyF� {bull , butterfly};

X � {(bull , butterfly, 0), (bull , cat, 0),(butterfly , cat, 1), (butterfly, butterfly, 1)};

Y � {(bull [ cat, 0.8), (cat, butterfly, 1), (butterfly[ butterfly, 1)}.

For example, the value associated with the atomic rela-tion bull [ cat of Q along they-direction, is 0.8, since theimage containsbull % cat and sim([, %)� 0.8 according toTable 2.

Once the most similar atomic relations were selected, it isnecessary to derive the similarity degree between the queryand the image in order to decide if the image belongs to theanswer set. Formally, the similarity degree is computedaccording to the following definition.

Definition 4. Let Q andimvi be defined as in Definition 3,then the similarity degree betweenQ and imvi, denoted bySim_deg(Q, imvi), is defined by the formula:

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980974

Fig. 7. A query specification.

Fig. 8. An image similar to the query of Fig. 7.

Sim_deg�Q; imvi� �uFu 1 uRelxF

u 1 uRelyFu 1 uG > Obimu 1

X�qgq0·s�[X

s1X

�qgq0·s�[Y

s

uFu 1 uGu 1 uRelxu 1 uRelyu

if �F # Obim� and�RelxF# Rxim

� and�RelyF# Ryim

� Sim_deg�Q; imvi� � 0 otherwise

More details about the notions ofquery and similaritydegreecan be found in [9].

As an example, let us compute the similarity degreebetween the query of Fig. 7 and the virtual image corre-sponding to the image of Fig. 10. We obtain

Sim_deg�Q; imvi� � 2 1 1 1 1 1 2 1 2 1 2:82 1 2 1 5 1 4

� 0:83:

SinceSim_deg(Q, imvi) $ t (the minimum required simi-larity degree), the image in Fig. 8 satisfies the user’s require-ments in terms of objects and spatial relationships.

5. Morphological–geometrical features

For a complete description of the visual content ofimages, additional information is often needed about themorphology of the objects they contain, besides their spatialrelationships. For example, an important aspect of diagnos-tic image analysis is the presence/absence of abnormalobjects with certain shape features. In this specific case-study, we consider only images containing one abnormality.Of course, the extension to more than one abnormality isstraightforward. Typical features used to identify an objectcontained in a picture include its spatial location, densityand geometrical measures. Such features are easilycomputed by means of well known statistics techniques.The features used in the implementation of the IME prototypeare briefly illustrated in the following. More detailed descrip-tions of these kinds of features and the methods available tocompute them can be found in the large traditional literatureon image processing (see, for example, [21]).

After the segmentation process, it is easy to obtain thecoordinates of the set P of the pixels that constitute the edgeof the object. Starting from them, we can consider the peri-meterp of the object under observation as the total numberof pixels of the edge. The coordinates (X, Y) of the centroidC (a sort of barycenter of the boundary) are the arithmeticaverage of all the coordinates in P:

X �

Px:�x;y�[p

x

uPu; Y �

Py:�x;y�[p

y

uPu:

The areaA of an object can be computed as the number ofthe pixels inside the boundary plus those in P. The valuesof A andp give us information not only about the dimensionof the object but also about its shape. Intuitively, an objectwith a little area and a large perimeter has a shape veryirregular. Therefore, we define theuniformity index Uofan object as the ratio between the square of its perimeterand its area:

U � p2

A:

The roundness of an object describes how much its shapeis similar to a circle. To measure the roundness of an objectwe use the variancer of the distances of the points of P from

the centroid. A value ofr close to 0 means that the object isnearly a circle.

Other features can be obtained considering moments ofhigher order. Consider, for example, the variablex thatdescribes the Euclidean distance of the points in P fromthe centroid C. The third momentSof that variable:

S� E x 2 Exÿ �3h i

gives us information about the symmetry of the object withrespect to C. A value ofS close to 0 means that the objecthas an high grade of symmetry [21].

Another aspect of interest is the symmetry with respect tothe Cartesian axes. The quantitiesSx and Sy (symmetryrespect to theX axes andY axes) can be measured usingquartiles [21]. They record the growth ofaccumulatedfrequencieson theX axis andY axis of points in P.

The last parameter we use to describe objects in ourapplication regards its texture. In fact, the representationof a CT scan is possible because a grey level is assignedto each point depending on the opacity in that point. If wecalculate the varianceD of the grey levels of an object on aCT scan, we obtain information about the homogeneity(density) of that object.

Summarizing, the seven features we consider in our fieldof application are:

SX symmetry w.r.t.X axis;SY symmetry w.r.tY axis;S symmetry;D density;r roundness;U uniformity;A area.

These features allow us to perform content-based image

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980 975

Fig. 9. An example of k-d-tree, wherek � 2.

Fig. 10. A k-d-tree storing binary values.

retrieval by shape. In particular, we associate a seven-dimensional shape vector with the abnormal object ofeach image. The distance, or similarity measure, betweentwo shape vectors is a weighted Euclidean distance definedas follows.

Definition 5. The weighted normalized Euclideandistancebetween two shape vectors

Svim � �SX:im; SY:im; S:im; D:im; r:im; U:im; A:im�and

SVq � �SX:q; SY:q; S:q; D:q; r:q; U:q; A:q�is given by

where, fori [ { Sx;SY;S;D; r;U;A} ; wi [ �0;1� is a weightthat reflects the importance of the corresponding feature(these weights are opportunely chosen so that

Pi wi � 1);

eachRi is the width of the range of possible values for theithfeature, so that the maximum distance between two shapevectors is 1 while the minimum is 0.

The introduction of the shape vector as a mean for simi-larity retrieval leads us to define an extended form of query.

Definition 6. An extended query Qis a 6-tuple (F, G, Rel,SV, t, 1) where ({F < G}, Rel) is a virtual images,SV is ashape vector,t [ [0,1] is a similarity degree threshold, and1 [ [0,1] is the maximum required shape vector distance.

In the rest of the article, we refer to an “extended query”simply as “query”. According to Definition 6, an imageimwill be included in the answer set of a given queryQ ifSim_deg(Q, imvi) is greater thant and the weighted normal-ized Euclidean distance between their shape vectors is lessthan1 . In the next section, we show how the virtual imageand shape vector are used as an index key in a SAM.

6. A spatial access method based on k-d-trees

The importance of efficient spatial information proces-sing has grown as the availability and use of spatial dataincreases. Spatial Access Methods manage a large collec-tion of k-dimensional points so that range queries can beanswered faster than sequential searching. In our case apoint in a k-dimensional space is the index of an imageand range query specifies a region in the address space.

The SAMs that were proposed so far can be grouped inthree classes [33,34]:

• Methods using rectangles as points in a higher dimen-sional space;

• Methods using linear quad-trees, or space filling curves;• Methods using trees.

An alternative approach can be found in [22], whichperforms a hashing of images based on the spatial relationsbetween objects and then uses a hash table to avoid sequen-tial search. In addition, several schemes are available toreduce the number of similarity measure calculationsbased on the triangle inequality [35].

One of the most interesting tree-based SAMs is the k-d-tree [1], which is the underlying indexing method we have

adopted in the actual implementation of the IME system.The k-d-tree can be seen as a variant of the BS-tree to storean ordered set of record, where at each non-leaf node adifferent field is tested to determine the direction in whicha branch is to be made. Therefore, the tree levels correspondto the record fields in a round way in order to partition thechildren of each node with respect to that field. In otherwords, the leveli corresponds to the fieldFi for i �1,2,…,k, while the levelk 1 1 corresponds toFl and soon. If the value of the fieldFi of a nodeN at level i isequal tox, then all its left descendants satisfy the conditionFi # x, while all its right descendants satisfy the conditionFi . x.

As an example, Fig. 9 shows a k-d-tree representing a setof pairs of integers, where each node has the tested compo-nent in bold and each arc is labelled with the correspondingcondition. For example, if we want to insert the pair (11,2),we have to follow the sequence of left/right branches below:

left since 11# 12,left since 2# 3,right since 11. 8.

As a second example, Fig. 10 shows a k-d-tree that storesa set of binary 4-tuple of the form (b1, b2,…,b4) wherebi [{0, 1} for i � 1,2,3,4.

The k-d-tree based method is more robust than classicalSAMs, such as R-trees, to handle two different kinds ofmultidimensional queries [36]:

• queries with one or more fields that may not be specified(partial match)

• queries with one or more fields specified by a rangerather than a single value.

In fact k-d-trees can handle queries like (3, [1–5], 4, ?, ?,6), where ? represents a dummy value and the interval [1–5]includes the numbers between 1 and 5. Therefore the

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980976

���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������wSX

SX:im 2 SX:qRSX

!2

1wSY

SY:im 2 SY:qRSY

!2

1wSS:im 2 S:q

RS

� �2

1wDD:im 2 D:q

RD

� �2

1wrr:im 2 r:q

Rr

!2

1wUU:im 2 U:q

RU

� �2

1wAA:im 2 A:q

RA

� �22

vuut

proposed SAM is very attractive because it can handlepartial matching efficiently, in terms of both time andspace [16]. To evaluate the time required to perform apartial matching, we have to take into account that a k-d-tree may not be balanced. However, several heuristic tech-niques are available to keep a k-d-tree balanced [36]. So weconsider only the case in which the tree is complete andcontainsn nodes. In such a case each path from the rootrequires the visit of log2n nodes at most. If we perform apartial matching in whichmfields are specified out of a totalk, then in exactly (k 2 m) nodes the search has to continue inboth subtrees so that the total number of visited leaves is atmost 2��k2m�=k�logn � n�k2m�=k

:

In the proposed index strategy each image is representedas point in ad-dimensional space, and then is stored in anode of a k-d-tree. The dimensiond of our tuples iscomputed as follows. Suppose we want to store a set ofimagesI � { im1, im2,…, imn}. Let Oimi

be the set of objectsin the imageimi and letO� Sn

i�1 Oimi: It is worth noting

that the set of the objects that an image can contain is ofteneasily determined by the application domain. For example,if we are dealing with lung CT scans then only the canonicalobjects and possibly one abnormality can appear in anyimage. Then, ifO� { o1, o2,…, oc}, we needc componentsto express the presence/absence of each object. In addition,we use spatial relationships as search keys. An imageimsatisfies a queryQ if their spatial relations verify the follow-ing inclusions:

RelXQ# RelXim

; �1a�

RelYQ# RelYim

�1b�Where RelXQ

(resp.,RelYQ) is the set of spatial relations

along thex-axis (resp.,y-axis) contained in the query, andRelXim

(resp.,RelYim) is the set of spatial relations along thex-

axis (resp.,y-axis) contained in the image. Proposition 1gives a necessary condition in order for (1a) and (1b) tobe satisfied.

Proposition 1. Let Q� (F, G, Rel, SV, t, 1) be a query,imvi � (Obim, Relim) a virtual image, Op the set of spatialrelations and num(g , Rel) the number of times that theoperatorg appears in Rel. A necessary condition in orderfor (1a) and (1b) to hold is:

num g;RelxQ

� �# num g;RelXim

� �;g [ Op; �2a�

num g;RelYQ

� �# num g;RelYim

� �;g [ Op: �2b�

Proof. Let (1a) be true, we prove that also (2a) holds.Suppose that there is an operatorg [ Op such thatnum�g;RelXQ

� . num�g; RelXim�: This implies thatRelXQ

contains at least one atomic relation that is not present inRelXim

; but this is absurd because of (1a). Similarly, (1b)implies (2b) and then the thesis.

We use the result of Proposition 1 to define the dimensionof the tuples. In fact, for each imageim we can store thenumber of atomic relations involving the spatial operatorg ,;g [ Op. The necessary condition that an image satisfies aquery if it contains at least the same number of relations isexploited to speed-up the search for solutions. Therefore, weneed one component for each spatial operator. In total, thedimensiond is obtained as the cardinality of the setO,namely c, plus the number of spatial relations with respectto both axes, namely 26, plus the dimension of the shapevector, namely 7. At this point, we can represent the set ofimageI by means of a k-d-tree whose nodes are vectors ofthe form (b1, b2,…, bd). The firstc components are defined asfollows:

bj � 1 iff oj [ Ovij [ �1; c�;

bj � 0 iff oj Ó Ovij [ �1; c�

wherebj denotes the presence (bj � 1) or the absence (bj �0) of the objectoj. The componentsbc11, bc12,…, bc126

denote the number of times that each spatial operatorgappears inRelim. The last 7 components represent theshape vector.

In our case study, we observed that a k-d-tree withk� dwould be strongly unbalanced, sinced < 40 and the size ofthe IDB is substantially smaller than 240 < 1012. Therefore,we decided to use only the 7 components of the shapevectors as index keys in the k-d-tree and set the dimensionk equal to 7. This choice is motivated by the considerationthat 7 is an appropriate value to keep the tree balanced, andthat queries on shape vector components are always speci-fied by a range of values rather than a precise value. Simu-lation studies have in fact demonstrated that this choiceleads to reasonably balanced k-d-trees.

Given a queryQ, our search strategy works in the follow-ing way. Letj be the shape vector component chosen as thekey for the first level of the k-d-tree.

1. First we check if the value of thejth component of theroot falls within the interval �j·Q 2 1�Rj =wj�; j·Q 11�Rj =wj��: The reader can easily verify that this is a neces-sary condition for the weighted normalized Euclideandistance between the query and the image to be lessthan1 .

2. If the condition above holds, we also check if the imagecontains the mandatory objects specified in the query.Observe that this can be verified by a simple logicaloperation on the binary components of the tuples.

3. Then we check if the necessary condition of Proposition1 is satisfied.

4. If the steps 1–3 succeed, the weighted normalized Eucli-dean distance and theSim_degbetween the query and the

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980 977

image are computed, and the image is retrieved accord-ing to the threshold parameterst and1 .

5. The search continues in the left subtree, right subtree orboth depending on the check of step 1, until the leaves ofthe k-d-tree are reached.

7. Performance evaluation

This section presents experimental results and perfor-mance evaluation obtained by testing the IME system on adatabase with a large number of lung CT scan imageschosen from digital archive of the Institute of Radiologyof the Second University of Naples. It should be notedthat such a set of images are relative to the same humanbody district, namely the lung district, and then all containapproximately the same objects in the same positions. Theonly relevant difference between two images is related to thepresence/absence of a similar abnormality, owing to severalpossible lung pathologies.

Frequently used evaluation criteria to measure effective-ness of retrieval system are therecall and theprecision[34]:

• the recall is the capability of system to retrieve all rele-vant images;

• the precisionis the capability of system to retrieve onlyrelevant images.

We have chosen an extension of the former, namely theNormalized Recall(NR), introduced in [34], to asses theperformances of IME. In particular, NR reflects how closeis the set of the retrieved images to an ideal retrieval, in

which the mostRELrelevant images appear in the firstRELpositions.

Formally, relevant images are ranked 1, 2,…, REL, whereREL is the number of relevant images, and Ideal Rank (IR)is given by

PRELr�1 r=REL: Now let

PRELr�1 Rankr =REL be the

Average Rank (AR) over the set of relevant imagesretrieved by the system, whereRanki represents the rankof relevant images. The difference between AR and IR,given byAR2 IR, represents a measure of the effectivenessof the system. This difference can range from 0, for theperfect retrieval (AR� IR), to (TOT2 REL), for the worstcase, whereTOT is the number of images in the collection.Hence we can normalize the above expression by dividingit by (TOT 2 REL) and then by subtracting the resultfrom 1. Finally, we obtain that the Normalized Recall(NR) as 12 �Ar 2 Ir =�TOT2 REL��: This measure rangesfrom 1 for the best case (perfect retrieval) to 0 for the worstcase.

In order to evaluate the performances of IME, withrespect to human perceptual similarity, we have selected asample of 15 heterogeneous images to be used as queries.For each of them, we first manually selected the 25 mostsimilar images from the database, that is a set of 25 imagescontaining a similar abnormality in the same position as inthe query. Then we contrasted them with the correspondingimages automatically retrieved. For each query, wecomputed the corresponding NR. We run experiments ona set of 2000 images, selected from an analogic medicalarchive, which had already been classified with respect tothe different pathologies. The results are shown in Table 3and demonstrate the effectiveness of IME. The value of NRis averaged over the 15 test queries.

In this test, we computed a NR value equal to 0.975,which is very close to 1. This reflects the fact that theranks of relevant images deviate very little in averagefrom the ideal case.

In order to achieve faster than sequential search, IMEuses k-d-trees as spatial access structures. To show howefficient this choice is, in terms of computing time, in the

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980978

Table 3Effectiveness of IME evaluated by normalized recall

Measure Value

Size of idb 2000Number of queries 15Normalized recall 0.975

Fig. 11. The response time as a function of the retrieved images set size.

following we illustrate the performance gains it offers withrespect to sequential scanning. We have run the IME proto-type system on a Pentium 100 in a medium configuration.

We have tested the IME system, considering an IDB of20 000 images and 50 typical queries randomly repeatedwith different values of the parameterst and 1 . Fig. 11plots the average response times of sequential and k-d-treesearches, as a function of the size of the retrieved set ofimages. Notice that the size of the retrieved set of imagesgrows ast decreases and1 increases. The figure shows anevident speed up to k-d-tree search over sequential search.Moreover the k-d-tree search never requires more than 20 s.

Another test considers the response time as a function ofthe IDB size. In this experiment we sett � 0.8 and issuedeach query with four different values for1 . Fig. 12 plots theaverage retrieval response times compared to sequentialsearch. Notice the k-d-tree achieves considerably bettercomputing time than sequential search and that the perfor-mance gap widens as the IDB grows.

8. Conclusions and future work

We have presented the IME system, a prototype environ-ment to manage image databases. IME is mainly composedof a user-friendly management environment for handlingimage storing and retrieval by content. We have also illu-strated the basic techniques developed to implement theimage processing tools, the iconic indexing of images inthe system and the SAM based on k-d-trees. The user-friendliness of the system is guaranteed by a visual browserthat follows the query-by-pictorial-example paradigm. Inorder to assess experimental results, we applied IME on amedical database including a large number of lung CTimages. We run several experiments to prove that our tech-nique is both efficient and reliable in terms of false alarmsand false dismissals. The IME system is currently beingused at the Radiology Institute of the Second Universityof Naples as an experimental application.

We are presently addressing the problem of implement-

ing our indexing methodology in real IDB applications. Tothis aim, additional effort will be devoted to integrate ourtechniques in an RDBMS to develop new applications. Theuse of an RDBMS allows a great amount of images and agreat number of users to be managed concurrently. We areplanning to use the Oraclee RDBMS because it is powerfuland fault tolerant, it uses the Web–Client-Server model fordistributed computations and supports various hardware/software platforms.

Acknowledgements

The authors express their appreciation of Prof. F.S. Sassoand the research staff he directs at the Radiology Institute ofthe Second University of Naples for their assistance in thechoice of the pathologies and images from the Institutearchive.

References

[1] L. Bentley, J.H. Friedman, Data structures for range searching, ACMComputing Survey 11 (4) (1979) 397–409.

[2] S.K. Chang, T.Y. Hou, A. Hsu, Smart image design for large imagedatabase, Journal of Visual Languages and Computing 3 (1992) 323–342.

[3] S.K. Chang, A. Hsu, Image information systems: where do we gofrom here? IEEE Transaction on Knowledge and Data Engineering4 (5) (1992) 431–442.

[4] A.D. Narasimhalu, Multimedia databases, Multimedia Systems 4(1996) 226–249.

[5] A. Schnep, D.H.C. Du, E. Russel Ritenour, Building future medicaleducation environment over ATM networks, Communications of theACM 38 (2) (1995) 54–69.

[6] S.K. Chang, Q.Y. Shi, C.W. Yan, Iconic indexing by 2-D string, IEEETrans. Pattern Anal. Machine Intell. 9 (3) (1987) 413–428.

[7] S.K. Chang, C.W. Yan, D.C. Dimitroff, T. Arndt, An intelligent imagedatabase system, IEEE Trans. Software Engineering 14 (1988) 681–688.

[8] S.Y. Lee, F.J. Hsu, 2D C-string: a new spatial knowledge representa-tion for image database systems, Pattern Recognition 23 (1990)1077–1087.

[9] G. Petraglia, M. Sebillo, M. Tucci, G. Tortora, A normalized index for

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980 979

Fig. 12. The average retrieval response time as a function of the IDB.

image databases, in: S.K. Chang, E. Jungert, G. Tortora (Eds.), Intel-ligent Image Database Systems, World Scientific Publishing, Singa-pore, 1996, pp. 43–69.

[10] L. Tanimoto, An iconic symbolic data structuring scheme, in: PatternRecognition and Artificial Intelligence, Academic Press, New York,1976, pp. 452–471.

[11] J.M. Corridoni, A. Del Bimbo, E. Vicario, Painting retrieval based oncolor semantics, Proceedings of First International Workshop on IDB-MMS, Amsterdam, The Netherlands, August 1996, pp. 9–16.

[12] M. De Marsico, L. Cinque, S. Levialdi, Indexing pictorial documentsby their content: a survey of current techniques, Image and VisionComputing 15 (1997) 119–141.

[13] C. Faloutsos, W. Equitz, M. Flickner, W. Niblack, D. Petkovic, R.Barber, Efficient and effective querying by image content, Journal ofIntelligent Inf. Systems 3 (3/4) (1994) 231–262.

[14] M. Flickner, et al., Query by image and video content: the QBICsystem, Special Issue on Content-Based Image Retrieval Systems,Computer 28 (9) (1995) 23–32.

[15] E.G.M. Petrakis, S.C. Orphanoudakis, Methodology for representa-tion, indexing and retrieval of images by content, Image and VisionComputing 11 (8) (1993) 504–521.

[16] J.D. Ullman, Principles of Database and Knowledge-based Systems,Computer Science Press, Rockville, Maryland, USA, 1988.

[17] K.J. Mitchell, J.B. Kennedy, P.J. Barclay, A framework for user-interface to databases, in: T. Catarci, M.F. Costabile, S. Levialdi,G. Santucci (Eds.), Proceedings of the 1996 Workshop onAdvanced Visual Interfaces, Gubbio, Italy, ACM Press, 1996, pp.81–90.

[18] A. Del Bimbo, M. Campanai, P. Nesi, A three-dimensional iconicenvironment for image database querying, IEEE Trans. Soft. Eng.19 (1993) 997–1011.

[19] S.K. Chang, E. Jungert, Symbolic Projection for Image InformationRetrieval and Spatial Reasoning, Signal Processing and its Applica-tions, Academic Press, New York, 1996.

[20] S. Kostomanolakis, M. Lourakis, C. Chronaki, Y. Kavaklis, S.C.Orphanoudakis, Indexing and retrieval by pictorial content in theI2C image database system, Technical Report, Institute of ComputerScience, Foundation for Research and Technology-HELLAS, Herak-lion. Crete, Greece, 1993.

[21] E.G.M. Petrakis, C. Faloutsos, Similarity searching in medical image

databases, IEEE Transactions on Knowledge and Data Engineering 9(3) (1997) 435–447.

[22] M.S. Costa, L.G. Shapiro, Scene analysis using appearance-basedmodels and relational indexing, Proceedings of the IEEE Symposiumon Computer Vision, 1995, pp. 103–108.

[23] A. Del Bimbo, P. Pala, Visual image retrieval by elastic matching ofuser sketches, IEEE Trans. Pattern Analysis and Machine Intelligence19 (2) (1997) 121–132.

[24] P.M. Kelly, T.M. Cannon, Query by image example: the CANDIDapproach, Proceedings of SPIE Storage and Retrieval for Image andVideo Database III, 1995, pp. 238–248.

[25] W.W. Chu, A. Cardenas, R.K. Taira, KmeD: a knowledge-basedmultimedia medical distributed database system, InformationSystems 20 (2) (1995) 75–96.

[26] M. Nappi, G. Polese, G. Tortora, FIRST: fractal indexing and retrievalsystem for image databases, Image and Vision Computing 16 (5)(1998) 677–693.

[27] M. Barnsley, Fractals Everywhere, Academic Press, New York, 1988.[28] C. Di Ruberto, M. Nappi, S. Vitulano, Different methods to segment

biomedical images, Pattern Recognition Letters 18 (11/13) (1998)1125–1131.

[29] M. Kobashi, L.G. Shapiro, Knowledge-based organ identificationfrom CT images, Pattern Recognition 28 (4) (1995) 475–491.

[30] E. Jungert, S.K. Chang, An image algebra for pictorial data manip-ulation, CVGIP: Image Understanding 58 (2) (1993) 147–160.

[31] H. Tamura, N. Yokoya, Image database systems: a survey, PatternRecognition 17 (1) (1984) 29–43.

[32] S.K. Chang, E. Jungert, Pictorial data management based upon thetheory of symbolic projections, Journal of Visual Languages andComputing 2 (3) (1991) 195–215.

[33] H. Samet, The Design and Analysis of Spatial Data Structures, Addi-son Wesley, 1989.

[34] G. Salton, M.J. McGill, Introduction to Modern Information Retrie-val, McGraw Hill, 1983.

[35] A.P. Berman, L.G. Shapiro, Efficient Image retrieval with multipledistance measures, Proceedings of the SPIE Conference on Storageand Retrieval for Image and Video Databases, February 1997.

[36] J.H. Friedman, J.L. Bentley, R.A. Finkel, An algorithm for findingbest matches in logarithmic expected time, ACM Trans. Mathemati-cal Software 3 (3) (1997) 209–226.

A.F. Abate et al. / Image and Vision Computing 17 (1999) 967–980980