6
IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES USING FUZZY SET THEORETIC EVALUATION M. Banerjee M. K. Kundu P. K. Das Machine Intelligence Unit Machine Intelligence Unit Dept. of Computer Science I.S.I I.S.I Jadavpur University 203, B. T. Road, Kolkata - 700 108 203, B. T. Road, Kolkata - 700 108 Jadavpur, Kolkata -700032 minakshi [email protected] [email protected] [email protected] Keywords: Content-based image retrieval, high curvature points, fuzzy feature evaluation index, color, invariant mo- ments. Abstract This paper proposes a new image retrieval scheme using visu- ally significant features. Clusters of points around significant curvature regions (high, medium, weak type) are extracted to obtain a representative image. Illumination, viewpoint in- variant color features are computed from those points for evaluating similarity between images. Relative importance of the features are evaluated using a fuzzy entropy based measure computed from relevant and irrelevant set of the retrieved images marked by the users. The performance of the system is tested using different set of examples from general purpose image database. Robustness of the system has also been shown when the images have undergone different transformations. 1 Introduction Effective image retrieval techniques from a large database is a difficult problem and still far being solved. Retrieval of relevant images, based on automatically derived imagery features (color, texture, shape etc.) is popularly known as content based image retrieval(CBIR). There are several popular CBIR techniques reported in the literature [19], [3], [5], [9]. The information in an image sometimes involve significant amount of reasoning about the meaning or the purpose of the objects or scenes depicted. As a result, it is still not possible to achieve desired accuracy from a fully automated CBIR system. The accuracy of a CBIR system may be improved by iterative process of refinement of queries and features, guided by users feedback [22] known as relevance feedback mechanism. Owing to these facts, derivation and selection of optimal set of features still remain a challenging issue in designing an efficient CBIR system. Human visual system is highly efficient in sorting and selecting similar images from a very large collection. In this process of selection, a limited number of visually prominent features may be used to evaluate similarity between images. High curvature points play a significant role in characterizing an object with limited number of pixels compared to the total number of pixels. Designing a system using such features for retrieval mechanism will be both fast and cost effective proposition. Image retrieval tasks based on visually significant points [10], [12] are reported in literature. In [11] local features ( color, texture etc. ) are computed on a window of regular geometrical shape surrounding the corner points. General purpose corner detectors [8] are also used for this purpose. However the curvature points may be of different types (sharp, medium, weak). The characteristics of sharp curvature points will be confined within a small region but for that of medium and weak type the region will be larger. These facts indicate that extracting the possible high curvature region of interest (roi) where shape and size of the extracted (roi) varies adaptively according to the nature of curvature type could be a better solution. It may be considered an alternative to segmenting an image and using it in overall scene matching applications. An efficient CBIR system should be able to handle imprecise image data, to some extent the difference arising due individual perception in evaluating similarity between images. A Fuzzy set theoretic based approach may be considered a good choice for handling uncertainties arising at different stages of process- ing and analysis of a CBIR system [4]. The proposed technique is based on the assumption that two visually similar images will have similar visual characteristics. Each feature has its individual significance where the importance of each feature may vary depending upon the query type and applications. Looking into these aspects, the basic contribution of the paper involves (i) A fuzzy set theoretic approach for extraction of clusters of different types of curvature points (sharp, medium, weak) whose centroid almost depicts a true corner. These points are considered as the candidate points for computation of features. (ii) The Invariant global moments of the extracted points set are source of similarity evaluation. Beside these features some global measurements have also been considered to improve the results further. (iii) A feature evaluation mecha- nism is provided to enhance the accuracy of the system further. The user marks the relevant images within the retrieved set. The individual feature weights are updated with a measure namely fuzzy feature evaluation index(FEI) [15] computed from ’intra set ambiguity’ and the ’interset ambiguity’ of the relevant and irrelevant set of images. The results of the proposed methodology is compared with that of some well known techniques (a) integrated region based approach [21], [4] (b) color histogram method [17]. The organization of the paper is as follows, The proposed methodology and results are

IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURESUSING FUZZY SET THEORETIC EVALUATION

M. Banerjee M. K. Kundu P. K. DasMachineIntelligenceUnit MachineIntelligenceUnit Dept. of ComputerScience

I.S.I I.S.I Jadavpur University203, B. T. Road,Kolkata - 700 108 203, B. T. Road,Kolkata - 700 108 Jadavpur, Kolkata -700032

[email protected] [email protected] [email protected]

Keywords: Content-basedimage retrieval, high curvaturepoints, fuzzy feature evaluation index, color, invariant mo-ments.

Abstract

This paperproposesa new imageretrieval schemeusingvisu-ally significantfeatures.Clustersof points aroundsignificantcurvature regions (high, medium, weak type) are extractedto obtain a representative image. Illumination, viewpoint in-variant color featuresare computedfrom those points forevaluatingsimilarity betweenimages.Relative importanceofthefeaturesareevaluatedusinga fuzzy entropy basedmeasurecomputedfrom relevant and irrelevant set of the retrievedimagesmarked by the users.The performanceof the systemis testedusingdifferentsetof examplesfrom generalpurposeimagedatabase.Robustnessof thesystemhasalsobeenshownwhenthe imageshave undergonedifferent transformations.

1 Introduction

Effective image retrieval techniquesfrom a large databaseis a difficult problem and still far being solved. Retrievalof relevant images,basedon automaticallyderived imageryfeatures(color, texture, shapeetc.) is popularly known ascontentbasedimageretrieval(CBIR).ThereareseveralpopularCBIR techniquesreportedin the literature[19], [3], [5], [9].The information in an image sometimesinvolve significantamountof reasoningaboutthe meaningor the purposeof theobjectsor scenesdepicted.As a result, it is still not possibleto achieve desiredaccuracy from a fully automatedCBIRsystem.The accuracy of a CBIR systemmay be improvedby iterative processof refinementof queries and features,guidedby usersfeedback[22] known as relevancefeedbackmechanism.Owing to these facts, derivation and selectionof optimal set of featuresstill remaina challengingissueindesigningan efficient CBIR system.Humanvisualsystemis highly efficient in sortingandselectingsimilar imagesfrom a very largecollection.In this processofselection,a limited numberof visually prominentfeaturesmaybeusedto evaluatesimilarity betweenimages.High curvaturepointsplay a significantrole in characterizingan objectwithlimited number of pixels comparedto the total number ofpixels. Designinga systemusing such featuresfor retrieval

mechanismwill be both fast and cost effective proposition.Imageretrieval tasksbasedon visually significantpoints[10],[12] are reportedin literature. In [11] local features( color,textureetc.) arecomputedon a window of regulargeometricalshapesurroundingthe cornerpoints. Generalpurposecornerdetectors[8] are also used for this purpose.However thecurvature points may be of different types (sharp,medium,weak). The characteristicsof sharpcurvature points will beconfinedwithin a small region but for that of medium andweak type the region will be larger. Thesefactsindicatethatextracting the possiblehigh curvatureregion of interest(roi)whereshapeand size of the extracted(roi) variesadaptivelyaccordingto the natureof curvature type could be a bettersolution. It may be consideredan alternative to segmentingan imageandusing it in overall scenematchingapplications.An efficient CBIR systemshouldbe ableto handleimpreciseimagedata,to someextentthedifferencearisingdueindividualperceptionin evaluatingsimilarity betweenimages.A Fuzzysettheoreticbasedapproachmaybeconsidereda goodchoicefor handlinguncertaintiesarisingat differentstagesof process-ing andanalysisof aCBIR system[4]. Theproposedtechniqueis basedon the assumptionthat two visually similar imageswill have similar visual characteristics.Each featurehas itsindividual significancewhere the importanceof eachfeaturemay vary dependingupon the query type and applications.Looking into theseaspects,the basiccontribution of the paperinvolves (i) A fuzzy set theoreticapproachfor extraction ofclustersof differenttypesof curvaturepoints(sharp,medium,weak) whose centroid almost depicts a true corner. Thesepointsareconsideredas the candidatepoints for computationof features.(ii) The Invariantglobal momentsof the extractedpoints set are sourceof similarity evaluation. Beside thesefeaturessomeglobalmeasurementshave alsobeenconsideredto improvetheresultsfurther. (iii) A featureevaluationmecha-nismis providedto enhancetheaccuracy of thesystemfurther.The usermarks the relevant imageswithin the retrieved set.The individual featureweights are updatedwith a measurenamely fuzzy feature evaluation index(FEI) [15] computedfrom ’intra set ambiguity’ and the ’interset ambiguity’ ofthe relevant and irrelevant set of images.The resultsof theproposedmethodologyis comparedwith that of somewellknown techniques(a) integratedregion basedapproach[21],[4] (b) color histogrammethod[17]. The organizationof thepaperis asfollows,Theproposedmethodologyandresultsare

Page 2: IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

describedin section2. The paperis concludedin section3.

2 The proposed methodology

The feature extraction and feature evaluation methodsareexplainedin the following subsections.

2.1 Extraction of corner signature

The proposedtechniqueis basedon a work reportedin [2].The potential fuzzy corner regions are extracted using theTopographiccharacteristicsof intensity surfaces[18], [16].The uncertaintyarising in locating such points are handledusing fuzzy set theoreticapproach.The discontinuitiesin theintensity surfaces are the possiblecandidatesfor detectingcurvaturepoints.Thesepointsarecharacterizedwith gradientmembership(��������� ) generatedby a typefunctionshown inFig. 1(a). The assignmentof the membershipvalue is basedon the local gray level contrast[1].

���������� � ��������������� (1)

, where � is determinedfrom, ratio of contrastbetweentwooppositepixels (����� ) over a specifiedwindow shown in Fig.1(b). ��� =[ � �"! #%$'&(#�)*!� �"! +�$,&(+-)�! , � �"! +�$'&.+-)�!�/�0! #%$'&.#�)�! , � �"! 1�$,&.12)�!� �"!

�$'&

�)�! , � �"!

�$'&

�)�!� �"! 1�$,&.12)�! ]

����� 4365-7"8 � �*9 (2)

The parameters� and � are determinedfrom the maximumand minimum value of ����� , which maps the membershipbetween0.0 to 1.0. Two more memberships(��: , �

1 ) arecomputedto estimatethe strengthof connectednesson bothsides of the curvature junction. The membershipsassignedare related to the value of the curvature subtendedat thecorrespondingpoints.Differentsetof curvaturepoints ;�< areobtained, by selecting different membershipvalues � � �����as thresholdand using fuzzy rules on the computedfeatures(� : , �

1 ). If the threshold is selectedat (� � �����>=@?(A B )both higher and medium type of curvature points will beextracted together. Thresholdingat values (� � �����DCE?(A B )are not consideredbecausealong with high and mediumtype of curvaturepoints lot of spuriouscurvaturepoints aresimultaneouslyselectedwhich may reducethe accuracy ofthe techniques.Experimentallyit has beenfound that betterresultsarewith values����������=F?(A B typically (.6, .7, .8). Thesignaturedoesnot changesignificantlyundervarying imagingconditionsasshown in Fig. 4(a), (b). The extractedsignaturefor someimagesareshown in Figs.2, 3. Theclusterof points(roi) aroundthe cornerswill not only carryshapeinformationbut also carry information about the spatial distribution ofthosepoints.The procedureis implementedfor color imagesby converting RGB plane to HSI and consideringonly theintensitycomponentfor detectingthe points.

2.2 Computation of global color moments atselective points

Among the different color models reported in literature,Normalizedrgb representation,the illumination and viewinggeometry invariant representationswhich mostly belong toHSI family of color model are popular. In addition to thesetraditionalcolor spaces,new invariantcolor models( � � ,

�,G , �'H )have been proposedin [6] which discounts the effects ofshading and shadows also. We have chosenthe ( � � ,

�,G , �'H )invariant feature model which are defined in the following(3). This model is able to denotethe differencebetweentwocolors basedon their perceptualdifference.Invariant colorrepresentationalthoughvery popularly used in CBIR, thesemodelshave short comings under certain situations,due tosomeloss of discriminationpower amongimages.The RGBplaneis convertedto ( � � ,

� G , � H )

��

� � �-I���70��JLK�3M�����'NO��PQ�R�� G � � �-I���70��NOK�34�����'JL��PQ�R�� H � � �-I���70��PSK�3M������JL��N��R� (3)

In thenext step,thecolorpropertyof theselectedcandidatesof(roi) is extracted(using(3)) from eachof thecomponentplanes( � � ,

� G , � H ) for computation of centralized second moments�UTV? ,�U?VT ,�0WVW . The normalized secondcentral momentsarecomputedusing (4).

X�Y[ZO � Y[Z��\]^] (4)

where _ Y�ZGa` W

Basedon these,the momentinvariant to translation,rotation,scaleis derived asshown in (5). Thesesetof featurecanalsobe consideredasglobal descriptorof a shapewith invariancepropertiesandwith anbuilt in ability to discernandfilter noise[14], [7].

b cXdG ] ` X ] G (5)

The image is characterizedin the following manner. Themoment(

b)of the extractedsignificantspatial locations(roi)

will help to identify color similarity of the identified regions.However for natural images(consistingof different objects)the representationobtained from shapesignature,althoughimportant but may not be sufficient for discriminatingthemfrom othercategories.Consideringthesefactsthreeadditionalsetof moments(consideringall points from eachof( � � ,

� G , � H )plane)arecomputed.Thecomponentsof thefeaturevector ef<= gih �

� h G%� h HVAjA Aj� h�k�l areas follows, h � , h G , h H representthe (b)

valuescomputed,consideringall points of eachcomponentplane obtainedfrom (3). Thesevaluesdonot vary with thethresholdinglevels. h�m , h�n , h�k contain the (

b) valuesas ob-

tainedfrom the representative locations(roi) of the generatedsignature; < from eachcomponentplane.

Page 3: IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

2.3 Estimation of relative importance of differentfeatures

For a given data base, different combination of features(color, texture, shape etc.) may be effective for handlingdifferent typesof queries[13]. Within a selectedfeatureset,the individual feature weights may be further updatedforspecifying its importancefor improvementin precision.Fora particularquery, the relative importanceof the featuresareestimatedfrom ameasure,fuzzy featureevaluationindex (FEI)proposedby Pal et al., [15]. The fuzzy measure(FEI) definedfrom interclassand intraclassambiguitiesare as follows. Let; � , ; G ,.... ;�o ... ;�� be the m pattern classesin an Ndimensional(� �

�'� G ��� Z ��A A A �qp ) featurespacewhere class ;�ocontains,7 o numberof samples.It is shown that the valueof the proposedmeasures[16], r Z o (index of fuzziness)ors Z o (fuzzy entropy) gives a measureof ’intraset ambiguity’ along the qth co-ordinateaxis in ;�o . For computing rand H of ;�o along qth component,an S-type function isconsideredand the parametersare set as, � = �t� Z o ����u , � =� ` 3M����8%vt� �qZ o �'��uxwy� �zZ o � 3M���{v , vt� �zZ o ����uxw|�t�zZ o � 365-7"v 9 , �=T(�dw}� , where �t�zZ o ����u , �t�zZ o � 3M��� , � �qZ o )min denotethemean,maximumand minimum valuesrespectively computedalongthe qth co-ordinateaxis over all the 7 o samplesin ; o . Since�0����� =�0� �qZ o �'��u =0.5, the values of r and H are 1.0 at�~��t� Z o ����u andwould tend to zerowhenmoved away fromb towardseither � or � of theS function.Thehigherthevalueof r or H the more would be the numberof sampleshaving�0� ��� equalto .5 andhencegreaterwould be the tendency ofthesamplesto clusteraroundthe meanvalue,resultingin lessinternal scatterwithin the class.After combiningthe classes;�o and ; < themean,maximumandminimumvalues� � Z < o � av,� � Z o < � max, �t� Z o < � min respectively of qth dimensionover thesamples(7 o + 7 < ) arecomputed.The criteria of a good feature is that it should be invariantto within class variation while emphasizingdifferencesbe-tween patternsof different types [15]. The value of r or Hwould thereforedecreaseas the goodnessof the qth featurein discriminatingpattern classes;�o and ; < increases.Themeasuredenotedas r Z o < or

s Z o < is called”Intersetambiguity”along ����� dimensionbetweenclasses;�o and ; < . Consideringthe two typesof ambiguities,the proposedFeatureevaluationindex (FEI) for the qth featureis,

� eO��� Zd�� � Z o <� Z o ` � Z < (6)

where d standsfor r , H. The lower the value of (FEI)q,the higher is the quality of importanceof the qth featureinrecognizinganddiscriminatingdifferentclasses.Theprecisionof retrieval can be improved by emphasizingthe weight ofthe featurethat helps in retrieving the relevant imageswhilereducingthe importanceof the featuresthat deterthe process.Squireet.al[20] proposedaweightadjustmenttechniquebasedon the varianceof the featurevalues.Since eachfeature isweighted by its relative importancesay ��� . The weightedfeaturevector is now representedas,

e p

�i� �� � h � (7)

where��� is the weight associatedwith the featurecomponenth�� Theimportanceof eachfeaturecomponent( h�� ) is evaluatedas follows. Initially all componentsare consideredto beequally importantandi.e., ��� = 1.0 andthe candidateimagesareretrievedusingEuclideandistancemetric asthe similaritymeasure.Imagesare ranked accordingto this distance.Fromthefirst retrievedsetof 20 images,theusermarksthe relevantset of images. The rest of the images from the set areconsideredirrelevant. The FEI for eachfeaturecomponentisevaluatedfrom the two obtainedclassesrelevant(intraclass),irrelevant(interclass).Betterresultsareobtainedif weightsareadjustedbasedon the FEI valuesas, ��� = � eO��� � G . A newretrieved setof the samequery is thenobtained.This processmay be followed over a numberof iterations.

2.4 Experiment

The performance of image retrieval system is testedupon two databases(a) SIMPLIcity images (b) corel10000 miscellaneous database down loaded from(http://bergman.stanford.edu/cgi-bin/����� � ��uz�*�*��� � ��� ).The SIMPLIcity databaseconsists of 1000 images from10 different categories ( Africa, Beach, Buildings, Buses,Dinosaurs,Elephants,Flowers, Mountainsand Food ). Eachcategory is having around100 imagesalongwithsomeimagesundergonechangesdueto (rotation,translation,scaling,noiseinjection, illumination) etc. Our main objective is to designa CBIR systemusing simple techniquesinvolving low costfeatureextraction mechanism.Momentsgeneratea compactrepresentationwith fewer number of featurescomparedtoother sophisticatedfeatures.It may at the sametime yieldpoor results if query complexity increases.The experimentsare performed in the following manner. We started withthe proposedfeature set as explained in section 2.2, andobtainedsatisfactory resultsfor almost all categories exceptfor few cases.In such casesbetter results are obtainedbycomputing invariant moments (

b) directly from the RGB

componentplaneswithout using(3). We designatethe featureset as computedfrom ( � � ,

� G , � H ) model using (3) as set(A), and those computeddirectly from RGB plane as set(B). Such differencesin performanceas can be explainedfrom the fact that, the RGB componentsare sensitive tovarying imaging conditions but have better discriminatingpower among images. For an unknown database,featureset(B) becomesa good choice when there is less variationin the imaging conditions or perception. The retrievalscore was further enhancedby combining both the set offeaturesin a hierarchicalfashion.This was testedon 10,000images, where Illumination invariant set (A) is used firstto get the short listed candidatesaround (100 images)anda secondset of retrieval is performed on the short listedcandidatesusing set (B). Each retrieved set can be further

Page 4: IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

subjected to feature updation scheme for generating stillbetterresults.The experimentalresultsare shown from Figs.2 to Figs.11. The resultsareexplainedasfollows : Theshapesignatureof the image in Fig. 2(a) is shown in Fig.2(b) at���d�'���{=F?.A � . Similarly for Fig. 3(a) thesignaturethresholdedat ���d������=�?.At���,?(A � respectively is shown in Fig. 3(b), (c).The representative cornersunder varying imaging condition(blurred, illumination change)of Fig. 3(a) are shown in Fig.4 (a), (b). The query results of SIMPLIcity data set areshown in ( Figs. 5 - Figs. 10). The signatureis generatedata thresholdvalueof � � �'����=�?.A � . Imagesaredisplayedfromleft to right accordingto the Euclideandistancewith the topleft imageas the query image.The queryasshown in Fig.5 is of a red flower, which is ableto identify similar images undergone illumination change.The image in Fig. 6 is from horsecategory. The precisionobtainedis very high in this case.The retrieved imagesareless dependentto shadows. Fig. 7 shows the result whenqueried with dinosaur. The precision obtainedis very highfor such imageshaving distinct objects.The featureis fairlyinvariant to linear transformations.The further improvementin precisioncanbeseenfrom Fig. 8 from its ability to retrieveblurred,noisy images(atposition 4th and 6th from left) afterupdating the weights calculatedfrom the FEI values. Theresults when queried with a flower(yellow) is shown inFig. 9. The imagesof this category have objectswith someregularity in shapeandbackground.The improvementis alsoobserved in Fig. 10 from retrieving imageswith its literalcolor properties.The resultsobtainedfrom database(b) 10000miscellaneousimagesareshown in Fig. 11 Theresultsof Fig. 11(a)show theretrieved candidatescombining the featuresin a hierarchicalfashion. The results after iterative refinement is shown inFig. 11(b). The result for a query (scene)is shown in Fig.11(c). The results obtained proves to be satisfactory forretrieving scenes.Since proposedschemecan be comparedand evaluatedbestwhen the resultsare testedand evaluatedover the samedatabase.We benchmarkour resultswith thewell known standardimage retrieval algorithmsnamely the[4], [21] and color histogrammatching[17] using the samedataset againstthe quantitative measuredefinedasweightedprecisionin (8). The weightedaverageof the precisionvalueswithin � � retrieved imagesarecomputedas,

��z� 5����W�K��,W�?V?�� � ]R]

< $ � �7 < K�W�?%? (8)

� � = 1,.....,100.7 < is the numberof matcheswithin the first� � retrieved images.The weightedprecisionasobtainedfromeachcategory areshown in Table.I

SIMPLIcity methodhasreportedbetter than color histogrammethod.It is difficult to obtainsatisfactory resultsfor retriev-ing from all categories using the sameset of features.FEIcanprovide a goodmeasurefor improvementin precision.In

TABLE I

COMPARATIVE EVALUATION OF WEIGHTED AVERAGE PRECISION

class Our method SIMPLIcity Histogrambased FIRMAfrica .45 .48 .30 .47Beach .35 .32 .30 .35

Building .35 .35 .25 .35Bus .60 .36 .26 .60

Dinosaur .95 .95 .90 .95Elephant .60 .38 .36 .25Flower .65 .42 .40 .65Horses .70 .72 .38 .65

Mountains .40 .35 .25 .30Food .40 .38 .20 .48

1.0

0.5

0ca b

S(X; a, b, c)

cb

d d

a

a1 1 1

2 1

22 2c b

Pi

(a) (b)

Fig. 1. (a) � type function (b) 3x3 neighborhoodof a pixel

mostof the casesthe averageprecisionfor eachcategory canbe madebetter than SIMPLIcity and FIRM via the featureupdation scheme.SIMPLIcity and FIRM methodsare seg-mentationbased.Betterresultis obtainedwith their algorithmwheneachobjectis having differenttexturedpropertyandthenumberof objectsmatchesthe numberof classes.We haveimplementedthealgorithmon a SUN Bladesystemwith a 700MHz Processor. The CPU time for computingthe featuresison theaverage10 secs.Thecomputationrequiredfor matchingand sorting our resultsis of the orderof O(NlogN), N is thenumberof imagesin the databaseanddoesnot dependuponthe numberof classes.

3 Conclusion

In the current work we have proposedan image retrievalscheme,wherethe the relative weightsof the featurescanbeupdatedadaptively to specify its importance.Although ourCBIR systemis not capableenoughfor handlingverycomplextype of queries,it canidentify relevant imageswhich visuallydiffer in some characteristicsdue to translation, rotation,

(a) (b)

Fig. 2. (a) Flower image(b) Fuzzycornersignatureat �.���¡ �¢(£ 0.8

Page 5: IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

(a) (b) (c)

Fig. 3. (a) Houseimage. (b) Fuzzy corner signatureat, � � �i �¢0£ 0.7 (c)� � �¡ �¢(£ 0.8

(a) (b)

Fig. 4. (a) Representative points (centroid) of the clusters(a) blurred (b)illumination change

Fig. 5. Retrieved results,with the top left imageas the query image.Testfor illumination invariancewith set (A).

Fig. 6. Retrieved resultsusingset (A), with the top left imageasthe queryimage.

Fig. 7. Retrieved results,with the top left imageas the query image.Testfor (rot, trans.scale,noiseinvariance),usingset(B)

Fig. 8. Retrieved resultswith thesamequeryof Fig.7.After featureupdationwith (FEI) the noisy andblurred imagesare retrieved. The top left imageisthe query image.

scaling, blurring, illumination change etc. We will try toincorporatesomeother mechanismlike surroundingtext, forweb imagesand study the results with MPEG-7 to obtainbetterapplications.

Acknowledgement

Minakshi Banerjeeis grateful to the Departmentof ScienceandTechnology, New Delhi, India, for providing her researchfellowship under Women Scientist Scheme, vide, grant(no.SR/WOS- A/ET-111/2003) to carry out her researchwork.

REFERENCES

[1] Minakshi Banerjeeand Malay K. Kundu, “Edge basedfeaturesforcontentbasedimage retrieval,” Pattern Recognition, vol. 36(11), pp.2649–2661,(2003).

[2] M. Banerjeeand M. K. Kundu, “Content basedimage retrieval withmultiresolution salient points,” Fourth Indian Conference ComputerVision, Graphics and Image Processing, ICVGIP 2004, India, pp. 399–404, (2004).

[3] C. Carson,M. Thomas,S .Belongie, J.M. Hellerstein, and J.Malik,“Blobworld a systemfor region - basedimageindexing and retrieval,”Proceedings of Visual Information Systems, pp. 509–516,(1999).

Page 6: IMAGE RETRIEVAL WITH VISUALLY PROMINENT FEATURES …malay/Papers/Conf/CBIR_VIE06.pdf · scale is derived as shown in (5). These set of feature can also be considered as global descriptor

Fig. 9. Retrieved resultsusingset (B), with the top left imageas the queryimage.

Fig. 10. Retrieved results,After featureupdationwith (FEI), for the queryof Fig. 9.

[4] Y. Chen and J. Z.Wang, “A region-basedfuzzy featureapproachtocontent-basedimageretrieval.,” IEEE Transactions on Pattern Analysisand Machine Intelligence, vol. 24(9), pp. 1–16,(2002).

[5] T. Gevers and A. W. M. Smeulders., “Combining color and shapeinvariant featuresfor image retrieval.,” Image and Vision computing.,vol. 17(7), pp. 475–488,(1999).

[6] T. GeversandA. W. M. Smeulders,“Color - basedobjectrecognition,”Pattern Recognition, vol. 32, pp. 453–464,(1999).

[7] R. C. Gonzalezand R. E. Woods, Digital Image Processing, Wiley,New York, (1985).

[8] C. HarrisandM. Stephens,“A combinedcornerandedgedetector,” 4thAlvey Vision Conference, pp. 147–151,(1988).

[9] A. K. JainandVailaya,“Imageretrieval usingcolor andshape,” PatternRecognition, vol. 29, pp. 1233–1244,(1996).

[10] D. Lowe, “Distinctive imagefeaturesfrom scaleinvariant keypoints,”International Journal of Computer vision, vol. 2(60),pp.91–110,(2004).

[11] EtinneLoupiasandNicu sebe,“Wavelet-basedsalientpoints:Applica-tions to imageretrieval using color and texture features,” in Advancesin visual Information Systems, Proceedings of the 4th IntenationalConference, VISUAL 2000, pp. 223–232,(2000).

[12] K. Mikolajczyk and C. Schmid, “Scale and affine invariant interestpoint detectors,” International Journal of Computer vision, vol. 1(60),pp. 63–86,(2004).

[13] H. Muller, W. Muller, D. Squire,S.M. Maillet, andT. Pun,“Performanceevaluation in content-basedimage retrieval: overview and proposals,”Pattern Recognition Letters, vol. 22, pp. 593–601,(2001).

[14] M. S. Nixon and A. S. Aguado, Feature extraction and ImageProcessing, ReedEducationalandProfessionalPublishingLtd., Oxford,England,(2002).

[15] S. K. Pal and B. Chakraborty, “Intraclassand interclassambiguities(fuzziness)in featureevaluation,” Pattern Recognition Letters, vol. 2,pp. 275–279,(1984).

[16] S. K. Pal and D. Dutta Majumder, Fuzzy mathematical Approach toPattern Recognition, Willey EasternLimited, New York, (1985).

[17] Y. Rubner, L. J. Guibas,and C .Tomasi, “The earthmover’s distance,

(a)

(b)

(c)

Fig. 11. Retrieved results,with top left imageasthe query image.(a) Firstsetof retrieved candidates(b) After featureupdationwith (FEI) (c) Retrievedresultsfor scenequery.

multi- dimensionalscaling,andcolor- basedimageretrieval,” Proceed-ings of DARPA Image understanding Workshop, pp. 661–668,(1997).

[18] A. Rosenfeld, “Fuzzy digital topology,” in Fuzzy Models For patternRecognition, JamesC. Bezdekand SankarK. Pal, Eds., pp. 331–339.IEEE press,(1991).

[19] J. R. Smith andS. F. Chang, “VisualSEEKa fully automatedcontent-basedimagequerysystem,” Proc. ACM Multimedia, pp. 87–98,(1996).

[20] D. M. Squire,W. Muller, H. Muller, andT.Pun,“Contentbasedqueryofimagedatabases:inspirationsfrom text retrieval,” Pattern RecognitionLetters, vol. 21, pp. 1993–1998,(2000).

[21] J. Z. Wang,J. Li, andG. Wiederhold,“Simplicity: Semantics-sensitiveintegratedmatchingfor picturelibraries,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol. 23(9), pp. 947–963,(2001).

[22] X. S. Zhou and T. S. Huang, “Relevance feedbackin contentbasedimageretrieval: somerecentadvances,” Information Sciences, vol. 148,pp. 129–137,(2002).