Wavelet-based Salient Points for Image Retrieval

8/2/2019 Wavelet-based Salient Points for Image Retrieval

1/4

WAVELET-BASED SALIENT POINTS FOR IMAGE RETRIEVALE. Loupias. N. Sebe

+Leiden Institute of Advanced Computer ScienceLeiden University, Th e [email protected]

ABSTRACTThe use of interest points in content-based image retrievalallows image index to represent local properties of theimage. Classic corner detectors can be used for thispurpose. However, they have drawbacks when applied tovarious natural images for image retrieval, because visualfeatures need not be comers and corners may gather insmall regions. In this paper, we present a salient pointdetector that extract points where variations occur in theimage, whether they are corner-like or not. The detector isbased on wavelet transform to detect global variations aswell as local ones. The wavelet-based salient points areevaluated for image retrieval with a retrieval system usingtexture features. In this experiment our method providesbetter retrieval performance comparing with other pointdetectors.

1. INTRODUCTIONWe are interested in content-based image retrieval ingeneral image databases. The query is an image (iconicsearch), and the retrieved images should be similar to thequery. We assume that high-level concepts (objects,feelings, etc.) cannot be extracted automatically from theimage without specific knowledge, and so use an imagesimilarity based on low-level features (such as color,texture and shapes).An image is summarized by a set of features, theimage index, to allow fast querying. Local features are ofinterest, since they lead to an index based on localproperties of the image. This approach is also attractivefor sub-image search.The feature extraction is limited to a subset of imagepixels, the interest points, where the image information issupposed to be the most important [9,2,12,1]. This paperfocuses on the selection of points that are significant tocompute features for indexing.Comer detectors are commonly used for indexing[9,12]. They are usually defined as points where gradientis high in multiple orientations. This definition leads to

S. Bres J. M. Jolion** Laboratoire Reconnaissance de Formes

et Vision, INSA L yon, France{ loup ias,sbresj olion ]@ rfv .insa-1yon .fr

detectors based on local derivatives [6 ,5] . Anotherapproach is based on local neighborhood properties [l o].Comer detectors are in general designed for robotics andshape recognition and therefore, they have drawbackswhen are applied to natural image retrieval.Visual focus points need not be corners: visualmeaningful feature is not necessarily located in a comerpoint. For instance in Figure 1, the fur is too smoothed tobe detected by a comer detector such as Harris [5].

~ ~~~~(a) Fox image (b) 100corners (Harris)Figure 1. Image with smoothed edges

Corners may gather in small regions: in variousnatural images, regions may well contain textures (trees,shirt patterns, etc.), where a lo t of corners are detected (cf.Figure 2) . As the number of points is preset to limit theindexing computation time, most of the corners are in thesame textured region. .

(a) Dutch image (b) 100corners (Harris)Figure 2. Image with texture in the Dutch dress

With comer detectors, both examples lead to anincomplete representation, where some parts of the imageare not described in the index.For these reasons, comer points may not represent themost interesting subset of pixels for image indexing.Indexing points should be related to an y visualinteresting part of the image, whether it is smoothed or

5180-7803-6297-7/00/$10.000 000 IEEE

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on November 3, 2008 at 04:28 from IEEE Xplore. Restrictions apply.


2/4

corner-like. To describe different parts of the image, theset of interesting point should not be clustered in fewregions.From now on, we will refer to these points as salientpoints, which are not necessarily corners. We will avoidthe term interest points, which is ambiguous, since it waspreviously used in the literature as corner. Waveletrepresentations, which express image variations atdifferent resolutions, are attractive to extract salient points.Previous point detectors make use of multiresolutionrepresentation. Chen et al. consider two differentresolutions to extract comers [3]. In image retrievalcontext, contrast-based points are extracted in [ 2 ] .However, a lot of points are also extracted in texturedregions because these regions are contrasted. Points areextracted with a specific wavelet in [I] but, since only agiven scale is used, different resolutions features cannot bedetected.

2. FROM WAVELETS TO SALIENT POINTSThe wavelet transform is a multiresolution representationthat expresses image variations at different scales. Forwavelet theory, see [ 8 ] . For wavelet description andalgorithms, see [11 .A wavelet is an oscillating and attenuated function (itsintegral is equal to zero). We study the image f t thescales (or resolutions) ?h,%, . 2 , j~ 2 and j I -1 . Thewavelet detail image W 2,f is the convolution of theimage with the wavelet function dilated at different scales.

(a) Cameraman image (b) Haar transformFigure 3. Haar transform

Here we consider orthogonal wavelets, which lead to acomplete and non-redundant representation of the image.A wavelet can also have a compact support: its value iszero outside a bounded interval. The simplest orthogonalcompactly supported wavelet is the Haar wavelet (seeFigure 3), which is the discontinuous step function.Daubechies proposed wavelets, with any regularity p0,> 1 ), that are also orthogonal and compactly supportedt41:

The wavelet representation gives information about thevariations in the signal at different scales. In our retrievalcontext, we would like to extract salient points from anypart of the image where something happens in the signalat any resolution. A high wavelet coefficient (in absolutevalue) at a coarse resolution corresponds to a region withhigh global variations. The idea is to find a relevant pointto represent this global variation by looking at waveletcoefficients at finer resolutions.

Since we use wavelets with a compact support, we knowfrom which signal points each wavelet coefficient at thescale 2 was computed. We can study the waveletcoefficients for the same points at the finer scale 2.Indeed there is a set of coefficients at the scale 2+computed with the same points as a coefficient W2J ( n )at the scale 2 (see [ 7 ] for details). We call this set ofcoefficients the children C(W2, ( n ) ) of the coefficientW z ,f n ) The children set in one dimension is:c(w,, f ( n > ) = f ( k ) , 2n s k I 2n+ 2 p I 1,

where 0 I n < 2N, with N the length of the signal and pthe wavelet regularity.Each wavelet coefficient W2, ( n ) is computed with

2 - p signal points. It represents their variation at thescale 2. Its children coefficients give the variations ofsome particular subsets of these points (with the number ofsubsets depending on the wavelet). The most salient subsetis the one with the highest wavelet coefficient at the scale2+, that is the maximum in absolute value ofC(W2,f n ) ) . In our salient point extraction algorithm, weconsider this maximum, and look at its highest child.Applying recursively this process, we select a coefficientW2-l f (n )at the finer resolution ?h cf. Figure 4). Hence,this coefficient only represents 2p signal points. To selecta salient point from this tracking, we choose among these2p points the one with the highest gradient. We set it ssaliency value as the sum of the absolute value of thewavelet coefficients in the track:saliency = -I (Ck(V2, /(n)l ,0 n < 2N, -log* N I I1

k=l

The tracked point and its saliency value are computedfor every wavelet coefficient. A point related to a globalvariation has a high saliency value, since the coarsewavelet coefficients contribute to it. A finer variation also

I For clarity we use one-dimensional signals. Extension to twodimensions and signals with length not restricted to a power of 2 ,in addition to algorithm complexity, are discussed in [7].

519



3/4

leads to an extracted point, but with a lower saliencyvalue. We then need to threshold the saliency value, inrelation to the -desired number of salient points. We firstobtain the points related to global variations; localvariations also appear if enough salient points arerequested.

(a) 100Haar salient points for the Fox image

~ ~~(a) 100Haarsalient points (b) Tracked coefficients

(c) Spatial support of tracked coefficientsFigure 4. Salient points extraction ,The salient points extracted by this process depend onthe wavelet we use. Haar is the simplest wavelet function,

so the fastest for execution. Some localization drawbackscan appear with Haar due to its non-overlapping waveletsat a given scale. This can be avoided with the simplestoverlapping wavelet, Daubechies 4.However, this kind ofdrawback is not likely in natural images.3. EXAMPLES

The salient points detected with the Haar transform arepresented for the images used in Figure 1and Figure 2(cf:Figure 5 ) . For each image the detected points aresuperimposed on the original image to evaluate salientpoints location.Salient points are detected for smoothed edges (cf:Figure 5.a) and are not gathered in textured regions ( c jFigure 5.b). Hence they lead to a more complete imagerepresentation than comer detectors. Similar behavior canbe observed with Daubechies 4 wavelets.Repeatability of the detection under typical alterations isa common evaluation criterion for corner detectors.Repeatability of our detector is comparable to otherdetectors. However this criterion may not be relevant inour context, because features stability is more importantthan geometric stability for image retrieval.

(b) 100Haar salient points for the Dutch imageFigure 5. Haar salient points examples

4. EVALUATION FOR IMAGE RETRIEVALThe best way to evaluate points detectors for imageretrieval is to compare retrieval results obtained with eachdetector. The retrieval system is constituted by theindexing (points extraction and computation of localfeatures to build image indexes) and the querying (basedon a similarity measure between indexes).Different retrieval features and image databases are usedin [7] to compare points detectors. Here we present resultswith an image retrieval system2 based on texture features[13]. Gabor features are computed for regions around theextracted points for 3 scales and 8 orientations. Maximumamplitudes are used to build a set of histograms, which isthe image index.

We use a database of 1505 various natural images. Eachimage belongs to an instinctive (and subjective) category(animals, flowers, landscapes, buildings, cities.. .). Veryheterogeneous categories and images too different fromthe rest of the category are removed from the test set.Finally, we have a test set of 577 images in 9 classes.We present the recall-precision graph, computed fromdifferent numbers of return images n. The system retrievesr images that belong to the same class C as the query(Y I). There are Nc images in the class C of the query.Then P = r I n is the precision and R = r I N , the recallfor this query. We use each test set image as a query, anduse the average recall and precision for the graph (4Figure 6).

* This retrieval system is available through KIWI, the Key-points Indexing Web Interface:http://telesun.insa-lyon.fr/kiwi520

http://telesun.insa-lyon.fr/kiwihttp://telesun.insa-lyon.fr/kiwihttp://telesun.insa-lyon.fr/kiwi


4/4

656055

Q 50453 4030

-.-

p' 35

25L--- I0 J0 5 10 15 20Recall (%)

Figure 6. Retrieval resultsWe observe that the wavelet-based salient pointsperform better than other detectors for these features andthis database. Daubechies 4 has better performances thanHaar but is computationally more expensive. Randompoints are also used in the experiment: we randomly selectpoints, and compute the Gabor features around these

points. Their good result can be explained by theirspreading in the image. For that reason they lead to a morecomplete representation of the image than some detectors.Obviously, the random points are very unlikely to belocated in corners or edges point, but they are spreadenough to represent these variations in the index. Goodresult of random points for indexing was observed withother databases and other local features [7]. Theseexperiments show that the points spreading can be asimportant as the points location for image indexing(depending on the features). However, wavelet-basedsalient points, which are simultaneously spread andlocated, perform better than random points.5. DISCUSSION

We presented a salient point detector based on wavelets.The wavelet-based salient points are interesting for imageretrieval, because they are located in many visual features(whether they are corner-like or not), without gathering intextured regions. We presented a retrieval experiment withGabor features where our method performs better thanother point detectors from the literature.We used the Haar transform for point extraction, whichis simple but may lead to bad localization. Daubechieswavelets avoid this drawback, but are not symmetric.Since orthogonality is not required in our approach, wecould extend it to other wavelets that are compactlysupported and symmetric.Since points performance for indexing depends on theimage database, detector choice for a specific databaseshould be investigated, as well as random points relevancefor local features extraction.

Wavelets are also attractive to extract image features forindexing. These local features would be more related toour salient points.6. ACKNOWLEDGMENTS

E. Loupias' guest period in Leiden University wassupported by the RhGne-Alpes Region, France(EURODOC grant). Thanks to Dr. D.P. Huijsmans andProf. F. Peters, Leiden University, for discussions we hadabout this topic.

7. REFERENCES[l ] S. Bhattacharjee and T. Ebrahimi, "Image Retrieval Basedon Structural Content ", Workshop on Image Analysis forMultimedia Interactive Services, Heinrich-Hertz-Institut (HHI)Berlin, Germany, May 31 - June 1 1999.[2] S. Bres and J.-M. Jolion, "Detection of Interest Points forImage Indexation ", 3rd Int. Con$ on Visual InformationSystems, VisuaZ99,Amsterdam, The Netherlands, June 2-4 1999,[3] C.-H. Chen, J.-S. Lee and Y.-N. Sun, "WaveletTransformation for Gray-level Comer Detection ", PatternRecognition, 1995, Vol. 28, No. 6, pp. 853-861.[4] I. Daubechies, " Orthonormal bases of compactly supportedwavelets ", Communications on Pure and Applied Mathematics,[5] C . Harris and M. Stephens, " A Combined Comer and EdgeDetector ",Proc. of 4th Alvey Vision Conference, 1988, pp. 147-151.[6] L. Kitchen and A. Rosenfeld, "Gray-Level ComerDetection ", Pattern Recognition Letters, December 1982, Vol.171 E. Loupias and N. Sebe, " Wavelet-based Salient Points forImage Retrieval", RR 99.11, Laboratoire Reconnaissance deFormes et Vision, INSA Lyon, November 1999. On-linehttp://rfv.insa-1yon.fr/-loupias/points/[8] S. Mallat, "A Theory for Multiresolution SignalDecomposition :The Wavelet Representation ",IEEE Trans. on191 C. Schmid and R. Mohr, "Local Grayvalue Invariants forImage Retrieval ",IEEE Trans. on PAMI, May 1997, Vol. 19,[lo] S. Smith, J. Brady, "SUSAN - A New Approach to LowLevel Image Processing ", International Journal of ComputerVision,May 1997,Vol. 23, No. 1 , pp. 45-78.[ I l l E. Stollnitz, T. DeRose and D. Salesin, "Wavelets forComputer Graphics: A Primer, part 1 ", IEEE ComputerGraphics and Applications, May 1995, Vol. 15, No. 3, pp. 76-84.[I21 T. Tuytelaars and L. Van Gool, "Content-based ImageRetrieval Based on Local Affinely Invariant Regions ", 3rd Int.Conf on Visual Information Systems, Visual99,Amsterdam, TheNetherlands, 2-4 June 1999, pp. 493-500.[I31 C. Wolf, "Content based Image retrieval using InterestPoints and Texture Features ", RR 99.09, LaboratoireReconnaissance de Formes et Vision, INSA Lyon, 1999. On-line demo :http://telesun.insa-lyon.fr/kiwi

pp. 427-434.

1988, Vol. 41, pp. 909-996.

1,NO. 2, pp. 95-102.

PAMI, July 1989, Vol. 11,NO. 7, pp. 674-693.

NO. 5, pp. 530-535.

52

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL Downloaded on November 3 2008 at 04:28 from IEEE Xplore Restrictions apply
http://rfv.insa-1/http://telesun.insa-lyon.fr/kiwihttp://telesun.insa-lyon.fr/kiwihttp://telesun.insa-lyon.fr/kiwihttp://rfv.insa-1/

Documents

Wavelet-based Salient Points for Image Retrieval