9
Content-based trademark retrieval system using a visually salient feature Y.S. Kim, W.Y. Kim* Hanyang University, Image Engineering Laboratory, Electronic Engineering, HeangDang-Doug, Seoul Sungdong-Ku, 133-791 South Korea Received 21 March 1997; received in revised form 6 October 1997; accepted 12 November 1997 Abstract The ever-increasing number of registered trademarks has created greater demand for an automatic trademark retrieval system. In this paper, we present a method for such a system based on the image content, using a shape feature. Zernike moments of an image are used for a feature set. To retrieve similarly shaped trademarks quickly, we introduced the concept of a ‘visually salient feature’ that dominantly affects the global shape of the trademarks. Experiments have Rbeen conducted on a database of 3000 trademark images. The retrieval speed was very fast and similar-shaped trademark retrieval results were very promising. q 1998 Elsevier Science B.V. All rights reserved. Keywords: Content-based; Image database; Image retrieval; Trademark; Zernike moment 1. Introduction An image retrieval system based on image content is a key area for building and managing large multimedia data- bases such as trademark and copyright, art galleries and museum, picture archiving and communication system (PACS), to name a few [1]. So, interest in the subject of content-based image retrieval has greatly increased for the past few years. In this paper, we address the problem of visually similar trademark retrieval from a large trademark database using shape features. Trademarks are considered valuable intellectual properties and a key component of the goodwill of a business, since they represent not only the quality of actual products and services, but also the reputation of the manufacturer or the company. A registered trademark is protected through legal proceedings from misuse or imita- tion. Until now, since the total number of registered trade- marks is over a million, the task of designing and registering a new trademark becomes more difficult without inadvertent infringement of copyright. So far, the current practice to classify trademarks is first by grouping the trademarks into several similar shapes according to a specific class order, followed by performing the matching process manu- ally by human operator [2]. Therefore, the development of an on-line automatic trademark retrieval system for similar shapes becomes crucial. In this paper, Zernike moment magnitudes (ZMMs) are used as a feature set. ZMMs are robust to noise or small variance of a pattern, and have rotation invariant character- istics. With a proper normalization method, scale invariance has also been achieved [3]. To retrieve similar shapes, we developed the ‘visually salient feature’ that dominantly affects the global shape of the trademarks by ignoring minor details. The visually salient feature was determined by the probabilistic distribution model of a trademark data- base. To verify the performance of our proposed similar- shaped trademark retrieval system, several trademarks were submitted as a query image to a trademark database that contains 3000 trademarks. We also considered pseudo-Zernike moments as a feature set. Pseudo-Zernike moments have properties analogous to Zernike moments. The performance of pseudo-Zernike moments was very similar to that of Zernike moments. 1.1. Terminology and research scope A trademark is a complex pattern, consisting of various text and image patterns. Trademarks can be divided into four types as shown in Fig. 1. Word-in-mark is a trademark that contains only characters or words in the mark. Charac- ter recognition or manual annotation is required to handle the type because the linguistic property (word structures and phonetics) is the key component of the type. On the other hand, a device-mark contains graphical or figurative elements only. Thus, the geometric shape is the key compo- nent for the type. Composite-mark consists of characters or 0262-8856/98/$ - see front matter q 1998 Elsevier Science B.V. All rights reserved. PII S0262-8856(98)00060-2 * Corresponding author. Tel: 0082 02 290 0351; fax: 0082 02 292 6316; e-mail: [email protected] Image and Vision Computing 16 (1998) 931–939 IMAVIS 1516

Content-based trademark retrieval system using a visually salient feature

  • Upload
    ys-kim

  • View
    214

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Content-based trademark retrieval system using a visually salient feature

Content-based trademark retrieval system using a visually salient feature

Y.S. Kim, W.Y. Kim*

Hanyang University, Image Engineering Laboratory, Electronic Engineering, HeangDang-Doug, Seoul Sungdong-Ku, 133-791 South Korea

Received 21 March 1997; received in revised form 6 October 1997; accepted 12 November 1997

Abstract

The ever-increasing number of registered trademarks has created greater demand for an automatic trademark retrieval system. In thispaper, we present a method for such a system based on the image content, using a shape feature. Zernike moments of an image are used for afeature set. To retrieve similarly shaped trademarks quickly, we introduced the concept of a ‘visually salient feature’ that dominantly affectsthe global shape of the trademarks. Experiments have Rbeen conducted on a database of 3000 trademark images. The retrieval speed wasvery fast and similar-shaped trademark retrieval results were very promising.q 1998 Elsevier Science B.V. All rights reserved.

Keywords:Content-based; Image database; Image retrieval; Trademark; Zernike moment

1. Introduction

An image retrieval system based on image content is akey area for building and managing large multimedia data-bases such as trademark and copyright, art galleries andmuseum, picture archiving and communication system(PACS), to name a few [1]. So, interest in the subject ofcontent-based image retrieval has greatly increased forthe past few years.

In this paper, we address the problem of visually similartrademark retrieval from a large trademark databaseusing shape features. Trademarks are considered valuableintellectual properties and a key component of the goodwillof a business, since they represent not only the quality ofactual products and services, but also the reputation of themanufacturer or the company. A registered trademark isprotected through legal proceedings from misuse or imita-tion. Until now, since the total number of registered trade-marks is over a million, the task of designing and registeringa new trademark becomes more difficult without inadvertentinfringement of copyright. So far, the current practice toclassify trademarks is first by grouping the trademarksinto several similar shapes according to a specific classorder, followed by performing the matching process manu-ally by human operator [2]. Therefore, the development ofan on-line automatic trademark retrieval system for similarshapes becomes crucial.

In this paper, Zernike moment magnitudes (ZMMs) areused as a feature set. ZMMs are robust to noise or smallvariance of a pattern, and have rotation invariant character-istics. With a proper normalization method, scale invariancehas also been achieved [3]. To retrieve similar shapes, wedeveloped the ‘visually salient feature’ that dominantlyaffects the global shape of the trademarks by ignoringminor details. The visually salient feature was determinedby the probabilistic distribution model of a trademark data-base. To verify the performance of our proposed similar-shaped trademark retrieval system, several trademarks weresubmitted as a query image to a trademark database thatcontains 3000 trademarks.

We also considered pseudo-Zernike moments as a featureset. Pseudo-Zernike moments have properties analogous toZernike moments. The performance of pseudo-Zernikemoments was very similar to that of Zernike moments.

1.1. Terminology and research scope

A trademark is a complex pattern, consisting of varioustext and image patterns. Trademarks can be divided intofour types as shown in Fig. 1.Word-in-markis a trademarkthat contains only characters or words in the mark. Charac-ter recognition or manual annotation is required to handlethe type because the linguistic property (word structuresand phonetics) is the key component of the type. On theother hand, adevice-markcontains graphical or figurativeelements only. Thus, the geometric shape is the key compo-nent for the type. Composite-mark consists of characters or

0262-8856/98/$ - see front matterq 1998 Elsevier Science B.V. All rights reserved.PII S0262-8856(98)00060-2

* Corresponding author. Tel: 0082 02 290 0351; fax: 0082 02 292 6316;e-mail: [email protected]

Image and Vision Computing 16 (1998) 931–939

IMAVIS 1516

Page 2: Content-based trademark retrieval system using a visually salient feature

words and graphical elements, while acomplex-markcontains a complex image. Our current system focuses onretrieving device-mark types only.

1.2. Related work and literature review

Content-based image retrieval can be categorized intothree parts: color-, texture- and shape-based retrieval. Anumber of techniques have appeared in the literature thatdeal with retrieval based on shape similarity. The QBIC(Query by Image Content) system allows queries on alarge image database using various image contents suchas color, texture, shape and position [4]. Jagadish pro-posed a similar shape retrieval method using the rectan-gular cover description [5,6]. Bigu¨n et al. proposed animage retrieval system using orientation radiogramswhich are similar to the histogram of the edge directions[7]. Bimbo et al. presented an image retrieval system using ahierarchical model of the curve which is derived from itsmulti-scale analysis [8]. Mokhtarian et al. proposed thesimilar shape retrieval method using the maxima of curva-ture zero-crossing contours in the curvature scale space [9].

Several researchers have applied shape-based retrievaltechniques to trademark images. Kato introduced a con-tent-based similar shaped-trademark retrieval system [10].This system used graphical features such as spatial outlineof the overall figures, spatial, frequency, local correlationmeasure and local contrast measure. Cortelazzo et al.presented the trademark shape description method using astring matching technique [11]. Jain et al. proposed ahierarchical image retrieval system and tested the system

on a trademark database [12–14]. Their system uses atwo-stage hierarchy: a fast screening stage using a histo-gram of the edge directions and invariant moments and adetailed matching stage using deformable template match-ing [15]. Eakins presented the SAFARI (shape analysis forautomatic retrieval of images) system with curvature-basedfeature [16]. He developed a later version of SAFARI,so called ARTISAN (automatic retrieval of trademarkimages by shape analysis) that utilized more complexfeatures: circularity, aspect ratio, discontinuity angleirregularity etc. [17]. Lam et al. presented a trademarkretrieval system, STAR (system for trademark archivaland retrieval) [18]. The system consisted of two partsto handle device-marks and word-in-marks. For device-marks, invariant moments and Fourier descriptorsextracted from manually isolated distinct objects wereused for shape features and the similarities among thetrademarks are measured by a fuzzy thesaurus. For word-in-marks, the system performed sub-string matching andphonetics matching to retrievetrademarks that have similarlinguistic properties.

Boundary based techniques such as boundary matching[11,16,17], Fourier descriptors [18] and multiscale curvematching [8,9] may not be suitable for similar-shaped trade-mark retrieval, as the boundary shape can be changeddrastically when there is a small crack like an openingor an object touching neighboring objects. For example,the shapes shown in Fig. 2(a) and (c) are very similar inhuman perception. The boundaries of these shapes, asshown in Fig. 2(b) and (d), however, are very differentwhether or not the inner star touches the outer circle.Furthermore, while most Fourier descriptor or curvature-based methods are based on a single boundary, a trademarkconsists of a complex pattern that has more than one bound-ary. Morphology-based preprocessing can be applied toremedy the problem, but it is not easy to determine thenumber of operations such as erosion or dilation to yieldthe optimum result for all trademark images. In addition,the resulting number of contours may also be very sensitiveto the number of preprocessing steps.

A histogram of the edge directions [7,12–14] has alsobeen used in many systems. The drawback of thistechnique lies in the lack of discernment, because the histo-gram alone does not contain the information of edge loca-tion. For example, the images shown in Fig. 3(a)–(c),although their shapes are very different, have similar histo-grams of the edge directions as illustrated in Fig. 3(d).

Fig. 1. Types of trademarks.

Fig. 2. (a) A symbol of touching star and a circle, (b) the boundary contourof (a), (c) A symbol of non-touching star and a circle (d) the boundarycontour of (c).

Fig. 3. (a), (b) and (c) are examples of different shapes whose histogramsof the edge directions are very similar as shown in (d).

932 Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 3: Content-based trademark retrieval system using a visually salient feature

1.3. Outline of paper

The rest of this paper is organized as follows. In Section 2,we overview Zernike moments as a feature set. In Section 3,we present a probabilistic distribution model of thefeature. Then, our retrieval method is described inSection 4. Experimental results are given in Section 5, andSection 6 summarizes the paper.

2. Zernike moments as a feature set

Zernike moments are complex orthogonal momentswhose magnitude has rotational invariant property[19–22]. Teh et al. compared several moments in terms of:

1. sensitivity to image noise;2. aspects of information redundancy;3. capability for image representation.

They reported that Zernike and pseudo-Zernike momentsoutperform the other moments, such as regular moments,Legendre moments, rotational moments and complexmoments, in all aspects [23]. Kim et al. has shown thatZernike moments outperforms invariant moments in termsof inter-class sensitivity and intra-class insensitivity [3].

2.1. Definition of Zernike moments

Zernike moments are defined inside the unit circle and theradial polynomial vectorR(r) is defined as:

Rnm(r) ¼∑n¹ lml

2

s¼ 0(¹1)s (n¹ s)!

s!nþ lml

2¹ s

� �!

n¹ lml2

¹ s

� �!rn¹ 2s

R(r) ¼ { RnmðrÞln¼ 0,1, 2, …,`, lml # n,

andn¹ lml is even}

Then the two-dimensional Zernike moment of an image,I(r,v), in polar coordinate is defined as:

A ¼nþ 1

p

∑r

∑v

[V(r, v)]pI (r, v), s:t:r # 1

Here,V(r,v) is a Zernike basis polynomial defined as:

V(r, v) ¼ R(r)exp( ¹ jmv):

The magnitude of Zernike moment (ZMM) is defined as,

z¼ kAk

So, thez denotes the vector ofznm. That is,

z¼ { znmln¼ 0, 1,2, …,`, lml # n, andn¹ lml is even}

2.2. Visual effects caused by each order of ZMM

The order of Zernike moments is determined by two para-meters;n andm. Each order of Zernike moment is computedby multiplying the radial polynomial to the phase term. Asillustrated in Fig. 4(a), the larger the difference betweenn and m, ln–ml, the higher the frequency components inthe radial polynomial. So, an image whose ZMM is large

Fig. 4. (a) Zernike polynomials of the order of (n ¼ 1,5, m ¼ 1) (b) The phase of Zernike moment.

Table 1List of ZMM, z up to the ordern ¼ 8

n ZMM No. of ZMM

0 z00 11 z11 12 z20,z22 23 z31,z33 24 z40,z42,z44 35 z51,z53,z55 36 z60,z62,z64,z66 47 z71,z73,z75,z77 48 z80,z82,z84,z86,z88 5

933Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 4: Content-based trademark retrieval system using a visually salient feature

for a large value ofln–ml, has a large degree of high fre-quency component in the radial direction, i.e., the image isradially complex image. On the other hand, an image withlarge ZMM for a small value ofln–ml can be said to be aradially simple image.

As shown in Fig. 4(b), the phase term of each Zernikemoment ism-fold circular symmetric. That is, an imagewith a large value of ZMM corresponds to anm-foldcircular symmetric image. For example, the ZMM ofm ¼

2 becomes a dominant feature for a narrow and elongatedpattern, the ZMM of m ¼ 3 for a triangular-shaped,the ZMM of m ¼ 4 for a square-shaped, and the ZMM ofm ¼ 5 for a pentagonal-shaped pattern, etc.

3. Trademark database and its distribution model

3.1. Trademark data collection

Three thousand Korean and world trademarks werecollected from the reference [24,25] by scanner. All trade-mark images were binarized and normalized to the sizeof 100 3 100 pixels by maximum extent circle (MEC)method [26]. Color was not considered in our currentsystem. ZMMs were computed by the lookup-table method[26] and stored in a database up to the order ofn ¼ 17. Thetotal number of moments corresponding ton ¼ 17, is 90[23].

3.2. Distribution model

The distribution model of features plays an importantrole in our system to determine the feature that best distin-guishes the query trademark from the other trademarksin the database. As mentioned in Section 2.2, the order ofZMM is determined by two parameters;n and m. NinetyZMMs were computed and stored in the database for all3,000 trademarks. The probabilistic distribution of ZMMfor each order was modeled by a Gamma distribution as

shown in Fig. 5, which is defined as

f (z; a,b) ¼1

baG(a)za ¹ 1 exp( ¹ z)U(z)

G(a) ¼

∫`0

xa ¹ 1 e¹ x dx:

The parameters of the Gamma distribution,a andb, can bereadily estimated with the mean and variance by solving[27]

E(z) ¼ ab,E(z2) ¼a(a þ 1)b2:

So, the parameters of the Gamma distribution are

a ¼{ E(z)} 2

E(z2) ¹ { E(z)} 2, b¼(E(z2) ¹ { E(z)} 2

E(z):

Here,a andb can be easily updated when a new trademarkis added to the database, because only the mean and var-iance of feature are used in the estimation.

The distribution model was verified by Kolmogorov–Smirnov goodness-of-fit test and accepted with significancelevel of 0.05 [27]. The overall probabilistic distribution ofZMMs of database is shown in Fig. 6.

4. The similar shape retrieval

4.1. Retrieving similar trademarks using the most salientfeature

With 90 Zernike moment features to use for retrievingsimilar trademarks from database, one of the common prac-tice is to make use of the Euclidean distance in featurespace, along with a proper weight on each feature [22].The one whose distance to the query is the minimum willbe selected. However, when the number of patterns in a

Fig. 5. Typical distribution of each order of ZMM. (ZMM of order (n ¼ 3,m ¼ 0), (a30 ¼ 1.42,b30 ¼ 0.05).

Fig. 6. The overall probabilistic distribution of ZMM of database.

934 Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 5: Content-based trademark retrieval system using a visually salient feature

database to compare is very large, the number of featuresshould also be increased, and this naive approach maypose a computational problem. In addition, as the size ofthe database grows, this approach may not be feasible,especially when the system needs to run in real-time foron-line applications. Furthermore, the determination ofthe weight for each feature to yield an optimum result isnot a trivial matter even with a neural-network basedapproach, since the weight should be updated as moredata merges to the database. For these reasons, instead ofusing all 90 features, we decided to use as few featuresfor each trademark as possible.

A well-known problem in pattern recognition is that morefeatures do not necessarily imply a better classification.Therefore, the effective selection of good features is animportant issue. As defined in [28], the most expressivefeature (MEF) has to yield the minimum squared error inreconstruction, while the most discriminating feature(MDF) has to best distinguish an image from the rest ofimages in the database. A specific feature for either MEFor MDF, or any specific orders of moments in our applica-tions, may not be the same for all queries. Therefore theselection scheme plays an important role because everytrademark might have its unique characteristics. For thatpurpose, we developed a new feature selection methodthat can comply with the requirements for both MEF andMDF.

To meet the requirements, we first introduce asalientfeature that dominantly affects the shape globally but notits minor details. Thedegree of saliency, DS(n,m), is definedfor each feature extracted from the query trademark as aparameter and indicates the degree of relative contributionof each order of moment to the global shape of the trade-mark;

DS(n, m) ¼ P[znm $ Z(q)nm] ¼

∫`ZðqÞ

nm

f (znm; anm,bnm) dznm

Here,Z(q)nm is the ZMM of (n,m)-th order of the query trade-

mark. As illustrated in Fig. 7, the larger the value ofZ(q)nm, the

more the shape is affected by the (n,m)-th order momentcompared with other moments. Note that this is a very simi-lar concept to MEF defined in [28]. That is, as mentioned

in Section 3.2, a moment with the large magnitude isresponsible for the global shape of the trademark. A similarargument holds for MDF. That is, the smaller the value forDS(n,m), the less probable and more unique it is for the restof trademarks to have (n,m)-th order of moment as theirMEF. Therefore, the most salient feature (MST) of thequery trademark is defined by the (N,M)-th order of momentthat yields the smallestDS(N, M). The degree of the simi-larity between the trademarks and the query trademark isthen determined by the Euclidean distance from the queryto all trademarks in the database using only the MSF or (N,M)-th order moment instead of 90 moments.

In order to retrieve visually similar trademarks fromthe database to the query trademark, the minor differencesof the shape should be allowed to some degree of user-specified level. Only the trademarks whose differencewithin the range are selected, i.e.

Ll # lZ(i)NM ¹ Z(q)

NMl # Lh;

where the superscript (i) indicates the trademark index.L l

andLh indicate the lower and upper bound, respectively, andshould satisfy the following constraint,

∫Z(q)NM

Ll

f (zNM; aNM,bNM) dzNM ¼

∫Lh

ZðqÞ

NM

f (zNM; aNM,bNM) dzNM

¼ user defined confidence level

so that the area to the left and right ofZ(q)NM remains the

same. As illustrated in Fig. 8, this confidence boundryalso determines the number of trademarks to be retrieved.

4.2. Further pruning of the selected trademarks

The similar-shape-retrieval method described in theprevious section, based on the MSF, depends on two factors;radial complexity andm-fold circular symmetry of theshape. Therefore, the retrieved trademarks will havethe similar radial complexity and the degree of circularsymmetry. Although the MSF is the dominant cue of theshape, no single feature may be sufficient to discriminatemany different shapes. As a result, several trademarks that

Fig. 7.DS(m,n) is defined asP(Z(q)NM $ znm). Fig. 8. Determination of the selection range from user specified confidence

level and inverse CDF.

935Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 6: Content-based trademark retrieval system using a visually salient feature

are not visually similar to the query will be selected. Themajor cause for this problem is due to the fact thatm-foldcircular symmetric images also respond to integer multiplesof m-fold circular symmetric images, ormk-fold images. Forexample, triangular-shaped images will be retrieved for ahexagonal-shaped image since 3 is a GCD of 6. To remedythis, the retrieved trademarks whoseZn,GCD exceeds athreshold are eliminated.

For more accurate retrieval, more elaborate additionalmatching techniques such as deformable template matchingor other pattern recognition techniques may be applied afterthis stage.

5. Experimental results

To verify the performance of our proposed similar-shapedtrademark retrieval scheme, several trademarks shown inFig. 9 were submitted as query image to the trademarkdatabase that consists of 3,000 trademarks. The perfor-mance is estimated by the following subjective and objec-tive criteria;

1. How well can similar-shaped trademarks be retrieved inaccordance with the human perception.

2. How well can the same trademarks be retrieved in thepresence of noise or deformation.

When a query was submitted to the system, its ZMMswere computed first. Using these ZMMs and the prob-abilistic distribution model of the database,DS(n,m) werethen calculated to determine the MSF. Next, the similaritymeasure between the query and all trademarks in the data-base were computed. The average elapse time for ZMMcomputation of the query and the similarity distance to alltrademarks in the database are listed in Table 2. Thepreliminary demo version of our system is available athttp://trademark.hanyang.ac.kr

5.1. Subjective criteria: evaluated by precision (similar-shaped trademark retrieval)

The first criterion for estimating the retrieval performanceis the retrieval precision defined as

Precision¼Number of relevant trademarksNumber of retrieved trademarks

3 100

This criterion can be rather subjective because the judge-ment of relevance (similarity) is very subjective and contextdependent. We determined the relevance by interviewing 10people working in this area. The precision of the query isshown graphically in Fig. 10, and candidates for relevanttrademarks retrieved among the top 30 trademarks areshown in Fig. 11. The average precision was about 65%.

Similar trademarks were retrieved regardless of theirorientation because the ZMM have the rotation invariantcharacteristics, and the retrieved results of TM3 in Fig. 11were good examples for such a desirable property.

5.2. Objective criteria: evaluated by recall (noisy ordeformed trademark retrieval)

Several sets of deformation transformation, as shownin Fig. 12(a)–(f), and various types of noise, shown inFig. 12(g),(h), were prepared in order to generate a set ofdeformed trademarks as shown in Fig. 13. The noisy ordeformed trademarks were rotated and scaled arbitrarilyand submitted as query images to the trademark databaseto examine whether the noisy or deformed trademarks canretrieve their original ones. Table 3 presents the results ofrecall rate in the top 30 candidates queried by noisy ordeformed trademarks. All noisy or deformed trademarkscan recall their original ones in the top 30 candidates, butin the case of spherical deformation, only TM4, TM5, TM9

Fig. 9. Queried trademarks.

Table 2Average elapse time for retrieving trademarks on a PC (200 MHz PentiumPro)

Moment computation Database query Total

0.1 s 0.5 s 0.6 s

Fig. 10. Precision versus number of retrieved image.

936 Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 7: Content-based trademark retrieval system using a visually salient feature

and TM10 are recalled successfully. The average recall rateof the original ones among the top 30 candidates queriedby noisy or deformed trademarks was 92.5%.

The noise or deformation can affect the value in everyorder of ZMM. The MSF of a trademark, however, is barelyaffected by the noise or the deformation if the global shapeof the trademark does not change substantially, becausethe MSF implies the global shape of a trademark. The rea-son that the recall rate of the spherically deformed trade-marks is relatively low is because the MSF of thosetrademarks are significantly changed.

6. Summary and discussion

In this paper, we presented a new content-based similarshape retrieval method for trademarks using Zernike

Fig. 11. Trademarks in the left most column are the query trademarks, the rest are the retrieved results in the order of similarity.

Fig. 12. (a)–(f) are several deformation transformations, (g) and (h) aredifferent types of noise.

937Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 8: Content-based trademark retrieval system using a visually salient feature

moments. The advantages of using MSF are twofold: quickretrieval of similar trademarks, and robustness to the minortransformation of the shape.

Since the radial complexity and the degree of circularsymmetry of the shape are reflected in the MSF, theretrieved trademarks using the MSF will have similar char-acteristics. The MSF of the trademark was barely affectedby noise or deformation if the global shape of the trademark

remained similar. In the experiment, we showed that it waspossible to retrieve the original image using the noisy or thedeformed ones. The average precision of retrieval was about65% and the average recall rate of the originals was amongthe top 30 candidates queried by noisy or deformed trade-marks was 92.5%.

The proposed method is suitable for a large trademarkimage database to retrieve a moderate number of similar

Fig. 13. Trademarks in the left most column are the original ones and the rest are noisy or deformed.

Table 3The recall rate of the original ones among the top 30 candidates queried by noisy or deformed trademarks

Original Pinch Punch Sphere Twirl Ripple Diffuse Tiles Impulse noise Total

Recall rate (%) 100 100 100 40 100 100 100 100 100 92.5

938 Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939

Page 9: Content-based trademark retrieval system using a visually salient feature

trademarks because the query process is very simple, fastand robust to noise or deformation.

We do not, however, expect our system to handle a data-base of more than 3000 trademarks with the MSF only. Toincrease the accuracy as well as the capability of the system,we are presently working on the development of an addi-tional shape feature and a data fusion scheme to handlemore diverse and complex shapes including colors.

Acknowledgements

This work was supported by the Electronics and Tele-communications Research Institute under grant 97202.

References

[1] V.N. Gudivada, V.V. Raghavan, Content-based image retrievalsystems, IEEE Comput. 40 (1995) 18–22.

[2] B. Andrews, U.S. Patent and Trademark Office ORBIT TrademarkRetrieval System, T-term User Guide, Examining Attorney’s Version,Oct. 1990.

[3] Kim W.Y., Yuan P.O., A practical pattern recognition system fortranslation, scale and rotation invariance, in: Proceedings of IEEEInternational Conference on Computer Vision and Pattern Recogni-tion, 1994.

[4] M. Flickner, H. Sawhney, Query by image and video content: theQBIC system, IEEE Comput. 23 (1995) 23–39.

[5] H.V. Jagadish, A retrieval technique for similar shapes, in: Proceed-ings of ACM SIGMOD, 1991, pp. 208–217.

[6] S.K. Chang, Y. Cheng, S.S. Iyengar, R.L. Kashyap, A new method ofimage compression using irreducible covers of maximal rectangles,IEEE Trans. Software Engng 14 (1988) 651–658.

[7] J. Bigun, S.K. Bhattacharjee. Michel S., Orientation radiograms forimage retrieval: an alterative to segmentation, in: Proceedings of IEEEInternational Conference on Pattern Recognition, 1996.

[8] A.D. Bimbo, P. Pala, Image indexing using shape-based visual fea-tures, in: Proceedings of IEEE International Conference on PatternRecognition, 1996.

[9] Mokhtarian, S. Abbasi and J. Kittler, Efficient and robust retrievalby shape content through curvature scale space, in: Proceedings ofInternational Workshop on Image Databases and Multimedia Search,1996, Amsterdam, The Netherlands.

[10] T. Kato, Database architecture for content-based image retrieval, in:Proceedings of SPIE Conference on Image Storage Retrieval Systems,1992.

[11] G. Cortelazzo, G.A. Mian, G. Vezzi, P. Zamperoni, Trademark shapesdescription by stringmatching techniques, Pattern Recog. 27 (8)(1994) 1005–1018.

[12] A.K. Jain, A. Vailaya, Image retrieval using color and shape, PatternRecog. 29 (1996) 1233–1244.

[13] A. Vailaya, A.K. ZhongYuJam, A hierarchical system for efficientimage retrieval, in: Proceedings of IEEE International Conferenceon Pattern Recognition, 1996.

[14] B. Kroepelien, A. Vailaya, A.K. Jain, Image database: a case studyin Norwegian silver authentication, in: Proceedings of IEEE Inter-national Conference Pattern Recognition, 1996.

[15] A.K. Jain, Y. Zhong, S. Lakshmanan, Object matching using deform-able templates, IEEE Trans. Pattern Anal. Mach. Intell. 18 (3) (1996)267–277.

[16] J.P. Eakins, Retrieval of trademark images by shape feature, in:Proceedings of Int. Conf. on Electronic Library and Visual Informa-tion Research, (1994) 101–109.

[17] J.P. Eakins, K. Shields, J. Boardman, ARTISAN — a shape retrievalsystem based on boundary family indexing, in: Proceedings of SPIE,Storage Retrieval Image Video Database, 1996.

[18] C.P. Lam, J. K. Wu, B. Mehtre, STAR — a system for trademarkarchival and retrieval, in: Proceedings of the 2nd Asian Conference onComputer Vision, 1995.

[19] R.J. Prokop, A.P. Reeves, A survey of moment-based techniques forunoccluded object representation and recognition, CVGIP: Graph.Models Image Process. 54 (1992) 438–460.

[20] A. Khotanzad, Y.H. Hong, Invariant image recognition by zernikemoments, in: IEEE Transactions on Pattern Analysis Machine Intelli-gence, vol. 12, 489–498, 1990.

[21] M.R. Teague, Image analysis via the general theory of moments,J. Opt. Soc. Am. 70 (1980) 20–45.

[22] A. Khotanzad, Y.H. Hong, Rotation invariant image recognition usingfeatures selected via a systematic method, Pattern Recog. 23 (1990)1089–1101.

[23] C.H. Teh and R.T. Chin, On image analysis by the methods ofmoments, in: IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 10, 496–513, 1988.

[24] E. Kim and J.M. Kim, Korean Symbol Marks and Logotypes, Kuna,Korea, 1992.

[25] J.M. Kim, World Symbol Marks, Logotypes and Pictographs, Kabul,Korea, 1986.

[26] W.Y. Kim, An Analytical and Experimental Study of Binary ImageNormalization for Scale Invaliance with Zernike Moments, Journal ofElectrical Engineering and Information Science, vol. 2, no. 6, Dec.,1997.

[27] J.L. Devore, Probability and Statistics for Engineering and theSciences, 3rd Ed, Bookes/Cole, Pacific Grove, Calif., 1991.

[28] D.L. Swets, J. Weng, Using discriminant eigenfeatures for imageretrieval, IEEE Trans. Pattern Anal. Mach. Intell. 18 (8) (1996)831–836.

939Y.S. Kim, W.Y. Kim/Image and Vision Computing 16 (1998) 931–939