Information Processing Letters 74 (2000) 209–214

A concept-based query evaluation with indefinite fuzzy triples

Jae Dong Yang1

Department of Computer Science, Chonbuk National University, Chonju, 561-756, South Korea

Received 28 March 1999; received in revised form 10 February 2000Communicated by F. Dehne

Abstract

Triple indexing technique provides us with more understandable specification of the spatial structure of images withoutcompromising retrieval time by a well-designed hash function. However, this technique has a serious drawback; itaccommodates neither a concept-based image retrieval facility nor does it allow indefinite object labeling. 2000 ElsevierScience B.V. All rights reserved.

Keywords:Information retrieval; Fuzzy logic; Query evaluation

1. Introduction

Triple representation [1] is a variation of the 2D(dimensional) string [2–4], which is one of the mostpromising techniques used for indexing images. Thistriple representation was proposed to enhance the se-mantic expressiveness of the 2D strings by translatingthem into conceptually equivalent triples. The noveltyof this technique is to provide us with more under-standable specification of the spatial structure of im-ages without compromising retrieval time by a well-designed hash function. However, the technique has aserious drawback; it accommodates neither a concept-based image retrieval facility nor does it allow indefi-nite object labeling. Retrieving images based on con-cepts is crucial to get answers relevant to user queries,whereas allowing the indefinite labeling is indispens-able to compensate for the poor recognition power ofextant image analyzers. The purpose of this paper isto propose a query evaluation supporting a concept-based image retrieval as well as the indefinite label-

1 Email: jdyang@cs.chonbuk.ac.kr.

ing. The evaluation is based on the I-triple (Indefinite-triple) framework adopting a fuzzy matching [5].

2. Image retrieval by I-triples

According to [1], the iconic imagep in Fig. 1 maybe indexed by ordinary triples:{〈w,c,northwest〉,〈r,w,east〉, 〈r, c,north〉, 〈w, s,north〉, 〈r, s,northeast〉}wherew is ‘working table’,r is ‘radio’, s is ‘speaker’andc is ‘clock’. However, if it is possible that the ra-dio can also be labeled as ‘recorder’, then indefinitetriples (I-triples) should be introducedand〈r, c,north〉may be represented as〈r ∨ re, c,north〉 where re is‘recorder’.

Additionally, we need to specify objects by theirnames in order to deal with an I-triple. It helps us dealwith occurrences of an object, which share the samename. The objectsshouldbe named by the followingterms.

Definition 2.1. A fuzzy linguistic term (or simply aterm)T is a fuzzy set characterized by the membership

0020-0190/00/$ – see front matter 2000 Elsevier Science B.V. All rights reserved.PII: S0020-0190(00)00066-1

210 J.D. Yang / Information Processing Letters 74 (2000) 209–214

Fig. 1. Iconic imagep.

functionµT :N → [0,1] whereN is a set of terms.c in N is a crisp term if there existsc′ such thatµC(c

′) = 1, for c′ = c and µC(c′) = 0, otherwise.A term that is not crisp is referred to as fuzzy term.

Definition 2.2. Let N be a set of terms,IDB be a setof all images in an image database andOp be a set ofall objects which can occur in an imagep ∈ IDB. Thena name function,fNAME is defined as follows.

For allo ∈Op, there existsT ∈N such that

fNAME(o)= T .

We can now define an indefinite triple with thisfunction.

Definition 2.3. Let Op be the set of all objects inp ∈ IDB andD = {east,northeast,north, . . .} be theset of eight directions. Then a set of all indefinitetriples (I-triples) forp, I_Tp is given by

I_Tp ={⟨

m1∨i=1

fNAME(oi),

m2∨j=1

fNAME(oj ), rij

⟩ ∣∣∣∣rij ∈D is a relative direction ofoj

with respect tooi for oi, oj ∈Op,

16 i 6m1, 16 j 6m2

}.

Note that the number of I-triples forn primitiveobjects of an image isn∗(n− 1)/2, since we treat anI-triple identical to

⟨m1∨i=1

fNAME(oi),

m2∨j=1

fNAME(oj ), rij

⟩,

if it is given as⟨m2∨j=1

fNAME(oj ),

m1∨i=1

fNAME(oi), rij

⟩,

whererij is the reverse direction ofrij .We next need term predicates to make our I-triple

framework complete. The following is a prerequisite.

Definition 2.4. A fuzzy linguistic term predicate (orsimply a term predicate)T corresponding to the termT in N is defined by

T :N→[0,1],where

T (c)={

1 if c= T ,

µT (c) for eachc 6= T ∈N .

For example, furniture(X) is a term predicate, whereasfurniture is its corresponding term. furniture(c) forsomec ∈ N is therefore interpreted as the degree ofsatisfaction ofc to furniture.

The following disjunction of term predicates en-ables us to introduce the final I-triple version whereterms are replaced by their corresponding term pred-icates. The term predicates serve as templates whichcan make it possible for terms to match each otherbased on concepts.

J.D. Yang / Information Processing Letters 74 (2000) 209–214 211

Definition 2.5. Let d_T (A) =∨mi=1Ti(A) be a dis-

junction of term predicates. Then an I-triplet ∈ I_Tpis defined as follows.(1) Let each ofd_T1, andd_T2 be a disjunction of

term predicates andd ∈D. Then〈d_T1, d_T2, d〉is an I-triple.

(2) Nothing else is an I-triple.If an I-triple t is of the form,t = 〈T1, T2, d〉 whereeach ofT1 andT2 is a term predicate, then we call it,simply, triple.

3. Exploiting thesauri for a concept-based match

The term thesaurus in Fig. 2 is used to evaluate termpredicates. It contains crisp terms in leaf nodes and inthe other ones, fuzzy terms, each taking lower levelfuzzy terms as its members with degrees specified onthe corresponding edges. Any degree between themis assumed 0 if unspecified. Additionally, a composedmembership function is provided for obtaining mem-bership values between two terms indirectly related.

Definition 3.1 [5]. Let T be a fuzzy term. Then

µT (c)=max(min(µai (c),α)

)for all c ∈N,

whereα = µT (ai) andai is a fuzzy term.

Note that sinceai is a fuzzy term,µai (c), ∀c ∈ Ncan also be calculated by the recursive applicationof Definition 3.1 even ifc is not directly connectedwith ai .

Note also that the degree of conceptual closenessbetween two terms not related with any edge cannotbe obtained by this definition. For example, it does

not define an edge of certain degree between electronicappliance and audio. In this paper, we do not considera similarity relation which may be used to quantifysuch degree.

Example 3.1. The degree of conceptual closenessbetween furniture and radio is calculated by

µfurniture(radio)

=max(min

(µelectronic_appliance(radio),

µfurniture(electronic_appliance)),

min(µaudio(radio),µfurniture(audio)

))=max

(min(0.96,0.87),min(0.7,0.91)

)= 0.87.

Hence, furniture(radio)= 0.87.

In the above example, the max operator enablesµfurniture(radio) to take the most informative value0.87 by passing through ‘electronic appliance’ be-tween ‘radio’ and ‘furniture’ instead of passing through‘audio’. Moreover, it states that either of the two pathsis available to quantifyµfurniture(radio).

We are now in a position to formally define ourimage retrieval system.

Definition 3.2. An image retrieval systemI_IR isdefined as follows.

I_IR = 〈IDB, I_T ,Tr, In〉,whereIDB is a set of all images,I_T is a set of all(I_Tp)s for p ∈ IDB,Tr is a term thesaurus, andInis an inverted file. We defineIn as a set of entries,each formed by attaching tot ∈ I_Tp, every image,p satisfyingt ∈ I_Tp, i.e., In = {〈t, {p}〉}. In is usedto search for images indexed byt .

Fig. 2. Term thesaurus.

212 J.D. Yang / Information Processing Letters 74 (2000) 209–214

4. Query evaluation

Evaluation of a user query to retrieve images in-volves translating it into equivalent triples and thenmatching the I-triple in each entry ofIn.

As a first step toward developing such an evaluation,we provide a definition to specify our query structure.

Definition 4.1. Let each ofQTj , j = 1, . . . , s, be atriple used as a basic unit of a query. Then the fol-lowing disjunction of thes triples is called disjunctivequeryQD .

QD =s∨j=1

QTj .

Definition 4.2. Let QDi , i = 1, . . . , n, ben disjunc-tive queries. Then a conjunctive normal queryQ (orsimply query) is defined as follows.

Q=n∧i=1

QDi .

We now formalize a fuzzy match involving morethan one disjunction of term predicates.

Definition 4.3. Let d_T =∨mi=1Ti be a disjunction

of term predicates. Then for allc ∈ N , d_T (c) =max(Ti(c), i = 1, . . . ,m).

Definition 4.4. Let d_T be a disjunction of termpredicates. Then|d_T | = {c ∈N | d_T (c)= 1}.

Definition 4.5. Let each of d_T1 and d_T2 be adisjunction of term predicates. Thend_T1 is moregeneral with a degreeα ∈ [0,1] thand_T2 iff

min(d_T1(c) for all c ∈ |d_T2|

)= α > 0.

It is denoted byd_T2⊆α d_T1.

Example 4.1. Let d_T1 = electronic_appliance andd_T2= radio∨ recorder. Thend_T2⊆0.92 d_T1 since

min(d_T1(c) for all c ∈ |d_T2| = {radio, recorder})=min

(electronic_appliance(radio),

electronic_appliance(recorder))

= 0.92> 0.

Definition 4.6. LetQ′ be a sub-query of the disjunc-tive queryQD =∨s

j=1QTj given as follows;

Q′ =ik∨j=i1

QTj , QTj = 〈T1,j , T2,j , d〉

for j = i1, i2, . . . , ik ∈ {1,2, . . . , s},sharingd with an I-triple t = 〈d_T1, d_T2, d〉. Thenwe call it common direction disjunctive query, orbriefly c-d query fort .

Definition 4.7. LetQ′ =∨sj=1〈T1,j , T2,j , d〉 be a c-d

query fort = 〈d_T1, d_T2, d〉. Then

t ⊆α Q′ with α =min(α1, α2) > 0⇔d_T1⊆α1

s∨j=1

T1,j and d_T2⊆α2

s∨j=1

T2,j .

In the sequel, the following proposition is pro-vided for testingd_T1 ⊆α1

∨sj=1T1,j or d_T2 ⊆α2∨s

j=1T2,j .

Proposition 4.1. Let each ofd_T1 andd_T2 be a dis-junction of term predicates. Supposed_T1=∨m

i=1Ti .Thend_T2 ⊆α d_T1 iff min(max(Ti(c/A), i = 1,2,. . . ,m) for all c ∈ |d_T2|)= α > 0.

Proof. It can be directly proved by Definitions 4.3and 4.5. We omit its proof.2Example 4.2. Letd_T1= radio∨receiver andd_T2=electronic_appliance∨ audio. Thend_T1 ⊆0.9 d_T2,since

min(max(electronic_appliance(radio),audio(radio)),

max(electronic_appliance(receiver),

audio(receiver))

=min(0.96,0.9)= 0.9> 0

by Proposition 4.1.

Consider next the condition that an I-triplet ∈ I_Tpexactly matches a queryQ guaranteeing a minimumpossibility regardless of its indefiniteness. Detecting

J.D. Yang / Information Processing Letters 74 (2000) 209–214 213

the condition may be crucial to obtain the followingexact answer set ofQ.

Definition 4.8. Let QD be a c-d query fort ∈ I_Tp .Then

I_IR`α∗ QD(p) iff t ⊆α QD, α > 0.

Definition 4.9. ‖Q‖α∗ = {p | I_IR`α∗ Q(p),α > 0}is a set of exact answers satisfying a queryQ with athreshold valueα > 0.

Example 4.3. Let a query be given as “search forimages where an electronic appliance locates at thenortheast of furniture”. Then it is converted intoQT1 =〈electronic_appliance, furniture,northeast〉. Now, fort = 〈d_T1, d_T2,northeast〉 ∈ I_Tp1 where d_T1 =radio∨ recorder,d_T2 = working_table,t ⊆0.91QT1

since d_T1 ⊆0.92 electronic_appliance (see Exam-ple 4.1) and working_table⊆0.91 furniture.

Hence,I_IR`0.91∗ QT1(p1).

Example 4.4. Let the queryQD1 be “search for im-ages where an electronic_appliance or audio is atthe northeast of furniture”. ThenQD1 = QT1 ∨ QT2

where QT1 is given in Example 4.3 andQT2 =〈audio, furniture,northeast〉. Now, fort = 〈d_T1, d_T2,

northeast〉 ∈ I_Tp2 where d_T1 = radio∨ receiver,d_T2 = table, the result isI_IR `0.9∗ QD1(p2) sinced_T1 ⊆0.9 electronic_appliance∨ audio (see Exam-ple 4.2) and table⊆0.91 furniture.

Proposition 4.2. LetQD =∨sj=1QTj be a disjunc-

tive query. Then for allp ∈ IDB,

p ∈ ‖QD‖α∗, α =max(α1, α2, . . . , αs) > 0,

iff there exists at least one c-d query,Q′i satisfying

p ∈ ‖Q′i‖α∗i , αi > 0, 16 i 6 s.

Proof. To bep ∈ ‖Q′i‖α∗i , there should exist at leastoneti ∈ I_Tp that satisfiesti ⊆αi Q′i , αi > 0, 16 i 6s. Without loss of generality, we can assume such c-dqueries areQ′1 andQ′2 for t1, t2 ∈ I_Tp, respectively.If t1 ⊆α1 Q

′1 or t2 ⊆α2 Q

′2, then p ∈ ‖Q′1‖α∗1 , or

p ∈ ‖Q′2‖α∗2 , α1, α2 > 0 from Definitions 4.8 and 4.9.We can now prove this proposition since eitherp ∈‖Q′1‖α∗1 or p ∈ ‖Q′2‖α∗2 implies p ∈ ‖QD‖α∗ , α =max(α1, α2) > 0. The same is true even ift1 ⊆α1 Q

′1

and t2 ⊆α2 Q′2. It is possible sincep can be indexed

simultaneously byt1 andt2 which are not necessarilythe same with each other.2Theorem 4.1. Let QD =∨s

i=1QTi be a disjunctivequery. Then

s⋃i=1

‖QTi‖α∗i ⊆ ‖QD‖α∗,

α =max(α1, α2, . . . , αs) > 0.

Proof. By using Proposition 4.2, it is straightforwardto show if p ∈ ‖QTi‖α∗i , i = 1,2, . . . , s, then p ∈‖QD‖α∗ . Hence, we only prove

s⋃i=1

‖QTi‖α∗i 6= ‖QD‖α∗

by showing that there exists somep ∈ ‖QD‖α∗ butp /∈ ‖QTi‖α∗i , i = 1,2, . . . , s.

Let

Q′ =ik∨j=i1〈T1,j , T2,j , d〉

be a c-d query ofQD for an I-triple,t = 〈d_T1, d_T2,

d〉 ∈ I_Tp. If such a c-d query does not exist for allt , any p cannot be considered as an answer. Henceφ ⊆ φ, which trivially holds. By Definition 4.8 andProposition 4.2, we can find thep if it is indexedby t that satisfiest ⊆α Q′ with α > 0, but, none oft ⊆αi QTi , i = 1,2, . . . , s, holds.

Without loss of generality, letQ′ = QT1 ∨QT2 =〈T1, T2, d〉 ∨ 〈T ′1, T ′2, d〉 andt = 〈T1∨ T ′1, T2∨ T ′2, d〉.Then, obviously,t ⊆1 Q

′ but t ⊆0 QTi , i = 1,2.Therefore, we can always find somep satisfyingp ∈‖QD‖α∗ butp /∈ ‖QTi‖α∗i , i = 1,2, . . . , s. 2Theorem 4.2. Let Q =∧n

i=1QDi be a query. Thenfor all p ∈ IDB,

p ∈ ‖Q‖α∗ with α =min(α1, α2, . . . , αn)⇔p ∈ ‖QDi ‖α∗i , αi > 0, i = 1,2, . . . , n.

Proof. (⇒) Supposep ∈ ‖Q‖α∗ . Then by Defini-tion 4.8,

I_IR`α∗ QD1(p)∧QD2(p)∧ · · · ∧QDn(p),

214 J.D. Yang / Information Processing Letters 74 (2000) 209–214

which meansI_IR `α∗i QDi (p) for all i = 1,2, . . . ,n, with α = min(α1, α2, . . . , αn). Accordingly, p ∈‖QDi ‖α∗i .

(⇐) Similarly, we can prove the (⇐) part. We omitit. 2Theorem 4.3. Let Q = ∧n

i=1QDi , and QDi =∨sij=1QTij be a query. Thenp ∈ ‖Q‖α∗ with

α =min(max(α11, . . . , α1s1), . . . ,

max(αi1, . . . , αisi ), . . . ,max(αn1, . . . , αnsn))

such that fori, 16 i 6 n, p ∈ ‖QTij ‖αij , j = 1, . . . ,si .

Proof. This theorem can be easily proved by Proposi-tion 4.2 and Theorem 4.2. We omit its proof.2Example 4.5. Let the queryQ beQD1 ∧QD2 whereQD2 = 〈furniture, table,northeast〉. Then, sincep2 ∈‖QD1‖0.9 (see Example 4.4) andp2 ∈ ‖QD2‖0.84 frommin(0.84,1)= 0.84, we getp2 ∈ ‖Q‖0.84.

5. Conclusion

In this paper, we proposed a concept-based queryevaluation against images indexed with indefinitetriples. As further research, the similarity betweenterms needs to be incorporated into the query evalu-ation for more sophisticated concept-based matching.

References

[1] C.C. Chang, S.Y. Lee, Retrieval of similar pictures on pictorialdatabases, Pattern Recognition (1991) 675–680.

[2] S.K. Chang, E. Jungert, C. Tortora, Intelligent Image DatabaseSystems, Series on Software Engineering and Knowledge Engi-neering, Vol. 5, World Scientific, Singapore, 1996.

[3] E. Jungert, S.-K. Chang, An image algebra for pictorial datamanipulation, CVGIP: Image Understanding 58 (2) (1993) 147–160.

[4] D. Papadias, T. Sellis, The semantics of relations in 2D spaceusing representative points: Spatial indexes, in: Proc. EuropeanConference on Spatial Information Theory, 1993.

[5] J.D. Yang, D.G. Lee, Incorporating concept-based match intofuzzy production rules, Inform. Sci. 104 (1998) 213–239.