Neighborhood sequences for comparing similarity vectors in image retrieval

Neighborhood sequences for comparing similarity vectors in imageretrieval

András Hajdu,

Tamás Tóth,

Krisztán Veréb22 August, 2006

page 2 of 22 22 August, 2006Information Technology in Agriculture and Rural Development

Neighborhood sequences (basic concepts and notation)

n N; q, r Zn; m Z, with 0 m n;Pri(q) is the i-th coordinate of q.

q and r are m-neighbors if|Pri(q) - Pri(r)| ≤ 1, 1 ≤ i ≤ n, and|Pr1(q)-Pr1(r)| + …+ |Prn(q)-Prn(r)| ≤ m.

B is an nD-neighborhood sequence ifB(bi)i N with 1 ≤ bi ≤ n, for i N-re

q = q0 q1 … qj = r is an B-path of length j if qi-1 and qi are b(i)-neighbors.

d(q, r; B) is the length of the shortest B-path


Measuring distance in RGB cube by 3D neighborhood sequences

24bit RGB cube, between black = (0, 0, 0) and white = (255, 255, 255).

Extendable to other color representation and higher dimensions.

Distance values highly depend on the chosen sequence:

d(C1,C2;{1})=360, d(C1,C2;{2})=180, d(C1,C2;{3})=120,

d(C1,C3;{1})=240, d(C1,C3;{2})=180, d(C1,C3;{3})=180,

d(C2,C3;{1})=300, d(C2,C3;{2})=150, d(C2,C3;{3})=150.

{3 }

{2 }{1 }

C

(1 7 0 ,1 7 0 ,1 7 0 )2

C

(5 0 ,5 0 ,5 0 )1

C

(2 3 0 ,2 0 ,8 0 )3


Periodic neighborhood sequences

nD neighborhood sequence:B = (Bi)∞

i=1

If for some jN, Bi = Bi+j for all i N, then B is called periodic with period j.

The brief notation of periodic neighborhood sequences:B = B1B2…Bj

If j = 1 then N is called a constant neighborhood sequence:{1} ≡ L1

{3} ≡ L∞, 3D case


„Ultimately periodic” neighborhood sequences

Let B(j) denote the neighborhood sequence obtained by omitting the first j elements of B periodic

neighborhood sequence. The sequence B = (Bi)∞

i=1 is called ultimately periodic, if B(k) is periodic for some k N.

If B(k) has period length l–k, then we writeB = B1B2…BkBk+1…Bl


CNS – Classic Neighborhood Sequences

B1, B2, B3 based on the well known 6-, 18-, 26-neighborhoods.

The number in the index shows how many coordinates may change (1-, 2-, 3-neighborhood).

There is no restriction about which coordinate should change, in which direction we can make a move.

B1 = {(0, 0, ±1), (0, ±1, 0), (±1, 0, 0)},B2 = B1 {(0, ±1, ±1), (±1, 0, ±1), (±1, ±1, 0)},B3 = B2 {(±1, ±1, ±1)}.

Naturally algorithms are presented for calculating the distance.


SNS – „Subspace” Neighborhood Sequences

It is explicitly given in which direction we can make a move. Bx = {(±1, 0, 0)}, By = {(0, ±1, 0)}, Bz = {(0, 0, ±1)},

Bxy = {(±1, ±1, 0)}, Byz = {(0, ±1, ±1)}, Bxz = {(±1, 0, ±1)},Bxyz = {(±1, ±1, ±1)}.

Numbers can be used against the names of the axis’s in applications:

x 4, y 2, z 1;in case of more axis's these numbers are added.

Distance measurement is trivial.


MNS – „Mixed” Neighborhood Sequences

Classic (CNS) and Explicit (SNS) neighborhood sequences can be combined.

Usable neighborhoods are: B1, B2, B3, Bx, By, Bz, Bxy, Byz, Bxz, Bxyz

We allow both explicit (SNS) and classic neighborhood movements too.

The distance measurement is the hybrid of the methods of CNS and SNS distance measurement.


Disadvantages of SNS

It is not sure that there is finite distance between two points if we use SNS and we do not change every coordinates directly:

e.g.: {Bxz} periodic sequent never reaches the point at (0, 1, 0).

Different sequences may give the same result if empty steps can be taken (if we could not converge to the goal the length of the way do not increase):

e.g.: t = (3, 1, 3)d(t) = xz, xz, xz, xz, xz, xz, xz, xz, 3 =d(t) = xz, xz, xz, 3 = 4


Empty step

There are three different possibilities to handle empty steps:

Empty step has none-zero weight (this is our main approach). We may hold our position for some cost at every step.

Empty step has zero weight. It means that we can decide to ignore any such elements of neighborhood sequences that cannot help us to reach a given point. The sequence can be permuted freely then.

We do not let empty steps neither for some cost nor for free, and force to move to some other position at every step.


Generative grammar

The suitable neighborhood sequences used in applications may be too long.

e.g.: we want to allow 10 B3 steps after 100 B1 step. G = <VN, VT, S, H> VT={N1,N2,N3,Nx,Ny,Nz,Nxy,Nxz,Nyz,Nxyz} S → {Ak}

S → SSA → SA → a

a [1…7] {a, b, c} k N {∞}


Generative grammar - examples

The previous neighborhood sequent with grammar notation:{a100}{c10}

The meaning of the {23}{{45}{71}2} formula given by grammar is:

{222 444447 444447} {a3}{{51}{21}∞} means:

{aaa5252525252…} ≡ {aaa52}


Oracle – Image Database

Storing of multimedia data (video, sound, image) is a real task of database managing systems.

Oracle developed a visual information retrieval (VIR) tool, which is standard feature of Oracle since 9i.

The feature vector contains the followings: Color histogram, Texture descriptor, Shape representation, Their location information

The similarity value derivation is built on the matching values of these properties.


Oracle – feature vectors

Location by itself is not a meaningful search parameter, so we won’t consider location as a feature.

A weight value between 0.0 and 1.0 can be assigned to every feature. This value denotes the relative

”importance” of the feature. 0.0 not considered at all. 1.0 highest importance at the end of the comparison, we have three real values

between 0.0 and 100.0 (c = color, t = texture, s = shape), and three normalized weights (w1, w2, w3), where w1+w2+w3 = 1.

The final distance is:d = w1 * c + w2 * t + w3 * s


Experimental results

Database of 1240 distinct images. Generating Oracle feature vectors

(both three feature has same importance) :comparing every image to each other.

The build matrix will be processed. Looking for images which are similar to the seed image.



Used sequence: {Nxz4}

{Ny40}

23 383732

46454544



Used sequence: {Nx4}

{Nz4}{Ny

30}

27 413936

49494544



Used sequence: {Nz4}

{Nx4}{Ny

30}

36 413937

48474645



Numerical comparison:

id1 id2 color shape texture860 850 83.6478 57.8826 2.4824860 851 6.16674 40.7421 2.2384860 852 82.1175 27.6878 1.6104860 853 9.20443 29.4861 1.396860 854 3.97565 28.4126 2.0864860 855 5.33467 26.8312 2.1816860 856 21.5916 33.8737 4.268860 857 2.51894 32.7586 1.598860 858 4.38217 33.5518 3.864860 859 3.01489 18.5306 2.9624


Conclusions

Applicability in arbitrary finite dimensions. Adaptation in other (non database) applications. More consideration for digitalizing. Assigning non-negative weights to each step.


References A. Fazekas, A. Hajdu, L. Hajdu: Lattice of generalized sequences in

nD and ∞D, Publ. Math. Derbecen 60 (2002), 405-427. A. Hajdu, B. Nagy, Z. Zörgő, “Indexing and segmenting colour

images using neighbourhood sequences”, IEEE ICIP 2003, Barcelona, Spain, pp. I/957-960

A. Hajdu, T. Tóth, K. Veréb and Z. Zörgő: Distance functions in multidimensional image processing applications, KÉPAF 4 (2004), Miskolc-Tapolca.

M.D. Levine, and A.M.Nazif: Dynamic measurement of computer generated image segmentations, IEEE trans. PAMI 7, pp. 155-164, 1985.


Thank you for your attention

Documents

Neighborhood sequences for comparing similarity vectors in image retrieval