Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada

Image Indexing and Retrieval using

Moment Invariants

Imran AhmadSchool of Computer Science

University of Windsor – Canada

I. Ahmad - Windsor, Canada 2

Outline

IntroductionShape-based RetrievalImage RepresentationMoment InvariantsProposed ApproachExperimental ResultsConclusions


Introduction

Information CharacteristicsNatureMultiple formatsComplexTypes

Image DatabaseTime dependent sequence

Video Database


Introduction (contd.)

Still information

Dynamic informationTemporal Evolution of informationChanges in features and characteristics


Introduction (contd.)

Image contentsReal world informationUnique featuresNeed retrieval based on contents


Image Retrieval

Exact matchSimilarity-based retrievals

Color / texture similarity-based retrievalsSpatial similarity-based retrievalsShape similarity-based retrievals


Shape-based retrievals

Model-based Object Recognition ApproachModels based on global and local featuresUnknown object compared against known ones

Data-Driven ApproachAn index for known shapesSearch utilizes such indices


Image Representation

Images can be defined in terms of:Global features

Based on overall image compositionEasier to compute

Local featuresBased on individual image componentsIncorporate spatial informationComputationally expensive


Shape

How to define a shape?A geometric property of a figureFormal definition – independent of language

Described in terms of properties invariant under a group of coordinate transformationsLet is a characteristic function such that

0

1),( yxf

For points in the figure

Otherwise

),( yxf


Shape (contd.)

Definition: Let be a group of coordinate transformations. The function I is invariant w.r.t. if

for all characteristic functions and all transformations

Definition: A shape of a figure is a pair <I, >, where I is invariant under the group of coordinate transformations

yxfIyxfI ,,

),( yxf


MomentsCan capture global information about image Do not require closed boundaries.

Regular moments – introduced by Hu.Invariant to translation, rotation and scaling

Algebraic momentsDo not depend on actual values of the coefficients

Central momentsEquivalent to regular moments of an image that has been shifted


Moments (contd.)

ApplicationsImage reconstructionShape identification such as aircrafts, etc.Shape recognitionClassifiers


Moments (contd.)

Let be the image intensity distribution functionp +q is the order of moments (for p, q =0, 1, 2, …)

the algebraic moment of functions are given as:

For a digital image of size M x N

yxf ,

R

qppq dxdyyxfyxm ,

M

x

N

y

qppq yxfyxm

0 0

,


Moments (contd.)

For centralized moments, we can write:

with its digital form as:

dxdyyxfyyxxqp

pq ,

M

x

N

y

qppq yxfyyxx

0 0

,


Moments (contd.)

M

x

N

y

M

x

N

y

M

x

N

y

M

x

N

y

mymymyy

mymmmmyy

mmmmyy

m

0 001

20203

303

01020 0

0020102

202

0 00000010101

0 00000

23

0

1

Central moments up to 2nd order are defined as:


Moments (contd.)

010 0

2201121

221

10200 0

0021020

220

100 0

2021112

212

10110 0

0001101111

0 00000101010

22

22

0

mxmymxmyyxx

mymmmmxx

mymxmymyyxx

mymmmmmyyxx

mmmmxx

M

x

N

y

M

x

N

y

M

x

N

y

M

x

N

y

M

x

N

y


Moments (contd.)

Algebraic moments by Hu

2

30122

210321033012

20321

21230123003217

03211230112

03212

123002206

23012

2210321032103

20321

21230123012305

20321

212304

20321

212303

211

202202

02201

33..........

33

4

33..........

33

33

4

M

M

M

M

M

M

M


Moments (contd.)

Moment Invariants:Time complexity in computing MI is directly proportional to the number of pixels in the silhouette or forming the boundary.Let N be the perimeter of the closed boundaryTo calculate 2nd order moments, we need:

4(N-1) real additions and 3N real multiplicationsTo calculate 3rd order moments, we need:

6(N-1) real additions and 12N real multiplications


Moments (contd.)

A Sample binary image & moment invariants

F={0.259179343138514, 0.00801986505055, 0.012354456089699, 0.00827468547136, -0.000000750728194, -0.00005777268349,

0.00000025369430}


Clustering

Main CategoriesHierarchical methods

A nested sequence of partitionsInvolves multiple iterations to cluster objects

Non-hierarchical methodsAssume desired number of clusters at the beginningData is reallocated until a particular clustering criteria is optimized.Objects in a cluster are more similar to each other.


Clustering (contd.)

K-means clusteringLet a set of N objects in d-dimensional space Rd

k is an integerDetermine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center.

Also known as squared-error distortion.


K-means Clustering (contd.)

ALGORITHM: K-means clustering 1. For pattern vectors P1, P2,…, Pm, set first

k pattern vectors to the initial clusters C1=P1, C2=P2, C3=P3,…, Ck=Pk, where m >= k

2. Assign each pattern vector to the nearest cluster

3. Compute new cluster means 4. If new cluster means = old cluster means

stop, else

go to step 2



K-means Clustering Tree (KCT)The KCT is a hierarchical data structure similar to a combination of binary search tree and B+ -tree.

Data pointers are stored only at the leaf nodes of tree. Non-leaf nodes that contain weight vectors. Non-leaf nodes have links to other nodes.Unidirectional linksLeft child nodes with lesser threshold valuesRight child nodes with greater threshold values.



K-means Clustering Tree (KCT) – contd.KCT has a single root nodeLeaf nodes of KCT contain features for an image shape and a pointer to the images. Non-leaf nodes of the tree correspond to the other levels of the index. The nodes correspond to disk pages and the structure is designed so that search requires visiting only a small number of pages.


K-means clustering (contd.)

KCT Tree creationAssume 2-D feature vector for each of 19 object as: { {-3.0, 3.0}, {-2.5, 3.0}, {-2.0, 2.0}, {-1.5, 2.0}, {-3.5, 1.5}, {-4.0, 1.0}, {-3.0, 0.5}, {-3.0, 0.0}, {-1.5, 0.5}, { 2.5,-0.5}, { 2.5,-1.0}, { 4.5,-1.0}, { 0.5,-1.5}, { 1.0,-2.0}, { 3.0,-2.0}, { 4.0,-2.0}, { 0.5,-2.5}, { 1.0,-2.5}, { 2.0,-3.0}}



Same 2-dimensional numerical values Each object value will be in leaf node of KCT Object 1 2 3 4 5 6 7 8 9

X -3.0 2.5 -2.0 -1.5 -3.5 -4.0 -3.0 -3.0 -1.5

y 3.0 2.0 2.0 2.0 1.5 1.0 0.5 0.0 0.5

Object 10 11 12 13 14 15 16 17 18 19

2.5 2.5 4.5 0.5 1.0 3.0 4.0 0.5 1.0 2.0

-0.5 -1.0 -1.0 -1.5 -2.0 -2.0 -2.0 -2.5 -2.5

-3.0



Corresponding KCT treeA

B C

D E F G

H I J

K

5 6 7 8 1 2

3 4

10 11

9 12

13

15 16 19

14 17 18: Non-leaf node

: leaf node



Insertion possibilities1. A leaf node is full and a new object is to be inserted

in that leaf node. In such case, an insertion results in overflow and, therefore, the node must split. As a result, a new non-leaf node and two leaf nodes are constructed and linked to that part of the tree. We also need to train the non-leaf node to get weight values.

2. When the node in which the object has to be inserted has only one object in it. In this case, object can be simply added into node without any additional cost.



Deletion possibilities1. When the sibling node is full-leaf node or non-leaf

node and we delete an object from full leaf node delete object from that node.

2. When the sibling node is leaf node with an object and we delete an object from full- leaf node delete object from that node, combine two leaf nodes, delete its parent non-leaf node, and connect remaining full-leaf node to deleted non-leaf node's parent node.



Deletion possibilities – contd.3. When the sibling node is full-leaf node and we

delete an object from leaf node with itself delete object from that node, delete its parent non-leaf node, and connect remaining full-leaf node to deleted non-leaf node's parent node.

4. When sibling node is non-leaf node and we delete an object from leaf node with one object Delete an object from that node, delete its parent non-leaf node, and connect deleted non-leaf node's child node to deleted non-leaf node's parent node.



Algorithm for retrieving images// Search an object with feature F using KCT1. b block containing root node of a KCT2. read block b3. while (b is not a leaf node of the KCT) do4. next recall Backpropagation using weights

in block b5. b next6. read block b7. search block b for the most similar object with

feature F // search leaf node8. if found then9. read index file block; display images with object


Experimental Results

Experimental SetupEnvironment: PC with Microsoft Windows 98Language: C / C++Images

Normalized to grayscale and 128 x 128Grayscale images to binary images for chain-codes

Data set size100 original images5 variants involving translation, rotation and scaling.


Experimental Results (contd.)

Sample image shapes and their seven moment invariants

a b c d

1 0.2136250052

0.2202478858

0.3404800296

0.6092586028

2 0.0007960114

0.0031965503

0.0836683672

0.2806988755

3 0.0061832619

0.0010778334

0.0009795464

0.0710062966

4 0.0000318972

0.0001001921

0.0004405216

0.0345626089

5 -0.000000014

1

-0.000000024

2

0.0000002893

0.0017118779

6 -0.000000892

1

0.0000047618

0.0001274228

0.0183105689

7 0.0000000000

0.0000000222

0.0000000000

0.0000339550



A subset of grouping results using database images at the root level of KCT. Objects in top row occupy the left subtree while the bottom row objects become right subtree.



Grouping results with database images at the 3rd level of KCT. The top row of objects forms the left subtree of KCT while the bottom row is the right subtree.



Results of sample queries. Query shape is given in row 1, column 1 of each image while the retrieved images include the query shape.




ConclusionsPresented a moment invariants based image indexing scheme.Chain codes are used to reduce size of databaseIndexing is based on K-means clustering with k = 2 and Backpropagation to get weights for each node.Retrieval of images was based on finding the leaf node that includes similar images.Limitation: Small image collection and limited training data.

Documents

Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada