Reverse image search (using matlab®)

Reverse Image Search (using MATLAB®) and Implementation for Web based Application

Abdullah (A4LE-38) Faisal Jamal (A4LE-33)

Under the Guidance of:Dr. A. A. Moinuddin

Dr. Omar Farooq

Contents• What we want to do

– Reverse Image Search v/s Conventional Search• Why we want to do it

– Significance• How we are going to do it

– Analysis• Statistical Properties• Object Oriented

– Tagging– Searching

• How far we have done– Analysis of statistical properties– Basic Algorithm for Tagging

• What is Left– Object Oriented Analysis & Related Tagging– Searching Algorithm– Implementation for Web Based Application

What we want to do

• Conventional Image Search– Input - Text– Search Based On -

• Name of the Image (Simplest)• PageRank (User Feedback)• Tagging

– Output – Relevant Images

What we want to do

• Conventional Image Search– Limitations

• Input text seldom gives specific results• Tagging depends highly on perception

What we want to do

• Reverse Image Search– Input – Image and/or Text

• Upload image• Enter relevant text

– Search Based on • Characteristics of the Image• Contents of the Image• Related and/or Generated Tags

– Output• Relevant Images• Relevant Data

– Address, Content, Etc.

Our Project

Why we want to do it

• Limitations of Conventional Search• Can be the next Big Thing in Searching on Internet• Can be extended to other media, like Music, Sounds, Videos,

etc.

Why we want to do it

• Example:

Image

• Where is the Image• What is in the Image• Who is in the Image• Find Related Images• Find Similar Images• ….

Questions

How we are going to do it

Basic Steps:

Database Creation

Searching Algorithm

Web Implementation


• Database Creation:

Image Acquisition

Statistical Analysis

Object Oriented Analysis

Tagging and Creating

Database


• Image Acquisition:– Currently, done for finding supported images in a

computer– Can be expanded for internet databases using

Crawlers– Currently supported images:

• .jpg , .tiff , .bmp , .png , .gif


• Analysis– Of Statistical Properties– Using Pattern Recognition (Object based)

• Tagging – Deriving an Alphanumeric Code for each image and saving in a Database

• Searching Algorithm• Web Implementation


Database of Tags

Database

Database Database

Database

Query Output

How far we have done

• Image Acquisition from System• Statistical Analysis

– Converting all images to Gray scale– Converting this to a Unique Size– Deriving Relevant Statistical Properties

• Histogram• Contour Plot

– Analyzing the Properties


• Histogram: Graphical Representation of color distribution in a digital image


• Contour Plot: Plots showing lines depicting boundaries of colors


• Analyzing the Properties– Variance, Skewness and Kurtosis of the Histogram– Mean, Variance, Skewness, and Kurtosis of the

Contour Plot– 2-D Mean of the Image

• Each Analysis result was saved in an ‘index table’, for each image, along with their location.


• Example:

Input Image Entry in Table


• Tagging:– Assigning an Alphabet to the value of each

analyzed property– Alphabets for each property for a single image

forms the alphanumeric ‘tag’ code for the image– Central database has the list of these

alphanumeric tags, along with address of the image

• No need to save the complete image


• Basic Algorithm for Tag Generation:– Each analyzed property further analyzed for range of

variation– Assigning alphabets keeping in mind:

• Most images have their property lying in a localized range• Similar images may vary in the value of certain properties,

although by a small amount• Certain unrelated images may also lie in the same localized range

– Taken care of by variations in other properties• All databases do not have the same range of variation

– Taken care of by implementing a centralized analyzing and tagging system, or, by predefining the supported ranges


• Basic Algorithm for Tag Generation:– Our Proposal:

• In a set k:• Divide the elements into two parts, P1 and P2 according to: (for i =

1,2,3, … , end(k))P1(j) = indices{value(k(i)) > {minval(k) + [(maxval(k) – minval(k))/2]}}P2(j) = indices{value(k(i)) < {minval(k) + [(maxval(k) – minval(k))/2]}}

• Further divide the part which has most number of elements, as above, considering the values of k whose indices are stored in that part

• Repeat the above step further, every time considering the part with most number of elements, unless 256 parts have been generated


min

max max max

min min

m1 m1

m2

n

n1

n2 n2

n3

n4

…


• Basic Algorithm for Tag Generation:– Our Proposal:

• Each partition has a unique range of variation, further assuring that ranges with most number of elements are divided in most number of parts

• Each element lying in a single partition is assigned a unique alphabet from the ASCII set

• Partition with larger number of elements is given the highest priority alphabet


• The above process is repeated for each property in the ‘index table’

• Finally, alphabets for each property form an alphanumeric code representing the tag

• Still in the phase of development– Keeping in mind necessary changes when more properties

are added for analysis

What is left

• Object Oriented Analysis & related tagging– Statistical analysis can act as the first stage– Would require feature extraction and pattern recognition– Fairly complex, and still in the study phase

• Searching Algorithm– Can be made fairly easy by using alphanumeric tags– Plan to use ‘user feedback’ in the training phase

• Implementation for Web Based Applications

Acknowledgements

• R. C. Gonzalez, R. E. Woods, S. L. Eddins; “Digital Image Processing Using MATLAB”; Pearson Education Inc. ; 2004

• J. Z. Wang, G. Wiederhold, O. Firschein, S. X. Wei; “Content-based image indexing and searching”; Int J Digit Libr (1997) 1: 311±328; 1997

• P. K. Mukherjee, M. Nasipuri, D.K. Basu, M.Kundu ; “Indexing and Searching in Multimedia Database Management System”; Indian Institute Oftechnolooy. Kharagpur 721302. December 20-22.2004

• Y. Jing, S. Baluja; “PageRank for Product Image Search”; WWW 2008 / Refereed Track: Rich Media; April 21-25, 2008.

• Principal Component• U. Sinha, H. Kangarloo; “Analysis for Content-based Image Retrieval“;

RadioGraphics 2002; 22:1271–1289• Dr. A. A. Moinuddin, Dr. Omar Farooq; Department of Electronics Engg,

ZHCET, AMU.

Thank You