FUZZ-IEEE 2003 1

Looking Beyond Region Boundaries:A Robust Image Similarity Measure Using Fuzzified Region Features

Yixin Chen and James Z. Wang

The Pennsylvania State Universityhttp://www.cse.psu.edu/~yixchen

FUZZ-IEEE 2003 2

Outline

Introduction

A robust image similarity measure

Experimental results

Conclusions and future work

FUZZ-IEEE 2003 3

Introduction

The driving force InternetStorage devicesComputing power

Two approachesText-based approachContent-based approach

FUZZ-IEEE 2003 4

Text-Based Approach

Input keywords descriptions

Text-BasedImage

RetrievalSystem

Elephants

ImageDatabase

FUZZ-IEEE 2003 5

Text-Based Approach

Index images using keywords(Google, Lycos, etc.)Easy to implementFast retrievalWeb image search (surrounding text)Manual annotation is not always availableA picture is worth a thousand wordsSurrounding text may not describe the

image

FUZZ-IEEE 2003 6

Content-Based Approach

Index images using low-level features

Content-based image retrieval (CBIR): search pictures as pictures

CBIRSystem

ImageDatabase

FUZZ-IEEE 2003 7

A Data-Flow Diagram

ImageDatabase

FeatureExtraction

ComputeSimilarityMeasure

Visualization

Histogram, colorlayout, sub-images,regions, etc.

Euclidean distance,intersection, shapecomparison, region matching, etc.

Linear ordering,Projection to 2-D, etc.

FUZZ-IEEE 2003 8

Region-Based Approach

An image is viewed as a collection of regionsRegions Objects Semantics

Difficulties Image segmentationRegion matching

Our goalRobust to image segmentation

FUZZ-IEEE 2003 9

UFM: Unified Feature Matching

MotivationHuman can identify complex objects in a

collection of points even when those points cannot always be assigned unambiguously to objects

HypothesisAllowing for blurry boundaries between

regions may increase the robustness

FUZZ-IEEE 2003 10

UFM: Unified Feature Matching

UFM

Fuzzified region features

Region matching fuzzy logic operation

Integrate region-level similarities

FUZZ-IEEE 2003 11

Image Segmentation

Wavelet,RGBLUV

K-means

Original Image

Segmentation Result

(Centroid, Inertia)=region features

FUZZ-IEEE 2003 12

Image Segmentation

Segmentation examples

FUZZ-IEEE 2003 13

Fuzzy Feature Representation

A segmented image regions {R1,…,RC}feature sets {F1,…,FC}.

Region Rj is represented by a fuzzy feature with membership function of the form

FUZZ-IEEE 2003 14

Region Matching

Two regions “AND” or intersection of two fuzzy sets

A region and an imageAn image “Union” of all its regions

Two imagesA vector of similarities

UFM measureA convex combination of region level

similarities

FUZZ-IEEE 2003 15

Experimental Results

Query Examples from 60,000-image COREL DatabaseNatural Out-Door SceneHorsesPeopleVehicleFlag

Natural Outdoor Scene

15 matches out of 19

People

15 matches out of 19

Horses

19 matches out of 19

Flag

19 matches out of 19

Vehicle

17 matches out of 19

FUZZ-IEEE 2003 17

Experimental Results

Robustness to Image Alterations Intensity VariationSharpness VariationColor DistortionCroppingShiftingRotation

FUZZ-IEEE 2003 19

Experimental Results

Performance on Image CategorizationSubset of the COREL database formed by

10 image categories, each containing 100 images

Africa, Beach, Buildings, Buses, Dinosaurs, Elephants, Flowers, Horses, Mountains, and Food

Comparison with EMD-based color histogram approaches

FUZZ-IEEE 2003 20

Experimental Results

Average Precision

FUZZ-IEEE 2003 21

Experimental Results

Robustness to Image Segmentation

FUZZ-IEEE 2003 22

Conclusions

A robust image similarity measureFuzzified region featuresRegion matching

Good retrieval performanceRobust to image segmentationRobust to image alterations

FUZZ-IEEE 2003 23

Future Work

Improving image segmentation

Generating fuzzy features directly form segmentation

Utilizing location information