Upload
srijan-kumar
View
140
Download
1
Tags:
Embed Size (px)
Citation preview
IIIT HYDERABAD
Towards Exhaustive Pairwise Matching In Large Image Collections
Kumar Srijan : IIIT – Hyderabad C.V. Jawahar : IIIT - Hyderabad
IIIT HYDERABAD
Community Photo Collections
• Anyone can take photographs!
• Sharing photographs is easy!
• Searching for photographs is easy!
• How to organize these collections ?
IIIT HYDERABAD
CPCs - Characteristics
• Totally unstructured collections!– Different resolutions– Different viewpoints…
• Contents unknown– Difficult to find relevant information
• Opportunity – Discover relationships between images which are
hard to find manually
IIIT HYDERABAD
Matching Images
• Establish correspondences – Ex: Matching SIFT features
• Discover transitive relationships– Discover Popular Images / Locations / Paths…– Filter irrelevant images
• Applications : – Automatic Annotation transfer – 3D reconstruction…
IIIT HYDERABAD
Representing Matching Images
• Image Match Graph – Images Nodes – Image Match Edges– Allows queries such as :• Connected components• Shortest path
– Image Match : Ascertained by verification• RANSAC based epipolar geometry estimation• Cosegmentation .. etc.• Verification is Expensive
IIIT HYDERABAD
Creating Match Graphs
• Collection of N images– Verify N x N matches : Ideally!– Impractical!
• Practical– Filter– Verify only promising candidates– Successful!
• Exhaustive Pairwise matching – A Step!
IIIT HYDERABAD
Current Techniques
• Image Retrieval– Quantization Image Features– Inverted Index Database Images– Filtering Reject lot of Matches– Verification On Top scoring matches– O(N) time for a single querying
• Min Hash – Grow seeds by Image Retrieval
• Image Webs – Compute cluster skeletons by Image Retrieval
IIIT HYDERABAD
Exhaustive Pairwise Matching - Why?
• Image matches beyond human scope!– No ground truth for large collections!
• Discover low scoring matches!
• No effort spent in Designing filter!
IIIT HYDERABAD
Exhaustive Pairwise Matching – How?• Image Retrieval• ‘ Query each image in turn ’ – Verify all potential matches• No shortlists
– Verification doable from Index retrievals• Index Geometry • More discriminative features
– IF : Constant time queries• THEN : Linear time complexity
IIIT HYDERABAD
High Order Features
• High Order Features– Combine nearby features• Primary with Secondary Features• Encode Affine Invariants
– Relative Orientation and Scale– Normalized distance– Baseline orientation
– Tuple• <VWp,VWs,g1,g2,g3,g4>
– Huge Feature Space• Helps in faster queries!
IIIT HYDERABAD
Constant Time Queries
• Regular Inverted Index – Fixed size : # Visual words• Posting lists grow with Database
• Our Case : Huge Feature Space– Hash Functions Reproject to custom size• Database size• Constant sized posting lists• Constant time queries
IIIT HYDERABAD
Bloom Filters
• Bloom Filter– Set Membership• Present• Not Present
– Bit array(m) – Hash Functions(k)– Elements(n)– False Positives• C is not present• D is a False Positive
IIIT HYDERABAD
Inverted Index over Bloom Filters• Database of N images• N Bloom Filters – Index and Query N images
• Inverted Index – Bloom filter size(m) Proportional to N• Use one Hash function (k = 1) No list Intersections!• Constant time queries
IIIT HYDERABAD
Spatial Verification
IIIT HYDERABAD
Results
• UK benchmark• 73.2 % recall
• Oxford Buildings– 78 Mn Hof– 2^25 index size– Extracting features : 27 minutes – Querying features : 2 minutes
IIIT HYDERABAD
• 317 clusters with 1375 images• Small Clusters
• Errors
IIIT HYDERABAD
Effect of doubling inverted index size
• On oxford 105k (100k + 5k Oxford Buildings)– 1480 Mn HoF
– While using 16 x 2^25 = query time = 2.5 hours– While using 32 x 2^25 = query time = 1.8 hours– 2147 clusters 7198 images– Largest 2265
IIIT HYDERABAD
Conclusions
• Indexing geometry gives a Bigger feature space
• Bloom Filters can be used for Match Graph construction
• Exhaustive Pairwise Matching is Feasible with efficient indexing