Error=15.98m

Error=29.82m

3. Single Image Localization

Accurate Image Localization Based on Google Maps Street View Amir Roshan Zamir† (aroshan@cs.ucf.edu), Mubarak Shah† (shah@cs.ucf.edu)

†University of Central Florida1. Problem Accurate Image Localization:

Finding Image Location in Terms of Longitude and Latitude Accuracy Comparable to Hand-Held GPS Devices

Applications: Vision-Based Navigation, Photo Organization, Crime-Scene Localization, etc.

3.2 Smoothing by Gaussian Attenuates Solitary Peaks Amplifies Multiple Close Peaks

4. Image Group Localization Geolocating a Group of Query

Images Accurate Location of Each Query

Image

3.3. Confidence of Localization (CoL) A Parameter Representing The Reliability of

Localization Task CoL=Kurtosis (Normalized Fourth Central Moment) of

Normalized Vote Distribution Function

2. Google Maps Street View Dataset Collected From Pittsburgh, PA and Orlando, FL Automatically. 4 Side Views And 1 Top View Per Place Mark. Total Number of 100,000 Images.

Group of Query Images

Individual Localization

Localization of Other Queries in Individual

Neighborhoods

Selection Based on CoLgroup

Location of Each Image

Input GPS-Tagged

Reference Images

Compute SIFT Descriptors for Interest Points

Index Using FLANN Tree

Block Diagram of Localizing a Single Query

Block Diagram of Processing The Dataset

Dataset (Green) and Test Set (Red), Orlando, FL Dataset (Green) and Test Set (Red), Pittsburgh, PA

Side Views Top View

V flag (d i )={1 ;¿|d i − NN (d i , 1 )|∨ ¿|¿d i − NN (d i , min { j })|∨¿<0.8 ¿

¿ 0 ;∀ j→∨Loc (NN (d i , 1 ))− Loc (NN ( d i , j ))∨¿ Dotherwise

Error1=10.2m

Error2=15.7m

Error3=11.4m

Input Query Image

Compute SIFT

Descriptors for Interest

Points

Query The Reference Tree and

Accumulate Votes

Geospatial Pruning

Smooth by Gaussian

Select Image with Highest Number of

Votes

CoLgroup=0.14k

CoLgroup=2.1k

CoLgroup=0.23k

CoLgroup (S )=∑i=1

N CoLi

N

Single Image Localization Results

5. Results Test Set: 521 GPS-Tagged User-Uploaded Images Downloaded From Panoramio,

Picasa, Flickr, etc. 311 Images as Test Set of Single Image Localization, 210 Images as Test Set of

Image Group Localization.

Image Group Localization Results

Illustration of CoL Value vs. Accuracy Breakdown of Group Image Localization Results Sample User-Uploaded Images in Test Set Along With Their Longitude () And Latitude ()

V smoothed(𝜆′ ,𝜑 ′ )=∑𝜆∑𝜑

e−( 𝜆

2+𝜑2

2𝜎 ′ 2 )V (𝜆′ −𝜆 ,𝜑 ′ −𝜑 )V flag(𝜆′ − 𝜆 ,𝜑 ′ −𝜑 )

❑

CoL=Kurt (V smoothed)=− 3+ 1𝜎 4 ∑

𝜆=−∞

+∞

∑𝜑=− ∞

+∞

[(𝜆−𝜇𝜆)2(𝜑−𝜇𝜑)

2]V smoothed (𝜆 ,𝜑)❑

Gaussian Smoothing

Kurt=1.18 Kurt=233.42 Kurt=29865.7

More-Peaked Vote Distribution Results in Higher Reliability in Localization &

Higher Kurtosis

3.1. Geospatial Pruning Helpful When:

I: Reference Images Have Overlap in Scene II: There Are Repeated Structures Such as in Urban

Area

Descriptor Ratio: 0.95

GPS Distance: 9m

Descriptor Ratio: 0.87

GPS Distance: 13mDescriptor Ratio: 0.56GPS Distance: 414m

1st NN 2nd NN

4th NN 3rd NN

Sample Query Interest Point

I: Reference Images Have Overlap in Scene II: Repeated Structure in Urban Area

Sample Query Interest Point

Descriptor Ratio: 0.44GPS Distance: 919m

Descriptor Ratio: 0.91GPS Distance: ~0m

Descriptor Ratio: 0.94GPS Distance: ~0m

2nd NN3rd NN

4th NN 1st NN

Retrieved ImageError =17.8m

Query Image Extension To Videos: Using Temporal Continuity of Videos Red: Found Trajectory, Green: Ground Truth

Error=44.77m

6 Sample Localization Results. From Left to Right, First Column: Query, Second Column: Vote Distribution, Third Column: Retrieved Image, Forth Column: Ground Truth in Green and Found Location in Blue on The Map Along With Error.

Error=39.61m

Error=10.87m

Error=12.95m

Query Vote Distribution

Retrieved Image Map Query Vote Distribution

Retrieved Image Map

=40.441191=-80.006622

=40.438880=-80.001690

=40.440116=-80.004033

=40.442799=-79.998840

=40.441421=-80.002497

=40.440645=-80.002112

=40.442565=-80.000493

=40.441809=-79.998938

=40.443281=-80.000584

=40.441745=-79.998760

=40.441559=-79.996395

=40.443652=-80.002507

=40.439211=-80.004400

=40.441978=-79.999055=40.441127

=-80.002821

=40.440715=-80.002836