Preliminary Exploration of the Use of Geographical Information for Content-based Geo-tagging of Social Video

08-04-2023

Challenge the future

DelftUniversity ofTechnology

Preliminary Exploration of the Use of Geographical Information for Content-based Geo-tagging of Social VideoXinchao Li, Claudia Hauff, Martha Larson, Alan Hanjalic

2Visual similarity measures for semantic video retrieval

System Overview

• Goal

derive location information from the visual content of videos

• Challenge

• no tags: 35.7%, only one tag: 13.1%

• improve metadata-based system

System Overview


Great Victoria Desert

South Pole

System Overview

• Assumption

divide the world map into regions that have a high within-region visual stability and a high between-region variability


Different Division Methods

• Baseline



• Temperature Data based



• Temperature Data based


6 temperature regions: from -20◦C to 40◦C with 10◦C intervals.


• Biomes Data based



Run Results

Run Results


Run Results

Run Results

22 Biomes classification: 12.17% (random, 4.55%)


Discussion

• Visual Content of Test Videos

• Indoor (42%)

• Outdoor Event (32%)

• Normal Outdoor (26%)

• Visual Content of Training Photos

458 photos from the 3M training set

• Indoor (27.5%)

Discussion

500 videos from the 4182 videos (12%)


Discussion

Indoor (42%)


Discussion

Outdoor Event (32%)


Discussion

Normal (26%)


• Recall our assumption

“we can divide the world map into regions

that have a high within-region visual stability and a

high between-region variability.”

• indoor images are noisy information

• Only use outdoor videos to train and test

Discussion

Conclusion and Future work


Thank you!

[email protected]