Interactive Video Search and Browsing Systems

Interactive Video Search and Browsing SystemsMarco Bertini, Alberto Del Bimbo, Andrea Ferracani, Daniele Pezzatini

Universita degli Studi di Firenze - MICC

The search engine

The Orione search engine has different query modalities and visualizations, resulting in an advanced tool forretrieval and exploration of video archives.

It uses an ontology created semi-automatically from a flat lexicon, using WordNet to create concept relations.

Automatic video annotations are provided by a system based on the bag-of-words approach, that exploits SIFT,SURF and MSER visual features and the Pyramid Match Kernel.

The web-based user interface

• Simple or advanced search interface tobuild composite queries that may includeBoolean and temporal operators, metadata andgeo tag.

• Integrated tool that allows to browse videoarchives navigating through the relationsbetween the concepts of the ontology.

• Manual annotation tool to providefree-text, ontology-based and geographicalannotations that can be used for correcting andintegrating automatic annotations.

The multitouch user interface

• Users can browse the ontology structurein order to select concepts and start the videoretrieval process.

• Users can inspect the results returned bythe search engine and organize them accordingto their specific purposes by performing simplegestures.

• The user interface adopts some visualizationprinciples derived from the discipline ofinteraction design and is equipped with a setof interaction functionalities to improve theusability of the system.

Usability test

http://www.micc.unifi.it/ This work was partially supported by the EU IST IM3I project (contract FP7-222267). http://www.im3i.eu