MULTIMEDIA DATABASESAND MPEG7

Rahmi Volkan Başar

Department of Computer Engineering

METU

May, 2013

Multimedia Databases

• Introduction

• Capabilities of DB Types

• Search on MMDB

• Multimedia Content Description

• Research Fields

Multimedia Data

• Text: using a standard language (SGML, HTML)

• Graphics: encoded in CGM, postscript

• Images: bitmap, JPEG, MPEG

• Video: sequenced image data at specified rates

• Audio: recordings in a string of bits in digitized form

Database vs Multimedia Database

• Databases– well structured data organization

– efficient storage of large amounts of data

– querying

– transactional support for concurrent users

– numbers, strings

• Multimedia Databases– large content

– different structures

– not easily searched/queried

Use Cases

• Repositories: central location for data maintained by DBMS, organized in storage levels

• Presentations: delivery of audio and video data, temporarily stored, ‘VCR-like functionality’

• Collaborative: complex design, analyzing data

Capabilities

• Relational Databases

– Atomic / Tables

– Data relation – Common Foreign Keys

– Record: Content – No meta information

– A predefined set of domains for columns

• Hard to extend

• BLOB data type exist

Capabilities

• Object Oriented Databases

– Schema is “Class”

– All data is “Object”

– References

– New data types

• Easy. New class is a new data type.– Appropriate for multimedia data

Capabilities

• Object Relational Databases

– In addition to RDBMS

• Object references

• New types– Multimedia

– MMDBMS

• Extensible ORDBMSs

Search

• Collection of data. How to search?

– Any standards?

– Workarounds?

• Search: Retrieve similar images…

– Fast, Correct

• Content-based

– New techniques?

Search

• Content Based Retrieval Facilities

– Supported by MMDBMS

• Organize and Manage accordingly

– Compare based on a number of features

• Shape/Color/Texture

• Meta-Data?

– Always.

Content Based Retrieval

• Accurate representation of the multimedia objects in the database

– For accuracy and efficiency

– Combination: Different features

• Similarity Search

– High-dimensional feature vectors

• Special multi-dimensional indexing structures

• Dimension reduction methods.

Multimedia Content Description Standard: MPEG-7

• Influential XML based multimedia meta-data standard• Description of the storage media:

– Format, Image Size, Audio Quality, Video Frames etc.

• Creation and production information:– Creation date and location, title, genre, etc.

• Content semantic description:– Events, concepts, objects, etc.

• Content structural description:– Shot and key frames with color, texture and motion

features, etc.

• Metadata about the description:– Author, version, creation date, etc.

MPEG-7

• Expression of multimedia data

• Missing: Search for Implicit Data

– The meaning of the structure: Not expressed

– Ex. A video: length, format, name, dates etc.

• Gender: Documentary, Interview, Movie

• Theme: Science, Sports, Horror

• No consideration on search engines

MPEG-7

• Search:

– XPath, XQuery

– Semantic Views Query Language

Simple MPEG7 Example<Mpeg7>

<Description xsi:type="SemanticDescriptionType">

<Semantics>

<Label>

<Name> Car </Name>

</Label>

<Definition>

<FreeTextAnnotation>

Four wheel motorized vehicle

</FreeTextAnnotation>

</Definition>

<MediaOccurrence>

<MediaLocator>

<MediaUri> image.jpg </MediaUri>

</MediaLocator>

</MediaOccurrence>

</Semantics>

</Description>

</Mpeg7>

MPEG7 Details

• Standardizes 3 parts:

– Description tools

• Descriptors (D)

• Description Schemes (DS).

– Description Definition Language (DDL)

• To specify these schemes

– System tools

MPEG7 Details

• Descriptors (D)– Representation of a feature

• Syntactic and Semantic

– Low-level audio or visual features • Color, motion, texture etc

– Audiovisual content • Location, time etc

• Objects can be described– Several descriptors.

MPEG7 Details

• Description Schemes (DS) describe

– Specification of the relations

• Between Descriptors

• Between Description Schemes

– Relations can be structural and semantics

– High-level audiovisual (AV) features

• Regions, segments, events etc

MPEG7 Details

• Description Definition Language

– Based on XML

• Defines the structural relations between descriptors– Creation and modification of description schemes

– Creation of new descriptors.

MPEG7 Details

• System Tools

– Deal with Descriptor management

• Binarization

• Synchronization

• Transport

• Storage

MPEG7 Details - Overview

MPEG7 Details

• Next Slide

– Description of a Video Segment

MPEG7 Details

• How to extract semantics?

– i.e. Intelligent Information Retrieval

– Drawback of the standard

– Ontology help required:

• Domain Specific Ontology (Football, Location)

• Automatically extract information

• Use for a better search result

Research Fields

• Design: still in research

• Queries: techniques need to be modified

• Rest:

– Modeling: complex objects, wide range of types

– Storage: representation, compression, buffering during I/O, mapping

– Performance: physical limitations, parallel processing

• Thank you!

• Questions?

References

• Wikipedia: Various Pages• Computer Science and Engineering Department

Resources:– University of Notre Dame– Northumbria University– Carnegie Mellon University– Boston College– Simon Fraser University– Georgia Institute of Technology

• Interview with A. Anil Sinaci