11/1/2005 StreetTIVO

StreetTIVO

Arjen P. de Vriesarjen@acm.org

11/1/2005 StreetTIVO

N5: Semantic Access

• Main goal:– Improve search system engineering efficiency

• Approach:– Given

• Declarative specification of collection, background information, and context

• Declarative specification of retrieval model

– Generate a parameterised search system

11/1/2005 StreetTIVO

Collection +

Search Strategy

System

11/1/2005 StreetTIVO

TRECVID example

• A query– //VideoSegment[

about(./transcript, “airplane taking off”) and about(./keyframe,

airplane-in-sky.jpg)]

• and a retrieval model specification… [avg(

[log((0.9*psd(s,d)) +(0.1*ps(s))) | s<$NSamples])

| d<$NDocs ] …

11/1/2005 StreetTIVO

Spiegle Prototype

Retrieval model specification (RAM) +

Collection schema (XML)+

Cirquid XML-IR (TIJAH) +

MonetDB/XQuery

11/1/2005 StreetTIVO

Waterland Demo

• Demonstration system querying “De Digitale Voorziening” at the Dutch broadcasters

MonetDB/XQuery

TIJAH 4 VIDEO

(IR engine)

NE

XI.c

gi

11/1/2005 StreetTIVO

11/1/2005 StreetTIVO

let $NEXI_QUERY:="//segment[about(.,metro)]"let $ranking := doc(fn:concat($NEXI_CGI,$NEXI_QUERY))let $vids := distinct-values($ranking//Result/@docID)

let $vranking :=element Vranking { for $v in $vids let $vshots := $ranking//Result[@docID=$v] let $vscore := for $f in $vshots/@score return $f cast as xs:double let $MAXvscore := max($vscore) let $MINvscore := min($vscore) order by $MAXvscore descending return element Vid { attribute id {$v}, attribute score {$MAXvscore}, for $s in $vshots[@score > $MINvscore] return $s }}

TIJAH’s NEXI query language ~ XPath+about

MonetDB/XQuery to present result fragments grouped by

video

11/1/2005 StreetTIVO

StreetTIVO

• Golden demo

“A demo not seen

anywhere else”

•Co-operation between projects:•N1: Multimedia Features Twente, UvA, Delft

•N3: Ambient Data Management CWI, Philips, Twente

•N5: Semantic Access CWI, Twente

11/1/2005 StreetTIVO

StreetTIVOPVR

+ MultimediaN Features • Teletekst• ASR (automatic speech recognition) • Scene- and/or shot-segmentation • Detecting repeated objects • Tracking a library of famous people• [your fav “detector” here…]

+ Information Retrieval

11/1/2005 StreetTIVO

StreetTIVO

Idea: P2P Collaborative Multimedia Indexing

Work distribution Result exchange Fault tolerance

Challenge: feature detection takes long time

11/1/2005 StreetTIVO

StreetTIVO

CWI Seed farm

10 workstations

11/1/2005 StreetTIVO

StreetTIVO

Work distribution Result exchange Fault tolerance

AmbientDB

StreetTIVO aims for the world-wide

MythTV community (starting in NL).

Scalability and robustness!

CWI Seed farm

10 workstations

11/1/2005 StreetTIVO

StreetTIVO Architecture

StreetTIVOmodule

MultimediaFeature Detection

AmbientDBP2P

database

TIJAH 4 VIDEO

(IR engine)

XQuery/embedded

SO

AP

/X

Qu

ery

Extending the search functionality corresponds to installing a new

detector and updating the TIJAH 4 VIDEO XQuery queries

11/1/2005 StreetTIVO

Key Persons

Thijs Westerveld (CWI) Multimedia search

Matthijs Mourits (CWI) mythTV boxes

Jennie Zhang (CWI) P2P data management

Djoerd Hiemstra (CTIT) TIJAH

Roeland Ordelman (CTIT) ASR (Dutch)

YOUR NAME IS WELCOME HERE!!!