Notube slides-120130071559-phpapp01

1 New trends in television: social and semantic

Project overview and results

TV on the Web: growing trend

TV on the Web: channel explosion

Source: Nielson Three Screen Report, March 2010

From „ Mobile Shopping Framework: The role of mobile devices in the

shopping process” by Yahoo! and the Nielson company, January 2011http://advertising.yahoo.com/industry-knowledge/mobile-shopping-insight.html

Including the Web in your TV

Yahoo! launches ConnectedTV platform for Web-based widgets on TV (e.g. Flickr, YouTube, facebook, twitter) – Jan 2009

Augmenting TV with the WebBlinkx BBTV makes

video information

and its textual

transcript clickable,

and links to Web

sources such as

IMDB and Wikipedia

www.blinkxbbtv.com

Also Mozilla has a

project on showing

content around

videos using HTML5

www.drumbeat.org

Some Web-TV solutions today

Stand alone boxes such as

• TiVo – original DVR, added on-demand video, YouTube, music and photos from the Web

• Boxee – STB offering its own store of apps

• AppleTV – relaunched as $99 product tied to iTunes content, and iPhone/iPad integration

+ Hybrid boxes tied to specific IPTV providers

+ Games consoles (Sony, Microsoft, Nintendo) also adding Internet and video services to TV!

Some Web-TV solutions today

First TVs with

integrated Web

and individual

app platforms

in 2011.

Future TVs will

be „connected“

as standard. LG SmartTV, pic courtesy

http://www.wired.com/gadgetlab/2011/01/lg-smart-tv/

State of the art in TV

• TV content shifting to the Web as delivery platform

– An explosion in available content at any time

• Web content shifting to the TV as augmentation of the TV experience

– An explosion in additional content at any time

Limitations of today‘s TV

• Too much content in one place

– How to find what you want to watch? Sort between live TV, TV on demand, archives, video portals and P2P-TV

• Too much functionality at any one time

– The whole Internet while you watch TV. But what do viewers really want to be able to do additionally (parallel) to watching TV?

Social TV

• Integrate the TV experience with the so-called Social Web

– Who are my friends and what do they watch?

– What do my friends like -> maybe I‘d like it too

– Where are my friends now -> connect via the shared TV experience

• Key goal for social TV

– Enhance my TV experience through my friends‘ TV experience

Semantic TV

• Add formal semantic descriptions for

– TV programmes

– TV schedules (EPGs)

• Link those descriptions to other semantic data on the Web, cf. Linked Data

• Two key use cases for semantic TV:

– Filtering of TV content -> personalisation, recommendation

– Augmentation of TV content with Web data

NoTube project

• Integrating TV & Web with help of semantics

– Open and interlink TV content in a Web fashion with Linked Open Data

• Putting the user back in the driving seat

– Connect multitude of distributed personal data with explicit semantics

• TV is not bound to the device– Computer as TV & vice versa– Mobile device as remote control

NoTube partners

Bridging Web and TV cultures

Rest of this slideset

• Technological background (Semantic Web, Linked Data)

• Semantic annotations for TV data (semantic TV)

• Extracting knowledge from my activities and social graph (social TV)

• TV content recommendation (personalized TV)

• The further future: finally … interactive TV

“If computers can understand the

meaning behind the information

they can

learn what we are interested in

better help us find what we want.”*

* Source: http://www.slideshare.net/HatemMahmoud/web-30-the-semantic-web

(1) Semantic Web, Linked Data

The Semantic WebThe vision of what was termed the “Semantic Web“ first came to public

attention through an article in Scientific American in May 2001.

* Source: T. Berners-Lee, J. Hendler, O. Lassila; “The SemanticWeb”, Scientific American, 284(5):34–43, May 2001.

“The Semantic Web is not a separate

Web but an extension of the current one,

in which information is given well-defined

meaning, better enabling computers and

people to work in cooperation.”*

HTMLHTML was too limited for Web documents – it is purely a presentation

format. The tags in HTML have no meaning outside how content should be

rendered in the browser, and so the meaning of the content must be

interpreted by a human, hence excluding any possibility of machine

processing.

James Bond

MI5

Her Majesty's Secret

Service

Secret HQ

007 England

James Bond MI5

Her Majesty's

Secret

Service

Secret HQ

007 England

<name>James Bond</name>

<fullname>Her Majesty's Secret

Service</fullname>

<address><street>Secret HQ</street>

<country>England</country>

</address>

</company>

The core idea of XML – Extensible Markup Language – is to provide for

definitions of markup which allows self-describing tags, i.e. tags which

describe the meaning of the content they mark up rather than its

presentation

James Bond MI5

Her Majesty's

Secret

Service

Secret HQ

007 England

RDFRDF provides a graph structure for making statements about things.

Individual things, and not just files, are given an URI identifier.

This is where the Semantic Web begins.

from is a child element of flight

(syntactic structure)

http://my.org/flightAI288

:from http://my.org/Vienna

:to http://my.org/Innsbruck

:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“

:currency http://my.org/euro

<flight>Flight AI288

<from>Vienna</from>-

<to>Innsbruck</to>

dep <dep>1.1. 1200</dep>

arr <arr>1.1. 1255</arr>

price <price>88€</price>

</flight>

from is a property of the resource

RDFSRDF Schema begins to formalise the meaning of things spoken about in

RDF on the basis of computational logic. RDFS permits simple ontologies

(models about concepts and their properties) to be defined, which can be

used to conclude new knowledge.

http://my.org/Vienna

is a http://my.org/City

http://my.org/City

subClass of http://my.org/PopulatedPlace

is a http://my.org/PopulatedPlace

:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“

OWLOWL broadens the possible expressivity of the ontology. This makes

richer models of knowledge about things possible, but at the cost of those

models being more complex for a computer to process.

isPlaceIn http://my.org/Austria

http://my.org/Austria

isPlaceIn http://my.org/Europe

isPlaceIn is a transitive property

isPlaceIn http://my.org/Europe

:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“

SPARQLThe final block of the Semantic Web that we will cover in this introduction is

SPARQL, the query language for semantic data using the RDF data model

(which includes OWL).

:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“

Is there a flight from Vienna to

somewhere in Austria for a price

under 100 euros?

SELECT ?flight

?flight :from http://my.org/Vienna

?flight :to ?place

?place :isPlaceIn

http://my.org/Austria

?flight :price ?price

?flight :currency http://my.org/euro

FILTER

(?price < 100)‏

Semantic Web principles• Every concept can be identified with URIs

• Resources and relationships are typed semantically

• Partial information is acceptable

• Absolute truth is not necessary

• Evolution as a development principle

Linked Data principles

• Use URIs as names of things

• Use HTTP URIs so that people can look up those names

• When someone looks up an URI, provide useful information

• Include links to other URIs, so that they can discover more things

Semantic Web vs Linked Data

“In contrast to the full-fledged Semantic Web vision, linked data is mainly about publishing structured data in RDF using URIs rather than focusing on the ontological level or inference. This simplification - just as the Web simplified the established academic approaches of Hypertext systems -lowers the entry barrier for data providers, hence fosters a widespread adoption.”

- Reference

Linked Data cloud

Linked Data for music & TV

DBPedia: Wikipedia as Linked Data

DBPedia Mobile

Pictures from revyu.com

Try yourself:

http://wiki.dbpedia.org/

DBpediaMobile

Resources and representations

http://dbpedia.org/resource/Berlin

.../page/Berlin .../data/Berlin

non-information resource

HTML representation RDF representation

Linking things, not documents

http://dbpedia.org/resource/ABBA

http://www.bbc.co.uk/music/artists/d87e52c5-

bb8d-4da8-b941-9f4928627dc8#artist

sameAs

Browsing things, not documents

http://dbpedia.org/resource/ABBA

http://dbpedia.org/resource/Knowing_Me%2C_

Knowing_You..._with_Alan_Partridge

themeMusicComposer

Asking for things, not documents

Which music artists have composed the theme music for a BBC comedy program?

(2) Semantic annotation for TV

• What can we annotate in TV?

– The program schedule

– The TV program

– TV program segments

• How can we annotate TV?

– Feature description (low level, analysis based)

– Metadata (date, creator, legal notice)

– Content description (title, summary, genre, concepts)

Why have metadata?

Archives from where content has to be found and retrieved have been the place where the need for accurate documentation first arose.

Broadcast metadata

• Data about data

– All digital resources (A/V, scripts, contracts, reports, pictures, etc.) are data

– Metadata is created at all stages in broadcasting from commissioning to playout

• Three main categories

– Administrative metadata

• Replacing project and asset management paperwork

– Technical metadata

• Format, processing, identification, location, database, network

– Descriptive metadata

• All asset related information, human readable

Need for common standards

Exchange of information

hampered by lots of proprietary

interfaces

TV Content

Creator 2

TV Content

Creator 3

TV Archive

NoTubeBroadcaster

Broadcaster

TV Content

Creator 1

Screenshot http://www.ifanzy.nl

EPG data

• An EPG is composed of two parts: content descriptions and broadcast description

• Content descriptions contain static data about television programmes such as a brand name (e.g. EastEnders), description or plot summary, type of programme, (e.g. series, movie, news), genre(s) (e.g. drama) actors, directors, recording data, etc.

• Broadcast description is expressed by variable data, such as channel (e.g. BBC ONE), format (e.g. 16:9) and broadcast media (e.g. digital television)

TVAnytime (1/2)

• Unique document structure

– Program description

– Program location

– Program segmentation

– User description & personalisation

– System aspects

– Content rights

TVAnytime (2/2)

• Advantages of TV-Anytime

– It is network and middleware independent

– Supports related material, segmentation, locators, group information etc.

• Applications of TV-Anytime

– ARIB

– DVB (MHP, DVB GBS, DVB IPI, DVB CBMS)

– Asian User Groups, Korea

– US’ Consumer Electronic Association

– HbbTV

TVAnytime schema

Other models in use

• egtaMETA - a unique metadata exchange schema dedicated for the exchange of ads between ads agencies and broadcasters. NoTube was an early tester of the schema in its personalised advertisements use case.

• BMF – an abstract semantic model designed for metadata exchange in the professional media production domain. ARD in Germany is starting to use BMF.

• Presto Space – format generated by the project of the same name to provide for digital preservation of audiovisual collections. Used by NoTube partner RAI.

Metadata interoperability via NoTube

http://notube.tv/tv-metadata-interoperability/ for more information

BBC /programmes

The BBC have made their EPG data machine-readable and published it on the Web

BBC /programmes: add .rdf

http://www.bbc.co.uk/program

mes/b00rl5y1

http://www.bbc.co.uk/program

mes/b00rl5y1.rdf

BBC /programmes ontology

From http://purl.org/ontology/po/

This may the first TV content

ontology, but certainly not the

Key organisations in the TV

standards domain are exploring

the publication of metadata in

RDF or SKOS:

• EBU (Core)

• TV-Anytime

• IPTC (NewsML)

The final step must be a

common shared ontology

integrating the different

schemas (cf.W3C Media

Ontology and API)

Channel identifiers

• Collected resolvable channel identifiers together with relevant metadata in RDF, e.g. 1700+ channel identifiers of Freebase

http://www.cs.vu.nl/~ronny/notube/tv-channels.rdf

Genre taxonomies

• BBC, TV Anytime, YouTube, IMDB, tvgids.nl …

• Convert them into RDF concepts and define SKOS relations between them, e.g. EBU has done this for the TV Anytime Classification scheme

Concept extraction

• NLP tools identify named entities in text and attach an unique identifier to them

e.g. OpenCalais, Zemanta

• Focus on key classes of entity such as person, place or organisation

• Use of Linked Data for common concept identifiers

• Ontotext developed specifically for TV metadata the tool LUPedia

LUPedia (http://lupedia.ontotext.com)

Concept extraction for TV

Linking TV content to Web content

starring

David Dickinson

Tim Wonnacott

birthplace

Barnstaple

(3) Extracting knowledge about the user

Idea: generating user profiles from data the user creates on the Social Web, and in this way facilitating a personalised TV experience without an intrusive user profiling process.

Facebook, Twitter & co.

Activity Streams

• RSS/Atom feeds include a title, description, link and some other metadata;

• Activity Streams extend this with a verb and an object type

– to allow expression of intent and meaning

– to provide a means to syndicate user activities

• Supported by Facebook, MySpace, Windows Live, Google Buzz and…

Getting TV into the Social Network„ BBC iPlayer adds

Twitter and Facebook to socialise TV”

– Share what you are watching on iPlayer

– Sync viewing with friends

– Real time chat

Techcrunch Europe, May 26 2010

TV viewer actions

• Recorded

• Consumed

• Loved

• Bookmarked

• …

Twitter activity

Bringing it all together

Eurovision example

• Analyse tweets with the #eurovision tag over a set time period (during the program)

• Extract country and positive/negative remark

Getting the user‘s interests

Beancounter architecture

• RDF based format

– Defines properties for describing a person and their relations to other people and objects

http://xmlns.com/foaf/spec/

Weighted Interests

• Add weighting to the foaf:interest property

See http://xmlns.notu.be/wi/

FOAF as common vocabulary

Beancounter web UI

Collecting user streams

Viewer profile (1/2)

Viewer profile (2/2)

(4) TV content recommendation• Recommender strategy

– Collaborative recommendation

• You share interests with your friends

• Statistical analysis: what content is liked/watchedquantitively more by others with similar interests/history

– Content-based recommendation

• An interest in X means a potential interest in Y

• Pattern-based analysis: what content has related conceptsto the content liked/watched by you

– Hybrid recommendation

• Best of both!

NoTube recommendationapproach

Recommendation lifecycle

Graphic by Libby Miller, BBC

Linked Data recommendations

• The content-based approach:

– Identify weighted sets (patterns) of DBPediaresources from user activity objects

– Compute distance between DBPedia concepts in the user profile and in the program schedulethrough its SKOS-based categorisation scheme

– Choose the matches above a certain threshold forTV programme recommendation

User interests (DBPedia concepts)

Match user interest and TV subjects

N-Screen http://n-screen.notu.be

Get recommendations

TV recommendation calculation

So, is this the future of television?More: http://notube.tv/showcases/personalised-news/

Or this?More: http://notube.tv/showcases/tv-guide-and-adaptive-ads/

Or this?More: http://notube.tv/showcases/tv-and-the-social-web/

And in the farther future?

Interested in the project results?

Find out more online at www.notube.tv

No re-use of any slides or content of slideswithout explicit acknowledgement of:

NoTube project, www.notube.tv &

this slideset, www.notube.tv/slides

Notube slides-120130071559-phpapp01

Design

NoTube: Metadata Interoperability

Whyrecruiterslovehirevue Slides 120529130501 Phpapp01

NoTube: User Profiling (Beancounter)

NoTube: Loudness Normalisation

NoTube project results. Bringing TV and Web together

The NoTube BeanCounter: Aggregating User Data for ...sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-520/pa… · The NoTube BeanCounter: Aggregating User Data for Television

NoTube User Model slides

Slides Fordismo 120627120330 Phpapp01

NoTube Project Presentation @ IBC2010

NoTube: BBC show case

NoTube: Stoneroos show case

NoTube: integrating TV and Web with the help of semantics

NoTube: Pattern-based Recommendations (part 1)

Highprotectionatm Slides 130604063813 Phpapp01

NoTube project poster, 2nd version

NoTube: Using the Synergy of Broadcast, Internet and Social TV

Ais Romney 2006 Slides 09 Auditing Computer Based is 091101092309 Phpapp01

Slides epistolaaosfesios-130425113759-phpapp01

Sondenabhängigkeit – in 5 Schritten erklärt – NoTube

NoTube: User Profiling @ Korea Telecom