Lemonia Argyriou, (ICCS - National Technical University of Athens)Charalampos Z. Patrikakis, (Technological Education Institute of Piraeus)Stuart CM Porter, (CTVC) Nikolaos Papaoulakis, (ICCS - National Technical University of Athens)Christina Androulaki, (ICCS - National Technical University of Athens)

Using media related user profiles to personalize multimedia access over social networks

The research leading to these results presented here, has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 248474 Socially Aware, collaboRative, scAlable Coding mEdia.Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• Personalized media access over the internet

• Use of media related, dynamic user profiles, - created and updated through the user behavior,- related to access to multimedia resources, - related also to tagging and commenting

• Mechanisms for implementing and deploying the profiles

What this paper is about

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• The amount of time spent on the internet is approaching (and in many countries, has surpassed) the number of hours spent watching television, or consuming more traditional media types.

• There is too much content.

• Personalized media platforms based on sophisticated recommendation engines and social networks are now starting to enter the market.

• The “orange film” problem.

• Recommendations (taking also into account social networking)

• Context awareness

The situation

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• Design and implement a prototype platform for offering Quality of Experience and personalisation,

- using scalable media streaming, - over P2P networks, - and making use of social networking

• An end to end platform covering all aspects of the media value chain (from production to consumption of media)

• Accommodating both broadcaster originating and user generated content

SARACEN project concept

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN platform general architecture

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Personalising the streaming experience

Device capabilities

Viewing preferences

Social networkinginfo

Access network capabilities

Location

User-centric design of the media access experience

1. Technological capabilities of the supporting infrastructures and devices

2. Personal preferences and general context (i.e. location)

3. Implicit and explicit methods and harnessing information and functionality of social networks (SARACEN will in fact implement a media centric social network)

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Personalising the streaming experience

• Personalized streams, through the integration of metadata related to the streamed media

• Produced by users or user groups and destined to users belonging to the same interest group

• Filter media related information such as tags, comments and metadata that will be streamed to the user

• Allow for the effective formation of a P2P distribution network so as to link users with similar preferences

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• Consists of a range of public and a private data.

• The level of privacy of each entry in his profile can be set/edited

• Data accessible by some specific “friends” or “groups”, or completely public

• Data making up the profile is gathered and, importantly, kept up to date, through both explicit and implicit data collection techniques

The SARACEN user profile

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• Provision of particular information contributed directly by the user.

• Targeted information can be used: Movies, Music, Books, TV series/shows, Sports and Other Interests.

• Import information from the user’s social networks

• Use of auto complete function

• Automatic creation of new records

Explicit profiles

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• After the creation of a user profile, SARACEN creates a record where all the keywords produced according to his activity are stored

• The user can select which of those keywords he wants to add to his profile

• The user activities that allow the system to update his personal record are:

1) Movies and music that he accesses,

2) Filtering information (addition and deletion of terms)

3) Comments and tagging of media that he performs

Implicit profiles

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• Each time a user selects to watch or interact with a video, the personalization module of the system increases or reduces a value parameter that represents the importance of the keywords that are being stored in his personal user record

• Refreshing a “keyword memory” mechanism similar to that of a cache

• When the user next selects to update his profile by selecting the “Suggest me” button the system ask him if he wishes to delete the keywords whose values has reached zero and if he wishes to add the new keywords the values of which have been increased and reached a threshold value for addition

Dynamic update of profiles

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Filtering Mechanism

^ƚĂŐĞϮ

h ƐĞƌƉƌŽĨŝůĞ

WƌĞĨĞƌĞŶĐĞƐ

ŬĞLJǁ ŽƌĚƐ

WĞƌƐŽŶĂůŝƐĂƚŝŽŶ

Wƌŝǀ ĂĐLJ

WƌĞĨĞƌĞŶĐĞƐ

ŬĞLJǁ ŽƌĚƐ

ŽŶƚĞŶƚŵĂƚĐŚŝŶŐ

ŐƌŽƵƉ

&ŽĐƵƐƐĞĚŐƌŽƵƉ

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Use of keywords for characterising a movie

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Two types of users: SARACEN users and the SARACEN broadcasters

Common users are offered the opportunity to make their uploaded video public and available to anyone who uses the SARACEN or characterize it as private and only send it to a group of friends that they select.

Keywords presented in a depictive way, that can easily visualize the profile (like in the different templates of wordlets/ slidebar for managing focus)

Multimedia uploading and sharing

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• Metadata is stored in a separate multimedia record at the SARACEN database and is translated into keywords that characterize the certain multimedia

• Keywords come out from the filtering of the users’ performing over the content but also from the rights owner’s page describing the multimedia content.

• User is also able to interact with the content, in other words he/she can tag the multimedia while he/she is watching it and make comments, producing metadata that are being stored following the record of the page in SARACEN database

Content access and media browsing

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

• The ideas presented here are being implemented.

• A LoFi prototype is available

• A first version of a HiFi prototype has just been released

• Demos of P2P streaming are being prepared

• User generated content with the ability to tag it is also being tested

• 3D is introduced since Oct 2011.

Making it happen

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN profile creation - SNSocial networks & subscriptions

Me

My favorite

Indicate credentials to acess to the different social networks you’re in

myIdUser name

*****Password

CheckYou didn’t indicate a valid user name or password to access to Primetel system

Back to SARACEN

home page

Profile information

Invite to use Saracen(we can place thissomwhere else)

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN profile creation - infoSocial networks & subscriptions

Me

Back to SARACEN

home page

Save

Profile information

My favorite

Alfred Hitchcock Presents

The String

Audrey HebpburnHung Up

Waka Waka Maria CallasThe Doors

comedy

war

pop

rock

bluesclassic

Tags from your profile

Tags from your profile that matches with your actions

Tags extracted from your actions

bollywood

IndiaFormentera

beachsailing

Ibiza

Mediterranean sea

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN player - infoBack to

SARACEN home page

Back to search

Info

Raiting & Social Networks

Tags & Comments

Title: Haan Maine Bhi Pyaar Kiya

Date: 2002

Directed by: Dharmesh Darshan

Duration: 113 min.

Country: India

Distributed byShree Krishna Productions

Written by: Dharmesh Darshan Raj Sinha

Genres: Romantic

Review

Comment: Rian:Best scene from the whole movieFriends tagged: Natalia is on this sceneCamera info: diafragma 5.6

CommentsFriends taggedCamera info….

Select extra info you want to see

Recommend something else

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN player - rateSNBack to

SARACEN home page

Info

Raiting & Social Networks

GeneralBollywoodFootball

Set Boxes

Yours :Your friends: ---General : 7.8

Raiting 6

Social network chat

Mine & friends boxes (when you can put content)

share social networks

Which of you friends have watched/is going to watch this movie

Comment: Rian:Best scene from the whole movieFriends tagged: Natalia is on this sceneCamera info: diafragma 5.6

CommentsFriends taggedCamera info….

Select extra info you want to see

Tags & Comments

Review

Allow other people know that I’m watchin this movie and that I’ve watched it

Back to search

Recommend something else

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN player – tags - tagBack to

SARACEN home page

Raiting & Social Networks

Tags & Comments

Comment: Rian:Best scene from the whole movieFriends tagged: Natalia is on this sceneCamera info: diafragma 5.6

CommentsFriends taggedCamera info….

Select extra info you want to see

Info

Review

Tag

Timeline

Drag & Drop the icon to add a newClick to view/edit existing

comedy

Romeo&julia

musiclove

belly-dancingdance

Bombay

Comment

Friend beeing tagged

Folk clothes

Location

Link

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

SARACEN player - recommendBack to

SARACEN home page

Back to search

Info

Raiting & Social Networks

Tags & Comments

Title: Haan Maine Bhi Pyaar Kiya

Date: 2002

Directed by: Dharmesh Darshan

Duration: 113 min.

Country: India

Distributed byShree Krishna Productions

Written by: Dharmesh Darshan Raj Sinha

Genres: Romantic

Review

Comment: Rian:Best scene from the whole movieFriends tagged: Natalia is on this sceneComments

Friends tagged….

Select extra info you want to see

Recommend something else

Haan Maine Bhi Pyaar

KiyaSangam

Mother India

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Search result

Uploader: name1/id1Type: movieTitle: Genghis BluesDate: 1999

Affinity with you: 93%

Uploader: name2/id2Type: home recordingTitle: Dubee dubee in concertDate: 20-july-2012

Remarks: one of your facebook firends is related to the video

Affinity with you: 85%

Uploader: PrimetelType: TV program / documentaryTitle: Red, white & bluesDate: 18:00 20-july-2012

Affinity with you: 84%Remarks: TV program only for primetel suscribers

Uploader: name4/id4Type: trailerTitle: Blues Brothers 2000Date: 1998

Affinity with you: 82%

Uploader: name4/id4Type: movieTitle: Blues Brothers 2000Date: 1998

Affinity with you: 82%

Uploader: name5/id5Type: movieTitle: The Blues BrothersDate: 1980

Affinity with you: 70%

Duration: 88 min.

Duration: 23 min.

Duration: 60 min.

Duration: 123 min.

Duration: 10 min.

Duration: 132 min.

Moreinfo

play

do not showagain

Search blues Search

Back to SARACEN

home page

Affinity Best rating Most watched NewestSort

Only my firends content Only my friends affinity, rating, watched

Include 3D content in restult Only 3D in result

3D

3D

3D

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Detailed Information Blues Brothers 2000 Movie - rateSN

Info

Raiting & Social Networks

Tags

Affinity with you: 82%

Uploader: name4/id4

Type: movie

Yours : ---Your friends: 8.2General : 7.8

Raiting GeneralBollywoodFootball

Set boxes

Deatiled votes and review from you friendsJohn Smith: 7.5Rita Mosoyan: 5.6Peter Hamilton: 8.1

It's a mix of loving homage to rhythm and blues, scattershot comedy, on-the-spot musical, and the most outrageous car chase film ever put to celluloid.

Patrik Monterde: 7.4The Blues Brothers ranks among my favorite films of all time -- not the best, just a favorite. With inspiration that never remotelycomes close to ceasing, Landis' (The Kentucky Fried Movie, An American Werewolf in London) film is more an experience than a conventional movie.

Deatiled votes and review from people that have same affinity than youLurta Nikolas: 8.5

A musical comedy action fantasy should not work, especially when one considers that it is the first SNL skit-to-screen adventure (which, history has shown us, is a decidedly mixed bag). But this one does.

Frida Robertsson: 7.4Dani Goya: 6.5

Back to SARACEN

home page

Back to search

add to queue

share social networks

play

Which of you friends have watched/is going to watch this movie

See what others are watching

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

BoxesBack to

SARACEN home page

Your boxes

General

Football

Family Favorite

Bollywood

Create

new box

Your friends boxes

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Configure Box - searchBack to

SARACEN home page

Search for movies to add

Privacy

Already added movies

Title: Haan Maine Bhi Pyaar Kiya

Title: Sangam

Title: Mother India

bollywood movies Search

add

Uploader: name1/id1Type: movieTitle: Hum Aapke Hain KaunDate: 1999

Affinity with you: 93%

Duration: 88 min.

add

Uploader: name1/id1Type: movieTitle: Mughal-e-AzamDate: 1999

Affinity with you: 86%

Duration: 88 min.

Bollywood

Go to boxes

Affinity Best rating Most watched Newest

Sort

Only my firends content Only my friends affinity, rating, watched

Include 3D content in restult Only 3D in result

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Publish - descriptionBack to

SARACEN home page

File

Title

Browse/user/files/movies/lordofrings/twoTowers.avi

The Lord of the Rings: The Two Towers

Frodo and Sam continue on to Mordor in their mission to destroy the One Ring. Whilst their former companions make new allies and launch an assault on Isengard.

Description

Description

Info & Tags

Preview image

Upload image for preview

Use image from uploaded video

Browse

/user/files/movies/lordofrings/t

Privacy & License

3/10

Duration The published file has a duration of 223 minutes.

3D It has been detected this content is 3D (please confirm ir can be detected)

Tagged contentBack to

SARACEN home page

Marianne, did the following review for

Vertical Limit meets real life in this deceptively simple documentary. The story of two British mountain climbers who stared death in the face while climbing a treacherous peak in the Peruvian Andes, Touching The Void is a gripping tale of ordinary men coping with extraordinary circumstances.

Approve selected

Alex added location to

Location: Time 1:34-1:45, Es Pujols

Alex added a comment to

Comment: Spent there my last summer, beautiful place.

Reject selected

Play that scene

Play that scene

Select All Select None Filter Alex

Select from filter

Unselect from filter

Dec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

Thank you and …

. . .stay tunedDec 1, 2011 SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

http://saracen-p2p.eu/

www.facebook.com/pages/Saracen/126085287431720

twitter.com/saracenP2P

www.youtube.com/user/SaracenP2P#p/u