Transcript
Page 1: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

Munich, 11.10.2016

Michael Muckel

IN SEARCH OF RELEVANCE

MODERN RECOMMENDER SYSTEMS FOR MEDIA INDUSTRY

Page 2: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

2

Glomex Overview

Publishers

Content providers

Video Value Service

Media Delivery Service

Media Exchange Service

Glomex

External broadcasters

Web-only content owners

Non-P7S1 publishers

Page 3: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

3

List Model for Recommendations

1 Item

2 Item

3 Item

4 Item

N Item

Top-N List

Decreasing Relevance

Page 4: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

4

Temporal Model for Recommendations

1 Item

Continuous Playlist

2 Item 3 Item 4 Item ∞ Item

Current Item

Page 5: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

5

Relevance – Search Perspective

Exploration

PrecisionSe

arch

Page 6: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

6

Relevance – Recommender Perspective

Exploration

PrecisionSe

arch

Recommender

Page 7: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

7

Two Sides of the Relevance Medal

Search

Recommender

Page 8: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

8

Glomex Recommender - Customer Perspective

Enable our Glomex Exchange Customers to efficiently select content with high potential for conversions

Metrics: Playlist Add, Pick Top-N

Page 9: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

9

Glomex Recommender – User Perspective

Keep attention of users as long as possible

Metrics: Video-View per Playlist, Video-Views per Session, Average View-Time

Page 10: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

10

Anatomy of a Recommender System

SCORE RANK FILTER

Page 11: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

11

Anatomy of a Recommender System

SCORE RANK FILTER

Page 12: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

12

Popular Recommender Models - in a nutshell

Page 13: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

13

SCORE - Content-Based Recommenders

Tags:

Publish Date:

Title:

Cast:

Description:

Brand:

Hierachy:

fun, joko, claas, comedy, pro7 ,circus halligalli, show, sport, interview, formula1, race of champions

11.12.2015

Ein unnötig kompliziertes Interview

Joko Winterscheidt, Klaas Heufer-Umlauf, David Coulthard, Sebastian Vettel, Nico Hülkenberg

Wir haben Joko endlich seinen Traum erfüllt, einmal in einemFormel 1-Wagen mitzufahren. Die einzige kleine Bedingung: Ermuss währenddessen ein Interview führen.

Prosieben

Episode: 13 Series: 6

Watch: http://bit.ly/2bO4Z3u

Page 14: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

14

SCORE - Content-Based Recommenders – Search Problem

Tags:

Publish Date:

Title:

Cast:

Description:

Brand:

Hierachy:

fun, joko, claas, comedy, pro7 ,circus halligalli, show, sport, interview, formula1, race of champions

11.12.2015

Ein unnötig kompliziertes Interview

Joko Winterscheidt, Klaas Heufer-Umlauf, David Coulthard, Sebastian Vettel, Nico Hülkenberg

Wir haben Joko endlich seinen Traum erfüllt, einmal in einemFormel 1-Wagen mitzufahren. Die einzige kleine Bedingung: Ermuss währenddessen ein Interview führen.

Prosieben

Episode: 13 Series: 6

Find similar content based on descriptive metadata

Page 15: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

15

SCORE - Context-Based Recommendation – Use Context

Palina Rojinski

Joko Winterscheidt

Ina Müller

Olli Schulz

Web Mining with:

Named Entity Recognition Word Embeddings

Page 16: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

16

SCORE - Collaborative Filtering

User U 3 - 5 1 - -

User U 1 0 1 1 0 0

Explicit Feedbackaka Ratings

Implicit Feedback

Circus HalligalliDawn of the Gag 2

Length of vector depends on number of items available

Page 17: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

17

SCORE - Collaborative Filtering

Item 1 Item 2 Item 3 Item 4 … Item N

User 1 x x x

User 2 x x

User X x x ? x ? ?

User-Rating Matrix

Page 18: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

18

SCORE – User-User Collaborative Filtering

Item 1 Item 2 Item 3 Item 4 … Item N

User 1 x x x

User 2 x x

User X x x ? x ? ?

Conceptual Model:

Find users with similar preferences1

Find and Score items from matching users that User X has not watched2

Page 19: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

19

SCORE – Item-Item Collaborative Filtering

Item 1 Item 2 Item 3 Item 4 … Item N

User 1 x x x

User 2 x x

User X x x ? x ? ?

Conceptual Model:

Find items with similar usage patterns for item 31

Find and Score items from matching users that User X has not watched2

Page 20: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

20

SCORE - Collaborative Filtering with Matrix Factorization

Item 1 Item 2 Item 3

User 1 x x

User 2 x

User X x x ?

≅ ✕

Conceptual Model:

Decompose rating matrix into dense, low-rank matrices (e.g. Alternating Least Squares)1

Use resulting “Taste-Space” for finding similar videos2

Matrix U Matrix V

Page 21: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

21

SCORE – Statistical Models

Detecting (Hidden) Periodicities

Statistical Trend Detection

Possible Methods

Page 22: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

22

Anatomy of a Recommender System

SCORE RANK FILTER

Page 23: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

23

RANK - Basic Ranking - Sorting

Modele.g. Collaborative

Filtering

1 Item

2 Item

3 Item

N Item

Sort by Relevance Score

Page 24: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

24

Which model should we then choose?

Page 25: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

25

RANK - Hybrid Recommenders – Static Ensembles

f(xi,wi)

Score: xi

Weight: wi

1 Item

2 Item

3 Item

N Item

Model 1Collaborative

Filtering

Model 2Collaborative

Filtering

Model 3Content-Based

Filtering

Model 4Trending

Genre

Model NNew

Content

Sort by Adjusted Relevance Score

Page 26: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

26

RANK - Hybrid Recommenders – Learning to Rank

Model 1Collaborative

Filtering

Model 2Collaborative

Filtering

Model 3Content-Based

Filtering

Model 4Trending

Genre

Model NMost-Watched

Content

f(xi,wi)

Score: xi

1 Item

2 Item

3 Item

N Item

Ranking Model

Weight: wi

User Feedback: ClicksUser Context: Location, DeviceSort by Adjusted Relevance Score

Page 27: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

27

Anatomy of a Recommender System

SCORE RANK FILTER

Page 28: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

28

FILTER – Basic Filtering

1 Content

2 Content

3 Content

N Content

1 Content

4 Content

6 Content

M Content

Ranked ListBy Relevance Score

Filtered List

2 Content

3 Content

5 Content

X

X

X

Filter Logic

Example Filter: Geo-Location

Page 29: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

29

FILTER – Multi-Dimensional Filters

1 Content

2 Content

3 Content

N Content

1 Content

4 Content

Ranked ListBy Relevance Score

Filtered List

2 Content

3 Content

5 Content

X

X

XFilter Logic

Real-world Filters

Geo-Location Device (type) Age-Restriction

Brands Genres ….

5 Content

5 Content

5 Content

X

X

X

!

Page 30: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

30

Alternate Approaches

SCORE RANK FILTER

Attribute XExamples: Genre, Rights (Geo-Restriction )

Integrate into Models Integrate into Filter?

Page 31: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

31

Glomex on AWS

Page 32: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

32

Glomex Data Service

Video Value Service Media Delivery Service Media Exchange Service

Data Service

Real-time-Monitoring Analytics Machine Learning

Page 33: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

33

Glomex Data Service

Page 34: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern

34

Thank you for your attention!


Recommended