Diversity and novelty for recommendation system

A simple survey of Diversity and A simple survey of Diversity and novelty metrics for recommender novelty metrics for recommender systemssystems

Reporter:孙建凯

2012.07.11

Copyright 2012 by IRLAB@SDU

Move beyond accuracy metricsMove beyond accuracy metrics

while the majority of algorithms proposed in recommender systems literature have focused on improving recommendation accuracy

other important aspects of recommendation quality, such as the diversity of recommendations, have often been overlooked.

The recommendations that are most accurate according to the standard metrics are sometimes not the recommendations that are most useful to users[1]

2


Diversity and Novelty

Accurate is not always good: How Accuracy Metrics have hurt Recommender Systems

GroupLensResearch,CHI'06


Accuracy does not tell the whole story


Diversity

Individual Diversity Aggregate Diversity


Individual Diversity

Diversity Difficulty[3]

Average dissimilarity between all pairs of items recommended to a given user(intra-list similarity) [2,4]


Diversity Difficulty

What We Talk About When We Talk About Diversity [DDR’12 Northeastern University USA]

Like query difficulty in IR

For a specific query and corpus, query difficulty is a measure of how successful the average search engine should be at ad-hoc retrieval.


Diversity Difficulty

Diversity Difficulty is defined with respect to a query and a corpus.

Describes diversity-the number of subtopics which are covered by a list;

Describes novelty-which is inversely proportional to the number of times a list repeats a subtopic


Finding needles in the haystack

Imagine a query with 10 subtopics ,1000 documents relevant to only the first subtopic, and each of the remaining subtopics covered by a single, unique document.

On the other hand ,if there are large numbers of documents relevant to multiple subtopics, it would be easy to produce a diversity list.


Diversity Difficulty function

The maximum amount of diversity achievable by any ranked list-dmax

The ease with a system can produce a diverse ranked list.-dmean

Harmonic function


Examples


Improving Recommendation Lists Through Topic Diversification

Introduce the intra-list similarity metric to access the topic diversification of recommendation lists and the topic diversification approach for decreasing the intra-list similarity

Average dissimilarity between all pairs of items recommended to a given user


Intra-list Similarity


Taxonomy-based similarity Metrics

Instantiate c with their metric for taxonomy-driven filtering.[5]


Topic Diversification Algorithm

Algorithm A brief textual sketch

Experiments

precision diversity


Aggregate Diversity

improving recommendation Diversity using ranking-based techniques[IEEE transaction’12]

Use the total number of distinct items recommended across all users as an aggregate diversity measure, define as follows:


General overview of ranking-based approaches for improving diversity


Re-Ranking Approach


Other Re-ranking Approach












Combining Ranking Approaches

Many possible ways to combine several ranking functions

In this paper , linear combination

Open issue: letor ? Neural network?


Entropy

A study of Heterogeneity in Recommendations for a social Music Service[6]


Open issue:probability


Entropy

Aggregate Entropy:

Item popularity between lists?

Individual Entropy:

subtopic popularity?


Bipartite network

Bipartite network projection and personal recommendation[Tao Zhou, Physical Review]

Solving the apparent diversity-accuracy dilemma of recommender systems[Tao Zhou]


Illustration of resource-allocation process in bipartite network


Solving the apparent diversity-accuracy dilemma

heats probs


Hybrid Methods

weight hybrid


Diversity Measure


Surprisal/novelty


Results-why better?


Surprise me

Tangent: A novel, ‘surprise me’, recommendation algorithm [kdd’09]


Framework of Tangent Algorithm

Suggest items which are not only relevant to user preference but also have a large connectivity to other groups.

Consisting three parts as follows:

1 Calculate relevance score(RS) for each node

2 Calculate bridging score(BRS) for each node

3 Compute the Tangent score by somehow merging two criteria above


Case study


Case study


Call for papers

September 20, 2012


Reference

1. Accurate is not always good: How Accuracy Metrics have hurt Recommender Systems

2.improving recommendation Diversity using ranking-based techniques

3. What We Talk About When We Talk About Diversity

4. Improving Recommendation Lists Through Topic Diversification

5. Taxonomy-driven computation of product recommendations


Reference

6. A study of Heterogeneity in Recommendations for a social Music Service

7. Bipartite network projection and personal recommendation

8.Solving the apparent diversity-accuracy dilemma of recommender systems

9. Tangent: A novel, ‘surprise me’, recommendation algorithm


thanks

Education

Diversity and novelty for recommendation system