Mining Social Networks for Recommendation€¦ · box on Yahoo's home page ... Facebook friend recommendations ... Jamali & Ester: Mining Social Networks for Recommendation, Tutorial

Mining Social Networks for

Recommendation

Mohsen Jamali & Martin Ester

Simon Fraser University

Tutorial at ICDM 2011

December 12th 2011

Intro Recommenders Recommendation in SNs Mining SNs Memory based Model based Link prediction Distrust Summary

Introduction

Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 2


Flood of information

• Conventional (industrial / mass) media

• Social media


Introduction



Introduction

Outline

• Introduction

• Recommender systems

• Recommendation in social networks

• Mining social networks

• Memory based approaches

• Model based approaches

• Link prediction

• Social networks with distrust

• Summary


Recommender Systems

4Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011


Web

search


Recommender Systems

• Search

Input: query keywords

Output: ranked list of results

• User needs to know what he is looking for.

� but content changes, keywords change

• Every user gets same result.

� but users have diverse interests

e.g., student, software developer, politician, . . .




Recommender Systems

• Users want to have personalized results.

• But are not willing to spend a lot of time to specify their personal information needs.

• Recommender systems automatically identify information relevant for a given user, learning from available data.

• Data - user actions,- user profiles.




Recommender Systems


Departed Star Wars Matrix Hurt Locker Titanic Terminator

Joe 2 5 4 2 ? ?

John 5 1 2 1

Susan 5 5 5 5

Pal 2 5 3

Jean 5 3 5 3

Ben 1 5

Nathan 2 4 1 4

Target User

Use

rs

Similar User

Items

Rating Matrix

Target Item

Ratings



Recommendation Tasks

• Rating prediction

Predict the rating of target user for target item, e.g. predict Joe’s rating for Titanic.

• Top-N recommendation

Predict the top-N highest-rated items among the items not yet rated by target user.

• Link recommendation (only if social network) Predict the top-N users to which the target user is most likely to connect.




Applications

Yahoo! news recommendations

• Recommendations of new articles for Today

box on Yahoo's home page

• 9,000 recommendations per minute

• Sophisticated personalization algorithm

• Based on demographic user attributes, the places they've visited when they've come to Yahoo in the past, and the stories they've already seen during that particular visit.




Applications

Yahoo! news recommendations (cont.)

• Team of editors prepare 50-100 news

packages, algorithm ranks packages for user.

• Has increased the click through rate by 270%

since 2009.

• Has helped editors to get better

understanding of the interests of different

user segments.http://www.fastcompany.com/1770673/how-yahoo-got-to-a-billion-clicks




Applications

Facebook friend recommendations

• „People you may know“

• “Based on mutual friends, work and education information, networks you’re part of, contacts and many other factors.”

• “Since our formula is automatic, you might occasionally see people you don’t know or don’t want to be friends with. To remove them from view, just click the X next to their names.”http://www.facebook.com/help/?page=199421896769556




Applications

Pandora music recommendation

• Internet radio service Pandora.com

• Music Genome Project:trained music analysts score each song based on hundreds of distinct musical characteristics.

• Recommend songs with similar scores

• Recommend sequence of songs




Applications


Item-based

collaborative

filtering



Applications

Netflix (movie recommendation)

� $1M prize for 10% accuracy improvement




Privacy Issues

• Recommender systems use a lot of personal data: movies watched, current location, . . .

• The more personal data shared, the better (more personalized) the recommendations.

• Serious threat to data privacy.

• Users need to be able to make an informed choice.

• E.g., Google users can shut off personalization features by deleting their Web history.




The Filter Bubble

• Users get less exposure to

conflicting viewpoints and are

isolated intellectually in their

own informational bubble.

• E.g., Google results for "BP"

• User 1: investment news about

British Petroleum

• User 2: information about the

Deepwater Horizon oil spill


http://www.kidsil.net/wp-

content/uploads/2011/05/filter-

bubble.jpg



The Filter Bubble

• [Pariser 2011]: “. . . creates the impression that

our narrow self-interest is all that exists. ..”.

Google and Facebook are offering users "too much

candy, and not enough carrots.”

• Book reviewer Paul Boutin: did a similar

experiment among people with differing search

histories with nearly identical search results.

• Harvard law professor Jonathan Zittrain: "the

effects of search personalization have been light."




Collaborative Filtering

• Set of items I, set of users U

• Users rate items.

• No need for information about content of items or attributes of users.

• Users with similar ratings on some items are likely to have similar ratings on further item.

• Items which are rated similarly by some users are likely to have similar ratings by further users.






AggregatorAggregator

Prediction

Target User




Nearest neighbor-based approach [Resnick et al., 1994]

• find users with history of agreement

(similar rating profiles),

• aggregate their ratings to predict unknown

rating.

Issues

• How to define user similarity?

• How many similar users?

• How to aggregate the ratings?20Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011




• : (observed) rating of user u for item i

• : mean rating of user u

• : predicted rating of user u for item i

• : set of users similar to user u

(who have rated item i)

• : similarity of users u and v

• : normalization factor


iur ,iur ,iur ,

iur ,

ur

iur ,ˆ

)(uN

κ

),( vusim




• Different users use the ratings scale differently.

� normalize ratings by the mean rating

• The more similar a user v, the higher the weight

of his rating.



∑∈

−×+=)(

,, )(),(ûNv

vivuiu rrvusimrr κ





How to define similarity of users?

: set of items rated by both users u and v

• Pearson correlation coefficient

• Cosine similarity

∑∑

∑

∈∈

∈

−−

−−

=

uvuv

uv

Ii

viv

Ii

uiu

viv

Ii

uiu

rrrr

rrrr

vusim2

,

2

,

,,

)()(

)()(

),(

∑∑

∑

∈∈

∈=

uvuv

uv

Ii

iv

Ii

iu

iv

Ii

iu

rr

rr

vusim2

,

2

,

,,

),(

uvI





• So far: user-based CF

• Item-based CF is dual approach [Sarwar et al., 2001].

• : mean rating of item i

• : set of items (rated by user u) similar to item i

• : similarity of items i and j


∑∈

−×+=)(

,, )(),(îNj

jjuiiu rrjisimrr κ

ir

)(iN

),( jisim





• So far: memory-based CF

• Model-based CF:

learn model in training phase,

apply model in test phase to predict rating.

• : set of ratings of user u

• Rating scale [1..n]

• Probabilistic model

• How to compute ?

))(|(ˆ,

1

, uRrrPrr iu

n

r

iu =×=∑=

)(uR

))(|( , uRrrP iu =





• Idea: ratings of items depend on their location in a

latent factor space.

[Koren et al., 2009]




Probabilistic matrix factorization [Salakhutdinov et al., 2007]

• Assumption: observed ratings are governed by

latent variables (factors).

• N: number of users

• M: number of items

• K: number of factors, K << M, K << N

• Uu: latent factor (vector) of user u

• Vi: latent factor (vector) of item i





Probabilistic matrix factorization (MF) (cont.)

• σu, σV , σR : normal priors

• Assumption: item ratings are independent fromeach other

•


),|(),|( 2

,, Ri

T

uiuiuiu VUrVUrrP σΝ==

=otherwise,0

observedif,1,

uiR

iu

rI




MF(cont.)

• Parameter learning through maximum

likelihood estimation

• Equivalent to minimizing the error function





MF(cont.)

• MF-based CF typically outperforms NN-based

CF [Koren et al., 2009].

• MF can naturally incorporate biases and

additional data sources [Koren et al., 2009].

• But latent factors are hard to interpret.




Content-based Recommenders

• Set of items I, set of users U.

• Given user profiles, describing the users‘

tastes, preferences and needs.

• Given item profiles, characterizing the

content of item.

• Top-N recommendation by ranking items

w.r.t. similarity of item profiles and user

profile [Balabanovic 1997].





• Item profile: typically frequencies of k selected keywords.

: frequency of keyword i in item j

: number of items containing keyword i

• Term frequency / inverse document frequency

• Profile for item i:32Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011

i

in

MIDF log=

jif ,

in

jzz

ji

jif

fTF

,

,

,max

=

),...,()( ,,1 iki wwicontent =ijiji IDFTFw ×= ,,




• User profile: typically importance or

frequencies of keywords, e.g. aggregation of

profiles of items liked by user.

• Similarity of item i and user u


∑∑

∑

==

==k

l

il

k

l

ul

k

l

ilul

ww

ww

iusim

1

2

,

1

2

,

1

,,

),(



Hybrid Recommender Systems

• Combine collaborative and content-based method.

• Approach 1: combine separate recommendersCombine results, e.g. using linear combination or voting. [Pazzani 1999]

• Approach 2: add aspects of content-based method to CF. [Pazzani 1999]

E.g., use profiles to compute similarities between users or items.





• Approach 3: add aspects of CF to content-

based method [Soboroff et al., 1999] .

E.g., perform dimensionality reduction on

group of content-based profiles.

• Approach 4: Unified recommendation model

E.g., combine topic model, i.e. Latent Dirichlet

Allocation, with MF.





• Latent Dirichlet Allocation (LDA) [Blei et al., 2003]

Assumption: documents have latent topic

distribution, topics have word distributions.

• Graphical model


θ (latent)

topic distribution

Z (latent) topic

W (observed) word

α prior for topic

distribution

βword distributions

N number of words

per document

M number of

documents




• Collaborative topic regression [Wang et al., 2011]

• Idea: latent item factors (V) depend on topic

distribution (θ) of item.




Performance Evaluation

Cross-validation on offline dataset

• Withhold subset of ratings (test set)

• Use remaining ratings to train recommender

(training set)

• Compare the withheld ratings against the

predicted ratings, compute error measure

� Standard evaluation in research


} . . ),.,( ),,{( i.e.),( jviuTestIUTest =×⊂




Cross-validation on offline dataset (cont.)

• Measures for rating prediction

• Mean absolute error

• Root mean square error


∑∈

−×=Testiu

iuiu rrTest

MAE),(

,, |ˆ|||

1

||

)ˆ(2

),(

,,

Test

rr

RMSETestiu

iuiu∑∈

−

=




Cross-validation on offline dataset (cont.)

• Measures for top-N recommendation

• Recall (or coverage)

TopN: set of the top-N recommendations

(by algorithm)

TestTop: set of all elements of the test set that

are among the top-N items for the user


||

||

TestTop

TestTopTopNRecall

∩=




Limitations

• Measures only accuracy of recommendations

• Does not measure other aspects such as

diversity

• Does not measure how recommendations

change user behavior

� this is the ultimate goal of a recommender!





In industry

• Want to evaluate user satisfaction and business profit

• A/B testing in online system

• Evaluation measures

click-through rateusage

return rate of customers

profit




Challenges

• Privacy-preservationHow to make good recommendations without violating privacy concerns?

• Diversity of recommendationsRecommenders tend to suggest “more of the same”.

• Explanation of recommendationsNecessary to build trust into the recommender.




Challenges

• Recommendations for cold-start usersi.e. users with very few ratings

• Typically, ~ 50% of users cold-start.

• CF fails, because there are no similar users (user-based CF) and no item ratings to aggregate (item-based CF).

• Recommendations for cold-start itemsi.e. items with very few ratings

• Typically, ~ 50% of items cold-start.

• CF fails, because there are not enough raters for the item.

• Content-based method works. 44Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011



Social Networks

• Social network [Wasserman et al., 1994]

• Used widely in the social and behavioral

sciences, in economics, marketing, . . .

• directed or undirected graph

nodes: actorsedges: social relationships or interactions


http://www.fiestamart.com/blog/wp-content/uploads/2011/02/socialnetwork.jpg



Social Networks

• Different types of social relationships

• Different types of interactions




Social Networks

• Explicit social network

relationships provided by users

• Implicit social network

relationships inferred from user actions

• Email network

• Co-worker network




Social Networks

• The formation and evolution of social networks is affected by many effects, including

– Self-interest,

– Social and resource exchange,

– Balance,

– Homophily,

– Proximity.

[Monge & Contractor 2003]




Trust Networks



• Related concept: trust network [Golbeck 2005]

• Trust network allows users to

– systematically document their trust-relationships,

– see which users have declared trust in another user.

• Connected users do not necessarily have a social relationship.

• Trust in a user may be based, e.g., on articles or reviews authored by that user.


Online Social Networks

• Emergence of online social networks

• Among the top websites [Alexa 2011]

. . .

2. FaceBook

. . .

9. Twitter

. . .

12. LinkedIn

� Availability of very large datasets50Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011



Social Rating Networks

• Social rating network (SRN):social network, where users are associated with item ratings.

• Item ratings can be numeric [1..5]or Boolean (bookmark photo, like article, . . .).

• Examples: Epinions, Flixster, last.fm, flickr, Digg.

• Social action: create social relationship, rating action: rate an item.




Social Rating Networks


0.8

0.7



Effects in Social Rating Networks

• Social influence: ratings are influenced by ratings of friends, i.e. friends are more likely to have similar ratings than strangers.

• Correlational influence:ratings are influenced by ratings of actors with similar ratings,i.e. if some ratings are similar, further ratings are more likely also to be similar.




Effects in Social Rating Networks

• Selection (homophily):

actors relate to actors with similar ratings,

i.e. actors with similar ratings are more likely

to become friends.

• Transitivity:

actors relate to friends of their friends,

i.e. actors are more likely to relate to indirect

friends.




Recommendation in Social Networks

• Benefits of social network-based

recommendation:

- Exploit social influence, correlational

influence, transitivity, selection.

- Can deal with cold-start users, as long as

they are connected to the social network.

- Are more robust to fraud, in particular to

profile attacks.





• Challenges

– Low probability of finding rater at small

network distance.

– Noisy ratings at large network distances.

– Social network data is very sensitive.

– Edges in online social networks are of

greatly varying reliability / strength.




Mining Social Networks

• Lots of research in various directions,

e.g. community identification,

maximization of social influence, etc.

• Here only mining methods relevant for

recommendation:

– Analysis of social influence,

– Models of social rating networks,

– Inference of social networks.57Jamali & Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011



Influence and Correlation [Anagnostopoulos et al., 2008]

• Goal: does a SN exhibit social influence?

• Discrete time period [0..T], consider only one

action, e.g. using a certain tag.

• At every time step, each user flips a coin to

decide whether he will get active.

• Probability of activation depends only on

number a of already active friends:


βα

βα

++

++

+=

)1ln(

)1ln(

1)(

a

a

e

eap



Influence and Correlation

• α measures social correlation

• Ya,t: number of users with a active friends at

time t-1 who get activated at time t

• Na,t: number of users with a active friends at

time t-1 who do not get activated at time t

•

• Compute α and β that maximize the data

likelihood


∑∑ ==t

taa

t

taa NNYY ,, ,

aa NY

a

apap ))((1()( −∏




• If social influence does not play a role, then the

timing of activation should be independent of

the timing of activation of other users.

• : set of active users at time T

• ti: activation time of user i

• Shuffle test

– Perform random permutation π of {1,. . ., l}.

– Set activation time of user i to .


},...,{ 1 lwwW =

)(' : ii tt π=




• Compute α for original activation times.

• Compute α’ for shuffled activation times.

• If α and α’ are close to each other, then the

model exhibits no social influence.


α for original activation times

vs. α’ for shuffled activation times

on Flickr dataset

� Social correlation,

but no social influence!



Feedback Effects Between Similarity

and Social Influence [Crandall et al., 2008]

• Goal: characterize how social influence and

selection work together to affect users’ actions

and interactions.

• Wikipedia dataset

– Actions: editing article,

– Interactions: editing the discussion page of another

user.





and Social Influence

• How does the similarity of two users vary

around the time of their first interaction?


Sharp increase in similarity

immediately before first

interaction

Continuing but slower increase

after first interaction





• Generative model for social network

• Users are associated with history of their

actions and corresponding time stamps.

• Options for generating next action for user u

– Sample from u’s own history.

– Sample from history of a friend of u.

– Sample from history of any user.

– Perform a new action.






• Options for generating next interaction for user u

– Sample from users with similar history of actions.

Use weighted Jaccard coefficient to measure similarity.

– Sample a random user to interact with.

• Estimate model parameters from data

– Some parameters can be observed.

– Others are estimated by maximum likelihood

estimation.





Modeling Social Rating Networks [Jamali et al., 2011]

• Goal: generative model considering all four

effects between social actions and rating

actions, i.e. social influence, selection,

transitivity and correlational influence.

• What about other effects?

E.g., demographic attributes, location.

Corresponding data not observed, modeled as

random background effect.




Modeling Social Rating Networks





• Generation of actions similar to [Crandall et al.,

2008], but also transitivity and correlational infl.

• Temporal dynamics of effects, e.g. new user

Epinions Flickr





• ak: kth action, ordered by timestamps

• St : state of SRN at time t

• ϴ : set of model parameters

• Data likelihood

• Likelihood of social action





• Likelihood of rating action

• Parameter learning

Maximum likelihood estimation

EM algorithm





• Experimental results

• Φt,1 =0.91 in Epinions and Φt,1 =0.9 in Flickr

– Transitivity is the strongest effect for social actions.

• Φr,1 =0.59 in Epinions and Φr,1 = 0.54 in Flickr

– Social influence is the strongest effect for rating

actions.

• Comparison partners include

– CrossModel, similar to [Crandall et al., 2008]

– SocialOnly, similar to [Leskovec et al., 2008]





• Growth of similarity after creation of social

relationshipFlickr





• Average network distance of users before

creation of social relationship

Epinions Flickr




Inferring Social Relationships

and Their Strength

• So far: (Boolean) social network given.

• Sometimes, no information about social

relationships, only user actions

� Inference of social network from user

actions [Gomez-Rodriguez et al., 2010]

� Inference of weighted social network

[Myers et al., 2010]




Inferring Social Networks [Gomez-Rodriguez et al., 2010]

• Goal: infer social relationships from user actions

with time stamps.

• Assumption: there is a latent, static network

over which influence propagates.

• tu: activation time of user u, i.e. time when user

u gets activated (“infected”) by a cascade

• Cascade c specified through activation times of

all users: ∞== in tttc possibly ],,...,[ 1




Inferring Social Networks

• Independent Cascade model:

activated node activates each of his friends with

a given probability

• : probability of cascade c spreading from

user u to user v

•

• decreases with increasing

uv tt −=∆

),( vuPc

αα

∆∝∝

∆− 1

),(or ),( vuPevuP cc

),( vuPc∆





• C: set of all given cascades

• G: inferred directed graph over node (user) set

U={1,…,n}

• T(G): set of all subtrees of G

• Problem: compute G with at most k edges

that maximizes the likelihood

∏∈

=Tvu

c vuPTcP),(

),()|(

∏∏∏∈∈

∈∈

∈==

Tvu

c

CcGTT

CcGTT

vuPTcPGCP),(

)()(),(max)|(max)|(

)|( GCP





• Improvement of log-likelihood over empty

graph E:

• Equivalent problem

• Problem is NP-hard.

• FC(G) is submodular, which means that a greedy

algorithm gives a constant-factor approximation

of the optimal solution.

� NetInf algorithm

)|(logmax)|(logmax)()()(

TcPTcPGFETTGTT

c∈∈

−=





• Precision and recall for synthetic datasets

Spreading probability

PL: power law,

Exp: exponential




Inferring Weighted Social Networks[Myers et al., 2010]

• NetInf is very accurate for homogeneous

networks, i.e. networks where all connected

nodes influence (“infect”) each other with the

same probability.

• For inhomogeneous networks, define

• Goal: learn the matrix A = [Aij] from the

observed set of cascades C={c1,. . . , cn}

infected) is node | node infects node( i j iPAij =




Inferring Weighted Social Networks

• If i becomes infected, then j will be infected

with probability Aij.

• w(t): transmission time model

probability distribution of the transmission

time from one node to a friend

time of infection of node i by cascade c

time of infection of i’s friend j by cascade c

)(~ where, twttc

i

c

j += ττ

:c

iτ





∏ ∏ ∏∏ ∏∈ ∞= ∞<∞< <

−

−−−=

Cc i

ji

ji

ji

j

c

j

c

ici

cj

ci

ci

cj

AAwACPτ ττ ττ

ττ: :: :

)1())(1(1)|(

• Likelihood of observed cascades C given a

weight matrix A

• First term: one factor for each infected node i,

assuming that at least one of his friends j who was

infected earlier infected him

• Second term: one factor for each non-infected node i,

assuming that none of the infected friends j infected

him





• Parameter learning through maximum

likelihood estimation

• If i never infected j, then and do not

need to learn it.

• Translate into a convex optimization problem:

• Finds globally optimal solution.

• Can use efficient convex optimization methods.

� ConNIe algorithm

0:=ijA





• Precision vs. recall and mean square errors vs.

number of edges for synthetic datasets

Transmission time model

PL: power law,

Exp: exponential,

WB: Weibull.



Mining Social Networks for

Recommendation

Tutorial at ICDM 2011

Mohsen Jamali & Martin Ester

Part 2



Outline

• Introduction

• Recommender systems

• Recommendation in social networks

• Mining social networks

• Methods for Recommendation in Social Netorks

– Memory based approaches

– Model based approaches

• Link prediction

• Social Networks with distrust

• Summary



Recommendation in SNs: Problem Definition

• Input

– Rating matrix

• Real valued or binary

– Social network

• Weighted or binary

– Social rating network (SRN)

• A social network in which users can express ratings on

items




Recommendation in SNs: Problem Definition

• Rating prediction problem– For a given user u and the target item i:

• Predict the rating ru,i .

• Top-N item recommendation– For a given user u recommend top N items desirable

for him [Deshpande et al. 2004].

– Mostly neglected in the literature

• Top-N recommendation has been investigated in traditional recommender system.

– In social networks, there are very few works [Jamaliet al. 2009.b].




Data Sets for Recommendation in SNs

• Epinions– Online product review

– Explicit notion of trust

– Users review and rate products in different categories.

– Users express trust on other reviewers.

– http://www.trustlet.org/wiki/Epinions_dataset• 50K users, 140K items, 650K ratings, 480K links

– http://alchemy.cs.washington.edu/data/epinions/• 70K users, 105K items, 575K ratings, 500K links

– 50 % cold start• Less than 5 ratings




Data Sets for Recommendation in SNs

• Flixster

– Social metworking service for rating movies

– Friendship relations

– http://www.sfu.ca/~sja25/datasets/

• 1M users, 50K items, 8M ratings, 26M links

• 85% of users have no ratings

• 50% of rater are cold start

– Less than 5 ratings




Models for Recommendation in SNs

• Memory based approaches

– Explore the social network for raters

– Aggregate the ratings to compute prediction

– Store the social rating network

– No Learning phase

– Slow in prediction

– Most pioneer works for recommendation in SN

are memory based approaches.




Models for Recommendation in SNs

• Model based approaches

– Learn a model

– Store the model parameters only

– Extra time for learning

– Fast in Prediction

– Most models are based on matrix factorization




Memory based Approaches for


• Explore the network to find raters in the

neighborhood

• Aggregate the ratings of rater to compute the

predicted rating.

• There are different methods to calculate the

top trusted neighborhood of users.




Advogato [Levien et al., 2002]

• A trust metric to compute the top N trusted

users

• Input

– n: the number of users to trust

– x: the source users who want to trust

• A maximum flow based approach

• Advogato can be used to find the

neighborhood in rating prediction.




Advogato (cont.)

• Node Capacities:

– source user: n

– user at level l


capacity at l-1

average out-degree from l-1



Advogato (cont.)

• To apply Ford-Fulkerson algorithm for maximum

flow analysis, we should have

– Single source, single sink

– Capacities on edges

• Graph transformation– Super sink

– Split nodes into two nodes

– Node capacity c

• Edge with c-1 from negative to positive node

• Edge with capacity 1 from negative to super sink

• Infinite capacity for regular edges




Advogato (cont.)

• Nodes with flow to super link

– Top n trusted users

• Recommendation

– No distinction among the top

trusted users


• Maximum flow computed from source-negative

to super sink



AppleSeed [Ziegler 2005]

• Trust Metric

• Intuition: Spreading activation model

• Source node u is activation through injection

of energy e.

• Energy is fully propagated through edges

– Proportional to the edge weights




AppleSeed (cont.)

• Nodes are ranked according to the energy

they receive

• Issue:

– Trust is considered to be additive

– Nodes with many weakly trusted paths are

considered to be highly trusted




TidalTrust [Golbeck 2005]

• Modified breadth first search in the network

• Consider all raters v at the shortest distance

from the source user u.

• Trust between u and v




TidalTrust (cont.)

• Predicted rating

• Only considers raters at the shortest distance:

– Loss of information

– Lower recall




MoleTrust [Massa et al., 2007]

• Similar to the idea in TidalTrust

• Considers raters up to a maximum-depth d.

• Backward exploration in trust computation

• Tuning d: Trade-off between accuracy and

recall




Memory based Approaches for


• How far to go into network?

– Tradeoff between Precision and Recall

• Far neighbors on the exact target item

• Trusted friends on similar items




TrustWalker [Jamali et al., 2009]

• Random walk based model

• Combines item-based recommendation and

trust-based recommendation.

• Performs several random walks on the social

network.

• Each random walk returns a rating on the

exact target item or a similar item.

• Prediction = aggregate of all returned ratings




TrustWalker – Single Random Walk

• Starts from source user u0.

• At step k, at node u:

– If u has rated i, return ru,i

– With Φu,i,k , the random walk stops

• Randomly select item j rated by u and return ru,j .

– With 1- Φu,i,k , continue the random walk to a

direct neighbor of u.




TrustWalker (cont.)

• Item similarities

• Φu,i,k

– Similarity of items rated by u and target item i.

– The step of random walk




TrustWalker (cont.)

• Special cases of TrustWalker

– Φu,i,k = 1• Random walk never starts.

• Item-based Recommendation.

– Φu,i,k = 0• Pure trust-based recommendation.

• Continues until finding the exact target item.

• Aggregates the ratings weighted by probability of reaching them.

• Existing methods approximate this.




TrustWalker – Extension

• TrustWalker can be applied to recommend a

list of top-N items [Jamali et al., 2009.b]

– Every random walk stops at a user v.

– All items ranked highly by v are returned as the

result of the random walk.

– Result of several random walks are merged into a

list of top-N items.




Memory based Approaches: Experiments

RMSE Results on Epinions





Result for cold start users on Epinions





Result for cold start users on Epinions





Result for all users on Epinions





Result for all users on Epinions






RMSE Results on Flixster



Memory based Approaches: Summary

• Not learning a model

– No Learning phase

– Explore the network to find raters

• Need to store the SRN

• Slow in the prediction phase due to

exploration




Model based approaches for

Recommendation in SNs

• Recently have attracted attention

• Most common approach: Matrix factorization

• Latent features for users

• Latent features for items




SoRec [Ma et al., 2008]

• Matrix factorization model

– Factorize the ratings and links together

– Social network as a binary matrix

• Latent factors for items (as in MF)

• Two latent factors for users

– One for the initiator

– One for the receiver




FIP [Yang et al., 2011]

• Factorizes both rating matrix and the social network

• Similar to the idea in SoRec

• Assumes undirected network

• FIP vs. SocRec

– SocRec: directed graph

– FIP uses features as priors

– Choice of the user factor determining the observed rating is arbitrary




Social Trust Ensemble [Ma et al., 2009]

• Social Trust Ensemble (STE)

• Linear Combination of

– Basic matrix factorization and

• Latent factors of the user and the item determine the

observed rating.

– Social network based approach

• Latent factors of the neighbors and the latent factor of

the item determine the observed rating.




Social Trust Ensemble (cont.)

• The STE model





• The STE model





• Issues with STE

– Latent factors of neighbors should influence the

latent factor of u not his ratings

– STE does not handle trust propagation

– Learning is based on observed ratings only




SocialMF [Jamali et al., 2010]

• Social influence � behavior of a user u is

affected by his direct neighbors Nu.

• Latent characteristics of a user depend on his

neighbors.

• Tu,v is the normalized trust value.




SocialMF (cont.)




SocialMF (cont.)

• Properties of SocialMF

– Trust propagation

– Learning the user latent factor is possible with

existence of the social network only

• No need to fully observed rating for learning

• Appropriate for cold start users and users with no

ratings

• Similar ideas [Ma et al. 2011]




Results for Epinions

• Gain over STE: 6.2%. for K=5 and 5.7% for K=10

Mohsen Jamali, Social Matrix Factorization 126



Results for Flixster

• SocialMF gain over STE (5%) is 3 times the STE gain over BasicMF (1.5%)




Experiments on Cold Start Users




Analysis of Learning Runtime

• SocialMF:

• STE:

• SocialMF is faster by factor


N # of Users

K Latent Feature Size

Avg. ratings per user

Avg. neighbors per user

r

t



Generalized Stochastic Block Model for

Recommendation in Social Networks [Jamali et al., 2011]

• Social influence and selection lead to

formation of communities/groups

• Users may belong to different groups in their

actions

– Teacher interacting with students or his/her son

– Digital Camera when being rated

• Clustering based methods for recommendation




GSBM (cont.)

• Extending mixed membership stochastic block

model [Airoldi et al. 2008]

• Users probabilistically act as a member of one

of the groups in their actions.

• Every item is considered to belong to a latent

group when it is being rated.

• The relation between users and items is

governed by the relation between groups.

131



GSBM – Graphical Model

132



GSBM (cont.)

� Sample the social relation,

133

P(g1-->g2)K1

K1

g1

g2

BT



GSBM : Experiments on Rating Prediction

134

FlixsterEpinions



GSBM: Experiments on Link Prediction

135

Epinions Flixster



Link Prediction

• Emergence of online social network– The need to get connected to other people led to link

prediction

• Problem Definition– Given a user pair (u,v), estimate the probability of

creation of the link u�v

– Given a user u, recommend a list of top users for u to connect to.




Link Prediction (cont.)

• Link prediction vs. Rating prediction


Strength of a relation between

a users and another user

Strength of a relation between

a users and an item


0.80.70.6 3 5 4


Link Prediction Methods

• Pair-wise similarity based approach

– Roots in social selection

– Users with highest similarity to u are

recommended to u.

– Every user u is represented by his/her observed

features, properties and past activities such as

ratings and clicks.




Link Prediction- Similarity based Methods

• Defining similarity measure between A and B– the ratio between the amount of information needed

to state the commonality of A and B and the information needed to fully describe what A and B are [Lin 1998]:

• Special Cases: – Cosine similarity

– Pearson correlation

– Jaccard’s coefficient




Link Prediction Methods (cont.)

• Network Topology based methods

– Common neighbors

– Jaccard’s coefficient

– [Adamic and Adar 2003]

– Preferential attachment [Newman 2001]

• Initially proposed for modeling network growth

• Measure similarity, based on direct neighbors




Link Prediction – Path based Methods

• Katz [Katz 1953]:

– pathlA,B: number of paths of length l from A to B

• Hitting time [Liben-Nowell et al., 2003]

– score(A,B): Average number of steps for a random

walk to reach B starting from A




Link Prediction – Path based Methods

• Random walk with restart [Pan et al., 2004]– A random walk starts from A. At each step, with

probability α the random walk restarts

– score(A,B): probability of being at B during the random

walk.

• SimRank [Jeh et al., 2002]– Two user are similar to the extent that they are joined to

similar neighbours.




Link Prediction Methods (cont.)

• MF based models [Rennie et al., 2005]

– Social network as a binary matrix

– Similar to MF methods for rating prediction

– Factorize the network matrix into product of lower rank matrices (representing user factors)

– Advanced version in [Yang et al., 2011]

• Latest advances:

– Supervised random walks [Backstrom et al., 2011]

• Random walk based approach

• Considering properties of links and user attributes




Analyzing Social Networks with Distrust

• Relations between users on social media sites

often reflect a mixture of positive and

negative interactions [Leskovec et al., 2010].

• User can express distrust on other users

– E.g. block some users in eBay, Google+




Analyzing Social Networks with Distrust (cont.)

• Few works have addressed negative relations

– [Leskovec et al., 2010]

– [Kunegis et al., 2009]

– [Brzozowski et al., 2008]

– [Guha et al., 2004]

• Prior work shifted the trust to avoid negative values

– [Kamvar et al., 2003]





• How does distrust propagate?

• Distrust propagates only one step [Guha et al., 2004]


a

c

b

d

e

g

f




• How does distrust propagate?

• Distrust propagates only one step [Guha et al., 2004]


a

c

b

d

e

g

f

a

a

a

a g

f

e

d

?

?

trust

distrust




• Signed network can be analyzed according to

two different theories [Leskovec et al., 2010]:

– Structural balance theory

• Originated in social psychology

• Triangles with three positive signs (three mutual

friends,T3) and those with one positive sign (two

friends with a common enemy, T1) are more plausible.


[Leskovec et al., 2010]




• Signed network can be analyzed according to

two different theories [Leskovec et al., 2010]:

– Theory of Status

• Positive(negative) directed link indicate that the creator of the link

views the recipient as having higher(lower) status

• These relative levels of status can be propagated along multi-step

paths of signed links

• Leads to different predictions than balance theory.





• Which theory does better explain the

relations among users in a signed social

network? [Leskovec et al., 2010]

– In undirected networks, structural balance theory

– In directed signed networks, theory of status




Recommendation in Social Networks with

Distrust

• How can distrust be exploited in

recommendation?

• Very few works addressed this problem [Ma

et al., 2009.b]

– Matrix factorization model

– Modified objective (error) function

– Maximizing the distance between factor of u and

his/her distrusted neighbor v




Recommendation in Social Networks with

Distrust (cont.)

• [Ma et al., 2009.b]

• D+: set of users that u distrusts

• : distrust score

• Experiments on Epinions show promising results




Summary

• State-of-the-art methods for recommendation in social networks– Memory based approaches

• ModelTrust [Massa 2007], Modified BFS

• TidalTrust [Golbeck 2005], Modified BFS

• TrustWalker [Jamali et al., 2009], Random Walk

– Model based approaches• SoRec [Ma et al., 2008], Matrix Factorization

• FIP [Yan et al., 2011], Matrix Factorization

• STE [Ma et al., 2009], Matrix Factorization

• SocialMF[Jamali et al., 2010], Matrix Factorization

• GSBM[Jamali et al., 2011], Stochastic BlockModel




Summary

• Link Prediction

– Pair-wise profile similarity approaches

• Information theoretic based definition of similarity

– Network topology based approaches

• Common neighbors

– Path based approaches

• Katz, Hitting time, RWR, SimRank




Summary

• Social Networks with distrust

– Propagation of distrust

– Theories behind distrust

– Recommendation with distrust [Ma et al., 2009.b]




Future Research Directions

• Exploring other machine learning models

• Privacy of recommendation in social networks

– How to preserve privacy while employing social networks?

• Improving the diversity of recommendations

– How to evaluate the diversity?

• Recommendation of cold-start items

– They are very important!




Future Research Directions

• Recommendation in mobile social networks

– Distributed algorithm

– How to exploit the user location?

• Recommendation in social networks with documents (posts)

– E.g., Twitter

– Integration with topic models




Thank You!



References

• Lada A. Adamic and Eytan Adar. Friends and neighbors on the web. Social Networks, 25(3):211–230, July 2003.

• Adomavicius G., and Tuzhilin A.: Toward the next generation of recommender systems: a surveyof the state-of-the-art and possible extensions, IEEE TKDE, 2005.

• E. M. Airoldi, D. M. Blei, S. E. Fienberg, and E. P. Xing. Mixed membership stochastic blockmodels. Journal of Machine Learning Research, 9:1981–2014, 2008.

• Alexa, http://www.alexa.com/topsites, accessed on November 26, 2011.

• Anagnostopoulus A., Kumar R., Mahdian M.: Influence and correlation in social networks, KDD

2008.

• Balabanovic, M. and Y. Shoham. Fab: Content-based, collaborative recommendation, CACM

1997.

• L. Backstrom, J. Leskovec. Supervised Random Walks: Predicting and Recommending Links in Social Networks. ACM International Conference on Web Search and Data Mining (WSDM), 2011.

• Blei D., Ng A., Jordan, M.: Latent Dirichlet allocation, Journal of Machine Learning Research,

2003.

• M. J. Brzozowski, T. Hogg, G. Szabó. Friends and foes: ideological social networking. Proc. ACM CHI, 2008.

• Crandall D., Cosley D., Huttenlocher D., Kleinber J., Suri S.: Feedback effects between similarity and social influence in online communities, KDD 2008.



References (cont.)

• Mukund Deshpande and George Karypis. Item based top-n recommendation algorithms. ACM

Transactions on Information Systems, 2004.

• Jennifer Golbeck. Computing and Applying Trust in Web-based Social Networks. PhD thesis, University of Maryland College Park, 2005.

• Gomez-Rodriguez M., Leskovec J., Krause A.: Inferring networks of diffusion and influence, KDD

2010.

• R. V. Guha, R. Kumar, P. Raghavan, A. Tomkins: Propagation of trust and distrust, WWW 2004.

• M. Jamali, T. Huang, and M. Ester, A Generalized Stochastic Block Model for Recommendation in Social Rating Networks, ACM RecSys 2011.

• Jamali M., Haffari G., Ester M.: Modeling the temporal dynamics of social rating networks using bidirectional effects of social relations and rating patterns, WWW 2011.

• M. Jamali and M. Ester: A Matrix Factorization Technique with Trust Propagation for Recommendation in Social Networks, ACM Recsys 2010.

• M. Jamali and M. Ester: TrustWalker : A Random Walk Model for Combining Trust-based and Item-based Recommendation, KDD 2009.

• M. Jamali and M. Ester: Using a Trust Network to Improve Top-N Recommendation for Cold Start Users, ACM RecSys 2009.

• Glen Jeh and Jennifer Widom: Simrank: a measure of structural-context similarity, KDD 2002.



References (cont.)

• S. D. Kamvar, M. T. Schlosser, and H. Garcia-Molina. The eigentrust algorithm for reputation management in P2P networks, WWW 2003.

• Leo Katz. A new status index derived from sociometric analysis. Psychometrika, 18(1):39–43, March 1953.

• Koren Y., Bell R., and Volinsky, C.: Matrix factorization techniques for recommender systems, IEEE

Computer, 2009.

• J. Kunegis, A. Lommatzsch, C. Bauckhage. The Slashdot Zoo: Mining a social network with negative edges, WWW 2009.

• J. Leskovec, D. Huttenlocher, J. Kleinberg. Signed Networks in Social Media. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 2010.

• Leskovec J., Backstrom L., Kumar R., Tomkins A.: Microscopic evolution of social networks, KDD 2008.

• Levien and Aiken. Advogato's trust metric. online at http://advogato.org/trust-metric.html, 2002.

• D. Liben-Nowell, J. Kleinberg. The Link Prediction Problem for Social Networks. Proc. 12th International Conference on Information and Knowledge Management (CIKM), 2003.

• Dekang Lin. An information-theoretic definition of similarity. In ICML 1998.

• Hao Ma, Irwin King, and Michael R. Lyu: Learning to recommend with social trustensemble, SIGIR 2009.



References (cont.)

• Hao Ma, Michael R. Lyu, Irwin King: Learning to recommend with trust and distrust relationships, ACM RecSys 2009.

• Hao Ma, Haixuan Yang, Michael R. Lyu, and Irwin King. Sorec: social recommendation using probabilistic matrix factorization, CIKM 2008.

• Paolo Massa and Paolo Avesani. Trust-aware recommender systems, ACM RecSys 2007.

• Myers S.A., Leskovec J.: On the convexity of latent social network inference, NIPS 2010.

• Monge, P. R., and Contractor, N. S.: Theories of communication networks, Oxford University Press, 2003.

• M. E. J. Newman. Clustering and preferential attachment in growing networks, April 2001.

• J.-Y. Pan, H.-J. Yang, C. Faloutsos, and P. Duygulu. Automatic multimedia cross-modal correlation discovery, KDD 2004.

• Pariser, E.: The Filter Bubble: What the Internet Is Hiding from You, Penguin Books, 2011.

• Pazzani, M.: A framework for collaborative, content-based and demographic filtering, Artificial

Intelligence Review, 1999.

• Jason D. M. Rennie and Nathan Srebro. Fast maximum margin matrix factorization for collaborative prediction, ICML 2005.

• Resnick, P., N. Iakovou, M. Sushak, P. Bergstrom, and J. Riedl. GroupLens: An open architecture for

collaborative filtering of netnews, CSCW 1994.

• Salakhutdinov, R. and Mnih, A.: Probabilistic matrix factorization, NIPS 2007.



References (cont.)

• Sarwar, B., G. Karypis, J. Konstan, and J. Riedl: Item-based Collaborative Filtering Recommendation

Algorithms, WWW 2001.

• Soboroff, I. and C. Nicholas: Combining content and collaboration in text filtering, IJCAI Workshops,

1999.

• Wang C., and Blei C.: Collaborative Topic Modeling for Recommending Scientific Articles, KDD 2011.

• Wasserman, S. and K. Faust: Social network analysis: methods and applications, Cambridge

University Press, 1994 .

• Yang, S.H., Long, B., Smola, A., Sadagopan, N., Zheng, Z., and Zha, H., Like like alike - Joint Friendship

and Interest Propagation in Social Networks, WWW 2011

• Cai Nicolas Ziegler: Towards Decentralized Recommender Systems, PhD thesis, University of Freiburg, 2005.


Documents

Mining Social Networks for Recommendation€¦ · box on Yahoo's home page ... Facebook friend recommendations ... Jamali & Ester: Mining Social Networks for Recommendation, Tutorial