View
112
Download
0
Category
Preview:
Citation preview
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
MATRIX FACTORIZATION TECHNIQUE FORRECOMMENDER SYSTEMS
Oluwashina Aladejubelo
Universite Joseph Fourier,Grenoble, France
June 6, 2015
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
About Me
Bachelor of Science, Ambrose Alli University, Nigeria(2004-2008)
IT Business Analyst, Virgin Nigeria Airlines (2009-2011)
Team Lead/Software Architect, Speckless InnovationsLimited (2011-2014)
Master of Informatics (M2 MOSIG), Universit JosephFourier, Grenoble (2014-2015)
Master Thesis on ”Distributed Large-Scale Learning” withPr. Massih-Reza Amini.
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Overview
1 Introduction
2 Matrix Factorization Methods
3 Netflix Prize Competition
4 Conclusion
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
1 IntroductionRecommender SystemsContent Filtering ApproachCollaborative Filtering ApproachContent vs Collaborative Filtering
2 Matrix Factorization MethodsMatrix Factorization Model (MFM)Stochastic Gradient DescentAlternating Least SquaresAdding BiasesAdditional Input SourceTemporal DynamicsVarying confidence levels
3 Netflix Prize Competition
4 Conclusion
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Recommender Systems
Recommender systems analyze patterns of user interest inproducts to provide personalized recommendations
They seek to predict the rating or preference that user wouldgive to an item
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Recommender Systems
Such systems are very useful for entertainment products suchas movies, music, and TV shows.
Many customers will view the same movie and each customeris likely to view numerous different movies.
Huge volume of data arise from customer feedbacks which canbe analyzed to provide recommendations
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Content Filtering Approach
creating profile for each user or product to characterize itsnature.programs associate users with matching products.
it requires gathering external information that may not beavailable
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Collaborative Filtering Approach
depends on past user behaviour, e.g. previous transactions orproduct rating
does not rely on creation of explicit profiles
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Collaborative Filtering Approach
the primary areas of collaborative filtering are neighborhoodmethods and latent factor models
neighborhood is based on computing the relationshipsbetween items or users
latent factor models tries to explain by characterizing bothitems and users on say, 20 to 100 factors inferred from theratings patterns
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Content vs Collaborative Filtering
Collaborative filtering address data aspects that are difficult toprofile.
it is generally more accurate
suffers from cold startup problem (new product / new user) inwhich case content filtering is better
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
1 IntroductionRecommender SystemsContent Filtering ApproachCollaborative Filtering ApproachContent vs Collaborative Filtering
2 Matrix Factorization MethodsMatrix Factorization Model (MFM)Stochastic Gradient DescentAlternating Least SquaresAdding BiasesAdditional Input SourceTemporal DynamicsVarying confidence levels
3 Netflix Prize Competition
4 Conclusion
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Matrix Factorization Model (MFM)
some of the most successful realizations of latent factormodels are based on matrix factorization
it characterizes both items and users by vectors of factorsinferred from item rating patterns
high correspondence between item and user factors leads to arecommendation
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Matrix Factorization Model (MFM)
MFM maps both users & items to a joint latent factor spaceof dimensionality f
the user-item interactions are modeled as inner products inspace f
each item i is associated with a vector qi ∈ Rf
each user u is associated with a vector pu ∈ Rf
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Matrix Factorization Model (MFM)
the approximate user rating is given by
r̂ui = qTi Pu (1)
carelessly addressing only the relatively few known entries ishighly prone to overfitting
observed ratings can be modeled directly with regularizationas follows
minq∗,p∗∑
(u,i)∈κ
(rui − qTi pu)2 + λ(||qi ||2 + ||pu||2) (2)
κ is a set of (u, i) pairs for which rui is knownOluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Stochastic Gradient Descent (SGD) - Simon Funk; 2006
SGD approach can be used for solving the equation (2)
For each given training case, the system predicts ru i andcomputes the prediction error
eui = rui − qTi pu
it modifies the parameters by a magnitude proportional to γin the opposite direction of the gradient, yielding∈ Rf
qi ← qi + γ.(eui .pu − γ.qi )pu ← pu + γ.(eui .qi − γ.pu)
combines ease with a relatively fast runtime
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Alternating least squares
Because both qi and pu are unknown, equation (2) is notconvex
if we fix one of the unknowns the quadratic optimization canbe solved optimally
when all pu are fixed the system recomputes the qi by solvinga least-squares problem and vice versa
each step decreases the minimization problem untilconvergence
massively parallelizable
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Adding Biases
rating values are also affected by biases independent of anyinteraction
a first-order approximation of the bias involved in rating rui is
bui = µ+ bi + bu (3)
µ denotes the average rating, bu and bi are the observeddeviations of user u on item i
therefore,
r̂ = µ+ bi + bu + qTi pu (4)
equation(2) also becomes,
minq∗,p∗,b∗
∑(u,i)∈κ
(rui−µ−bu−bi−qTi pu)2+λ(||qi ||2+||pu||2+b2u+b2
i ) (5)
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Additional Input Sources
cold start problem could be as a result of user supplying veryfew ratings-difficulty to conclude on their taste
behavioural information such as purchase and browsing historycan be used for implicit feedback
let’s say N(u) denotes the set of itels for which user uexpressed an implicit preference
a new set of item factors is given by xi ∈ Rf
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Additional Input Sources
a user who showed a preference for items in N(u) ischaracterized by the vector
∑i∈N(u)
xi
normalizing the sum we have,
|N(u)|−0.5∑
i∈N(u)
xi
another information source is known as user attribute, e.g.demographics, gender, age, income level and so on
let A(u) denote set of attributes of a user u
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Additional Input Sources
a distinct factor vector ya ∈ Rf corresponds to each attributeto describe a user through the set of user-associatedattributes: ∑
a∈A(u) ya
the matrix factorization model should intergrate all signalsources, with ehanced representation:
r̂ui = µ+ bi + bu + qTi [pu + |N(u)−0.5∑
i∈N(u)
xi +∑
a∈A(u)
ya] (6)
items can get a similar treatment
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Temporal Dynamics
in reality customers’ inclinations evolve, leading them toredefine their taste
it is therefore important to accommodate this temporal effectsreflecting the dynamic, time-drifting nature of user-iteminteractions
the following terms vary over time: item biases, bi (t); userbiases, bu(t); and user preferences, pu(t)
equation (4) therefore becomes,
r̂(t) = µ+ bi (t) + bu(t) + qTi pu(t) (7)
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Varying Confidence Level
other factors like massive advertisement can influenceobserved ratings, which do not reflect long-term characteristics
hence the need for a weighting scheme or confidence
confidence can stem from available numerical values thatdescribe the frequency of actions, e.g. how much time theuser watched a show
in matrix factorization less weight is given to less meaningfulaction
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Varying Confidence Level
if confidence in observing ru i is denoted as cu i , then the modelenhances equation (5) to account for confidence as follows
minq∗,p∗,b∗
∑(u,i)∈κ
cui (rui−µ−bu−bi−qTi pu)2+λ(||qi ||2+||pu||2+b2u+b2
i ) (8)
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
1 IntroductionRecommender SystemsContent Filtering ApproachCollaborative Filtering ApproachContent vs Collaborative Filtering
2 Matrix Factorization MethodsMatrix Factorization Model (MFM)Stochastic Gradient DescentAlternating Least SquaresAdding BiasesAdditional Input SourceTemporal DynamicsVarying confidence levels
3 Netflix Prize Competition
4 Conclusion
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Netflix Prize Competition
in 2006, Netflix announced a contest to improve the state ofits recommender system
training data comprised of 100 million ratings sapnning500,000 annonymous customers’ rating of 17,000 movies
each movie was rated on a scale of 1 to 5 stars
test data was 3million ratings
the metrics was 10 percent or more root-mean-square error(RMSE) performance better than Netflix algorithm
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Netflix Prize Competition
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
1 IntroductionRecommender SystemsContent Filtering ApproachCollaborative Filtering ApproachContent vs Collaborative Filtering
2 Matrix Factorization MethodsMatrix Factorization Model (MFM)Stochastic Gradient DescentAlternating Least SquaresAdding BiasesAdditional Input SourceTemporal DynamicsVarying confidence levels
3 Netflix Prize Competition
4 Conclusion
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Conclusion
matrix factorization techniques have become a dominantmethodology within collaborative filtering recommenders
experience with the Netflix competion has shown that theydeliver accuracy superior to classical nearest-neighbortechniques
they integrate many crucial aspects of the data, such asmultiple forms of feedback, temporal dynamics and confidencelevels.
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
Reference
Y. Koren, R. Bell and C. Volinsky: Matrix Factorization Techniquesfor Recommender Systems, AT&T Labs-Research, 2009
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
IntroductionMatrix Factorization Methods
Netflix Prize CompetitionConclusion
THANK YOU!
Oluwashina Aladejubelo Matrix Factorization Techniques for Recommender Systems
Recommended