Click here to load reader

Deep Matrix Factorization Models for Recommender · PDF fileProceedings of the Twenty-Sixth International JointConference onArtificial Intelligence (IJCAI-17) 3203 Deep Matrix Factorization

  • View

  • Download

Embed Size (px)

Text of Deep Matrix Factorization Models for Recommender · PDF fileProceedings of the Twenty-Sixth...

  • Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17)


    Deep Matrix Factorization Models for Recommender Systems

    Hong-Jian Xue, Xin-Yu Dai, Jianbing Zhang, Shujian Huang, Jiajun ChenNational Key Laboratory for Novel Software Technology; Nanjing University, Nanjing 210023, China

    Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 210023, [email protected], {daixinyu,zjb,huangsj,chenjj}

    AbstractRecommender systems usually make personalizedrecommendation with user-item interaction ratings,implicit feedback and auxiliary information. Ma-trix factorization is the basic idea to predict a per-sonalized ranking over a set of items for an indi-vidual user with the similarities among users anditems. In this paper, we propose a novel matrixfactorization model with neural network architec-ture. Firstly, we construct a user-item matrix withexplicit ratings and non-preference implicit feed-back. With this matrix as the input, we present adeep structure learning architecture to learn a com-mon low dimensional space for the representationsof users and items. Secondly, we design a new lossfunction based on binary cross entropy, in whichwe consider both explicit ratings and implicit feed-back for a better optimization. The experimentalresults show the effectiveness of both our proposedmodel and the loss function. On several bench-mark datasets, our model outperformed other state-of-the-art methods. We also conduct extensive ex-periments to evaluate the performance within dif-ferent experimental settings.

    1 IntroductionIn the era of information explosion, information overload isone of the dilemmas we are confronted with. Recommendersystems (RSs) are instrumental to address this problem asthey help determine which information to offer to individualconsumers and allow online users to quickly find the person-alized information that fits their needs [Sarwar et al., 2001;Linden et al., 2003]. RSs are nowadays ubiquitous in e-commerce platforms, such as recommendation of books atAmazon, music at, movie at Netflix and referenceat CiteULike.

    Collaborative filtering (CF) recommender approaches areextensively investigated in research community and widelyused in industry. They are based on the simple intuition that

    Xin-Yu Dai is the corresponding author. This work wassupported by the 863 program(2015AA015406) and the NSFC(61472183,61672277).

    if users rate items similarly in the past, they are likely to rateother items similarly in the future [Sarwar et al., 2001; Lindenet al., 2003]. As the most popular approach among variouscollaborative filtering techniques, matrix factorization (MF)which learns a latent space to represent a user or an item be-comes a standard model for recommendation due to its scal-ability, simplicity, and flexibility [Billsus and Pazzani, 1998;Koren et al., 2009]. In the latent space, the recommendersystem predicts a personalized ranking over a set of items foreach individual user with the similarities among the users anditems.

    Ratings in the user-item interaction matrix are explicitknowledge which have been deeply exploited in early rec-ommendation methods. Because of the variation in ratingvalues associated with users on items, biased matrix factor-ization [Koren et al., 2009] are used to enhance the rat-ing prediction. To overcome the sparseness of the ratings,additional extra data are integrated into MF, such as socialmatrix factorization with social relations [Ma et al., 2008;Tang et al., 2013], topic matrix factorization with itemcontents or reviews text [McAuley and Leskovec, 2013;Bao et al., 2014], and so on.

    However, modeling only observed ratings is insufficient tomake good top-N recommendations [Hu et al., 2008]. Im-plicit feedback, such as purchase history and unobserved rat-ings, is applied in recommender systems [Oard et al., 1998].The SVD++ [Koren, 2008] model firstly factorizes the ratingmatrix with the implicit feedback, and is followed by manytechniques for recommender systems [Rendle et al., 2009;Mnih and Teh, 2012; He and McAuley, 2015].

    Recently, due to the powerful representation learning abil-ities, deep learning methods have been successfully appliedincluding various areas of Computer Vision, Audio Recogni-tion and Natural Language Processing. A few efforts havealso been made to apply deep learning models in recom-mender systems. Restricted Boltzmann Machines [Salakhut-dinov et al., 2007] was firstly proposed to model users ex-plicit ratings on items. Autoencoders and the denoising au-toencoders have also been applied for recommendation [Li etal., 2015; Sedhain et al., 2015; Strub and Mary, 2015]. Thekey idea of these methods is to reconstruct the users ratingsthrough learning hidden structures with the explicit historicalratings. Implicit feedback is also applied in this research lineof deep learning for recommendation. An extended work pre-

  • Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17)


    sented a collaborative denoising autoencoder (CDAE) [Wu etal., 2016] to model users preference with implicit feedback.Another work of neural collaborative filtering (NCF) [He etal., 2017] was proposed to model the user-item interactionswith a multi-layer feedforward neural network. Two recentworks above exploit only implicit feedback for item recom-mendations instead of explicit rating feedback.

    In this paper, to make use of both explicit ratings andimplicit feedback, we propose a new neural matrix factor-ization model for top-N recommendation. We firstly con-struct a user-item matrix with both explicit ratings and non-preference implicit feedback, which is different from otherrelated methods using either only explicit ratings or only im-plicit ratings. With this full matrix (explicit ratings and zeroof implicit feedback) as input, a neural network architecture isproposed to learn a common latent low dimensional space torepresent the users and items. This architecture is inspired bythe deep structured semantic models which have been provedto be useful for web search [Huang et al., 2013], where it canmap the query and document in a latent space through multi-ple layers of non-linear projections. In addition, we design anew loss function based on cross entropy, which includes theconsiderations of both explicit ratings and implicit feedback.

    In sum, our main contributions are outlined as follows. We propose novel deep matrix factorization models with

    a neural network that map the users and items into acommon low-dimensional space with non-linear projec-tions. We use a matrix including both explicit ratingsand non-preference implicit feedback as the input of ourmodels. We design a new loss function to consider both explicit

    ratings and implicit feedback for better optimization. The experimental results show the effectiveness of our

    proposed models which outperform other state-of-the-art methods in top-N recommendation.

    The organization of this paper is as follows. Problem state-ment is introduced in Section 2. In Section 3, we present thearchitecture and details of the proposed models. In Section4, we give empirical results on several benchmark datasets.Concluding remarks with a discussion of some future workare in the final section.

    2 Problem StatementSuppose there are M users U = {u1, ..., uM}, N itemsV = {v1, ..., vN}. Let R RMN denote the rating ma-trix, where Rij is the rating of user i on item j, and we markunk if it is unknown. There are two ways to construct theuser-item interaction matrix Y RMN from R with im-plicit feedback as,

    Yij =

    {0, if Rij = unk

    1, otherwise(1)

    Yij =

    {0, if Rij = unk

    Rij , otherwise(2)

    Most of the existing solutions for recommendation applyEquation 1 to construct the interaction matrix of Y [Wu et

    al., 2016; He et al., 2017]. They consider all observed rat-ings the same as 1. In this paper, we construct the matrix ofY with the Equation 2. The rating Rij of user ui on itemvj is still reserved in Y . We think that the explicit ratings inEquation 2 is non-trivial for recommendation because they in-dicate the preference degree of a user on an item. Meanwhile,we mark a zero if the rating is unknown, which is named asnon-preference implicit feedback in this paper.

    The recommender systems are commonly formulated asthe problem of estimating the rating of each unobserved en-try in Y , which are used for ranking the items. Model-basedapproaches [Koren, 2008; Salakhutdinov and Mnih, 2007] as-sume that there is an underlying model which can generate allratings as follows.

    Yij = F (ui, vj |) (3)

    where Yij denotes the predicted score of interaction Yijbetween user ui and item vj , denotes the model parameters,and F denotes the function that maps the model parameters tothe predicted scores. Based on this function, we can achieveour goal of recommending a set of items for an individualuser to maximize the users satisfaction.

    Now, the next question is how to define such a function F .Latent Factor Model (LFM) simply applied the dot productof pi, qj to predict the Yij as follows [Koren et al., 2009].Here, pi and qj denote the latent representations of ui and vj ,respectively.

    Yij = FLFM (ui, vj |) = pTi qj (4)

    Recently, neural collaborative filtering (NCF) [He et al.,2017] presented an approach with a multi-layer perceptron toautomatically learn the function of F . The motivation of thismethod is to learn the non-linear interactions between usersand items.

    In this paper, we follow the Latent Factor Model whichuses the inner product to calculate the interactions betweenusers and items. We do not follow the neural collaborativefiltering because we try to get the non-linear connection be-

Search related