1

Focused Matrix Factorization For AudienceSelection in Display Advertising

Bhargav Kanagal 1 Amr Ahmed 1 Sandeep Pandey 3 Vanja Josifovski 1 Lluis Garcia-Pueyo 1 Jeff Yuan 2

1 {bhargav,amra,vanjaj,lgpueyo}@google.com, 2 { yuanjef}@yahoo-inc.com, 3 spandey@twitter.com1 Google Inc, 2 Yahoo! Research, 3 Twitter Inc

1,3 Work done while at Yahoo! Research

Abstract— Audience selection is a key problem in displayadvertising systems in which we need to select a list of users whoare interested (i.e., most likely to buy) in an advertising campaign.The users’ past feedback on this campaign can be leveraged toconstruct such a list using collaborative filtering techniques suchas matrix factorization. However, the user-campaign interactionis typically extremely sparse, hence the conventional matrixfactorization does not perform well. Moreover, simply combiningthe users feedback from all campaigns does not address this sinceit dilutes the focus on target campaign in consideration. To resolvethese issues, we propose a novel focused matrix factorizationmodel (FMF) which learns users’ preferences towards the specificcampaign products, while also exploiting the information aboutrelated products. We exploit the product taxonomy to discoverrelated campaigns, and design models to discriminate betweenthe users’ interest towards campaign products and non-campaignproducts. We develop a parallel multi-core implementation ofthe FMF model and evaluate its performance over a real-worldadvertising dataset spanning more than a million products. Ourexperiments demonstrate the benefits of using our models overexisting approaches.

I. INTRODUCTION

From its meager beginnings in the 1990’s, online displayadvertising has grown into a $10 billion dollar industry in20111. Today, an increasing number of advertisers launchconversion-oriented ad campaigns where the goal is to displayads to those users who are most likely to show interest andrespond positively to the campaigns (e.g., in terms of productpurchases). Hence, the problem for the advertising network isto select such a list of users. This problem is typically referredto as “audience selection” or “audience retrieval” [5], [18].

Traditionally, advertisers have selected audiences using gen-eral demographic attributes such as gender, location and age.However, lately, the trend has been to avoid making suchvague generalizations, and instead customizing the campaignaudience using the set of users that have responded positivelyto the campaign in the past (as identified by the advertiser).This type of information is reminiscent of the user responsematrix in the recommender systems [3], [14], [15], [17]settings. Here, the input data is matrix in which the rows of thismatrix represent users, the columns are products, and matrixcells indicate whether a user has purchased a given product.The only difference being that now we need to recommendusers for a given campaign product(s), instead of the otherway around. For instance, for a target ad campaign from an

1According to emarketer.com, including video advertising.

electronics seller that seeks to advertise a set of electronicitems from that seller, we want to recommend users who, giventheir past product purchases, are likely to buy electronic itemslike cameras, cell phones, etc.

The state of the art techniques in recommender systemssuch as matrix factorization [14], [17] and other approachescould be used to solve the audience retrieval task. However,as we show in our experimental analysis, they perform poorly– this is because the data set (user-product response matrix)in our application consists of real product purchases and isvery sparse (on average, one in a million advertisement leadsto a product purchase). The problem is even more pronouncedfor campaigns that are new or small in size in terms of thenumber of products spanned. One approach to deal with thesparsity challenge is by borrowing information from otherongoing as well as historical advertising campaigns. The keyintuition is that by exploiting other campaigns, we can enrichthe user-product response matrix, thus leading to better matrixfactorization and better audience retrieval in turn. Note that inthe extreme case, this would mean considering the completeuser-product response matrix using all the present and pastcampaigns to perform audience retrieval for the given targetcampaign. However, as shown later in the paper, this doesnot perform very well since it treats each campaign equallyand loses focus on the target campaign in consideration (sincethe optimization minimizes error over all the campaigns, ratherthan just this campaign). In other words, it is important to keepfocus while borrowing information from other campaigns, thusmotivating the focused matrix factorization approach proposedin the paper.

There are several challenges that need to be addressedin building such a framework. First, the information shouldbe borrowed from those campaigns which are similar to thetarget campaign while ignoring those which are not. Second,the campaign similarity and dis-similarity must themselvesbe learned using the sparse response matrices. Third, theinformation transfer needs to be adaptive, i.e., when the targetcampaign is new and does not have enough response values ofits own, it should borrow more information from others (i.e.,cold start). But as the campaign gets older and accrues moredata, it should gradually decrease the information transfer andmaintain its specificity. Fourth, the approach must scale tomillions of users and products as is the case with real worldadvertising campaigns.

2

Our ApproachIn this paper we propose a novel focused matrix factorizationtechnique, FMF model, which learns users’ preferences to-wards the specific campaign products while also exploiting theinformation about related campaigns/products. More specifi-cally, our FMF model advances the existing audience retrievalsystems in the following way. We discriminate between theusers’ preferences in the target campaign, which we call focuspreferences, and the non-focus preferences. By this, akin to thetransfer learning approaches [19], we allow for focusing on thetarget campaign, while also drawing from the data availablein other campaigns. In particular, we learn two different setsof user preferences/factors as well as their importance towardspredicting the user response to the given campaign. Simulta-neously, we estimate the similarity/dis-similarity between thetarget campaign and other campaigns using the user-productresponse matrix to facilitate the information transfer. Also, weshow how a taxonomy over the products can be integratedwith our approach to aid in this task.

We develop an efficient parallel multi-core implementationof our FMF model and evaluate its performance over a real-world shopping dataset spanning more than a million users andproducts. Such implementation requires different processingthreads to share the model parameters with both parameterreads and updates going to shared values. This leads to bot-tlenecks with some of the model parameters that are updatedmore frequently. To resolve this issue we provide an updatecaching scheme that batches the changes of individual threadsto avoid conflicts. Our experiments demonstrate the significantbenefits of using our FMF models over existing approaches.

ContributionsIn summary, the contributions of this paper are as following:

• we introduce the Focused Matrix Factorization modelwhere we discriminate between the user interests in thetarget campaign and other campaigns, while using all theavailable information.

• we show how FMF can be applied to the audienceselection problem in display advertising.

• we have provided a parallelized implementation for multi-core machines that caches the updates of the modelparameters to avoid bottlenecks and speeds up the eval-uation by orders of magnitude.

• we provide an experimental evaluation over a real-worlddataset showing the merits of the proposed approach.

We note here that although we present FMF model in thecontext of audience selection, the approach is general and isapplicable to all the recommendation scenarios. For example,FMF can be used to train models for movie recommendationfor specific users or specific genres and so on.

Outline: The rest of the paper is organized as follows. First weformally describe the audience selection problem in Section II.Next, we describe our focused collaborative filtering model inSection III. We illustrate the algorithms to train the model inSection IV. We conclude with experiments in Section VI.

II. PROBLEM FORMULATION

We start by providing a brief overview of audience selectionin display advertising. Subsequently, we formulate it as acollaborative filtering problem.

A. Audience Selection

Display advertising constitutes graphical ads and media thatare displayed alongside the main content of the web page. Aswith traditional advertising, the advertisers’ intent is to reachusers that might be interested in their products. In particular,the performance oriented advertisers are interested to targetusers that are most likely to convert, i.e, make a purchasecorresponding to the advertisement. To this end the advertiserlaunches a campaign with the advertising network promotingits products. The advertising network now needs to select theset of users as desired by the advertiser. This task of retrievingrelevant users by the advertising network is referred to asaudience selection. For instance, an electronics company maylaunch a campaign with the network, targeting users who buyportable media players. A commonly used approach by theadvertising networks is to select users who have previouslyconverted on media players and related electronics goods.

B. Matrix Factorization for Audience Selection

As described before, each campaign consists of a set ofitems (products) that are being advertised as part of thecampaign. For each item the advertising network maintainsthe set of customers that have bought it. In other words,we have a sparse (partially populated) response matrix forcampaign c, say Xc, where Xc(i, j) = 1 if user i haspurchased item j. Given a item p, the audience selectionproblem requires us to select the set of users that are mostlikely going to purchase p. One such approach is using matrixfactorization [3], [16], [17], [21]. By factorizing this responsematrix, we project each user and item into a lower dimensionalspace (such lower dimensional projections are called latentfactors) which essentially clusters similar users and itemstogether; subsequently we measure the affinity between a userand an unpurchased item as the dot product between thecorresponding factors and finally select the top-k users withthe highest affinity for each item. We explain how to learn theuser and item factors given the historical purchases data.

The input to the audience selection system is the user-itemresponse matrix Xc (size m× n, m users and n items in thecampaign c). The goal of the system is to predict, for eachproduct p, a ranked list of users in the matrix. We assume thateach user u can be represented by a latent factor vU

u which is avector of size 1×K (K is commonly referred to as the numberof factors in the model, typically much smaller than m andn). Similarly, each item i can be represented by a latent factorvIi (also, a vector of size 1×K). User u’s affinity/interest in

item i is assumed to follow this model:

x̂ui = 〈vUu ,v

Ii 〉

Here, xui is the affinity of user u to item i, 〈vu,vi〉 representsthe dot product of the corresponding user and item factors. The

3

fj ,  

X1

X1 Xj

≈ ≈

≈ ≈

XjX1

V U

V I

V IG

V UG

XG =

V S

V I1

V NV S

V IJ

FMF

MF GMF

Fig. 1: Illustrating the different approaches considered in this paper. X1 is the user-item matrix for the target campaign. X2 · · ·XJ arethe user-item matrices for the non-target campaigns. In the Matrix Factorization (MF) approach, we just factorize the target campaign toget the user and item factors. In this case we do not utilize any non-target campaigns data. In the GMF approach, we poll together datafrom all campaigns, to get the global user-item matrix XG. We then factorize this matrix to get the user and item factors, V U

G and V GI ,

respectively. In the last approach, Focused Matrix Factorization (FMF), each user has two two factors, a target campaign factor V S andnon-target campaign factors V N . The target campaign user-item matrix, X1, is factorized using user factors V S and item factors V I1 .Each non-target campaign user-item matrix Xj is also factorized using user factors fj(V S , V N ) , and item factors V Ij . The user factorsfj(V

S , V N ) for non-target campaign j combines the V S which is shared with the target-campaign and V N which is not shared with thetarget campaign. The combination function fj() depends on the campaign j such that the more similar it is to the target campaign, the morepronounced the role of V S int this combination and as such information is transfered into the target campaign. In the text, we will discussthree different models, each of which utilizes different form of the function fj().

learning problem here is to determine the best values for vUu

and vIi (for all users u and all items i) based on the given

rating matrix Y , these parameters are usually denoted by Θ.A widely used approach to learning Θ is Bayesian personal-

ized ranking (BPR) proposed by Rendle et al. [20]. In BPR, thegoal is to discriminate between items bought by the user anditems that were not bought. In other words, we need to learna ranking function Ri for each item i that ranks i’s interestingusers higher than the non-interesting users. In other words, ifuser u1 has purchased item i and user u2 has not purchasedthe item, we must have Ri(u1) > Ri(u2). For this, we needto have: xu1i > xu2i. Based on the above arguments, ourlikelihood function p(Ri|Θ) is given by:

p(Ri|Θ) =∏i∈I

∏u1∈Li

∏u2 /∈Li

σ(xu1i − xu2i)

Here, Li is the list of users who have purchased item i.Following Rendle et al. [20], we have approximated the non-smooth, non-differentiable expression xu1i > xu2i using thelogistic sigmoid function σ(xu1i−xu2i), where σ(z) = 1

1+e−z .We use a Gaussian prior N(0, σ) over all the factors in Θ and

compute the MAP ((maximum aposteriori) estimate of Θ. Theposterior over Θ (which needs to be maximized) is given by:

p(Θ|Ri) = p(Θ)p(Ri|Θ)

= p(Θ)∏i∈I

∏u1∈Li

∏u2 /∈Li

σ(xu1i − xu2i)

We need to maximize the above posterior function, (or its log-posterior), shown below.

log p(Θ|Ri) =∑i

∑u1∈Li

∑u2 /∈Li

lnσ(xu1i − xu2i)− λ||Θ||2

The first summation term corresponds to the log-likelihood,i.e., log p(Ri|Θ) whereas the second term corresponds to thelog of the Gaussian-prior, i.e., log p(Θ). Here, λ is a constant,proportional to 1

σ2 . ||Θ||2 is given by the following expression:

||Θ||2 =∑u

||vUu ||2 +

∑i

||vIi ||2

The second term is commonly called as the regularizationterm, and is used to prevent overfitting by keeping the learnedfactors vU

u and vIi sparse.

4

1) Stochastic Gradient Descent (SGD): SGD is typicallyused to optimize objective functions that can be writtenas sums of (differentiable) functions, e.g., in the objectivefunction above, we have one function per training data point(i, u1, u2). The standard gradient descent method is an iter-ative algorithm: Suppose that we want to maximize a givenobjective function. In each iteration, we compute the gradientof the function and update the arguments in the directionof the gradient. However, computing the overall gradientrequires computing the derivative for each function in thesummation and is quite expensive when we have a largetraining dataset. In SGD, we approximate the derivative bycomputing it only at a single (randomly chosen) term in thesummation and update the arguments in this direction. Despitethis approximation, SGD has been shown to work very wellin practice [9], [7], [8], often outperforming other methodsincluding the standard gradient descent. In the above example,a given training data point (i, u1, u2) defines a term in thesummation. The derivatives with respect to the vI

i , vUu!

andvUu2

variables are shown below. Denote ci,u1,u2 to be equal to(1− σ(xu1i − xu2i)).

∂L(U, V )

∂vIi

= ci,u1,u2(vU

u1− vU

u2)− λvI

i

∂L(U, V )

∂vu1

= ci,u1,u2vIi − λvU

u1

∂L(U, V )

∂vu2

= −ci,u1,u2vIi − λvU

u2

Now, we use the above derivatives to update the appropriatefactors. Note that since we have a maximization problem, weneed to move in the same direction as the derivative. Thecorresponding update equations are shown below:

vIi = vI

i(1− ελ)− εci,u1,u2(vU

u1− vU

u2)

vUu1

= vUu1

(1− ελ)− εci,u1,u2vIi

vUu2

= vUu2

(1− ελ) + εci,u1,u2vIi

Here, ε is the learning rate which is set to a small value. Theregularization term λ is usually chosen via cross-validation.An exhaustive search is performed over the choices of λ andthe best model is picked accordingly. The overall algorithmproceeds as follows. A training data point (u, i, j) is sampleduniformly at random. The gradients are computed at thisparticular data point and the variables are modified accordingto the update rules shown below. An epoch is roughly definedas a complete pass over the data set (i.e., over all the non-zeroentries in the rating matrix). By deriving the factors for usersand items from solving the above optimization problem, wecan perform audience selection for campaign c. In particular,for each product j in campaign c, we compute the dot product(〈vUu , vIi 〉) for every user and recommend the ones with thehighest values. We refer to this approach in the rest of thepaper by Matrix Factorization (MF).

While the MF approach works well when the campaignhas a rich response matrix, it is known to struggle in thepresence of sparsity. This is particularly apparent in our casebecause (a) most tail campaigns are small in terms of the

number of items and have a “narrow” response matrix, (b) andfor new campaigns where we do not have enough historicalpurchase data (i.e., cold-start problem). One possible approachto mitigating these issues is to borrow information from othercampaigns. We describe this in the next section.

III. FOCUSED MATRIX FACTORIZATION MODEL

By bringing in other campaigns, we can enrich the user-product interaction matrix and find better insights into userpreferences. For instance, if a user often buys items fromcampaigns on computer/gaming accessories, then her buyingpattern can be leveraged for performing audience selection forrelated campaigns on cell phones, music players etc.

GMF: Global Matrix FactorizationA simple way to borrow information is by combining theresponse matrix of all campaigns. In other words, we constructglobal matrix XG which includes user-product data from allcampaigns. The global response matrix is dense and can befactorized under the BPR framework (as described before) toderive the item and user factors, V GU and V GI , respictively.Then, given any target campaign, these factors can be usedto perform audience selection. We refer to this approach byGlobal Matrix Factorization (GMF). Note that in this approach,we learn one set of user factors in this approach, irrespectiveof the target campaign, unlike the MF approach, where welearn user factors for each target campaign separately.

While GMF resolves the sparsity issue, it has other draw-backs. Since we are factorizing the global matrix, the deriveduser factors represent the global user preferences, e.g., userbuys electronics often. Such global preferences give us insightsinto the general buying pattern of the user, but they do notcapture the campaign-specific user preferences accurately. Thisis especially true for the small campaigns which are likelyto be overwhelmed by large campaigns in the global matrix.(Note that the whole idea of borrowing information is tohelp the small campaigns.) Hence, we propose our focusedcollaborative filtering models which allow us to appropriatelyborrow relevant information from other campaigns while stillretaining focus on the target campaign.

A. Focused Matrix Factorization Models (FMF)

We present three focused collaborative filtering modelsFMF1, FMF2 and FMF3 with varying degrees of sharingbetween campaigns. Subsequently, we present details abouthow to incorporate a taxonomy over the products into ourFMF framework.

FMF1

Let T denote the items in the target campaign for which wewant to perform audience selection. Let N denote the itemsin the campaigns that are not part of the target (i.e., non-targetcampaigns, T ∪ N = I). This includes the campaigns whichare currently running as well as old campaigns which wererun in the past. The key idea behind FMF1 is that instead oflearning one set of user preferences (i.e., factors) like GMF,we allow each user to have two sets of preferences: one set

5

for the target campaign (focus preferences) and the other forthe non-target campaigns (non-focus preferences). We definethis formally next.

Let v1u and v2

u denote the factors for user u for the targetand non-target campaigns respectively. To allow informationtransfer between the target and non-target campaigns, weconstrain these factors in the following manner: v1

u = vSu +vTuand v2

u = vSu + vNu . Here vSu captures the information thatcan be shared between the target and non-target campaigns,vTu captures the residual specific to the target items and vNufor the non-target items. Then, the affinity xui between useru and item i in the FMF1 model is written as:

xui =

{〈vSu + vTu , v

Ii 〉 if i ∈ T

〈vSu + vNu , vIi 〉 if i ∈ N

Our FMF1 model is stringent in terms of model variables butstill general enough to capture both MF and GMF models thatwere proposed before. For instance, if the target and non-target items are completely independent and do not share anyinformation, then vSu can be set to 0 and only vTu will beused to compute user affinity for the target items (i.e., the MFmodel). On the other hand, if the target and non-target itemsare completely alike, then the residual vectors vTu and vNu canbe set ot 0 and the shared vector, vSu , is used for the affinitycomputation (i.e., the GMF model).

Without loss of generality, we can simplify the FMF1 modelby setting vTu to 0. Effectively, the shared factors in the newmodel, say v′Su , can be thought of as vSu +vTu and the residualvector v′Nu as vNu − vTu . Doing this not only reduces thenumber of factors that need to be estimated, but also makesthe model mathematically identifiable (by breaking symmetrybetween the targeted and the non-targeted campaigns) andhence interpretable. Thus, our final FMF1 model is:

xui =

{〈vSu , vIi 〉 if i ∈ T〈vSu + vNu , v

Ii 〉 if i ∈ N

Thus in FMF1, we have fj(V S , V N ) = vSu + vNu .

FMF2

While FMF1 captures the sharing between the target campaignand the non-target campaigns (through vSu ), it borrows identi-cal amounts of information from all the non-target campaigns.This will “confuse” the model if the non-target campaigns aredifferent and represent a diverse set of items. For instance,a target campaign on “electronic” items must borrow morefrom a non-target campaign on “computers” than the one on“furniture”.

Essentially, we need to take into account the similaritybetween the target and non-target campaigns while sharinginformation. To allow this, we introduce an αj variable forevery non-target campaign j from which we want to borrow(letNj be the set of items in the non-target campaign j) to controlthe degree of sharing. Note that we also need to learn the αvalues from the data. A large positive value for αj wouldindicate a large correlation between the target and the jth

non-target campaign, a small value indicates the absence of

interaction, and a negative value suggests anti-correlation. Themodel FMF2 is shown below:

xui =

{〈vSu , vIi 〉 if i ∈ T〈αjvSu + vNu , v

Ii 〉 if i ∈ Nj

Thus in FMF2, we have fj(V S , V N ) = αjvSu + vNu .

We pictorially illustrate this model using a plate-based graphi-cal model [12] representation in Figure 2. In the user plate, wehave random variables vNu and vSu that denote the user factors.In the item plate, we have random variables αj and vIi thatdenotes the sharing factor and the item factor respectively. Theshaded variable xui corresponds to the observed entities in theuser-item matrix. The other random variables wIj correspondto the taxonomy, which we discuss in Section III-B.

FMF3

The FMF2 model allows borrowing information differentlyfrom different campaigns by keeping αj variables. We canfurther generalize this model to not only have different αj butalso different vNj

u for each non-target campaign j. In otherwords, each non-target campaign has its own residual vector tocapture its specificity. However, this results in too many userfactors (K factors per campaign, multiplied by the numberof non-target campaigns) and can be difficult to estimate inpractice. To avoid this, we set each v

Nju to be (1 − αj) · vNu

and hence, keep the number of user factors to be 2 · K asbefore.

Essentially, for each campaign, target or non-target, theeffective user factor is a linear combination of vSu and vNuwith αj being the mixture coefficient. Since αj for the targetcampaign is 1 (i.e., the campaign is completely correlated withitself), we get the effective user factor for the target campaignto be vSu . Thus, the FMF3 model looks like:

xui =

{〈vSu , vIi 〉 if i ∈ T〈αjvSu + (1− αj)vNu , vIi 〉 if i ∈ Nj

Thus in FMF3, we have fj(V S , V N ) = αjvSu +(1−αj)vNu .

Moreover, we restrict αj to be in the range [0, 1]. We picto-rially illustrate the FMF model in relation to the MF and GMFapproaches in Figure 1.

We would like to point out here that while using additionalfactors to model user preferences doubles the number ofparameters that we need to train, we use appropriate regular-ization to keep them correlated – thereby the effective numberof parameters is much less. As we mention in Section VI-A.4,we use cross-validation to select the best parameters for theFMF models.

B. Leveraging Item Taxonomy

In this section, we show how a taxonomy over prod-ucts/items can be integrated in the FMF framework. Thisalleviates the sparsity issue and improves the performance ofour models (as shown in the experiments).

As described in Section I, sparsity is a prevalent issuewith ad campaigns. Tail campaigns contain very few items

6

vSu

vu vIi

wIp0(i) wI

p1(i) wIp2(i) wI

p3(i)

σ

ITEM FACTOR

xuiUSER FACTOR

vNu

αj

σ

Fig. 2: Graphical model description of FMF2 model. Random vari-ables in the user plate include user factors vNu and vSu . Randomvariables in the item plate include item factor vIi and the taxonomy-node factors wI (Section III-B). Note that αj is in the item platesince the campaign index j is unique for a given item i.

and are therefore sparse as a result, while large campaignsstruggle with sparsity when they are new. Our FMF modelsfrom Section III exploit information from other campaigns toresolve this problem, but we can further boost the performanceof our models by leveraging an additional source of infor-mation: taxonomy over the products, which are commonlyavailable [2], [13]. The taxonomy provides lineage for aproduct in terms of the parent categories that it belongs to,e.g., an Apple iPod is a portable media player, which itself isan electronics item.

We incorporate taxonomy using a hierarchical additivemodel [10], [11] over the item factors. In particular, weintroduce item factors wIj for all nodes in the taxonomy(including the actual items, which are considered as leavesof the taxonomy tree) and define the item factors using thefollowing equation:

vIi =

L∑`=0

wIpl(i)

where pl(i) denotes the lth ancestor of item i. Now, every itemfactor has a contribution from each of its ancestors. Note thatthis allows us to derive factors for even those items whichmay not have any purchases (and are hence not trained) byusing the estimate of their parent in the taxonomy. And as weget more purchases for this item, the models allows the itemfactor to get away from the parent if necessary.

IV. MODEL LEARNING AND INFERENCE

In this section, we describe algorithms for learning the FMFmodels. Specifically, we illustrate the approach for the FMF2model. As mentioned in Section III, we use the Bayesianpersonalized ranking (BPR) objective function of Rendle etal. [20]. We use the stochastic gradient descent optimizationalgorithm [7] for training our models.

The output of our model needs to be a ranking functionRi for each item i in the target campaign T that ranks theusers according to who is most likely going to purchase the

item. From the training data, if we know that user u1 hasbought item i and that another user u2 has not bought itemi, then we must have Ri(u1) > Ri(u2). The BPR objectivefunction essentially enforces this criterion for all items andevery pair of users within a given item. Before discussing theobjective function, we present some of the notations used. Theparameters that we need to learn are the user factors vSu , vNu ,and αj for all campaigns and the item factor matrix vIi . Wedenote the union of these parameters using Ψ. Also, denoteB(u1) to be the set of items that are bought by user u1. Also,let T denote the set of items in the target campaign and let Njdenote the set of items in the non-target campaign j. The log-likelihood function F (vS , vN , vI , α) for this case (w/ BPR) isgiven by:

F (ψ) =∑i∈I

∑u1:

i∈B(u1)

∑u2:

i/∈B(u2)

log σ(xu1i − xu2i)− λΨ||Ψ||2

We wish to treat the campaign items differently from the non-target campaign items, i.e., we penalize the errors on the targetcampaign much more than the non-target campaigns by usingweights A and B for respective terms in the summation.Thelog-likelihood expression is now given by:

A∑i∈T

∑u1:

i∈B(u1)

∑u2:

i/∈B(u2)

[log σ(〈vSu1

− vSu2, vIi 〉)

−λ||vS ||2 − λ||vI ||2]

+

B∑i/∈T

∑u1:

i∈B(u1)

∑u2:

i/∈B(u2)

[log σ(〈αj(vSu1

− vSu2) + vNu1

− vNu2, vIi 〉)

−λS ||vS ||2 − λI ||vI ||2 − λN ||vN ||2 − λα||α||2]

where λ’s denote the regularization constants. To use SGD,we sample a term from the above summation, which we denoteusing (i, u1, u2). Depending on whether the item is fromthe target campaign (i.e., i ∈ T ) or from some non-targetcampaign j (i.e., i ∈ Nj), we obtain two sets of gradientswhich are shown in Figure 3.Even though the objective function is highly non-convex,stochastic gradient descent has been shown to work well [22],[21], [20] with BPR on real datasets. To handle the localminima, we use multiple initializations of our factor matricesand select the best performing model via cross-validation.

Training model FMF3

The gradient rules for training FMF3 model is similar to theones shown in Figure 3 and we do not present it owing tolack of space. However, for this model to be meaningful,we enforce that all values of αj ∈ [0, 1] by using projectedgradient [6] over the αj . Essentially, whenever the gradientrules force the value of αj to be less than 0 (more than 1),we force it to be exactly 0 (1). This enables us to intuitivelyinterpret the αj parameter as determining the fraction of theuser’s local and global interests.

7

ci,u1,u2= 1− σ(〈vSu1

− vSu2, vIi 〉)

∂F

∂vIi= A(ci,u1,u2(vSu1

− vSu2)− λvIi )

∂F

∂vSu1

= A(ci,u1,u2vIi − λvSu1

)

∂F

∂vSu2

= A(−ci,u1,u2vIi − λvSu2

)

(i) item i is in the target campaign (i.e., i ∈ T ).

c′i,u1,u2= 1− σ(〈αj(vSu1

− vSu2) + vNu1

− vNu2, vIi 〉)

∂F

∂vIi= B(c′i,u1,u2

(αj(vSu1− vSu2

) + vNu1− vNu2

)− λvIi )

∂F

∂vSu1

= B(αjc′i,u1,u2

vIi − λvSu1)

∂F

∂vSu2

= B(−αjc′i,u1,u2vIi − λvSu2

)

∂F

∂vNu1

= B(c′i,u1,u2vIi − λNvNu1

)

∂F

∂vNu2

= B(−c′i,u1,u2vIi − λNvNu2

)

∂F

∂αj= Bc′i,u1,u2

〈vSu1− vSu2

, vIi 〉

(i) item i is in the non-target campaign j (i.e., i ∈ Nj).Fig. 3: Update rules for the FMF2 model.

GLOBAL STATE VARIABLES

Thread T1 Thread T2 Thread T3 Thread T4

Computing gradient Update step

Global variables are used to compute gradient

V S V N

if(αL4 − αG

4 ) > τ

αG = αG + αL4 − αG

4

αG V I

αG1

αL1 αL

2

αG2 αG

3

αL3 αL

4

αG4

Fig. 4: Pictorial description of the multi-threaded implementation.Note that the global variables are used to compute the gradients.The global variable is updated whenever the difference exceeds thethreshold.

V. IMPLEMENTATION

We developed a multi-core implementation of our focusedcollaborative models in C++. We used the BOOST [1] librarypackage for storing the factor matrices.

A. Parallelizing Training & Evaluation

Our multi-threaded approach is developed using locks. Theglobal state maintained by the SGD algorithm consists of the 3factor matrices {vS , vN , vI} and the α vector. We introduce alock for each row in our factor matrices. In the SGD algorithm,in each iteration of training, we execute 3 steps. In the firststep, we sample a 3-tuple (i, u1, u2). In the second step, weread the appropriate user and item factors and compute thegradients with respect to them. Before reading, we obtaina read-lock over the factor and release it after reading. Inthe third step, we update the factor matrices based on thegradients. Before writing we obtain a write lock on the itemfactor and subsequently release the lock once we update thefactor.

Each user and item factor matrix has a large number of rows(over 500,000). Even with a fairly large number of threads,the contention over such rows is fairly small. However, the αvector is relatively small (equal to the number of campaigns,as in FMF2 and FMF3 models). Therefore, there is much morecontention on this vector. As we show in Section VI, usinglocks over such small vector can result in significant increasein the processing time. To alleviate this problem, we proposeto use a novel caching technique which we illustrate below.

Caching:We illustrate the caching technique for a scalar α value (thiscan be easily generalized to vectors). In this technique, eachthread Ti maintains two values in its cache: αGi which is thevalue with which the current thread started off, and αLi whichis the locally cached value. We maintain that the |α−αG| <=τ where τ is a given tolerance threshold. In each iteration,the threads read the value of αG from the global value byobtaining a read lock. Whenever a thread needs to update thevalue of α, it updates αLi using the update rules. Whenever|αGi − αLi | exceeds τ , we reconcile the locally cached copywith the global value in the following manner:

αG = αG + (αLi − αGi )

αLi = αGi = αG

We pictorially illustrate the caching in Figure 4. We use asimilar technique for caching the α vector for FMF2. We alsoparallelize evaluation similarly. Each thread takes a subset ofitems in the target campaign and ranks the users within eachitem independently. Note that we only need to read the factormatrices in this case.

VI. EXPERIMENTAL EVALUATION

In this section, we present the results of our experimentalevaluation. We compare our proposed FMF techniques with thealternative approaches MF and GMF. We start with a descriptionof the data set and the evaluation metric.

A. Experimental Setup

1) Dataset: To evaluate our proposed models, we use thelog of previous advertising campaigns obtained from a majoradvertising network. The dataset contains information aboutthe items corresponding to various advertising campaigns and

8

an anonymized list of users who actually responded to thecampaign by making a purchase of the campaign item. Inaddition, we have a taxonomy over the various items in thecampaigns. For our experiments we sample a fraction ofcampaigns from this dataset. In this sample, we have about50,000 users and around a million items in the taxonomy.The taxonomy itself is 3 levels deep, with around 1500 nodesat lowest level, 270 at the middle level and 23 top levelcategories. As described in Section II, a campaign is a set ofrelated items. We have a total of 23 campaigns in our dataset.For each of the campaigns, we do the following: we select itas a target campaign and put the remaining set of campaignsas the additional non-target campaigns. Each campaign ischaracterized by the number of items that is targeted by it,number of purchases that are present, how homogeneous theitems are and how much it is related to other campaigns. We donot report the campaign identifiers due to anonymity reasons,we denote them using C1, C2 and so on.

0.73

0.74

0.75

0.76

0.77

0.78

0.79

0.8

0.81

0.82

0.83

30 40 60 80

AU

C

Number of factors

MF FMF

Fig. 5: Figure shows the improvement in performance over allcampaigns using our proposed FMF model over the baseline matrixfactorization approach.

To construct the user-item data matrix necessary for collabo-rative filtering, we select all the purchases made by the users inthe above set of campaigns and sort them by their timestamps.For each item we select a random timestamp and select allthe data prior to this timestamp into the training dataset. Therest of the data is used for evaluation. The last T transactionsin the training dataset are used for cross-validation and firstT transactions in the test dataset are used for prediction andreporting the error estimates. In all experiments, we use T = 1.

2) Comparison Systems: We compare our proposed modelsFMF1, FMF2 and FMF3 against the following methods.

1) MF: Here, we use the MF approach described in Sec-tion III as a baseline.

2) GMF: We also compare against the global matrix factor-ization (Section III).

3) GMF(t), MF(t): In addition, we use the above modelsalong with the taxonomy extension (proposed in Sec-tion III-B).

3) Metrics: We use the area under the ROC curve (AUC)to compare our model with the above systems. AUC is a com-

monly used metric for testing the quality of rank orderings. Itgives the probability with which a random positive example isranked higher than a negative example in the ordering (i.e., apure random ordering achieves an AUC of 0.5). Suppose thelist of users to rank (for a given item) is U and our groundtruth (i.e., the set of users that actually bought the item) isB. Also suppose r(u) is the numerical rank of the user uaccording to our model (from 1 . . . n). Then, the formula tocompute AUC is given by:

1

|B||U \B|∑

u∈B,v∈U\B

1(r(u) < r(v))

Here, 1 is the indicator function that is 1 if the conditionis satisfied and 0 otherwise. An alternative metric could be tomeasure precision/recall at a certain rank in the list. However,different campaigns may have different requirements in termsof precision and recall. Hence, selecting a rank at whichto evaluate precision such that it would be suitable for allcampaigns is not possible. Instead, we use AUC since itcombines the prediction performance over all ranks into asingle number.

4) Cross-validation/Parameter sweep: For each of the ex-periments that we illustrate below, we executed a parametersweep over our MapReduce cluster. The parameters we sweepover included λU , λI , λN and K, the number of factors. Inaddition, since our objective function is non-convex, for eachsetting of the parameter we evaluated 4 different initializationsand picked the best initialization for each configuration, interms of performance on the validation dataset. Finally, wechose the AUC over the test set for a given number of factorsto report in our experiments.

B. Experimental Results

1) Improvement Over the Baselines: In the first experiment,we compare the GMF, MF and FMF2 techniques for differentcampaigns. For each campaign, we learn all the above modelsand evaluate the accuracy of prediction over the campaignitems using the AUC metric. We report the average AUCacross all the campaigns in our data set in Figure 5. As shownin the figure, the AUC for FMF2 is higher than the AUC valuesfor the MF model. Next, we examine the performance of theFMF models over the individual campaigns. We drill downacross four different campaigns C1, C2, C3 and C4 whichare chosen such that they representative of all the campaignsin the dataset. We show the results for the above campaigns inFigure 6 (i-iv). We make the following observations: First, forcampaigns C1, C3 and C4, the improvement of FMF over theMF model is substantial (over 7% for campaign C1), howeverthe improvement for campaign C2 is less than 2%, which isnot statistically significant. We attribute this to the fact thatthe number of purchases in C2 is high, and subsequently, theresponse matrix is not as sparse as those for other campaigns.Hence, the additional benefit of FMF is not as pronounced.

Second, we note that between the GMF and MF methods,there is no clear winner. In Figure 7, we plot the best AUCvalues across all factor sizes for each campaign. As shown inthe figure, in campaign C2, MF outperforms GMF, whereas

9

0.68

0.7

0.72

0.74

0.76

0.78

0.8

30 40 60 80

AU

C

Number of factors

MF FMF

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

30 40 60 80

AU

C

Number of factors

MF FMF

(i) Campaign C1 (ii) Campaign C2

0.75

0.76

0.77

0.78

0.79

0.8

0.81

0.82

0.83

0.84

30 40 60 80

AU

C

Number of factors

MF FMF

0.69

0.7

0.71

0.72

0.73

0.74

0.75

0.76

30 40 60 80

AU

C

Number of factors

MF FMF

(iii) Campaign C3 (iv) Campaign C4

Fig. 6: Improvement over baseline: In parts (i-iv) we show the performance of the MF and FMF models over campaigns C1, C2, C3 and C4

respectively. As shown here, FMF models consistently outperform the baseline models.

in the other campaigns, GMF outperforms MF. This occursbecause the number of purchases in C2 is high and that C2

is a large campaign which helps MF. This figure demonstratesthat blind sharing of information as performed by GMF doesnot always help, as in the case of campaigns C2.

0.6

0.65

0.7

0.75

0.8

0.85

0.9

C1 C2 C3 C4

AU

C

MF FMF GMF

Fig. 7: We summarize the best performing model (across factor sizes)for each case. Note that between GMF and MF, there is no clearwinner, i.e., in campaign C2, MF outperforms GMF

Note that our FMF2 model performs fairly well for everycampaign, since it can adapt in terms of how the information

0.84

0.86

0.88

0.9

0.92

0.94

0.96

GMF TMF FMF

AU

C

Without TaxonomyWith Taxonomy

Fig. 8: We show the improvement obtained by all the models usingthe taxonomy, and the improvement of the FMF(t) models over theGMF(t) and MF(t) baselines.

from non-target campaigns is borrowed, the extent of infor-mation that needs to be borrowed, etc. We further dissect theperformance behavior of FMF models in detail in Section VI-B.2.

Improvement Using TaxonomyNext, we compare the FMF(t) model (i.e., the FMF model

10

augmented with information about the taxonomy) against thetaxonomy-aware MF(t) and GMF(t) models. As before, weexecute a parameter sweep and determine the best AUC valuesfor each of the models, for different values of the number offactors. We show the results in Figure 8 for campaign C2. Asshown in the figure, all models benefit from using informationabout the product taxonomy, just as expected. However, theFMF(t) model still outperforms the GMF(t) and MF(t) models.Also, it is worth noting that MF draws the largest benefitfrom the taxonomy – this can be explained based on thefact that MF factorizes the target campaign matrix only andhence, is affected by sparsity the most. The taxonomy-basedsmoothing of item factors alleviates this issue, thus leading tolarge improvement in MF.

2) Understanding Focused Matrix Factorization: In thissection we conduct a thorough analysis to investigate theperformance behavior of the FMF models. We aim to identifykey characteristics that affect the FMF models.

Comparison of the different FMF ModelsWe start by comparing the FMF1, FMF2, FMF3 models witheach other. For each of the four campaigns we train the FMF1,FMF2 and FMF3 models. We show the AUCs metrics, asbefore, over the test dataset in Figure 9 (i-iv). As shownin the figure, the models FMF1 and FMF2 perform muchbetter than FMF3. We reason this is because FMF3 is muchmore constrained than the other two models (as αj values areconstrained to be between 0 and 1). However, with FMF2,the αj values are unconstrained leading to greater freedom oflearning. Indeed, in our analysis we observed some αj valueswere positive and others negative, reinforcing our belief thatsome campaigns may be negatively correlated. For brevity, wefocus on FMF2 in the remaining experiments in this section.

Effect of Campaign SizeIn this experiment, we understand the performance of FMF2model as a function of the target campaign size, i.e., thenumber of items in the target campaign. We select 8 differentcampaigns with varying sizes (ranging from a few hundredsto many thousands of items). We show the AUCs achievedby the FMF2 model in Figure 9(i). The x-axis denote thecampaign index in increasing order of campaign size. Notethat the performance of FMF2 is robust and largely unaffectedby the campaigns size.

Effect of Intra-Campaign relationship (CampaignHomogeneity)In this experiment, we explore the performance of FMF2models as a function of the homogeneity (user purchase patternsimilar across all products) of the target campaign. To test this,we create a set of hypothetical target campaign C ′(p) from anexisting campaign C for different values of p. The campaignC ′(p) is constructed in the following way: with probability p,we select an item from C and with probability 1−p, we selecta random item from the complete collection of items. The sizeof C ′(p) is kept the same as C. In the experiment, we usedp = { 0, 0.34, 0.60, 1}. We train the FMF2 model over thesecampaigns for four different values of K (number of factors),and plot the AUCs in Figure 9(ii). As shown in the figure,

the AUC scores increase as we increase the homogeneity ofthe campaign. This is expected since information transfer toa campaign is much easier (through αj) if the campaign ismuch more coherent and homogeneous.

Effect of Inter-Campaign Relationship (InformationTransfer)Next, we illustrate the effect of inter-campaign relationshipfor information transfer in the FMF2 model. Intuitively, weexpect that as we increase the positive correlation betweenthe target and the non-target campaigns, we should see moreinformation transfer and improved AUC values. We ran acontrolled experiment to verify this as follows: We picked afairly homogeneous campaign X (e.g., electronics) and split itrandomly into two parts X1 and X2. Then we picked anothercampaign Y and constructed two configurations using X1, X2

and Y as follows. In both configurations, called config 1 andconfig 2, X1 is made the target campaign, while X2 is the non-target campaign in config 1 and Y is the non-target campaignin config 2. We ignore the rest of the campaigns from thedata for this experiment. After running the FMF2 model withthese configurations, we plot their AUCs in Figure 9 (iii). Asshown in the figure, config 1 has much higher AUC than config2 since it has X2 as the non-target campaign which is highlysimilar to X1. In other words, our FMF2 model successfullymanages to transfer the information from the X2 campaign toachieve better performance.

3) Efficiency: As described in Section V, we developa multicore implementation of the training and evaluationalgorithms. To deal with lock contention over frequentlyaccessed matrix entries, we use our caching techniques. In thisexperiment, we demonstrate the trade-offs that is obtained byusing the caching technique. Recall that we update the globalvariable only if we exceed a specified threshold value. Whenthis threshold is set to 0, there is complete synchronization. Asthe threshold is increased, the synchronization with the globalcopy is performed less often, resulting in faster runtime butless accuracy. In other words, caching allows us to trade-offaccuracy for efficiency.

For the FMF2 model, we measure the time taken per epochof training with caching enabled and without caching. Theresults are shown in Figure 9(iv). We make two observationshere: First, we obtain substantial improvement using caching.For instance, the run-time is cut to almost half by enablingcaching. Second, even with a reasonably large value of thresh-old = 0.001, we did not observe any significant loss in theaccuracy of the model.

RELATED WORK

Recommender SystemsClassical approaches in collaborative filtering are basedon item-item/user-user similarity, these are nearest-neighbormethods where the response for a user-item pair is predictedbased on a local neighborhood mean [26], [24]. However,modern methods based on matrix factorization have shownto be very successful and outperform nearest neighbor meth-ods [23]. In this paper we showed how matrix factorizationcan be used for audience selection in display advertising.

11

0.68

0.7

0.72

0.74

0.76

0.78

0.8

30 40 60 80

AU

C

Number of factors

FMF1 FMF2 FMF3

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0.89

0.9

30 40 60 80

AU

C

Number of factors

FMF1 FMF2 FMF3

(i) Campaign C1 (ii) Campaign C2

0.75

0.76

0.77

0.78

0.79

0.8

0.81

0.82

0.83

0.84

30 40 60 80

AU

C

Number of factors

FMF1 FMF2 FMF3

0.69

0.7

0.71

0.72

0.73

0.74

0.75

0.76

30 40 60 80

AU

C

Number of factors

FMF1 FMF2 FMF3

(iii) Campaign C3 (iv) Campaign C4

Fig. 9: Study of FMF models: In part(i-iv), we study the relationship between the various FMF models that we propose. Essentially, we seethat FMF1 and FMF2 models outperform FMF3.

Transfer LearningThe idea of using focused collaborative filtering draws inspi-ration from transfer learning / multi-task learning. Zhang etal. [27], however our work differs from these work since weonly care about increasing the performance of the focused taskrather than learning the task structure between all tasks as in[27], which reduces the number of correlation parameters tobe learned from a quadratic number of parameters betweenall tasks to just a linear number of parameters between thefocused task and all other tasks. Another related area is collec-tive matrix factorization [25] which shares structure betweenmultiple matrix factorization tasks, however, our focus here ison efficient way of training the resulting model.

Audience Selection in Display Advertising Display adver-tising is increasingly becoming more and more performanceoriented where the goal is to identify and target customersmost suited to the advertising campaigns. Existing work onthis topic focuses on building models to characterize userinterests based on their past activities, e.g., search queries,pages browsed [18], [5]. The advertising network can trackand construct user history to build these models for targetingpurposes. In contrast, our work does not require user onlineactivities to be given, instead we mine the past purchaserecords to bring together similar users and advertisers. Tothe best of our knowledge, this is the first work to applymatrix factorization approach to display targeting. Also, it ispossible to directly extend this work for the case where user

history is given. For instance, similar to [4], we can derive userfactors by regressing on the user features, and thus combinethe purchase history as well as past online activities of theuser.

CONCLUSIONS

Display advertising has grown into a multi-billion dollarindustry in recent years. To this end, there has been extensivework on behavioral targeting and user personalization to solvethe key problem of audience selection for ad campaigns. Muchof the previous work use users’ search logs, browsing historyand other features for targeting ads to user. In this paper,we propose to use information from previous campaigns,which contains information about a user’s actual purchasesand is hence feature rich. We propose a novel focused matrixfactorization model, in which we develop techniques to borrowinformation from related campaigns while ignoring informa-tion from unrelated campaigns. As shown in our extensiveempirical study, the FMF model consistently outperforms thetraditional matrix factorization techniques over all kinds ofcampaigns. In addition, in our experimental study, we charac-terized the conditions under which we can obtain significantimprovements from our approach.

REFERENCES

[1] Boost c++ libraries. http://www.boost.org/.[2] Pricegrabber. http://www.pricegrabber.com/.

12

0.72

0.74

0.76

0.78

0.8

0.82

0.84

0.86

0.88

0 1 2 3 4 5 6 7

AU

C

size of campaign

FMF2

0.76

0.78

0.8

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0 20 40 60 80 100

AU

C

Percentage of homogeneity in campaign

30 40 60 80

(i) Effect of campaign size (ii) Effect of campaign homogeneity

0.7

0.72

0.74

0.76

0.78

0.8

30 40 60 80

AU

C

Number of factors

Config 1 Config 2

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0 0.5 1 1.5 2 2.5 3 3.5 4threshold

AUC Time (min)

(iii) Effect of inter-campaign relationship (iv) Study of caching

Fig. 10: In part(i), we study the effect of campaign size over the models. In part(ii), we show that as the campaigns are more homogeneous,we improve the recommendation accuracy. In part (iii), we show the effects of inter-campaign relationships over the recommendation. Inpart (iv), we show the accuracy-time trade-offs provided by our caching technique.

[3] D. Agarwal and B.-C. Chen. Regression-based latent factor models. InKDD, pages 19–28, 2009.

[4] D. Agarwal and B.-C. Chen. Regression-based latent factor models.In Proceedings of the 15th ACM SIGKDD international conference onKnowledge discovery and data mining, KDD ’09, pages 19–28, NewYork, NY, USA, 2009. ACM.

[5] A. Bagherjeiran, A. Hatch, A. Ratnaparkhi, and R. Parekh. Large-scale customized models for advertisers. Data Mining Workshops,International Conference on, 0:1029–1036, 2010.

[6] D. P. Bertsekas and D. P. Bertsekas. Nonlinear Programming. AthenaScientific, 2nd edition, 1999.

[7] L. Bottou. Stochastic learning. In Advanced Lectures on MachineLearning, pages 146–168, 2003.

[8] L. Bottou and O. Bousquet. The tradeoffs of large scale learning. InNIPS, 2007.

[9] L. Bottou and Y. LeCun. Large scale online learning. In NIPS, 2003.[10] J. Eisenstein, A. Ahmed, and E. P. Xing. Sparse additive generative

models of text. In ICML, 2011.[11] A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian Data

Analysis. Chapman and Hall/CRC Texts in Statistical Science, 2003.[12] M. I. Jordan. Learning in Graphical Models (ed). MIT Press, 1998.[13] N. Koenigstein, G. Dror, and Y. Koren. Yahoo! music recommendations:

modeling music ratings with temporal dynamics and item taxonomy. InRecSys, pages 165–172, 2011.

[14] Y. Koren. Factorization meets the neighborhood: a multifaceted collab-orative filtering model. In KDD, pages 426–434, 2008.

[15] Y. Koren. Collaborative filtering with temporal dynamics. In KDD,pages 447–456, 2009.

[16] Y. Koren and R. M. Bell. Advances in collaborative filtering. InRecommender Systems Handbook, pages 145–186. 2011.

[17] Y. Koren, R. M. Bell, and C. Volinsky. Matrix factorization techniquesfor recommender systems. IEEE Computer, 42(8):30–37, 2009.

[18] Y. Liu, S. Pandey, D. Agarwal, and V. Josifovski. Finding the rightconsumer: optimizing for conversion in display advertising campaigns.In Proceedings of the fifth ACM international conference on Web searchand data mining, WSDM ’12, pages 473–482. ACM, 2012.

[19] S. J. Pan and Q. Yang. A survey on transfer learning. IEEE Transactionson Knowledge and Data Engineering, 22:1345–1359, 2010.

[20] S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-thieme. L.s.:

Bpr: Bayesian personalized ranking from implicit feedback. In In: Pro-ceedings of the 25th Conference on Uncertainty in Artificial Intelligence(UAI), 2009.

[21] S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme. Factorizingpersonalized markov chains for next-basket recommendation. In WWW,2010.

[22] S. Rendle and L. Schmidt-Thieme. Pairwise interaction tensor factor-ization for personalized tag recommendation. In WSDM, 2010.

[23] R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. InNIPS, 2007.

[24] B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl. Item-basedcollaborative filtering recommendation algorithms. In WWW, 2001.

[25] A. P. Singh and G. J. Gordon. Relational learning via collective matrixfactorization. In Proceedings of the 14th ACM SIGKDD, KDD ’08,pages 650–658. ACM, 2008.

[26] J. Wang, A. P. D. Vries, and M. J. T. Reinders. Unifying user-basedand item-based collaborative filtering approaches by similarity fusion. InIn SIGIR 06: Proceedings of the 29th annual international ACM SIGIRconference on Research and development in information retrieval, pages501–508. ACM, 2006.

[27] Y. Zhang, B. Cao, and D.-Y. Yeung. Multi-domain collaborative filtering.In UAI, 2010.