Factorization Machine

2016/12

Outline• FM

• vs

• vs

• vs

• FM FFM FM

FM

•

• sigmoid

•

•

• w_ = 0.5 y =

•

• CTR etc

•

gbdt DNN deep-width model

•

• 0-18 19-25 25-30 30+

• onehot

• eg: =20: [0, 1, 0, 0] [1, 1, 0, 0]

•

•

•

•

• gini

• + = += etc

•

• + +

•

•

• SVM, d=2

•

•

•

• xi xj != 0

• xi xj

• CTR xi xj

•

•

•

• k << n

• W

• 0

•

• Rendle S. Factorization Machines[C]. international conference on data mining, 2010. • Rendle S. Factorization Machines with libFM[J]. ACM Transactions on Intelligent Systems and Technology, 2012, 3(3).

• ( )

• 0

• SGD

• ALS

• MCMC

•

• libFM

• Spark FM DMSPA

• difacto DMLC PS FM

• tensorflow FM https://github.com/kopopt/fast_tffm

Rendle S. Factorization Machines with libFM[J]. ACM Transactions on Intelligent Systems and Technology, 2012, 3(3).

• movielens

FM vs

review CF•

• FM

CF FM• (userID, itemID) label score

• f(userID, itemID) = ID onehot ++ ID one hot

123 45 3

[0,0,…,1,0,…,0] [0,0,…,1,0,…,0]

[0,0,…,1,0,…,1,0,…0]

3

3

CF FM

FM

• FM

• poi

FM vs

FM vs •

•

• FM

•

x1=1

x2=1

100

40 60

40

x1=1 x1=0

x2=1x2=0

x 2 = 1

! FMFM

FM vs

• Hintonbottleneck dark knowledge

•

dark knowledge

embedding•

•

• FM embedding

• word2vecFM

• FM embedding

deep & wide •

•

Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.

deep & wide

Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.

FFM• (Field)

• FM FFM

•

x_i w_i

x_iw_i1

w_i2

w_if

x_jw_j(w_i · w_j)

x_j

w_j1

w_j2

w_jf

w_i,fj w_j,fi

(w_i,fj · w_j,fi)

FFM•

M. Jahrer, A. To ̈scher, J.-Y. Lee, J. Deng, H. Zhang, and J. Spoelstra, “Ensemble of collaborative filtering and feature engineered model for click through rate prediction,” in KDD Cup 2012 Workshop, ACM, 2012.

FFM• O(k·n0^2), n0 0

• k_ffm << k_fm, n0 k_ffm 10

•

• libFFM ffm Spark DMSPA

Juan Y, Zhuang Y, Chin W, et al. Field-aware Factorization Machines for CTR Prediction[C]. conference on recommender systems, 2016: 43-50.

KDD CUP 2012• KDD Cup 2012 CTR

• 234M

• AUC

M. Jahrer, A. To ̈scher, J.-Y. Lee, J. Deng, H. Zhang, and J. Spoelstra, “Ensemble of collaborative filtering and feature engineered model for click through rate prediction,” in KDD Cup 2012 Workshop, ACM, 2012.

• CTR Criteo Avazu Kaggle

Juan Y, Zhuang Y, Chin W, et al. Field-aware Factorization Machines for CTR Prediction[C]. conference on recommender systems, 2016: 43-50.

FM• d-way FM d

• High order FM d 2,3...,d-1

• Rendle S. Factorization Machines[C]. international conference on data mining, 2010. • Blondel M, Fujino A, Ueda N, et al. Higher-Order Factorization Machines[C]. neural information processing systems, 2016: 3351-3359.

FM• FM

• Blondel M, Fujino A, Ueda N, et al. Higher-Order Factorization Machines[C]. neural information processing systems, 2016: 3351-3359.

ANOVA kernel

ANOVA kernel 递推关系

©2016-2017 XX