Context-Aware Service

Recommendation

Yueshen Xu(徐悦甡)

Xidian University; Zhejiang University

xyshzjucs@zju.edu.cn

ysxu@xidian.edu.cn

[Recommender System and Service Computing]

Knowledge

Engineering

&

E-Commerce

Background➢Why do those Web services need to be recommended?

➢What’s the challenge?

Context-aware Recommendation➢What kind of context information can we use?

➢Basic method

Context-aware feature learning➢User context-aware features learning

➢Service service-aware features learning

Experiment ➢ Experimental Result

➢ Performance Comparison

Reference

Outline

2017/6/15 2XDU & ZJU

Background

Web Service

➢ A Web service is a self-describing programmable

application used to achieve interoperability and

accessibility over a network ~ ‘Services Computing’, Zhang, L. etc.

2017/6/15 3

Other similar component

➢ Open API:

− Douban, Sina Weibo, RenRen, 51.com

− Amazon, Facebook, MySpace, Twitter, eBay, Google Maps

WSDL(XML) SOAPInterface

WS=WSDL+Interface+SOAP Popularity

➢ Amazon Relational Database Service

➢ Amazon Simple Storage Service etc.

They are almost

the same

Generalrestricted

XDU & ZJU

Background

Backgroud

➢ Cloud Computing the number of Web services is

exploding

2017/6/15 4

Amazon Relational

Database Service

Amazon Simple

Storage Service

Problem➢ Everyone wants to invoke the one whose qos is the best.

➢ Is there a service suitable for everyone? No. Why? Context

➢ So the real problem is : which one should I invoke?

Personalization & Prediction

Decision Basis

➢QoS(response time, throughput, availability etc.)

XDU & ZJU

Background

Formalization of the Problem

➢ Personalized QoS prediction

➢ User-Service Invocation Matrix

2017/6/15 5

service1 service2 service3 service4

user1 q11

user2 q22

user3

user4 q41 q44

user5 q53

➢ Large Scale Feasibility

Challenge

➢ Data Sparsity Cold Start Problem

million

million

Sufficient

Features

+Effective

Model

Cold-Start

Problem

Similar with rating prediction

in e-commerce systems

XDU & ZJU

Context-aware recommendation

Contextual Features Selection

➢ Basis: factors dominating QoS: physical configuration

2017/6/15 6

Service

User

Service

Provider

Network

Connection etc.

CPU, Memory,

Bandwidth etc.

Bandwidth etc.

Feature Quantification

➢ User-User: Geographical Distance

➢ Service-Service: Service ProviderWhy?

XDU & ZJU

Context-aware recommendation

Feature Quantification (Con.)

➢Observation

– For the exactly same service or user:

2017/6/15 7

Assumption:

➢Users located nearly with each other have similar IT

infrastructure

➢Services provided by the same company have similar physical

configuration

✓ Users in different locations usually experience different QoS

✓ Users in the same location usually experience similar QoS

city, town, community

Just the same with

browsing in

Internet

✓ Services operated by different providers usually offer different QoS

✓ Services operated by the same providers usually offer similar QoS

Reasonable?

XDU & ZJU

Context-aware recommendation

Assumption: Reasonable?

2017/6/15 8

Comapratively

Reasonable

Web service invocation scenario

Algorithm: the basic model

➢ Collaborative Filtering/Matrix Factorization traditional

recommender systemXDU & ZJU

Related Work Collaborative Filtering

➢ Explore similar users and services through their historical

invocation records much like recommender systems

Context-aware recommendation

2017/6/15 9

Similar Service

Invocation

Experience

Whose invocation is similar with mine?

➢ Pearson Correlation Coefficient Similarity Computation

Ii uuiIi aai

Ii uuiaai

qqqq

qqqquaSim

22 )()(

))((),(

Uu jujUu iui

Uu jujiui

qqqq

qqqqjiSim

22 )()(

))((),(

Predicted Results

➢ Self Average + Weighted Average

)(

)(

),(

))(,(

aSu a

uSu aua

ui

a

a a

uuSim

uruuSimup

)(

)(

),(

))(,(

iSi k

iSi kik

ui

k

k k

iiSim

iriiSimip

Memory-based

&

Heuristic

XDU & ZJU

Context-aware recommendation

Probabilistic Matrix Factorization(PMF)

2017/6/15 10

m

i

n

j

I

Qj

T

iijS

Qij

SUqNSUQp1 1

22 ,|),,|(

Generative ModelStatistical Machine

Learning

m

i

UiU IUNUp1

22 ),0|()|(

n

j

SjS ISNSp1

22 ),0|()|(

)()(),|()|,( SpUpSUQpQSUp

n

j

Sj

m

i

Ui

m

i

n

j

I

Qj

T

iij

ISN

IUN

SUqNQSUpQij

1

2

1

2

1 1

2

),0|(

),0|(

,|)|,(

Gaussian

Distribution

0/1

Zero-Mean Spherical

Gaussian Prior

Bayesian

Inference

Maximize Posterior

Probability(MAP)

CNDMDI

SSUU

SUqI

QSUp

SUQ

m

i

n

j

ij

n

j

j

T

j

S

m

i

i

T

i

U

m

i

n

j

j

T

iijij

Q

SUQ

)lnlnln)((2

1

2

1

2

1

)(2

1

),,,|,(ln

222

1 1

12

12

1 1

2

2

222

Logarithm

Transformation

Maximization

Minimization

XDU & ZJU

Context-aware recommendation

Probabilistic Matrix Factorization

➢MF can factorize the high-rank user-service invocation

feature space into the joint low-rank feature space

➢Q={qij}: m×n user-service invocation matrix

➢ U∈Rdm, S∈Rdn: user and service feature matrices

2017/6/15 11

j

T

iij SUq

m

i

n

j

j

T

iijijSU

SUqI1 1

2

,)(

2

1min

22

1 1

2

22)(

2

1min

F

S

F

Um

i

n

j

j

T

iijij SUSUqIL

Inner product of two d-rank feature vectors

Minimization function

Objective Optimization Function

Indicator function(0/1)

How can the context

information be

merged into the

basic model?

Regularization Term

XDU & ZJU

Context-aware recommendation

Matrix Decomposition

➢ Tri-angle

➢ LU

➢QR

➢ Spectral

➢ SVD

2017/6/15 12

Matrix Factorization

➢Basic MF

➢Non-negative

➢PMF

➢BPMF

➢pLSA, LDA

Matrix

Theory

Machine

Learning

XDU & ZJU

XDU & ZJU

Context-aware features learning

----User Side

User Neighborhood Identification

➢ Latitude and Longtitude Distance K

Nearest Neighbors (KNN)

2017/6/15 13

Euclidean Distance Top-K Selection

Similarity Computation

➢ Distinguish each neighbor’s significance

➢ The similarity of user i and user j can be defined as

)exp( ijij dSim

✓Simij = 1: i and j live in the exactly same place

✓Simij 0: i and j live extremely far

KNNg

ig

ilil

Sim

Simw

K

l

j

T

liljKNN SUwq1

(Ul∈KNN of user i)

Multiple choice Not exclusive

Context-aware features learning

----User Side

Reasonable Combination

2017/6/15 Middleware, CCNT, ZJU 14

k

l

j

T

lilj

T

iijij SUwSUqq1

)1(ˆ Learned from his/her neighbors’

invocation experience

Regulating Factor

Final Objective Function

22

1 1

2

1 22)))1(((

2

1F

S

F

Um

i

n

j

k

l

j

T

lilj

T

iijij SUSUwSUqIL

jS

k

l

lili

m

i

k

l

ijj

T

lilj

T

iij

j

iU

n

j

k

l

ijj

T

lilj

T

ijij

i

SUwUqSUwSUIS

E

UqSUwSUSIU

E

))1(())1((

))1((

11 1

1 1

(Stochastic) Gradient Descent Local Minimum

Global Minimum Closed-form solution NP Hard

Context-aware features learning

----Service Side

Service Neighborhood Identification

➢Operated by the same service provider

2017/6/15 15

Reasonable Combination

)(

1)1(ˆ

jCc

cTij

Tiijij SU

jCSUqq Learned from its neighbors’

invocated experience

Regulating Factor

Sc and Sj are operated by the

same company

Final Objective Function

22

1 1

2

)( 22))

1)1(((

2

1F

S

F

Um

i

n

j jCc

c

T

ij

T

iijij SUSUjC

SUqIL

(Stochastic) Gradient Descent Local Minimumthe same with the computation process of user-side

XDU & ZJU

Context-aware features learning

----Combination

Model Unification ==

➢Model Fusion: too parameters, too complicated

➢ Result Aggregation: simple and effective

2017/6/15 16

Result Aggregation

➢ Strategy 1: Fixed Proportion

sideservicesideuserijij ppqq )1(ˆ

too simple, too naive

Ensemble

Learning

λ: a fixed decimal

➢ Strategy 2: Dynamic Proportion

sideservicejsideuseriijij pwpwqq ˆ

ji

j

j

ji

ii

concon

conw

concon

conw

)1(

)1(,

)1(

neighborsuser

neighborsuser

igiSim

giSimcon

),(

),( 2

neighborsservice

neighborsservice

jgjSim

gjSimcon

),(

),( 2

Zibin Zheng etc.

XDU & ZJU

Context-aware features learning

----Combination

A Unified Framework

➢Offline and Online

➢ Preparation for System Building

➢ It needs further improvement

2017/6/15 17

update when the info. In

databases has been updated

update when the offline

step has been updated

Building the system has

been on the schedule

XDU & ZJU

Experiment

Preparation

➢ Dataset: real world dataset

➢Matrix Density: 5%~20%

➢ Evaluation Metrics: RMSE and MAE

➢ Result: average of multiple testing

2017/6/15 18

Memory-

based

Basic ML-

based

User-side &

Service-side

Unification

Large Improvement

v.s.

Large computation

Offline

Online(linear)

Our methods

XDU & ZJU

➢ The parameter regulates the

weight of user-side model model

and service-side model

➢ User’s context is more important

Why?

Experiment

Impact of

2017/6/15 19

Impact of

➢ The parameter controls the

individual contributions of the

user/service and their neigh-

bors to the predicted value

➢ Neighbors’ Contributions are

dominating

XDU & ZJU

Experiment

Impact of TopK

➢ The parameters TopK(S) and TopK(U) determine the

number of neighbors involved in the learning process.

➢ Too few or too much neighbors are both not so proper

2017/6/15 20

Smoothness

XDU & ZJU

Furture Work

Explore more efficient strategies for model

aggregation

Trying to find conduct the unification method of

model fusion (model fusion)

Build a practical service recommender system

Refining the similarity function between users

Collecting a real world dataset by ourselves

Others

2017/6/15 25XDU & ZJU

Reference

[1] L.-J. Zhang, J. Zhang, H. Cai, "Services Computing," In Springer and Tsinghua University Press ,

2007

[2] M.P. Papazoglou, "Service-Oriented Computing: Concepts, Characteristics and Directions," In

Proc. of International Conference on Web Information System Engineering (WISE) , pp. 3-12,

2003

[3] Amazon Relational Database Service, "Using the SOAP API," In

http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/using-soap-api.html

[4] Amazon Simple Storage Service, "SOAP API," In

http://docs.aws.amazon.com/AmazonS3/latest/API/APISoap.html

[5] Amazon Elastic Compute Cloud, "Pricing," In https://aws.amazon.com/ec2/

[6] Windows Azure, "Pricing," In http://www.windowsazure.com/en-us/pricing/calculator/

[7] Akamai, "The State of the Internet," In http://www.akamai.com/stateoftheinternet/

[8] R.M. Sreenath, M.P. Singh, "Agent-based service selection," In Journal of Web Semantics , pp.

261-279, 2004

[9] S. Ran, "A model for web services discovery with QoS," ACM SIGecom Exchanges , vol. 4, no. 1,

pp. 1-10 (2003)

[10] Netflix. Netflix Prize Competition. http://www.netflixprize.com/ (2006)

[11] P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, "GroupLens: An Open

Architecture for Collaborative Filtering of Netnews," In Proc. of ACM Conference on Computer

Supported Cooperative Work (CSCW) , pp. 175-186, 19942017/6/15 26XDU & ZJU

Reference

[12] B.M. Sarwar, G. Karypis, J.A. Konstan, and J. Riedl, "Item-based collaborative filtering

recommendation algorithms," In Proc. of International World Wide Web Conference (WWW) , pp.

285-295, 2001

[13] M.R. McLaughlin, J.L. Herlocker, "A Collaborative Filtering Algorithm and Evaluation Metric that

Accurately Model the User Experience," In Proc. of ACM SIGIR (SIGIR) , pp. 329-336, 2004

[14] L. Shao, J. Zhang, Y. Wei, J. Zhao, B. Xie, and H. Mei, "Personalized QoS Prediction for Web

Services via Collaborative Filtering," In Proc. of International Conference on Web Services

(ICWS) , pp. 439-446, 2007

[15] X. Chen, X. Liu, Z. Huang, and H. Sun, "RegionKNN: A Scalable Hybrid Collaborative Filtering

Algorithm for Personalized Web Service Recommendation," In Proc. of International Conference

on Web Services (ICWS) , pp. 9-16 (2010)

[16] Z. Zheng, H. Ma, M.R. Lyu, and I. King, "WSRec: A Collaborative Filtering Based Web Service

Recommender System," In Proc. of International Conference on Web Services (ICWS) , pp. 437-

444, 2009

[17] Y. Koren, R.M. Bell, and C. Volinsky, "Matrix Factorization Techniques for Recommender

Systems," In IEEE Computer , 30-37, 2009

[18] H. Ma, D. Zhou, C. Liu, M.R. Lyu, and I. King, "Recommender systems with social regularization,"

In Proc. of ACM Conference on Web Search and Data Mining (WSDM) , pp. 287-296, 2011

[19] Z. Zheng, H. Ma, M.R. Lyu, I. King, "Collaborative Web Service QoS Prediction via Neighborhood

Integrated Matrix Factorization," In IEEE Transactions on Services Computing , volume: PrePrint,

issue: 99, 20112017/6/15 27XDU & ZJU

Reference

[20] W. Lo, J. Yin, S. Deng, Y. Li, and Z. Wu, "An Extended Matrix Factorization Approach for QoS

Prediction in Service Selection," In Proc. of International Conference on Services Computing

(SCC) , pp. 162-169, 2012

[21] Z. Zheng, Y. Zhang, M.R. Lyu, "Distributed QoS Evaluation for Real-World Web Services," In

Proc. of International Conference on Web Services (ICWS) , pp. 83-90, 2010

[22] Salakhutdinov, R., Mnih, A.: Probabilistic Matrix Factorization. In: Advances in Neural Information

Processing Systems, volume 20, (2008)

[23]S.Purushotham, Y.Liu, and C.J.Kuo, Collaborative Topic Regression with Social Matrix

Factorization for Recommendation Systems. In Proceedings of CoRR. 2012.

[24]J.L.Herlocker,J.A.Konstan,L.G.Terveen,and J.T.Riedl,"Evaluating collaborative filtering

recommender systems,“ ACM Transactions on Information Systems,vol.22,no.1,pp.5-53,2004.

[25]R.Sinha and K.Swearingen,"Comparing recommendations made by online systems and friends," in

Proceedings of the DELOS-NSF Workshop on Personalization and Recommender Systems in

Digital Libraries, 2001.

[26]S.M.McNee,J.Riedl,and J.A.Konstan,"Accurate is not always good：how accuracy metrics have

hurt recommender systems," in Proceedings of the ACM Conference on Human Factors in

Computing Systems(CHI'06),2006}

[27]R.B.Statnikov and J.B.Matusov,Multicriteria Optimization and Engineering. Chapman \& Hall,1995

[28]Steffen Rendle,Zeno Gantner,Christoph Freudenthaler,Lars Schmidt-Thieme.Fast context-aware

recommendations with factorization machines. SIGIR 2011: 635-644

2017/6/15 28XDU & ZJU

Thank You !

Q&A

Context-Aware Web Service

Recommendation