Agent Technology for e-Commerce

Chapter 6: Recommender Systems

Maria Fasli

http://cswww.essex.ac.uk/staff/mfasli/ATe-Commerce.htm

2Chapter 6

Agent Technology for e-Commerce

Recommender systems: The problem

Too much information: information overload – consumers have too many options

A recommender system is a system which provides recommendations to a user

Applications: Books, music CDs, movies. Even documents, services and other products such as software games

3Chapter 6

Agent Technology for e-Commerce

Recommender

Provider n

Provider 1

RecommenderRequest for service

Advertisementof capabilities

Sorted description of P1,..Pn

Service delegation

Results of service request

Requester

4Chapter 6

Agent Technology for e-Commerce

Information needed

Information used for recommendations can come from different sources:

browsing and searching data purchase data feedback explicitly provided by the users textual comments expert recommendations demographic data

5Chapter 6

Agent Technology for e-Commerce

Providing recommendations

Recommendations can take the following forms: Attribute-based recommendations: based on syntactic attributes

of products (e.g. science fiction books) Item-to-item correlation (as in shopping basket

recommendations) User-to-user correlation (finding users with similar tastes) Non-personalized recommendations (as in traditional stores, i.e.

dish of the day, generic book recommendations etc.)

6Chapter 6

Agent Technology for e-Commerce

Recommendation technologies

Information retrieval (IR) systems: allow users to express queries to retrieve information relevant to

a topic of interest or fulfil an information need they are not useful in the actual recommendation process they cannot capture any information about the users’ preferences they cannot retrieve documents based on opinions or quality as

they are text-based

To address these issues two techniques have been developed: Content-based filtering (Information filtering) Collaborative-based filtering

7Chapter 6

Agent Technology for e-Commerce

Content-based filtering

The system processes information from various sources and tries to extract useful elements about its content

keyword-based search (keywords sometimes in boolean form) semantic-information extraction by using associative networks of

keywords, or directed graphs of words

8Chapter 6

Agent Technology for e-Commerce

Each user is assumed to act independently and the system requires a profile of the user’s needs or preferences

The user has to provide information on her personal interests on starting to use the system for the profile to be built

The profile includes information about the items of interest, i.e. movies, books, CDs etc.

Content-based filtering techniques try to identify similar items which are returned as recommendations

They do not depend on having other users in the system

9Chapter 6

Agent Technology for e-Commerce

Issues

Pure content-based filtering systems are not capable of exploring new items and topics

Over-specialization: one is restricted in viewing similar items Difficult to apply in situations where the desirability of an item is

determined in part by aesthetic qualities that are difficult to quantity – it is difficult to apply content-based analysis to such items

10Chapter 6

Agent Technology for e-Commerce

The user profiles For the system to produce accurate recommendations, the user

has to provide constant feedback on the returned suggestions – users do not like providing feedback

Consist entirely of ratings of items and topics of interest: the fewer the ratings, the more limited the set of possible recommendations

As the user’s interests change, these changes need to be tracked

11Chapter 6

Agent Technology for e-Commerce

Collaborative filtering

Collaborative-based filtering systems can produce recommendations by computing the similarity between a user’s preferences and the preferences of other people

Such systems do not attempt to analyse or understand the content of the items being recommended

They are able to suggest new items to user who have similar preferences with others

12Chapter 6

Agent Technology for e-Commerce

Basic mechanism

A large group of people’s preferences are registered A subgroup of people is located whose preferences are similar of

the user who seeks the recommendation An average of the preferences for that group is calculated The resulting preference function is used to recommend options

to the user who seeks the recommendation The concept of similarity needs to be defined in some way

13Chapter 6

Agent Technology for e-Commerce

Example user-item matrix

What would be the recommendation for user D?

14Chapter 6

Agent Technology for e-Commerce

Neighbourhood-based algorithms

Three steps

(i) The degree of similarity of the active user and the others in the database is calculated (positive or negative)

(ii) A set of users is chosen as the basis for making the prediction. This is determined based on the degree of similarity and differs from system to system

(iii) The set of users chosen in the previous step is used to make the recommendation. A user with high degree of similarity may be assigned higher weight

15Chapter 6

Agent Technology for e-Commerce

Pearson’s correlation coefficients

It reflects the degree of linear relationship between two variables and ranges –1 to +1

The degree of correlation between an active user a and another user u is:

16Chapter 6

Agent Technology for e-Commerce

Next, the neighbourhood of users based on which the recommendation will be provided is selected

The weighted average of the ratings of the neighbourhood of users for the item of interest is then calculated as follows:

17Chapter 6

Agent Technology for e-Commerce

Example

wD,A= 0.9 wD,B= - 0.7 wD,C= 0

pD,Item4= 4.5

Using Pearson’s correlation coefficients:

18Chapter 6

Agent Technology for e-Commerce

Issues

A critical mass of users is needed in order to create a database of preferences: first-rater or cold start problem

New items cannot be recommended until someone has rated them The scarcity of ratings (the user profiles are sparse vectors of

ratings) also presents a problem Recommendations will come from users with which the active

user shares ratings (or votes) – this presents a problem to methods such as Pearson’s correlation coefficients; potential solutions: default voting

19Chapter 6

Agent Technology for e-Commerce

Scalability: in systems with a large number of items and users, computation grows linearly; appropriate algorithms that scale up are needed

Reliability, especially in reputation systems: content providers inflate their ratings

Lack of transparency: the user is given no indication whether to trust a recommendation – incorporating explanation systems would help address this concern

Privacy – once a system has built your profile, who else can have access to it?

20Chapter 6

Agent Technology for e-Commerce

Combing collaborative and content-based filtering

The underlying idea is that the content is also taken into account when attempting to identify similar users for collaborative recommendations

A number of systems have been developed: Fab, Tango, the Recommender system, GroupLens’ approach

21Chapter 6

Agent Technology for e-Commerce

Recommender systems in e-commerce

Turning browsers into customers: they can stimulate the users’ needs (need identification stage)

Cross-selling: suggest additional products which may match the user’s interests or current shopping basket

Personalization: personalized services, or the site can be personalized to the user’s liking – unique shopping experience

Keeping customers informed Retaining customer loyalty

22Chapter 6

Agent Technology for e-Commerce

Personalization

Vendors can identify exactly who is visiting their store through registration, cookies, spyware

Vendors can personalize their websites for their customers They can keep track of preferences, actions, they can build

profiles of their users. These can be used for marketing Vendors can measure the users’ desires – dynamic pricing When the consumer is unaware, then problems arise, possible

breaches of the user’s privacy. Who else gains access to these profiles?

Negative impact on consumer confidence