Improving aggregate recommendation diversity

Embed Size (px)


Dear StudentsIngenious techno Solution offers an expertise guidance on you Final Year IEEE & Non- IEEE Projects on the following domainJAVA.NETEMBEDDED SYSTEMSROBOTICSMECHANICALMATLAB etcFor further details contact us: 044-42046028 or 8428302179.Ingenious Techno Solution#241/85, 4th floorRangarajapuram main road,Kodambakkam (Power House)

Text of Improving aggregate recommendation diversity

  • 1.This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, MANUSCRIPT ID 1Improving Aggregate Recommendation Diversity Using Ranking-Based Techniques Gediminas Adomavicius, Member, IEEE, and YoungOk Kwon Abstract Recommender systems are becoming increasingly important to individual users and businesses for providing personalized recommendations. However, while the majority of algorithms proposed in recommender systems literature have focused on improving recommendation accuracy (as exemplified by the recent Netflix Prize competition), other important aspects of recommendation quality, such as the diversity of recommendations, have often been overlooked. In this paper, we introduce and explore a number of item ranking techniques that can generate recommendations that have substantially higher aggregate diversity across all users while maintaining comparable levels of recommendation accuracy. Comprehensive empirical evaluation consistently shows the diversity gains of the proposed techniques using several real-world rating datasets and different rating prediction algorithms. Index Terms Recommender systems, recommendation diversity, ranking functions, performance evaluation metrics, collaborative filtering. 1 Introduction In the current age of information overload, it is becomingincreasingly harder to find relevant content. This problem is recommended items, while maintaining an acceptable level of accuracy [8], [33], [46], [54], [57]. These studies measure rec-t.c omnot only widespread but also alarming [28]. Over the last 10-ommendation diversity from an individual users perspective ompo t.c 15 years, recommender systems technologies have been intro- (i.e., individual diversity).gs po duced to help people deal with these vast amounts of informa- In contrast to individual diversity, which has been exploredlo s.b og tion [1], [7], [9], [30], [36], [39], and they have been widely usedin a number of papers, some recent studies [10], [14] startedts .bl in research as well as e-commerce applications, such as the examining the impact of recommender systems on sales diver-ec tsoj c ones used by Amazon and Netflix.sity by considering aggregate diversity of recommendationspr oje The most common formulation of the recommendation across all users. Note that high individual diversity of recom-re rlo rep problem relies on the notion of ratings, i.e., recommender sys- mendations does not necessarily imply high aggregate diversi-xp lo tems estimate ratings of items (or products) that are yet to be ty. For example, if the system recommends to all users theee xp .ie ee consumed by users, based on the ratings of items already con- same five best-selling items that are not similar to each other,w e sumed. Recommender systems typically try to predict the rat-the recommendation list for each user is diverse (i.e., high in- w .iw w ings of unknown items for each user, often using other users dividual diversity), but only five distinct items are recom- :// wtp //w ratings, and recommend top N items with the highest pre-mended to all users and purchased by them (i.e., resulting inht ttp: dicted ratings. Accordingly, there have been many studies onlow aggregate diversity or high sales concentration). h developing new algorithms that can improve the predictive While the benefits of recommender systems that provide accuracy of recommendations. However, the quality of rec- higher aggregate diversity would be apparent to many users ommendations can be evaluated along a number of dimen-(because such systems focus on providing wider range of items sions, and relying on the accuracy of recommendations alone in their recommendations and not mostly bestsellers, which may not be enough to find the most relevant items for eachusers are often capable of discovering by themselves), such user [24], [32]. In particular, the importance of diverse recom-systems could be beneficial for some business models as well mendations has been previously emphasized in several studies[10], [11], [14], [20]. For example, it would be profitable to Net- [8], [10], [14], [33], [46], [54], [57]. These studies argue that one flix if the recommender systems can encourage users to rent of the goals of recommender systems is to provide a user with long-tail type of movies (i.e., more obscure items that are highly idiosyncratic or personalized items, and more diverselocated in the tail of the sales distribution [2]) because they are recommendations result in more opportunities for users to get less costly to license and acquire from distributors than new- recommended such items. With this motivation, some studiesrelease or highly-popular movies of big studios [20]. However, proposed new recommendation methods that can increase the the impact of recommender systems on aggregate diversity in diversity of recommendation sets for a given individual user, real-world e-commerce applications has not been well- often measured by an average dissimilarity between all pairs of understood. For example, one study [10], using data from on- line clothing retailer, confirms the long tail phenomenon that refers to the increase in the tail of the sales distribution (i.e., the G. Adomavicius is with the Department of Information and Decision Sciences, Carlson School of Management, University of Minnesota, Minneapolis, MN increase in aggregate diversity) attributable to the usage of the 55455. E-mail: recommender system. On the other hand, another study [14] Y. Kwon is with the Department of Information and Decision Sciences, Carl-shows a contradictory finding that recommender systems ac- son School of Management, University of Minnesota, Minneapolis, MNtually can reduce the aggregate diversity in sales. This can be 55455. E-mail: explained by the fact that the idiosyncratic items often haveManuscript received (insert date of submission if desired). Please note that all ac- limited historical data and, thus, are more difficult to recom-knowledgments should be placed at the end of the paper, before the bibliography. xxxx-xxxx/0x/$xx.00 2009 IEEEDigital Object Indentifier 10.1109/TKDE.2011.15 1041-4347/11/$26.00 2011 IEEE

2. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.2 IEEE TRANSACTIONS ON KNOWLEDGE & DATA ENGINEERING, MANUSCRIPT IDmend to users; in contrast, popular items typically have moremetric (i.e., the percentage of truly high ratings among thoseratings and, therefore, can be recommended to more users. Forthat were predicted to be high by the recommender system)example, in the context of Netflix Prize competition [6], [22],is 82%, but only 49 popular items out of approximately 2000there is some evidence that, since recommender systems seekavailable distinct items are recommended across all users. Theto find the common items (among thousands of possible mov- system can improve the diversity of recommendations from 49ies) that two users have watched, these systems inherently tendup to 695 (a 14-fold increase) by recommending the long-tailto avoid extremes and recommend very relevant but safe rec-item to each user (i.e., the least popular item among highly-ommendations to users [50].predicted items for each user) instead of the popular item.As seen from this recent debate, there is a growing aware- However, high diversity in this case is obtained at the signifi-ness of the importance of aggregate diversity in recommender cant expense of accuracy, i.e., drop from 82% to Furthermore, while, as mentioned earlier, there has The above example shows that it is possible to obtain higherbeen significant amount of work done on improving individual diversity simply by recommending less popular items; howev-diversity, the issue of aggregate diversity in recommender sys-er, the loss of recommendation accuracy in this case can betems has been largely untouched. Therefore, in this paper, wesubstantial. In this paper, we explore new recommendationfocus on developing algorithmic techniques for improving ag- approaches that can increase the diversity of recommendationsgregate diversity of recommendations (which we will simply with only a minimal (negligible) accuracy loss using differentrefer to as diversity throughout the paper, unless explicitly spe- recommendation ranking techniques. In particular, traditionalcified otherwise), which can be intuitively measured by therecommender systems typically rank the relevant items in anumber of distinct items recommended across all users. descending order of their predicted ratings for each user andHigher diversity (both individual and aggregate), however, then recommend top N items, resulting in high accuracy. Incan come at the expense of accuracy. As known well, there is a contrast, the proposed approaches consider additional factors,tradeoff between accuracy and diversity because high accuracysuch as item popularity, when ranking the recommendationmay often be obtained by safely recommending to users thelist to substantially increase recommendation diversity whilet.c om ommost popular items, which can clearly lead to the reduction in maintaining comparable levels of accuracy. This paper pro-po t.cdiversity, i.e., less personalized recommendations [8], [33], [46].vides a comprehensive empirical evaluation of the proposedgs poAnd conversely, higher diversity can be achieved by trying toapproaches, where they are tested with various datasets in alo s.b oguncover and recommend highly idiosyncratic or personalized variety of different settings. For example, the best results showts .blitems for each user, which often have less data and are inhe-up to 20-25% diversity gain with only 0.1% accuracy loss, up toec tsoj crently more