Research recommendations at Mendeley

Research recommendations at Mendeley Marco Rossetti Data Scientist @ross85 24/11/2015 Elsevier, Amsterdam

2

Outline

Research recommendations at Mendeley

• What is Mendeley

• Recommender Systems at Mendeley – Why

– Data Sources

– Algorithms

– Business Logic & Analytics

– User Interface 24/11/2015

3

What is Mendeley

Research recommendations at Mendeley 24/11/2015

4

Mendeley builds tools tohelp researchers …


Read &

Organize

Search &

Discover

Collaborate &

Network

Experiment &

Synthesize

24/11/2015

5

Read & Organize


Reference management

Cite-as-you-

write

Full-text article search

Digitalised

annotations

24/11/2015

6

Search & Discover


Mendeley Suggest

Literature

Search

Related Documents

24/11/2015

7

Collaborate & Network


Research network

Groups

24/11/2015

8

Mendeley & Elsevier


9

Recommender Systems


10

Why Recommender Systemsat Mendeley?


Vision: “To build a personalised research advisor that helps you to organise your work, contextualise it within the global body of research, and connect you with relevant researchers and artifacts.”

24/11/2015

11

Recommender Systemsat Mendeley – Related Documents


12

Recommender Systemsat Mendeley – Mendeley Suggest

Research recommendations at Mendeley https://www.mendeley.com/suggest/

24/11/2015

13

Recommender SystemComponents


Algorithms

Business Logic and Analytics

User Experience

Data Sources Algorithms

Business Logic

& Analytics

User Interface

24/11/2015

14

• Mendeley – User Libraries

• What the users have in their libraries (what they read, what they annotate, what they highlight, what folders they have, etc. etc.)

– Articles metadata (title, authors, abstract, keywords, tags, etc. etc.) – Groups

• Scopus – Citation network

• Science Direct – Logs

• …

Data Sources


15

Algorithms


1.  Collaborative filtering User-based

If Alice read X, Y, Z and Bob read X, Y, Z and W, we recommend W to Alice + Efficient for us because users << items - Only for users with enough articles in the library

Item-based Users who read X also read Y + Item-item similarity matrix is useful to model last n articles read - Expensive in our setting (millions of items)

24/11/2015

16

Algorithms [2]


1.  Collaborative filtering (still) Matrix factorization

+ Best CF model in literature - A lot of latent factors, generate recommendations on a catalog of million of items is too slow

1 1 11 1 1? ? 1 ? 1 ?

1 1 11 1

1 1 1

U n x k

V k x m

X n x m

X ≈

24/11/2015

17

Algorithms [3]


2.  Content-based I read articles about text mining, show me other stuff about text mining + Good for semi-cold users (users with only a few articles) - Overspecialisation: items recommended are too similar

3.  Popularity/Trending I work in Computer Science, show me popular/trending articles in Computer Science + Perfect for cold users - Non personalised, discipline too broad

24/11/2015

18

Algorithms [4]


4.  Citation Network Articles similar to articles I cited

Articles that cite me

Articles from my co-author

+ Good for some kind of users

- Young researchers do not have (enough) publications

24/11/2015

19

Offline experiments


Offline Evaluation of 100+ algorithms variations on an historical dataset

• Split data into training and testing based on timestamps: train until day X, try to predict what users will add in the next day/week/month

• Computed different metrics to measure different dimensions: • Accuracy (precision, recall, f-score, nDCG, MAP) • Diversity • Recency • Popularity • Consistency • Coverage

24/11/2015

20

Offline results


Warm Users

Cold Users

24/11/2015

User Based CF Item Based CF Content Based

Citation Network Popularity Trending

Content Based Popularity Trending

21

Business Logic / Analytics


• Business put some constraints that could have an impact on the recommendation experience

– Don’t show articles outside the user discipline – Show articles only with a minimum readership – Show only recommendations that you can explain (especially for people recommendations, a different matter)

• Analytics – Dashboard on the recommender statistics:

• Number of recommendations served • Number of users with recommendations • …

24/11/2015

22

User Interface


• Original idea: One list fits all

Create a single list with the best recommendations for the user: use advanced methods to take into account every signal and provide what is best for you!

24/11/2015

23

User Interface [2]


• However… – Different kinds of users can have different information needs!

– The same user in different contexts can have different information needs!

VS

24/11/2015

24

User Interface [3]


• Solution: different lists! • Provide multiple lists that satisfy different information needs • More likely for a user to find something he is interested in

24/11/2015

25

Online Survey


Survey with Mendeley Advisors (pre-launch)

24/11/2015

Based on all the articles

Good

Bad

Popular

Good

Bad

Based on the last article

Good

Bad

Trending

Good

Bad

26

Online Statistics


Different statistics collected:

• overall and list

• click on title oradd to library • different metrics:

– # users – CR – CTR

24/11/2015

27

What’s next


• New lists! – Based on your research interests – …

• Improve current lists

• Researchers you may want to follow

24/11/2015

28 Research recommendations at Mendeley 24/11/2015

Thank you!

Data & Analytics

Research recommendations at Mendeley