Evaluation of the impact of library discovery technologies on usage of academic content

Valérie Spezi, LISU (Loughborough University, UK)

UKSG Webinar – 14th May 2014

Why this study?

• Commissioned by UKSG/Jisc in July 2013Lots of interest in library discovery technologiesQuestions about whether libraries, publishers and other

stakeholders should be engaging with those technologies

• Small-scale study A UK perspective No previous usage data analyses at the time - fills in the

gap Complements the 2 other studies that are currently taking

place

• Report available on UKSG website (Dec 2013)

Objectives of the research Evaluation of the impact of library discovery technologies

on usage of academic resources Provide evidence to determine if there is a case for

Investment in library discovery technologies by librariesEngagement with library discovery technologies by

publishers and other stakeholders in the information supply chain

Provide recommendations for stakeholders to best support the discovery of academic resources

Identify additional research, data, discussion and initiatives that will support the findings of the study

Methodology

Phase 1: survey of UK HE libraries Objective: determine the current RDS landscape

Online questionnaire to UK HE library directors – 62 respondents

Phase 2: case studies of libraries and publishers Objective: collect usage data + views and perceptions on the

impact of library discovery technologies 8 publishers and content providers; 6 case study libraries; Data received from

6 libraries & 4 content providers - COUNTER JR1, BR2 and DB1 or close equivalent (2 years pre and post-RDS implementation)

Phase 3: interviews with stakeholders Objective: obtain a bigger picture on the perceived impact of library

discovery technologies and an insight of where the sector is going

Findings

1. UK RDS landscape (survey findings)

2. Libraries – usage trends & experiences

3. Publishers and content aggregators – usage trends & perceptions

RDS landscape20

07-0

8

2008

-09

2009

-10

2010

-11

2011

-12

2012

-13

2013

-14

0

5

10

15

20

25

2 1

4

7

10

21

1

2

5

In process Implemented

No

of re

spon

dent

s

• 77% of UK HE libraries already using an RDS at their institution

• A further 11% are in the process of implementing an RDS

• RDS implementation in HE libraries had probably reached its peak in the last 12 months

• Summon, Primo and EDS – 76% of systems in use

2% 2%

24%

3%2%

26%

36%

2%2%

2%AquaBrowser

Blacklight

Ebsco Discovery Service

Encore

Endeca

Primo

Summon

VuFind

WorldCat Local

Other RDS

Journals – mixed picture, possibly some positive influence to varied extent

E-books – positive correlationDatabase results were inconclusive

LIBRARIES –usage data and experiences• Improved user experience through a single search interface

linked to full-text – high level of satisfaction• One stop shop experience for users• Better use of subscriptions – no silos

• Possibly a positive influence of RDS on content usage, most visibly for e-books

Multi-dimensional

environment*****

Difficult to isolate the sole effect of RDS

*****Multitude of other factors at

play?

LIBRARIES –usage data and experiences

Perceived challenges***********************

• Lack of clarity in coverage - RDS coverage of subscribed resources ‘believed’ to be 50% or more – gaps in some disciplines

• Lack of cooperation between some vendors is a concern – not helpful according to libraries

• Interoperability between library systems – ‘ecosystem’• No routine analysis of the RDS usage data (yet)• RDS searching aimed at undergrads?

• Starting point? • Can researchers benefit from RDS too?

PUBLISHERS –usage data and experiences

Key motivation - improving discoverability & visibility of content

• Publishers have no clear evidence of their usage is being affected by RDS

• difficult to isolate traffic mediated by RDS• Still low traffic compared to search engines

• Our usage study shows a very mixed picture for publishers• Some publishers may benefit more from RDS than bigger publishers

Perceived challenges:• Metadata –compatibility and optimisation for improved discoverability• Dilution of the publisher’s brand within the RDS• Lack of feedback/communication from RDS suppliers• Lack of clarity and understanding of how data are used

Relevancy ranking

Engagement - can publishers afford to wait

and see where this is going?

Conclusions

• There is a lot of data out there but it is imperfect

• Fantastic tool for library end-users but more work needs to be done to take full advantage of RDS technology

• Collaboration is key to success

What next?

Recommendations

Recommendations

Libraries

Library community working closely with bodies such as SCONUL, RLUK, UKSG, Jisc

Consider issue of interoperability between products from different vendors vs. moving into a particular vendor’s ecosystem

Engage in cross-sectorial talks to understand better how minor changes in the RDS settings can affect usage of certain resources

RDS suppliers

Working towards an open communication with libraries and content owners/providers

Consider user-testing for publishers and content providers Provide clearer information about what is indexed by the RDS Support the development of working relationships between competing

suppliers on the issue of disclosure and exchange of data for the benefit of end-users

… etc.

Recommendations

Publishers Engage and work closely with libraries and RDS suppliers to

optimise content discoverability Voice the need for more communication and feedback from RDS

suppliers

Other stakeholders

Monitor developments led by COUNTER 4, particularly in the area of database usage

Inclusion of usage data from RDS suppliers and link resolvers in initiatives such as JUSP or KB+

Development of a COUNTER code of practice for RDS usage data COUNTER, NISO, ODI to work together and establish industry

standards Support new research

THANK YOU!

Report available on the UKSG website: http://www.uksg.org/researchstudy

Get in touch with us!v.c.l.spezi@lboro.ac.uk or lisu@lboro.ac.uk