Web-Based Usage Surveys MINES for Libraries TM Brinley Franklin University of Connecticut [email protected] Terry Plum Simmons GSLIS [email protected]

Web-Based Usage SurveysMINES for LibrariesTM

Brinley FranklinUniversity of Connecticut

[email protected]

Terry PlumSimmons GSLIS

[email protected]

CNI Fall 2005 Task Force Meeting Phoenix, AZ

December 5, 2005

2CNI Fall 2005 Task Force Meeting

Overview

• Research problem– Assessment of networked electronic services– Open access – beyond vendor supplied data– Issues with web-based surveys in assessing the use of networked

electronic services – Digital libraries – how to assess

• Project– MINES for LibrariesTM

• Web based usage and user survey– Assessment infrastructure and gateway

• Examples – data from:– Medical libraries– Large academic libraries– Consortia


Assessment of networked services• Vendor supplied data - standards

– COUNTER - Counting Online Usage of Networked Electronic Resources

• http://www.projectcounter.org/

– ICOLC – International Coalition of Library Consortia • http://www.library.yale.edu/consortia/

– ISO – International Standards Organization• ISO 11620 Library Performance Indicators • http://www.iso.org/

– NISO – National Information Standards Organization• NISO Z39.7 Library Statistics• http://www.niso.org/

– ARL E-Metrics• http://www.arl.org/stats/newmeas/emetrics/index.html

– are helpful and trusted• LibQUAL+TM

– Suite of services that libraries use to solicit, track, understand, and act upon users’ opinions of service quality.

• http://www.libqual.org/index.cfm

• DigiQUALTM

– Digital library service quality – set of 180 items tested against five collections


Open access --Beyond vendor supplied data

• ISO standards for the electronic collection include e-books, electronic databases, ejournals, and digital documents. – Free internet resources are counted separately. – Yet, library liaison and subject web pages include free resources.

• Other electronic resources are important, including:– Digital libraries – local, archival resources– Pre-print and post-print servers– Open access journals– Open access repositories (e.g., institutional repositories)– Electronic theses and dissertations servers

• Can we assess use of these resources as well as vendor supplied databases and ejournals?


Web-based usage surveys• Advantages

– Can go beyond gross usage counts, using sampling– User and use information

• E.g.; can query purpose of use, the discipline or affiliation, the status of the patron– Opinion

• Service quality perception– Can be tied to usage

• Point of use– Can fold in other services (e.g., ILL, Ask A Librarian)

• Methodological issues– Sample, not census– Replicable?– Truly random sample is difficult to obtain– High non-response rate – are the responses representative of the population?

• Survey not seen• Survey seen, but not filled out

– Users, not usage– Remembered, predicted or critical incident reports – perceived to be not as

reliable as vendor census based on usage– Browsers can influence the display of the web survey

• And more. See for example:• Holly Gunn, “Web-based Surveys: Changing the Survey Process”, First Monday, Issue

7, 2002. • D.A. Dillman, Mail and Internet Surveys, The Tailored Design Method. 2nd ed., 2000.


MINES for LibrariesTM

• MINES is a transaction-based research methodology consisting of a web-based survey form and a random moments sampling plan.

• MINES typically measures who is using electronic resources, where users are located at the time of use, and their purpose of use.

• MINES measures usage as well as users.

• MINES is administered at the point of use, and surveys repeat usage during the sample period.

• MINES was adopted by the Association of Research Libraries (ARL) as part of the “New Measures” toolkit in May, 2003.

• MINES is different from other electronic resource usage measures that quantify total usage (e.g., COUNTER, E-Metrics) or measure how well a library makes electronic resources accessible (LibQUAL+TM).




Assessment Infrastructure

On Campus Patrons

In Library Patrons

Off campus patrons

?


Assessment Gateway

online catalogfederated searching


Demographics by Location of UserU.S. Main Libraries

24%

25%

8%

43%

Graduate Students

Faculty, Staff,Research Fellows

UndergraduateStudents

All Other Users

40%

31%

25%

4%

28%

11%

21%

40%

On Campus, Not in the Libraryn = 8,838

Inside the Libraryn = 9,172

Off-Campusn = 7,688

MINES for Libraries™


31%

46%

6%

17%

Graduate Students



All Other Users

Off-Campusn = 5,133

Demographics by Location of UserU.S. Medical Libraries



20%

53%

1%

26%

34%

32%

5%

29%

MINES for Libraries™



14%

14%

69%

3%

Graduate Students



All Other Users

Off-Campusn = 9,163

Demographics by Location of UserOntario Council of University Libraries



45%

24%

29%

2%31%

16%

48%

5%


25%

27%9%

39%

SponsoredResearch

Instruction

Patient Care

Other

On-Campus, Not in the Libraryn = 15,423

In the Libraryn = 6,312

Purpose of Use By LocationMedical Libraries

2003 – 2005

Off-Campusn = 4,375

*83% of sponsored research usage occurred outside the library. 92% of this use took place on-campus.

Overall Usen = 26,110

31%

14%10%

45%

34%37%

13%16%

14%

49%

18%19%


Purpose of Use By Location Main Campus Libraries

2003 – 2005

5%

66%

3%

26%

Sponsored Research

Instruction

Other

Other SponsoredActivities

In the Libraryn = 7,618

On-Campus, not in the Libraryn = 6,641

11%63%

24%

2%

All Usagen = 19,271

•72% of sponsored research usage of electronic resources occurred outside the library; 83% of this took place on campus.

6%

59%

33%

2%

Off-Campusn = 5,012

21%63%

14%2%



64%

36%

Inside the Library

Outside the Library

U.S. Main LibrariesTotal Users

n = 25,698

21%

79%

U.S. Medical Libraries

Total Usersn = 31,883

80%

20%

OCUL (Canada)

LibrariesTotal Users

n = 20,300

Location of Electronic Resources Users



OCUL Data from MINES


Web-Based Usage SurveysMINES for LibrariesTM

[email protected]@simmons.edu

http://www.arl.org/stats/newmeas/mines.html

Documents

Web-Based Usage Surveys MINES for Libraries TM Brinley Franklin University of Connecticut [email protected] Terry Plum Simmons GSLIS [email protected]