Upload
griffin-wade
View
220
Download
0
Embed Size (px)
Citation preview
http://www.minesforlibraries.orghttp://www.arl.org/stats
MINES for Libraries™
Presented by
Martha KyrillidouDirector of the ARL Statistics and Service Quality Programs
Association of Research Libraries
at
Rutgers University LibraryJune 1 2007
New Brunswick, NJ
www.arl.org/stats/
http://www.minesforlibraries.orghttp://www.arl.org/stats
ARL Overall
http://www.minesforlibraries.orghttp://www.arl.org/stats
Libraries Remain a Credible Resource in Libraries Remain a Credible Resource in 2121stst Century Century
98% agree with statement98% agree with statement, “My … library , “My … library contains information from credible and contains information from credible and known sources.”known sources.”
Note. Digital Library Federation and Council on Library and InformationResources. (2002). Dimensions and Use of the Scholarly Information Environment.
http://www.minesforlibraries.orghttp://www.arl.org/stats
Changing BehaviorsChanging Behaviors
Only Only 15.7% agreed with the statement15.7% agreed with the statement “The “The Internet has not changed the way I use the Internet has not changed the way I use the library.”library.”
Note. Digital Library Federation and Council on Library and InformationResources. (2002). Dimensions and Use of the Scholarly Information Environment.
http://www.minesforlibraries.orghttp://www.arl.org/stats
ARL Toolkit…
• StatsQUAL+™– ARL Statistics – LibQUAL+®– E-Metrics– DigiQUAL+™
– MINES for Libraries™
http://www.minesforlibraries.orghttp://www.arl.org/stats
What is MINES?
• Action research– Historically rooted in indirect cost studies– Set of recommendations for research design– Set of recommendations for web survey
presentation– Set of recommendations for information
architecture in libraries– Plan for continual assessment of networked
electronic resources– An opportunity to benchmark across libraries
http://www.minesforlibraries.orghttp://www.arl.org/stats
MINES for LibrariesTM
• MINES is a transaction-based research methodology consisting of a web-based survey form and a random moments sampling plan
• MINES typically measures who is using electronic resources, where users are located at the time of use, and their purpose of use in the least obtrusive way
• MINES was adopted by the Association of Research Libraries (ARL) as part of the “New Measures” toolkit in May, 2003.
• MINES is different from other electronic resource usage measures that quantify total usage (e.g., Project COUNTER, E-Metrics) or measure how well a library makes electronic resources accessible (LibQUAL+™).
http://www.minesforlibraries.orghttp://www.arl.org/stats
Questions Addressed By MINES for Libraries™ for the OCUL Scholars Portal
• How extensively do sponsored researchers use OCUL’s Scholars Portal? How much usage is for non-funded research, instruction/education, student research papers, and course work?
• Are researchers more likely to use the Scholars Portal from inside or outside the library? What about other classifications of users?
• Are there differences in Scholars Portal based on the user’s location (e.g., in the library; on-campus, but not in the library; or off-campus)?
• Could MINES, combined with usage counts, provide an infrastructure to make Scholars Portal usage studies routine, robust, and easily integrated into OCUL’s administrative decision-making process for assessing networked electronic resources?
http://www.minesforlibraries.orghttp://www.arl.org/stats
MINES for LibrariesTM Survey Form Five Questions and a Comment Box
http://www.minesforlibraries.orghttp://www.arl.org/stats
Methodological considerationsExperience with the MINES Survey
Terry PlumAssistant Dean
Simmons GSLIS
Rutgers UniversityJune 1 , 2007
http://www.minesforlibraries.orghttp://www.arl.org/stats
Issues with web surveys
• Research design– Coverage error
• Unequal access to the Internet• Internet users are different than non-users
– Response rate • Response representativeness
– Random sampling and inference– Non-respondents
• Data security
http://www.minesforlibraries.orghttp://www.arl.org/stats
MINES strategy• A representative sampling plan, including sample size, is
determined at the outset. Typically, there are 48 hours of surveying over 12 months at a medical library and 24 hours a year at a main library.
• Random moment/web-based surveys are employed at each site.
• Participation is usually mandatory, negating non-respondent bias, and is based on actual use in real-time.– IRB waiver or approval
• Libraries with database-to-web gateways or proxy re-writers offer a comprehensive networking solution for surveying all networked services users during survey periods.
http://www.minesforlibraries.orghttp://www.arl.org/stats
MINES strategy• Placement
– Point of use– Not remembered, predicted or critical incident
• Usage rather than user– What about multiple usages– Time out ?– Cookie or other mechanism with auto-population or
more recently counting invisibly with a time out. • Distinguish patron association with libraries.
– For example, medical library v. main library. – But what if the resources are purchased across
campus for all. Then how to get patron affiliation?
http://www.minesforlibraries.orghttp://www.arl.org/stats
Web Survey Design Guidelines
• Web survey design guidelines that MINES followed: – Presentation
• Simple text for different browsers – no graphics– Different browsers render web pages differently
• Few questions per screen or simply few questions• Easy to navigate• Short and plain• No scrolling• Clear and encouraging error or warning messages• Every question answered in a similar way - consistent
– Radio buttons, drop downs• ADA compliant• Introduction page or paragraph• Easy to read
– Must see definitions of sponsored research. • Can present questions in response to answers – for example if
sponsored research was chosen, could present another survey
http://www.minesforlibraries.orghttp://www.arl.org/stats
Quality Checks• Target population is the population frame – surveyed the patrons
who were supposed to be surveyed - except in libraries with outstanding open digital collections.
• Check usage against IP. In this case, big numbers may not be good. May be seeing the survey too often.
• Alter order of questions and answers, particularly sponsored and instruction.
• Spot check IP against self-identified location• Spot check undergraduates choosing sponsored research –
measurement error• Check self-identified grant information against actual grants • Content validity – discussed with librarians and pre-tested. • Turn-aways – number who elected not to fill out the survey• Library information architecture -- Gateway v. HTML pages – there
is a substantial difference in results.
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
Documenting the Purpose and Use of Electronic Resources:Experience with the MINES Survey
Brinley FranklinVice Provost for University Libraries
University of Connecticut
Rutgers UniversityJune 1 , 2007
http://www.minesforlibraries.orghttp://www.arl.org/stats
“It is useless to tell the acquisitions librarian that half the monographs ordered will never be used,unless we can specify which 50% to avoid buying.” (Galvin and Kent, 1977)
http://www.minesforlibraries.orghttp://www.arl.org/stats
Reliance on Vendor Statistics
Vendor statistics, while more reliable than in the past, are still maturing.
http://www.minesforlibraries.orghttp://www.arl.org/stats
Measuring Digital Content Use
The most popular current method of measuring usage of electronic resources by libraries is not through web-based usage surveys, but through vendor supplied data of library patron usage or transaction usage.
Web-based usage surveys are increasingly relevant in the collection of usage data to make collection development and service decisions, to document evidence of usage by certain patron populations, and to collect and analyze performance outputs.
Brinley Franklin and Terry Plum, “Successful Web Survey Methodologies for Measuring the Impact of Networked Electronic Services (MINES for LibrariesTM)” IFLA Journal 32 (1) March, 2006
http://www.minesforlibraries.orghttp://www.arl.org/stats
A web-based transactional survey that collects data on users’ demographics and their purpose of use. It is administered in real time over the course of at least a year using a random moments sampling plan.
MINES for LibrariesTM has been administered at 40 North American universities in the last four years. More than 100,000 North American networked services users have been surveyed using a standard protocol.
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
Library User SurveyPatron Status
http://www.minesforlibraries.orghttp://www.arl.org/stats
Library User SurveyAffiliation
http://www.minesforlibraries.orghttp://www.arl.org/stats
Library User SurveyLocation
http://www.minesforlibraries.orghttp://www.arl.org/stats
Library User SurveyPurpose
http://www.minesforlibraries.orghttp://www.arl.org/stats
Sample Survey Data File Generated
Other UConnInstruction/Education/Departmental (Non-Funded) Researchhttp://newfirstsearch.oclc.org/done=referer;dbname=WorldCat;autho=100122319;FSIP12:36:5012/3/2004Off CampusUConn Faculty12.101.104.84
Family StudiesInstruction/Education/Departmental (Non-Funded) Researchhttp://www.jstor.org/cgi-bin/jstor/gensearch12:37:4312/3/2004Off CampusUConn Undergraduate Student12.101.104.84Non-UConnOther Activitieshttp://homerweb.lib.uconn.edu/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First13:08:4112/3/2004Off CampusNon-UConn12.18.36.40
Non-UConnInstruction/Education/Departmental (Non-Funded) Researchhttp://magic.lib.uconn.edu/index_real.html13:31:2912/3/2004Off CampusNon-UConn12.76.131.246
Non-UConnOther Activitieshttp://magic.lib.uconn.edu/index_real.html12:11:0612/3/2004Off CampusNon-UConn130.132.86.156Agriculture & Natural ResourcesInstruction/Education/Departmental (Non-Funded) Researchhttp://magic.lib.uconn.edu/index_real.html12:33:5712/3/2004
Off CampusNon-UConn131.128.89.54EducationInstruction/Education/Departmental (Non-Funded) Researchhttp://www.euromonitor.com/womdas/12:57:4412/3/2004
Off CampusNon-UConn134.241.135.70Non-UConnInstruction/Education/Departmental (Non-Funded) Researchhttp://homerweb.lib.uconn.edu/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First13:28:5212/3/2004
Off CampusNon-UConn134.74.42.82Business AdministrationOther Activitieshttp://homerweb.lib.uconn.edu/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First12:56:4612/3/2004In the LibraryUConn Faculty137.99.1.122
Liberal Arts & SciencesOther Activitieshttp://www.siam.org/journals/simax/simax.htm12:52:1712/3/2004On Campus - StorrsUConn Graduate Student137.99.1.122
EngineeringInstruction/Education/Departmental (Non-Funded) Researchhttp://homerweb.lib.uconn.edu/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First12:04:3112/3/2004On Campus - StorrsUConn Graduate Student137.99.1.219
Business AdministrationInstruction/Education/Departmental (Non-Funded) Researchhttp://proquest.umi.com/pqdweb?RQT=31812:16:3312/3/2004On Campus - StorrsUConn Graduate Student137.99.100.105
Business AdministrationInstruction/Education/Departmental (Non-Funded) Researchhttp://www.jstor.org/journals/07322399.html12:16:5212/3/2004On Campus - StorrsUConn Graduate Student137.99.100.105
Business AdministrationInstruction/Education/Departmental (Non-Funded) Researchhttp://homerweb.lib.uconn.edu/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First12:29:5312/3/2004On Campus - StorrsUConn Graduate Student137.99.100.175
Business AdministrationInstruction/Education/Departmental (Non-Funded) Researchhttp://homerweb.lib.uconn.edu/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First12:48:4112/3/2004On Campus - StorrsUConn Graduate Student137.99.100.175
Business AdministrationInstruction/Education/Departmental (Non-Funded) Researchhttp://proquest.umi.com/login?COPT=SU5UPTAmVkVSPTImREJTPTE3MjErMysxNkJD&clientId=4899612:04:2312/3/2004On Campus - StorrsUConn Graduate Student137.99.100.175
http://www.minesforlibraries.orghttp://www.arl.org/stats
Demographics by Location of UserU.S. Main Libraries
24%
25%
8%
43%
GraduateStudents
Faculty, Staff,Research Fellows
UndergraduateStudents
All Other Users
40%
31%
25%
4%28%
11%
21%
40%
On Campus, Not in the Libraryn = 6,391
Inside the Libraryn = 9,172
Off-Campusn = 4,953
MINES for Libraries™
http://www.minesforlibraries.orghttp://www.arl.org/stats
14%
14%
69%
3%
Graduate Students
Faculty, Staff,Research Fellows
UndergraduateStudents
All Other Users
Off-Campusn = 9,163
Demographics by Location of UserOntario Council of University Libraries
Inside the Libraryn = 4,047
On Campus, Not in the Libraryn = 7,090
45%
24%
29%
2%
31%
16%
48%
5%
http://www.minesforlibraries.orghttp://www.arl.org/stats
Purpose of Use
Are users engaged in coursework, funded (or unfunded) research, public service, patient care, or other activities?
http://www.minesforlibraries.orghttp://www.arl.org/stats
5%
26%3%
SponsoredResearch
Instruction
Other
OtherSponsoredActivities
Purpose of Use By LocationU.S. Main Campus Libraries
2003 – 2005
*72% of sponsored research usage of electronic resources occurred outside the library; 83% took place on campus.
63%
14%2%
21%
11%
62%
25%2%
6%
58%
34%2%
On-Campus, Not in the Libraryn = 9,460
In the Libraryn = 9,733
Off-Campusn = 7,790
Overall Usen = 26,983
66%
http://www.minesforlibraries.orghttp://www.arl.org/stats
OCUL Scholars Portal Users Purpose of Use
In a sample of 20,300 electronic resources uses at OCUL libraries,
there were four uses outside the library for each use in the library.
8%6%2%
16%
26% 42% Sponsored Research
Coursework
Other Research
Other Activities
Teaching
Patient Care
http://www.minesforlibraries.orghttp://www.arl.org/stats
Questions?
Learn more about LibQUAL+TM, DigiQUALTM, & MINES for LibrariesTM at: www.statsqual.org
http://www.minesforlibraries.orghttp://www.arl.org/stats
Analysis
• Web deliverables:– Crosstabulations in html for all OCUL data– Interactive crosstabs for all OCUL and
institutions
• Print deliverables:– summary tables for OCUL – summary tables for each institution– Final report
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
OCUL Scholars Portal UsageAffiliation
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
Affiliation by Purpose of Use
Purpose of Use
Affiliation CourseworkOther
ActivitiesOther
ResearchPatient
Care Sponsored Teaching Total
Applied Sciences 24.0% 7.6% 17.7% 0.6% 46.3% 3.7% 100.0%
Business 34.8% 7.6% 30.0% 0.9% 10.8% 16.0% 100.0%
Education 40.9% 5.4% 17.1% 0.8% 11.8% 24.0% 100.0%
Environmental Studies 43.5% 2.5% 24.0% 0.3% 23.3% 6.3% 100.0%
Fine Arts 56.3% 6.9% 20.6% 1.3% 5.6% 9.4% 100.0%
Humanities 51.5% 10.8% 21.0% 0.5% 9.5% 6.7% 100.0%
Law 67.5% 6.8% 12.8% 0.9% 2.6% 9.4% 100.0%
Medical Health 29.7% 5.5% 18.4% 8.6% 32.0% 5.7% 100.0%
Other 51.9% 22.8% 10.9% 2.1% 7.4% 5.0% 100.0%
Sciences 44.6% 9.7% 11.1% 0.4% 31.8% 2.4% 100.0%
Social Sciences 62.6% 4.5% 14.4% 0.7% 13.6% 4.2% 100.0%
Total 42.0% 7.5% 16.2% 2.4% 26.2% 5.6% 100.0%
http://www.minesforlibraries.orghttp://www.arl.org/stats
User Status by Purpose of Use
Purpose of Use
User Status CourseworkOther
ActivitiesOther
ResearchPatient
Care Sponsored Teaching Total
Faculty 1.5% 4.7% 21.2% 4.4% 42.6% 25.6% 100.0%
Graduate Professional 19.5% 3.9% 25.5% 2.5% 45.4% 3.2% 100.0%
Library Staff 23.5% 24.1% 13.1% 16.5% 17.7% 5.2% 100.0%
Other 6.0% 35.2% 20.8% 8.7% 26.8% 2.5% 100.0%
Staff 3.5% 9.5% 20.6% 2.1% 51.6% 12.7% 100.0%
Undergraduate 75.8% 7.8% 7.7% 0.9% 5.9% 1.9% 100.0%
Total 42.0% 7.5% 16.2% 2.4% 26.2% 5.6% 100.0%
http://www.minesforlibraries.orghttp://www.arl.org/stats
Location by Purpose of Use
Purpose of Use
Location CourseworkOther
ActivitiesOther
ResearchPatient
Care Sponsored Teaching Total
Library 52.8% 14.9% 10.8% 1.2% 12.3% 7.9% 100.0%
Off-campus 47.2% 7.0% 17.3% 4.1% 19.9% 4.6% 100.0%
On-campus 29.2% 4.0% 17.9% 0.9% 42.2% 5.7% 100.0%
Total 42.0% 7.5% 16.2% 2.4% 26.2% 5.6% 100.0%
http://www.minesforlibraries.orghttp://www.arl.org/stats
Reason for Use
Reason for Use (n=20293) Frequency Reason for Use (n=20293) Percent
Important Journal 10219 Important Journal 50.4%
Recommended Colleague 2436 Recommended Colleague 12.0%
Reference/Citation 6090 Reference/Citation 30.0%
Recommended Librarian 620 Recommended Librarian 3.1%
Course Reading 925 Course Reading 4.6%
Other 4388 Other 21.6%
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
http://www.minesforlibraries.orghttp://www.arl.org/stats
Library Assessment Conference
August 4-6, 2008
Seattle, WA, USA