Beyond Lists and Guides - Bowling Green State...

Beyond Lists and Guides

Using Usability to Help Students Get the Most Out of E-Resources

Amy Fry, Electronic Resources CoordinatorBowling Green State University

http://personal.bgsu.edu/~afry/

BGSU Project

• Identify best practices

• Recommend updates based on these

• Implement updates

• Test updated pages with users

BGSU’s “All Research Databases” Landing Page

BGSU’s Databases A-Z List

Business “Databases-by-Subject” List

Resource Record for Business Source Complete

• Databases A-Z list

• Databases-by-subject lists

• Full resource records

• Software

• Discovery layer or federated search

• Link name

• Order of databases-by-subject lists

• Use of icons/graphics

Other surveys of ARL libraries

• Cohen and Calsada (2003)Found that 66 of 114 academic ARLs used database-driven webpages to present their e-resources in 2002.

• Shorten (2006)Found that 88.6% of ARL libraries had databases A-Z lists in 2003, and 10.5% also categorized them by type.

• Caudle and Schmitz (2007)Found that 97% of the 99 American academic libraries in ARL had a databases A-Z list, 96% had databases-by-subject lists and 27% had federated searching.

Type of system # of libraries %*

Homegrown 81 71.1%

Metalib 14 12.3%

Innovative 8 7%

LibGuides 4 3.5%

Xerxes 4 3.5%

WebFeat 2 1.75%

LibData 1 <1%

Type of system Homegrown: 71.1%

*Percentages are based on

114 libraries (excluding 7

national/special libraries

and 4 libraries whose

databases pages were

behind a login)

Type of system # of libraries %

Homegrown 81 71.1%

Metalib 14 12.3%

Innovative 8 7%

LibGuides 4 3.5%

Xerxes 4 3.5%

WebFeat 2 1.75%

LibData 1 <1%

Type of system Metalib: 12.3%

Metalib at Yale University

Type of system # of libraries %

Homegrown 81 71.1%

Metalib 14 12.3%

Innovative 8 7%

LibGuides 4 3.5%

Xerxes 4 3.5%

WebFeat 2 1.75%

LibData 1 <1%

Type of system Innovative (III): 7%

Innovative (III) at the University of Western Ontario

Type of system # of libraries %

Homegrown 81 71.1%

Metalib 14 12.3%

Innovative 8 7%

LibGuides 4 3.5%

Xerxes 4 3.5%

WebFeat 2 1.75%

LibData 1 <1%

Type of system LibGuides: 3.5%

LibGuides at the University of Louisville

Type of system # of libraries %

Homegrown 81 71.1%

Metalib 14 12.3%

Innovative 8 7%

LibGuides 4 3.5%

Xerxes 4 3.5%

WebFeat 2 1.75%

LibData 1 <1%

Type of system Xerxes: 3.5%

Xerxes at New York University

Types of access# of

libraries%

Databases A-Z 111 97

Databases-by-subject lists

91 80

Full resource records

83 73

All three 73 64

Queen’s University

Types of access# of

libraries%

Databases A-Z 111 97

Databases-by-subject lists

91 80

Full resource records

83 73

All three 73 64

University of Missouri-Columbia

Types of access# of

libraries%

Databases A-Z 111 97

Databases-by-subject lists

91 80

Full resource records

83 73

All three 73 64

Full resource record from SUNY Buffalo

Subject list order # of libraries %

By relevance 38 41.8%

By format 7 7.7%

Alphabetical only 46 50.5%

Order of subject lists

Subject list order # of libraries %

By relevance 38 41.8%

By format 7 7.7%

Alphabetical only 46 50.5%

Order of subject lists

Full text databases for Art – SUNY Albany

Subject list order # of libraries %

By relevance 38 41.8%

By format 7 7.7%

Alphabetical only 46 50.5%

Order of subject lists

Metalib at Northwestern University

Icon # of libraries

Access restrictions 38

More information 27

Full text 9

Magnifying glass (Metalib: search in database)

5

Tutorials 4

Funding source 3

Format (audio, etc.) 3

Plus sign (Metalib: add to a set)

2

Social media 2

Metasearch 2

Logo/screenshot 2

RefWorks 1

New 1

Plus-star 1

SFX 1

Libraries using icons or graphics: 64 (56%)

Use of icons and graphics

Name of LinkLink title begins with… # of libraries % Examples

“Databases” 47 41% Databases (30), Databases A-Z (8)

“Articles” 22 18.6% Article Databases (4)Articles & Databases (8)

“E” or “Electronic” 16 13.6% E-Resources (7), Electronic Resources (5)

“Find” 8 6.8% Find Articles (3)Find Articles & Databases (1)

“Research” 8 6.8% Research Databases (3)

“Search” 4 3.4% Search & Find (2), Search a Database (1)

“Indexes” 2 1.7% Indexes & Databases (1)Indexes & Databases (Articles) (1)

“Journal” 2 1.7% Journal Articles (2)

Branded names 2 1.7% Vera: E-Journals & Databases Galileo @ UGA

Other 4 < 1% each

Resource Gateway – ResourcesMore DatabasesAll Databases A-Z and Database FinderOnline Research Resources (Databases)

new

old

BGSU resource records: changes

implemented summer 2009

old

new

Databases-by-Subject lists: changes implemented summer 2009

ISO definition of usability:

“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”

Five attributes:

• Ease of learning

• Efficiency of use

• Memorability

• Error frequency and severity

• Subjective satisfaction

Lehman & Nikkel, 2008

Foster & Gibbons, 2007

Krug, 2006

Other library usability studies

• Hammill (2003)Did common task testing with 52 users at Florida International University Libraries, including finding a named database.

• Krueger, Ray and Knight (2004)Did common task testing with 134 users at the University of the Pacific Library.

• Fuller, Livingston, Brown, Cowan, Wood and Porter (2009)Did three rounds of testing with five users each on the databases pages at the University of Connecticut Libraries.

Linda Rich

Reference Coordinator, Bowling Green State University Libraries, and co-investigator for our

usability study.

BGSU usability study timeline

• Identify goals (December 2009)

• Complete Human Subjects Review Board (HSRB) training (January 2010)

• Submit HSRB application, including script, recruitment materials, consent form (January 2010)

• Obtain funding for incentives (January 2010)

• Test the instrument (February 2010)

• Recruit participants (February 2010)

• Complete the testing (February-March 2010)

• Analyze results (March-April 2010)

• Present findings and recommendations (April-May 2010)

Format of the Study

• Part I: the home page (on paper)

• Part II: five tasks on the computer

• Part III: the databases pages (on paper)

• Part IV: Likert scale for research guide names

• Part V: open-ended questions

Part I

Links I’ve used before / # students chosen

EBSCO 9

OhioLINK Catalog 9

Course Reserves 7

My Library Account 7

All Research Databases

5

Hours 5

BGSU Libraries Catalog

4

Search box for catalog

4

E-Journals, etc. 3

By Course (Class Web Pages)

2

Jobs at the Library 2

11 other links chosen once each

Links I find confusing / # students chosen

Digital Resource Commons

8

ILLiad 8

News links 5

WorldCat 5

A-Z Library Website Index

4

RefWorks 4

E-Journals, etc. 3

Evaluating and CitingSources

2

17 other links chosen once each

Links I’ve used before / # students chosen

EBSCO 9

OhioLINK Catalog 9

Course Reserves 7

My Library Account 7

All Research Databases

5

Hours 5

BGSU Libraries Catalog

4

Search box for catalog

4

E-Journals, etc. 3

By Course (Class Web Pages)

2

Jobs at the Library 2

11 other links chosen once each

Student descriptions of the All Research Databases link:

“List of subjects for databases”

“Locating academic resources for papers”

“List A-Z, JSTOR, ARTstor”

“JSTOR, Dissertations, LexisNexis”

Usability Study, Part II

• Task 1: find a named book (given title and author last name)

• Task 2: find scholarly articles on (your topic)

• Task 3: find another source for scholarly articles on (your topic)

• Task 4: find a named database

• Task 5: find a named journal article (given an APA-formatted citation)

1

5

32

Task 1: find a named bookSession # Starting point Initial search # tries Successful?

Session 1 Search box for catalog Title 2 Yes

Session 2 Search box for catalog Title 3 Yes

Session 3 Reserves Title 2 Yes

Session 4 Search box for catalog Title & Author 1 Yes

Session 5 Search box for catalog Author 4 Yes

Session 6 BGSU Libraries Catalog link Title 9 Yes

Session 7 Search box for catalog Title 3 Yes

Session 8 Search box for catalog Title 7 Yes

Session 9 Search box for catalog Title & Author 2 Yes

Session 10 BGSU Libraries Catalog link Title 1 Yes

Session 11 Advanced Search link Title 1 Yes

Session 12 Search box for catalog Title 3 Yes

Session 13 Search box for catalog Title 2 Yes

Session 14 A-Z Library Website Index Title 3 Yes

Session 15 BGSU Libraries Catalog link Author 2 Yes

Task 2: find scholarly articles on your topicSession # Starting point # tries Successful?

Session 1 EBSCO link on home page 1 Yes

Session 2 EBSCO link on home page 1 Yes

Session 3 E-journals, etc. 6 No

Session 4 All Research Databases 1 Yes

Session 5 EBSCO link on home page 1 Yes

Session 6 EBSCO link on home page 1 Yes

Session 7 EBSCO link on home page 1 Yes

Session 8 All Research Databases 1 Yes

Session 9 EBSCO link on home page 1 Yes

Session 10 EBSCO link on home page 1 Yes

Session 11 EBSCO link on home page 1 Yes

Session 12 EBSCO link on home page 3 Yes

Session 13 EBSCO link on home page 1 Yes

Session 14 EBSCO link on home page 1 Yes

Session 15 EBSCO link on home page 1 Yes

Task 3: find another source for scholarly articles

Session Strategy Starting point

2 Accessed a known DB All Research Databases

4 Accessed a known DB All Research Databases

9 Accessed a known DB EBSCO link on home page

11 Accessed a known DB All Research Databases

12 Accessed a known e-journal OhioLINK EJC

14 Accessed a known DB Course guides

Students who knew how to complete this task

Session Strategy Starting point

3 Used the subjects on the All Research Databases page and chose the first one in her subject list

All Research Databases

7 Used the subjects on the All Research Databases page and chose a DB she recognized

All Research Databases

8 Googled her topic and “Springer” Google

10 Used the E-Journals link to find a scholarly journal in his subject area

E-Journals

Students who did not how to complete this task but were successful

Task 3: find another source for scholarly articles

Session Strategy Starting point

1 Chose an inappropriate subject and resource from the All Research Databases page

E-Journals, then All Research Databases

6 Chose an inappropriate subject from the All Research Databases page then became lost in the catalog

All Research Databases

13 Browsed by subject under E-Journals E-Journals

15 Chose an inappropriate resource from the All Research Databases page

E-Journals, then All Research Databases

Students who could not successfully complete this task

Task 4: find a named database

Session Starting point Ending point # tries Successful?

2 Databases A-Z list --- 1 Yes

7 Databases A-Z list --- 1 Yes

9 Databases A-Z list --- 1 Yes

11 Databases A-Z list --- 1 Yes

4 Databases-by-subject lists --- 1 Yes

15 Databases-by-subject lists --- 1 Yes

3 Databases-by-subject lists Search box for catalog 2 Yes

1 Search box for catalog --- 1 Yes

10 Search box for catalog --- 1 Yes

6 E-Journals, etc. link All Research Databases title search box

4 Yes

14 All Research Databases page

Google 3 Yes

Students who successfully completed this task

Session Starting point Ending point # tries Successful?

5 E-Journals, etc. link All Research Databases page 2 No

8 Databases-by-subject lists Google 2 No

12 A-Z Library Website Index --- 1 No

13 Databases-by-subject lists --- 2 No

Students who could not successfully complete this task

Task 4: find a named database

Task 5: find a named article

Link # tried # successful

E-Journals, etc. 6 3

EBSCO 4 4

Catalog 2 1

All Research Databases 2 0

Google 1 1

Chosen as initial starting point

Chosen as search

Link # successful

EBSCO 4

Google 4

E-Journals, etc. 3

Catalog 3

Not successful 1

Successful path

Search # used as first search # successful with this search

Article title 7 5

Author 5 2

Journal title 3 7

BGSU’s “All Research Databases” Landing Page

BGSU’s Databases A-Z List

Business “Databases-by-Subject” List

Resource Record for Business Source Complete

“I try to avoid this page.”-- graduate student

“I’m not sure what these are. I

feel like I should know this.”-- graduate student

“I’d probably quit after I got this.”-- undergraduate student

“If I were a business

major, I would definitely

use this page.”-- undergraduate student

Important fields #

Description 14

Dates 10

Full text 7

License information 6

Subjects 6

Title (resource name)

5

Local contact 4

Coverage load 2

Resource format 2

User support 2

Five other fields chosen once each

Confusing fields #

Mobile access 10

Coverage load 6

On-campus access 4

Index to journal articles

3

License information 2

Subjects 2

User support 2

Three other fields chosen once each

Unimportant fields #

User support 3

Local contact 2

Public note 2

Title (resource name)

2

Authorized users 1

Coverage load 1

Description (long) 1

Index to journalarticles

1

“MIS” (in record) 1

Mobile access 1

Subject 1

“I usually pay attention to dates and descriptions.”

“Do I do all this down here, too?” (about coverage load)

“I don't pay attention to certain things but that doesn't mean they shouldn't be in there.”

Conclusions• Students stick with what they know.

Conclusions• Students stick with what they know.

• Students do what their professors tell them to do.

Conclusions• Students stick with what they know.

• Students do what their professors tell them to do.

• “As long as you can search for anything, you can search for it.”

Conclusions• Students stick with what they know.

• Students do what their professors tell them to do.

• “As long as you can search for anything, you can search for it.”

• “Google is powerful.”

Conclusions• Students stick with what they know.

• Students do what their professors tell them to do.

• “As long as you can search for anything, you can search for it.”

• “Google is powerful.”

• Subject lists are for librarians.

Recommendations

• Get specific.

+

+

+

• Get specific.

• Use relevance ranking.

Recommendations

+

• Get specific.

• Use relevance ranking.

• Implement a discovery layer!

Recommendations

+

View slides/contact me: http://personal.bgsu.edu/~afry/