Using LibQUAL+™ ResultsObservations from ARL Program “Making

Library Assessment Work”

Steve HillerUniversity of Washington Libraries

ARL Visiting Program Officer 2004-06

Making Library Assessment Work: Practical Approaches for Developing and

Sustaining Effective Assessment

• 2 year ARL project to assist libraries with moving assessment efforts forward

• Led by Steve Hiller (UW) and Jim Self (UVa) • Recognition that libraries understand value of assessment

but have trouble sustaining efforts and using results• Site visits conducted to evaluate assessment and develop

practical approaches to effective local practices• Identify common barriers and facilitators to assessment• 7 libraries participating in Phase I Spring 2005; 12-14

during 2005-06 academic year; final report in 2006

Phase I Libraries and Latest LibQUAL+™ Survey

• Spring 2005 Participants Latest Survey Year– University of Arizona 2004 (2005 too)

– Arizona State University 2004

– University of Connecticut 2004

– University of Illinois U-C 2002

– New York University 2002

– University of Notre Dame 2002

– University of Oregon 2005

Using LibQUAL+™ ResultsSome General Observations

• Response numbers low, especially for faculty• Comparisons important

– Between internal group– By academic area– Peer institutions– Over time

• Uncertain how to analyze• Uncertain how to follow-up unless very negative• Difficulty in using for planning and decision-making• Comments help tell the story and pinpoint problems

SIZE MATTERS! Maximize the Number of Survey Responses

Institution (year) Undergrad Grad Faculty

Arizona (2004) 118 122 82

Arizona State (2004) 107 200 205

Connecticut (2004 304 195 96

NYU (2002) 322 86 146

Notre Dame (2002) 308 255 183

Oregon (2004) 114 87 63

LibQUAL+ ™ Number of Survey Respondents by Group

Using Negative Results Positively University "Q" and ARL

"Comfortable & Inviting Environment"

Q

Q

ARL

Q

Undergrads

ARL

Graduate

ARL

Faculty

5.5

5.7

5.9

6.1

6.3

6.5

6.7

6.9

7.1

7.3

7.5

7.7

7.9

8.1

Minimum

Perceived

Desired

6.5

6.6

6.7

6.8

6.9

7

7.1

7.2

7.3

Undergrad Grad Faculty

What's Going On? Overall Satisfaction by Group University "Z" 2002-2004

2002 2003 2004

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

Affect Info Control Place

Adequacy Gaps by Dimension & Group University "Z" 2003 and 2004

Grad 2003 Grad 2004 Fac 2003 Fac 2004

Academic Area Grad 2003

Grad 2004

Grad Pop

Faculty 2003

Faculty 2004

Faculty Pop

Fine Arts/Arch 5% 8% 4% 9% 15% 5%

Humanities 9% 14% 6% 8% 15% 9%

Social Sciences 21% 22% 11% 19% 18% 16%

Science 7% 16% 11% 23% 16% 19%

Engineering 12% 9% 12% 10% 4% 7%

Respondent Composition by Selected Academic Area & Group 2003-04 - University “Z”

Some Complementary Information

• Fine Arts/Architecture libraries “temporarily” merged• Budget reductions impacted collections and staff

– Book budget cut

• Some hours reduced in branch libraries

• Comments corroborated the above as important concerns for faculty and graduate students

University “X” Peer Comparisons

UGTreat

UG Support

UGOver

all

GradTreat

Grad Support

GradOver all

Fac Treat

FacSupport

FacOver all

“X” 7.46 7.33 7.40 7.40 7.09 7.26 7.41 6.67 7.00

Peer 1 7.12 6.77 6.98 7.42 6.95 6.99 7.50 6.99 7.27

Peer 2 6.96 6.75 6.81 6.90 6.72 6.82 7.53 7.02 7.27

Peer 3 7.12 6.77 6.98 7.42 6.95 6.99 7.50 6.99 7.27

Peer 4 6.92

6.73 7.02 7.30

6.91 7.06 7.69

7.12 7.37

University “X” Overall Library Quality:Mining the Data - Faculty Frequency Responses By Area

Other Areas

Liberal Arts

0%

10%

20%

30%

40%

50%

1 2 3 4 5 6 7 8 9

0%

10%

20%

30%

40%

50%

University “X” Comments

• Likes– Librarian liaisons*****

– ILL**

• Concerns– Copiers

– Loan periods/renewals– Print collection (Faculty)

– Journal collection (Faculty)

– Noise

– Student assistants

– Off-campus online access

– Web site

Service Adequacy Gaps by Dimension

University “X” Undergrads by Year in School

1st Year

3rd year

4th Year

2nd year

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

Affect Place Info Control

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

Information Control “Y”/ARL Adequacy Gaps By Group 2005

"Y" Grad

"Y" Grad

"Y" Faculty

"Y" Faculty

ARL Faculty

ARL Faculty

ARL GradARL Grad

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Remote Web site Access tools Independent OnlineResouces

Print Journals

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

"Y" Grad"Y" FacultyARL FacultyARL Grad

Journal Collections I RequireFaculty/Grad Student Adequacy Gaps (ARL Ranks 61-110)

-1.5

-1.3

-1.1

-0.9

-0.7

-0.5

-0.3

-0.1

-1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5FACULTY

GR

AD

ST

UD

EN

TS

ARL Rank 61-80 "Y" ARL Rank 81-110 ARL Mean