DEFINING NEW METRICS FOR LIBRARY SUCCESS

PULLING IT ALL TOGETHER!CARL THOMPSON

COUNTING OPINIONS (SQUIRE) LTD

PULLING IT ALL TOGETHER!

  "GREAT SYMPOSIUM .... 'NOW WHAT'?“

WHY SHOULD WE DO THIS?

MEASURESEFFORT

• INPUTS – STAFF, EQUIPMENT, RESOURCES

• PROCESS/ACTIVITY – TRANSFORMING INPUTS INTO OUTPUTS, ACQUISITIONS, SPEED AND RELIABILITY OF SERVICE DELIVERY – TYPICALLY THE KEY RATIOS MEASURING EFFICIENCIES

• OUTPUTS – QUANTITY/UNITS OF SERVICE PROVIDED

EFFECT

• OUTCOMES – MEASURES OF IMPROVEMENT(S) TO CONSUMERS LIVES

Results courtesy of ACRLMetrics

Results courtesy of ACRLMetrics

Results courtesy of PLAmetrics

PLDS SURVEY RESULTS

CONTINUOUS LIBRARY RESPONDER GROUP

COMPARING FY2011 TO FY2012, ON AVERAGE PER $1000 SPENT:

1.63% FEWER VISITS

1.41% FEWER CIRCULATIONS

6.25% FEWER REFERENCE TRANSACTIONS

22.63% FEWER IN-LIBRARY USES

4.36% MORE REGISTERED BORROWERS

3.36% MORE PROGRAM ATTENDANCE

PURPOSE

IMPROVING THE LIVES OF THOSE YOU SERVE

THE SHIFT TO OUTCOMES-BASED THINKING

OUTCOMES-BASED APPROACHES TO PUBLIC SERVICE DELIVERY

AN APPROACH USED BY THE ORGANISATION

--OR--

PRESCRIBED BY AGREEMENT WITH FUNDING PARTNERS & OTHER STAKEHOLDERS

OUTCOMES BASED EVALUATIONEVIDENCE-BASED MANAGEMENT

IMLS HAS RESPONDED TO YOUR NEED TO KNOW WHAT DIFFERENCE OUR SERVICES MAKE BY DEVELOPING A MODEL OF OUTCOMES-BASED PLANNING AND EVALUATION AS ONE FOUNDATION OF OUR GRANT MAKING.

SUSAN HILDRETHDIRECTOR, INSTITUTE OF MUSEUM AND LIBRARY SERVICES

ACADEMIC LIBRARIES AND OUTCOMES

OUTCOMES … THE WAYS IN WHICH LIBRARY USERS ARE CHANGED AS A RESULT OF THEIR CONTACT WITH THE LIBRARY'S RESOURCES AND PROGRAMS. OUTCOMES THAT ALIGN CLOSELY TO THE MISSIONS OF LIBRARIES AND THE INSTITUTIONS THEY SERVE. THE IMPORTANT OUTCOMES OF AN ACADEMIC LIBRARY PROGRAM INVOLVE THE ANSWERS TO QUESTIONS LIKE THESE:

• IS THE ACADEMIC PERFORMANCE OF STUDENTS IMPROVED THROUGH THEIR CONTACT WITH THE LIBRARY? 

• BY USING THE LIBRARY, DO STUDENTS IMPROVE THEIR CHANCES OF HAVING A SUCCESSFUL CAREER? 

• ARE UNDERGRADUATES WHO USED THE LIBRARY MORE LIKELY TO SUCCEED IN GRADUATE SCHOOL? 

• DOES THE LIBRARY'S BIBLIOGRAPHIC INSTRUCTION PROGRAM RESULT IN A HIGH LEVEL OF "INFORMATION LITERACY" AMONG STUDENTS? 

• AS A RESULT OF COLLABORATION WITH THE LIBRARY'S STAFF, ARE FACULTY MEMBERS MORE LIKELY TO VIEW USE OF THE LIBRARY AS AN INTEGRAL PART OF THEIR COURSES? 

• ARE STUDENTS WHO USE THE LIBRARY MORE LIKELY TO LEAD FULLER AND MORE SATISFYING LIVES?

CALIFORNIA'S STATEWIDE SUMMER READING OUTCOMES

• DESIGNED TO CAPTURE AND HELP LIBRARIES DEMONSTRATE THE VALUE, THE POTENTIAL, AND THE IMPACT OF CALIFORNIA'S SUMMER READING PROGRAMS.

• OUTCOME ONE: CHILDREN [TEENS/ADULTS/FAMILIES]

BELONG TO A COMMUNITY OF READERS AND LIBRARY USERS

• OUTCOME TWO: UNDERSERVED COMMUNITY MEMBERS

PARTICIPATE IN THE SUMMER READING PROGRAM

CALIFORNIA SUMMER READING OUTCOMES PROJECT

1. HOW MANY PEOPLE IN YOUR FAMILY PARTICIPATED IN THE SUMMER READING PROGRAM? ______

2. IS THIS YOUR FIRST YEAR TAKING PART IN THE LIBRARY’S SUMMER READING PROGRAM AS A FAMILY?

O YES O NO, WE HAVE TAKEN PART AS A FAMILY BEFORE

3. DO YOU TALK ABOUT THE BOOKS YOU READ?

O YES O NO O DON’T KNOW

4. DID YOU ENJOY THE LIBRARY’S SUMMER READING PROGRAM?

O YES O NO O DON’T KNOW

4A. WHY? / WHY NOT? ______________________________________________________

5. WILL YOU COME BACK TO THE LIBRARY AFTER THE SUMMER?

O YES O NO O DON’T KNOW

5A. WHY? / WHY NOT? ______________________________________________________

6. IS THERE ANYTHING ELSE YOU WANT TO TELL US?

PERFORMANCE MEASURES CONTINUUM

Inputs Processes Outputs

Outcomes Impacts

FROM THE PERSPECTIVE OF THE USER

• PERSONAL OUTCOMES• EXPECTATIONS (EXPECT)

• ASPIRATIONS (HOPE)

• MOTIVATIONS (REASON(S))

• LIBRARY VIEW• PRESCRIPTIVE / ASPIRATIONAL

• COMPARE TO OURSELVES, TO OTHERS, TO STANDARDS

TO INFORM

• PEOPLE, PROCESSES, STAKEHOLDERS

• DECISION MAKERS

• CUSTOMERS AND SERVICE DELIVERY PARTNERS

• PRIORITIES, PLANNING

• GOALS AND OBJECTIVES

• CONTINUOUS IMPROVEMENT

SELF ASSESSMENT

1)LIST WAYS YOU DO/CAN IMPROVE THE LIVES OF THOSE YOU SERVE?

2) WHAT ARE SOME OUTCOMES YOU CAN/SHOULD MEASURE?

3) WHAT IMPACTS ARE YOU ULTIMATELY TRYING TO ACHIEVE?

4) WHAT MESSAGES WILL ENHANCE ADVOCACY / CONTINUOUS IMPROVEMENT?

OUTCOMES IMPROVE HOW USERS?

Users

Changes

QUANTIFYING OUTCOMES

• BASELINE/CONTEXTUAL (STARTING POINT)

• THE RIGHT SET OF MEASURES (E.G., PRE-ASSESSMENT/POST-ASSESSMENT)

• IMMEDIATE RESULTS (FEELINGS/PERCEPTIONS)

• USER ACTIONS TAKEN + RESULTS

• CORRELATION TO SERVICES PROVIDED

• OTHER FACTORS/INFLUENCES (TEMPERED BY)

POTENTIAL CHALLENGES

•AMBIGUITY

• INTERPRETATION

•ATTRIBUTION

•RELIABILITY / CONSISTENCY

•COMMITMENT

ACTION PLANARE USERS ARE BETTER OFF? FIND WAYS TO MEASURE THIS.

CONTINUE TO GATHER EFFORT DATA (INPUTS, PROCESS AND OUTPUTS)

FOCUS ON MEASURES THAT SUPPORT ADVOCACY AND CONTINUOUS IMPROVEMENT.

MAKE SURE OTHERS IN YOUR ORGANIZATION

UNDERSTAND AND EMBRACE OBE IN EVERY

ASPECT OF WHAT THEY DO.