Upload
shadi
View
20
Download
0
Embed Size (px)
DESCRIPTION
The READ Scale. G. Lynn Berard Bella Karr Gerlich. The READ Scale. R eference E ffort A ssessment D ata A six point (1 - 6) sliding scale that asks librarians to assign a number based on effort / knowledge / skill / teachable moment instead of a hash mark after a reference transaction. - PowerPoint PPT Presentation
Citation preview
Lynn Berard / Bella Gerlich
The READ Scale
G. Lynn Berard
Bella Karr Gerlich
Lynn Berard / Bella Gerlich
The READ Scale Reference Effort Assessment Data
A six point (1 - 6) sliding scale that asks librarians to assign a number based on effort / knowledge / skill / teachable moment instead of a hash mark after a reference transaction. The READ (Reference Effort Assessment Data) Scale Creative Commons License This work is licensed under a Creative Commons Attribution- Noncommercial- No Derivative Works 3.0 Unported License. Bella Karr Gerlich
Lynn Berard / Bella Gerlich
Objective
Test the validity of the READ Scale as an additional tool for gathering reference statistics to record and recognize the effort / knowledge / skills / value-added service required during a reference transaction.
READ Scale Study
Lynn Berard / Bella Gerlich
READ Scale Study
Recruit 9 - 15 academic libraries Public and private Geographically diverse Various enrollment size
Lynn Berard / Bella Gerlich
14 Institutions, 12 States
Lynn Berard / Bella Gerlich
The InstitutionsEnrollment under 5,000
Eastern Virginia Medical SchoolEdward E. Brickell Medical Sciences Library, Norfolk, VA
Lawrence UniversitySeeley G. Mudd Library, Appleton, WI
Lewis & Clark CollegeAubrey R. Watzek Library, Portland, OR
Clarke CollegeClarke College Library, Dubuque, IA
Our Lady of the Lake University San Antonio (OLLUSA)Sueltenfuss Library, San Antonio, TX
Lynn Berard / Bella Gerlich
The InstitutionsEnrollment over 5,000
Georgia College & State UniversityGCSU Library & Instructional Technology Center (2 Service Points) Reference & Special Collections, Milledgeville, GA
Carnegie Mellon UniversityHunt Library, Arts,& Spec Coll, Science Library, Mellon Inst, Music Listening, Software Engineering Institute (6 Service Points) Pittsburgh, PA
Robert Morris UniversityPatrick Henry Center, Pittsburgh Center (2 Service Points)Moon Township, PA
Washburn UniversityMabee Library, Topeka, KS
Lynn Berard / Bella Gerlich
The InstitutionsEnrollment over 15,000
West Virginia University (3 Libraries)Health Sciences Library, Downtown Campus Library,Evansdale Library, Morgantown, WV
University of California, San DiegoScience & Engineering Library, La Jolla, CA
New York UniversityBusiness & Documents Center – Bobst Library, New York, NY
Georgia Institute of TechnologyGeorgia Tech Library, Atlanta, GA
University of NebraskaLove Library (Chat Service only), Lincoln, NE
Lynn Berard / Bella Gerlich
Participant Data
14 Institutions 24 Service Points 179 Participants Full / PT / Faculty / Staff Varying experience levels
Lynn Berard / Bella Gerlich
Study Components
IRB / Consent Forms Timeline (3 week and/or semester long) Pre-test / local calibration Blog Online Survey
Lynn Berard / Bella Gerlich
Study Components - IRB
IRB (Institutional Review Board) approval Done at GCSU and Carnegie Mellon Done at Institutions if required Consent forms delivered electronically,
signed & returned with data at end of 3 week study period
Lynn Berard / Bella Gerlich
Study Components - Timeline
3 Week and / or Full Semester Feb. 2 - Feb. 24, 2007 - 14 Institutions (all) Full Semester - 7 Institutions
February dates selected to give institutions time to test as well as minimize chances for spring break, holidays etc.
Lynn Berard / Bella Gerlich
Study Components - Pre-test, Local Calibration
Sample Questions created to choose from Encouraged to include questions typical to
their home institution (ie collection specific) On-Site Coordinator distributed locally and
calibrated, creating an ‘example key’ for participants
Asked to record time for each during test phase
Lynn Berard / Bella Gerlich
Study Components - Pre-test, Local Calibration
Pre-test allowed for study-wide calibration - test questions, responses, time and READ Scale category assignments were the same across institutions for the most part
Institutions used their own recording sheets
Lynn Berard / Bella Gerlich
Study Components - Blog
Blog set up Only one question received during study Online survey responses suggest that the
READ Scale was easy to apply, could explain why blog was not utilized
Lynn Berard / Bella Gerlich
Study Components - Online Survey
Online Survey sent at the conclusion of the three week study period to all participants
Response rate was high - 102 responses out of 179, or 56%
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results
Question 1: Please rank your degree of difficulty using the READ Scale.
Responses
not difficult somewhat difficult
moderately difficult
difficult very difficult
Skipped Question
Number responded
52 (51.0%) 38 (37.3%) 10(9.8%)
2 (2.0%)
0 0 102
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results
Question 2: Was the READ Scale easy to apply?
Responses
very easy to apply
easy to apply
moderately easy
somewhat easy
not easy Skipped Question
Number responded
16 (15.7%)
39 (38.2%)
38 (37.3%)
8 (7.8%)
1(1.00%)
0 102
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results Question 3: Please rank the level of perceived "added
value" the READ Scale placed on statistics gathering for reference transactions.
Responses
extreme value added
high value added
moderate value added
minimal value added
no value added
Skipped Question
Number responded
7(6.9%)
46 (45.5%)
35 (34.7%)
9 (8.9%)
4 (4.0%)
1 (.99%)
101
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results Question 4: Did you have difficulty in deciding between
ratings? If so check all that apply.
Scale
1-2 2-3 3-4 4-5 5-6 no difficulty
Resp
Total
Skip
Resp
12 (7.6%)
32 (20.4%)
46 (29.3%)
31 (19.7%)
15 (9.6%)
21 (13.4%)
157 99 3
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results Question 5: How did you feel about evaluating your own
efforts?
Responses
Extremely Comfortable
Very Mod. Min. NotSkipped Question
Number responded
12 (11.9%)
50 (49.5%)
35 (34.7%)
4 (4.0%)
0 (0%)
1 (.99%) 101
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results
Question 6: Would you recommend the READ Scale to another reference librarian?
Response Response Total
Yes 68 (67.3%)
No 12 (11.9%)
Yes, but with some modifications
21 (20.8%)
Skipped Question 1
Number responded 101
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results
Question 7: Would you like to see this scale adopted for use in your library?
Response Response Total
Yes 50 (50.5%)
No 18 (18.2%)
Yes, but with some modifications
31 (31.3%)
Skipped Question 3 (2.97%)
Number responded 99
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results - Likes
Question 8 asked participants what they liked about the Scale.
The likes listed by the participants where coded into the six most common reoccurrences:
Effort / Value; (17)Approach to Evaluation; (13)Types / Levels; (9)Time; (5)Staffing Levels; (6)Reporting to Administration (5)
Lynn Berard / Bella Gerlich
Likes
“It gave me a quick visible check of my recent efforts. This made my desk work more rewarding, since I sometimes feel like I do so many 1s and 2s- but I could see that I was actually doing a higher level of reference than I realized. It added value to the statistics - literally.”
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results - Dislikes
Question 9: Participants were also asked to list their dislikes. These were coded into the six most common reoccurrences:
Difficult to Apply / Subjectivity; (19)Types / Levels; (16)Approach to Evaluating; (9)Knowledge of Staff; (6)Effort / Value; (4)
Lynn Berard / Bella Gerlich
Dislikes
“At times it was difficult to rate effort.”
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results
Question 10: Do you have any modifications you would like to suggest to improve the READ Scale?
Response Response Total
No 67 (72.8%)
Yes (please describe) 25 (27.2%)
Total Respondents 92 (90.1%)
Skipped Question 10 (9.90%)
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results - Modifications
Modifications were put into the following categories:
Delivery Method/READ Scale Appearance; (9)
Time Element; (5)
Skill Level Element; (4)
Clarity of Categories; (4)
Discussion Component; (2)
Comments / Observations. (2)
Lynn Berard / Bella Gerlich
Modifications
“Perhaps, clarify who is answering the questions” (skill level).
“Collect length of time as another way to gauge level of difficulty”
Lynn Berard / Bella Gerlich
Study Components - Online Survey - Results
Question 11: Were there any changes in your personal approach to reference service while you were using the READ Scale?
Responses Response Total
No 88 (89.8%)
Yes (please describe) 10 (10.2%)
Total Respondents 98 (96.0%)
Skipped Question 4 (3.9%)
Lynn Berard / Bella Gerlich
Personal approaches
“More likely to think about the level of service being provided.”
“I gave more conscience thought to the processes or steps involved in order to rate each interaction.”
Lynn Berard / Bella Gerlich
Survey Results - Overall
No difficulty using the Scale Easy to apply Ranked perception of added value to reference statistics
as ‘high’ Staff comfortable with rating their own efforts 68% would recommend as is, 20% with modifications 50% would adopt as is, additional 31% with
modifications Low percentage of changes in approach (10%)
reinforces ease of use / local adaptability
Lynn Berard / Bella Gerlich
READ Scale - What works
Local approach to using the READ Scale Pre-testing / common questions Easy to use Adds value to data gathering Adds value to work / satisfaction Records previously unrecorded effort /
knowledge / skills (service point and off-desk)
Lynn Berard / Bella Gerlich
READ Scale - preliminary data
Comparisons per service point (READ Scale) Comparisons off-desk (READ Scale) Approach type, service points Approach type, off-desk
Lynn Berard / Bella Gerlich
Three Week Data, Service PointsService Point Comparisions, READ Scale Category Percentages
0% 20% 40% 60% 80% 100%
Service Point 1
Service Point 2
Service Point 3
Service Point 4
Service Point 5
Service Point 6
Service Point 7
Service Point 8
Service Point 9
Service Point 10
Service Point 11
Service Point 12
Service Point 13
Service Point 14
Service Point 15
Service Point 16
Service Point 17
Service Point 18
Service Point 19
Service Point 20
Service Point 21
Service Point 22
Service Point 23
Service Point 24
1
2
3
4
5
6
Lynn Berard / Bella Gerlich
Three Week Data, Off-Desk
Off-desk READ Study Category Comparisons
0% 20% 40% 60% 80% 100%
1 Off-Desk
2 Off Desk
3 Off-Desk
4 Off-Desk
5 Off-Desk
6 Off-Desk
7 Off-Desk
8 Off-Desk
9 Off-Desk
10 Off-Desk
11 Off-Desk
12 Off-Desk
13 Off-Desk
14 Off-Desk
15 Off-Desk
16 Off-Desk
17 Off-Desk
1
2
3
4
5
6
Lynn Berard / Bella Gerlich
Full Semester Participants Data, Service Points
Semester Category Percentages READ Scale, Service Points
0% 20% 40% 60% 80% 100%
Semester 1
Semester 2
Semester 3
Semester 4
Semester 5
Semester 6
Semester 7
Semester 8
Semester 9
Semester 10
Semester 11
Semester 12
Semester 13
Semester 14
Service Points
1
2
3
4
5
6
Lynn Berard / Bella Gerlich
Full Semester Participants Data, Off-Desk
Off-Desk Semester Percentages
0% 20% 40% 60% 80% 100%
Semester Off-Desk1
Semester Off-Desk2
Semester Off-Desk3
Semester Off-Desk4
Semester Off-Desk5
Semester Off-Desk6
Semester Off-Desk7
Semester Off-Desk8
123456
Lynn Berard / Bella Gerlich
Approach Types, Service Points, All Transactions
In Person81%
Phone 9%
Email8%
Chat2%
Lynn Berard / Bella Gerlich
Approach Types, Off-Desk, All Transactions,
In Person46%
Phone18%
Email36%
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Training / Continuing Education Renewed Personal & Professional Interest Outreach Reporting / Statistics
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Training / Continuing Education
“I felt it was very useful because it challenged me to come up higher in those areas where I need improvement in certain concentrations like ____ which is not my specialty. I need to learn so much more.”
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Training / Continuing Education
New staff. Develop a training regimen with outcomes, with a similar series of questions to be given at a later date to insure that the staff is developing the necessary skills / knowledge.
Continuous learning. Writing down any questions that elicit an assignment of a category of 4 or higher at the service point, then sharing with colleagues, there by sharing strategies and learning from others.
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Renewed Personal & Professional Interest
“Using the READ Scale added to my sense of accomplishment!”
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Renewed Personal & Professional Interest Self Assessment / Reference as Activity Reference
staff can rate their effort / knowledge / skills as appropriate. Gives recognition to primary function and can be compared to other libraries / librarians using the Scale. Acknowledges the two activities most important to reference staff in terms of job satisfaction (Gerlich): helping users and detective work.
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Outreach
“It gives ME a tangible scale on which to rate my efforts, ultimately spurring me to strive for better service.”
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Outreach Recording Liaison Activity Off-desk statistics are often
not recorded, or if they are, given the same hash mark as a directional question. Using the READ Scale in these cases would show case the subject specialization knowledge / needs of the campus. In cases where off-desk statistics are low or READ Scale assignments are in the low end range, outreach activity could be re-examined, surveys taken, etc in that particular area and services redesigned as needed.
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Research / Statistics
“An assessment tool that does a better job of reflecting how reference librarians spend their time. It gives more value than tick marks on a page. It's a tool we can use with administrators to show what we really do.”
Lynn Berard / Bella Gerlich
READ ScalePractical Applications
Research / Statistics Staffing Strategies. Who staffs desk, when. Develop Narrative Statistics. Records hidden
work. Time. Estimate or actual real time statistics for
effort working with patrons. Comparisons with like institutions who use the
Scale.
Lynn Berard / Bella Gerlich
For further exploration Discussion must be part of implementation /
continued use - integrate with Scale Categories to include more clear examples that
emphasize effort / teachable moment Consider alternate format easier to read (bulleted or
rubric form) Time element more prominent and / or recorded for
sample periods More clear distinction between categories or consider
fewer categories Consider ways to identify levels of skill between staff
at service points
Lynn Berard / Bella Gerlich
Next Steps
Consider modifications based on feedback Publish articles Host @ Creative Commons Invite other Libraries to use Continue data gathering Create communication tool for users /
continued development of Scale
Lynn Berard / Bella Gerlich
Questions?
Thank you!
If you are interested in trying the READ Scale, please contact us at:
Lynn Berard, [email protected]
Bella Gerlich, [email protected]