Academic Libraries and Virtual Reference

  • View
    24

  • Download
    0

Embed Size (px)

DESCRIPTION

Academic Libraries and Virtual Reference. User Preferences & Usability of Reference Services. Rebecca Croxton, MLIS Anthony S. Chow, Ph.D. July 12, 2011. Purpose of Study. Better understand information seeking preferences of library users in an academic setting - PowerPoint PPT Presentation

Transcript

Slide 1

Academic Libraries and Virtual ReferenceUser Preferences & Usability of Reference ServicesJuly 12, 2011Rebecca Croxton, MLISAnthony S. Chow, Ph.D.Purpose of StudyBetter understand information seeking preferences of library users in an academic setting

Develop an understanding of user perceptions of usability of different virtual reference mediums offered by an academic library

Research QuestionsWhich virtual reference mediums do library reference users prefer? (Email, Telephone, Online Chat, Video Conferencing, Text Messaging)

Are there differences in user preferences for virtual reference mediums based on type of question?

Are there differences in user preferences for virtual reference mediums based on type of user and type of question?

Which virtual reference mediums do library users find most usable?

Findings in the LiteratureFindings vary across studies

Washington State University (2 campuses) (N=197)South Atlantic (2 Universities) (N=276)1st Choice When Seeking Library Assistance1st Choice When Seeking Reference Help with ResearchLibrary Web Site (42.4%)Face-to-Face (66.4%)Reference Librarian (14.5%)Email (20.2%)Friend (10.9%)Telephone (9.2%)Online Chat (10.3%)Online Chat (4.2%)Telephone (10.3%)Email (7.2%)Other (3.6%)Joel Cummings, Lara Cummings, and Linda Frederiksen, "User Preferences in Reference Services: Virtual Reference and Academic Libraries," Portal 7, no. 1 (January 2007).Corey M. Johnson, "Online Chat Reference," Reference & User Services Quarterly 43, no. 3 (Spring 2004): 241.Study MethodologyCampus-wide Survey (936 participants)

Usability Study (31 university student participants)

Focus Group of Reference Library Staff (13 participants)

Interview with Head of Reference at Academic Library

Participants Campus Wide SurveyCampus-Wide Survey - 936 respondentsSent to a randomly selected sample of university students (N=2,552 / 14.2% response rate)All permanent faculty and staff (N=3,840 / 14.3% response rate)

Instrumentation Campus Wide Survey11-item online surveyQuestions:Prior experience with and knowledge of reference services

Rank order virtual reference seeking preferences for answering a procedural based, research question and a quick, factual question

Choose one reference service for procedural, research question and quick, factual question this time including face-to-face reference as option

Key Findings Campus-Wide SurveyFifty-four percent of respondents had prior experience with at least one reference medium.

Key Findings - Campus-Wide SurveyResearch QuestionsEmail is most popular choice for research questions.

Key Findings Campus-Wide SurveyResearch QuestionsIf face-to-face is an option, survey respondents prefer face-to-face for research questions.Faculty prefer email over face-to-face reference.After face-to-face, undergraduates nearly equally prefer email and online chat.

Key Findings Campus-Wide SurveyQuick, Factual QuestionsTelephone is most popular for quick, factual questions.

Key Findings Campus-Wide SurveyQuick, Factual QuestionsWith face-to-face as an option, telephone is most preferred for factual questions. Online chat and telephone were nearly equally preferred for undergraduates.Faculty preferred email over all other choices.

Participants Usability StudyUsability Study - 31 participants12 Randomly Selected Undergraduate Students 19 Randomly Selected Graduate Students

Of these participants:15 Students Tested Reference Mediums at University A16 Students Tested Reference Mediums at University B

Universities Utilized for Study:Two mid-sized, doctoral granting state universities in the Southeast

Instrumentation Usability StudyParticipants given written instructions via email to contact a designated university library and ask a pre-assigned procedural, research based question and a quick factual question via:Email

Telephone

Online Chat

Skype Video*

Text Messaging

*University B did not offer Skype Video ReferenceInstrumentation Usability Study (Cont.)Participants completed online surveys after testing each medium and at completion of study

Rank order virtual reference seeking preferences for answering a procedural based, research question and a quick, factual question

Choose one reference service for procedural, research question and quick, factual question this time including face-to-face reference as option

Rate usability of virtual reference mediums using measures defined by ISO 9241-11:1998 of Effectiveness, Efficiency, and SatisfactionKey Findings Usability StudyResearch QuestionsOnline chat is most preferred for research questions.

Key Findings Usability StudyResearch QuestionsWith face-to-face as an option, online chat is still most preferred for research questions

Key Findings Usability StudyQuick, Factual QuestionsOnline chat is most preferred for quick, factual questions.

Key Findings Usability StudyQuick, Factual QuestionsWith face-to-face as an option, online chat is still most preferred for quick, factual questions.

Key Findings Research QuestionsCampus-Wide Survey

Email is most preferred virtual reference medium

When face-to-face reference is an option, users prefer face-to-face over all other choicesException Faculty preferred email over F2F

Usability Study (students only)

Online chat is preferred reference medium over all other choices (including face-to-face)

Key Findings Quick, Factual QuestionsCampus-Wide Survey

Telephone is most preferred virtual reference medium

When face-to-face reference is an option, users prefer telephone over all other choices

Exceptions:Faculty prefer email over face-to-face referenceStudents equally prefer telephone and online chat

Usability Study (students only)

Online chat is preferred reference medium over all other choices (including face-to-face)

Summary of PreferencesResearch Question1st ChoiceQuick, Factual Question1st ChoiceVirtual ReferenceOverall(w/ Face-to-Face)Virtual ReferenceOverall(w/ Face-to-Face)All Users CombinedEmailFace-to-FaceTelephoneTelephoneUndergraduatesOnline ChatFace-to-Face*Telephone* (followed closely by Online Chat)Online ChatGraduate StudentsEmail*Face-to-Face*Telephone* (followed closely by Online Chat)Telephone* (followed closely by Online Chat)FacultyEmailEmail (followed closely by F2F)Telephone (followed closely by Email)EmailStaffEmailFace-to-FaceTelephoneTelephone*In usability study, both graduate and undergraduate students rated Online Chat as their first choice, regardless of the availability of Face-to-Face Reference.Usability - DefinedUsability is a quality attribute relating to how easy something is to use. it refers to how quickly people can learn to use something, how efficient they are while using it, how memorable it is, how error-prone it is, and how much users like using it. If people cant or wont use a feature, it might as well not exist.

J. Nielsen & H. Loranger, Prioritizing Web Usability. (Berkeley, CA: New Riders, 2006).Usability - OperationalizedPer ISO 9241-11:1998, usability is measured using three factors:EfficiencyEffectivenessSatisfactionUsability - EffectivenessEffectiveness (Jordan, 2001)Task Completion: Were you able to successfully use [reference medium] to ask your questions and have them answered? Quality of Output: How satisfied were you with the overall quality of the [reference medium] responses and the extent to which they answered your questions?

P.W. Jordan, An Introduction to Usability.(Philadelphia, PA: Taylor & Francis, 2001).

Effectiveness Usability Ratings(1=lowest satisfaction / 10=highest satisfaction)Task CompletionQuality of OutputOverall Effectiveness Email (n=31)8.296.037.16Telephone (n=30)8.778.538.65Online Chat (n=30)9.509.709.60Skype Video (n=15)*6.336.076.20Text Messaging (n=29)6.216.006.10*Skype reference was only available at University A

Usability - EfficiencyEfficiency (Jordan, 2001)Deviations from Critical Path: How satisfied are you that you were able to use [reference medium] without any problems?Error Rate: How satisfied were you that you accomplished your goal with very few or no errors?Time-on-Task: How satisfied are you that you accomplished your goal with minimal time wasted?Mental Effort: How satisfied are you that you accomplished your goal with minimal mental effort?

Efficiency Usability Ratings(1=lowest satisfaction / 10=highest satisfaction)Deviations from Critical PathError RateTime-on-TaskMental EffortOverall Efficiency Email (n=31)8.658.777.268.458.28Telephone (n=30)9.079.078.038.538.68Online Chat (n=30)9.49.479.339.279.37Skype Video (n=15)*5.075.475.275.205.25Text Msg (n=29)6.456.526.006.486.36*Skype reference was only available at University A

Usability - SatisfactionSatisfaction (Jordan, 2001)Overall, how satisfied were you with the general usability of using [reference medium] to ask reference questions?

Usability Overall ResultsOverall, research subjects found online chat to the be most usable virtual reference medium.

Overall Usability Ratings(1=lowest satisfaction / 10=highest satisfaction)Overall EffectivenessOverall EfficiencyOverall SatisfactionOverall Usability (mean)Email (n=31)7.168.288.037.82Telephone (n=30)8.658.688.708.68Online Chat (n=30)9.609.379.609.52Skype Video (n=15)*6.205.255.675.71Text Messaging (n=29)6.106.366.386.28*Skype reference was only available at University AInstrumentation Focus Group & InterviewFocus GroupReference Library Staff (n=13) at University A7 Questions re: thoughts, preferences, and future trends surrounding virtual referenceSample Question:Based on your experience, which of the referen