Summon Use at Dartmouth College

Summon Use at Dartmouth CollegeUSERS RESPONSE

1Dartmouth College Library

Dartmouth College is located in Hanover, New Hampshire

Smallest of the Ivies, there are approximately 4100 undergraduates; 1700 graduate students

Looking at Baker-Berry LibraryEight libraries within the system

Summon use at Dartmouth:One of two beta test sites (along with Oklahoma State University)Summon went live to library staff in January, 2009Open to all users in the fall of 20092Triennial SurveySummary of resultsundergrad = 65% use Summon at least once a term (64% use Google Scholar at least once a term)

grad = 25% once a term (61% Google Scholar)

faculty = 25% once a term (74% Google Scholar)

Undergraduate students feel it is important to their research, but it is not important to graduate students or faculty.

When compared with other resources such as the Library Catalog, Library website, and the Research Guides all three groups ranked it last for meeting their research needs.

Triennial Survey:In house survey tool, replacing LibQual+ as an institution wide response to research tools

3750 students (graduate and undergraduate), faculty and college staff were invited to participate

21% undergrads responded, 24% graduate students, 18% faculty and 11% of college staff.

Overall a 20% response rate higher than weve ever had and considered solid for responses.

Google Scholar becoming a favored tool, especially among faculty and graduate students.3Importance: Comparison of Search ToolsLibrary OPACSummonResearch GuidesLibrary WebsiteCourse Web PagesRefWorksEndnote

IMPORTANCE At the heart of the Triennial Survey were pairs of questions one asking about the importance of a Library resource, collection, or service, the other asking to what extent it met the users needs.

4Criteria for responsesHow important are the following to your research?

4: Very important

3: Important

2: Somewhat important

1: Not important

0: Do not use

For ease of calculation, we rated the responses as follows:

How important are the following to your research?4: Very important3 Important2: Somewhat important1 Not important

5Undergraduates

This table illustrates the question, the answer choices, and the corresponding numbers for the importance questions. 6Graduate Students

Faculty

Meets Needs: Comparison of Search ToolsTo what extent do the following resources meet your needs?

4: Completely meet needs

3: Mostly meet needs

2: Somewhat meet needs

1: Rarely/never meet needs

0: Do not use

MEET NEEDSThe following table illustrates the question, the answer choices, and the corresponding numbers for how different resources meet the needs of the various groups of those surveyed.Note that Summon received the lowest meeting needs score for this question from all three groups (undergraduate students, graduate students, and faculty).

9Undergraduates

10Graduate Students

Faculty

12How often do you use a resource?Library catalogLibrary databasesLibrary website Research GuidesAsk Library staff

SummonGoogle ScholarCourse Web PagesCourse ReservesWeb search engines

These are the ten resources those surveyed chose from.13Undergraduates

The percentages indicate those who use that resource more than once a week.

14Graduate Students

Faculty

16

Librarians on the Street:

What are our users telling us?

In addition to the results from our Triennial Survey, I also solicited responses from my colleagues in Research & Instruction Services who are working with students and faculty on a daily basis.

17ObservationsSummonPros

Cons

The observations primarily focus on undergraduates.

Pros:

Love the results they get that start them on their wayWhen a search is successful it is a smooth transition from needing information to finding and getting information.Students tend to use Summon most effectively after a class with a librarian or after having come to the reference desk for helpOnce shown how to use the facets, many will begin to use them successfully on their own.

Others will do the best on their own from the beginning. There are many successes, but of course we dont hear about or experience those as often.

Once I have the opportunity to work with a student they tend to feel excited about discovering a new search tool.

Cons:What is it supposed to do? Heard from faculty and students. Students will just try something, not always with success, but you often dont hear of their failure.Students may have used this on their own, but only clicked on results.Use of facets is not clear to our users, across undergrads, graduate students and faculty; will end up with a large search result with no real idea how to limit down to a successful set of records.Want immediate results, and the failures happen frequently.We see the Summon "failures at the desk, i.e., students aren't happy with their experience and results OR Summon says it will link to full text and it doesn't One of the greatest failures is students seeing a full text tab but having no success with getting the full text.

Another colleague observed: Full text linking failures, especially to ProQuest content. is the biggest problem. Many people have a difficult time interpreting and using the helper banner to go to the link resolver and find additional full text options. They expect Google Perfect Linking. They expect the link from Summon to take them directly to the full text with no intermediate pages and no additional effort.Few people understand how the Subject facets work. When choosing subjects from the main interface, it ANDs the terms, narrowing results. When choosing subjects from the "more options" pop-up, it performs a boolean OR between the terms. This is a subtle difference that 99% of users don't understand.The Scholarly Publications limit applies to the journal in which the article appears, not the article itself, and all of this is based on Ulrich's. There are a large number of non-scholarly articles that appear in results sets that have been limited to "scholarly" sources because the criteria used by Ulrich's includes many publications that do not really meet this criteria (e.g. Junior Scholastic). It also includes articles in journals such as Nature that are news pieces written by staff writers, not articles that were submitted through the peer review process. And while this may be a reason why Summon uses "scholarly" instead of "peer reviewed," this distinction is lost on most students who let the search engine do the thinking for them when judging whether an article is scholarly or not.

Yet another colleague: The main thing that people just do not seem to see is the limit in the upper-left corner that says "Limit to articles from scholarly publications, including peer-review." I ask in classes for people's thoughts and experiences with Summon, and invariably the majority answer is "there's too much irrelevant stuff." Then I run a search and show them that limit, and it's like the sun has come out from behind the clouds -- "Ohhh! Now I get it!"

18Current Library Search Box

A year ago we made a change to our Library home page search box

We made Summon the default search (previously it was the Library Catalog)Since this change, there has been a large outcry from many of our users asking us to make the Library Catalog the default search again.Reasons noted in their comments:The catalog is more preciseToo many irrelevant results in SummonLack of confidence in finding the actual full text no matter what the record says

Will soon be making a few more changes to our website where a search in the box will bring up the top five results from Summon, the Library Catalog and the Librarys website, with a link for each to further results. We are hoping this will make visually evident the choices one have and the strengths of each choice.

19New Directions for Searching

We are currently in the process of redesigning our library home page to try and incorporate our searches to alleviate this specific issue.

Search box will be a general box, similar to what one comes up in many other sites (including Google, of course).

In this case, a search for World War II has been put in.20Results from three sources:

Click on Search and the results from three of our primary jumping off points appear.21To Sum Up:

To Sum Up:

There is no doubt Summon has become an important tool in our arsenal of resources.Provides results in a variety of formats, sources, etc.Often provides sources to beginning researchers that would have taken much longer to find otherwiseWhen the stars are aligned, and everything works as it should, its a miracle and users go happily on their way

Users want ease of use, they want tools to be easy to figure out and they want them to workFacets are wonderful in concept and librarians and power users get them (and sometimes want more!)Most users are overwhelmed by the list, dont know how to use it or, with many students, never even see it.When a new tool fails, it is hard to regain enthusiasm. The request for full text fails over and over. The reason for the hue and cry when we changed the primary search to Summon was due to this lack of confidence in the results and the ability to obtain them easily. When compared with other resources such as the Library Catalog, Library website, and the Research Guides all three groups ranked it last for meeting their research needs.Confusion was the term most used in comments22To Sum Up:

There is no doubt Summon has become an important tool in our arsenal of resources.Provides results in a variety of formats, sources, etc.Often provides sources to beginning researchers that would have taken much longer to find otherwiseWhen the stars are aligned, and everything works as it should, its a miracle and users go happily on their wayUsers want ease of use, they want tools to be easy to figure out and they want them to workFacets are wonderful in concept and librarians and power users get them (and sometimes want more!)Most users are overwhelmed by the list, dont know how to use it or, with many students, never even see it.When a new tool fails, it is hard to regain enthusiasm. The request for full text fails over and over. The reason for the hue and cry when we changed the primary search to Summon was due to this lack of confidence in the results and the ability to obtain them easily. When compared with other resources such as the Library Catalog, Library website, and the Research Guides all three groups ranked it last for meeting their research needs.Confusion was the term most used in commentsQuestions?

Ridie Wilson Ghezziridie.w.ghezzi@dartmouth.edu