Collection Analysis and Evaluation: Fundamentals of Collection-Centered Assessment Techniques

2nd Marina G. Dayrit Lecture Series 2016

Collection Analysis and Evaluation: Fundamentals of Collection-Centered Assessment Techniques

March 11, 2016 Ateneo de Manila UniversityLoyola Heights, Quezon City

Marilou N. Andres

Philippine Association of Academic/Research Librarians, Inc.(PAARL)

Outline Rationale Different collection-centered assessment

techniques Framework of identifying useful data for collection assessment Preparation of collection assessment studies/reports

05/01/23

The Collection Development Process

05/01/23

intended curriculum

implementedcurriculum

attained curriculum

G A P

managementof research

& instructional

materials

COLLECTION ANALYSIS

techniques in

evaluation

statusof

collection

05/01/23

Assessm

ent

• examines or describes collections either in their own terms or relative to other collections and checklists.

Evaluation

• determines how well the collection supports the goals, needs, and curriculum of the parent organization.

COLLECTION ANALYSIS

(Johnson ,2009)

05/01/23

Collection Assessment1. Analyzing what you have

a. How much?b. How old?c. How does it compare to others?

2. Analyzing how it is useda. Circulation/in-library useb. Comparing usec. Electronic counting-website, databases

(Kachel, D. E.)

05/01/23

“Culture of Evaluation”

Way to demonstrate Relevance Value Impact

Considered from the view of: Users Stakeholders

Lakos, A. & Phipps, S. (2004)

05/01/23

A mechanism to determine:

If the collection is meeting its objectives How well it is serving its users In which ways or areas it is deficient, and

what remains to done to develop the collection

If selectors are performing their responsibilities effectively

How to allocate collections/access funds

(Johnson ,2009)

05/01/23

Who is the audience? Accreditation agencies Parent organization (administration,

board, senior management) Library administration Selector User community or communities Consortial partners

(Johnson ,2009)

05/01/23

Collection-based Measures Look at:

Size Growth Coverage (depth, breadth, balance)

(Johnson ,2009)

05/01/23

Misconception A “balanced” collection means having

resources on every topic.Does NOT Mean:

• You purchase a few resources on every topic known to mankind

•Replacing each weeded title with something on the same topic (1 to 1 replacement)

•Purchasing resources “just in case” there might be a need

Does mean:

•Having a balance of opinions on controversial topics studied in the curriculum

•Representing diversity in the topics collected- includes “ethnic and racial backgrounds, age, physical and cognitive abilities, family status, sexual orientation, socioeconomic status, religious and spiritual values, and geographic location.”

(Kachel, D. E.)

05/01/23

Collection-based Measures

Checking lists Evaluating the collection directly Comparative statistics Application of collection standards

05/01/23

Research MethodsQualitative Research

- A process of inquiry that draws data from the context in which events occurs . . . using induction to derive possible explanations based on observed phenomena. Gorman and Clayton (2005)

Quantitative Research -Involves the collection and analysis of data that is quantifiable.

05/01/23

DifferencesQualitative Quantitative

Strengths Weaknesses

Count things (Titles, Circulation transactions, Expenditures, E-metrics, ILL transactions, Ratios

Provides context Offers a way to understand the attitudes that inform the statistics

Online or printed surveysInterviews (structured or unstructured) ObservationFocus groups

(Johnson ,2009)

05/01/23

Commonalities

Qualitative (non-numerical)

Quantitative (numerical data)

• Logic of triangulation• Qualitative research

facilitates quantitative research

• Quantitative research facilitates qualitative research

• Provide a general picture• Researchers' and

subjects' perspectives Hughes, C.

05/01/23

Different collection-centered assessment techniquesMethod Technique

Use and user Centered Collection-Centered

Quantitative Interlibrary loan statistics Collection size and growthCirculation statistics Materials budget and growthIn-house use statistics Collection size standards and formula

Qualitative User opinion survey List checkingFocus groups Citation analysis

Direct collection checking Commercial ProductsCollection mapping

(assigning conspectus levels)

Evans, (2012), Gregory, (2011), Johnson (2009)

STANDARDS FOR LIBRARY COLLECTION IN ACADEMIC LIBRARIESPAASCU

RequirementsPACU-COA

RequirementsAACCUP

RequirementPAARL

RequirementsCHED

Requirements

Basic book collection of 6,000 well selected titles with an enrollmentof 500 or less to supportits educational programs

Core book collection of 5,000 titles for a college and 10,000 for a university● A quality core book collection of 10,000 titles is maintained*

Core collection of at least 5,000 titles for a college and 10,000 for a university

Core book collection of 5,000 well selected titles for college libraries and 10,000 for university libraries

Minimum of 2 selected titles should be provided for every student

Minimum of 5 professional titles per student*;

3-5 book/journal titles for each professional subjects

Minimum of 5 professional titles per student shall be provided

Book Ratio 2- 5 titles per professional course

For reserve books, 1 copy for every 30 students

For reserve books 1 copy per 25 students For reserve books 1 copy per 25 students

The provision of textbooks is not theresponsibility of the library but a maximum of 5 copies of frequently used materials shall be provided● For reserve books, 1 copy per 25 students

Periodicals (local and foreign)50 titles – 1,000 enrollees75 titles – 1,001-3,000 enrollees100 titles – 3,000 and more● Every major field should be covered adequately by relevant and appropriate journals and periodicals

Periodicals (local and foreign)50 titles – 1,000 enrollees75 titles – 1,001-3,000 enrollees100 titles – 3,000 and more● For every major field-6 titles of undergraduate concentration in the major field-6 titles for graduate course-10 titles for doctoral course● Core periodical/journal collection of at least 100 titles are provided*

Periodicals (local and foreign)50 titles – 1,000 enrollees75 titles – 1,001-3,000 Enrollees 100 titles – 3,000 and more● Every major field should be covered by at least 3 titles of undergraduate concentration of majorsubject field, 6 titles of graduate concentration and 10 titles of doctoral work or equivalent

3-4 professional journals

Obille, K.L.B. (2007). An Evaluation of standards for academic libraries in the Philippines. Journal of Philippine Librarianship, 27(1&2), 109-150AACCUP Survey Instrument

COLLECTION MAPPING

• EXISTING COLLECTION

Input

• COLLECTION MAPPING

Process

• COLLECTION DEVELOPMENT POLICY

Output

Elements in common

1. Subject based rather than descriptive of total.2. Number of items.3. Type or formats represented.4. Age of items – mean, mode, medium.5. Level, scholarship or audience for items.6. Means of identifying appropriateness.7. Necessity to view the collection –shelf scanning viewed as essential.

(Bushing, 2005)

Appropriateness determined by . . .

1. Comparison with other similar or ideal library collections.2. Comparison with standard lists, bibliographies.3. Relation to mission of the library.4. Use statistics where applicable.

(Bushing, 2005)

3 Basic Assumptions About Collections that Relate to Collection Mapping

The collection should have breadth

It should have depth based on the needs of your students and teachers

The collection should be well understood in order to be effectively developed and used.

(Lamb and Johnson, 2014)

Collection maps are used for many collection-related projects

showing strengths and weaknesses in the collection evaluating whether the strengths of the collection match

the curriculum focus tailoring purchasing decisions planning for future directions suggesting areas of weeding demonstrating areas of need and areas of excellence developing short and long term goals building a budget plan to match curriculum goals creating selection procedures that material collection

goals(Lamb and Johnson, 2014)

What is a base collection and core collection?

BASE• number of works

recommended for a minimal level collection

CORE• associated with specific

titles

(Lamb and Johnson, 2014)

Procedure for Collection Mapping

Divide the total size of the collection by the number of students in the school

Divide the total size of each emphasis collection by the number of students and chart the resulting

items/student on a collection map

Decide which specific emphasis collection

Decide the general emphasis collections

http://libraryadmin.wikispaces.com/Collection+Mapping

Conspectus: a step in the evolution

• strengths of the collection

• acquisitions commitment

Input

• assessment

process• desired collecting

intensity

output

How are results analyzed?

• examine to determine if the collection is evenly distributed

• consider limiting selection• develop promotional strategies

strength

• Select from retrospective selection bibliographies

• Watch for new itemsweakness

• Circulation statisticsCOMPARE

http://libraryadmin.wikispaces.com/Collection+Mapping

Benefits of Conspectus a standard means of describing current collection

strengths a standard means of describing future collection

intentions assistance with collection rationalization by informing

librarians who else is collecting in a particular subject area assistance with resource sharing by listing collection

strengths for user who may then visit other libraries, and for library staff, as a guide to interlibrary loans

assistance with the prioritization of materials for preservation activity, and recording of these priorities.(Clayton ,2001)

05/01/23

Collecting LevelsCollection depth indicators

Information Level Definition

Collecting Level Conspectus

A 5 Comprehensive A collection in a specifically defined field of knowledge that strives to be exhaustive, as far as isreasonably possible (i.e., a "special collection"), in all applicable languages.

B 4 Research A collection that contains the major published source materials required for doctoral study andindependent research

C 3 Study

Collections that provide information about a subject in a systematic way, but at a level of less than research intensity, and support the needs of general library users through college and beginning graduate instruction

05/01/23

Collection depth indicatorsInformation Level Definition

Collecting Level Conspectus

D 2 Basic

Collections that serve to introduce and define a subject, to indicate the varieties of information available elsewhere, and to support the needs of general library users through the first two years of college instruction

E 1 Minimal Collections that support minimal inquiries about this subject

O Out of Scope Library does not intentionally collect materials in any format for this subject.

http://www.ifla.org/files/assets/acquisition-collection-development/publications/gcdp-en.pdf


SAMPLE OF CONSPECTUS WORKSHEET FOR BUSINESS AND ECONOMICS

LC CLASS(Dewey Dec.)

Subject Group Collecting Levels and Language Coverage

Comments

ECS CCI DCI

HD4801-8942(331)

HFI-4050(380-382)

HF5001-6351(650-659)

HG(332)

Labor

Commerce

Business

Finance

2

2

2

1

2

2

2

2

2

3

2

2

Good GDoc Collection; Meets most needs

Ave. Copyright 1979;Good GDoc CollectionLow circ., low demand

47% Coverage/BPI30% Harvard Core List;High use/demand

Ave. Copyright 1981;Low use/demand

(Wood and Strauch)

05/01/23

Conspectus experience . . .Like collection policies, the process is as valuable as the end product.

– Discussing the appropriate data & gathering it.– Sorting out the differences in information cultures

from discipline to discipline, from subject to subject.– Deciding upon appropriate segments or subjects for

mapping.– Identifying consistent vocabulary, codes, & notes.

The process is adaptable for every size & type of collection.

(Bushing, 2005)

End results & benefits of mapping

Extensive knowledge of collection details & character. Clarity of purpose. Identification of “problems” to be resolved. Evidence that can be communicated to others. Documentation for policy decisions. Staff with in-depth knowledge of discipline specific information

cultures. Confidence in making day-to-day decisions regarding information

resources.

(Bushing, 2005)

05/01/23

Evaluating Electronic Resources

Data elements

No. of sessions

No. of Queries

No. of Menu selections No. of full-

content units

examined

No. of turn-aways, peak simultaneou

s users

05/01/23

Where to start? Define the question or problem Determine metrics to use Decide:

Where to locate the information Who will collect the information Who will analyze and report the information Who will act on the information

Framework of identifying useful data for collection assessment

Problem Technique Data needed1. Profile of the collection

List checking Collection profiling (Inventory)

2. Compare collection profile and bibliographies

Bibliographies by recognized subject experts, catalogs of libraries with strong collections in the area, or standard lists in a discipline.

Customized list may be prepared by compiling references from a number of authoritative sourcesCollection profiling (Inventory)

05/01/23

List Checking

05/01/23

Framework of identifying useful data....Problem Technique Data needed

1. Profile of the collection

Citation analysis: books

Collection profiling

2. Compare collection profile and bibliographies

List of the bibliographic references that have been cited by researchersList of references to monographs representing the materials that should be available in a research-level toxicology collectionCirculation dataCollection profiling (Inventory)

05/01/23

COLLECTION EVALUATION AND DEVELOPMENT USING CITATION ANALYSIS TECHNIQUES

Elizabeth Fuseler- McDowellColorado State University Libraries

Fort Collins, Colorado

Objectives:1. Gather data for better collection management decisions;2. Assess how well collection meets user needs;3. Act as a check on the implementation of collection

development policies;4. Review performance of current collection management

policies;5. Reduce the inherent subjectivity of the selection/deselection

process.

05/01/23

05/01/23

05/01/23



Citation analysis: journals

Cost-effectiveness factor

a. Subscription costs

b. Circulation data compiled for a

specified time period.

c. List of serials, ranked by annual

cost

2. Compare cost-effectiveness and impact factor

Impact factor (impact factor, along with other data, can be found in ISI's Journal Citation Reports (JCR). List of serials

05/01/23

05/01/23

Framework of identifying useful data....

Problem Technique Data needed

Profile of the collection

Direct collection checking

Collection profilingShelf-list title countSubject heading on the local catalogCirculation recordsDetailed information about imprints-age, language of publication, percentage of duplication, and subject coveragePercentage of holdings by call number, by copyright date

05/01/23



Commercial Products

Collection profiling (Inventory)

2. Compare collection profile and analysis tool

Collection analysis tools

-WorldCat Collection

-Bowker’s Book Analysis System

- Ulrich’s Serial Analysis System

-Follet Library Resources’ Title Wise

- Sagebrush BenchMARC

- Spectra Dimension (Comparative Study) Comparative Statistics and Collection Profile Gap Analysis

05/01/23

Journals: Ulrich's Serials Analysis System

Wiersma, Gabrielle (2010). Collection Assessment in Response to Changing Curricula: An Analysis of the Biotechnology Resources at the University of Colorado at Boulder. Retrieve from http://www.istl.org/10-spring/refereed1.html

05/01/23

Journals: Ulrich's Serials Analysis System


05/01/23

Collection Profile using Spectra Dimension


05/01/23

Collection Profile using Spectra Dimension


05/01/23

Gap Analysis in Spectra Dimension

05/01/23



Collection mapping


a. Number of items in the collection treating the general topic

b. number of items in the collection treating the specific topic.

2. Compare collection profile and analysis tool

Student population

05/01/23



Collection mapping


a. Number of items in the collection treating the general topic

b. number of items in the collection treating the specific topic.

2. Collecting Level (Conspectus)

Strengths of the collectionAcquisition Commitment

3. Total size of the collection by the number of students in the school

Results of assessment (Problem No. 2)Student population

Preparation of Collection Assessment Studies/Reports

Set purpose and objectives

Review previous research

Select data to be collected and methodology

Select population sample

Carry out a pilot study

Collect and analyze the data

Facilitate replication Clayton (2001)

05/01/23

References: Clayton, Peter. Managing Information Resources in Libraries: Collection Management in Theory and Practice. London:

Library Association Publishing, 2001.

Evans, G. Edwards. Collection Management Basic. 6th ed. California: Libraries Unlimited, 2012.

Gregory, Vicki L. Collection Development and Management for 21st Century Library Collections: An Introduction . New York: Neal-Schuman Publishers, Inc., 2011.

Gorman and Clayton (2005). Qualitative Research for the Information Professional: A Practical Handbook, 2 nd ed. London: Facet.

Hoffman, Frank and Wood, Richard. Library Collection Development Policies: Academic, Public and Special Libraries. Maryland: The Scarecow Press, Inc. 2005.

Hughes, Christina. Qualitative and Quantitative Approaches to Social Research. Retrieve from www2.warwick.ac.uk/.../researchprocess/quantitative_and_qualitative_ap

IFLA. Guidelines for a Collection Development Policy Using the Conspectus Model http://www.ifla.org/files/assets/acquisition-collection-development/publications/gcdp-en.pdf

Johnson, Peggy. Fundamentals of collection development and management. 2nd ed. Chicago: American Library Association, 2009.

Johnson, Peggy. Collection Analysis: Overview. Retrieve from www.ala.org/alcts/sites/ala.org.alcts/files/content/events/.../Johnson.ppt

Kachel, Debra E. Collection Evaluation: Knowing What You Have and What You Need, 2011. Retrieve from https://www.webjunction.org/content/dam/.../ParticipantsDirections.pdf

Lakos A., Phipps S. Creating a culture of assessment: A catalyst for organizational change. Portal: Libraries & the Academy , 2004

Wood, Richard J. and Strauch, Katrina. Collection Assessment: A Look at the RLG Conspectus. New York: Haworth Press, 1992.

IFLA. Guidelines for a Collection Development Policy Using the Conspectus Model http://www.ifla.org/files/assets/acquisition-collection-development/publications/gcdp-en.pdf



05/01/23

Thank you very much

Education

Collection Analysis and Evaluation: Fundamentals of Collection-Centered Assessment Techniques