1

Assessing for Improvement

Diane HarveyJune 16, 2011

2

Learning outcomes for today

At the end of this session, participants will be able to:

1. Describe the learning outcomes assessment process

in libraries.

2. Write learning outcomes for an information literacy instruction session.

3. Choose appropriate assessment methods.

4. Define ways to work with assessment results.

Why assess student learning in libraries?3

Affirm commitment to student learning

Improve effectiveness of instructional programs

Align with campus, disciplinary, and higher ed efforts

Be recognized on campus for contributions to student learning

What’s happening on campus?

SACS re-accreditation: required departments and programs to create and assess learning outcomes (SLOs/objectives/goals)

Undergraduate programs have done this; graduate programs are in the process

Every department and program website must post learning outcomes

4

Example: Classical Studies

Majors develop:

Comprehensive knowledge of disciplinary fundamentals through study of Greek and Latin (CLLA), courses in both Greek and Roman history, and a variety of courses in literature, art and archaeology, history, philosophy (CLCIV), with ample scope in both domains for developing advanced-level proficiencies through more concentrated study. 

Proficiency in writing in the discipline of Classical Studies, enjoying opportunities to write research and analytical papers in in both CLLA and CLCIV courses. In addition, all majors take the Capstone Seminar. 

Research experience, through the Capstone seminar; majors are also encouraged to write a Senior Thesis. 

Expertise outside the classroom, including study abroad, excavation experience, summer abroad programs, and other varieties of hands-on experience, including other experiential learning opportunities.

5

Assessing for improvement6

We don’t assess to prove…

…but to improve.D. Stufflebeam

What will we do in the library?

Summer 2011: workshops on LOA and shared learning outcomes for WR20 library instruction.

Fall 2011: begin assessing student learning in WR20 and subject specific instruction.

Spring 2012: continue assessing, gather feedback from librarians

7

What is LOA: learning outcomes assessment?8

Systematic look at what students are learning

Moving from “What am I going to teach today?” to “What do I want students to learn today?”

What LOA is NOT9

NOT evaluation of teaching

NOT evaluation of program

10

Assessment loop

Develop student learning

outcomes

Set criteria

Devise assessment measures

Do assessment

Work with results

11

Levels of assessment

Institutional (University wide)

Program (e.g. Department)

Course

Session (e.g. IL instruction)

LOA in libraries: constraints12

Lack of consistent, reliable access to students

IL instruction: is it a “program”?

Program assessment vs. session assessment

Difficult to isolate effects of library instruction

Where will you assess student learning?13

On the instruction session level?

In a for-credit library skills course?

Across a series of library instruction sessions?

Broadly across the student population?

Some simple approaches14

Develop learning outcomes for your instruction program, and look at where they are taught – “snapshots”

Focus on large programmatic efforts e.g. first year writing

Highlight specific academic departments

Assess individual library instruction sessions – develop shared learning outcomes, compare/contrast

15

What is a learning outcome?

A learning outcome is one sentence that indicates what students should represent, demonstrate or produce as a result of what they learn.

- source: Peggy Maki

16

Good learning outcomes:

Focus on what students will learn/know/be able to do

Describe actions or behaviorsAre results orientedAre observable and measurableInclude a time frame

17

ACRL Standards

3 levels: standards, performance indicators, outcomes

Can be used for LOA

Examples: (1)Identifies keywords, synonyms and related terms for the information needed. (2) Selects an appropriate documentation style and uses it consistently to cite sources

Writing outcomes18

Today, we’ll write learning outcomes for a library instruction session.

19

Exercise: What do you cover?

Think of an information literacy instruction session that you teach on a regular basis.

List what you would cover in that session.

20

Bloom’s Taxonomy

Classification of educational objectivesPublished in 1956, revised in 2001 (changes: noun to

verb, synthesis/creating becomes highest level)

Taxonomy = classificationCognitive levels (lower higher)Provides way to express outcomes

21

Bloom’s levels (lowest highest)

Knowledge/RememberingComprehension/UnderstandingApplication/ApplyingAnalysis/Analyzing Evaluation/EvaluatingSynthesis/Creating

22

Verbs for information literacy

Knowledge/Remembering: define, list, recognize Comprehension/Understanding:characterize, describe,

explain, identify, locate, recognize, sort Application/Applying: choose, demonstrate, implement,

perform Analysis/Analyzing: analyze, categorize, compare,

differentiate Evaluation/Evaluating: assess, critique, evaluate, rank, rate Synthesis/Creating: construct, design, formulate, organize,

synthesize

Verbs to avoid23

Understand

Appreciate

Know about

Become familiar with

Learn about

Become aware of

24

Learning outcomes formula

1. Time frame2. Student focus3. Action verb4. Product/process/outcome

25

Using the formula

Time frame: “At the end of the library session…”

Student focus: “…students will be able to…”Action verb: “…identify…”Product/process: “…a relevant database for

their term paper research.”

26

Another example:

Time frame: “After completing the online tutorial…”

Student focus: “…students will be able to…”Action verb: “…differentiate between…”Product/process: “…scholarly journals and

popular magazines.”

27

One more example:

Time frame: “By the time they complete their undergraduate program…”

Student focus: “…students will be able to…”Action verb: “…construct…”Product/process: “…a research question that

can be investigated using primary archival resources.”

28

Examples: bad and better

Bad: Students will understand how to use social science databases.

Better: Students will perform a search in Sociological Abstracts that retrieves relevant items.

Bad: Students will be able to search the catalog.Better: Students will use the catalog to identify a book on their

topic.

Bad: Students will appreciate the importance of correct citation.Better: Students will produce citations in correct APA format.

29

Now, write your own!

Choose two items from the list of what you cover in an information literacy session.

Transform them into learning outcomes, using the formula.

Find a partner and critique your outcomes using the following checklist.

30

Checklist for learning outcomes

Includes a time frame?Focuses on students?Uses action verbs?Names a product or process?Is measurable/observable?Prompts a measure/method?Will be useful for you to assess?

Setting Targets

Ask yourself

“What is success?”

31

Assessment Methods

Ask yourself

“How will I know?”

32

33

Assessment methods

Knowledge testOne Minute paper & variationsBibliography analysisConcept InventoryStandardized test

Knowledge tests34

Tests knowledge and/or skills before and/or after library instruction session.

Can be given at end of library session or later in semester.

Can use clickers to gather data during library session.

Sample questions:1. What is the difference between a library catalog

and a database?2. The Boolean operator “or” narrows a search statement (true/false).

One minute paper & variations35

Sample questions:

1. What is the most important thing about library research you learned today?

2. 3-2-1 (three things you learned, two things you’re still confused about, one thing you’d change about session)

3. What is one question you still have?

4. In your research, what will you do differently after today’s session?

Bibliography analysis36

Look for citations from scholarly/peer reviewed journals

Look for citations for books and journals owned by your library

Look for articles retrieved from your databases.

Concept inventory37

Make a checklist of 3-12 important concepts students need to master.

Ask students to explain each concept in a sentence or two. If a concept is unfamiliar, they should leave blank.

Count good responses for each concept, then plan future instruction.

Standardized tests38

I-Skills

ILAT (James Madison University)

SAILS

39

Using assessment information

Rewrite learning outcomesChange what you do/how you teachWork collaboratively with colleagues

(librarians and faculty)Revise assessment measuresShare the news in the library and on campus

40

Learning outcomes are:

Fundamental components of a learning assessment program

Focused on student learningIndicate what students will know/be able to doMeasurable, observable, overt

Learning Outcomes Assessment 41

A tool to help librarians

improve student learning

42

LOA resources

Angelo, Thomas (1993). Classroom assessment techniques. Maki, Peggy (2004). Assessing for learning : building a

sustainable commitment across the institution. Middle States Commission on Higher Education(2003).

Developing research & communication skills: guidelines for information literacy in the curriculum http://msche.org/publications/devskill050208135642.pdf

Neely, Teresa Y(2006). Information literacy assessment : standards-based tools and assignments.

Radcliff, Carolyn et.al (2007). A practical guide to information literacy assessment for academic librarians.

Student Learning Outcomes Assessment (University of Virginia)http://www.web.virginia.edu/iaas/assessment/outcomes.htm

43

Let’s assess this workshop!

44

Thank you!

Diane Harvey