Information Literacy: Information Literacy: Assessing Your Assessing Your

Instruction Instruction Texas Library AssociationTexas Library AssociationDistrict 10 Fall WorkshopDistrict 10 Fall Workshop

October 15, 2005October 15, 2005

Michelle Millet Michelle Millet

Information Literacy Coordinator Information Literacy Coordinator

Trinity University, San Antonio, TXTrinity University, San Antonio, TX

OverviewOverview

History of Assessment and Information History of Assessment and Information LiteracyLiteracy

Best PracticesBest Practices Assessment at TrinityAssessment at Trinity Programmatic assessmentProgrammatic assessment Other opportunitiesOther opportunities

Assessment of IL: a brief historyAssessment of IL: a brief history

1. Higher education assessment 1. Higher education assessment movementmovement

2. Strategic planning and TQM adoption2. Strategic planning and TQM adoption

3. Evolution of information literacy3. Evolution of information literacy

Activity 1Activity 1

What does assessment mean to you? What does assessment mean to you?

Best practices documentsBest practices documents ACRL Best Practices Initiative: Characteristics of ACRL Best Practices Initiative: Characteristics of

Programs of Information Literacy that Illustrate Best Programs of Information Literacy that Illustrate Best Practices, 2003Practices, 2003

Standards for Libraries in Higher EducationStandards for Libraries in Higher Education

9 Principles of Good Practice for 9 Principles of Good Practice for

Assessing Student LearningAssessing Student Learning From American Association of Higher EducationFrom American Association of Higher Education

Overarching principles to guide assessmentOverarching principles to guide assessment

6.6. Assessment fosters wider improvement Assessment fosters wider improvement when representatives from across the when representatives from across the educational community are involved.educational community are involved.

8. Assessment is most likely to lead to 8. Assessment is most likely to lead to improvement when it is part of a larger set of improvement when it is part of a larger set of conditions that promote change.conditions that promote change.

What to assess?What to assess?

Student learning outcomesStudent learning outcomes

Refers to the measurement of what a student Refers to the measurement of what a student actually learned in the course. actually learned in the course.

Pre-tests and post-testsPre-tests and post-tests

Annotated bibliographiesAnnotated bibliographies

Classroom assessment techniques (Angelo Classroom assessment techniques (Angelo and Cross)and Cross)

Assignments!Assignments!

What to assess? continuedWhat to assess? continued

Student perceptionsStudent perceptions

MMeasures the perceptions that students have easures the perceptions that students have about the course content, the instructor’s about the course content, the instructor’s knowledge and teaching ability, and what the knowledge and teaching ability, and what the students feel they learned from the course. students feel they learned from the course.

Student evaluation formsStudent evaluation forms

SurveysSurveys

Focus groupsFocus groups

What to assess? continuedWhat to assess? continued

Progress towards goals and objectivesProgress towards goals and objectivesRefers to what some know as ‘inputs and Refers to what some know as ‘inputs and

outputs.’outputs.’

Statistics on number/duration of courses, Statistics on number/duration of courses, instruction sessionsinstruction sessions

Number of hits to a web site Number of hits to a web site Student follow-up appointments, drop in visits, Student follow-up appointments, drop in visits,

emailsemails

Small steps can take you farSmall steps can take you far

Consult your library’s mission/values/goals Consult your library’s mission/values/goals documents. documents. Is IL articulated in library documents?Is IL articulated in library documents?

Do you already keep basic statistics on students Do you already keep basic statistics on students reached? Time spent preparing classes?reached? Time spent preparing classes? This can be valuable evidence.This can be valuable evidence.

What kind of in-class activities do you already What kind of in-class activities do you already do?do? Collecting student work can be assessment.Collecting student work can be assessment.

Activity 2.Activity 2.

Name one “small step” that you would like Name one “small step” that you would like to take next.to take next.

Research chart: measuring student learningResearch chart: measuring student learning

Student evaluation forms:Student evaluation forms:measuring student perceptionmeasuring student perception

Question Category

Strong Disagree Disagree Neutral Agree

Strong Agree TOTAL Mean score

cat 1 Respect forstudents 11 8 24 106 512 661 4.66

cat 2 Facilitatedlearning 14 5 34 113 540 706 4.64

cat 3 Appropriate to Assignment/class 13 5 26 75 560 679 4.71

cat 4 Affective/attitudinalchanges 12 11 65 167 402 657 4.42

Assessment or Evaluation?Assessment or Evaluation?

What do we mean by assessment What do we mean by assessment anyway?anyway? Assessment focuses on the end resultAssessment focuses on the end result Evaluation focuses on the overall processEvaluation focuses on the overall process

Students

Instruction/Pedagogy

Instruction Program

Learning Outcomes

Assessment

Student-centered LearningStudent-centered Learning

Assessment and CollaborationAssessment and Collaboration

Teaching faculty want to know…Teaching faculty want to know… Assessment in the classroom Assessment in the classroom

Use it as outreach to the teaching facultyUse it as outreach to the teaching faculty Share results and use as dialog for Share results and use as dialog for

assignment designassignment design Build an integrated information literacy Build an integrated information literacy

program within a distinct disciplineprogram within a distinct discipline Build tailored first-year experience programsBuild tailored first-year experience programs

Activity 3.Activity 3.

Who are your allies? Name two Who are your allies? Name two prospective collaborators. prospective collaborators.

Trinity University and ILTrinity University and IL

Information Literacy is relatively newInformation Literacy is relatively new Challenges to changeChallenges to change 2400 FTE, primarily liberal arts campus2400 FTE, primarily liberal arts campus

Trinity’s AnswerTrinity’s Answer

Build an IL program and assess along the Build an IL program and assess along the wayway What is an IL What is an IL programprogram?? Assessment was internally and externally Assessment was internally and externally

focusedfocused

The end result is a program, fully The end result is a program, fully integrated into the curriculumintegrated into the curriculum

Where do I begin?Where do I begin?

How do you know what you are How do you know what you are doing/building is effective? doing/building is effective?

What is most important to administrators?What is most important to administrators? How can you help those who may be How can you help those who may be

working against you?working against you? Example: FYS and Lunches at TrinityExample: FYS and Lunches at Trinity

First StepFirst Step

Where are you now? Where are you now? State of Information Literacy and Instruction State of Information Literacy and Instruction

What do you want to do next?What do you want to do next? Short term and long term goals. Know the Short term and long term goals. Know the

difference. difference.

Go for the most impact!Go for the most impact!

Activity 4.Activity 4.

What is your goal? What do you want to What is your goal? What do you want to achieve next?achieve next?

Low Effort/High ImpactLow Effort/High Impact

Creating programmatic assessment takes Creating programmatic assessment takes time and worktime and work Begin with low effort/high impact itemsBegin with low effort/high impact items

• Start tracking hours spent preparing for classStart tracking hours spent preparing for class• Start tracking NEW classes coming in for Start tracking NEW classes coming in for

instructioninstruction

Best Practices as a Guideline for Best Practices as a Guideline for AssessmentAssessment

Great tool to use as a guidelineGreat tool to use as a guideline Collaboration/Integration Goal of DocumentCollaboration/Integration Goal of Document Check off what you’ve doneCheck off what you’ve done Check what you can do now (short term)Check what you can do now (short term) Which characteristics will be your guide for Which characteristics will be your guide for

the future? the future?

Activity 5.Activity 5.

Is your “small step” LE/HI?Is your “small step” LE/HI?

If not, do you have an alternative idea?If not, do you have an alternative idea?

Creating a PlanCreating a Plan

From your long term and short term goalsFrom your long term and short term goals Internal focusInternal focus

• (teaching, hours, # of classes)(teaching, hours, # of classes) External focusExternal focus

• (who are the active faculty, creating outcomes and (who are the active faculty, creating outcomes and standards, long-term studies)standards, long-term studies)

Focus on pieces: It WILL come together!Focus on pieces: It WILL come together!

Activity 6.Activity 6.

One short term goal:One short term goal:

One long term goal:One long term goal:

Outcomes for your PlanOutcomes for your Plan

Example: Assessing Program Example: Assessing Program Effectiveness (External)Effectiveness (External) Outcome: Encourage broad faculty Outcome: Encourage broad faculty

participation in instruction in order to ensure participation in instruction in order to ensure information literacy reaches across the information literacy reaches across the curriculum.curriculum.

Assessment: Collect data concerning overall Assessment: Collect data concerning overall number of classes and departments using number of classes and departments using instruction, looking for increases in numbers instruction, looking for increases in numbers of classes or departmental participation.of classes or departmental participation.

Outcomes ContinuedOutcomes Continued

Assessing Program Effectiveness Assessing Program Effectiveness (Internal)(Internal) Outcome: Hold workshop for librarians about Outcome: Hold workshop for librarians about

integrating outcomes assessment in classes integrating outcomes assessment in classes in order to help them be more effective in their in order to help them be more effective in their teaching.teaching.

Assessment: Have each librarian create three Assessment: Have each librarian create three outcomes for every instruction session…outcomes for every instruction session…Collect data from their assessments.Collect data from their assessments.

Activity 7.Activity 7.

Write one outcome using either your long Write one outcome using either your long term or short term goal.term or short term goal.

What do you want to know? Why?What do you want to know? Why?• Don’t forget “in order to”Don’t forget “in order to”

Program Assessment Can:Program Assessment Can:

Improve TeachingImprove Teaching Improve Student LearningImprove Student Learning Provide Validation Provide Validation

Five QuestionsFive Questions

What do you want the instruction program to be What do you want the instruction program to be able to do?able to do?

What factors does the program demonstrate or What factors does the program demonstrate or exhibit if they are meeting the outcomes?exhibit if they are meeting the outcomes?

How will you gather your data or evidence?How will you gather your data or evidence? How will you determine if you have achieved How will you determine if you have achieved

your outcomes?your outcomes? How will the discussion or evaluation of the data How will the discussion or evaluation of the data

occur? Who will be involved?occur? Who will be involved?

Questions Translate Into…Questions Translate Into…

Outcome Outcome What does an effective program do?What does an effective program do?

Indicator Indicator What will be happening? What will be happening?

Assessment Assessment How will the data be collected?How will the data be collected?

Criteria Criteria How will you know?How will you know?

Immersion 2004

An Ongoing Culture of AssessmentAn Ongoing Culture of Assessment

““Assessment is an ongoing process aimed Assessment is an ongoing process aimed at understanding and improving student at understanding and improving student learning.”learning.”

Tom Angelo, AAHE Bulletin, 1996

ACRL Immersion 2004

Assessment Plans: Individual or Assessment Plans: Individual or Group WorkGroup Work

Questions? Questions?

Thank You!Thank You!