View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Practical Strategiesfor Assessing Student Learningin Departments and Programs:
A Utilization-Focused Approach
Jo BeldSt. Olaf College
Guiding questions
• Who are the users of the evidence?
• How might the evidence be used?
• How will you gather the evidence?
• How will you animate the evidence?
Theoretical perspectives
Utilization-focused assessment
(Patton, 2008):
Focus on intended uses
by intended users
Theoretical perspectives
Backward design
(Wiggins & McTighe, 2005):
“Beginning with the end in mind”
Theoretical perspectives
Traditional instructional design:
Choose texts
Develop classroom activities
Make up tests
Theoretical perspectives
Backward instructional design:
Identify intended learning outcomes
Determine appropriate evidence
Plan instruction and practice
Theoretical perspectives
Traditional assessment design:
Choose an assessment instrument
Gather and summarize evidence
Send a report to someone
Theoretical perspectives
Backward assessment design:
Identify intended users and uses
Define and locate the learning
Choose assessment instrument
Who are the users?
Faculty roles, commitments, and disciplinary identities offer both incentives and disincentives to engage assessment
What are the uses?
What kind of outcome do you want to investigate…
–Knowledge (“Understanding of…”)
–Proficiencies (“The ability to…”)
–Practices (“The habit of…”)
–Values or attitudes (“A concern for…”)
What are the uses?…and why?
• Affirming current practices• Tweaking the content of key courses• Extending a specific pedagogy• Enhancing “scaffolding”• Piloting innovations• Supporting grant applications• Setting future assessment agendas
How will you gather evidence?
Assessment is like any other kind of investigation; use strategies that fit the questions you are trying to answer
How will you gather evidence?
“Direct” Assessment”
“Direct” Assessment
Evidence of what students actually know, can do, or care about
“Indirect”Assessment
Evidence of learning-relatedexperiences or
perceptions
How will you gather evidence?
Common direct assessment “artifacts”
Papers, essays, abstracts
Presentations and posters
Oral or written examination items
Analytic journals
Responses to survey or interview questions that ask for examples of knowledge, practice, or value
How will you gather evidence?
Common indirect assessment “artifacts”
Course-taking patterns or transcript analysis
Responses to survey or interview questions about experiences, perceptions, self-reported progress, or impact of program experiences
Reflective journals
How will you gather evidence?
But wait!! Aren’t we observing student work all the time anyway? What’s the difference between grading and assessment?
How will you gather evidence?
Gradingsummarizes
many outcomes
for
one student
Assessmentsummarizes
one outcome
for
many students
How will you gather evidence?
The purpose of an assessment instrument is to provide systematic, summarized information about the
extent to which a group of students has realized one or more intended learning
outcomes
How will you gather evidence?
Options to consider:
Use an instrument developed by someone else
Adapt an existing instrument
Add to something you’re already doing
Connect to institutional-level evidence
Invent something new
How will you gather evidence?
Where possible, pair indirect observations of processes and perceptions with direct observations of outcomes (pp. 16-17).
How will you gather evidence?
The dual goal of sampling:
Representativenessand
Manageability
How will you gather evidence?
Examples involving comprehensive sampling:
Survey of all senior majors
Application of rubric to all research abstracts in all seminars
Application of rubric to all work submitted for senior art show
How will you gather evidence?
Examples involving selective sampling:
Application of rubric to randomly-selected subset of final papers in capstone course
Pre/post administration of locally-developed quiz in required sophomore methods course
End-of-course survey in one introductory and one senior-level course
Aggregation of results on selected items in an evaluation form for student work
How will you animate the evidence?
Evidence never speaks for itself; be intentional in summarizing, interpreting, and responding to the findings
How will you animate the evidence?
Plan intentionally for summarizing and interpreting findings:
• Delegate and distribute• Enlist help from staff and students• Use technology to save time• Balance individual and collective work• Dedicate time for discussion
How will you animate the evidence?
Plan intentionally for responding to the findings:
• Borrow strategies from past successes in collective departmental action
• Focus reporting on planned actions, not on the evidence itself
• Weight Watchers trumps The Biggest Loser
• Dedicate resources for action
A final thought….
Less really is more!