Embed Size (px)
Citation preview
The “Self Regarding Institution” Using Assessment for Accountability and Improvement
Douglas R. Davenport
Dean, College of Arts and Sciences
Truman State University
Kirksville, Missouri 63501
Indiana University South BendAssessment WorkshopMarch 28, 2008
What are your goals?
List three things you hope to gain by attending this workshop.
Rank them in terms of importance.
Questions to Consider…
Why does higher education engage in assessment?
What motivates YOU to do assessment?
Assessment at IUSB…
What are the best examples of assessment you know of on campus?
What might you learn from them? What are the key challenges facing the
implementation of assessment on campus? Who is responsible for assessment at IUSB? (Everybody!)
Where Do We Begin?
“Begin with the end in mind.” (Covey)
What are our Learning Goals?” Write down the student learning goals for…
Your favorite course to teach Your academic program The university
Can they be assessed?
Writing Student Learning Outcomes
Learning outcome statements should: reflect essential knowledge, skills or
attitudes; focus on results of the learning experiences; reflect the desired end of the learning
experience, not the means or the process; Learning outcomes statements may be
considered to be exit behaviors.(Adapted from Lesch, 2008)
Writing Student Learning Outcomes
Key Components: Action word (performance) Learning statement (the learning) Criterion (the conditions of the performance)
(Adapted from Lesch, 2008)
Writing Student Learning Outcomes
Action words to avoid: Appreciate Enjoy Know Realize Be aware of Perceive
Appropriate action words: Use Develop Analyze Express Evaluate Organize Create Write Plan Apply
(Adapted from Lesch, 2008)
Exercise…Write one or more learning outcome statements
for the following goals…
1. Graduates will be able to communicate effectively in writing.
2. Graduates will have an understanding of cultures beyond the United States.
3. Graduates will appreciate the fine and performing arts.
Time for a Temperature Check…“What is the most important thing you
have learned thus far?”
“What important questions remain unanswered?”
How Do We Assess Outcomes?General design principles… Know your purposes (accountability and improvement) Use multiple measures and mixed-methods Make it your own Don’t reinvent the wheel Start slowly Don’t assess every outcome every year Involve faculty Use authentic assessment wherever possible (e.g., embed
assessment in courses, etc)
The Truman Experience Began assessment program in 1972 Started with senior tests Added a writing assessment (SWE) and portfolio
(1980s) Emphasized value-added assessment (1990s) Added self-reports (CSEQ, GSQ, CIRP) (2000s)
Truman Assessment Today Nationally-normed senior tests in every major CLA MAPP and CAPP CSEQ CIRP Portfolio Project Student Interview Project Graduating Student Questionnaire http://assessment.truman.edu
Using Portfolios for Multiple Assessments at Truman Writing Critical Thinking “Modes of Inquiry” Interdisciplinary Thinking http://assessment.truman.edu/components/P
ortfolio/index.asp
Writing Assessment & Portfolios:Rationale “Leverage” current assessment activities A natural location Retains the benefit of embedded assessment Minimizes additional student work
Writing Assessment & Portfolios:Putting it into Practice Tested in 2004, using Critical Thinking
Submissions Incorporated in 2005, using a new “Critical
Thinking and Writing” prompt
Portfolio Findings
The vast majority of students demonstrated competence in all three areas
Organization – 81% Style – 81% Mechanics – 84%
Portfolio Findings
Organization, 2004-2005
2
25
44
28
1
18
47
34
0
10
20
30
40
50
None Weak Competent Strong
Rating
Perc
ent
2004
2005
Portfolio FindingsStyle, 2004-2005
2
21
46
31
1
18
53
28
0
10
20
30
40
50
60
None Weak Competent Strong
Rating
Per
cen
t2004
2005
Portfolio FindingsMechanics, 2004-2005
40
2
16
42
1
15
45
39
0
10
20
30
40
50
None Weak Competent Strong
Rating
Perc
ent
2004
2005
Why Assessment? - Accountability External Forces…
Spellings Report, A Test of Leadership (NCLB, Part 2)
Public Funding Priorities Rising Cost of Higher Education Call for Objective, Comparative Measures “It's proper for the government to define ‘legally
drunk,’ but not ‘legally intelligent.’” (Eubanks, 2006)
College Portrait: Voluntary System of Accountability (VSA) Sponsored by NASULGC and AASCU Provides consumer information regarding
institutional measures, student experiences and perceptions, and student learning outcomes.
Learning outcomes– CLA, MAPP, and CAPP Experiences and perceptions – CSEQ, CIRP,
NSSE, CSS
Why Assessment? - Improvement Assessment for Learning The Learning Organization (Senge) Turbulent Times “It is knowledge creation, not knowledge
reproduction, that creates competitive advantage.” (Banta, 2007)
Are These Motivations Mutually Exclusive? Not at all… Accountability is a hallmark of democratic
institutions. Higher education requires transparency. However, assessment to merely pacify is
problematic – it reinforces shallow learning. So…
Characteristics of Assessment by Purpose Accountability Improvement
Audience External Internal
Stakes – Institution High High
Stakes – Programs Low High
Stakes – Students Low Varies
Methods StandardizedNationally-NormedCriterion Referenced
CustomizedLocally-developed
Unit of Analysis Institution Program or Student
Assessment Type Summative Formative
Turbulent Times Did you know?…
Today’s learner will have 10-14 jobs by age 38. The top 10 jobs that will be demand in 2010 did not
exist in 2004. We are preparing students for jobs that do not yet
exist, using technologies that have not been invented in order to solve problems that we have not yet recognized.
Nationally Normed/Criterion Referenced Assessment Instruments – General Collegiate Learning Assessment (CLA - CAE) College Basic Academic Subjects Examination (C
BASE – Univ. of MO) Collegiate Assessment of Academic Proficiency
(CAAP - ACT) Measure of Academic Proficiency and Progress
(MAPP - ETS) iSkills (formerly ICT Literacy Assessment - ETS) Standardized Assessment of Information Literacy
Skills (SAILS – Kent State) WorkKeys (ACT)
Nationally Normed/Criterion Referenced Assessment Instruments – Subject Specific Area Concentration Achievement Test
(ACAT) Major Field Test (MFT – ETS) Praxis II (ETS) A vast array of subject specific exams are
administered through accrediting bodies.
Nationally Normed Assessment Instruments – Self Report National Survey of Student Engagement
(NSSE) College Student Experiences Questionnaire
(CSEQ) Cooperative Institutional Research Program
(CIRP) Freshman Survey (HERI) College Senior Survey (CSS – HERI)
Local Instruments – Some Examples Portfolios Exit Surveys Capstone Projects Juried Theses, Exhibits, Performances
What Do Our Constituents Think?Employers deem both multiple-choice tests of general
content knowledge and institutional assessment that show how colleges compare in advancing critical-
thinking skills of limited value for evaluating applicants’ potential for success in the workplace …
faculty-assessed internships, community-based projects, and senior projects would be the most useful in gauging graduates’ readiness for the
workplace.P. Hart Research Associates, AAC&U (January, 2008)
