Upload
orinda
View
28
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Assessment 101. PNAIRP Conference Thursday October 1, 2009. Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302 [email protected] http://faculty.whatcom.ctc.edu/InstResearch/index.htm. Outcomes. Define assessment. - PowerPoint PPT Presentation
Citation preview
Anne Marie Karlberg Director of Institutional Research and Assessment
(360) [email protected]
http://faculty.whatcom.ctc.edu/InstResearch/index.htm
Assessment 101
PNAIRP Conference Thursday October 1, 2009
Outcomes
1. Define assessment.
2. Identify the components of a successful assessment program.
3. Articulate three types of assessment information or data and provide examples of each.
4. Identify three levels of assessment.
5. Identify two phases in creating an effective outcomes process.
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
What is assessment?
“ The systematic collection of information about student learning…to inform decisions about how to improve learning.”
(Walvoord, 2004, p. 2)
Purposes
1. improvement (formative)1. improvement (formative)
2. accountability (summative)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
Educational Evaluation Model: CIPPBy Daniel Stufflebeam
Using CIPP to Create an Effective Assessment Program
Using CIPP to Create an Effective Assessment Program: Context
Context: What do we know about the context of our
colleges? At Whatcom Community College…
8.3% of our employees
18.9% of our degree and certificate seeking students
16.0% percent of Whatcom County residents
…indicate they are of color.
Using CIPP to Create an Effective Assessment Program: Inputs
Resources
• human resources
• financial support
• technical support
• administrator and faculty support
Plans and Strategies
Whatcom will…
•revise its mission statement and familiarize faculty / staff with mission
•implement the strategic plan
•implement the assessment plan
Using CIPP to Create an Effective Assessment Program: Processes
Implementation
Link assessment program to mission statement • redirect resources towards priorities • increase responsiveness to community needs•measure the extent to which we are successful
Provide opportunities for meaningful, collaborative college-wide conversations, by structuring assessment• in an ongoing, simplified, participatory way • relevant and meaningful
Embed assessment throughout college
Learning and teaching practices
Apply meaningful, relevant and contextualized experiences for students, such as…
→ using self-reflection→ applying concepts to a relevant context→ teaching material to peers→ discovering connections between subjects
Using CIPP to Create an Effective Assessment Program: Products
1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations
Assessment data
1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations
2.Indirect indicators (perceptions): e.g., surveys, focus groups, interviews
Assessment data
1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations
2.Indirect indicators (perceptions): e.g., surveys, focus groups, interviews
3.Institutional data: e.g., retention, graduation, enrollment, transfer trends
Assessment data
A.College level
B.Program level
C.Course level
Assessment levels
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
3. Institutional data (rates and numbers)
Assessment Products: Examples
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
3. Institutional data (rates and numbers)
Assessment Products: Examples
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Using CIPP to Create an Effective Assessment Program
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products
Two phases of an outcomes process
(1) Development of an outcomes process
(2) Implementation of an outcomes process
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
Using CIPP to Create an Effective Assessment Program
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Direct indicators: College outcomes plan
Goal (# of CLAs)
Whatcom will…
Baseline (May 2008) 2008 - 2009
2009 - 2010
2010 - 2011
2011 - 2012
(1) Development of the college outcomes processa. educate faculty / staff / students about assessment No formal
initiativesongoing ongoing ongoing ongoing
b. revise Core Learning Abilities (CLAs) and outcomes Identified 5 CLAs in 1998
Revise CLAs and outcomes
refine review review
c. develop assessment tools (e.g., scoring guides / rubrics) to measure the outcomes
5 review review
d. determine which courses will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)
5 review review
e. include outcomes on syllabi 1 2f. collect the instructional assignments, activities, projects, or experiences, in required courses that will be used to teach outcomes at entry, midway, and exit
1 2
g. collect the activities, experiences, projects, essays, or assignments in required courses that will be used to assess outcomes at entry and exit
1 2
h. attach anchor papers (i.e., examples) for each level of the scoring guide/rubric scale
1 2
(2) Implementation of the college outcomes processa. assess students at (entry and) exit for outcomes 1b. analyze the (entry and) exit assessment data 1c. present analysis to faculty and students and consult on the results 1d. use the data to improve and revise curriculum 1e. document the process; create an assessment report about how the data were used to improve learning
1
Direct indicators: College outcomes plan
Goal (# of CLAs)
Whatcom will…
Baseline (May 2008)
2008 - 2009
2009 - 2010
2010 - 2011
2011 - 2012
(1) Development of the college outcomes processa. educate faculty / staff about assessment No formal
initiativesOngoing
b. revise Core Learning Abilities (CLAs) and outcomes
Identified 5 CLAs in
1998
Revise CLAs and outcomes
Refine Review
c. develop assessment tools (e.g., scoring guides / rubrics, etc.) to measure the outcomes
Review
d. determine which courses will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)
Review
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Indirect indicators:College survey schedule
Goal
Survey
Baseline data (May 2008)
2008 – 2009
2009 – 2010
2010 – 2011
2011 – 2012
2012 – 2013
Student Opinion Survey (every 6 years – alternate with the CCSSE)
Conducted January 2008
Community College Survey of Student Engagement (every 6 years – alternate with the Student Opinion Survey)
Faculty Survey of Student Engagement (every 6 years)
Staff satisfaction survey (every 4 years) Conducted in 2007)
Alumni Survey (ACT with Whatcom-specific questions) (every 10 years)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Institutional data: College-level plan
Type of Data
Baseline May 2008
2008 – 2009
2009 – 2010
2010 – 2011
2011 – 2012
Enrollment numbers
Profile of all students
Grade distribution
Graduation rates / numbers
Retention rates
Success after transfer to WWU
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
Website
•WCC Homepage
• “About Whatcom”
• “Assessment / Inst Research”
http://www.faculty.whatcom.ctc.edu/InstResearch/
Comments and questions?
Using CIPP to Create an Effective Assessment Program
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Anne Marie Karlberg (Director of Institutional Research and Assessment)(360) 383-3302
[email protected]://faculty.whatcom.ctc.edu/InstResearch/index.htm