View
213
Download
0
Category
Preview:
Citation preview
1
Fred Trapp, Ph.D.Administrative Dean, Institutional Research/Academic Services (Retired)
Long Beach City CollegeCambridge West Partnership, LLC
Robert Pacheco, Ed.D.Director Of Institutional Planning, Research and Resource Development
Barstow College
Comprehensive Assessment Reports
2
Outcomes for the Session
The participant will be able to Describe the comprehensive assessment report concept. Locate best practice examples from other colleges
through web links. Discuss how the comprehensive report idea can be used
as part of the institution’s learning process and as a means by which the institution provides quality assurance to the public.
Indicate national trends and efforts of consortia/national organizations to provide quality assurance about student experiences and learning outcomes.
Please hold questions until the end.
8
ACCJC Institutional Effectiveness Rubric
Part III Student Learning Outcomes Proficiency
Comprehensive assessment reports exist and are completed on a regular basis.
9
What Did We Look At, With Whom Did We Consult?
ACCJC. Institutional Effectiveness RubricACCJC. 2002 StandardsACCJC. ThemesACCJC. Guide to Evaluating InstitutionsProfessional literatureEfforts of national groups/institutesInstitutional web sites, listservs and
colleagues
10
Guiding Questions
How can the report writing experience: Help faculty explore the student learning process? Determine the extent to which the curriculum is
working? Where can time, energy and/or money be
allocated for continuous improvement in learning? Exploit the writing process and dialogue about
results to gain broader institutional learning experiences?
Help meet our quality assurance pledge to the community?
11
Illustration Selections
Council for Higher Education Accreditation (CHEA) Annual Award for Outstanding Institutional Practice in
Student Learning Outcomes Demonstrated commitment to & developed highly
effective practice to use SLO assessment Willingness to share the practices they developed
Selection committee Selection criteria
Articulation & evidence of outcomes Success with regard to outcomes Information to the public about outcomes Using outcomes for improvement
12
Illustration selections (continued)
Cited by scholars and peers Schools with assessment work cited in scholarly books
and articles Schools with assessment work selected for presentation
at national conferences Web presentations publicly available for you to ease
drop upon
Prominent national movements/initiatives regarding learning outcomes & assessment documentation (including public quality assurance)
13
What Might Be Included?
Assessment focus- course, program, general ed, etc.
What outcomes were assessed?How and when were they assessed?Who was assessed?What were the results?Who reviewed the results, made sense of the
them and what conclusions were reached?What are the implications for practice and/or
policy or future assessment work?
14
CC of Baltimore County (MD)Course-level Reporting
Middle States Commission on Higher Education
Community College Futures Assembly, Bellwether Award, 2008 Instructional Programs & Services for High Impact
Course Level AssessmentCHEA award winner, 2006
Institutional Progress in Student Learning Outcomes
National Council on Student Development (NCSD) Exemplary Practice Award Winner
15
CC of Baltimore County (MD)Course-level Reporting
Projects are at least three semesters long Individual and high-impact courses (all sections)
included Project proposal by a faculty group
Measurable objectives External review & approval in selecting
methods/instrument & analyzing results. Benchmarking should be included if possible.
Controls and sample size considered. Course improvements based on data analysis Reassessment expected Results/report shared across the college and web
posted
16
CC of Baltimore County (MD)
Learning Outcomes Assessment Final Report Template1. Design & proposal for the LOA project
2. Implementation of design & data collection
3. Redesign of the course to improve student learning
4. Implementation of course revisions & reassessment of student learning
5. Final analysis and results
eavesdropping http://www.ccbcmd.edu/loa/CrseAssess.html Two-page executive summaries available
17
CC of Baltimore County (MD)
CHEM 108 An initial “failure” turned to success and collaboration
with a four-year school
HLTH 101 Addressing an achievement gap with professional
development and increased communication with students
CRJU 101 and 202 Statewide group assessment development effort and
creativity in the interventions used
18
Program-level Reporting
North Central Association of Colleges & Schools, Higher Learning Commission
CHEA award winner, 2008 Institutional Progress in Student Learning
Outcomes
19
Hocking College (OH)
Learning outcomes data collected in a student E-portfolio Direct internal and external evidence (1 to 10 measures) Indirect evidence (1 to 4 measures)
Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric Eight general education outcomes (student success skills) Discipline-specific exit competencies or outcomes
See the Hocking wheel
21
Hocking College (OH)Program-level Report
eavesdropping Cloud reference, not college URL as links
are broken there Various reports available in each program
profile Curriculum matrix Criteria statements (exit competencies) Instructional Program Outcomes (assessment
plan) Trend Charts for performance criteria
Hocking College (OH)Program-level Report
Example reports and analysis Culinary Arts Technology Forestry Management Technology Nursing Technology
22
23
Mesa College (AZ)General-Education Reports
North Central Association of Colleges & Schools, Higher Learning Commission
CHEA Award winner, 2007 Institutional progress in Student Learning
Outcomes
24
Mesa CollegeGeneral Education Report
Multiple outcomes assessed
AnnuallyAnnual Report elements
Executive SummaryMethodologyResults & observations (GE & workplace)Indirect measures findingsAppendices of past results
25
Mesa Community College (AZ)
General education studies completed 2007-08; 2005-06 Numeracy Scientific inquiry Problem solving/critical thinking (2008-09) Information literacy Workplace skills (CTE) (2009-10)
General education studies completed 2006-07; 2004-05 Arts & humanities Cultural diversity Oral communication Written communication
26
Mesa College (AZ)General Education Reports
eavesdropping http://www.mesacc.edu/about/orp/assessment/
index.html
Annual reports and summaries available Nine years of history and experience 14 years of assessment work
27
Capital CC (CT)General-Education Reports
New England Association of Schools and Colleges, Commission on Institutions of Higher Education
Cited in the Art and Science of Assessing General Education Outcomes: A Practical Guide (AAC&U, 2005)
eavesdropping http://www.ccc.commnet.edu/slat/ Annual reports and summaries available
28
Portland CC (OR)General Education Reports
Northwest Accrediting Commissioneavesdropping
http://www.pcc.edu/resources/academic/learning-assessment/
One general education theme a year Learning Assessment Focus for 2009-10- Critical
Thinking & Problem Solving Physical Science, Geology and General Science Bioscience Technology Management and Supervisory Development Culinary Assistant Program
29
Truman State University (MO)Various Reports
Southern Association of Schools and Colleges, Commission on Colleges
eavesdropping Assessment work began in 1970 http://assessment.truman.edu/
Assessment Almanac- A compilation of results from each year’s assessment work (versions from 1997 to 2009 are posted)
General Education outcomes are assessed in the context of the major field of study Portfolio Project- required of all seniors to show best work
assessed by faculty for the nature & quality of the liberal arts and sciences learning outcomes (versions from 1997 to 2008 are posted)
30
Authorship
Course-level, program-level & general education Teaching faculty study team with technical assistance
from institutional research or assessment committee
No “lone ranger” authors
Institutional summary Academic administrator with assistance from
Learning outcomes coordinator or assessment committee Compilation of work accomplished in one or two
academic years across the institution
31
Putting the Reports to “Work”
To all affected participantsCampus committees
Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership
Campus fairs, brown-bag lunches, poster sessions for information sharing
Faculty professional development programsAccreditation self-study committee work groupsLocal governing board presentationCollege web site for the public
32
Reports & a Learning Organization
Learning organization Environment that promotes a culture of learning Individual & group learning enriches & enhances the
organization as a whole Systematic problem solving using data for decisions Learning from experiences in assessing organizational
performance Comparing yourself to others (benchmarking) and
borrowing ideas
Adriana Kezar ed. Organizational Learning in Higher Education New Directions for Higher Education. No. 131, Fall 2005. Jossey-Bass.
33
Reports as Institutional Learning & Resource Allocation
The assessment data sense-making process = a faculty learning experience
Linking results to future interventions = a learning experience
Using results to inform an intervention, then reassess = a learning experience (accomplished one or more terms later)
Reference for future assessment work and other groups on campus
34
Reports as Institutional Learning & Resource Allocation
Hocking College (OH) Annual summary
Improvements in the program in the previous year brought on by study of assessment results
Expenditures of time, money & materials for the assessment program
Requests for assistance in implementing assessment Recommendations for altering the institution’s
assessment process
Transition from evaluating individual students to assessing groups of students & the curriculum experience
35
Reports as Institutional Learning & Resource Allocation
Community College of Baltimore County (MD) Learning Outcomes Assessment Advisory Board
Links findings in assessment reports to other college-wide initiatives and professional development opportunities
Use of assessment processes and (findings) results Challenged faculty to reexamine prompts used in assessment
Clarity of written prompt & extent it supports program goals
Common assignment options and common rubric increases faculty understanding and buy-in
Builds faculty unity toward common goals Public web page enhances communication and accessibility
to information
36
Reports as Institutional Learning& Resource Allocation
CHEA award winner, 2010 Institutional Progress in Student Learning
Outcomes
37
Reports as Institutional Learning& Resource Allocation
Feedback & recognitionFeedback rubric for annual assessment
reports Conversations and action Collection and analysis of evidence Implementation of findings
Recognition (achievement & excellence)
38
Reports as Institutional Learning& Resource Allocation
Seal of Assessment AchievementAcademic programs earning this recognition
have demonstrated in their annual report that learning outcomes have been assessed through two or
more methods, and findings have been discussed among
the faculty.
39
Reports as Institutional Learning& Resource Allocation
Seal of Assessment Excellence Academic programs earning this recognition
have demonstrated a thorough implementation of assessment plan(s) the reporting of meaningful assessment data the discussion of findings among faculty and perhaps students the use of findings to showcase student achievements and to make curricular adjustments.
40
Reports as Institutional Learning& Resource Allocation
Mesa College (AZ)Results Outreach Committee
Promotes use of outcomes data in relation to faculty development, pedagogy and academic climate
Groups of faculty offer a proposal for summer or academic year work above the course level
Resulting report placed on the web and used for campus discussion and action
Report as Quality Assurance
NILOA 2010 Webscan report Exploring the Landscape: What
Institutional Websites Reveal About Student learning Outcomes Assessment Activities
2010 Connecting State Policies on Assessment with Institutional Assessment Activity
2011 Transparency Framework (online)
eavesdropping www.learningoutcomeassessment.org
41
42
Report as Quality Assurance
Promising Vehicles for Expanding Information to the PublicBrief narrative report from annual
assessment reportsSimple statistical reports on learning
outcomes or surveysBest practices stories supported by
assessmentPeter Ewell. Accreditation & the Provision of Additional Information to the Public about
Institutional and Program Performance, CHEA, May 2004
43
Quality Assurance to the Public
Voluntary System of Accountability APLU & AASCU (520 public institutions, award 70% of
bachelor’s degrees in the US each year)
College Profile (includes learning outcomes & links to campus) Proactive initiative to document learning gains and average
institutional scores (choice of 3 national instruments) Proactive initiative to illustrate unique campus learning
outcomes assessment work Promoting a learning institution
eavesdropping http://www.collegeportraits.org/
44
Quality Assurance to the PublicVSA Example, CSU Monterey Bay
http://www.collegeportraits.org/map Cal Poly Pomona
http://www.collegeportraits.org/CA/CPP/learning_outcomes Learning Assessment Examples
45
Quality Assurance to the Public
National Association of independent Colleges and Universities Assessment programs on campus tied to institution’s
mission eavesdropping
http://www.naicu.edu/special_initiatives/accountability/Student_Assessment/id.514/default.asp
Pepperdine University http://services.pepperdine.edu/oie/learning-outcomes/
learning-outcomes-overview.aspx
46
Where Can I Go?Resource
Filesanywhere.com
http://www.filesanywhere.com/fs/v.aspx?v=8a69668b5c6773a96f6d
Contacts & Questions
Robert Pacheco (Barstow College)Rpacheco@Barstow.edu
Fred Trapp (Cambridge West Partnership)fredtrapp@gmail.com
Questions and Comments
47
48
Session EvaluationOutcomes for the Session
The participant will be able to Describe the comprehensive assessment report
concept. Locate best practice examples from other colleges
through web links. Discuss how the comprehensive report idea can be used
as part of the institution’s learning process and as a means by which the institution provides quality assurance to the public.
Indicate national trends and efforts of consortia/national organizations to provide quality assurance about student experiences and learning outcomes.
Recommended