1

Fred Trapp, Ph.D.Administrative Dean, Institutional Research/Academic Services (Retired)

Long Beach City CollegeCambridge West Partnership, LLC

Robert Pacheco, Ed.D.Director Of Institutional Planning, Research and Resource Development

Barstow College

Comprehensive Assessment Reports

2

Outcomes for the Session

The participant will be able to Describe the comprehensive assessment report concept. Locate best practice examples from other colleges

through web links. Discuss how the comprehensive report idea can be used

as part of the institution’s learning process and as a means by which the institution provides quality assurance to the public.

Indicate national trends and efforts of consortia/national organizations to provide quality assurance about student experiences and learning outcomes.

Please hold questions until the end.

3

4

Curriculum Map4

5

6

7

8

ACCJC Institutional Effectiveness Rubric

Part III Student Learning Outcomes Proficiency

Comprehensive assessment reports exist and are completed on a regular basis.

9

What Did We Look At, With Whom Did We Consult?

ACCJC. Institutional Effectiveness RubricACCJC. 2002 StandardsACCJC. ThemesACCJC. Guide to Evaluating InstitutionsProfessional literatureEfforts of national groups/institutesInstitutional web sites, listservs and

colleagues

10

Guiding Questions

How can the report writing experience: Help faculty explore the student learning process? Determine the extent to which the curriculum is

working? Where can time, energy and/or money be

allocated for continuous improvement in learning? Exploit the writing process and dialogue about

results to gain broader institutional learning experiences?

Help meet our quality assurance pledge to the community?

11

Illustration Selections

Council for Higher Education Accreditation (CHEA) Annual Award for Outstanding Institutional Practice in

Student Learning Outcomes Demonstrated commitment to & developed highly

effective practice to use SLO assessment Willingness to share the practices they developed

Selection committee Selection criteria

Articulation & evidence of outcomes Success with regard to outcomes Information to the public about outcomes Using outcomes for improvement

12

Illustration selections (continued)

Cited by scholars and peers Schools with assessment work cited in scholarly books

and articles Schools with assessment work selected for presentation

at national conferences Web presentations publicly available for you to ease

drop upon

Prominent national movements/initiatives regarding learning outcomes & assessment documentation (including public quality assurance)

13

What Might Be Included?

Assessment focus- course, program, general ed, etc.

What outcomes were assessed?How and when were they assessed?Who was assessed?What were the results?Who reviewed the results, made sense of the

them and what conclusions were reached?What are the implications for practice and/or

policy or future assessment work?

14

CC of Baltimore County (MD)Course-level Reporting

Middle States Commission on Higher Education

Community College Futures Assembly, Bellwether Award, 2008 Instructional Programs & Services for High Impact

Course Level AssessmentCHEA award winner, 2006

Institutional Progress in Student Learning Outcomes

National Council on Student Development (NCSD) Exemplary Practice Award Winner

15

CC of Baltimore County (MD)Course-level Reporting

Projects are at least three semesters long Individual and high-impact courses (all sections)

included Project proposal by a faculty group

Measurable objectives External review & approval in selecting

methods/instrument & analyzing results. Benchmarking should be included if possible.

Controls and sample size considered. Course improvements based on data analysis Reassessment expected Results/report shared across the college and web

posted

16

CC of Baltimore County (MD)

Learning Outcomes Assessment Final Report Template1. Design & proposal for the LOA project

2. Implementation of design & data collection

3. Redesign of the course to improve student learning

4. Implementation of course revisions & reassessment of student learning

5. Final analysis and results

eavesdropping http://www.ccbcmd.edu/loa/CrseAssess.html Two-page executive summaries available

17

CC of Baltimore County (MD)

CHEM 108 An initial “failure” turned to success and collaboration

with a four-year school

HLTH 101 Addressing an achievement gap with professional

development and increased communication with students

CRJU 101 and 202 Statewide group assessment development effort and

creativity in the interventions used

18

Program-level Reporting

North Central Association of Colleges & Schools, Higher Learning Commission

CHEA award winner, 2008 Institutional Progress in Student Learning

Outcomes

19

Hocking College (OH)

Learning outcomes data collected in a student E-portfolio Direct internal and external evidence (1 to 10 measures) Indirect evidence (1 to 4 measures)

Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric Eight general education outcomes (student success skills) Discipline-specific exit competencies or outcomes

See the Hocking wheel

20

21

Hocking College (OH)Program-level Report

eavesdropping Cloud reference, not college URL as links

are broken there Various reports available in each program

profile Curriculum matrix Criteria statements (exit competencies) Instructional Program Outcomes (assessment

plan) Trend Charts for performance criteria

Hocking College (OH)Program-level Report

Example reports and analysis Culinary Arts Technology Forestry Management Technology Nursing Technology

22

23

Mesa College (AZ)General-Education Reports

North Central Association of Colleges & Schools, Higher Learning Commission

CHEA Award winner, 2007 Institutional progress in Student Learning

Outcomes

24

Mesa CollegeGeneral Education Report

Multiple outcomes assessed

AnnuallyAnnual Report elements

Executive SummaryMethodologyResults & observations (GE & workplace)Indirect measures findingsAppendices of past results

25

Mesa Community College (AZ)

General education studies completed 2007-08; 2005-06 Numeracy Scientific inquiry Problem solving/critical thinking (2008-09) Information literacy Workplace skills (CTE) (2009-10)

General education studies completed 2006-07; 2004-05 Arts & humanities Cultural diversity Oral communication Written communication

26

Mesa College (AZ)General Education Reports

eavesdropping http://www.mesacc.edu/about/orp/assessment/

index.html

Annual reports and summaries available Nine years of history and experience 14 years of assessment work

27

Capital CC (CT)General-Education Reports

New England Association of Schools and Colleges, Commission on Institutions of Higher Education

Cited in the Art and Science of Assessing General Education Outcomes: A Practical Guide (AAC&U, 2005)

eavesdropping http://www.ccc.commnet.edu/slat/ Annual reports and summaries available

28

Portland CC (OR)General Education Reports

Northwest Accrediting Commissioneavesdropping

http://www.pcc.edu/resources/academic/learning-assessment/

One general education theme a year Learning Assessment Focus for 2009-10- Critical

Thinking & Problem Solving Physical Science, Geology and General Science Bioscience Technology Management and Supervisory Development Culinary Assistant Program

29

Truman State University (MO)Various Reports

Southern Association of Schools and Colleges, Commission on Colleges

eavesdropping Assessment work began in 1970 http://assessment.truman.edu/

Assessment Almanac- A compilation of results from each year’s assessment work (versions from 1997 to 2009 are posted)

General Education outcomes are assessed in the context of the major field of study Portfolio Project- required of all seniors to show best work

assessed by faculty for the nature & quality of the liberal arts and sciences learning outcomes (versions from 1997 to 2008 are posted)

30

Authorship

Course-level, program-level & general education Teaching faculty study team with technical assistance

from institutional research or assessment committee

No “lone ranger” authors

Institutional summary Academic administrator with assistance from

Learning outcomes coordinator or assessment committee Compilation of work accomplished in one or two

academic years across the institution

31

Putting the Reports to “Work”

To all affected participantsCampus committees

Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership

Campus fairs, brown-bag lunches, poster sessions for information sharing

Faculty professional development programsAccreditation self-study committee work groupsLocal governing board presentationCollege web site for the public

32

Reports & a Learning Organization

Learning organization Environment that promotes a culture of learning Individual & group learning enriches & enhances the

organization as a whole Systematic problem solving using data for decisions Learning from experiences in assessing organizational

performance Comparing yourself to others (benchmarking) and

borrowing ideas

Adriana Kezar ed. Organizational Learning in Higher Education New Directions for Higher Education. No. 131, Fall 2005. Jossey-Bass.

33

Reports as Institutional Learning & Resource Allocation

The assessment data sense-making process = a faculty learning experience

Linking results to future interventions = a learning experience

Using results to inform an intervention, then reassess = a learning experience (accomplished one or more terms later)

Reference for future assessment work and other groups on campus

34

Reports as Institutional Learning & Resource Allocation

Hocking College (OH) Annual summary

Improvements in the program in the previous year brought on by study of assessment results

Expenditures of time, money & materials for the assessment program

Requests for assistance in implementing assessment Recommendations for altering the institution’s

assessment process

Transition from evaluating individual students to assessing groups of students & the curriculum experience

35

Reports as Institutional Learning & Resource Allocation

Community College of Baltimore County (MD) Learning Outcomes Assessment Advisory Board

Links findings in assessment reports to other college-wide initiatives and professional development opportunities

Use of assessment processes and (findings) results Challenged faculty to reexamine prompts used in assessment

Clarity of written prompt & extent it supports program goals

Common assignment options and common rubric increases faculty understanding and buy-in

Builds faculty unity toward common goals Public web page enhances communication and accessibility

to information

36

Reports as Institutional Learning& Resource Allocation

CHEA award winner, 2010 Institutional Progress in Student Learning

Outcomes

37

Reports as Institutional Learning& Resource Allocation

Feedback & recognitionFeedback rubric for annual assessment

reports Conversations and action Collection and analysis of evidence Implementation of findings

Recognition (achievement & excellence)

38

Reports as Institutional Learning& Resource Allocation

Seal of Assessment AchievementAcademic programs earning this recognition

have demonstrated in their annual report that learning outcomes have been assessed through two or

more methods, and findings have been discussed among

the faculty.

39

Reports as Institutional Learning& Resource Allocation

Seal of Assessment Excellence Academic programs earning this recognition

have demonstrated a thorough implementation of assessment plan(s) the reporting of meaningful assessment data the discussion of findings among faculty and perhaps students the use of findings to showcase student achievements and to make curricular adjustments.

40

Reports as Institutional Learning& Resource Allocation

Mesa College (AZ)Results Outreach Committee

Promotes use of outcomes data in relation to faculty development, pedagogy and academic climate

Groups of faculty offer a proposal for summer or academic year work above the course level

Resulting report placed on the web and used for campus discussion and action

Report as Quality Assurance

NILOA 2010 Webscan report Exploring the Landscape: What

Institutional Websites Reveal About Student learning Outcomes Assessment Activities

2010 Connecting State Policies on Assessment with Institutional Assessment Activity

2011 Transparency Framework (online)

eavesdropping www.learningoutcomeassessment.org

41

42

Report as Quality Assurance

Promising Vehicles for Expanding Information to the PublicBrief narrative report from annual

assessment reportsSimple statistical reports on learning

outcomes or surveysBest practices stories supported by

assessmentPeter Ewell. Accreditation & the Provision of Additional Information to the Public about

Institutional and Program Performance, CHEA, May 2004

43

Quality Assurance to the Public

Voluntary System of Accountability APLU & AASCU (520 public institutions, award 70% of

bachelor’s degrees in the US each year)

College Profile (includes learning outcomes & links to campus) Proactive initiative to document learning gains and average

institutional scores (choice of 3 national instruments) Proactive initiative to illustrate unique campus learning

outcomes assessment work Promoting a learning institution

eavesdropping http://www.collegeportraits.org/

44

Quality Assurance to the PublicVSA Example, CSU Monterey Bay

http://www.collegeportraits.org/map Cal Poly Pomona

http://www.collegeportraits.org/CA/CPP/learning_outcomes Learning Assessment Examples

45

Quality Assurance to the Public

National Association of independent Colleges and Universities Assessment programs on campus tied to institution’s

mission eavesdropping

http://www.naicu.edu/special_initiatives/accountability/Student_Assessment/id.514/default.asp

Pepperdine University http://services.pepperdine.edu/oie/learning-outcomes/

learning-outcomes-overview.aspx

46

Where Can I Go?Resource

Filesanywhere.com

http://www.filesanywhere.com/fs/v.aspx?v=8a69668b5c6773a96f6d

Contacts & Questions

Robert Pacheco (Barstow College)Rpacheco@Barstow.edu

Fred Trapp (Cambridge West Partnership)fredtrapp@gmail.com

Questions and Comments

47

48

Session EvaluationOutcomes for the Session

The participant will be able to Describe the comprehensive assessment report

concept. Locate best practice examples from other colleges

through web links. Discuss how the comprehensive report idea can be used

as part of the institution’s learning process and as a means by which the institution provides quality assurance to the public.

Indicate national trends and efforts of consortia/national organizations to provide quality assurance about student experiences and learning outcomes.