Institutional Effectiveness and Student Learning: Integration of Student and Learning Support Services ASCCC Accreditation Institute 2013 Kevin Bontenbal,

Institutional Effectivenessand Student Learning: Integration of Student and Learning Support

Services

ASCCC Accreditation Institute 2013

Kevin Bontenbal, Cuesta CollegeCatherine Webb, Monterey Peninsula College

Roberta Eisel, Citrus College, facilitator

Standard I – Institutional Mission and Effectiveness

I. A. - Mission

I. B. – Improving Institutional Effectiveness

Standard II – Student Learning Programs and Services

II. A. Instructional ProgramsII. B. Student Support ServicesII. C. Library and Learning Support Services

Standard II. B. Student Support Services

II. B. “…The institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measure in order to improve the effectiveness of these services.”

II. B. 4. “… Evaluation of these services provides evidence that they contribute to the achievement of student learning outcomes. The institution uses the results of these evaluations as the basis for improvement.”

Standard II. C. Library and Learning Support Services

II. C. “… The institution systematically assesses these services using student learning outcomes, faculty input, and other appropriate measures in order to improve the effectiveness of the services.”

II. C. 2. “… Evaluation of these services provides evidence that they contribute to the achievement of student learning outcomes. The institution uses the results of the evaluations as the basis for improvement.”

Accreditation Reference HandbookJuly 2012 edition

Where do I find the current handbook?

http://www.accjc.org/wp-content/uploads/2012/09/Accreditation-Reference-Handbook_2012.pdf

Rubric for Evaluating Institutional Effectiveness (2011)

Part I - Program ReviewPart II - PlanningPart III - Student Learning Outcomes

Where do I find the entire rubric?

http://www.accjc.org/wp-content/uploads/2011/10/CoverMemoAndRevisedRubric_10-28-2011.pdf

Rubric – Part I: Program Review

• Program review processes are ongoing, systematic and used to assess and improve student learning and achievement.

• The institution reviews and refines its program review processes to improve institutional effectiveness.

• The results of program review are used to continually refine and improve program practices resulting in appropriate improvements in student achievement and learning.

Rubric– Part II: Planning

• The institution uses ongoing and systematic evaluation and planning to refine its key processes and improve student learning.

• There is dialogue about institutional effectiveness that is ongoing, robust and pervasive; data and analyses are widely distributed and used throughout the institution.

• There is ongoing review and adaptation of evaluation and planning processes.

• There is consistent and continuous commitment to improving student learning; and educational effectiveness is a demonstrable priority in all planning structures and processes.

Rubric Part III: Student Learning Outcomes

• Student learning outcomes and authentic assessments are in place for courses, programs, support services, certificates and degrees.

• There is widespread institutional dialogue about the results of assessment and identification of gaps.

• Decision-making includes dialogue on the results of assessment and is purposefully directed toward aligning institution-wide practices to support and improve student learning.

• Appropriate resources continue to be allocated and fine-tuned.

Rubric Part III: Student Learning Outcomes, continued

• Comprehensive assessment reports exist and are completed and updated on a regular basis.

• Course student learning outcomes are aligned with degree student learning outcomes.

• Students demonstrate awareness of goals and purposes of courses and programs in whichthey are enrolled.

Citrus CollegeInstitutional Effectiveness Committee

Standing Committee of Steering

Co-chaired by Program Review Coordinator and Director of Institutional Research

Citrus CollegeInstitutional Effectiveness Committee

Purpose:Review and make recommendations on matters regarding institutional effectiveness, i.e. the review and assessment of the integrated planning process and the means by which it is linked to the budget process.

Charge:Advance the college mission by promoting an ongoing, collegial, self-reflective dialogue based on a culture of evidence and data-driven decision-making.

Integrated Planning Model

Mission, Vision, Values

Strategic Plan Board Goals

Annual and Comprehensive

Program Reviews

Educational and FacilitiesMaster Plan

Annual and Comprehensive

Program Reviews

Institutional Support PlansAnnual Implementation

Plans/Progress Reports

Resource Allocation

Plan Implementation

Assessment

Program Improvement

Student Services Program Review

Like academic program review, student services program review consists of on-going, high quality peer reviews of all the college's student service areas on a five-year cycle to determine strengths and areas needing improvement.

Instructional Support Program Review

Annual

Analyzelibrary, learning center, honors, learning communities

Develop unit plans

Strategies to remedy weaknesses,Contributions to institutional goals and strategic objectivesBudget requests for equipment, staff and facilities

Report on assessment of student learning outcomes

Comprehensive

Verify content of curriculum is current

Student Services Program Review

AnnualAnalyze

student services programs to identify strengths and weaknesses

Develop unit plans to identify

Strategies to remedy weaknessesContributions to institutional goals and strategic objectivesBudget requests for equipment, staff and facilities

Report on assessment of student learning outcomes

ComprehensiveVerify content of curriculum is current

Library & Learning Support Services

ACCJC Recommendation

Library and Learning Support Services

• To meet the standard, the team recommends the college and its libraries develop and implement an on-going formal assessment of their library services, on-line services, and student learning outcomes.

(Standards II.C.1.b, II.C.1.c, II.C.2)

Library SLOs

• A Cuesta College information literate student is one who should know or be able to do the following:– Conceptualize and communicate a research

topic or information need, and know when expert assistance is necessary.

– Synthesize material and evaluate whether information need has been successfully satisfied.

– Locate, use, and evaluate library and information resources relevant to class assignments and personal information needs.

Assessment Strategies

• Library Assessment Committee– Direct and formalize the assessment of

library services, collections, online services, and SLOs.

• Data, Data, Data– Usage statistics for online and print

collections– Reserve materials– Interlibrary request– Reference desk usage

• Even more data…– English Composition Library Research

Workbook– Online Information Competency Tutorial– Research Skills Class– Library Satisfaction Survey– Library Orientation Surveys• Pre & post

• Program Review – Assessments – Improvements – Resource Allocation

Assessment Strategies

Institutional Effectiveness &

Learning Support Services

Library & Learning Support Services @ MPC

• Library Services Division • Academic Support

Center (tutoring)• ESL Center• English & Study Skills

Center• Reading Center

• High Tech Center for Students with Disabilities

• TRIO Learning Center• Nursing Learning

Resource Center• Math Learning Center

“Dialogue about institutional effectiveness is ongoing, robust, and pervasive…”

• Dialogue around SLO assessment, program review, and planning/resource allocation are all part of the same, cyclical process

• Mission-driven, data-informed, & student learning-focused

What this looks like at the institutional level:

SLOs live here:

What this looks like at the institutional level:

What this looks like from inside a division:

Every Semester: • Instructor & Program Reflections

Every Year: • Program Review Annual Updates & Action Plans• Academic Affairs / Student Services

Component Goals• Resource Allocation Discussions

Every 3 Years: • Institutional Goals / Mission Statement Review

Every 6 Years: • Full Program Review







A quick aside about Reflections:

Instructor & Program Reflections are the heart of our SLO efforts• Keystone of our SLO Evidence• Designed to prompt

o Dialogo Collaborationo Improvement Effortso Action Plan Rationale







How the cycle works:

Reflections (SLO assessment data/dialogue) identification of needs

action plan items for the Annual Update planning/resource allocation

Reflections

How the cycle works:

Reflections (SLO assessment data/dialogue) identification of needs

action plan items for the Annual Update planning/resource allocation

Reflections

Sample Student Learning Outcomes from the MPC Library

Users of the MPC Library will:

…be able to access and use information resources in a variety of formats;

…have access to information and instruction services

Sample Program Reflections dialoguearound those specific SLOs

“The internal advances in tracking statistics and providing information (using LibStats and LibGuides for these, respectively) have been major successes.”

“Budget issues will be challenges in the coming years, and we need to analyze usage patterns to make the best use of our funds.”

MPC Library’s Nursing LibGuide

Sample language from the Library’s Program Review Annual Update narrative :

“…significant increases continue to be recorded in the use of the library’s online resources. Some representative databases and their growth in usage:

Academic Search Premier +44%History Reference Center +45%Nursing Reference Center +177%

The last figure is a good example of the impact of specific instruction and curricular changes, which can result in many students “discovering” a database that they had never been exposed to before.

The explosive growth in physical usage experienced by the library in the first several years after the move to the Library and Technology Center building has stabilized, but demand for library resources is clearly continuing to grow.”

Path through the cycle:

Reflections (SLO assessment data/dialogue)Data confirms that usage of both LibGuides & databases

appears to be up, and that LibGuides use has a positive impact on database discovery

identification of needs

action plan items for the Annual Update

planning/resource allocation

Reflections

Path through the cycle:Reflections (SLO assessment data/dialogue)Data confirms that usage of both LibGuides & databases appears to be up, and that

LibGuides use has a positive impact on database discovery

identification of needsGrowth in demand for electronic resources, further

incorporation of LibGuides into instruction

action plan items for the Annual Update


Reflections

Path through the cycle:Reflections (SLO assessment data & dialogue)Data confirms that usage of both LibGuides & databases appears to be up, and that


identification of needsGrowth in demand for electronic resources, further incorporation of LibGuides into

instruction

action plan items for the Annual UpdateIncrease budget for subscription databases (budget dependent) Expand

development of LibGuides (not budget dependent)


Reflections



identification of needsGrowth in demand for electronic resources, further incorporation of LibGuides into instruction

action plan items for the Annual UpdateIncrease budget for subscription databases (budget dependent)Expand development of LibGuides (not budget dependent)

planning/resource allocationBudget requests submitted to College Council (via Academic Affairs); non

budget-dependent requests put into action

Reflections



identification of needsGrowth in demand for electronic resources, further incorporation of LibGuides into instruction

action plan items for the Annual UpdateIncrease budget for subscription databases (budget dependent)Expand development of LibGuides (not budget dependent)

planning/resource allocationBudget requests submitted to College Council (via Academic Affairs); non budget-dependent

requests put into action

ReflectionsAssessment / evaluation of action items; discussion of further

improvements

Top Deficiencies Causing Sanctions(ACCJC Newsletter, June 2012)

Colleges onSanction

ProgramReview

PlanningInternal

GovernanceBoard

Financial Stability or

Management

2009 Sanctions(n=24)

71%(17)

92%(22)

46%(11)

46%(11)

54%(13)


68%(13)

89%(17)

42%(8)

58%(11)

58%(11)


19%(4)

71%(15)

24%(5)

67%(14)

62%(13)


21%(6)

71%(20)

18%(5)

71%(20)

50%(14)

Trends in Deficiencies Leading to Sanctions

• The proportion of institutions with deficiencies in program review work has decreased considerably from 71% of those on sanction in 2009 to 19% of those on sanction in 2012.

• The proportion of institutions with deficiencies in planning practices has decreased somewhat from 92% of those on sanction in 2009 to 71% of those on sanction in 2012.

ACCJC Newsletter June 2012

Trends, continued

• Internal governance deficiencies have decreased from 46% of those institutions on sanction in 2009 to 18% of those on sanction in 2011.

• Of most concern, the proportion of institutions with

deficiencies in governing board practices has increased sharply from 46% of on sanction in 2009 to 71% of those on sanction in 2012.

• The proportion of institutions on sanction with deficiencies in financial stability or management has remained at or slightly above 50% since 2009.

ACCJC Newsletter June 2012

What to look for when we all go home….

Evidence of Institutional Effectiveness in Student and Learning Support Services

Let’s hear from our colleagues…

47

Documents

Institutional Effectiveness and Student Learning: Integration of Student and Learning Support Services ASCCC Accreditation Institute 2013 Kevin Bontenbal,