Student Affairs Assessment: Using Student Learning Outcomes for

Continuous Improvement

Evaluating Institutional Learning-Centeredness ConferenceCenter for Educational Leadership, Innovation, and Policy in

Postsecondary EducationDepartment of Administration, Rehabilitation and

Postsecondary Education, SDSUJuly 13, 2007

CSU Channel IslandsCSU Sacramento

University of the Pacific

1

Workshop Presenters:

Elizabeth Griego, VP for Student Life, Pacificegriego@pacific.edu (209) 946-2365

George Morten, Asst VP for Student Affairs, CSUCIgeorge.morten@csuci.edu (805) 437-8510

Lori Varlotta, VP Student Affairs, Sacramento Statevarlotta@csus.edu (916) 278-6060

CSU Channel IslandsSacramento State

University of the Pacific

2

Learning Outcomes for Conference Participants

This session will help participants:• Understand how the University’s mission, values,

traditions, and aspirations guide the co-curricular assessment effort

• Articulate the potential benefits of assessment in student affairs

• Write direct learning outcomes that measure learning goals

• Identify useful types of assessment and evidence-gathering tools

CSU Channel IslandsCSU Sacramento

University of the Pacific

3

CSU Channel IslandsSacramento State

University of the Pacific

44

Potential Value of Assessment in Student Affairs :

• Target the learning we expect students to achieve via their participation in the co-curriculum

• Demonstrate the value-added nature of our programs and the difference they make in the lives of students (accountability)

• Focus staff on learning rather than on mere satisfaction with services; on results rather than number of activities

• Build a culture of inquiry in student affairs• Engage creatively what really matters • Become a partner with academic affairs in the learning

process• Help programs continuously improve

CSU Channel IslandsSacramento State

University of the Pacific

55

What is learning?

For the purposes of this workshop, learning is defined as:

“a comprehensive, holistic, transformative activity that integrates academic learning and student development, processes that have often been considered separate, and even independent of each other. When we say learning then, we do not mean exclusively or primarily academic instruction, the acquisition of disciplinary content, or classroom learning.”

Learning Reconsidered: A Campus-wide Focus on the Student Experience

CSU Channel IslandsSacramento State

University of the Pacific

66

Process for Measuring Learning

1. Identify expected outcomes2. Plan programs and services that accomplish

outcomes3. Implement programs and services4. Assess the learning responsive to the outcomes

through sound data collection and interpretation5. Use results to improve learning opportunities

CSU Channel IslandsSacramento State

University of the Pacific

CSU Channel IslandsSacramento State

University of the Pacific

CSU Channel IslandsSacramento State

University of the Pacific

77

What is assessment?

“Assessment is an ongoing process aimed at understanding and improving student learning.

It involves making our expectations explicit and public;

setting appropriate criteria and high standards for learning quality;

systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards

and using the resulting information to document, explain, and improve performance….

CSU Channel IslandsSacramento State

University of the Pacific

88

Division LevelDivision Level

Department LevelDepartment Level

Program LevelProgram Level

Modes of DeliveryModes of Delivery

Division of Student Affairs, Assessment of Strategic Plan

Housing & Resident Life, Athletics, etc. Department level Reviews

Student Leadership Training, Visual Impaired, etc. Program Review

Workshops, Presentations, Role-plays, and others strategies

WASC Review, OCR Review, etc.Institutional LevelInstitutional Level

Organizational Context

EXAMPLESASSESSMENT LEVELS

CSU Channel IslandsSacramento State

University of the Pacific

9

10

Writing Good Learning Outcomes:

• Be clear about what you are assessing: Am I assessing program learning outcomes or student learning outcomes?

• Clearly state what you want students to know and be able to do.

• Make sure that your outcomes link to mission, values, strategic plan.

• Ensure that your outcome is measurable: Are my outcomes observable or operationally defined?

• Make sure that your outcome is meaningful: Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about (9 principles of good Practice for Assessing Student Learning (AAHE 1998)?

CSU Channel IslandsSacramento State

University of the Pacific

10

Components of Learning Outcomes

• Audience: For whom is the program aimed?

• Behavior: What do you expect the audience to know/be able to do?

• Conditions: Under what conditions or circumstances will the learning occur?

• Degree: How much will be accomplished, how well will the behavior need to be performed, and at what level?

An Example of a Learning Outcome

Students who successfully complete the communications workshop will be able to 1) demonstrate giving and receiving accurate feedback during a role-play exercise and 2) score at least 75% on a feedback survey administered by a facilitator.

12

An Example of a Learning Outcome

Students who successfully complete this Problem Solving Workshop will be able to 1) demonstrate knowledge of effective problem solving using a pencil paper exam, and 2) correctly identify the three elements of problem solving: empathy, invention, and selection.

13

An Example of a Learning Outcome

Roommates who successfully complete this exercise will be able to demonstrate knowledge of their roommate’s salient life experiences, cultural values, and unique likes and dislikes by correctly identifying 80% of the correct answers found on their Roommate’s Characteristics Checklist.

14

Indirect Learning Outcomes

Indirect Learning Outcomes - self-reported statements/ comments that reveal a perceived increase in understanding or appreciation. The perception is not verified through any demonstration of knowledge acquisition or observed behavioral / attitudinal change.

CSU Channel IslandsSacramento State

University of the Pacific

15

Methods to Assess IndirectLearning Outcomes

• Satisfaction surveys • Program / evaluation surveys • Questionnaires• Inventories• Face-book response• Informal peer-to-peer conversations (e.g. with RAs,

orientation leaders)

CSU Channel IslandsSacramento State

University of the Pacific

16

Direct Student Learning Outcomes

Direct Student Learning Outcomes - Demonstrate the abilities, information, knowledge, attitudinal or behavioral changes student can demonstrate after participating in a program or utilizing a service

CSU Channel IslandsSacramento State

University of the Pacific

17

Methods to Assess DirectLearning Outcomes

• Portfolios• Capstones• Performances• Common assignments• Narratives with reflection• Tests• Competency observations• Observations of employee or student behaviors• Juried art exhibits• National licensure examinations• Standardized tests• Work samples

CSU Channel IslandsSacramento State

University of the Pacific

18

Differentiating Direct and Indirect Learning Outcomes

Direct or Indirect?1) T / F This orientation session has helped me understand the

foreign language at Sacramento State.2) Which of the following examples fulfill the foreign language

requirement necessary to graduate from Sacramento State?A) Demonstrated fluency in a language other than EnglishB) Passed the AP foreign language exam with a score of 3 or

higherC) Successfully completed, with C- or better, 3 years of high

school foreign languageD) All of the above

CSU Channel IslandsSacramento State

University of the Pacific

19

Changing Indirect Learning Outcomes to Direct Learning Outcomes

Distribute Orientation Survey Handout

Initiate group activity

CSU Channel IslandsSacramento State

University of the Pacific

20

Direct Student Learning Instrument

Distribute Post-Test Orientation Assessment Handout

CSU Channel IslandsSacramento State

University of the Pacific

21

“Best Advice” Examples

CSU Channel IslandsSacramento State

University of the Pacific

2222

Resources

WASC website http://www.wascweb.orgCAS Standards http://www.cas.edu/gags.htmlKansas Study Website http://Kansasstudy.orgNational CC Benchmark Project http://www.NCCBP.orgPowerful Partnerships: A Shared Responsibility for

Learning http://www.aahe.org/assessment/joint.htmStudent Learning Imperative

http://www.acpa.nche.edu/sli/sli.htmAssociation of Theological Schools http://www.ats.eduLearning Reconsidered, National Association of Student

Personnel Administrators (NASPA) http://www.naspa.org

CSU Channel IslandsSacramento State

University of the Pacific

2323