Student Learning Outcomes for Student Services Modesto Junior College January 2005 Sue Granger-Dickson Counselor Bakersfield Community College Terrence

Student Learning Outcomes Student Learning Outcomes for Student Servicesfor Student ServicesModesto Junior College

January 2005

Sue Granger-DicksonCounselor

Bakersfield Community CollegeTerrence Willett

Director of ResearchGavilan Community College

Learning Outcomes for Learning Outcomes for TodayToday

Summarize history of accountability and how new standards came to be

Define learning outcomes and explain the underlying philosophy

Describe the assessment cycle and its role in the college

Write appropriate student learning outcomes for your program

Explain at least two methods to assess at least one student learning outcome including needed resources and potential barriers

Identify at least 3 resources to help you succeed in the learning outcomes challenge

Your expectations and Your expectations and needsneeds

HistoryHistory 1960’s– In the past, year-end reports demonstrated proper

management but not whether the programs directly aided student learning

– Colleges measured enrollments, participation rates, budget expenditures

1990’s– More recently have expanded to measure success,

retention, persistence, graduation, and transfer– Focus has also been on instructional delivery versus

student learning 2000’s

– Student Learning Outcomes Present state of MJC

Standard IIBStandard IIB The institution recruits and admits diverse students who

are able to benefit from its programs, consistent with its mission. Student support services address the identified needs of students and create a supportive learning environment. The entire student pathway through the institutional experience is characterized by a concern for student access, progress, learning, and success. The

institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services.

What are Learning What are Learning Outcomes?Outcomes? “Learning outcomes” not a new concept

– Used in California in K-12, special ed, voc programs– Used in other college systems (i.e. Maryland,

England)– Barr and Tagg called for learning outcomes in 1995

Focuses more on students’ learning rather than just how we instruct or support them; outcome based

Emphasis on higher more complex level of learning

Clearly states educational intentions for students

See example SLO in Appendix E in handout

Sounds like behavioral Sounds like behavioral objectives!objectives!

Learning outcomes differ from behavioral objectives by – Being broadly stated– Having instructors assess gains in skills rather

than creating detailed list of specific topics and abilities to be mastered (Harden 2002)

Some authors do not perceive that the difference between learning outcomes and behavioral objectives is important (Prideaux 2000)

Important to distinguish the two to gain acceptance

6 Principles – O’Banion6 Principles – O’BanionThe Learning College…The Learning College…

…creates substantive change in individual learners …engages learners in the learning process as full

partners who must assume primary responsibility for their own choices

…creates and offers as many options for learning as possible

…assists learners to form and participate in collaborative learning activities

…defines the roles of learning facilitators in response to the needs of the learners

…succeeds only when improved and expanded learning can be documented for learners

Levels of AnalysisLevels of Analysis

InstitutionalDepartment or ProgramClass or ServiceAssignment or Student Interaction

New PerspectiveNew Perspective “We must treat the college as a learning

laboratory” – John Baker, VP SS Gavilan Dealing with whole student-their

uncertainty, response to that uncertainty, and their wisdom and courage-all matters of being (Harden 2002)

Shift from instruction as “core of the college” towards learning as central focus

True marriage between academic and life skills learning

Institutional ConcernsInstitutional Concerns

Students have a fundamental right to services and if these services are not or cannot be related to learning outcomes, their existence could be unfairly jeopardized (Collins 2002)

Uncritical application of business models and concepts to education

Fear all this measuring will be a waste of effort and not enhance student achievement – Luna, Gavilan College Academic Senate

Do students really care about learning? - Borden

Assessment at Bakersfield Assessment at Bakersfield CollegeCollege

2001-Outside audit-impetus to become a learning college

2002-Program & Budget review transformed into IEC

2002-Faculty sent for trainingJuly, 2002-New accreditation

standards

BC Assessment BC Assessment continued…continued…

2003-Campus wide audit– Development of mission statement– Description of how they IMPROVE

student learning– Opening Day workshop-each faculty

member write one SLO


2003-Campus-wide cross disciplinary instruction workshops on writing SLO’s

Academic Senates adopts assessment as one of its goals– Philosophy statement developed– Creation of assessment plan at AAHE– Interviews of 41 concerned faculty

members


2004-Student services SLO Workshops

General Education Committee adopts draft SLO’s

Critical Thinking Committee adopts draft SLO’s

Assessment coordinator selected with reassigned time

Gavilan’s ApproachGavilan’s Approach

Discussion began in Student Services Had SLO presentations at convocation and

meetings of department and program leads as well as attending external workshops

Institutional learning outcomes selected Course level learning outcomes written Currently developing program level

outcomes Next steps include publishing outcomes in

catalog and implementing assessment of outcomes

Gavilan Institutional Gavilan Institutional Learning Outcomes Learning Outcomes

(from Palomar College)(from Palomar College)CommunicationCognitionInformation CompetencySocial InteractionsAesthetic ResponsivenessPersonal Development and

Responsibility

Gavilan Institutional Gavilan Institutional Learning OutcomesLearning Outcomes

Communication– Listening– Reading– Writing

Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes

Cognition– Analysis & Synthesis– Problem Solving– Creative Thinking– Quantitative Reasoning– Transfer of Knowledge & Skills

to a New Context


Information Competency– Research– Technological


Social Interaction– Teamwork– Effective Citizenship


Aesthetic Responsiveness– Differentiate between people who

can sing and people who can’t e.g. Brittney Spears vs. Norah Jones

– Distinguish between art that offends you, art that is cheesy, art you can’t understand, and art that is too expensive


Personal Development and Responsibility– Students will develop individual responsibility, personal

integrity, and respect for diverse people and cultures– Self-management: Students will demonstrate habits of

intellectual exploration, personal responsibility and physical well being

– Ethics and Values: Students will demonstrate an understanding of ethical issues that will enhance their capacity for making sound judgments and decisions

– Respect for Diverse People and Cultures: Students will respect and work with diverse people including those with different cultural and linguistic backgrounds and different abilities

Service →Learning Outcome ↓

Communication

Student understanding and explaining goals/ plans; use advice from counselor

Student discussing and understanding requirements with staff

Student explaining what is desired: registering, transcripts, adding, etc.

CognitionUnderstand GE patterns; Develop ed plan

Calculating income, need, and unit loads

Calculating units and fees

Information Competency

Using web to gather program information

Comprehension of forms and instructions

Comprehension of forms and instructions

Social Interaction

Working with college personnel

Working with college personnel/ mentoring

Working with college personnel/ mentoring

Aesthetic Responsiveness

Form design, web page layout

Form design, web page layout

Form design, catalog, schedule web pages layout

Personal Development

And Responsibility

Understand concept of balancing school, work, and family

Provide accurate information; meet deadlines

Provide accurate information; meet deadlines

Counseling Financial Aid Admissions and Records

Opportunities for Student Opportunities for Student ServicesServices

Support students in taking more responsibility for their own learning = framework for action (Barr and Tagg 1995)

Acknowledges that Student Services personnel are also educators

Your Turn! Your Turn!

Think of an instance you directly observed a student learning while interacting with your program or service outside of a classroom experience

Write a mission statement for your program or service

Writing Learning Writing Learning OutcomesOutcomes

Should be based on our mission and values

In an environment that nurtures creativity and intellectual curiosity, Gavilan College serves the community by providing a high quality learning experience which prepares students for transfer, technical and public service careers, life-long learning, and participation in a diverse global society

Writing Learning Writing Learning OutcomesOutcomesOutcomes must be observable so

you can measure or record themCourse level learning outcomes

sound like behavioral objectives (or really are the same) but LO tie into institutional learning outcomes

Keep in mind that outcomes are tools to guide evidence collection for assessment

Assessment CycleAssessment Cycle

Create or Modify Goals and Outcomes

Implement Programs or Strategies

Evaluate

Draft BC Assessment PlanDraft BC Assessment Plan

Class and Course Assessment Senate Assessment Committee & Curriculum Committee

Program Assessment Program Review

Institutional Assessment Institutional Effectiveness Stu

dent

Service

s &

Learn

ing

Support

Service

s

Assess

ment

Closing the LoopClosing the Loop

Develop, modify, or review a curriculum, course/program.

Develop Student Learning Outcomes

Design & Measure Student Learning as a result of the Curriculum, Course, or Program

Collect, discuss, and analyze data.

Determine refinements based on outcomes data.

Closing the Assessment Loop

Dimensions of EvidenceDimensions of Evidence

Quantitative or qualitative– Not everything that can be counted counts and not

everything that counts can be counted -Einstein Direct or indirect Norm- or criterion-referenced Should be representative and relevant Need several pieces of evidence to point to a

conclusion– e.g. Student complains of fever and aches, their

temperature is 102º F, tonsils are not inflamed, eyes are red and irritated, posture appears weak. Notice mix of types of evidence that all point to same conclusion…flu!

Common Assessment Common Assessment MethodsMethods Tests

– Locally developed or Standardized Performances

– Recital, Presentation, or Demonstration Cumulative

– Portfolios, Capstone Projects Surveys

– Attitudes and perceptions of students, staff, employers Database Tracked Academic Behavior

– Grades, Graduation, Lab Usage, Persistence Embedded Assessment

– Using grading process to measure ILO Narrative

– Staff and student journals, interviews, focus groups

Example Method

Randomly assign

students to service

Randomly assign

students to receive

information about services

Correlate use with

performance or skills

measures

SurveysCase study and

journals

Strength of Evidence

Can claim causality

Weaker causality claim

Causality cannot be claimed, useful in conjunction with other information

Causality cannot be claimed, useful in conjunction with other information

Causality cannot be claimed but complex and difficult to measure effects can be noted

Ethical Consideration

Denies access to a service that may or may not be effective for some students

All students have access but some receive less information

No restriction of access or information

Use student time to complete survey

Confidentiality

Embedded Assessment Embedded Assessment ExampleExampleCriteria Joe Jane Liam Celeste Avg

Arithmetic with fractions

3 4 5 4 4

Calculating basic areas

3 4 3 5 3.75

Order of operations

3 4 5 5 4.25

Right triangles

3 4 4 4 3.75

Total 12 16 17 18

Student Grade

C B B A

Down for grades, across for outcomes assessment

–after Nichols

Assessment TipsAssessment Tips Collect data from a representative sample

rather than everyone in population Collect only a few well chosen pieces of

data Have assessment stem from activities you

already do if possible Team up with others to share assessment

work Design activities and outcomes with

assessment in mind Ask for advice from your colleagues -

especially vocational faculty who relate to learning outcomes as core competencies

Writing Learning Writing Learning OutcomesOutcomes Good verbs (Blooms’ Taxonomy):

– Compile Arrange Classify– Analyze Identify Operate– Design Solve Write– Apply Differentiate Calculate– Demonstrate Formulate Compose– Explain Predict Assess– Compare Estimate Critique

Bad verbs– Know Understand Appreciate Learn

Good verbs are clear and observable, bad verbs are vague states of mind

Example Learning Example Learning Outcomes at Outcomes at

program/service levelprogram/service level Disabled Student Services

– The student will be able to explain his/her individual academic strengths and weaknesses

Maryland Health Education – Students will demonstrate an understanding

of health promotion and disease prevention concepts to establish a foundation for leading healthy, productive lives.

Admissions and Records– Students can “work the system”– Demonstrate patience while waiting in line

More Student Service More Student Service ExamplesExamples

From Ventura College SLO Workshop– DSPS = Students will demonstrate self-

advocacy skills with instructors and staff– EOPS = Student will identify and describe

resources available on the campus. – Counseling = Student will be able to state

informed academic goal(s). – Financial Aid = Faculty and staff will be able to

describe basic aspects of financial aid available to students and how students can apply for financial aid.

Gavilan Counseling SLO’sGavilan Counseling SLO’s

Students will identify college resources, procedures, and policies that support their academic success. (Cognition)– Students will complete the college orientation and score

at least an 80% on the orientation quiz. Students will use technology to gather, process,

and articulate career options and choices. (Info Comp)– Students referred to the Eureka system (online/center)

will complete the assessments and discuss outcomes with counselor. (track completion on SARS)

– Students enrolled in Guidance 1 will utilize the internet for research and declare a major at the end of the course.

Instructional Program Level Instructional Program Level SLO ExamplesSLO Examples

Sinclair College– Math = Apply mathematical models

to real world problems– Amer. Gov. = Communicate the

significance of facts, concepts, and ideas in spoken and written English, which is clear, precise, and logical

– ECE = Demonstrate professionalism in the child care setting

More Instructional PSLOMore Instructional PSLO

Parkland Community College– English = Students will create college-level

written text for multiple purposes and diverse audiences that demonstrate depth of critical thought and that observe the conventions of standard written English.

– Psychology = Our students will apply major theoretical perspectives and methodological approaches used in the discipline of psychology to particular social phenomena or autobiographical circumstances.

Your Turn! Your Turn!

Write at least 2 learning outcomes for your program and at least 2 assessment strategies for each outcome

Reflect on Your SLO’s:Reflect on Your SLO’s:

Will the student understand? Is this truly an overarching outcome? Can you blend? Does this take into consideration the

other areas your program interfaces with?

If you had only 6 outcomes for your program, is this one of them?

In reality, can this be done?

Questions to ContemplateQuestions to Contemplate

Why should I use student learning outcomes for evaluating my program?

What are some strategies for writing effective learning outcomes?

What are some differences between qualitative and a quantitative measurements?

How can anecdotal reports or case studies be used in program assessment?

What are the pros and cons of using sampling methods for assessment?

Remember:Remember:Everyone is a

learner when it comes to

assessment

Creating Venues for Creating Venues for DialogueDialogue

Invite outside speakersUse department/committee

meetingsHold lunch meetingsSend people for training to create

championsHold cross disciplinary workshopsConduct campus-wide workshops

AcknowledgementsAcknowledgements Gavilan Student Services Division

especially:– John Baker, Margery Regalado, Jane Maringer

Gavilan Instructional Division especially:– Marty Johnson– Marlene Dwyer & her Curriculum Committee– Department Chairs– Academic Senate, Enrique Luna

Research and Planning Group especially:– Harriett Robles, Brad Philips, Fred Trapp, Bob

Gabriner, Jerry Rudman WASC

– Darlene Pacheco, Barbara Beno

Thank YouThank You

Documents

Student Learning Outcomes for Student Services Modesto Junior College January 2005 Sue Granger-Dickson Counselor Bakersfield Community College Terrence