Overview of this Session

In Search of New SLO Measures in Student Services: An In Search of New SLO Measures in Student Services: An Exploratory StudyExploratory Study

Presented byPresented byJerry Rudmann, PhD and Shañon Gonzalez, MA

Coastline Community College

Strengthening Student SuccessAssessment Conference

October 2007

Overview of this Session

Procedure, findings, conclusions and recommendations from a recent exploratory study.

Purpose of the Study

The study was designed to explore whether assessment tools used to measure cognitive variables -- e.g., goal clarity, self-efficacy -- could serve as learning outcome measures in Student Services.

The Spark for this Study

The need for truly appropriate and really useful assessment measures in Student Services.

Ideas generated by interviews with counselors.

Possible Relationships

Attributes of New Students•Academic Self-Efficacy

•Self-Regulation

•Goal Clarity (academic and career)

•Self-Regulation

•Hope

•Optimistic/Pessimistic Explanatory Style

Students’ Academic OutcomesShort term outcomes

•Semester GPA

•Units earned

•% units completed

•Return next semester

•Long range outcomes

•GPA

•Units earned

•Certificate, degree, and/or transfer


Attributes of New Students•Academic Self-Efficacy

•Self-Regulation


•Self-Regulation

•Hope


Students’ Academic OutcomesShort term outcomes

•Semester GPA

•Units earned




•GPA

•Units earned


Student Services

College success courses

Academic counseling

Career Center presentations

Career counseling

Career course

Club, team, chorus, band, student government or other form of social connectedness

Formal and informal recognition for progress

Non-academic counseling

Transfer Center programs

Peer advisors

Tutoring center

University tours


Student’s Attributes•Academic Self-Efficacy

•Self-Regulation


•Self-Regulation

•Hope


Student’s Academic OutcomesShort term outcomes

•Semester GPA

•Units earned




•GPA

•Units earned


Student Services

College success courses

Academic counseling

Career Center presentations

Career counseling

Career course

Club, team, chorus, band, student government or other form of social connectedness

Formal and informal recognition for progress

Non-academic counseling

Transfer Center programs

Peer advisors

Tutoring center

University tours

Procedure Counselor interviews (preliminary

brainstorming) Literature survey for promising

assessments tools Recruitment presentations at Region 8

DSPS and EOPS meetings Website created having all

assessments online

Study Website

Measures We Tried

Academic and Career Goal Clarity Academic Self-Efficacy Dispositional Hope Self-Regulation Optimism Positive Affect Negative Affect

Data Collection Seven colleges participated in the DSPS version Six colleges participated in the EOPS version Pre-test – start of fall 2006 semester

• College code• ID code• 7 different assessments• Demographic information (age, gender, ethnicity, etc.)

Post-test – end of fall 2006 semester• College code• ID code• 7 different assessments• Services used during the semester

Short term academic outcomes for all students – spring 2007• Semester GPA• Units earned • Percentage of units earned in reference to units attempted

Participation

DSPS study• Seven colleges

• Students• Pre-test - 142

• Post-test -127

EOPS study• Six colleges

• Students• Pre-test - 276

• Post-test - 154

Descriptive Statistics from the DSPS Study

Intercorrelations Among Scales and Academic Outcomes (DSPS study)

Predicting Academic Outcomes (Based on Correlation Matrix and Stepwise Regression Analyses)

GPA % Units Earned

DSPS Study Self-RegulationAcademic Self- Efficacy

Academic Self- EfficacySelf-RegulationAcademic and Career Goal Clarity

EOPS Study Academic Self-EfficacySelf-Regulation

Academic Self-EfficacySelf-RegulationHopeAcademic and Career Goal Clarity

Impact of Services on Student Outcomes

STUDY Service Outcome

DSPS Academic OR Career Counseling

GPA

% of Units Earned

EOPS Transfer Assistance Goal Clarity Gain

Peer Advisement Goal Clarity Gain

DSPS Study Self Regulation, Receipt of Counseling, and Semester GPA

1

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

Low High

Level of Initial Self-Regulation

Se

me

ste

r G

PA

Yes-Received Academic orCareer Counseling Services

No-Did Not Receive Academicor Career Counseling Services

Low SR - Yes Counseling = 32Low SR - No Counseling = 31High SR - Yes Counseling = 20High SR - No Counseling = 20(N = 103)

DSPS Study Self Regulation, Receipt of Counseling, and Percentage of Units Earned for Units Attempted

35

40

45

50

55

60

65

70

75

80

85

Low High


% U

nit

s E

arn

ed

of

Un

its

Att

em

pte

d

Yes-Received Academic orCareer Counseling Services

No-Did Not Receive Academicor Career Counseling Services

Low SR - Yes Counseling = 32Low SR - No Counseling = 32High SR - Yes Counseling = 20High SR - No Counseling = 20(N = 103)

EOPS Study Self Regulation, Peer Advising, and Semester GPA

2.25

2.35

2.45

2.55

2.65

2.75

2.85

2.95

3.05

3.15

Low High


Se

me

ste

r G

PA

Yes - Received Peer Advising

No - Did Not Receive PeerAdvising

Low SR -Yes Peer Adv = 17Low SR - No Peer Adv = 51High SR -Yes Peer Adv = 15High SR - No Peer Adv = 55(N = 138)

Academic and Career Goal Clarity correlated with units earned (both studies). EOPS students who were low on goal clarity at the start of the semester but who reporting receiving transfer assistancetransfer assistance, had significantly high pre to post-test gains in goal clarity.

-10

-5

0

5

10

15

20

25

Low High

Level of Initial Goal Clarity

Ch

ang

es i

n G

oal

Cla

rity

Ove

r th

e S

emes

ter

No Transfer Assistance

Yes--Received TransferAssistance

Academic and Career Goal Clarity correlated with units earned (both studies). EOPS students who were low on goal clarity at the start of the semester but who reporting receiving peer advisementpeer advisement, had significantly high pre to post-test gains in goal clarity.

-10

-5

0

5

10

15

20

25

Low High

Level of Initial Goal Clarity

Ch

ang

es i

n G

oal

Cla

rity

Ove

r th

e S

emes

ter

No Peer Advisement

Yes--Received PeerAdvisement

Limitations of Study

Lack of random selection and assignment to treatments

Self-selection bias Results are correlational, not causal Data are an aggregate from the

participating colleges, but there may be significant differences among colleges, procedures, services, personnel, etc.

Recommendations

Use these instruments• Inexpensive, easy to complete and score• Can help identify “at risk” students• Can help formulate appropriate ways to assist

students• Gain scores derived from pre to post-test

assessments can be useful• Instruments can serve as SLO assessment

instruments that “fit” the services provided by many functions within Student Services

Recommendations

“Map” your services to the constructs measured by these instruments.

Develop new interventions where none currently exist.

Create an assessment referral system.

To SummarizeHere are Suggested SLO Measures

Academic and Career Goal ClarityAcademic and Career Goal Clarity (Tucker & Rudmann, 2006)

Academic Self-Efficacy Academic Self-Efficacy (Chemers, Hu, & Garcia, 2001) Confidence in reaching positive academic outcomes

Efficacy for Self-Regulated Learning Efficacy for Self-Regulated Learning (Zimmerman, Bandura, & Marinez-Pons, 1992)Confidence is managing and regulating academic tasks encountered in college

Hope Hope (Shorey & Snyder, 2004)

HHope that one will achieve short and long term academic goals. “Hope” involves goals, pathways and agency (i.e., having clear goals, with alternative pathways for reaching those goals, and agency as the motivation to strive for one’s goals)

Current / Potential Services for Enhancing These SLO Domains

SLO Domains

What Now Do or Could Do Increase

Low Scores Assessment Tool

Academic Self-Efficacy

Recognize & acknowledge little successes

Program students for success

Efficacy scale

Self-Regulation College success course

Academic counseling

Peer mentors / tutors

Self-regulation scale

Academic and Career Goal Clarity

Career Counseling

Career Center programs

Careers Course

SEP for all students

Goal clarity scale

Research Team

Jerry Rudmann, PhD – Formally at Coastline College, now at Irvine Valley

[email protected] Kari Tucker, PhD – Irvine Valley College

(Psychologist)

[email protected] Shañon Gonzalez, MA – Coastline College

[email protected]

Academic Self-Efficacy(Albert Bandura)

Beliefs about one’s capabilities to learn or perform at designated levels.

Compared with students who doubt their learning capabilities, those who feel efficacious for learning or performing a task participate more readily, work harder, persist longer when they encounter difficulties, and achieve at a high level.

Much research shows that self-efficacy influences academic motivation, learning, and achievement.

Addendum

Four Sources of Efficacy Beliefs

Mastery experiences -- Outcomes interpreted as successful raise efficacy, those interpreted as failures lower efficacy

Vicariously – success or failure of models Verbal persuasions by others – positive or negative

appraisals by others Physiological states (e.g., anxiety, stress, arousal,

fatigue, mood) act as information about efficacy beliefs and can raise or lower efficacy

Hope for Academic Success(CR Snyder)

The three components of the HOPE model:

Goals – that which we want to have happen Pathways – routes to get to the goal Agency – motivation to work toward goal

Hope for Academic Success(CR Snyder)

Hope scores reliably predict academic success:• GPA• graduation rate• lower drop rates

High Hope versus Low Hope students:• More readily find multiple pathways• Are more motivated to pursue goals• Can more readily articulate their goals• Less sidetracked by self-deprecatory thinking• Less counterproductive negative emotions (e.g., stress,

test-taking anxiety)

Optimism (Seligman)

Explanatory style• Optimists and pessimists differ in how they

interpret both positive and negative events

• Pessimists have tendency to underachieve and to become depressed

Documents

Overview of this Session