Presentation Outline

• The Future of Institutional Research (IR) & Technology in improving first-year students’ success.

• Example 1: Demonstration of an IR innovation.

• Example 2: Demonstration of a Technology innovation.

The Future of IR and Technology

• IR’s future is moving beyond reporting to analysis. This means converting data into ‘actionable’ information that FYE personnel can use.

• Technology’s future is moving beyond data management to production of tools that directly facilitate and improve student success.

Example 1:Student-at-Risk Prediction Model

• Also known as a predictive model, or enrollment forecasting model.

• Helps answer questions like:– Which student variables are most useful for

predicting freshmen retention?– What is the “best” combination of variables to

optimize predictions?– How useful is this combination for identifying at-

risk students?

Relevant Previous ResearchAstin, A. W. (1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass.

Bean, J. P. (1985). Interaction effects based on class level in an explanatory model of college student dropout syndrome. American Educational Research Journal, 22(1), 35–64.

Caison, A. L. (2006). Analysis of institutionally specific retention research: A comparison between survey and institutional database methods. Research in Higher Education, 48(4), 435-451.

Herzog, S. (2006). Estimating student retention and degree-completion time. Decision trees and neural networks vis-à-vis regression. New Directions for Institutional Research, 131, 17-33.

Pascarella, E., and Terenzini, P. (2005). How College Affects Student: Volume 2, A Third Decade of Research. San Francisco: Jossey-Bass.

Sujitparapitaya, S. (2006). Considering student mobility in retention outcomes. New Directions for Institutional Research, 131, 35-51.

Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89-125.

4 Steps to Modeling Retention

1. Get Freshmen

Data.

(i.e. We used fall 2009 & 2010 data

to build our “training” data

set.)

4. Check the actual 2011 retention outcomes to see how well the model performed.

2. Build Model. 3. Apply model

parameters to new data.

(i.e. model validation, scoring)

RETENTION

Examples of Student Variables AnalyzedGenderAgeEthnicityResidencyGeographic Origin

High School GPA & RankSATAP CLEPEducational GoalsTransfer GPA# Transfer Credits

MajorCredit LoadCredits Earned First Term GPADistance EducationDual EnrollmentHigh Failure Rate CoursesCourses Taken (including Math & English)

On Campus EmploymentHousingStudent Life ActivitiesAthleticsSTAR UsageAverage Class Size

Need Based AidNon-need Based AidPell GrantWork Study% of Aid Met

Ethnicity by Geographic OriginEmployment by HousingHigh School GPA by First Term GPAResidency by Need Based AidRatio of Successful Adds to Drops

PERSISTENCE

DEMOGRAPHICS

ACADEMIC

CAMPUS EXPERIENCE

FINANCIAL NEED

INTERACTIONS

Credits earnedCredits attemptedCredit Completion RatioMath/English Enrollment/CompletionContinuous EnrollmentMilestone metrics

MILE-STONES

PRE-COLLEGE

Continental US

High School GPA

% Need Met

Educational Goals

AP/CLEP Credit

FYE Class

15 Credits

On Campus Work

RETENTION IN YEAR 1

Strongest

Weakest

These variables account for approximately 39% of the variance in a student’s likelihood of returning for a third semester (Pseudo R Square = .387).

*Wald statistic (sig.)The Wald test statistic was used to indicate strength of the variable instead of the coefficient, standardized beta. Because of the nature of the logistic regression, the coefficient is not easily interpretable to indicate strength.

262.804 (.000)*101.368

(.000)*

7.817 (.005)*

20.292 (.000)*13.486 (.000)*

7.134 (.008)*

4.419

(.036)*

3.791

(.052)*

7 Strongest Predictors of Retention

Predictors in Regression Equation

B S.E. Wald df Sig. Exp(B)

Step 1a

ED GOALS .575 .157 13.486 1 .000 1.778

HS GPA 1.761 .175 101.368 1 .000 5.817

CONTINENTALUS -2.544 .157 262.804 1 .000 .079

FYE CLASS .393 .147 7.134 1 .008 1.481

FIN NEED MET 1.021 .227 20.292 1 .000 2.777

ON CAMPUS WORK .411 .211 3.791 1 .052 1.508

FIFTEENCREDITS .267 .127 4.419 1 .036 1.306

AP/CLEP .453 .162 7.817 1 .005 1.573

Constant -6.623 .628 111.200 1 .000 .001a. Variable(s) entered on step 1: EDGOALS, HSGPA, MAINLAND, CAS110, FINNEED, EMPLOY, FIFTEENCREDITS, APCLEP.

Pseudo Rsquare = .387

• Scoring of relative dropout/retention risk

p = exp(a+b1x

1+b

2x

2+b

3x

3+b

4x

4….)

1 + exp(a+b1x

1+b

2x

2+b

3x

3+b

4x

4….)

Where: p = probability of enrollment/non-enrollment exp = base of natural logarithms (~ 2.72)

a = constant/intercept of the equationb = coefficient of predictors (parameter

estimates)

Scoring Students

• John: – is from the continental U.S. (0)– has a below average high school GPA (2.65)– is enrolled in 9 credits (9)– has a low % of financial need met (.45)– isn’t not working on campus (0)– isn’t enrolled in CAS 110 (0)– didn’t specify any educational goals in survey (0)

• Probability of Dropping: 0.77

Example: John is at risk of dropping

Sample Data for FYE Advisors

UH ID AGE GENDER ETHNICITY COLLEGE DEPT MAJOREd Goal

SpecifiedRelative Risk

ValueRisk Level

 001 18 F CH CA&H ART BA Yes 14.92 LOW

 002 18 F HW CSS SOC BA Yes 36.88 MEDIUM

 003 19 M AA CENG EE BS No 89.18 HIGH

UH IDLAST NAME

FIRST NAME

EMAILCURRENT CREDITS

RESIDENTAP/

CLEP

HS GPA

WORK ON

CAMP

1st YR EXP

CLASS

% FIN NEED MET

STAR LOGINS

ADVISOR PREVIOUS CONTACT

 001       12 HI 6 3.80 Y Y 77% 0 Y

 002       15 HI 0 3.13 N N 43% 3 N

 003       16 CA 6 2.59 Y Y 65% 2 N

• 407 freshmen from 2011 dropped out in year one.

• Retaining just 22 students from 2011 would have improved Mānoa’s overall retention rate from 78.8% to 80%.

• Additional Revenue from Tuition and Fees = $210,000 (for 16 HI, 6 WUE, excludes out-of-state!).

• Are there 22 students in this group that we can help/retain?

Impact on Campus

Gary RodwellDirector of Advanced Technology &

Lead Architect of ‘STAR’University of Hawaii at Manoa

Example 2: ‘STAR’ Technology

Essential engagement

Essential engagement

Reed DasenbrockVice Chancellor for Academic Affairs

John StanleyInstitutional Analyst

Gary RodwellDirector of Advanced Technology

University of Hawaii at Manoa

Questions: miro@hawaii.edu

Mahalo