• •

Completing Learning Support:

Using Predictive Modeling to Determine Best Practices

Gregory SchutzRion McDonald

Chris Tingle

TASSR Fall ConferenceOctober 24, 2013

Successful academic entry is a necessary

component for degree attainment.

The Complete College Tennessee Act mandated that public higher education institutions in Tennessee improve student success.

Meanwhile, approximately two-thirds of incoming freshmen at Tennessee Board of Regents institutions required some sort of pre-college learning support in fall 2011.

Student abilities in math, reading, and

writing skills impact successful entry.

The TBR adopted new learning support (LS-precollege) guidelines with the goal that all students be able to move through LS in a timely manner.

Holding demographic and academic preparedness constant, the authors find one-year, LS completion rates differ by institution.

Presentation Flow

Purpose Background Findings Recommendations

What Matters?

Introduce evaluation data being collected a the Tennessee Board of Regents (TBR) for their Learning Support (developmental education) initiative.

Illustrate a predictive model that includes academic preparation, demographics, and institutional components for completing LS.

Provide a method and tools for researching the impact of Learning Support (developmental education)

Discuss implications on best implementation practices and future research.

For this Study Completion of LS is defined as:

Completion of all learning support competencies within the first semester or first year of enrollment.

Outcomes for Learning Support Math

* No significant difference

Status Recom. MathComplete All

LS MathRetain Fall GPA 3 or Over

24+ Hours (12 PT)

Full-time No 69% 42% 64%

Full-time Yes No 37% 9% 13%

Full-time Yes Yes 74% 32% 60%Part-time No 48% 27% 35%

Part-time Yes No 27% 16% 13%

Part-time Yes Yes 81% 54% 73%

Status Recom. MathComplete All

LS MathRetain Fall GPA 3 or Over

24+ Hours (12 PT)

Full-time No 57% 37% 45%

Full-time Yes No 26% 9% 6%

Full-time Yes Yes 65% 35% 37%Part-time No 39% 36% 39%

Part-time Yes No 23% 17% 15%

Part-time Yes Yes 58% 47% 59%

Universities

Community Colleges

Outcomes for Learning Support Writing

* No significant difference

Status Recom. WriteComplete All

LS WriteRetain Fall GPA 3 or Over

24+ Hours (12 PT)

Full-time No 71% 43% 64%

Full-time Yes No 61% 22% 43%

Full-time Yes Yes 71% 26% 54%Part-time No 40% 24% 32%

Part-time Yes No 44% 29% 32%

Part-time Yes Yes 75% 54% 71%

Status Recom. WriteComplete All

LS WriteRetain Fall GPA 3 or Over

24+ Hours (12 PT)

Full-time No 53% 35% 40%

Full-time Yes No 32% 12% 11%

Full-time Yes Yes 63% 31% 31%Part-time No 35% 34% 32%

Part-time Yes No 24% 16% 18%

Part-time Yes Yes 59% 41% 60%

Universities

Community Colleges

Outcomes for Learning Support Reading

Status Recom. ReadComplete All

LS ReadRetain Fall GPA 3 or Over

24+ Hours (12 PT)

Full-time No 68% 41% 62%

Full-time Yes No 55% 21% 35%

Full-time Yes Yes 71% 26% 54%Part-time No 40% 27% 27%

Part-time Yes No 33% 12% 20%

Part-time Yes Yes 88% 25% 63%

Status Recom. ReadComplete All

LS ReadRetain Fall GPA 3 or Over

24+ Hours (12 PT)

Full-time No 51% 33% 38%

Full-time Yes No 32% 13% 13%

Full-time Yes Yes 63% 27% 30%Part-time No 33% 33% 31%

Part-time Yes No 24% 15% 17%

Part-time Yes Yes 55% 36% 59%

Universities

Community Colleges

* No significant difference

Literature Review: Student Background

Part-time students are significantly less likely to complete the learning support program.

Adult students and under-represented minority students are both more likely to need remediation and also less likely to complete all required learning support.

The lower the high school grade point average, the less likely a student is to complete all required learning support.

Students that are required to take more learning support are less likely to complete the learning support curriculum.

Literature Review: Best Practices

Mainstreaming learning support students into college level courses while providing supplemental instruction.o Immediate credit accumulation, increased retention.o May not work for high need students.

Condensing semester long learning support courses into shorter timeframes and stacking courses for quickest completion.o Reduced withdrawals from learning support courses.

Modularizing learning support competencies into specific skills in order to reduce a student’s needed learning support.o Increased success of first college level course and increased retention.

Increasing student support, especially additional advising or tutoring, positively affects learning support outcomes.

Learning Support Guidelines

All students must meet ACT college benchmarks or be diagnosed for and placed into appropriate learning support.

Institutions will design learning support curriculum so that full-time students can satisfy pre-college level requirements in one semester.

Institutions must structure learning support so that a student who has demonstrated mastery of a competency will not be required to repeat support in that area.

Delivery of learning support must be based on proven methods of integrating technology as a tool for instruction.

Universities will not award credit that is less than college level.

Learning Support Study

http://www.highereducation.org/crosstalk/ct0105/news0105-virginia.shtml (retrieved 8/3/13)

Logistic Regression Model

Log (Odds of Completion)=constant + coefficient(ACT Score) + coefficient(HS GPA) +coefficient(# Competencies Required) +coefficient(Attendance Status) + coefficient(Age) +coefficient(Minority Status) +coefficient(School1) + coefficient(School2) + coefficent(School3) +….etc. +error

Probability of CompletionOdds of Completion =

(1 –Probability of Completion)

Math Model: Student-Related Factors

* ceteris paribus

Math Model: School Effects

Change in Completion Odds Per

School*

School First-Term First-Year

Univ1 415% 276%

CommColl1 189% 55%

Univ2 31% -21%

Univ4 insignificant 31%

CommColl3 insignificant 23%

Negative Effects -35% to -77%

-20% to -46%

* deviation from average, ceteris paribus

Math Model: Predicted Probabilities

Probability of Completion for

Average Student*

School First-Term First-Year

Univ1 58% 76%

CommColl1 44% 56%

Univ2 26% 39%

CommColl2 26% 51%

Univ4 25% 52%

Bottom Eight Schools

6% to 20% 31% to 42%

* constant student-related factors used across schools

Writing Model: Student-Related Factors

Change in Completion Odds Per Unit Increase*

VariableFirst-Term

First-Year

ACT English 13% 7%

HS GPA (.5 increase) 46% 58%

1 vs. 2 Competencies

711% 242%

FT vs. PT 63% 86%

Age 3% 4%

Non-White vs. White -24% -24%

* ceteris paribus

Writing Model: School Effects

Change in Completion Odds Per

School*

School First-Term First-Year

Univ1 953% 443%

CommColl11 251% 84%

CommColl1 213% 49%

CommColl4 162% 49%

Negative Effects -24% to -68%

-20% to -64%

* deviation from average, ceteris paribus

Writing Model: Predicted Probabilities

Probability of Completion for

Average Student*

School First-Term First-Year

Univ1 76% 83%

CommColl11 51% 63%

CommColl1 48% 58%

CommColl4 44% 58%

Schools 5 through 8

18% to 29%

46% to 49%

Bottom Eight Schools

9% to 17% 20% to 39%

* constant student-related factors used across schools

Reading Model: Student-Related Factors

Change in Completion Odds Per Unit Increase*

VariableFirst-Term

First-Year

ACT Reading 7% 7%

HS GPA (.5 increase)

48% 59%

1 vs. 2 Competencies

646% 208%

FT vs. PT 65% 75%

Age 2% 2%

Non-White vs. White

-15% insignificant

* ceteris paribus

Reading Model: School Effects

Change in Completion Odds Per

School*

School First-Term First-Year

CommColl11 411% 130%

Univ1 297% 180%

Univ3 292% 101%

CommColl1 71% insignificant

CommColl4 41% insignificant

Negative Effects -39% to -60%

-36% to -48%

* deviation from average, ceteris paribus

Reading Model: Predicted Probabilities

Probability of Completion for

Average Student*

School First-Term First-Year

CommColl11 65% 68%

Univ1 59% 72%

Univ3 58% 65%

CommColl1 38% 47%

CommColl9 16% 53%

Bottom Eight Schools

13% to 23%

33% to 44%

* constant student-related factors used across schools

The data implies a set of recommendations:

The ultimate aim of graduation and the near future goal of completing entry level math, writing and reading courses need to be evaluated further to confirm the strength of completing learning support as a leading indicator.

Continue to follow if one-semester completion rates impact college success more strongly than one-year completion rates.

The literature implies another set of recommendations:

Continue to investigate mainstreaming learning support students into college level courses while studying where the level of unpreparedness may make the method unfeasible.

Look more closely at age to see how age is impacting learning support completion.

Pilot services like learning support to first-time college students just meeting TBR college readiness goals.

Target LS delivery and services for students of different backgrounds, demographics, and majors.

Institution Best Practices

Institution should commit to finishing in one semester.

Student and instructor interactions. Facilitate LS competency completion as a

prerequisite or corequisite for college level courses. Create a data driven decision-making culture. Coordinate LS functions on campus. Maintain a faculty-led classroom while implementing

new technology. Require professional development for full-time and

part-time faculty. Promote use of full-time faculty in the classroom.

Conclusion

TBR has changed policy and practice to meet literature recommendations on college readiness and initial indications show improved results for institutions implementing these policies.

The completion of learning support in a timely fashion is a leading indicator for underprepared first-time college students.

The variation of success in completing learning support across campuses suggests that best practice institutions can be identified by results.

The success of learning support with underprepared students suggests that the process of identifying and delivering competency levels could be piloted for other university and college populations.

TBR can use system level data sources to help campuses with their 2015-2020 strategic planning and for evaluating the possible impact on college completion (trajectories).

• •

Gregory Schutz – Greg.Schutz@tbr.eduRion McDonald –

RMcDonald11@columbiastate.eduChris Tingle – Chris.Tingle@tbr.edu

TASSR Fall ConferenceOctober 24, 2013

Completing Learning Support:

Using Predictive Modeling to Determine Best Practices