Multiple Measures Assessment Project: Math & English Models from Phase II

Multiple Measures Pilot College Convening - North

Cañada CollegeTerrence Willett

October 28, 2015

MMAP variables for predicting CC successEnglish• Cumulative HS GPA• AP English class grade• Score on English CST• Grade in last English class

Math• Cumulative HS GPA • Grade in Algebra I,

Algebra II, Statistics, Trigonometry, Calculus

• Taking a challenging CST

• Score on math CST

Why Decision Trees?

• Provide easy-to follow and easy-to-code rule sets• Create discrete categories• No distributional or relational assumptions• Visual presentation of decisions/decision-making• They’re fun!

A B ZZ A BB Z A

A B AB A B

AAA BBB

ZZZ

Increasing Homogeneity with each split

Branch

Internal Node

Leaf Node

Root Node

Statistics Tree – Direct Matriculants

Statistics Tree – NDM

English Decision Rules Direct matriculants Non-direct matriculants

Transfer-level (criterion = 0.70 / 0.70)

HS 11 GPA >= 2.6 HS 12 GPA >= 2.6

One-level below (criterion = 0.65 / 0.65)

HS 11 GPA >= 2.3 HS 12 GPA >= 2.2 &HS 12 English course GPs >= 1.8

Two-levels below (criterion = 0.60 / 0.55)

HS 11 GPA >= 2.0

HS 12 GPA >= 1.8 &HS 12 English course GPs >= 1.1 HS 12 GPA >= 1.8 & CST >= 288

Three-levels below (criterion = 0.55 / 0.55)

HS 11 GPA >=1.4

HS 12 GPA >=1.5 & CST>=268

HS 12 GPA >=1.7

Four-levels below Everyone else Everyone else

Level (CB21 code, criterion)

Hardcoded Pre-Requisite

Direct Matriculants (Up through 11th grade, no CST)

Non-Direct Matriculants With CST

Non-Direct Matriculants Without CST

Pre-Calculus(criterion = 0.70)

Algebra II (or better)

HS GPA>=3.4 HS GPA>=2.6 AND Calculus

HS GPA>=3.3 HS GPA>=3 AND CST>=340

HS GPA>=3.3 HS GPA>=3 AND Calculus C or better

Statistics(criterion=0.70)

Algebra I (or better)

HS_11_GPA >= 3.0 HS_11_GPA >= 2.3 AND Pre-Calculus with C or better

HS_12_GPA >= 3.0 HS_12_GPA >= 3.0

General Education Math(Transfer Level)(criterion=0.70)

Algebra I (or better)

HS GPA >=3.3

HS GPA>=3.2 HS GPA>=3.2ORHS GPA>=2.9 AND Stat C or better

College Algebra(criterion=0.70)

Algebra II (or better)

HS GPA >=3.2 HS GPA >=2.9 ANDPre-Calculus C or better

HS GPA>=3.2 HS GPA>=3.2

Level (CB21 code, criterion)

Hardcoded Pre-Requisite

Direct Matriculants (Up through 11th grade, no CST)

Non-Direct Matriculants With CST

Non-Direct Matriculants Without CST

Calculus I(criterion = 0.70)

Pre-Calculus or Trigonometry (or better)

HS GPA>=3.6 HS GPA>=3.2 AND Pre-Calculus C or better

HS GPA>=3.1 AND Calculus C or better HS GPA>=3.5

HS GPA>=3.1 AND Calculus HS GPA>=3.5

Higher level math rules

Algebra II (CB21=A, criterion=0.65)

HS 11 GPA >=2.8 HS 12 GPA >=2.9 HS 12 GPA >=2.5ANDAlgebra II CST >= 302

HS 12 GPA >=2.9 HS 12 GPA >=2.5 ANDPre-Calculus with C or better

Algebra I (CB21=B, criterion = 0.55)

HS 11 GPA>=2.4 HS 12 GPA>=2.5 HS 12 GPA>=2.0ANDAlgebra I CST >= 302

HS 12 GPA>=2.3ANDAlgebra II C or better

Pre-Algebra (CB21=C, criterion 0.50)

HS 11 GPA>=2.0 HS 12 GPA >=2.1 HS 12 GPA >=1.6ANDAlgebra 1 Cluster 1 (Number Properties, Operations, and Linear Equations) >= 8

HS 12 GPA >=2.1

Arithmetic (CB21=D)

Everyone else Everyone else Everyone else

Basic Skills Math Decision Rules

Correlations with College Grade:11th grade Cumulative GPA vs. Accuplacer

English 11th Grade GPA Accuplacer

Transfer 0.27 0.101 level below 0.24 0.122 levels below 0.25 0.123 levels below 0.18 0.124 levels below 0.21 0.07

Math 11th Grade GPA Accuplacer

Transfer - STEM 0.24 0.19Transfer – Stats 0.31 0.16Transfer – LAM 0.26 0.091 level below 0.28 0.212 levels below 0.26 0.113 levels below 0.23 0.114 levels below 0.19 0.05

English (n=103,510)

Math (n=143,253)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

38%31%

61%

42%

Placement into Transfer-level with Current Placement Systems vs. Strong Multiple Measures (Statewide)

Current Disjunctive MM

Afr Am Latino Asian White0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

24%30%

41%

53%

40%

51%

73% 74%

Transfer Level English Placement

Current Disjunctive MM

Afr Am Latino Asian White0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

15%21%

41%

51%

22%

32%

53%

65%

Transfer Level Math PlacementCurrent Disjunctive MM

How long is High School GPA good for?

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

f(x) = − 0.00763859649122807 x + 0.341964912280702R² = 0.620115928375629

f(x) = − 0.0116122807017544 x + 0.363070175438596R² = 0.833614197361731

Decay function for the predictive utility of HSGPA on English grade

HS 11 GPA Linear (HS 11 GPA) HS 12 GPALinear (HS 12 GPA) Accuplacer

Semesters of delay (approx. 6 months each)Corr

elati

on b

etw

een

HSPG

A an

d 1s

t CC

Engl

ish g

rade

Mallory NewellThe RP Groupnewellmallory@deanza.edu

Terrence WillettThe RP Group twillett@rpgroup.org

Craig HaywardThe RP Groupchayward@rpgroup.org

John HettsEducational Results Partnershipjhetts@edresults.org

Ken SoreyEducational Results Partnershipken@edresults.org

Daniel LamoreeEducational Results Partnershipdlamoree@edresults.org

MMAP Research Team Contacts

http://www.rpgroup.org/projects/multiple-measures-assessment-project

Predicting transfer-level gradesTest Correlation with

GPsCorrected

correlationCTEP Grammar .10 .16CTEP Reading .06 .09CTEP Syntax .08 .12Math Assessment Test .21 .2911th Grade GPA - English .38 .3911th Grade GPA – Math .37 .38

Cost

Old model Acceleration MIH + Acceleration

Semesters 3 2 1

Units 10 6 4

Cost to get all to Engl 1A $1,677,552 $1,084,682 $208,851

Placement at Transfer level 18% 29% 57%