Effects of Math Tutoring - Cabrillo College · 19.25% of under-represented minority (URM) students used tutoring for intermediate algebra while 23.95% of non-URM students used tutoring

Planning & Research Office Report 20150185

1

Requestor: Math Department

Researcher(s): Steve Blohm and Terrence Willett

Date: 6/30/15

Title: Effects of Math Tutoring

Effects of Math Tutoring Introduction

The purpose of this study is to measure the effects of math tutoring at Cabrillo College. This is an

observational study in which we looked at student grades, success rates, and completion rates for

students who received tutoring and those who did not in all math classes in which at least one student

was tutored.

From a research point of view, it would have been better to use an experiment where we could examine

the effects of tutoring while holding other variables constant. In order to perform an experiment to

measure the effect of math tutoring, we would need to randomly select students to be tutored from a

pool of students who seek tutoring, while denying other students tutoring. However, due to practical,

ethical, and regulatory concerns about denying tutoring students to students, an experimental approach

was not possible. The direct comparisons between those receiving tutoring and those not receiving

tutoring while useful are subject to self-selection bias. To help control for bias from students self-

selecting to participate in tutoring, regression and propensity score matching (PSM) techniques were

employed to equate participants and non-participants on a variety of background variables. The results

are of this study are consistent with the belief that tutoring is helpful for students.

Sample

The sample analyzed consists of students who used tutoring at the Math Learning Center (MLC) from

summer 2009 through summer 2013 along with those who were enrolled in a course where at least one

of the students used tutoring. The MLC is overseen by a math faculty member and provides free, drop in

math tutoring for Cabrillo students in a study hall setting including closed study rooms and computer

stations1. Math tutors go through a training class with regular workshops to provide general tutoring

skills and to ensure students with learning disabilities and other special needs can be accommodated.

The MLC allows students to check out math textbooks, graphing and scientific calculators, laptops,

1 https://www.cabrillo.edu/services/mlc/

https://www.cabrillo.edu/services/mlc/


2

software, audio visual materials, and “learning manipulatives” such as blocks to aid understanding of

fractions and proportions.

We looked at six courses at Cabrillo where enrollment and tutoring use were both relatively high

compared to other courses. The courses we looked at were Statistics (Math 12), Intermediate Algebra

(Math 152), Basic Algebra (Math 154), Essential Mathematics (Math 254A), Pre-calculus (Math 4), and

Calculus (Math 5A).

Summary Findings

Many students at Cabrillo have had difficulty succeeding in Intermediate Algebra. We look in detail at

the relationship between hours tutored and success in this course while controlling for other factors

that are predictive of success. We came up with an equation that indicates the number of tutoring hours

that are required on average for a particular probability of success for a given course given certain

demographic indicators. While our study indicated that students in general will have a greater chance of

success if they receive tutoring, using this equation we can come up with some concrete numbers. For

example, to have at least a 75% chance of success in intermediate algebra, an underrepresented

minority (URM) male would need, on average, 29.5 hours of tutoring in a semester or about two hours

of tutoring per week. A URM female would need 23.24 hours of tutoring for the same probability of

success.

Detailed Findings

The following figure looks at the success rates in each class for these three groups

1. Students who did not log tutoring time or log time at the MLC - Nothing

2. Students who used the MLC, but did not use tutoring – MLC Only

3. Student who used some tutoring – Tutoring


3

Figure 1: Percentage of Math Students Earning a C or Better

In every case, those who were tutored outperform those who were not. The sample sizes were large so

even small difference would show up as significant. However, in many cases the differences are

substantial. For example, pre-calculus students who were tutored succeed 58% of the time and those

who were not succeeded 48% of the time. Below we will investigate the effect of tutoring among

specific among different gender, ethnic groups, and other special populations.


4

Figure 2: Percentage of Math Students Earning a C or Better by Gender

Both female and male students perform better with tutoring than without. Female students tend to

outperform male students in all cases except for those not tutored in Calculus I. In every case other than

Math 154, Elementary Algebra, the higher success rate for female tutored students was statistically

significant at the 0.05 level of significance. For male students the higher success rate for tutored

students was statistically significant in all cases other than Math 12 (Statistics) and Math 5A (Calculus).

Chi-Square details are in Appendix A.


5

Figure 3: Percentage of Math Students Earning a C or Better by URM Status

While under-represented minority students (URM) do tend to have lower success rates compared to

other students, those URM students who receive tutoring perform better than URM students who do

not receive tutoring. For Math 152, Math 254A and Math 5A the higher success rates are statistically

significant at the 0.05 level significance. This information is based on a two-tailed significance level

where the hypothesis is whether or not success rates are different for tutored students. If we were to

instead consider these one-tailed tests where our hypothesis is whether or not tutored students have a

higher success rate then the results would be significant for all six courses2. Chi-Square details are in

Appendix A.

2 While a chi-square test is generally only a two-tailed test because the chi-square statistic is not symmetrical, in this case our chi-square is mathematically equivalent to a Z-statistic. We can therefor take half the p-value of the two tailed test and compare that to the 0.05 level of significance for a one-tailed test.


6

Figure 4: Tutoring Use By Ethnicity

Of the 8,082 Students enrolled in math 152 over the years studied, 1,757 (21.74%) used tutoring.

19.25% of under-represented minority (URM) students used tutoring for intermediate algebra while

23.95% of non-URM students used tutoring. The table below shows tutoring use by ethnicity for Math

152, Intermediate Algebra. While URM students in general used less tutoring than other students, black

students used tutoring at the highest rate.

23.68%

13.45%

29.82%

8.11%

18.68%

23.34%

18.52%

25.63%24.14%

AmericanIndian,Alaskan

Nativ

Asian Black Non-Hispanic

Filipino Latino MultipleEthnicities

PacificIslander

Unknown White Non-Hispanic

Percent of Students Enrolled in Intermediate Algebra that Used Tutoring by Ethnicity


7

Figure 5: propensity score matching analysis of tutoring use

The above chart shows performance differences between students who were tutored and a comparison

group of students who were not tutored but have similar attributes based on one to many propensity

score matching (PSM). The criteria used for matching were gender, ethnicity (as a binary variable

indicating URM or not), age, and prior overall grade point average (GPA). The chart shows that on

average those who were tutored had better success rates than those who were not. The chart also

shows a 95% confidence interval for the difference. If the confidence interval does not include zero then

we can be at least 95% confident that the difference did not occur by random chance. In every case

except for Math 154, Elementary Algebra the results were statistically significant. The largest difference

in success rates between the two groups ids for Math 254, Pre-algebra.


8

Figure 6: Distribution of Hours Tutored for Math 152 Intermediate Algebra

For Math 152, Intermediate Algebra, the maximum number of hours someone was tutored for a

semester was 72 hours and the minimum was less than an hour. The mean time for tutoring was about 3

hours per semester with a standard deviation of 5.5 hours. But as can be seen in the histogram above,

the data are not normally distributed. The median for hours tutored was 1.15 hours; 50% of students

tutored received 1.15 or less hours of tutoring in the semester. Seventy five percent of students tutored

received 3.16 or less hours of tutoring during the semester.

Looking at these numbers separately for those who succeeded vs those who did not succeed we see the

average number of tutoring hours for those who succeed was 3.58 and the average number of tutoring

hours for those who do not succeed was 2.29. This difference is statistically significant at p < 0.001


9

Logistic Regression

We used a Logistic Regression to estimate the number of tutoring hours required to succeed in Intermediate algebra and to attempt to at least partly control for self-selection bias.

Below we estimate the ln(odds of succeeding) using hours tutored, gender, URM status, tutoring use

and overall GPA .

𝑝 = 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑒𝑑𝑖𝑛𝑔 𝑖𝑛 𝑀𝑎𝑡ℎ 152

𝑙𝑛 (𝑝

1 − 𝑝) = 𝛼 + 𝛽1𝑥1 + 𝛽2𝑥2 + 𝛽3𝑥3 + 𝛽4𝑥4 + 𝛽5𝑥5 + 𝜀

𝑙𝑛 (𝑝

1 − 𝑝) = 𝛼 + 𝑇𝑢𝑡𝑜𝑟𝑖𝑛𝑔𝐻𝑜𝑢𝑟𝑠𝑥1 + 𝑈𝑅𝑀𝑥2 + 𝐺𝑒𝑛𝑑𝑒𝑟𝑥3 + 𝐺𝑃𝐴𝑥4 + 𝑈𝑠𝑒𝑇𝑢𝑡𝑜𝑟𝑥5 + 𝜀

Variables in the

Equation Beta

Std.

Error Wald df P-Value

Constant -0.541 0.055 95.802 1 P < 0.001

Tutoring Hours 0.046 0.012 16.288 1 P < 0.001

URM (Y = 1) -0.420 0.046 83.445 1 P < 0.001

Gender (F = 1) 0.339 0.046 54.537 1 P < 0.001

Overall GPA 0.229 0.017 172.508 1 P < 0.001

Use Tutoring (Y = 1) 0.104 0.063 2.668 1 P = 0.102

Because we are looking at the log odds of succeeding, it is difficult to interpret the Betas. However,

positive values mean that the variable has a positive influence on succeeding and negative values mean

the variable has a negative impact on succeeding. Of the variables coded dichotomously, URM status

has the largest influence.

We could use the equation below to predict the probability of success for a particular student that student’s information for each of the variable in the equation.

𝑙𝑛 (𝑝

1 − 𝑝) = − 0.541 + 0.046𝑥1 − 0.420𝑥2 + 0.339𝑥3 + 0.229𝑥4 + 0.104𝑥5 + 𝜀


10

Alternatively we could use the equation to estimate the Number of Tutoring Hours Required to Succeed in Intermediate Algebra

The equation below can be used to predict the ln(odds of succeeding) using hours tutored, gender, URM status and overall GPA as a covariate.


𝑙𝑛 (𝑝

1 − 𝑝) = 𝛼 + 𝛽1𝑥1 + 𝛽2𝑥2 + 𝛽3𝑥3 + 𝛽4𝑥4 + 𝜀

𝑙𝑛 (𝑝

1 − 𝑝) = 𝛼 + 𝑇𝑢𝑡𝑜𝑟𝑖𝑛𝑔𝐻𝑜𝑢𝑟𝑠𝑥1 + 𝑈𝑅𝑀𝑥2 + 𝐺𝑒𝑛𝑑𝑒𝑟𝑥3 + 𝐺𝑃𝐴𝑥4 + 𝜀

Variables in the

Equation Beta

Std.

Error Wald df P-Value

Constant -0.142 0.133 1.15 1 P = 0.284

Tutoring Hours 0.048 0.012 17.39 1 P < 0.001

URM (Y = 1) -0.610 0.100 37.15 1 P < 0.001

Gender (F = 1) 0.303 0.100 9.21 1 P = 0.002

Overall GPA 0.148 0.040 13.43 1 P < 0.001

𝑙𝑛 (𝑝

1 − 𝑝) = − 0.142 + 0.048𝑥1 − 0.610𝑥2 + 0.303𝑥3 + 0.148𝑥4 + 𝜀

Note: Sample only includes those who used tutoring.


11

If we want to estimate the number of tutoring hours required for success for a group, it is simpler to exclude GPA (a continuous variable) as a covariate. Here we estimate the ln(odds of succeeding) using hours tutored, gender, and URM status.


𝑙𝑛 (𝑝

1 − 𝑝) = 𝛼 + 𝛽1𝑥1 + 𝛽2𝑥2 + 𝛽3𝑥3 + 𝜀

𝑙𝑛 (𝑝

1 − 𝑝) = 𝛼 + 𝑇𝑢𝑡𝑜𝑟𝑖𝑛𝑔𝐻𝑜𝑢𝑟𝑠𝑥1 + 𝑈𝑅𝑀𝑥2 + 𝐺𝑒𝑛𝑑𝑒𝑟𝑥3 + 𝜀

Variables in the Equation Beta Std. Error Wald df P-Value

Constant 0.213 0.091 5.499 1 p = 0.019

Tutoring Hours 0.052 0.012 20.32 1 p < 0.001

URM (Y = 1) -0.645 0.099 42.169 1 p < 0.001

Gender (F = 1) 0.322 0.099 10.52 1 p = 0.001

𝑙𝑛 (𝑝

1 − 𝑝) = 0.213 + 0.052𝑥1 − 0.645𝑥2 + 0.322𝑥3 + 𝜀

Note: Sample only includes those who used tutoring.


12

Now we can estimate the number of tutoring hours required for success in Math 152 for a particular population of interest. For example, if we want to know the number of hours of tutoring estimated for under-represented minority male success in Intermediate Algebra we could use the equation below.

The equation for an under-represented minority male simplifies to:

𝑙𝑛 (𝑝

1 − 𝑝) = − 0.432 + 0.052𝑥1 + 𝜀

• Among the 1,467 Male URM students who were not tutored, 560 succeeded (38%) • Among the 316 Male URM students who were tutored at all, 129 succeeded (41%) • For p = 0.5 we estimate that a URM male needs 8.3 hours of tutoring (about 0.5 hours

per week) • 29 Male URM students were tutored for at least 8.3 hours and 17 succeeded (59%) • For p = 0.75 we estimate that a URM male needs 29.5 hours of tutoring (about 2 hours

per week) • The two URM male students who had at least 29.5 hours of tutoring both succeeded.


13

English Placement Scores

Other covariates such as English Placement scores could be used. However, comparing English

Placement scores for those who used tutoring and those who do not use tutoring we see no significant

difference.

Used Tutoring

N Mean English

Placement Score

Std. Deviation

Std. Error Mean

No 5,134 45.85 8.143 0.114

Yes 1,271 45.48 8.183 0.230

The difference of 0.37 is not significant (p = 0.149). Given that the sample size is over 6,000, this is

evidence that English placement scores are not related to the decision to use tutoring for Math 152,

Intermediate Algebra.

There is a significant difference between English placement scores for those who succeed in Math 152,

Intermediate Algebra vs those who do not succeed in Math 152; English placement is a significant

predictor of success in Math 152.

Success in

Math152

N Mean English

Placement Score

Std. Deviation

Std. Error Mean

No 3,314 44.57 8.111 0.141

Yes 3,091 47.07 7.996 0.144

The difference of 2.5 is statistically significant (p < 0.001)

When we add English placement scores to the equation it is a significant predictor of success. However,

the fit of the equation does not change in any practical way and the sample size is reduced by almost

1,700 students when we use this information.

The primary implication is that more tutoring did appear to lead to greater success in math classes in

general. Even those students who use tutoring tend to use very little. Encouraging students to use

tutoring and to use it often would likely lead to greater success in math courses. Some of the

underrepresented minority groups used tutoring at a disproportionately lower rate than their white

counterparts. In particular, Latino, Filipino, and Pacific Islander students used tutoring at less than 80%

of the rate of use by White students. Other non-URM Asian students tended to use less tutoring as well.

No more than 30% of any ethnic group used tutoring and this speaks to the fact that all students,

regardless of their background, could probably benefit by using more tutoring.


14

Appendix A

Female Students

Course Name Chi-Square N df Sig. (2-sided)

MATH-12 14.661 2,593 1 0

MATH-152 17.304 4,139 1 0

MATH-154 3.089 2,936 1 0.079

MATH-254A 11.353 1,000 1 0.001

MATH-4 9.816 723 1 0.002

MATH-5A 12.188 578 1 0

Table 1: Chi-Square analysis tutored vs non-tutored female students.

Male Students


MATH-12 1.254 1,659 1 0.263

MATH-152 18.303 3,943 1 0.000

MATH-154 13.847 2,730 1 0.000

MATH-254A 21.522 712 1 0.000

MATH-4 5.644 1,053 1 0.018

MATH-5A 2.790 1,067 1 0.095

Table 2: Chi-Square analysis tutored vs non-tutored male students.

URM Students


MATH-12 3.146 1,637 1 0.076

MATH-152 4.842 3,802 1 0.028

MATH-154 3.174 2,901 1 0.075

MATH-254A 26.060 1,103 1 0.000

MATH-4 3.481 634 1 0.062

MATH-5A 5.446 550 1 0.020

Table 3: Chi-Square analysis tutored vs non-tutored URM students.

Documents

Effects of Math Tutoring - Cabrillo College · 19.25% of under-represented minority (URM) students used tutoring for intermediate algebra while 23.95% of non-URM students used tutoring