AVID Middle School 2015-2016 DISTRICT FINDINGS: FINAL …...L139 Education Building, 1000 Bascom Mall, Madison, WI 53706-1326 School of Education, University of Wisconsin-Madison avid.org

L139 Education Building, 1000 Bascom Mall, Madison, WI 53706-1326 School of Education, University of Wisconsin-Madison avid.org | wihopelab.com | wiscape.wisc.edu

A V I D M i d d l e S c h o o l 2 0 1 5 - 2 0 1 6

DISTRICT FINDINGS: FINAL REPORT

A V I D M I D D L E S C H O O L 2 0 1 5 - 2 0 1 6 D I S T R I C T F I N D I N G S : F I N A L R E P O R T

Authors

Dr. Peter KinsleySenior Researcher

Wisconsin HOPE Lab

Jamie HawkinsProject Assistant

Wisconsin HOPE Lab

Dr. Jed RichardsonActing Director

Wisconsin HOPE Lab

Acknowledgments

This report is the product of the Wisconsin HOPE Lab under the directorship of Dr. Jed Richardson. Dr. Peter Kinsley, Senior Researcher, conducted primary data analysis and drafted the report. Jamie Hawkins, Project Assistant, contributed to data preparation and analysis, and created all charts and figures. Alison Bowman, Wisconsin HOPE Lab Associate Director, and Minhtuyen Mai, Research Specialist, created the layout design. The authors would also like to thank Dr. Sara Goldrick-Rab for her help and guidance throughout the report preparation process.

Tables provided by the Madison Metropolitan School District are courtesy of Langston P. Evans, AVID District Coordinator.

ii


i i i

Table Of Contents

Executive Summary ................................................................................................................................................. 1

Introduction .............................................................................................................................................................. 3

The Wisconsin HOPE Lab Evaluation of Student Outcomes ........................................................................ 3

Overview of AVID ..................................................................................................................................................... 3

History of AVID and the Partnership with the Madison Metropolitan School District and Boys and Girls Club of Dane County ..................................................................................................... 4

AVID National Certification ..................................................................................................................................... 5

AVID Middle School Implementation..................................................................................................................... 6

AVID Middle School Program Retention ................................................................................................................ 6

Study Methodology ................................................................................................................................................. 8

AVID Eighth-Grade Impacts .................................................................................................................................... 9

Eighth-Grade Impacts: Combined Cohort ................................................................................................. 11

Eighth-Grade Impacts: 2016 Cohort ............................................................................................................ 15

AVID Ninth-Grade Impacts ................................................................................................................................... 20

Ninth-Grade Impacts: Combined Cohort ................................................................................................... 22

Comparison of Annual 9th Grade Impacts: 2014 to 2016 ......................................................................... 26

Ninth- Grade Impacts: 2016 Cohort ............................................................................................................ 30

Summary ................................................................................................................................................................. 36

Appendix: AVID Middle School Impact Tables ................................................................................................... 37


1

Executive Summary

The Madison Metropolitan School District (MMSD) has partnered with the Wisconsin HOPE Lab to conduct an assessment of the Advancement via Individual Determination (AVID) program in district middle schools. AVID is a national college readiness system designed to increase academic achievement, college preparation, postsecondary educational access, and degree attainment for students in the academic middle (i.e. grade point averages between 2.0 and 3.5) who are traditionally underrepresented in higher education. Since 2009, the AVID program has operated in all four comprehensive MMSD high schools. In 2012, the program was also expanded to 11 of 12 district middle schools. The 2015-2016 AVID Middle School report analyzes measurable student outcomes for the AVID program that reflect the program’s stated goals. The assessment focuses on AVID Middle School program impacts in two areas—8th grade academic outcomes and 9th grade academic outcomes. Research questions guiding these analyses are as follows:

1. How does the 8th grade academic achievement of students who participated in the middle school AVID program compare to academically and demographically similar non-AVID peers?

a. What have been the program’s overall impacts on 8th grade academic achievement over the past four years?

b. What were the program’s impacts on 8th grade academic achievement in 2016?

2. How does the 9th grade academic achievement of students who participated in the middle school AVID program compare to academically and demographically similar non-AVID peers?

a. What have been the program’s overall impacts on 9th grade academic achievement over the past three years?

b. Have 9th grade achievement impacts remained consistent over the past three years?

c. What were the program’s impacts on 9th grade academic achievement in 2016?

Methodology

Estimates of program effects were computed using propensity score matching. This statistical method matches AVID and non-AVID students based on the individual probability of middle school AVID participation using 6th grade pre-participation data. Groups were matched within cohort and high school feeder pattern, and balanced within racial and socioeconomic categories. This means that rather than comparing selected AVID students to all of their grade-level peers, they were compared to other students who had similar academic and demographic profiles but chose not to participate. We conducted this matching process on two separate samples—district 8th graders and district 9th graders. Students in our 8th grade sample were drawn from four 8th grade cohorts: 2013, 2014, 2015 & 2016. Students in our 9th grade sample were drawn from three 9th grade cohorts: 2014, 2015 & 2016. The 8th grade student sample is 71% low-income and 75% students of color. Slightly less than half are male, 39% are English language learners, and 8% are in special education. The 9th grade student sample is 70% low-income, 74% students of color, and 47% male. Approximately 40% are English language learners and 9% are in special education.

Findings

The impacts of the AVID middle school program are broadly similar to those from prior years. However, with increased sample sizes gained from conducting a multi-cohort analysis, statistical significance could be better detected when impacts were smaller. Highlights include:

1. On average, the AVID program has had modest positive impacts on 8th grade academic achievement and school attendance, particularly for low-income students and students of color.

• District-wide, AVID students have earned cumulative GPAs around .05 of a grade point higher on average than their non-AVID


2

counterparts. Core GPAs have been similarly higher at .07 of a grade point. The impact has been slightly larger for low-income students (.07 of a grade point higher cumulative GPA and .09 of a grade point higher core GPA), and for students of color (.06 of a grade point and .08 of a grade point higher for cumulative and core GPA respectively).

• AVID students have had fewer unexcused days absent overall than similar non-program peers—around three quarters of a day on average. This effect has been generally larger among low-income AVID students, who have been absent about a full day less on average than similar low-income students who did not participate in the program.

2. Participation in AVID during the 8th grade has helped students remain on track academically during 9th grade, particularly for low-income students and students of color.

• Students who participated in AVID during the 8th grade have earned higher cumulative and core GPAs in the 9th grade than their peers, amounting to just over a tenth of a grade point on average. Impacts have been larger for low-income students at nearly two-tenths of a grade point, and for students of color at around .15 of a grade point.

• Eighth-grade AVID students have logged fewer unexcused days absent in the 9th grade—nearly 1.5 days fewer on average than those who did not participate. Among low-income students and students of color, this number was around two full days fewer absences.

• Eighth-grade AVID students have also earned more credits in the 9th grade (6.57 for AVID vs. 6.20 for the comparison group overall), and have failed fewer courses (.89 for AVID vs. 1.43 for the comparison group). Again, for low-income students and students of color, these impacts were somewhat larger.

• The positive impacts of 8th grade AVID on 9th grade academic outcomes appear to have remained generally consistent over the past three years. However, program impacts on attendance may have increased somewhat in 2016.

3. Participation in 8th grade AVID appears to have increased early college math readiness among 9th graders in 2015-2016.

• Students who participated in 8th grade AVID scored significantly higher on the ASPIRE math test the following year as 9th graders than similar non-participants—about a point higher on average. Among students of color, however, this impact approached two full points higher.


3

Introduction

The Madison Metropolitan School District (MMSD) is a large and increasingly diverse urban school district, serving 27,000 students in 48 schools. Nearly half of the students in the district come from low-income families, and over half identify as students of color. Equity is a serious concern in MMSD as the district has grappled for many years with persistent socioeconomic gaps in academic achievement and high school graduation rates. In 2007, as part of an effort to close these achievement gaps, MMSD began to implement the national AVID (Advancement via Individual Determination) system at East High School. In 2008, the district joined forces with the Boys and Girls Club of Dane County (BGCDC) to pair AVID with BGCDC’s Teens of Promise (TOPS) program, and the following year the combined AVID/TOPS program was offered in all four of the district’s comprehensive high schools. Starting in 2012, the AVID program was expanded to 11 district middle schools, and BGCDC also began implementing the College Club program at selected middle schools that same year.

Since 2014, the Madison Metropolitan School District has partnered with the Wisconsin HOPE Lab at the University of Wisconsin-Madison to conduct an annual assessment of the district’s AVID program. The 2015-2016 AVID assessment focuses on estimating

AVID Middle School program impacts in twoareas—8th grade academic outcomes and 9th grade academic outcomes. Research questions guiding these analyses are as follows:

1. How does the 8th grade academic achievement of students who participated in the middleschool AVID program compare to academicallyand demographically similar non-AVID peers?

a. What have been the program’s overallimpacts on 8th grade academicachievement over the past four years?

b. What were the program’s impacts on 8thgrade academic achievement in 2016?

2. How does the 9th grade academic achievement of students who participated in the middleschool AVID program compare to academicallyand demographically similar non-AVID peers?

a. What have been the program’s overallimpacts on 9th grade academicachievement over the past three years?

b. Have 9th grade achievement impactsremained consistent over the past threeyears?

c. What were the program’s impacts on 9thgrade academic achievement in 2016?

Overview of AVID

Advancement via Individual Determination (AVID) is a national college readiness system consisting of two interrelated components designed to increase academic achievement, college preparation, and postsecondary educational access and degree attainment for underrepresented students in the academic middle:

1. An elective class aimed at preparing a selectgroup of students for entrance and enrollmentin post-secondary education. AVID aims to servestudents in the academic middle (2.0-3.5 grade-point average) who may be first generation,low-income, historically underrepresented in

post-secondary education, and/or have special circumstances.

2. AVID also functions as a school-wide system oftransformation. A strong AVID system focuses onincreasing college readiness for all AVID electivestudents, as well as improving academicperformance for all students, by transformingthe instruction, systems, leadership, and cultureof a school.

The elective class includes a curriculum that focuses on developing students’ writing, inquiry, organization, and reading skills to help students


4

become college ready. Additionally, students develop critical reasoning and thinking skills through work in small, collaborative tutorial groups. The purpose of the elective class is for students to gain the skills and receive the support to enroll and graduate from a post-secondary institution.

All schools adhere to the 11 AVID Essentials that the national AVID College Readiness System requires.

Essential 1: Student Selection – AVID student selec-tion must focus on students in the academic mid-dle, with academic potential, who would benefit from AVID support to improve their academic re-cords and begin college preparation.

Essential 2: Voluntary Participation – AVID program participants, both students and staff, must choose to participate in the program.

Essential 3: Scheduling – The school must commit to fully implementing AVID by providing and enrolling students in the AVID yearlong elective classes of-fered within the regular school day.

Essential 4: College Readiness – AVID students must enroll in a rigorous course of study that will en-able them to meet the requirements for university enrollment.

Essential 5: Organization – The AVID elective class must provide instruction aimed at developing stu-dents’ organizational skills to promote their aca-demic self-management.

Essential 6: Reading and Writing – The AVID elective class must rely on strong, relevant writing and read-ing curriculum as its basis for instruction.

Essential 7: Inquiry and Collaboration – The AVID elective class must promote critical thinking through inquiry and collaboration.

Essential 8: Tutorial – The program must provide a sufficient number of tutors in the AVID classroom to facilitate student access to a college-preparatory curriculum. Tutors should be enrolled in college and universities, be able to mentor students and facil-itate tutorials, and they must be trained to imple-ment AVID methodologies with students.

Essential 9: Data Use – AVID program implementation and student progress must be monitored through the AVID data entry system, and results must be an-alyzed to ensure student success. Progress monitor-ing must also inform instruction.

Essential 10: Professional Learning – The school or district must implement the AVID Essentials, partici-pate in the annual AVID certification process, and provide ongoing AVID professional development for both AVID and non-AVID teachers.

Essential 11: Site Teams – The school must support an active, interdisciplinary AVID site team that col-laborates to help students access and succeed in rigorous college preparatory courses.

History of AVID and the Partnership with the Madison Metropolitan School District and Boys and Girls Club of Dane County

MMSD piloted the AVID program at East High School in 2007. AVID was implemented in the district middle schools starting in 2012. At that time, MMSD and BGCDC also established the AVID College Club in Cherokee and Wright middle schools. College

Club middle schoolers receive additional supports including after-school tutoring, cultural field trips, and family engagement events. In addition, Cherokee and Wright receive additional funding and personnel support from BGCDC.


5

AVID National Certification

The national AVID Center monitors the AVID elective class through an extensive certification process encompassing the 11 AVID Essentials. The Essentials focus on student selection into the program, the use of AVID strategies, student access to rigorous course work, and effective site team practices. In addition, schools must show evidence of using AVID strategies beyond the AVID elective classroom and throughout the general academic program.

All MMSD middle schools received AVID certification during their third year of implementation in 2015.

Each middle school received at least a “1” (meets certification) in all 11 Essentials. In 2016 in the fourth year of middle school implementation, scores improved in 10 of the 11 essentials (Figure 1). Overall, the MMSD AVID program has demonstrated strengths in the areas of student selection, course scheduling, and site team leadership. Areas for improvement include the development and implementation of AVID Writing, Inquiry, Collaboration, Organization, and Reading (WICOR) within the elective class, use of data for monitoring and improvement, and greater access to advanced coursework.

Legend

2015-2016

2014-2015

Each Essential can receive a maximum score of 3. Figures provided by MMSD.

FIGURE 1: AVERAGE MIDDLE SCHOOL CERTIFICATION SCORES BY THE 11 AVID ESSENTIALS

2.64

3.00

1.64

2.00

1.27

1.82

1.64

1.45

1.64

1.55

1.64

1.91

Student S

electio

n

Voluntary

Parti

cipatio

nSc

heduling

College R

eadinessO

rganiza

tion

Writ

ing And R

eading

Inquiry

Tuto

rial

Data U

se

Profe

ssional L

earning

Site Te

ams

0.00

0.50

1.00

1.50

2.00

2.50

3.00

1.36

1.90

1.09

1.36

1.54

1.36

1.63

1.18

1.45

1.45

Ce

rtifi

ca

tion

Sc

ore


6

AVID Implementation

During the 2015-16 school year MMSD offered 24 sections of AVID in grades 7 and 8, serving 510 students (Table 1). Generally, one class per grade was offered in middle school.

In 2015-16, AVID served approximately 15% of MMSD’s 7th and 8th grade student population.

Seventy-six percent of AVID middle school students were eligible for free or reduced price lunch, and 56% were female (Table 2). While the district is 19% African American and 21% Hispanic, these groups represent 25% and 41% of the AVID population respectively. Following the national trend, boys (44%) are under-enrolled in AVID.

Program Retention

AVID and wraparound AVID programs (e.g. College Club) seek to retain all students who enroll in the AVID elective class. Students who are struggling academically are identified and closely monitored by a school-based AVID site team. Student attrition from AVID typically occurs through three avenues: (1) they leave the district or transfer to a district school that does not offer AVID programming such

as Badger Rock Charter Middle School; (2) they voluntarily drop the AVID elective course, or; (3) they are asked to exit from the program because their cumulative grade point average drops below 2.0 for more than one semester.

Among AVID 7th graders in 2015-16, 83% enrolled in the program again as 8th graders for the 2016-17

TABLE 1: 2015-16 MIDDLE SCHOOL AVID ENROLLMENT AND NUMBER OF COURSE SECTIONS

Grade Level Enrollment Sections Per Grade7 273 118 237 13

Total 510 24

Source: MMSD. Notes: Enrollment based on 3rd Friday September counts. Sennett MS offers three AVID elective courses that are multi-age 7th and 8th grade students.

TABLE 2: 2015-16 MIDDLE SCHOOL AVID ENROLLMENT BY STUDENT BACKGROUND CHARACTERISTICS

TotalLow-

IncomeFemale Black Hispanic Asian White ELL

Special Education

7th Grade District

273 74% 54% 25% 39% 11% 19% 48% 7%

8th Grade District

237 78% 59% 25% 43% 10% 13% 47% 5%

District Total

510 76% 56% 25% 41% 11% 16% 48% 6%

Source: MMSD.


7

FIGURE 2: 2015-16 AVID RETENTION FROM 7th GRADE TO 8th GRADE

FIGURE 3: 2015-16 AVID RETENTION FROM 8th GRADE TO 9th GRADE

83%

83%

86%

80%

79%

88%

93%

80%

71%

90%

Distric

tLo

w-in

come

Fem

ale

Male

African A

meric

an

Hispanic

Asian

White

Multir

acial

ELL

81%

Specia

l Ed

0%

20%

40%

60%

80%

100%

AV

ID R

ete

ntio

n

58%

64%

62%

54%

58%

69%

69%

35%

48%

69%

Distric

tLo

w-in

come

Fem

ale

Male

African A

meric

an

Hispanic

Asian

White

Multir

acial

ELL

50%

Specia

l Ed

0%

20%

40%

60%

80%

100%

AV

ID R

ete

ntio

n

academic year (Figure 2). Seventh to 8th grade retention rates of between 80% and 90% were common across most demographic subgroups. African American students were retained in the program at a slightly lower rate of 79%, while English language learners and Asian students were retained at higher rates of 90% and 93% respectively.

Overall retention in the AVID program as students

made the transition from middle school to high school was considerably lower at 58% (Figure 3). However, there was variation in retention rates between students from different backgrounds. Hispanic, Asian and ELL students had notably higher rates of retention at 69% for each group. Male students and White students, on the other hand, were much less likely to enroll in AVID during their 9th grade year at 54% and 35%.

Source: MMSD.


8

Study Methodology

We estimated the effects of middle school AVID participation using propensity score matching. This statistical method allowed AVID students to be compared with other students who had similar ac-ademic and demographic profiles but who did not participate in the program--a more “apples to ap-ples” comparison. Our approach matched students based on the individual probability of middle school AVID participation computed for each student within each MMSD high school feeder pattern (Table 3).1 Because we used multiple cohort years in our anal-yses, we performed this matching process separate-ly for each cohort to ensure that AVID students and their matched non-AVID counterparts were exposed to the same school environments (e.g. school poli-cies, leadership & peers) which might otherwise bias our estimates. Matching within cohort also enabled us to compare program effect estimates on 9th grade outcomes across cohorts.

TABLE 3: FEEDER MIDDLE SCHOOLS

High School Middle SchoolsEast Blackhawk; O'Keefe; ShermanLa Follette Badger Rock Charter; Sennett;

WhitehorseMemorial Jefferson; Spring Harbor; TokiWest Cherokee; Hamilton; Wright

To maximize the overall quality of the matching process, we allowed AVID students to be matched

with more than one similar non-AVID comparison student. All final estimates were weighted to adjust for the number of matched partners. Pairing was based on the probability of AVID program participation as calculated from the student’s sixth-grade pre-program characteristics. These baseline characteristics were derived from district administrative records, and included student demographics (e.g. gender, race/ethnicity, low-income status, parent education,2 English language learner status, special education status), academic and behavioral records, attendance, and WKCE Reading and Math scores.3 In cases where particular baseline information for a given student was missing, we substituted predicted (imputed) values based on other observed characteristics.4 This allowed us to keep these students in our analyses rather than drop them and risk biasing our results.

We conducted this matching process on two separate samples—district 8th graders and district 9th graders. Students in our 8th grade sample were drawn from four 8th grade cohorts: 2013, 2014, 2015 & 2016.5 Students in our 9thgrade sample were drawn from three 9th grade cohorts: 2014, 2015 & 2016. We used the matched 8th grade sample to estimate the impacts of AVID middle school participation on the focal 8th grade outcomes of cumulative and core6

GPA, attendance, and spring MAP Reading and Math scores. With the matched 9th grade sample we examined how participation in middle school AVID impacted 9th grade outcomes once students

1 Due to the relatively small number of traditionally underserved students targeted by the AVID program at some district middle schools, matches could only be conducted within feeder patterns, and not within individual schools.2 The use of parent education in our matching process represents a change to our analysis strategy from prior years. Although the relationship between parent education and school outcomes is well established in the research literature, missing data previously prevented the use of parent education for matching. However, recent published guidance on dealing with missing values in propensity score matching allowed us to account for the influence of parent education in all of our impact estimates while also accounting for missing values. For more information see: http://link.springer.com/chapter/10.1007/978-3-319-41259-7_5. 3 We use 6th grade WKCE scores instead of MAP scores for matching due to their availability for all cohorts used in our analyses. 4 Approximately 10% of the students in our samples had missing values for parent education. We used the technique of multiple imputation to handle these missing values in our matching and analyses. 5 Students in the 2013 cohort were enrolled for the 2012/13 academic year. Each cohort year we reference follows this same pattern.6 Core classes include math, science, language arts and social studies.


9

made the transition to high school. We estimated the impacts of middle school AVID participation on the outcomes of 9th grade cumulative and core GPA, attendance, and credit accumulation as represented by the average number of earned credits and failed courses in the 9th grade. Throughout the report we note significant program impacts using a 95% confidence level (α=.05) unless specifically stated otherwise.

Like all non-experimental studies relying on observational data, the present study is subject to

certain limitations. Most notably, AVID and non-AVID students could only be matched on the pre-program characteristics present in district administrative records. While these records allowed us to “control” for a comprehensive array of academic and demographic pre-program information that would otherwise bias estimates of program impact, remaining bias from other preexisting but unobserved factors is still possible. The analysis therefore bears this risk in absence of better strategies for estimating program effects.

AVID 8th Grade Impacts

Eighth-Grade Sample

The statistical matching process for the combined cohort 8th grade sample resulted in a matched comparison group of non-AVID students who

closely resembled their AVID counterparts on 6th grade characteristics. Figure 4 presents the demographic makeup of the two groups, as well as the demographic profile of the full district for comparison. Among both AVID students and their

0% 20% 40% 60% 80% 100%

English Language Learner

Parents: Four-Year Deg.

Parents: Two-Year Deg.

Parents: High School or Less

Special Education

Low-income

White

Students of Color

Male

MMSD AVID Comparison Group

FIGURE 4: DEMOGRAPHIC CHARACTERISTICS OF 8TH GRADE SAMPLE


1 0

matched comparison group peers, students of color made up three-quarters of the sample. Low-income students also represented more than two-thirds of the matched sample. Slightly less than half of AVID and comparison group students were male, while nearly 40% were English language learners. Students in the matched sample most commonly had parents who had never attended college. However, just over a quarter had parents who had earned a four-year college degree or higher.

Table 4 shows that AVID students and their comparison group counterparts had nearly

identical academic histories at the start of middle school. This was true in terms of their 6th grade GPA, as well as their 6th grade WKCE Reading and Math scores. As determined by AVID selection, students in the matched 8th grade sample came from the academic middle, with an average GPA of around 3.0 as 6th graders. All baseline measures presented in Figure 4 and Table 4 were statistically equivalent between AVID students and their comparison group peers.7 Nevertheless, we make statistical adjustments for any residual imbalance between the two groups in all of our impact estimates.8

TABLE 4: DEMOGRAPHIC CHARACTERISTICS OF EIGHTH-GRADE SAMPLE BY MIDDLE SCHOOL AVID AND MATCHED COMPARISON GROUP

MMSD AVIDComparison

GroupSixth Grade Cumulative GPA 3.08 3.00 3.00

Sixth Grade Core GPA 2.97 2.86 2.86

Sixth Grade Attendance Rate 95% 96% 96%

Sixth Grade Behavior Events 2.03 0.86 0.81

Sixth Grade WKCE Math Score 511.07 500.62 499.31

Sixth Grade WKCE Reading Score 496.61 489.61 488.94

7 Throughout the report, statements of statistical significance are based on a 95% confidence level, with α=.05.

8Impact estimates are derived from regression models, which control for any residual imbalance in baseline characteristics be-tween AVID and comparison group students.


1 1

Impact: Eighth-Grade Grade Point Average

There is evidence of positive average program impacts on 8th grade academic performance. AVID students earned higher cumulative and core GPAs than their non-AVID peers. This was true both at the district level and across most focal subgroups with the exception of white students (Figures 5

& 6). Differences in cumulative and core GPAs were generally statistically significant, with the exception of core GPA at the district level which was only marginally non-significant.

FIGURE 5: EIGHTH-GRADE CUMULATIVE GPA BY AVID PARTICIPATION

Legend

AVID

COMPARISONGROUP

DIFFERENCE ISSTATISTICALLYSIGNIFICANTAT THE 90%LEVEL


0.50

0.00

1.50

1.00

2.50

2.00

3.50

4.00

3.00

2.95

2.94

2.95

2.93

2.94

2.90

2.90

2.87

2.89

2.93

2.87

2.87

District Low-income WhiteStudentsof Color

Low-incomeStudentsof Color

Male Studentsof Color

AVID Comparison Group

Cu

mu

lativ

e G

PA

Eighth-Grade Impacts: Combined Cohort (2013-2016)

Impact estimates in this section are based on analyses combining four eighth-grade cohorts: 2013-2016. They thus represent the average impacts of AVID participation over that four-year period. Our evaluation of the middle school AVID program indicates that it has had positive average effects on academic

achievement. We also found positive average effects on school attendance. In both cases, however, effects were generally modest. We did not find any effects of the program on standardized test performance as measured by eighth-grade MAP scores. We detail each of these findings below.

Key Finding: AVID students earned higher cumulative and core 8th grade GPAs than their peers.


1 2

Legend

AVID

COMPARISONGROUP



0.50

0.00

1.50

1.00

2.50

2.00

3.50

4.00

3.00

2.81

2.77

2.81

2.79

2.77

2.67

2.74

2.68

2.73

2.76

2.69

2.69

District Low-income Low-incomeStudentsof Color



Co

re G

PA

WhiteStudentsof Color

FIGURE 6: EIGHTH-GRADE CORE GPA BY AVID PARTICIPATION


1 3

Legend

AVID

COMPARISONGROUP



Impact: Eighth-Grade Attendance

There is evidence that AVID participation reduced the number of unexcused absent days among 8th grade participants, and increased the general attendance of participants. At the district level and across most focal subgroups, AVID 8th graders had significantly fewer unexcused absent days than their comparison group peers (Figure

7). Attendance rates for AVID students were also significantly higher than their c o m p a r i s o n g r o u p counte rpar t s (Figure 8).

FIGURE 7: EIGHTH-GRADE UNEXCUSED DAYS ABSENT BY AVID PARTICIPATION

FIGURE 8: EIGHTH-GRADE ATTENDANCE BY AVID PARTICIPATION

Key Finding: AVID students had significantly fewer unexcused absent days and higher attendance rates in the 8th grade than their peers.

1.30

1.41

1.40

1.01

1.56

1.14

2.01

2.35

2.14

1.57

2.38

2.12



0

1

2

4

3


Un

exc

use

d D

ays

Ab

sen

t


95%

95%

95%

95%

95%

96%

95%

94%

95%

95%

94%

95%



0%

20%

40%

60%

80%

100%


Att

en

da

nc

e R

ate



1 4

Legend

AVID

COMPARISONGROUP



Impact: Eighth-Grade MAP Test Scores

We did not find evidence of average AVID effects on 8th grade MAP test performance. At the district level and across most subgroups, AVID students earned statistically similar scores to their

comparison group peers on the MAP math and reading tests (Figures 9 & 10). The one exception to this was white AVID participants whose MAP reading scores were about 1.7 points lower on average than their matched peers—a significant difference between the two groups.

FIGURE 9: EIGHTH-GRADE MAP MATH SCORES BY AVID PARTICIPATION

0.00

100.00

50.00

200.00

250.00

150.00

229.

86

229.

94

229.

64

230.

57

229.

77

231.

44

230.

06

230.

09

229.

76

231.

02

229.

70

230.

79




MA

P M

ath

Sc

ore


0.00

100.00

50.00

200.00

250.00

150.00

220.

42

220.

37

220.

66

219.

68

220.

35

220.

48

220.

44

220.

04

220.

16

221.

35

219.

84

220.

04




MA

P R

ea

din

g S

co

re


FIGURE 10: EIGHTH-GRADE MAP READING SCORES BY AVID PARTICIPATION


1 5

0.50

0.00

1.50

1.00

2.50

2.00

3.50

4.00

3.00

2.96

2.93

2.95

2.96

2.92

2.86

2.94

2.91

2.94

2.96

2.91

2.95




Cu

mu

lativ

e G

PA


Eighth-Grade Impacts: 2016 Cohort

While the previous section presented average AVID impacts over the four-year period of 2012 to 2016, here we present a more fine-grained picture of impacts for the 2016 cohort alone. This picture includes impacts at the district level, as well as within demographic subgroups and high school feeder patterns. Our evaluation of the

2016 8th grade AVID cohort indicates that the program had few effects on 8th grade academic performance as measured by GPA, attendance and MAP test scores. This was true both at the district level, and within key demographic subgroups. We detail our findings from the 2016 cohort below.

Impact: Eighth-Grade Grade Point Average (2016)

There was no evidence of AVID program impacts on eighth-grade academic performance in

2016. AVID students and their comparison group counterparts had statistically similar cumulative and core GPAs (Figures 11 & 12). This was true both at the district level and across all focal subgroups.

FIGURE 11: 2016 8th GRADE CUMULATIVE GPA BY AVID PARTICIPATION

Legend

AVID

COMPARISONGROUP




1 6

Legend

AVID

COMPARISONGROUP



0.50

0.00

1.50

1.00

2.50

2.00

3.50

4.00

3.00

2.77

2.71

2.77

2.77

2.70

2.59

2.75

2.71

2.74

2.79

2.70

2.76




Co

re G

PA


FIGURE 12: 2016 8th GRADE CORE GPA BY AVID PARTICIPATION


1 7

Legend

AVID

COMPARISONGROUP



Impact: Eighth-Grade Attendance (2016)

There is some limited evidence that AVID participation reduced the number of unexcused absent days and increased attendance in 2016. At the district level and across most focal subgroups, AVID 8th graders had fewer unexcused absent days

than their comparison group peers (Figure 13). However, these differences were not large enough to achieve statistical significance. Eighth-grade attendance rates were also slightly higher for AVID students than for comparison group students, yet were similarly non- significant (Figure 14).

FIGURE 13: 2016 8th GRADE UNEXCUSED DAYS ABSENT BY AVID PARTICIPATION

FIGURE 14: 2016 8th GRADE ATTENDANCE BY AVID PARTICIPATION

1.59

1.68

1.65

1.34

1.81

1.09

1.81

2.06

1.93

1.20

2.20

1.23



0

1

2

4

3


Un

exc

use

d D

ays

Ab

sen

t


96%

95%

96%

96%

95%

96%

95%

95%

95%

96%

95%

96%



0%

20%

40%

60%

80%

100%


Att

en

da

nc

e R

ate



1 8

Legend

AVID

COMPARISONGROUP



Impact: Eighth-Grade MAP Test Scores

There is no evidence that AVID participation impacted 8th grade MAP scores in 2016. At the district level and across all subgroups,

AVID students earned similar average scores on the 8th grade spring MAP math and reading tests to their comparison group peers (Figures 15 & 16).

FIGURE 15: 2016 8th GRADE MAP MATH BY AVID PARTICIPATION

0.00

100.00

50.00

200.00

250.00

150.00

230.

32

229.

98

229.

97

231.

93

229.

77

231.

67

230.

51

230.

15

230.

24

231.

77

229.

70

231.

78




MA

P M

ath

Sc

ore


0.00

100.00

50.00

200.00

250.00

150.00

220.

61

220.

08

221.

08

218.

48

220.

28

221.

77

221.

03

220.

40

221.

16

220.

46

220.

60

220.

97




MA

P R

ea

din

g S

co

re


FIGURE 16: 2016 8th GRADE MAP READING BY AVID PARTICIPATION


1 9

Eighth-Grade Impacts by High School Feeder Pattern

An examination of AVID 8th grade impacts by high school feeder pattern (i.e. which high school the middle school feeds into) revealed little variation in effects in 2016 (Table 5). We did find some evidence of a positive effect on core GPA equivalent to a tenth of a grade point among East middle school

students—an effect that was generally absent in the other feeder patterns. However, due to the rel-atively small sizes of the feeder pattern samples, this effect did not achieve statistical significance. Performance on all other outcomes was statistically identical between AVID students and their compar-ison group peers within each of the four high school feeder patterns.

TABLE 5: 2016 8TH GRADE IMPACTS BY HIGH SCHOOL FEEDER PATTERN

OutcomeEast Schools

La Follette Schools

Memorial Schools

West Schools

AVID Comp. AVID Comp. AVID Comp. AVID Comp.Cumulative GPA 2.99 2.99 2.99 2.97 2.92 2.90 2.93 2.87Core GPA 2.96 2.86 2.87 2.89 2.61 2.62 2.61 2.56Unexcused Days Absent 2.15 2.89 1.03 1.24 1.17 1.50 1.75 1.43Attendance Rate 95.48 94.54 95.00 95.02 96.36 95.28 95.47 95.69MAP Math Score 219.27 221.25 221.50 222.77 217.84 216.29 224.01 223.37MAP Reading Score 231.25 231.44 231.39 233.77 227.30 225.00 230.96 231.32

Note: Bold indicates statistically significant difference (p<.05)


2 0

AVID 9th Grade Impacts

Overall, our statistical matching process resulted in a 9th grade combined cohort research sample with extremely similar AVID and comparison group students. Figure 17 presents the demographic makeup of the two groups compared to the general population of district students prior to matching. Both the AVID and comparison group students were over

two-thirds low-income and three-quarters students of color. Slightly less than half of each group was male, and approximately two-fifths were English language learners. Around 40% of both groups had parents who had never attended college. Just under a third of both groups had at least one parent with a four-year degree or higher.

FIGURE 17: DEMOGRAPHIC CHARACTERISTICS OF 9th GRADE SAMPLE

0% 20% 40% 60% 80% 100%

English Language Learner

Parents: Four-Year Deg.

Parents: Two-Year Deg.

Parents: High School or Less

Special Education

Low-income

White

Students of Color

Male

MMSD AVID Comparison Group


2 1

TABLE 6: ACADEMIC CHARACTERISTICS OF 9TH GRADE SAMPLE

MMSD AVIDComparison

GroupSixth Grade Cumulative GPA 3.11 3.01 3.00

Sixth Grade Core GPA 3.01 2.87 2.86

Sixth Grade Attendance Rate 96% 96% 96%

Sixth Grade Behavior Events 1.54 0.88 0.87

Sixth Grade WKCE Math Score 513.45 500.32 499.91

Sixth Grade WKCE Reading Score 498.45 489.77 489.24

Consistent with the AVID selection process, students in the matched 9th grade sample came from the academic middle, with an average 3.0 cumulative GPA in sixth grade. Table 6 shows that overall, the AVID students and their comparison group counterparts

in our sample looked very similar to each other academically in the 6th grade in terms of their GPA, as well as their WKCE Reading and Math scores. All measures in Figure 17 and Table 6 were statistically equivalent between the two groups.


2 2

Impact: Ninth-Grade Grade Point Average

There is evidence of positive program impacts on 9th grade academic performance. On average, students who participated in middle school AVID earned 9th grade cumulative and core GPAs over a tenth of a grade point higher than their peers (Figures 18 & 19). This positive effect was present across most subgroups, and was strongest among low-income students of color, where it reached approximately

a fifth of a grade point. One exception was white AVID p a r t i c i p a n t s who had similar average cumulative and core GPAs to their non-AVID peers.

FIGURE 18: NINTH-GRADE CUMULATIVE GPA BY AVID MIDDLE SCHOOL PARTICIPATION

Legend

AVID

COMPARISONGROUP



0.50

0.00

1.50

1.00

2.50

2.00

3.50

4.00

3.00

2.70

2.66

2.69

2.71

2.65

2.61

2.57

2.47

2.52

2.72

2.45

2.48




Cu

mu

lativ

e G

PA


Ninth-Grade Impacts: Combined Cohort (2014-2016)

The impact estimates in this section are based on analyses combining three 9th grade cohorts: 2014-2016. They thus represent the average impacts of middle school AVID participation over a three-year period.9 We find evidence that AVID participation has

had positive average effects on 9th grade academic achievement, school attendance, and credit accumulation over the past three years. We detail each of these findings below.

Key Finding: AVID students earned significantly higher cumulative and core 9th grade GPAs than their peers.

9 Our 9th grade impact estimates more strictly represent the effects of participation in the AVID middle school program combined with the partial effects of participation in the AVID/TOPS 9th grade program.


2 3

Legend

AVID

COMPARISONGROUP



0.50

0.00

1.50

1.00

2.50

2.00

3.50

4.00

3.00

2.54

2.49

2.53

2.55

2.50

2.43

2.43

2.33

2.38

2.58

2.31

2.32




Co

re G

PA


FIGURE 19: NINTH-GRADE CORE GPA BY AVID MIDDLE SCHOOL PARTICIPATION


2 4

Legend

AVID

COMPARISONGROUP



Impact: Ninth-Grade Attendance

Estimates suggest that AVID middle school participation increased 9th grade attendance while reducing the number of unexcused absent days. Ninth graders who participated in middle school AVID had significantly fewer unexcused absent days and higher attendance rates in the 9th grade than their matched peers (Figures 20 & 21). This was true both at the district level where AVID participants had approximately 1.5

fewer unexcused absent days on average than c o m p a r i s o n group students, and across most subgroups where AVID students had one to two fewer days absent.

FIGURE 20: NINTH-GRADE UNEXCUSED DAYS ABSENT BY AVID MIDDLE SCHOOL PARTICIPATION

FIGURE 21: NINTH-GRADE ATTENDANCE BY AVID MIDDLE SCHOOL PARTICIPATION

2.23

2.38

1.75

3.58

4.12

4.06

2.09

4.53

2.90



0

2

4

8

6


Un

exc

use

d D

ays

Ab

sen

t

2.13

2.24

2.08


95%

95%

95%

94%

95%

96%

94%

94%

94%

94%

94%

95%



0.0

100%

80%

40%

60%

20%


Att

en

da

nc

e R

ate


Key Finding: AVID students had significantly higher attendance rates and fewer unexcused absent days in the 9th grade than their peers.


2 5

Legend

AVID

COMPARISONGROUP



Impact: Ninth-Grade Credit Accumulation

There is evidence that AVID middle school participation increased the number of credits earned in the 9th grade. AVID middle school participants earned significantly more credits in the 9th grade than comparison group students (Figure 22). This difference was significant for all subgroups except white students and male students of color who nevertheless earned slightly more credits if they were AVID participants. Middle school AVID participants also failed significantly fewer

courses during ninth grade than their comparison group counterparts (Figure 23). This difference, which amounted to about half a course on average, was significant at the district level and across all subgroups except white students and male students of color.

FIGURE 22: NINTH-GRADE EARNED CREDITS BY AVID MIDDLE SCHOOL PARTICIPATION

FIGURE 23: NINTH-GRADE FAILED COURSES BY AVID MIDDLE SCHOOL PARTICIPATION

6.57

6.53

6.58

6.53

6.52

6.48

6.20

6.10

6.17

6.33

6.07

6.28



0

8

6

4

2


Earn

ed

Cre

dits


0.89

0.96

0.89

0.89

1.01

0.93

1.43

1.67

1.56

1.02

1.77

1.30



0.0

2.0

1.6

1.2

0.8

0.4


Faile

d C

ou

rse

s


Key Finding: AVID students earned significantly more credits and failed fewer courses in the 9th grade than their peers.


2 6

Comparison of Annual 9th Grade Impacts: 2014 to 2016

One objective of this report is to examine whether the effects of middle school AVID participation on 9th grade outcomes have remained consistent over the past three years of program implementation. Here, we present program impact estimates for each of three 9th grade cohorts—starting with the 2014 cohort, and ending with the 2016 cohort. We also note whether effects over the three-year period were statistically different from each other. In general, we found that program effects remained consistent during that period. We discuss general effect consistency for the district below.

Interpreting Annual Effects Comparison Charts

We illustrate annual comparisons of middle school AVID effects on 9th grade outcomes using standard bar charts. In these charts, each colored bar represents the estimated program effect on a particular outcome in a given year. In Figure 24, for example, the effect of 0.11 in the first blue bar indicates that AVID participants had an average cumulative GPA 0.11 grade points higher than their comparison group counterparts in 2014.

Because these numbers are estimates calculated from a single sample of students, the line overlaying each bar represents the range of values where we can be confident the “true” population effect actually lies. If the line does not overlap the zero value on the left side of the chart (Y-axis), we can say that the estimate is “significant” (i.e. not zero) with 95% confidence.


2 7

GPA Effects (2014-2016)

Estimates indicate that the AVID program has had positive annual effects on cumulative and core GPA in the 9th grade (Figures 24 & 25). In 2014 and 2015, the effects were relatively small and did not reach statistical significance. In 2016, however, the effects on both cumulative and core GPA were larger, and were both significant. Nevertheless,

when we compared 2016 effects to those of prior years we found no significant difference that would allow us to say with confidence that actual program effects were indeed different from year to year.

FIGURE 24: EFFECTS ON CUMULATIVE GPA (2014-2016)

FIGURE 25: EFFECTS ON CORE GPA (2014-2016)

Key Finding: The AVID program has had positive annual effects on ninth-grade cumulative and core GPA.

0.30

-0.10

0.18

0.12

0.11

0.20

0.10

0.00

2014

Cu

mu

lativ

e G

PA

Eff

ec

t

2015 2016

0.40

-0.20

0.20

0.080.

06

0.30

0.20

0.10

-0.10

0.00

2014

Co

re G

PA

Eff

ec

t

2015 2016


2 8

Attendance Effects (2014-2016)

We also find evidence that the AVID program has had positive effects on school attendance each year, though with some variation. Though positive, effects on attendance were relatively small and non-significant in both 2014 and 2015 (Figures 26 & 27). In 2016, the program effect appeared to increase, with AVID students logging two and a half fewer unexcused days absent than their peers. That year, they also had an overall attendance rate more than a percentage point higher than similar

non-AVID students. When we statistically compared 2016 estimates to those of prior years, we found the difference in effects between 2014 and 2016 to be non-significant, but only marginally so (p=.06). This suggests that the program may have had somewhat larger effects in 2016 compared to those in 2014.

FIGURE 26: EFFECTS ON UNEXCUSED DAYS ABSENT (2014-2016)

FIGURE 27: EFFECTS ON ATTENDANCE RATE (2014-2016)

Key Finding: The AVID program has had positive annual effects on school attendance.

1.00

-5.00

-2.5

4

-0.9

6

-0.7

6

0.00

-1.00

-2.00

-4.00

-3.00

2014

Un

exc

use

d D

ays

Ab

sen

t Ef

fec

t

2015 2016

3.00

-1.00

1.49

0.730.

43

2.00

1.00

0.00

2014

Att

en

da

nc

e R

ate

Eff

ec

t

2015 2016


2 9

Credit Accumulation Effects (2014-2016)

Finally, there is strong evidence that the AVID program has had consistent positive effects on 9th grade credit accumulation over the past three years (Figures 28 & 29). Students who participated in middle school AVID earned a third of a credit more on average in the ninth-grade than their matched non-program

counterparts. Further, they failed significantly fewer courses on average as 9th graders. In the case of both earned credits and failed courses, AVID impacts were consistent between 2014 and 2016.

FIGURE 28: EFFECTS ON CREDITS EARNED (2014-2016)

FIGURE 29: EFFECTS ON FAILED COURSES (2014-2016)

Key Finding: The AVID program has had consistent positive annual effects on credit accumulation.

0.80

-0.10

0.38

0.33

-0.3

7

0.70

0.60

0.40

0.50

0.30

0.10

0.20

0.00

2014

Nin

th G

rad

e E

arn

ed

Cre

dits

Eff

ec

t

2015 2016

0.20

-1.40

-0.5

3

-0.4

7

-0.6

0

0.00

-0.40

-0.20

-0.60

-1.00

-0.80

-1.20

2014

Nin

th G

rad

e C

ou

rse

s Fa

iled

Eff

ec

t

2015 2016


3 0

Impact: Ninth-Grade Grade Point Average (2016)

Among 9th graders in 2016, students who had participated in the AVID middle school program earned significantly higher cumulative and core GPAs than their matched peers (Figures 30 & 31). This was true at the district level and across all focal subgroups except white students. Effects were particularly concentrated among

l o w - i n c o m e students of color, where middle school AVID p a r t i c i p a t i o n was associated with around a quarter of a GPA increase in the 9th grade.

FIGURE 30: 2016 9TH GRADE CUMULATIVE GPA BY AVID PARTICIPATION

Legend

AVID

COMPARISONGROUP



2.77

2.69

2.81

2.67

2.71

2.59

2.47

2.56

2.70

2.44

2.47



0.00

4.00

3.50

3.00

2.50

1.50

2.00

1.00

0.50


2.70

Cu

mu

lativ

e G

PA


Ninth-Grade Impacts: 2016 Cohort

As we did with the 8th grade impacts, here we break out 2016 9th grade impacts by demographic subgroup and high school feeder pattern. Our evaluation of the middle school AVID program indicates that it has had significant positive average effects on 9th

grade academic achievement school attendance, and credit accumulation. We also find evidence of positive program effects on ninth-grade ASPIRE math performance, but no similar effect on ASPIRE reading performance. We detail each of these findings below.

2016 Cohort Key Finding: Students who were in middle school AVID had significantly higher achievement in the 9th grade than their peers.


3 1

Legend

AVID

COMPARISONGROUP



2.62

2.54

2.68

2.48

2.58

2.43

2.29

2.39

2.52

2.28

2.27



0.00

4.00

3.50

3.00

2.50

1.50

2.00

1.00

0.50


2.56

Co

re G

PA


FIGURE 31: 2016 9th GRADE CORE GPA BY AVID PARTICIPATION


3 2

Legend

AVID

COMPARISONGROUP



Impact: Ninth-Grade Attendance (2016)

There is evidence that AVID middle school participation had a positive effect on 9th grade attendance among students in the 2016 cohort. Ninth graders who participated in middle school AVID had significantly fewer unexcused absent days—about two and a half fewer days on average (Figure 32). They

also had higher overall attendance rates than their matched peers (Figure 33). These patterns held true both at the district level, and across focal subgroups.

FIGURE 32: 2016 9th GRADE UNEXCUSED DAYS ABSENT BY AVID PARTICIPATION

FIGURE 33: 2016 9th GRADE ATTENDANCE BY AVID PARTICIPATION

1.95

2.17

1.99

1.81

2.63

4.49

5.29

5.21

2.52

6.19

3.62



0

7

6

5

3

4

2

1


1.31

Da

ys A

bse

nt


95%

95%

95%

94%

95%

96%

93%

93%

93%

94%

93%

94%



0%

100%

80%

60%

40%

20%


Att

en

da

nc

e


2016 Cohort Key Finding: Students who were in middle school AVID had significantly higher 9th grade attendance than their peers.


3 3

Legend

AVID

COMPARISONGROUP



Impact: Ninth-Grade Credit Accumulation (2016)

Our estimates indicate that middle school AVID participation led to more earned credits and fewer failed courses in the 9th grade. On average, this amounted to a fifth of a credit more for AVID students, though with somewhat higher gains among low-income students and students of color (Figure 34). AVID students also failed fewer courses as 9th graders

than their non-program peers on average. Among low-income students of color, this d i f f e r e n c e amounted to nearly a full course.

FIGURE 35: 2016 9th GRADE FAILED COURSES BY AVID PARTICIPATION

6.63

6.63

6.67

6.51

6.63

6.53

6.24

6.13

6.18

6.41

6.07

6.39



0

8

6

7

5

4

3

2

1


Earn

ed

Cre

dits


0.79

0.90

0.72

0.96

0.86

0.79

1.31

1.57

1.40

1.08

1.65

1.14



0.0

2.0

1.6

1.2

0.8

0.4


Faile

d C

ou

rse

s


2016 Cohort Key Finding: Students who were in middle school AVID earned significantly more credits and failed fewer courses in the 9th grade than their peers.

FIGURE 34: 2016 9th GRADE EARNED CREDITS BY AVID PARTICIPATION


3 4

Legend

AVID

COMPARISONGROUP



Impact: Ninth-Grade ASPIRE Scores (2016)

AVID middle school participants earned significantly higher scores on the 9th grade ASPIRE math test than their matched peers. The effect at the district level amounted to just over a point on the test’s unstandardized scale (Figure 36). For certain subgroups, the effect was considerably larger. Among male students of color, for example, AVID participants scored more than two and a half points higher than their matched counterparts. However, we

found no similar effect of program participation on ASPIRE reading test scores (Figure 37), where AVID and comparison group students earned nearly the same scores.

FIGURE 36: 2016 9th GRADE ASPIRE MATH BY AVID PARTICIPATION

FIGURE 37: 2016 9th GRADE ASPIRE READING BY AVID PARTICIPATION

423.

58

423.

66

423.

63

423.

45

423.

36

424.

17

422.

59

422.

32

421.

99

424.

18

421.

79

421.

51



0

450

400

350

300

250

200

150

100

50


ASP

IRE

Ma

th


421.

13

420.

72

420.

87

421.

82

420.

26

420.

37

420.

92

420.

82

420.

55

421.

88

420.

13

419.

36



0

450

400

350

300

250

200

150

100

50


ASP

IRE

Re

ad

ing


2016 Cohort Key Finding: Students who were in middle school AVID scored significantly higher on the 9th grade ASPIRE math test than their peers.


3 5

Ninth-Grade Impacts by High School Feeder Pattern (2016)

Our examination of AVID 9th grade impacts by high school feeder pattern revealed that district-level analyses obscured some important variation in effects (Table 7). In particular, we found that program effects on 9th grade attendance appeared to vary by feeder pattern. The strongest effects were observed among students from middle schools feeding into East High School and La Follette High School. Students from West High middle schools also experienced positive effects on 9th grade attendance, though there was little effect evident among students coming from Memorial High middle schools.

Further, we found that the positive average program impacts on 9th grade credit accumulation were

largely centered among students coming from middle schools feeding into West High School and East High School. This was true both in terms of earned credits and in terms of failed courses. Although program effects on cumulative and core GPA were not statistically significant at the feeder level as compared to those the district, they were nevertheless substantial at between a tenth and a quarter of a GPA. GPA effects were particularly concentrated among students coming from East, La Follette and West middle schools.

TABLE 7: 2016 9TH GRADE IMPACTS BY HIGH SCHOOL FEEDER PATTERN

OutcomeEast Schools

La Follette Schools

Memorial Schools

West Schools

AVID Comp. AVID Comp. AVID Comp. AVID Comp.Cumulative GPA 2.87 2.72 2.90 2.68 2.60 2.51 2.69 2.43Core GPA 2.72 2.50 2.79 2.55 2.44 2.35 2.53 2.28Unexcused Absent Days 2.18 6.51 1.39 4.53 2.33 2.59 1.60 4.17Attendance Rate 94.67 92.29 95.24 93.01 94.59 94.69 95.26 93.40Earned Credits 6.52 6.16 7.05 6.76 6.20 5.94 6.76 6.06Failed Courses 0.52 1.18 0.91 1.45 0.92 1.12 0.76 1.59ASPIRE Math 422.43 421.40 422.52 423.09 425.55 424.37 423.98 422.03ASPIRE Reading 418.40 417.90 421.87 423.10 423.84 422.31 421.41 421.47

Note: Bold indicates statistically significant difference (p<.05)

2016 Cohort Key Finding: AVID student gains in 9th grade attendance and earned credits differed considerably between high school feeder patterns.


L139 Education Building, 1000 Bascom Mall, Madison, WI 53706-1326 School of Education, University of Wisconsin-Madison

avid.org | www.wihopelab.org | wiscape.wisc.edu

Summary

The results presented in this evaluation indicate that AVID participation leads to modest positive gains in 8th grade academic performance and attendance. Over the course of the past four years of program implementation, AVID 8th graders have earned significantly higher average GPAs than comparable non-participants. They have also been less likely to log unexcused absences—a reduction of nearly a day overall. These findings persist across subgroups of interest and are somewhat stronger for low-income students and students of color.

The results also indicate that participation in AVID during the 8th grade helps students get on track academically for 9th grade. In each of the past

three years, students who participated in AVID earned higher 9th grade GPAs the subsequent year. They were also less likely to have unexcused absences during their freshman year, and tended to earn more credits on average than similar peers who did not participate in the middle school AVID program. Annually, these impacts were not always large enough to reach statistical significance. However, when we estimated aggregate program impacts over the past three years, we found them to be highly significant, suggesting that the middle school AVID program has had a net positive effect on the 9th grade academic success of participants during its’ existence.

The Wisconsin HOPE Lab Mission

The Wisconsin HOPE Lab was established in 2013 on the University of Wisconsin–Madison campus to engage in translational research aimed at improving equitable outcomes in postsecondary education.

The lab is housed in the School of Education and is led by Acting Director Dr. Jed Richardson. For more information, see www.wihopelab.com.

3 7


Appendix: AVID Middle School Impact Tables

Appendix Table 1: Eighth-Grade Impacts: Combined Cohort (2013-2016)

District Low-Income Students of Color White LI SOC Male SOC

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Cumulative GPA

2.90 0.05** 2.87 0.07** 2.89 0.06** 2.93 0.01 2.87 0.07** 2.87 0.03

(0.01) (0.02) (0.01) (0.03) (0.01) (0.02)

Core GPA

2.74 0.07* 2.68 0.09** 2.73 0.08** 2.76 0.03 2.69 0.08** 2.69 -0.01

(0.03) (0.03) (0.03) (0.06) (0.03) (0.04)

Unexcused Days Absent

2.01 -0.70** 2.35 -0.94** 2.14 -0.75** 1.57 -0.56 2.38 -0.82** 2.12 -0.98**

(0.26) (0.31) (0.25) (0.63) (0.29) (0.37)

Attendance Rate

94.54 0.76** 94.27 1.06** 94.51 0.93** 94.64 0.23 94.42 1.01** 95.03 0.78**

(0.20) (0.25) (0.24) (0.48) (0.29) (0.31)

Earned Credits

230.06 -0.20 230.09 -0.15 229.76 -0.12 231.02 -0.45 229.70 0.07 230.79 0.65

(0.35) (0.58) (0.47) (0.72) (0.65) (0.79)

Failed Courses

220.44 -0.02 220.04 0.33 220.16 0.51 221.35 -1.68** 219.84 0.51 220.04 0.44

(0.48) (0.66) (0.58) (0.52) (0.68) (0.51)

Sample Size 1029 1029 731 731 785 776 244 253 660 653 327 335

Appendix Table 2: Eighth-Grade Impacts: 2016 Cohort District Low-Income Students of

Color White LI SOC Male SOC

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Cumulative GPA

2.94 0.02 2.91 0.01 2.94 0.02 2.96 0.01 2.91 0.01 2.95 -0.08

(0.03) (0.04) (0.04) (0.04) (0.04) (0.07)

Core GPA

2.75 0.02 2.71 0.00 2.74 0.03 2.79 -0.03 2.70 0.00 2.76 -0.17

(0.09) (0.10) (0.11) (0.11) (0.11) (0.16)


1.81 -0.21 2.06 -0.39 1.93 -0.29 1.20 0.13 2.20 -0.40 1.23 -0.14

(0.26) (0.36) (0.31) (0.21) (0.37) (0.44)

Attendance Rate

95.12 0.43 94.95 0.51 94.98 0.56 95.81 -0.19 94.89 0.54 96.29 -0.48

(0.38) (0.46) (0.42) (0.39) (0.42) (0.48)

Earned Credits

230.51 -0.20 230.15 -0.17 230.24 -0.27 231.77 0.16 229.70 0.07 231.78 -0.10

(0.92) (0.94) (0.97) (1.94) (1.13) (2.14)

Failed Courses

221.03 -0.42 220.40 -0.32 221.16 -0.08 220.46 -1.97 220.60 -0.32 220.97 0.81

(0.80) (0.86) (0.86) (1.76) (1.07) (1.70)

Sample Size 241 241 179 173 201 197 40 44 166 164 74 78

Notes: *p<.10, **p<.05. Standard errors in parentheses. Reported means and impact estimates are covariate adjusted.

3 8


Appendix Table 3: Ninth-Grade Impacts: Combined Cohort (2014-2016)


Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Cumulative GPA

2.57 0.13** 2.47 0.19** 2.52 0.18** 2.72 -0.01 2.45 0.21** 2.48 0.13**

(0.04) (0.05) (0.04) (0.08) (0.05) (0.05)

Core GPA

2.43 0.11** 2.33 0.16** 2.38 0.16** 2.58 -0.04 2.31 0.18** 2.32 0.11*

(0.04) (0.04) (0.03) (0.08) (0.04) (0.05)


3.58 -1.46** 4.12 -1.87** 4.06 -1.98** 2.09 0.14 4.53 -2.14** 2.90 -1.15

(0.45) (0.60) (0.54) (0.47) (0.58) (0.66)

Attendance Rate

94.19 0.87** 93.60 1.29** 94.11 1.15** 94.44 0.01 93.70 1.34** 95.00 0.85

(0.28) (0.34) (0.23) (0.70) (0.23) (0.54)

Earned Credits

6.20 0.36** 6.10 0.43** 6.17 0.42** 6.33 0.20 6.07 0.45** 6.28 0.21

(0.08) (0.12) (0.10) (0.14) (0.12) (0.14)

Failed Courses

1.43 -0.53** 1.67 -0.71** 1.56 -0.66** 1.02 -0.14 1.77 -0.76** 1.30 -0.37

(0.12) (0.16) (0.14) (0.15) (0.17) (0.23)

Sample Size 745 745 530 524 567 553 178 192 483 467 254 245

Appendix Table 4: Annual 9th Grade Impacts: 2014 to 2016

2014 2015 2016

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Cumulative GPA

2.50 0.11 2.59 0.12* 2.59 0.18**

(0.07) (0.06) (0.05)

Core GPA

2.38 0.06 2.46 0.08 2.43 0.20**

(0.07) (0.07) (0.05)


2.80 -0.76 3.37 -0.96 4.49 -2.54**

(0.62) (0.58) (0.77)

Attendance Rate

94.80 0.43 94.42 0.73 93.37 1.49**

(0.36) (0.47) (0.58)

Earned Credits

6.11 0.37** 6.26 0.33** 6.24 0.38**

(0.13) (0.14) (0.11)

Failed Courses

1.62 -0.60** 1.36 -0.47** 1.31 -0.53**

(0.24) (0.20) (0.16)

Sample Size 239 239 246 246 260 260



3 9

Appendix Table 5: Ninth-Grade Impacts: 2016 Cohort


Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Control Mean

AVID Impact

Cumulative GPA

2.59 0.18** 2.47 0.23** 2.56 0.25** 2.70 -0.03 2.44 0.27** 2.47 0.23*

(0.05) (0.05) (0.03) (0.14) (0.05) (0.12)

Core GPA2.43 0.20** 2.29 0.25** 2.39 0.29** 2.52 -0.04 2.28 0.30** 2.27 0.29*

(0.05) (0.05) (0.04) (0.15) (0.06) (0.14)


4.49 -2.54** 5.29 -3.12** 5.21 -3.22** 2.52 -0.71 6.19 -3.56** 3.62 -2.30**

(0.77) (0.98) (0.99) (1.06) (1.17) (1.00)

Attendance Rate

93.37 1.49** 92.56 2.01** 93.29 2.03** 93.59 0.01 92.66 2.33** 94.26 2.03**

(0.58) (0.69) (0.52) (1.32) (0.58) (0.76)

Earned Credits6.24 0.38** 6.13 0.50** 6.18 0.48** 6.41 0.10 6.07 0.56** 6.39 0.15

(0.11) (0.15) (0.11) (0.28) (0.15) (0.18)

Failed Courses1.31 -0.53** 1.57 -0.67** 1.40 -0.68** 1.08 -0.12 1.65 -0.79** 1.14 -0.35

(0.16) (0.22) (0.14) (0.36) (0.21) (0.31)

ASPIRE Math422.59 0.99** 422.32 1.34**. 421.99 1.64** 424.18 -0.73 421.79 1.57** 421.51 2.66**

(0.43) (0.56) (0.59) (0.71) (0.61) (0.92)

ASPIRE Reading

420.92 0.22 420.82 -0.10 420.55 0.32 421.88 -0.06 420.13 0.13 419.36 1.01

(0.41) (0.55) (0.60) (0.69) (0.64) (1.16)

Sample Size 260 260 176 185 191 189 69 71 155 161 82 80


Documents

AVID Middle School 2015-2016 DISTRICT FINDINGS: FINAL …...L139 Education Building, 1000 Bascom Mall, Madison, WI 53706-1326 School of Education, University of Wisconsin-Madison avid.org