Is Education Reform in New Orleans Working? A Few Facts Swimming in a Sea of Unknowns Michael Schwam-Baird and Laura Mogg The Cowen Institute for Public

Is Education Reform in New Orleans Working? A Few Facts Swimming in a Sea of Unknowns

Michael Schwam-Baird and Laura MoggThe Cowen Institute for Public Education InitiativesOctober 16, 2009

• Situation Overview• Claims made Regarding School

Reform• What the Data can tell us about

School Reform in New Orleans• Conclusions

Presentation Outline

Situation Overview

• NOPS was widely recognized as one of the lowest-performing school systems in the country. In addition the school system was poorly managed and had significant problems with corruption.

• Just before Katrina, NOPS was declared broke and its finances were handed over to a corporate turnaround firm.

• Voters in 2003 approved a constitutional amendment giving the state the power to intervene in chronically failing schools. With its new power, the state established the Recovery School District (RSD) as a mechanism for this intervention.

The State of NOPS before Hurricane Katrina

Hurricane Katrina and the Transformation of NOPS

• The confusion and uncertainty after the storm were, to some, an opportunity to overhaul public education governance in New Orleans.

• The legislature, with Gov. Blanco’s support, passed Act 35, expanding the RSD’s authority to take over low-performing schools in New Orleans.

• With its new authority, the state took over more than 100 New Orleans public schools that were now considered failing.

Charter Schools

• Chartering was a way for the OPSB to re-open schools quickly as well as take advantage of federal charter school start-up grants.

• Despite having the authority to operate schools, the RSD viewed its role as overseeing charters. A rapid return of students to the city made RSD-run schools a necessity.

• The plan to charter schools was initially very controversial, though this has abated somewhat and the number and proportion of all public schools that are charters continues to grow.

School Choice

• School choice options prior to Katrina were limited.

• In a system with a significant number of charter schools, choice is inevitable

• After Katrina, both the RSD and OPSB were forced to institute choice for all schools out of necessity.

• City-wide school choice was eventually codified as official policy by the RSD and OPSB.

Human Capital Policies

• Charter and RSD-run schools were not subject to the teachers union contract and selected school-based personnel at the school site.

• Charter schools are not subject to state tenure and seniority laws.

• RSD-run schools’ work rules and a salary schedule are established at the district level.

• Many OPSB teachers are veterans from before Katrina while the RSD has used national organizations to attract inexperienced new teachers.

Resource Allocation

• OPSB-run schools continue to be funded according to the prior year’s staffing needs.

• RSD-run schools are funded on a per-pupil basis according to a weighted student formula in amounts significantly more than before Katrina.

• All charter schools receive a set amount per pupil, regardless of the needs of the students they enroll. These schools have greater budget autonomy.

The Effect of the Reforms

• There is disagreement about the extent to which these reforms have been implemented and the effects the reforms have had.

• It is difficult to assess these claims with the information available.

• Recent public opinion polls have shown that New Orleans voters see the changes to the school system as largely positive.

Claims Made Regarding School

Reform

School Choice/ Open Enrollment

► Choice has created an educational shopping mall. Parents will not choose low-performing schools and they will forced to close. (Charter advocates)

► Choice can be a strategy to meet the diverse needs of all children. (Community and political leaders)

► Many students are still unable to access the city’s better public schools creating a system of high and low performing schools. (Parent advocates & Union)

► Parents lack the information and resources to take advantage of school choice. (Parent advocates)

Introduction of Charter Schools► Charter autonomy in hiring and other functions

allows them to be higher quality schools. (Charter advocates)

► Charters have seen greater gains in student achievement than traditional public schools. (Charter advocates and leaders)

► Charter schools “cream” the best students, and access restrict access to some students. (Parent advocates and researchers)

► Charter schools enroll a smaller percentage of special education students than traditional public schools. (Parent advocates)

Human Capital

► The union’s resistance to change caused problems pre-Katrina. The elimination of collective bargaining provided a fresh start. (RSD and charter advocates)

► New teachers are motivated, willing to collaborate, and open to new strategies. (RSD)

► Evidence from national research finds that experienced teachers are more effective at raising student achievement. (Union)

► The RSD schools are seen as a “proving ground” for teachers who will either improve and move to other schools or leave the profession. (Union and teachers)

Resource Allocation

► The increase in per-pupil funds being spent by the RSD was necessary given the chronic under-funding of the district as well as the increased cost of educating children in poverty. (RSD)

► Many of these funds are one-time recovery dollars and the RSD will be unable to sustain its reforms once this money dried up. (Researchers and advocates)

► Charter schools, because of their ability to access private funding sources, are creating a system with large disparities in per-pupil spending between schools. (Researchers and parent advocates)

What the Data on Public Education Can Tell Us about School Reform in

New Orleans

System-wide Data

Year Percent Basic & Above Year Percent Basic & Above

4th Grade ELA 4th Grade Math2003 38.0% 2003 34.0%2005 47.0% 9.0% 2005 43.0% 9.0%

2007 48.9% 2007 43.4%2009 61.0% 12.1% 2009 52.9% 9.5%

(Post Katrina) - (Pre Katrina) Growth Rate 3.1%


8th Grade ELA 8th Grade Math2003 27.0% 2003 26.0%2005 30.0% 3.0% 2005 36.0% 10.0%

2007 38.6% 2007 36.6%2009 46.6% 8.0% 2009 44.4% 7.8%


(Post Katrina) - (Pre Katrina) Growth Rate -2.2%

GEE ELA GEE Math2003 29.0% 2003 33.0%2005 40.0% 11.0% 2005 39.0% 6.0%

2007 37.3% 2007 43.4%2009 48.6% 11.3% 2009 57.8% 14.4%



Pre- & Post-Katrina Public School Student Achievement in New Orleans

Significantly, proficiency rates grew in every testing area in the 3-year periods before and after Katrina.

Year Louisiana NOLANOLA -

Louisiana Year Louisiana NOLANOLA -

Louisiana

4th Grade ELA

4th Grade Math

2003 - 2005 6.0% 9.0% 3.0% 2003 - 2005 3.0% 9.0% 6.0%

2007 - 2009 3.9% 12.1% 8.2% 2007 - 2009 2.2% 9.5% 7.3%

(Post Katrina) - (Pre Katrina) Difference in Growth Rates

5.2%(Post Katrina) - (Pre Katrina) Difference in Growth Rates

1.3%

8th Grade ELA

8th Grade Math

2003 - 2005 0.0% 3.0% 3.0% 2003 - 2005 2.0% 10.0% 8.0%

2007 - 2009 6.0% 8.0% 2.0% 2007 - 2009 5.9% 7.8% 1.9%


-1.0%(Post Katrina) - (Pre Katrina) Difference in Growth Rates

-6.1%

GEE ELA GEE Math

2003 - 2005 8.0% 11.0% 3.0% 2003 - 2005 3.0% 6.0% 3.0%

2007 - 2009 6.1% 11.3% 5.2% 2007 - 2009 8.7% 14.4% 5.7%


2.2%(Post Katrina) - (Pre Katrina) Difference in Growth Rates

2.7%

Growth in Students Performing Basic & Above in Louisiana and New Orleans

On every state test, both before and after the storm, New Orleans outgrew the state in terms of the percentage of students scoring basic and above.

City-wide Performance Conclusions

• The growth rates in the three years after Katrina show that in three out of six tests the growth rate of public schools has increased significantly. In two of the six tests the growth rate grew by less than one percentage point. In 8th grade Math, the rate of growth has declined since Katrina.

• In four out of the six state tests, New Orleans outgrew the state at a faster pace after Hurricane Katrina. However, in two out of the six tests, New Orleans outgrew the state at a slower pace after the storm.

What does it tell us?

• The growth in all areas of testing since the storm is certainly good news.

• In most areas, growth since the storm has outpaced growth before the storm. In a few areas, growth has not been greater.

• It is difficult to tie these trends to any particular aspect of reform. It’s not clear what the effects should be. We take these results to be generally positive, but not mind-blowing by comparison to growth before the storm. But you could make a lot of different arguments based on these numbers and others, and many do.

Teachers by Years of Experience Before and

After Katrina

New Orleans Teachers by Years of Experience before and after Katrina

0-1 Years 2-3 Years 4-10 Years 11-14 Years 15-19 Years 20-24 Years 25+ Years

2004-2005 9.7% 7.3% 24.7% 9.0% 8.9% 10.9% 29.5%

2007-2008 36.7% 17.2% 19.3% 4.8% 5.5% 4.9% 11.6%

Change (04-05

and 07-08) 27.0% 9.9% -5.4% -4.2% -3.4% -6.0% -17.9%

The proportion of teachers with 0-1 and 2-3 years of experience has increased significantly.


• The proportion of new and less experienced teachers has grown dramatically.

• This does not seem to have had a generally negative effect on the school system, whose results have generally improved since the storm.

• Predictions of dire consequences from new teachers seem to have been unfounded. Predictions of radical improvement due to new teachers have yet to be proven.

Student Population

• The percentages of students who were eligible for free or reduced lunch has changed only slightly since before the storm (77 percent in 2004-2005; 85 percent in 2008-2009).

• Within the category of students who are eligible for the school lunch program, there is a lot of room for variation.

• The proportion of families living in extreme poverty in New Orleans has declined according to the US Census Bureau (12.3% of families had less than $10K in income in 2005; 6.4% in 2007).

• This may mean that the proportion of very poor students, who are most likely to have educational challenges, has also declined.


• Not much at the macro-level.

• These differences are suggestive because poverty is so closely correlated with achievement. But they are not a definitive explanation for higher achievement.

Resources:Total Current Expenditures

Year NOLA Current Expenditures Per Pupil LA Current Expenditures Per Pupil

2002 - 2003 $6,571 $6,9062003 - 2004 $7,296 $7,2482004 - 2005 $7,893 $7,630

Public School Spending: Pre-Katrina

Year Current Expenditures Per Pupil LA Current Expenditures Per Pupil

2006 - 2007 $14,122 $8,8812007 - 2008 $15,557 $9,966

Public School Spending: Post-Katrina

Salaries per Pupil

All RSD Schools $4,198All OPSB Schools $3,914

All NOLA Schools $4,074

All Louisiana Schools $3,522

2007-2008 Teacher Salaries per Pupil

All RSD Schools $6,056All OPSB Schools $6,547

All NOLA Schools $6,182


2007-2008 Total Salaries per Pupil

Expenses per Pupil and Instruction Spending

More resources are being spent on instruction post-Katrina, but most of this extra money is not being spent on teachers.

All NOLA Schools All Louisiana Schools

Total Supplies $2,352 $994Purchased Technical and Professional Services $2,401 $439Repairs and Maintenance $772 $230

2007-2008 Expenses per Pupil

All RSD Schools $9,641All OPSB Schools $10,655All NOLA Schools $9,862


2007-2008 Total Instruction Spending

Spending Conclusions

• Schools are spending much more than before the storm on school operations.

• Some of these extra resources are being used for instructional purposes, though not necessarily on teachers.

• It is unclear how much extra resources might be responsible for better performance. The role of resources in schools is hotly debated given all of the other inputs that affect achievement.

• Extra resources are, however, a major difference between the pre-Katrina and the post-Katrina environment. If they are critical to sustaining improvements in education, then we should be concerned now that most of those extra resources are gone.

Data by School Type

4th Grade School Type

2007 ELA: Basic & Above


07 to 09 Growth

2007 Math: Basic & Above


07 to 09 Growth

OPSB Charter 79.9% 88.9% 9.0% 74.0% 82.1% 8.1%OPSB-run 80.2% 94.3% 14.1% 76.0% 89.5% 13.5%RSD Charter 48.6% 59.7% 11.1% 41.8% 55.1% 13.3%RSD-run 18.6% 43.7% 25.1% 14.0% 30.3% 16.3%All NOLA 48.9% 61.0% 12.1% 43.4% 52.9% 9.5%

8th Grade School Type



07 to 09 Growth



07 to 09 Growth


GEE School Type2007 ELA: Basic

& Above2009 ELA:

Basic & Above07 to 09 Growth



07 to 09 Growth


2007-2009 Performance by School Type in New Orleans

School Type Performance Conclusions

• The school type with the highest growth rate varies by grade-level and subject. No particular school type seems to outperform the others in terms of growth.

• Large differences in absolute scores between RSD-run schools and RSD charters in 2007 indicates that RSD charter students probably entered school with higher levels of preparation, giving some support to “creaming” claims among non-selective schools.

• Recent Stanford study does provide evidence that charter students as a whole outperform their peers in non-charter schools across Louisiana. The data for this is statewide and from before and after Katrina.

Teachers by Years of Experience

The RSD-run schools have relied heavily on new teachers, while the OPSB-run schools have a large population of veteran teachers.

0-1 Years 2-3 Years 4-10 Years 11-14 Years 15-19 Years 20-24 Years 25+ Years

OPSB-run 4% 1% 16% 7% 12% 12% 48%OPSB-charter 18% 11% 20% 9% 11% 12% 20%RSD-run 60% 26% 13% 0% 0% 0% 1%RSD-charter 27% 14% 26% 8% 7% 5% 13%

All NOLA 37% 17% 19% 5% 6% 5% 12%

2007-2008 Teachers by Years of Experience by School Type in New Orleans


• No arrangement of experience levels seems to be particularly associated with better or poorer performance.

• Schools with large proportions of veterans have shown promising results, as have schools with high proportions of new teachers.

• Perhaps site selection is the more important variable?

Students Eligible for Free and Reduced

Lunch

The real differences in eligibility between RSD charter schools, RSD-run schools, and even OPSB-run schools are probably relatively negligible

October 2007 February 2008 October 2008 February 2009OPSB Charter 68.3% 69.81% 65.51% 66.21%OPSB-run 74.3% 77.92% 76.44% 89.40%RSD Charter 93.7% 94.77% 92.87% 94.02%RSD-run 77.4% 82.79% 86.90% 87.38%

All NOLA 79.7% 81.18% 83.30% 83.35%

Percentage of Students Eligible for Free and Reduced Lunch

Student with Disabilities

There is some evidence for the claims that students with special needs are more likely to end up in RSD-run schools by comparison to charter schools.

2007-2008 2008-2009

OPSB Charter Schools 4.54% 4.66%OPSB-run Schools 6.46% 7.31%RSD Charter Schools 6.10% 6.27%RSD-run Schools 8.44% 10.45%

All NOLA Schools 6.52% 7.54%

Percentage of Students with Disabilities


• There are not great differences in the proportion of students eligible for free and reduced lunch among non-selective schools, though there could be some variation within income levels that this measure does not capture.

• There are important differences in the percentage of special education students educated by non-selective charters versus non-charters.

Average Expenditures per

Pupil

RSD-run schools, a category that includes the district’s central office, are spending much more per student than the city’s charter schools, regardless of type.

[Though they cannot be broken out the same way, OPSB-run schools and the OPSB central office also probably spend much more per-pupil than charters.]

RSD-run Schools $20,770RSD Charter Schools $10,449OPSB Charter Schools $9,671All Charter Schools $10,224

2007-2008 Average Expenditures per Pupil

Expenditures by Object in RSD-run and RSD Charter Schools

Teacher Salaries All Salaries Benefits

Professional and Technical Services Supplies

RSD-run Schools $4,955 $6,752 $1,847 $4,204 $3,934RSD Charter Schools $3,323 $5,252 $1,324 $1,003 $793

Louisiana Average $3,540 $5,871 $2,063 $439 $994

2007-2008 Expenditures by Object in RSD-run and RSD Charter Schools

Though RSD-run schools do spend more on teacher salaries than RSD charter schools, the biggest difference is in contracted services and supplies.


• There is little evidence to support claims that charters have access to more resources than district-run schools.

• The RSD is spending a lot of money on operations. While contracted services and supplies may be able to be cut back, the extra money spent on teachers may be more painful to cut when federal money runs dry.

Conclusion

• Many of the claims made by advocates, union officials, researchers, and public officials are based on shaky evidence. This is due to both the politics of school reform and a lack of reliable data.

• Overall, student achievement in the city’s public schools has improved since the storm and, in many cases, at a faster rate than before the storm. However, achievement growth is not radically greater than before Katrina.

• The majority of New Orleanians report that the system of schools that has emerged since Katrina is better than what existed before.

• There is evidence from at least one study that students in Louisiana charter schools are outperforming their peers in district-run schools.

What We Know

Some of the Many Questions

That Remain• What effect are the many new teachers who have entered the system

having on student achievement? Are some more effective than others?

• To what extent are demographic shifts in New Orleans caused by Katrina affecting aggregate scores across the system?

• How are the differences in student and family characteristics (including prior student achievement) affecting achievement patterns across different types of schools?

• How mobile are students? How often do special education students switch schools?

• What role are additional one-time resources playing? Will their eventual loss negatively impact the performance of public schools?

• Which reforms are driving achievement the most? With finite resources, this is important!

Is Education Reform in New Orleans Working? A Few Facts in a Sea of Unknowns

Cowen InstituteOctober 16, 2009

Documents

Is Education Reform in New Orleans Working? A Few Facts Swimming in a Sea of Unknowns Michael Schwam-Baird and Laura Mogg The Cowen Institute for Public