Forty Years of Public School Reform: What the Data Tell Us Randy Keyworth Jack States The Wing Institute

Forty Years of Public School Reform:

What the Data Tell Us

Randy Keyworth

Jack States

The Wing Institute

A value of 1.0 = perfect flatnessA value of 1.0 = perfect flatness

Pancake = .957Pancake = .957

Kansas = .999Kansas = .999

Texas State UniversityTexas State University

Arizona State UniversityArizona State University

Is Kansas Flatter Than a Pancake?Is Kansas Flatter Than a Pancake?

National Assessment of Educational Progress (NAEP)National Assessment of Educational Progress (NAEP)

Only nationally representative and continuing assessment of what America’s students know and can do in: math, reading, science, writing, the arts, civics, economics, geography, and U.S. History.

Common metric for all states

Stays the same each year with carefully documented changes

Administered by National Center for Education Statistics (NCES) within the Institute of Education Sciences within the U.S. Department of Education

Provides a clear picture of student academic progress over time

Tests are continually scrutinized for reliability and validity by panels of technical experts within NCES and by external groups.




40 years NAEP reading data = .99640 years NAEP reading data = .996


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1992, 1994, 1998, 2000, 2002, 2003, 2005, and 2007 Reading Assessments.


Grade 4

Grade 8

Grade 12




40 years NAEP reading data = .99640 years NAEP reading data = .996

5 years IES graduation rate data = .9925 years IES graduation rate data = .992


IES, National Center for Education Statistics

Today’s Question:Today’s Question:

What’s going on????

1978 - 20041978 - 2004

Operated a large nonpublic school in SF Bay AreaOperated a large nonpublic school in SF Bay Areasix campuses adult programs

residential programs employment supportive services

public school consultation teacher training campus

CAPSES Board MemberCAPSES Board Member

President 1993-95 Regional Coordinator

GAC Committee local SELPA liaison

CAPSES “Street Cred”CAPSES “Street Cred”

The Wing InstituteThe Wing Institute

2004 - present2004 - present

independent, non-profit operating foundation

promote evidence-based education policies and practices

act as a catalyst to facilitate communication, cooperation and collaboration between individuals and organizations currently engaged in evidence based education

The Wing Institute’s Strategic Vision

Identify exemplars in evidence-based education

Develop networks to facilitate collaboration

Provide support for new ideas, research, and publications

Facilitate cross-discipline cooperation

www.winginstitute.org

Today’s DiscussionToday’s Discussion

What does performance monitoring data tell us about the effectiveness of 40 years of school reform?

What does education research data tell us about the effectiveness of 40 years of school reform?

REACHING AMERICA'S HEALTH POTENTIAL: A STATE-BY-STATE LOOK AT ADULT HEALTH Commission to Build a Healthier America May 2009 U.S. Census Data: American Community Survey (2007) U.S. Census Data: Behavioral Risk Factor Surveillance System Survey Data (2005-2007)

REACHING AMERICA'S HEALTH POTENTIAL: A STATE-BY-STATE LOOK AT ADULT HEALTH Commission to Build a Healthier America May 2009 U.S. Census Data: American Community Survey (2007) U.S. Census Data: Behavioral Risk Factor Surveillance System Survey Data (2005-2007)

SOURCE: Department of Health and Human Services (2003)

SOURCE: Department of Health and Human Services (2003)

U.S. Census Bureau, 2004

University of Maryland, Department of Sociology

Source: U.S. Department of Justice (2003)

Education and DelinquencyEducation and Delinquency

The odds of delinquency, given low academic performance, were about 2.1 times higher than those given high academic performance.

Intervention Studies showed that improvements in academic performance co-occurred or followed improvements in the prevalence of delinquency.

Maguin & Loeber, 1996

SOURCE: U.S. Department of Commerce, Census Bureau, Current Population Survey (CPS), October 1967 through October 2007.

• IES, National Center for Education Statistics

SOURCE: U.S. Department of Commerce, Census Bureau, Current Population Survey (CPS), October 1967 through October 2007.

Education is the civil rights issue of our generation.

Arne DuncanArne Duncan Secretary of Education Secretary of Education

20092009

PERFORMANCE FEEDBACKPERFORMANCE FEEDBACK

Performance feedback is one of the most important tools we have to assess and correct performance at all levels of the education system:

Student performance

Staff performance

Systems performance

STUDENT STUDENT PERFORMANCE FEEDBACKPERFORMANCE FEEDBACK

No Child Left Behind:

Adequate Yearly Progress (AYP)

graduation rates

state test scores in reading and math

States can establish standards and goals

The Education Trust (2005)

127,292 high school students dropped out....70% were black or hispanic

1,252,396 high school students dropped out....53% were black, hispanic, or native american

Comparison of State AYP RequirementsComparison of State AYP Requirements

THOMAS B. FORDHAM I NSTITUTE (2008)


Department of Education InitiativesDepartment of Education Initiatives

Adopting common standards and assessments

Building longitudinal data systems that measure student growth and success to support instruction


Standardized Graduation Rate calculation

DOE amended regulations require states to adopt the “cohort model” by 2010-11:

four-year adjusted cohort graduation rate

disaggregated by subgroups


Standardized Tests

Nation-wide standardized tests

43 States (so far)

2014 - 2015

Race to the TopRace to the Top

Build robust data systems to aggregate and disaggregate student outcome data

link teachers to students outcomes -- to distinguish between effective and ineffective teachers

to link effective teachers to their colleges of education – to distinguish between effective and ineffective programs

STAFF STAFF PERFORMANCE FEEDBACKPERFORMANCE FEEDBACK

1. Performance measures must be reliable and valid.

2. Staff should know the variables being measured

3. Skills must be identified to attain these measures

4. Staff feedback should be frequent and specific

5. Feedback should trigger and guide effecting coaching.

Hiring, Assignment, and Transfer in Chicago Public Schools Report from The New Teacher Project July 2007


Only 3 out of 1,000 teachers rated unsatisfactory

93% of teachers received superior or excellent ratings


87 Schools met criteria for being identified as “failing schools”

69 (79%) of these schools did not issue a single “unsatisfactory rating”



TRADITIONAL:

Performance measures must be reliable and valid.

Staff should know the variables being measured.

Skills must be identified to attain these measures.

Staff feedback should be frequent and specific.

Feedback should trigger and guide effecting coaching.

NoNo

NoNo

NoNo

NoNo

NoNo


LAUSD

Performance measures must be reliable and valid.

Staff should know the variables being measured.

Skills must be identified to attain these measures.

Staff feedback should be frequent and specific.

Feedback should trigger and guide effecting coaching.

Yes*

No

No

No

No

SYSTEM PERFORMANCE FEEDBACK

Most systemic school reform strategies have focused on

“structural fixes”

Structural Fixes….Commonly Held BeliefsStructural Fixes….Commonly Held Beliefs

SCHOOL RESOURCES

increase funding for students

higher pay for teachers

TEACHER QUALITY

more “Highly Qualified Teachers” (NCLB)

more teachers with credentials

more teachers with advanced degrees

more teachers with professional certifications (NBPTS)

comprehensive induction for new teachers

SCHOOL MODELS

smaller class size

school choice

charter schools

SCHOOL REFORM INITIATIVES

A Nation at Risk

Goals 2000

No Child Left Behind

Structural Fix: Increased Funding for Education

SOURCE: U.S. Department of Education, National Center for Education Statistics. (2009). Digest of Education Statistics, 2008 (NCES 2009-020), Chapter 2 and Table 179.

IES National Center for Education Statistics Digest of Educational Statistics: 2008

Structural Fix: Increased Pay for TeachersStructural Fix: Increased Pay for Teachers

Structural Fix: More Highly Qualified TeachersStructural Fix: More Highly Qualified Teachers

Structural Fix: More “Credentialed” TeachersStructural Fix: More “Credentialed” Teachers

SOURCE: National Education Association, Status of the American Public School Teacher, 2000-01. (This table was prepared August 2003.)

Structural Fix: Advanced Degrees for Teachers Structural Fix: Advanced Degrees for Teachers

Master’s Degrees

Bachelor’s Degrees

Structural Fix: More Board Certified TeachersStructural Fix: More Board Certified Teachers

Smith & Ingersoll, 2004

Structural Fix: More Induction Support for TeachersStructural Fix: More Induction Support for Teachers

Structural Fix: Class Size Reduction

32 States now have class size reduction programs in law:

Texas (1983) California (1996)

Tennessee (1989) New York (1997)

Minnesota (1990) Florida (2002)


Structural Fix: School Choice

TRENDS IN THE USE of SCHOOL CHOICE: 1993 To 2007, IES

There are currently over 5,000 Charter schools in the United States

U.S. Department of Education, National Center for Education Statistics

Structural Fix: More Charter SchoolsStructural Fix: More Charter Schools

A History of Structural Fixes

SCHOOL RESOURCES

increase funding for students

higher pay for teachers

TEACHER QUALITY

more “Highly Qualified Teachers” (NCLB)

more teachers with credentials

more teachers with advanced degrees

more teachers with professional certifications (NBPTS)

comprehensive induction for new teachers

SCHOOL MODELS

smaller class size

school choice

charter schools

SCHOOL REFORM INITIATIVES

A Nation at Risk

Goals 2000

No Child Left Behind

✔✔

✔✔✔

✔

✔

✔

✔✔✔

✔

✔


National Assessment of Educational Progress (NAEP)National Assessment of Educational Progress (NAEP)

Basic denotes partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.

Proficient represents solid academic performance. Students reaching this level have demonstrated competency over challenging subject matter.

Advanced represents superior performance.

Grade 4Grade 4 Grade 8Grade 8 Grade 12Grade 12

Basic = 208 Basic = 243 Basic = 265

Proficient = 238 Proficient = 281 Proficient = 302

Advanced = 268 Advanced = 323 Advanced = 346

1.2 million students below basic2.4 million students below proficient

347,900 Students


Legislatively mandated four year comprehensive study.

Major CA research orgs: AIR, RAND, PACE, WestEd, EdSource

Conclusions (Final Report):

1. The relationship of CSR to student achievement was inconclusive…attribution to any gains in scores to CSR is unwarranted.

2. Students in CSR received more individual attention, but similar instruction and curriculum.

3. CSR remains very popular with parents and teachers.

CSR Research Consortium

PERFORMANCE FEEDBACK

we have had lousy performance feedback systemswe have had lousy performance feedback systemsstudent performance

teacher performance

system performance

we have have been focusing on the wrong thingswe have have been focusing on the wrong thingsstructural fixes haven’t worked

enormous time, energy and money has been wasted

there is hope on the horizonthere is hope on the horizonbetter student longitudinal data systems

standardized formulas for graduation rate data

uniform standardized tests

better tools for evaluating teacher performance

What’s going on????

THE END

The Education CultureThe Education Culture

Education has been a non-evidence-based culture:

Teaching is an “art”, not a “science”

Constructivism, anti-science, anti-measurement

Ruled by ideology, preferences, hunches, fads

The teaching profession has been stuck in a “guild” mentality

Teacher tenure

Resistance to performance appraisal

Most school reform strategies are not based on scientific evidence

Lack of fundamental research into teaching, curriculum, systems

Special interests drive initiatives

The culture drives simplistic, commonly held “beliefs”

Lack of performance feedback

What is the relationship between What is the relationship between teacher experience and effectiveness?teacher experience and effectiveness?

Kane, Rockoff, Staiger, 2006

What is the relationship between What is the relationship between teacher experience and effectiveness?teacher experience and effectiveness?

STUDENT PERFORMANCE FEEDBACKSTUDENT PERFORMANCE FEEDBACK

Sample of Graduation Rate calculations:

counting only seniors for graduation rates

(not freshman entering high school)

percentage of kids who got their diplomas in four years or less

number of graduates compared to 12th grade enrollment on the last day of school

Structural Fix: Major School Reform InitiativesStructural Fix: Major School Reform Initiatives

A Nation at Risk (1984)

Goals 2000 (1994)

No Child Left Behind (2001)

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1978, 1982, 1986, 1990, 1992, 1994, 1996, 1999, 2004, and 2008 Long-Term Trend Mathematics Assessments.

What’s Going On????What’s Going On????

TOOLS:

RoadmapRoadmap: a model for identifying critical components of evidence-based education

Data-mining: Data-mining: a mechanism for identifying, analyzing, and disseminating evidence

ResearchR

eplicabilityS

ustainability

Evidence-based Education Roadmap

Practice

Keys to Evidence-Based Education

RESEARCH

selecting interventions

PERFORMANCE MONITORING

evaluating interventions

The Wing Institute ServicesThe Wing Institute Services

• Publications

• Research

• Public Policy Analyses

• Information Clearinghouse

• The Evidence-based Education Knowledge Network

• Professional Forums

Documents

Forty Years of Public School Reform: What the Data Tell Us Randy Keyworth Jack States The Wing Institute