Teacher and Principal Effectiveness: What Do We Know?

Teacher and Principal Effectiveness:What Do We Know?

Philanthropy Roundtable Atlanta, GA September, 2009

© 2009 THE EDUCATION TRUST© 2008 THE EDUCATION TRUST

1. There are much bigger differences among our teachers than

we ever knew. And those differences matter hugely to

students.

© 2009 THE EDUCATION TRUSTSource: Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job. Washington, D.C.: The Brookings Institution.

10 Point Average Difference Between Top and Bottom Teachers

© 2009 THE EDUCATION TRUST

Comparing the Average Student in the Classrooms of Bottom Quartile and Top

Quartile Teachers

-6

-4

-2

0

2

4

6

Pe

rce

nti

le P

oin

ts G

ain

ed

/Lo

ss

Bottom QuartileTop Quartile

Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job. Washington, D.C.: The Brookings Institution.

10


Students in Dallas Gain More in Math with Effective Teachers: One Year Growth From 3rd-4th Grade

Source: Heather Jordan, Robert Mendro, and Dash Weerasinghe, The Effects of Teachers on Longitudinal Student Achievement, 1997.


LOW ACHIEVING STUDENTS IN TN GAIN MORE WITH EFFECTIVE TEACHERS: One

Year Growth

14

53

0

10

20

30

40

50

60

low high

Sanders and Rivers, Cumulative and Residual Effects of Teachers on Future Academic Achievement, 1998.


2. Impact of effective teachers swamps almost every other “intervention,” including

class size reduction.


Cumulative Teacher Effects On Students’ Math Scores in Dallas (Grades 3-5)

Source: Heather Jordan, Robert Mendro, and Dash Weerasinghe, The Effects of Teachers on Longitudinal Student Achievement, 1997.

Beginning Grade 3Percentile Rank= 55

Beginning Grade 3Percentile Rank= 57


3. Though there are large differences among our teachers we

pretend that there aren’t.


The Widget Effect

“When it comes to measuring instructional performance, current policies and systems overlook significant differences between teachers. There is little or no differentiation of excellent teaching from good, good from fair, or fair from poor. This is the Widget Effect: a tendency to treat all teachers as roughly interchangeable, even when their teaching is quite variable. Consequently, teachers are not developed as professionals with individual strengths and capabilities, and poor performance is rarely identified or addressed.”

• The New Teacher Project, 2009

Source:

Source:


4. Good teachers are not fairly distributed.


More Classes in High-Poverty Secondary Schools Taught by Out-of-Field* Teachers

High- Poverty

Low-poverty

Note: High Poverty school-75% or more of the students are eligible for free/reduced price lunch. Low-poverty school -15% or fewer of the students are eligible for free/reduced price lunch.

National

*Teachers with neither certification nor major. Data for secondary-level core academic classes (Math, Science, Social Studies, English) across USA.Source: Analysis of 2003-2004 Schools and Staffing Survey data by Richard Ingersoll, University of Pennsylvania 2007.


Math Classes at High-Poverty and High- Minority Schools More Likely to be Taught by Out of Field* Teachers

Note: High Poverty school-75% or more of the students are eligible for free/reduced price lunch. Low-poverty school -15% or fewer of the students are eligible for free/reduced price lunch. High minority school-75% or more of the students are Black, Hispanic, American Indian or Alaskan Native, Asian or Pacific Islander. Low-minority school -10% or fewer of the students are non-White students.

*Teachers with neither certification nor major. Data for secondary-level core academic classes (Math, Science, Social Studies, English) across USA.Source: Analysis of 2003-2004 Schools and Staffing Survey data by Richard Ingersoll, University of Pennsylvania 2007.


Students at High-Minority Schools More Likely to Be Taught By Novice* Teachers

*Novice teachers are those with three years or fewer experience.Source: Analysis of 2003-2004 Schools and Staffing Survey data by Richard Ingersoll, University of Pennsylvania 2007.

Note: High minority school-75% or more of the students are Black, Hispanic, American Indian or Alaskan Native, Asian or Pacific Islander. Low-minority school -10% or fewer of the students are non-White students.

© 2009 THE EDUCATION TRUST1998 by The Education Trust, Inc.


Tennessee: High poverty/high minority schools have fewer of the “most effective” teachers and more “least effective”

teachers

Source: Tennessee Department of Education 2007. “Tennessee’s Most Effective Teachers: Are they assigned to the schools that need them most?” http://tennessee.gov/education/nclb/doc/TeacherEffectiveness2007_03.pdf

Note: High Poverty/High minority means at least 75% qualify for FRPL and at least 75% are minority.


Low-Achieving Students are More Likely to be Assigned to Ineffective Teachers than Effective Teachers

Source: Sitha Babu and Robert Mendro, Teacher Accountability: HLM-Based Teacher Effectiveness Indices in the Investigation of Teacher Effects on Student Achievement in a State Assessment Program, AERA Annual Meeting, 2003.


4. We don’t know nearly enough about the characteristics of effective teachers…and even less about their

practices.


In last few years, volume of studies has grown rather dramatically. And we are

beginning to learn some of the basics.


What are we learning?• What Seems to

Matter• Experience, but only

for first year or two;• Content knowledge,

at least in math and science;

• Teachers’ own test performance.

• What Doesn’t Seem to Matter

• Traditional vs. Alternate Certification;

• Masters Degrees;• Experience beyond

the first several years.


But there is still a lot more digging to do, if only to

understand what seem to be exceptions…


For example, while novices usually aren’t as good as teachers with at

least 3 years experience….

• Studies in Louisiana show that some programs produce teachers who are more effective in year one, than veteran teachers in same district;

• Recent study in North Carolina found Teach For America Corps Members who taught in secondary schools as effective as veteran teachers in those same schools.


And within any “type” of teacher, there is a wide range of

effectiveness.

© 2009 THE EDUCATION TRUSTSource: Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job. Washington, D.C.: The Brookings Institution.

Similar Effectiveness, Regardless of Certification


Some nuances…

• Traditional a little better with younger children, especially in reading;

• Alternates a little better with older children, especially in math;

• Most differences in lower grades wash out by year 3.


Effectiveness More Important than Certification

“The difference between the 75th percentileteacher and the 50th percentile teacher for all

three groups of teachers was roughly five times as large as the difference between the average certified teacher and the average uncertified teacher.”

Three groups = traditionally certified, alternatively certified, and uncertified

Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job. Washington, D.C.: The Brookings Institution.


Bottom line: Most proxies for teacher effectiveness—especially

things like Masters Degrees or average years of experience-- don’t turn out to mean very

much.• That’s why incorporating and using

value-added measures is so important.


5. Principals are critical—at least in part because they are the ones who

attract and hold good teachers.


But if you think we know shockingly little about good

teachers…


You will be thoroughly despondent about how little we know about effective principals,

other than that they are RELENTLESS and GOOD TEAM

BUILDERS, and that ed leadership programs DON’T PRODUCE VERY

GOOD ONES.


6. The bottom line, though, is very clear: if we are going to improve

achievement and close gaps, we’ve got to act on what we know, while

also digging deeper for insights and answers.


And we’ve got to provide generous support for the break-the-mold

teacher and principal selection and preparation programs.


Why?


A few years ago, we got a wake up call when the 2000 PISA results were

published.


Rankings are for the 26 OECD countries participating in PISA in 2000, 2003, and 2006.

PISA PerformanceU.S.A. Ranks Near Bottom, Has Fallen Since 2000

Subject 2000 Rank(out of 26)

Mathematics 17th

Science 13th

Source: Organization for Economic Cooperation and Development (OECD), PISA 2006 Results , http://www.oecd.org/

2003 Rank(out of 26)

22nd

Tied for 17th

2006 Rank(out of 26)

22nd

19th


PISA 2003 Math Of 29 OECD Countries, U.S.A. Ranked 24th


U.S.A.


PISA 2006 Science Of 30 OECD Countries, U.S.A. Ranked 21st

U.S.A.

Source: NCES, PISA 2006 Results, http://nces.ed.gov/surveys/pisa/

Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average


PISA 2003 Problem-SolvingU.S. Ranks 24th Out of 29 OECD Countries


U.S.A.


Only place we rank high?

Inequality.


*Of 29 OECD countries, based on scores of students at the 5th and 95th percentiles.

PISA 2003: Gaps in Performance Of U.S.15 Year-Olds Are Among the Largest of OECD

Countries

Rank in Performance Gaps Between Highest and Lowest Achieving

Students *

Mathematical Literacy 8th

Problem Solving 6th

Source: Organization for Economic Cooperation and Development (OECD), PISA 2003 Results, data available at http://www.oecd.org/

http://www.oecd.org/


Among OECD Countries, U.S.A. has the 4th Largest Gap Between High-SES and Low-SES Students

PISA 2006 Science

Source: OECD, PISA 2006 Results, table 4.8b, http://www.oecd.org/

U.S.A.


Fortunately, we are making some progress—especially at the

elementary and middle school levels.


4th Grade Reading:Record Performance with Gap Narrowing

• NAEP 2008 Trends in Academic Progress, NCES

*Denotes previous assessment format


4th Grade Math:Record Performance with Gap Narrowing




8th Grade Reading: Recent Gap Narrowing for Blacks, Less for Latinos




8th Grade Math: Progress for All Groups, Some Gap Narrowing




And much of that progress is being led by urban school

districts, including the one right here in Atlanta.


Note: Data refer to the percentage point increase in the percent of students at Basic and Above between 2003 and 2007.

NAEP Grade 4 Reading – African American

Districts Outperforming Their States in Movement into Basic (2003-2007)

• NAEP Data Explorer, NCES

District District Movement State Movement

Atlanta 9 6

Los Angeles 7 5

San Diego 6 5



NAEP Grade 4 Math – African American Districts Outperforming Their States in Movement into Basic (2003-2007)


District District Movement State Movement

Boston 16 13

New York City 14 11

San Diego 11 7

Atlanta 10 8

Houston 7 5

Charlotte 2 0


NAEP Grade 8 Reading – African American Districts Outperforming Their States in Movement into Basic (2003-2007)


District District Movement State MovementHouston 9 5

Boston 7 3

Atlanta 6 2

Cleveland 6 -3

San Diego 2 -1

Charlotte 1 -3



We can’t go the distance, though, without strong teachers.


“Massive Impact”

“If the effects were to accumulate, having atop-quartile teacher rather than a bottom

quartile teacher four years in a row would be:

– enough to close the black-white test score gap…; and,

– Have twice the impact of reducing class size from 22 to 16.”

Source: Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job. Washington, D.C.: The Brookings Institution.

For more information, visit www.edtrust.org

1250 H Street N.W. Suite 700Washington, D.C. 20005202/293-1217

November 12-14, 2009, in Arlington, Va.

Send a team to our 2009 ConferenceWHEN THE GOING GETS TOUGH: Smart Choices and Bold Action

to Raise Achievement and Close Gaps

Documents

Teacher and Principal Effectiveness: What Do We Know?