The State of Teacher Evaluation Reform

7/30/2019 The State of Teacher Evaluation Reform

1/62

The State of Teacher Evaluation ReformState Education Agency Capacity and the Implementation

o New Teacher-Evaluation Systems

Patrick McGuinn November 2012

WWW.AMERICANPROGRESS.O


2/62


3/62

The State of TeacherEvaluation ReformState Education Agency Capacity and the

Implementation o New Teacher-Evaluation Systems

Patrick McGuinn November 2012


4/62


5/62

Contents 1 Introduction and summary

7 Implementation and state capacity gaps

11 Six state case studies

11 Tennessee

17 Colorado

21 New Jersey

27 Pennsylvania

31 Delaware

33 Rhode Island

37 Key lessons and challenges

49 Recommendations

51 Conclusion

52 About the author and acknowledgements

53 Appendix

54 Endnotes


6/62


7/62

Introduction and summary | www.americanprogress.

Introduction and summary

Te Obama adminisraions Race o he op compeiive gran program iniiaed

an unprecedened wave o sae eacher-evaluaion reorm across he counry.1 o

dae, mos o he scholarly analysis o his aciviy has ocused on he design o he

evaluaion insrumens2 or he implemenaion o he new evaluaions by disrics

and schools.3 Bu litle research has explored howstates are managing and sup-

poring he implemenaion o hese reorms. As U.S. Deparmen o Educaion

Secreary Arne Duncan has remarked: because eacher evaluaion sysems are

sill a work in progress, i is vial ha school leaders and adminisraors coninueo solici eedback, learn rom heir misakes, and make improvemens.4 I has

become increasingly clear ha he role o sae educaion agencies will be crii-

cal as school disrics ener wha or mos will be unchared erriory. As Edward

Crowe argued in his recen Cener or American Progress repor on eacher

preparaion, Te capaciy and commimen o saes o implemen hese Race o

he op aciviies will deermine success or ailure.5 And as highlighed in recen

news repors, many saes are sruggling o implemen heir new eacher-evalua-

ion sysems and mos o he Race o he op winners have asked o exend heir

imeables or compleing his work.6

Tis paper oers an assessmen o how early adoper saes deparmens o

educaion have underaken he preparaion and implemenaion o new evalu-

aion sysems. I also idenies challenges and lessons ha can be used o guide

uure reorm eors in his area. Developing new eacher-evaluaion sysems has

been idenied by scholars and policymakers alike as a crucial par o improving

eacher qualiy and raising suden academic perormance across he counry.7 I

is imperaive ha we learn more abou he mos eecive way or sae educaion

agencies o suppor disrics in his dicul work.

Tis assessmen o he aciviies o sae deparmens o educaion is based

on comparaive case sudies o six saes: Colorado, Delaware, New Jersey,

Pennsylvania, Rhode Island, and ennessee. Tese paricular saes were seleced

because hey are early adopers in he area o eacher-evaluaion reorm and


8/62

2 Center or American Progress | The State o Teacher Eval uati on Reor m

because heir saes and/or educaion agencies have underaken dieren

approaches o implemening he reorms. wo o he saesennessee and

Delawarewere iniial Race o he op winners, while he oher saes won

smaller grans in laer rounds. Research consised o a review o he scholarly and

hink ank research on sae educaion agency capaciy and eacher-evaluaion sys-

ems; analysis o repors and daa rom he sae educaion deparmens websiesand rom organizaions such as he Council o Chie Sae School Ocers; a sudy

o media coverage o he reorm eors in he six saes; and 15 inerviews wih

naional expers on eacher-evaluaion reorms and sae educaion agency and

local educaion agency sa in each sae.

Te cenral quesions probed and answered in his repor include:

How are sae educaion deparmens adjusing o heir new, more ambiious

roles and responsibiliies in he wake o Race o he op?Wha seps are sae educaion agencies aking o resrucure hemselves or

hese new responsibiliies?Wha kinds o capaciynancial, personnel, echnicalhave sae educa-

ion agencies added o suppor he implemenaion o new eacher-evaluaion

sysems, and wha kinds o capaciy are sill lacking? o wha exen and in wha ways are sae educaion agencies relying on exernal

capaciy by conracing ouside consulans o provide echnical assisance wih

his work?Wha is he role o philanhropic organizaions in supporing sae educaion

agencies in his work? How rapidly and eecively are saes implemening heir new eacher-evalua-

ion sysems? How are saes approaching his work dierenly rom one anoherdo some

approaches appear o be more or less eecive han ohers?Wha challenges are emerging and how are saes addressing hem?Wha lessons can be learned rom hese early-adoper saes ha can inorm

eacher-evaluaion reorm in he res o he counry?

I is clear ha sae educaion agencies are working hard o realign heir organiza-

ions wih he many new responsibiliies ha have been hrus upon hem in hewake o he ederal No Child Le Behind Ac and Race o he op programs.8

Sae eors o implemen new eacher-evaluaion reorms oer excellen exam-

ples o he ways ha sae educaion agencies are adaping o heir new role as well

as he ways in which ongoing capaciy gaps coninue o impede heir work.


9/62


Improving eacher qualiy has become he cenerpiece o he Obama adminisraions

educaion agenda and o he conemporary school-reorm movemen. Te many

challenges ha have already emerged, however, also highligh how dicul his work

is and how i is complicaed by shor imelines and limied sae educaion agency

sang and unding. A number o key challenges o implemening new eacher-evalu-

aion sysems have emerged rom he work o he early-adoper saes. Some o hesechallenges, which can inorm he eors o oher saes going orward, include:

The philosophical/statutory/constitutional debate over the proper role of state

education agencies. I is imporan o recognize ha all sae educaion agencies

are no he sameeach agency has a unique hisory and operaes in a dieren

scal, poliical, sauory, and consiuional conex. In paricular, saes vary

signicanly in heir atachmen o local conrol o schools and he proper role

o he sae in educaion. Tis has a major impac on how sae educaion agen-

cies approach eacher-evaluaion reorm. A relaed issue revolves around he

radiional ocus o sae educaion agencies on compliance and accounabiliyaciviies, which has made local educaion agencies wary o being candid abou

wheher and how hey migh be sruggling o implemen reorm and made hem

relucan o seek ou assisance.

The amount of flexibility in state evaluation systems varies greatly. Saes vary

widely in he amoun o cenralizaion and sandardizaion hey have man-

daedhrough saue or regulaionin he new eacher-evaluaion sysems.

Tis variance has a major impac on he sae educaion agencys approach o

supporing implemenaion. A clear ension is emerging beween a saes desire

o give disrics fexibiliy o selec or adap evaluaion insrumens ha are bes

suied o heir paricular circumsances, and he sae educaion agencys limied

capaciy o provide implemenaion suppor or a wide array o insrumens.

State education agency restructuring and the human capital demands. Sae

educaion agencies in many saes are undergoing a radical resrucuring and re-

sang as hey embrace a shi rom being compliance moniors o service delivery/

school-improvemen organizaions. Tis resrucuring is dicul and ime-consum-

ing work and, while necessary o carry ou new responsibiliies over he long erm,

creaes a number o shor-erm challenges. I will ake some ime or his organiza-ional shake ou o be compleed and or new srucures and sa o acclimae o

heir new roles. Many sae educaion agencies have creaed new eacher-eecive-

ness unis, bu he degree o which hese unis have been well-inegraed wih oher

unis appears o vary and longsanding concerns abou agency siloing persis.


10/62


Internal versus external capacity. In he shor erm, sae educaion agencies

are dealing wih heir inernal capaciy gaps by relying on wo dieren kinds o

exernal capaciy: ouside consulans and oundaions. Tere is some concern,

however, ha reliance on ouside grans and consulans may preclude or delay

he developmen o he scal sel-suciency and inernal capaciy ha can sup-

por hese sysems over he long erm.

Funding streams and the fiscal cliff. Tere is a grea deal o concern abou sae

educaion agencies lack o capaciy o implemen hese reorms, paricularly

or saes ha did no win a Race o he op gran or secure oundaion suppor

(which is he majoriy o saes). Given he curren igh scal climae, mos saes

have been unable or unwilling o allocae new money o suppor he implemena-

ion o hese reorms. Sae educaion agencies appear o vary widely in he way

ha hey have spen exernal unds, he degree o which hey are dependen on

hem, and he exen o which hey have begun o bring hese expenses on budge.

As a resul he evenual end o ederal and oundaion granspar o he upcom-ing scal cliis likely o aec saes in dieren ways.

Evaluating the evaluators. One o he primary aciviies o sae educaion

agencies in supporing heir local educaion agencies wih eacher-evaluaion

reorm has been providing raining o he adminisraors ha will be conduc-

ing he new observaions. Saes vary widely in heir approach here, however,

or boh philosophical and capaciy reasons wih some sae educaion agencies

(such as ennessee) direcly raining all evaluaors, some (such as Colorado

and Pennsylvania) adoping a rain-he-rainer model, and ohers (such as New

Jersey) leaving he raining enirely up o disrics.

Implementation timetables and sequencing. Mos sae reorm saues have

esablished rapid imeables or he insallaion o new eacher-evaluaion

sysems. While all saes are sruggling o mee hese imeables, i is becoming

clear ha some saes are sruggling more han ohers due o he ac ha saes

vary in erms o heir experience wih saewide evaluaion sysems. A relaed

challenge ceners on he exen o which evaluaion reorms areor are no

being conneced o he implemenaion o oher reorms such as new principal

evaluaions and new common core sandards and assessmens.

Value-added/growth scores for teachers in nontested subjects. Perhaps he

single bigges challenge in implemening new evaluaion sysems ha has emerged

rom he eld is he ac ha he majoriy o eachers do no each in esed subjecs


11/62


or grades and as a consequence sandardized suden achievemen daa is no avail-

able o be used in heir raings. Disrics are working independenly o develop heir

own suden-learning objecives, bu he qualiy o he resuls appears o be mixed

and messy boh wihin and across saes. Tis is an enormous problem and i is clear

ha many sae educaion agencies are sruggling o address i.

Networks, policy learning, and politics. Policy learning and coninuous

improvemen requires ha local educaion agencies, sae educaion agencies,

and he U.S. Deparmen o Educaion be ransparen and orhcoming abou

wha is working and wha is no and ha lessons learned be regularly shared

wihin and beween saes. Bu on he ground he realiy appears o be ha no

enough communicaion and sharing o inormaion abou eecive measures is

happening ye. Balancing heir suppor and compliance monioring uncions

will coninue o require a delicae balancing ac or sae educaion agencies and

he Deparmen o Educaion, bu geting he balanceand he communica-

ionrigh will be crucial o he evaluaion reorm eor going orward.

Te lessons derived rom hese challenges orm he basis or he ollowing

recommendaions:

Individual saes need o hink careully abou he work ha needs o be done

o implemen a new eacher-evaluaion sysem, assess he exising capaciy a

he local and sae educaion agency levels, and dene an appropriae role or

he sae educaion agency ha is commensurae wih sae consiuional and

sauory provisions.

Given heir limied resources, sae educaion agency leaders have o hink

careully abou how bes o reallocae exising sa and budges o ocus on

new responsibiliies, build capaciy, and evenually bring work ha is unded

by exernal grans on budge. Federal regulaions and sae budgeing and civil

service requiremens ha consrain he abiliy o sae educaion agencies o do

so should be revised wih an eye oward permiting greaer managerial fexibiliy.

Sae educaion agencies need o hink abou comparaive advanage and

economies o scalewhere he sae can provide somehing disrics canno.Providing echnical assisance and policy inerpreaion, creaing communica-

ion neworks or inormaion sharing, expanding assessmen porolios, and

esablishing online raining modules are several areas where sae educaion

agencies could add real value.


12/62


Sae legislaures and sae educaion agencies should ailor heir implemena-

ion imelines o he unique needs and resources o heir paricular sae. Tey

should also deermine how he evaluaion work ough o be sequenced wih and

conneced o he roll ou o oher relaed educaion reorms, paricularly hose

reorms around eacher preparaion, proessional developmen, principal evalua-

ion, and common sandards and assessmens.

Saes need o hink long erm abou how o produce a large and sable sup-

ply o adminisraorssae educaion agency sa as well as school principals

and disric superinendenswih he raining, echnical experise, and eld

experience o address heir curren human-capial challenges around eacher-

evaluaion reorm. Parnering wih a saes higher educaion sysem or wih

managemen consulans o devise new raining and cericaion programs ha

refec he dieren work and skill se required is crucial.

Te learning curve or local educaion agencies, sae educaion agencies, andhe U.S. Deparmen o Educaion during he implemenaion o new eacher-

evaluaion sysems will be seep and misakes will ineviably be made, bu i

is crucial ha he work be ransparen and ha inormaion abou eecive

mehods be shared up and down he educaion delivery chain. Sae educaion

agencies and he Deparmen o Educaion need o creae a sae space where

praciioners wihin and across saes can be candid abou he misakes hey are

making and he suppor hey need wihou ear o riggering puniive oversigh

or inervenions by a higher auhoriy.

Te remainder o he paper will provide a review o previous research on sae

educaion agency capaciy and eacher-evaluaion reorm, analyze sae educaion

agency implemenaion eors in he six case sudy saes, and elaborae on he

lessons and challenges ha have emerged rom he early-adoper saes.


13/62

Implementation and state capacity gaps | www.americanprogress.

Implementation and

state capacity gaps

Race o he op inauguraed an unprecedened wave o sae eacher-evaluaion

reorms.9 Te Naional Council on eacher Qualiy repors ha 36 saes and

he Disric o Columbia have changed heir eacher-evaluaion policies since

2009. Tere has been a large increase in he number o saes ha require annual

eacher evaluaions (currenly 43 saes), and hose incorporaing suden achieve-

men (32 saes), diereniaed levels o perormance (26 saes), annual class-

room observaions (39 saes), muliple observaions each year (22 saes), and

perormance-based enure decisions (9 saes).10 Te Naional Council o eacherQualiy ound ha he landscape is quickly and dramaically changing when i

comes o rehinking and rebuilding eacher evaluaions in school sysems in he

Unied Saes. Tere is a grea deal o promise and poenial in hese policy rends.

A he same ime, however, i is clear ha policy is only par o wha is necessary

Even he bes evaluaion sysem can be implemened poorly or undermined.11

I is exraordinarily dicul o drive change rom he sae capiol all he way down

o he classroom level. For eacher-evaluaion reorm o succeed, sae policy

changes mus resul in changes in disric pracice. In urn, changes in disric

pracice mus change he behavior o principals and eachers a he school level,

and changes a he school level mus deliver improved suden perormance. As

a resul he vigor and eeciveness o sae and disric implemenaion eors

will be criical. Bu commimen alone may no be sucien, or, as Harvards

Richard Elmore argues, saes suer rom a capaciy gap ha undermines heir

abiliy o monior and enorce mandaes and provide echnical guidance.12 A 2011

Cener or Educaion Policy survey, or example, ound ha operaing budges

or a majoriy o sae educaion agencies have declined by 10 percen or more

since 2007, and ha only a handul o saes believe hey have all hree elemens

in placeadequae experise, sang levels, and undingo carry ou keyAmerican Recovery and Reinvesmen Ac relaed reorm aciviies.13

Paul Pasorek, ormer Louisiana sae superinenden o educaion, has expressed

concern ha he U.S. Deparmen o Educaion and many saes have been insu-


14/62


cienly atuned o hese capaciy decis. I hink some [saes] may be underes-

imaing he resources and energy ha hese kinds o iniiaives require . . . sae

deparmens o educaion are no designed o implemen hese programs, says

Pasorek.14 Furhermore, a recen sudy by he Daa Qualiy Campaign ound

ha sae daa sysems are woeully inadequae, poining ou ha only 11 saes

naionwide (and only 4 o he 12 Race o he op winners) have all he compo-nens hey deem essenial.15 And many saes and disrics have litle experience

implemening some o he reorm approaches conained in heir Race o he op

applicaions. A 2011 sudy by he U.S. Governmen Accounabiliy Oce con-

cluded ha saes are sruggling o implemen he reorms in heir Race o he op

applicaions and ha heir overly opimisic imelines are unlikely o be me.16

Te dozen winners rom he compeiion have ormally amended heir Race o

he op plans more han 25 imes, usually o scale back proposed reorms or push

back imeables.17 Disric eors o circumven compliance wih sae mandaes

are a urher challenge, as are he debaes over memoranda o undersanding and

implemening he Obama adminisraions new school-resrucuring approaches.18

Adam ucker, senior program ocer a he Bill & Melinda Gaes Foundaion,

observes, Te work is moving o anoher phase. Now ha he challenges o adop-

ing he evaluaion reorms have been done, he nex phase o work is design and

implemenaion. Tis is very challenging work o implemen a he school level

where he rubber mees he road. As noed by NP (originally Te New eacher

Projec), Now comes he hard par. As saes across he counry have already

learned, srong implemenaion will deermine wheher a new evaluaion sysem

lives up o is poenial. Even he mos eleganly designed evaluaion sysem won

succeed unless schools implemen i consisenly and accuraely.19 NP idenied

ve ses o invesmens ha saes should make o suppor schools wih implemena-

ionools and sysems (including a eacher value added model, suden learning

measures, assessmen rubrics, and daa sysems); raining or evaluaors and disric

sa; communicaions wih sakeholders; monioring and suppor (including

suppor eams, merics o success, and evaluaor accounabiliy); and susainabil-

iy (hrough eedback and improvemen and he reallocaion o sae and disric

resources o suppor implemenaion over he long haul).

While we are beginning o undersand he kinds o capaciies ha sae educa-ion agencies will need o implemen eacher evaluaion reorms eecively, we

have much less undersanding abou wheher sae educaion agencies possess

hese capaciies and how hey are acually going abou his work in he eld. In

a recen survey o sae educaion agencies, Cynhia Brown o he Cener or


15/62

Implementation and state capacity gaps | www.americanprogress.

American Progress and her colleagues noe ha a wave o recen reorms has pu

immense sress on agencies ha were originally conceived as iny deparmens

primarily designed o unnel money o local school disrics. Ye i is no a all clear

ha sae educaion agencies are prepared or his demanding new role.20 Sara

Mead, Andrew Roherham, and Rachael Brown o Bellweher Educaion Parners

recenly cauioned ha policymakers mus be careul o avoid a policy hangoverand ha i advocaes o 2.0 eacher evaluaion rush oo quickly o creae new sys-

ems or do so wihou appropriae humiliy abou wha we do and do no know

he naions eacher evaluaion spree could urn ino a big headache.21 Les urn

now o he sae case sudies and analysis, which will hopeully provide some

insighssome ideaional aspirin, i you willha can help preven he onse o

he poenial eacher-evaluaion hangover or a he very leas, miigae is eecs.


16/62


17/62

Six state case studies | www.americanprogress.o

Six state case studies

Tennessee

Number of teachers: 66,558

Number of schools: 1,803

Number of districts: 137

How Tennessees teacher evaluations are structured:

Tennessee has a statewide teacher evaluation model that local districts are required

by law to adopt. The system uses three components to arrive at a teachers level

o eectiveness: observation data (50 percent); student growth (35 percent); and

student-achievement data selected by the educator and his/her supervisor rom a

list o state approved options (15 percent). Teachers with three or less years in the

classroom are observed six times per year, while more experienced teachers are

observed our times per year. The evaluation system was implemented statewide in

the 2011-2012 school year.

Quick facts

ennessee was a rs-round Race o he op winneri received a $500 million

granand had a pre-exising saewide eacher-evaluaion sysem in place prior o

Race o he op. I was one o he earlies saes in he counry o iniiae eacher-

evaluaion reorm and is value-added suden-growh sysem, he ennessee

Value-Added Assessmen Sysem enaced in 1992, inormed much o he subse-

quen debae across he counry. Sara Heyburn, assisan commissioner or eachersand leaders, poins ou, however, ha he old evaluaion conained inrequen and

subjecive observaions, and ha he deliy o implemenaion was compleely on

he disric. As a consequence, resuls were no colleced by he sae and here was

limied accounabiliy. ennessees recen reorms esablished a saewide eacher-


18/62

12 Center or American Progress | The State o Teacher Eval uation Reor m

evaluaion model ha local disrics are required by law (Firs o he op Ac o

January 2010) o adop. A waiver opion is available or he observaion insrumen

bu 126 o 137 disrics are currenly using he sae model. As Heyburn noes, Te

sae has a lo o cenralized decision-makinghere is a signican sae role in

erms o design, developmen and implemenaion around evaluaion.

ennessees evaluaion sysem has hree componens: observaion daa (50

percen), suden growh (35 percen), and suden-achievemen daa seleced by

he educaor and his/her supervisor rom a lis o sae board approved opions

(15 percen). Apprenice eachers (hose wih hree years or less in he classroom)

are required o be observed six imes per year, while proessional eachers are

observed our imes per year. During he 2010-11 school year he eacher-evalu-

aion advisory commitee piloed our dieren evaluaion mehods in approxi-

maely 30 disrics and recommended he AP (sysem or eacher and suden

achievemen) eaching sandards22 as he model or he observaion componen

o he evaluaion. Mos disrics have adoped ha model, which is he only oneha ennessees deparmen o educaion is providing implemenaion suppor

or. Te evaluaion sysem was implemened saewide in 2011-2012.Note: Te

Memphis school district, which is part o the Gates Foundations Intensive Partnerships

or Eective eaching grant, and a ew other districts have devised their own models.

Te saes deparmen o educaion worked wih NP, ormerly known as

Te New eacher Projec, o survey school disrics and ideniy capaciy needs

around eacher evaluaion. Unlike many oher saes ha are relying on a rain-

he-rainer model, he ennessee deparmen o educaion rained all 5,000

plus observers and evaluaors isel, hrough a our-day summer raining session

in groups o 40 o 50 (a oal o 102 cohors.) Te deparmen conraced wih

he Naional Insiue or Excellence in eaching o deliver he raining and also

provide ongoing and on-demand suppor o schools during he academic year.

imohy Gaddis, he assisan superinenden or eaching, learning, and assess-

men or he Williamson Couny Schools (and he ormer direcor o evaluaion

or he ennessee deparmen o educaion), calls he insiues raining and

suppor really, really srong. Te sae creaed an online raining poral wih

videos and oher insrucional resources, along wih an online cericaion es

(ha measures iner-raer reliabiliy) ha all observers mus complee. Niney-seven percen o principals passed he es, hough many apparenly had o ake i

muliple imes beore passinga poenial red fag.23


19/62


Assessing he new evaluaion sysem, Gaddis remarks ha he wishes he had one

more year or so o roll ou he new sysem and noes ha in order o mee Race

o he op imelines we had o roll i ou wih a lo o bugs in he sysem. Gaddis

recalls ha he had o say I don know a huge amoun o he ime and his ae

away a our credibiliy.

Assisan Commissioner Heyburn erms he rs year o he saes new evaluaion

sysem a success, saying, Te big win rom our sandpoin is ha we have imple-

mened a muli-measure evaluaion sysem in all disrics and all schools across

he sae ha provides regular eedback o eachers and ocuses hem on insruc-

ional pracice and suden resuls in a very real way. Te conversaion change

ha we have seen a he school and disric level over he pas year has been really

asounding. Heyburn admis ha here were many challenges associaed wih

iniial implemenaion, bu ha ulimaely ennessee was able o ully implemen

he new sysem in he eld in he 2011-2012 school year.

Te implemenaion o ennessees new sysem encounered some poliical urbu-

lence he resul o media coverage ha highlighed serious concerns raised by sae

legislaors early on in he process as well as an eor o roll back some elemens o

he new evaluaion sysem.24 Tis underscores how sae educaion agencies mus

be cognizan o boh he policy and poliical challenges involved in implemening

and susaining dicul and conroversial reorms.

Aer winning is Race o he op gran, ennessee conraced wih he U.S.

Educaion Delivery Insiue o conduc a capaciy review o he saes depar-

men o educaion. Teir review concluded ha he organizaion and he work

wasn organized in a way ha suppored implemenaion[and] reinorced ha

inenional change had o happen in order o improve capaciy, regardless o how

ha would aec componens, deparmens, and people in he agency.25

In April 2012 Kevin Human came on board as ennessees new educaion com-

missioner and reorganized he sae educaion agency. A new division o eachers

and leaders was creaed wihin he saes deparmen o educaion o bring ogeher

all o he dieren elemens o he human-capial coninuumrom educaor

preparaion, licensure and cericaion o recruimen, sang, and compensaiono evaluaion and proessional developmenwhich in he pas had all been unc-

ions o dieren oces ha were no conneced and didn necessarily sync wih

one anoher. Commening on he new division, Heyburn says she believes ha i

Unlike many

other states

that are relying

on a train-the

trainer model

the Tennessee

department o

education trai

all 5,000 plus

observers and

evaluators itse


20/62


has really helped us o have he sae level human capial managemen work under

he same division o ensure ha we have he same vision or eaching and leading

across he specrum o an educaors career. o ge his righ, we have o work in

andem across hese oces. ennesseelike oher saeshas sruggled o nd

he appropriae sa o ll some posiions in his new division. Heyburn wen on o

noe, Tese new evaluaion models require unique skills and experise bu here areno a lo o people ye wih experience in his work.

Heyburn repors ha while she has had a good bi o reedom o recrui and hire

he people she needs, several changes o hand around he evaluaion work during

he rs phase o implemenaion have been a challenge.

In addiion o geting he deparmen capaciy righ, ennessee has used Race

o he op money o hire consulans o suppor some ime-limied iniial work,

including year-one raining and he sae daa sysem evaluaion. Gaddis expresses

concern, however, abou sa urnover a he sae educaion agency during sucha criical ime. I is ough o no have ha consisency a a ime o grea change

and uncerainyhe new olks have a lo o caching up o do beore hey can be

eecive. He also says i is criical ha he olks who are driving and are he ace

o big change really have credibiliy a he disric level bu noes ha he hiring

o nonradiional sae educaion agency sa and ou o sae consulans has

creaed a credibiliy problem. Heyburn has a slighly dieren assessmen, saying

she believes ha hiring new sa and drawing on selec exernal suppor, while no

perec, has enabled he sae deparmen o educaion o build he inernal capac-

iy needed o suppor evaluaion reorm while avoiding unding new posiions

wih Race o he op money in a way ha migh no be susainable.

Gov. Bill Haslam (R) asked a nonpro organizaion, he Sae Collaboraive on

Reorming Educaion, or SCORE, o provide an independen assessmen o he

rs-year rollou o he new sysem. Te nonpro led a ve-monh lisening and

eedback process ha included nine regional roundables and an online eacher

quesionnaire. Teir repor concluded ha overall, ennessees evaluaion sysem

is improving boh he qualiy o insrucion in he classroom as well as he esablish-

men o accounabiliy or suden resuls.26 Tey ound, however, ha educaors

quesioned wheher principals had he ime and abiliy o eecively assess each-ers and believed ha here was inconsisen inerpreaion and implemenaion

o he rubric. Te repor ound ha wo-hirds o eachers do no have individual

value-added suden growh daa or heir grades and subjecs, while also encourag-

ing he sae o develop beter merics or eachers in nonesed subjecs. Te repor


21/62


also criicized he lack o high-qualiy proessional learning opporuniies ied o

eachers perormance eedback and he lack o atenion o linking he new evalu-

aion sysem o he pending implemenaion o he common core sae sandards.

Assisan Superinenden Gaddis cauions ha here have been serious growing

pains wih he implemenaion o he new sysem along wih some undamenal

unairness relaed o he requiremen ha every eachers evaluaion be 35 percenbased upon a growh score. He noes ha since 60 percen o our eachers have

no sandardized es, he sae had o pu measures in place ha really aren a good

direc indicaor o growh in he area o he eachers responsibiliy.

In response o a reques rom he sae legislaure, he ennessee deparmen o

educaion also conduced is own inernal analysis o he rs-year implemenaion,

which i released in July 2012.27 I conduced more han 120 sakeholder meeings

across he sae, me wih all o he saes 136 direcors o schools, and compared he

daa rom he eacher observaions and value-added scores o suden es scores.

Te review was conduced by he ennessee Consorium on Research, Evaluaion,and Developmen, which is unded by he saes Race o he op gran. Overall, he

repor concluded ha evaluaors successully idenied high-perorming eachers

bu sysemaically ailed o ideniy he lowes-perorming eachers, leaving hese

eachers wihou access o meaningul proessional developmen and heir sudens

and parens wihou a reasonable expecaion o improved insrucion in he uure.28

Assisan Commissioner Heyburn noes ha he bigges challenges were a he school

level and no he disric level and poins ou ha 72 schoolsacross 50-plus dis-

ricshave been argeed or addiional suppor in year wo o he program.

According o he eacher surveys i was clear ha here were communicaion

challenges and ha eachers and school leaders sruggled o nd he suppor

and guidance needed o navigae he early sages o implemenaion. Te sae

educaion deparmen pledged o develop a more cenralized communicaion

sraegy o address his issue going orward.29 Heyburn acknowledged ha

communicaion is key bu i is somehing ha sae educaion agencies as a rule

don do well, paricularly in erms o working wih disrics. As she explained,

We had o rese our [communicaions] sraegy early on in erms o our abiliy

o respond o disrics so ha hey could ge answers o heir quesions abou

implemenaion in real ime. Heyburn says sang an online quesion and answerholine and holding regional echnical assisance meeings helped accomplish

his sraegy. In addiion, he sae educaion deparmen announced plans o hire

ve ull- and par-ime evaluaion coaches, in collaboraion wih he Naional

Insiue or Excellence in eaching, who are being deployed regionally o suppor


22/62


disric implemenaion eors during he 2012-13 school year. Tese coaches will

devoe special atenion o hose schools where year-one scores were mos ou o

alignmen. Te deparmen also inends o expand he online resources avail-

able hrough he AP poral o include more model lessons and oher resources

ha ie direcly o rubric indicaors. Recognizing he crucial bu ime-consuming

role ha principals play as evaluaors in his new sysem, he sae deparmen oeducaion is also working wih disrics o help hem reallocae managerial asks in

order o ree up more ime or hem o work as insrucional leaders.

While he various repors generaed a grea deal o concern and criicism o

ennessees evaluaion reorms, hey also demonsraed he imporance o rans-

parency in he implemenaion o eacher-evaluaion reorms. Heyburn noes he

need o refec on year one and make some changes and coninue he cycle o

improvemen o his sysem and our suppor a he sae level.

As in oher saes, he sae educaion agency in ennessee is sruggling o denean appropriae and consiuional role in implemening eacher evaluaion reorm

and o balance being supporive and fexible around disric eors o ensure com-

pliance. Aer he SCORE repor was released, newspaper coverage repored he

sae is looking or clariy regarding when ocials can inervene in disrics wih

a wide gap beween value-added and observaion scores.30 Given ennessees

long experience wih value-added daa and he ac ha is eor was relaively

well-unded and suppored, he challenges ha have emerged here should serve

o underscore he diculy o his work. Finally, Heyburn emphasizes ha i is

imporan o rame his rom he beginning as coninuous improvemen o he

evaluaion sysem, jus as we are alking abou coninuous improvemen o each-

ing pracice. We are in he early sages o learning abou how o build eecive

evaluaion sysems and learning more and more all he ime.


23/62


Colorado

In 2010 Colorado enaced one o he mos sweeping educaor evaluaion reorm

saues in he counry (Senae Bill 10-191), direcing school disrics o adop new

eacher-evaluaion sysems based on wo major componens: 50 percen on suden

academic growh and 50 percen on observaions and/or oher mehods ha mea-

sure proessional pracice. Beginning in 2011 and coninuing hrough o he sar o

he 2013 school year, Colorado is piloing elemens o is new evaluaion sysem ha

will go ino eec saewide in 2013-14. Kay Anhes, he execuive direcor o educa-

or eeciveness in he Colorado Deparmen o Educaion, emphasizes he impor-

ance o he wo-year pilo program versus he single-year pilos being used elsewhere,

poining ou ha i allows hem more ime o learn and rene rom misakes. And

unlike many saes, Colorados legislaure has been acively involved in he rollou

o he new evaluaion sysem, including being required by saue o approve plans

developed by he saes deparmen o educaion relaed o is eacher-evaluaion

sysem. Observers also repor ha Sae Sen. Michael Johnsonhe sponsor o he

reorm billhas been acively involved in he implemenaion o he pilo.

In conras o ennessees more cenralized approach, Colorados emphasis on

local conrol has led o an opional sae model or eacher evaluaion ha disrics

can choose o adop wholly or in par, or hey can develop heir own evaluaion




How Colorados teacher evaluations are structured:

Senate Bill 10-191 directs school districts to adopt new teacher-evaluation systems

that are based 50 percent on student academic growth and 50 percent on observa-

tions and/or other methods that measure proessional practice. The system incorpo-

rates our perormance level ratings or educators. Local school districts can adopt

the state model wholly or in part, but district evaluations must meet or exceed the

state criteria and be subject to state review. The new evaluation system is to go into

eect statewide in the 2013-2014 school year ollowing a two-year pilot.

Quick facts


24/62


sysems. Disric-creaed evaluaion sysems mus mee or exceed he sae crieria

and mus be submited o he saes deparmen o educaion as par o he annual

assurances, and are subjec o sae review as needed. Te Colorado deparmen o

educaion, however, only oers raining and suppor or he sae model, inceniv-

izing disrics o adop i. Going orward i will be ineresing o see how much

variaioni anyexiss in he various evaluaion sysems. Anoher area o wachwill be how requenly and in wha way he sae uses is audi auhoriy.

Anoher unique eaure o Colorados approach o his work is ha i decided o

implemen a new principal-evaluaion sysem, also as a pilo, beore embarking on

he pilo phase o he new eacher-evaluaion sysem, a move ha Anhes believes

has really helped he later. Boh new evaluaion sysems will be implemened

saewide a he same ime.

Prior o he passage o Senae Bill 191, he Colorado Deparmen o Educaion did

no have an educaor eeciveness uni and hough i has expanded a bi, i remainssmall. Due o sa and resource consrains, he saes deparmen o educaion has

adoped a rain-he-rainer model. Anhes noes, We know we can be everywhere

and do his work wih only eigh rainers so we are parnering wih our regional ser-

vice delivery agencies and disric personnel o go ou and rain people a he school

level. She also says ha he saes deparmen o educaion plans o build long-erm

capaciy by creaing crieria and an approval process ha will allow programs and

disrics o become sae approved raining programs, where sa rom proessional

associaions, boards o cooperaive educaional services, school disrics, and insiu-

ions o higher educaion can assis wih he work. Te rainingone and a hal days

longprovides an orienaion o he law and a deep dive ino he rubrichow i

should be used and scoredollowed by simulaed pracice evaluaions and eedback

sessions. Te Colorado Deparmen o Educaion has also devoed one saer o

communicaion and oureach along wih creaing online modules ha walk disrics

hrough he implemenaion process. In order o ensure compliance, i requires ha

disrics repor every July (he assurances noed above) deailing he ype o evalua-

ion sysem ha each disric is using and how disrics are meeing sae sandards.

One way he saes deparmen o educaion will monior disric progress is by look-

ing a he evaluaion daa o deermine i here is a meaningul correlaion o educaor

raings and suden achievemen. Anhes admited o capaciy issues a boh hesae educaion agency and disric level, saying doing his work can be exhausing.

Nina Lopez, vice presiden a he Colorado Legacy Foundaion, a privae non-

pro organizaion supporing educaion reorm, emphasizes ha local conex


25/62


maters or implemening hese kinds o reorms. An ineresing dynamic in

Colorado, or example, has been guring ou an accepableand consiuional

role or he sae educaion agency. Colorado is a local conrol sae and he sae

consiuion explicily declares school curriculum o be he sole province o

local disrics and places limis on wha he sae educaion agency may require.

Lopez observes here were decades where disrics jus old he sae o leave usalone. Bu aer some iniial resisance o he saes evaluaion reorms, Lopez

says disrics are now requesing suppor rom he sae educaion agency, a siu-

aion she calls unprecedened. Many disrics have come o recognize ha hey

do no have he capaciy o do his work on heir own and are concerned abou

he poenial legal challenges ha would emerge i personnel decisions are made

by an evaluaion sysem ha is no valid and reliable. Colorado conains a large

number o small, rural, and geographically isolaed school disrics ha according

o Lopez have limied resources available o creae a valid and reliable evaluaion

insrumen. Tese small disrics emphasis on personal relaionships beween

principals and eachers and heir diculy in recruiing qualied new eachers andprincipals urher complicaes atemps o creae rigorous evaluaion sysems.

Lopez noes ha here are huge challenges in doing his work and ha no one

could have ully conemplaed all o he deails, all o he seps involved and wha

would be required because no one had ever really pu all o hese pieces ogeher

beore a he sae level. In paricular, she says ha he imporance o daa and ana-

lyics and assessmens can be oversaed: Te magniude and primacy o doing

solid analysis o he daa boh or he evaluaions and he eedback or eachers is

really, really going o need o work well or his hing o realize is ulles poenial

and he capaciy across saes and our sae is highly variable a bes. Tere is a wide

variey o labor-inensive and ime-consuming asks ha are necessary o aciliae

he roll ou o hese new sysems, including he creaion o common course codes,

eacher-suden daa links, roser vericaion, daa analysis emplaes, elecronic

version o observaion rubric, and proessional developmen resources conneced o

he rubric. Te sae educaion deparmens iniial esimae was ha only hree sa

would be necessary o implemen he new eacher-evaluaion reorms bu Lopez

says i has now become he core ocus o he agency. She repors ha he American

Insiues or Research was hired o provide echnical experise o he deparmen.

One o he mos uniqueand successulaspecs o Colorados approach o

educaor-evaluaion reorm has been he parnership beween he saes depar-

men o educaion and he Colorado Legacy Foundaion, a privae, nonpro

oundaion creaed in 2008 wih philanhropic suppor rom a variey o sources,

Another

unique feature

of Colorados

approach to

this work is tha

it decided to

implement a

new principal-

evaluation

system, also as

a pilot, before

embarking on

the pilot phase

the new teache

evaluation syst


26/62


including he Gaes and Ford oundaions (i does no receive any sae or ederal

unding). Te oundaion does no engage in advocacy (unlike groups elsewhere

such as ennessee SCORE) bu raher sees is role as supporing educaion reorm

in he sae and being a criical riend o he Colorado deparmen o educaion.

Te oundaion, which currenly has 40 saers, has played a crucial role in provid-

ing supplemenal unding and sang or he deparmen o educaion and heSae Council on Educaor Eeciveness. Te oundaion has also helped convene

sae sakeholders or conversaions and hires consulans o provide experise and

guidance ha can assis he saes deparmen o educaion wih sraegic planning

around he launching o new iniiaives. Lopez says he sae educaion agency oen

does no have he resources or agiliy o be able o quickly urn on a dime and ocus

on new hings bu we are well-posiioned o give hem momenum quickly. She also

noes ha he oundaion can work wih disrics in a way ha he Colorado depar-

men o educaion canno because he oundaion does no have a compliance role.

In addiion, he oundaion is able o creae a sae place or disrics o ail and learn

and is in a posiion o be candid abou disrics eors and challenges. Tere isregular communicaion and coordinaion beween he oundaion, he deparmen

o educaion, and he Sae Council on Educaor Eeciveness, aciliaed by weekly

meeings beween he oundaion and he educaion commissioner, no o menion

he ac ha Lopez ormerly worked or he Colorado deparmen o educaion and

served on he council on educaor eeciveness.

Te oundaion is also working wih 13 inegraion disrics on a projec unded

by he Gaes oundaion ha is rying o provide ocused suppor and analysis or

how educaor evaluaion reorms are inegraed wih oher reorms such as he

implemenaion o new principal evaluaions and new insrucional ools aligned

wih common core sandards and assessmens. Lopez noes ha he oundaion is

meeing wih he sae monhly o share inormaion gleaned rom he inegraion

disrics, which in urn can be used o inorm heir own work. An example o he

kind o work being done by he oundaion is he adapaion o he Harvard ripod

suden survey or use in he collecion o suden eedback as par o eacher evalua-

ions, which he oundaion unded. I will launch a pilo program using he ripod

evaluaion ool and will conduc a validaion sudy as well as develop proocols or

is use. All o he inormaion and resources ha are developed hrough he pilo

process will be shared wih he Colorado deparmen o educaion or use saewide.


27/62


New Jersey




How New Jerseys teacher evaluations are structured:

New Jerseys reorm legislation created requirements or our categories or teacher

ratings based on multiple measures o student learning and growth, and multiple

observations. It does not address how much student achievement should be

incorporated into teacher evaluations. Districts can choose rom one o the state-

approved models or develop their own evaluation model, with evidence o validity.

Statewide implementation is scheduled or the 2013-2014 school year.

Quick facts

Te saes reorm saue, EACHNJ, creaed muliple raings caegories or

eachershighly eecive, eecive, parially eecive, and ineeciveha are

based on muliple objecive measures o suden learning and growh and muliple

observaions. Te saue also called or he creaion o an Educaor Eeciveness

ask Force during he 2010-2011 school year, which recommended a eacher-

evaluaion implemenaion pilo in 10 school disrics in he 2011-2012 school

year. Anoher 20 disrics joined he original 10 during he pilo or 2012-2013,

and oher disrics were insruced o build capaciy or he saewide implemena-

ion scheduled or he 2013-2014 school year. Te pilo guidance purposely was

ambiguous on how and how much suden achievemen should be incorporaed

ino eacher evaluaions, noing only ha sandardized assessmens mus be used,

bu mus no be he predominan acor. As o Ocober 12, 2012 he New Jersey

deparmen o educaion had approved 14 dieren eaching-pracice-evaluaion

insrumens, which me he saes echnical requiremens.31 Disrics can choose

rom one o he sae-approved models or develop heir own evaluaion model

(wihin one year) and presen daa o show is validiy.32

One unique aspec o New Jerseys implemenaion o eacher evaluaion reorm

was he use o a compeiive applicaion process or disrics ineresed in parici-

paing in he pilo program. For he second year o he pilo, he sae deparmen


28/62


o educaion received 49 applicaions and 10 disrics were seleced o receive

grans oaling $1 million o suppor implemenaion. Anoher ineresing eaure

was he requiremen ha all disrics in he sae engage in a year o ocused capac-

iy building in advance o he roll ou o he new evaluaion sysem. I is imporan

o noe ha as complicaed and labor inensive as eacher evaluaion reorm is, he

reorm is hardly he only major iniiaive being implemened by he New JerseyDeparmen o Educaion. Te agencys capaciy is being urher srained by he

simulaneous roll ou o a new principal-evaluaion sysem as well as a wide array

o oher reorms, including charer school expansion, school urnarounds, and

he coninued sae deparmen o educaion conrol o New Jerseys hree larges

disrics: Newark, Jersey Ciy, and Paerson.

New Jerseys Deparmen o Educaion creaed a Sae Evaluaion Advisory

Commitee and Disric Evaluaion Advisory Commitees o solici eedback on

he pilo program, wih he disric commitees meeing monhly o discuss imple-

menaion challenges. Te educaion deparmen also conraced wih he RugersUniversiy Graduae School o Educaion o conduc an independen evaluaion

o he year one pilo ha includes sie visis and adminisraor surveys.33 Te

Rugers evaluaion has wo componensa review o he process ocusing on he

implemenaion o he pilo and sakeholders response o i; and a review o ou-

comes, which looks a he disribuion o evaluaion raings and a suden growh

daa. Anoher ineresing aspec o New Jerseys work in his area is ha he sae

began piloing a new eacher-evaluaion sysem beore he sae legislaure enaced

he new evaluaion and enure law. Tis enabled he lessons learned rom ha

pilo o be incorporaed ino he saue.

Te rs year o he pilo program revealed a number o challenges. Te majoriy o

eachers do no have sandardized assessmens or heir sudens, and even or hose

who do, he saes daa sysem is only now saring o link suden scores o indi-

vidual eachers. Based on he experience in he rs year o he pilo, he sae made

a number o changes or he second year. I increased he number o observaions

or core eachers, inroduced he use o double-scoring (wo independen observa-

ions), shied o unannounced observaions, and increased fexibiliy in weighing

or esed and nonesed grades and subjecs. In addiion i required schools o use

exernal evaluaors o provide a second se o observaions o deal wih he issue oprincipal bias. Te exernal evaluaors mus come rom anoher school, rom he

disric oce, or are reired bu sill reaining cericaion; and every evaluaor has o

be rained and calibraed on he rubric being used by he disric. Evaluaor raining

is no done by he sae deparmen o educaion bu is le o he disrics, many o


29/62


which are ousourcing he raining, in some cases hrough new mulidisric con-

soria.34 Peer Shulman, chie alen ocer/assisan commissioner o eacher and

leader eeciveness or he New Jersey Deparmen o Educaion35 observes, We are

being less prescripive because here is no perec way o do his work. I we knew

he answers we would ell hem bu we aren sure. We will hold hem accounable

or oucomes and resuls and are rying o provide ools and guidelines o help hemge here. He adds ha local conrol is a big par o he abric in New Jersey, which

has more han 600 school disrics.

im Maheney, direcor o evaluaion in he division o eacher and leader eec-

iveness a he sae deparmen o educaion, says ha he implemenaion o

evaluaion reorm is a really heavy li and described he deparmen as ocusing

on hree waves o regulaory work: requiremens around building disric capaciy,

using inormaion rom he pilos o improve observaion proocols, and feshing

ou how he new evaluaions will inersec wih he new enure law. Tey have

also been working wih privae vendors o ensure ha he raining and evaluaioninsrumens he vendors are providing o disrics mee sae expecaions2 o

he 15 insrumens submited were rejeced. Maheney noes, We have o read

careully i we are going o be really specic because we are a local conrol sae.

We wan o honor he local knowledge ha comes wih ha bu we also have o

have some common saewide expecaions.

In he summer o 2011 he New Jersey Deparmen o Educaion surveyed is

580 superinendens and ound ha almos hree-quarers o hem believed he

deparmen did no play a role in helping o improve suden achievemen.36 Ta

same summer Chris Cer, he saes new educaion commissioner, iniiaed a

radical redesign o he sae educaion deparmen wih he expressed purpose o

beter enabling i o suppor disric reorm eors. He has resrucured he orga-

nizaional char and reassigned sa around our areas: academics, perormance,

alen, and innovaionand all our oces are ocused on service delivery.

According o chie alen ocer Shulman: For our prioriy and ocus (low-

perorming) schools, we wan o have direc inervenion suppor. He says sae

educaion agencies have radiionally allen ino he one-size-s-all manra bu

now we are rying o provide suppor a he granular schooli no classroom

level or abou 250 (10 percen) o he lowes-perorming schools in he sae.

Te sae has also creaed seven new Regional Achievemen Ceners, each wih a

sa o 10 o 15 drawn rom he sae educaion agency who each specialize in di-

eren areas. Shulman says he idea is o make sure ha you have he righ cure or

In the summe

of 2011 the

New Jersey

Department

of Education

surveyed its 5

superintende

and found th

almost three-

quarters of th

believed the

department d

not play a role

in helping to

improve stud

achievement


30/62


he righ ailmen and ha he regional ceners have creaed an unprecedened

opporuniy or wo-way dialogue. I remains unclear exacly how he regional

ceners will operae in pracice. Shulman admis ha hey are sill hinking abou

how all he pieces connec, bu he regional ceners have he poenial o connec

he dieren srands o school improvemen work wih eacher evaluaion reorm.

Te New Jersey Deparmen o Educaion has creaed implemenaion eamsha provide suppor o disrics, boh hose in pilo and hose in he preparaion

phase. Wih only our or ve sa (oal) assigned o his work, however, hey

don have he capaciy o be in all disrics so hey use a riage approach o sup-

por hose disrics mos in need.

Te insiuional reorganizaion has been accompanied by a major overhaul o

personnel in he sae deparmen o educaion. Shulman noes ha sae educa-

ion agency sa are no widges and ha he needed o ge he righ people wih

he righ skills on my eam. He has reconsiued he oce wih 50 percen o he

original senior sa gonesome people have been moved ou or moved o oherareas, while ohers were counseled ou. We needed o ge a good combinaion

o New Jersey educaors as well as olks who are philosophically reorm-minded

and have an undersanding o where he indusry is moving, says Shulman. Te

sae deparmen o educaion hired an individual rom he Harvard Sraegic

Daa Projec o provide needed quaniaive experise or he eacher-evaluaion

sysem. All o his resang has come a a price and Shulman admis ha he has

spen a grea deal o ime on deparmenal personnel maters.

Par o he challenge is he very small pool o people who are qualied and experi-

enced in he area o eacher evaluaion and he demand or heir services has creaed

a poaching problem. (Shulman, or example, led he eacher-evaluaion work in

Delaware beore being recruied o New Jersey.) Shulman noes ha he New Jersey

Deparmen o Educaion is rying o creae a high-perorming organizaionlike

a Google or a McKinsey & Companywih a culure ha can consanly recrui,

rain, and recycle good sa. He argues ha human capial changes will ake some

ime, poining ou ha you can only bring people in rom he ouside as here is a

need or insiuional knowledge. Bu Shulman believes ha sraegically bringing

ouside consulans onboard or a limied engagemen is a scally pruden way o

bring alen o he deparmen and i avoids he need o hire a ull-ime employeewho may no be needed or aordable over he long erm. He conceives o his work

as being done in sages and noes ha building new evaluaion sysems is dieren

han susaining hese sysems and may require dieren skill ses.


31/62


Overall, Shulman says ha he and he deparmen have learned our key lessons

in New Jersey hus ar: Firs and oremos we have learned abou he imporance

o sakeholder engagemen and communicaion. We have heard rom he eld

how imporan he Disric Evaluaion Advisory Commitees have been hav-

ing his governing or advisory board has paid signican dividends in building

ransparency and rusand so every disric is going o have his. Te secondlesson, according o Shulman was he need or raining, raining, and more rain-

inghigh quaniy and high qualiy. Selecion o he observaion insrumen was

le o disrics (rom a sae-approved lis) bu mos have chosen he Danielson

Framework or eaching. Shulman poins ou ha he specic insrumen is less

imporan han how i is implemened. Te hird lesson ha has emerged is he

major challenge ha exiss around nonesed grades and subjecs, where san-

dardized es scores do no exis and which require some oher meric or calcu-

laing a eachers growh or value-added score o be developed. Te New Jersey

Deparmen o Educaion is rying o srike he righ balance beween giving

disrics fexibiliy while also providing guidance along wih some level o sand-boxing esablishing ouer boundaries ha consrain fexibiliyo he process

o ensure a degree o qualiy conrol. Bu as Shulman poins ou, We are rying

o ge ou o he compliance business and ino he service-delivery business. Te

ourh and nal lesson is he capaciy issues or principalshey are being asked

o spend signicanly more ime conducing evaluaions and providing eedback

and guidance or eachers bu are sruggling o do his along wih heir oher

responsibiliies. Te sae educaion agency is rying o help principals wih ime

managemen o allow hem o ge i all done.37

Maheney, he saes direcor o evaluaion, observes ha he early pilo dis-

rics were all high-capaciy disrics ha voluneered o ake par, and ha hese

disrics were erile soil or implemening he reorms. He believes ha mos

embraced he reorms and were well-prepared o implemen he iniiaive hough

some sruggled due o compeing demands, problems wih he iming and qualiy

o raining, or inadequae communicaion wih sakeholders. Sill, here are con-

cerns abou wha will happen when he reorms are aken saewide. As Maheney

noes, Some disrics have overburdened leadership, or don embrace innova-

ion and are suck in heir ways. We anicipae ha here will be some recalciran

disrics ha don see as much value in his work as we do. We are concerned hahey migh have litle inenion o do his work well.

For Shulman he success o he reorms is pegged o buy-in: We are rying o

weave his ino he abric o he disric, and ha buy-in by olks a he local level


32/62


is very imporan. Tere is a bridge beween heory and pracice ha needs o

be crossed so ha he undersanding on he ground and in he eld maches

ha a he sae level. I comes down o communicaion. Even as he New Jersey

Deparmen o Educaion ries o shi oward more service delivery and suppor,

Shulman says here is sill an elemen o compliance and monioring, ha doesn

go o he able compleely. Te deparmen is developing new regulaions andrequires disrics o submi wo progress repors during he year. Shulman ideni-

ed hree key ears or challenges going orward. Te rs is ha he reorms are a

major culural change and i akes ime and he poliical and operaional oriude

o see i hrough, especially in he ace o gubernaorial ransiions. Second is he

need o beter connec evaluaion o proessional developmeno nd ways

o use daa o ideniy eachers srenghs and weaknesses and drive growh. Te

hird akeaway, according o Shulman, is removing bad eachers, bu jus as impor-

an is raising he bar and expanding he pipeline or pool o new eachers.


33/62


Pennsylvania




How Pennsylvanias teacher evaluations are structured:

Pennsylvanias new teacher evaluation system is based on traditional teacher

practices and classroom observations (50 percent) and multiple measures o student

achievement and growth scores (50 percent). The new system is to be implemented

statewide during the 2013-2014 school year, ollowing three years o piloting in

numerous school districts. Districts are allowed to use any state-approved model.

Quick facts

In Pennsylvanias new eacher-evaluaion sysem 50 percen o he evaluaion comes

rom radiional eacher pracices and classroom observaions and he oher 50

percen rom muliple measures o suden achievemen and growh scores. One

unique aspec o he Pennsylvania approach o implemening he new sysem is ha

hey are spending a ull hree years piloing he new evaluaions insead o he single

pilo year being employed in mos saes. Te rs round o Pennsylvanias pilo

program ook place during he 2010-11 school year wih 10 disrics; he second

year (2011-12) he pilo expanded o 100 disrics; and in he hird year (2012-13)

i has gone o 300 disrics. Te legislaure has called or implemening he new

sysem saewide during he 2013-2014 school year. Carolyn Dumaresq, depuy

secreary in he saes Oce o Elemenary and Secondary Educaion, says ha as a

resul o he longer pilo process we have he luxury o more ime o do i righ. We

weren hurried ino jumping ino one measure. She views losing in he rs round

o Race o he op as a mixed blessing since i reed hem rom being orced o mee

an acceleraed imeline or implemenaion.

Disrics in he sae are allowed o use any sae-approved model bu he sae edu-caion agency only provides raining and suppor or one model know in educaional

circles as Danielson, an evaluaion ool creaed by Te Danielson Group, a highly

respeced educaional consuling rm. As o May 2012, however, no oher models

had been added o he lis as he sae deparmen o educaion was waiing o review


34/62


he resuls o disric pilos. Tis approach has enabled he sae educaion agency

o concenrae limied resources in providing greaer deph o suppor or a single

model han would have been possible or muliple models. Te Pennsylvania depar-

men o educaion is providing echnical assisance o suppor implemenaion o

he new sysem in wo dieren ways: hrough wo-day ace o ace coaching rain-

ing sessions or classroom observers, which is provided hrough regional inerme-diae unis38 and hrough he creaion o a ree, online proessional developmen

cener. Pennsylvanias deparmen o educaion is in he process o developing a

module or each o he 22 componens in he Danielson rubric ha includes videos

o procien eaching. Te videos enable eachers and evaluaors alike o gain a be-

er undersanding o wha hey should be looking or in heir own work. According

o he saes deparmen o educaion hese rainings will help paricipans un-

pack he componens o each domain and ideniy wha evidence looks like a

each level o prociency.39 eachers also ge proessional developmen credi or

compleing he online modules. Te sae is drawing on inormaion gleaned rom

he hree years o piloing o rene heir model and he associaed raining. eacherswere candid abou heir srong belie ha he model used during he rs pilo did

no work. Te willingness o eachers, principals, and sae educaion agency sa

o be open and hones abou he srenghs and weaknesses ha emerged rom he

pilo was very imporan and led o a swich o a new model. Dumaresq also repors

having learned a lo rom he regional educaion labs, which have provided webinars

and virual learning around evaluaion reorm.

Dumaresq noes ha a grea deal o reraining, reooling, resrucuring, and reshu-

fing o responsibiliy has gone on a he Pennsylvania Deparmen o Educaion

o enable i o beter suppor he school reorms enaced in recen years. She says

he need o creae a muliple measures model ha incorporaed suden achieve-

men daa required he sae educaion agency o bring in psychomeric kinds o

olks because ha kind o experise wasn here. As a resul, he sae educaion

agency is parnering wih exernal consulans, including Mahemaica and SAS Inc.,

o provide his experise and o connec he saes new value-added model o he

common core sandards. Some o his work was unded by he $41 million ederal

gran Pennsylvania received in round wo o Race o he op. A small Gaes ounda-

ion planning gran also enabled he saes deparmen o educaion o hire hree

addiional saone ocused on qualiaive evaluaion, anoher on quaniaiveevaluaion, and a hird dedicaed o raining. Depuy Secreary Dumaresq believes

hey now have a good eam o inernal and exernal olks in place:

We are going to leave capacity behind here aer we fnish the development phase

which is especially important when the development money and consultants go


35/62


away. We need to make sure that our own sta inside the department under-

stands the process and the program to be able to continue to support it. What we

have started to do incrementally each year is move dierent pieces o the work

into our departmental budgets.

One o Pennsylvanias larges school disrics, Pitsburgh, began is own evaluaionreorms even prior o he sae eor hanks o a Gaes Foundaion iniiaive. One

dierence beween he wo reorm programs is ha Pitsburgh is using dier-

en value-added and muliple-measures models, he implemenaion o which is

suppored no by he sae bu by he Gaes gran. As a resul o is earlier sar i

appears ha Pitsburgh is arher ahead o he sae in doing his work, bu i has

also given rise o some concerns, specically wheher he sae educaion agency is

aciliaing, obsrucing, or simply playing a neural role vis-a-vis he disric?

Sam Franklin, execuive direcor o he Oce o eacher Eeciveness or he

Pitsburgh schools, sees he Pennsylvania Deparmen o Educaion as havingbeen supporive o he disrics eors around eacher-evaluaion reorm. He

believes he bigger challenge is how he sae educaion agency suppors he saes

500 oher disrics, paricularly he many smaller ones ha lack resources and

are much urher behind in doing his work han Pitsburgh. No oher disric in

Pennsylvania, he poins ou, has moved beyond piloing new observaion models

and some haven even done ha ye. Franklin did noe ha i would be helpul or

he sae o move more quickly and comprehensively on expanding he assess-

men porolio an area where mos disrics lack compeency. You don wan o

have disrics across he sae developing los o dieren assessmens a dieren

levels o qualiy. Teres an advanage o having ess ha are consisen across

he sae where you can have comparable daa. And developing new assessmens

shouldn be driven jus by he need o evaluae eachers.

For her par, Dumaresq says he sae has beneed rom he Pitsburgh experience

bu has had o gure ou which pars o he [Pitsburgh] pilo could be expanded

saewide and which could no. Franklin has concerns ha scal susainabiliy issues

may be an obsacle o saewide adopion o his disrics eacher-evaluaion model,

noing ha ouside consulans such as Mahemaica and Cambridge Educaion

are no cheap. He says he Pitsburgh disric is working o deermine wha heongoing coss are and looking or ways o incorporae hese coss ino our budge.

He noes, however, ha large or medium sized urban disrics wih more sizable

operaing budges such as Pitsburgh may have more abiliy o reallocae resources

and sa ime o suppor his work han smaller disrics.


36/62


37/62


Delaware

Delaware has a single saewide evaluaion modelhe Delaware Perormance

Appraisal Sysem bu gives disrics he opion o using an alernaive evalua-

ion model in conjuncion wih he sae model. As o Sepember 2012 no disric

has exercised ha opion. Delaware piloed heir new eacher-evaluaion sysem

during he 2011-2012 school year and is implemening i saewide his school

year (2012-2013). Te sae has hree disinc advanages over oher saes in he

implemenaion o eacher-evaluaion reorm. Firs, as a rs-round Race o he

op winner, Delaware received a large gran ($100 million), which has enabled i o

dedicae more sae educaion agency sa and resources o he work. Te second

advanage is ha Delaware already had a saewide eacher evaluaion sysem in

place since he 1980s. Diane Donohue, special assisan or educaor eeciveness in

he saes deparmen o educaion, allows ha as a resul Delaware is ahead o he

game compared o oher saes and hereore is able o ocus on rening he exising

sysem raher han saring rom scrach. A hird advanage enjoyed by Delaware isis small sizei only has 19 disrics and abou 230 schoolswhich has enabled

he sae educaion agency o operae almos like a local educaion agency, giving i

more direc conac wih disric and school leadership han is possible elsewhere

and making implemenaion o is evaluaion program more manageable overall.


Number of schools: 218


How Delawares teacher evaluations are structured:

Delaware has a single statewide evaluation model but local districts have the option

to use an alternative evaluation model in conjunction with the state model. The new

statewide evaluation system: establishes our levels o educator perormance; uses

multiple valid measures in establishing perormance levels; requires no more than

fve components with one dedicated exclusively to student improvement (growth)

and weighted at least as high as any other component. The new teacher-evaluation

system was piloted during the 2011-2012 school year with implementation in the2012-2013 school year.

Quick facts


38/62

Te Delaware Deparmen o Educaion is very smallonly 270 peoplecom-

pared o oher sae educaion agencies. Te deparmen was reorganized in he

wake o Race o he op and developed a eacher leader-eeciveness uni (which

has our saers) along wih a delivery uni ha serves as he projec manager

or heir Race o he op gran. Te saes deparmen o educaion has used is

Race o he op undsas well as some repurposed sae undso develop anexensive nework o suppors or schools and disrics o aciliae he implemen-

aion o he new eacher-evaluaion sysem. Delawares Deparmen o Educaion

also hired nine developmen coaches or a wo-year programin parnership

wih he Universiy o Delawares Academy or School Leadershipo work wih

evaluaors o improve he accuracy o he evaluaion sysem. Te deparmen also

used $8.2 million o is Race o he op unds o hire 29 daa coaches o work wih

eachers, principals, and adminisraors o expand heir capaciy o analyze su-

den daa as par o he growh porion o he new evaluaion sysem. Te depar-

men o educaion conraced wih Wireless Generaion, a leading provider o

educaion soware, o manage his program. In addiion, i parnered wih he U.S.Educaion Delivery Insiue, an innovaive nonpro organizaion ha ocuses on

implemening large-scale sysem change in public educaion, o provide echnical

assisance o disrics or planning around he Race o he op reorms.

Te Delaware Deparmen o Educaion asked each school o creae a school

eam ha included (a a minimum) he principal, a school specialis, and wo

eachers (one o whom was a union represenaive) and hen did counywide,

ull-day raining on he new evaluaions. Te idea was o ge he inormaion o a

eam in each school who in urn would ake he program inormaion back o he

res o he building sa. Te Delaware Deparmen o Educaion is rying o visi

every school in he sae o solici eedback and address any quesions ha arise.

I has also conraced wih an ouside vendor o evaluae he evaluaion sysem

every year hrough surveys and ocus groups.40 As par o he monioring process a

eam rom he deparmen visis every school o assess he way hey are evaluaing

eachers. Te U.S. Educaion Delivery Insiue has praised Delawares local educa-

ion agency suppor program as a model or he res o he counry.41



39/62


Rhode Island

Rhode Island has aken a dieren approach o educaor evaluaion, one ha sands

ou rom mos oher saes. I has esablished a deaul sae modelhe Rhode

Island Model Educaor Evaluaion Sysemha all disrics are required o use

unless he sae says oherwise (disrics can propose alernaive plans, bu hose

plans mus be approved by he sae educaion agency). Mary Ann Snider, he chie o

educaor excellence and insrucional eeciveness a he Rhode Island Deparmen

o Educaion, repors ha abou 80 percen o disrics have adoped he sae model,

wih six urban disrics using an American Federaion o eachers model, and one

disric using is own locally developed model. Bu even he alernaive models have

o implemen he suden-learning componen ha is eaured in he saes evaluaion

model. As a resul all eachers mus wrie suden-learning objecives wih some ge-

ing a growh score in an eor o, as Snider noes, limi variaion in he suden learn-

ing componen. As she says, We are rying o keep hings as igh as possible hese

rs ew years so we can ge a more accurae disribuion o eacher eeciveness.

Unlike mos saes, Rhode Islands evaluaion reorm is no writen ino sae saue

bu insead in regulaions se by he saes Board o Regens or Elemenary and

Secondary Educaion ha require every disric o have an approved evaluaion


Number of schools: 325


How Rhode Islands teacher evaluations are structured:

Rhode Islands evaluation reorm is written into the regulations o the State Board o

Regents or Elementary and Secondary Education which established a deault state

model that all districts are required to use unless they propose an alternative model

approved by the state education agency. Educator evaluations must contain three

components based on evidence o proessional practice, proessional responsibili-

ties, and student learning. The state is silent on the percentage that each compo-

nent contributes to the fnal rating. Statewide implementation o the ull evaluationsystem is taking place during the 2012-2013 school year.

Quick facts


40/62


sysem and o evaluae every eacheron evidence o suden growh and achieve-

menevery year. Snider believes he boards se o regulaions have been exremely

helpul: I gives us he orce o law bu wih addiional fexibiliy so ha i we need

o revise hem, i is much easier han having o go back o he saehouse.

Parial implemenaion o Rhode Islands evaluaion reorms occurred in he2011-2012 school year, when every disric was required o conduc wo o he

our annual eacher observaions required by he new evaluaion model. Te

saes deparmen o educaion solicied signican eedbacksurveying every

eacher and principal in he sae and conducing ocus groups. Te deparmen

hen revised heir evaluaion model based on ha eedback. All raining on he

Rhode Island educaor evaluaion model and he suden-learning componen or

disrics using alernaive models was planned and delivered by he saes depar-

men o educaion, which hired ouside consulans o help disrics wih he

oher pars o he sysem. Te deparmen o educaion also conduced a our-day

summer academy or evaluaors and paricipans who had o complee abou 25hours o online work o calibrae heir observaion abiliy wih he rubric, includ-

ing a calibraion es a he end o he raining session. Based on he iniial resuls

and eedback he sae educaion agency decided o have wo addiional days o

raining or evaluaors during his school year. Saewide implemenaion o he

ull evaluaion sysem is aking place now, during he 2012-2013 school year.

In order o beter suppor he implemenaion, he Rhode Island deparmen

o educaion was compleely resrucured, creaing divisions o unie dier-

en unis ha had previously no communicaed or coordinaed very much.

Creaing a connecion beween educaor evaluaion and curriculum, insruc-

ion, and assessmen is really unusual among saes bu i has really helped us

wih alignmen and preparaion or he ransiion o he common core and

common assessmens, says Snider. Tere has been no new sae money o und

he evaluaion work in Rhode Island bu he sae was awarded a second-round

Race o he op gran o $75 million in Augus 2010 and hey have relied on he

ederal unds o hire new deparmen o educaion sa. We hired a ew new

people bu we way underesimaed he li i would ake o ge a new educa-

ion evaluaion sysem ready, admis Snider. She says ha miscalculaion has

mean ha he sa in he deparmen has been working exensive hours overhe pas ew years. Despie he heavy workload, Snider believes ha par o he

success in Rhode Island is due in par o avoiding he insabiliy in sae educa-

ion agency sa ha has been common in oher saes. She says ha he Rhode

Island Deparmen o Educaion is proacive abou checking in wih sa o

enhance produciviy and limi urnover.


41/62


Limiing sa urnover, according o Snider has been imporan because here are

no a whole lo o people ou here wih deep evaluaion experise. Moreover, she

noes ha here is a lo o ear and anxiey when ousiders are brough in. She says

he percepion is ha hey are here o re people. In an atemp o ease ears,

when he deparmen o educaion sough o hire some 20 inermediary service

providers (ouside consulans), who were evenually deployed in he eld o sup-por disrics and work wih principals and eachers on evaluaion, i ocused on

in-sae recruiing or hose posiions. Some o he consulans ulimaely came o

work ull ime or he deparmen on educaor evaluaion. Going orward, how-

ever, he sae is aking seps o build and susain heir capaciy over ime, in par

by working wih he Rhode Island higher educaion secor o ensure ha he nec-

essary skills are being augh in eacher and adminisraor cericaion programs.

Developing eecive lines o communicaion and building rusing relaionships

wih disrics has been crucial according o Snider. We have been successul in

changing he mindse abou he role o he SEA [sae educaion agency]prin-cipals and superinendens say ha i has been a dieren ride over he pas ew

years. We have gone ou o our way o be presen in he school disric and o

provide as much o a parnership as we can.

Te Rhode Island Deparmen o Educaion mees on a monhly basis wih every

assisan superinenden and curriculum direcor in he sae o check in and share

inormaion and build heir capaciy. One example ha Snider highlighs is he

Collaboraive Learning or Oucomes process, where a eam rom he saes depar-

men o educaion visis every disric and gives hem a sel-refecion rubric o

promp hem o hink abou wha good implemenaion looks like and o sel-assess

heir progress. We don pass any judgmen, Snider noes. We jus ask olks o ell

us where hey are and wha hey are sruggling wih and wha is unclear o hem. Te

eams ha do he disric sie visis hen wrie up hose repors and give hem o he

program direcors in he deparmen o educaion who deermine where more suppor

migh be needed. Snider noes he disrics like his approach: Wha we are saying is

ha we undersand ha his is hard work and i disrics are open o rying o ge beter

hen par o he change is we are no shuting hem down, asking or repors, monior-

ing hem, being a big broherwe are rying o provide he suppor ha hey need.

Snider says he sae educaion agency ends o be more insisen when disrics

pull back rom he work alogeher: Our reorm model depends on everyone

working o coninuously improve. We won negoiae our eors o close suden

achievemen gaps. A his ime, everyone undersands ha and his approach

seems o be working in our sae.

Partial

implementatioof Rhode Islan

evaluation

reforms occur

in the 2011-20

school year, w

every district

was required

to conduct

two of the fou

annual teache

observations

required by th

new evaluatio

model.


42/62


43/62

Key lessons and challenges | www.americanprogress.o

Key lessons and challenges

Role of state education agencies: A philosophical/statutory/

constitutional debate over centralization

I is imporan o recognize ha all sae educaion agencies are no he sameha

each sae agency has a unique hisory and operaes in a dieren scal, poliical,

sauory, and consiuional conex. In paricular, saes vary signicanly in heir

atachmen o local conrol o schools and he proper role o he sae in educaion.

Tis has a major impac on how sae educaion agencies approach eacher-evalu-aion reorm. Janice Poda, sraegic iniiaive direcor or he educaion workorce

division a he Council o Chie Sae School Ocers, says ha many wesern saes,

or example, are simply philosophically opposed o an acive role by sae educaion

agencies and are resisan o he idea o sandardizing he eacher-evaluaion process

across disrics. Addiionally, here are also consiuional limiaions on he role o

he sae educaion agency in some saes such as Colorado. Being amiliar wih he

naional educaion landscape, ennessees assisan commissioner or eachers and

leaders Sara Heyburn observes, Te sae role varies drasically rom sae o sae

in erms o how much local conrol exiss. I has huge implicaions or wha he sae

atemps o do or doesn do and he kinds o suppor you oer a he sae level

versus how you aciliae he righ hings o be happening a he disric level.

A relaed issue revolves around he radiional sae educaion agency ocus on

compliance and accounabiliy aciviies, which has made local educaion agencies

wary o being candid abou wheher and how hey migh be sruggling o imple-

men reorm, causing relucance o seek ou assisance. Tere has been a lo o alk

abou he atemp o sae educaion agencies o shi rom a compliance role o a

service-delivery role, bu he agencies are sruggling o gure ou how o ulll boh

uncions simulaneously. Adam ucker o he Gaes Foundaion acknowledges,however, ha his is hard o do and requires a dieren approach o implemenaion.

Dan Weisberg, execuive vice-presiden and general counsel or perormance man-

agemen a NP reieraes his poin, noing ha here is his percepion ha he

agency ha has he abiliy o ake money away and ake oher puniive acion agains


44/62


disrics can also be a suppor o he eniies hey regulae. He draws a parallel o

he challenge ha healh and saey agencies such as he Occupaional Saey and

Healh Adminisraion have in balancing is inspecion and ning uncion wih is

workplace saey improvemen uncion. Weisberg argues, Tere is nohing muu-

ally exclusive abou he wo uncionsin ac he dual role is

Documents

The State of Teacher Evaluation Reform