TNTP First Round Data Report 012710 Final Draft

8/14/2019 TNTP First Round Data Report 012710 Final Draft

1/29

Preliminary Data Findings: EvaluationComponents Survey Results

For New Haven Public Schools | January 2010


2/29

2 The New Teacher Project 2010

Agenda

Background

Preliminary Findings


3/29


Survey Overview

Teacher Survey

All NHPS teachers and instructional

support staff

Online distribution

Topics covered:

o Underlying teacher beliefs andattitudes

o Evaluation tool componentpreferences

o Evaluation process preferences

o Peer assistance and reviewpreferences

o Evaluation system use preferenceso School leadership and colleague

quality

o Retention factors

Response rate: 74 percent

Administrator Survey

All current NHPS principals, assistant

principals, and subject supervisors

Online distribution

Topics covered:

o Underlying administrator beliefsand attitudes


o Evaluation process preferences

o Peer assistance and reviewpreferences

o Evaluation system needso Current priorities and time on task

o Teacher remediation and dismissalexperiences

Response rate: 94 percent*

*Response rate calculated for principals and assistant principals only.All administrator responses in this document omit supervisor responses.


4/29


Survey analysis is ongoing, but many of the critical issues have beenaddressed and will be used to inform the development process.

Teacher Survey

Topics completed include:


o Peer assistance and review

preferenceso Evaluation process preferences

(related to system credibility)

Administrator Survey

Topics completed include:


o Peer assistance and review

preferenceso Evaluation process preferences

(related to system credibility)

All survey results will be used to inform the discussions and decisions of theteacher evaluation committee charged with developing the new evaluation system.

The results do not represent formal policy positions.

The teacher evaluation committee is comprised of both district and unionleadership, as well as current teachers, coaches, and school administrators.


5/29


Agenda

Background


NHPS currentevaluation system does

not sufficiently

promote studentachievement or

address ineffectiveinstructionalperformance.

#1


6/29


Both teachers and administrators express dissatisfaction with thecurrent teacher evaluation system.

38%

22%

Percentage of teachers whostrongly agree or agree

Percentage of administrators whostrongly agree or agree

Overall, I am satisfied with the current teacher evaluation process.


7/29


In addition, only a small majority believe teachers are held to highinstructional standards, including during the tenure-granting process.

51% 59%

Percentage of administratorswho strongly agree or agree

Percentage of teachers whostrongly agree or agree

My principal communicates and enforcesa high standard of instructionalperformance for all teachers in my school.

NHPS enforces a high standardof instructional performance forall teachers.

Only 33 percent of teachers agree or strongly agree that in order toattain tenure in New Haven, teachers are evaluated rigorously.


8/29


Teachers and administrators also acknowledge that poor instructionexists in their schools.

15%

32%

23%

17%

12%

1%7%

39%

26%

7%

17%

3%

0% 1-5% 6-10% 11-20% 21-50% 51-100%

Teachers Administrators

What percentage of teachers at your school areineffective*?

*Ineffective defined on survey as teachers who deliver poor instruction. Complete question reads: In your opinion, whatpercentage of teachers at your school are performing at the following levels: Exemplary, Effective, Somewhat Effective, Ineffect ive?

54 percent of teachers and administratorsreport that more than 5 percent of theircolleagues/staff are ineffective teachers.


9/29


The current evaluation system, however, fails to identify and remedypoor performance, nor does it promote student achievement.

47% 46%

36% 37%

48%

55%

36% 37%

Recognize exemplaryperformance

Help teachers improvetheir instructional

performance by providingspecific and useful

feedback

Identify and offerconcrete steps to remedy

poor performance

Promote studentachievement

Teachers and administrators who agree or stronglyagree that the current evaluation system accomplishes the

following objectives



10/29


And teachers and administrators both recognize that not all NewHaven students are learning at an appropriate pace.

53%

Only a slight majority of teachers and administratorsstrongly agree or agree that their students are

learning at an appropriate rate.

A fifth of teachers and administrators disagree*.

To what extent do you agree with the following statement:My students / students in my school are learning at an appropriate rate

at least a years worth of learning for a years worth of schooling.

*Respondents who answered somewhat disagree, disagree, or strongly disagree.

Actual percentages are:19 percent of teachers and 21 percent of administrators.


11/29


But a strong consensus on the purpose of a teacher evaluationsystem maps a path forward for NHPS.

89%

96% 95%99% 100% 100%

Recognize exemplaryperformance

Help teachers improve theirinstructional performance byproviding specific and useful

feedback

Identify and offer concretesteps to remedy poor

performance

Teachers and administrators who believe that it is veryimportant or important that an evaluation system

accomplish the following objectives



12/29


Agenda

Background


While there is broadconsensus that student

learning is an important

criterion in teacherevaluation, clear

communication aboutstudent learning measureswill be critical to building

trust and legitimacy.

#2


13/29


Teachers and administrators are open to various competenciesbeing used in teacher evaluations.

87%

93%90%

95%

84%

95%92%

83%

99% 100%

93%98%

100% 100% 99% 98%

Studentlearning

measures

Studentengagement inthe classroom

Instructionaltechnique

Teachermastery of

contentknowledge

Evidence ofpreparation

and planning

Classroommanagement

Professionalism Improvementsince last

evalutation

Teachers and administrators who rated the following criteria to be veryimportant or important in a fair and comprehensive evaluation system



14/29


While teacher support varies greatly for different measures ofteacher impact on student learning, teachers in tested grades andsubjects are more likely to be open to using test score growth.

71% 67%

40% 36%

24%

17%

Studentacademic

growth based onteacher- or

school-generated

assessments

Studentengagement

levels

Standardizedtest score

growth

School-widestudent

achievement

Standardizedtest scores

Studentgraduation rates

Teachers who report that the following metrics should beused to measure teacher impact on student learning

52 percent of teachers who teach tested subjects in

grades 3-5 report that test score growth should beused, and 46 percent of teachers in tested subjects ingrades 6-8 report the same.

Only 22 percent of special education teachers reportthat test score growth should be used.


15/29


Many of the individual concerns about using test scores raised byteachers can be addressed through the creation of a growth model.

54%

37%

15% 12%11%

External factors Validity of test Measurementmethodology

Use of tests inevaluation

Considerationof special

populations

What, if any, are your greatest concerns or questions regarding theuse of student learning measuresas part of an evaluation system?*

External factors: Tests do not account for factors out of the teachers control, such as studentmobility, attendance, and home life

Validity of tests: Tests are flawed and do not accurately measure true student ability, criticalthinking skills, or character development; cultural bias of tests

Measurement methodology: Whether tests will measure growth relative to a starting point andnot measure absolute positions or compare different classes from different years

Use of tests in evaluation: Test results given too much weight or used for punitive actions Special populations: ELL, Special Education, gifted student populations may skew results

*Open-ended question. All responses were coded and organized into eight categories, the five categories listed in the chart plus

Timing, Consequences,, and Other.


16/29


Administrators are more open to various methods of measuringteacher impact on student learning, although few believe studentgraduation rates should be used.

93%87%

84%

58%

52%

33%

Studentacademic growth

based onteacher- or

school-generatedassessments

Studentengagement

levels

Standardized testscore growth

Standardized testscores

School-widestudent

achievement

Studentgraduation rates

Administrators who report that the following metrics shouldbe used to measure teacher impact on student learning


17/29


Agenda

Background


A robust implementationplan that includes

meaningful training,

support, andaccountability for

evaluators, as well as aclearly defined evaluation

process, is critical to

establishing credibilitywith teachers andadministrators.

#3


18/29


Teachers welcome an evaluation process that clearly identifies andcommunicates their performance level and development areas.

3 out of 4 teachers strongly agree or agreethat all teachers should receive annualformative evaluations that identify clear areasfor development.

Over two-thirds (70 percent) of teachers stronglyagree or agree that all teachers should receivean annual summative evaluation rating that is aclear measure of their performance as a teacher.

2 out of 3 of teachers strongly agree or agreethat all teachers should be observed regularlythroughout the year.


19/29


If implemented, a variety of processes, particularly those monitoringevaluator effectiveness, would increase teachers confidence in thecredibility of the evaluation system.

Evaluators are themselves evaluated in a transparent process that includessimilar factors of effectiveness ...

Comprehensive training of evaluators and quality checking of their work ...

Annual survey of teacher satisfaction with the evaluation process .

Clear timeline and expectations for the evaluation process ..

Clear performance standards with evaluation rubrics and ratings alignedwith those standards

District implementation of a process to check the consistency of evaluationresults across the district .

Observation by multiple evaluators prior to assigning a final rating ..

Greater transparency on the district-wide distribution of evaluation ratings

Frequent observation of your classroom teaching ..

Suggested process to increase teachersconfidence in evaluation system credibility

Percentage of

teachers who selectedthis process

77%

70%

70%

68%

68%

46%

43%

38%

30%


20/29


A variety of processes would, if implemented, also influenceadministrators to invest more time in the evaluation process.

An electronic system or database that would make it easier to record andview observation and/or evaluation records...

The use of teacher evaluation results in key personnel decisions.

Annual surveys of all administrators on their satisfaction with various aspectsof the evaluation process..

Including the frequency and quality of teacher observations and evaluationsas a primary component in your own evaluation as an administrator.

Comprehensive training on conducting effective evaluations and qualitychecks of your work .

Including the frequency and quality of teacher observations and evaluationsas a component in any sort of potential rating or ranking of your school

More frequent accountability checks for completion and quality ofobservations and evaluations..

Greater transparency on the district-wide distribution of evaluation ratings.....

Suggested process to influence administrators

to invest more time in the evaluation process

Percentage ofadministrators whoselected this process

79%

64%

58%

45%

45%

34%

33%

33%


21/29


Ultimately, providing administrators with adequate time and support toevaluate teachers is a critical consideration.

Less than half of teachers agree or stronglyagree that their administrator has the time

to be an effective instructional leader.48%

64%

70 percent of teachers agree or strongly agree that their administrator has the

ability to be an effective instructional leader. However:

In fact, nearly two-thirds of administrators report lackof sufficient time and support as the strongest barrier

to conducting high quality evaluations of all teachers intheir school.

81 percent cite lack of sufficient time and support asthe strongest or second strongest barrier.


22/29


Agenda

Background


Discussion and Next Steps

Teacher andadministrator support forpeer evaluation varies by

grade level and subjectarea. Teachers inparticular have a wide

range of opinionsregarding how peer

evaluation should be usedand who should beinvolved in a peerevaluation process.

#4


23/29


Overall, teachers express confidence in multiple types of evaluators.

60% 59%55% 54% 53%

49% 49%

32%

Independentmasterteacher

Peer teacher Principal Mentor AssistantPrincipal

InstructionalCoach

Content AreaSupervisor

Director

Percentage of teachers who responded very confident or confident to thequestion: Rate your level of confidence in the ability of each of these people to

provide you with a fair and accurate assessment of your teaching performance.*

24 percent of teachers are very confident in principals, the highestpositive rating percentage of all answer options, but 21 percent of teachers

responded not confident, also the highest negative rating percentage.

Comparatively, 20 percent responded very confident and only 8 percentnot confident for peer teachers.

*Respondents were asked to rate their level of confidence on a 5-point scale, including a no opinion option.


24/29


The vast majority of teachers, however, do not believe that peersshould play an evaluative role that impacts a teachers final rating.

45%

29%

13% 12%1%

Individual whoprovides additional

support or

development forteachers but does not

evaluate them

Evaluator whoseevaluation is used for

the purpose of

ensuring consistencyand accuracy in

evaluations conductedby administrators

Evaluator whoseevaluation is used in ateacher's final rating

Should not play a roleat all

Evaluator whoseevaluation is used as

the sole determinant of

a teacher's final rating

What is the most appropriate role for peers toplay in teacher evaluation?

86 percent of teachers report that peers should not play role in teacher ratings.


25/29


Teacher opinion about peer evaluation varies greatly, particularlyby subject area and grade level.

Overall, 58 percent of teachers agree* that teachers should have a role in

evaluating their peers. However:Tenured teachers are less

likely than their non-tenuredcolleagues to agree that

teachers should have a rolein evaluating their peers.

Special education teachersare also less likely thantheir general educationcolleagues to agree that

teachers should have a rolein evaluating their peers.

Elementary school teachers are lesslikely to agree that teachers should

have a role in evaluating theirpeers.

54%

63%

60%

Elementary Middle High

54%

65%

Tenured Non-tenured

50%

59%

SpecialEducation

Non-SpecialEducation

*Question posed as a Yes/No question. Agreement reflects percentage of teachers who responded Yes.


26/29


Administrator opinion about peer evaluation also varies by gradelevel.

High school administrators are significantly more likely toagree that having a peer review component could lead to a

stronger teacher evaluation system.

71 percent of administrators agree that having a peer review component could

lead to a stronger teacher evaluation system. However:

83%

62%

High schooladministrators

Elementary and middleschool administrators


27/29


In general, administrators believe that peer evaluation would bemost helpful for new teachers and for struggling teachers.

75%

70%

49%

71%

53%

Teachersidentified as

needingadditional

support

Teachers onan Intensive

Plan forImprovement

Teachersidentified as

highlyeffective

Noviceteachers

Tenuredteachers

For which teachers would peer review be helpfulin the evaluation process?


28/29


If peer evaluation is used, teachers feel strongly that peer evaluatorsshould be effective teachers with significant knowledge of their district,subject area and school.

Percentage of respondents who report that thefollowing teachers would be effective orhighly effective as a peer evaluator

88%

80%

76%

60%

44%

Teachers withinthe same

content area

Teachers whohave proven to

be effectiveteachers

Teachers withsignificantteaching

experience

Teachers whoare not

currentlyaffiliated with

NHPS

Teachers whoare not

affiliated withtheir school

Percentage of respondentswho report that the following

teachers should not play arole in peer evaluation


29/29


Administrators generally prefer peer evaluators to be colleaguesfrom other NHPS schools, although variations exist by grade level.

Only 14 percent of administrators agree or strongly agree that peer evaluators should besupplied by an outside vendor, with 61 percent disagreeing or strongly disagreeing.

Only 45 percent of administratorsagree or strongly agree that peerevaluators should be colleagues from

the same school. However:

52%

37%

High school Elementaryand middle

school

41%

83%

High school Elementaryand middle

school

70 percent of administrators agree orstrongly agree that peer evaluatorsshould be teachers from other NHPS

schools in the same license area.However:

Documents

TNTP First Round Data Report 012710 Final Draft