Data Summary.3.29 T+ Network Event

8/2/2019 Data Summary.3.29 T+ Network Event

1/18

BE PART OF THE EQUATION

Memphis T+ Network Event

Your Evaluations, Your Voice: Building the TEM to Last

Data Summary

Ms. Tamala R. Boyd

Executive Director, Teach Plus Memphis


2/18


Audience Demographics

240 teachers RSVPd for the event

85 attended

Of the 85 teachers in attendance:

3% had 1-2 years experience

26% had 3-5 years of

experience

25% had 6-10 years experience

47% had 11+ years of

experience

53% - Elementary

23% - Middle

21% - High

3% Combination

For 37% of the teachers this was their first T+ Network Event.


3/18


Collecting Candid Feedback

The TEM WorkingGroup expressed

concern that someteachers do not feelcomfortable being

honest on districtsurveys.

Teachers reaction to In-Service Survey Results is that they maybe skewed positively.

Teachers were asked at the T+ Network Event to describe themost effective ways to gather honest feedback from teachers.

SolutionsSuggested by T+

NetworkTeachers

Remove all personal information from the district surveymanyteachers believe a survey is the best way to reach all teachers, but areafraid their responses will be tracked back to them

In-school faculty discussions

Having a one-on-one dialogue with administrator to give feedback onthe experience

Host series of small T+ Network Like Events, i.e. focus groups tocollect feedback

An anonymous mode of communication to reportconcerns/grievances


4/18


Your Thoughts on the TEM

4

On which of the following indicators do you feel you need more

training?1. Teach 1 Engage students in

objective-driven lessons

2. Teach 2 Explain content

clearly and accurately

3. Teach 3 Engage students at all

learning levels in appropriately

challenging work

4. Teach 4 Provide students

multiple ways to engage in

content

5. Teach 5 Use strategies that

develop higher-level thinking

skills

3% 0%

22%13%

62%

0%

10%

20%

30%

40%50%

60%

70%

8


5/18


Do you agree the TEM has made you

a more effective teacher?

Teachers are more reflective of

their own practice, and students

know what is expected of them

and the teacher.

Allows students to see what

mastery looks like. More thoughtful when planning

lessons.

Teachers are able to receive

feedback in a non-threatening

environment.

Teachers are more aware ofdata and student growth.

What you get from the TEM is

what you put in.

Teachers are more interactive

with students.

Feedback from the administrators is

not the driving force behind teachers

improvement.

Observers have preset agendas and

may not be open to a variety of

teaching styles.

Observers need more training on

what constitutes Level 5 teaching.

Observers should be trained on how

to complete an observation without

disrupting student work a teacher

should not have to restart the lesson

when the observer arrives. This data is ambiguous many

teachers who believe they were

effective before the TEM was

implemented may skew the results.

Stating an objective doesnt make

you more effective. It is just protocol.


6/18


Additional TEM Training

6

How much additional training on the rubric indicators do you

need?

1. LotsI still do not know

what these are or how

they relate to myobservation

2. SomeThere are specific

indicators I am uncertain

about

3. A LittleI have only one

indicator that is not clear

4. NoneI am satisfied with

my knowledge level of

the rubric indicators

17%

48%

23%

12%

0%

10%

20%

30%40%

50%

60%

8


7/18


Teachers Thoughts on the TEM Rubric

The rubric isexplicit and clearregarding what isexpected of

teachers.

The rubric doescapture effectiveteaching if anobserver can stay inthe classroom for the

entire lesson.

The rubric is great,but there is observererror regarding howto properly assessteachers with therubric.

The rubriccaptureseffectiveteaching.

The observationprocess doesntallow variancefrom the rubric for

teachablemoments.

Attaining a Level 5 in theCLE domain seemsunrealistic would liketo see more authenticdescriptors for gradelevels and subject areas.

The observer may notunderstand thepedagogy of a contentarea enough toaccurately interpret andmeasure teaching onthe rubric.

The rubricdoes not

captureeffectiveteaching.


8/18


Preparing Teachers for the

TEM

Methods for Continuous TrainingCurrent Teachers

Video presentations with commentary/closed captioning to give rational for teacher rating

Mock observations

Should hire staff other than principals to complete observations

Content areas should have their own professional development to review rubric and watchteaching videos

Create a step-by-step training handbook available online (like for the old evaluation process)

Would like specific Randa Tower training

Make the MCS and MCSTEI websites easier to navigate

Have a qualified team available to model lessons and observe

Introduction to the TEMNew Teachers

First year teachers should not be observed until November

Would like to see the PAR program used to prepare teachers for the TEM observation process andto provide on the job training for new teachers

In-depth TEM training during new teacher orientation


9/18


The TRIPOD Survey

9

Next year, how should MCS determine the 5% Stakeholder

Perceptions score?

1. Average the scores

from both the falland spring TRIPOD

administrations

2. Only use the scores

from the Fall TRIPOD

survey

3. Only use the scores

from the Spring

TRIPOD survey

67%

2%

32%

0%10%20%30%40%50%60%

70%80%

8


10/18


Feedback on the TRIPOD

Survey should beshortened (too long even

for students at highgrade levels)

The Springadministration wentmuch smoother for

teachers and students.

Create a TRIPOD trainingvideo for teachers andstudents so all parties

understand the surveyspurpose.

Collect feedback fromstudents on their survey

experience

Students should betrained on the surveylanguage OR languageshould be made more

student friendly

Teachers should be ableto explain difficult

vocabulary to students

Ensure that studentsknow who they are

evaluating

Next year, use the Fallscores to help teachers

improve their Springscores

Survey should always beadministered by theteacher to their own

students


11/18


Suggested TEM Summative Models

MCS should petition thestate to flip the

weightings for studentgrowth and student

achievement:

15% should be studentgrowth

35% should be studentachievement

Teacher Knowledge should

be assessed appropriately(not through AVATAR course

on Common Core, butthrough portfolios) and

worth more i.e. worth 10-15%, reduce observations

to 35%

10% Student Growth

10% Student Achievement

70% Teacher Observations5% Stakeholder Perceptions

5% Content Knowledge

*Teachers are concerned toomany variables affect results of

state tests: observations aremore reliable

Re-Design of the TEM

Summative Score

No group suggested increasing the value of Stakeholder Perceptions

Teachers concerned about their ability to attain high student growth vs. high

student achievement

Content knowledge is crucial, should be a larger portion of Summative Score


12/18


TEM Training Sessions

12

Would you agree to attend TEM Training

Sessions with Principals?

1. Yes2. No

96%

4%0%

20%

40%

60%

80%100%

120%

8


13/18


Professional Growth and Support

13

The support provided in my PG&SP is aligned

with my observation feedback.

1. Strongly agree

2. Somewhatagree

3. Somewhat

disagree

4. Stronglydisagree

5. I never created a

PG&SP with an

observer

16%

34%

13% 13%

22%

0%5%

10%15%20%25%30%35%

40%

8


14/18


Plan and Reflect

14

How should the TEM observation process include the Plan and

Reflect domains next year?

1. Satisfactory/Unsatisfactory

Rating instead of full rating

2. One formative observation

in the fall only for feedback

(no ratings), one formal

observation in the spring

3. Full implementation, equal

to the Teach & CLEdomains

4. No implementation until at

least 13-14 school year

5%

25%

12%

58%

0%

10%

20%

30%

40%

50%

60%

70%

8


15/18


Your Observers

15

How do you feel that your observation scores this year have

impacted your initial opinion of TEM?

1. My support for the TEM

has significantly increased

after seeing my scores2. My support for the TEM

has somewhat increased

after seeing my scores

3. My opinion has not

changed

4. My support for the TEMhas somewhat decreased

after seeing my scores

5. My support for the TEM

has significantly

decreased after seeing my

scores

19% 23%

35%

13% 11%

0%5%

10%15%20%25%30%

35%40%

8


16/18


Reality vs. Expectations

At times teachers feel observers dont take the observations seriously and are just

going through the motions as was done in the past.

The level of stress on administrators is not conducive to effective observations.

Would like to see the Level 5 Video library be more content and grade specific.

Many teachers were nervous due to uncertainty, but are fine now that they know the

expectations of them. Restating what you already have gone over with students for the observer when

he/she enters the room is not what is intended with the TEM.

CLUE teacher thought would be observed by supervisor, but that did not happen

Teachers thought they would have multiple observers, most teachers have the same

observer for most, if not all of the observations.

Teachers feel observations are subjective, when they were intended to be objective.

Observation time should be consistent and adequate.

Teachers believed the order of the Announced/Unannounced observations would be

consistent per semester but the order varied greatly for each teacher

Many technical difficulties and lost observations.


17/18


Themes from Discussion

Much of the feedback fromteachers stems from two key

pieces of the TEM:Implementation and

Communication

Teachers who have had positive

experiences with the TEM,seem to have observations thatare consistent with the TEMpolicy and training theyreceived.

Principals are supportive andfollow the TEM observationschedule

Teachers who have not had apositive experience citeinconsistency and subjectivityas key barriers to success.

The implementation of the TEMfrom school to school is not

consistent (teachers are having awide variety of experience both

good and bad with the TEM)

Principals are using awidely variant amount oftime to completeobservations

Experiences across gradelevel and content area arewidely variant teachersbelieve observers aresubjective due to a lack ofcontent knowledge

Principals show a variantamount of engagementand investment in theobservation process.

Teachers seem to mostly get theirinformation from word of mouth: teacher

to teacher or teacher to administrator.Many teachers openly admit to not reading

their MCS email. Teachers are either notfully informed or misinformed.

Teachers believe what isoccurring in their school asindicative of the entiresystem

Teachers are put off bydifferent websites to getinformation and differentemail addresses to sendfeedback teachers havelow stamina when it comesto searching forinformation


18/18


Overview of Teacher Feedback

Teacher Feedback has been collected from teachers in a variety of forums

informed or facilitated by Teach Plus:

T+ Network Event - The TEM: Teachers Ensuring Effective

Implementation November 17, 2011 Facilitated December TEI Ambassador session to collect additional

feedback on the TEM

Collected feedback and facilitated discussion from almost 275 teachers

across 4 TEM Summative Sessions in March

Teaching Policy Fellows informed the Teacher Evaluation In-ServiceSurveys for MCS teachers which garnered roughly 6,110 responses

Meeting with TEM Evaluation Working Group to determine content for

March T+ Network Event

T+ Network EventYour Evaluations, Your Voice: Building the TEM to

Last

Documents

Data Summary.3.29 T+ Network Event