Copyright © Tulsa Public Schools 2011 © 2012, Tulsa Public Schools Teacher and Leader...

Copyright © Tulsa Public Schools 2011© 2012, Tulsa Public Schools

Teacher and Leader Effectiveness (TLE)Observation and Evaluation System / Process

EVALUATOR TRAINING – 2012 – Segment 5Getting the Rankings Right

2

Calibration Sensation!!Getting the Scores Right

Importance of Each and Every Training Piece…

3

Each piece of the training represents an EQUALLY important component of the TLE system and process.

One missing component can derail the process.

TLE Evaluation System

Observations

Evaluations

Conferences

Feedback / Support

4

Rubrics and the Horse’s Tale

Which direction is thehorse’s head facing?

5

The Goal of Rater Accuracy

To move from subjectivity to higher levels

of objectivity and insight.

How?

Practice, discipline and intentional study

6

Warm Up Question

Why is it important that raters be calibrated correctly with regard to the rankings/expectations in the rubric?

7

Calibration Exercise Basics

• Focusing (separately) on indicators 1, 6, 7, 11, 12 & 15, we will…– reflect upon the requirements of the indicator– watch a short clip of a teacher performance– individually rank the performance– discuss our rankings, attempting to reach

agreement on a score– compare our score against the target score

8

Context of Calibration

• Only one performance/observation– Disadvantages: not exactly comparable to how

principals evaluate teachers (i.e., doesn’t allow us the opportunity to draw upon multiple observations and our history/personal knowledge of the teacher)

– Advantages: forces us to critically examine the rubric’s expectations and our understanding of those expectations

9

Determining an Indicator’s Rating

Each Indicator often has several definitional narratives for each level of proficiency. However, Principals must enter only one (1) score as to each Indicator (1, 2, 3, 4, 5, NO, NA).

10

3

Determining an Indicator’s Rating

“Best fit” approach

11

12

Indicator 1

Indicator 1—Planning and Preparation at a “3”

• Plans for instructional strategies that encourage the development of performance skills.

• Ensures materials and equipment are ready at the start of the lesson or instructional activity (most of the time).

13

Planning and Preparation: What Do you Expect…

• teachers to be doing/saying if they are performing at an “effective” level…

• students to be doing/saying if teachers are performing at an “effective” level…

14

15

Indicator 1…Ready to Cue

Music?

What Level of Planning and Preparation is This? (Clip 1, Indicator 1)

16

Click on the link to view:Clip 1 - Indicator 1--revo, reaction, reform.wmv

What do you think?

• Study the rubric for Indicator 1 and write down your score for this Indicator relying on the language in the rubric.

• Show your score to the table.• What was the most frequent (“modal”)

score?• Discuss our modal scores• Is there a consensus for the room after our

discussion?18

Target score…. Clip 1, Indicator 1

5 - Superior

19

20

Indicator 15

Indicator 15—Closure at a “3”

• Ends the day’s learning / activity by summarizing the lesson and or asking students to summarize the lesson.

• Connects what is learned to prior learning.

21

Indicator 15: What Do you Expect…

• teachers to be doing/saying if they are performing at an “effective” level…

• students to be doing/saying if teachers are performing at an “effective” level…

22

23

Indicator 15…Ready to Cue

Music?

What Level of Closure is this?

24

Click on the link to view:Clip 2_Indicator 15--absent kid.wmv

What do you think?

• Study the rubric for Indicator 15 and write down your score for this Indicator relying on the language in the rubric.

• Show your score to the table.• What was the most frequent score?• Is there a consensus for the room after

our discussion?

26

Target score…. Clip 2, Indicator 15

2 or 3* – Needs Improvement or Effective

*depending on whether he confirmed that the students had actually summarized the lesson. (We can’t tell for sure from the clip whether the true summary of the lesson was in the students’ written work.)

27

28

Indicator 7

Indicator 7—Embedding Literacy at a “3”

• Literacy, the practice of reading, writing, spelling, listening, and speaking, is embedded in ALL content as an explicit learning objective.

• Displays basic recognition of the importance of literacy as the “bonding agent” for all learning.

29

Indicator 7: What Do you Expect…

• teachers to be doing/saying if they are performing at an “effective” level…

• students to be doing/saying if teachers are performing at an “effective” level…

30

31

Indicator 7…Ready to Cue

Music?

What Level of Embedded Literacy is This?(Clip 3, Indicator 7)

32

Click on the link to view:Clip 3-Indicator 7--math manipulatives.wmv

What do you think?

• Study the rubric for Indicator 7 and write down your score for this Indicator relying on the language in the rubric.

• Show your score to the table.• What was the most frequent score?• Is there a consensus for the room after

our discussion?

34

Target score…. Clip 3, Indicator 7

4 – Highly Effective

35

36

Indicator 11

Indicator 11—Directions at a “3”

• Provides directions and procedures, in a variety of delivery modes, e.g., verbal, modeling, visual, demonstration, etc., that are clearly stated / presented and relate to the learning objectives.

• Gives students directions for transitions and includes transitioning in the planning process to optimize academic learning time.

• Uses spoken and written language that is clear and correct, conforms to standard English, vocabulary, and is appropriate to students’ ages and interests.

37

Indicator 11: What Do you Expect…

• teachers to be doing/saying if they are performing at an “effective” level…

• students to be doing/saying if teachers are performing at an “effective” level…

38

39

Indicator 11…Ready to Cue

Music?

What Level of Directions is This? (Clip 4, Indicator 11)

40

Click on the link to view:Clip 4-Indicator 11--evaporation.wmv

What do you think? (Clip 4, Indicator 11)

• Study the rubric for Indicator 11 and write down your scores for this Indicator relying on the language in the rubric.

• Show your score to the table.• What was the most frequent score?• Is there a consensus for the room after

our discussion?

42

Target score…. Clip 4, Indicator 11

2 –Needs Improvement

43

44

Indicator 6

Indicator 6—Teacher’s Interactions with and Expectations of her Students at a “3”

• Oral, written and nonverbal communications with students are considerate and respectful.

• Consistently conveys a generally positive view of learning and of the curriculum, demonstrating high expectations for most students.

45

Indicator 6: What Do you Expect…

• teachers to be doing/saying if they are performing at an “effective” level…

• students to be doing/saying if teachers are performing at an “effective” level…

46

47

Indicator 6…Ready to Cue

Music?

What Level of Interactions and Expectations Is This? (Clip 5, Indicator 6)

48

Click on the link to view:Clip 5- Indicator 6--star of the week.wmv

What do you think?

• Study the rubric for Indicator 6 and write down your score for this Indicator relying on the language in the rubric.

• Show your score to the table.• What was the most frequent score?• Is there a consensus for the room after

our discussion?

50

Target score…. Clip 5, Indicator 6

5 - Superior

51

52

Indicator 12

Indicator 12 —Modeling at a “3”

• Provides demonstrations and modeling of the desired skill or process that are clear and precise to students.

53

Indicator 12: What Do you Expect…

• teachers to be doing/saying if they are performing at an “effective” level…

• students to be doing/saying if teachers are performing at an “effective” level…

54

55

Indicator 12…Ready to Cue

Music?

What do you think? (Clip 6, Indicator 12)

56

Click on the link to view:Clip 6-Indicator 12--Climate Change in a

bottle.wmv

What do you think?

• Study the rubric for Indicator 12 and write down your score for this Indicator relying on the language in the rubric.

• Show your score to the table.• What was the most frequent score?• Is there a consensus for the room after

our discussion?

58

4 – Highly Effective

59

Target score…. Clip 6, Indicator 12

Our Closure: Interview a Friend!

60

• Pick one of the following questions–What behaviors and processes support

accurate calibration?–What did this exercise reveal about my

expectations? Was I calibrated a little low/high? Just right? Ideas?

– How might evaluators deconstruct the words of the rubric with videos, or other materials, with their teachers to ensure that have agreement in their expectations?

Final Notes

How does the Tulsa model and its processes support accurate calibration?

61

Questions and Next Segment…

The 6th Segment of Evaluator InformationFile: PP6 Modeling & Presentation

Topic: Modeling / Presentations

62

For more information: www.tulsaschools.orgOffice of Teacher and Leader Effectiveness

918-746-6800

Tulsa Framework: http://sde.state.ok.us; left column: Teacher and Leader Effectiveness; Scroll to Frameworks; Tulsa Framework

Talia Shaull – shaulta@tulsaschools.org Gene S. Kleindienst - kleinge@tulsaschools.org

Jana Burk – burkja@tulsaschools.org

63