Embed Size (px)
Citation preview
Washington State Teacher and Principal
Evaluation Project
Conducting High-Quality Observations
1 June 2013
} As you enter, please have a brief discussion with your district team to decide your level of confidence in whether the following statements are true for your district: 1. Our evaluators know how to minimize bias when collecting
observation data. 2. Our evaluators know how to capture observation evidence
using a running record approach to note-taking. 3. Our teachers trust our observers to capture accurate
evidence that will provide them with useful feedback for professional growth.
Write your district name on three sticky notes and place them on the confidence scales posted on [INSERT LOCATION] for each statement.
2
Entry Task: Confidence Conversation
3
} Introductions } Logistics } Agenda
} Agenda } Connecting } Learning I } Learning II } Learning III } Learning IV } Reflecting } Wrap-Up
Welcome!
} Introduction to Educator Evaluation in Washington } Using Instructional and Leadership Frameworks in Educator
Evaluation } Preparing and Applying Formative Multiple Measures of
Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring
} Including Student Growth in Educator Evaluation } Conducting High-Quality Observations and Maximizing Rater
Agreement } Providing High-Quality Feedback for Continuous Professional
Growth and Development } Combining Multiple Measures Into a Summative Rating
4
Modules
5
The Evaluation System Components
} Pausing } Paraphrasing } Posing Questions } Putting Ideas on the Table } Providing Data } Paying Attention to Self and Others } Presuming Positive Intentions
} What Else?
6
Session Norms
Connecting
Builds community, prepares the team for learning, and links to prior knowledge, other modules, and current work
7
} Conducting High-Quality Observations } Maximizing Rater Agreement
Reminder! } This module provides an orientation to the basic
concepts. } This module does not go into great depth about evidence
relating to any of the specific instructional or leadership frameworks and instead leaves it up to the districts to seek additional training.
}
8
Module Overview: 2 Parts
Participants will know and be able to: } Describe the role of observation as a methodology for
gathering evidence of educator performance and its role in achieving rater agreement.
} Utilize the revised WACs to identify the number and length of observations for different categories of staff.
} Identify common observation biases in their own and others’ practice.
} Utilize appropriate strategies for minimizing bias in the observation.
} Utilize best practices in collecting observation data.
9
Overview of Intended Participant Outcomes
Remember: Observation is one measure within a comprehensive evaluation system that includes multiple measures, including the following: } Self-Assessment } Goal-Setting } Artifact Review
10
Observation as One Piece of the Puzzle
Observation is another piece, helping us move closer to enacting TPEP’s Core Principles and the new system
“We Can’t Fire Our Way to Finland” } The critical importance of teacher and leadership quality } The professional nature of teaching and leading a school } The complex relationship between the system for teacher
and principal evaluation and district systems and negotiations
} The belief in professional learning as an underpinning of the new evaluation system
} The understanding that the career continuum must be addressed in the new evaluation system
} The system must determine the balance of “inputs or acts” and “outputs or results”
11
TPEP Core Principles
12
Rater Agreement Depends on High- Quality Evidence Collection Practices
Criterion 2
Criterion 1
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Criterion 7
Criterion 8
Frameworks +
Student Growth Rubrics
• Observation • Student
growth data • Artifacts • Other
evidence relevant to the frameworks
District- determined process Distinguished Proficient Basic Unsatisfactory
State- determined process Distinguished Proficient Basic Unsatisfactory
Student Growth Measures
(From 3 specific criteria)
Student Growth Impact
Ratings: Low, Average, High
13
The extent to which the scores between the raters exhibit consistency and accuracy against predetermined standards. The predetermined standards are the instructional and leadership frameworks and rubrics that define the basis for summative criterion-level scores.
Rater Agreement
} Evaluators need multiple opportunities and settings to observe and assess educator practice.
} Multiple observations paired with timely feedback are a key part of a strong evaluation system.
} Using multiple observers and multiple observations improves the reliability of scores more than having longer observation periods.
} Using a mix of shorter and longer observations can decrease costs while maintaining increased reliability of scoring.
} Using a mix of internal and external observers (school-based and not) can help determine the validity of practice scores.
(Sources: Ho & Kane, 2013; Sartain et al., 2011)
14
What the Research Says…
ê Handout 1: Studies link classroom observations to student achievement, by Stephen Sawchuck
ê Handout 2: Feedback for Better Teaching, by The Measures of
Effective Teaching (MET) Project. (pp. 1-2 and 6-7; skip pp. 3-5)
15
Voices on Observation
} What are the key messages and ideas in the two articles with regard to observations?
} Based on what we just read, what are some best practices for observation?
16
Voices on Observation Wrap-Up
Learning
Understand the new requirements for educator observation in Washington, as well as best practices and strategies for
conducting high-quality observations
17
} 1. In the 2012-2013 school year, Mr. Davis was in his eighth year of teaching at your school. He has continuing contract status and was rated “satisfactory” in his last four evaluations.
True or False: Mr. Davis is eligible for focused evaluation in 2013-2014.
} Answer: True RCW 28A.405.100 (12)(b)(iv)(c)(i)
18
Learning Introduction: Meet the New Rules
} 2. Fill in the blank: Mr. Garvey has been a principal in North Thurston for the past five years. He is starting his first year as a principal in your district in September. Mr. Garvey should have a ___________ evaluation [focused or comprehensive] this year.
} Answer: Comprehensive RCW 28A.405.100 (12)(b)
19
Learning Introduction: Meet the New Rules
} 3. Fill in the blank: Mr. Burke has taught for ten years in your district and has continuing contract status. He is currently on a comprehensive evaluation cycle. Mr. Burke must be observed ____ times this year for no less than _____ minutes total.
} Answer: 2 and 60 WAC 392-191A-070
20
Learning Introduction: Meet the New Rules
} 4. Fill in the blank: Mrs. Wright is a 20-year veteran teacher but just started teaching at your school this year. As a new employee, Mrs. Wright must be observed once for ___minutes during the first ____calendar days of the employee period.
} Answer: 30 and 90 WAC 392-191A-070 (2)
21
Learning Introduction: Meet the New Rules
} 5. Mr. Jones is starting his third year in teaching and is a provisional teacher. This year, he must be observed: } A. At least two times for a total of 120 minutes } B. At least three times for a total of 90 minutes } C. At least four times for a total of 150 minutes
} Answer: B WAC 392-191A-070 (3)
22
Learning Introduction: Meet the New Rules
} 6. Ms. Thomas has been on a focused evaluation for the past two years. In both years, she was rated as proficient. This year: } A. She must have a comprehensive evaluation. } B. She and her supervisor can decide whether to continue with
a focused evaluation or switch to a comprehensive evaluation. } C. She must have a focused evaluation.
} Answer: B WAC 392-191A-070(12)(c)(iv)
23
Learning Introduction: Meet the New Rules
} 7. Mr. Tucker had a comprehensive evaluation last year and was rated as “proficient”; however, he and his evaluator identified Criteria 4 as an area that needed improvement.
} True or False: Mr. Tucker is required to use Criterion 4 for his focused evaluation next year.
} Answer: False WAC 392-191A-120(2)
24
Learning Introduction: Meet the New Rules
} 8. Mr. Tanyu has continuing contract status and is on a focused evaluation cycle. He selected Criteria 7: “Communicating and collaborating with parents and school community.” You notice that Criteria 7 does not need to be assessed through observation; other methods, such as artifact review, may be more appropriate. Mr. Tanyu: } A. Does not need to be observed this year. } B. Must be observed this year. } C. Districts can decide whether or not to require observation in this
situation.
} Answer: False WAC 392-191A-110
25
Learning Introduction: Meet the New Rules
} 9. Part I: Principal Smith completed an observation of Ms. Williams on Tuesday and promptly wrote up the results of the observation later that day. Principal Smith must give a copy of the report to Ms. Williams within: } A. four days. } B. no more than one week. } C. three days.
} 9. Part II: Principal Smith and Ms. Williams must hold a minimum of ____ confidential conferences during each year.
} Answers: C and 2 WAC 392-191A-130
26
Learning Introduction: Meet the New Rules
} 10. True or False? The purpose of observation is to: } (a) collect accurate, relevant evidence to evaluate against
instructional and/or leadership frameworks, and } (b) provide educators with quality feedback.
} Answer: True!
27
Learning Introduction: Meet the New Rules
} Frequent: research agrees that a single observation of teacher or leader practice is insufficient for capturing evidence to generate accurate and reliable ratings.
} Comprehensive: captures evidence appropriate for assessing the educator against the required criteria.
} Accurate: minimizes bias and error to ensure that collection and review of the data produces an authentic assessment of the educator’s performance.
} Useful & Timely Feedback: generates detailed, focused feedback for improvement.
28
Learning Content I: High-Quality Observation
} What does it mean to be [INSERT DESCRIPTOR] in your observations?
} How does this principle of high-quality observations better support the improvement of teaching and learning?
29
Learning Content I Activity: High-Quality Observation
} Share out your two key ideas for your Guiding Principle
} Frequent
} Comprehensive
} Accurate
} Feedback
30
Learning Content I Activity: Debrief
A skilled observer: } 1. Understands each component and indicator on the
district rubric thoroughly and deeply.
} 2. Gathers and sorts sufficient evidence of practice as it happens in the classroom or school.
} 3. Recognizes and puts aside preferences and biases.
} 4. Interprets the evidence appropriately to give an accurate rating using the evaluation instrument.
31
Learning Content II: Strategies for Reducing Bias in Observation
(McClellan, Atkinson, & Danielson, 2012)
} Bias enters the process in two places:
} Observation bias: while recording data during an observation, every observer has a “lens” altered by: } Specializations: content or grade level } Personal instructional preferences } Past observations or impressions of the educator } Individual bias or experiences.
} Rater error and bias: observers rate performance too high or too low based on considerations other than the instructional or leadership framework.
32
Common Sources of Bias in Observation
} Record evidence, not judgment or opinion: } Evidence describes a concrete behavior, practice, interaction, or
material that can be used to support a rating decision and provide clear feedback.
} Judgment or opinion comments on or interprets what is being observed but does not provide concrete evidence.
} Observation notes are written as evidence statements: } Evidence statement: A factual statement, free from opinion or
bias, that describes teacher practice.
33
Reducing Bias in Observation
} EXAMPLE: Opinion Statement: “Yes! Good questioning, engages the students.” Evidence Statement: T: “How do you know, or how did you get that answer?” S: (Multiple hands shoot up) T: Calls on Alissa S: “I went back and used Pythagorean’s formula.” S writes out process on board and explains each step.
34
Reducing Bias in Observation
} Record: } Direct quotes } Questions asked } Observed actions or movements } Literal descriptors
} Avoid: } Jotting down extraneous information } Jotting down initial thoughts on ratings
35
Strategies for Recording Evidence
} Examine Handout 4
} Highlight factual statements
} Underline judgment statements or statements not based on evidence
36
Learning Activity II: Evidence Versus Opinion
} Holding the Pre-observation Conference } Review the proposed lesson plan } Listen to the teacher’s perspective and learn about the
classroom context } Understand the teacher’s professional goals and expectations } Identify any specific areas the teacher would like feedback on
37
Learning Content III: Best Practices for Collecting Observation Data
} Before Observing: } Focus: Review pre-conference notes and the educator’s
professional goals
} Clean Your Slate: Purposefully identify and set aside previous impressions from earlier observations
38
Best Practices for Collecting Observation Data
} While Observing: } Be “a fly on the wall”: Avoid interacting with the participants to
minimize your influence on what you are observing. } Record only information you actually observe—do not “fill in
the blanks” or assume other practices are occurring without actually observing them.
} Record the depth and frequency of key “look-fors” (e.g., types of questioning, student behaviors, re-direction to task).
39
Best Practices for Collecting Observation Data
} Balance taking detailed notes with observing the action in front of you. } Use abbreviations to increase efficiency (e.g., “T” for teacher,
“S” for student, or components by their first two letters—“LD” for lesson delivery).
} Paraphrase around short quotes to convey the spirit and tone of the interaction.
40
Best Practices for Collecting Observation Data
41 41
} Example Observation Notes
Best Practices for Collecting Observation Data
42
Learning Activity III: Practicing Collecting Observation Data
} Video Background } “Ms. K” is teaching a sixth-grade science lesson. } 25 students, ranging from 11 to 13 years old, are in her class. } Two students are English language learners and several
students have IEPs for accommodations in reading.
NBPTS Video #161: Probing Student Understanding (http://198.101.195.20/video.php?p=15346487)
} Compare your notes for the following: } Step 1: Do the notes capture the same data?
} What are you missing?
} Step 2: Do the notes record evidence, such as: } quotes? } questions? } actions or movements?
} Step 3: Do the notes include judgments or opinions? } Step 4: Did you identify any personal biases while you were taking
notes?
43
Learning Activity III: Practicing Collecting Observation Data
} How many of you know what eVAL is? } Is anyone currently using eVAL? } Is anyone considering using eVAL?
44
Learning Content IV: Overview of eVAL
} A web-based tool designed to manage the evaluation process and documentation
} A free resource! } Personalized for each district for their instructional
framework, resources, and documents } Voluntary for all districts, who can use as many or as
few of eVAL’s features as they’d like (or none at all) } Extremely secure, with limited access physically and
virtually to its servers
45
eVAL
} eVAL helps evaluators and educators in five specific ways: } Practice: Enter formative evaluation data regarding videos or
classroom observations and engage in a dialogue about shared (or different) perceptions of quality instruction.
} Conference: Plan for pre-conferences, observations, and post-conferences—including sending questions in advance of meetings and scheduling observations.
46
eVAL
} Observe: Principals can… } Enter notes during observations (or upload them from other
software). } Align evidence from notes to instructional and leadership
frameworks and provide feedback to teachers. } Score and annotate the rubric and manage all aspects of
evidence gathered in the observation. } Create post-conference observation reports.
47
eVAL
} Summary & Reports: Principals can… } Submit final summative scores for teachers to their district office. } Reflect on all the evidence gathered for the year. } Score final summative rubrics. } Print final summative reports, including schoolwide evaluation
summaries, individual evaluation trends, and discrepancies between teacher self-assessments and evaluator scores.
48
eVAL
} For more information, check out the eVAL website: } http://tpep-wa.org/resources/eval/
} Watch eVAL video walkthroughs: } http://tpep-wa.org/resources/eval/eval-video-walkthroughs/
49
eVAL
Reflecting
50
Select one of the Debrief Cards and share:
} One key observation I made during this session
} One key question I have as a result of this session
} One way someone else helped me out during this session
} Something new I have learned as a result of this session
} One feeling I experienced during this session
51
Whip Around and Plus/Delta Debrief
Wrap-Up
Understand the options for homework
Preparation for the Maximizing Rater Agreement module
52
} Bring a copy of: } Your observation notes from Learning Activity III. } Your district’s instructional framework.
You will use these to become familiar with best practices in analyzing observation data and achieving rater agreement.
53
Preparation for the Maximizing Rater Agreement Module
} Discuss and plan an annual schedule of observations: } Consider the number of evaluators, the number of teachers,
and the number of school days. } Consider how many teachers and principals are on focused
versus comprehensive evaluations. } Consider setting deadlines for completing the observation of
all teachers.
54
Suggested Homework
