Enhancing Professional Practice: Observation Skills using the Framework for Teaching

www.DanielsonGroup.org

DOMAIN 1: Planning and Preparation

1a Demonstrating Knowledge of Content and Pedagogy

1b Demonstrating Knowledge of Students1c Setting Instructional Outcomes1d Demonstrating Knowledge of Resources1e Designing Coherent Instruction1f Designing Student Assessments

DOMAIN 4: Professional Responsibilities

4a Reflecting on Teaching4b Maintaining Accurate Records4c Communicating with Families4d Participating in a Professional Community4e Growing and Developing Professionally4f Showing Professionalism

DOMAIN 2: The Classroom Environment

2a Creating an Environment of Respect and Rapport

2b Establishing a Culture for Learning2c Managing Classroom Procedures2d Managing Student Behavior2e Organizing Physical Space

DOMAIN 3: Instruction

3a Communicating With Students3b Using Questioning and Discussion

Techniques3c Engaging Students in Learning3d Using Assessment in Instruction3e Demonstrating Flexibility and

Responsiveness

The Danielson Framework for Teaching

Enhancing Professional Practice

Observation Skills using the Framework for Teaching

All rights reserved. No part of this manual may be copied, reproduced, translated or transmitted inany form or by any means, electronic or mechanical, including photocopying, recording, or by anyinformation storage or retrieval system now known or to be developed, without written permissionfrom The Danielson Group, except for the inclusion of brief quotations or for classroom educationalpurposes, or in connection with reviews.

Copyright © 2012 by The Danielson Group

1

Agenda:• Framework Review• Plan-Teach-Reflect-Apply• The Collaborative Observation Process• The Planning Conference• Evidence vs. Opinion• Minimizing Bias• Evidence Collection• Evidence Interpretation• The Reflection Conference• Writing Rationales for Levels of Performance• Reflection

Outcomes: An opportunity to acquire the skills needed to conduct classroomobservations using the Framework for Teaching (2 day session)

Participants will:• Distinguish between evidence and opinion• Be aware of (and control for) personal biases• Analyze a planning conference• Collect classroom-based evidence• Align evidence to components• Examine evidence against critical attributes• Determine level of performance• Analyze a reflection conference• Write a rationale for the level of performance

Norms:• Equity of voice• Attentive Listening• Safety to share different perspectives• Commitment to the work

2

Professional Improvement is characterized a cycle that includes four steps: Plan—Teach—Reflect—Apply:

PLAN:What are my goals?What do I know about my students?Information gatheringDeveloping a plan

REFLECT:What have I learned ...

about my students?about my teaching?about the community of

learners at my school?

TEACH:Implementation in the classroom

APPLY:How will I apply what I’ve learned in my classroom to enhance student learning?

Professional Growth Cycle

3

Collaborative Observation Process

4

COMPONENTObservable Evidence from the

Written Lesson Plan Additional Specificity Needed

1a: DemonstratingKnowledge ofContent andPedagogy

1b:DemonstratingKnowledge ofStudents

1c: SettingInstructionalOutcomes

1d:DemonstratingKnowledge ofResources

1e:Designing Coher-ent Instruction

1f: Designing Student Assessments

Analyzing Evidence of Domain 1

5

Questions/Comments for the Teacher during the Planning Conference

6

The Planning ConferenceConsider the teacher’s planning conference ...

What do we know about the teacher’s student learning priorities?

Is there anything “missing” from the plans?

What are two important questions that you have for the teacher during the pre-observation confer-ence that will support professional reflection and learning about the lesson?

7

Evidence or Opinion? E O1 Some students have difficulty staying engaged.

2 The teacher asked five yes/no questions in the first five minutes.

3 The teacher said that the Civil War was a tragedy for U.S. civilization.

4 The seating arrangement should be flexible because it is kindergarten.

5 “I assure you that today’s lesson will be quite interesting.”

6 The last activity, discussion of the key scene, was rushed.

7 The teacher clearly has planned and organized for maximum effect.

8 As the activity progressed, students started calling out, “What should we do next?”

9 The teacher says today’s activities are an extension of the math unit.

10 The new table arrangement encourages concentration and controlled interactionwith neighbors.

11 The pacing of the lesson was slow, allowing many possibilities for student restlessness, and disruptive behavior.

12 Students worked with a classmate in choosing key scenes and discussing the reasons for their choice.

13 The class was chaotic and out of control.

14 Seventy-five percent of the students were out of their seats and were not workingon the assignment.

15 The teacher spent most of the class period talking to students in the front half of the classroom.

16 The teacher prefers to work with female students rather than with male students.

17 The students were bored and uninterested.

18 Five students had their heads down during the teacher’s lecture.

19 All students wrote in their journals.

20 The students were unclear about the objective of the social studies activity.

21 The teacher was fair and consistent in her discipline management strategies.

22 The teacher circulated throughout the entire room while the students worked in theirgroups.

Evidence vs. Opinion

8

Many observers of teaching confuse descriptions of classroom practice with opinions about class-room practice. Opinions can be a reflection of one!s biases and personal preferences, particularlywhen they are not supported by a collection of evidence. Descriptions (evidence) and opinions differin the following way:

Description of Classroom Practice (Observer records an event with no interpretation)

Versus Opinion About Classroom Practice

(Observer interprets an event based on own beliefs about good teaching)To consistently apply the rubric to observations of classroom practice, it is essential to be able tomake observations of evidence that stand independent of opinions (premature interpretations of evi-dence that are based on personal beliefs).Evidence and Opinion

Evidence Types:

Verbatim scripting of teacher or student comments:

“Would one person from each table come to collect the materials?” “We have five more minutes to finish. Let’s look over our work before we hand it in!“

Non-evaluative statements of observed teacher or student behavior:

The teacher stood by the door, greeting students as they entered. Students were seated at tables in groups of four, working independently.

Numeric information about time, student participation, resource use, etc.:

Three students offered 80% of the comments during the discussion. Fifteen minutes was spent in circle time.

An observed aspect of the environment:

The assignment was on the board for students to do while attendance was being taken. There were three centers designed for independent work.

EVIDENCE OPINION

observable draws conclusionsobjective subjective

free of value judgment may include value judgment

Evidence vs. Opinion

9

TEACHER A TEACHER B

TEACHER C TEACHER D

What words come to mind...when you look at each picture of a teacher?What does each image tell you about the teacher?

Bias Errors

BIAS ERRORS

A bias rating error is any attitude, tendency to respond in a certain way, or inconsistency on the partof the supervisor which impedes objectivity and accuracy in the evaluation process. There are anumber of biases that may affect judgments. For example, a person from one region of the countrymay regard people from another region as holding different values or not as urbane. Visual or auralcues concerning a teacher’s religious or sexual orientation may trigger biased judgments. None ofthese influences, it should be noted, are directly related to teaching effectiveness, but they can sub-consciously affect your judgment as an observer if you allow them to. The following types of errorsare the most common:

First Impression Error

The tendency of a rater to make an initially favorable or unfavorable judgment about a person andthen ignore or distort subsequent information so as to support the initial judgment. For Example:Mrs. A’s desk is really neat and well-organized, so the rater assumes a well managed classroom. Mr.B’s shirt is always untucked, so the rater looks for other examples of sloppy performance.

Halo/Horns Error

The tendency of a rater to make generalizations from one aspect of a person’s performance to all as-pects of a person’s job performance. For Example: Mr. Z always volunteers for extracurricular tasks, sothe rater sees only the positive performance examples in his classroom. Ms. H loses instructional timerepeating task directions, so the rater looks for other low performance examples in her classroom.

10

Compare/Contrast Error

The tendency of a rater to evaluate a person relative to other individuals rather than on how well theperson fulfills the requirements of the job. For Example: Mrs. T uses the same routines in her class-room as Mrs. V who is a master teacher. The rater generalizes from this similarity and looks for addi-tional positive examples during her observation. Mr. G is the only teacher who did not participate inthe voluntary staff development session. The raters negative judgment about this absence influenceshis observation of the classroom.

Similar/Dissimilar to Me Error

The tendency of a rater to evaluate more favorably those people whom they perceive as similar tothemselves. For Example: Mrs. J’s has created a classroom that is just like the rater’s last work setting.Therefore, the rater is more likely to look for positive examples during an observation. Mr. D frequentlyreads the same books as the rater. This leads the rater to view Mr. D’s performance more positively.Mrs. A has a similar accent to the rater. This leads the rater to view Mrs. A in a positive light.

Central Bias Error and/or Negative-Positive Error

The tendency of a rater to score people either at the extreme ends of the scale (too harshly or too le-niently) or close to the midpoint of a scale when their performance justifies a substantially higher orlower rating. For Example: The rater never gives more than a 2 on anything or always gives 3’s oneverything.

I Know You Error

The tendency of a rater to allow previous history or present relationship to influence the score. ForExample: Mr. L has a long standing friendship with the rater, as they formerly co-taught a humanitiescourse. As a result, the rater is likely to look for positive performance examples. Mrs. P is argumen-tative during staff meetings, so the rater is likely to look for negative examples in her classroom.

Emotional State Error

The tendency of a rater to allow their present emotional state to influence the score. For Example:The rater had an argument with her teenage daughter before leaving for work and her resulting moodcauses a negative lens on her observations. The rater got engaged over the weekend and his goodmood influences his observations positively.

Tips for Avoiding Bias

All performance ratings should be established against fixed standards, not compared to previousperformance, the ratings of others, initial impression or any factors other than the observable evi-dence and the specific rubric.

Bias Errors

11

Some tips to guide non-biased scoring include:• Consider whether the person being evaluated has done anything unusually good or bad in the

last few months. Either situation can influence your thinking.• Ask yourself whether you feel the person has a particularly pleasant or unpleasant personality

and whether this might be influencing your perspective regarding their job performance.• Remember that it is unlikely that any staff member either “exceeds expectations” or “need

improvement” in every component.• Study your ratings to determine whether you might be giving higher ratings to individuals more

similar to yourself. Be particularly alert for this problem when rating a staff member who is a goodfriend or with whom you socialize.

• Study your ratings to determine whether you might be giving lower ratings to staff members whoare very dissimilar to you or whom you dislike.

What bias errors are in play when you observe images of “famous” teachers?

Cameron Diaz in Bad Teacheras Elizabeth Halsey

Edward James Olmos inStand and Deliveras Jaime Escalante

Ben Stein in Ferris Bueller’s Day Offas the Economics teacher

Maggie Smith inthe Harry Potter seriesas Minerva McGonagall

Richard Dreyfuss inMr. Holland’s Opusas Glenn Holland

Arnold Schwarzenegger inKindergarten Cop as John Kimble

Bias Errors

12

Data Collection Sheet

TIME EVIDENCE COMPONENT

13

Data Collection Sheet

TIME EVIDENCE COMPONENT

14

The 3 Rs ...

REMAIN ... to clean up data

• be sure the data is truly representative• check that only facts are recorded• whenever possible, quantify words such as few, some, most• use quotation marks when quoting teachers or students• add any missing information

REVIEW ... data while it’s still fresh

• clarify anything that needs further explanation• be sure there are no opinion statements or gaps• check that the documentation does not indicate personal or professional preferences

REFLECT ... to align the evidence statements to the Framework components

• check that interpretations are free from bias• be sure that interpretations are based on a preponderance of evidence• accurately compare the preponderance of evidence against the levels of performance• check that interpretations are aligned to the critical attributes• share with colleagues using the language of the Framework• be sure any summary statements are clear and reflect strengths as well as growth opportunities• check the tone of summary statements so they are neither positively or negatively framed

Collecting Evidence

15

In 2009, the Bill and Melinda Gates Foundation embarked on the large research project “Measures ofEffective Teaching (MET),” which entailed the video capture of over 23,000 lessons, analyzed ac-cording to five observation protocols, with the results of those analyses (together with other meas-ures) correlated to value-added measures of student learning. The aim of the study was to determinewhich aspects of a teacher’s practice were most highly correlated with high levels of studentprogress.

The Framework for Teaching was one of the models selected for this study, which, because ofits size, entailed the (online) training and certification of hundreds of observers for the purpose of rat-ing the quality of teaching in the lessons. In order to fulfill this obligation, it became necessary tosupply additional tools to aid in the training of observers, so that they could make accurate and con-sistent judgments about teaching practice as demonstrated in the large numbers of videotaped les-sons. The tools required were of several types:

• Rubric language tighter even than that of the 2007 edition of The Framework for Teaching.Furthermore, the levels of performance in the 2011 revision are written at the component, ratherthan the element, level. While providing less detail, the component level rubrics capture all theessential information from those at the element level and far easier to use in evaluation than arethose at the element level.

• “Critical attributes” for each level of performance for each component. These critical attributesprovide essential guidance for observers in distinguishing between practice at adjacent levels ofperformance. They are of enormous value in training and in the actual work of observation andevaluation.

• Possible examples for each level of performance for each component. These examples serve toillustrate the meanings of the rubric language. However, they should be regarded for what theyare: possible examples. They are not intended to describe all the possible ways in which a certainlevel of performance might be demonstrated in the classroom; those are, of necessity, particularto each grade and subject. The possible examples simply serve to illustrate what practice canlook like in a range of settings.

These enhancements to The Framework for Teaching, while created in response to the de-mands of the MET study, have turned out to be valuable additions to the instrument in all its applica-tions. Practitioners have found that the enhancements not only make it easier to deterine the level ofperformance reflected in a classroom for each component of The Framework but also contribute tojudgments both more accurate and more worthy of confidence. As the stakes in teacher evaluationbecome higher, this increased accuracy is absolutely essential.It should be noted that there are absolutely no changes to the architecture of The Framework forTeaching in the 2011 to the 2007 edition: it contains the same 4 domains, the same 22 components,and all of the same elements. Therefore, those educators who have invested resources in learningthe language of the 2007 edition will find nothing to confuse them. They should expect to discoverthat the additional tools, added initially in response to the demands of a large research project, as-sist them in the challenging work of applying the framework to actual classroom teaching.

The 2011 Framework for Teaching Evaluation Instrument

16

UNSATISFACTORY BASIC3c E

NG

AG

ING

ST

UD

EN

TS

IN

LE

AR

NIN

G

Few students are intellectually engaged inthe lesson.

Learning tasks require only recall or have asingle correct response or method.

The materials used ask students to performonly rote tasks.

Only one type of instructional group is used(whole group, small groups) when varietywould better serve the instructionalpurpose.

Instructional materials used are unsuitableto the lesson and/or the students.

The lesson drags or is rushed.

Some students are intellectually engaged inthe lesson.

Learning tasks are a mix of those requiringthinking and recall.

Students are in large part passivelyengaged with the content, learning primarilyfacts or procedures.

Students have no choice in how they com-plete tasks.

The teacher uses different instructionalgroupings; these are partially successful inachieving the lesson objectives.

The materials and resources are partiallyaligned to the lesson objectives and only insome cases demand student thinking.

The pacing of the lesson is uneven—suitable in parts, but rushed or dragging inothers.

The learning tasks and activities, materials,resources, instructional groups andtechnology are poorly aligned with theinstructional outcomes or require only roteresponses.

The pace of the lesson is too slow or toorushed.

Few students are intellectually engaged orinterested.

The learning tasks and activities arepartially aligned with the instructional out-comes but require only minimal thinking bystudents, allowing most to be passive ormerely compliant.

The pacing of the lesson may not providestudents the time needed to be intellectu-ally engaged.

Critical Attributes

17

PROFICIENT DISTINGUISHED

Most students are intellectually engaged in thelesson.

Learning tasks have multiple correctresponses or approaches and/or demandhigher-order thinking.

Students have some choice in how they com-plete learning tasks.

There is a mix of different types of groupings,suitable to the lesson objectives.

Materials and resources support the learninggoals and require intellectual engagement, asappropriate.

The pacing of the lesson provides students thetime needed to be intellectually engaged.

In addition to the characteristics of “proficient”:

Virtually all students are highly engaged in thelesson.

Students take initiative to modify a learningtask to make it more meaningful or relevant totheir needs

Students suggest modifications to the groupingpatterns used.

Students have extensive choice in how theycomplete tasks.

Students suggest modifications or additions tothe materials being used.

Students have an opportunity for both reflec-tion and closure after the lesson to consolidatetheir understanding.

The learning tasks and activities are alignedwith the instructional outcomes and designedto challenge student thinking, the result beingthat most students display active intellectualengagement with important and challengingcontent and are supported in that engagementby teacher scaffolding.

The pacing of the lesson is appropriate, provid-ing most students the time needed to beintellectually engaged.

Virtually all students are intellectually engagedin challenging content through well-designedlearning tasks and suitable scaffolding by theteacher and fully aligned with the instructionaloutcomes.

In addition, there is evidence of some studentinitiation of inquiry and of student contributionto the exploration of important content.

The pacing of the lesson provides students thetime needed to intellectually engage with andreflect upon their learning and to consolidatetheir understanding.

Students may have some choice in how theycomplete tasks and may serve as resourcesfor one another.

18

Data Collection Sheet

TIME EVIDENCE COMPONENT

19

Data Collection Sheet

TIME EVIDENCE COMPONENT

20

Evidence Organization

21

ReflectionHow has your work today addressed a paradigm shift in teacher evaluation?

• Indicate at least one new LEARNING from the session:

• Indicate a CONCERN you have about Observation Skills:

• Indicate a QUESTION you still have about Observation Skills:

22

The Role of TrustIn the center, write the names or initials of teachers with whom you work and for whom you have ahigh level of trust. With these educators in mind, fill in the other sections as indicated

Definition Characteristics

Examples Non-examples

TRUST

Trustworthy leaders strike the right balance between pushing too hard and pushing too little. They are soft on people and tough on

values/goals/expectations. They combine personal humility— exercising restraint and modesty—with tenacity and the professional will to see that the work of teaching is accomplished and accomplished well.

Trust matters to successful leaders and their schools

23

Research Spotlight: Trust Building in Teacher Evaluation Process

A culture of cooperation and trust between the evaluator and teachers must exist before effectivelyutilizing teacher evaluation to link to teaching practices regarding student learning. During this timeof high-stakes testing and increasing teaching accountability, the relationship between levels of theschool system are often strained. The evaluator must foster a culture of shared leadership and pro-fessional commitment with all stakeholders in the school community for successful school-widelearning improvement while implementing complex change such as a new teacher evaluationprocess (Wahlstrom & Louis, 2008). Inherent in collaborative, trusting relationships is an element ofcommitment, defined by Chhuon, Gilkey, Gonzalez, Daly, and Chrispeels (2008) as “one’s willingnessto participate in a relationship that involves being vulnerable to another person” (p. 228). Facets oftrust include risk, communication, benevolence, reliability, competence, integrity, openness, and re-spect (Daly & Chrispeels, 2005).Trust building around teacher evaluation requires that 1) teacher effectiveness criteria be clearly understood by all staff that ultimately will be measured

by it 2) teachers have input and shared conversation with evaluators around teaching and professional

practices3) evaluators be knowledgeable about instruction and core curriculum/assessments in order to

provide valid feedback and professional recommendations4) teaching data be collected through multiple measures over multiple times in order to come to a final

rating decision. (Ashby & Krug, 1998; Danielson & McGreal, 2000; Kimball & Milanowski, 2009)

Trust and Performance Evaluation

Supervision is one aspect of an evaluator’s role as instructional leader in which the establishment oftrust and the language of coaching are especially important. Not surprising, trust has been found toplay a significant role in employee reactions to supervision. Not surprising, trust has been found toplay a significant role in employee reactions to supervision. In a study of perceptions of fairness ofperformance evaluations by supervisors, the level of trust in the supervisor was more important in re-gard to perceived fairness than any other characteristics of the performance evaluation process. Thetraditional norms of schools have allowed teachers a great deal of autonomy and little supervision.Teachers have enjoyed the trust, or at least the neglect, of their supervisors. In the age of accounta-bility, however, this state of affairs is changing. New systems of teacher evaluation require greater in-spection of teachers’ classroom practice. Greater scrutiny may be perceived as a lessening of trustby both teachers and administrators and might in fact lead to less trust. However, if supervision ispracticed in such a way that the greater attention is perceived as increased care with a focus onproblem solving and coaching, principals will have an opportunity to demonstrate their competenceand expertise. Trustworthy leadership is likely to lead to more active a constructive supervision thatcontributes to improved instruction in the school.

Source: Trust Matters, by Megan Shannon-Moran, 2004

The Role of Trust

24

Quality and Quantity of Observation Data

Teacher Quote Framework Alignment

1. “Okay folks, let’s go over the homework. Jay,what did you get for number 1? (293). Right.Shannon, take number 2. 404? I don’t think so.Anyone else? Tom? (419) Yup. Next one, Will?Someone help him. Tim?

2. “This homework was a piece of cake, rightpeople? If you couldn’t do this, then you ought tobe thinking about another level of math for sure.Okay, any questions on any of the problems?None? Good. Exchange papers. Check the an-swers as I read. 3.9, 2.437, 6… That’s it. Add upthe incorrect items and put the total at the top.Now, moving right along. Today we…

3. “Okay. Homework check, gang. Pair up withthe person next to you. Compare solutions.Argue your point of view. In five minutes be pre-pared to nominate one problem you think it wouldbe helpful to discuss. I’ll want to know why youthought it was interesting, challenging—or whyyou considered it the headache problem of theday---and believe me there were some stinkershere, I know.”

4. Okay folks, let’s go over the homework. Jay,what approach did you take to number 1? Orderof operations? Yes. Why? How’d you figure thatout? Shannon, take number 2. 404? Tell us howyou got there. Keep talking…see something new?Okay, want to revise your response? 419? Ok,everyone, talk to your partner and decide what youthink. (Wait) Shannon, they agree with you. Lizyou are up next. Tell us what was challengingabout #3. I heard lots of you complaining aboutthat one.

25

Quality and Quantity of Observation Data

Comments

26

Data Collection Sheet

TIME EVIDENCE COMPONENT

27

Data Collection Sheet

TIME EVIDENCE COMPONENT

28

29

Read the following quote by Carl Glickman and write something about your learning over the pasttwo days:

“The goal of the evaluator should ALWAYS be to use approaches that strengthen a teacher’scapacity for greater reflection and self-reliance in making improvements in classroom teach-ing and learning.” (Glickman, 2002)

Reflection

30

31

Observation Skills using the FrameworkFormative Evaluation

DATE: _____________________________________________

LOW HIGH

Organization(Coherent and easy to follow) 1 2 3 4Presentation(Engaging and interesting delivery) 1 2 3 4Materials(Clear, useful, and aligned to presentation) 1 2 3 4Activities (Meaningful and allowed opportunities for dialogue) 1 2 3 4Pacing(Coherence supported through flow and pace) 1 2 3 4Usefulness to My Work(Applicable in my work setting) 1 2 3 4

Please explain all ratings of 1 or 2:

What information/concepts/learning will be most useful to you?

What revisions or improvements could you suggest to strengthen this learning process?