PRINCIPALS’ ATTITUDES AND INSIGHTS CONCERNING THE ...cnweb.cn.edu/tedu/NCATE-State/Examples-ProfMajor/EDUC600Sample[3].pdfPrincipal’s Attitudes 3 Definitions Within the context

PRINCIPALS’ ATTITUDES AND INSIGHTS CONCERNING THE EVALUATION

OF TEACHER EFFECTIVENESS

An Action Research Project Submitted in Candidacy for the degree of Master of

Education in Educational Leadership

Carson-Newman College

School of Education

EDUC 600

April 23, 2009

Table of Contents

Page

Acknowledgements……….………………………………………………… iii

Abstract……………………………………………………………………… iv

Chapter

I. Introduction……..……………………………………………………….. 1

Personal Perspective………………………………………………… 1

Statement of Problem……………………………………………….. 2

Importance of Study………………………………………………… 2

Definitions…………………………………………………………... 3

Limitations………………………………………………………….. 3

II. Literature Review……………………………………………………….. 4

Introduction…………………………………………………………. 4

The Importance of Teacher Evaluation……………………………... 4

Collecting Meaningful Data………………………………………… 7

Traditional Approaches: Applications and Implications……………. 10

Progressive Approaches: Appraisal and Professional Enhancement.. 14

Conclusion…………………………………………………………... 16

III. Method………………………………………………………………...... 19

Choice of Method…………………………………………………… 19

Development of the Survey…………………………………………. 19

Participants and Data Collection……………………………………. 21

Data Analysis………………………………………………………... 21

IV. Results…..……………………………………………………………….. 24

The Attitude of the Principal………………………………………… 24

The Confidence of the Principal…………………………………….. 25

The Principal and Methods of Evaluation…………………………… 27

The Principal and Frequency of Evaluation…………………………. 29

The Principal and Data………………………………………………. 31

Open Responses……………………………………………………… 32

V. Discussion………………………………………………………………… 39

The Administrator’s Feelings About Teacher Evaluation…………… 39

Evaluation in Practice………………………………………………... 40

Data-Based Decisions………………………………………………... 42

Implications for Action………………………………………………. 43

Suggestions for Further Research……………………………………. 44

References……………………………………………………………………. 47

Appendices…………………….……………………………………………… 50

A. Administrator Survey: Measuring Teacher Effectiveness………... 51

B. Survey Participant Descriptive Data……………………………… 55

C. Constructed-Response Questions, Results, and Research Notes…. 57

D. Survey Summary Report…………………………………………. 72

Principal’s Attitudes iii

Acknowledgements

I must acknowledge the principals and assistant principals of the Knox County

School system for their overwhelming response and participation in this study. Their

candid answers made my project a success. The faculty at Carson-Newman, especially

, must be recognized for providing me with the

feedback and clarity needed to complete a project of this nature. I must also thank my

family for their support and understanding as being involved in a graduate program like

this takes time away from one’s normal routine. They have given a great deal of

themselves so that I might accomplish my goals. My wife, in particular, has been patient

and considerate of the time that I have been required to dedicate to this work. I must also

individually recognize my sister who has pursued this degree right by my side. Having a

partner and friend in a sibling is a gift and being in her company made this experience

even more meaningful.

Principal’s Attitudes iv

Abstract

The purpose of this action research project was to investigate how public school

administrators view the teacher evaluation process in terms of its importance, their

feelings about who should be evaluated and how often, their confidence in both their

abilities and the evaluation model that is currently used, and their beliefs about being

ready and equipped to complete the task. A survey consisting of 30 items was used to

collect information from 93 principals and assistant principals from the Knox County

School (KCS) system in Tennessee. The information from both the Likert scale and

open-response questions was examined for trends.

The results of the survey showed that while most administrators were confident in

their abilities to effectively assess their teachers, some dissatisfaction existed among them

with regards to the current evaluation model or tool. Most administrators identified the

task of effectively evaluating teachers as an important one, but were less certain about

which types of data should be used to measure effectiveness. Participants regarded the

amount of time involved in the evaluation process and the number of responsibilities held

by administrators as hindrances to frequently and consistently assessing one’s staff. At

the same time, most agreed there was a need for increased evaluation for teachers of all

experience levels. Administrators made several suggestions about the types of

commitments that must be made to effectively evaluate and positively impact the

teaching in their schools.

Principal’s Attitudes 1

Chapter One: Introduction

Personal Perspective

I began working as an eighth grade science teacher in 2001. As a novice teacher,

being evaluated was a part of my life for the first three years that I was employed. Early

on, my administrators and supervisors praised me for doing a “good job” and told me that

I would have no trouble getting tenure. While this type of reinforcement made me feel

better about my career choice, it did very little for me in terms of making me better at my

job. As part of my first cycle of evaluations, I remember being told by the evaluator that

his recommendation for improvement was that I should add more decorations to my

room. While, I am sure that this person meant well, I could not see then, and cannot see

now, how that would make me more effective as an instructor or increase the

achievement scores of my students.

For the past three years I have participated in a school reform program in my

school system that centers on the evaluation and coaching of teachers, professional

development, student-instructional strategies, and performance-pay for teachers. Within

this system, all teachers, whether tenured or non-tenured, are evaluated and coached

multiple times each year using a set of instructional standards that are based upon known

effective teaching practices. I have been evaluated using this model and have also

conducted evaluations on the staff in my school as part of a leadership team. My

experience with this type of ongoing and intense evaluation model has increased my

capacity as a professional and reinforced the feelings I had as a novice teacher that

something was missing from those initial evaluations in which I took part.


Statement of Problem

As I began to see things from the perspective of the evaluator, I became interested

in how administrators feel about the importance of evaluating teachers, their ability to do

so effectively, and the current model that they use to assess the effectiveness of their

staff. Participating in a different evaluation system solidified many of the feelings that I

had about the shortcomings of my initial evaluations. I was curious to see how

administrators felt about assessing the quality of their staff. I hoped to better understand

the principals’ and assistant principals’ perceptions about the current evaluation model

and their abilities as an evaluator.

Importance of the Study

In today’s educational climate, accountability for the learning taking place in

every classroom is at an all-time high. Accounting for the gains made by each student

relates directly to verifying that high quality instruction is taking place in each class.

This project is important because it will help clarify what steps to take to improve the

quality of teacher evaluation protocols in our schools. Those steps may include training

for teachers and administrators, support staff to fulfill additional administrative duties, or

a modification of the current evaluation model. If a school system or a state can improve

the ability of its leaders (supervisors and administrators) to identify quality teaching and

refine those teachers in need of improvement, then it will equate with a teacher work

force that is teaching more effectively and academically impacting the students with

whom they work.


Definitions

Within the context of the study and the survey that was provided to participants,

any statement relating to the “quality” or “effectiveness” of a teacher or a staff of

teachers is defined in terms of student achievement. For example a teacher that is

“effective” or whose practices were of “higher quality” would have students who

achieved at higher levels. The terms “evaluating” and “observing” are used to describe

time spent in a teacher’s classroom collecting information about their practices or

instructional habits. This may be part of a formal evaluation or an informal visit to the

classroom. “Administrators” in this study are defined as principals and assistant

principals working in the KCS system.

Limitations

This study is limited because of the number of responses that were received. Of

the 179 administrators that were provided with the survey, 93 were completed and

returned within the specified time-frame. There were some issues with the online survey

format that prevented some participants from completing the survey correctly.

Approximately 12 to 15 additional surveys would have been completed if not for the

participant’s technological issues with the online format. Also, because the research

questions guiding this study could apply not only to administrators in Knox County, but

other counties as well, the results could have been considered more significant if more

school systems had been involved in the study.


Chapter Two: Literature Review

Introduction

States and school districts across the country spend copious amounts of time and

effort evaluating the teachers that occupy their classrooms. The methods that are used,

the processes that teachers participate in, and the data that is collected vary greatly from

situation to situation. While there is a high degree of disagreement about how to evaluate

America’s teachers, everyone agrees that it is necessary. This review of the literature on

teacher evaluation is centered on several guiding questions. Why is teacher evaluation

important? What type of data should be used to determine effective practice? How

efficient are school administrators at identifying effective teaching practice? How do

more traditional models for evaluation compare to newer models in terms of frequency of

evaluation, collection and uses of data, and professional implications for the teacher?

The Importance of Teacher Evaluation

The No Child Left Behind Act of 2001 (NCLB) is the latest reauthorization of the

primary federal law effecting the establishment and function of public education, the

Elementary and Secondary Education Act (ESEA). ESEA, which has been reauthorized

and revised in six-year intervals since 1965, was modified to contain drastic changes to

the way schools and states were held accountable for the achievement of their students.

In the statement of purpose for this piece of legislation, Public Law 107-110, are

suggestions for how the federal government intended to provide each student with a fair

chance of receiving a high quality education. One suggestion is the use of “high quality

accountability systems.” “Improving and strengthening accountability, teaching, and

learning by using state assessment systems” is also considered essential to providing high


quality education. Finally, “significantly elevating the quality of instruction” is

imperative to the effort (No Child Left Behind Act of 2001).

The authorization of NCLB was not the first occasion upon which accountability

became an issue in the realm of public education. It does however represent the latest

and possibly most consequential piece of legislation aimed at reforming the practices of

schools that are failing to meet the needs of their students. Accountability exists in many

forms and at many levels. Regardless of the level where accountability begins, it

ultimately filters down to a teacher in a classroom with a group of students. Researchers

have drawn several conclusions about the impact that teachers have on their students.

Eric Hanushek (2007) found, while studying the variance in teacher quality, that

teachers performing at the highest levels of quality could get an additional year’s worth

of growth out of students when compared to students instructed by a teacher from the

lowest quality category. Another study revealed that a teacher performing at a quality

level one standard deviation higher than an average teacher would produce effects in the

students equivalent to those observed when class size was reduced by ten students. This

would have represented a 50% decrease in the size of the classes observed (Hanushek,

2007). Tennessee is one of the states that pioneered the collection of long-term data

connecting students and their achievement to the districts, schools, and teachers to which

they were assigned. The Tennessee Value-Added Assessment System (TVAAS) records

and tracks the achievement of students enrolled in Tennessee public schools. The data

generated through this system suggests that over time, the best predictor for a student’s

academic growth is not socio-economic status, race, or class size. The most impacting


factor upon the academic growth of a student is the effectiveness of the teacher in the

classroom (Sanders & Horn, 1998).

A study conducted in Los Angeles schools found that students assigned to a

teacher in the top quartile of effectiveness would show a growth of five percentile points

when compared to their peers. At the same time, students with teachers from the lowest

quartile would, on average, lose five percentile points. This study focused on the needs

of low income students and the achievement gap between white and black students. The

findings suggested that assigning all black students to highly effective teachers for four

consecutive years could essentially close the achievement gap between these groups

(Haycock & Crawford, 2008).

The TVAAS study of 1998 also hinted at the cumulative effects of teacher

instruction. It found that residual effects from either effective or ineffective instruction

were still measurable in terms of student performance two years after the student moved

on from the teacher’s class (Johnson, Kahle, & Fargo, 2007). If improving student

achievement, closing the achievement gaps between different groups, and making good

on the promise of a fair and equitable chance at a high-quality education are the goals of

NCLB, then accountability for what teachers are doing in the classroom cannot be

overlooked. Based on the aforementioned studies and others like them, there are

sufficient reasons to examine the practices of teachers, to target teacher efficiency as an

area in need of increased accountability, and to concentrate on the quality of classroom

instruction as part of the reform efforts in underperforming schools.


Collecting Meaningful Data

Different from the sorts of structural systemic elements (i.e. class size, gender-

based instruction) that have traditionally been addressed in reform efforts, focusing on

the elements of teacher quality means clearly defining effective practice in terms of

teaching standards that are connected to what teachers should know and be able to do

(Ingvarson & Rowe, 2008). This is more than a checklist of behaviors. Process-product

research in the field of education was often centered on checklists of behaviors that were

used to evaluate the practices of teachers that produced successful students. From this

type of research came the idea that a behavior present in a classroom where students

displayed success would create success in every classroom in which it was used.

Because this type of research defined success in terms of performance on a standardized

test, the measurements of effectiveness were focused on the type of low-level information

often found on these assessments. These fail to fully portray effective practice, because

the use of only a few specific teaching behaviors could produce successful results

(Danielson, 1996). Charlotte Danielson (1996, p. 17) believes that “good teachers may

accomplish many of the same things (but) they do not achieve them in the same way.”

More important than a list of behaviors are context dependent actions based on common

themes.

Fenstermacher and Richardson (2005) warned against over simplifying the

relationship between teaching and learning. They argued that while good teaching (age

appropriate, morally defensible, intended to enhance the learner’s competence in the

subject) may occur, successful teaching requires that actual learning take place and

depends on many factors outside of the teacher’s activities. Like Danielson, they doubted


that quality teaching was such that it could be reduced to a step-wise process made

directly applicable from situation to situation.

There is currently a considerable policy focus on quality teaching, much of it

rooted in the presumption that the improvement of teaching is a key element in

improving student learning. We believe that this policy focus rests on a naive

conception of the relationship between teaching and learning. This conception

treats the relationship as a straightforwardly causal connection, such that if it

could be perfected, it could then be sustained under almost any conditions,

including poverty, vast linguistic, racial, or cultural differences, and massive

differences in the opportunity factors of time, facilities, and resources. Our

analysis suggests that this presumption of simple causality is more than naive; it is

wrong. (p. 191)

In order to measure effective practice, a teacher’s actions must be held up to a set

of standards. Each standard must meet several criteria in order to be considered a valid

tool for assessing performance or making judgments of observed teacher actions. The

first criterion is that the standard identifies something that is a substantial and meaningful

part of a teacher’s work. Deeper than a personality trait, the standard represents a valid

example of the work the teacher does. The second requires that the standard can be

applied to any context. Of course, the context in which it is applied will affect its

practice. The third condition is that the standard does not require teachers to standardize

their practice. In other words, the standard may identify an essential element of teaching

but does not specifically state how it will look in practice. This allows for a similar level

of professional creativity in teaching that is afforded to so many other professionals. The


final criterion for a valid standard is that it identifies an observable or measurable

phenomenon (Ingvarson & Rowe, 2008).

The validity of the standards and the degree to which they are agreed upon

becomes even more important when high-stakes decisions are being made based on their

measurements. Quite often teachers are dependent upon these evaluations for job

promotion, professional certification, or employment. The methods used and standards

applied to measuring the quality of the teachers must have a sound basis that is well

promoted and understood by all parties involved. Applying standards like the ones

mentioned above to observations of actual teaching will be necessary to provide a fair

assessment of teacher quality (Ingvarson & Rowe, 2008).

In contrast, and quite often in complement to the types of data acquired from the

application of standards-based protocols is value-added modeling (VAM). These types

of measurements identify the teacher impact on student achievement when measured by

standardized tests by examining multiple years of test score data. As mentioned before,

as an element of process-product research, these types of measures have their weaknesses

in accurately identifying high quality teaching. According to Sanders and Horn (1998),

connecting teacher effectiveness to student outcomes is an important component of any

comprehensive evaluation system. However, in isolation, this type of value-added data is

questionable when looked at as an indicator for effective practice. McCaffrey, Lockwood,

Koretz, and Hamilton (2003) studied the application of VAM in a variety of settings

across the states. They concluded that VAM currently contained too many complex

issues resulting from sampling errors and discrepancies across schools and classrooms to

reliably rank teachers in terms of effectiveness. VAM is also not considered to reliably


generate data for making high-stakes decisions (McCaffrey, et al., 2003). Measurements

that rely only on student outcomes, like the ones generated in VAM, are inappropriate as

high-stakes measures of teacher quality because they fail to measure what teachers are

trying to achieve, and they do not provide useful information so that teachers can refine

their methods and knowledge to teach in a more effective manner (Ingvarson & Rowe,

2008). While VAM is not ready to be used for any high-stakes decisions, it could be

used as an indicator for further investigation, especially in terms of the very high and

very low effect scores. This may provide systems with sources of information about best

practices or begin the process of intervention into a weaker teacher’s classroom

(McCaffrey, et al., 2003).

As determined in the previous information, the types of data that are collected to

measure teacher effectiveness can have a wide range. They can be the result of a search

for specific behaviors that have proved effective across a variety of classrooms. Data

may result from the application of a commonly held set of standards for high quality

instruction that is more dependent upon the context of the lesson. Also, they may be

derived from the student outcomes on a high stakes achievement test. In some form, each

of these types of data either is part of a current model for teacher evaluation or has been

in the past. Teacher evaluation models, just like the data they are based upon, vary

greatly between states and districts.

Traditional Approaches: Applications and Implications

The discrepancies that exist across the country in the way teachers are evaluated

are based on the decisions made in state legislatures and the rules that they set forth. The

different issues include how often the evaluations should occur, who should be


conducting the evaluations, what data are collected, and how the data collected from

evaluations can and should be used. The National Council on Teacher Quality (NCTQ)

examined teacher evaluations in terms of how they impacted who gets tenure and how

they earn it. They found that most states only require teachers to be evaluated on a two

or three year basis. If teachers in Texas receive at least a rating of proficient, they are

allowed to choose whether or not they are evaluated again during the next five year

period. States often establish a minimum number of times that a teacher is to be

evaluated. This minimum is most often the policy that is followed by the districts within

the state, making it the “unstated” maximum number (Cohen, Walsh, & Biddle, 2008).

There are only 14 states that require a teacher to be evaluated on a yearly basis.

Once they are tenured, Tennessee requires that teachers only be evaluated twice in a ten-

year period. For untenured teachers, an evaluation of the 50 largest school districts in the

United States found that more than half only required the teacher be evaluated once per

year. (Education Sector, Rush, 2008) The task of maintaining high-quality instruction in

the classroom is dependent on characterizing which teaching is effective and ineffective.

The variability with which states and districts require their teachers to be observed

highlights one of the deficits in the condition of teacher evaluation across the country.

Traditionally, a building level administrator carries the bulk of the responsibility

when it comes to evaluating their faculty. Medley and Coker (1987) found that when

comparing the relationship between 46 elementary principals’ judgments of teacher

effectiveness and their students’ gains in reading and arithmetic, virtually no relationship

existed. Also, there was very little variance between principals in the ability to accurately

predict achievement. In another study, principals were shown to be most effective at


rating their teachers’ abilities when those teachers fell in the top 10 % or bottom 10% of

effectiveness (Education Sector, Rush, 2008).

The New Teacher Project (2007) has conducted studies of the staffing practices of

many of the nation’s largest school districts in an attempt to analyze how to best serve

school employees and students. In terms of evaluation, they also found issues with how

principals ranked their faculty’s performance. They labeled the Chicago public schools

(CPS) system of teacher evaluation as ineffective. In its current state, they found that it

failed to identify low quality teachers and did not sufficiently make any progress towards

removing ineffective practitioners from the classroom. During the course of their study,

they found that only 0.3 % of the evaluations conducted by principals in the CPS system

resulted in a teacher being rated unsatisfactory. Also, during the four-year period of the

study, 88% of the schools in the CPS had not issued a single unsatisfactory rating. In the

district’s worst performing schools, as test scores declined steadily, teacher ratings

remained at the normal levels of superior or excellent, according to their evaluations.

The principals that participated in the study expressed a lack of confidence in the

evaluation tool.

The lack of faith in the standards used to assess teachers is often well deserved.

What some have deemed “drive-by” (Education Sector, Rush, 2008, p. 2) evaluations

result in high scores for teachers independent of student learning. These types of

evaluation models reduce the complex art of teaching down to a frequency of specific

behaviors akin to the process-product research discussed previously. Mannatt and

Benway (1998) identified some of the central issues with single-source evaluations and

how the use of evaluation teams could be used to address them. Some of the problems


that Mannatt and Benway (1998) found with single administrator evaluations were a lack

of useful data, favoritism, higher scores for behaviors the evaluator viewed as being more

“like them,” and evaluators that fail to address poor performance. Most teachers and

administrators agree that these sorts of one-shot single-source evaluations are based on

the rituals of district requirements and provide little benefit for the parties involved

(Mannatt & Benway, 1998). Principals are often guilty of meeting the requirements of

the law, but failing to use the information they gain from conducting evaluations in a way

that is beneficial to teachers, instruction, or student achievement (Education Sector, Rush,

2008).

As stated before, meaningful observed data is most often based on a set of agreed

upon standards. It also provides the best picture of effective practice when paired with

student outcomes that measure the gains in achievement that students made while

assigned to a teacher. Yet to be discussed is how the observed or measured data is used.

Traditionally, one would assume one of the primary uses of data would be to establish

whether or not a teacher should continue working in the profession. While this seems

simple, this is often determined by the established contracts that teachers work under,

contracts that are controlled by districts and manipulated by teacher unions. An example

of the policies that can alter the practice of using data for high stakes decisions occurred

in New York. When the New York City public schools tried to change the way tenure

was awarded, the teachers union lobbied the state legislature in protest. The result was a

provision to the school budget that made it illegal to consider a teacher’s job performance

as a factor in the tenure process (Cohen et al., 2008).


Progressive Approaches: Appraisal and Professional Enhancement

Based on various studies and research, some included herein, there exist more

progressive and comprehensive models for teacher evaluation and the use of the data that

it generates. While they are not considered to be perfect, they are intended to provide

examples of methodologies that stray from the traditional models and purposes for

teacher evaluation. These models are being used not only to address the quality of

teachers in the classroom, but to impact other areas of the profession as well.

One of the more comprehensive models is Minnesota’s Quality Compensation for

Teachers program (Q Comp). This model consists of five components: Career

ladder/advancement options, job-embedded professional development, teacher

evaluation, performance pay, and alternative salary schedules. Of interest are the yearly

evaluations of teachers by multiple evaluators, many of whom are classroom teachers that

receive extra compensation to work as lead teachers in their schools. Data from teacher

evaluations is used to financially reward teacher performance. The data used to calculate

performance awards comes from a combination of in-class standards-based observations

of teacher practice and value-added measures from the state achievement test. Evidence

collected from teacher evaluations is also used at the school level to conduct professional

development that addresses identifiable areas of instructional need. (Minnesota

Department of Education [DOE], 2008) Models like this one move forward from

traditional ones and are more closely related to appraisal than evaluation. Kedian (2006)

sees appraisal as a process that centers on the one being appraised. It begins with an

evaluation but extends into conversations, practice, feedback, and exploration. This


ultimately results in the growth of the individual through professional learning and

practice.

Another similar model is the Teacher Advancement Program (TAP). This

program, which is currently practiced in more than 220 U.S. schools, is operated by the

National Institute for Excellence in Teaching (NIET). Like Q Comp, elements included

in this school reform model are multiple career paths for teachers, performance pay,

professional growth, and multiple evaluations. As a school reform effort, their focus is

on improving student achievement by attracting, retaining, developing, and motivating

teachers. NIET claims to be seeing the results of this program. In a comparison of

teacher-effect (value-added) scores between teachers in TAP schools and a control group

of teachers in non-TAP schools, fewer TAP teachers had students with below average

gains, and TAP teachers more often had students scoring better than one year’s growth.

TAP, like Minnesota’s Q Comp model, requires that decisions about professional

development activities be based on the needs of teachers and students. These decisions

rely heavily on the data collected through the application of the TAP instructional rubric

to multiple teacher lessons throughout the year. The desired result of this pairing of data

and professional growth opportunities is increased student achievement (Solomon, White,

Cohen, & Woo, 2007).

While both Q Comp and TAP involve elements of evaluation, they apply it to a

broader reform effort. The teacher’s role in the achievement of students is an important

one. Any reforms aimed at increasing student achievement would be remiss not to

include evaluations as an element in need of attention. The advantage of the evaluation


components in these models over the more traditional approaches is in what data are

generated and how it is used to develop the skills of teachers.

Meaningful professional development is but one benefit for teachers participating

in more progressive evaluation models. Another advantage, quite often, is the chance to

receive extra compensation when data reveals appropriate levels of performance. There

has been a variety of opinions on the matter of merit pay for teachers. Some researchers

feel that a large portion of the country’s school districts that are operating on a single

salary schedule are stuck with a system that works against improving the teacher

workforce (Hanushek, 2007). Others feel that there is still no reliable way to measure the

true quality of a teacher’s work. (Education Sector, Rush, 2008) However, opinions of

educators have remained fairly consistent over the past few years. In a 2007 survey of

teachers on a variety of issues in education, 58% agreed with providing incentive pay for

teachers who consistently score high on their evaluations (Education Sector, Waiting,

2008) The Basic Education Plan (BEP) for the state of Tennesee was reconfigured in

2007 to include increases in the number of evaluations that are required of teachers, as

well as a requirement that local education agencies develop a differentiated pay scale to

attend to the issues of teacher quality, especially in underperforming or hard to staff

schools. Accountability for schools has resulted in changes to the way teachers are

measured, trained, and even compensated.

Conlusion

Most researchers would agree that the teacher plays the most significant role in

the achievement of students. While the relationship between teaching and learning is a

tenuous one, the effective evaluation of teachers is necessary to any comprehensive


school reform effort. However, the traditional models of evaluation have historically

failed to reliably provide any concrete evidence about which teachers are effective and

which ones are not. Principals and teachers agree, quite often viewing evaluation as a

formality and a meaningless part of the teaching profession.

Data about the effectiveness of teachers comes from a variety of sources. High

stakes decisions about teachers must be made with caution, as the picture painted by data

can be misleading. Compiling and comparing data collected over longer periods of time

and from numerous sources has been found to be most effective in terms of identifying

the quality of a teacher.

Currently and in the future, more policy decisions will be based on the types of

data that evaluators are collecting. These policies will most certainly play a role in who

receives tenure and to whom it is denied. The ways in which teachers are compensated

will most likely be affected as well. Maybe most importantly, professional development

will be made more meaningful by data based decision making. This ensures that the

training teachers receive meets the needs observed in the classroom.

As school districts across the country will undoubtedly use evaluation of teachers

to change the way that teachers are retained, fired, paid, and promoted in the next decade,

there are ample opportunities for continued research. Possible topics of interest might

include a study of how evaluations coupled with feedback and professional development

impact teacher attitudes, practice, or value-added scores. Mathematically correlating the

scores received on a traditional evaluation (process-product oriented, drive-by, related to

teacher behavior) with a more progressive standards-based model and value-added data

could also provide information on the effectiveness of each model in predicting success


in terms of standardized test data. Over multiple trials with multiple models,

characteristics of the more effective predictors might be identified, and links between

effective practice and value-added could be more strongly related. However, as

evaluation models are examined and refined, the most important implication could be in

the everyday professional practice of America’s schools.

As an outcome of the reviewed literature, two guiding questions for a qualitative

study may be posed. Research proposes that traditionally principals do an ineffective job

of identifying teachers’ respective levels of effectiveness. This lack of accuracy has been

attributed to many different factors. Therefore, the first question is “How do principals

and assistant principals feel about measuring the effectiveness of their staff?” Based on

the literature it is also of importance to gain an understanding for what administrators see

as being significant elements of the evaluation process. The second guiding question is,

“What do principals and assistant principals feel is important in regards to the evaluation

of teachers?” These questions could lead to an understanding for how appealing the task

is to administrators or to their perceived abilities to effectively assess their teachers. It

could also draw on their expertise to address issues about how often teachers should be

evaluated, who should be evaluated, and changes that need to be made to address

deficiencies in the evaluation models they use.


Chapter Three: Method

Choice of Method

Data was collected using a survey that was offered online through a web-service,

Surveymonkey.com. Because the data of interest in this project consisted of the feelings

and perceptions of the participants, using a survey was an appropriate assessment tool

and it allowed the respondents to remain anonymous. By using the website to offer the

survey, I was able to reduce the time involved in distributing and collecting paper copies.

The website provided each participant with an e-mail invitation to complete the survey by

following an imbedded hyperlink which connected them to a secure version of the

instrument.

Development of the Survey

I constructed this survey in three basic parts that were meant to serve different

purposes. The first part of the survey included five questions designed to describe the

participant that was completing the survey. The answers to these initial questions on the

survey allowed respondents to describe their current state of employment as either a

principal or an assistant principal, the grade levels served by the school in which they

work, the number of certified teachers on their staff, the number of years they had

worked as an administrator, and their total number of years experience working in

education. The intent of including these questions was to later examine any relationships

that might exist between these types of descriptors and the attitudes or perceptions of the

respondent.

The second part of the survey consisted of 22 Likert scale items. These

statements each addressed different aspects of teacher evaluation process such as


principal attitude, confidence, and methods. Participating administrators were able to

rank each statement according to how strongly they agreed or disagreed. These questions

were preceded by several explanatory statements so that those taking the survey would

clearly understand how to respond to the items.

First, the rating scale was explicitly labeled so that each numerical score had a

defined meaning. The response rankings were explained as follows:

5 – Strongly Agree

4 – Agree

3 – Neutral or Undecided

2 – Disagree

1 – Strongly Disagree

Next, operational definitions were established for respondents to explain the language of

the items in the survey. Several items were statements about the “quality” or

“effectiveness” of a teacher. These two terms were defined in terms of student

achievement. According to these definitions, if a teacher was more effective or of a

higher quality, their students’ achievement scores would be higher. Two other terms that

required defining were “evaluation” and “observation.” Both of these terms were meant

to describe time that was spent in a classroom collecting information about a teacher’s

practice.

The third and final part of the survey was made up of three constructed response

questions. These three questions allowed respondents to speak to their own personal

issues with evaluating their teachers, discuss resources that might improve the evaluation

model or process, and clarify any other responses that they had provided on the Likert


scale portion of the survey. My original intention was to use the Likert scale items only.

I was then going to randomly select some participants to participate in a small focus

group of five to ten people. I decided that these last few questions could serve a similar

purpose without the time involved for myself and the participants. It also increased the

amount of input on the part of whole group and not just the select few. The complete 30-

item survey can be found in Appendix A.

Participants and Data Collection

The school-level administrators (assistant principals and principals) of the KCS

district were chosen to participate in this study. While issues of proximity and

availability influenced my decision to use this group, the context of my interest in this

topic also began and has developed as a member of the KCS faculty. On March 22,

2009, e-mail invitations were sent to 179 Knox County administrators asking for their

participation in the project. They were advised that the survey would be available until

4:00 p.m. March 27, 2009. When the survey was closed, 93 complete responses had been

recorded. There were six partially completed responses which were left out of the study

due to their incomplete status. Of the participants who submitted complete surveys, 46

were principals and 47 were assistant principals. The group was comprised of 54

administrators working at the elementary school level, 20 from the middle schools, and

19 from the high schools. The data describing the participant group can be found in

Appendix B.

Data Analysis

Analysis began with identifying the make-up of the participant group. Of interest

was the number of principals that responded versus assistant principals. Also, there was a


comparison of the grade levels at which these individuals worked. I calculated the range

and average number of years the participants had worked as administrators and as

professional educators.

The items from the Likert scale portion of the survey were organized by themes.

Some of the questions spoke about the administrator’s confidence in his or her own

ability or in the evaluation tools he or she had been provided. Another set of items dealt

with the positive or negative feelings the principal had about evaluating teachers. I

considered these types of questions to relate to the principal’s attitude about evaluation. I

organized the questions thematically into groups like this so that I could look for areas

where the mean scores were higher or lower. This might indicate a particular pattern in

terms of the administrators’ perceptions of themselves or in regards to the evaluation

protocol. I could also examine situations where one or more questions failed to match up

with the other questions that it was associated with. This might have indicated a

misunderstanding of the question on the part of the participants, or a discrepancy in the

expected response of the group.

As the scale items were grouped, it became necessary to alter the way in which

the average scores for responses were tabulated. Within the categories, there were

instances where a high score on one item would show agreement with a low score on a

different item. In this case, the averages were reconfigured to apply them to the same end

of the scale. This was done by reversing the scoring scale so that any response indicating

of strongly disagree became a score of five numerically, disagree became a four, neutral

stayed the same, agree became a two, and strongly agree became a score of one. This

made the reporting of data easier to understand and the comparative nature of the graphs


more significant. These items are identified by an asterisk in the chapter four results

section. Reconfigured scores were indicated in red in the data tables.

Next I examined the responses for each of the final three constructed-response

questions. A complete list of responses for each of the constructed-response questions

along with my review notes may be found in Appendix C. I looked for similarities in the

responses within the context of each individual question to identify some commonalities

in the thinking of the respondents. I also looked across all three questions to establish

some themes to help explain the data collected from the Likert scale portion of the

survey. Those same themes could be used to make suggestions about current perceived

issues with the evaluation model and about possible improvements that could be made to

the process.


Chapter Four: Results

A complete summary report for the administrator survey scale items is located in

Appendix D.

The Attitude of the Principal

The first items related to the attitude of the principal were indicators of how

positively or negatively the administrators felt about the task of evaluation and their own

experiences being evaluated. The attitudes of the participants were measured in terms of

how strongly they agreed with the following items from the survey:

1. Evaluating teachers is an enjoyable part of my job.

9. I enjoy coaching teachers about ways to refine their practice.

19. My experiences being evaluated as a classroom teacher were mostly positive.

Also considered in regards to the attitude of the participant was the level of importance

that they placed on evaluating teachers in terms of student achievement, their own

responsibilities as an administrator, and overall school improvement. Items from the

survey related to the importance of evaluating teachers were as follows:

15. Evaluating teachers is an important part of being an instructional leader.

16. Teacher evaluation can be an important element of school improvement.

17. Teacher effectiveness is the most important influence on student

achievement.

All six questions relating to attitude are written so that a higher score reflects a more

positive view of the evaluation process. Table 1 and Figure 1 summarize the responses

for the attitude portion of the rating scale.


Table 1 - Attitude Item Response Data

Item Average Score

Most Common Score Received (number of

times received)

# 1 3.9 4 (44)

# 9 4.32 5 (44)

# 19 4.23 5 (43)

# 15 4.55 5 (63)

# 16 4.47 5 (54)

# 17 4.3 5 (51)

Figure 1 - Attitude Item Responses

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

# 1 # 9 # 19 # 15 # 16 # 17

Item

Ave

rag

e S

co

re

The Confidence of the Principal

Several of the items included in the survey were designed to measure the

principal’s confidence based on how strongly he or she agreed or disagreed with the

statement. A few items related to whether or not the participant felt the teachers in their

building were being evaluated thoroughly. Other items were connected to the


administrator’s comfort level when assessing instruction in content areas that were less

familiar to them, and their abilities to identify examples of strong and weak instructional

practices. The items relating to the principal’s confidence are listed below:

2. I know high quality teaching when I see it.

3. I could easily identify the teachers in my building who are most effective.

4. I observe the teaching in my building often enough to have an accurate picture

of each teacher’s strengths.


of the types of improvements each teacher might consider making.

14. It is difficult to evaluate teachers in academic areas that are less

familiar to me.

22. With a fair degree of accuracy, I could rank the teachers on my staff from

least effective to most effective.

With the exception of item 14, again the items are arranged so that a higher score would

indicate a greater degree of confidence. Responses to item 14 would be considered to

exhibit less confidence as the score increased. The response data from these items are

summarized in Table 2 and Figure 2. Item 14 is identified with an asterisk because the

average score had to be reconfigured to align with the other responses for comparison.

All reconfigured scores are listed in red and are found in brackets below the actual

average score.


Table 2 - Confidence Item Response Data

Item Average Score

Most Common Score Received (number of times

received)

# 2 4.57 5 (61)

# 3 4.66 5 (70)

# 4 4.2 4 (45)

# 7 4.09 4 (46)

# 14* 2.41 [3.59]

2 (49) [4(49)]

# 22 4.18 4 (49)

Figure 2 - Confidence Item Responses

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

# 2 # 3 # 4 # 7 # 14* # 22

Item

Ave

rag

e S

co

re

The Principal and Methods of Evaluation

Four scale items completed by the participants centered on the methods used by

the administrator when evaluating their staff. Some of these items labeled specific

practices such as providing feedback, while others provided suggestions about the

respondents’ feelings toward the current evaluation tool. One scale item related to how


much the administrator depended upon their own classroom experience to assist them in

accurately evaluating teachers. Below is the list of items relating to the methods used by

principals:

5. I use my experience as a classroom teacher to help me evaluate a teacher’s

practice.

6. The evaluation model supplied by my school system allows me to accurately

evaluate my teachers’ effectiveness.

8. When I observe a teacher, I give them specific feedback about the strengths

and weaknesses of their lesson.

18. I use evaluation methods or tools other than those prescribed by my district to

evaluate the effectiveness of my staff.

The data from the Methods portion of the survey are represented below in Table 3 and

Figure 3.

Table 3 - Method Item Response Data

Item Average Score


received)

# 5 4.28 5 (44)

# 6 3.48 4 (38)

# 8 4.49 5 (59)

# 18 3.89 4 (44)


Figure 3 - Method Item Responses

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

# 5 # 6 # 8 # 18

Item

Avera

ge S

co

re

The Principal and Frequency of Evaluation

The scale items relating to the frequency of evaluations were intended to measure

the opinion of the principal as to which teachers should be evaluated and how frequently

these evaluations should occur. These statements also allowed for the collection of data

relating to how much time the principal is spending observing and evaluating teachers

when compared to the minimum requirement. Below are the statements relating to the

frequency of evaluation:

10. I spend more time observing instruction than is required by my district.

13. I observe / evaluate only those teachers who are scheduled for school-district

evaluations during that school year.

20. Tenured teachers should be evaluated more often than is currently required.

21. Non-tenured teachers are the only teachers that should be evaluated yearly.


Items 13 and 21 had to be reconfigured for comparison with the other two items. These

two items were adjusted so that for each item a higher score reflected a higher frequency

of evaluation. Items 13 and 21 are identified with an asterisk because of the changes

made to the scoring. All reconfigured scores are listed in red and are found in brackets

below the actual average score. Table 4 and Figure 4 summarize the responses collected

for the items relating to frequency.

Table 4 - Frequency Item Response Data

Item Average Score


received)

# 10 3.99 4 (38)

# 13* 1.84

[4.16] 2 (42) [4 (42)]

# 20 3.56 4 (31)

# 21* 2.09 [3.91]

2 (37) [4 (37)]

Figure 4 - Frequency Item Responses

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

# 10 # 13* # 20 # 21*

Item

Ave

rag

e S

co

re


The Principal and Data

Two items that were part of the survey related specifically to the principal’s

identification of what constituted meaningful data in terms of teacher evaluation. Both

were written in a way that a higher scoring response indicated a greater belief in the

accuracy of that specific type of data. The items were as follows:

11. Formal evaluations of teachers are the most accurate way to measure teacher

effectiveness in the classroom.

12. The analysis of students’ standardized test scores (TCAP assessments, EOCs,

Gateway exams) is the most accurate way to measure teacher effectiveness in the

classroom.

Table 5 and Figure 5 summarize the responses collected for these two items.

Table 5 - Data Item Response Data

Item Average Score


received)

# 11 2.57 2 (37)

# 12 2.88 2 (29)


Figure 5 - Data Item Responses

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

# 11 # 12

Item

Ave

rag

e S

co

re

Open Responses

The open- response items made up the last three questions on the survey and allowed the

participants to share details about their feelings, perceptions, and answers to previous

scale items. A complete set of responses for each question can be found in Appendix C.

Item 23 was the first of the open-response questions and can be read below:

What concerns do you personally have in regards to evaluating and or measuring

the effectiveness of your staff?

Of the 93 respondents, 23 (24.7%), expressed no personal concerns about the evaluation

process in their school. More than one-quarter of the respondents cited time as an issue

affecting their ability to consistently evaluate their staff. According to the principals

there were several reasons that time was factor in evaluation. The amount and variety of

duties required of the principals was identified as a problem affecting the time they had


available to be in classrooms. Size of the staff and dealing with student discipline were

identified as time–related issues. Principals also found it hard to regularly schedule

themselves to conference or be available for teachers because of their responsibilities.

Excessive paperwork with the current evaluation model was identified as a time related

factor as well. One participant wrote that the paperwork “took away from the process.”

Many of the responses dealt with the accuracy or fairness of the evaluation

process. These responses related to the principals concern with being fair when

observing a teacher. There was concern over collecting the right amount of data, the

right type of data, and whether or not the present evaluation tool and the observation

itself could effectively measure all aspects of the “high-quality teacher.” Below are some

sample responses related to the accuracy or fairness of evaluation:

(I am concerned with) “being fair to all staff with an unbiased approach and

making sure I have enough information to accurately evaluate them.”

“Much of the art of teaching and relating to students is difficult to quantify in an

observation.”

“Often times the evaluation process does not accurately measure concerns such as

professional conduct.”

“I know good instruction when I see it. The domains and indicators don’t always

allow me to relay that information accurately.”

“If a very strong effort to point out needed improvements is not made by the

observer, the evaluation process we use is not very effective. Two or three

scheduled visits are not a true picture of what is really happening in the

classroom.”


There was a definite concern on the part of the principals that the types of

teaching they observed during an announced observation were not authentic

measurements of effectiveness. One administrator stated, “observations that are

announced are not true measures of regular teaching practice…unannounced observations

should be the norm.” Several others cited the need for more unannounced drop-in

evaluations to see “authentic teacher/student performance.”

Other less-frequent concerns expressed by the respondents included playing the

role of coach and administrator simultaneously. They also expressed concerns in terms

of teachers who were struggling and the type of support that was available. The opinion

existed that ineffective tenured teachers remained that way because of resistance to

offered assistance, a lack of options for the principal in terms of targeting these

individuals for intervention, and the burden of going through the process to have them

removed from the classroom.

Item 24, the second open-response question, asked the respondents to consider the

needs of the school system and their schools in order to improve the quality of teacher

evaluation. Item 24 is listed below:

In your opinion, what resources are needed in your district to support

administrators in accurately assessing the quality of their staff?

Approximately 20% of the administrators felt that the county’s resources were sufficient.

Time was again identified as an issue affecting the evaluation of teachers. For this reason

there were several responses that expressed a desire for extra staff. Some administrators

specifically identified the need for additional assistant principals, instructional specialists,

lead teachers to assist with evaluations, and the use of evaluation teams. There were


several responses that identified additional support from the central office and content

supervisors in assisting with evaluations and struggling teachers. A commitment at the

school level was identified by one administrator as a needed resource. This participant

stated “a commitment by all administrative teams to be in the classrooms everyday”

would increase the quality of teacher assessment.

Administrators commonly identified the need for training or professional

development. One respondent wished to have more training on how to assist the

struggling teacher, while another wanted additional training on how to effectively coach

teachers. The need for assistance on how to effectively give negative feedback was also

identified. Another participant said:

“Administrators can never get enough training in this area. If we are going to put

a lot of emphasis on evaluations, we need to know what we are looking for in a

setting.”

Several responses suggested that the system should reevaluate the use of the

current model or make modifications to the evaluation tool in order to improve its

effectiveness. One administrator stated, “The evaluation tool is too straight-line; a

teacher could be a bad teacher and still score appropriately.” Another felt that

improvements could be made to the informal and formal documents used by

administrators so that feedback on instruction would be more specific. One principal

suggested making other evaluation instruments available. Two other participants

recommended adopting a model like the one used in the county’s Teacher Advancement

Program (TAP) schools.


The final item on the survey allowed the participants to express their ideas about

the principal’s role regarding teacher effectiveness and evaluation. Item 25 is listed

below:

Please add any additional comments you may have in regards to the

administrator’s role in teacher effectiveness, evaluation, or assessing teacher

quality. You may also consider one item from the scale above upon which you

would like to expand your answer.

Many of the previously mentioned themes emerged from the responses to this item such

as a need for more time, reduced responsibilities, and needed modifications to the current

evaluation tool. However, some administrators made direct and indirect statements about

the identifiable beliefs of an effective leader and the impacts a school leader can and

should have on their teachers. A sample of these responses is found below:

“Administrators must make teacher effectiveness a priority in the school in order

to impact student achievement.”

“An administrator can help teachers be effective in the classroom in a lot of ways

other than the evaluation instrument-listening, collaboration, etc.”

“Administrators are responsible to assess the overall effectiveness of the staff and

make the decisions that are best for each student and not what is convenient for

the staff.”

“The administrator needs to be out in the school and in the classrooms.”

“I think that it is important for teachers and administrators to view the formal and

informal observation process as a means of growth.”


“Building relationships with my teachers as well as trust is critical to my

effectiveness as an instructional leader.”

“The administrator’s response to teacher observations is so important. They are

the person who can make a difference in the teacher by pointing out their areas of

strength and areas to strengthen. However, they must be willing to provide

support for those areas of weakness.”

“The administrator’s role is key in setting the overall expectations and tone for a

building. If expectations are high, the majority of teachers will aspire to those

expectations.”

“We must be in the classrooms everyday.”

“Teachers and administrators must understand that the process of evaluation is a

collaborative effort, and the goal is to improve student learning.”

“The administrator must understand that evaluation is a never-ending process, and

we can always improve.”

“Administrators should be in teachers’ classes daily.”

“Administrators need to make it a commitment to visit all teachers multiple times

during a school year to help assist and encourage teaching improvement.”

“As the instructional leader for the school, a principal must inspect what he/she

expects.”

“The administrator has a vital role in not only assessing teacher effectiveness, but

improving teacher quality as well.”


“It is important to evaluate those untenured teachers and to be honest. It is critical

to help those not really cut out for this job to find that out early on and help them

find another avenue to pursue.”

Another new theme that came from this question was the issue of experience.

One respondent stated that moving to a new school this year had left him with limited

knowledge concerning the effectiveness of the teachers on the staff. This had affected his

answer to item 22 that related to ranking one’s teachers by their effectiveness. Another

participant felt that administrators who have a lot of classroom experience can help

teachers better in a coaching role. In the mind of this participant having less than ten

years experience in the classroom left you with a “handicap” in terms of helping teachers

grow professionally.


Chapter Five: Discussion

The Administrator’s Feelings About Evaluation

The data collected through the course of this project can be used to describe the

general feelings of Knox County administrators in regards to the evaluation of teachers.

First of all, most principals and assistant principals had a positive attitude about the

process of teacher evaluation. They had positive experiences as teachers when they were

evaluated in the past and enjoyed conducting evaluations and coaching teachers.

Principals also felt that this was an important task. They felt that evaluating teachers was

an important part of an instructional leader’s job. Administrators saw evaluation and

improving teacher effectiveness as factors affecting overall school improvement and

student achievement. The overall positive attitude held by the participants was also

supported by the responses to the final three questions. Principals reinforced the

importance of effectively evaluating teachers and made suggestions about the

commitments that administrator’s needed to make to improve the efficiency of this

process.

Items that measured the confidence of the principal resulted in similar positive

outcomes. Principals felt confident that they knew what high quality teaching looked like

and that they could identify the teachers in their building that were most effective.

Administrators also felt that they had an accurate picture of the types of strengths and

weaknesses possessed by teachers on their staff. They were fairly sure that they could

accurately rank their staff members from least effective to most effective based on their

observations. While still a positive response, the one item that principals seemed less

confident about dealt with evaluating teachers of content outside their area of expertise.


Thirty-four percent of the administrators responded with either a neutral or negative

response. This item could point to the participants’ expressed desire for additional

training and professional development relating to evaluation and coaching. This reduced

confidence could also relate to those respondents who felt that the evaluation tool was not

specific enough or did not allow them to adequately convey all that they observed.

Others expressed concerns about their familiarity with unfamiliar curricula and

accounting for student learning in the evaluation model.

Evaluation in Practice

Certain portions of the data collected pointed to the habits of the principals as they

conducted evaluations. This data related to the methods they currently used and ideas

about the frequency of evaluation. In terms of methods, nearly all the participants

provide teachers with specific feedback about areas of strength and weakness from the

lessons that they observe. This is in agreement with their confidence in being able to

identify these high and low areas in a lesson. Most participants use methods other than

the adopted system model and their own experience as a classroom teacher to evaluate

the effectiveness of their teachers. Principals only weakly agreed that the current

evaluation model allowed them to accurately evaluate their teachers with 43% providing

a neutral or negative response. Again this is supported by the responses to the final three

open-response survey items. Principals pointed out the need for additional or more

specific indicators that allow them to measure a broader range of teacher and student

behaviors within the current evaluation model. Several respondents called for an increase

in the variety of available evaluation tools or a switch to a new model altogether.


Items related to the frequency of evaluations highlighted both the beliefs and

habits of the participants. In terms of their beliefs, administrators felt that the number of

evaluations for teachers should be greater. They indicated a belief that yearly evaluations

should not be reserved for non-tenured staff and a weaker feeling that tenured teachers

should be evaluated more often. Based on the open-response data, several principals

expressed concerns about specific tenured teachers that were resistant to coaching,

needed targeted professional development, or were ineffective but too difficult to remove

due to system policy and procedure. When comparing the responses from the scale items

to the open response, it may be suggested that principals feel that there are some tenured

teachers (based on effectiveness and growth) that need to be evaluated more often than is

required. However, there may not have been an overwhelming feeling that all tenured

teachers need to be evaluated more often.

Principals in Knox County agree that they spend more time observing instruction

than is required of them. They also regularly visit the classrooms of teachers to observe

or evaluate whether they are participating in the current state evaluation cycle or not.

Committing to do more than the minimum requirement is certainly a challenge for these

administrators due to several factors identified from the open-response questions. Time

was identified as the primary needed resource and concern for administrators. They also

cited the need for extra support from the school system central office and additional staff

to alleviate the duties of the principal so that more instructionally focused activities could

take precedent over issues of management.


Data-Based Decisions

While administrators expressed positive feelings about the evaluation process,

confidence in their abilities to effectively assess their faculty, and a commitment to go

beyond the minimum requirements, their feelings about what type of data was most

meaningful were not as strong. Participants failed to identify formal evaluations or

standardized test scores as the most accurate way to measure teacher effectiveness. Both

items relating to the accuracy of data resulted in average scores below three, which

indicated a slightly negative response.

This can be viewed as contrasting with the highly positive responses regarding

other areas of evaluation such as attitude or confidence. For example, one would expect

that an evaluator who can rank their teachers in terms of effectiveness, or give them

specific feedback about their teaching would be able to strongly correlate these

judgments back to observed and measured data. These scale items suggest that the link to

data when determining teacher effectiveness is weak. The question that results from this

suggestion would then be: What types of data are administrators using to determine

which teachers are effective or ineffective? Several open responses stated the principals

wish to measure performances of duties and skills outside of the classroom within the

evaluation model. Some respondents felt that the types of relationships that a teacher

formed with students were important to their teaching and could not be measured by an

evaluation. While these attributes speak to the professionalism and quality of the teacher,

they may fail to capture the instructional capacity of the teacher. Especially in the case of

new teachers, these identified concerns could indicate misplaced focus on the part of


some administrators whose task should be increasing their teachers’ instructional

effectiveness.

There are also some responses that may suggest an explanation for a lack of faith

in the data collected in a formal evaluation or from standardized tests. Throughout the

open responses, there was an expressed need for more unannounced or drop-in

evaluations. Administrators expressed concern that what they were able to see in an

announced formal evaluation could be uncharacteristic of the type of teaching that takes

place everyday in that same classroom. This may explain why participants did not

identify this type of data as the most accurate way to define effectiveness. In relation to

standardized test data, one respondent saw the use of this type of data as an indicator of

effectiveness as a “bad idea” because test scores did not exist for every teacher and

provided only one aspect of measurement. Other participants indicated that outside

factors from the student’s home life impact their education and may interfere with the

work of the teacher. This could produce test results for some students that are

misrepresentative of the type of teaching that takes place on a daily basis. The identified

complexity of the relationship between teacher practice and student learning relates to the

previously discussed ideas of Fenstermacher and Richardson (2005) and the differences

between good teaching and successful teaching.

Implications for Action

The data collected during this project provides a general description of the

building-level administrators from a single school district. The results of this data could

be used to affect the evaluation process in Knox County in a variety of ways.

Discussions about how to better divide the duties of the administrative team could help


address issues of time and the quality of feedback and coaching cycles from school to

school. Also looking at school staffing or scheduling could help principals identify ways

to use experienced teachers to assist with evaluations and coaching by providing them

with scheduled time out of the classroom. Participants expressed the need for continuous

professional development for teachers as well as administrators. The school system

could use the feedback from administrators to guide them in making professional

development decisions. Training could be used to support areas of need in terms of

effectively evaluating staff and in the types of data to use for making building-level

instructional decisions. Ultimately, the data from this project provides a starting point for

school system officials to examine a wide range of administrative needs from staffing and

professional development to the use of data and modifications to the current evaluation

protocol.

Suggestions for Further Research

Through this process, I have learned that the administrators of Knox County

maintain positive philosophies relating to the evaluation of teachers. Their candid

responses reveal the thoughts of a reflective group of professionals who are willing to

analyze their own practice in order to improve the outcome of this process and

thoughtfully make suggestions that could positively impact their teachers and students.

The quick and large response that I received from them indicated that this topic was valid

and of current importance to them as administrators.

The questions that guided this study could have been applied to a broader group

of administrators from other counties and districts. Many of the Knox County

administrators’ concerns were related to the evaluation protocol which is the adopted


Tennessee State model. Similar concerns might have arisen among principals from other

parts of the state. A larger number of participants from a variety of districts could have

expanded the implications of this study to a larger context.

Data for this study was collected over a five day period. Extending the window of

opportunity for participants may have allowed more participation. The short window for

completing the online survey was further complicated by the technological difficulties

experienced by some who attempted to participate. The technological problems alone

resulted in the loss of more than 10 potential participants.

Survey construction may have posed a problem for some of the participants.

There was one participant in particular that appeared to answer each question almost

exactly the opposite of the group. While this is possible, it is also possible that he or she

misread the scale and therefore completed the survey incorrectly. Clearer directions

could have made a difference for this participant. Changes to the content of the survey

items could have further clarified some of the more ambiguous responses. In hindsight,

more information on the types and uses of data could have provided beneficial

information considering the context of this study.

If research were continued with this same group, a goal would be to better

understand the types of data principals use to make decisions about the quality of

teachers. It would be interesting to see what types of factors lead to decisions of

retention and firing among non-tenured staff. These factors could be studied to see if a

set of characteristics that principals identified as high-quality or low-quality existed.

These characteristics could then be examined to see how they impacted instruction and

related to the actual classroom practices of the teacher. This continued research would


stem from the perception that principals look at many other factors besides effective

practice when making decisions about the quality of staff.

If this research were extended to a larger group, it could be continued in the same

manner that the present study was conducted. The researcher might consider the addition

of a scale item that measures whether or not the participant agrees that a combination of

evaluation data and standardized test data provides an accurate measure of teacher

effectiveness. The addition of an open-response item that allowed administrators to

explain what type of data they use to make decisions about teacher effectiveness could

also be added to help clarify information relating to this element of evaluation.


References

Cohen, E., Walsh, K., & Biddle, R. (2008). Invisible ink in collective bargaining: Why

key issues are not addressed. Washington, D.C.: National Council on Teacher

Quality. Retreived on November 2, 2008 from http:// www.nctq.org/cb/

Danielson, C. (1996). Enhancing professional practice: A framework for teaching.

Alexandria, VA: Association for Supervision and Curriculum Development.

Education Sector. (2008). Rush to judgment: Teacher evaluation in public education.

Washington, D.C.: Toch & Rothman. (Eric Document Reproduction Service No.

ED502120). Retrieved September 29, 2008 from EBSCOHost ERIC database.

Education Sector. (2008). Waiting to be won over: Teachers speak on the profession,

unions, and reform. Washington, D.C.: Duffett, Farkas, Rotherham, & Silva. (Eric

Document Reproduction Service No. ED502154). Retrieved September 29, 2008

from EBSCOHost ERIC database.

Fenstermacher, G., & Richardson, V. (2005). On making determinations of quality in

teaching. Teachers College Record, 107(1), 186-213.

Hanushek, E. (2007). The single salary schedule and other Issues of teacher pay. The

Peabody Journal of Education, 82(4), 574-586. Retrieved September 23, 2008

from Education Research Complete database.

Haycock, K., & Crawford, C. (2008). Closing the teacher quality gap. Educational

Leadership, 65(7), 14-19. Retrieved September 23, 2008 from Education

Research Complete database.


Ingvarson, L., & Rowe, K. (2008). Conceptualising and evaluating teacher quality:

substantive and methodological issues. Australian Journal of Education, 52, 5-35.

Retrieved September 23, 2008 from Education Research Complete database.

Johnson, C., Kahle, J., & Fargo, J. (2007). Effective teaching results in increased science

achievement for all students. Science Education. 91(3), 371-383. Retrieved

September 30, 2008, from Education Research Complete database.

Kedian, J. (2006). Appraisal and Evaluation: Professional Learning or Box-Ticking.

Education Today, Retrieved September 30, 2008, from Education Research

Complete database.

Mannatt, R. & Benway, M. (1998, Spring). Teacher and administrator performance

evaluation: Benefits of 360-degree feedback. ERS Spectrum. (no volume or issue)

18-23. Retrieved on November 3, 2008 from http://www.rise.hs.iastate.edu/

uploads/reports/360%20manatt&benway.pdf.

McCaffrey, D., Lockwood, J.R., Koretz, D., & Hamilton, L. (2003). Evaluating value-

added models for teacher accountability. Santa Monica, CA: RAND Corporation.

Medley, D. & Coker, H. (1987). The accuracy of principals’ judgments of teacher

performance. Journal of Educational Research. 80(4), 242-247.

Minnesota Department of Education. (2008). Quality compensation for teachers (Q

Comp) – Program Components. Retrieved November 14, 2008 from

http://www.education.state.mn.us/MDE/Teacher_Support/QComp/index.html

No Child Left Behind Act of 2001, Pub. L. No. 170-110, 115 Stat.1425 (2002).


Sanders, W., & Horn, S., (1998). Research findings from the Tennessee value-added

assessment system (TVAAS) database: Implications for educational evaluation

and research [Abstract]. Journal of Personnel Evaluation in Education, 12(3),

247-256. Retrieved November 2, 2008 from

http://www.springerlink.com/content/x277216564462554/

Solomon, L., White, J., Cohen, D., & Woo, D. (2007). The effectiveness of the teacher

advancement program [Executive summary]. Santa Monica, CA: National

Institute for Excellence in Teaching. Retrieved November, 14 2008 from

http://www.talentedteachers.org/tap.taf?page=results

The New Teacher Project (2007). Hiring, assignment, and transfer in Chicago public

schools. New York: The New Teacher Project. Retrieved on November 15, 2008

from http://www.tntp.org/files/TNTPAnalysis-Chicago.pdf

Appendix A

Administrator Survey: Measuring Teacher Effectiveness

Principals’ Attitudes 51

Appendix A

Measuring Teacher Effectiveness

Please complete the profile below to the best of your ability. This information will be used only to describe

the general profile of survey respondents and will not be used to identify individual respondents. All

answers are confidential.

A. Current position: (Assistant Principal / Principal)

B. Type of school you currently work in: (Elementary / Middle / High)

C. Number of certified teachers currently employed in your school: _____________

D. Number of years working in an administrative capacity: ____________________

E. Total number of years working in the field of education: ____________________

Please rate how strongly you agree or disagree with each of the following statements by circling the

appropriate number. Responses are ranked as follows:

1 – Strongly Disagree / 2 – Disagree / 3 – Neutral or Undecided / 4 – Agree / 5 – Strongly Agree

*Consider any statement relating to the “quality” or “effectiveness” of a teacher to be in terms of student

achievement. For example, the more effective a teacher is or the higher the quality of the teaching is then

the higher the achievement levels or scores will be for students.

*“Evaluating” and “observing” teachers consists of any time an administrator spends in a teacher’s

classroom collecting information about their practice whether part of an evaluation model or an informal

drop-in.


1 2 3 4 5


1 2 3 4 5


1 2 3 4 5

4. I observe the teaching in my building often enough to have an accurate picture of each teacher’s

strengths.

1 2 3 4 5

5. I use my experience as a classroom teacher to help me evaluate a teacher’s practice.

1 2 3 4 5

6. The evaluation model supplied by my school system allows me to accurately evaluate my teachers’

effectiveness.


1 2 3 4 5

7. I observe the teaching in my building often enough to have an accurate picture of the types of

improvements each teacher might consider making.

1 2 3 4 5

8. When I observe a teacher, I give them specific feedback about the strengths and weaknesses of their

lesson.

1 2 3 4 5


1 2 3 4 5

10. I spend more time observing instruction than is required by my district.

1 2 3 4 5

11. Formal evaluations of teachers are the most accurate way to measure teacher effectiveness in the

classroom.

1 2 3 4 5

12. The analysis of students’ standardized test scores (TCAP assessments, EOCs, Gateway exams) is the

most accurate way to measure teacher effectiveness in the classroom.

1 2 3 4 5

13. I observe/evaluate only those teachers who are scheduled for school-district evaluations during the

current school year.

1 2 3 4 5

14. It is difficult to evaluate teachers in academic areas that are less familiar to me.

1 2 3 4 5


1 2 3 4 5


1 2 3 4 5

17. Teacher effectiveness is the most important influence on student achievement.

1 2 3 4 5

18. I use evaluation methods or tools other than those prescribed by my district in order to evaluate the

effectiveness of my staff.


1 2 3 4 5


1 2 3 4 5


1 2 3 4 5

21. Non-tenured teachers are the only teachers that should be evaluated yearly.

1 2 3 4 5

22. With a fair degree of accuracy, I could rank the teachers on my staff from least effective to most

effective.

1 2 3 4 5

23. What concerns do you personally have in regards to evaluating and or measuring the effectiveness of

your staff?

24. In your opinion, what resources are needed in your district to support administrators in accurately

assessing the quality of their staff?

25. Please add any additional comments you may have in regards to the administrator’s role in teacher

effectiveness, evaluation, or assessing teacher quality. You may also consider choosing one item from the

scale above upon which you would like to expand your answer.

Appendix B

Survey Participant Descriptive Data


Appendix B

The tables below represent the survey questions and respective responses that describe

the participant group for the study.

What is your current position?

Answer Options

Response

Frequency

Response

Count

principal 49.5% 46

assistant principal 50.5% 47

Which type of school do you currently work in?

Answer Options

Response

Frequency

Response

Count

Elementary 58.1% 54

Middle 21.5% 20

High 20.4% 19

Approximately how many certified teachers are currently employed in

your school?

Response Data

Average Response 55

Most Common 60

Range (Min - Max) 16-128

How many years have you been working in an administrative capacity

(as a principal or assistant principal)?

Response Data

Average Response 7.2

Most Common 6

Range (Min - Max) 1-25

How many total years have you been working in the field of education?

Response Data

Average Response 2.3

Most Common 23

Range (Min - Max) 8 - 41

Appendix C

Constructed-Response Questions, Results, and Research Notes


Appendix C

23. What concerns do you personally have in regards to evaluating and or measuring the

effectiveness of your staff?

*Research notes are listed in red and in capital letters.

Having enough time to be in the classroom on a regular basis is difficult to come by.

TIME(T) – REGULARITY OF SCHEDULE (ROS)

I would like to see more sursprise drop ins as a part of the formal system. MORE SURPRISE DROP INS (DI)

I wish other things coul be taken off of my plate so I could be in classrooms more

often. T – TOO MANY OTHER RESPONSIBILITIES (RESP)

time to make the feedback beneficial for student achievement T – HELPING STUDENT ACHIEVEMENT (ST AC)

Ensuring visits to each classroom regularly TIME – ROS

None

#####################################################

Paper-intensive model PAPERWORK (PW)

Size of the school and time available to observe all teachers adequately. T-ROS

Currently, I do not have any concerns

too much paperwork that takes away from the process PW

Sometimes the formal evaluation of teachers becomes an exercise in paperwork. The

real evaluation comes from a cycle of coaching and observing. PW – NEED COACHING AND OBSERVING

We have to have the summative conferences earlier than I would like, especially for

non-tenured teachers and when making tenure/license decisions. SUM TO EARLY FOR DECISION MAKING

#####################################################

None specifically

Being fair to all staff with an unbiased approach and making sure I have enough information to accurately evaluate them. ACCURACY/ FAIRNESS (AF) – QUALITY OF

DATA

I know there are one or two tenured teachers on my staff that need some additional assistance in areas and there is sometimes resistance by one in particlar to accept the

assistance TEACHER RESISTANCE (TR)

Much of the art or teaching and relating to students is difficult to qunatify in an observation AF – OBSERVATION DOESN’T MEASURE

present evaluation instrument does not cover all of teacher's methods concerning student learning AF – OBSERVATION DOESN’T MEASURE

Often times the evaluation process does not accurately measure concerns such as

professional conduct AF – OBSERVATION DOESN’T MEASURE

Time to be in enough classrooms T

n

I believe that observations that are announced are not a true measure of regular

teaching practice and therefore unannounced observations should be the norm to get

the true measure of teaching practice occuring in the building. MORE DI

Making sure I do not use my opinion, that I use the facts. AF – BIAS FREE

none

None


none

TIME required for paperwork and conferencing with staff T PW

The process is relatively time consuming. With a large staff, it can be difficult to give

feedback consistently. T - CONSISTENT FEEDBACK

psychological fitness

I know good instruction when I see it. The domains and indicators dont always allow

me to relay that information accurately. AF – QUALITY OF TOOL

The time it takes to be in classrooms. An administrator must know what is going on in the classrooms for optimum learning and teaching to take place. This is much easier

when there is an assistant(s) in the school and not just one administrator. T

being able to get to everyone's room! T – ROS

Formal observations need to occur more often, but time is a factor. T – NEED MORE

OBS

Any trained professional can "jump through the hoops" three times a year. I am more

interested in what occurs in each classroom on a daily basis and what each teacher

contributes to the school (outside the classroom). AF OBS DOESN’T MEASURE

Using TVAAS scores as part of the formal evaluation process is a very bad idea. We do

not have scores for all teachers and this is only one measure of effectiveness.

I believe that having a more specific/effective observation instrument could improve

the impact that observations have upon classroom practices. AF – QUALITY OF TOOL

– ST ACH

It is very subjective and I must remind myself each time that I need to look at it with an open mind. AF – SUBJECTIVE - QUAL OF TOOL

Do not have enough time to really evaluate my teachers, especially new staff members

due to the number of disciplinary referrals that are having to be dealt with on a daily basis. T – OTHER RESP

NONE

As a first year administrator non-renewal issues are difficult. – NON RENEWAL

teachers that put on a good show for their observation and do not continue that kind

of teaching after the observation AF – DOG AND PONY

I feel it would be beneficial to have more detailed informal evaluation procedures for

teachers that may be having trouble. – ADDITIONAL SUPPORT PROCEDURES FOR

STRUGGLING TEACHERS

I fortunately do not have any major concerns. Being in a small school, I am able to get in the classrooms on a regular basis.

I have concerns with the frequency of teacher evaluations. Evaluations should be

more frequent. Evaluations should be unannounced in order to see authentic teacher/student performance. – FREQUENCY – MORE DI

Personally, I have no concern. I observe the teachers at least once a week by walking

through the classroom.

If a very strong effort to point out needed improvements is not made by the observer,

the evaluation process we use is not very effective. Two or three scehduled visits are not a true picture of what is really happening in a classroom. AF – QUAL OF TOOL –

SCHED OBS PROBLEMS

It is difficult to "measure" classroom climate and the relationship that the teacher has

built with students most in need. State evaluation model does not necessarily

measure effectiveness of the teacher as a mentor and student advocate. AF – OBS DOEN’T MEASURE


Wearing both the coaching and evaluator hat is extremely difficult. Ideally, they are 2

different people. – CHALLENGE TO COACH AND EVAL

THE TIME TO DO EVALUATING T

Excessive paperwork for the results on evaluations. PW - INEFFICIENT

Lack of efficiency

It's a dog and pony show evaluation and you cannot possibly get an accurate idea of a

teacher. – AF – DOG AND PONY

None

If a staff member is ineffective AND have tenure, the process to get them where they need to be (either more effective in the classroom or not in the classroom at all) is too

laborious and takes entirely too long! – PROCESS FOR CORRECTING PROBLEM

TEACHERS

#####################################################

Tenured teachers need to be evaluated. I am sure I will get a visit from KCEA this year when I let the teacher know that I am requesting he go through the evaluation

process next year. NEED MORE TENURED EVAL

None

None

#####################################################

None

the model we employ in my school is approved by the state but is different from the

state model. To use the new model requires specific training by the evaluator and the evaluee in regard to expectations and feedback to be effective. – ALTERNATIVE

TRAINING OF EVALUEES

My biggest concern is finding the time to failry and accurately assess the effectiveness of our staff. – T

Learning about curriculum different from the area in which I taught. – CHALLENGES

CURRICULUM UNFAMILIAR

I don't believe the current model is specific enough to accurately evaluate teachers.

Teachers can't see strengths and areas to strengthen as clearly as possible. There is no place for a narrative description in the summative evaluation. QUAL OF TOOL –

NOT SPECIFIC ENOUGH – NEED OPPORTUNITY FOR COMMENT

The time involved in processing the State Evaluation Model is cumbersome, but I have

not seen a better formal tool. - T

I think that at some point it needs to become evident that how teachers work

"collaborate" with others is key in the success of individuals, team and ultimately

teachers. I do not think this is addressed enough – AF – OBS DOESN’T MEASURE

As an instructional leader, I need to continue to seek developmental opportunities that will enhance my teachers' effectiveness.

As an administrator, the evaluation of staff is one of the most important responsibilities. On-going walk-through evaluations are critical in developing a strong

assessment tool to identify a teacher's strengths and weaknesses in the classroom. MORE DI

Too many other duties interfere with my being in the classrooms enough. T- OTHER

RESP

I would like to see a component of the evaluation tool for evaluation of management skills outside of the academic areas. QUAL OF TOOL – OBS DOESN’T MEASURE

None

Teachers who can teach everyday,not just on the scheduled observations days – AF DOG AND PONY


NONE

Evaluations of new teachers are due to early in the year. NEW TEACHERS DUE EARLY

EFFECTS MEASUREMENT

None

It is important that teachers are evaluated beyond a formal evaluation. Unexpected

walkthrough evaluations effectively help measure a teachers true teaching strengths and weaknesses. – MORE DI

Being the only administrator in the building, it is very difficult to find the time to get

into the classrooms often enough. - T

Amount of time that it can take up with larger staffs. T

Not enough time to do the job adequately T

None

first year here

too soon to be confident in my own perceptions of teacher effectiveness QUALI OF DATA – CONFIDENCE -

May need more of a variety of instruments to assess regarding teachers and students -

VARIETY

Time not enough to look at the total teacher. T – OBS DOESN’T MEASURE

Specifying certain teachers for evaluation limits my ability to choose teachers that need

to be evaluated. NEED TENUR OBS, PRINCIPAL CHOICE

rubrics should be more user friendly-the wording is a pit ambiguous QUAL OF TOOL – AMBIG LANGUAGE

My concern is that the process is time consuming. It is often difficult to schedule

observation/meeting times when there is a large number of staff to evaluate in a year. T

Time! T

Some of the indicators are not specific enough for me. I would like an indicator that directly addresses classroom management. OBS DOESN’T MEASURE – AMBIG LANG

Time is always a factor. You plan on getting into classrooms and discipline issues or

IEP meetings come up and the day is gone before you get to get into those rooms. – T OTHER RESP

Stress it puts on the teacher TEACHER STRESS

The evaluation process needs to include data on student learning. NEED TO INCLUDE

STUDENT DATA

time constraints T

none


24. In your opinion, what resources are needed in your district to support administrators

in accurately assessing the quality of their staff?


More people to do the evaluating, such as lead teachers. Extra Staff (ES) Peer Eval (PE)

More training on "how to assist the struggling teacher". Struggling Teachers Extra Help

Time T

coaching on coaching TRAINING (TR) - COACHING

Time!! The assessment is not what takes so much time, it is coaching the teachers in a manner to make them more effective. Self reflective practices are difficult to instill in

tenured teachers, some of whom need great improvement. T – RESISTANCE TO IMPROVEMENT

More input from supervisors for teachers you are concerned about CENTRAL OFFICE SUPPORT – STRUGGLING TEACHERS

If principals could have some things taken off their plates, then we could be more available

to work individually with teachers in order to help them grow and become the most effective teacher for the students. – T – TOO MANY RESPONSIBILITIES

Ongoing PD focusing on teacher effectiveness and administrative coaching support TR-

EFFECTIVE TEACHING AND COACHING

######################################

Refresher classes on using the state model for local evaluation would be helpful. TR

specific professional development for teachers TARGETED TEACHERS FOR PD

A team of evaluators is more effective than one evaluator. If teacher growth is the true

issue, then you need a model like TAP or some other model that allows for frequent

observing and coaching by a team of professionals. ES AND PE

######################################

######################################

A validation test of raters scores to verify accuracy of scores throughout the system. RELIABILTY OF SCORING IN SYSTEM

High quality training. TR

professional development opportunities that are required when the principal says to go TARG TEACH FOR PD

More assistants so that focus on teacher eval is possible ES – ASST PRINCIPALS

resources are appropriate, the evaluation instrument is to "straight-line"; a teacher good be

a bad teacher and still score appropriately – MODIFICATION TO EVAL TOOL

None. The new formal walk through evaluations have addressed some of the issues of the

long periods some teachers would go through without being evaluated. ++WALKTHROUGHS HELP

Additional training - TR

n

Every school should have at least two administrators so instruction can be supported every

day in the classrooms. ES – MORE AP

The training is very useful. TR

More time T


Funding for traning TR

none

More input/participation from supervisors CENTRAL OFFICE SUPPORT

It would be nice to see more methods for getting teachers involved in helping each other in

non-threatening ways. Our system has made great strides in this with the current

professional learning community movement. TEACHER COLLAB SYSTEM

central office personnel – CO SUPPORT

An instructional specialist in our school. Additional administrator. ES

Great informal as well as formal documents to use when going in the classrooms that help

pinpoint areas of strength and need. MOD EVAL TOOL

principal inservices on dealing with giving negative feedback TR – NEGATIVE FB

Outside evaluators (supervisors, directors) CO SUPPORT

More money for staff development, more current technology, more and better use of

department chairs so that they too can assist in the evaluation process TR & CO SUPPORT

More drop in visits. – MORE DI

######################################

Administrators can never get enough training in this area. If we are going to put a lot of

emphasis on evaluations, we need to know what we are looking for in a setting. TR – WHAT TO LOOK FOR

######################################

THE RESOURCES ARE AVAILABLE IN OUR SYSTEM.

A different observation instrument that allows more for classroom management issues MOD

TO EVAL TOOL

classroom management training Common assessment training/ how to test/ how to make and read tests training TR CLASS

MGMT – ASSESSMENT

There is already a great deal of training.

I feel that Knox County do a great job of explaining and modeling the current evaluation

model.

A would like to see meaningful, weekly professional development provided by an appropriate intelligent experienced professional. Teachers should be exposed to best practices on a daily

basis while being evaluated on implementing best practices. ONGOING PD FOR TEACHERS IN EFFECTIVE PRACTICE

######################################

Better Evaluation Tool Enough time to spend on evals TIME - MOD TO TOOL

We have sufficient resources.

The number of evaluations is really increasing when the performance assessments are add

in. Time to complete is a challenge. T

N/A


Ability to remove constantly disruptive studnets that take excessive time and energy away

frrom productive aspects of the school without having to be penalized for graduation rate. – PENALIZING FOR – STUD IMPACTING CLASS

not sure

build capacity within the school to support principal time in the classroom – ES - T

More teacher mentors- literacy coaches, math coaches, etc. It would also help if every school had an assistant principal to assist with assessing staff. I would also like to see other

instruments made available to accurately assess teachers. ES – CO STAFF – MOD TO TOOL

More time in classrooms instead of completing other busy work type tasks. – REDUCE RESP

Supervisor support, maybe as a secondary evaluator, in evaluating a tenured teacher. CO SUPPORT

ongoing professional development in-services

conferences workshops - TR

Time. It is impossible to properly evaluate the entire staff with all the other responsibilities

given to administrators. – T – REDUCE RESP

A commitment by all Admin. Teams to be in the classrooms, everyday. – SCHOOL LEVEL

COMMITMENT

Professional Development - TR

Staff development involving using data from the classroom as part of the evaluation model. The growth plan which follows the evaluation must be a living document that becomes the

focus of the teacher's development. MOD TO EVAL TOOL – TARGETED AND COMPREHENSIVE GROWTH PLAN – TR FOR DATA USE

Our system does a nice job of providing the proper tools to effectively assess our staff.

More time T

A better evaluation model. – MOD TO EVAL TOOL

More assistant principals and Curriculum Instruction Facilitators. ES

Mentors, CIF being in the building more to serve in that "coaching" role. ES

Concerning teacher effectiveness, we need to continue to provide professional growth

opportunities for all administrators. TR

Continuous professional development in the area of changes in curriculum standards. – TR

IN CURRICULUM

More alternative school seats so disruptive students can be removed so teachers and

administrators can focus on students who want to learn. – DISR STUD ALT SCHOOL

More time - TIME

excellent staff development - TR

Time - T

TRAINING TO THOSE WHO NEED IT. – TR TARGET PD FOR TEACHERS


We have the proper tools.

There is a need for more available time to assess teaching. Too often discipline interferes with my time. T RED RESPO

Time is always a factor. T

Our school system is currently doing a good job. It would be nice to have supervisors do

routine observations periodically. CO SUPPORT

Different levels of professional development on evlauating teachers. What I needed to know

in my first year is defferent than in my 6th. TR THAT DEVELOPS WITH ADMIN EXPERIENCE

Training for new administrators - TR

none

secondary evaluators for all needing evaluation - ES

more computer information update

I think teachers need to have more opportunities to observe and engage in dialogue with

effective teachers. This is an indirect response to the question but it would help me assist teachers who need role models. – INC TEACHER COLLAB OPPORTUNITIES

The training I received on the instrument that Knox County uses to evaluate teachers was well done. I am comfortable with the format we use.

more time and less paper work-also, all observations should be unannounced T – REDUCED

PAPER – MOD TO EVAL (UNANN)

Professional Development training for novice teachers before they go through the evaluation

process. T TRAINING BEFORE PROCESS

Additional Assistant Principals to aid in other duties so the Principal can spend more time in the classroom. ES - APS

Effective training and consistency TR

######################################

Time T

We need an enhanced district-wide evaluation framework such as the Teacher Advancement Program (TAP) evaluation model, which is designed for improvement of targeted areas of

instructional practice. MOD TO EVAL TOOL - TAP

staff development and support when needed by an extra evaluator ES - TR

there are plenty of resources available now


25. Please add any additional comments you may have in regards to the administrator’s

role in teacher effectiveness, evaluation, or assessing teacher quality. You may also

consider one item from the scale above upon which you would like to expand your

answer.


The state evaluation system is built on the premise that self reflection and coaching

from an administrator will encourage the teacher to grow more effective. Just

having the time to do more than that is difficult. T

I firmly believe teacher quality would improve, if the tenure system were changed.

Teachers are too comfortable and know that it is difficult to remove an ineffective

tenured teacher thus creating a system that does not reward excellence but in many ways, rewards complacency. CHANGE THE TENURE SYSTEM

none

the observation process must go beyond the system required observations, ongoing

evaluation and ongoing feedback is imperative PRIN MUST DO MORE THAN THE

SYSTEM REQUIRES

No additional comments

None

I personally do not like formal evaluations because many times you get a "show" and not the true picture of the teacher. ANNOUNCED OBSERVATION NOT AUTHENTIC

It is important that administrators receive PD in this area as schedules, discipline,

etc. often take immediate precedence and focus. T – OTHER RESP TAKE PRECEDENT

For question #22, my answer last year (before moving to a new school) would have been strongly agree. It is only my unfamiliarity with and large number of teachers at

my present school that places my answer where it is. EXPERIENCE IN CURRENT SETTING EFFECTED RANKING ABILITY

N/A

all teachers should be evaluated yearly – INCREASED EVALUATIONS

Administrators must make teacher effectiveness a priority in the school in order to

impact student achievement. – PRIN COMMITMENT TO TEACHER EFFEC

Too much documentation required for non-renewal of non-tenured teachers. NON-

RENEW IS TOO DIFFICULT OF A PROCESS

My concern is for those schools that will be losing some of it's administrators due to budget cuts. Inorder for an administrator to be effective, the duties should be

shared. It is impossible to be effective when parent meetings, S-teams, M-teams consume most of your daily routine. As I was once told, "being and administrator is

an interruption on top of interruptions." Each day is a new experience. – STAFFING AND TOO MANY INTERRUPTIONS TO BE EFFECTIVE

I have no additional comments

None.


If I could hire my own teachers and not be forced to accept some administratice

transfers, the teacher effectiveness in my building might be higher – PRIN - MORE CONTROL OVER WHO WORKS IN YOUR BUILD - ADMIN TRAN

None

an administrator can help teachers be effective in the classroom in alot of ways

other than the evaluation instrument--listening, collaboration, etc. – PRIN - USE OF MANY STRATEGIES NOT JUST EVAL TO IMPROVE PRACTICE

Informal observations are also key to ensuring the administrator has an accurate picture of teacher effectiveness. – PRIN - USING INFORMAL TO GET PICTURE

N/A

n

Administrators are responsible to assess the overall effectiveness of the staff and

make the decisions that are best for each student and not what is convenient for the staff. PRIN – MAKE TOUGH DECISIONS AND MEET RESPONSIBILITIES

The administrator needs to be out in the school and in the classrooms. PRIN – OUT

AND VISIBLE

none

Too many teachers can just play the game. – MOD EVAL TOOL

nothing

None at this time

I think that it is important for teachers and administrators to view the formal and

informal observation process as a means of growth. - CULTURE

administrators should be friendly

Administrators are too bogged down with discipline, parents, s-teams, meetings, testing, etc. We would be nuch more effective if we could get into the classrooms

more frequently and do our own in-house staff development. Not enough time in the day. – REDUCED RESP – PRIN – MORE CONTROL OVER IN HOUSE PD

For me, building relationships with my teachers as well as trust is critical to my

effectiveness as an instructional leader. You need to build your relationships so they feel comfortable coming to you with ideas and value your opinion of their

effectiveness in the classroom. – BUILDING RELAT AS INSTRU LEADER

administrators need to provide positive feedback on a daily basis – PRIN -INC POS

FB

None

TIME....When I am principal of a building that has only two assistants and we must

evaluate 20-25 teachers...time becomes the issue. T

None


I believe that teacher effectiveness has a great impact on student achievement. In

addition, I believe that there are some factors that play a role in student

achievement that are outside of the teacher's control. I understand that one system in Ohio is offering the opportunity for homeless students to attend a boarding

school, funded by private foundations or businesses. I think that this is an innovative way to help support students whose homelessness ( a situation the

teacher cannot control) might impact their achievement. – HOME FACTORS

This is a very time consuming process! With one building level administrator and the

numerous evaluations and walk throughs, it gets overwhelming. T

"To protect, support, encourage, instruct, and defend" my teachers is my mission.

NONE

I feel tenured teachers should be evaluated more frequently and that classroom observations should be a part of all evaluations, including focused. INC TEACH

EVALUATIONS

none

Technology is a great tool that new teachers are using effectively and it shows when

they are formally evaluated.

The administrator's response to teacher observations is so important. They are the

person who can make a difference in the teacher by pointing out their areas of strength and areas to strengthen. However, they must be willing to provide support

for those areas of weakness. – PRIN - AFTER POINTING OUT S AND W, MUST BE WILLING TO SUPPORT

All schools (urban and suburban) would benefit greatly by using a variation of the

TAP model. This model is very effective in driving teacher growth. Observing/Evaluating only a few teachers formally is not an effective way in which to

monitor teacher effectiveness. – MOD TO EVAL TOOL - TAP

I feel the evaluation system does not accurately apply to special area teachers. – TOOL DOES NOT MEASURE SPEC AREA CLASS

I would like to see one scheduled evaluation and the other(s) be on a drop-in basis.

MORE DI

It would be somewhat helpful to have more leeway in determining how to deal with ineffective teachers. Some teachers just shouldn't be with students. From time to

time, an administrator sees that and doesn't always have the freedom to make a change. PRIN – MORE CONTROL TO REMOVE INEFFECTIVE TEACHERS

none

HAVE TEACHERS WRITE A SUMMARY.

Paper work requirements and redundant improvement plans require excessive time

and materials for the results they produce. More efficient and effective plans would allow time for implementatioin – T AND PW

.none

education can not stop with the school. the guardian must be involved and support

student learning – HOME FACTORS PARENTAL SUPPORT


No additional comments.

The administrator's role is key in setting the overall expectations and tone for a

building. If expectations are high, the majority of teachers will aspire to those expectations - it is difficult to inspire teachers who are buried beneath paperwork

and data reflection forms etc. – PRIN - SET HIGH EXPE

Frequent classroom observation is one of the ways you are able to measure the effectiveness of your teachers. I feel that along with classroom data analysis

throughout the year, seeing the growth students make, and observation of the classroom with student engagement, an administrator can pretty well predict the

effectiveness of each teacher – PRIN – FREQUENT CLASSROOM VISITS

Continuation of #24 The TAP rubric should be used during professional development to help teachers

become more effective teachers. In-services, conferences, and workshops based on the TAP rubric should be provided. – TR FOR TEACHERS ON TAP RUBRIC

A teacher's effectiveness is the result of a variety of factors not just test scores. –

EFFEC IS MORE THAN TEST SCORES

We must be in the classrooms everyday. – PRIN COMM TO BE IN CLASS EACH DAY

none

Effective leaders develop highly effective teachers. They also cull out ineffective teachers with their feedback and raised expectations. Needy schools must have

effective leadership and effective teachers to build capacity.

Accurately assessing a teacher's effectiveness is crucial to the overall success of any school. Teachers and administrators must understand that the process of evaluation

is a collaborative effort and the goal is to improve student learning. – PRIN RE EVAL GOALS OF EVALUATION

None

I believe with a better evaluation model (one similar to the TAP model) we can improve teachers effectiveness and student performance. – BETTER EVAL TOOL -

TAP

I use many different informal methods to evaluate teachers, such as "managing by wandering around." PRIN – DIFF STRATEGIES TO ASSESS

My role is extremely important in setting high standards in student achievement but

also giving teachers some ability to decide what are the best strategies. There has to

be a balance. – PRIN – HIGH EXPEC

The administrator must understand that evaluation is a never ending process and we can always improve. – PRIN0 – RE EVAL GOAL – ALWAYS IMPROVE

n/a

Administrators need to focus on high quality training for teachers that will add to their focus on student learning. PRIN – TR FOR TEACHERS, STUD FOCUS

I feel that acquiring tenure should be 5 years instead of 3. PRIN – CHANGE TENURE

SYSTEM

none

utilizing data driven decisions – PRIN – MAKE DATA DECISIONS


ADMINISTRATORS SHOULD BE IN TEACHER'S CLASSES DAILY. – PRIN – COMMIT

TO BE IN CLASSROOM

More time for improvement should be allowed for new teachers. Evaluations are due too early to show the growth they might make during that year. – NEW TEACH

EVALS DUE TO EARLY TO REFLECT GROWTH

Test Data is the most effective means of evaluating teacher effectiveness, but it cannot stand alone. – USE DATA, BUT NOT ALONE

Administrators need to make it a commitment to visit all teachers multiple times

during a school year to help assist and encourage teaching improvement. PRIN – COMM ON PART OF ADMIN TO BE IN CLASS

Teacher evaluations involve much more that classroom observations. It is just as

important to see the teacher interact with the students outside of the classroom as well as to be aware of how the teacher is utilizing his/her time with respect to

professional development. – MANY ELEMENTS TO GOOD TEACHER

An indicator needs to be developed that deals more directly with classroom management issues. The language in the current standards is vague in this area. –

MOD TO TOOL MGMT INDICATOR

As the Instructional Leader for the school, a principal must inspect what he/she expects. – PRIN – BE OUT IN CLASS

none

need more time and support in evaluating teachers -T

continue to provide updated information as we receive it from the state department.

Not really have any comments.

I have none at this time.

none

NA

I plan to have more unannounced formal observations next year. – MORE UNANNOUNCED

NA

I do feel that the administrators who have a lot of classroom experience can help

teachers better in the coaching role. If you have 10 years or less classroom experience, you have a handicap with being able to offer suggestions and help the

teachers to grow professionally. – LESS CLASS EXP – HARDER TO COACH TEACHERS

I believe that there needs to be a combination of evaluation tools used to accurately

assess teachers and staff members. – ADDITIONAL TOOLS FOR EVAL

The administrator has a vital role in not only assessing teacher effectiveness, but

improving teacher quality as well. An evaluation tool that informs and provides feedback on specific areas of instructional practice for improving student learning will

improve teacher quality. – PRIN- MUST IMP TEACHER QUALITY – MOD TO EVAL TOOL

Teacher evaluation is necessary but very time consuming. I think informal walk

throughs are good ways of evaluating teachers. Some of the paper work is very

time consuming for both the administrator and teacher. – T – MORE DI - PW


It is important to evaluate those untenured teachers and to be honest. It is critical to help those not really cut out for this job to find that out early on and help them

find another avenue to pursue. It is also critical to honestly evaluate interns before they are hired by a system to teach. PRIN – BE HONEST, MAKE HARD DECIS, CUT

BAD TEACH AND INTERNS

Appendix D

Survey Summary Report


Appendix D

Measuring Teacher Effectiveness


Answer

Options 1 2 3 4 5

Rating Average

Response Count

A1 1 6 18 44 24 3.90 93


Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A2 2 1 0 29 61 4.57 93


Answer Options 1 2 3 4 5

Rating Average

Response Count

A3 3 0 0 20 70 4.66 93


of each teacher's strengths.

Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A4 2 3 6 45 37 4.20 93

5. I use my experience as a classroom teacher to help me evaluate a teacher's practice.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A5 2 3 6 38 44 4.28 93

6. The evaluation model supplied by my school system allows me to accurately

evaluate my teacher's effectiveness.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A6 5 13 22 38 15 3.48 93


of the types of improvements each teacher might consider making.

Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A7 2 3 11 46 31 4.09 93

8. When I observe a teacher, I give them specific feedback about the strengths and weaknesses of their lesson.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A8 2 2 3 27 59 4.49 93



Rating

Average

Response

Count

A9 1 4 3 41 44 4.32 93

10. I spend more time observing instruction than is required by my school system.


Answer

Options 1 2 3 4 5

Rating Average

Response Count

A10 2 8 12 38 33 3.99 93

11. Formal evaluations of teachers are the most accurate way to measure teacher

effectiveness in the classroom.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A11 8 37 36 11 1 2.57 93

12. The analysis of students' standardized test scores (TCAP assessments, EOCs,

Gateway exams, etc.) is the most accurate way to measure teacher effectiveness in the classroom.

Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A12 9 29 24 26 5 2.88 93

13. I observe/evaluate ONLY those teachers who are scheduled for school-system evaluations during the current year.

Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A13 37 42 7 6 1 1.84 93

14. It is difficult to evaluate teachers in academic areas that are less familiar to me.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A14 13 49 16 15 1 2.41 93


Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A15 2 1 4 23 63 4.55 93


Answer

Options 1 2 3 4 5

Rating Average

Response Count

A16 2 0 4 33 54 4.47 93

17. Teacher effectiveness is the most important influence on student achievement.

Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A17 2 5 7 28 51 4.30 93

18. I use evaluation methods or tools other than those prescribed by my school system in order to evaluate the effectiveness of my staff.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A18 2 7 15 44 25 3.89 93


Answer

Options 1 2 3 4 5

Rating

Average

Response

Count

A19 3 3 7 37 43 4.23 93


Answer

Options 1 2 3 4 5

Rating Average

Response Count

A20 7 13 18 31 24 3.56 93


21. Non-tenured teachers are the only teachers that should be evaluated on a

yearly basis.


Rating

Average

Response

Count

A21 35 37 6 8 7 2.09 93

22. With a fair degree of accuracy, I could rank the teachers on my staff from least effective to most effective.

Answer

Options 1 2 3 4 5

Rating Average

Response Count

A22 0 5 6 49 33 4.18 93

23. Open-response; see

Appendix C

24. Open-response; see

Appendix C

25. Open-response; see Appendix C

Documents

PRINCIPALS’ ATTITUDES AND INSIGHTS CONCERNING THE ...cnweb.cn.edu/tedu/NCATE-State/Examples-ProfMajor/EDUC600Sample[3].pdfPrincipal’s Attitudes 3 Definitions Within the context