Taking a Second Look: Expert and Novice Differences when Observing the Same Classroom Teaching Segment a Second Time

Journal of Personnel Evaluation in Education 14:1 5±24, 2000

# 2000 Kluwer Academic Publishers, Boston±Manufactured in The Netherlands

Taking a Second Look: Expert and Novice Differenceswhen Observing the Same Classroom Teaching Segmenta Second Time

JUDITH A. KERRINS

California State University, Chico

KATHERINE S. CUSHING

California State University, Chico

Abstract

Expert and beginning principals viewed a classroom teaching episode of a seventh grade mathematics classroom

for teacher supervision purposes. After the ®rst viewing participants were asked questions about classroom

management and instruction, about the skills of the teacher, and about recommendations they would make for

improvement. After a second viewing, similar questions were asked. Differences between groups were found in

their understanding of classroom teaching and their abilities to evaluate and make recommendations for the

improvement of teaching. This study indicates that important differences will occur in the supervision experience

for teachers depending on whether an expert or novice principal supervises them. Findings from this study have

implications for the development of preservice and inservice training programs and induction experiences for new

principals.

Introduction

The research is quite clear: what principals believe, expect, emphasize, do, and say; how

they spend their time; and how they mold the culture of a school in¯uences student learning

(Brookover, 1981; Cawelti, 1987; Good & Brophy, 1986; Murphy & Hallenger, 1988;

Purkey & Smith, 1983). Known as the school effectiveness research, many of the ®ndings

from these studies are not surprising to those of us who have worked in schools. Over the

years, we've been privileged to have observed a few excellent principals in actionÐto see

how they spend their time; how they interact with students, staff, parents, and community

members; how they problem-solve; and how they move the school community toward a

vision of excellence for all. They were experts. We've also observed many beginning

principals. Recently credentialed, these new principals had studied and worked hard and

had been eager to assume their ®rst principalship. We were able to see how they, too,

interacted with others and how they focused their time and energy. The more we observed,

the more differences we noted between members of these two groups, and the more

intrigued we've become. What does expertise look like? we wondered, under what

conditions do experts act like experts? We began our search for answers to these questions

by reviewing the job expectations for school principals and the research on expertise.

The Work of School Principals

Historically, principals have been responsible for the management functions within

schools, including establishing school policies, dealing with attendance and discipline

issues, communicating with parents and community members, and managing the day-to-

day operation of the school. In the 1970s and 1980s, there was a call for principals to

become the instructional leaders of their schools. Though de®nitions of instructional

leadership vary somewhat, the focus is a shift away from the heavy emphasis on

management functions to a greater emphasis on the teaching/learning process (Oliva,

1984). Instructional leaders are concerned about what happens in the classroomÐteaching

strategies, student±teacher interactions, and student±student interactionsÐand how that

results in student learning.

While the idea of principal as instructional leader dominates many conversations about

principal responsibilities, in survey research principals repeatedly report that they spend

the preponderance of their time each day doing tasks related to school management. Oliva

(1984) notes that

although principals have responsibility for the curriculum and instruction of the school,

supervision of those aspects is only one of their many tasks. It is unfortunately true that

instructional supervision is often a secondary task to many school principals who

commonly lament that they do not have time to devote to curriculum and instructional

leadership because they are too busy with the day-to-day operation of the school.

( p. 10)

Even so, one of the major responsibilities of school principals is the supervision

and evaluation of certi®cated staff membersÐnamely, classroom teachers. What is

meant by the term instructional supervision? Alfonso, Firth, and Neville (1981) de®ne

instructional supervision as ``behavior of®cially designated by the organization that

directly affects teacher behavior in such a way as to facilitate pupil learning . . .''

(p. 43).

Since most principals have previous experience as classroom teachers, the supervision

process is not unfamiliar to them. Yet the roles of teacher and principal are different in the

supervision process. While teachers have knowledge and/or expectations based on their

being supervised, it seems reasonable to expect such knowledge to be different from that

of principals, since it was formed from a different perspective in a different role. This

might be especially true in comparison with principals with several years of experience

and/or expertise in the supervisory process. What, we wondered, are the differences in the

supervisory experience for those teachers who work with and are supervised by beginning

principals and those who work with and are supervised by experts? And if such differences

exist, what are the policy implications for staff to consider? Thus, with respect to

supervision, we wondered, What does expertise look like? and Under what conditions do

experts act like experts?

6 J.A. KERRINS & K.S. CUSHING

Expert±Novice Research

From studies of expertise, we have learned that experts in areas as diverse as physics,

medicine, board games, and baseball differ from novices in the way they perceive

information related to their area of expertise, encode and retrieve that information, and

solve problems in their area of expertise (Chi, Glaser, & Rees, 1982; Chiesi, Spilich, &

Voss, 1979; de Groot, 1965; Engle & Bukstel, 1978).

The early studies of expertise investigated disciplines not directly related to education.

Expert±novice teacher research emerged in the mid-1980s, and, for the most part,

replicated ®ndings of expert±novice differences in other ®elds. David Berliner and his

colleagues identi®ed differences in, among other things, teacher perception and

understanding of classroom environments and events, teacher knowledge and expectations

of students, teacher planning activities, and classroom instruction (Carter et al., 1988;

Carter et al., 1987; Sabers, Cushing, & Berliner, 1991). Summarizing ®ndings over

multiple studies, these researchers concluded that expert and novice teachers perceive and

understand classroom events differently, assume different roles in classroom instruction,

and have a different notion of what is ``typical'' in the classroom environment (Cushing,

Sabers, & Berliner, 1992).

A more recent study of teacher expertise (Ellett et al., 1994) found that teacher

nominations of superior colleagues identi®ed teachers who were rated more positively

through classroom observations and assessment procedures than those receiving no such

nominations. Statistical and meaningful differences were largest for teaching students to

think.

It wasn't until the late 1980s and early 1990s that studies investigating expertise of

school principals and superintendents emerged. Leithwood and Steinbach (1995)

published several studies that investigated the perceptions and problem-solving skills of

expert school principals and superintendents. Their initial work dealt with the frequency,

time frame (when problems occurred), and nature of problems (whether routine or

nonroutine; whether well or ill structured) encountered by principals and superintendents.

More recently, these authors studied principals' cognitive ¯exibility and in¯exibility in

problem solving. Leithwood's studies are of interest because he compared expert

experienced principals with typical experienced principals rather than with novices or

beginners. He did so because he believes it is ``relevant domain-speci®c knowledgeÐ

which many beginners have acquired in prior roles, and many experienced principals have

not acquired for reasons best known to them'' that distinguishes expertise from mere

experience (K. Leithwood, personal communication, October 31, 1995).

Others have argued that there is a relationship between experience and expertise.

Kolodner (1983) describes the evolution from novice to expert, writing

When a person has only gone to school and acquired book knowledge, he is considered a

novice. After he has experience using the knowledge he has learned, and when he

knows how it applies both to common and exceptional cases, he is called an expert. . . .Experience serves to turn unrelated facts into expert knowledge. ( p. 498)

EXPERT AND NOVICE DIFFERENCES WHEN OBSERVING THE SAME CLASSROOM 7

Berliner, too, has argued that experience is an important component of expertise and has

proposed a ®ve-stage theory about the development of expertise in teaching (Berliner,

1989). Berliner posits that teachers move from novice (stage 1) to advanced beginner

(stage 2) primarily due to the experience they gain as they develop both episodic and

strategic knowledge (knowing when to follow rules, and when to ignore or break them).

The third stage, competent teacher, is distinguished by two important characteristics:

teachers make conscious choices about what they are going to do, and because of that,

they feel more responsibility for what happens. At the pro®cient (fourth) stage, intuition

and know-how assume prominence, whereas experts (stage 5) ``have both an intuitive

grasp of the situation and a nonanalytic and nondeliberative sense of the appropriate

response to be made'' (Berliner, 1989, p. 43). In this model, experience helps beginning

teachers move towards the second and perhaps even the third stage of expertise; most

likely, it is re¯ected-upon experiences that move teachers beyond the stage of competent

teacher. It is also, and we suspect Berliner would agree, through relevant domain-speci®c

experiences that beginners acquire both knowledge and expertise.

The Research Question

From the studies cited above, one can expect that expert and beginning (novice) principals

differ in their understanding of the job roles and responsibilities of a principal and, very

likely, in their actual performance in that leadership position. Although differences may be

less discernible in areas where novices have extensive experience and/or relevant domain-

speci®c knowledge, this assumption still seems reasonable. Identifying the nature and

extent of those differences in a classroom teacher supervisory experience was the focus of

this study. Speci®cally, for this study we were interested in differences between the

classroom observations of experts and novices upon viewing a classroom teaching

performance for a second time.

This study is important for several reasons. First, classroom supervision is one of the

most important jobs principals do and, therefore, if differences exist it is critical to identify

and understand them. Beginning principals may be familiar with supervision from a

teacher's perspective. In fact, Leithwood believes that supervision may be the one aspect

of the principal's job where beginners have the greatest amount of domain-speci®c

knowledge. However, it is quite possible that the understanding and domain-speci®c

knowledge of teachers about teacher supervision may be characteristically different from

that of principals, who have different roles and responsibilities in the supervisory process.

If there are real differences in the supervision experiences of teachers based on the

expertise of the principal with whom they work, then teachers who work with beginning

principals may be at a disadvantage in terms of the support, encouragement, assistance, or

direction they receive during the supervision and evaluation process. It is important to

determine whether this is so, and to consider the possible effects of such a situation.

Second, from a policy perspective, it is interesting to consider observational differences

among expert and novice principals and whether a second look helps novices to see and

understand teaching in ways similar to experts. A ®nding such as this would have


important policy implications for the supervision, evaluation, and retention decisions of

classroom teachers. For example, a district might want to consider a policy that requires

two supervisors, at least one of whom is a recognized expert, when there is a question

about whether to renew a teachers' contract because of concerns about classroom

management or pedagogy.1

Third, there are a large number of school administrators retiring or eligible to retire and

many new, unseasoned administrators entering the profession (Adams, 1999; Lovely,

1999). In order to better understand the training, internship, and induction needs of

beginning school principals, it is important that we study expert±novice differences and

also investigate how expertise can be developed and fostered over time. If expert

principals see and understand classroom teaching differently than do beginning principals,

perhaps there are ways to help beginners develop these skills more quickly and ef®ciently.

Finally, from a cognitive psychology perspective, it is interesting to consider whether

expert school principals resemble experts in other ®elds, to determine under what

conditions they act like experts, and to study the cognitions of expert principals as they go

about the regular tasks of school leadership, especially teacher supervision.

Participants

Participants for this study were all drawn from the Rocky Mountain area. Expert principals

�n � 5� were building principals with ®ve or more years of leadership experience who

were identi®ed by their supervisors, colleagues, and the education community as being

outstanding school administrators. Two of the ®ve experts identi®ed for this study

appeared on every nomination list as an expert. The other three names appeared

frequently, although not on all lists. Names that appeared on only one or two nomination

lists were not included in this sample. Three of the experts were female; all were Anglo.

Experts averaged more than eight years of experience as a principal (range: ®ve to ten

years) and ten years of teaching experience prior to becoming a principal. Although the

nomination method is typically used for the identi®cation of experts, it does have certain

limitations. For example, we have no way of knowing whether these individuals were

nominated because they were expert managers or because they were expert instructional

leaders, since we did not probe the nomination criteria used by those who submitted the

names of expert principals. Still, the method has strong external validityÐthese individual

were perceived by three or more of their supervisors and colleagues as being experts as

de®ned by those individuals.

Novices �n � 6� were ®rst-year principals or principal candidates who had

satisfactorily completed, or almost completed, administrative certi®cation requirements.

Novices were identi®ed by their supervisors (district administrators, university

professors, or internship/®eld supervisors) as demonstrating the potential to become

outstanding school administrators. In all but one case, novices were experienced

classroom teachers and were widely recognized by colleagues as very good or

outstanding classroom teachers. The one novice exception was a well-respected school

psychologist who had completed administrative certi®cation requirements and was


seeking an administrative position. This novice served as acting principal in the buildings

to which he was assigned when the principal was away attending meetings (this was true

about several of our novices and was considered ``good experience,'' since they had

completed or were in the process of completing certi®cation requirements); he, too, was

highly respected by colleagues.

The use of two contrasting groups, expert and novice principals, is the appropriate

methodology for the exploration of the nature and development of expertise in school

administration. However, because of the small sample size, the ®ndings from this study

should be interpreted cautiously. Still, it is important to note that although only ®ve experts

and six novices were included in this study, the number of subjects is comparable to the

number involved in other studies of experts and novices.2 In addition, in-depth studies of

qualitative differences between groups requires the use of small samples.

Procedures

One class period (forty-®ve minutes) of a middle-school seventh-grade mathematics

lesson was videotaped for an entire week. Both teacher and students were asked to act as

they typically would and, as much as possible, to ignore the video camera (one) in the

classroom. No attempts were made to alter classroom conditionsÐrather, the hope was to

capture classroom instruction and activities as they naturally occurred. The taping was

made from two locations in the classroom: the back corner served as the primary location,

and about three quarters of the class period was taped from that position; then the video

recorder was moved to the front far corner of the classroom for the remainder of the class.

From both locations, the camera focused on both the teacher and students, following the

teacher as she moved around the room, focusing on students when they were answering

questions or working independently, and panning the room periodically to show all the

students and the entire classroom.

One videotape (one day's lesson) was selected for use in this study; no editing of the

original videotape was undertaken. The selection was based on the quality of the recording

and the teacher's perception that it re¯ected a typical day in her classroom. During the

forty-®ve-minute class, the teacher was primarily at the front of the classroom, with

students seated in two rows of desks arranged in a horseshoe shape. The front of the

classroom (the open end of the horseshoe) included a screen and overhead projector. The

video showed students entering the classroom and interacting with the teacher for about

three-and-a-half minutes. Then the teacher got students started on a sponge activity while

she interacted individually with them and took attendance at her desk in the back of the

classroom. About ®ve minutes later, the teacher moved to the front of the class and began

an oral review of the sponge activity, which lasted eight minutes. She then began an

around-the-world ¯ash card review of math facts. This competitive activity lasted about

four minutes. A one-minute explanation of a bonus question followed before she moved to

the overhead projector and reviewed the previous day's lesson, the area of a circle, for

eight minutes. Homework was then collected. Following that, the teacher began an

instructional activity on equivalent fractions, which she prefaced by saying they were


getting ready to ``play Bingo, but had to learn the rules ®rst.'' This instruction was teacher

led and required individual student responses to the teacher's questions. During this period

of instruction, the lesson was interrupted six times by announcements over the intercom.

The last four minutes of the class were spent with students working independently on what

appeared to be a homework assignment, although many students began getting ready to

leave while there was still about three minutes of instructional time left. The teacher's

offer of help, `Ì'll be in at lunch or after school if you have some problems,'' was shouted

to students as they left the classroom.

The Second-Look Supervision Task

For this study, participants were asked to do some role playing and to assume the role of

principal (though for our experts this was, in fact, their current position). They were told

that due to a major illness a colleague was unable to complete his classroom observations

for teacher supervision, and they were asked to help out by observing a seventh-grade

teacher. After viewing the videotape, participants were asked questions about their general

impressions of the classroom, perceived strengths of the lesson, and areas needing

improvement. During a second viewing of the same video, participants were asked to talk

aloud about what they were observing, and what drew their attention during the lesson.

After this viewing, participants were asked speci®c questions about classroom

management and instruction, about their rehiring recommendations, and about suggestions

for the teacher.3 Finally, participants were asked to judge the validity of this task in terms

of what principals really do. We should note here that all our experts were elementary

principals and most of our novices were elementary teachers. When asked about the

validity of the task, the dif®culty of observing at the middle-school level was speci®cally

probed and no participant indicted that this transition was dif®cult. In fact, most stated that

seventh grade was ``near enough'' to elementary that it didn't matter and that ``good

teaching is good teaching'' wherever you see it.

Data Analysis

Because of the nature of this study and the kind of data gathered during this task, primarily

qualitative data analysis procedures were employed. Participant interviews and talk-alouds

were audiotaped and then transcribed for analysis. Two researchers read and coded the

data independently using a multistep iterative process to determine patterns, trends, and

differences in both the kind and quality of responses. Coding focused on the nature of the

comments, whether descriptive, interpretative, evaluative, a recommendation, a summary,

or a quali®ed comment. Comments that stated what was happening on the video such asÐ

``Now she's moved to the back of the room,'' or ``She pulled down the screen and turned

on the overhead''Ðwere coded as descriptive; comments using words such as `ìt

appears'' or `ìt seems'' were coded as interpretative. When participants used words such

as `Ì think that is positive'' or ``some good examples,'' it was coded as evaluative. A


comment such as ``she needs to'' or ``she might want to'' was coded as a

recommendation. Summary comments tended to be listsÐ``so now this has happened

and then this, and then this''Ðor overriding statements, such as ``the whole ®rst part you

would have to categorize as a review.'' Finally, quali®ed comments included words such

as `Ì assume . . .'' or `Ì'm not sure but . . .''. Researchers sought to compare not only the

differences between groups but also the similarities and differences between the ®rst and

second viewing for each group as well as between groups.

After initial coding, researchers met to identify tentative ®ndings and to re®ne coding

procedures. Protocols were read an additional time to identify the best exemplars to either

support or refute individual understandings.

Data analyzed for this study were drawn primarily from the talk-aloud portion during

the second viewing and from participant answers to questions after they watched the video

a second time.

Findings from this study have been summarized below in the form of two statements

regarding differences in the perception and understanding of expert and beginning school

administrators about classroom supervision. Explanations of the ®ndings, as well as

examples from individual protocols that support these explanation, are reported below.

Results and Discussion

Overall, it seems that when experts view classrooms and teachers for supervision

purposes, they see the events of the classroom and both teacher and student behavior

differently than do novices. Experts view the big picture and provide interpretative

comments regarding teacher behavior. They are concerned about the coherence of the

lesson and wonder about the teacher's ability to self-evaluate and be re¯ective about the

lesson and her own needs and growth potential. They make recommendations about what

might improve the lesson, but qualify their comments with respect to both their

interpretation of the evidence and the limitations of the videotape format. Novices, on the

other hand, tend to be descriptive rather than interpretative about what they see during the

classroom observation. They often provide a series of statements about what they see

happening, a list of sorts, without questioning either the sequencing or coherence of the

lesson as a whole. Their responses are more varied, and sometimes contradictory, about

what they see happening. On average, they make about half as many evaluative comments

as the experts, but qualify their comments less than a third as often. They, too, are

interested in the teacher's self-evaluation.

Our observations about novice behavior are not surprising; in many ways, these novice,

would-be principals act like the novice teachers in the Berliner studies. Perhaps because

their experience base is one of teaching and, for the most part, they have not been asked to

judge or evaluate their peers, they are hesitant, uncomfortable, and awkward at doing so.

Instead, even though they are assigned the role of supervisor, and perhaps in part because

they know this is a simulation, they act more like colleagues than supervisors.

We offer two ®ndings to support this overall conclusion.


Finding 1: When expert principals view classroom teachers for supervision purposes,

they see the events of the classroom and both teacher and student behavior differently

than do novices.

To illustrate this ®nding, we offer data analyses in four areas: (1) differences in the types

of comments made, (2) descriptive/interpretative differences in understanding lesson

coherence, (3) descriptive/interpretative differences in understanding the strengths of the

lesson, and (4) descriptive/interpretative differences in understanding the management

strategies using during the lesson.

Differences in the Type of Comments Made

As reported earlier, comments made during the talk-alouds were coded into six different

categories: descriptive, interpretative, evaluative, recommendation, summary, or quali®er.

The number of comments in each category was changed to a percentage because the

experts, just like those studied by Peterson and Comeaux (1987) and Sabers, Berliner, and

Cushing (1991), gave more elaborated answers. These data are reported in table 1.

These data show that differences in two of the categories, namely, recommendations

and summaries, were negligible; small differences were found in three categories, namely,

interpretations, evaluations, and quali®ers; large differences were found in the descriptive

category. Taken together, these data suggest that experts are somewhat more balanced in

their view of and reactions to classroom observations. They clearly recognize that their job

is to do more than to describe what they are seeing; they have an evaluative and

improvement role, but also are cautious in over- or underinterpreting what they are seeing.

The novices, who in reality are teachers, are more comfortable with describing what they

see and less comfortable with the evaluative role they have been asked to play for this

study.

Table 1. Percentage of Each Groups' Comments by Category During the Second-Viewing Talk-Aloud.

Group

Type of Comment Expert Novice

Descriptions 39 55

Interpretations 28 24

Evaluations 16 10

Recommendations 7 7

Summaries 2 4 1

Quali®ers 8 3


Descriptive/Interpretative Differences Re: Coherence

Table 1 reports quantitative differences in the percent of descriptive and interpretative

comments made by experts and novices during the talk-aloud portion of the study.

Qualitative analysis reveals an interesting and important difference between experts and

novices: the focus of experts' observations was on students and learning, whereas novices

focused their observations on teaching.

Overall, experts demonstrated a desire to understand how the parts of the lesson ®t

together. They seemed to take a broad view of the lesson, trying to understand its

coherenceÐhow it tied together and contributed to student learning. They wanted to ask

the teacher broad, comprehensive questions about the purpose and focus of the lesson. In

fact, experts often acknowledged trouble identifying how the lesson parts were connected

and expressed concern that students, too, might not understand what they were supposed to

be learning. For example, when asked about general impressions of the teacher, experts

responded as follows:

Expert 1: I thought she had a rather unfocused lesson. I wasn't sure if she knew where he

was going. I would be curious to . . . ask her, if her students were successful, what they

would be able to do on the basis of that lesson. That was pretty unclear to me.

Expert 2: I would say that students didn't really know what it was they were supposed

learn that day.

Expert 3: You did get a sense that [the students] didn't think that they learned the lesson.

Expert 4: I felt a lack of . . . any centeredness. She did a lot of things which she must

have planned out, but it didn't seem to relate . . . I thought the kids were pretty tuned out.

. . . and by the time we got through the lesson, I was tuned out myself.

Expert 5: It was a sort of rambling, unconnected lesson. [She needs to focus on] what it

is we are trying to do today. Where's our beginning, where's our middle, where's our

end . . . Where are we going?

This concern about the focus of the lesson and how it tied together and made sense to

students permeated the protocols of experts regardless of the question to which they were

responding or the amount of time they spent observing the lesson. For example, after a

second viewing of the video, Expert 4 expressed frustration because the lesson ``didn't go

anywhere.''

Novices, on the other hand, tended to focus on discrete teaching aspects of the lesson,

expressing little interest or concern about student learning or about the lesson's overall

coherence. For example, when responding to the same question about general impressions

of the lesson, novices commented as follows:

Novice 1: Her relationships with the kids were appropriate. . . . her knowledge of what

she was teaching was pretty solid. . . . Behaviorally, the lesson was poorly organized.

Novice 2: I thought the teacher did a really good job. She switched activities quite often.

The children were very interested, they behaved really well.

Novice 3: I thought there were some things she could do to improve her overall teaching


style. . . . She needed material really badly. . . . The kids seemed to like her; she seemed

to have pretty good control in terms of behavior.

Novice 4: I thought she had an understanding of the content and needed to be more

aware of the students.

Novice 6: I would suggest she might focus on and work on classroom management.

Novices' tendency to focus on the separate components of the lessonÐcontent, classroom

management, discipline, and student±teacher interactionsÐsuggests that they saw each

component but were not troubled about the coherence of the lesson. Their focus was not on

understanding the lesson and how it made sense to students, but rather on understand-

ing the events. Analysis of novice protocols indicates that they were not very concerned

about the coherence of the lesson as supervisors or from a student-centered perspective.

In addition, novices seemed a bit inconsistent (as a groupÐthough none was probably

aware of this individually), as evidenced by the fact that novices sometimes contradicted

themselves and frequently contradicted one another, interpreting their observations quite

differently. For example, Novices 1 and 4, above, contradicted one another about the

teacher's relationships with her students, and Novices 3 and 6, above, about classroom

management. Such inconsistency suggests that novices either didn't know what to focus

on or had very different standards for what positive relationships or effective management

(among other things) looked like. Such contradictions would likely lead to very different

feedback to the classroom teacher, depending on who (which novice, let alone whether

feedback is from a novice or an expert) is providing the feedback. No contradictions were

noted among the protocols of the experts.

Descriptive/Interpretative Differences re: Strengths of Lesson

Descriptive/interpretative differences were also noted in expert and novice responses to

speci®c questions that were asked after the video had been watched twice.

After watching the video once, participants were asked about the strengths of the lesson.

Both experts and novices spoke positively about the relationship between teacher and

students. Several members of both groups spoke of the rapport the teacher had with her

students and noted that ``the kids like her'' (Expert 2) and that she ``likes the kids''

(Expert 2, Expert 3, Expert 4, Expert 5, Novice 2). Almost all the novices' comments

focused on the nature of the student±teacher relationship. Novices noted that she

`ìnteracted individually with each kid . . . [and her] interactions were upbeat and positive''

(Novice 1), that she ``knew the kids and . . . walked around and talked to them in the

beginning when they were coming into the classroom'' (Novice 2), that she ``seemed

friendly'' (Novice 5), and that she ``circulates around and helps the kids'' (Novice 6).

Using these terms, they explained and expanded on the notion of student±teacher rapport.

The positive nature of student±teacher interactions was the primary strength of the

teaching lesson as identi®ed by experts; however, there were ``¯ickers'' (Expert 4), or

strengths in other areas that experts also addressed. Experts often couched their comments

around suggestions that would result in improved teacher performance, seeming to


recognize that many of the teacher's actions needed ®ne-tuning. For example, Expert 2

commented that the teacher checked for student understanding but that this process ``was a

little faulty . . . She should have known there were problems someplace when not very

many kids would raise their hands for most questions.'' Expert 3 noted that the teacher

used competition but wasn't ``tuned into the cooperative aspects'' of the lesson, whereas

Expert 5 liked her ``real-life perspective [example . . . but it] wasn't developed.'' These

kinds of comments by experts suggested a need for the development of a deeper, more

conscious understanding of effective lesson development on the teacher's part. While

experts applauded the positive rapport, they saw a need to work with the teacher to develop

these additional skills.

Unlike experts, novices tended to identify and comment on the strengths of this teacher

without qualifying or supporting them. For example, referring to strengths, Novice 1

noted, ``she used visual aids.'' Novice 2 talked about the teacher's use of ``nonverbal cues

to start the class . . . She had quite a routine, the children are used to that routine and follow

it very well . . . She changed her activities . . . She moved [changed the content] a lot.''

Novice 5 noted that the teacher `ìs persistent,'' and Novice 6 commented that she

``circulates around and helps the kids.'' Perhaps this ®nding is to be expected, given that

the experience base of these novices is that of classroom teachers. It is as if they were

excusing the behavior by describing it, rather than evaluating the behavior from a

principal's perspective. It is also interesting to note that when novices did qualify their

comments, their quali®cations were usually related to the limitations of the videotape

(Novice 3: `Ìt was dif®cult to see with the video . . .''; Novice 4: `Ìt wasn't really clear

from the video . . .''; Novice 5: `Ìt's kind of hard to see in the video . . .''), rather than to

any issues or concerns they identi®ed about the lesson.

After watching the video a second time, other interesting differences between groups

were noted. Responding to the same question (``What do you see as the strengths of this

teacher and this lesson?''), experts merely noted that the teacher and students had a

positive relationship and left it at that. It was as if, upon a second viewing, the needs for

improvement were so great that experts had trouble seeing beyond (or at least commenting

upon) the greatest strength (rapport) and were, instead, ready to move on to suggestions for

improvement. Expert 4 commented on this, saying, `Ìt was harder to watch the second

time. I guess because I already knew that it didn't go anywhere.'' This ®nding was

corroborated in ®eld notes kept by the researchers, which indicated that every expert

responded negatively (with a grimace, rolled eyes, a ``wail,'' or a shake of the head) when

asked to view the video a second time. On the other hand, novices tended to repeat both the

number and kind of comments made after the ®rst viewingÐcon®rming what they'd seen

the ®rst time again, without making evaluative or interpretative comments about the lesson

itself. Table 2 reports these data.

Descriptive/Interpretative Differences re: Management Strategies

After viewing the video a second time, participants were asked, ``What speci®c

management strategies did you observe being used?'' Participant responses, by type


of comment, are reported in table 3 and represent only four of the six categories:

description, interpretation, evaluation, and recommendation. Novices provided no

recommendations.

These data suggest that novices are more comfortable with describing what they are

seeing, literally, when they watch classroom teachingÐand this makes sense, because they

are teachers. To some degree, they are also willing to interpret what they see, but they are

much less comfortable with the evaluative and interpretative role that is the responsibility

of the supervisor (typically, the school principal). Since the total number of comments was

similar (22 for experts, 18 for novices) but the percent in each category different, we

believe these data support Leithwood's notion of domain-speci®c knowledge and our

notion that the roles and responsibilities of ``supervisor'' account for differences,

regardless of past experiences.

These differences in how expert and novice principals perceive and understand teacher

behaviors during a second viewing of a classroom lesson indicate that experts are looking

deeply and carefully at what it is that teachers do and at how those actions are likely to

in¯uence student learning. Further, it appears that experts understand that when even small

parts of a lesson are off, it may make learning more dif®cult and contribute to a lack of

coherence for students. Novices did not appear to have developed such a ®nely tuned

understanding of lesson coherence or a focus on the learning aspects of the lesson. When

novices were concerned, their concerns dealt with the technical limitations of what they

saw ( perhaps as a means to defend a colleague), rather than with their own observation

skills and understandings of the teaching/learning process or of lesson coherence. From

data such as these, we would expect that recommendations from expert and novice

principals about the purpose and sequence of a lesson and about how to improve it,

Table 2. Number of Each Groups' Comments to Strengths Question.

Group

Expert Novice

After the ®rst viewing 25 25

After the second viewing 8 20

Table 3. Percentage of Each Groups' Comments to Management Strategies Question by Category.

Group

Type of Comment Expert Novice

Descriptions 50 66

Interpretations 27 28

Evaluations 14 6

Recommendations 9 0


especially if the lesson is only in need of thoughtful re®nement (®ne tuning), would be

signi®cantly different.

Finding 2: Experts have a different focus and provide qualitatively different feedback to

teachers based on the classroom observation experience than do novices.

Because experts and novices had different observation experiences, even though they

saw the same video of teaching performance, experts provided different feedback and

focused on different areas for improvement than did novices. Experts provided more

holistic feedback and focused on the instructional components of the lesson. Their

comments focused on instructional strategies that would improve student learning. Any

concerns they might have had about classroom management and student behavior were

addressed through increasing students' active participation in learning activities and the

interactive nature of the lesson itself, rather than through speci®c classroom management

strategies. Examples of this approach are re¯ected in the recommendations that experts

made for this teacher, including the following:

Expert 1: . . . being very, very clear in terms of what she wants to teach and focus on, and

what she thinks will get her there . . . lay[ing] out a sequence . . . and follow[ing] up with

a closing and a reinforcement or practice activity . . . I think it would be important that

she either generate discussion or give some models of how this would be applied.

Expert 2: I'd try to keep the lesson simpler . . . and ®nd a way to keep the kids more

actively engaged . . . [and check] for how well the students were understanding the

different things that she was talking about.

Expert 3: . . . if she intended for this to be an inquiry lesson . . . I think [I could] help her

with some different ways of setting up inquiry . . . with the idea of critical attributes . . .

strategies for doing that. Another improvement [would be] equitable response

opportunity . . . [and] a way of checking for understanding.

Expert 4: I would suggest that she not try to do as much. Focus on one area . . . and to

devise a method to ®nd out if the kids understood what she was talking about. . . . I

would try to help her focus on one speci®c skill that she wanted to talk on that day.

Hopefully the homework and the activities in class would re¯ect that instead of jumping

around quite so much.

Expert 5: What was the objective? What are we trying to get at here . . . engagement . . .

check for understanding . . . more immediate feedback . . . closure.

Novices, on the other hand, had a clear focus on management concerns and classroom

organization; their recommendations were either descriptive or listyÐas if by saying

``do this, and this, and this,'' the quality of teaching and the learning experience for

students would improve. Novices usually addressed the instructional focus after they

had made recommendations intended to improve management concerns, or embedded

this focus within their discussion of management strategies. For example, when asked

about making recommendations about what needed to be improved, novices replied as

follows:


Novice 1: I would talk to her about classroom management. . . . She didn't deal well with

interruptions . . . the door should have been closed. She needs to structure her classroom

to minimize some of these things. . . . There should be a rule about sharpening pencils.

. . . I would say she needs to address organization, classroom management . . . and she

needs to have greater clarity and focus on what she's doing.

Novice 2: I think the main part of her lesson was on area of circles . . . and she didn't

seem to have enough things to do to really reteach that in a better way.

Novice 3: She seemed to be calling more on boys than girls. . . . The lesson . . . might

have lent itself well to presenting the materials in other modalities than the overhead. . . .I think the main thing is just being aware of the on-task issue. . . . She always started off

by walking down the left side of the room.

Novice 4: I would want her to work on time-on-task. . . . One recommendation I think

would be to have more purpose to her lesson and knowing where she is going. . . . I felt

there was a lot of off-task time and attention-getting behavior [and] I would like her to

work on awareness of that.

Novice 5: What is it you're going to accomplish and how you're going to get there. . . .Have your examples ready beforehand. . . . Get rid of the ¯ash cards.

Novice 6: Have some kind of structured, organized way to start the class. . . . I think I'd

stick to one or two concepts rather than overpowering. . . . She needs to have, maybe, a

tighter rein on the class . . . and then closure.

Perhaps this focus on management is not so surprising, given that teachers know that

classroom management is the number one concern of teachers (Veenman, 1984). Being

themselves teachers, they focus on management before they focus on anything else.

When experts and novices were asked what they would look for if observing near the

end of the school year, differences between the two groups were also noted. Experts

continued their emphasis on the focus of instruction, whereas novices continued to express

concerns about management and time issues. The following statement by Expert 1

summed up experts comments:

I would want her to be much more focused in terms of what she was after. I would want

her to be much more evaluative in terms of determining what the students were learning.

And I would hope to see many more learning activities that had more direct and

interactive involvement with the students.

All but one expert spoke to these three aspects of an improved lesson. Only two novices

spoke to the purpose or focus of the lesson; most emphasized classroom management

concerns and time-on- and time-off-task.

These kinds of differences in recommendations for improvement have major

implications for the improvement of classroom teaching and student learning. While

there was some variability within groups, it was much smaller than the variability of

comments between groups. From these data, it is clear that the kind and quality of

feedback teachers receive from colleagues, even those credentialed and searching for

positions as school administrators, will be different than the feedback they receive from


expert school principals. It is not so far a leap to conclude that feedback and

recommendations from beginning school principals, novices if you will, about improving

classroom instruction will be qualitatively different feedback from that of expert

principals.

Conclusion and Implications

What do the ®ndings from this study mean, and what implications do they have for the

training of school principals and for the supervision of classroom teachers?

First, two words of caution. Although every attempt was made to simulate a real-

classroom supervision experience, and even though experts acknowledged that this

simulation pretty much re¯ected what they really do, it is important to remember that data

were collected in a laboratory setting using videotapes as the stimulus material. The use of

the videotape format for something as contextually driven as classroom observation and

supervision must be noted (though, in fact, these limitations were the same for all

participants). The laboratory situation and the small, nonrandom sample limits the

generalizability of these propositions to other populations and settings. However, if the

®ndings from this study are similar to ®ndings in other studies of expertise, then this will

provide additional support for the general nature of expertise regardless of the speci®c

domain of study. Further, in this simulation, novices were asked to play the role of

principal. Because they were primarily teachers who had never been in a principal role for

more than a day or two at a time, it is possible and perhaps even likely that they were

unable to play this role well. If that is true, it may account for their tendency to describe

and/or defend the teacher and their inability to provide more evaluative comments or to

focus on learning rather than teaching. To the degree that this actually happened, these data

more accurately re¯ect collegial feedback and recommendations rather than those of

novice principals. Still, we believe that the differences are worthy of notice.

We are encouraged that the results of this study are consistent with the ®ndings of

studies of expert±novice teacher differences concerning pedagogy reported by Berliner

(1989) and his colleagues. From this study, it appears that novice principals have

developed a knowledge base similar to that of experts around typical instructional

methodology and classroom management practices. However, the ability of expert and

novice principals to use that knowledge in meaningful, integrative, and contextually

appropriate ways is different. When experts observe a lesson, they are looking for

coherence and meaning; they are interested in the purpose of the lesson and how all the

parts ®t together and make sense to students. They ask about student involvement and

interaction, methodology, meaning, and relevance. They tie all their questions to student

learningÐhow is what's happening in the classroom likely to further student engagement

and learning of the discipline? This focus on student learning by these experts appears to

be quite unique: two national reviews of teacher evaluation instruments and procedures

around the country document a focus on teacher behaviors and performances rather than

on student learning (Ellett & Garland, 1987; Loup, Garland, & Ellett, 1996). Yet these

experts were consistent in their student-centered focus on learning. Further, in observing


experts, their observations, comments, and responses to questions demonstrated the

automaticity that Bloom (1986) writes about and the intuitive grasp and nondeliberative

sense of the appropriate response that Berliner (1989) explains de®nes expertise.

Novices, on the other hand, appear to be busy trying to understand each of the parts of

the lesson and never get to the issue of connectedness and coherence. In many instances

they focus, see, and comment on similar teaching events as experts. However, their

comments tend to be descriptive rather than interpretativeÐthey see what is happening,

but they do not look at the broader issue of what is needed to connect the lesson together

and how this correction will make sense to students. Nor do they appear able to evaluate

what they see or make recommendations for improvement.

Differences such as these have important implications for the supervision of classroom

teachers and for the training of school principals. Perhaps we should look carefully at who

is doing classroom supervision. Regardless of their level of experience or expertise, all

building principals are expected to observe in classrooms and to work with teachers to

improve the teaching/learning process. Data from the present study suggest that a

principal's ability to do this varies signi®cantly depending on his or her level of experience

and expertise. Teachers who work with a novice principal are likely to receive

qualitatively different feedback than are teachers who work with experts. While this

issue might not be terribly important to expert teachers, it may be critical to beginning

teachers, teachers struggling with issues of classroom management or instruction, or

average teachers who want to improve and who may not receive the help, insight, or

recommendations they need if their supervisor is a novice principal.

If we want beginning principals to consistently demonstrate the conceptual, integrative,

and contextually appropriate skills of experts, then they must have qualitatively different

training programs than many are currently experiencing. Perhaps some of the case-based

and problem-based learning experiences now emerging in our educational leadership and

administration programs will foster this kind and level of understanding. From our data,

we believe that novices are likely to need multiple opportunities to experience

performance responsibilitiesÐclassroom observations, supervisory conferences, guided

practice opportunities, etc.Ðprior to assuming those responsibilities if we want them to

perform more like experts than they presently do. Further, we would argue that novices

should truly have an evaluative role in these training opportunities, since this role may be

an important variable that mediates their behavior. Note that, in many instances, novices

were not too far off the mark from experts, and more experiences with qualitatively

different feedback might make a difference in their level of expertise as supervisors of

instruction.

One way to provide this kind of support is through a mentoring or induction experience

for beginning administrators. Daresh and Playko (1992) note that ``the majority of

mentoring schemes . . . have tended to focus on helping people learn critical technical

skills'' ( p. 5). Our data suggest that novices have already developed many of the technical

skills but have not yet learned to integrate those skills in a meaningful, contextually

appropriate, and more supervisory way. Mentoring and induction programs should focus

on this level of integration and should provide many opportunities for novices to view the

world of teaching and learning from a broader, more holistic perspective, to re¯ect upon


those opportunities, and to receive feedback from experts regarding their performance.

Arrendondo et al. (1995) note that ``the supervisory process is at its best when it

encourages re¯ective practice'' ( p. 75). Re¯ection on the part of novices, paired with

discussion and feedback from mentors, will likely improve novice performance.

We might also want to consider changing our expectations of beginning principals and

start to develop year-long or multiyear apprenticeship experiences. In a manner similar to

the mentoring and induction experiences suggested above, novice principals could be

paired with experts who can provide ongoing feedback and facilitate re¯ection and growth

of both novice principals and teachers about what is observed in classroom visits and how

that leads to instructional improvement. This kind of coaching is likely to improve both

what happens in the classroom and the novice principals' ability, in the long run, to give

useful and comprehensive feedback to teachers. This is not unlike the training

(apprenticeship) experiences of other professions, such as medicine, or of the training

required for high-stakes judging, such as AKA or cat-show judges.

We believe that the ®ndings from this study have important policy implications for both

school districts and university certi®cation/credential programs. Districts should look

carefully at the training experiences of those they hire for building principal positions and

at the performance expectations placed upon them. Districts should pair up beginning and

expert administrators to ensure that novices have an opportunity to work closely with

expert models who are skilled peer coaches, in the hope that this experience will help

further novice development. Perhaps formative classroom supervision could become a

joint responsibility, shared among expert and novice principals. In particular, it is not very

realistic to expect a novice principal to be of much help to a struggling teacher or to

improve the teaching/learning process in any signi®cant way. Finally, university

preparation programs for principals need to ensure continued and ongoing opportunities

for students to re¯ect upon and integrate technical information in a more holistic way

when talking about classroom teaching and learning. If we are serious as a nation about

improving student achievement for all students, then we must also improve the skill level

of our educational leaders.

Notes

1. The Miami±Dade County Public Schools has been developing this kind of program for the past three years.

New teachers are assessed (for professional development) by a three-member team that includes a principal

or designee, a teacher colleague, and a third person. While the evaluation decision is solely the principal's to

make, this model mixes various levels of expertise and offers assessment team members the opportunity to

learn from one another (Chad Ellett, personal communication, November 1, 1999).

2. See, for example, the work of Berliner and his colleagues (1989); Chi, Glaser, & Farr (1988); or Mazzeo

(1985).

3. Illustrative examples of the speci®c questions that participants were asked after the second viewing include

the following: Q1: `Òkay, now that you've watched the video twice, what do you see as the greatest

strengths of this teacher?'' Q2: `Ànd what do you think is the area of greatest concern?'' Q3: ``What speci®c

management strategies did you observe being used?'' Q4: ``What about the speci®c instructional strategies

you observed being used?'' Q5: ``This is the middle of the year. If you were to observe this teacher near the

end of the year, what speci®cally would you be looking for?''


4. We should note that all our experts were elementary principals (interestingly, only elementary principals were

nominated) and that all but one of our novices worked at the elementary level (the one exception was a

middle-school ®rst-year assistant principal). We speci®cally asked participants whether they had ``concerns''

observing at the middle level, since it was ``different'' from where they typically worked. To a person, they

commented that the level was ``close enough'' to their experience that it was not an issue and that they were

looking for teaching qualities that they believed were not dependent on the speci®c grade levels taught (or

observed).

References

Adams, J.P. (1999). Good principals, good schools. Thrust for Educational Leadership, 29(1), 8±11.

Alfonso, F.J., Firth, G.R., & Neville, F.F. (1981). Instructional supervision: A behavior system. Boston: Allyn

and Bacon.

Arrendondo, D.E., Brody, J.L., Zimmerman, D.P., & Moffett, C.A. (1995). Pushing the envelope in

supervision. Educational Leadership, 53(3), 74±78.

Berliner, D.C. (1989). Implications of studies of expertise in pedagogy for teacher education and evaluation.

In New directions for teacher assessment: Proceedings of the 1988 ETC invitational conference. Princeton, NJ:

Educational Testing Service.

Bloom, B. (1986). Automaticity. Educational Leadership, 43, 70±77.

Brookover, W.B. (1981). Effective secondary schools. Philadelphia, PA: Research for Better Schools (Ed 231

088).

Carter, K., Cushing, K., Sabers, D., Stein, P., & Berliner, D.C. (1988). Expert±novice differences in

perceiving and processing visual information. Journal of Teacher Education, 39(3), 25±31.

Carter, K., Sabers, D., Cushing, K., Pinnegar, P., & Berliner, D.C. (1987). Processing and using information

about students: A study of expert, novice and postulant teachers. Teaching and Teacher Education, 3, 147±157.

Cawelti, G. (1987). How effective instructional leaders get results. Paper presented at the Annual Meeting of

the American Association of School Administrators, New Orleans, LA, February, 1987 (ED 328 935).

Chiesi, H.L., Spilich, G.J., & Voss, J.R. (1979). Acquisition of domain-related information in relation to high

and low domain knowledge. Journal of Verbal Learning and Behavior, 18, 257±273.

Chi, M.T.H., Glaser, R., & Farr, M. (1988). The nature of expertise. Hillsdale, NJ: Erlbaum.

Chi, M.T.H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. Sternberg (Ed.) Advances inthe psychology of human intelligence. Vol. 1, (pp. 7±75). Hillsdale, NJ: Erlbaum.

Cushing, K.S., Sabers, D.S., & Berliner, D.C. (1992). Olympic gold: Investigations of expertise in teaching.

Educational Horizons, 70, 108±114.

Daresh, J.C., & Playko, M.A. (1992, April). What do beginning teachers need? Aspiring and practicingprincipals' perceptions of critical skills for novice administrators. Paper presented at the Annual Meeting of the

American Educational Research Association, San Francisco.

de Groot, A.D. (1966). Perception and memory vs. thought: Some old ideas and some recent ®ndings. In B.

Kleinmuntz (ed.), Problem solving (pp. 392±425). New York: Wiley.

Ellett, C.D., & Garland, J.S. (1987). Teacher evaluation practices in our largest school districts: Are they

living up to ``state-of-the-art'' systems? Journal of Personnel Evaluation in Education, 1(1), 69±92.

Ellett, C.D., Loup, K.S., Evans, R.I., Chauvin, S.W., & Naik, N.S. (1984). A study of teachers' nominations of

superior colleagues: Implications for teacher evaluation programs and the construct validity of classroom-based

assessment of teaching and learning. Journal of Personnel Evaluation in Education, 8(1), 7±28.

Engle, R.W., & Bukstel, L. (1978). Memory processes among bridge players of differing expertise. AmericanJournal of Psychology, 91, 673±689.

Good, T.L., & Brophy, J.E. (1986). School effects. In M.C. Wittrock (Ed.), Handbook of Research onTeaching, 3rd ed. (pp. 570±602). New York: Macmillan.

Kolodner, J. (1983). Towards an understanding of the role of experience in the evolution from novice to

expert. International Journal of Man±Machine Studies, 19, 497±518.


Leithwood, K., & Steinbach, R. (1995). Expert problem solving: Evidence from school and district leaders.

Albany, NY: State University of New York Press.

Loup, K.S., Garland, J.S., & Ellett, C.D. (1996). Ten years after: Findings from a replication of a study of

teacher evaluation practices in our 100 largest school districts. Journal of Personnel Evaluation in Education,

10(3), 203±226.

Lovely, S. (1999). Developing leaders from within. Thrust for Educational Leadership, 29(1), 12±13.

Mazzeo, J. (1985). An evaluation of the level of skill required of operators of a computer-assisted radiologictotal lung capacity measurement system. Unpublished doctoral dissertation, University of Arizona, Tucson.

Murphy, J., & Hallenger, P. (1988). Characteristics of instructionally effective school districts. Journal ofEducational Research, 81(3), 175±181.

Oliva, P.F. (1984). Supervision for today's schools. New York: Longman.

Peterson, P.L., & Comeaux, M.A. (1987). Teachers' schemata for classroom events: The mental scaffolding of

teachers' thinking during classroom instruction. Teaching and Teacher Education, 3(4), 319±333.

Purkey, S.C., & Smith, M.S. (1983). Effective schoolsÐa review. Elementary School Journal, 83(4), 427±

452.

Sabers, D.S., Cushing, K.S., & Berliner, D.C. (1991). Differences among teachers in a task characterized by

simultaneity, multidimensionality, and immediacy. American Educational Research Journal, 28, 63±88.

Veenman, S. (1984). Perceived problems of beginning teachers. Review of Educational Research, 54(2), 143±

178.


Documents

Taking a Second Look: Expert and Novice Differences when Observing the Same Classroom Teaching Segment a Second Time