Mathematics and Science Achievement of U.S. Fourth and ... · Annual Colloquium Journal vol. XIV, Spring 2009 Highlights from The Trends in International Mathematics and Science Study(TIMSS)

i

Annual Colloquium Journal vol. XIV, Spring 2009

Highlights from The Trends in International Mathematics and Science Study (TIMSS) 2007:

Mathematics and Science Achievement of U.S. Fourth and Eighth-Grade Students in an International ContextDecember 2008

Executive Summary

The 2007 Trends in International Mathematics and Science Study (TIMSS) is the fourth administration since 1995 of this inter-national comparison. Developed and implemented at the international level by the International Association for the Evaluationof Educational Achievement (IEA)—an international organization of national research institutions and governmental researchagencies—TIMSS is used to measure over time the mathematics and science knowledge and skills of fourth- and eighth-graders.TIMSS is designed to align broadly with mathematics and science curricula in the participating countries. This report focuses onthe performance of U.S. students relative to that of their peers in other countries in 2007, and on changes in mathematics andscience achievement since 1995. Thirty-six countries or educational jurisdictions participated at grade four in 2007, while 48participated at grade eight. This report also describes additional details about the achievement of U.S. student subpopulations.All differences described in this report are statistically significant at the .05 level. No statistical adjustments to account for mul-tiple comparisons were used.

Key findings from the report include the following:

• In 2007, the average mathematics scores of both U.S. fourth-graders (529) and eighth-graders (508) were higher than theTIMSS scale average (500 at both grades). The average U.S. fourth-grade mathematics score was higher than those of studentsin 23 of the 35 other countries, lower than those in 8 countries (all located in Asia or Europe), and not measurably differentfrom those in the remaining 4 countries. At eighth grade, the average U.S. mathematics score was higher than those of studentsin 37 of the 47 other countries, lower than those in 5 countries (all of them located in Asia), and not measurably different fromthose in the other 5 countries.

• Compared to 1995, the average mathematics scores for both U.S. fourth- and eighth-grade students were higher in 2007. Atfourth grade, the U.S. average score in 2007 was 529, 11 points higher than the 1995 average of 518. At eighth grade, the U.S.average mathematics score in 2007 was 508, 16 points higher than the 1995 average of 492.

• In 2007, 10 percent of U.S. fourth-graders and 6 percent of U.S. eighth-graders scored at or above the advanced internationalbenchmark in mathematics. At grade four, seven countries had higher percentages of students performing at or above theadvanced international mathematics benchmark than the United States: Singapore, Hong Kong SAR, Chinese Taipei, Japan,Kazakhstan, England, and the Russian Federation. Fourth-graders in these seven countries were also found to outperform U.S.fourth-graders, on average, on the overall mathematics scale. At grade eight, a slightly different set of seven countries had high-er percentages of students performing at or above the advanced mathematics benchmark than the United States: Chinese Taipei,Korea, Singapore, Hong Kong SAR, Japan, Hungary, and the Russian Federation. These seven countries include the five coun-tries that had higher average overall mathematics scores than the United States, as well as Hungary and the Russian Federation.

• In 2007, the average science scores of both U.S. fourth graders (539) and eighth-graders (520) were higher than the TIMSSscale average (500 at both grades). The average U.S. fourth-grade science score was higher than those of students in 25 of the35 other countries, lower than those in 4 countries (all of them in Asia),and not measurably different from those in the remain-ing 6 countries. At eighth grade, the average U.S. science score was higher than the average scores of students in 35 of the 47other countries, lower than those in 9 countries (all located in Asia or Europe), and not measurably different from those in theother 3 countries

• The average science scores for both U.S. fourth- and eighth-grade students in 2007 were not measurably different from thosein 1995. The U.S. fourth-grade average science score in 2007 was 539 and in 1995 was 542. The U.S. eighth-grade averagescience score in 2007 was 520 and in 1995 was 513.

• In 2007, 15 percent of U.S. fourth-graders and 10 percent of U.S. eighth-graders scored at or above the advanced internation-al benchmark in science. At grade four, two countries had higher percentages of students performing at or above the advancedinternational science benchmark than the United States: Singapore and Chinese Taipei. Fourth-graders in these two countrieswere also found to outperform U.S. fourth-graders, on average, on the overall science scale. At grade eight, six countries hadhigher percentages of students performing at or above the advanced science benchmark than the United States: Singapore,Chinese Taipei, Japan, England, Korea, and Hungary. These six countries also had higher average overall eighth-grade sciencescores than the United States.

http://nces.ed.gov/pubs2009/2009001.pdf

ii


GUIDELINES FOR SUBMISSION

The papers submitted for the Journal must discusspsychological and pedagogical issues and trendsrelated to mathematics and science education.

WHEN SUBMITTING A PAPER, PLEASE USE THEFOLLOWING GUIDELINES:

1. Submit an electronic version of the paper, anabstract, approximately 150 words, and abiographical sketch, about 30 words. All pictures and diagrams must be submitted in a separate document.

2. Use double spacing with one-inch margins.

3. For references, tables, and figures follow thestyle described in the Publication Manual ofthe American Psychological Association (APA),Fifth Edition.

4. Paper length must not exceed 30 pages,including pictures, tables, figures, and list of references.

5. Paper must be submitted by December 1.

6. Authors will be notified about the status oftheir papers by January 15.

7. The Colloquium is scheduled in April.

SUGGESTIONS TO THE AUTHORS:

When preparing a research paper include:

a) a rationale and an identification of theresearch question(s),

b) a conceptual framework or brief statement ofrelationship to the literature,

c) an identification of research methodology,

d) a summary of the analytical technique(s),

e) a summary of preliminary findings.

SUBMIT PAPERS AND CORRESPONDENCE TO:

Dr. Regina M. PanasukProfessor of Mathematics EducationGraduate School of Education

University of Massachusetts Lowell61 Wilder Street, O’ Leary 5th FloorLowell, MA 01854

Phone: (978) 934-4616Fax: (978) [email protected]

2009 Annual Spring Colloquium JournalThe Graduate School of Education, University of Massachusetts Lowell

iii


TABLE OF CONTENTS

Inquiry-Based Teaching for American and Japanese Middle-School Science Teachers: TwoViews, Two PracticesSachiko Tosa ......................................................................................................................1

Enhancing the Teaching of the Nature of ScienceMichael Wadness..............................................................................................................11

A Study of the Relationship Between Group Perception of School Climate and Incidencesof Bullying at the Junior High/Middle School LevelKimberly Douglass ..........................................................................................................24

Qualifying Papers

Increasing the Effectiveness of Mathematics Instruction Through Lesson PlanningNoreen Flanagan Johnson ................................................................................................39

Interactive-Engagement and Reflective WritingMichael Doherty ..............................................................................................................53

iv


CONTRIBUTORS

Sachiko Tosa earned a PhD in Physics from the University of Rochester in 1986. She has been teaching sciencecourses for children at the Museum of Science since 1995, and is involved in a technology-based educational proj-ect in the Computer Science Department at UML. Sachiko is a recipient of Graduate Research Scholar award in2008 and currently working on the final stage of her dissertation.

Kimberly Douglass is currently completing her dissertation for the Leadership in Schooling Program. She graduatedwith a B.S. in Special Education in 1994 and an M.S. in School Psychology in 2000. Kimberly is currently employedas a school psychologist for the Northborough, Massachusetts School Department

Michael Wadness is a doctoral student in mathematics and science education and is making his final preparationsto begin his dissertation research. Michael has been teaching physics for eleven years at Medford High School.

Noreen Flanagan Johnson teaches Mathematics at the Greater Lawrence Technical School. She is currently workingon her dissertation proposal in the Math and Science Education doctoral program at UML.

Michael Doherty has been teaching Physics, Chemistry, and Electronics at Andover High School. For the past twoyears he has been involved in UML Teams Academy teaching Baseball Bat Design Engineering. He is currentlyworking on his dissertation proposal in the Math and Science Education doctoral program at UML.

2008–2009 Academic YearMathematics and Science Education Program

2009 Annual Colloquium Journal on Research in Mathematics and Science EducationThe Graduate School of Education, University of Massachusetts Lowell

DISSERTATION DEFENSE STAGENeil Hatem Wesley JohnsonSachiko Tosa Matthew BeynavenardMichael Wadness

DISSERTATION PROPOSAL STAGEKate McLaughlin Jaimie StoneAlex Ballantyne Noreen Flanagan JohnsonValerie Finnerty Michael Doherty

For information about Graduate Programs in Mathematics and Science Education at UML,

please contact the appropriate faculty member:

Vera Ossen, Ed.D. Regina Panasuk, [email protected] [email protected] Director of Educator Licensure Programs Mathematics Education

David Lustick, Ed. D. Michelle Scribner-MacLean, [email protected] [email protected] Education Science Education

Application packets may be obtained on line or by contacting the Graduate School of Education at (978) 934-4601

1


ABSTRACTThis study examines similarities and differences inhow Japanese and American middle-school scienceteachers think and feel about inquiry-based teaching.Their attitudes toward the use of inquiry in scienceteaching were measured through a survey instrument(N=191). The results showed that American teachersvalue the importance of inquiry-oriented teachers’actions more highly than Japanese teachers do.Japanese teachers reported feeling less comfortablehelping students with their questions and activities.These findings were further examined thro u g hschool visits in Japan (N=15) and the United States(N=9). Preliminary analysis of data indicated thatJapanese teachers were not generally helping stu-dents construct their own understanding of scientificconcepts in spite of well-organized lesson structuresand activity set-ups. On the other hand, theAmerican lessons often lacked meaningful sciencecontent in spite of the high level of inquiry-orientedteaching strategies that many teachers exhibited.Implications regarding the cultural differences in sci-ence education will be discussed.

Since the publication of the National ScienceEducation Standards (NSES) (National Researc hCouncil [NRC], 1996), the importance of scientificinquiry has been clearly emphasized in K-12 scienceeducation in the United States. Scientific inquiry isregarded as the heart of science education becauseinquiry helps students develop deeper understanding ofscientific concepts (NRC). Students construct their ownunderstanding of scientific ideas when they make senseof what they are learning (Bybee, 2000; Krajcik,Blumenfeld, Marx, & Soloway, 2000). Through theactive process of inquiry, students are expected to devel-op “the knowledge and understanding of scientific con-cepts and processes required for personal decision mak-ing, participation in civic and cultural affairs, and eco-nomic productivity” (NRC, p. 22). The effectiveness ofinquiry-based teaching for developing students’ deeperunderstanding of scientific concepts has been suggestedby various re s e a rchers (Anderson, 2002; Colburn ,2004; Ertepinar & Geban, 1996; Krajcik et al.;Shymansky, Kyle, & Alport, 1983; Von Secker, 2002).

In spite of the emphasis on scientific inquiry in thescience education research community, the literaturesuggests that very few inquiry-based lessons are taking

place in actual science classrooms (Lotter, 2004; Reiff,2002; Simmons et al., 1999). The Third InternationalMathematics and Science Study (TIMSS) 1999 VideoStudy of science (National Center for EducationStatistics [NCES], 2006) showed that 66% of 88 U.S.8th-grade science teachers’ lessons developed sciencecontent mainly by student acquisition of facts, defini-tions, and algorithms. The study showed that only 17%of the U.S. lessons developed science content throughinquiry. It appears that U.S. teachers are not translatingthe guidelines for inquiry-based teaching described inthe NSES.

In contrast to the situation in the United States, theliterature suggests that Japanese science lessons seem tobe more inquiry-oriented (NCES, 2006). In the TIMSS1999 Video Study (NCES), 72% of Japanese science les-sons for 8th graders developed science content mainlyby making connections between students’ activities andthe explanations. The study further showed that 57% ofthe Japanese lessons developed content by making con-nections through inquiry. However, there is some evi-dence that Japanese science teachers are just followingprescribed ways of teaching science as inquiry, and thattheir teaching is not inquiry-based teaching in the sensedescribed by the NSES (Ogura, 2004). In the independ-ent research study based on the TIMSS 1999 video-taped lessons, Ogura found that U.S. science teacherswere critical about Japanese lessons; U.S. teachers whowatched the Japanese video-taped lessons found thatJapanese teachers were typically so concerned with giv-ing students “the right answer” as the result of theiractivity that they rushed toward the conclusion at theend of the lessons.

Given a set of contradicting findings, it is importantto conduct a study on science teachers’ thoughts andfeelings about inquiry-based teaching in Japan and theUnited States. The TIMSS Video Study (NCES, 2006)showed that inquiry was more widely used in middleschool science in Japan than in the United States basedon observational data collected by the governmentalagencies. The next step may be to more closely studywhat science teachers think and feel about inquiry-based teaching in these countries. By studying individ-ual teachers’ thoughts and feelings toward inquiry, we

Inquiry-Based Teaching for American and Japanese Middle-School Science Teachers:Two Views, Two Practices

This work was in part supported by CAREER grant provided by the National Science Foundation under the grant number 0546513.

Sachiko TosaUniversity of Massachusetts Lowell

2


may be able to understand the situation of inquiry-based teaching from teachers’ points of view. By observ-ing lessons in a more private setting, we may be able tocollect data from their actual daily practices. Moreover,by matching up teachers’ thoughts and feelings withtheir practice, we may be able to draw a better pictureof the current status of inquiry-based teaching in mid-dle school science in each country. Furthermore, byunderstanding how the meaning of inquiry - b a s e dteaching is similar and different to American teachersand Japanese teachers, we may be able to identify whatelements are missing for promoting inquiry - b a s e dteaching in both countries.

C o n s e q u e n t l y, the following re s e a rch questionsregarding U.S. and Japanese middle-school scienceteachers guided the present study:

1. What is their understanding of inquiry-basedteaching?

2. What are the similarities and differences betweentheir understandings of inquiry-based teaching?

3. How do their teaching practices reflect theirunderstanding of inquiry-based teaching?

4. What are their attitudes toward the use ofinquiry-based teaching?

5. What are the similarities and differences betweentheir attitudes toward the use of inquiry-basedteaching?

6. How do their teaching practices reflect their atti-tudes toward inquiry-based teaching?

The significance of this study is to inform scienceeducators what might be lacking in science teaching inJapan and the United States. As Stigler, Gallimore, andHiebert (2000) described in their paper of a compara-tive study, teaching is a cultural activity. It is difficult tosee some of the ubiquitous teaching practices andbeliefs in one’s own culture. It is often the case that edu-cators are not aware of lacking elements because theymay be buried in the tradition of teaching in the partic-ular country. By comparing U.S. and Japanese teachersin terms of their understanding of, and attitudes towardthe use of inquiry, this study reveals what is taken forgranted as inquiry-based teaching in each place. Thisstudy further provides U.S. and Japanese science educa-tors in higher education with information about whatshould be more strongly emphasized in teacher educa-tion programs in order to increase the use of inquiry inmiddle school science.

METHODResearch MethodsThis study measured U.S. and Japanese science

teachers’ understanding of, and attitudes towardinquiry-based teaching. Teachers’ attitudes were meas-ured by a survey instrument. The survey data were ana-lyzed statistically. Data on teachers’ understanding ofi n q u i ry-based teaching were collected qualitativelyt h rough lesson plans, classroom observations, andinterviews. The qualitative data provide informationabout the context of each of the lessons, including thecontent, the teacher, and the students.

ParticipantsFifty-seven American and 134 Japanese teachers

completed the survey. They were the members of list-serve networks to which the researcher had access.Participants for classroom visits were first recruitedthrough mailings to randomly selected middle schools.Participants were also recruited through a request thatwas posted at the end of the survey. Fifteen Japaneseand ten American teachers agreed to participate in class-room observations and interviews. Four of theAmerican teachers dropped from the participation inthe process, and three more American teachers wererecruited using a snowball sampling technique.

InstrumentationInstruments to measure teachers’ attitudes toward

inquiry-based teaching were not available in literature(Smolleck, Zembal-Saul, & Yoder, 2006). Thus, prior tothis study, the researcher created and piloted a surveyinstrument by clearly defining attitudes toward inquiry-based teaching into four categories. The scale measuredteachers’ (a) general preference of inquiry-based teach-ing, (b) personal evaluation toward performing typicalbehaviors in inquiry-based teaching, (c) beliefs aboutthe possible obstacles to inquiry-based teaching, and(d) self-efficacy about the use of inquiry in scienceteaching. The instrument contains 45 questions: 37closed-ended questions with ordered response option,one open-ended question about inquiry-base teaching,and seven demographic questions. The survey instru-ment items and their categories can be found in theAppendix.

The researcher also created an interview protocol.The interview protocol included questions regardingthe observed lessons as well as questions regardingteachers’ thoughts about inquiry-based teaching. Theinterview started with a question about the lesson thatwas just completed. In this way, the interviewer wasable to have the participants describe their thoughtsabout the use of inquiry on a concrete basis.F u rt h e rm o re, using the interview protocol theresearcher was able to collect data that could be sorted

3


out in six categories for participants’ understanding ofinquiry-based teaching: (a) definitions of inquiry-basedteaching, (b) what constitutes student learning, (c)teachers’ depth of understanding of subject-specificknowledge, (d) teachers’ understanding of effectivenessof different instructional strategies in inquiry-basedteaching, (e) teachers’ depth of understanding of theways to transform scientific concepts into teachableforms in inquiry-based teaching (pedagogical contentknowledge), and (f) teachers’ understanding of thenature of scientific inquiry.

The rubric for analyzing classroom observationswas also created by the researcher. The observed lessonswere coded by using a rubric that describes how each ofthe essential features of inquiry-based teaching (NRC,2000) were practiced. The rubric included three points(X) based on who completed the activity (teachers 1,students and teachers 2, students alone 3), and fivepoints (Y) for the depth of conceptual link displayed.The two kinds of points were given for each of theessential features of inquiry-based teaching.

Newly-created instruments were reviewed by agroup of experts for content validity. Following theexperts’ comments, the instruments were revised sever-al times before they were submitted to the InstitutionalReview Board for permission for conducting research onhuman subjects.

ProceduresThe request for middle-school science teachers to

participate in the survey was distributed through U.S.

and Japanese listserve networks separately in late May,2008. The responses were collected through the use ofonline survey instrument provided by a commercialcompany. Both American and Japanese data collectiontook place for two-and-a-half weeks.

The researcher visited each of the interview partici-pants’ schools once to conduct a classroom observation,an interview, and a collection of artifacts including thelesson plan and worksheets. The researcher observed15 Japanese science teachers’ lessons at 13 public mid-dle schools in the Tokyo area in June and July, 2008.Nine American teachers’ classes at eight public middleschool schools in the state of Massachusetts wereobserved in September and October in 2008.

RESULTSCronbach’s alpha for the whole set of survey data

(N=191) was .796, indicating high internal consistencyreliability. Alpha for the Japanese data alone (N=134)was .815, indicating even higher internal consistencyreliability. The Cronbach’s alpha for the U.S. data(N=57) alone was slightly lower, yet still reliable at.771.

Factor analysisA factor analysis was performed for the whole set of

data (N=191). The analysis identified 10 factors witheigenvalues greater than one. The matrix rotation con-verged and the items that share similar characteristicswere identified. The percentage variation explained is65. The scree plot is shown in Figure 1. It shows a dis-

Figure 1. Scree plot for the factor analysis

4


tinctive elbow.By considering common features of the items

grouped in the factors, each of the factors can be namedas they are shown in Table 1.

The numbers in Table 1 indicate the averagenumerical values (and standard deviation) for the itemsgrouped in the factors. The numerical values 5 to 1were given to each of the response options for the 36items in the instrument, where strongly agree = 5, agree= 4, neither agree nor disagree = 3, disagree = 2, andstrongly disagree = 1. The asterisk (*) in the last columnin Table 1 indicates that the difference between the U.S.and Japan is statistically significant for the factor(p<.01).

Four of the items that address the obstacles toinquiry-based teaching were factored out in Factors 3and 8. The items that ask teachers’ self-efficacy aboutinquiry-based teaching were loaded on particular fac-tors (Factors 2, 4, and 5). It is interesting to observe thatunderlying categories in defining teachers’ attitudestoward the use of inquiry in teaching was revealed inthe analysis. This result implies that the scale well rep-resents the underlying latent variables in the researchproject.

The results of the factor analysis indicate that bothAmerican and Japanese teachers highly value the

importance of the elements of inquiry-based teaching(Factor 1). They both agreed, for example, that it isimportant individual students formulate their explana-tions from evidence in their science classes (Q13). Thefactor analysis also shows that American teachers agreewith the importance of inquiry-oriented teachers’actions more strongly than Japanese teachers. It alsoshows that American teachers were more satisfied withthe extent to which they use inquiry in their scienceclasses than Japanese teachers (Factor 9). One of thereasons for the Japanese teachers’ reservations towardthe use of inquiry might be the time constraint in theJapanese curricula; Japanese teachers have only 105 sci-ence lessons in a school year (Ministry of Education,C u l t u re, Sports, Science, and Technology [MEXT],2007), whereas American teachers typically have 135hours (NCES, 2008). The Japanese teachers’ concernabout the time constraint as an obstacle to inquiry-based teaching is clearly shown in the factor analysisresults (Factor 3).

The factor analysis also reveals that Japanese teach-ers felt less comfortable helping students with theirquestions and activities (Factors 4 and 5). This some-what surprising result will be discussed in the next sec-tion.

The observed lessons were coded using the rubric

Factor Name of the factor Mean Score (SD) Significance(survey items grouped in the factor) of

differenceFactor 1 Standard-based concepts of inquiry-based

teaching (Q10, Q12, Q13, Q14, Q18, Q21)Factor 2 Students’ self-directedness and teachers’

confidence in teaching inquiry-based lessons (Q4, Q9, Q25, Q36, Q37)

Factor 3 Time and material constraints as obstacles to inquiry-based teaching (Q33, Q34, Q35)

Factor 4 Helping students and the importance of inquiry-based teaching (Q2, Q24, Q29)

Factor 5 Dealing with students’ questions and higher level of thinking (Q22, Q26, Q27, Q28)

Factor 6 Correctness of the scientific concepts (Q16, Q23)Factor 7 Secondary important concepts of

inquiry-based teaching (Q6, Q19, Q20)Factor 8 Presence of mandate tests as an obstacle

to inquiry-based teaching (Q30)Factor 9 Teachers’ satisfaction (Q3)Factor 10 Importance of whole class understanding (Q17)*p<.01

US Japan4.46(0.37) 4.20(0.53) .006*

4.17(0.51) 3.93(0.59) .065

2.76(0.85) 3.48(0.79) .000*

4.46(0.72) 3.99(0.57) .000*

4.41(0.41) 3.95(0.52) .000*

3.35(0.69) 3.41(0.62) .5684.35(0.46) 4.24(0.47) .389

3.49(1.16) 3.19(1.17) .204

3.42(1.08) 2.93(0.92) .002*4.18(0.87) 4.33(0.68) .224

Table 1.Name of Factors and Mean Scores for the Items in Each Factor by Country.

that describes how each of the essential features ofinquiry-based teaching was practiced. Two numberswere given to each essential feature found in the lesson.When the particular features were not found in theobserved lesson, the researcher asked in the interview ifthe teacher intended on including the feature in the fol-lowing lesson. If no intention was indicated, the lessonreceived (0, 0) as a set of scores for the feature. If theteacher indicated his/her intension to carry over the fea-ture into the next lesson, it was left as blank for the fea-ture. Table 2 shows the preliminary results of the meanscores for each of the essential features observed inAmerican and Japanese lessons.

Table 2 indicates that on average it was the teacherswho gave students the questions to be investigated dur-ing the lesson in both American and Japanese lessons (Xscores of Feature 1). It was also the teachers, on theaverage, who connected explanations to scientificknowledge (X scores of Feature 4). In contrast, when itcame to collecting data, students did the activities inboth countries (X scores of Feature 2). However, for for-mulating explanations from data, more American teach-ers let students do the task (X scores of Feature 3). Table3 also shows that on the average little conceptual linkswere displayed in both American and Japanese lessons(Y scores of overall mean). However, it is interesting tonotice that the American lessons received higher Yscores for the features that were exhibited in the begin-ning of the lessons (Features 1 and 2) than those exhib-ited later in the lessons (Features 3 and 4) . On the

other hand, the Japanese lessons received the sameaverage scores throughout the features from 1 to 4.

DISCUSSIONThis study examined differences and similarities

between U.S. and Japanese science teachers in terms oftheir thoughts and feelings toward the use of inquiry-based teaching. The findings from the survey analysisshowed that American teachers agreed more stronglywith inquiry-based teaching than Japanese teachers. Thefindings also showed that there was a statistically signif-icant difference between U.S. and Japanese teachers interms of the willingness of helping students with theirquestions and problems during their science classes.

These findings make more sense when descriptiveobservation data were taken into consideration. Theresearcher found that except for one lesson, which wasdevoted to skills training of connecting an electric volt-age meter and a current meter in a circuit, the observedJapanese lessons started with scientifically meaningfulquestions. The questions were examined by students inexperiments or observations. The Japanese middle-school students were often instructed to perf o rmsophisticated experiments using equipments such asgas burners and microscopes. After the experiments,Japanese students were instructed to write down theconclusions that they drew from the activities. At theend of the lessons, teachers made connections betweenwhat they found from the activities and scientific con-

Mean scores (SD) for who complete the activity X and how conceptual links were made YEssential Feature of Inquiry X Y

US(N=9) Japan (N=15) Sig. US (N=9) Japan (N=15) Sig.1. Students engage in scientifically

oriented questions2. Student gives priority to evidence

in responding to questions (Student collects and analyzes data)

3. Student formulates explanations from evidence

4. Student connects explanations to scientific knowledge

5. Student communicates and justifies explanations

Overall Mean

1.06(0.17) 1.13(0.52) .264 2.56(1.59) 2.00(0.76) .118

2.06(0.18) 2.00(0.00) .264 2.56(0.68) 2.00(0.00) .009*

2.25(0.76) 1.58(0.90) .007* 2.00(1.16) 2.00(0.95) .554

1.14(0.90) 1.00(0.00) 1.000 1.21(1.15) 2.00(0.87) .132

0.67(0.52) 1.00(0.00) 0.75(0.76) 1.00(0.00)

1.49(0.30) 1.38(0.33) 1.87(0.57) 1.87(0.46)

5


Table 2.Mean Scores for Essential Features of Inquiry Found in US and Japanese Classroom Practice

Note. X scores indicate who did the activity: 1 = teacher only, 2 = teacher and students together, 3= students only. Y scores indicate the depth of concep-tual links displayed by teacher and students during the lesson, ranging from 1= no conceptual links to 5 = multiple links as conceptual networks. Sig. indi-cates the significance of difference between countries.

6


cepts in a whole-class discussion. In spite of the well-o rganized lesson stru c t u re, the re s e a rcher did notobserve incidences in which Japanese teachers helpedstudents construct their own meaning of the scientificconcepts. This trend was indicated by the flat Y scoresof 2.00 for the features 1 through 4 in Table 2. It showsthat Japanese teachers were making connectionsbetween activities and concepts at each step of the con-tent development. However, the connections weremade not through logical reasoning steps, but by givenfacts. The Japanese teachers rarely took time to relatethe questions to students’ prior experiences or conceptsthat they have learned already. Many of the teacherseven omitted stating what the purpose of the experi-ment was before students started the task. Many stu-dents often expressed in private conversations that theydid not know what they were expected to do. Duringthe student activities, the Japanese teachers were oftenbusy checking if the experiments were performed in theright way. During the interviews, however, the Japaneseteachers expressed the importance of helping studentsmake connections between the outcomes of their activ-ities and the scientific concepts. It seems that theJapanese teachers are unfamiliar with actual ways howto help students construct their knowledge.

Compared to the Japanese lessons, the observedAmerican lessons had more variations in the Y scores.The larger standard deviations in Y for U.S. teachers inTable 2 support this point. Some of the teachers in theobserved American lessons helped students well in theirconstruction of the meaning of the activities through awhole-class discussion. These teachers related the topicwith what students learned already. Some of the teach-ers also prompted students to come up with the exper-imental procedures by a series of questionings. In spiteof the high level of pedagogical skills that some of theteachers exhibited, many American lessons did not con-tain scientifically meaningful content. The researcherfound little scientific concepts involved in thre eAmerican lessons, and they received low Y scores; ifthere were no scientific concepts to be connected withthe results of the student activities, there would be noconceptual links made. It also happened in some of theU.S. lessons that the experiments did not show what theteacher wanted to show. When the experiments did notwork, students were having a hard time generatingexplanations for the phenomena and the lessonsreceived low Y scores for Features 3 and 4.

The differences between U.S. and Japanese sciencelessons that were observed in this study may be attrib-uted to the different experiences that the teachers hadduring the course of their education in terms of the

amount of science content they have learned. All of theJapanese teachers hold undergraduate degrees in eitherscience or science education. Undergraduate studentsin science education in Japan are required to take sci-ence courses in a similar way as students who are in sci-ence and engineering departments in order to be certi-fied as middle-school science teachers (MEXT, 2008).They are often expected to develop teaching materialsas undergraduate research projects. For example, aJapanese teacher who participated in the interviewreported that as his undergraduate research project hedeveloped a way to measure time when a falling objectpassed two distant points using sensors. Even at thegraduate level, Japanese students in science educationare required to take science classes. A typical researchproject at the Master’s level in Japanese science educa-tion is to study the effectiveness of the use of materialsthat were developed to teach specific science content.Three of the Japanese teachers who participated in theclassroom observations hold master’s degrees in sciencerelated fields. It was clear that all the Japanese teacherswho participated in the observations had strong back-ground in science content and materials in the course oftheir education. On the other hand, American teacherswho participated in this research had less experiencesstudying science at higher education institutions. Fourof the American teachers did not have science back-ground as undergraduate or graduate degrees; two ofthem majored in elementary education, while the othertwo studied humanity subjects and were certified toteach science after graduation. The other five teachersstudied science as undergraduates and studied educa-tion at the graduate level. However, none of them tookscience courses at their graduate schools.

Teachers’ experiences of learning science contentare not the only place where differences between U.S.and Japanese middle-school science teachers are found.When the demographic data is examined in terms of theamount of teachers’ experiences in learning pedagogy,another difference is revealed. Seven of the Japaneseteachers who hold degrees only in science did not studyeducation as their major subject of study. In contrast,only one of the American teachers did not have experi-ence of learning education as the major field. As far asthe observation participants were concerned, theAmerican middle-school science teachers had moreexperiences than the Japanese teachers in learning ped-agogy in their own course of education.

Table 3 shows the result of the analysis when demo-graphic data of survey participants and observation par-ticipants was examined in terms of their educationalbackground.

7


The numbers of respondents for the survey partici-pants shown in Table 3 were much smaller than thenumbers of whole survey participants due to the factthat many teachers did not provide the names of thefields they studied. Table 3 indicates that the trend thatwas found for the observation participants also holdsfor the survey participants: all the Japanese participantshave science in their educational background, whereasmore than 40% of American teachers did not have sci-ence as the major field of study in their own course ofeducation. In terms of background of learning peda-gogy, 63% of Japanese teachers did not have muchexperience, whereas only 15% of American teacherswere in this category.

The findings of this study imply that the lack ofcontent in U.S. middle-school science lessons might bedue to the lack of science education in teachers’ back-ground. The findings suggest that one of the ways toimprove U.S. science teachers’ practice would be toinclude more science courses for teacher preparationprograms. Without solid content knowledge, teacherswould not be able to use pedagogical knowledge forinquiry-based teaching that they have acquired throughtheir education trainings. On the other hand, one wayto improve Japanese teachers’ practice for promotinginquiry-based teaching would be to put more emphasison pedagogy than on materials and procedures in theirteacher preparation programs. It should be noted, how-ever, that these conclusions drawn from the observa-tional data were based on small sample sizes and thusshould be taken cautiously

Further analysis of observation data and interviewdata is underway. It is expected that interview data willreveal what teachers assumed as inquiry-based teachingwithin their culture. It would be interesting to find outhow teachers’ assumptions about inquiry-based teach-ing were connected with their practice. Through theseanalyses, it may be possible to further suggest effectiveways to improve teacher education programs in bothcountries.

ReferencesAnderson, R. D. (2002). Reforming science teaching: What

research says about inquiry. Journal of Science Teacher

Education, 13(1), 1-12.

Bybee, R. (2000). Teaching science as inquiry. In J. Minstrell & E.

H. van Zee (Eds.), Inquiring into inquiry learning and teaching in

science (pp. 21-46). Washington, DC: American Association

for the Advancement of Science.

Colburn, A. (2004). Inquiring scientists want to know. Educational

Leadership, 62(1), 54-68.

Ertepinar, H. & Geban, O. (1996). Effect of instruction supplied

with the investigative-oriented laboratory approach on

achievement in a science course. Educational Research, 38(3),

333-341.

Krajcik, J., Blumenfeld, P., Marx, R., & Soloway, E. (2000).

Instructional, curricular, and technological supports for

inquiry in science classrooms. In J. Minstrell & E. H. van Zee

(Eds.), Inquiring into inquiry learning and teaching in science (pp.

283-315). Washington, DC: American Association for the

Advancement of Science.

Lotter, C. (2004). Preservice science teachers’ concerns through

classroom observations and student teaching: Special focus on

inquiry teaching. Science Educator, 13(1), 29-38.

Ministry of Education, Culture, Sports, Science, and Technology.

(2007). The Course of Study. Retrieved on October 26, 2007

from http://www.mext.go.jp/b_menu/shuppan/sono-

ta/990301c/990301d.htm

Ministry of Education, Culture, Sports, Science, and Technology.

(2008). Overview of Teacher Licensure System in Japan.

Retrieved on November 23, 2008 from

http://www.mext.go.jp/a_menu/shotou/kyoin/main13_a2.htm

National Center for Education Statistics. (2006). Teaching science in

five countries: Results from the TIMSS 1999 video study (NCES

Publication No. 2006-011). Washington, DC: U. S.

Government Printing Office.

Educational Background Survey Participants % (N) Observation Participants % (N)US (N=45) Japan (N=46) US (N=9) Japan (N=14)

Science and Education 44% (20) 37% (17) 56% (5) 50% (7)Science only 11% (5) 63% (29) 0% (0) 50% (7)Education only 40% (18) 0% (0) 33% (3) 0% (0)Neither Science nor Education 4% (2) 0% (0) 11% (1) 0% (0)

Note. The numbers in the parentheses show the actual numbers of respondents in the categories.

Table 3.Percentage of Teachers Who Have Academic Degrees in Science and/or Education in the USA and Japan

8


National Center for Education Statistics. (2008). Instructional prac-

tices and time spent teaching science in eighth grade, by country:

2003 (NCES Table No. 399). Retrieved on November 18,

2008 from

http://nces.ed.gov/programs/digest/d07/tables/dt07_399.asp

National Research Council. (1996). National science education stan-

dards.. Washington, DC: National Academic Press.

National Research Council. (2000). Inquiry and the National Science

Education Standards: A guide for teaching and learning.

Washington, DC: National Academic Press.

Ogura, Y. (2004). Video studies on Japanese science lessons and science

lessons in 4 other countries. Report at National Institute of

Educational Policy in Japan, Project Report #12308007 (in

Japanese).

Reiff, R. (2002, January). If inquiry is so great, why isn’t everyone

doing it? Paper presented at the Annual International

Conference of the Association for the Education of Teachers in

Science, Charlotte, NC.

Shymansky, J. A., Kyle, W. C. Jr., & Alport, J. M. (1983). The

effects of new science curricula on student performance.

Journal of Research in Science Teaching, 20(5), 387-404.

Simmons, P. E., Emory, A., Carter, T., Coker, T., Finnegan, B.,

Crockett, D., et al. (1999). Beginning teachers: Beliefs and

classroom actions. Journal of Research in Science Teaching,

36(8), 930-954.

Smolleck, L. D., Zembal-Saul, C., & Yoder, E. P. (2006). The devel-

opment and validation of an instrument to measure preservice

teachers’ self-efficacy in regard to the teaching of science as

inquiry. Journal of Science Teacher Education, 17, 137-163.

Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using video sur-

veys to compare classrooms and teaching across cultures:

Examples and Lessons from the TIMSS Video Studies.

Educational Psychologist, 35(2), 87-100.

Von Secker, C. (2002). Effects of inquiry-based teacher practices

on science excellence and equity. The Journal of Educational

Research, 95(3), 151-160.

Item Statement CategoryQ2 I think it is important that the use of inquiry is emphasized in middle-school science education.

Q3 I am satisfied with the extent to which I use inquiry in my science classes.Q4 Students learn scientific concepts with deeper understanding through inquiry-based lessons than

through lecture type lessons.Q5 I prefer to teach lecture type lessons rather than to teach lessons in which

students engage in hands-on activities.Q6 In my science classes, it is important to me that students acquire the skills for

conducting scientific investigations.

Q7 In my science classes, it is important to me that students have opportunities to work on open-ended questions whose answers are not known.

Q8 In my science classes, it is important to me that students have opportunities to discover scientific concepts by engaging in hands-on activities.

Q9 It is important to me that students have opportunities to investigate their own scientific questions in my science classes

Q10 It is important to me that students have opportunities to generate their own hypotheses in my science classes.

Q11 It is important to me that students have opportunities to collect data to test their hypotheses in my science classes.

Q12 Variations in students’ experimental results are an important element of discussion in my science classes.

Q13 It is important to me that individual students formulate their explanations from evidence in my science classes.

A. General preference

AA

A

B. Personal evaluationtowardinquiry-orientedbehaviors

B

B

B

B

B

B

B

APPENDIXTable A1.Survey Instrument and Categories for Measuring Middle-School Sicence Teachers’ Attitudes Toward Inquiry-Based Teaching.

9


Q14 It is important to me that individual students generate their explanations through discussions in my science classes.

Q15 In my science classes, it is important to me that students’ explanations are consistent with data they have collected through experiments or observations.

Q16 In my science classes, it is important to me that students test their explanations against descriptions found in the textbooks or other reliable sources.

Q17 It is important to me to guide the whole class to achieve understanding of scientific concepts that I teach in a lesson.

Q18 It is important to me to help students make connections between outcomes from their hands-on activities and scientific concepts in my science classes.

Q19 It is important to me to explicitly state the scientific concepts I was trying to teach in the lesson at the end of the lesson.

Q20 It is important to me to help students make connections between scientific concepts and real-life situations during the course of a science unit.

Q21 It is important to me that students learn to communicate their findings in my science classes.Q22 It is important to me that students learn to justify their explanations in my science classes.Q23 It is important to me to follow suggestions described in the teacher guidebooks in my science lessons.Q24 In my science classes, it is important to me to offer help to individual students that

matches their level of understanding of scientific concepts.Q25 It is important to me that students are encouraged to explore their own ways of

investigating scientific questions in my science classes.Q26 It is important to me that students are able to recognize the strengths and

weaknesses of the methods they took to explore scientific questions in my science classes.Q27 I feel comfortable in handling students’ questions that I cannot answer immediately.

Q28 I use students’ questions which I cannot answer as an opportunity for further student explorations.Q29 I feel comfortable helping students when they have trouble with their experiments or observations.Q30 Curricular demands placed by high stakes exams administered by the

state make it difficult for me to use scientific inquiry in my science classes.Q31 Students get out of control easily in an inquiry-based classroom.Q32 Assessment of student learning outcomes in inquiry-based teaching is difficult.Q33 Teaching inquiry-based lessons takes too much preparation time.Q34 Teaching inquiry-based lessons takes too much instructional time.Q35 Managing materials for hands-on activities is an issue for me in inquiry-based teaching.Q36 I feel comfortable teaching inquiry-based science lessons to middle-school students.Q37 Inquiry-based teaching is feasible in my middle-school science classes.

Q38 What are your thoughts and feelings about inquiry-based teaching? Please write freely.

B

B

B

B

B

B

B

BBBB

B

B

D. Teacher’sself-efficacy

DDC. Obstacles

C.C.C.C.C.D.D.

Please provide the following information by marking ? in the appropriate box.Q39 Are you male or female?

Male Female

Q40 How old are you? 20s 30s 40s 50s 60s

Q41 How many years have you been teaching? Less than 1 1-2 years 3-5 years 6-10 years More than 10 years

Q42 What grade(s) are you currently teaching? 6th 7th 8th Other: _____________________________

Q43 What subject(s) are you teaching? Science Mathematics Both science and mathematics Other: ______________________________

Q44 What are the academic degrees you earned during the course of your higher education and what were the fields of study?

1st Degree: _________________________ Field of study: _______________________________________

2nd Degree: _________________________ Field of study: _______________________________________

If you have more than 2 degrees, please indicate degree and field: _______________________________________Q45 What area(s) and level(s) are you certified to teach?

Area: _________________________ Level(s): _______________________________________

If you are certified to teach more than one area, please specify: _______________________________________

11


ABSTRACTThis paper addresses the problem of science literacy,focusing specifically on students’ lack of understand-ing about the nature of science. In February of 2009,re s e a rch will be conducted to determine ifQuarkNet’s Particle Physics Masterclass provides afruitful program for students to learn about thenature of science. The Masterclass is a national pro-gram where students come to a local area researchinstitute and interact with particle physicists throughlectures, informal discussions, and work together toanalyze real particle physics data. This paper high-lights the research question and methodology for theFebruary, 2009 study.

INTRODUCTIONOne of the major goals of K–12 science education is

to create scientifically literate citizens. The AmericanAssociation for the Advancement of Science (AAAS)suggests that a scientifically literate citizen is someonewho: understands major principles and concepts in sci-ence; understands the nature of science; recognizes theconnections between science, mathematics, and tech-nology; maintains a positive attitude towards being ableto understand science; and is able to use scientificknowledge and habits of mind for solving personal andsocial purposes (1995). AAAS further delineates the rea-sons for cultivating a scientifically literate population bysuggesting that science provides the knowledge base forcitizens to develop solutions for local and global prob-lems, that science creates a respect for nature, and thatscience fosters the habits of mind needed in life whenmaking informed, critical decisions. Most importantly,the possible future life-enhancing benefits of scienceand technology may never occur unless there is a newgeneration of scientists to carry on the scientific endeav-or along with a scientifically literate population whounderstands the decisions that need to be made to sup-port scientists’ work (AAAS, 1989).

There are many scientific issues that have turnedinto controversial social and political issues. Cloning,stem cell research, pesticide use, global warming, andevolution are all examples of issues that demand anunderstanding of scientific evidence. All citizens have aresponsibility to be able to understand and interpret theclaims made by both scientists and politicians (NationalResearch Council, 1996). As one examines the meaningand importance of being scientifically literate, the case

for educating the public on the science that their taxdollars support is strengthened. Science educators haveargued that achieving the goal of science literacy amongthe populous is greatly aided by the teaching of theNature of Science (referred to as NOS) throughout theK–12 curriculum (Smith & Scharmann, 1999).

During the last three years, one attempt to increaseunderstanding of NOS for high school physics studentshas been through a program known as the ParticlePhysics Masterclass. The Masterclass allows students theopportunity to interact with physicists at a researchinstitution where students attend lectures, tour the lab-oratory, work alongside physicists to analyze real parti-cle physics data, and communicate their findings toeach other in a conference-type atmosphere .Specifically, the Particle Physics Masterclass seeks toincrease interest in science and to demonstrate theprocesses of real-life scientific research (Johansson,Kobel, Hillebrandt, Engeln, & Euler, 2007).

PURPOSE OF THE STUDYThe purpose of this study is to evaluate the 2009

U.S. Particle Physics Masterclass to determine if it isindeed an effective program for student learning ofNOS. Although research has been conducted regardingstudent learning of NOS (Irwin, 2000; Scharmann,Smith, James, & Jensen, 2005), and anecdotal claimshave been made about science outreach programs aid-ing in the learning of NOS (Croft, 1999; Abraham,2002; MJ Young & Associates 2008, July), rigorousresearch has not been conducted to determine if studentexposure to working scientists can aid in the learning ofNOS. Specifically, research has not been conducted todetermine if the 2009 U.S. Particle Physics Masterclassis an effective program for student learning of NOS.

RESEARCH QUESTIONThe following research question will be examined

in an effort to determine if the 2009 U.S. ParticlePhysics Masterclass is an effective program for studentlearning of NOS.

As a result of attending the 2009 U.S. ParticlePhysics Masterclass at Fermilab, do secondaryscience students change their views about thenature of science?

Enhancing the Teaching of the Nature of ScienceMichael J. WadnessMedford High School

12


SIGNIFICANCE OF THE STUDYAlthough much can be learned from science text-

books and classroom laboratory activities, these twoapproaches focus almost exclusively on the acquisitionof content knowledge and skills. Learning scienceknowledge and skills may not be enough for today’s stu-dents to understand and appreciate the scientific enter-prise. What is generally studied in high school sciencec l a s s rooms are the finished products of science.Although science content does provide the foundationfor science literacy, it may not be enough for students tounderstand the basis for which scientists make claims.Unfortunately, due to the social controversies surround-ing the teaching of Evolution and the Big Bang Theory,there is a need to teach students what scientific knowl-edge represents and how scientific knowledge is devel-oped. In order for the problem of science literacy to beaddressed, there is a need for the development of avail-able programs for the instruction of NOS. Research hasshown that although traditional classroom activitiesteach students about scientific knowledge, they do littleto convey the socially constructed nature of science(Lederman, 1999). The information eventually printedin textbooks almost never includes the altern a t i v ehypotheses, the anomalous data, the dissenting views,the many confirming studies, and the social processesthat occur in discussing the meaning of the observa-tions. What is primarily taught in science classroomsare the final products of the scientific process and notNOS (Abd-El-Khalick, Bell, & Lederman, 1998; Abd-El-Khalick, 2005). Although there are a number of pro-grams involving scientists in K–12 education, very fewof them have been formally evaluated to determine ifthey provide an adequate learning of NOS. Therefore,this study is significant because if student learning ofNOS occurs during the 2009 U.S. Particle PhysicsMasterclass then it will suggest that other science out-reach programs may also be a useful context for studentlearning of NOS.

OPERATIONAL DEFINITIONSThe following operational definitions will be used

in this study:

Nature of ScienceAlthough there are many variations on the tenets of

the nature of science, the tenets used in this study willbe drawn upon NSES (NRC, 1996), Project 2061Science for all Americans (AAAS, 1989), and from Abd-El-Khalick, Bell, & Lederman (1998). The nature of sci-ence describes the epistemology of scientific knowledge

and of the scientific enterprise itself. The following willbe defined as the tenets of the nature of science.

The Nature of Science includes the nature of scientif-ic knowledge and the nature of the scientific enterprise.

1. Scientific knowledge is tentative yet stable.a. Scientific knowledge is subject to on-going

testing and revision.b. Scientific knowledge occasionally undergoes

major revolutionary changes.c. Scientific knowledge often encounters minor

changes.d. New theories that conflict with curre n t

accepted theories may encounter harsh criti-cism.

2. Scientific knowledge is based on observationsthat are indirect, subjective, and theory laden.

a. Scientific knowledge attempts to explain andpredict observations.

b. Scientific knowledge is subject to observa-tions.

c. Until consensus, conflicting theories canresult from identical data.

d. The goal of observations is to be objective yetit is impossible to be completely objective.

e. The interpretation and measurement ofobservations is based on existing theories.

f. Personal, cultural, social, and metaphysicalfactors can influence the interpretation ofdata.

3. Science is a social, human enterprise withstrengths and weaknesses.

a. Scientists work alone and in teams but neverin isolation.

b. Scientists publish and evaluate each other’swork (peer review) and suggest alternativeexplanations for observations.

c. New ideas often encounter harsh criticism.d. Scientific knowledge is subject to social and

cultural bias.e. Research is guided by the availability of fund-

ing.f. Scientific knowledge is constructed by people

who are not significantly different from thegeneral population.

4. Scientific knowledge involves human inference,imagination, and creativity.

5. There is no universal scientific method.6. Scientific knowledge incorporates an under-

standing of the function and re l a t i o n s h i pbetween hypotheses, models, laws, and theories.

13


Learning of NOSThe learning of NOS will be defined as an increase

in score from the pre- and posttest instruments usedduring this study.

REVIEW OF THE LITERATUREAlthough there is disagreement among philoso-

phers of science concerning NOS (Alters, 1997), someof the commonly agreed upon tenets include: the use ofevidence; the stability and tentativeness of scientificknowledge; that science is a human enterprise withstrengths and weaknesses; the use of imagination andcreativity by scientists; that science is a complex socialactivity; that science is theory laden and incorporates anunderstanding of the functions and relationships ofmodels, laws, and theories (AAAS, 1989; Abd-El-Khalick & Akerson, 2004; NRC, 1996). Essentially,NOS addresses how scientific knowledge is developed,what scientific knowledge represents, and the recogni-tion of science as being a social, human enterprise.

Explicit Instruction in NOSResearch has demonstrated that many students and

preservice teachers with backgrounds in science whohave not had explicit instruction in NOS hold a view ofNOS that can be considered far from adequate (Abd-El-Khalick, 2005; Samarapungavan, Westby, & Bodner,2006; Scharmann et al., 2005; Sere et al., 2001; Tao,2003). These studies suggest that a comprehensive viewof NOS may not occur implicitly through the instruc-tion of science content. To test this hypothesis, Irwin(2000) used historical cases in teaching science to 14year-old students in London, UK. Irwin taught two sci-ence classes consisting of relatively equivalent students.These two classes were identified as a control group anda treatment group. Both groups were given instructionon atoms and the periodic table in the same number oflessons. However the treatment group included NOSinstruction using a historical context. After instruction,Irwin reported that there was no statistically significantdifference in the mean scores for the two groups’ under-standing of science content, but following focus groupinterviews, he determined that members of the treat-ment group used historical examples to demonstratetheir understanding of many of the tenets of NOS, whilethe control group maintained an inadequate view ofNOS. Although this study did not include a large, ran-dom sample of students it still demonstrates severalpoints. These points include: an understanding of NOSwas not implicitly conveyed through the instruction ofscience content alone in the control group; NOS

instruction was in fact explicitly conveyed in the treat-ment group by providing a historical context; becauseboth groups received the same instruction time, NOScan be integrated within the historical context to teachcontent without a sacrifice to classroom learning time;and the inclusion of explicit NOS instruction did notaffect the students’ understanding of content comparedto the control group. Therefore, this study demonstratesan explicit method for addressing the problem of stu-dents’ understanding of NOS.

To further investigate explicit methods for NOSi n s t ruction, some re s e a rchers (Abd-El-Khalick &Akerson, 2004; Scharmann, et al., 2005; Tsai, 2006)have drawn upon conceptual change theory (Posner,Strike, Hewson, & Gertzog, 1982). Although discussionof conceptual change theory will not be addressed inthis literature review, the main points refer to studentsresisting accommodation of new concepts until there issignificant dissatisfaction with their preconceptions. Inaddition, the new concepts must be considered intelli-gible, plausible, and fruitful. Conceptual change theoryalso addresses factors that affect students’ resistance todiscarding preconceptions within their conceptual ecol-ogy, which includes prior experiences and epistemolog-ical views. Therefore, researchers suggest that studentswill only undergo conceptual change to accommodate amore comprehensive view of NOS if their original viewsare challenged within the context of explicitly designedinstruction. Scharmann et al. (2005) reported in theirstudy of undergraduate students in their last semesterbefore student teaching, that when including explicitNOS instruction within a context and providing severalopportunities for reflection and disequilibrium, stu-dents gained a more comprehensive view of NOS. Thestudents were enrolled in a course that emphasizedstudying NOS. Throughout the course, students weregiven multiple opportunities to discuss issues in NOSwithin a context of historical examples. Although thestudents were undergraduates, it is possible that theresults may also be applied to K–12 education.

It is important to recognize the use of good peda-gogy in NOS instruction. Tao’s study (2003) used his-torical science stories to teach NOS without the use ofdisequilibrium and reflection. The researcher reportedvery little, if any, improvement between students’ pre-and posttests. In fact, it was determined that many stu-dents used the stories to confirm and reinforce inade-quate views of NOS. The students did engage in dis-course; however, the co-construction of ideas did notfoster conceptual change. This study suggests that stu-dents have entrenched ideas concerning NOS that needto be overcome through skilled pedagogy. Therefore,

14


although the previously mentioned study by Irwin(2000) demonstrated successful NOS instru c t i o nthrough a historical context, Tao’s study suggests that bymerely exposing students to stories without properexplicit instruction, students may not go through theprocess of conceptual change and the problem of stu-dents’ inadequate views of NOS will not be properlyaddressed.

One way to foster conceptual change for NOS maycome from introducing high school students to contem-porary scientists and exposing them to their work. In astudy by Abraham (2002), students worked alongsidescientists in a summer internship. Abraham reportedthat students had a positive change in their views of sci-ence and scientists. The researcher claims that studentssaw scientists as real people and recognized science as ahuman endeavor as a result of this experience.

Research suggests that other programs in which stu-dents interact with scientists have also led to a betterunderstanding of NOS. The programs in which theseclaims have been made include the exposure of studentsto scientists through research programs, mentor pro-grams, classroom inquiry projects, guest lectures, andlaboratory tours (Sabo, Sarquis, & Ennis, 1997; Croft,1999; Berkson & Harrison, 2002; Siegel, Mlynarczyk-Evans, Brenner, & Nielsen, 2005). Although not all ofthe tenets of NOS described were addressed by theabove science outreach programs, many have discussedhow the interactions between students and scientistsaffected how students viewed science as a human enter-prise and have changed their view of the nature ofobservations.

Unfortunately many of these outreach programshave questionable evaluation methods, therefore pre-senting the need for more rigorous research. However,the common ingredient in the above programs is thatthey provide a context for students to experience first-hand the tenets of NOS. Because the Particle PhysicsM a s t e rclass provides multiple contexts for explicitlearning of NOS, it may provide a suitable vehicle forstudents to change their view of NOS.

The Particle Physics Masterclass is a program inwhich students spend one day at a research facilityinteracting with particle physicists to learn about parti-cle physics. This is accomplished primarily through lec-tures and an inquiry-based activity where students ana-lyze real data alongside physicists and present theirresults in a scientific conference environment. In addi-tion, students will tour the research facility to learn howcurrent data is collected and analyzed. The ParticlePhysics Masterclass is a program run by QuarkNet, inassociation with Fermilab. QuarkNet is a program par-

tially funded by the National Science Foundation andthe U.S. Department of Energy. The goal of QuarkNet isto provide particle physics educational outreach to sec-ondary science students.

The Particle Physics MasterclassThe Particle Physics Masterclass began in 1997 in

the U.K. and has expanded to 18 European countriesadministered at 58 universities and research centers,attracting over 3,000 students (Johansson et al. 2007).In 2005, the Masterclass was administered by theEuropean Particle Physics Outreach Group (EPPOG) inan effort to provide opportunities for students to partic-ipate in an authentic research process to improve theirunderstanding of scientific research. Specifically, theaims of the Masterclass were to “stimulate interest in sci-ence, demonstrate the scientific research process, makedata of modern particle physics experiments availablefor students, and to explore the fundamental forces andbuilding blocks of nature” (p. 637).

To accomplish these goals, students spent the day ata university or research center attending an introducto-ry lecture to learn about particle physics and how par-ticle physicists analyze data from high-energy experi-ments. Following the lectures, students analyzed datafrom CERN’s Large Electron Positron Collider, whichcollected data in the 1990’s outside of Geneva,Switzerland. The students analyzed data alongside thephysicists in an attempt to understand Z-decays, usinga web-based program similar to what physicists actual-ly use. At the conclusion of the day, an internationalvideoconference occurred between four to six sites at atime to collaborate the students’ data. Physicists atCERN moderated the videoconference in an attempt toprovide an environment similar to current internationalcollaborations carried out by research physicists.

To evaluate the effectiveness of the Masterclass, aquestionnaire was provided to the students at the con-clusion of the 2005 Masterclass. Unfortunately, theauthors do not discuss their instrument in detail ormake claims of validity or reliability. The sample sizeconsisted of 1,291 high school students from 18European countries with most students being between16 and 19 years old. The students’ understanding ofparticle physics before the program was smoothly dis-tributed with one third of students reporting that theyknew little to nothing about particle physics, one thirdclaiming to know something about particle physics, andone third of the students claiming to know very muchabout particle physics. Of the sample, 69% of the stu-dents were male and 31% were female. The majority ofthe students were either nominated by their teacher to

15


attend or attended on their own initiative. This impliesthat the sample may not be representative of the gener-al population.

The authors claim that 82% of the students report-ed liking the Masterclass or very much liking it and didnot find a correlation between previous knowledge ofparticle physics and enjoyment of the program. In addi-tion, the authors found no significant difference inresponses between males and females. Therefore, theauthors claim that an understanding of particle physicswas not necessary to enjoy the program and was notgender biased. The majority of the students reportedthat the Masterclass increased their interest in physicsand their desire to learn more about particle physics. Inregards to NOS, 70% of the students claimed that theMasterclass helped them to learn much or very muchabout the organization of scientific research. In a corre-lation of the students’ replies, a weak correlation of .15was found between students’ appreciation ofMasterclass and learning about the organization ofresearch. Interestingly, a moderate correlation of .510was found between the appreciation of Masterclass andfinding the lectures interesting. However, 68% of thestudents reported the lectures to be easy or very easy tounderstand. This suggests that although the focus of thedesign of the Masterclass is in the data analysis portion,the design of the lectures may hold a significant role instudents’ reporting their appreciation of the Masterclass.

Although the above study only reports on the stu-dents’ perceptions in regards to how Masterclass helpedthem to learn about the organization of scienceresearch, this study is significant because it is related tothe nature of science. It would be interesting to investi-gate what the students’ understanding of the organiza-tion of science was before and after the Masterclass, inaddition to other tenets of NOS.

In a related study, a U.S. Masterclass was conduct-ed in 2008. The U.S. Masterclass was conducted byQuarkNet, a program partially funded by the NationalScience Foundation and the U.S. Department of Energy.QuarkNet is a particle physics educational outreachprogram for high school physics teachers. The format ofthe U.S. Masterclass was similar to the Euro p e a nMasterclass with the exception of high school physicsteachers conducting an hour or more of preparationwith their students to provide a basic familiarity of par-ticle physics terminology. In addition, U.S. studentswere given a tour of a research lab on the Masterclassday (MJ Young & Associates, 2008). The U.S.M a s t e rclass had similar goals to the Euro p e a nMasterclass with the addition of a broader goal of sci-ence literacy. An evaluation of the program was con-

ducted by MJ Young and Associates (2008). The pur-pose of the evaluation was to determine if the goals ofthe Masterclass were met, to compare U.S. studentresponses to European student responses from theabove mentioned 2005 study, and to determine if theroles of the teachers created any difference in studentresponses. The instruments used in this study were aquestionnaire identical to the one used by the abovementioned 2005 study, a pre- and posttest to assess stu-dents’ gains in content knowledge about part i c l ephysics and science literacy in the areas of data organi-zation, analysis, and interpretations, and a transmittalform for teachers to report on the students contextualdata. The pre- and posttest was reviewed for face valid-ity but was never piloted. Analysis of the data demon-strated that some test items might have been too easy,too difficult, or too confusing.

The sample of students in the U.S. was similar tothe European students. In fact, the ratio of males tofemales was identical to the European numbers, 69%and 31% respectively. The previous knowledge of parti-cle physics was also reported in a similar distribution ofapproximately one-third of the students claiming tohave little or no knowledge of particle physics, one-third having some knowledge, and one-third knew alot. Slightly more than half of the students reported hav-ing taken an advanced science course and 75% report-ed taking an advanced math class. Therefore, the sam-ple is not representative of the general population.

The questionnaire had 91 students reporting from 6of the 13 sites. The results of the questionnaire statedthat students reported a strong agreement for wantingto learn more about physics, had an increased interestin physics, and learned how science research is organ-ized and carried out as a result of the Masterclass. Theseresults strongly support the European study’s results.However, the responses of the students are questionabledue to the fact that, similar to the European study, thequestionnaire was administered to the students at theconclusion of the Masterclass and that the questionnairedid not significantly probe the students’ conceptions.

The pre- and posttest used in this study includeditems related to science literacy including process itemsand items about how science is constru c t e d .Unfortunately, after analyzing the data, it was deter-mined by the evaluators that some of the questions weretoo easy, too difficult, or confusing. However, otheritems involving content demonstrated students havingstatistically significant gains re g a rdless of pre v i o u sknowledge. Therefore, the evaluators concluded that allstudents did benefit in some manner from theMasterclass. It was also noted that a correlation similar

16


to the European study arose between students’ enjoy-ment of Masterclass and their understanding of the lec-tures. This study also suggests that the lectures mayprovide a significant role in the students reporting theirappreciation of Masterclass. Unfortunately, the evalua-tors reported that there was not enough data collectedto determine how the teachers’ instruction before theMasterclass affected the outcomes of Masterclass.

The significance of the above studies suggests theneed for further investigation of the Masterclass activity.The studies suggest that students report a greaterunderstanding of how science research is organized, amain item related to NOS. However, there remains aneed to determine what students’ specific understand-ing of NOS is both before and after Masterclass beforeconcluding whether or not Masterclass provides a con-text for increased understanding of NOS.

RESEARCH HYPOTHESISThe research hypothesis for this study is that sec-

o n d a ry science students attending the 2009 U.S.Particle Physics Masterclass at Fermilab will changetheir views of NOS towards the accepted view asdescribed in this study.

RATIONALETo examine the rationale for this hypothesis, it is

i m p o rtant to briefly summarize the re s e a rch thatdemonstrates successful learning of NOS. The researchreviewed suggested that students are more likely tochange their view of NOS towards the comprehensiveview, as described in this paper, when explicitly taughtwithin a specific context. The Particle PhysicsMasterclass at Fermilab provides many experiences,which may provide a context for learning NOS. Theseinclude: a tour of Fermilab, introductory lectures with-in a historical context in particle physics, a data analy-sis activity, interactions with working research physi-cists, and a conference for students to present theirresults. Many of the individual ingredients of theMasterclass may act as a treatment for NOS. One fruit-ful context for NOS instruction has been through theuse of historical examples (Irwin, 2000; Scharmann etal., 2005). Although the Masterclass does not focus onexplicit teaching of NOS through historical examples,the introductory lessons do explicitly include the tenetsof NOS and present particle physics content andmethodology in a historical context. Therefore, theintroductory lessons may promote a better understand-ing of NOS. Research also suggests that programs in

which students interact with working scientists havealso led to a better understanding of NOS. The pro-grams reviewed in which these claims have been madeto affect the students’ views of NOS include the expo-sure of students to scientists through research pro-grams, mentoring programs, classroom inquiry proj-ects, guest lectures, and laboratory tours (Sabo et al.,1997; Abraham, 2002; Berkson & Harrison, 2002;Siegel et al., 2005). Due to the Masterclass being a com-bination of fruitful experiences for NOS instruction, it ispossible that the Masterclass will also be fruitful forNOS instruction.

Although not all of the tenets of NOS described inthis paper were addressed by the above science out-reach programs, many have discussed how the interac-tions between students and scientists affect how stu-dents viewed science as a human enterprise. In addi-tion, the above studies have also suggested that studentsinvolved in inquiry projects with scientists have alsochanged their view of the nature of observations.Although a number of the programs involving scientistsin secondary education have questionable re s e a rc hmethods, the common ingredient to the above pro-grams is that they provide a context for students toexperience the tenets of NOS firsthand. Because theParticle Physics Masterclass provides multiple contextsfor the explicit learning of NOS, it may provide a suit-able vehicle for students to change their view of NOS.

PARTICIPANTSThe students participating in this study will be

attending the 2009 U.S. Particle Physics Masterclass atFermilab, outside of Chicago. This Masterclass will con-sist of approximately 100 students. The type of sam-pling for this study will be a nonrandomized conven-ience sample. The Fermilab site will be subdivided intofive subgroups consisting of approximately 20 studentseach for the data analysis and presentation of results.The sampling bias present is that the sample is not rep-resentative of the population due to the fact that someof the students attending Masterclass have been nomi-nated by their teacher and some have volunteered. Thismight indicate that the students may be more advancedthan the average high school student.

METHODOLOGY AND ANALYSISTo determine if the 2009 U.S. Particle Physics

Masterclass is an effective program for student learningof NOS, a mixed methodology will be employed. Theprimary methodology will be based on a quasi-experi-

17


mental time series design (Campbell & Stanley, 1963;Charles & Mertler, 2002). This design includes a single,nonrandomized group of participants observed multi-ple times over a period of time and exposed to a treat-ment in an interval between two of the measurements.The design suggests that if a statistically significantchange occurs after the introduction of the treatment,then the treatment may be judged to be the cause of thatchange. This design is diagramed as:

O O O X1 O O O

In this study however, due to the limitations of theavailability of the participants, there will not be as manyobservations as implied by the above design. Instead,the participants will be observed once before the treat-ment and twice after the treatment. Therefore, thedesign of this study is diagramed as:

O X1 O O

This variation on the quasi-experimental time seriesdesign has been used in studies by Chua (2008), Ertenand Tekin (2008), and Madden (2006).

The instrument will be a Likert-type survey meas-uring student understanding of the tenets of NOS. Ifstudents show an increase in their score on the instru-ment then it can be concluded that learning of NOSoccurred. In addition, focus group interviews will occurto further probe students’ understanding of NOS andwhat specific activities had the greatest impact on theirunderstanding of NOS. Lastly, portions of theMasterclass day will be videotaped to analyze interac-tions between scientists and students in order toobserve evidence of student learning of NOS. Theanalysis will consist of Case II dependent means, one-tailed t-tests between the means on the instrumentdesigned for this study, and grounded theory methodsfor the focus group interviews.

InstrumentationThe instrument under development for this study is

a Likert-type survey with open-ended questionsdesigned to evaluate students’ understanding of NOS(see Appendix A). The majority of the items on theinstrument were adapted from two different pre-exist-ing instruments by Liang et al. (2008) and Chen (2006).These instruments have previously been tested forvalidity and reliability. Due to the specific definition ofNOS used for this investigation, questions were eitherdiscarded, reworded or added. In order to test for valid-ity and reliability, the instrument used for this study willbe piloted with local area high school students.

Focus Group InterviewsThe focus group interviews will be conducted and

analyzed using grounded theory methods as describedby Charmaz (2006). Approximately 10 students will beselected to participate in the focus group interview. Thepurpose of the interviews will be to provide a greaterunderstanding of students’ views of NOS towards theend of the Masterclass and how the Masterclass wasassociated with that view of NOS. A portion of the ques-tions are designed to attempt to elicit if and how theMasterclass has changed the participant’s view of sci-ence. The remainder of questions will be used to deter-mine if participants can use specific examples from theMasterclass in expressing their current view of NOS.The majority of the questions are based on the open-ended questions from the survey. However, these inter-view questions ask participants to use specific examplesfrom the Masterclass as examples to elaborate their viewof NOS (Appendix B).

LIMITATIONS OF THE STUDYOne significant limitation of this study includes the

bias whereby the researcher who is evaluating theMasterclass is also involved with the planning andimplementation of the Masterclass. Another limitationof the study is that the students participating in thisstudy are not representative of the general population.As previously discussed, the students participating fromthe Chicago area have volunteered or have been nomi-nated by their teacher. The nonrandomized conven-ience sample creates a significant limitation to thisstudy. These students are more likely to be higherachieving students who have a greater interest in sci-ence. Thus, the largest threat to the external validity ofthis study is population validity. Therefore, it will bemore difficult to generalize the results of this study tothe general population. Although many of the threats tointernal validity are addressed by the design of thisstudy, the threats of history, fatigue, and regressionremain.

CONCLUSIONThe purpose of this study is to determine if as a

result of attending the 2009 U.S. Particle PhysicsMasterclass at Fermilab, do secondary science studentschange their views about the nature of science. Thea ff i rmative re s e a rch hypothesis is based on theMasterclass being a combination of programs that haveclaimed to be successful for student learning of NOS. Inaddition, the Masterclass functions as an explicit expe-

18


rience for student learning of NOS. The methodologyemployed in this study is a mixed methodology, whichincludes a variation of a quasi-experimental time seriesdesign and focus group interviews. The analysis willconsist of Case II dependent means, one-tailed t-testsbetween the means on the instrument designed for thisstudy, and grounded theory methods for the focusgroup interviews. Although many of the threats to inter-nal validity are controlled, history, fatigue, and regres-sion are not. In addition, because the sample is not rep-resentative of the general population, the threat of pop-ulation validity exists towards external validity.

Adequate science literacy requires the developmentof programs for the instruction of NOS. Although thereare a number of programs involving scientists in K–12education, very few of them have been formally evalu-ated to determine if they provide adequate learning ofNOS. Therefore, this study is significant because if stu-dent learning of NOS occurs during the 2009 U.S.Particle Physics Masterclass then it will suggest that sim-ilar science outreach programs may also be a useful con-text for student learning of NOS.

ReferencesAbd-El-Khalick, F. (2005). Developing deeper understandings of

nature of science: The impact of a philosophy of science

course on preservice science teachers’ views and instructional

planning. International Journal of Science Education, 27(1), 15-

42.

Abd-El-Khalick, F., & Akerson, V. (2004). Learning as conceptual

change: Factors mediating the development of preservice ele-

mentary teachers’ views of nature of science. Science Education,

88, 785-210.

Abd-El-Khalick, F, Bell, L, & Lederman, N. (1998). The nature of

science and instructional practice: Making the unnatural nat-

ural. Science Education, 82(4), 417-436.

Abraham, L. (2002). What do high school students gain from

field-based research apprenticeship programs? The

Clearinghouse, 75(5), 292-32.

Alters, B. (1997). Whose nature of science? Journal of Research in

Science Teaching, 34(1), 39-55.

American Association for the Advancement of Science, Project

2061. (1989). Science for All Americans Online. Retrieved

November 1, 2004, from

http://www.project2061.org/tools/sfaaol/sfaatoc.htm

American Association for the Advancement of Science, Project

2061. (1995). Science for all americans summary. Retrieved

November 1, 2004 from http://www.project2061.org/publica-

tions/articles/2061/sfaasum.htm

Berkson, J., & Harrison, A. (2002). Refocusing natural resource

management. Journal of College Science Teaching, 31(7), 464-

469.

Campbell, D. & Staney, J. (1963). Experimental and quasi-experi-

mental designs for research. Chicago: Rand McNally.

Charles, C., & Mertler, C. (2002). Introduction to educational

research. Boston: Allyn and Bacon.

Charmaz, K. (2006). Constructing grounded theory: A practical guide

through qualitative analysis. Los Angeles: Sage.

Chen, C. (2006). Development of an instrument to assess views on

nature of science and attitudes toward teaching science.

Science Education, 90, 803-819.

Chua, S. (2008). The effects of the sustained silent reading pro-

gram on cultivating students’ habits and attitudes in reading

books for leisure. The Clearing House, 81(4), 180-184.

Croft, P. (1999, May). Assessing “the excitement of meteorology!”

for young scholars. Bulletin of the American Meteorological

Society, 80(5), 879-891.

Erten, I., & Tekin, M. (2008). Effects on vocabulary acquisition of

presenting new words in semantic sets versus semantically

unrelated sets. System, 36(3), 407-422.

Irwin, A. (2000). Historical case studies: teaching the nature of

science in context. Science education, 84(1), 5-26.

Johansson, K., Kobel, M., Hillebrandt, D., Engeln, K., & Euler, M.

(2007). European particle physics masterclasses make stu-

dents into scientists for a day. Physics Education, 42(6), 636-

644.

Lederman, N. (1999). Teachers’ understanding of the nature of sci-

ence and classroom practice: Factors that facilitate or impede

the relationship. Journal of Research in Science Teaching, 36(8),

916-929.

Liang, L., Chen, S., Chen, X., Kaya, O., Adams, A., Macklin, M., et

al. (2008). Assessing preservice elementary teachers views on

the nature of scientific knowledge: A dual-response instrument.

Asia-Pacific Forum on Science Learning and Teaching, 9(1), article

1.

Madden, C. (2006). Undergraduate nursing students’ acquisition

and retention of CPR knowledge and skills. Nurse Education

Today, 26(3), 218-228.

MJ Young & Associates. (2008, July). Particle Physics Masterclass

2008, QuarkNet (U.S.) Report, Draft. Tucson, AZ.

National Research Council. (1996). National science education stan-

dards. Washington, DC: National Academic Press.

Posner, G., Strike, K., Hewson, P. & Gertzog, W. (1982).

Accommodation of a scientific conception: Toward a theory

of conceptual change. Science Education, 66(20), 211-227.

19


Sabo, M., Sarquis, M., & Ennis, C. (1997). The pact ambassador

outreach program: More than just a bunch of “old white-

haired scientists.” Journal of Chemical Education, 74(4), 451-

452.

Samarapungavan, A., Westby, E., & Bodner, G. (2006). Contextual

epistemic development in science: A comparison of chem-

istry students and research chemists. Science Education, 90,

468-495.

Scharmann, L., Smith, M., James, M., & Jensen, M. (2005).

Explicit reflective nature of science instructions: Evolution,

intelligent design, and umbrellaology. Journal of Science

Teacher Education, 16, 27-41.

Sere, M., Fernandez-Gonzalez, M., Gallegos, J., Gonzalez-Garcia,

F., De Manuel, E., Perales, F., & Leach, J. (2001). Images of

science linked to labwork: A survey of secondary school and

university students. Research in Science Education 31, 499-523.

Siegel, M., Mlynarczyk-Evans, S., Brenner, T., & Nielsen, K. (2005,

October). A natural selection: Partnering teachers and scien-

tists in the classroom laboratory creates a dynamic learning

community. The Science Teacher, 42-45.

Smith, M., & Scharmann, L. (1999). Defining versus describing

the nature of science: a pragmatic analysis for classroom

teachers and science educators. Science Education, 83(4), 493-

509.

Tao, P. (2003). Eliciting and developing junior secondary students’

understanding of the nature of science through a peer collab-

oration instruction in science stories. International Journal of

Science Education, 25(2), 147-171.

Tsai, C. (2006). Reinterpreting and reconstructing science:

Teachers’ view changes toward the nature of science by cours-

es of science education. Teaching and Teacher Education 22,

363-375.

20


1. Scientific knowledge is subject to on-going revision.

2. Scientific explanations may be completely replaced by new explanations in light of new evidence.

3. Scientific explanations may be changed because scientists reinterpret existing observations.

4. Scientific explanations based on accurate experimentation will not be changed.

5. Scientists who present evidence that conflicts with established scientific explanations will encounter harsh criticism.

6. Some scientific explanations may encounter minor changes due to new evidence.

7. With examples, explain why you think scientific explanations do not change OR how (in what ways) scien-tific knowledge may change.

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

8. Scientists’ observations of the same event may be different because the scientists’ prior knowledge may affect their observations.

9. Scientists’ observations of the same event will be the same because scientists are objective.

10. Scientists’ observations of the same event will be the same because observations are facts.

11. Conflicting scientific explanations can result from identical observations.

12. Scientists’ interpretations of data are based on pre-existing scientific explanations.

13. The existence of atoms is known because atoms have been directly observed.

14. The existence of atoms is known because of indirect observations that can only be explained by atoms.

15. With examples, explain why you think scientists’ observations and interpretations of the same event couldbe the same OR different.

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

APPENDIX AMasterclass SurveyViews on Science

Directions: For the following questions please circle to the best of your ability your position on the followingviews on science. Use the code below in your responses. This survey may take you 20-30 minutes to complete.We thank you in advance for your time.

SD = Strongly Disagree D = Disagree U = Unsure A = Agree SA = Strongly Agree

21


For questions 16-20 use the following definition. Societal and cultural influences include: religious, political, eco-nomical, and other values and expectations influenced by ethnic and cultural backgrounds.

16. Scientific research is not influenced by society and/or culture because scientists are trained to conduct pure, unbiased investigations.

17. Cultural values determine what science is conducted and accepted.

18. Cultural values determine how science is conducted and accepted.

19. All cultures conduct scientific research the same way because science is universal and independent of society and culture.

20. With examples, explain how society and/or culture affect OR do not affect scientific research.

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

21. Most scientists work by themselves

22. Most scientists work in teams

23. It is important for a scientist(s) to be secretive about their work.

24. It is important for scientists to be critical of each other’s conclusions.

25. It is important for scientists to share their results.

26. With examples, explain why a scientist(s) would or wouldn’t share his/her results.

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

27. Scientists use their imagination when making a hypothesis.

28. Scientists use their imagination when designing an investigation.

29. Scientists use their imagination when they analyze data.

30. Scientists use their imagination when they interpret the meaning of data.

31. Scientists do not use their imagination because this can conflict with their logical reasoning.

32. Scientists do not use their imagination because this can interfere with objectivity.

33. Scientists use their creativity when making a hypothesis.

34. Scientists use their creativity when designing an investigation.

35. Scientists use their creativity when they analyze and interpret data.

36. Scientists use their creativity when they interpret the meaning of data.

37. Scientists do not use their creativity because this can conflict with their logical reasoning.

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

22


38. Scientists do not use their creativity because this can interfere with objectivity.

39. With examples, explain how and when scientists use imagination and creativity OR do not use imaginationand creativity in the scientific process.

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

__________________________________________________________________________________________

40. Scientists follow the same step-by-step scientific method.

41. When scientists use the scientific method correctly, their results are true and accurate.

42. There is no so-called the scientific method. Scientists use different types of methods to conduct scientific investigations

43. Experiments are the only means used in the development of scientific knowledge.

Use the following phrase to answer questions 44-47:In comparison to laws, theories have less evidence to support them.

44. Yes, theories are not as definite as laws.

45. Yes, if a theory stands up to many tests it will eventually become a law,therefore, a law has more supporting evidence.

46. Not quite, some theories have more supporting evidence than some laws.

47. No, theories and laws are different types of ideas. They cannot be compared.

48. Approximately how many scientists do you think fit the following characteristics?a. Male None Some Half Most Allb. Female None Some Half Most Allc. Young None Some Half Most Alld. Middle aged None Some Half Most Alle. Old None Some Half Most Allf. Black None Some Half Most Allg. White None Some Half Most Allh. Asian None Some Half Most Alli. Hispanic None Some Half Most Allj. Wear glasses None Some Half Most Allk. Facial hair None Some Half Most Alll. Clean shaven None Some Half Most Allm. Messy hair None Some Half Most Alln. Bald None Some Half Most Allo. Wears a lab coat None Some Half Most Allp. Wears a suit None Some Half Most All

49. What other characteristics or traits do you associate with scientists that may not be included in this list?

__________________________________________________________________________________________

__________________________________________________________________________________________

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

SD D U A SA

23


APPENDIX BFocus Group Questions

1. Many of you came to the Masterclass with a preconception about what science is and how it works. Since com-ing to Fermilab and experiencing the Masterclass, what surprised you about how science really works? Why?

2. Today you heard a few talks on particle physics, toured Fermilab, analyzed data, presented and discussed results,and interacted informally with particle physicists. Out of those items, what made the biggest impact how youview science really works? Why?

3. With examples from today’s Masterclass can you explain why you think scientific explanations do not changeOR how (in what ways) scientific knowledge may change?

4. With examples from today’s Masterclass can you explain why you think scientists’ observations and interpreta-tions of the same event could be the same OR different?

5. With examples from today’s Masterclass can you explain how society and/or culture affect OR do not affect sci-entific research.

6. With examples from today’s Masterclass can you explain why a scientist(s) would or wouldn’t share his/herresults?

7. With examples from today’s Masterclass can you explain how and when scientists use imagination and creativi-ty OR do not use imagination and creativity in the scientific process.

8. What types of methods do scientists use besides experimentation to develop new knowledge?

9. What is the difference between a theory and a law?

10. As a result of today’s Masterclass how has your view of a scientist changed?

24


ABSTRACTBullying at school has become a significant societalissue. Many students in American schools experiencevictimization during their school careers. Often, thevictims are the students who appear different or havea handicap. Bullies make victims feel unsafe in schooland as a result, the education of the victims may becompromised, as these students are less available forlearning.

This study focused on the potential link betweenperception of school climate and incidences of bully-ing at the junior high/middle school level. The pri-mary research question was what, if any, was the rela-tionship between group perception of the type ofschool climate identified by the surveyed participantsand their perceptions of incidences of bullying in aschool. One hundred two teachers/specialists alongwith186 7th-grade students and their parents fromfive schools located in the Commonwealth ofMassachusetts were surveyed.

The findings suggest that school climate cannot beused to predict whether a school has an issue withbullying. Therefore, the assumption cannot be madethat a school that is viewed by teachers/specialists,students, and parents as having a positive school cli-mate has fewer issues with bullying than does aschool with a more negative climate. However, thedata do suggest that the perceptions among all teach-ers/specialists, 7th-grade students, and their parentsoffer valuable information that educators, adminis-tration, and teachers can use to further address bul-lying issues in their schools.

A serious issue facing students in schools world-wide today is bullying.

For two years, Johnny, a quiet 13-year-old, wasa human plaything for some of his classmates.The teenagers badgered Johnny for money,forced him to swallow weeds and drink milkmixed with detergent, beat him up in the rest-room and tied a string around his neck, leadinghim around as a ‘pet.’ When Johnny’s torturerswere interrogated about the bullying, they saidthey pursued their victim because it was fun.(Olweus, 2001, p. 24)

The scenario described above is just one of manyinvolving a student who has suffered at the hands ofhis/her peers. Bullying is the occurrence in which “a stu-dent is repeatedly exposed to negative actions on thepart of one or more other students” (Olweus, 2001, p.

24). Several of the first studies on bullying were con-ducted outside of the United States in countries such asNorway, Great Britain, and Sweden (Stein, 2001). Forexample, in the early 1970s, Dan Olweus, a professorfrom Norway, first began studying the issue of bullyingin Scandinavian countries in response to the suicides ofseveral adolescent boys, which were believed to havebeen a reaction to victimization (Olweus, 2001). Inrecent years, however, researchers in the United Stateshave recognized that bullying has become a major issuein American schools (e.g., school shootings, suicides)and have attempted to identify why bullying occurs. Forexample, four years ago, the Indicators of Crime andSafety 2005 Report indicated that 28% of students, overa six-month period, reported having been bullied whileat school (Dinkes, Cataldi, Lin-Kelley, & Snyder, 2007).Educators, therefore, continue to seek and understandcontributing causes to the incidence of bullying.

RATIONALE AND IDENTIFICATION OF THERESEARCH QUESTION(S)

The purpose of this study was to determine if a rela-tionship existed between teachers/specialists, students,and parents’ perceptions of both climate and levels ofbullying in schools by surveying teachers/specialists,students, and parents in different schools. For example,did a school deemed welcoming and supportive bypolls of the parent community report incidences of bul-lying? Did a school in which teachers/specialists report-ed being supported by the administration and connect-ed to the parent community have issues with bullying?Did students who felt supported by the adults and theirpeers experience less bullying?

The hypothesis of this study was that in juniorhighs/middle schools that were identified byteachers/specialists, parents, and students as having amore positive school climate, the three groups wouldalso report fewer incidences of bullying than those injunior highs/middle schools with a negative school cli-mate. By establishing a relationship between group per-ceptions of school climate and incidences of bullying,the belief was that this information could then be usedto assist educators in reducing the percentage of bulliedstudents. By identifying differing interpretations of pos-itive and negative environments, educators may be able

A Study of the Relationship Between Group Perception of School Climate andIncidences of Bullying at the Junior High/Middle School Level

Kimberly R. DouglassNorthborough Public Schools, MA

25


to assess what changes can be made within the schoolsetting, and therefore to the school climate, to reduceincidences of bullying.

RELATIONSHIP TO THE LITERATUREBullyingCertain characteristics of a school can perpetuate

bullying. One study found that unmonitored halls leadto more incidences of bullying (DeVoe & Kaffenberger,2005). Another study found that when playgroundsand cafeterias were not well supervised, bullying wasmore likely to occur (Leff, Power, Costigan, & Manz,2003). When adult supervision is lacking, students arenot held accountable for their behavior. Additionally, ifthere is no observation of the students, the teachers, orsupervisors in the school will likely lack direct dataabout the occurrence of bullying. The Bradshaw study(2007) found that many teachers, while concernedabout the occurrence of bullying in their schools, actu-ally underestimate the issue. Staffs report that they areless likely to intervene if an incidence is reported tothem rather than when they directly observe the act(Bradshaw, Sawyer, & O’Brennan, 2007). Therefore,students who report bullying may find no assistance.Students, therefore, learn to avoid certain areas ofschools in an attempt to avoid victimization (e.g., stairs,parking lots) (DeVoe & Kaff e n b e rg e r, 2005).Supervision in schools appears to be critical to prevent-ing bullying.

Unfortunately, the victims are also usually the stu-dents least likely to respond to the bully, usually endur-ing the bully’s actions (Eslea et al., 2003). Some victimsmay believe that they deserve to be the target of a bully(Smith, Talamelli, Cowie, Naylor, & Chauhan, 2004).School then becomes a battleground of sorts for thesechildren, as they are either unable to defend themselvesagainst the bullies or resign themselves to the victimiza-tion. These students are also the ones who most oftenlack a strong support system which could assist them instopping the bullying (Smith et al., 2004). The researchsuggests that if peers stand by each other then a reduc-tion in the number of bullying incidences will mostlikely be observed (Eslea et al., 2003). The inability tostop the bullying may foster the perception in these vic-timized students of an overall school climate that is neg-ative.

Furthermore, as a result of bullying, as noted above,the victims become less engaged in learning as theirattention is diverted from academics to violence atschool (DeVoe & Kaffenberger, 2005). The studentsmust be on guard; they have to develop their own safe-

ty plan in order to be less vulnerable, which may fostera negative school climate for these students (Brown,Birch, & Kancheria, 2005). The victims want to escapefrom the abuse and eventually, school may lose its pur-pose for the victims.

School ClimatePerceptions of individual members and groups

within the school are also affected by school climate,which is defined as “the set of internal characteristicsthat distinguish one school from another and influencethe behavior of each school’s member” (Hoy & Miskel,2005, p. 185). Several studies have been conducted thatexamine the role of school climate on student perform-ance (Baker, 1999; Dobransky, 2004; Wilson, 2004).Peer relationships help students establish a sense ofbelonging (Wilson, 2004). Student-teacher relation-ships, parent-teacher relationships, and leader influenceall impact the overall morale of, or feelings associatedwith, the school building (Peterson & Deal, 1998).None of these relationships exist in isolation for all havethe potential to impact the overall climate of the school,either positively or negatively.

Research has found that students go to adults as alast resort (Rock, Hammond, & Rasmussen, 2002). Asense of community may be lacking for the students.

METHODParticipantsThe surveyed population consisted of four groups:

seventh grade students, their parents, school adminis-trators, and junior high/middle school teachers and spe-cialists. All participants were from the Commonwealthof Massachusetts at the time the surveys were adminis-tered.

One hundred eighty-six students from fivemiddle/junior high schools completed the surveys. Atthe time of the study, they were enrolled as seventhgraders. Two factors that were used in selecting thispopulation were that 1) research showed that bullyingbehaviors decrease as students enter the later juniorhigh/middle school years and 2) that the majority of thestudents were enrolled in their current school for atleast one school year and therefore had more experiencewith staff and fellow students (Bradshaw et al. 2007;Seals & Young, 2003).

The parent population was defined as the mother,father, or legal guardian of each seventh grade student.The teachers and specialists included sixth, seventh,and eighth grade teachers as well as special subject areateachers and school specialists. Special area teachers

26


included those who taught art, music, library, comput-ers, physical education, and technology education. Theschool specialists category included: school guidancecounselors, school psychologists, school adjustmentcounselors, social workers, special education teachers,behavior specialists, occupational therapists, speechand language therapists, and reading specialists.Building administrator referred to either principal orassistant principal.

Over 30 principals throughout the Commonwealthwere contacted by e-mail, seeking permission to con-duct student, teacher/specialist, and parent surveys intheir schools. Five schools granted permission toadminister surveys to all three groups.

School ProfilesSchool One. School One is located in central

Massachusetts. Six hundred sixteen students in gradessix through eight were enrolled in this suburban schoolat the time of the study (Massachusetts Department ofEducation, 2008). The principal is female. The schoolhas both an anti-bullying curriculum and policy. Aschool safety officer is available on an as-needed basis.

School Two. School Two is a suburban school locat-ed in central Massachusetts. Five hundred fourteen stu-dents in grades six through eight attended the school atthe time of the study (Massachusetts Department ofEducation, 2008). The principal is female. There is acurriculum that was designed and is implemented bythe school staff to address issues of bullying. There isalso a policy against bullying. There is no school safetyofficer.

School Three. School Three is also located in centralMassachusetts. Grades six and seven are housed in thissuburban school with an enrollment of 413 students atthe time of the study (Massachusetts Department ofEducation, 2008). The principal is female. The schoolhas an anti-bullying policy and an anti-bullying cur-riculum. The school has a safety officer; however, thisperson is rarely on site.

School Four. School Four is located in the NorthShore area of Massachusetts. The school is consideredan urban school. Enrollment at the time of the studyconsisted of 429 students in grades six through eight(Massachusetts Department of Education, 2008). Theprincipal is a female. While the school has an anti-bul-lying policy, there is not a curriculum in place toaddress issues of bullying. A school safety officer isavailable on an as-needed basis.

School Five. School Five is a suburban school locat-ed near Lowell, Massachusetts. Three hundred ninety-three students in grades six through eight attended theschool at the time of the study (MassachusettsDepartment of Education, 2008). The principal is amale. While there is an anti-bullying policy in place,there is not a curriculum program in place that address-es issues of bullying. The school does not have a schoolsafety officer.

Race/ethnicity. Table 1 breaks down the race/ethnic-ity for each school for the 2007-2008 school year. Thedata include all students enrolled in the school at thetime, not only the seventh graders that participated inthe surv e y. Tables are from the MassachusettsDepartment of Education Website (2008).

InstrumentsSurveys were developed to examine teacher/special-

ist, parent and students’ perceptions of school climateand incidents of bullying within their school. The ques-tions relating to school climate were developed as aresult of the review of the literature of leadingresearchers in the field including Hoy and Miskel(2005), Peterson and Deal (1998), Fullan (2001),Kelley, Thornton, and Daugherty, (2005), Kallested andOlweus (2003), and Orpinas, Horne, and Staniszewski(2003). The bullying items for the student andteacher/specialist questionnaires were published byBradshaw et al. (2007). The parent items were an adap-tation of the teacher/specialist items.

Teacher/Specialist survey. The questionnaire for the

Table 1.School Populations by Race/Ethnicity

Race (% of) School 1 School 2 School 3 School 4 School 5African American 1.6 0.8 1.9 3.5 2.0Asian 5.8 8.0 2.2 2.3 12.7Hispanic 1.4 1.8 4.4 32.2 3.1Native American 0.0 0.3 0.0 0.0 0.0White 90.9 88.0 91.0 59.4 80.7Native Hawaiian, Pacific Islander 0.0 0.6 0.2 0.7 0.0Multi-Race, Non-Hispanic 0.4 0.5 0.2 1.9 1.5

27


teachers/specialists also included questions with Likertscale response options on school climate and bullying.Four questions were used to determine how long teach-ers and specialists had worked in a particular school,their position (i.e., classroom teacher, special are ateacher, specialist), their gender, and how many stu-dents they currently taught. Eighteen questions wererelated to school climate. The questions were used tocollect data concerning the following:

1. How and how often teachers communicated withparents of students in their classes,

2. How supported the teachers/specialists felt bytheir colleagues and the building administration,

3. The teacher/specialists’ feelings regarding the cli-mate or tone (e.g., positive, negative) of the school,

4. Assigned duties and responsibilities.

The next 19 questions were directly linked to bul-lying. The teacher/specialist items were taken from theteacher questionnaire presented in the Bradshaw study(2007). The questions were used to collect data on thefollowing:

1. Prevalence of bullying in their school,2. Types of bullying,3. Attitudes towards students who bully,4. How seriously bullying reports were taken by

school personnel,5. How safe they felt at their school,6. Their skill in addressing/handling bullying

issues.

Student survey. The student survey included bothopen- and close-ended questions and true/false state-ments. The first nine questions/statements related toschool climate. The questions/statements sought todetermine the following:

1. How connected to the staff the students were,2. How available adults were when students need-

ed to seek assistance,3. How safe the students felt within the school.

The next 14 questions examined issues regardingbullying. The items were taken from the student ques-tionnaire presented in the Bradshaw study (2007).Thequestions were used to collect data about the following:

1. How students were bullied,2. The students’ perceptions about bullying,3. Adult responses/reactions to bullying.

Parent survey. The questions for the parent surveyexamined both school climate and bullying. The ques-tions were all close-ended. Eighteen questions wererelated to school climate. The questions were used tocollect data concerning the following:

1. How welcome parents felt at the school,2. If parents felt their child’s teacher was willing to

communicate,3. How often parents communicated with the

school principal,4. The number of opportunities for parents to

become involved in their children’s education(e.g., volunteering, attending school functions,reading the school newsletter, parent organiza-tion, helping with homework).

There were eighteen questions that related to bully-ing. They were based on the teacher questionnaire itemstaken from the Bradshaw study (2007). The questionswere used to collect data on the following:

1. Prevalence of bullying in their child’s school,2. Types of bullying,3. Attitudes towards students who bully,4. How seriously bullying reports were taken by

school personnel,5. How safe their child felt at school.

Five other questions were included in order to gaina better understanding of 1) whether mothers, fathers,or legal guardians were the more frequent reporters oftheir child’s victimization, 2) the sex of the parent’s childthat was victimized, 3) work schedules of individualparents, and 4) whether the parent was bullied as achild.

Administrator survey. In addition, a survey wasdeveloped for the school administrator (principal orvice principal) to complete. Twenty-one questions wereused to collect information regarding:

1. Exact job title,2. Years in the position,3. Any recent changes in administration,4. If the administration held regular staff meetings,5. If their school held grade level meetings and if

so, did they, as administrators, attend,6. If there was a school newsletter,7. If there were all-school meetings,8. If there was an anti-bullying curriculum and if

so, which one was currently being implemented,9. Supervision procedures,10. Whether the school had a safety office and, if

so, for how many hours a week,11. For how long incidences of bullying had been

tracked,12. Number of incidences of bullying reported to

the administration.

28


ProceduresSurveys were distributed to the school principals or

the person designated as the contact in the school.Student and parents surveys were sent home with thestudents; envelopes were also provided with the surveysso that students could return both theirs and their par-ents’ surveys to their homeroom teacher. Teacher/spe-cialists’ surveys were distributed by the principal orcontact and an envelope was placed in the main officeinto which the completed surveys were returned, unlessother arrangements were made (i.e., being distributedand completed at a staff meeting). Students in the studywho returned the survey and consent form were offeredas an incentive of being eligible to receive a gift certifi-cate for a local store such as Borders or Target (one stu-dent from each participating school received a gift cer-tificate). (Note: Some school principals expressed thatthey preferred that an incentive not be offered.)

RESULTSThe computer-based, data program SPSS was used

to analyze the survey data.

Confirmatory Factor Analyses The findings from each confirmatory factor analysis

are discussed below. However an exploratory analysiswas conducted prior to the confirmatory analyses. Theitems that factored into each factor during the initialanalysis were used to run the confirmatory analysis.Bullying and school climate were the two factors thatappeared across all three analyses.

Teacher/Specialist items. The teacher data resulted insix factors with Eigenvalues over 1.0. However, theelbow in the scree plot indicates that the first five com-ponents account for most of the variance in the data.When looking at the components, it becomes clear thatonly four of those identified factors consist of two ormore unique items. When examining how the itemsclustered together, parent contact, school climate, directexperience with student bullying, and adult response toreported bullying seemed most appropriate for the fourfactor titles.

Student items. The student data resulted in four fac-tors over 1.0. All four of the identified factors consist oftwo or more unique items. When examining how theitems clustered together, direct bullying, school climate,o b s e rved bullying, and adult response to bullyingseemed most appropriate for the four factor titles. Thedirect bullying and school climate factors correspondmost closely with the factors regarding direct experi-ence with student bullying and school climate in theanalysis of the teacher data.

Figure 1.Confirmatory factor analysis teacher/specialist survey scree plot

Figure 2.Confirmatory factor analysis student survey scree plot

Figure 3.Confirmatory factor analysis parent survey scree plot

29


Parent items. The confirmatory analysis resulted insix factors over 1.0. However, the elbow in the Screeplot indicates that only the first four of those compo-nents account for most of the variance in the data.When looking at the components, it became clear thatonly those ones consist of two or more unique items.When examining how the items clustered together,school bullying, school climate, parent contact, and rea-

sons for bullying seemed most appropriate for the fourfactor titles. The school climate factor corresponds mostclosely to the school climate factors from the teacherand student data. The bullying factor corresponds mostclosely to the direct experience with student bullyingfactor from the teacher data and the direct bullying fac-tor from the student data.

Directexperience Adultwith response tostudent reported Parent Schoolbullying bullying contact climate

Reasons for contacting parents/ Social .765Reasons for parents contacting teachers/Behavior .625Reasons for parents contacting teachers/Social .869Witnessed bullying .926Sense of safety .905Sense of belonging .919Location of incidences/ Classroom .695Location of incidences/ Hallway .888Location of incidences/ Cafeteria .689Location of incidences/ Other .765Type of bullying/Name calling .854Type of bullying/ Pushing .720Type of bullying/Other .699Type of bullying/ Verbal Threats .693Type of bullying/ Exclusion .741Reason/Students’ race or skin color/Student’s race or skin color .414Reason/ Individual differences .407Reason/ Other .486Response to observed incidence/Intervened with bully .881Response to observed incidence/ Ignored it .897Response to observed incidence/Other .784Response to observed incidence/ Referred to school psychologist/ guidance .704Response to observed incidence/ Referred to an administrator .707Response to report/Talked to administrator .835Response to report/Talked to bully’s parents .860Response to report/ Intervened with victim .860Response to report/ Other .812

Table 2. Confirmatory Factor Analysis Teacher/Specialist Survey Factors

30


CorrelationsIn order to help establish if perceptions between

teachers, parents, and/or students were similar amongthe groups of surveyed participants the totals for thebullying and school climate factors, established duringthe factor analysis, were calculated for each participant.These factor scores from each survey type were com-bined at the group and school level and then correlated.

Correlations at the individual factor level. First thedata was analyzed for correlations between groups atthe individual factor level. When the data was correlat-ed by group only one correlation existed. Parent andstudent perceptions of bullying correlated at the .01level.

Direct Observed School Adultbullying bullying climate response

Feel welcome .686Like going .727Adults help when asked .487 -.434Caring teacher .664 -.449Adults help handle peer conflict .554 -.494Dislike of school -.781Witness to bullying .939Response to another/Joined in .928Response to another/Ignored it .849Response to another/Told an adult .907Response to another/Other .885Response to another/I tried to stop it .872Sense of safety .728Sense of belonging .821Location of incidences/Classroom .912Location of incidences/Hallway .888Location of incidences/Cafeteria .907Location of incidences/ Other .917Reason/Student’s race or skin color .905Reason/ Student’s gender .933Reason/ Looks .908Reason/ Other .916Own response/Bullied that person back .916Own response/Did nothing .922Own response/Told an Adult .886Own response/Other .878Proactive approach to bullying -.558Adults ignoring bullying .688Bullying worsened with adult intervention .593No response to request for help .606

Table 3. Confirmatory Factor Analysis Student Survey Factors

31


Correlations between the school climate and bullyingfactors by school. The next part of the data analysissought to establish any correlations between percep-tions of bullying and school climate at the individual

school level. A correlation between perceptions ofschool climate and bullying existed at both the student(.01 level) and teacher (.05) levels for School 5 only. Noother significant correlations existed among the groups.

Reasons for School ParentBullying bullying climate contact

Involved with PTO .751School Council membership .450Contact with administration .800Child reported seeing bullying .690Child self reported bullying .544Sense of safety .848Sense of belonging .905Location of incidences/Classroom .904Location of incidences/Hallway .892Location of incidences/Cafeteria .887Location of incidences/Other .886Type of bullying/Name calling .834Type of bullying/Pushing .827Type of bullying/ Verbal Threats .863Type of bullying/Other .874Reason/Student’s race or skin color .887Reason/ Student’s gender .877Reason/ Physical differences .888Reason/ Other .864Response/ Talked to administrator .473Response/ Talked to bully’s parents .519Response/ Talked to school psychologist, behavior specialist or guidance counselor .516Response/ Talked to my child’s teacher .459Response/ Talked to my child .595Response/ Other .545

Table 4. Confirmatory Factor Analysis Parent Survey Factors

Table 5. Correlations for Bullying Factor by Group

Student Parent TeacherStudent 1.000 .615** .156Parent .615** 1.000 .124Teacher .156 .124 1.000**Correlation is significant at the 0.01 level (2-tailed).

Table 6. Correlations for School Climate Factor by Group

Student Parent TeacherStudent 1.000 .039* -.106Parent .039 1.000 -.088Teacher -.106 -.088 1.000

32


A correlation between parents’ perceptions ofschool climate and bullying was identified only forSchool 5. This correlation is significant at the .05 level.

Correlations between the school climate and bullyingfactors by group. When the bullying and school climate

factors were correlated by group, only two correlationsexisted. There was a correlation between the bullyingand school climate factors for students (.01 level) andparents (.05 level).

Table 7. Teacher Correlations by SchoolSchool Climate Bullying

School 1 School Climate Pearson Correlation 1.000 -.304Bullying Pearson Correlation -.304 1.000



School 4 School Climate Pearson Correlation 1.000 .057Bullying Pearson Correlation .057 1.000

School 5 School Climate Pearson Correlation 1.000 .031Bullying Pearson Correlation .031 1.000

Table 8. Student Correlations by SchoolSchool Climate Bullying





School 5 School Climate Pearson Correlation 1.000 -.556**Bullying Pearson Correlation -.556** 1.000

**Correlation is significant at the 0.01 level (2-tailed).

Table 9. Parent Correlations by SchoolSchool Climate Bullying





School 5 School Climate Pearson Correlation 1.000 -.514*Bullying Pearson Correlation -.514* 1.000

*Correlation is significant at the 0.05 level (2-tailed).

33


Conclusion. Perceptions about school climate andincidences of bullying are not similar when teacher, stu-dent, and parent responses are correlated. Only isolatedcorrelations existed. The data does not support the ear-lier hypothesis that schools which are perceived to havemore positive school climates are also perceived to haveless bullying incidences.

MeansMean scores by group. Teacher/specialist, student,

and parent perceptions of school climate and bullyingdiffer across groups and schools. When the mean scoresfor each group were examined, teachers identified theclimate as more positive than the other groups andidentified bullying as more of an issue than did studentsand parents. In each school, the groups had differingviews of the levels of bullying and school climate whentheir responses were compared to the others schools.For example, in School One, parents saw bullying asmore of an issue than students and teachers. Studentssaw school climate as more negative than did their par-ents and teachers/specialists. In Schools Two and Three,parent responses indicated that they felt that theirchild’s schools had a more positive school climate andless incidences of bullying. In School Three, however,teachers/specialists viewed the school climate as morenegative than did the parents and students. In SchoolFour, parents saw the school as having a poorer schoolclimate with higher amounts of bullying. Teachers sawbullying as an issue but not the school climate, where-as for students the reverse was true. (Note: Incidencereports indicated that this school reported the highestnumber of bullying incidences per school year com-

pared to the other four schools.) In School Five, parentssaw bullying as less of an issue than did teachers/spe-cialists and students. However, students saw the schoolclimate as being better than did their teachers/ special-ists and parents. Therefore, it is important for adminis-trators to be cognizant of the fact that how staff viewsthe school climate and level of bullying is not necessar-ily the same as how students and parents perceive them.Attempting to change either would require input fromall three groups.

Mean scores by individual items. When means werecollapsed by group (i.e., teachers/specialists, students,and parents), information was learned that could helpadministrators address school climate and bullyingissues. When analyzed, individual group perceptions ofeach may impact how bullying and school climateissues are addressed.

The teacher/specialist data revealed that the major-ity of the surveyed educators have witnessed bullying.Bullying was more likely observed in the hallways thanin other school locations. Also the most frequent typesof bullying were name calling and exclusion. Teacherswere likely to intervene during an observable bullyingsituation as well as provide assistance to the victimwhen a situation was reported.

The student data revealed that most students feelwelcome at their respective schools; however, that didnot correspond with whether they like going to school.Most students felt that adults are supportive and helpwhen asked. Most students either tried to stop bullyingwhen they witnessed it or did not get involved in thesituation.

Table 10. Correlation Between Bullying and School Climate Factors for Teachers

Bullying School climateBullying 1.000 -.082School climate -.082 1.000

Table 11. Correlation Between Bullying and School Climate Factors for StudentsBullying School climate

Bullying 1.000 -.267**School climate -.267** 1.000**Correlation is significant at the 0.01 level (2-tailed).

Table 12. Correlation Between Bullying and School Climate Factors for Parents

Bullying School climateBullying 1.000 -.246*School climate -.246* 1.000*Correlation is significant at the 0.05 level (2-tailed).

34


Table 13. Means for Teachers/SpecialistsHighest point value per item Statistic Standard error

Reasons for contacting parents/Social 1.0 .4020 .04879Reasons for parents contacting teachers/Behavior 1.0 .4118 .04897Reasons for parents contacting teachers/Social 1.0 .4412 .04941Witnessed bullying 1.0 .8137 .03874Sense of safety 3.0 2.6765 .07262Sense of belonging 3.0 2.6139 .07171Location of incidences/Classroom 2.0 1.2529 .07167Location of incidences/Hallway 2.0 1.6782 .07402Location of incidences/Cafeteria 2.0 1.3103 .07368Location of incidences/Other 2.0 1.0000 .05423Type of bullying/Name calling 2.0 1.6705 .07178Type of bullying/Pushing 2.0 1.3182 .07146Type of bullying/Other 2.0 1.0114 .05480Type of bullying/Verbal Threats 2.0 1.1477 .06371Type of bullying/Exclusion 2.0 1.5227 .07577Reason/Students’ race or skin color 2.0 .9796 .03547Reason/Individual differences 2.0 1.6837 .06085Reason/Other 2.0 1.0102 .03971Response to observed incidence/Intervened with bully 2.0 1.6556 .07605Response to observed incidence/Ignored it 2.0 .8889 .04311Response to observed incidence/Other 2.0 .8889 .04311Response to observed incidence/Referred to school psychologist/ guidance 2.0 1.1889 .07223Response to observed incidence/Referred to an administrator 2.0 1.1556 .06872Response to report/Talked to administrator 2.0 1.2323 .08351Response to report/Talked to bully’s parents 2.0 .8485 .05808Response to report/Intervened with victim 2.0 1.3200 .08514Response to report/Other 2.0 .9100 .06371

Table 14. Means for StudentsHighest point value per item Statistic Standard error

Feel welcome 3.0 2.4239 .04459Like going 3.0 1.8478 .05551Adults help when asked 3.0 2.3333 .04921Caring teacher 3.0 2.2787 .05163Adults help handle peer conflict 3.0 2.3261 .05008Dislike of school 3.0 .6831 .05737Witness to bullying 1.0 .5055 .03797Response to another/Joined in 2.0 .5394 .04078Response to another/Ignored it 2.0 .7879 .06535Response to another/Told an adult 2.0 .5879 .04786Response to another/Other 2.0 .5697 .04567Response to another/I tried to stop it 2.0 .7333 .06148Sense of safety 3.0 2.3405 .04834Sense of belonging 3.0 2.3027 .05057

35


More parents reported that their child had wit-nessed bullying than was involved in a bullying inci-dence. Parents felt that their children felt safe and wel-come at school. They reported the cafeteria as being thelocation of most incidences of bullying. Name callingwas the most frequent form of bullying. More parentsresponded to bullying by talking to their child ratherthan contacting the school.

DISCUSSIONLimitationsIn reviewing the survey development, process for

school selection, participants, survey administration,and item responses, certain limitations were identified.Identifying these limitations prior to discussing theimplications of the knowledge learned from the surveyis important because different conclusions may bedrawn from future studies when these research limita-tions are addressed.

School selection. The first issue that may have affect-ed the research results was school selection. The schools

chosen to participate in the survey were not randomlyselected. Rather, the Massachusetts Department ofEducation website and professional connections wereused as resources to recruit the schools. Over 30 schooldistricts were contacted to participate in the study andonly five actually took part in the study. Thus, the prin-cipals and superintendents who chose to have theirschools participate in the study may have had positivepreconceptions about their school climate and rates ofbullying and as a result felt more comfortable participat-ing. Also the school administrators that were involvedin the study each had personal knowledge of theUniversity of Massachusetts Lowell doctorate programand/or this particular research study, and the schooladministrators may have had more confidence regard-ing the confidentiality that would be afforded to the sur-vey results which may have made them more likely toparticipate than other schools with negative school cli-mates and higher incidences of bullying.

Timelines. The second limitation identified with theresearch methodology was the timeline in which theschools received the surveys. Difficulty with identifying

Table 15. Means for ParentsHighest point value per item Statistic Standard error

Involved with PTO 2.0 1.1257 .03383School Council membership 1.0 .0811 .02012Contact with administration 3.0 .6703 .06433Child reported seeing bullying 2.0 1.2732 .06110Child self reported bullying 2.0 .6000 .06468Sense of safety 4.0 3.0973 .07554Sense of belonging 4.0 3.1514 .07387Location of incidences/Classroom 3.0 1.1288 .09644Location of incidences/Hallway 3.0 1.1439 .09806Location of incidences/Cafeteria 3.0 1.1667 .09925Location of incidences/Other 3.0 1.0758 .09038Type of bullying/Name calling 3.0 1.1795 .10065Type of bullying/Pushing 3.0 1.0449 .08787Type of bullying/ Verbal Threats 3.0 1.0385 .08718Type of bullying/Other 3.0 1.0192 .08506Reason/Student’s race or skin color 3.0 1.4728 .05916Reason/ Student’s gender 3.0 1.6196 .07092Reason/ Physical differences 3.0 1.4348 .05537Reason/ Other 3.0 1.6576 .07340Response/ Talked to administrator 2.0 .5839 .06031Response/ Talked to bully’s parents 2.0 .4631 .04520Response/ Talked to school psychologist, behavior specialist or 2.0 .5302 .05434guidance counselorResponse/ Talked to my child’s teacher 2.0 .5705 .05890Response/ Talked to my child 2.0 .7383 .07340Response/ Other 2.0 .4667 .04602

36


sites and securing agreement from the schools to partic-ipate impacted overall survey distribution. The surveyswere not all distributed to the schools at the same time.Students and parents who took the surveys at a laterdate, therefore, had more opportunities for experiences,positive or negative, with the school. The results, there-fore, do not capture a snapshot of bullying and schoolclimate at the same time within all five schools.

Respondents. Only five schools participated in thestudy. Of those five, four were suburban and one urban.No rural schools agreed to participate. The majority ofthe schools were located in what is considered theMetro-West portion of Massachusetts. Massachusetts asan entire commonwealth is not well represented in thestudy. Additionally, the sample size is small. There are328 middle/junior high schools in the commonwealth(Massachusetts Department of Education, 2008) andonly five participated in this study. However, it isimportant to note that, 102 teachers/specialists alongwith 186 7th-grade students and their parents did par-ticipate in the study for a total of 474 participants. Eachgroup’s (i.e., all teachers/specialists, students, and par-ents) responses offered valuable information regardingperceptions of school climate and incidence of bullyingfor those groups. School climate and bullying may belinked when a wider representation of the population issampled.

Some parents also may have not received the sur-veys as the students were given the survey at school andasked to take it home to their parents. If the parents didnot receive the surveys then their children did not com-plete them either as written consent was required fromall respondents.

R a c e / E t h n i c i t y. Based on the population of theschools, the surveys could have been translated intoSpanish, Portuguese, and Asian languages in order toensure that the survey was accessible to and representa-tive of all ethnicities. No data were collected regardingrace of respondents so it is unknown if most respon-dents were white or of other ethnic backgrounds. Thisgap in data collection may have skewed the resultswhen examining the impact school climate has on inci-dences of bullying.

Survey development. One limitation of the surveydevelopment had to do with different items across sur-veys. In some cases, like worded questions had differingnumber of responses. For example, in the parents andstudent surveys, the question that asked if bullyingoccurred in the cafeteria had three possible answers onthe student survey (i.e., “I have not been bullied,” “No,”and “Yes”) yet four possible answers on the parent sur-vey (i.e., “Not applicable,” “I don’t know,” “Yes,” and

“No). Also, the consistency of questions across the threesurveys had discrepancies. That is, questions that couldhave applied to all groups were not asked on all surveys.For example, the item, “I feel welcome at my school,”could have been asked of all three groups but was onlyasked on the student survey.

S u rvey administration. Envelopes were includedwith each survey packet in order to return the surveysand permission slips as the parents’ permission wasrequired for the students to participate. By returningboth documents in the same envelope, the authorwould be able to link the respondent with the respons-es although such links were to be held confidential andwould not be revealed as part of the published results.However, a number of parents complained to theirrespective principal that the survey was not trulyanonymous as their and their children’s responses couldbe linked to their names despite assurances that thiswould not happen. Response numbers were low andmay have been due to this inability to create a complete-ly anonymous survey because of the age of the studentsand the need to get the parents’ permission.

Also it is unclear how many parents actuallyreceived the survey. As mentioned above, students weregiven the survey and asked to take it to their parents; asthe students were the messengers, they may not havedelivered the envelopes to their parents. Likewise, someparents may have completed the survey and their childdid not return the envelope to school. Thus, the actualnumber of parents that received the survey is unknownand the number of parents who did receive the surveybut chose not to participate is also not known.

Data Sharing at the Individual School LevelIt is important that school administrators have an

understanding of how school climate is perceived bystaff, parents, and students along with how the issue ofthe bullying is perceived within the community.Ultimately, the frequency of responses will be sharedwith the individual school principals that participatedin the surveys used for this research so that they haveaccess to this information. The expectation is that byknowing how teachers/specialists, students, and parentsresponded to the survey items, then principals can fur-ther investigate practices in their school buildings thatmight lead to change regarding the prevalence of bully-ing. Below are two examples:

For example, in School One, 55.3 percent of thestudents reported seeing someone bullied in the previ-ous month. Additionally, when asked if the studentshad seen adults observe bullying without intervening,38.3 percent answered “yes.” Over 10% of the same stu-

37


dents polled shared that they had reported bullying toan adult and received no assistance. The survey resultsfrom School One also revealed that more studentsreported being bullied while in a classroom than in thecafeteria and hallway.

In School Three, 19.1 percent of the parents sur-veyed would not report an incidence to school staff if heor she heard about his or her child being bullied. Whenasked if their child had reported adults watching andnot responding to a bullying incidence, 12.8% respond-ed “yes.” When asked about bullying incidences, 59.6%of the School 3 parents responded that their childrenhad seen another student being bullied and 30.4%responded that their children had been bullied. Whilethese same parents reported positive school climate atSchool 3, the fact that parents were aware of bullyingoccurring at their children’s school and yet would notreport bullying to school staff is, while not statisticallycorrelated, still another interesting observed responseon why and how bullying occurs.

Instruments for Schools to Assess Perceptions of SchoolClimate and Bullying

The three surveys used for this research could beadministered in any middle/junior school in order toassess perceptions of school climate and bullying. As aschool psychologist it is important to develop an under-standing of how members view their school. For exam-ple, if a high percentage of students and their parentsfeel that staff do not readily respond to bullying inci-dences then how can that be addressed? If students donot feel welcome at school, what practices may beimplemented in order that the students develop a senseof belonging? By further developing an understandingof perceptions of school climate and bullying, then inci-dences of bullying may be able to be lessened.

CONCLUSION Although this study did not indicate a link between

perceptions of school climate and incidence of bullying,bullying is clearly an important issue in these schoolsand thus is a topic that must continue to be taken seri-ously. Educators need to be committed to ensuring thatall students in a classroom/school are treated respectful-ly. Professional development regarding conflict resolu-tion skills must also be provided to educators. Teachersneed to be taught how to teach conflict resolution skillsto their students so that the students have a skill set touse when responding to a bully. Teacher and adminis-trator attitudes towards students also impact how a stu-dent is received within the school setting. Students

sense if the adults in the school do not regard a peer ina favorable light. Training to assist teachers in recogniz-ing their own reactions to/behavior towards their stu-dents may also be needed. The provision of support toteachers and students by administration and studentsupport teams is critical in aiding students. Teachersshould not feel that they have to deal with a challengingstudent alone. Students should be encouraged to tell ifthey are being bullied. All students are entitled to aneducation and an educational environment in whichthey are free from harm.

Consideration should be given to making thechanges discussed in the Limitations section of thischapter and rerunning the study in order to furtherunderstand potential links between school climate andincidences of bullying. More respondents from a widerpool (i.e., ethnicity, socio-economic background, geo-graphic location) may yield diff e rent re s u l t s .Additionally, data sharing with the principals from eachschool that participated in the survey will also hopeful-ly assist with identifying and eliminating the bullyingthat is occurring in the surveyed schools. While thisstudy may ultimately impact students in five schools,until bullying is addressed and controlled withinschools nationwide, all angles of bullying must contin-ue to be examined.

ReferencesBaker, J. (1999). Teacher-student interaction in urban at-risk class-

rooms: Differential behavior, relationship quality, and student

satisfaction with school. The Elementary School Journal 100(1),

58-70.

Bradshaw, C., Sawyer, A., & O’Brennan, L. (2007). Bullying and peer

victimization at school: Perceptual differences between students

and school staff. School Psychology Review, 36(3), 361-382.

Brown, S., Birch, D., & Kancheria, V. (2005). Bullying perspec-

tives: Experiences, attitudes, and recommendations of 9 to 13

year olds attending health education centers in the United

States. Journal of School Health 75(10), 384-393.

DeVoe, J. & Kaffenberger, S. (2005). Student reports of bullying:

Results from the 2001 school crime supplement to the nation-

al crime victimization survey. Statistical Analysis Report.

Washington, DC: National Center for Education Statistics.

Dinkes, R., Cataldi, E., Lin-Kelley, W., & Snyder, T. (2007).

Indicators of school crime and safety: 2007. Retrieved

November 12, 2008 from http://www.ncjrs.gov

Dobransky, N. (2004). Developing teacher-student relationships

throughout class communication. Communication Quarterly

52(3), 211-223.

38


Eslea, M., Mensini, E., Morita, Y., O’Moore, M., Mora-Merchan, J.,

Pereaire, B., et al. (2003). Friendship and loneliness among

bullies and victims: Data from seven countries. Aggressive

Behaviors, 30, 71-83.

Fullan, M. (2001) Leading in a culture of change. San Francisco:

Jossey Bass.

Hoy, W., & Miskel, C. (2005). Education administration: Theory,

research, and practice (7th Ed.). New York: McGraw-Hill.

Kallested, J. & Olweus, D. (2003). Predicting teachers’ and

schools’ implementation of the Olweus bullying program.

Prevention and Treatment 6(7), Retrieved June 1, 2008 from

http://web.ebscohost.com.libproxy.uml.edu/ehost/pdf?vid=3&

hid=17& sid=5df8f396-ad20-4705-8c36-

6b736b18a7e3%40sessionmgr7

Kelley, R., Thornton, B., & Daugherty, R. (2005). Relationships

between measures of leadership and school climate. Education,

126(1), 17-25.

Leff, S. Power, T., Costigan, T., & Manz, P. (2003). Assessing the

climate of the playground and lunchroom: Implications for

bullying prevention programming. School Psychology Review

32(3), 418-431.

Massachusetts Department of Education (2008). Retrieved

December 1, 2008 from http://profiles.doe.mass.edu/

Olweus, D. (2001). Bullying at school: Tackling the problem.

OECD Observer. Retrieved February 9, 2009 from

http://www.oecdobserver.org/news/fullstory.php/aid/434/

Bullying_at_school:_tackling_the_problem.html

Orpinas, P., Horne, A., & Staniszewski, D. (2003). School bullying:

Changing the problem by changing the school. School

Psychology Review, 32(3), 431-445.

Peterson, K. & Deal T. (1998). How leaders influence the culture

of schools. Educational Leadership, 28-30.

Rock, E., Hammond, M., & Rasmussen, S. (2002). School based

program to teach children empathy and bully prevention.

Paper presented at the annual conference of the American

Psychological Association.

Seals, D. & Young, J. (2003). Bullying and victimization:

Prevalence and relationship to gender, grade level, ethnicity,

self-esteem, and depression. Adolescence, 38(152), 735-747.

Smith, P., Talamelli, L., Cowie, H., Naylor, P. & Chauhan, P.

(2004). Profiles of non-victims, escaped victims, continuing

victims and new victims of school bullying. British Journal of

Educational Psychology, 74(4), 565-581.

Stein, N. (2001). What a difference a discipline makes: Bullying

research and future directions. In Geffner, R., Loring, M. &

Young, C. (Ed.) Bullying behavior: Current issues, research, and

intervention. New York: The Haworth Press.

Wilson, D. (2004). The interface of school climate and school con-

nectedness and relationships with aggression and victimiza-

tion. The Journal of School Health 74(7), 293-299.

39


ABSTRACTIt is imperative that mathematics teachers make theirinstruction more effective. This paper argues thatcomprehensive lesson planning is crucial for effectiveteaching. A review of the literature identifies threetraits of effective teaching: (1) teachers’ level of math-ematical content knowledge, (2) class instructiondeveloped explicitly on student learning objectives,and (3) focus on student prior knowledge. A defini-tion of comprehensive lesson planning is provided,and research is reviewed into teachers planninghabits, the decisions teachers make when planning,and planning in countries outside the United States.A brief summary of common lesson planning meth-ods is provided. Finally, research is identified thatshows a possible link between one specific method oflesson planning and increased effectiveness of math-ematics instruction.

INTRODUCTION TO THE PROBLEMMathematics is a fundamental part of every stu-

dent’s education, with many hours devoted specificallyto the development of mathematical knowledge. Yet, agreat number of students in the United States leaveschool with only a superficial understanding of mathe-matics concepts (Stigler & Heibert, 1999). That somuch effort and time on the part of the teacher and thestudent is used to little or no avail should be a concernto all educators. Not only will a mathematically illiter-ate population be at a disadvantage in our increasinglytechnologically-advanced society, students’ failure tol e a rn mathematics signals a greater failure in ournations’ schools.

Data published by the Massachusetts Department ofEducation for the school year 2004–2005(Massachusetts Department of Education, 2006) showthat a significant number of sixth- and eighth-grade stu-dents achieve a score of Warning, essentially a failinggrade, on the mathematics portion of the MassachusettsComprehensive Assessment System (MCAS). Of thesestudents, those who go on to repeat at that grade leveldo not improve enough to achieve a level of NeedsImprovement when tested again at the end of the

repeated year. The extra year of instruction given to stu-dents fails to provide the desired outcome of improvedstudent achievement. This may, in part, be a directresult of the ineffectiveness of the classroom instructionprovided to the students. It is imperative that classroomteachers find a way to make their mathematics instruc-tion more effective.

Since the early 1980s, there has been a persistentshortage in the number of qualified mathematics andscience teachers. Because of this shortage, many effortshave been made to bring more qualified mathematicsand science teachers into the profession. The No ChildLeft Behind federal legislation includes several provi-sions for the recruitment of mathematically competentindividuals to the teaching profession through alterna-tive licensure routes (No Child Left Behind [NCLB],2002). In Massachusetts, the Massachusetts Departmentof Education has provisions in its licensure proceduresfor those teachers who do not take a traditional path tobecome mathematics educators. As Laczko-Kerr andBerlinger (2003) note, “research provides convincingevidence that subject-matter knowledge is necessarybut not sufficient for teaching well” (p. 35). While it isimperative that all mathematics classrooms include amathematically competent teacher, this rush to certifynontraditional applicants has created a situation wheremany teachers enter the classroom with little or notraining in education. As a consequence, new mathteachers may have little knowledge of how to plan les-sons, let alone plan effective lessons that focus on theuse of inquiry methods as recommended by NCTM(2000). Perplexingly, those teachers who do completetraditional teacher preparation programs are reported tocease planning in the way that they were instructed afterleaving those programs (Searcy & Maroney, 1996). It isa common assumption that the longer one stays in theteaching profession, the less time one needs to devote tolesson planning. However, this paper will argue thatconsistent and comprehensive lesson planning is crucialfor effective teaching.

Qualifying PapersIncreasing the Effectiveness of Mathematics Instruction Through Lesson Planning

Noreen C. Flanagan JohnsonGreater Lawrence Regional Vocational Technical High School, Andover, MA

40


THE PROBLEMThe problem that this paper will address is the fail-

ure of many middle and high school students to leavethe classroom with more than a superficial understand-ing of mathematical concepts; in effect, they fail to learnmathematics. It is suggested here that the primary fac-tor that influences student learning in the classroom isthe effectiveness of the instruction provided to the stu-dent. It is an oft-stated fact that most teacher-prepara-tion programs include methods classes in which pre-service teachers are expected to plan lessons (for exam-ple, see Reiser & Mory, 1991). However, research showsthat teachers either stop formal lesson planning alto-gether or do not continue to plan in the manner whichthey were taught (Searcy & Maroney, 1996). Alternately,those teachers who have not completed a traditionalteacher preparation program may have little to noknowledge of lesson planning methods when they enterthe classroom.

If the state of instruction in secondary mathematicsclassrooms is to be systematically improved, then it isimperative to assess what teaching activities and/ortechniques are most effective at improving mathematicsinstruction. Whether or not comprehensive lesson plan-ning can be linked to behaviors consistent with effectiveteaching is the purpose of this review. This topic isimportant because of the plethora of factors that canaffect the ability of students to become proficient inmathematics. Aspects such as the students’ home lives,socioeconomic statuses, societal influences, and cogni-tive abilities, however, are nearly impossible for teach-ers to change. Therefore, these factors will not be dis-cussed in this paper. It is primarily through theimprovement of classroom instruction that teachers,administrators, and teacher educators can have themost impact on the teaching and, hopefully, the learn-ing that takes place in the classroom.

PERTINENT AREAS OF RESEARCHThis paper consists of two strands. The first details

several characteristics of instruction identified in the lit-erature to be effective. The second strand reviews therelevant research on lesson planning. Specifically, itsearches for evidence of ways in which teachers mightimprove their instruction through comprehensive les-son planning.

REVIEW OF RESEARCH LITERATURE This review of the literature will focus on the argu-

ment that instructional effectiveness in the mathematics

classroom can be improved through comprehensive les-son planning. The first part of this review will focus onthree characteristics of effective teaching identified bythe literature. The importance of teachers’ contentknowledge, instruction based on student learning objec-tives, and students’ prior knowledge will be outlined.Research will be presented that links these characteris-tics to increased instructional effectiveness. A discussionon lesson planning will be presented in the second partof the review. The analysis will focus on the decisionsteachers make when they plan, any common themes inthe types or amounts of planning teachers engage in,and several common lesson planning formats currentlyavailable to teachers. Finally, research will be reviewedthat points to a possible relationship between a specificmodel of planning for mathematics lessons andincreased instructional effectiveness in the classroom.

CHARACTERISTICS OF EFFECTIVE MATHEMATICSTEACHING

“The teaching profession does not have enoughknowledge about what constitutes effective teaching”(Stigler & Hiebert, 1999). Many different factors havebeen put forth to explain what constitutes effectiveteaching. Those that will be addressed in this paper areteachers’ level of mathematical content knowledge, classinstruction developed explicitly on student objectivesfor learning, and teachers taking student prior knowl-edge into account during instruction. For the purposeof this paper, the personality traits of teachers that havebeen linked to levels of teacher effectiveness will not bediscussed.

According to Mager (1975), “instruction is success-ful or effective to the degree that it accomplishes what itsets out to accomplish” (p. 1). This definition, thoughvague, does illustrate the ongoing argument over thenotion of what constitutes effective teaching. The cur-rent NCLB legislation has placed a significant emphasison student scores on standardized exams, thereforeequating effective teaching with student achievements c o res. Morine-Dershimer (1977) and Good andG rouws (1979), among others, have conductedresearch that attempts to link teacher effectiveness withan increase in students’ scores on various types of tests.Other educators may wish to define student learning inother ways, taking, for example, student participationand attitudes towards learning into account. For thispaper, the precise measurement of student learning isnot as important as is identifying the elements thatmany different researchers and educators can agree areshared by effective teachers.

41


Cruickshank, Bainer, and Metcalf (1999) argue that,in order to teach effectively, teachers must “organizetheir lesson content and activities logically” (p. 347).This requires teachers to have strong content knowl-edge of the subject they teach. Cruickshank et al. con-tinue that effective teaching must “inform students ofthe objectives of the lesson,” and also “determine themost logical way to introduce content based on theirstudents’ abilities, previous learning, and the naturalstructure of the content” (p. 347). As such, two addi-tional characteristics of effective teaching have beenestablished—student learning objectives must be iden-tified, as well as the prior knowledge students need inorder to achieve those objectives. It is feasible to con-clude that classroom instruction which includes allthree of these characteristics will be more effective thaninstruction that is lacking one or more of the identifiedtraits. It is the literature on these three characteristics ofeffective teaching that will make up the first part of thispaper.

Teachers’ Content Knowledge“To be effective, teachers must know and under-

stand deeply the mathematics they are teaching”(NCTM, 2000, p. 17). It seems obvious that to teachmathematics one needs to possess the requisite contentknowledge. Yet, research indicates that approximatelyhalf of eighth grade math classes in the United States aretaught by teachers who did not major in mathematics(National Center for Educational Statistics, United StatesDepartment of Education, 2000) and one-third of highschool mathematics teachers do not possess a degree inmathematics, math education, or a related field(Ingersoll, 2001). While all teachers are now required tobe highly qualified under the No Child Left Behind leg-islation (NCLB, 2002), this is far from reality. TheMassachusetts Department of Education allows teachersto teach a maximum of two classes per day outside oftheir area of licensure, and in extreme cases provides aone-year waiver that allows uncertified, and possiblyunqualified, teachers to teach a majority of their dailyschedule in a subject for which they do not hold a validteaching license. This process is in direct contradictionto some researchers’ findings (Fowler & Poetter, 2004;Manouchehri & Goodman, 2000; Wenglinsky, 2002),which indicate that a teacher’s level of content knowl-edge is a primary determinant of effective teaching.

The link between content knowledge and effectiveteaching was explored by Wenglinsky (2002) using datafrom the 1996 National Assessment of EducationalProgress (NAEP) in mathematics. A nationally represen-tative sample, consisting of 7,146 eighth-grade students

from the United States, was investigated to determinewhich aspects of classroom instruction impact studentachievement. It was found that, when controlling forclass size and socioeconomic status, a teacher’s major incollege was “modestly associated with academicachievement” (p. 18). Further support for the linkbetween content knowledge and effective teachingcomes from a case study of two seventh-grade mathe-matics teachers conducted over seven months.Manouchehri and Goodman (2000) looked at theteacher characteristics that affected the successfulimplementation of a new, reform-based mathematicstextbook. The researchers concluded that a teacher’scontent knowledge, and by extension, the teacher’s abil-ity to see connections between mathematical concepts,is at the heart of effective instruction.

The Trends in International Mathematics andScience Studies (TIMSS) alerted educators in the UnitedStates to the problems with mathematics education inthis country (National Center for Education Statistics,2000). As a result, researchers have examined whyother countries appear to be succeeding where we arefailing. Fowler and Poetter (2004) examined why stu-dents in France scored much higher than their U.S.counterparts. They collected 60 hours of observationsin primary schools, conducted formal and informalinterviews with educators, and examined school docu-ments. The researchers noted that a strong mathemati-cal content knowledge is required of teachers in Francebefore they are allowed to enter teacher preparationprograms. The authors conclude that this is the primarycontributor to the high achievement of French mathe-matics students.

While few researchers doubt the effect of teachers’mathematics content knowledge on effective classroominstruction, a survey sent to New York principals didnot reveal content knowledge in general to be one oftheir major concerns. Torff and Sessions (2005) mailed300 surveys to secondary school principals in New YorkState, of which 242 were completed and returned. Ofthe principals who responded, the least likely cause ofteacher ineffectiveness to be indicated was the content-knowledge deficiency of teachers. Instead, these princi-pals listed classroom-management skills, lesson imple-mentation skills, rapport with students, and lessonplanning skills as more important for teacher effective-ness than content knowledge. However, had this studybeen focused on mathematics teachers and sent todepartment heads rather than principals, the resultsmight have been quite different. Only a small portion ofschool principals start their careers as mathematicsteachers, and as such they may not be competent

42


enough in the secondary mathematics curriculum toevaluate whether or not a teacher has adequate mathe-matical content knowledge.

Similar to the principals in the Torff and Sessions(2005) study, preservice teachers have been shown tobelieve that content knowledge is less important thanteacher enthusiasm and teaching skills when it comes toinstructional effectiveness. Minor et al. (2002) surveyed134 preservice teachers during their first week of anintroductory-level education class, using the PreserviceTeachers’ Perceptions of Characteristics of EffectiveTeachers Survey. The participants were asked to identi-fy, rank order, and define what they perceived as char-acteristics of effective teachers. Eighty percent of theparticipants did not indicate that being knowledgeableabout subject-matter was a trait of effective teachers,while more than one-half of the participants identifiedstudent-centered themes relating to teacher effective-ness. However, since these preservice teachers wereenrolled in an introductory-level education class, it isprobable that they had not yet had any significant expe-rience as teachers in the classroom. It would be interest-ing to follow-up with these same teachers once they hadexperienced student teaching to see if their perceptionsof effective teaching had changed. It seems reasonableto hypothesize that, once exposed to real-life teachingexperiences, the importance of content-knowledge, par-ticularly in mathematics, may become a larger compo-nent of what teachers perceive as a characteristic ofeffective teaching.

For the purposes of this review, one major limita-tion of the work done by both Minor et al (2002) andTorff and Sessions (2005) is that it did not focus explic-itly on effective mathematics instruction. More conclu-sive evidence is needed on the link between teachers’mathematical content knowledge and effective mathe-matics teaching.

As Laczko-Kerr and Berlinger (2003) note,“research provides convincing evidence that subject-matter knowledge is necessary but not sufficient forteaching well.” (p. 35). Teachers must also possessinstructional strategies that have been shown to posi-tively influence student learning in the classroom.Literature that identifies two of these possible strategieswill be reviewed next.

Objectives-Based InstructionOver six decades ago, Ralph Tyler (1949) stated the

following:

If an educational program is to be planned andif efforts for continued improvement are to be

made, it is very necessary to have some concep-tion of the goals that are being aimed at. Theseeducational objectives become the criteria bywhich materials are selected, content is out-lined, instructional procedures are developedand tests and examinations are prepared. (p. 3)

This demand for objectives-based instruction hasbeen repeated by many other authors and researchersthroughout the years (Mager, 1975; Gagne & Briggs,1979; Airasian, 2001; Panasuk & Todd, 2005). Prior towriting lesson plans for instruction, it is “important tobe able to state clearly just what you intend the resultsof that instruction to be” (Mager, 1975, pre f a c e ) .Panasuk and Todd (2005) note that the purpose of writ-ing student objectives is “to guide the lesson planningprocess” (p. 219).

Student learning objectives should be the startingpoint for planning the instruction and assessment of thelearner. This idea is concurrent to the statement made byGagne and Briggs (1979) that “objectives should guidethe instruction and the evaluation, not the other wayaround……the objectives should be determined beforethe lesson plans or evaluation instruments” (p. 32).According to Mager (1975), there are a number of reasonsthat objectives are important. If there is no clear objective,there is no basis for selecting materials or methods forinstruction. If there is no clear objective, it is not possibleto identify when the end of instruction has been reached.Finally, if there is no clear objective, students are less like-ly to be able to “organize their own efforts toward accom-plishment of those objectives” (p. 6).

Mager posits that “an objective is a description of aperformance you want learners to be able to exhibitbefore you consider them competent. An objectivedescribes an intended result of instruction, rather thanthe process of instruction itself” (p. 5). According toTyler (1949), there are three basic rules for writingobjectives. The first is that objectives should not be stat-ed as actions that the teachers will complete; these arenot educational objectives. The purpose of an objectivedoes not lie in what the teacher does, but in what thestudents learn. “It becomes important to recognize thatany statement of the objectives …… should be a state-ment of changes to take place in students” (p. 44). Withthese statements, it is possible to determine which activ-ities the teacher should perform so as to facilitate thestudents’ attainment of the stated objectives. The sec-ond rule is that objectives should not be stated in theform of content topics or generalizations that studentsare to encounter. “These are not satisfactory objectivessince they do not specify what the students are expect-

43


ed to do” (p. 45). Merely stating what topics are to becovered in a lesson does not indicate the change in stu-dents learning that is expected to have occurred at theconclusion of the lesson. The final rule is that objectivesshould not be stated in “the form of generalized patternsof behavior” (p. 46) such as ‘to learn to be a better citi-zen’ or ‘to become a critical thinker’, since they do notindicate the area of “content to which this behaviorapplies, or the area in life in which such behavior is tobe used” (p. 46).

“It is best that objectives be explicit, clear, andmeasurable” (Airasian, 2001, p. 74). Tyler notes that“the most useful form for stating objectives is to expressthem in terms which identify both the kind of behaviorto be developed in the students and the content or areaof life which this behavior is to operate” (p. 46–7). Thiscoincides with Gagne and Briggs (1979) definitions of awell-written student objective. “An objective is precise-ly described when it communicates to another personwhat he would have to do to observe that a stated lessonpurpose has in fact been accomplished” (p. 118).

Most teacher preparatory programs contain directinstruction for preservice teachers in how to constructstudent learning objectives (McCutcheon, 1980;Kennedy, 2004; Stangis, Pringle, & Knopf, 2006).However, researchers have reported that, although pre-service teachers acknowledged the importance of stan-dards and lesson objectives when planning lessons, thepreservice teachers were more concerned with firstfinding the instructional activities to take place duringthe lesson (Stangis, Pringle & Knopf, 2006). In thisstudy, a majority of the preservice teachers reportedhaving difficulty writing student learning objectives.Student teachers re p o rted that writing objectives“would become easier over time” (p. 78) and noted thatdeficits in their own content knowledge made it difficultto begin planning lessons by writing objectives.

In research performed in Newfoundland, many in-service teachers rated themselves as adequate at instruc-tional design strategies, but most could not “develop aspecific objective on request” (Kennedy, 1994, p. 20).Primary and elementary teachers studied showed afamiliarity with student learning objectives (To b i n ,1990, as cited in Kennedy, 1994), and secondary teach-ers reported “extensive use” of student learning objec-tives (Graham, 1992 and Thomey, 1991, as cited inKennedy, 1994, pp. 20–1), yet most used those objec-tives that were provided in their curriculum materials.Two-thirds of the secondary teachers studied did notuse objectives as the basis for assessment of studentlearning. In a study by Bol & Strage (as cited in Delong,Winter, & Yackel 2005), identified a distinct disconnect

between high school biology teachers’ stated lessongoals and the goals that were suggested by their lessonassessments. This study demonstrates that even whenteachers begin their lesson planning with student-learn-ing objectives, they do not necessarily base their subse-quent classroom activities and assessments on thoseobjectives.

Strangis, Pringle, and Knopf (2006) concluded that,although preservice teachers do not always start plan-ning with student objectives, identifying student learn-ing objectives at some point in the planning process iskey to good teaching. This is in direct opposition toresearch conducted by Panasuk and Todd (2005), whofound that when teachers planned using a specificobjectives-first method, writing and explicitly statingstudent learning objectives was closely linked to a logi-cal flow of the lesson through multiple phases, as wellas the homework correctly assessing the lesson’s objec-tives. “The core proposition [of objectives-basedinstruction] is that well-structured lessons flow fromwell-specified objectives” (p. 228). In a study per-formed by Morine-Dershimer (1977), it was noted thatteachers who stated behavioral objectives in their writ-ten plans for a reading lesson were more frequentlythose teachers whose students had high gain scoresthan those with low gain scores. In this study, teacherswere allowed to choose prewritten student objectives orto construct their own. She noted that “when original[objectives] were developed, the only teachers to state[these] were teachers with high pupil gain scores” (p.19). In light of the conclusions drawn by Panasuk andTodd (2005) and Morine-Dershimer (1977), it is rea-sonable to conclude that objectives-based lessons canlead to more effective classroom instruction than les-sons that are not explicitly and directly based on stu-dent learning objectives.

Students’ Prior KnowledgeIn order for classroom instruction to be effective,

teachers must first know what tasks and skills studentshave already mastered; they must take into considera-tion the students’ prior knowledge. Robert Gagne(1970/1973) stated, “the most dependable condition forthe insurance of learning is the prior learning of prereq-uisite capabilities” (p. 111). During his research, resultssuggested that the performance of students in a self-paced mathematics program that was “otherwise effec-tive” (p. 163) was highly dependent on whether or notthey had previously learned the necessary subordinateknowledge to complete the program and was independ-ent of other factors (Gagne, 1963/1973). Based onGagne’s ideas of prior knowledge, Panasuk and Todd

44


(2005) advocate for teachers to determine the necessaryprior knowledge, or “constituent parts” (p. 222) of themathematics concept to be learned by the students.This allows teachers to present a gradual developmentof the concept from what the students are already ableto do on their own to what they are desired to learn. Italso provides teachers a lens through which to previewthe types of student errors or misconceptions that theymay develop as the lesson progresses. The teacher canthen plan to introduce examples in the lesson that willchallenge students’ misconceptions and increase thelikelihood of learning for understanding.

The method of identifying the subordinate skillsthat are required by students to learn a new concept iscalled task analysis. “Task analysis refers to several dif-ferent, though interrelated, procedures which are car-ried out to yield the systematic information needed toplan and specify the conditions for instruction,” (Gagne& Briggs, 1979, p. 114). Two types of task analysis thatGagne details are the learning task analysis and theinformation processing task analysis. A learning taskanalysis is the act of identifying the prerequisite knowl-edge and skills necessary for the objective or task that isto be learned. An information processing task analysisidentifies each individual step which, when taken as awhole, makes up the totality of the task of the primaryobjective to be completed.

One means of performing a task analysis is forteachers to work out all of the homework and classworkproblems that students will be required to complete dur-ing a class lesson. The purpose of this is for teachers toanalyze the underlying concepts and skills to define theprerequisite skills and knowledge needed by the stu-dents to learn the new material (Panasuk & Todd,2005). The act of working out homework before devel-oping the class activities allows teachers to determinewhich sub-skills must be reinforced, as well as to predictwhere students may have misconceptions during theinstruction. This allows the teacher to tailor the lessonaccordingly to the maximum benefit of the students.

During the Missouri Mathematics Eff e c t i v e n e s sProject, Good and Grouws (1979) instructed teachers ina treatment group to spend some time at the beginningof each class focusing on the recall of prior knowledgethat students would need to use during the new lesson.While the posttest means of both the treatment andcontrol groups were higher than the pretest means, stu-dents of teachers in the treatment group had largergains on standardized achievement tests than studentsof the teachers in the control group. For the purpose ofthis paper, a major limitation of this study is that thefocus on prior knowledge was only one of the teaching

behaviors explicitly requested of the treatment teachers.As such, it is not possible to isolate the effects of thisone specific treatment behavior from the entire treat-ment. However, the results are supported by otherresearch. Wiegand (as cited in Gagne, 1970/1973)asked students to perform a task with which they wereunfamiliar. At first, nearly all of the students could notcomplete the task. Wiegand then tested the students atsuccessively simpler levels of the prerequisite skills nec-essary to complete the task until a base-level was foundwhere all students possessed the same basic knowledge.The students were then taught, beginning at that levelof commonality, all of the necessary sub-skills requiredto complete the task. However, they were not instruct-ed in how to complete the task itself. Nine out of tenstudents were able to complete the final task withoutdirect instruction once they possessed all of the neces-sary prior knowledge.

These findings are substantiated by those ofCarnine (1980), who studied the effects of preteachingversus concurrent teaching of component skills of anew concept. Fifteen lower-achieving first-graders,identified by teachers, were assigned to either a treat-ment or control group. The two groups were bothtaught a multiplication algorithm. However, thepreteaching group first taught the component skills ofthe algorithm and the concurrent group was taught thecomponent skills along with the algorithm at the sametime. Time measures, as well as correct responses to thefinal task, were recorded. “Children in the pre-teachinggroup reached criterion in significantly less time thanchildren in the concurrent group,” (p. 378) and alsoscored significantly better on the final task. The resultsof this study seem to indicate that the order of theteaching of components skills of a new concept is a rel-evant factor to be considered by the teacher. Students inthis study learned in less time and provided more cor-rect responses to transfer facts when pre-taught thecomponent skills of multiplication before being taughtthe multiplication algorithm itself. While these studentswere in first grade, it would seem likely that the morecomplicated the mathematical task, the more likely thismethod of insuring that students have the necessaryprerequisite knowledge and skills needed to master anew concept prior to instruction would be beneficial.More research into whether or not these results wouldbe transferrable to other levels of mathematics instruc-tion, such as middle and high school, is necessary.

SummaryThere is no one single method to produce effective

mathematics teaching; instructional strategies that work

45


for one class of students may not work for another.There are, however, some elements which can be shownto be common to effective teaching. These are teachers’level content knowledge, teachers’ use of objectives-based instruction, and teachers’ recognition of the priorknowledge and skills re q u i red of the students.Classroom instruction that includes these characteris-tics is more likely to be effective than instruction that islacking in one or more of these traits. However, theremust be some method of structuring and implementingthese characteristics so that they may be more useful toteachers in the classroom. In order to facilitate effectivec l a s s room instruction, teachers should incorporatethese three elements into their lessons during the plan-ning process. Therefore, lesson planning will be thefocus of the second part of this review.

LESSON PLANNINGFrom a search of online databases, it appears that

research on lesson planning was at its peak during thelate 1970s to the early 1980s, which coincided with thespread of instructional design strategies for educationbeing taught in teacher training programs. This researchidentifies many different definitions of lesson planning.Peterson, Marx, and Clark (1978) state that planning isthe “preactive decisions the teacher makes prior to theact of teaching” (p. 418). Clark and Yinger (1979)“define planning as a process of preparing a frameworkfor guiding teacher action……[which] involves teacherthinking, decision making, and judgment” (pp. 8-9).More general definitions involve any and all thoughts ordecisions that teachers make before they begin to teach(Fernandez & Cannon, 2005; Panasuk, Stone, & Todd,2002). Panasuk and Todd (2005) state that “planning alesson involves teachers’ purposeful efforts in develop-ing a coherent system of activities that facilitates theevolution of students’ cognitive structures” (p. 215). Forthe purposes of this paper, comprehensive lesson plan-ning will be defined as the act of planning for classroominstruction that:

• is written explicitly based on student learningobjectives,

• includes assessments based on the stated objec-tives,

• requires teachers to take into consideration theprior knowledge of students as well as the prereq-uisite skills and knowledge necessary for comple-tion of the stated objectives, and

• has consistency throughout all phases of the les-son.

The questions remain, however, to what extent doteachers plan comprehensively, and does comprehen-sive planning improve the effectiveness of their class-room instruction? It is these topics that will make upthe second part of this paper.

Teachers’ Planning ActivitiesResearch suggests that the majority of teachers do

not consistently and thoroughly plan for instruction(Clark & Yi n g e r, 1979; Morine-Dershimer, 1977;McCutcheon, 1980; Stigler & Hiebert, 1989; Kagan &Tippins, 1992; Searcy & Maroney, 1996; Reiser & Mory,1991). Much research on teacher lesson planning seemsto have been done ex post facto, with in-service teach-ers interviewed or surveyed about previous planningactivities (Clark &Yinger, 1979; Sardo-Brown, 1990;Reiser, 1994; Searcy & Maroney, 1996; Sanchez &Valcarcel, 1999; Yildirim, 2001). Other studies haveasked teachers to participate in a staged lesson-planningactivity in which they plan, and sometimes subsequent-ly teach, a concept outside the scope of their usual cur-riculum (Morine-Dershimer, 1977; Peterson, Marx, &Clark, 1978). Still other studies are built around theplanning that occurs throughout the school year (Sardo-Brown, 1988, 1993; Reiser & Mory, 1991; Reiser, 1994;Panasuk, Stone, & Todd, 2002; Kagan & Tippins,1992), observing and collecting artifacts from in-serviceteachers as they go about their regular teaching duties.

Several studies have relied on teachers’ self-report-ing of their own planning habits and decisions throughthe use of survey instruments (Clark &Yinger, 1979;Searcy & Maroney, 1996). In a survey of special educa-tion teachers in Iowa, Searcy and Maroney attempted tocollect information about how teachers plan lessons.The majority of the teachers who returned the surveystated that they did not write lesson plans for every les-son. However, this may be a function of the special edu-cation teacher not being the primary instructor of theclass, deferring planning responsibilities to the regulareducation teacher in the room. In an attempt to answerthe question “how do teachers plan” (p. 11), Clark andYinger surveyed 300 elementary school teachers inMichigan. They found that many teachers self-reportedthat they did not write lesson plans, but rather relied onmental plans. This work is supported by previousresearch. The process of staged lesson planning wasused by Morine-Dershimer (1977) whose goal was todocument “the kinds of planning that teachers engagein before a lesson begins” (p. 1). In this study, 40 sec-ond- and fifth-grade teachers were given a topic andmaterials to create one lesson each in mathematics andreading. In the follow-up interviews, two-thirds of the

46


teachers acknowledged that the lesson plans that theyhad written for this study were done in much greaterdetail than their usual plans. These teachers also notedthat, for the most part, the regular planning that theydid was not of a written form; rather, it was ‘in theirheads” (p. 5). Taken together, these studies indicate awidespread lack of written planning being performedby classroom teachers. However, one limitation of theliterature reviewed here is that none of these studiesfocused explicitly on the planning activities of mathe-matics teachers.

Research has shown that, when teachers do createwritten lesson plans, the most common form of theseplans are outlines or lists, followed by narratives. Thiswas noted in the work of Clark and Yinger (1979), whofound that “the most common form of written plan wasan outline or list of topics to be covered” (p. 15).Additionally, Sardo-Brown (1988) noted that at theyearly, unit, and weekly planning stage, teachers’ planswere ‘sketchy outlines” or “lists and notes” (p. 77). Atthe daily planning level, teachers’ plans were “detailednotes” (p. 77) about what was to happen in the courseof the class. Furthermore, McCutcheon (1980) notedthat teachers planning resembled a grocery list, just ajotted list of reminders, as opposed to a guide forinstruction. According to research, teachers often indi-cate that they do not use the lesson plan format thatthey were taught in their teacher preparation programsor those that were created and marketed by experts(Searcy & Moroney, 1996). Rather, they develop theirown method and system for lesson planning that ishighly individualized and not based on research.

Taking into consideration the focus most teacher-preparation programs put on the development of lessonplans, it seems that once teachers strike out on theirown and are no longer required by instructors to writeplans, many alter the form of their planning or stopwriting plans altogether. However, this doesn’t meanthat teachers aren’t planning for instruction beforeentering the classroom. Nevertheless, it does make itextremely difficult to study the planning habits of teach-ers if they are not in written form. In order to study theeffectiveness of planning, the researcher needs first toknow what decisions teachers entertain, dismiss, andaccept as they think about each lesson.

Planning for Student Learning Over the course of a twenty-year period, multiple

studies have been conducted that indicate a pervasivetrend in the lesson planning habits of teachers. Thesestudies show that a majority of teachers focus their les-son planning on classroom activities rather than on stu-

dent learning objectives (Peterson, Marx & Clark, 1978;Clark & Yi n g e r, 1979; Mintz & Ya rg e r, 1980;McCutcheon, 1980; Sardo-Brown, 1988). While eachstudy has its own set of limitations, taken as a wholethey provide evidence that teachers may be unknowing-ly undermining their own best efforts at effective math-ematics teaching by placing the focus of instruction onwhat is to be done by the students and teacher during thelesson rather than what must be learned by the studentas a result of the lesson.

Peterson, Marx, and Clark (1978) recruited twelveexperienced in-service teachers to plan and teach asocial studies lesson to three staged classes, each day forthree days, outside of the regular school schedule.“Each teacher was given a list of objectives and socialstudies text material, and was told to teach the materialin any way he or she wished” (p. 419). Each teacher hadexactly 90 minutes to plan at the beginning of each dayand was asked to use the ‘think aloud’ method so thatinvestigators could analyze their thinking and decisionmaking processes while planning for instruction. Theresults were stunning. “Even though the teachers wereprovided with a list of desired cognitive and affectivestudent objectives, they did not refer to them in theirplanning, nor did they relate their choice of instruction-al processes to learning objectives” (p. 424). It wasfound that teachers focused most of their planning onthe content and activities, and the least amount ofthought was given to student learning objectives.

Clark and Yinger (1979) conducted both a surveyof elementary school teachers as well as a field study ofteacher planning activities. In the teacher survey, 78 ofthe 300 teachers who were mailed a survey completedand returned it, for a response rate of only 26%. Thoseteachers returning the survey self-reported that studentlearning objectives are not the basis of developing les-son plans; activities lead what happens in the classroomduring any given lesson. In the field-study, six volunteerelementary school teachers (four individual teachersand one teacher-pair) were followed over the course offive weeks while they planned and implemented a two-week reading lesson that they had never taught before.Teachers were asked to keep a journal of their planningand were also interviewed on a bi-weekly basis. Clarkand Yinger found that “rather than moving from wellspecified and carefully stated objectives and proceedingto designing activities to meet these objectives, ourteachers more commonly began with a general idea” (p.18).

Supporting this work by Clark and Yinger is a studyof 68 elementary school teachers conducted by Mintzand Yarger (1980). They attempted to analyze written

47


lesson plans by asking in-service teachers to plan read-ing instruction for a simulated class of students.However, the researchers did not expect the teachers toimplement the plans as part of this study and, therefore,were able to have the participants plan for one of threetime periods: an entire school year, a span of severalweeks, or a single class period. The researchers con-cluded that teachers focus their efforts during lessonplanning on what content is to be taught by the teacher,rather than on what content is to be learned by the stu-dent. “There was little mention of objectives” (p. 8). Inaddition, there is a study completed by McCutcheon(1980), focusing on the nature of the planning done byelementary school teachers. During this research, it wasreported that “teachers usually listed objectives for a les-son in their planbooks only if required to do so by theprincipal.” (p. 6). The teachers commented that objec-tives were provided in the teachers’ manuals, thus theyfelt that they didn’t need to indicate in their plans whichobjectives they were going to attend to in each lesson.Also, teachers in the study noted that their planningwas impeded by the fact that they lacked sufficientcoursework on planning in both preservice educationand in-service professional development.

In Sardo-Brown’s (1988) research study on theplanning habits and decisions of 12 in-service middleschool teachers, participants were required to docu-ment their planning throughout the course of a schoolyear in an effort to describe the real decisions aboutactual course curricula that teachers implement. Thisresearch involved multiple collections of different typesof data over a period of several months. Each of the 12middle-school teachers who participated were inter-viewed a total of four times, once each at the yearly,unit, weekly, and daily levels of planning. The teachersalso completed a written questionnaire, were asked to‘think-aloud’ while planning their daily lesson and havethe planning audio-taped, and submitted copies of theiryearly, unit, weekly, and daily lesson plans to theresearcher. Over the course of the four-month study, theresearcher gleaned from the data that teachers did notwrite student learning objectives while planning dailylessons. Instead, the teachers’ “first considerations inplanning were what instructional strategies to use” (p.78).

In contrast, Searcy and Maroney (1996) conducteda survey of special education teachers in Iowa, asking556 teachers to self-report on their own lesson planningmethods. Of the 207 teachers who completed andreturned a survey, approximately one-quarter noted thatthey did not write down student objectives when creat-ing lesson plans. On the other hand, over half of the

teachers claimed that they consciously plan studentlearning objectives; however, they just don’t write themout. As this study encompassed teachers from preschoolthrough twelfth grade, Searcy and Maroney were able tonote that, for this study, planning practices were not sig-nificantly different at the different grade levels. Onceagain, this result may be because of the special educa-tion teacher not always being the primary instructor inthe classroom.

In addition, during a survey of 33 K–12 teachers,many of the teachers reported making decisions basedon student learning objectives (Sardo-Brown, 1990). Amultiple choice survey, a questionnaire, and an inter-view were all completed by the participants. However,several factors may have influenced these findings.These teachers were concurrently enrolled in a graduatedegree program, with half of the participants of thestudy enrolled in a class taught by the researcher. Theseteachers were not volunteers; rather, they were requiredto participate as part of their degree program. Theseteachers also were all employed by a single school dis-trict which required teachers to use a specific lesson-plan template. It was noted that this template includedthe setting of student objectives. It is, therefore, plausi-ble that the data collected from these teachers does notrepresent the teachers’ preferred approach to lessonplanning, but instead is a response either to the schooldistrict’s lesson planning requirements or the desire tomake themselves look better in the eyes of theresearcher.

Limitations aside, some re s e a rch does supportSardo-Brown’s assertion that many teachers do refer tostudent learning objectives when planning for instruc-tion. During a case-study of two experienced elemen-tary school teachers, Reiser and Mory (1991) notedthat, of the two teachers, the one who had completed agraduate-level course in instructional design tended touse the systematic lesson planning model that she wastaught as she planned her own lessons. The teacher whohad not received formal training in instructional designdid not use any systematic method for lesson planning.However, for the purpose of this study, the researchersonly observed the planning activities of these two teach-ers as they related to the teaching of science lessons. Aselementary school teachers, these teachers were respon-sible for teaching most of the subjects that their stu-dents studied. While it is plausible that the teachersused a consistent method of planning for all subjectsthat they taught, it is possible that the teacher who didnot use a systematic method for planning her sciencelessons may have been influenced by factors other thana lack of formal training, such as a deficiency in her

48


content knowledge or a general dislike of the subject ofscience. Finally, if the focus of this study had been onplanning for mathematics instruction rather than sci-ence, the results may have been significantly different.

It should be noted that, although there have beenmany studies conducted in an effort to describe thedecisions that teachers make during the lesson planningp rocess, none of the literature reviewed here hasfocused on mathematics teaching. While it can beimplied that research into the planning practices of ele-mentary school (Clark & Yinger, 1979; Mintz & Yarger,1980; Reiser & Mory, 1991), middle school (Sardo-B rown, 1988), and special education (Searcy &Maroney, 1996) teachers has implicitly included plan-ning for mathematics classes, any effort to improvemathematics instruction should include research con-ducted explicitly on the lesson planning practices anddecisions of mathematics teachers.

Given the overwhelming disregard for writing stu-dent objectives, it is plausible that there is little congru-ency between what students learn and what is intendedto be implemented in the classroom. If teachers do notconsider student learning outcomes and subsequentassessments as they plan, they will not be able to recog-nize whether or not they are teaching effectively. Fewstudies have examined the link between comprehensivelesson planning and instructional effectiveness but,from those which have, there are indications of a closeassociation. This connection will be discussed later inthe paper.

Planning in Countries Other Than the United StatesResearch from other countries shows a similarly

uneven approach to lesson planning. The internationalliterature does not reveal any one universally acceptedsystem of lesson planning. In Spain, Sanchez andVa l c a rcel (1999) perf o rmed re s e a rch that involvedinterviewing 27 science teachers. These teachers taughteither 11–13 year-olds or 13–17 year-olds and heldeither a diploma in education or a science degree.According to the researchers, there was no reported dif-ference between the groups as pertains to planning.Analysis of the interviews showed that 92% of teacherswrote some type of document to go along with their les-sons, even if the entire lesson plan was not written out.Of these teachers, 76% stated that what they wrote“consisted of notes and guidelines for the teacher’s ownuse on the content to be taught and how the lessonswere to run” (p. 500). However, this number may beartificially high based on the context in which the inter-views were conducted. The interviewed teachers wereattending “an in-service training course on teaching

unit design……following the latest reforms in……edu-cation” (p. 496). Asking teachers to describe their ownplanning decisions in the midst of teaching them are f o rm planning method may have influenced theteachers to over-report their own use of the desiredattributes in an effort to please the researchers.

Research has shown that, even when teachers arerequired to plan around student learning objectives,they may not hold much faith in the usefulness of thispractice. Yildirim (2001) studied the responses ofalmost 1,200 Turkish school teachers to a surv e y“designed to explore teachers’ perceptions of their plan-ning at the primary school level” (p. 7). He noted thatthe Turkish education system had strict rules for thewriting of lesson plans at the yearly, unit, and daily lev-els, and that all of these planning levels included stu-dent learning objectives. In his analysis, it was foundthat when teachers were asked to rate what componentsof daily lesson plans they perceived as most important,writing student learning objectives rated lowest. “Manyteachers did not find writing behavioral objectives use-ful in planning” (p. 18) even though they were requiredto do it at all levels of instructional planning. Teacherswere more concerned with how and what to teach, thecontent of the lessons, than with writing student learn-ing objectives for the lesson or the evaluation of stu-dents based on those objectives. While Yildirim notesthat the planning of teachers in Turkey is strictly moni-tored, he does not mention whether or not the class-room teachers are required to have had formal trainingin the components of lesson planning, such as develop-ment of student objectives for instruction and assess-ment.

Preservice teachers at Oxford University in Englandwere studied by John (1991) in an effort to identify bothhow they plan and what they take into considerationwhile planning. Five preservice teachers were chosen atrandom; they participated in open-ended interviewsand think-aloud activities, as well as provided all les-son-planning materials to the researcher. John notedthat the writing of objectives was not seen as havinggreat importance to the student teachers. Similar toresearch previously identified, these teachers tended tofocus on the topics, resources, and activities of a lessonwhen planning. John notes that the Tyler (1949)method of lesson planning does not resemble the think-ing and actions of teachers in practice. He notes that“teacher educators should begin to question the Tylerianor overtly theoretical approach to planning in the lightof the evidence cited in this paper and elsewhere” (ForTeacher Education section, ¶¶1). What it seems he issuggesting is that teacher preparation programs and

49


schools of education stop teaching lesson planning asan objectives-first process and begin teaching it so as tom o re closely resemble what teachers have beenobserved to already do while planning. However, Johndoes not take into account in this research two veryimportant issues, specifically, whether or not these fivepreservice teachers were properly taught how to planfollowing an objectives-first method, and whether ornot they had successfully mastered this skill. Also, thereis no evidence that these teachers, who did not correct-ly use the Tylerian method of lesson planning, wouldfind more success with other methods.

In summary, the Turkish teachers were required tofollow a strict formal structure for their lesson planswhile the Spanish teachers had a much more flexiblesystem. The English teachers were found to engage inlesson planning activities that were quite different fromthose prescribed by educational theorists. The diversityof lesson planning formats and the complexity of indi-vidual teachers’ planning activities point to some of thedifficulties in studying lesson planning. While obvious-ly not an exhaustive review of the literature of interna-tional lesson planning, this small sample does identifysome themes that are common to teaching in class-rooms across the globe. Teachers in other countriesseem to be comparable to those in the United States interms the structure of their written plans, their use ofstudent learning objectives to guide instruction andassessment, and their perception of the usefulness ofthose objectives.

Common Lesson Planning ModelsAs stated previously, any effort to improve mathe-

matics instruction should include research conductedexplicitly on the lesson planning practices and decisionsof mathematics teachers. The term lesson planning, how-ever, is very broad and can include any number of dif-ferent activities that teachers carry out before instructionoccurs. Therefore, it is necessary to briefly summarize anumber of different methods for lesson planning provid-ed in the literature as well as a range of formats commer-cially available to teachers. Cruickshank, Bainer, andMetcalf (1999) identified a sample of six different typesof lesson planning formats including those by Callahan,the Center for Vocational Education, the Air Force, Eby,Gange and Briggs, and Hunter, as well as their own “rec-ommended lesson plan format” (p. 147). While there areseveral components that many of these models have incommon, such as resources and assessments, there isonly one that they all have in common. Each of theselesson planning formats includes an objectives compo-nent. It is presumed that each of the methods referenced

by Cruickshank, Bainer, and Metcalf have tailored theirplanning method to a specific audience, for example,vocational school instructors, Air Force instructors, oracademic teachers and, therefore, each method willinclude different lesson planning components specific tothat audience’s content. For the purpose of this paper,planning models not specifically designed for the aca-demic classroom will not be considered.

Much has been written about daily lesson planningfor academic school teachers. An early book publishedon the topic is Tyler’s Basic Principles of Curriculum andInstruction (1949). In it, he states that there is a four-step process to planning before instruction. First, statethe objectives. Second, select learning experiences forthe students that are likely to promote the attainment ofthese objectives. Third, organize the learning experi-ences so as to best facilitate the attainment of the statedobjectives. Fourth, evaluate the learning experiences todetermine whether or not the stated objectives weremet. Decades later, this process for lesson planning wasrenewed and re i n f o rced by the writings of Mager(1975), Gagne and Briggs (1979), and Airasian (2001).These are all authors that have included approximatelythe same steps in their respective recommendations forteachers to attend to while planning daily lessons.

Many of the various formats include a section onmaterials or resources, and/or include a section for eval-uation or assessment. The act of stating lesson objec-tives seems to be the only common element in manydifferent methods for lesson planning that have beendescribed here. However, each planning procedureplaces the writing of objectives at a different ‘step’ intheir plan. Many of the lesson plan formats do not beginwith stating students learning objectives. Interestingly,each lesson plan format mentioned is presented in themanner that the plan will be enacted in the classroom.Since no notation is made to the contrary, it can beassumed that the steps presented in each form are in theorder that the authors also recommend the lesson beplanned.

In an effort to focus explicitly on improving theinstruction provided to students in mathematics classes,Panasuk, Stone, and Todd (2002) present a lesson plan-ning format that is tailored specifically for the dailyplanning of mathematics lessons. The first, and mostimportant, difference to note between this method andthe other previously mentioned is that the order forplanning instruction is different than the order fordelivery. The components for developing a lesson planas stated in the Four Stages of Lesson Planning (FSLP)model are (in order) objectives, homework, develop-mental activities, and mental mathematics. This order

50


for development proceeds directly from the literature onobjectives-based instruction and students’ prior knowl-edge. The literature reviewed previously shows possibleconnections between teachers’ use of objectives-basedi n s t ruction and task analysis, along with teachers’awareness of students’ prior knowledge, and more effec-tive classroom instruction.

During a statewide Middle School MathematicsInitiative, middle-school teachers were trained in theFSLP model. Panasuk, Stone, and Todd (2002) chose apurposive sample of three teachers from the larger pop-ulation to complete a questionnaire and be interviewedabout the implementation of the FSLP strategy in theirown classrooms. The researchers found that “thought-fully constructed……lesson plans……helped theteachers to become effective classroom leaders and facil-itators of efficient use of classroom time” (p. 825). Theyalso noted that “the teachers accomplished instructionbased upon established [student learning] objectives”(p. 825). One of the teachers noted that, as she planned,she began to recognize the gaps in her own contentknowledge. Also, using the FSLP caused her to “thinkmore often about connections among mathematics con-cepts” (p. 819).

In further research conducted into the FSLP strate-gy by Panasuk and Todd (2005), a link was shownbetween the explicit writing of student learning objec-tives and effective teaching as determined by directobservation of researchers. Thirty-nine teachers fromthe original Middle School Mathematics Initiative par-ticipated in a smaller study in which they engaged in apre-observation conference, an observation by a trainedspecialist teacher, and a post-observation conference. Inaddition, a total of 261 written lesson plans were devel-oped and collected for the study; each lesson plan andobservation was then individually scored using a rubric.Panasuk and Todd found that when written lesson plansincluded explicitly stated learning objectives for stu-dents, lessons also included multiple representations ofconcepts, teaching was in line with state curriculumframeworks, and possible or common student miscon-ceptions had been considered by the teacher beforebeginning instruction. Thorough lesson planning in thisstudy led to lessons that were considered by theresearchers to be highly effective in achieving theintended learning outcomes.

SummaryEach of the lesson planning formats described here

recognizes the necessity of teachers identifying studentlearning objectives prior to instruction. These formatsalso rely on teachers having a strong content knowledge

background, a large supply of possible instructionalactivities, and the creativity to design appropriate les-sons for their students. However, only the FSLP methodput forth by Panasuk, Stone, and Todd (2002) meets thepreviously stated definition of comprehensive lessonplanning. Lessons developed using the FSLP method:

1. are written explicitly based on student learningobjectives;

2. include assessments of the students based on thestated objectives;

3. require teachers to perform task analyses to iden-tify the prior knowledge of students, as well asthe prerequisite skills and knowledge necessaryto complete the stated objectives; and

4. have consistency throughout all phases of thelesson.

The case is therefore made that, all other thingsbeing equal, lessons planned using the FSLP methodwill lead to more effective mathematics instruction thanthose that are not.

CONCLUSIONS AND POTENTIAL RESEARCHQUESTIONS

In order for teachers to provide the best possibleeducation to their students, classroom instruction needsto be effective. The research has provided several char-acteristics that are common to effective mathematicsinstruction. The first is the need for teachers to possessa high level of content knowledge. The second is thatinstruction in mathematics must be based explicitly onstudent learning objectives. Lastly, the prior knowledgeof the students and the necessary subskills of the con-tent must be taken into account. Instruction that con-tains these three characteristics is likely to be moreeffective than instruction that is lacking in one or moreof these.

Mathematics lessons planned in a comprehensivemanner may lead to more effective classroom instruc-tion. This is because this method of planning forcesteachers to recognize and attend to their own content-knowledge deficiencies. It requires that teachers base allinstructional activities and assessments in the lesson onexplicitly stated student learning objectives. Also, thismethod of planning leads teachers to analyze the statedobjectives in order to determine the best methods tobridge the gap between student prior knowledge andthe new learning. Because of the focus on these threecharacteristics identified by researchers as being effec-tive at improving student achievement, it is possible tohypothesize that teachers may improve their classroom

51


instruction by implementing comprehensive planningmethods in their classrooms.

Colleges of teacher education expect pre-serviceteachers to plan lessons and assume that these noviceteachers will continue this practice as they begin theirteaching careers. Yet, the literature indicates that theydo not. In addition, teachers who have entered theclassroom through alternative methods of licensure mayhave little to no training or knowledge of lesson plan-ning methods. Unfortunately, once in the classroom,these teachers are often left to their own devices whenit comes to planning for instruction. Research into pro-fessional development programs for in-service teachersaimed at improving lesson planning in mathematics iswarranted due to the potential for positive effects onboth classroom instruction and student learning. Thefollowing research questions are suggested:

1. Does the particular method that teachers use toplan lessons impact the effectiveness of theirclassroom instruction?

2. What attitudes or habits held by teachers mayaid or impede their adoption of comprehensivelesson planning methods?

3. Is there a significant difference in achievement ona specific unit of content between the students ofteachers who have been taught to plan using theFour Stages of Lesson Planning method andthose teachers that have not?

ReferencesAirasian, P.W. (1994). Classroom assessment (2nd ed.). New York:

McGraw Hill.

Carnine, D. (1980). Preteaching versus concurrent teaching of the

component skills of a multiplication algorithm. Journal for

Research in Mathematics Education, 11(5), 375-379.

Clark, C. M., & Yinger, R. J. (1979). Three studies of teacher plan-

ning. research series no. 55. Paper presented at the Annual

Meeting of the American Educational Research Association,

San Francisco, CA. (ERIC Document Reproduction Service

No. ED175855)

Cruickshank, D.R., Bainer, D.L. & Meltcalf, K.K.(1999). The act of

teaching, (2nd ed.). Boston: McGraw-Hill College.

DeLong, M., Winter, D., & Yackel, C.A. (2005). Student learning

objectives and mathematics teaching. Primus: problems,

resources, and issues in mathematics undergraduate studies, 15(3),

226-258. Retrieved July 12, 2007, from the WilsonWeb data-

base.

Fernandez, C., & Cannon, J. (2005). What Japanese and U.S. teach-

ers think about when constructing mathematics lessons: A pre-

liminary investigation. Elementary School Journal, 105(5), 481-

498.

Fowler, F. C., & Poetter, T. S. (2004). Framing French success in

elementary mathematics: Policy, curriculum, and pedagogy.

Curriculum Inquiry, 34, 283-314.

Gagnéé, R. M. (1973). Learning and proficiency in mathematics. In

F. Crosswhite, et al. (Eds.), Teaching mathematics: psychological

foundations (pp. 158-165). Worthington, OH: Charles A. Jones

Publishing Company. (Original work published 1963)

Gagnéé, R. M. (1973). Some new views of learning and instruc-

tion. In F. Crosswhite, et al. (Eds.), Teaching mathematics: psy-

chological foundations (pp. 107-115). Worthington, OH:

Charles A. Jones Publishing Company. (Original work pub-

lished 1970)

Gagne, R.M. & Briggs, L.J. (1979). Principles of instructional design

(2nd ed.) New York: Holt, Rinehart, and Winston.

Good, T.L. & Grouws, D.A. (1979). The Missouri mathematics

effectiveness project : An experimental study in fourth-grade

classrooms. Journal of Educational Psychology, 71(3), 355-362.

Ingersoll, R. (2001). Teacher turnover and teacher shortages: An

organizational analysis. American Educational Research Journal,

37, 499-534.

John, P.D. (1991). A qualitative study of british student teachers’

lesson planning perspectives. Journal of Education for Teaching,

17(3), 301-320. Retrieved July 12, 2007, from the EBSCOhost

database.

Kagan, D. M., & Tippins, D. J. (1992). The evolution of functional

lesson plans among twelve elementary and secondary student

teachers. The Elementary School Journal, 92(4), 477-489.

Kennedy, M. F. (1994). Instructional design or personal heuristics

in classroom instructional planning. Educational Technology,

34, 17-25.

Laczko-Kerr, I., & Berliner, D. C. (2003). In harm’s way: How

undercertified teachers hurt their students. Educational

Leadership, 60(8), 34-39.

Mager, R.F. (1975). Preparing instructional objectives (2nd ed.).

Belmont, CA: Fearon Publishers, Inc.

Manouchehri, A., & Goodman, T. (2000). Implementing mathe-

matics reform: The challenge within. Educational Studies in

Mathematics, 42(1), 1-34.

Massachusetts Department of Education (2006). Retrieved April

22, 2008 from

http://www.doe.mass.edu/infoservices/reports/retention/0405/r

eport.pdf

52


McCutcheon, G. (1980). How do elementary school teachers plan?

The nature of planning and influences on it. Elementary School

Journal, 81(1), 4-23.

Minor, L. C., Onwuegbuzie, A. J., Witcher, A. E., & James, T. L.

(2002). Preservice teachers’ educational beliefs and their per-

ceptions of characteristics of effective teachers. Journal of

Educational Research, 96(2), 116-127.

Mintz, S. L., & Yarger, S. J. (1980). Conceptual level and teachers’

written plans. Paper presented at the Annual Meeting of the

American Educational Research Association, Boston, MA.

(ERIC Document Reproduction Service No. ED189030)

Morine-Dershimer, G. (1977). What’s in a plan? Stated and unstated

plans for lessons. Paper presented at the Annual Meeting of the

American Educational Research Association, New York, NY.

(ERIC Document Reproduction Service No. ED139739)

National Center for Education Statistics, United States Department

of Education. (2000). Pursuing Excellence: Comparisons of

International Eighth-Grade Mathematics and Science Achievement

from a U.S. Perspective, 1995 and 1999. NCES 2001–028.

Figure 26. Washington, DC: U.S. Government Printing Office.

National Council of Teacher of Mathematics. (2000). Principles and

standards for school mathematics. Reston, VA: National Council

of Teachers of Mathematics.

No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107-110,

§§115, Stat. 1425. (2002).

Panasuk, R., Stone, W., & Todd, J. (2002). Lesson planning strate-

gy for effective mathematics teaching. Education, 122(4), 808-

827, 714.

Panasuk, R. M., & Todd, J. (2005). Effectiveness of lesson plan-

ning: Factor analysis. Journal of Instructional Psychology, 32(3),

215-232.

Peterson, P. L., Marx, R. W., & Clark, C. M. (1978). Teacher plan-

ning, teacher behavior, and student achievement. American

Educational Research Journal, 15(3), 417-432.

Reiser, R. A., & Mory, E. H. (1991). An examination of the system-

atic planning techniques of two experienced teachers.

Educational Technology Research and Development, 39(3), 71-82.

Reiser, R. A. (1994). Examining the planning practices of teachers:

reflections on three years of research. Educational Technology,

34, 11-16.

Sáánchez, G., & Valcáárcel, M. V. (1999). Science teachers’ views

and practices in planning for teaching. Journal of Research in

Science Teaching, 36(4), 493-513.

Sardo-Brown, D. (1988). Twelve middle-school teachers’ planning.

Elementary School Journal, 89(1), 69-87.

Sardo-Brown, D. (1990). Experienced teachers’ planning practices:

A U.S. survey. Journal of Education for Teaching, 16(1), 57-71.

Searcy, S., & Maroney, S. A. (1996). Lesson planning practices of

special education teachers. Exceptionality, 6(3), 171-187.

Stigler, J. W., & Hiebert, J. (1999). The teaching gap: Best ideas from

the world’s teachers for improving education in the classroom. New

York, NY: The Free Press.

Strangis, D. E., Pringle, R. M., & Knopf, H. T. (2006). Road map

or roadblock? Science lesson planning and preservice teach-

ers. Action in Teacher Education, 28(1), 73-84.

Torff, B., & Sessions, D. N. (2005). Principals’ perceptions of the

causes of teacher ineffectiveness. Journal of educational psychol-

ogy, 97, 530-537.

Tyler, R.W. (1949). Basic principles of curriculum and instruction.

Chicago: The University of Chicago Press.

Wenglisky, H. (2002). How schools matter: The link between

teacher classroom practices and student academic perform-

ance. Education Policy Analysis Archives, 10(12). Retreived April

24, 2007 from http://epaa.asu.epaa/v10n12/

Yildirim, A. (2001). Instructional planning in a centralized school sys-

tem: An assessment of teachers’ planning at primary school level in

Turkey. Paper presented at the Annual Conference of American

Educational Research Association, Seattle, WA. (ERIC

Document Reproduction Service No. ED453192)

53


ABSTRACTThis paper discusses the use of pedagogicala p p roaches (called interactive-engagement (IE)strategies) that have been found to be more effectiveat facilitating conceptual understanding than tradi-tional instructional techniques. Commonalitiesbetween IE approaches are identified. The claimsre g a rding diff e rences between IE and traditionalinstructional strategies have been justified by theanalysis of Force Concept Inventory (FCI) pretestand posttest data through the computation of averagenormalized gain <g>. The average normalized gainfor a course is the ratio of the average actual gain <G>to the maximum possible average gain. The FCI is aformative assessment that has been used as an instru-ment to assess the effectiveness of instru c t i o n a lmethodologies at facilitating conceptual understand-ing among high school students taking introductoryphysics courses that cover Newtonian Mechanics. Itis acknowledged that interactive-engagement tech-niques require more time than traditional teachingtechniques and therefore result in less content cover-age for equal student-teacher contact time.

Reflective journal writing, an activity shown tofacilitate conceptual understanding in other learningenvironments, is also reviewed. The evidence report-ed regarding graded reflective writing by introducto-ry level physics students suggests that reflective jour-nal writing can facilitate understanding, promotemetacognitive engagement leading to the learnermaking connections between learning events andprior knowledge, and create a space for dialoguebetween the teacher and student. These are alsoattributes of IE techniques. It is suggested that stu-dent engagement in daily reflective journal writing,coupled with a weekly reflective summary, each ofwhich is a graded component of the course, mayfacilitate a more time-efficient use of IE techniques.This could effect a reduction in the content coveragedeficit resulting from implementing IE strategies,while facilitating the increased conceptual under-standing associated with IE methods.

INTRODUCTION TO THE PROBLEMAccording to the American Institute of Physics

(AIP) the trend in physics education is toward increasedenrollment. There has been an increase in the percent-age of high school graduates that have completed aphysics course, from less than 18% in 1986 to approx-imately 33% in 2001 (AIP Statistical Research Center, ascited in Hehn & Neuschatz, 2006) . The developmentof the conceptual approach to teaching physics, which

allows for new instructional techniques involving moreactive student engagement in the learning process, is afactor contributing to the increased high school physicsenrollment. Historically, high school physics classeswere likely to be populated by students that had exhib-ited the highest academic perf o rmance (Hehn &Neuschatz, 2006). The implication here is that theincreased enrollment comes from the segment of thestudent population that is more dependent upon theeffectiveness of instructional technique to develop con-ceptual understanding than has historically been thecase. Other trends identified were an increase inemphasis on what is learned, rather than what is taught,and increased use of formative assessments to guideinstruction (Hehn & Neuschatz, 2006).

Newton’s laws of motion, a central topic of intro-ductory physics courses, are mediated through the con-cept of force. Many students enter into a study ofNewton’s laws with an existing schema that interactswith the instruction, and thereby may modify theintended learning outcome (Watts & Zylberztajn, 1981;Halloun & Hestenes, 1985a). These commonly heldand frequently scientifically inaccurate beliefs are theresult of personal observations and intuitive thoughtprocesses that have not been exercised in a rigorous andthorough manner aimed at deriving a comprehensiveunderstanding of the underlying principles responsiblefor producing the observed phenomena (DiSessa, 1988;Driver, 1989; Driver, Guesne, & Tiberghien, 1985; West& Pines, 1985). In both science education research lit-erature and physics education research literature, onewill encounter many similar terms, each referring to theexistence of notions regarding the concept of force priorto instruction. The abundance of essentially synony-mous terms (see Appendix A for a partial listing) refer-ring to the same phenomenon is indicative of the preva-lence of its occurrence.

Research indicates that the pre-instruction under-standings derived from a common-sense approach tooperating within the demands of daily living can con-tain internal inconsistencies, which do not seem to berecognized by the individual (Clement, 1982; Halloun& Hestenes, 1985a; Redish, 1994). Traditional intro-ductory physics instruction produces minimal changein the beliefs derived from common-sense (Clement,1982), even when students have successfully learnedp roblem-solving algorithms (Halloun & Hestenes,

Interactive-Engagement and Reflective WritingMichael Doherty

Andover High School

54


1985b). Traditional instruction refers to a classroom inwhich the teacher presents the facts, and students uti-lize drill and practice activities leading to end of thechapter assessments (Russell, 1993).

The recognition that existing student understand-ings can mediate the effect of instruction, and that thetraditional model of introductory physics instruction isineffective at replacing student preconceptions withNewtonian alternatives for the majority of students(McDermott, 1984; Halloun & Hestenes, 1985b), led tothe development of the FCI (Hestenes, Wells, &Swackhamer, 1992). The FCI (in Appendix B) is a thir-ty item multiple choice assessment that offers fourAristotelian (common-sense) choices, and oneNewtonian response to each question. The FCI hasbecome the standard formative assessment and instruc-tional technique evaluation instrument within thephysics education research community, when an evalu-ation of learning with respect to Newtonian Mechanicsis conducted. It is commonly utilized in a pre-instruc-tion, post-instruction pairing that is analyzed to meas-ure student gain with respect to learning Newtonianconcepts. Gain <g> (the average normalized gain) is cal-culated by dividing the actual average gain (% posttest- % pretest) by the maximum possible average gain(100 - % pretest) (Hake, 1998a). The purpose of aformative assessment is to probe the state of studentlearning; then the teacher can adjust instruction andlearning activities to meet student needs (Dufresne &Gerace, 2004). A summative assessment exists to pro-vide a measure for a person outside of the classroom(Starkman, 2006). This could be a report card grade fora parent or an SAT score for college admission consid-eration.

Interactive teaching strategies can produce greatergains in conceptual understanding than traditionalmethods of instruction (Hake, 1998a; Mazur, 1997).Interactive teaching strategies are defined as “thosedesigned at least in part to promote conceptual under-standing through interactive engagement of students inheads-on (always) and hands-on (usually) activitieswhich yield immediate feedback through discussionwith peers and/or instructors” (Hake, 1998a, p. 65).Other advantages associated with interactive-engage-ment pedagogies are these: student-teacher and stu-dent-student discussions probe student conceptualiza-tions, allowing for the recognition of and reflection onstudent prior notions (Mestre, 1994); the constructionof explanations of ideas by students promotes concep-tual change (Hewson & Beeth, 1993); the instructorgains insight into what students are thinking becausethey express it in words (Redish, 1994); more effective

development of complex reasoning skills occurs(Johnson, Johnson & Smith, 1991). In addition, there isimproved grasp of material (Crouch & Mazur, 2001);improved quantitative problem solving (Hake, 1998a;Thacker et al., 1994); significant reduction of the per-formance gap associated with gender (Lorenzo, Crouch,& Mazur, 2006); and higher success rates, particularlyfor females and minorities (Beichner, et al., 2007).

If all other factors were equal then there would existlittle incentive for teachers to use traditional instruc-tional techniques. But all other factors are not equal.Non-traditional instructional techniques require moretime than traditional instruction and therefore result inless content coverage within a physics course (Crouchet al, 2007; Laws, 1991; Meltzer & Manivannan, 2002).In the high school environment there are physics cours-es that are often followed by nationally administereds t a n d a rdized tests. Examples include AdvancePlacement (AP) Physics leading to the AP Examination,and honors physics leading to the SAT II PhysicsExamination. These assessments draw upon a broadsurvey curriculum, and those students that are notexposed to the range of potential question topics maybe disadvantaged compared to students that havereceived instruction in the entirety of the prescribedcurriculum. Performance on these assessments mayhave consequence within the college admission process(Starkman, 2006).

In summary, re s e a rch indicates that interactiveengagement practices result in greater gains in concep-tual understanding of Newtonian mechanics than tradi-tional instructional techniques. The trend of increasingenrollment in high school physics courses brings with itan increased burden upon the pedagogical practicesused within the classroom. The interactive engagementstrategies require a slower pace of instruction than tra-ditional instructional techniques. The competing yetreasonable demands of choosing instructional strategiesthat favor breadth of coverage to maximize the chanceof a high score on the high stakes, end-of-course,assessment versus teaching less content to facilitate con-ceptual understanding of fewer topics through interac-tive-engagement methods is a decision that the highschool physics teacher encounters.

THE PROBLEMSome high school introductory physics courses

have an end-of-course, national, high-stakes assessmentassociated with them that may influence the collegeacceptance process. These assessments are based on abroad range of curriculum topics that require a rapid

55


rate of progress through the curriculum if all contenttopics are to be covered. Traditional teaching strategiesallow for more content coverage in a given time inter-val, making this instructional approach desirable in anenvironment where breadth of content coverage is ofparamount importance. However, traditional instruc-tion encourages rote memorization of facts and fails toconnect classroom lessons with the students’ existingunderstandings (Mestre, 1994).

Changes in the post-secondary employment markethave resulted in a greater percentage of the high schoolstudent population taking introductory physics courses.The broader range of student capabilities within intro-ductory courses places an even greater burden on themethod of instruction in the classroom with respect tofacilitating student understanding. If the desired out-come of instruction is to go beyond rote memorizationand include conceptual understanding for a greater per-centage of the students within the classroom, then non-traditional instructional methodologies are required.The new pedagogical practices inspired by physics edu-cation research are called interactive-engagement strate-gies. IE teaching strategies result in greater conceptualunderstanding for students learning Newtonianmechanics in intro d u c t o ry physics courses (Hake,1998a). The competing demands of a faster rate of con-tent coverage to maximize the broad range of subjectmatter tested by high stakes national assessments and aslower rate of content coverage to maximize studentconceptual understanding through interactive-engage-ment techniques create a dilemma for the instructor.This paper will consider the possibility that instructionthat incorporates reflective journal writing by studentsmay be able to reduce the content coverage differentialbetween traditional and interactive-engagementinstructional methodologies in introductory physicscourses, while sustaining the types of gains associatedwith interactive-engagement instructional methodolo-gies.

PERTINENT AREAS OF RESEARCHInteractive-engagement strategies used during

introductory physics instruction are discussed. TheForce Concept Inventory, a formative assessment devel-oped within the physics instructional reform movementas a tool to give feedback to the instructor upon com-pleting instruction regarding Newtonian mechanics, isreviewed with respect to its more generalized role with-in the physics education research community. That roleis as an instrument used to assess the effectiveness of aninstructional methodology. Pedagogical reforms (called

IE strategies) inspired by the same research data thatresulted in the creation of the FCI are reviewed in rela-tionship to their relative success at affecting gain in stu-dent understanding of Newtonian Mechanics.Commonalities between these interactive-engagementstrategies are discussed.

Reflective journal writing, a student learning activi-ty shown to contribute to conceptual understanding inother learning environments, such as pre-service train-ing of teachers (Bell, 2001), business management, clin-ical psychology and higher education (Bolton, 2005),but currently underutilized in physics instruction, isreviewed. Interactive-engagement methodologies arenot restricted to verbal interaction but may also includewriting that reflects upon interpersonal as well asintrapersonal dialogue. The attributes associated withreflective journal writing are compared to those identi-fied within interactive-engagement instru c t i o n a lmethodologies. A research project involving the use ofgraded reflective journal entries, to be evaluated withrespect to conceptual gain through the use of the FCI asa pre-instruction, post-instruction assessment, is pro-posed. The effect of reflective journal writing withregards to minimizing loss of curricular content cover-age would also be evaluated.

INTERACTIVE-ENGAGEMENT STRATEGIESFACILITATE CONCEPTUAL UNDERSTANDING

The Force Concept Inventory Assesses ConceptualUnderstandingWithin the science education community physicists

have led the way in measuring how well students learnthe fundamental concepts in intro d u c t o ry courses(Stokstad, 2001). Within the physics education research(PER) community the most commonly utilized forma-tive assessment following instruction on Newtonianmechanics is the Force Concept Inventory (FCI),(Hestenes, Wells, & Swackhamer, 1992). The instru-ment was developed to probe student beliefs regardingforce, the central concept in Newtonian mechanics. TheFCI consists of thirty multiple choice questions thatcover the concept of force with respect to kinematics,Newton’s Laws of motion, gravitation, contact forces,and the superposition principle. Each question requiresa choice from four answers associated with commonsense understandings and one choice based on theNewtonian conceptualization. The four distractors werechosen from a compilation of the most common mis-conceptions articulated by students in exit interviewsfollowing instruction in introductory courses, over anumber of years, at Arizona State University (Halloun &

56


Hestenes, 1985b; Hestenes, Wells, & Swackhamer,1992). The conceptual nature of the questions, whichare based on easily understood situations, allows theinventory to be administered prior to formal instructionin mechanics.

The FCI is published with a table that correlates thefive previously identified Newtonian concepts with thespecific questions that test for understanding regardingthese concepts. Item analysis permits a view intowhether Newtonian or non-Newtonian thinking regard-ing these concepts is exhibited across questions testingfor understanding of the same concept. When the FCIis used by the instructor as a pre-instruction assessmentof students’ existing beliefs regarding force and motion,the existence of common sense notions regarding forceand motion is revealed, and a baseline is establishedagainst which instructional effectiveness can be calibrat-ed through the comparison of pretest/posttest scores.When used exclusively following instruction, the classdata can be analyzed to inform the instructor regardingthe effectiveness of instruction comparatively betweeneach of the five Newtonian concepts by which the ques-tions are grouped. The FCI has been used in bothinstructional and research settings (Hestenes, Wells, &Swackhamer, 1992).

Upon publication, the instrument immediatelywent into use as a metric to measure the effectiveness ofinstruction; in fact, the majority of the original journalpublication (Hestenes, Wells, & Swackhamer, 1992) isdevoted to the analysis of the pretest/posttest data foreight high schools (N > 1500), and three universities (N> 500) that were the subjects of the study. The effective-ness of instruction was assessed through the analysis ofthe difference between the pretest scores and posttestscores, with the class average difference being consid-ered the measure of instructional effectiveness. Theauthors contend that a score of 80% indicates thethreshold of mastery of Newtonian concepts (regularlyengaging in Newtonian thinking). A score of 60% canbe considered to be the threshold indicator ofNewtonian thinking (beginning to engage in Newtonianthinking) for students in an intro d u c t o ry physicscourse, and that without achieving a minimum under-standing of Newtonian mechanics, quantitative prob-lem-solving activities are meaningless exercises involv-ing rote memorization of the algorithms associated withforce and motion (Hestenes, Wells, Swackhamer, 1992).These percentages were arrived at by correlating theposttest FCI scores with a diagnostic mechanics test(the Mechanics Baseline Test; Hestenes & Wells, 1992)developed to test quantitative problem-solving skillsfollowing intro d u c t o ry instruction in Newtonian

mechanics. Each participant in the study took both theFCI and the mechanics test upon completion of instruc-tion in mechanics. The fact that the class average of cor-rect responses on the post-instruction administration ofthe FCI for more than 40% of the participants in thisstudy was below the minimum Newtonian threshold (ascore of 60%), and that no class scored a class averageof 80% correct on this test, caught the attention of thephysics education research community (Mazur, 1997).

The Hake Gain Analysis Assesses Effectiveness ofInstructional MethodologiesAfter the initial publication of the FCI, Richard

Hake, Physics Professor at Indiana University, collecteddata over the next six years from high school, college,and university teachers of introductory physics courses,that had used the FCI in the prescribed pretest/posttestmode to gauge student learning regarding force. Thepublication of his meta-analysis of the 62 data sets (N =6542) reports on the average effectiveness of a course atpromoting conceptual understanding, which is definedas the average normalized gain <g> (Hake, 1998a).Average normalized gain for a course is defined as theratio of the posttest class average % – pretest class aver-age % to 100% – pretest class average. In reporting theresults of the meta-analysis, Hake (1998a) grouped thedata by instructional methodology for these courses.The range of scores for course averages was categorizedas: high gain (>�.7); medium gain (.7 > g >�.3); and lowgain (< .3). The fourteen traditionally taught coursesyielded an average gain of .23 while the forty-eightcourses incorporating some form of interactive-engage-ment (IE) strategies yielded an average gain of .48, closeto two standard deviations above the traditionallytaught courses. Other re s e a rchers re p o rting similaraverage gain score differences between courses utilizingtraditional and interactive-engagement methodologiesinclude: Redish & Steinberg (1999); Meltzer &Manivannan (2002); Fagen, Crouch, & Mazur (2002);and Royuk & Brooks, (2003). These findings are cred-ited by a number of introductory physics instructorswith influencing curricular reforms regarding instruc-tional methodologies (see Appendix C for a partial list-ing).

There were fourteen high school data sets reviewedby Hake (1998a, 1998b). Four of these courses werecategorized as Traditional. The average normalized gainfor these courses was .255. The ten IE data sets had anaverage normalized gain of .545 (which exceeds thegain average for all IE courses because a few of the com-munity college and university IE courses yielded gainscores in the low category). Two of the traditionally

57


taught courses administered the Mechanics Baseline(MB) test (Hestenes & Wells, 1992) following instruc-tion in mechanics; the average of the two class averageswas 34.5%. Four of the IE taught courses administeredthe MB at the completion of mechanics instruction. Theaverage of the class averages was 54.25%. These resultsare similar to those reported by Hestenes, Wells, andSwackhamer (1992) in that higher scores on the FCIcorrelated with higher scores of the MB, and that theMB scores ran about 15% on average lower than thescores on the FCI. Hake (1998a) reports a positive cor-relation coefficient (r = +0.91) between the course aver-age MB score and course average FCI score. In summa-rizing the comparative results for the entire meta-analy-sis, Hake (1998a) states that “The conceptual and prob-lem-solving test results strongly suggest that the use ofIE strategies can increase mechanics-course effective-ness well beyond that obtained with traditional meth-ods” (pg 71).

Examples if IE Strategies Used in High School PhysicsInstruction Two IE instructional approaches were used within

all ten IE introductory high school physics courses ana-lyzed in the meta-analysis (Hake, 1998a): peer instruc-tion and microcomputer-based labs (MBL) (Thornton& Sokoloff, 1990). Peer instruction, as described byMazur (1997), involves both requiring students to bringa written response to the assigned reading when report-ing to class, and group discussion in lecture. A concep-tual question based upon a common misconception fol-lows each topic described during lecture .Approximately five minutes of small group discussionterminates when the teacher polls the class for answersand explanations, then moves on. The discussions andanswers are not graded. The active engagement of thestudents with each other and with the teacher, as well asthe students’ construction of explanations, has beenc redited with promoting conceptual understanding(Mazur, 1997). It was not reported if the collaborativepeer instruction, as utilized in the ten high school class-es, exactly mirrored the implementation described byMazur, or some form of personalized adaptation byindividual instructors that included small group discus-sion and whole class discussion around questions relat-ed to common student misconceptions. An advantageof group discussion which is focused on a questiondesigned to draw out the existence of a common senseunderstanding, if the student holds such a notion, isthat the conflict that exists between the Newtonian con-ceptualization and the student belief is brought to theforefront, and optimally to resolution. Requiring stu-

dents to articulate and defend an understanding, in amutually respectful manner, within first the group andthen the class, striving for consensus, engages studentssimultaneously with both the concept being discussedand their personal belief re g a rding that concept(Crouch et al., 2007). A disadvantage of using the peerinstruction technique in conjunction with traditionallecture presentation and conceptual tests, as practicedat Harvard University, is that the slower pace that allowsfor student discussion results in a 10-15% loss of con-tent covered within the course. The Harvard physicscourses reported on do have a high-stakes, end-of-course assessment associated with them, a somewhatanalogous condition to many high school introductoryphysics courses. The high-stakes assessment for theHarvard students is the Medical College AdmissionTest. At Harvard the instructor’s solution to this dilem-ma, not covering the entire range of topics from whichtest questions can be selected, is advising the studentsat the beginning of the course that they will have tolearn the other 10-15% of the physics on their own(Crouch, et al., 2007). The combination of the caliber ofstudent matriculating at Harvard University and the useof traditional lecture within the course results in lesscontent coverage loss (10-15%) than might be antici-pated in many other instructional environments. Askinghigh school students to learn some part of the physicscourse on their own prior to sitting for a nationallyadministered examination may not yield the same resultthat occurs at Harvard University.

The micro-computer based laboratory (MBL) is theproduct of Tufts University’s Center for Mathematicsand Science Education and was developed by Thorntonand Sokoloff (1990) as a means of using technology tocrystallize the conflict between student understandingand real-world outcomes for specific cases of motion.

These investigations employ the capacity of com-puters interfaced with motion detectors to displaygraphs of motion in real time. The laboratory investiga-tions, which are used in conjunction with small groupdiscussion surrounding results, and predict and com-pare activities, were found to be effective at reducingstudent misconceptions and improving students’ con-ceptual understanding of force and motion (Laws,1991; Thornton & Sokoloff, 1990). Again Hake(1998b) does not report on the degree of alignment ofthe high school implementations with respect to howthese materials were described in educational researchpublications (Laws, 1991; Thornton & Sokoloff, 1990;Sokoloff & Thornton, 1997). Hake (1998a) does reportthat he collected the original data by soliciting FCIpretest/posttest data sets at colloquia and meetings as

58


well as through e-mail postings on the PHYS_L andPhysLrnR nets between 1992 and 1998. It seems fair toassume that most if not all of the high school instructorsresponding to these requests had probably becomeaware of the MBL program through involvement withPER literature or by participating in National ScienceFoundation funded summer workshops during thattime period that trained teachers to use the MBL mate-rials.

There are multiple advantages associated with usingthe MBL with its capacity to graph, in real time, thecomplete set of motion graphs (displacement, velocity,and acceleration, each with respect to time) stacked foreasy comparison. These include elimination of studente rror that results from poor transposition of datarecorded in table form to graphical representation, andinstantaneous results which allow for immediate discus-sion between lab team members rather than the stu-dents at some later time constructing the graphs (atime-involved activity) without lab team members pres-ent and available for discussion. The time gainedthrough the computer program’s instantaneous graph-ing capability is used in discussion both informallywithin the groups and formally by responding to thes t ru c t u red questions associated with these labs.Discussion of and reflection on the three aspects ofmotion represented graphically present the opportunityfor the student to experience a discrepant event. A dis-crepant event is an outcome that contradicts the out-come the student expects based upon the existingunderstanding (Sokoloff & Thornton, 1997). Placingstudents in the position of resolving discrepanciesbetween the common-sense based predictions and awitnessed outcome is an objective of physics reformmaterials in general, and the MBL activities are anexample of this approach to promoting conceptualunderstanding. In relating the content coverage for auniversity introductory course taught traditionally oneyear to the same course taught by the same instructorthe following year using the MBL activities and otherPER inspired reforms, a 25% reduction in content cov-erage was noted (Laws, 1991).

Half of the high school IE courses in this studyemployed the modeling theory approach to teachingphysics which has become the focus of the physics edu-cation research group at Arizona State University. Themodeling theory of teaching physics states that physicsis based on a small set of models that represent thestructure seen in the world. Models are explicitly iden-tified as models of re a l i t y, not physical re a l i t i e s(Hestenes, 1987). Modeling was developed to addressidentified conceptual deficiencies commonly present in

students that have completed traditional physics cours-es. These deficiencies include fragmented physicsknowledge and the persistence of alternative concep-tions which are attributable to the tendency of studentsto focus on the memorization of facts and algorithmsrather than understanding the processes involved(Hestenes, 1987). Through group discussion studentsare required to identify a question that might be askedof nature by a scientist. Then, in small groups, experi-ments are designed and conducted. The experiment isrun, data are collected and analyzed, and students makeclaims based on their findings. The instructor is expect-ed to be knowledgeable with respect to common mis-conceptions, and able to employ Socratic questioning toprompt and guide the inquiry. The instructor must alsotake care in choosing as few paradigms as possible toinvestigate so that students will have enough time tolearn and understand the concepts as well as have thetime to resolve conflicts between their misconceptionsand the accepted scientific conceptualizations(Hestenes, 1998). As all of the IE high school coursesreviewed in the study (Hake, 1998a) used the MBLmaterials, modeling is a good pedagogical fit thatshould interface seamlessly between the classroom andlab setting. Once again this specific reform is based onacknowledging the prevalence of common sensenotions of force and motion and using group discussionto mediate between those understandings and theaccepted scientific conceptualization. The requirementof more time per topic to allow for the discussion andreflection associated with promoting conceptual changeresults, however, in less content coverage.

Just after the publication of Hake’s meta-analysis inJanuary of 1998, a detailed comparison between tradi-tional introductory physics instruction and introducto-ry physics instruction utilizing active learning strategieswas undertaken. This study occurred during the fallsemester of 1998 at California Polytechnic StateUniversity (Knight, 2002). California Polytechnic con-tributed four data sets to the Hake (1998a) meta-analy-sis. Three of the sets reported were categorized as IE,and each of these courses was taught by Randall Knight,who conducted but did not teach any of the sections forthe comparative study (Knight, 2002). The other dataset from Cal Poly reported on by Hake (1998b) was cat-egorized as traditional in format. For the comparativestudy (Knight, 2002) the students were randomlyassigned to one of six introductory sections (N = 225).For this investigation three professors each taught onetraditional section and one studio classroom session.The studio classroom is designed to eliminate theboundary between lecture and laboratory and promote

59


an active learning environment. This is facilitated by theelimination of lecture and having lab and other activi-ties occur in the same classroom. The active learningstrategies included MBL (Thornton & Sokoloff, 1990),group problem solving involving worksheets based onPER findings, and group discussion in thet h i n k / p a i r / s h a re format. The studio class used aresearch inspired text for outside of class readings. Thetraditionally taught class used a traditional introductoryphysics text. As a historical aside, the elimination of lec-ture from introductory physics courses as a reformaimed at improving students’ conceptual understandingcannot be called a recent innovation. In an autobiogra-phy, Robert Millikan, the man credited with determin-ing the quantity of electrical charge on an electron,describes elimination of lecture from the introductorycourses he taught, because he found lecture ineffectiveat helping students understand physics. Millikan insti-tuted this reform at the University of Chicago in 1896(Millikan, 1950).

For all students in the Knight (2002) study the FCIwas administered as a pretest on the first day of class,and no differences were found between the traditionaland non-traditional sections prior to instruction. At theend of the quarter all students were given the FCI as aposttest and all students took the same final exam. Thefinal exam, described as a typical quantitative examina-tion, was written by a professor not involved in thecomparison. Student examinations were identified by anumber and each problem on the exam was correctedby only one faculty member to assure fairness in cor-recting. The findings of this comparison were similar toH a k e ’s meta-analysis (1998a). The students wholearned physics in the active-learning environment didsignificantly better on both the FCI posttest (11% aver-age difference) and on the quantitative final exam (7.5%average difference). A comparison of overall instructorrating was also tabulated through the use of the stan-dard course evaluation questionnaire, which used a 1-5scale. The same three instructors taught one section ineach format yet received an average evaluation of 3.51for the traditional course and an average of 3.92 for thestudio class. The same instructors were evaluated high-er in the active-learning environment. The results of thisstudy support the contention of Hake (1998a) that IEmethods facilitate greater gains than traditional instruc-tion in conceptual understanding related to theNewtonian concept of force, and also provide evidencefor the claim that greater conceptual understanding ofNewtonian mechanics results in improved quantitativeproblem-solving skills within this part of an introducto-ry physics course.

This study did not report on the comparative time-rate of content coverage, or on the number of topicsstudied during the semester. The fact that all six sec-tions took the same final examination is indirect evi-dence that the content coverage was essentially equiva-lent. The elimination of lecture by the professors in theactive learning courses may account for some of thereduction of time differential reported in other compar-isons of content coverage between the discussion-richIE courses and the lecture-based traditional instructionmethod. In another university setting the elimination oflecture and a move to MBL based investigations sup-ported by student discussion resulted in a 25% reduc-tion of content coverage (Laws, 1991), so merely elimi-nating lecture doesn’t always result in a pace compara-ble to that associated with traditional introductoryphysics instruction.

SummaryThe three studies reviewed (N≈ 9000) have each

concluded, through analysis of assessment data, thatstudents who develop a better conceptual understand-ing of force as measured by the FCI (Hestenes, Wells, &Swackhamer, 1992) will also exhibit superior quantita-tive problem-solving skills re g a rding Newtonianmechanics. Each of the three instructional reformsreviewed is designed to facilitate conceptual under-standing by addressing existing common-sense basednotions that were derived from life experiences prior tof o rmal instruction in physics. The mechanism foraddressing existing misconceptions that each of theseinteractive-engagement strategies employ involves stu-dents actively discussing their reasoning. The processincludes evaluating discussion as well as constructingknowledge, and developing the ability to constructarguments. The give and take that occurs between stu-dents during discussion requires that students reflectupon the reasoning they have used in constructing theunderstanding that they present and defend during thediscussion process. The role of the teacher is to act as afacilitator rather than as a transmitter of information. Alarge percentage of class-time is utilized to develop con-ceptual understanding. This focus on negotiating mean-ing through the articulation of student understandingsresults in less content being covered than in coursestaught in a traditional manner.

The physics instructional reform movement grewout of the recognition that students enter into instruc-tions with existing understandings that do not alignwith scientific conceptualizations regarding the topicsstudied. Newtonian mechanics is one of the central con-cepts of introductory physics courses. Research has

60


established that traditional physics instruction does lit-tle to change the existing understandings that studentshold, both in general and specifically in the case of theconcepts of force and motion. The instructional reformsaimed at improving conceptual understanding, whichhave been found to be highly correlated with computa-tional problem solving ability, have been called interac-tive-engagement methods. These reforms are rooted inthe constructivist view of learning that posits that stu-dents construct their understandings through socialinteractions with each other and the instructor, as wellas through reflection on the relationship between theirunderstanding and the views of others being presented(Piaget, 1964, 1970; Vygotsky, 1962, 1978).

In describing the role of instruction within interac-tive-engagement pedagogies, Mestre (2005) relates thatthe teacher must probe student understanding to checkthat instruction is leading to students constructingknowledge that aligns with accepted scientific concep-tualizations. It has been argued (Mazur, 1997) thatwithout probing student understanding the instructor isleft with illusions regarding student learning rather thaninsight that can inform instruction. The discussioncomponent of the specific interactive-engagementmethodologies previously described is designed in partto probe student understandings. Verbal dialogue is notthe only mode through which the instructor can probestudent conceptualizations. In a publication relatingfindings from cognitive studies to the teaching ofphysics, Redish (1994) states that “to find out what ourstudents know we have to give them the opportunity toexplain what they are thinking in words” (p. 798).

REFLECTIVE WRITING FACILITATES CONCEPTUALUNDERSTANDING

Student reflective writing offers the instructor theo p p o rtunity to appraise student understanding.Reflective writing is informal writing that aids studentsin acquiring ownership of ideas (Connally & Vilardi,1989). Researchers reporting on reflective writing in thesecondary science classroom (Keys, et al., 1999) haveconcluded that when teachers encourage students towrite, the students engage in metacognition and thatwriting facilitates the generation of meaning.Metacognition has been defined by Baird as “knowledgeawareness and control of one’s own learning” (as citedin Etkina, 2000, p. 604). Metacognition also includesthe ability to decide when one’s own level of under-standing is inadequate (Mestre, 2005).

Graded Reflective WritingResearch has reported that unless something is

graded students are not likely to buy into it (Resnick &Resnick, 1992) and that writing for credit yields a bet-ter quality of writing, which correlates with higher testscores than writing that receives credit intermittently, ornot at all (Hautau et al., 2006). The benefits associatedwith students engaging in reflective writing combinedwith the motivational advantage of grading activitiesthat students engage in suggest that graded reflectivewriting may be a productive interactive-engagementstrategy. One specific activity that combines reflectivewriting with grading is the graded reflective journal.The National Research Council (1995) describes reflec-tive journal writing as an activity in which studentsassess their learning. In a later publication (NationalR e s e a rch Council, 2001) reflective journaling isdescribed as a formative assessment as well as an oppor-tunity for students to make connections between thetopic of reflection and real world understandings. Areview of the literature has revealed that reflective writ-ing is an under-utilized strategy in introductory physicscourses. The few studies reporting on student reflectivewriting in introductory physics courses all involved theuniversity setting.

Research regarding the use of a structured journalin an introductory university physics course has beenreported (Etkina, 2000). Students were required tocomplete a weekly reflection. Students were guided interms of what to reflect upon by responding to threequestions within their reflections. The questions weresuggested by Arons (1990) in a book that summarizedresearch, observations, and conclusions gleaned frommore than thirty years of teaching introductory physicsto college and university students. The questions were:What did you learn? What remains unclear? If you werethe professor, what questions would you ask to deter-mine if students understand the material? The maxi-mum score on each weekly report was six points, whichwas equal in value to the homework component of thecourse during each week. The final grade for the coursewas based on a points system. Extra credit was given forinteresting questions.

The reports allowed for communication with eachstudent on an individual basis through written com-mentary. The findings include a 90% match betweenwhat students reported having learned and the teacherexpectations re g a rding these statements. Fre q u e n t l ystudents reported some concept as learned and fol-lowed that with an indication that it had not really beenlearned because the same idea reported on as learnedwas identified in the response to the question on what

61


remains unclear. In the conclusion Etkina (2000)reports on how the weekly reports impacted her teach-ing. Specific instances cited include: developing anunderstanding of topics that presented conceptual diffi-culties for the students; being allowed insight into themisconceptions held by those students; and reducingthe amount of lecture in the course to allow for morediscussion around questions culled from the weeklyreports.

It was also stated that the weekly reports made stu-dent thinking visible by allowing insight into how stu-dents were thinking about their learning as well asdetailed exploration of student’s knowledge creation.The contention is made that weekly reports are a prac-tical system to facilitate the development of metacogni-tive skills. There is no evaluation given to the weeklyreporting requirement’s impact on course content cov-erage, but it was noted that the grading of the reportsincreased the correcting time for the instructor. It wasalso reported that the mismatch between what theinstructor assumed the students had learned and whatthe reports revealed as remaining unclear prompted thereduction in the lecture component of the instruction(Etkina, 2002). There was no end of course, high-stakesexamination of the kind previously identified associatedwith other courses. This study was the first of three thatoccurred during a three year period that utilized thesame question set, responded to in a weekly, gradedreport. Slightly different introductory physics universi-ty student populations were reported on in each study.

In the second study of the series (May & Etkina,2002) the subjects were honors engineering students(SAT mathematics test average score = 680) in an intro-ductory physics course that spent ten weeks studyingmechanics. This is 50% of the course’s allotted time,which is between 10% and 20% longer than would beexpected in a high school course teaching students ofsimilar ability. In addition to requiring reflective writ-ing, other interactive-engagement strategies were exten-sively employed. A fourth question, “What did youlearn in lab this week, and how did you learn it?” wasadded to previously used guiding questions. A codingscheme of fourteen items grouped into three categorieswas developed as a grading rubric. The reports wereevaluated regarding the quantity and quality of writing.The weekly reflective reports accounted for 10% of thecourse grade. All students were given the FCI in thepretest/posttest manner and the Hake gain analysis wasemployed. From the half of the class that had the low-est pretest scores twelve students were selected, sincethese students had the greater opportunity to exhibitgain in conceptual understanding, as measured by the

FCI. The twelve subjects selected for evaluation werethe six highest and six lowest gain values achieved with-in this half of the group. The average gain of the six low-est was .228, a result comparable to students receivingtraditional introductory instruction in the studies thatcompared gain to method of instruction (Hake, 1998a,1998b; Knight, 2002). The average gain of the six high-est was .71 which Hake (1998a) would have classifiedas high gain. The average gain for the entire group wasnot reported but if the gain for these twelve studentswas averaged the .469 result would mask a significantdifference between the two groups.

For these twelve students, the scores achieved onthe weekly reports were compared to the gain in con-ceptual understanding as indicated by the Hake gainscore from the FCI. The results were interpreted to sug-gest a possible correlation between the conceptual gainand a student’s ability to reflect on learning. Specificfindings reported that the high gain students wrotemore about how they learned, they referred to the rea-soning involved in developing understanding more fre-quently and they also showed a greater concern for thecoherence of knowledge than the lower gain students.In contrast, it was observed that the lower gain studentstended to focus on knowing formulas without indicat-ing a comprehension of their meaning as evidence ofnew knowledge, and relied upon teacher authority asthe source of understanding. More extensive research iswarranted if these claims are to be accepted. A follow-up study was begun to investigate possible corrobora-tion of these findings.

The third study in this chronological sequenceinvolved two introductory physics courses for engineer-ing students designated as at-risk (Etkina & Eisner,2003). This study was designed to determine if theresearch finding of the previous study (May & Etkina,2002) could be generalized. The populations of the twostudies were of dissimilar abilities as exhibited by the at-risk designation as opposed to the honors designationand the difference in mathematical performance as indi-cated by the mathematics average score of 480 on theSAT for the at risk group. The research investigated howstudents of lower math and reading skills would reporton their learning, as well as investigating the result ofreflection upon learning for this population. Therequired graded weekly report as previously described(Etkina, 2000; May & Etkina, 2002) and the codingrubric developed during the previous study (May &Etkina, 2002) were employed. The introductory cours-es for this study differed from previous studies in thatthe courses took place over two semesters covering thefull year of introductory physics rather than only the

62


first semester, which includes mechanics, and thecourse focused on lab investigations as the basis forclaims of knowledge. The FCI was used as pretest, butas concepts covered by the FCI accounted for only 10%of the course time, a novel method of calculating gainwas employed. Gain was calculated as the quotient ofthe difference between the decimal average of the finalexams for each semester and the decimal score of theFCI to the difference between one and the FCI decimalscore. This interpretation supposes that the pre-instruc-tion understanding regarding mechanics is an accurateindicator of a generalized baseline of understanding forall concepts studied in an introductory course. There isno discussion within the report regarding assigning theFCI pretest result as the baseline of understanding forall understanding of topics studied in an introductoryphysics course. This is certainly a debatable supposi-tion. The range of scores that were categorized as lowand the range of scores that were categorized as highwere not reported. No indication of the relationship ofthe low gain versus high gain to those categorizations asidentified by Hake (1998a) with numerical score wasreported either.

It was reported that both categories of studentsindicated concepts were more important than equationsfor developing understanding, and that reflection on thereasoning used to understand experimental resulted ini m p roved conceptual understanding. Diff e re n c e sreported were that high gainers more frequently citedlearning from authority (lecture and formulas) moreoften than low gainers (which stands in contrast to find-ings for these categories within the study of honors stu-dents), and that low gainers tended to focus on whatthey learned rather than how they learned it. Etkina andEisner (2003) claim that this focus is evidence that lowgain students lack metacognitive skills, which may con-tribute to the lower gain for this grouping. The studentpopulation that is the focus of this study may be similarto the segment of the overall student population fromwhich the increased introductory physics enrollmentreferred to by Hehn and Neuschatz (2006) is beingdrawn. If that is so then the findings regarding thisstudy are of particular interest to instructors in academ-ic environments where introductory physics courses areexperiencing increasing enrollment.

Each of the studies that have been referenced to thispoint in the paper comes from the PER community. Thisgroup of researchers is familiar with the same body ofresearch findings reported on over the past forty yearsrelating to physics instruction and learning. Within thiscommunity a great deal of credence is placed on thefindings reported through the use of the FCI. The

acceptance of Hake’s (1998a) contentions regarding thecomparative effectiveness of traditional instruction ver-sus interactive-engagement methodologies is near uni-versal. Evidence for this contention includes the citationof the same core of researchers in the reference sectionof the published findings, and the experiments con-ducted all attempt to increase conceptual understand-ing with respect to Newtonian mechanics throughinstructional reform that includes attempting to addresscommon-sense understandings through some form ofexternal or internal dialogue. The last study reviewedcomes out of a different research tradition, the writingacross the curriculum movement.

A study investigating how students perceived andaccomplished reflective writing focused on assignedi n t ro d u c t o ry physics text readings was conducted(Kalman et al., 2008). The readings and reflectionsoccurred prior to instruction, which was traditional lec-ture format mechanics course. At the beginning of thecourse one class period was dedicated to instructionregarding the reflective writing. The goal of the reflec-tive writing was to capture metacognitive reflectionoccurring during the reading activity. Each week thewriting assignments, which were not graded, and threesentences, which were graded, were collected. The threesentences were based on important ideas identified inthe jottings recording the reflections upon the readings.The sentences were graded and the best nine sentencessubmitted during the semester accounted for 15% ofthe course grade.

From the half of the class (class total ≈ 100) thatvolunteered to participate in the study five studentswere chosen. The reflective writing was evaluated withrespect to Bereiter and Scardamalia’s knowledge trans-formation model of writing (1987). This model pur-ports that students increase their conceptual under-standing through engaging in reflective writing.Reflective writing will involve knowledge transforma-tion which is characterized by structures that includecomparison, argument, and explanation that includesthe identification of new ideas. Writing that is based ona recitation of facts from the reading or existing knowl-edge is categorized as knowledge-telling and is not con-sidered evidence of metacognitive reflection. Coding ofreflective writing samples was done for only 25% of thestudent samples. The five subjects were interviewedabout their reflective writing at the third, seventh, andthirteenth (final) week of the course.

All students claimed that engaging in reflectivewriting helped them prepare for class. Analysis of thewriting samples led to the claim that three of the fivestudents showed evidence of recognizing conceptual

63


difficulties through the act of reflective writing and thesame number showed evidence of a positive outcome,evidence of an ability to inductively solve new problemswhile engaging in reflective writing. It is claimed(Kalman et al., 2008) that reflective writing promotesself-dialogue about existing knowledge and new con-cepts encountered in textbook reading.

This is a very small sample size. The report does notoffer insight into why only five of the fifty students thatvolunteered to participate in the investigation wereselected. The time required evaluating writing samples,grading the submitted sentences, and interviewing par-ticipants may be involved in the restriction of theresearch to only five participants. No reporting regard-ing course or test grades, the relationship of gradedwriting to course grade, or the relationship of studentperceptions, as revealed through interviews, to coursegrade was included. The omission of the potential linkbetween metacognition as indicated by reflective writ-ing and the overall course performance, or the relation-ship of writing indicating a lack of metacognition andoverall course performance suggests an area for furtherresearch. This paper makes no reference to any researchreporting on differences between traditional instructionand interactive-engagement methods of instruction. Asthe course involved what has been described as tradi-tional instruction, other than the use of the reflectivewriting, no reporting on the content coverage aspect ofthe course was expected or reported.

SummaryResearchers have reported that graded reflective

writing, structured by questions or instructions thatguide students with respect to focus, has multiple ben-efits for students when learning Newtonian mechanics.A particular benefit described is the facilitation of thedevelopment of metacognitive skill, a critical capabilityfor learning in this topical area because of the preva-lence of common-sense derived understandings thatresist traditional instruction and oppose the acceptedNewtonian conceptualizations. Evidence has been pre-sented that reflection can and does lead students to therecognition of a discrepancy between the understandingheld and the scientifically supported conceptualization.Recognition of the discrepancy is the first step towardreconciliation of conflicting notions. It is also reportedthat reflective writing facilitates the resolution of con-flicting notions by promoting self-dialogue about theexisting knowledge and the new concept. Another ben-efit claimed is that graded reflective writing facilitatesthe development of inductive problem solving capabili-ties. It was reported that all students benefit from grad-

ed reflective writing focused on learning, and that manystudents perceive reflective writing as contributing toknowledge transformation.

Differences were reported regarding the type ofwriting exhibited as well as the focus of the writingwithin each study. When the focus of the reflective writ-ing was correlated to gains in conceptual understandingas quantified through the use of the FCI and the Hakegain analysis, it was found that the high gain studentswithin a class of high academic achievers tended towrite about the reasoning they used to develop under-standing. This apparent correlation between gain inconceptual understanding and reflection on learningmay indicate that concern for coherence of knowledgeis a factor involved in achieving larger Hake gains. Thelower gain students within this same class often citedauthority (formulas or teacher statements) as the basisfor developing new understandings.

In a parallel study of much lower academic achiev-ers it was found that the high gain students of thisgroup often wrote about authority as the source ofunderstanding. This result which contradicts the find-ings for students with higher mathematical and verbalskills, as measured by the SAT, led to the conjecture thatstudents of different academic abilities may have differ-ences in the way that they reflect on learning and thatinstruction should take this into account.

It is also reported that reflective writing creates aspace for dialogue between the teacher and student.The writing can make students’ thinking visible whichpermits insight and comment by the teacher that maynot occur otherwise. Changes in student perception canbe captured, and if a particular learning activity isextremely effective the instructor could receive feed-back through examination of student re f l e c t i o n s .Another attribute of corrected reflective writing is that itallows the instructor to have insight into difficultiesbeing experienced by a large number of class partici-pants which can result in adjustment to the pace ofinstruction to accommodate resolution of the problembefore moving on. Due to the informative capability ofthe reflective journal it is clear that a student reflectivejournal is a formative assessment, as well as a learningactivity. In summary, graded reflective writing can ben-efit both the learner and the instructor.

CONCLUSION AND POTENTIAL RESEARCHQUESTIONS

Interactive-engagement strategies were developed torespond to the less than optimal effectiveness of tradi-tional instruction at facilitating conceptual understand-

64


ing regarding Newtonian mechanics for students ini n t ro d u c t o ry courses, as measured by the FCI.Interactive-engagement strategies seem to be more effec-tive at moving students from the common-sense basedunderstandings regarding force and motion that theyentered into instruction believing, to the Newtonianunderstandings of force and motion. Physics educationresearchers have presented evidence that interactive-engagement strategies are more effective at facilitatingconceptual understanding and quantitative pro b l e msolving capabilities in students than are traditionalinstructional strategies, for students studying Newtonianmechanics in an intro d u c t o ry course. Interactive-engagement strategies can use a number of events togenerate discussion among students. Interactive-engage-ment strategies use student discussion as the mediumt h rough which a discrepancy between an existingunderstanding held by a student and the understandingbeing presented, discussed, and defended by other stu-dents is crystallized. In reflecting on this discrepancy,the holder of the misconception is forced to negotiatebetween the competing notions through the construc-tion of arguments to defend a position. The constructionof the arguments parallels the construction of knowl-edge. This is a time involved process that requires exter-nal arguments to become internalized, and optimally thediscrepancy is resolved through achieving a Newtonianunderstanding. The discussion process takes up a signif-icant portion of the time that the students and theinstructor share. Metacognitive reflection, during andafter discussion, is required if the student is to gain theNewtonian understanding.

Metacognitive reflection is also the goal of gradedstudent reflective writing. Research indicates that whenthe students are guided to reflect on the learningprocess, metacognitive skills can be developed. Whenthe students engages in self-dialogue regarding the dis-crepancy between what they have held as true and thenew information being presented, and reflected upon,knowledge transformation can occur. Graded reflectivewriting moves the intrapersonal discussion outside ofthe classroom and conserves some of the precious stu-dent-teacher contact time for progress through the cur-riculum. Both interactive-engagement strategies andgraded reflective writing require the student to engagein metacognitive reflection in order to transition from acommon-sense based understanding to the Newtonianconceptualization. Because the reflective process isengaged in outside of the classroom for the reflectivewriting strategy, it may be worth examining the effectthat grading student reflective journals has on improv-ing conceptual understanding and upon reducing the

time differential that exists between traditional andinteractive-engagement teaching strategies.

Research indicates that the trend of increasingenrollment in introductory physics courses will contin-ue. Researchers have found that all students benefit byengaging in graded reflective writing. It has also beensuggested that students’ metacognitive skills, skillswhich are required to move from common-sense basedunderstanding to Newtonian thinking, are enhancedthrough the practice of reflective writing focused onlearning. Research reporting on the effect of gradedreflective writing on students that have academic capa-bilities spanning a wide range suggests that studentswhose mathematics and verbal skills, as indicated bySAT scores, are toward the opposite ends of the rangemay learn differently. Allowing the instructor insightinto the students’ perceptions upon learning, as cap-t u red in the reflections re c o rded, may permit theinstructor to adjust instruction to the advantage of thelearner. Because of the expanding enrollment withinintroductory physics courses the effectiveness of theinstruction at accommodating students of all capabili-ties becomes of even greater importance than has his-torically been the case.

Further research on the potential of reflective writ-ing to facilitate the development of conceptual under-standing for students studying Newtonian mechanics inan introductory course is suggested. Based upon theresearch findings cited, a proposed study using gradedstudent reflective writing is proposed. Reflective writingengaged in on a daily basis and summarized through aweekly summative reflective paragraph may lead toinsight regarding the following questions:

1. How will graded reflective journals compare tointeractive-engagement strategies, with respect toconceptual gain for students in an introductoryphysics course, as measured with the FCI usingthe Hake gain analysis?

2. What effect will using graded reflective journalshave upon the rate of content coverage withinintroductory physics courses using interactive-engagement strategies?

3. What effect will grading reflective journals haveupon the instructor regarding pace of instruc-tion?

4. What will be found regarding the type of reflec-tion (knowledge building or knowledge telling)and focus of reflection (internal dialogue orexternal authority) for high gain versus low gainstudents as grouped by the Hake gain analysis ofpre/posttest FCI scores.

65


ReferencesArons, A. B. (1990). A Guide to Introductory Physics Teaching. New

York: Wiley.

Beichner, R.J., & Saul, J. M. (2004). Introduction to the scale-up

project.

Proceedings of the International School of Physics “Enrico Fermi”

Course, 2004, Varena Italy. Retrieved August 24, 2008, from

http://www.ncsu.edu/per/Articles/Varenna_SCALEUP_Paper.pdf

Beichner, R.J., Saul, J. M., Allain, R. J., Deardorff, D. L., & Abbott,

D. S. (2007). Student-centered activities for large enrollment

undergraduate programs (scale-up) project. Research-Based

Reform of University Physics, 1 (1), 2-39.

Bell, G.L. (2001). Reflective journal writing in an inquiry-based

science course for elementary pre-service teachers. Paper pre-

sented at the Annual International Meeting of the National

Association for Research in Science Teaching, St. Louis.

Bolton, G. (2005). Reflective practice: Writing and professional devel-

opment. London: Sage Publications.

Carramazza, A., McCloskey, M., & Green, B. (1980). Curvilinear

motion in the absence external forces: Naïïve beliefs about the

motion of objects. Science, 210 (4474), 1139-1141.

Champagne, A.B., Klopfer, L.E., &Anderson, J.H. (1980). Factors

influencing the learning of classical mechanics. American

Journal of Physics, 48 (12), 1074-1079.

Clement, J. (1982). Student’s preconceptions in introductory

mechanics. American Journal of Physics, 50 (1), 66-71.

Connally, P. & Vilardi, T. (Eds.). Writing to learn mathematics and

science. New York: Teachers College Press.

Crouch, C. H. & Mazur, E. (2001). Peer instruction: Ten years of

experience and results. American Journal of Physics, 69 (9),

970-977.

Crouch, C. H., Watkins, J., Fagan, A. P., & Mazur, E. (2007). Peer

instruction: Engaging students one-on-one, all at once.

Research-Based Reform of University Physics, 1 (1), 40-95.

DiSessa, A. (1988). Knowledge in pieces. In G. Forman & P. Pufall

(Eds.) Constructivism in the computer age. (pp. 49-70).

Hillsdale, NJ: Lawrence Erlbaum Associates.

Dori, Y.J. & Belcher, J. (2004). How does technology-enabled

active learning affect undergraduate students’ understanding

of electromagnetism concepts? The Journal of Learning Sciences,

14 (2), 243-279.

Driver, R. (1990). Students’ conceptions and the learning of sci-

ence. International Journal of Science Education, 11 (5), 481-

490.

Driver, R. & Easley, J. (1978). Pupils and paradigms; A review of

literature related to concept development in adolescent sci-

ence students. Studies in Science Education, 5 (1) 61-84.

Driver, R., Guesne, E., & Tiberghien, A. (1985). Childrens ideas in

science. Philadelphia: Open University Press.

Dufresne, R., & Gerace, W. (2004). Assessing to learn: Formative

assessment in physics instruction. The Physics Teacher, 42 (7),

428-433.

Etkina, E. (2000). Weekly reports: A two-way feedback tool.

Science Education, 84 (5), 594-605.

Etkina, E. & Eisner, A. (2003). What did I learn and why do I

believe in it? Proceedings of the 2003 Physics education

research conference. In Franklin, S., Marx, J., & Cummings,

K (Eds.), Vol. 720 (pp. 27-30). Rochester, NY: retrieved

September 10, 2008, from

http://scitation.aip.org/dbt/dbt.jsp?KEY=APCPCS&Volume=72

0&Issue=1

Etkina, E., & Van Heuvelen, A. (2001). Investigative science learn-

ing environment: Using the processes of science and cognitive

strategies to learn physics.

Proceedings of the 2001 Physics education research Conference,

Franklin, S., Marx, J., & Cummings, K (Eds.), (pp. 17-20).

Rochester, NY: Physics Education Research Conference.

Fagan, A., Crouch, C.H., & Mazur, E. (2002). Peer instruction:

Results from a range of classrooms. Physics Teacher, 40 (4),

206-209.

Hake, R. R. (1987). Promoting student crossover to the Newtonian

world. American Journal of Physics, 55 (10), 878-884.

Hake, R. R. (1998a). Interactive-engagement versus traditional

methods: A six-thousand-student survey of mechanics test

data for introductory physics courses. American Journal of

Physics, 66 (1), 64-78.

Hake, R. R. (1998b). Interactive-engagement methods in introduc-

tory mechanics courses. Retrieved August 4, 2008, from

http://www.physics.indiana.edu/~hake/

Halloun, A., & Hestenes, D. (1985a). The initial knowledge state

of college physics students. American Journal of Physics, 53

(11), 1043-1055.

Halloun, A., & Hestenes, D. (1985b). Common sense concepts

about motion. American Journal of Physics, 53 (11), 1056-

1065.

Hautau, B., Turner, H. C., Carroll, E., Jaspers, K., Parker, M.,

Krohn, K., & Williams, R. L. (2006). Differential daily writing

contingencies and performance on major multiple-choice

exams. Journal of Behavioral Education, 15 (4), 256-273.

Hehn, J., & Neuschatz, M. (2006). Physics for all? A million and

counting! Physics Today, 59 (2), 37-43.

66


Hestenes, D. (1987). Toward a modeling theory of physics instruc-

tion. American Journal of Physics, 55 (5), 440-454.

Hestenes, D. (1998). Who needs physics education research?

American Journal of Physics, 66 (6), 465-467.

Hestenes, D., & Wells, M. (1992). A mechanics baseline test. The

Physics Teacher, 30 (3), 159-166.

Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force Concept

Inventory. The Physics Teacher, 30 (3), 141-158.

Hewson, P. W., & Beeth, M. E. (1993). Teaching for conceptual

change: Examples from force and motion. Paper presented at

the Iv Congreso Internacional sobre Investigacion en la

Didactica de la Ciencias y de las Matematicas. Barcelona,

Spain.

Hoellwarth, C., Moelter, M.J., & Knight, R.D. (2005). A direct

comparison of conceptual learning and problem solving abili-

ty in traditional and studio style classrooms. American Journal

of Physics, 73 (5), 459-463.

Johnson, D.W., Johnson, R.T., & Smith, K.A. (1991). Active learn-

ing: Cooperation in the college classroom. Edina, MN: Interaction

Book Company.

Kalman, C., Aulls, M.W., Rohar, S., & Godley, J. (2008). Students

perceptions of reflective writing as a tool for exploring an

introductory textbook. Journal of College Science Teaching,

March. Retrieved from

http://www.accessmylibrary.com/coms2/summary_0286_

34940779.

Keys, C. W., Hand, B., Prain, V., & Collins, S. (1999). Using the

science writing heuristic as a tool for learning from laboratory

investigations in secondary science. Journal of Research in

Science Teaching, 36 (10), 1065-1084.

Knight, R.D. (2002). Five easy lessons: Strategies for successful physics

teaching. San Francisco: Pearson Education Publishing.

Laws, P. W. (1991). Calculus-based physics without lectures.

Physics Today, 44 (12), 24-31.

Lorenzo, M., Crouch, C.H., & Mazur, E. (2006). Reducing the

gender gap in the physics classroom. American Journal of

Physics, 74 (2), 118-122.

May, D., & Etkina, E. (2002). College physics students’ epistemo-

logical self-reflection and its relationship to conceptual learn-

ing. American Journal of Physics, 70 (12), 1249-1258.

Mazur, E. (1997). Peer Instruction: A User’s Manual. Upper Saddle

River, NJ: Prentice Hall.

McDermott, L. C. (1984). Research on conceptual understanding

in mechanics. Physics Today, 37 (7), 24-32.

Meltzer, D. E., & Manivannan, K. (2002). Transforming the lec-

ture-hall environment: The fully interactive physics lecture.


Mestre, J. P. (1994). Cognitive aspects of learning and science

teaching. In Fitzsimmons, S. J., & Kerpelman, L. C. (Eds.),

Teacher Enhancement for Elementary and Secondary Science and

Mathematics: Status, Issues, and Problems. (3.1-3.53).

Washington, DC: National Science Foundation.

Mestre, J. P. (2005). Facts and myths about pedagogies of engage-

ment in science learning. Association of American Colleges

and Universities: Peer Review, 7 (2), 1-4.

Millikan, R.A. (1950). The autobiography of Robert A. Millikan. New

York: Prentice-Hall.

National Research Council. (1995). National science education

standards. Washington, DC: National Academy Press.

National Research Council. (2001). Classroom assessment and the

national science education standards: Commission on class-

room assessment. Washington, DC: National Academy Press.

Piaget, J. (1964). Part I, Cognitive development in children: Piaget

development and learning. Journal of Research in Science

Teaching, 2 (3), 176-186.

Piaget, J. (1970). The child’s conception of movement and speed.

Translated by Holloway, G.E.T & Mckenziw, M.J. London,

England: Routledge and Kegan.

Pollock, S. (2004). No single cause: Learning gains, student atti-

tudes, and the impacts of multiple effective reforms. 2004

Physics Education Research Conference: AIP Conference

Proceedings Vol 790 (pp 137-140), Marx, J., Heron, P., &

Franklin, S. (Eds.) Retrieved September 24, 2008 from

http://www.tinyurl.com/9tfk4

Posner, G., Strike, K., Hewson, P., & Gertzog, W. (1982).

Accommodation of a scientific conception: Toward a theory of

conceptual change. Science Education, 66 (2), 211-227.

Preece, P.F.W. (1984). Force and motion: Pre-service and practicing

secondary science teachers’ language and understanding.

Research in Science and Technological Education, 15 (1), 123-

128.

Redish, E. F. (1994). Implications of cognitive studies for teaching

physics. American Journal of Physics, 62 (6), 796-803.

Redish, E. F., & Steinberg, R. N. (1999). Teaching physics:

Figuring out what works. Physics Today, 52 (1), 24-30.

Resnick, L. B. (1983). Mathematics and science learning: A new

conception. Science, 220 (4596), 477-478.

Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking cur-

riculum: New tools for educational reform. In Gifford, B. R., &

O’Connor, M. C. (Eds.), Changing Assessments: Alternative Views

of Aptitude, Achievement, and Instruction (37-75). Boston:

Kluwer.

67


Russell, T. (1993). Learning to teach science: Constructivism,

reflection, and learning from experience. In Tobin, K. (Ed.),

The Practice of Constructivism in Science Education (247-258).

Hillsdale, NJ: Erlbaum.

Royuk, B. & Brooks, D.W. (2003). Cookbook procedures in MBL

exercises. Journal of Science Education and Technology, 12

(3),317-324.

Sokoloff, D. R., & Thornton, R. K. (1997). Using interactive lec-

ture demonstrations to create an active learning environment.

The Physics Teacher, 35 (6), 340-347.

Starkman, N. (2006). Formative assessment: Building a better stu-

dent. Technology Horizons in Education Journal, 33 (14), 41-46.

Stokstad, E., (2001). Reintroducing the intro course. Science, 293

(5535), 1608-1610.

Thacker, B., Kim, E., Trefz, K., & Lent, S.M. (1994). Comparing

problems solving performance of physics students in inquiry-

based and traditional physics courses. American Journal of

Physics, 62 (7), 627-633.

Thornton, R. K., & Sokoloff, D. R. (1990). Learning motion con-

cepts using real-time microcomputer-based laboratory tools.


Thornton, R. K. (1999). Using the results of research in science

education to improve science learning. Paper presented at the

International Conference on Science Education, Nicosia:

Cypress.

Vygotsky, L. (1962). Thought and language. Cambridge, MA: MIT

Press.

Vygotsky, L. (1978). Mind in society: the development of perception

and attitude. Cambridge, MA: Harvard University Press.

Watts, M. & Zylberztajn, A. (1981). A survey of some children’s

ideas about force. Physics Education, 16 (6), 360-365.

West, L.H.T., & Pines, E.L. (1985). Cognitive structure and conceptu-

al change. Orlando, FL: Academic Press.

APPENDIX ATERMS USED TO DESCRIBE “COMMON-SENSE” UNDERSTANDINGS

Term Publication

Alternative conception Driver & Easley, 1978

Alternative framework Driver & Easley, 1978

Intuitive science Preece, 1984

Naïïve belief Carramazza, McCloskey, & Green, 1981

Naïïve conception Champagne, Klopfer, & Anderson, 1980

Naïïve theory Resnick, 1983

Misconception Posner, Strike, Hewson, & Gerzog, 1982

Phenomenological primitive (p-prim) DiSessa, 1988

Preconception Clement, 1982

68


APPENDIX BTHE FORCE CONCEPT INVENTORY

69


70


71


72


73


74


75


76


77


78


APPENDIX CRESEARCHERS REPORTING INTRODUCTORY PHYSICS CURRICULAR

REFORMS AS RESPONSES INFORMED BY FCI & HAKE GAIN FINDINGS

Institution Implementing Reform Researcher(s)

California Polytechnic State University Hoellwarth, Moelter, & Knight (2005)Harvard University Crouch & Mazur, (2001)Massachusetts Institute of Technology Dori & Belcher, (2004)North Carolina State University Beichner & Saul, (2004)Rutgers, State University of New Jersey Etkina & Van Heuvelen, (2001)Tufts University Thornton (1999)University of Colorado Pollock (2004)

79


2010 Annual Colloquium on Research inMathematics and Science Education

Call for PapersEducators, researchers and graduate students are invited to submit papers that will be present-ed at the Fifteenth Annual Colloquium on Research in Mathematics and Science Education andpublished in the Colloquium Journal, vol. XV. The papers must discuss issues and trends inMathematics and Science Education.

WHEN SUBMITTING A PAPER, PLEASE USE THE FOLLOWING GUIDELINES.1. Submit an electronic version of the paper and one hard copy, an abstract, approximately 150

words, and a biographical sketch, about 30 words. All pictures and diagrams must be sub-mitted as separate documents.

2. Use double spacing with one-inch margins.

3. For references, diagrams, etc., follow the style described in the Publication Manual of theAmerican Psychological Association (APA), Fifth Edition.

4. Paper length must not exceed 30 pages, including pictures, tables, figures, and list of refer-ences.

5. Papers must be received by December 1, 2009.

6. Authors will be notified about the status of their papers by January 15, 2010.

7. The Colloquium will be scheduled for April 2010.

SUBMIT PAPERS AND CORRESPONDENCE TO:Dr. Regina M. PanasukProfessor of Mathematics EducationThe Graduate School of EducationUniversity of Massachusetts Lowell61 Wilder Street, O’Leary 5th FloorLowell, MA 01854

Phone: (978) 934-4616Fax: (978) [email protected]

Documents

Mathematics and Science Achievement of U.S. Fourth and ... · Annual Colloquium Journal vol. XIV, Spring 2009 Highlights from The Trends in International Mathematics and Science Study(TIMSS)