Evidentiary Competence: Sixth Graders’ Understandingfor Gathering and Interpreting Evidencein Scientific Investigations

Heisawn Jeong & Nancy B. Songer & Soo-Young Lee

Received: 1 July 2004 /Accepted: 25 April 2006 /Published online: 3 August 2006# Springer Science + Business Media B.V. 2006

Abstract With the growing emphasis on the development of scientific inquiry skills,there is a strong need for more research on students’ ability to collect and interpretevidence. This paper calls attention to the notion of evidentiary competence thatrefers to the concepts and reasoning skills involved in the collection, organization,and interpretation of data. We proposed a set of concepts and skills involved inevidentiary competence and examined sixth of them—the priority, relevancy,objectivity, replicability of evidence, and the interpretation of examples andtables—using a written instrument contextualized in atmospheric science. Analysesof 40 sixth grade students’ answers and explanations revealed that their under-standing of scientific evidence and the data collection process was quite weak inseveral respects. For example, many students neither appreciated the role ofempirical evidence in scientific inquiry, nor distinguished relevant evidence fromirrelevant evidence, nor understood the importance of reliable and objectiveobservations, nor interpreted examples and tables appropriately. Results suggestthat more explicit instructions are needed in order to strengthen students’ ability tocollect and interpret data, especially in the current data rich information age.

Key words evidentiary competence . inquiry . scientific reasoning . evidence .

data collection . interpretation . sixth graders . atmospheric science

Res Sci Educ (2007) 37:75–97DOI 10.1007/s11165-006-9014-9

H. Jeong (*)Department of Psychology,Hallym University,Okchun-dong 1, Chunchon,Kangwon-do, 200-702, South Koreae-mail: heis@hallym.ac.kr

N.B. SongerUniversity of Michigan, Ann Arbor, MI, USA

S.-Y. LeeTERC, Cambridge, MA, USA

Details of the cognitive activities employed by scientists vary depending on theproblem specifics or the standards and protocols within the disciplines, butaddressing scientifically oriented questions using empirical data is a common keydimension of scientific inquiry (Chinn & Malhotra, 2002; Gott & Duggan, 1996;Klahr, 2002; Klahr & Dunbar, 1988; Kuhn, 1989; Minstrell & van Zee, 2000). Inscience, theory is on trial, but it is the evidence that determines the outcome of thetrials. It is often assumed that evidence is just provided and its collection andinterpretation is a straightforward process. However, collecting and interpretingevidence is a complicated process involving a number of considerations. Althoughdeveloping a mature understanding and necessary skills of data collection andinterpretation is an essential component of scientific literacy (Driver, Leach, Millar,& Scott, 1996; Gott & Duggan, 1996; Lehrer & Schauble, 2002; National ResearchCouncil, 2000), relatively little attention has been paid to investigating students’conceptions and related skills involved in the collection and interpretation of data.

Much of the earlier research efforts on students’ understanding of evidence werecarried out in the context of whether they could differentiate theories from evidence(e.g., Carey, Evans, Honda, Jay, & Unger, 1989; Koslowski, 1996; Kuhn, Amsel, &O’Loughlin, 1988; Zimmerman, 2000). However, as Kanari and Millar (2004) havepointed out, these works have ignored one important step in the process of scientificreasoning, namely getting the measured primary data to the statement of Fresults_(also see Chinn & Brewer, 2001, for a similar criticism). For example, Kuhn et al.(1988) defined theory–evidence coordination as consisting of the following threecomponents: (1) The ability to think about a theory, rather than only think with it,(2) the ability to encode and represent the evidence to be evaluated as an entitydistinct from representation of the theory, and (3) the ability to set aside acceptanceor rejection of a theory in order to assess what the evidence by itself would mean forthe theory, were it the only basis for making a judgment. As one can see, the focuswas on how children conceptualize and evaluate theories rather than on how theyconceptualize and evaluate data. Children only need to Fencode_ and Frepresent_ thedata that was already collected, processed, and summarized for them. Anunderstanding for scientific evidence is closely tied to an understanding of theories,but an adequate understanding of scientific evidence is more than just knowing thedifference between theory and evidence.

We introduce the notion of evidentiary competency in order to refer to theconcepts and reasoning skills required to collect good, reliable, and valid data and toorganize and interpret them up to a point where they can be readily used forevaluating theories and explanations. What might be the conceptions and skills thatconsist of evidentiary competency? Norris (1985) proposed that observationalcompetence consists of three broad proficiencies: (1) Making observations well, (2)reporting them well, and (3) correctly assessing the believability of reports ofobservations. He further proposed a set of principles for each category ofproficiencies. For example, he proposed nine principles that an observer shouldadhere to in order to make good observations such as Fnot allow his or her emotionsto interfere with his or her making sound judgments_ or Fbe skilled at observing thesort of thing observed and in the technique being used._ His principles emphasizethe importance of making skillful and objective observations that are not influencedby outside influences such as preconceived notions or emotions. In spite of his effortto make these principles explicit, the notion of Fgood_ observation needs to beexplicated further in terms of the kinds of conceptual knowledge and reasoning

76 Res Sci Educ (2007) 37:75–97

skills that students need to develop in order to be skilled at making, reporting, andassessing observations well.

Gott and Duggan (1996) argued that in order to be scientifically literate studentsneed to have a sound knowledge base and skills related to the collection, validation,representation, and interpretation of evidence. They proposed a list of Fconcept ofevidence_ that is connected directly and necessarily with the understanding ofscientific evidence. These concepts are grouped into three broad categories. First,there are concepts associated with design such as the notions of variableidentification, fair test, sample size, and variable types. Second, there are conceptsassociated with measurement such as the notions of relative scale, range of interval,choice of instrument, repeatability, and accuracy. Lastly, there are conceptsassociated with handling data such as tables and graphs. Although these conceptsare grouped in three categories, they are considered to be manifestations of thenotion of validity and reliability, since evaluating the validity of evidence, forinstance, requires a consideration of the design, measurement, and data handlingwhich produced the evidence.

According to Lubben and Millar (1996), validity concerns the question ofwhether what one observes or measures is what he or she intends to observe ormeasure. Reliability concerns the question of whether one’s observation ormeasurement is a good representation of what he or she intends to observe ormeasure. They investigated the procedures that 11, 14, and 16 year old children usedin making measurements and their ability to use data as evidence to supportconclusions. They identified a pattern of progression concerning students’ ideas ofexperimental data. Students’ understanding of the process of measuring went fromthe denial of the need to repeat measurements, via a search for recurring results,and a deliberate variation of control variables to collect a guaranteed variety ofresults, to the determination of the likely range of results. A parallel progressionwas identified for the evaluation of measurements, from assessing a single value onthe basis of Flikely_ or Fexpected_ measurements in that context, to selecting a valueaccording to the place of the measurement in a sequence, to selecting a recurringvalue, to using the whole set of data to work out an average. As one can see, theprogression of understanding focused on the notion of repeated measurements, andtheir conceptualization of validity and reliability was about how students viewed andcarried out replications (e.g., whether they looked for recurring values or computedan average). Appreciating the role of replication and knowing how to handle dataobtained through repeated measurements is a necessary component of evidentiarycompetence, but evidentiary competence is more than that. Students should be ableto understand how to avoid observer bias during data collection, set up and useinstruments, and achieve desired accuracy, for instance.

Masnick and Klahr (2003) examined children’s understanding of experimentalerror. Across the five stages of experimentation – design, setup, execution, outcomemeasurement, and interpretation – four types of experimental errors were assumedto occur: Design error (e.g., design of a confounded experiment), execution error(e.g., unexpected factors influence the outcome, such as sudden wind changing thedirection of the ball), measurement error (e.g., errors in calibrating an instrument),and interpretation error. Ability to understand and handle experimental errors,especially measurement and execution errors, is closely associated with evidentiarycompetence. In order to collect reliable and valid data, children should understandthe origins of errors and how to avoid them. According to Masnick and Klahr, errors

Res Sci Educ (2007) 37:75–97 77

originate from various stages of experimentation. Measurement errors occur notonly when students measure outcomes, but also when students prepare and set upthe experiments (e.g., incorrect setting of an instrument). Execution errors occurwhen something not considered or anticipated in the design (e.g., imperfection inthe ball) influences the outcome during the experiment. Thus, in order to avoidthese errors, students need to be careful not only during the execution andmeasurement stage, but also during the design and setup stage as well.

Based on the concepts identified in the literature, especially the work by Gottand Duggan (1996) and others (Chinn & Brewer, 2001; Driver et al., 1996; Lubben& Millar, 1996; Masnick & Klahr, 2003), we identified concepts/skills that aredeemed essential in the development of students’ evidentiary competence (seeTable 1). These concepts and skills are grouped according to the stages of inves-tigation that they are most relevant to. Although they are grouped according to thephase of investigation, note that they often concern more than one stage ofinvestigation. For instance, considerations about accuracy not only concern the datacollection stage, but also the planning stage since concerns about accuracy wouldforce one to design and plan studies in certain ways (e.g., use certain equipment in

Table 1 Concepts and Skills Involved in Evidentiary Competence.

Phase of

investigation

Concepts and

skills

Explanations

Planning Priority* Understand that science accepts knowledge claims when

they are verified with empirical evidence

Relevancy* Identify data or variables relevant to the investigation

Operationalization Operationalize variables so that the effect of the variables

can be tested appropriately

Fair test Design un-confounded data collection

Sample size Understand the significance of sample size and plan

enough cases in the data

Variable type Understand different variable types and their effects

Collection Objectivity* Know that one’s own perception and judgment may not be

reliable and one needs to objectify observation by

employing a system of measurement

Replicability* Understand the inherent variability and uncertainty

associated with measurement process and seek reliability

through replication

Accuracy Know how much accuracy is desired and how to

achieve it

Instrument Know how to choose, set up, and use instruments

appropriately

Execution of

collection

protocols

Carry out the experimental procedures in the right order

and at the right moment

Interpretation Example* Recognize the relationship of an example to a given

knowledge claim or hypothesis

Table* Identify patterns and locate information from tables

Graph Identify patterns and locate information from graphical

representations

Anomalous data Know how to treat anomalous data

*Concepts and skills assessed in this study.

78 Res Sci Educ (2007) 37:75–97

order to achieve desired accuracy). Thus, in order to develop evidentiarycompetence students need to develop a comprehensive understanding about theentire process of investigation, especially the planning, collection, and interpretationphase. To be more specific, students first need to develop an understanding of thedata collection process itself. If students fail to use instruments appropriately,include enough repetitions, or observe experimental protocols, the quality of thedata will be compromised. Second, students need to develop an understanding forthe planning stage of the investigation. Well planned investigation is a cornerstoneof valid and reliable data collection. The decisions that students make while theyplan investigations affect the quality of the data. If students fail to include relevantdata, operationalize variables, or control variables appropriately, their data wouldbe flawed and biased regardless of how well they carry out the rest of theinvestigations. Lastly, students need to develop an understanding of the interpre-tation stage. Once data is collected, the raw data needs to be organized andinterpreted in such a way that logical inference could be made about the influencesof specific variables or features of the problems. In order to do that, students needto know how to organize and summarize the data in an appropriate format andinterpret such summaries in their own right.

Not all concepts and reasoning skills that consist of evidentiary competence havebeen researched systematically yet. In order to obtain more detailed informationabout students’ ability to collect and interpret data, this study assessed six conceptsand skills from Table 1, two per each phase of the investigation. The conceptsassessed in the planning phase were the notion of priority and relevancy of evidence.Students’ understanding for the priority of evidence was examined in Driver et al.(1996). They studied 9, 12, and 16 year old children and found that children’swarrants for beliefs (e.g., the earth is round) became more sophisticated with age,but even at age 16, only about one third of the warrants relied on more sophisticatedreasoning about evidence or authority. The other warrants relied on directperceptual evidence, blind authority or did not distinguish between observationand explanation. Considering the amount of schooling and the emphasis on sciencein the general culture, it seems that students should be able to better appreciate therole of evidence, even if they might not know the specifics of the data collection. Wespeculated that the results might have been due to the fact that the students couldnot remember how they learned those specific beliefs. In the hope that they mightvalue empirical evidence more when they could plan a new investigation, we askedstudents to plan an investigation in this study. We also asked them to evaluate a setof claims in order to find out whether they could differentiate claims backed byempirical evidence from other types of claims.

There exist only a few episodic observations about students’ understanding ofthe notion of relevancy. For example, in an episode with fifth graders, Lehrer andSchauble (2002) observed that in the course of measuring the length of tobaccohornworms, many students insisted on including the name of the data collector foreach piece of data and whether that individual was a girl or a boy. Although thedebate was resolved after a heated discussion, it seems that children have a tendencyto want to include all available information. Grim, Pace, and Shopkow (2004)pointed out that like scientists, historians also need to sift through multitudes ofevidence and select the ones that are relevant to their questions and claims.According to their observation, even college students often fail to recognizerelevant evidence in primary historical sources. In this study, we examined this

Res Sci Educ (2007) 37:75–97 79

ability more systematically by asking sixth graders to differentiate irrelevant datafrom relevant data and also to plan a data collection in science investigations.

The concepts assessed in the collection phase were the notion of objective andreliable data collection. Students’ understanding for objective and unbiased datacollection has not been researched well. In this study, we examined whetherstudents appreciated the importance of objective and unbiased data by asking themto evaluate and plan data collections. Unlike students’ understanding of objectivity,students’ understanding for repeated measurement has been researched quite well(e.g., Lubben & Millar, 1996; Kanari & Millar, 2004; Masnick & Klahr, 2003;Schauble, 1996; Varelas, 1997). The conclusion is somewhat mixed, however. On theone hand, studies indicated that students around this age did not typically planreplications, collecting data with no apparent awareness of the uncertaintyassociated with the measurement process or of the need to be able to defend theirdata as reliable. They looked confused and disturbed when they inadvertentlygenerated replications (Schauble, 1996). Even when they repeated measurements,they saw it as a matter of correcting a flawed initial measurement without anunderstanding of the inherent uncertainty related to measurement and did not knowwhat to do with the different values obtained through replication (Lubben & Millar,1996; Varelas, 1997).

At the same time, studies also reported that students possessed some rudimentaryunderstanding for measurement errors. Masnick and Klahr (2003), during a guidedexploration, asked second and fourth grade students questions such as BWhat wouldhappen if the identical experiment were to be repeated?^, BCan you think of somereasons why the results came out differently even though we rolled the same balldown the same ramp five times?^ Students were able to name sources of errorswhen replication failed. The researchers concluded that elementary studentsunderstood quite a bit about errors, especially measurement and execution errors,although their understanding was not well integrated into a coherent view yet. In aneffort to replicate and extend Masnick and Klahr’s finding in a more open-endedand unguided investigation, Kanari and Millar (2004) studied 10, 12, and 14 yearolds using a different task. Like Masnick and Klahr, they found that many childrenhave some awareness of measurement error, but concluded that only a minority ofstudents could be said to have developed their awareness of the variability ofmeasurements into a notion, however embryonic it is. Clearly more research isneeded, but so far it seems that children around this age have some rudimentaryunderstanding of the measurement errors and replication, although it is unlear howrudimentary or integrated their understanding is. Earlier studies mostly examinedstudents’ understanding of replication and measurement errors as they concern themeasurement stage, by observing students as they engaged in data collection orasking questions about their data. In this study, we examined students’ understand-ing of replication and measurement error as they plan an investigation and evaluatean interpretation, so that we could assess how well their understanding ofmeasurement error is integrated across different phases of the investigation.

As for students’ understanding regarding the interpretation phase, we examinedstudents’ ability to interpret examples and tables.1 Not much research has beencarried out on students’ ability to interpret examples in scientific investigation. In

1 Although we focused on the interpretation of data in this study, note that, as we discussed earlier,ability to organize and summarize data is also a part of evidentiary competence.

80 Res Sci Educ (2007) 37:75–97

this study, we examined students’ ability to interpret examples as they evaluate agiven example or generate their own. As for students’ ability to interpret tables,Wainer (1992) proposed three levels of questions that can be asked to measureproficiency in understanding quantitative phenomena: (1) Questions involving dataextraction (e.g., what was the US birth rate in 1980?), (2) questions involving trendsin parts of the data (e.g., between 1970 and 1985, has the US birth rate changed?),and (3) questions involving the comparison of trends and the formation of groups(e.g., which states show the same pattern of birth rate change?). Guthrie, Weber, andKimmerly (1993) investigated college students’ ability to interpret graphs and tablesand reported that in line with Wainer’s proposal, identifying patterns was moredifficult than locating specific information. In this study, we examined how wellsixth graders were able to locate and identify trends from a table.

In examining students’ understanding for evidence, our aim was not just to un-derstand children’s understanding per se, but eventually to integrate it into class-room instructions. We thus developed a written, in-class assessment that could beadministered to a relatively large group of students. Two questions were constructedfor each of the concepts or skills. We mixed open-ended and multiple choice withjustification format. All questions were contextualized in atmospheric science, thatis, weather domain. It is familiar enough to all children so that they are all expectedto have some knowledge and experience, but at the same time provides enoughopportunity for new investigations and learning. The test was initially designed in aneffort to gauge students’ understanding in order to tailor a specific curricular programcalled Kids as Global Scientists (KGS) (Songer, 1996), but it can be usedindependent from KGS, as long as students have some basic concepts about watercycle and weather phenomena.

Method

Participants

Forty students (18 girls and 22 boys) from two sixth grade classes in one urbanschool participated in this study. Both of the classes were taught by the same scienceteacher who had been teaching middle school science for more than 13 years.

Evidentiary Competence Test

The test contained 12 questions, two for each of the six concepts and skills addressedin this study: Priority, relevancy, replicability, objectivity of evidence, and exampleand table interpretation. In order to contextualize the questions and to help studentsrelate to the questions more personally, each item started with a situation orproblem posed by a peer student (e.g., Juan experienced a tornado for the first time),followed by a request to evaluate or help the peer’s reasoning and inquiry. Half of thequestions asked students first to choose between Fyes/no_ or other alternatives andthen justify their choice (in notation, for instance, Question 1A refers to the choicepart and Question 1B refers to the justification part). The other half of the questionssimply asked students to evaluate or help their peer’s reasoning and write theirresponses in an open-ended question form. The order of the questions wasrandomized in the test except for Question 7 and 8 which used the same data setand had to be presented next to each other.

Res Sci Educ (2007) 37:75–97 81

The science teacher and another teacher administered the test during the sameclass period. Questions were presented two per page in order to provide space forwriting (see Appendix for the Inquiry Test; the spaces for answers were removed inthe Appendix). Each category of questions is described below.

Priority of evidence. Question 1 and Question 2 examined whether studentsunderstood the priority of evidence. In Question 1, students were presentedwith a weather prediction that it would snow the next day along with fourdifferent rationales. These rationales included a dream, a remark from anauthority figure (i.e., father), someone’s intuition, and a weather news forecast.Students were asked to pick the alternative with the most scientific reasoningand to explain their answer. In Question 2, students were presented with a claimmade by a friend’s grandmother that her arthritis gets worse before it rains.Students were asked to help his friend to find out whether her claim is valid.

Relevancy of evidence. Question 9 and Question 6 examined whether studentsunderstood what kinds of evidence were relevant to their question. In Question9, a student, Chantel, wanted to know how the distance from the equator(latitude) affected the daily high temperature of a city during winter. Shecollected three kinds of data from four cities: (1) The latitude of the city, (2) theaverage maximum temperature, and (3) the month of data collection. Studentswere asked to determine which data among the four choices was not relevant toanswer the question and explain their choice. Question 6 was about Rodneywho wanted to find out whether he should bring his sled with him when hevisited his uncle during the winter break, anticipating snowy weather. Studentswere asked to list two kinds of weather data that Rodney needed to check inorder to decide if he should take his sled or not.

Objectivity of evidence. Question 11 and Question 4 examined whetherstudents understood that data needed to be collected in such a way to ensurea fair comparison across different observers and situations. In Question 11,students were presented with two kinds of cloud coverage data that twostudents, Peter and Shanice, collected. Shanice used narrative or descriptivewords to describe the amount of cloud coverage (e.g., Flots of cloud_ or Flittlecloud_), whereas Peter used percentages (e.g., F30% coverage_). Students wereasked to pick whose data they preferred and explain their choice. In Question 4,students were presented with a disagreement between two students. The twostudents, one living in Chicago and the other living in Denver, each claimedthat their city had the strongest wind. Students’ task was to come up with a wayto resolve this conflict and determine who was right.

Replicability of evidence. Question 3 and Question 12 examined whetherstudents understood the importance of replications in planning and evaluatingdata collection. In Question 3, a student named Juan had just experienced atornado for the first time in his life. He noticed that there was a lot of lightningin the sky immediately before the tornado passed. Based on this singleobservation, he concluded that lighting always accompanies a tornado.

82 Res Sci Educ (2007) 37:75–97

Students’ job was to decide whether they agreed with his generalization andexplain their choice. The word Falways_ was underlined in the question toemphasize that a generalization was made. In Question 12, students were askedto make two suggestions about how to collect monthly precipitation data fortheir city.

Example interpretation. Question 5 and Question 10 examined whetherstudents could evaluate an example with regard to a hypothesis or anexplanation. In Question 5, a student named Danny learned about the existenceof water molecules in the air and wondered whether water droplets formedoutside a bowl containing ice cubes are from the air. Students were asked todecide whether this example supported what Danny learned and to explaintheir answer. In Question 10, a student named Shante learned that wind makespeople feel colder than the actual temperature. Students were asked to generatesupporting examples.

Table interpretation. Question 7 and Question 8 examined whether studentscould read and interpret a simple table. Both questions used the same data setthat a student named Jackson collected for cloud coverage and humidity duringa five-day period. Throughout the period, the humidity increased steadily andreached 100% on the last day. Cloud coverage also increased steadily althoughit did not reach 100% coverage on the last day. In Question 7, students wereasked to judge whether the humidity increased as the cloud coverage increasedand to explain their answers. In Question 8, Jackson came up with a conditionalrule that precipitation occurs when the humidity level is close to 100%, which isscientifically correct. Students were asked to pick a day that was most likely tohave precipitation if Jackson were correct.

Coding

Answers and coding schemes for the justifications and explanatory answers aredescribed in Tables 2 through 4. Justifications of students’ choices and answers toopen questions were coded into either Fgood_, Fpoor_ or Firrelevant_ categories.Justifications and answers were coded Fgood_ if students showed some understand-ing of the target concepts. Justifications and answers were coded Fpoor_ if students_explanations were too vague or students just reiterated the questions. Justificationsand answers were coded Firrelevant_ if students_ answers were incomprehensible,confused, or inappropriate. For example, when asked to explain why they decidedthat the weather data from Boston was unnecessary (Question 9), a studentanswered, BI got my answer by looking at the chart. Chantel is not just looking forthe weather data of Boston; she is looking for all of it.^ Although one might gothrough such steps to arrive at an answer, such a response did not answer whyBoston’s data was not needed and was thus coded as irrelevant/inappropriate. Notethat the coding focused on the target concepts and reasoning skills. As long asstudents show some understanding for the target concepts or skills (e.g., objectivityof evidence), their answers were coded as good even if their data collection wasflawed in other respects (e.g., inclusion of irrelevant data). In other words, an

Res Sci Educ (2007) 37:75–97 83

answer could be good in one aspect of data collection or interpretation, but couldbe erroneous in other aspects of data collection.

Two coders independently coded students’ answers. Initial agreement was 89%(Question 1B), 80% (Question 2), 83% (Question 3B), 75% (Question 4), 64%(Question 5B), 90% (Question 6), 60% (Question 7B), 95% (Question 8), 80%(Question 9B), 78% (Question 10), 74% (Question 11), and 76% (Question 12).The disagreements were resolved through discussion. When verbatim quotes of

Table 2 Coding Schemes for Priority and Relevancy of Evidence.

Question Choice Justification/Explanatory answers

Good Poor

Priority 1A d Students show understanding

about the role of evidence

or the data collection

process behind the news

forecast or understand the

problems with other types

of justifications (e.g.,

Fsomeone dream[s] or

suggest[s] it is going [to

happen], it may not happen

at all_).

Students simply state that

news can be trusted (e.g.,

Fweather people are most

of the time right_) or

describe what the news

does (e.g., Fthe news will

tell you when it is going to

snow or rain or something

like that_).

2* Students plan empirical

investigation (e.g., FTony

could wait and see if it rains

and then see whether his

grandmother’s arthritis

get[s] any worse_).

Students do not plan

empirical investigation

(e.g., FTony could go and

ask his parents if it is true.

Or he could ask his

grandmother in a nice

way_).

Relevancy 9 Boston Students point out that

Boston’s data was irrelevant

because it was collected in

the summer (e.g., Fbecause

it is in August the

summer_).

Students simply rephrase

the question or their

choice (e.g., Fthe weather

data is not needed because

it doesn’t need to be

Boston_).

6 Students collect relevant data

such as precipitation and

temperature data (e.g.,

Fhumidity_).

Students collect irrelevant

weather data (e.g.,

Fwhether it is dry_), non-

weather data (e.g.,

Fwhether they have hills_),

or vague data (e.g.,

Fweather_).

*Students frequently mentioned news or TV in their answer. Note that TV or news can play adifferent role in the process of inquiry: It could serve as a source of scientific data, but at the sametime works as an authority source. That is, news can serve either as a data source to obtain dailyweather information such as precipitation or as an authority source to get the final form ofknowledge. We distinguished these two different uses of news in coding students’ answers. Ifstudents mentioned using specific weather information from the news (e.g., Fturn to the weatherchannel and get ...... data_), we coded their answer as a form of data collection. On the other hand, ifstudents use news as the source of information to verify grandmother’s claim (e.g., Fwatch the newsto find out if it really happens_) or were unclear about how they are going to use the information inthe news (e.g., Fwatch the news_), we coded their answers as a form of authority consulting.

84 Res Sci Educ (2007) 37:75–97

students’ answers are presented in the Results section, we corrected students’spelling and grammatical errors (e.g., when students missed words or madegrammatical errors, appropriate or missing words were provided in brackets) butotherwise preserved students’ responses.

Results

Planning: Priority and Relevancy of Evidence

How well did sixth grade students in our study understand the role of empiricalevidence? In Question 1, students were presented with four different reasons for a

Table 3 Coding Schemes for Objectivity and Replicability of Evidence.

Question Choice Justification/Explanatory answers

Good Poor

Objectivity 11 Peter Students state the advantage

of using Peter’s numeric

recording system (e.g.,

FPeter’s data gives you a

clearer thought of cloud

coverage than Shanice’s_).

Students rephrase what

was already obvious in the

problem (e.g., FBecause

he has numbers_) or

provide vague answers

(e.g., FI like Peter’s best

because he tell[s] you the

coverage_).

4 Students mention using

measuring instruments,

measurement units (e.g.,

mph), or other

measurement methods to

ensure objectivity (e.g.,

FThey could both use a

wind measuring tool and

call one another and find

out the wind by mph._).

Students do not specify a

way of objectifying the

process of measuring

wind data (e.g., Fgo stand

outside and see if it push

[es] her down._).

Replicability 3 No Students understand that a

single incidence is not

enough to formulate a

general conclusion.

Students rely on other

rationale such as

knowledge or experience

as a basis of their

rejection/acceptance of

the conclusion (e.g., FI

agree with this choice

because usually you see

lighting before you hear

thunder or before a storm

occurs_).

12 Students plan repeat

measurement across

different locations or

different time points (e.g.,

F...Go outside every day

for two weeks with a

weather meter_).

Students plan data

collection without repeat

measurement (e.g., FGo

outside and feel the air_).

Res Sci Educ (2007) 37:75–97 85

weather prediction and were asked to choose the most scientific one. As can be seenfrom Table 5, 92.5% of the students selected the correct choice, but only 37.8% ofthem provided good justifications. Even in the students who provided goodjustifications, the majority of them (79%) focused on the problems with otherchoices (e.g., BIt has to be d. You can’t just know it. Her father could have messedup on it and say snow when they said rain. People dream a lot of things that don’thappen^), and only a small portion of students manifested some understandingabout the role of evidence or the data collection process behind the news forecast(e.g., B[It is] most likely the weather forecaster[s] are right and in order for them tocollect data they use scientific methods^). In some cases, students even seemed toaccept the information in the news blindly without question (e.g., BJulie think it willsnow tomorrow because the T.V. said so^).

In Question 2, students were asked what they could do to evaluate a claim aboutarthritis and rains. Only 50% of the students planned to collect empirical data toverify the claim. The rest of the students did not plan empirical investigation andinstead chose to find an answer by consulting an authority source such as a doctor ora teacher. It seems that students do not fully understand the role of empirical evidence

Table 4 Coding Schemes for Example and Table Interpretation.

Question Choice Justification/Explanatory answers

Good Poor

Example 5 Yes Students explain how the

example supports the

knowledge about where the

water droplets came from

(e.g., FYes, because if

something is too cold so if

the water forms on the

outside of the bowl which is

condensation_).

Students simply rephrase the

question (e.g., Fthe example

is supposed to show air

being moist._) or provide

vague answers (e.g., Fthe ice

making the water which is

dropping and it’s somewhat

similar to what waters

vapors are_).

10 Students provide appropriate

examples that demonstrate

the role of the wind in how

we feel (e.g., Fwhen I was

playing ball and the wind

blew I was cold_ or Fin the

summer you use a fan to

help cool_).

Students do not provide

appropriate examples, just

reiterating the information

in the question (e.g., Fin

Denver, Colorado that wind

plays a role in how cold you

feel when you are outside_).

Table 7 Yes Students point out how each

variable behaved (i.e.,

whether it was increasing or

decreasing) and how the

two variables behaved

together (i.e., when one

variable increased/

decreased, whether the

other variable increased/

decreased as well).

Students just rephrase the

question (e.g., FBecause as

the cloud coverage goes up

so does the humidity_),

simply refer to the table

(e.g., FYou can see it on the

chart_), or mention the

behavior of only one

variable (e.g., Fhe said it was

increase of cloud coverage

[and] he was right_).

8 Answers including Friday Other answers

86 Res Sci Educ (2007) 37:75–97

yet. One could argue, however, that empirical investigation was not preferred bystudents in this question because consulting an authority is not only very naturalaround this age, but also it might actually be an easier way to find out whether rainreally affects arthritis than conducting an empirical investigation by themselves. SinceFpoor_ answers in Question 2 did not include data collection, we examined the Fgood_answers where students planned empirical investigation in order to examine moreclosely whether students had some understanding for the data collection process. Theresults showed that most of the data collection was naı̈ve and intuitive, relying on theirsense experiences and direct observations (e.g., Bsee whether it is going to rain’’).Some data collection (about 20% of them) contained serious problems and did notreally make sense other than the fact that they planned empirical investigations. Thus,it seems that failure to plan an empirical investigation in this question was not out ofconvenience but rather out of ignorance.

As for students’ understanding of the relevancy of data, Question 9 provided a setof data collected to answer a question and asked students to choose the data notneeded to answer the question. Fifty percent of the students correctly chose the

Table 5 Average Percentage (Number) of Students in Each Coding Category.

Concepts and

skills

Choice questions* Open questions

Question Correct/

Good

Other/

Poor

None/

Irrelevant

Question Good Poor None/

Irrelevant

Planning

Priority 1A 92.5%

(37)

7.5%

(3)

0.0%

(0)

2 50.0%

(20)

30.0%

(12)

20.0%

(8)

1B 37.8%

(14)

51.4%

(19)

10.8%

(4)

Relevancy 9A 50.0%

(20)

45.0%

(18)

5.0%

(2)

6 51.3%

(41)

22.5%

(18)

26.2%

(21)

9B 30.0%

(6)

5.0%

(1)

65.0%

(13)

Data collection

Objectivity 11A 77.5%

(31)

10.0%

(4)

12.5%

(5)

4 35.0%

(14)

35.0%

(14)

30.0%

(12)

11B 12.9%

(4)

67.7%

(21)

19.4%

(6)

Replicability 3A 57.5%

(23)

42.5%

(17)

0.0%

(0)

12 3.8%

(3)

40.0%

(32)

56.2%

(45)

3B 0.0%

(0)

87.0%

(20)

13.0%

(3)

Data interpretation

Example 5A 70.0%

(28)

27.5%

(11)

2.5%

(1)

10 20.0%

(8)

25.0%

(10)

55.0%

(22)

5B 17.9%

(5)

60.7%

(17)

21.4%

(6)

Table 7A 87.5%

(35)

7.5%

(3)

5.0%

(2)

8 55.0%

(22)

7.5%

(3)

37.5%

(15)

7B 45.7%

(16)

22.9%

(8)

31.4%

(11)

*In the choice questions, the first line refers to the data about students’ choice, and the second linerefers to data about students’ justifications of their choice.

Res Sci Educ (2007) 37:75–97 87

irrelevant data, but only 30% of them provided adequate explanations. In Question6 where students were asked to list two kinds of weather data they needed to gatherin order to find out whether it would snow during a visit to an uncle, only 51.3% ofthe suggestions were relevant out of the 80 suggestions made. Question 9 might havebeen too difficult for the students because it dealt with an unfamiliar situation. Onthe other hand, Question 6 dealt with a familiar situation, and students couldprobably answer the question correctly even though they did not know much aboutprecipitation. However, their performance was still poor. Ability to determine therelevancy of data seems to be quite poor around this age regardless of whether ornot students were familiar with the situation.

Data Collection: Objectivity and Replicability

As for students’ understanding for the need to collect objective and unbiasedobservation, Question 11 asked students to choose between two formats of datarepresentation, numeric and descriptive. There were 77.5% of the students who chosethe numeric format, but only 12.9% of them were able to articulate the advantage ofusing numeric data. Others mostly rephrased the question (e.g., BI like Peter’s betterbecause it has numbers and Shanie’s just said Lots, Little, some^) or provided vagueanswers. Question 4 asked students to decide how to resolve a conflict over which cityhad the strongest wind. The conflict could be resolved by measuring the wind speed inan objective way that both could agree upon. Only 35% of the students answered thatthey would use data collected with some kinds of tool or system of measurement (e.g.,BThe[y] could both use a wind measuring tool and call one another and find out thewind by mph^). The rest of the students were vague about their data collectionmethod (e.g., Bgo stand outside and see if it push[es] her down’’) or replied on othersources for the answer (e.g., Bask her science teacher’’).

As for students’ understanding of replicability of data, Question 3 providedstudents with a flawed generalization and asked them to determine whether theyagreed or disagreed with it. There were 57.5% of the students who correctlydisagreed with the faulty generalization, but none rejected the generalizationbecause it was unreliable evidence. There was only one student who pointed outthat a generalization based on a single incidence was problematic. In this case, thestudent did understand that a generalization based on a single case could underminean argument, but did not consider it important enough to override her experienceand ended up agreeing with the faulty generalization (e.g., BEvery time there is atornado I see lightning first. My prediction may be wrong because I have onlyexperienced one tornado.^). In Question 12, when students were asked to make twosuggestions about how to collect monthly precipitation data of their city, only 3.8%of the suggestions mentioned making multiple observations. Most of the studentsdid not plan to repeat measurements, indicating that they did not understand thatrepeated observation is needed in order to compensate for the uncertainties thataccompany the measurement and observation process.

Interpretation: Examples and Tables

How well can students interpret or judge examples to be evidence of a claim orhypothesis? In Question 5 where students were presented with an example andasked to decide whether the example supported the explanation, 70% of the

88 Res Sci Educ (2007) 37:75–97

students correctly selected the right choice, but only 17.9% of the students were ableto provide reasonable justifications for their choice. The majority of the students didnot seem to understand how the water droplets outside the bowl demonstrate theexistence of water vapors in the air. In Question 10, when students were asked togenerate an example that demonstrated the fact that wind plays a role in how coldwe feel, only 20% of the students were able to do so. In Question 5, students clearlyneeded some content knowledge. Although the topic was covered in school, it isabstract and students might have experienced difficulty remembering the appropri-ate content knowledge. However, Question 10 dealt with a situation where studentswould all have relevant experience and knowledge. Nonetheless, students stillshowed limited understanding of generating specific evidentiary examples. Thedifficulty with this question suggests that even when students possess relevant first-hand experience, most of them are not yet capable of interpreting and usingexamples, unable to make the connections between the example and the question athand.

As for students’ ability to interpret a table, Question 7 asked students to judgewhether there is a linear relationship between the amount of cloud coverage and theamount of humidity in the data collected over a five-day period. There were 87.5%of the students who correctly responded Fyes,_ and 45.7% of them provided goodjustifications. Question 8 provided students with a conditional rule about when it isgoing to rain and asked them to pick a day that was likely to have precipitation.There were 55.0% of the students who correctly answered Friday when the humiditylevel was 100%. In spite of the fact that the data in the table was quite simple,students still experienced difficulty with this task. One of the reasons for students’poor performance seems to be the fact that some of the students had difficultyreading the table correctly. For example, in the explanation of the choice inQuestion 7, one student read the table backwards and thought that the cloudcoverage and humidity decreased as the days went by from Friday to Monday (e.g.,Bthe reason I chose the answer is, ’cause take a look at the data table. The cloudcoverage, humidity doesn’t increase, it decrease[s]^). Another student failed torecognize that the two variables behaved in a similar fashion (e.g., BEvery time thecloud coverage increases doesn’t mean that the humidity increases also^).

Discussion

How competent are sixth grade students in terms of their understanding of scientificevidence and data collection? How well do they understand features of scientific dataand how to engage in systematic and unbiased data collection and interpretation?

Priority and Relevancy of Evidence

There are different ways of knowing. People can know things because it has alwaysbeen that way, because an authority figure says so, because it is logical or makes sense,or because they experienced it. What differentiates science from other ways ofknowing is that it accepts knowledge claims when they are verified with empiricalevidence. To be skilled at data collection and interpretation, students first need tounderstand that science is an endeavor to build knowledge that is verified withempirical evidence, using evidence to evaluate hypothesis or arguments. Whenstudents were faced with various justifications in this study, the majority of the

Res Sci Educ (2007) 37:75–97 89

students were able to differentiate claims backed by scientific evidence from othertypes of claims, but when it came to explaining their choice, few were able to do so.Students selected the right choice, but it was often because they had a blind trust in thenews media. In line with Driver et al. (1996), few understood and appreciated thenotion of evidence or the data collection process behind the weather forecast. Such alack of understanding for the role of evidence was also reflected in the method ofinvestigation students choose in evaluating a claim. When students were faced with aclaim and were asked to evaluate it, only about a half of them answered that theywould collect empirical data. It seems students have some intuitive understandingabout what kinds of justifications are more Fscientific_, but their understanding is notexplicit or elaborate. Students are constantly surrounded by scientific accomplish-ments in and out of schools, but an appreciation of evidence does not develop merelyas a result of such exposure. Doing lots of practical work or learning about scientificachievements does not seem to be enough. A more systematic and explicit instructionabout the role of evidence is needed in order to help students to develop anappreciation for empirical evidence and their role in science.

When planning an investigation, there are potentially infinite amounts of datathat one can collect for a given question. Although it is tempting to collect andanalyze all available data, more data is not necessarily a good thing. When collectingand analyzing data, it is important to differentiate data that are relevant to answerthe question from data that may be related tangentially but are not necessary toanswer the question. When students were faced with this task of selecting relevantdata in this study, only about a half of them correctly identified irrelevant data, andeven fewer students were able to explain their answer. Students’ performance wassimilarly poor when they planned an investigation themselves, collecting relevantdata in slightly more than half of the cases. Although this ability has not receivedmuch attention, we believe that one of the goals of inquiry teaching should be tohelp student to develop abilities to determine the relevance of evidence. Studentsshould be able to sift through different kinds of information and focus oninformation and data pertinent to their question at hand. Developing such skills isespecially needed in this current information age where people are constantlysurrounded with enormous data.

Results from this study suggest that students need two different kinds of learningin order to develop this ability. First, students need to develop at least someknowledge base about the topic that the question addresses. They don’t need to bean expert in the domain, but they need to know some rudimentary concepts aboutthe domain such as what latitude or average daily temperature is. We believe thatthe lack of such knowledge might have been the reason why some of the studentsexperienced difficulty with Question 9. Second, students need to develop reasoningskills required for filtering and evaluating data. Inadequate reasoning skills mighthave been why students’ performance was low in Question 6 even though it wasabout a domain where students have relevant experience. It seemed that their wholescheme about the topic (e.g., skiing) got activated once students read the question,and they just could not filter it based on what the question asks.

Objectivity and Replicability

The notion of objectivity is viewed differently depending on the field of study andrequires a quite sophisticated understanding, but an important first step towards a

90 Res Sci Educ (2007) 37:75–97

mature understanding of this notion is the realization that one’s own perception andjudgment are subjective in nature. A statement such as Fit is long_ or Fit producesenough enzymes_ are not considered appropriate in science because judgment oflength or enoughness can differ depending on the observer and the context. In orderto make data more objective and less biased, scientists often resort to a system ofstandardized measurement such as calibrated instruments and/or quantifications.Although use of instruments and quantification can also produce biased data, itallows data to be compared across different observers and situations.

In this study, when students were faced with numeric and descriptive data, manystudents preferred numeric data, but again few understood why quantification waspreferred. Only a fraction of them were able to articulate the advantage of objectivedata even in a rudimentary form. In addition, when they planned an investigation,only about a third of the students relied on using tools or some other system ofstandardized measurement and the rest relied on intuitive judgment or othermethods. As was the case with the priority of evidence, students seem to have someintuitive understanding for the value of quantification. However, without anunderstanding for why numeric data is preferred, a blind trust in numeric data canmislead students to think that numeric data is always better. Explicit instructions areneeded about why quantification or some other system of observing and measuringvariables is desired over human sense experiences and judgments along with theadvantages and disadvantages of various measurement methods and tools.

Due to measurement errors, one trial or observation is often not sufficient toyield representative or Ftrue_ value. Because of the inherent variability anduncertainty associated with the measurement process, it is important to replicatethe observation so that the obtained data is not a random occurrence or acoincidence. In this study, when students were asked to plan a data collection, only afew students planned to repeat their observation or measurement. When studentshad to evaluate a generalization made on a single observation, in spite of the factthat the word Falways_ was underlined, practically no students pointed out the flawsin the generalization or considered it important enough to constrain theirinterpretation of evidence. Students did not understand the inherent variabilityand uncertainty associated with measurement and observation process and that anisolated event might be a chance outcome and may not reflect the true phenomenonunder study. Lack of such understanding made students to rely on their personalexperience and knowledge that was often incorrect. Earlier research suggested thatstudents understand the need for replication to some extent as it applies to datacollection process per se. The results from this study indicate that level of theirunderstanding is quite poor and that their understanding, however rudimentary it is,is not mature or integrated yet to a level where they could consider measurementerror as they plan and evaluate data collection.

Example and Table Interpretation

During their investigation, students often encounter a specific example or case suchas a specific insect in a certain location or a specific weather phenomenon in acertain region. They then need to be able to link the specific cases or examples totheir investigation. When we asked students to evaluate whether a piece of datasupported the explanation, students’ performance was quite poor and only fivestudents were able to justify their choice correctly. Students’ performance remained

Res Sci Educ (2007) 37:75–97 91

at a similar level even when the question dealt with a weather phenomenon wherethe topic was easier. Interpreting example is not a simple task. It requires acomplicated process of representing and evaluating the examples in relation to thegiven hypothesis or explanation. Although students naturally use examples and theirown experiences for various reasoning (Kolodner, 1997), it seems that their abilityto use them in a more constrained fashion during scientific investigation is notdeveloped yet. In order to develop this ability, they first should be able to representthe example appropriately regardless of whether they encounter it duringinvestigation or retrieve it from memory. In addition, students should be able tomap the specific example to the given hypothesis, knowledge claims, or otherexamples. Developing students’ ability to identify and interpret examples wouldeventually form the basis for building student’s ability to make evidence–theorycoordination.

Even when all the data is collected in a flawless manner, one still needs toorganize and interpret it. Even before one formulates explanations from evidenceand/or uses it to evaluate specific theory and hypothesis, data needs to be organizedand interpreted in a way that their inherent characteristics and patterns could beidentified. When students’ ability to locate and interpret trend was examined in thisstudy, even though the table used in these questions was quite simple, sixth gradestudents’ ability to read and interpret data was quite weak. The majority of thestudents were able to identify a simple pattern from the table, but less than half ofthem were able to articulate the relationship between the two variables. In addition,only a little more than half of the students correctly used the table in order to locatean answer. Although students simply might not have known how to articulate thetrend because it was so obvious or have difficulty with the conditional rule, theirability to interpret a table was clearly problematic. Students seemed to havedifficulty in interpreting a trend or how to relate the behavior of one variable toanother. Explicit instructions on these aspects of table interpretation would helpstudents to interpret tables in a more skillful manner.

When one generalizes the results of this study, one needs to bear in mind that thestudents who participated in this study are from a disadvantaged urban school district.Urban students experience many challenges and barriers to the development of theirinquiry abilities. They suffer from problems such as large class sizes, inadequateclassroom space, outdated materials, little social or parental support, less state and localfunding, and difficulty with recruiting and keeping qualified teachers (Haberman, 1991;Songer, Lee, & Kam, 2002). As a result, there exist large achievement gaps betweenhigh-poverty urban students and students from less resource-poor schools, especiallywhen the tests measure higher-order thinking abilities (Atwater, 2000; Labov, 1995;Rodriguez, 1997). In this study, students were from a school located in a large districtserving about 180,000 students. Like many other American urban school districts,many students in this district came from middle or low-income households, and abouthalf of the students lived in households at or below the poverty line. Students’standardized test scores were also among the lowest, both across the state andnationwide. Thus, the poor performance found in this study may have been due tosuch conditions and environments that these students have been exposed to for manyyears.

Students are more likely to fall into biased thinking when they are ignorant of thescientific method. One of the best ways to overcome the biases in thinking andincorrect knowledge is to learn about the ways science goes about evaluating and

92 Res Sci Educ (2007) 37:75–97

testing knowledge claims. This study suggests that one of the areas that needstrengthening is students’ evidentiary competence, that is, their conception andreasoning skills involved in collecting and interpreting data. Recent advancement oftechnology such as the Internet has changed the nature of school science in manyways. Through the Internet, real-time, real-life scientific data are readily and directlyaccessible to students more than ever. In those technology-integrated scienceclassrooms where students work with such data, evidentiary competence is becominga more critical aspect of scientific literacy than ever, specially compared to atraditional science class where students mostly deal with well-organized and cleaned-up data. By developing evidentiary competence, students would be better equipped todeal with the challenges of the future classrooms.

Acknowledgement This research was supported in part by a grant from the Korea Science andEngineering Foundation (M10102040003-03B2204-00211) awarded to the first author and by a grantfrom the US National Science Foundation (REC-0089283) awarded to the second author. Anyopinions, findings, conclusions or recommendations expressed in this publication are those of theauthors and do not necessarily reflect the views of the funding agencies.

Appendix

KGS Inquiry Questionnaire

In this test, we are asking you to tell us what you would do and think about variousweather-related questions. You may not have studied these things before in school.So, do not worry if you are not sure about the answers. Try to answer them the bestyou can. Please note that your response will NOT affect your grade.

1. Rachel is a sixth grader living in Detroit. It is December, but it hasn’t snowedyet. Rachel and her friends all predict that it will snow tomorrow, but fordifferent reasons.

1-A. Which reason do you think is most scientific? Please circle your choice.

a) Mark thinks it will snow tomorrow because he dreamed he wassledding.

b) Rachel thinks it will snow tomorrow because that’s what her fathersaid this morning.

c) John thinks it will snow tomorrow because he just knows these things.d) Julie thinks that it will snow tomorrow because she heard it from

the news on TV.

1-B. Using complete sentences please explain your answer.

2. Tony’s grandmother says that her arthritis gets worse before it rains. Tonywants to find out whether this really happens to his grandmother.

What could Tony do to find out whether this really happens?

3. Yesterday Juan experienced a tornado for the first time in his life. He noticedthat there was lots of lightning in the sky right before the tornado passed. Fromthis experience, he thinks that lightening always occurs right before a tornado.

Res Sci Educ (2007) 37:75–97 93

3-A. Do you agree with his thinking? Please circle your choice.

3-B. Using complete sentences please explain your answer.

4. Latonya lives in Chicago. It is very windy where she lives. Last week, she metSam on the KGS Message Board. Sam lives in Denver, Colorado. Latonya andSam talked about their weather. Sam claimed that Denver is the windiest cityin US. Latonya thinks Chicago is very windy also. She wants to find out whichcity has the strongest wind.

What could Latonya do to find out which city has the strongest wind?

5. Denny learned in his science class the following concept:There are water vapors in the air.

Denny’s teacher asked his class to think of an example of this concept. Dennyremembered the following example:

Droplets of water are formed on the outside of a bowl containing ice cubes.5-A. Does his example support what he learned? Please circle your choice.

5-B. Using complete sentences please explain your answer.

6. Rodney is visiting his uncle during winter break. He likes sledding. He wants toknow whether he should pack his sled. Please list two kinds of weather datathat he needs to collect to decide if he should take his sled or not.

Data 1)Data 2)<Question 7 and 8>Jackson collected the following weather data for a week.

Yes No

Yes No

Mon (%) Tue (%) Wed (%) Thu (%) Fri (%)Cloud coverage 30 40 50 60 70Humidity 60 70 80 90 100

94 Res Sci Educ (2007) 37:75–97

7. After examining the data he collected, he concluded that as the cloud coverageincreases, the humidity increases also.

7-A. Do you agree with his reasoning? Please circle your choice.

7-B. Using complete sentences please explain your answer.

8. Jackson thinks that when precipitation occurs, the humidity level is close to100%. Based on the data he collected in the table, which day is most likely tohave precipitation, if he is correct?

9. Chantel wants to know how the distance from the equator (latitude) affects thedaily high temperature of a city during winter. She collected the following datain four cities.

9-A. Which of the following data is NOT needed to answer her question?Please circle your choice.

a) The month during which the data was collectedb) Latitude (the distance from the equator)c) Weather data of Bostond) Average Maximum temperature

9-B. Using complete sentences please explain your answer.

10. Shante learned in her science class that wind plays a role in how cold you feelwhen you are outside. Can you think of an example that shows this?

11. Shanice’s class decided to collect cloud coverage data. Shanice and Peter eachcollected the following data for three days.

Latitude (Distance from theEquator: -N)

Average Max.Temperature (-F)

Month

Miami 25 72 JanuaryDallas 32 55 FebruaryDetroit 42 30 DecemberBoston 44 80 August

Yes No

Shanice’s data Peter’s dataDay 1 Lots of cloud 70% coverageDay 2 Little clouds 20% coverageDay 3 Some clouds 40% coverage

Res Sci Educ (2007) 37:75–97 95

11-A. Whose data collection do you like better? Please circle your choice.

11-B. Using complete sentences please explain your answer.

12. Emily’s class wants to collect data on how much precipitation they have inFebruary in their city. However, the class is unsure about how they couldcollect the precipitation data to answer their question.

Can you make two suggestions about how Emily’s class should collect theprecipitation data to answer their question?

Suggestion 1)Suggestion 2)

Acknowledgement This research was supported in part by a grant from the Korea Science andEngineering Foundation (M10102040003-03B2204-00211) awarded to the first author and by a grantfrom the US National Science Foundation grant (REC-0089283) awarded to the second author. Anyopinions, findings, conclusions or recommendations expressed in this publication are those of theauthors and do not necessarily reflect the views of the funding agencies.

References

Atwater, M. (2000). Equality for black Americans in pre-college science. Science Education, 84(2),154–179.

Carey, S., Evans, R., Honda, M., Jay, E., & Unger, C. (1989). An experiment is when you try it andsee if it works: A study of grade 7 students’ understanding of the construction of scientificknowledge. International Journal of Science Education, 11(special issue), 514–529.

Chinn, C. A., & Brewer, W. F. (2001). Models of data: A theory of how people evaluate data.Cognition and Instruction, 19(3), 323–393.

Chinn, C. A., & Malhotra, B. A. (2002). Epistemologically authentic inquiry in schools: Atheoretical framework for evaluating inquiry tasks. Science Education, 86(2), 175–218.

Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young people’s image of science. Buckingham,UK: Open University.

Gott, R., & Duggan, S. (1996). Practical work: Its role in the understanding of evidence in science.International Journal of Science Education, 18(7), 791–806.

Grim, V., Pace, D., & Shopkow, L. (2004). Learning to use evidence in the study of history. NewDirections for Teaching and Learning, 98, 57–65.

Guthrie, J., Weber, S., & Kimmerly, N. (1993). Searching documents: Cognitive processes anddeficits in understanding graphs, tables, and illustrations. Contemporary Educational Psychol-ogy, 18, 186–221.

Haberman, M. (1991). The pedagogy of poverty versus good teaching. Phi Delta Kappa, 73, 290–294.Kanari, Z., & Millar, R. (2004). Reasoning from data: How students collect and interpret data in

science investigations. Journal of Research in Science Research, 41(7), 748–769.Klahr, D. (2002). Exploring science: The cognition and development of discovery processes.

Cambridge, Massachusetts: MIT.Klahr, D., & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12,

1– 48.Kolodner, J. L. (1997). Educational implication of analogy. American Psychologist, 52(1), 57– 66.Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning. Cambridge,

Massachusetts: MIT.Kuhn, D. (1989). Children and adults as intuitive scientists. Psychological Review, 96:674–689.

Shanice Peter

96 Res Sci Educ (2007) 37:75–97

Kuhn, D., Amsel, E., & O’Loughlin, M. (Eds.) (1988). The development of scientific thinking skills.San Diego, California: Academic.

Labov, W. (1995). Can reading failures be reversed: A linguistic approach to the question. In V. L.Gadsden & D. A. Wagner (Eds.), Literacy among African-American youth: Issues in learning,teaching and schooling (pp. 39–68). Creskill, New Jersey: Hampton.

Lehrer, R., & Schauble, L. (Eds.) (2002). Investigating real data in the classroom. New York:Teachers College.

Lubben, F., & Millar, R. (1996). Children’s ideas about the reliability of experimental data.International Journal of Science Education, 18, 955–968.

Masnick, A. M., & Klahr, D. (2003). Error matters: An initial exploration of elementary schoolchildren’s understanding of experimental error. Journal of Cognition and Development, 4(1),67–98.

Minstrell, J., & van Zee, E. (Eds.) (2000). Inquiring into Inquiry Learning and Teaching in Science.Washington, District of Columbia: American Association for Advancement of Science.

National Research Council (2000). Inquiry and the National Science Education Standards: A guidefor teaching and learning. Washington, District of Columbia: National Academy.

Norris, S. P. (1985). The philosophical basis of observation in science and science education. Journalof Research in Science Teaching, 22(9), 817–833.

Rodriguez, A. J. (1997). The dangerous discourse of invisibility: A critique of the National ResearchCouncil’s National Science Standards. Journal of Research in Science Teaching, 3, 19–37.

Schauble, L. (1996). The development of scientific reasoning in knowledge-rich contexts.Developmental Psychology, 32, 102–119.

Songer, N. B. (1996). Exploring learning opportunities in coordinated network-enhanced class-rooms: A case of kids as global scientists. The Journal of The Learning Sciences, 5(4), 297–327.

Songer, N. B., Lee, H.-S., & Kam, R. (2002). Technology-rich inquiry science in urban classrooms:What are the barriers to inquiry pedagogy? Journal of Research in Science Teaching, 39(2), 128–150.

Varelas, M. (1997). Third and fourth grader’s conceptions of repeated trials and best representativesin science experiments. Journal of Research in Science Teaching, 34(9), 853–872.

Wainer, H. (1992). Understanding graphs and tables. Educational Researcher, 21, 14 –23.Zimmerman, C. (2000). The development of scientific reasoning skills. Developmental Review, 20,

99–149.

Res Sci Educ (2007) 37:75–97 97