Advancing reflective judgment through Socioscientific Issues · JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 46, NO. 1, PP. 74–101 (2009) Advancing Reﬂective Judgment through

JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 46, NO. 1, PP. 74–101 (2009)

Advancing Reflective Judgment through Socioscientific Issues

Dana L. Zeidler,1 Troy D. Sadler,2 Scott Applebaum,1 Brendan E. Callahan1

1College of Education, University of South Florida, 4202 E. Fowler Avenue, Tampa,Florida 33620-5650

2School of Teaching and Learning, College of Education, University of Florida, Gainesville,Florida 32611-7048

Received 20 September 2007; Accepted 14 July 2008

Abstract: The purpose of this investigation was to explore possible relationships between Socioscientific Issues

instruction and students’ development of reflective judgment. The usefulness of the Reflective Judgment Model as a tool

for assessing the value of SSI is established in the parallels that can be drawn between them. Both involve ill-structuredproblems requiring evidence-based reasoning subject to differing interpretations by students, and both require

examination, analysis and the blending of scientific and normative evidence, as students use that evidence to support a

reasoned position. Results demonstrated both qualitative evidence revealing more sophisticated and nuanced

epistemological stances toward higher stages of reflective judgment, as well as statistically significant gains withintreatment groups with a moderately large effect size. Theoretical implications for advancing students’ epistemological

beliefs about evidence-based argumentation and pedagogical implications for rethinking how to connect science with

topics that are fundamentally meaningful to students are discussed. ! 2008 Wiley Periodicals, Inc. J Res Sci Teach 46:

74–101, 2009Keywords: Socioscientific Issues; reflective judgment; epistemology; nature of science; ethical issues

Purpose and Theoretical Framework

The Role of Socioscientific Issues in Science Education

The Socioscientific Issues (SSI) movement seeks to engage students in decision making regardingcurrent social issueswithmoral implications embedded in scientific contexts (Sadler, 2004; Zeidler&Keefer,2003; Zeidler&Sadler, 2008; Zeidler, Sadler, Simmons,&Howes, 2005). These issues provide studentswitha context that encourages active reflection and examination of relevant connections among science, their ownlives and the quality of life in their community (Driver, Leach, Millar, & Scott, 1996; Driver, Newton, &Osborne, 2000; Kolstø, 2001, 2006; Sadler, 2004; Zeidler, 2003).

The focus on utilization of SSI is based upon a theoretical framework drawn from areas ofdevelopmental psychology, sociology, and philosophy. It also includes a focus on discourse andargumentation intricately connected to moral and ethical issues, as well as emphasizing the developmentof character formation. It should be noted that this is where the SSI movement diverges from past Science-Technology-Society (STS) agendas. STS tended to emphasize student understanding of the interactionsamong science, technology, and society, but paid scant attention, if any, to the quality of social interactionsand reflective discourse most closely aligned with the formation of conscious and principles of justice(Zeidler & Sadler, 2008; Zeidler et al., 2005b). Within the SSI framework, however, students are exposed tomoral problems that involve a number of discrepant scientific, social ormoral viewpoints,manyofwhichmayconflict with the student’s own closely held beliefs. The scientific knowledge that forms as a result of socialknowledge construction and discourse becomes personally relevant and socially shared. A SSI curriculum

Correspondence to: D.L. Zeidler; E-mail: [email protected]

DOI 10.1002/tea.20281

Published online 5 December 2008 in Wiley InterScience (www.interscience.wiley.com).

! 2008 Wiley Periodicals, Inc.

focuses upon scientific knowledge that is obtained from data interpretation, analysis of conflicting evidenceand arguing viewpoints that may conflict with previousmisconceptions (Chinn&Brewer, 1993; Kuhn, 1993;Lin, 2007). For this reason, many science educators in the international community include SSI in scienceclassrooms to encourage the development of social consciousness and develop scientific habits of mind(Driver et al., 2000; Kolstø, 2001, 2006; Levinson, 2006; Ratcliffe, Harris, &McWhirter, 2004; Sadler, 2004,2006; Zeidler & Keefer, 2003; Zeidler, Sadler, & Applebaum, 2007). The advancement of concepts likesocial justice and nurturing scientific habits of mind have compelled researchers to consider the role of affectand emotion during discourse with respect to the role each plays in moral decision-making and characterformation (Berkowitz, 1997, 1998; Nucci, 2001; Sadler & Zeidler, 2004, 2005; Turiel, 1998; Zeidler &Schafer, 1984; Zeidler, Walker, Ackett, & Simmons, 2002).

Central to this approach is the concerted effort to provide opportunities for students to reflect on issues inorder to evaluate claims, analyze evidence, and assess multiple viewpoints regarding ethical issues onscientific topics through social interaction and discourse. These opportunities necessarily evoke nature ofscience (NOS) tenets in that SSI discourses are de facto tentative, creative, evidence-driven, culturallyembedded, etc. Further, Abd-El-Khalick (2003) has suggested that epistemological stances and NOS aspectsmay be developmentally linked to meaningful critical discourse regarding controversial SSI. However, thereis an absence of research frameworks that allows for this possible interaction. Therefore, in this study, weexamined how instruction driven by SSI over the course of an academic year affected students’ personalepistemological growth. Specifically, the purpose of this investigation was to explore possible relationshipsbetween SSI instruction and students’ development of reflective judgment.

Utilizing the Reflective Judgment Model to Evaluate SSI Instruction

Inasmuch as the development of reasoned argumentation is one of several desirable goals of utilizingSSI, a useful method of evaluating the efficacy of a SSI curriculum for epistemological development ofstudents in science classroomsmay be found in theReflective JudgmentModel (RJM), developed and refinedby 20 years of research by King and Kitchener (King & Kitchener, 1994, 2002; Kitchener, 1983; Kitchener,King, Wood, & Davison, 1989). The longitudinal and cross sectional studies included in this researchsuggests that individual views of knowledge change in developmental sequences over time, starting in youngadolescence and continuing through adulthood. The RJM is analogous to other developmental models(Broughton, 1978; Fischer, 1980; Perry, 1970; Piaget & Inhelder, 1969) in the organization of stages, theconsistency of varying subject responses, the discreet thought processes within stages, hierarchicalintegration among stages, and in the sequential order of stages. Reflective judgment involves the reasoningpatterns individuals use to support their approach to ill-structured problems (Hofer & Pintrich, 1997; King&Kitchener, 1994). In using the RJM as an organizing framework, we do not deny the influence of domain-specific experiences on students’ reasoning. To the contrary, we affirm such experiences are essential todevelopment of the SSI framework. However, the RJM provides an overarching developmental view ofsimilarities in patterns of reasoning useful in understanding epistemic trends in reasoning.

There are logical parallels that can be drawn between the RJM and the SSI framework. Both involve ill-structured problems and issues that entail many differing opinions, require the ability to analyze positions,use evidence to support a position, and recognize the role of constructed knowledge (particularly inmatters ofmoral sensitivity) in consensus building. This positions the RJM as an ideal means to assess the efficacy oflong-term SSI instruction. Accordingly, the quality of sociomoral discourse is intricately linked to both thequality of SSI instruction, and the epistemological sophistication of a student’s reflective reasoning.Instruction, therefore, should be aimed at facilitating structural changes in reasoning, judgment and discoursethat are increasinglymore robust, but developmentally appropriate (Berkowitz, Oser,&Althof, 1987;King&Kitchener, 1994).

The seven distinct stages of the RJM are organizedwithin three general groups. The first three stages arepre-reflective thought, stages four and five comprise quasi-reflective stages, and stages six and sevenconstitute mature reflective thought. The pre-reflective stages are defined by a single, concrete truth. Pre-reflective thinkers place a great deal of confidence in ‘‘authorities’’ to determine absolute truth, and thesepeople believe ‘‘truth’’ that has been (or will be) determined for them. Authorities, in this use of the term, canrefer to anyone whose knowledge is believed to be absolute, such as politicians, teachers, scientists, or

ADVANCING REFLECTIVE JUDGMENT THROUGH SSI 75

Journal of Research in Science Teaching

religious officials. As a result, people in these stages do not examine disconfirming evidence, and in fact, tendto believe the authority over contrary evidence.

The quasi-reflective stages are marked by an uncertainty in one’s belief system. Quasi-reflectivethinkers have advanced sufficiently to recognize that authorities are not necessarily or always correct, but theyhave not established a satisfactory paradigm to replace the naı̈ve, authority-driven epistemology. Althoughthe role of authority is diminished, there is an acceptance that authority is important, although potentiallybiased; however, authorities are no longer the single determinants of truth. Quasi-reflective thinkers arerelatively cynical, as they tend to believe that evidence is presented and molded to fit previous beliefs, ratherthan altering beliefs to fit the evidence. A person in this stage can examine evidence frommultiple viewpointsbut does not have the critical thinking skills needed to integrate various interpretations of evidence into acohesive knowledge base.

The reflective stages are characterized by a shift from contextualizing knowledge based on evidenceconsistent with prior beliefs to the construction and production of new knowledge, based on the evaluation ofevidence that is grounded within normally accepted criteria for valid argumentation (i.e., conceptual clarity,internal coherence, degree of fit with the data, utility, parsimony, etc.). These criteria parallel and areconsistent with more sophisticated NOS viewpoints. Reflective thinkers perceive authorities as experts withthe potential to produce viable solutions. This usage of the term ‘‘authorities’’ differs from the usage of pre-reflective thinkers in that the authorities tend to be experts directly in the field under consideration. There is anunderstanding that the experts are contributing to the production of new knowledge in the field. They alsounderstand that knowledge changes, based on available evidence. In these stages, people recognize thatevidence and knowledge originate from a variety of sources and they are capable of analyzing probabilisticclaims of contentious evidence. Whereas pre-reflective thinkers adopt relatively simple views of knowledgeand standards for the justification of knowledge, reflective thinkers assume more sophisticatedepistemological perspectives. A summary tablewhich provides detailed descriptions ofmajor characteristics,role of authority, role of evidence, views of knowledge and concepts of justification for knowledge that isfound within each stage is located in Appendix A.

Design and MethodsSample and Population

The participants in this study were members of four intact classes of 11th and 12th grade students (ages16–18) enrolled in Anatomy and Physiology classes in a large suburban high school in Florida. Two of theclasses were classified as ‘‘honors’’ made up of students who had excelled academically, and two of theclasses were classified as ‘‘regular’’ made up of students with diverse histories of academic performance.Each of the classes contained between 29 and 31 students. One of each of the honors and regular classes wasassigned to the comparison group; the other classes within each designation were assigned to the treatmentgroup. An experienced teacher who holds terminal degrees (DMD and JD) taught all four sections to controlfor variation in teacher attributes.

The students who were enrolled in the four classes (standard curriculum and treatment sections) wererandomly assigned to those classes by school administrators; however, their placement within honors andnon-honors sectionswas based upon the quality of grades earned in previous science classes andmembershipin the International Baccalaureate (IB) magnet program. The teacher noted that a comparable number ofstudents, including an equal number of males and females were represented in both the honors and non-honors sections. Additionally, both treatment and standard curriculum honors sections contained equalnumbers of IB students. It is significant to note that students enrolled in the IBmagnet programhad completedcollege level biology and chemistry courses. Whereas it is beyond the scope of this paper to include adescription of the grade transcripts of each student and class, the teacher/observer noted that the rankings ofthe students within their graduating class was comparable and equally distributed for both of the honors andnon-honors classes, and treatment and standard curriculum sections. The two honors classes (one of whichwas assigned to the treatment group and the other was assigned to the comparison group) demonstrated moreethnic diversity; specifically, the numbers of Asian and Indian students were better represented. The non-honors sections had few students that were not European-American. The school at which the research wasconducted is situated in a suburban, middle to upper middle class neighborhood, and most students attending

76 ZEIDLER ET AL.


the school, including those in the classes under investigation, live in the surrounding neighborhood. Theschool population has very few students from lower socioeconomic status neighborhoods. Most of thestudents from transported to the school from lower income areas of the county were African American;however, few African American students enrolled in this particular course. Only four African Americanstudents participated the SSI project; consequently, the student demographic representing this subset is notwell represented in the study.

Learning Conditions

Both the comparison and treatment groups received explicit NOS instruction. Like much of the scienceeducation community, we believe strongly that NOS should be a fundamental aspect of all science courseinstruction. The significance of NOS as a goal for science education is well established (Harding & Hare,2000; Irez, 2006; Khishfe & Lederman, 2006; McComas, Clough, & Almazroa, 1998), so we took explicitNOS instruction as a given for any high quality science curriculum as well as a necessary component of ameaningful comparison treatment. Therefore, NOS instruction was explicitly integrated throughout thelearning environments of both the comparison and treatment groups. Our approach to explicit NOSinstruction was consistent with the field’s dominant NOS framework (Lederman, 2007) and involved severalstandard NOS activities (Lederman & Abd-El-Khalick, 1998) including ‘‘black box’’ exercises in whichscientific processes are modeled and guided reflections on NOS themes.

Comparison Group

The curriculum and pedagogy for the comparison group was driven by the anatomy and physiologytextbook (Cummings, 2001) approved for use at the school site, as well as the NOS instruction previouslydescribed. Instruction in the comparison classes was consistent with traditional anatomy and physiologyinstruction in high school settings. The curriculum and lesson plans were organized on the basis of thedeveloping complexity of living organisms, beginning with cell and tissue structures and characteristics, andprogressing through the major organ systems (e.g., skeletal, muscular, nervous, etc.). Lecture was thedominant pedagogical strategy, with occasional laboratory activities that were scripted in preprinted labmanuals. The primary objectives were student mastery of structure, function, and pathology of anatomicalsystems. Preventing crossover was not difficult because the course was a yearlong project and the schooloperates on a semester calendar; therefore, tracking students was not an issue. Though students generallymaintain the same schedule throughout the school year, some students were moved from a non-treatmentclass to a treatment class (or vice versa) because of schedule changes that were external to the study. In eachincident, the data provided by these few students were removed from further analysis.

Treatment Group and SSI Framework

Design of the treatment instruction was informed by a general framework of eight ‘‘contenttranscending’’ (Kolstø, 2001) themes for examining the science dimension of SSI in science education andinclude: (1) Science-in-the-making and the role of consensus in science; (2) Science as one of several socialdomains; (3) Descriptive and normative statements; (4) Demands for underpinning evidence; (5) Scientificmodels as context-bound; (6) Scientific evidence; (7) Suspension of belief; and (8) Scrutinizing science-related knowledge claims. These themes were not used to match a given activity in a checklist manner to thecurriculum; rather they provided a template that served as a pedagogical mind-set for the researchers, andespecially for the classroom instructor. Additionally, the treatment group, with explicit focus onargumentation and discourse, followed a general organizational framework for decision-making usingcontroversial issues derived fromKeefer (2003), Ratcliffe (1997), and Pedretti (2003) that includes clarifyinggeneral and scientific knowledge, and criteria for evidence, considering alternative scenarios that argue fordifferent conclusions, and identifying and evaluating moral consequences.

We found many parallels between this framework and the characteristics of the RJM, including the useof evidence-based reasoning, consideration of the role of authority, understanding the relationship betweenthe role of knowledge and the status of epistemic beliefs. These parallels make the RJM an effective tool forevaluating the efficacy of SSI in the classroomover a long-termbasis. Selected features for utilizing reflectivejudgment, derived from Baxter Magolda (1999), Kegan (1994), and King and Baxter Magolda (1996), that



Chris McDonald

have the potential to impact the pedagogy of incorporating SSI in the classroomwere identified and served asa guide for practice and included the following strategies:

(1) Show respect for students’ assumptions, regardless of the developmental stage(s) they exhibit.

Their assumptions are genuine, sincere reflections of their ways of making meaning, and aresteps in a developmental progression. If students perceive disrespect or lack of emotional

support, they may be less willing to engage in challenging discussions or to take the intellectual

and personal risks required for development.

(2) Discuss controversial, ill-structured issues with students throughout their educational activities,and make available resources that show the factual basis and lines of reasoning for several

perspectives.

(3) Create many opportunities for students to analyze others’ points of view for their evidentiary

adequacy and to develop and defend their own points of view about controversial issues.(4) Teach students strategies for systematically gathering data, assessing the relevance of the data,

evaluating data sources, and making interpretive judgments based on the available data.

(5) Help students explicitly address issues of uncertainty in judgment-making and to examine theirassumptions about knowledge and how it is gained (King & Kitchener, 2002, p. 55).

The activities selected and developed for the treatment group were designed to move students towardbetter understandings of scientific concepts (Roth, 1990) and their application to SSI and developed inconsultation with the instructor. Figure 1 provides an overview of the SSI driven curriculum created for thisproject. This figure reveals the interrelationships between classroom science content knowledge and thecomplex social framework in which these concepts are embedded.More specifically, the figure illustrates theten main SSI units highlighted in the curriculum, the primary anatomical systems related to the issues, andthe connected scientific content and concepts developed by the corresponding arguments, debates, anddiscussion. The issues were carefully chosen to align with students’ interests. The course content wasembedded within the SSI and designed to challenge core beliefs and apply new content knowledge toappropriate scientific contexts in a manner that was personally relevant and meaningful. Topics ranged fromorgan transplant allocation, the safety of marijuana and fluoridated water, the morality of stem cell research,to euthanasia, quality of life issues, fast food consumption and other contemporary subjects that were sociallyrelevant. Each SSI required between three and seven classes to complete; however, connections betweencontext and content were reiterated on multiple occasions during teachable moments throughout theacademic year. The treatment curriculumwas formulated to offer unique opportunities to confront, defend, orreject contentious information.

Appendix B illustrates one example of a selected SSI (Marijuana) from Figure 1 and provides a closerview of the pedagogical details for using this topic as a controversial issue. In the activity regarding the safetyofmarijuana, students were provided an opportunity to explore the drug’s active ingredients and its effects onspecific areas of the brain. Further, the participation in classroom debates as well as small and large groupdiscussion encouraged students to confront core beliefs about recreational drug use and their personalthresholds of risk. In this particular activity, students were actively involved in reading and evaluatingconflicting evidence from credible sources and negotiating their conclusions within and among other groupsof students. This type of activity also provided opportunities to observe the criteria students used in theirselection of ‘‘credible’’ evidence. Through several phases of this activity, students were required to workindividually, in small groups, and interact as a whole class. Their challenges included reading articles withconflicting evidence from varied sources, identifying important data and arguments, ranking the importanceof evidence, fromgroup consensus positions, debate of positions, and serving as editors for amock ‘‘scientificresearch journal’’ with the goal of evaluating other groups’ presentations of positions and evidence. Thus, thecurriculum included multiple activities that required participants to evaluate claims, analyze evidence andtheir sources, come to a decision on a personal position, make moral decisions, and present the informationwithin a group of peers to negotiate a consensus opinion.

The goal of engagement in the learning process was essential to the success of the curriculum; thecontext of the subject matter served as a stimulus to learning the content, which provided the framework fordecision-making and understanding. Additionally, overcoming misconceptions was a major factor forattracting students to become involved in designed activities. Cigarettes, alcohol consumption, recreational

78 ZEIDLER ET AL.


Figure 1. Overview of socioscientific issues driven curriculum.



drugs, and steroid abusewere easily accepted topics; however, students were also attracted to debating issuesthatwere peripherally relevant, such as fluoridatedwater, organ transplants, animal rights, and bacterial/viralepidemiology. The advantage of using this format was the ability to adjust the themes to accommodate boththe academic abilities and interests of the students, as well as the different science disciplines.

It is important to note that a concerted effort was made to select issues that would drive the Anatomy &Physiology curriculum, and provide a context for the development of scientific content understandings.Accordingly, the teacher’s responsibility was to ensure that the students focused their learning through theSSI context by serving as a facilitator and guide throughout the learning treatments. Appendix C reflects theteacher’s role by illustrating the pedagogical relationships between teacher and students’ SSI discourse. Ourworking assumption was that SSI units afforded the context for students to understand, through carefullycrafted experiences, that scientific knowledge is theory-laden and socially and culturally constructed. Anoverview of the learning conditions is further summarized in Table 1 below.

Wewere keenly aware that the novel format of the SSI curriculummandated that researchers and teachermaintain a continuous and open involvement on daily, weekly, andmonthly assessments of the progress of theclass, student engagement, class discussion and writing performances, as well as the quality of the activitydesign. The teacher had extensive knowledge of standardized curriculum and the textbook version of thesubject. He was also well versed regarding the details of current information for each SSI and possessed anunderstanding of the historic and contemporary frameworks of the contentious topics selected. However,while the teacher had been exposed (through coursework) to the theoretical underpinnings that were the focusof the research, the pedagogical practices necessary for the delivery of the SSI approach was a new strategyfor the teacher to work through.Without historical guidelines, personal dialogue with the researchers helpedto provide a level of confidence necessary to provide the impetus for the teacher to continue instituting thisunique presentation of science. Classroom assessment of science knowledge and robust descriptions ofcontent understanding by the students throughout the project provided an additional sense of confidence inthismethodology.Adjustments in topic selection, age appropriate information and format construction by theresearchers, were occasionally instituted to ensure student interest and engagement.

Data Collection

As previously mentioned in the Introduction, the RJM was formulated from cross sectional andlongitudinal studies over the past 20 years. The primary instrument for assessing reflective judgment is thePrototypic Reflective Judgment Interview (PRJI; King & Kitchener, 1994, 2004). To conduct the PRJI, aninterviewer presents a scenario describing an ill-structured problem to a participant. After the participantreads through the brief scenario, the interviewer asks seven standard questions that encourage the participantto describe his/her position on the issue as well as a justification for that position. The sequence of interviewquestions is also designed to elicit epistemological perspectives including ideas about the certainty ofknowledge, the adequacy of alternative interpretations, and the significance of ‘‘expert’’ opinions. The PRJI

Table 1

Pedagogical framework for SSI study

Comparison Group Treatment Group

Approach Traditional approach: content topics followtextbook chapter topics

Socioscientific issues approach: content-relatedcourse topics embedded with SSI

Teaching methods Lecture, lab, discussion of content-relatedconcepts, worksheets, pre-designed labactivities

Focus on argumentation and discourse, smallgroup activities, role-play and student researchinto SSI. Limited lectures and traditional labs

Nature of science Explicit activities and connections are made Explicit activities and connections are madeIntended outcomes Mastery of structure, function, and pathology

of anatomical systems; more sophisticatedviews of NOS

Improved critical thinking and decision-makingparticularly in the context of SSI; engagementin scientific discourses; sociomoraldevelopment; content mastery; moresophisticated views of NOS

Classes 2 Classes: 1 regular and 1 honors 2 Classes: 1 regular and 1 honors

80 ZEIDLER ET AL.


scenarios used in this project (related to chemical additives in food, religion and science, and geneticdetermination of alcoholism) as well as the interview questions are presented in Appendix D. Following thisprotocol helped to ensure data collection continuity across all classes. The PRJI has been reported to producehigh test–retest reliability (e.g., 0.87) and high internal consistency across multiple approaches, whileconstruct validity of the RJM has been documented extensively (King & Kitchener, 1994).

The PRJI was administered with students at the beginning of the school year and again, to the samestudents, 7 weeks prior to the end of the school year (elapsed time: approximately 7 months). Since the PRJIinvolves an extensive interviewing process taking between 30 and 40 minutes, a random sample of tenstudents per class (initially n! 40) was selected for the pre- and post-test interviews. Deviation from thisnumber in the posttest results reflects students’ attrition associated with an academic yearlong study.Following initial administration, some students left the class, or switched class sections from a comparison totreatment group, or vice versa. (Student mobility out of the class altogether and between sections was a verytypical phenomenon in this school setting.)

Three doctoral students (who were blinded to which students belonged to treatment or comparisongroups)with coursework in epistemology and cognition and a professor of science education familiarwith thePRJI conducted interviews privately in a teacher’s office/prep area apart from the main classroom. Thestudents’ responses were recorded and later transcribed following the recommended format by King andKitchener (1994, pp. 264–265). Responses were then assessed according to the students’ views on the natureof knowledge and their concepts of justification for that knowledge, consistentwith the framework previouslydescribed (see Appendix A).

Data Analysis

We employed a mixed-methods analysis strategy for the PRJI data, a strategy consistent with extensiveresearch related to reflective judgment (King & Kitchener, 1994, 2002; Kitchener, 1983; Kitchener et al.,1989). Initially, interview responses (to all seven main questions) for each of the three scenarios werequalitatively analyzed for correspondencewith the seven developmental stages postulated by theRJM.Whileit was theoretically possible for a student to offer responses representative of widely varying stages ofreflective judgment, the developmental nature of the construct under investigation tended to restrict the rangeof represented stages to a maximum of three. However, most students demonstrated evidence of only one ortwo stages across the three scenarios. The results of these analyses were summarized in a three-digit code(e.g., 3-2-4). The first digit represents the dominant stage, and the second and third numbers representsecondary and tertiary stages (i.e., stages that were expressed less frequently). A 3-2-4 score indicates that theparticipant expressed stage 3 reflective judgment in a majority of his/her responses. This participant alsoexpressed stage 2 reflective judgment, but at a lower frequency than stage 3, aswell as an even lower incidenceof stage 4 reflective judgment. If only one stage was demonstrated (e.g., stage 3), then the coding schemewould only represent the single stage (e.g., 3-3-3). Often, one stage represented themajority of responses, butan additional stage was revealed in a small minority of responses. If, for instance, a participant expressedstage 3 reflective judgmentmost of the time but occasionally reverted to stage 2, their scorewould be reportedas 3-3-2. It should be noted that most PRJI responses (in this project as well as other investigations using thisframework) reflect only one or two stages. The display of three stages (e.g., 4-5-3) occurs much lessfrequently, but is consistent with the range of developmental judgment possible and the continuous nature ofreflective thought.

Three raters, who were very familiar with the PRJI protocols, randomly selected three transcripts toindependently code. Next, they collaboratively compared and reviewed their findings. These raters wereblinded as towhich transcripts belonged to students from the treatment or comparison classes. The first authormediated discussions regarding each rater’s rationale for an assigned score. This session served as anopportunity for the raters to ‘‘calibrate’’ their analyses to ensure consistent application of the RJM. Anadditional two transcriptswere then scored in common (again independently for each rater)with an inter-rateragreement of 100% on the dominant stage and over 90% for the less dominant stages. The remainingtranscripts were then scored individually by one of the three raters.

The PRJI provided a window into the epistemological beliefs concerning how students come to makesense of and evaluate the status of knowledge claims, allowing a rich interpretation of the data.We found that



while analyses of students’ responses provided quantitative interpretations useful for examining and codinggroup trends (described above), comparisons of pre- and post-test qualitative data provided indicators ofdifferences in a single student’s reflective judgment over the course of the school year. We describe suchindicators in the analysis of individual responses below.

Results

Qualitative Analysis of Individual Responses

Qualitative analysis of individual student responses to the three PRJI scenarios provided the foundationfor the investigation. In this section, we present examples of students’ interview responses to commonquestions in the pre- and post-interviews. These particular exampleswere selected because they represent andtypify a spectrum (slight to considerable) of developmental shifts in epistemological perspectives.Incremental shifts are consistent with a developmental model of reflective judgment whereby students movefrom more naı̈ve to more robust orientations (note Appendix A).

Six pre–post comparisons (two examples from each of the three scenarios) are presented in Table 2.Researchers’ interpretations of the data are also included. It should be emphasized that a single quote does notindicate a significant conceptual shift in and of itself; rather epistemological differentiation and progression isinferred over a minimum of 21 interview questions found in Appendix D. (It is important to note thatadditional probing questionswere embeddedwithin the interviewquestions.)Weprovide these data as ‘‘casesin point’’ to give insight into the subtle and developmental nature of reflective stage progression. The rationalefor our interpretation of pretest and posttest indicators may be found in the a priori framework of the RJMsummarized in Appendix A.

Upon close inspection of these cases in point, a general trend toward higher stage attainment includinggreater differentiation within a given broad epistemological stance becomes evident (e.g., quasi-reflectiveorientation).Our inferences are based on the developmental assumptions thatmovement toward higher stagesof reasoning represents structural cognitive changes that allow for the consideration of multiple forms ofevidence from varied sources, and that evidence, arguments, and sources can be checked against logicalstandards for internal coherence and/or a priori criteria for the soundness of empirical evidence. This isclearly the case for student ID-26 (Table 2; Chemical Additive Scenario), who is able to make a distinctionbetween circumstances in which scientists seek only to ‘‘prove’’ self-fulfilling prophecies (see the pretestinterview), from his/her recognition of intervening variables that may account for varied empiricalexperimental outcomes (see the posttest column). Similarly, we find student ID-11 (GeneticDetermination ofAlcoholism) has a narrow and limited viewpoint during the pretest interview by a belief that experts just have‘‘different opinions’’ on the same set of facts because of how they were raised, whereas during the posttestinterview, there emerges a broader consideration for contrasting experiments that form an empirical basissupporting a given stance on an issue. Here again, we observe developmental shifts toward greaterepistemological differentiation.

In general, these two scenarios more frequently revealed subtle degrees of developmental progressionthan the Science and Religion Scenario. In Table 2, we presented two responses to the Science and Religionissue; one that showed no stage progression (ID-38) and another that demonstrated a shift from pre-reflectiveto reflective reasoning (ID-35). The latter tended to be the exception to the rule inasmuch as the topic at-handwas one that challenged deep-structure core beliefs, which are customarily well protected from the ‘‘assault’’of external arguments or evidence. Here, we infer that there are contextual factors that may inhibit thedevelopment of reflective judgment toward more robust stages of reasoning. We consider this next in moredetail.

While Table 2 illustrates our assessments of students’ responses used to derive reflective judgmentscores, we present more extensive dialogue from two students during the posttest interview to illuminateexamples of nuanced epistemological orientations that are contextually tempered and appear to challengecore beliefs. In these discussions, the Researcher (R) attempts to probe the Student’s (S) epistemicassumptions about how she/he constructs and supports seemingly conflicting beliefs. Consider thefollowing exchanges with two different students (independently) during discussion of the Religion andScience Issue:

82 ZEIDLER ET AL.


Table 2

Pre-post comparisons of individual student responses to the PRJI

ID Pretest Key Indicator Posttest Key Indicator Researchers’ Interpretation

Scenario: Chemical Additives26 Researcher (R): Now how do you think that people have such

different viewpoints on this issue?R: That some people might say that they’re safe, and other

people might say they’re not, how do you think the generalpopulation might come to those conclusions?

During the course of the pre-intervention interview, the studentacknowledged that experts in the field-test their ideas.She/he notes in the pretest that differing conclusions arecaused by researchers using research (data) to fit their priorconceptions, which is indicative of low quasi-reflectivethinking (stage 4). During the course of the post-testinterview, the subject does not mention fitting data to fitpreconceived ideas, rather the emphasis is on theunderstanding that testing for all variables causes difficulty.While this student remains in the quasi-reflective stages,there is evidence of structural change (i.e., stage 5) withsome progression toward reflective thinking

Student (S): Umm. Because artificial sweeteners probably havecaused health problems in the past, but at other timessomeone probably used their entire life and never had aproblem with it. So they want to study that, just to, sothey . . . I don’t know . . . (laughing)

S: Since it wasn’t added before and now they find there’schanges in what’s added

R: How is it possible that experts in the field disagree about thissubject?

S: Disagree that they’re harmful?R: How is it possible that experts in the field disagree about this

subject?R: Disagree whether they’re harmful. Some scientists say

they’re harmful, other scientists say they’re not. How canwe have these different opinions?

S: Because they’re directing their research to prove their side.And maybe that haven’t looked at the other side of it, sothey’re partial to one side maybe

S: I would say it’s a pretty hard thing to test, like just excludingartificial sweeteners from someone’s diet, there could beother factors, they could be doing something else harmful

1 R: Could you ever know for sure that your position on the issueis correct?

R: Could you ever say which is the better position? The student progresses from a low quasi-reflective stage that ismarked by ambiguity of varying viewpoints to a highquasi-reflective stage where one examines evidencesupporting both sides and makes a decision based on thatevidence. In this interview exchange, one cannot determinethe student’s ability to decipher data and make a validjudgment from that data; but the fact that the student usesevidence to make an informed decision indicatesprogression toward becoming a reflective thinker

S: No, I don’t think so S: I couldn’t from this evidence at least because there’s notreally any evidence. All it says is there have been studiesthat show that it’s bad and studied that show that it’s good sofrom this amount of information I couldn’t really say whichis good enough

R: Why not? R: How would you go about making a decision about this issue?S: Cause there is always going to be something else that is

contradicting to what I am believingS: I guess I would just research more into both sides of the

argument and try and put a few things together and decidewhich one is more convincing

Scenario: Science and Religion38 R: Could you ever say which one was the better position? R: So how did you come to hold your point of view on this

topic?In this instance, the student has not progressed from the

pre-reflective stage regarding religion and science, (althoughs/he exhibited progress on other scenarios). This is one issuethat appears to be greatly influenced by authority, perhaps dueto the relative lack of ability to experimentally design and testvariables on the spot. This issue deals with core beliefs that arefundamental in nature and typically resist scientific evidence

S: (pause) Without offending anyone probably not, but I wouldbelieve in that I believe in God, so I don’t know

S: Just the way I was raised with religion

R: And how did you come to hold that point of view? R: How do you think this experience influences your thinking?S: I’ve been raised and taught that was what how we were

created and what happenedS: (Silence)

R: Can you ever know for sure that your position on this issue iscorrect?

R: Just the way you were raised, how did it affect your way ofthinking?

S: I think it affected it a lotS: No R: Can you ever know for sure that your position on this issue is

correct?R: Why not? S: NoS: Well everyone usually thinks they’re right and what they

believe in is correct and I don’t think I’m right over anyoneelse and if that’s what someone believes that they’re rightand they have evidence for like being evolved from apes orsomething then that’s what they think and that’s what I thinkI would say that they were wrong and I was right

R: Why not?S: I don’t know for sure if we were evolved, I believe in

evolution, but I’ve never been told that that’s the way, andthis is why and never been shown something to make mebelieve that’s exactly what happened

(Continued)

ADVANCIN

GREFLECTIV

EJU

DGMENTTHROUGH

SSI

83

Journalof

Research

inScience

Teaching

35 R: So how did you come to hold that point of view? R: How did you come to hold that point of view? These responses indicate a shift from pre-reflective to reflectivethought. The pre-test response simply indicated that s/hewas raised a Christian so that is why s/he believes whats/he does. The post-test response indicates that s/he hasstudied and evaluated evidence from both views. (It isimportant to note that a given stance is not associated withreflective thought; rather it is process by which the studentcame hold that stance.)

S: I was brought up Christian and I really believe that and it isbasically what I base my life on, so it says that in the Bibleso that is why I believe it to be true. And it says that it wascreated, so

S: I have actually researched this before and I have seen bothsides, obviously the scientific side that states that weevolved from lower animal forms in school, cause that’sbasically what they teach and but I have study the scientificside of the religious point of view that says the world wascreated and coming now to have more credibility in thescientific world and people are getting into it andresearching it more

Scenario: Genetic Determination of Alcoholism20 R: How did you come to hold that viewpoint? R: So how did you come to hold that point of view on the topic? In this instance, the student takes the same stance on the issue,

that alcoholism is socially determined. However, herreasoning becomes more robust as s/he offers a scenariothat could be used to test the viewpoints. This represents ashift toward higher quasi-reflective stages

S: Not all people with parents that are alcoholics becomealcoholics, or parents that are alcoholics doesn’t necessarilymean their children are going to be [alcoholics]

S: I’m sure that like you can have a family, and some of themcan live in New York, and others can live in Florida orsomething, and there could be a family with the parents arealcoholic and grew up to be alcoholics, and the other half ofthe family were never shown that a kids, and alcoholism andthey grow up believing you shouldn’t do that

11 R: How is it possible that experts in the field disagree about thissubject?

R: How is it possible that experts in the field disagree about thissubject?

In this case, the student shows a shift from the lowquasi-reflective stages to a higher level of quasi-reflectivethinking. Low-level quasi-reflective thought ischaracterized by a belief that experts are biased, and maymanipulate research to support previously held beliefs. Thehigher level of reasoning utilizes the research as the basisfor forming new beliefs, and for contradicting previouslyheld beliefs

S: I just think that, I don’t really know . . . but just how theywere brought up. But they just might automatically justthink they have the same information they might take itdifferently because they were brought up differently andwere taught different things. Even though they might havethe same facts, they might have different opinions on itbecause of the way they were brought up

S: Because they have contrasting experiments and beliefs orwhat not that they believe and support

Table 2

(Continued)

ID Pretest Key Indicator Posttest Key Indicator Researchers’ Interpretation

84ZEID

LERETAL.

Journalof

Research

inScience

Teaching

(R) What do you think about these statements?

(S) I don’t like them, I think that things are created from a Divine Being, because I don’t think we

can. . .from evolution to the Big Bang Theory . . . can just show up and things are put into place. I

think that we might be similar to apes, but we are not apes, and if we were apes then we wouldn’tbe humans.

(R) How would you go about making a decision about this issue?

(S) Well, personally I am religious, so that affects some of my reasoning, but it—you just have to think

in common sense. We didn’t come from apes, you know. They don’t have a brain like us, or all the

same features—they may have similar features. We are more complex than they are:

(R) Will we ever know for sure which is the better position?

(S) It’s really personal, because if you’re a Christian or you’re really religious, then you are going to

have the really strong knowing that is like God or a divine being, you’re not going to think oh that’s

evolution and that we just evolved.

(R) How did you come to hold that point of view?

(S) Because I am religious so I was taught growing up that a divine like God created Earth and that I

learned about evolution verses creation in Christian school so I’ve learned the pros and cons, and

how you can catch evolutionists lying—not lying, but being proven wrong like all their statementsbeing proved wrong.

And now consider another exchange with a different student:

(R) What do you think of these statements?

(S) It’s absolutely correct that many religions have a creation story and believe a divine being, a

supernatural being . . . and many scientists believe we evolved from lower forms. I personally

believe a mixture of the both, that a divine being did create the universe and the earth and did startlife, and then set up a system that would work in such a way that eventually we have life forms and

then change these into what we became because the differences are so vast, so different, and the

differences of our cognitive capacity, the way our brains work, is so different than others that it

seems something tampered with it between apes and humans.

(R) How did you come to hold that viewpoint? What do you base it on?

(S) Over many years, I grew up listening to the scientists’ view because it seemed to make sense that

slow change could eventually lead to larger change, and then eventually I started looking at the

other viewpoint and accepted what it said, and then I began to examine evolutionary theory moreclosely. Sure it makes sense on a microscale, microevolution makes sense completely because my

dog, a Doberman, was microevolution guided by human hands, so new subspecies can form. But

macroevolution, upon examining that by the many books I picked up, by well known doctors and

scientists, there are loopholes in the evolution story that can not be readily explained, and thatDarwinism doesn’t really work full-out, and the creationist theory makes more sense, and by

taking physics for three years now, you coincidence with the science of physics how lucky we are,

and what a coincidence we just happened to be here, and everything is just so, everything is so niceand neat and perfect that something must have created it.

The above exchanges are significant when viewed in the following comparative perspective: bothstudents were rated as reflective thinkers at the completion of the study on the scenarios of ChemicalAdditives and Genetic Determination of Alcoholism. However, they remained quasi-reflective in theirreasoning in regard to their epistemological positions and concept of justification for knowledge on theReligion and Science issue,where beliefs, in the absence of evidence that is consistentwith highly entrenchedpersonal beliefs, are defended against the viewpoint of personal opinion and dogmatic authority. Hence, thecontextual embeddedness of subject matter would seem to influence the quality of epistemological stances.Our interpretation is consistentwith other researchers who view the strict content separation from structure astoo rigid and not supported by the facts of development (Rest, 1979; Rest, Narvaez, Bebeau, &Thoma, 1999;King & Kitchener, 1994). From a neo-Kohlbergian framework (i.e., a perspective that allows for examiningmediating factors impacting moral reasoning beyond developmentally guided shifts in ‘‘content-free’’structural logic), the influence of social institutions and social knowledge on structural change is recognized;further, this finding supports the results of our previous work that examined how students’ normative



experiences mediated their moral judgments (Zeidler & Schafer, 1984) and epistemological beliefs inargumentation related to SSI (Fowler, Zeidler, & Sadler, in press; Zeidler et al., 2002). Thus, epistemologicalreasoning is not content-free but is influenced by situational (and by extension, educational) contexts.

An additional indicator pivotal in understanding the kinds contextual-based reflective judgment thatstudents learn to make in the realm of SSI may be revealed if we consider more specifically, the nature ofpatterned-reasoning by students in a given context. For example, reflective reasoning (stages 6 and 7) is likelyto develop first in domains in which students have greater interests and are subsequently more familiar withthe complexities of those issues (King&Kitchener, 2002). Consider the following exchange below. Here, theissue under consideration is whether alcoholism is predetermined from a genetic basis or stems fromenvironmental factors:

(R) Could you ever say which is the better position?

(S) No.

(R) Why not?

(S) Uhhh. Simply I’m sure there are people that have had a history of alcoholism in their families that

aren’t alcoholics just as much as there are people who have had numerous history of alcoholism intheir family who are alcoholics and what you have to take into account are these people’s

experiences like their home life . . .You can say that a person whose parents are alcoholics has hada terrible home life, could become an alcoholic, or someone who is predisposed to alcoholism

who’s had a great home life and never becomes an alcoholic so once there’s some form of researchor data that shows definitively that someone who’s predisposed to alcoholism becomes an

alcoholic then I couldn’t say which side is right.

(R) How did you come to hold the point of view that you have?

(S) (pause) . . .Simply for the fact that like some family members that I have are, one or two arealcoholics and my father’s father was and my father’s not an alcoholic so there can’t be a

generalized statement saying simply because someone, or relatives of an alcoholic, that does not

mean that they’re going to be an alcoholic. I mean it’s possible to be predisposed to be an alcoholic

but you have to take into account different experiences, so it’s not one extreme it’s a mix of thetwo.

(R) Can you ever know that your position on this issue is correct?

(S) Umm . . .Without any data or research, no, I can’t know whether my position on this issue is

correct. Simply for the fact that it might be true that I don’t have any data to say that it is true thatall those who are alcoholics are alcoholics because of genetic factors, but on the other hand I

cannot say that another person is an alcoholic simply because of their experiences so . . . I take themoderate viewpoint simply because the two extreme viewpoints are I think that are pretty marginal

as far as what they’re claiming . . . so I think that a moderate viewpoint is probably the safestviewpoint to take simply because it makes the most sense out of the two and also it combines both

factors of the two and makes a credible case for it I think.

In the above exchange, elements of normative reasoning temper this student’s judgment about theimplicit nature versus nurture debate. In his particular case, the student begins the conversation alluding tofamilies in general, but it becomes evident that his reasoning has been impacted by first-hand knowledgeabout the abuse of alcohol by (at least) his paternal grandparent. The viewpoint he adopts strikes a balancedmind-set; given his tacit experience with an issue that is personally relevant, he reaffirms the need foradditional data and research to add credibilityto a given position. This student had progressed from a quasi-reflective stage (5-5-5) to a reflective orientation (6-6-5) at the endof the study. This alcoholism issue revealedthemost consistent emphasis on the need for credible data.While it is important to remember that the final RJscores are derived from a composite of at least 21 questions over three scenarios, this finding adds support tothe notion that the complexities of epistemic reasoning represented in the highest reflective judgment stagesmay be more readily assessable in domains that align with students attitudes, interests, and propensities.

In the final example that follows here, the issue of trying to resolve the discrepancy between reports thatsuggest chemical additives to food canmake themsafer to eat and reports that suggest potential cancer links tothese chemicals is under consideration by a student who has some degree of vested interest in the debate.Consider the nature of the argument as it unfolds during the interview process:

86 ZEIDLER ET AL.


(R) What did you think about these statements?

(S) I felt that, um, like I kind of treated them as fact. I know that there’s still quite a bit of controversy

over artificial sweeteners. In my opinion I think that in normal quantities artificial sweeteners are

healthy and safe to use. It’s only in larger quantities that an artificial sweetener can potentially beharmful. I know in several studies they give rats an artificial sweetener and they’ve been

developing cancer but the doses in which they’re giving them is far higher than the dose that a

human would naturally consume. I feel that, just because of my experience, that artificial

sweeteners are relatively safe as long as they are used in controlled amounts.

(R) And how did you come to hold that point of view?

(S) I know that ummy mom has actually been on my case about it because I lost 30 pounds and a lot of

foods that I substituted for old ones contained artificial sweeteners and I started using a lot more

artificial sweeteners so I became interested in the issue and my mom was very concerned that itcould cause cancer and I looked at some things online and felt as though there were no conclusive

that showed that artificial sweeteners in appropriate amounts could be harmful to you.

(R) Can you ever know for sure that you opinion on this issue is correct?

(S) I guess I can’t know absolutely for certain in my position is correct but I think I’ve done more than

other people have done to try and look up different information on different studies to justify why Ichose to use artificial sweeteners . . . I’m not the scientist. I’m not the one doing the tests. I’m not

the one doing the studies. And even information because it’s printed online doesn’t necessarily

mean that it’s true. I try to look for credible sources that I really can’t know unless I’m the onedoing the study.

(R) Can you say that one opinion is in some way better than the other?

(S) I think it has to just be based on facts. Whoever has the most facts supporting their viewpoint?

Whoever can prepare the most convincing argument. I tend to lead towards that side. In the case ofme and my mom, my mom really hadn’t looked anything up whereas I had, that’s why I felt that I

was right!

(R) How’s it possible that people have such different points of view about this subject?

(S) Just the experience and with . . . just environmental factors. I’m sure my mom probably heard, you

know, when they were first testing artificial sweeteners that they could potentially be harmful andthat stuck in her mind. Whereas I’m going on my own and finding new information. It just depends

on what kind of environment you’re exposed to, what type of information you’re exposed to and

what you just picked up.

(R) How’s it possible that experts in the field disagree about this subject?

(S) I think that everyone has different motivations. Obviously the people who make artificial

sweeteners, they don’t want to . . . so they’re obviously going to do everything they can and

publish studies to support the side that artificial sweeteners aren’t harmful, whereas other

scientists if they have motivation against the artificial sweetener company, I’m not sure, butmaybe they’re looking out for the general welfare.

In this exchange, it is interesting to note the extent to which this student is pressed into evaluatingscience-related knowledge claims to advance her argument (in contrast to her mother’s position). Here she isable to distinguish the underdeveloped assertions of her mother’s position from scientific evidence withrespect to the topic at-hand. She recognized the importance of contextual factors relative to her claim aboutthe safety of chemical additives that reflects what Kolstø (2001) refers to as ‘‘content transcending’’ subjectmatter in the pursuit of SSI. Further, she tempers her position with the realization that one needs to weigh thecredibility of on-line sources as well as the motivation of those conducting scientific research. This studenthad more elements of higher elements of quasi-reflective thinking (stage 5), and to a lesser extent, someelements of reflective thought (stage 6) in her posttest scores. Again, we recognize that the student’s finalscore (5-5-6) is a composite variable over three scenarios, clearly the Chemical Additive Scenario provided arich domain-specific arena to exercise more advanced reasoning where her personal stake in the issue washigh. Instances such as this buttress the findings of past research on SSI where the use of conflicting positionsor anomalous data fostering conceptual shifts in reasoning patterns (Simmons & Zeidler, 2003).

Examining these lines of reasoning reveals three important points. First, in trying to understand thenuanced epistemic reasoning of a student, the interview process provides an excellent means to engagestudents in a manner that provides robust opportunities to evaluate a controversial issue in contrast to ‘‘paper



and pencil’’ assessments only that do not allow for students to create their own meanings—a concern alsoraised byWood,Kitchener, and Jensen (2002). Second, our data also are consistent with the research of others(Bell &Linn, 2002; Linn&His, 2000) that support domain-specific accounts of student learning in science incontrast to developmental views that are more essentialist in nature. It seems clear from our experiencesworking closely with these children for an academic year, and from our data, that there are significantcontextual influences on their thinking. Students often develop a plethora of idiosyncratic views of scientificinquiry and, as Bell and Linn (2002) have pointed out, are not likely to gather them up into coherent structureswithout the systematic challenges posed by the deliberate pedagogical approaches bound up in disciplinaryknowledge that a framework like SSI has to offer. A further presupposition in this regard is that beforeteachers can facilitate the development of coherent views of the NOS, students need to experience science inthe kind of personalized contexts that SSI may provide. Finally, advanced developmental reasoning seemsmore accessible to students in areas that are both controversial and personally or socially relevant to them.Such matters lie close to the heart of SSI.

Quantitative Comparisons of Groups

Comparisons of student responses were made, based on the quantification scheme described in theDesign andMethods Section inwhich levels of an individual student’s reflective judgment are summarized bya three-digit code. This strategy was useful as a basis for examining group trends. The pre- and post-scoringcodes for each student organized by group (i.e., comparison or treatment group) are presented in Table 3.

For statistical analysis consistent with related past research (King &Kitchener, 1994, 2002; Rest, 1979;Rest et al., 1999; Zeidler, 1985; Zeidler & Schafer, 1984), each of the three digit codes was converted to avalue between one and seven by using a weighted average with the primary score contributing 50%, thesecondary digit contributing 30%, and the tertiary digit contributing 20% of the total value. This is consistentwith the notion that an individual’s reflective judgment is best represented as a developmental continuum incontrast to being conceptualized as either ‘‘in’’ one stage or another. Thus, a score of 3-3-5would yield a score

Table 3

Summary of reflective judgment scores by groups

ID Pre Post

Comparison group1 4-4-5 5-5-52 5-5-6 3-4-53 4-4-2 4-4-25 4-4-3 3-3-46 5-5-6 5-6-411 4-4-5 5-5-412 3-3-4 3-3-213 4-3-5 3-3-414 3-3-4 2-2-315 2-2-3 4-4-3

Treatment group20 5-5-5 5-5-621 5-5-5 6-6-522 2-2-3 4-3-524 3-3-5 6-6-525 3-3-4 4-4-326 4-3-5 5-5-427 5-5-6 5-5-628 5-5-6 5-4-630 3-3-4 3-3-333 2-2-2 2-2-335 3-3-3 7-7-637 3-3-4 4-4-538 2-2-4 2-3-4

88 ZEIDLER ET AL.


of 3(0.50)" 3(0.30)" 5(0.20)! 3.40. Table 4 provides the descriptive statistics for pre- and post-results ofeach group.

Since the groups were pre-determined from existing classes, there was no way to ensure that thecomparison and treatment groupswere equivalent at the beginningof the study,which diminished the efficacyof drawing inferences from between group data. We, therefore, took the more conservative approach ofexamining within group differences for both the comparison and treatment groups. The descriptiveinformation presented in Table 4 revealed some trends that deserved further study. It was apparent thatthere was virtually no difference between the pre-test and post-test scores for the comparison group, whilethe scores appeared to increase from the pre-test to post-test administrations for the treatment group.

As we consider this research to be exploratory in nature (no previous research reports using SSI as thecontext to teach a content-rich course over an academic year), an alpha of 0.10 was selected in order to detectpossible trends that might otherwise be overlooked in such a complex social setting over such a prolongedperiod of time. Given the small sample sizes and unequal n’s (an unavoidable necessity stemming from thefocused sample of the PRJI and real-world classroom logistics), a non-parametric Wilcoxon test wasperformed. The ranked mean scores revealed a significant difference for the SSI treatment group (p! 0.09,alpha #0.10), but no discernable differences (p! 0.27, alpha #0.10) for the comparison group.

Effect sizewas calculated for the statistically significant finding between the pre-test and post-test scoresin reflective judgment in the treatment group. Using Cohen’s d (1988) as a guide, an effect size of 0.74 wascalculated indicating that the mean of the treatment group’s post-test responses was at the 78th percentile ofthe pre-test scores. The corollary to this is that the pre- and post-test distributions of scores revealed a non-overlapping area of 45%. This represents a moderately large effect size, adding further support that the SSItreatment effects, in all likelihood, had a meaningful and tangible impact with respect to providingopportunities consistentwith the development of reflective judgment. Values of thatmagnitude in this contextare not likely to be attributed to spurious factors.

Discussion

Studying Reflective Reasoning in Classroom Contexts

We chose to situate our research in four, yearlong classes because of the current gap in the literature baseregarding extended SSI treatments. While there is still much to learn, the effects of short-term SSIinterventions have been explored particularly with respect to outcomes such as student learning of sciencecontent (Barab, Sadler, Heiselt, Hickey, & Zuiker, 2007; Keselman, Kaufman, Kramer, & Patel, 2007;Pedretti, 1999), NOS (Khishfe & Lederman, 2006; Walker & Zeidler, 2007), and argumentation (McNeill,Lizotte, Krajcik, & Marx, 2006; Zohar & Nemet, 2002). However, to our knowledge, effects of extendedinterventions (on the order of an academic year or more), informed by the SSI framework discussed earlier,have not been researched. The only way to do this kind of research is long-term partnering with teachers andextended work in actual school settings. This kind of collaboration necessarily entails dealing with therealities of modern school contexts including issues that challenge traditional notions of social scienceresearch such as applying treatments to intact classes with a degree of student mobility that often leads tosample mortality. In our case, we began the study with 40 participants but were only able to collect completedata setswith a focused sample of 23 participants.Obviously,we are cautiously optimisticwith our inferentialclaims and appropriately adopted a non-parametric statistical test for this part of the study.

Table 4

Descriptive statistics for the PRJI results

Mean n Std. Dev. Std. Error Mean

Comparison pre 3.87 10 0.914 0.289Comparison post 3.74 10 0.968 0.306Treatment pre 3.62 13 1.18 0.325Treatment post 4.48 13 1.34 0.372



Another major challenge for the research we conducted was the nature of the construct underinvestigation: reflective judgment. Reflective judgment is a complex, difficult-to-assess construct, whichcaptures epistemological development. It cannot be measured with traditional exams that provide an‘‘objective’’ outcome score. The PRJI provides themost valid assessment of reflective judgment and has beendemonstrated to be an effective strategy for revealing subtle developmental shifts in epistemologicalperspectives (King & Kitchener, 2004). However, administration of the PRJI is time-consuming for theinterviewer and subject, and scoring is very labor-intensive requiring full transcription of the interviews anddetailed analyses of the transcripts by trained investigators. Coordinating, conducting, and analyzingextensive interviews with high school students, who had to be removed from class to participate, required ahuge effort.

We recognize the relatively small sample size limits generalizability of the findings; however, we feelthat our focus on an important, but hard-to-measure outcomevariable, reflective judgment, in the context of ayearlong intervention study justifies the research design employed. As with most investigations that span anacademic year, there exists a trade-off between generalizability (and possibly transferability) and theopportunity to conduct research with intact but realistic classroom settings. Nonetheless, we are encouragedby the resultswhen the quantitative tests are examined in concert with the qualitative data. The findings of thisresearch identify an important result in the context explored. Students who experience sustainedopportunities to negotiate scientific content in the context of SSI showed evidence of epistemologicaldevelopment, whereas a comparison group of students who experienced amore traditional science educationexperience without any focus on SSI showed no such evidence.

Advocates of socioscientific issue instruction and, in previous years, science-technology-societyeducation, have long argued that contemporary social issues with conceptual links to science can serve as a‘‘hook’’ to motivate students’ learning of science content (Cajas, 1999; Myers, 1996; Sadler, Amirshokoohi,Kazempour, & Allspaw, 2006; Solbes &Vilches, 1997). Student mastery of science content knowledge is nodoubt important, and as a part of the broader project in which this report is situated, we compared sciencecontent learning between the treatment and comparison groups.We found that the students, who experiencedthe SSI-driven curriculum, learned more basic anatomy and physiology concepts than their peers in thecomparison group (Zeidler, Sadler, Applebaum, Callahan,&Amiri, 2005). This finding certainly bolsters theSSImovement’s agenda, butwe actually see the reflective judgment findings documented in the current reportas an even more significant result. The current report provides evidence of SSI instruction affecting theepistemological development of students in the span of a single academic year. An academic year is quitelengthy for a classroom-based research project, but it represents a very limited time span from adevelopmental perspective. Documented student growth in reflective judgment in the context of scienceeducation is a very encouraging result especially when the aims of science education are framed withcontemporary notions of scientific literacy. If the aims of science education entail improved understandingsof the epistemological foundations of science, abilities to consider scientific evidence in the context of real-world problems, and engage in sophisticated discourse concerning matters of personal and public interest(Driver et al., 2000; NRC, 1996; Zeidler & Keefer, 2003), then an approach to science teaching whichpromotes development of reflective judgment is critically important. The findings reported herein positioninstruction guided by the SSI framework as just such an approach. Accordingly, we advance a central claimthat the justification of moral decisions in general, and decisions related to SSI in particular, must rely ondiscussion, rhetoric and argument embedded in the contextual normativity of different values related to thetopics at-hand.

Significant Aspects of SSI Instruction

The importance of selecting topics that are contemporary and personally relevant cannot beoveremphasized in the creation of classroom interest in learning science (Rivet & Krajcik, 2008). Forexample, in the activity regarding the safety of marijuana, students were provided an opportunity to discoverthe drug’s active ingredients and its effects on specific areas of the brain. Further, the involvement inclassroom debates and discussion encouraged students to confront core beliefs about recreational drug useand their personal threshold of danger. In this particular activity, students were actively involved in readingand evaluating conflicting evidence from credible sources and negotiating their conclusions within and

90 ZEIDLER ET AL.


against other groups of students. This type of activity also provides opportunities to observe the criteriastudents use in their selection of ‘‘credible’’ evidence. The importance of our SSI approach is echoed byPouliot (2008), who writes:

It is now a commonplace in science education that the study of socioscientific issues by students

constitutes a prime avenue for fostering scientific literacy of a kind that will prompt young people tofamiliarize themselves with science in action, to develop their capacity for evaluating the information

made available to hem on a daily basis, to make decisions concerning controversial sociotechnical

issues, and to take part in debates and discussions on sociotechnical controversies of concern to them.

(p. 545)

We obviously concur with this depiction of the practical utility of SSI, and can readily apply this idea toour own work. For example, through several phases of this classroom experience, students were required towork individually, in small groups, and interact as a whole class. Their challenges included reading articleswith conflicting evidence from varied sources, identifying important data and arguments, ranking theimportance of evidence, form group consensus positions, debate of positions, and serving as editors for amock ‘‘scientific research journal’’ with the goal of evaluating other groups’ presentation of positions andevidence. It is these types of experiences that we believe likely affected the developmental shifts inepistemological perspectives documented in this report.

Reflective Judgment and Nature of Science in the Context of SSI

The parallels (previously noted) between more advanced stages of reflective judgment and moresophisticated views ofNOS are important to recognize inasmuch as both require epistemological frameworksthat conceptualize and justify knowledgevia a process of inquiry, are based on data-driven evidence, allow forthe probabilistic nature of data, and possess an openness to reevaluation. Abd-El-Khalick (2003) seemed tohave anticipated the possible reciprocal nature of epistemological views centered on SSI, critical discourse,and NOS. Our research helps to inform the nature of that reciprocal relationship, though more work needs tobe done that directly examine NOS orientations under an SSI framework.

However, an important theoretical (and to some extent, pedagogical) difference between NOS andreflective judgment conceptualizations is that the RJM is guided by a cognitive-developmental progression.An equivalent model has not been advanced in NOS research; although, current work is beginning to addressdevelopmental processes for NOS ideas (Akerson & Hanuscin, 2007; Khishfe, 2008). Zeidler and Lewis(2003) and Lederman (2003) proposed a theoretical template that connected developmental attributes ofcognitive stages, moral reasoning patterns, and sociomoral discourse, with indicators of NOS (see pages296–305). For example,wemay consider that a studentwith nascent formal reasoning structures (a necessarybut not sufficient condition for post-conventional moral reasoning (Zeidler, 1985)), may have the ability toengage in sociomoral discourse that reflects participation of all discussants toward optimal solutions, andsupport the inclusion of scientifically based evidence while recognizing the tentative and cultural-embeddedness of that evidence. While it may be the case that students at less mature stages of developmentmay be able to be guided by similar NOS constructs, ourworkwith reflective judgment suggests that there aredevelopmental limitations with respect to applying NOS at higher stages of reflective judgment. In otherwords, certain NOS attributes may be sufficient for quasi-reflective thinkers, but the same NOS attributes, atleast in their current developmental form, may not be sufficient in terms of their ability to guide and informjudgments about SSI, for reflective thinkers. For example, it is one thing to understand that scientificknowledge is impacted by cultural factors and is tentative in nature (sufficient of quasi reflective thinkers), butit is quite another to know how to synthesize empirically variant data, theories or philosophical conjecturesinto coherent, robust epistemological world orientations, all governed by the highest stages of reflectivejudgment. NOS instruction, in isolation or in the absence of the kinds of sociomoral discourse and contextsthat SSI has to offer, may not tap such developmental potentialities of students. This, of course, is a claim thatwarrants further empirical investigation.

An advantage of using SSI to frame science instruction seems to be that SSI instruction may have theadded benefit of embedding NOS into a scientific context that is, de facto, theory laden and driven by data, aswell as socially and culturally embedded. The context of the SSI-driven curriculum provides an anchor on



which reason may be exercised. Reasoning about ill-structured problems requires that students care toexercise reasoning. If structural development shifts in epistemological orientations are to occur, thenreasonable opportunities need to be created for the type of social interaction necessary to advance solutions toissues that need to be sensitive to new evidence, perspectives or modes of inquiry.

SSI and Character Education

Another benefit of a SSI approach is that opportunities are afforded for the exploration of character. Thatcharacter development should be the purview of science education is fathomable when understood in an SSIcontext. As Berkowitz (1998) notes, ‘‘Aristotle argued that the central virtue that provided coherence to theother virtues was practical reason, essentially the ability to reason well’’ (p. 19). To the extent that a generalgoal for science educators is for their students to employ reflective judgment on SSI as they evaluate evidence,to ‘‘reason well’’ as it were, and to exercise this skill most effectively with due consideration to a sense ofsocial justice and personal caring, the pursuit of character certainly seems like a tenable part of scienceeducation (Zeidler & Sadler, 2008; Zeidler, Sadler, & Applebaum, 2007).

Our claim that SSImayprovide the educational context for the formation of character is supported otherswho have explored the connection between reflective judgment and character development. For example,King and Kitchener (1994) have advanced the argument that epistemological knowledge and its justificationmay shape a student’s internalization and representation of moral problems and influence identity formation.To the extent that reflective thinking tempers one’s ability to form moral judgments, the pursuit of reflectivereasoning skills becomes educationally defensible. As King and Kitchener (1994) write: ‘‘. . .An educationalinvestment in teaching students to think more reflectively also sets a foundation that enables them to makewise ethical and moral choices’’ (p. 221).

Our approach would also be consistent with a humanistic perspective of school science. Aikenhead(2006) advances the claim that a humanistic perspective to science education refers to ‘‘. . .values, the NOS,the social aspects of science, the culture of science, and the human character of science. . .’’ (p. 2). Included inAikenhead’s notion is that citizenship preparation for everyday life and moral reasoning integrated withscientific reasoning necessitates the recognition of individual, group and cultural values. Again, we endorsethis perspective and propose that an SSI perspective is consistent with the humanistic perspective (Zeidler,2007).

Summary

This research has shown that the use of SSI can promote reflective judgment, which is, in many ways,analogous to the development of conceptual understanding of NOS. The quasi- and reflective stages dependon the use of evidence rather than uncritical reliance on authority, with the highest stages of reflectivejudgment involving true analysis, synthesis and evaluation of data and claims. The ability to considermultipleviewpoints, and integrate various strands of evidence into an informed data-driven position, is not only animportant scientific skill, but also a fundamental life-skill all students should be able to utilize. From themedia stories to political debates, a barrage of competing claims, some worthy of serious consideration andothers worthy of serious contempt, inundates people. Through the use of SSI, students mirror the normativeactivity of scientific argumentation by evaluating the knowledge claims put forward by others. As Driver,Newton, and Osborne remind us:

Observation and experiment are not the bedrock on which science is built, but rather they are the

handmaidens to the rational activity of generating arguments in support of knowledge claims. But it is

on the basis of the strength of the arguments (and their supporting data) that scientists judge competing

knowledge claims and work out whether to accept or reject them (2000, p. 297).

A guiding premise of our work with SSI is students themselveswant to be able to reason well. Given theopportunity, students can and do develop their ability to make increasingly robust reasoned judgments usingevidence to support their claims. The development of a full-year SSI curriculum has been shown to bepromising in promoting these desired outcomes. In the context of this study, the SSI framework hashighlighted the importance of utilizing argumentation and evidence-based reasoning in a manner that canfoster reflective judgment in students.

92 ZEIDLER ET AL.


Appendix A: Summary of Reflective Judgment Stages (Adapted from King & Kitchener, 1994; 2002)

Epistemiccognition Stages

Majorcharacteristic

Role ofauthority

Role ofevidence

View ofknowledge

Concept ofjustification

Pre-reflective Stage 1 Belief isconcrete andsingle-category(there are noalternatives)

Authority andobservationare thesource ofknowledge

Disconfirmingevidence isdenied.Belief doesnot dependon evidence

Knowledge isabsolute andconcrete

Beliefs donot needjustification

Stage 2 There is a truereality, but noteveryoneknows it

Authoritiesknow thetruth, thosewhodisagreeare wrong

Evidence is notneeded toconfirmbelief, andcannot beused todisconfirmbelief

Knowledge isabsolute, butnot apparentto everyoneat every time

Justification byagreementwith authorityfigure

Stage 3 Belief thatauthorities maynot know thetruth, but willsomeday

Authority is thesource ofrightanswers, butthere is noway to justifyclaims inareas ofuncertainty

Evidence mustbe concreteand lead to asingleanswer

Knowledge iscertain inareas that areknown, ortemporarilyuncertain inareas that areunknown

Right answersareprovided byauthority,other areasare unclearand defendedby personalopinion

Quasi-reflective Stage 4 Understanding thatone cannotknow withcertainty

Authority isoften biased,they fit theevidence totheir beliefs

Evidence isused toconfirmsubject’sprior beliefs

Knowledge isuncertain,there isalways someambiguity

Justificationprovided byevidence thatsupportsprior belief

Stage 5 Understanding thatpeople cannotknow directly,but can withina context basedon subjectiveinterpretationof evidence

Authorities areseen asexperts intheir field,perhapslimited bytheirperspective

Evidence can becomparedfor differentbeliefs, butcannotintegrate theevidence

Knowledge iscontextualbecause it isfilteredthrough aperson’sperspective

Beliefs arejustified byevidence as itpertains to aparticularcontext

Reflective Stage 6 Knowing is aprocess thatrequires actionon part of thelistener

Authorities areinvolved inconstructingsolutions

Plausibility ofevidence andargumentcan be usedto basebeliefs forself

Knowledge isbased oninformationfrom avariety ofsources

Justificationprovided bycomparingevidence andopinion,utility ofsolution

Stage 7 Interpretations ofevidence andopinion can besynthesizedinto justifiableconjectures

Subject isinvolved inconstructingknowledge,and is awarethatknowledgechanges inlight of newevidence

Evidenceprovideslogicalsolutions toproblems,but maychange inface of newor betterevidence

Knowledge isconstructedby criticalinquiry andevaluatingevidence

Beliefs arejustified onthe basis ofprobability,we cannotknow forsure, butwealth ofevidencesupportsview



Appendix B: Pedagogical Details Using Marijuana as a Controversial SSI.

Identify Identify

Describe/Discuss

Introduce

Introduce

Introduce

Leads to Leads to

Identify

Considerationsinclude

Discuss / clarify

Considerationsinclude

Introduction

Marijuana as a controversial topic of

science

Student core beliefs about

marijuana safety

Studentexperience and

knowledge

Invertigation of relevant SSI characteristics of topic

Phase I Activity

Group Students 3-4Read 5 articles representing

mixed perspectives

Phase II Activity

Provide group ranking of evidence and sourcesGroup presentation of findings

Phase III Activity

Groups review evidence selected by peer groupsDecision to accept or reject evidence

Reviewers provide rationale for decisions

Class discussion: Anatomy & Physiology needed to resolve SSI

Class discussion identifying conflicting evidence and consensus

opinions

Medical/healthsafety issues

Head and neck cancer

Investigatedescription of

cancercharacteristics

Nervoussystem / brain

pathology

Nervous system / brain

anatomy & physiology

Anatomy & physiology affected

Medical / Health Issues

Potential Pathology

Medical Benefits

Level of Participation

IndividualActivity within

groups

ClassDiscussion

Small Group Activity

GroupPresentations

followed by Whole Class Dicscussion

94 ZEIDLER ET AL.


Appendix C: Pedagogical Relationships Between Teacher and Students’ SSI Discourse.

Draws upon

Draws upon Draws upon

Facilitates

InvestigatePersonally Relavant

Use & Develop

Use & Develop

Use & Develop

Develop

Use & Develop

Use & Develop

Use & Develop

Uses appropriate Embedded in

Teacher

MultidisciplinaryLife Experiences

Subject Matter Knowledge

PedagogicalContent

Knowledge

SocioscientificIssues

ScientificContent

Scientific & Social

Contexts

Students

InformalReasoning

SubjectMatter

Knowledge

Decision-Making

Characterand

Reflective Judgment

Life Experiences

MoralReasoning

Argumentation



Appendix D: Interview Protocol, Reflective Judgment Issues and InterviewProbe Questions. (Adapted from King & Kitchener, 1994,

pp. 100–103; 260)

Interview Protocol Methodology

‘‘During this session, wewill be talking about several issues that are of general concern and about whichmost people are at least vaguely familiar. I am not concerned with howmuch information you have about anyissue, but how you think about them. In order to standardize what we talk about, I will be asking the sameseries of questions for each of the three issues; I am not repeating the questions because I am looking for aparticular answer. For each issue, I will read a statement aloudwhile you follow along on a card. After I finishreading the statement, I’ll give you a minute or so to think about the issue and then we will talk about it. Arethere any questions before we begin?’’

(Inform participant of the fact that the interviewwill be tape-recorded. Give the participant a copy of thestory to read to him or herself as you read it aloud.)

Reflective Judgment Issues

Chemical Additives Issue. There have been frequent reports about the relationship between chemicalthat are added to foods and the safety of these foods. Some studies indicate that such chemical can causecancer, making these foods unsafe to eat. Other studies, however, show that chemical additives are notharmful, and actually make the foods containing them safer to eat.

Religion and Science Issue. Many religions of the world have creation stories. These stories suggestthat a divine being created the earth and its people. Scientists claim, however, that people evolved from loweranimal forms (some of which were similar to apes) into the human forms known today.

Alcoholism Issue. Some researchers contend that alcoholism is due, at least in part, to genetic factors.They often refer to results from a number of family studies to support this contention. Other researchers,however, do not think that alcoholism is in any way inherited. They claim that alcoholism is psychologicallydetermined. They also claim that the reason that several members of the same family often suffer fromalcoholism is due to the fact that they share common family experiences, socioeconomic status, oremployment.

Field Caveat 1: If the response to a given probe question (below) is incomplete, ambiguous, orcontradictory to earlier statements, ask for clarification or elaboration by asking such questions as:

$ ‘‘What do you mean by ‘the most reasonable’ explanation?’’

$ ‘‘What’s the difference between ‘knowing for sure’ and ‘being fairly certain’?’’

$ ‘‘How did that experience change your thinking about [x]?’’

Field Caveat 2: If the participant replies that experts disagree because they had access to differentinformation, or held different personal beliefs, then ask:

$ ‘‘What if they both had access to the same information?’’

$ ‘‘What if they both believed in God [or were both atheists] and still held different views about

whether or not evolution occurred?’’

96 ZEIDLER ET AL.


Reflective Judgment Interview Standard Probe Questions

References

Abd-El-Khalick, F. (2003). Socioscientific issues in pre-college science classrooms. In D.L. Zeidler(Ed.), The role of moral reasoning and discourse on socioscientific issues in science education (pp. 41–61).Dordrecht, Netherlands: Kluwer Academic Press.

Akerson, V.L., &Hanuscin, D.L. (2007). Teaching nature of science through inquiry: Results of a 3-yearprofessional development program. Journal of Research in Science Teaching, 44, 653–680.

Aikenhead, G.S. (2006). Science education for everyday life: Evidence-based practice. New York:Teachers College Press.

Barab, S.A., Sadler, T.D., Heiselt, C., Hickey, D.T., & Zuiker, S. (2007). Relatingnarrative, inquiry, andinscriptions: Supporting consequential play. Journal of Science Education and Technology, 16, 59–82.

Baxter Magolda, M.B. (1999). Creating contexts for learning and self-Author(s)ship: Constructive-developmental pedagogy. Nashville, TN: Vanderbilt University Press.

Bell, P., & Linn, M.C. (2002). Beliefs about science: How does science instruction contribute? In B.K.Hofer & P.R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge andknowing (pp. 321–346). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Probe Question Purpose

1. What do you think about these statements? (Note: If noparticular point of view is endorsed, ask: 1a) Could youever say which was the better position? How?Why not?How would you go about making a decision about thisissue? Will we ever know for sure which is the betterposition? How/Why not?

To allow participant to share an initial reaction to theproblem presented. Most state which point of view iscloser to their own

2. How did you come to hold that point of view? To find out how the respondent arrived at the point of view,and whether and how it has evolved from otherpositions on the issue

3. On what do you base that point of view? To find out about the basis of the respondent’s point ofview, such as a personal evaluation of the data,consistency with an expert’s point of view, or a specificexperience. This provides information about therespondent’s concept of justification

4. Can you ever know for sure that your position on thisissue is correct? How or why not?

To find out about assumptions concerning the certainty ofknowledge (e.g., whether issues like this can be knownabsolutely and what the respondent would do in orderto increase the certainty, or why that would not bepossible)

5. When two people differ about matters such as this, is itthe case that one opinion is right and one is wrong?

Assesses the adequacy of alternative interpretations; to seeif dichotomous either/or view of the issue(characteristic of the early stages) is held; to allow theparticipant to give criteria by which she or he evaluatesthe adequacy of arguments (information that helpsdifferentiate high-from middle-level stage responses)

If yes, what do you mean by ‘‘right’’?If no, can you say that one opinion is in some way betterthan the other? What do you mean by ‘‘better’’?

6. How is it possible that people have such different pointsof view about this subject?

To elicit comments about the respondent’s understandingof differences in perspectives and opinions (what theyare based on and why there is such diversity of opinionabout the issue)

7. How is it possible that experts in the field disagree aboutthis subject?

To elicit respondent’s understanding of how he or she usesthe point of view of an expert or authority in makingdecisions about controversial issues (such as whetherexperts’ views are weighted more heavily than others’views, and why or why not)



Berkowitz, M.W. (1997). The complete moral person: Anatomy and formation. In J.M. DuBois (Ed.),Moral issues in psychology: Personalist contributions to selected problems (pp. 11–41). New York:University Press of America, Inc.

Berkowitz, M.W. (1998). Finding common ground to study and implement character education:Integrating structure and content in moral education. Journal of Research in Education, 8(1), 3–8.

Berkowitz, M.W., Oser, F., & Althof, W. (1987). The development of sociomoral discourse. In W.M.Kurtines & J.L. Gewirtz (Eds.), Moral development through social interaction (pp. 322–352). New York:Wiley.

Broughton, J.M. (1978). Development of concepts of self, mind, reality, and knowledge. In W. Damon(Ed.), Social cognition: New directions for child development (Vol. 1, pp. 75–100). San Francisco: Jossey-Bass.

Cajas, F. (1999). Public understanding of science: Using technology to enhance school science ineveryday life. International Journal of Science Education, 21, 765–773.

Chinn, C.A., &Brewer,W.F. (1993). The role of anomalous data in knowledge acquisition: a theoreticalframework and implications for science instruction. Review of Educational Research, 63(1), 1–49.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. (2nd ed.). New Jersey: LawrenceErlbaum.

Cummings, B. (2001). Human Anatomy and Physiology, 5th edition. Marieb, Elaine N.: PearsonEducation, Inc.

Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young people’s images of science. Bristol, PA: OpenUniversity Press.

Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation inclassrooms. Science Education, 84(3), 287–312.

Fischer, K.W. (1980). A theory of cognitive development: the control and construction of hierarchy ofskills. Psychological Review, 87(6), 477–531.

Fowler, S.R., Zeidler, D.L., & Sadler, T.D. (in press) Moral sensitivity in the context ofsocioscientific issues in high school science students. International Journal of Science Education.

Harding, P., & Hare, W. (2000). Portraying science accurately in classrooms: Emphasizing open-mindedness rather than relativism. Journal of Research in Science Teaching, 37, 225–236.

Hofer, B., & Pintrich, P. (1997). The development of epistemological theories: Beliefs about knowledgeand knowing and their relation to learning. Review of Educational Research, 67, 88–140.

Irez, S. (2006). Arewe prepared?: An assessment of preservice science teacher educators’ beliefs aboutnature of science. Science Education, 90, 1113–1143.

Keefer, M.W. (2003). Moral reasoning and case-based approaches to ethical instruction in science. InD.L. Zeidler (Ed.), The role of moral reasoning and discourse on socioscientific issues in science education(pp. 241–259). Dordrecht, Netherlands: Kluwer Academic Press.

Kegan, R. (1994). In over our heads: The mental demands of modern life. Cambridge, MA: HarvardUniversity Press.

Keselman, A., Kaufman, D.R., Kramer, S., & Patel, V.L. (2007). Fostering conceptual change andcritical reasoning about HIVand AIDS. Journal of Research in Science Teaching, 44, 844–863.

Khishfe, R. (2008). The development of seventh graders’views of nature of science. Journal of Researchin Science Teaching, 45, 470–496.

Khishfe, R., & Lederman, N.G. (2006). Teaching nature of science within a controversial topic:Integrated versus nonintegrated. Journal of Research in Science Teaching, 43, 377–394.

King, P.M., & Baxter Magolda, M.B. (1996). A developmental perspective on learning. Journal ofCollege Student Development, 37, 163–173.

King, P.M., & Kitchener, K.S. (2002). The reflective judgment model: Twenty years of researchon epistemic cognition. In B.K. Hofer & P.R. Pintrich (Eds.), Personal epistemology: Thepsychology of beliefs about knowledge and knowing (pp. 37–61). Mahwah, NJ: Lawrence ErlbaumAssociates, Inc.

King, P.M., & Kitchener, K.S. (2004). Reflective judgment: Theory and research on the development ofepistemic assumptions through adulthood. Educational Psychologist, 39(1), 5–18.

98 ZEIDLER ET AL.


King, P.M., & Kitchener, K.S. (1994). Developing reflective judgment: Understanding and promotingintellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass.

Kitchener, K.S. (1983). Educational goals and reflective thinking. Educational Forum, 48(1), 75–95.Kitchener, K.S., King, P.M., Wood, P.K., & Davison, M.L. (1989). Sequentiality and consistency in the

development of Reflective Judgment: A six-year longitudinal study. Journal of Applied DevelopmentalPsychology, 10, 73–95.

Kolstø, S.D. (2001). ‘To trust or not to trust,. . .’-pupils’ ways of judging information encountered in asocio-scientific issue. International Journal of Science Education, 23, 877–901.

Kolstø, S.D. (2006). Patterns in students’ argumentation confronted with a risk-focused socio-scientificissue. International Journal of Science Education, 28(14), 1689–1716.

Kuhn, D. (1993). Science as argument: Implications for teaching and learning scientific thinking.Science Education, 77(3), 319–337.

Lederman, N.G. (2003) Personal communication.Lederman, N.G. (2007). Nature of science: Past, present, and future. In S.K. Abell & N.G. Lederman

(Eds.), Handbook of research on science education (pp. 831–880). Mahwah, NJ: Lawrence ErlbaumAssociates.

Lederman, N.G., & Abd-El-Khalick, F. (1998). Avoiding de-natured science: Activities that promoteunderstandings of the nature of science. InW.F.McComas (Ed.), The nature of science and science education:Rationales and Strategies (pp. 83–126). Dordrecht, Netherlands: Kluwer.

Levinson, R. (2006). Towards a theoretical framework for teaching controversial socio-scientific issues.International Journal of Science Education, 28(10), 1201–1224.

Lin, J.Y. (2007). Responses to anomalous data obtained from repeatable experiments in the laboratory.Journal of Research in Science Teaching, 44(3), 506–528.

Linn, M.C., & His, S. (2000). Computers, teachers, peers: Science learning partners. Mahwah, NJ:Erlbaum.

McComas,W.F., Clough,M.P.,&Almazroa,H. (1998). The role and character of the nature of science inscience education. InW.F.McComas (Ed.), The nature of science in science education (pp. 3–39).Dordrecht,Netherlands: Kluwer Academic Publishers.

McNeill, K.L., Lizotte, D.J., Krajcik, J., & Marx, R.W. (2006). Supporting students’ construction ofscientific explanations by fading scaffolds in instructional materials. Journal of the Learning Sciences, 15(2),153–191.

Myers, L. (1996). Mastery of basic concepts. In R.E. Yager (Ed.), Science/Technology/Society asreform in science education (pp. 53–58). Albany, NY: State University of New York Press.

National Research Council. (1996). National science education standards. Washington, DC: NationalAcademy Press.

Nucci, L.P. (2001). Education in the moral domain. Cambridge: Cambridge University Press.Pedretti, E. (1999). Decision making and STS education: Exploring scientific knowledge and social

responsibility in schools and science centers through an issues-based approach. School Science andMathematics, 99, 174–181.

Pedretti, E. (2003). Teaching science, technology, society and environment (STSE) education:Preservice teachers’ philosophical and pedagogical landscapes. In D.L. Zeidler (Ed.), The role of moralreasoning on socioscientific issues and discourse in science education (pp. 219–239). Dordrecht,Netherlands: Kluwer Academic Press.

Perry,W.G. (1970). Forms of Intellectual and ethical development in the college years: A scheme. Troy,Mo.: Holt, Rinehart & Winston.

Piaget, J., & Inhelder, B. (1969). The psychology of the child. New York: Basic Books.Pouliot, C. (2008). Students’ inventory of social actors concerned by the controversy surrounding

cellular telephones: A case study. Science Education, 92(3), 543–559.Ratcliffe, M. (1997). Pupil decision-making about socioscientific issues within the science curriculum.

International Journal of Science Education, 19(2), 167–182.Ratcliffe, M., Harris, R., & McWhirter, J. (2004). Teaching ethical aspects of science—Is cross-

curricular collaboration the answer? School Science Review, 86, 39–44.



Rest, J. (1979). Development in judgingmoral issues.Minneapolis,MN:University ofMinnesota Press.Rest, J., Narvaez, D., Bebeau, M.J., & Thoma, S.J. (1999). Postconventional moral thinking: A

neo-Kohlbergian approach. New Jersey: Lawrence Erlbaum Associates.Rivet, A.E., & Krajcik, J.S. (2008). Contextualizing instruction: Leveraging students’ prior knowledge

and experience to foster understanding of middle school science. Journal of Research in Science Teaching,45, 53–78.

Roth, K.J. (1990). Developing meaningful conceptual understanding in science. In B.F. Jones & L. Idol(Eds.), Dimensions of thinking and cognitive instruction (pp. 139–175). Hillsdale, NJ: Erlbaum.

Sadler, T.D. (2004). Informal reasoning regarding socioscientific issues: A critical review of research.Journal of Research in Science Teaching, 41(5), 513–536.

Sadler, T.D. (2006). Promoting discourse and argument in science teacher education. Journal of ScienceTeacher Education, 17(4), 323–346.

Sadler, T.D.,Amirshokoohi,A.,Kazempour,M.,&Allspaw,K. (2006). Socioscience and ethics in scienceclassrooms: Teacher perspectives and strategies. Journal of Research in Science Teaching, 43, 353–376.

Sadler, T.D., & Zeidler, D.L. (2004). The morality of socioscientific issues: Construal and resolution ofgenetic engineering dilemmas. Science Education, 88, 4–27.

Sadler, T.D., & Zeidler, D.L. (2005). The significance of content knowledge for informal reasoningregarding socioscientific issues: Applying Genetics knowledge to genetic engineering issues. ScienceEducation, 89(1), 71–93.

Simmons, M.L., & Zeidler, D.L. (2003). Beliefs in the nature of science and responses to socioscientificissues. In D.L. Zeidler (Ed.), The role of moral reasoning and discourse on socioscientific issues in scienceeducation (pp. 81–94). Netherlands: Kluwer Academic Press.

Solbes, J., & Vilches, A. (1997). STS interactions and the teaching of physics and chemistry. ScienceEducation, 81, 377–386.

Turiel, E. (1998). The development ofmorality. InW.Damon&N. Eisenberg (Eds.), Handbook of childpsychology. (pp. 863–932.) New York: John Wiley & Sons, Inc.

Walker, K.A., & Zeidler, D.L. (2007). Promoting discourse about socioscientific issues throughscaffolded inquiry. International Journal of Science Education, 29(11), 1387–1410.

Wood, P., Kitchener, K., & Jensen, L. (2002). Considerations in the design and evaluation of a paper-and-pencil measure of epistemic cognition. In B.K. Hofer&P.R. Pintrich (Eds.), Personal epistemology: Thepsychology of beliefs about knowledge and knowing (pp. 277–294). Mahwah, NJ: Lawrence ErlbaumAssociates, Inc.

Zeidler,D.L. (1985).Hierarchical relationships among formal cognitive structures and their relationshipto principled moral reasoning. Journal of Research in Science Teaching, 22(5), 461–471.

Zeidler, D.L. (2003). The role of moral reasoning on socioscientific issues and discourse in scienceeducation. Dordrecht, Netherlands: Kluwer Academic Press.

Zeidler, D.L. (2007). An inclusive view of scientific literacy: Core issues and future directions. PaperPresented at: Promoting scientific literacy: Science Education Research and Practice in Transaction—LSLSymposium, May, Uppsala University, Uppsala, Sweden.

Zeidler, D.L.,&Keefer,M. (2003). The role ofmoral reasoning and the status of socioscientific issues inscience education: Philosophical, psychological and pedagogical considerations. In D.L. Zeidler (Ed.), Therole of moral reasoning and discourse on socioscientific issues in science education (pp. 7–38). Dordrecht,Netherlands: Kluwer Academic Press.

Zeidler, D.L., & Lewis, J. (2003). Unifying themes in moral reasoning on socioscientific issues anddiscourse. InD.L. Zeidler (Ed.), The role ofmoral reasoning and discourse on socioscientific issues in scienceeducation (pp. 289–306). Dordrecht, Netherlands: Kluwer Academic Press.

Zeidler, D.L., & Sadler, T.D. (2008). The role of moral reasoning in argumentation: Conscience,character and care. In S. Erduran&M.Pilar Jimenez-Aleixandre (Eds.), Argumentation in science education:Perspectives from classroom-based research (pp. 201–216). New York: Springer Press.

Zeidler, D.L., Sadler, T.D., & Applebaum, S. (2007, January). Character, Critique and Controversy inScience Teacher Education. Paper Presented at the Association for Science Teacher Education, ClearwaterBeach, Florida.

100 ZEIDLER ET AL.


Zeidler, D.L., Sadler, T.D., Applebaum, S., Callahan, B., & Amiri, L. (2005a, April). SocioscientificIssues in Secondary School Science: Students’ Epistemological Conceptions of Content, NOS, and EthicalSensitivity. Paper Presented at the National Association for Research in Science Teaching, Dallas, TX.

Zeidler, D.L., Sadler, T.D., Simmons, M.L., & Howes, E.V. (2005b). Beyond STS: A research-basedframework for socioscientific issues education. Science Education, 89(3), 357–377.

Zeidler, D.L., & Schafer, L.E. (1984). Identifying mediating factors of moral reasoning in scienceeducation. Journal of Research in Science Teaching, 21(1), 1–15.

Zeidler, D.L.,Walker, K.A., Ackett,W.A., & Simmons,M.L. (2002). Tangled up in views: Beliefs in thenature of science and responses to socioscientific dilemmas. Science Education, 86(3), 343–367.

Zohar, A., & Nemet, F. (2002). Fostering students’ knowledge and argumentation skills throughdilemmas in human genetics. Journal of Research in Science Teaching, 39, 35–62.



Documents

Advancing reflective judgment through Socioscientific Issues · JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 46, NO. 1, PP. 74–101 (2009) Advancing Reﬂective Judgment through