In our November column (Libarkin and Kurdziel,2001) we discuss various types of qualitative data, howthese data can be collected, and a variety of methods foranalysis and use in classroom assessment. However, weonly touch upon the notion of appropriateness; that is,when is it appropriate to use qualitative techniques, andwhen are quantitative techniques more suitable to astudy? This issue is a long-debated and much discussedtopic in educational research, and is commonly calledthe Qualitative-Quantitative Debate.
Most researchers would agree that data, even fromthe most controlled experimental study, are neverpurely quantitative. The context of the study and theperspective of the researcher always affect the way dataare collected and interpreted. Where then does thedivision between qualitative and quantitative lie?Qualitative research focuses on the context of aphenomenon, while quantitative research seeks todevelop phenomenological generalizations that can beapplied to a range of contexts. Eisner (1991) discusses acontinuum of data types ranging in quality fromfictional to structured. In this framework, purelyqualitative data sit near the fictional end of the spectrumand are directly related to the individual or settingunder study, while quantitative data are highlycontrolled. For issues of science education assessment,data can be classified similarly as a spectrum rangingfrom anecdotal to statistical (Figure 1). Qualitative datastemming from classroom observations, interviews, orwritten documents can be shifted into the realm ofquantitative data through content analysis (Libarkin andKurdziel, 2001). Similarly, quantitative data ineducational settings are collected with subjective tools,such as Likert-scales. Data are further quantifiedthrough statistical analysis. Some researchers wouldargue that no data are purely quantitative, especiallysince the interpretation of statistical analyses is oftensubjective (Lincoln and Guba, 1985; Creswell, 1994).
Although this continuum between qualitative andquantitative research exists, the assumptions inherent toeach type of study are dramatically different (Trochim,2001). Quantitative researchers study social interactionsjust as they would study natural phenomena. In aquantitative world, everything can be reduced to atheoretical framework that describes the social context.Thus, the quantitative researcher treats socialphenomena as a set of interconnected variables, and
every social phenomenon is the result of interactionsbetween these variables. On the other side of the debate,qualitative researchers reject the idea that the social andnatural world can be studied in similar ways. Instead,qualitative researchers believe that human behavior isalways a function of the setting in which behavior isbeing observed, and that different individuals may actdifferently in identical settings. Qualitative researchersdo not deconstruct social interactions into compositevariables. Indeed, the most strident qualitativeresearchers would argue that generalizations can neverbe made; each setting is unique from every other.
QUALITATIVE RESEARCH: PROS AND CONS
Qualitative research is an unconstrained approach tostudying phenomena. Although a number of standardapproaches to collecting and interpreting qualitativedata exist (Bogdan and Biklen, 1992), qualitativeresearch is strongly dependent upon the researcherconducting the study. The researcher decides upon thetype of data gathered and the methods used to analyzethose data. Herein lies the power and the weakness ofqualitative research. Qualitative data is typically rich indetails and context, interpretations are tied directly tothe data source, and research validity and reliability arebased upon the logic of the study interpretations, ratherthan statistical tests (Table 1). Qualitative studies,therefore, provide a window into a contextual setting,and a logical picture of events within that setting.However, the attention to detail central to qualitativeanalysis also means that study conclusions will typicallyonly apply to a very narrow range of circumstances.Additionally, the training and beliefs of the qualitativeresearcher may themselves shape the research structureand findings. As a result, qualitative findings may notprovide any correlation between cause and effect on abroad scale.
QUANTITATIVE RESEARCH: PROS AND CONS
Quantitative research focuses on a set of narrowlydefined research methodologies. The tools andtechniques used for gathering and analyzing data arewell established, and the validity and reliability of astudy typically depend upon following pre-existing
78 Journal of Geoscience Education, v. 50, n. 1, January, 2002, p.78-86
Research Methodologies in Science Education:The Qualitative-Quantitative Debate
Julie C. LibarkinScience Education Department, Harvard-Smithsonian Center for Astrophysics,60 Garden Street, MS-71, Cambridge MA 02138, email@example.com
Josepha P. KurdzielTeaching and Teacher Education, University of Arizona, Education Room 703,
Tucson AZ 85721, firstname.lastname@example.org
methodologies (Table 1). The wide range of availablestatistical methods (Creswell, 1994) allows researchersto develop explanatory models that can account forphenomena occurring in similar settings. These modelsallow for the development of theories of cause andeffect, and can have significant predictive power inclassroom settings. Additionally, because data analysisis governed by statistics, the personal beliefs of theresearcher will have only minimal impact on studyfindings. However, this modeling approach can result inan artificial categorization of phenomena that has littlecorrelation with the real world. That is, unlikequalitative research, the context in which data wasoriginally collected may be lost beneath the layers ofstatistical analysis inherent to quantitative research.
Choosing one type of data over another ultimatelyhinges on the objectives of your assessment efforts. Thearguments for and against the use of each researchparadigm suggest a number of questions that can beused to determine which methodology will be mostuseful for a particular study. Researchers tend tocategorize these questions depending upon their ownresearch perspective. The following questions can befound in a variety of forms in Trochim (2001), Lincoln
and Guba (1985), Bogdan and Biklen (1992), and Milesand Huberman (1994).
I. Generating versus Testing Theories: Do Iunderstand the phenomenon under study well enoughto develop a theory about the interactions betweenvariables, or do I need to explore the setting further?
Both quantitative and qualitative methods areconcerned with exploring phenomena. However,qualitative analysis is primarily concerned with gainingdirect experience with a setting, while quantitativeanalysis seeks to document occurrences passively.Because qualitative research is an inherently exploratoryendeavor, the potential for generating new theories andideas is significant. Quantitative data, on the other hand,are most valuable when hypotheses and theories havealready been established and are being evaluated.
II. Single versus Multiple Variables: Are the variablesthat could potentially affect student outcomes alreadyidentified, or is the goal of your project to documentfactors influencing student outcomes?
Qualitative studies can be thought of as univariate,where the single variable is the context underobservation. In fact, a goal of qualitative research may be
Libarkin and Kurdziel - Research Methodologies in Science Education 79
Pros Cons Pros Cons
Issues can be studied ingreat detail. Analytical
Results may beapplicable to only a
narrow range ofindividuals or settings.Often no connection to
Results from a varietyof individuals or
settings can be used todevelop a single
Analytical approach isconstrained by
Individuals may beartificially forced into
Interpretation is oftenbased on manipulation
of raw data and istherefore tied directly
to the data source.
Individual beliefs ofthe researcher may
shape the datainterpretation.
Statistical analysis,although not perfectlyfree of subjectivity, istypically independent
of the researcherspersonal belief system.
By the time aquantitative study
reaches theinterpretation stage,the context in which
the data was collectedmay be lost.
Validity and reliabilityare establishedthrough logicalreasoning and
consensus; statisticsnot required.
Researcher acts as theinstrument; training
and skill ofpractitioner can bias
Validity and reliabilityare highly controlledvariables establishedstatistically; limitedtraining required.
Establishing validityand reliability is time
Table 1. Comparison of some aspects of qualitative and quantitative research.
80 Journal of Geoscience Education, v. 50, n. 1, January, 2002, p.78-86
the identification of variables affecting the phenomenonunder study (Creswell, 1994). For instance, in a study ofurban mothers, Barton and others (2001) used groupinterviews, observations, and surveys to identify bothperceptions about science and the factors that appearedto influence these viewpoints. Once key variables havebeen identified, quantitative methodologies can be usedto explore the relationship between variables andbroader ramifications.
III. Detail versus Generalization: How much detail isnecessary to meet the study objectives? Is generalizationto other settings an objective of the study?