Quantitative and Qualitative Data Analysis:

What’s the Difference?

Jim Smith &Christine Maidl Pribbenow

2012 Research Residency

Session Outline

• Difference between Quantitative and Qualitative Data

• Using Mixed Methods• Statistics and their Use in Education

Research• Analyzing Qualitative Data• Questions??

What are some of the assumptions that you have about educational

research?

How are they helping or hindering the development of your study?

Research in the sciences vs. research in education

• “Soft” knowledge• Findings based in specific

contexts• Difficult to replicate• Cannot make causal

claims due to willful human action

• Short-term effort of intellectual accumulation– “village huts”

• Oriented toward practical application in specific contexts

• “Hard” knowledge• Produce findings that are

replicable • Validated and accepted as

definitive (i.e., what we know)

• Knowledge builds upon itself– “skyscrapers of knowledge”

• Oriented toward the construction and refinement of theory

Sources of Quantitative Data

• Multiple choice or closed questions• GPA, grades• Concept inventories• Rubrics• SAT/ACT/GRE - standardized tests• Anything you can count (stars, population)!

Sources of Qualitative Data

• Lab notebooks• Field notes from observations• Open-ended exam or survey questions• Written papers, homework• Journal entries, reflections• On-line discussions, blogs• Email• Texts or notes from interviews or focus groups

Qualitative Data: Oxymoron or inherent tensions?

• Hard vs. soft (mushy)• Rigor• Validity and reliability• Objective vs. subjective• Numbers vs. text• What is The Truth?

Mixed Methods Designs:Taking the Best of Both!

QUANData & Results

QUANData & Results Interpretatio

n

Interpretation

QUALData & Results

QUALData & Results

QUANPre-test Data & Results

QUANPre-test Data & Results

QUANPost-test Data & Results

QUANPost-test Data & Results

Intervention

qualProcess

qualProcess

Interpretation

Interpretation

Convergent Parallel Design

Embedded Design

Concurrent Mixed Methods Designs

10

QUANData & Results

QUANData & Results Interpretati

on

Interpretation

qualData & Results

qualData & Results

Following up

QUALData & Results

QUALData & Results

quanData & Results

quanData & Results Interpretati

on

InterpretationBuilding

to

Before-intervention

qual

Before-intervention

qual

QUANInterventio

nTrial

QUANInterventio

nTrial

After-

intervention qual

After-

intervention qual

Interpretation

Interpretation

Exploratory Design

Explanatory Design

Sequential Embedded Design

Quantitative Data and Statistics

Back to Jim…

Qualitative Data Analysis

Qualitative analysis is the

“interplay between researchers and data.”

Researcher and analysis are

“inextricably linked.”

Qualitative Data Analysis

• Inductive process– Grounded Theory

• Unsure of what you’re looking for, what you’ll find• No assumptions• No literature review at the beginning• Constant comparative method

• Deductive process– Theory driven

• Know the categories or themes using rubric, taxonomy• Looking for confirming and disconfirming evidence• Question and analysis informed by the literature, “theory”

Definitions: Coding and Themes

• Coding process: – Conceptualizing, reducing, elaborating and

relating text– i.e., words, phrases, sentences, paragraphs.

• Building themes:– Codes are categorized thematically to describe

or explain phenomenon.

Let’s Code #1

Read through the reflection paper written by a student from an Ecology class and highlight words, parts of sentences, and/or whole sentences with some “code” attached and identified to those sections.

What did you highlight?

Why?

Let’s Code #2

Read through this reflection paper and code based on this question:

What were the student’s assumptions or misconceptions before taking this course?

What did you highlight?

Why?

Let’s Code #3

Read through this reflection paper and code based on this question:

What did the student learn in the course?

What did you highlight?

Why?

Can we say that the students learned something in the course

using reflection papers?

Why or why not?

Ensuring “validity” and “reliability” in your research

• Use mixed methods, multiple sources.• Triangulate your data whenever possible.• Ask others to review your design methodology,

observations, data, analysis, and interpretations (e.g., inter-rater reliability).

• Rely on your study participants to “member check” your findings.

• Note limitations of your study whenever possible.

Questions?

References

• Designing and Conducting Mixed Methods Research, Creswell, J.W., and Plano Clark, V.L., 2006, Sage Publications.

• Discipline-Based Education Research: A Scientist’s Guide, Slater, S.J., Slater, T.F., and Bailey, J.M., 2010, WH Freeman.

• “Educational Researchers: Living with a Lesser Form of Knowledge,” Labaree, D.L., 1998, Educational Researcher, 27(8), 4-12.

Resources• Atlas.ti and NVivo (qualitative analysis software)