Mixed methods: Integration of qualitative &

quantitative data collection and analysis in a single inquiry

Arden M. Morris, MD, MPH Associate Professor of Surgery

Center for Health Outcomes and Policy University of Michigan

No relevant financial disclosures

Partly supported by the Michigan Institute for Clinical & Health Research Summer

Disparities Immersion Program

Associated with the UW Collaborative to Improve Native Cancer Outcomes

(CINCO)

Raven: A Trickster Tale from The Pacific Northwest by Gerald McDermott

Objectives

• Describe MMR: rationale, design, benefits and challenges

• Review concepts using a systematic literature review example

• Discuss how to present and evaluate MMR studies

What is MMR?

Quantitative • Small number variables, large datasets • Closed ended Qs • Statistical analyses of numbers: Point

estimate for the mean of the population • Emphasizes generalizability • Concerns for bias

Qualitative • Small sample size • Open ended Qs • Inductive or deductive analysis of words • Emphasizes explanation (meaning,

model, mechanism, relationships, context) • Concerns for bias

MMR: What why when

• Data must be integrated, compared, contrasted, appraised, & synthesized

• Quantitative & qualitative approaches are complementary

• Neither quantitative nor qualitative approach is sufficient in capturing all relevant information

Six core characteristics of MMR

• Collection of qualitative and quantitative data (open- and closed-ended) in response to research Qs

• Analysis of both qualitative and quantitative data • Persuasive and rigorous procedures for the qualitative

and quantitative methods • The integration of these two data sources (merging,

connecting, embedding) • The use of a specific MM design using concurrent or

sequential integration • A philosophical foundation

9

Creswell, 2010

10

Post-positivism Determination Reductionism Empirical observation and measurement Theory verification

Constructivism Understanding Multiple participant meanings Social and historical construction Theory generation

Participatory Political Empowerment issue-oriented Collaborative Change-oriented

Pragmatism Consequences of actions Problem-centered Pluralistic Real-world practice oriented

Creswell, 2010

Mixing the methods Timing, implementation, priority

11

Connect data

Converge data

Embed the data

Quan Qual

Results Qual Quan

Results Qual Quan

Mixed methods designs

• Convergent parallel design • Explanatory sequential design • Exploratory sequential design • Embedded design • Transformative design • Multiphase design

Creswell, 2010

13

Convergent Parallel Design

Embedded Design

Concurrent MMR Designs

Creswell, 2010

QUAN Data collection

& analysis

Interpretation QUAL

Data collection & analysis

Interpretation

Quan Pre-test data & analysis Qual

process

Quan Post-test data

& analysis

intervention

14

Explanatory Design

Exploratory Design

Sequential MMR Designs - 1

Creswell, 2010

QUAN Data collection

& analysis

Interpretation qual

Data collection & analysis

QUAL Data collection

& analysis

Interpretation quan

Data collection & analysis

15

Sequential Embedded Design

Sequential MMR Designs - 2

Creswell, 2010

Qual Pre-

intervention

Interpretation QUAN

Intervention trial

Qual Post-

intervention

16

Sequential Embedded Design

Transformative Design

Qual Pre-

intervention

Interpretation QUAN

Intervention trial

Qual Post-

intervention

Multiphase (or Multi-project) Design

17

Interpretation quan

Data collection & analysis

QUAL Data collection

& analysis

QUAN Data collection

& analysis

Interpretation QUAL

Data collection & analysis

Technique Developments

• Types of designs; diagrams, detailed procedures, notation

• Scripts for purpose statements • Mixed methods research questions • Analysis strategies for merging data • Point of interface strategies for

sequential data

18

Typical Scenarios

• Survey & focus groups data merged and compared • Survey completed then focus groups used to

explain the quantitative results • Focus groups conducted and data used to construct

follow-up survey with a large sample • Qualitative data are collected before and after an

experiment is conducted • A longitudinal study with multiple quantitative and

qualitative studies to address a single overarching research objective

Example project: Systematic literature review

and meta-synthesis

Evidence of a problem • Planning CINCO MMR project

– Focus groups (tribe members) – survey/interviews (MDs)

• What is known about shared decision making among AI/AN patients?

• How should we incorporate the domain of SDM into survey?

Shared decision making

Patient choice

Shared decision making

Provider choice

Starting with a question

How does SDM happen among AI/AN ca pts?

What is the process & who makes

decisions among AI/AN ca pts?

Starting with a question

What is the process & who makes

decisions among AI/AN ca pts?

What is the process & who makes

decisions among minority ca pts?

Developing the taxonomy

• SDM: 2-way information exchange btw MD & pt followed by discussion of tx preferences until they reach consensus on a tx decision

• Racial/ethnic or cultural minority group: A racial, ethnic, or cultural minority population in the country where the study took place.

Inclusion Criteria Exclusion Criteria

Fit the definition of SDM DM for cancer treatment in children

Reported data on cancer treatment DM

Data not stratified by race/ethnicity or no minority group

Collected primary data Palliative or end-of-life care in cancer treatment

Results related to DM in a minority group

Peer-reviewed publication

Data acquisition

Data abstraction • A data abstraction tool developed and

iteratively revised • Across-method and method-specific study

features, sampling strategy, methods, results, conclusions, and assessments of methodological quality

• 2 reviewers independently data points and results were combined

Data abstraction

Data abstraction - 2

Data synthesis • Iterative thematic analysis • Conceptual model development • Findings synthesized for all cancer sites, sorted

and summarized by theme and study type (quan vs. qual).

• Confidence intervals were reported if available • Racial, ethnic, and gender differences described

Thematic analysis 1. Familiarization w data: 2 reviewers independently

read each paper in-depth 2. Data were independently coded 3. Reviewers searched the data for significant

themes 4. Reviewers discussed, compared and contrasted

the themes across studies and revised until 5. Consensus reached on final data-driven themes 6. Organized into a conceptual model

Level Theme - 1 Quan Qual

DM Process

Decisional role 8 11

Conflict btw actual & pref decisional role 2 5

Satisfaction/regret w DM or decision 3 6

Patient Factors

Spirituality and fatalism 1 6

Attitudes about tx & decision 3 3

Self-efficacy 2 1

Level of acculturation & language 6 5

Level Theme - 2 Quan Qual

Family/ Impt Others Factors

Family & others’ participation or advocacy & DM 6 8

Influence of family, friends, & others’ experiences 1 3

Participation or advocacy of others and receipt of treatment 6 0

Consequences of ceding decision-making autonomy 0 8

Community Factors

DM as a collective experience 0 3

Cultural/community norms & values 0 6

Communication w the community 0 4

Level Theme - 3 Quan Qual

Provider Factors

Provider preferences and recommendations

4 5

Provider communication & information-giving

3 8

Conflict and cultural congruence in the patient-provider relationship

0 3

A revised conceptual model

Linkages between themes

• Social support • Communication • Cultural congruence

Reporting and Assessing MMR

Challenges to reporting MMR

• Audiences familiar with only one approach • Different language and terminology • Journal word counts • Reviewer inexperience and bias

6 criteria • Identify content or context • Rationale for mixing • Describe data types & sampling design • Priority of qualitative & quantitative • Implementation sequence • Design model

Wisdom et al 2012 Creswell et al 2004

Reporting MMR

• Hard to satisfy both experts & novices • Educate reviewers—consider providing

similar published studies as appendices • Trial run figures/tables among a few people

you trust • Onus is on you to communicate clearly &

within bounds

Final thoughts • Clarify your research question • Access expertise in both methods • Take the time to create and test conceptual

models • Understand the taxonomy and consider a

pragmatic approach • Refer to successfully published MMR studies

Thanks!