Synthesizing Mixed-methods Synthesizing Mixed-methods Findings from Studies of Findings from Studies of Primary Care-based Primary Care-based Interventions:Interventions:Preliminary Insights from the Preliminary Insights from the Prescription for Health EvaluationPrescription for Health Evaluation

Deborah Cohen, PhDDeborah Cohen, PhD

Department of Family MedicineDepartment of Family Medicine

UMDNJ-RWJ Medical SchoolUMDNJ-RWJ Medical School

IntroductionIntroductionMulti-Method SynthesisMulti-Method Synthesis

ObjectivesObjectives To describe the design of a mixed To describe the design of a mixed

method cross project synthesismethod cross project synthesis To articulate some preliminary To articulate some preliminary

findingsfindings To discuss the benefits and To discuss the benefits and

drawbacks of this approachdrawbacks of this approach

IntroductionIntroductionQualitative and Mixed-Method Meta-Qualitative and Mixed-Method Meta-SynthesisSynthesis

DefinitionDefinition

Qualitative Meta-synthesis Qualitative Meta-synthesis Interpretive synthesis of data across Interpretive synthesis of data across studies using qualitative methods studies using qualitative methods

Mixed-method meta-synthesisMixed-method meta-synthesisInterpretive synthesis of data across Interpretive synthesis of data across studies using mixed methodsstudies using mixed methods

IntroductionIntroductionBackground on Prescription for HealthBackground on Prescription for Health

The Prescription for Health initiative The Prescription for Health initiative

sought to develop and pilot new sought to develop and pilot new

models for promoting and integrating models for promoting and integrating

health behavior change into routine health behavior change into routine

primary careprimary care

MethodologyMethodologyBackground on Prescription for HealthBackground on Prescription for Health

Evaluation GoalsEvaluation Goals

To understand project and practice To understand project and practice characteristics that supports the characteristics that supports the successful implementation and successful implementation and adherence to interventions in the adherence to interventions in the primary care settingprimary care setting

To conduct comparative case analysis of To conduct comparative case analysis of the overall initiative, summarizing new the overall initiative, summarizing new insights and patterns that transcend insights and patterns that transcend individual modelsindividual models

MethodologyMethodology Focus of the Analysis Team’s EvaluationFocus of the Analysis Team’s Evaluation

ConceptualizationConceptualization Studying implementation as way to Studying implementation as way to

evaluate what ‘works’ in real-life setting evaluate what ‘works’ in real-life setting (i.e. effectiveness)(i.e. effectiveness)

Mixed-Method, Nested Case study Mixed-Method, Nested Case study DesignDesign

Practice SurveysPractice Surveys

Site Visits/InterviewsSite Visits/Interviews

Online diaryOnline diary

Patient OutcomesPatient Outcomes

AnalysisAnalysis Focus of the Analysis Team’s EvaluationFocus of the Analysis Team’s Evaluation

Data Analysis – Qualitative Data Analysis – Qualitative DataData

First IterationFirst Iteration: Read diaries aloud in ‘real time’ : Read diaries aloud in ‘real time’ as a group. This informed site visits and as a group. This informed site visits and interaction with granteesinteraction with grantees

Second IterationSecond Iteration: Re-read full dataset for each : Re-read full dataset for each individual project tagging relevant textindividual project tagging relevant text

Third IterationThird Iteration: Read tagged data, discerning : Read tagged data, discerning themes and sub-themes and investigating themes and sub-themes and investigating new ideas across projectsnew ideas across projects

AnalysisAnalysis Focus of the Analysis Team’s EvaluationFocus of the Analysis Team’s Evaluation

Data Analysis – Quantitative Data Analysis – Quantitative DataData

First IterationFirst Iteration: Cleaning process to ascertain : Cleaning process to ascertain missing data, followed by request to fill gapsmissing data, followed by request to fill gaps

Second IterationSecond Iteration: Looked for consistency within : Looked for consistency within data across different variables; rectified data across different variables; rectified inconsistenciesinconsistencies

Third IterationThird Iteration: Created PIF and PAT narratives; : Created PIF and PAT narratives; integrated into qualitative databaseintegrated into qualitative database

AnalysisAnalysis Integrating Data SourcesIntegrating Data Sources

Translating data sourcesTranslating data sources– Qualitative Qualitative → → QuantitativeQuantitative– Quantitative Quantitative → → QualitativeQualitative

Interpretive integrative analysis Interpretive integrative analysis – Developing within project insightsDeveloping within project insights– Developing cross-project insightsDeveloping cross-project insights– Using common data to foster cross Using common data to foster cross

project learningproject learning

Preliminary FindingsPreliminary FindingsWhat does it take to bridge primary What does it take to bridge primary care practice to community care practice to community resources?resources?

Primary Care ElementsPrimary Care Elements

Identifying patients at riskIdentifying patients at risk

Knowledge of how to Knowledge of how to access community access community resourcesresources

Capacity to refer patientsCapacity to refer patients

As our survey data show, practices did not necessarily have the processes in place for identifying patients with health behavior risk

Primary Care ElementsIdentifying Patients at Risk

Use of HRA in P4H practices

56.47

38.8234.12

50.59

32.94

0

20

40

60

80

100

tobacco alcohol diet physicalactivity

none

% p

ract

ices

Practice ElementsUse of Health Risk Assessments

Limited time was another barrier to Limited time was another barrier to

identifying at risk patients.identifying at risk patients.

The discussion between clinician and patient The discussion between clinician and patient is designed to support patient is designed to support patient interest in the in the behavioral change and enhance patient behavioral change and enhance patient engagement with the intervention. We believe engagement with the intervention. We believe it should take 2 minutes or less to complete it should take 2 minutes or less to complete this discussionthis discussion

Primary Care ElementsIdentifying Patients at Risk

Practices also had limited capacity to make Practices also had limited capacity to make a a

referral:referral:

If we need to print referrals for faxing to the If we need to print referrals for faxing to the

community resource maybe someone in the community resource maybe someone in the

research team’s office could do it (some concern research team’s office could do it (some concern

about stressing the referral person, although about stressing the referral person, although one one

doctor thought it was okay)doctor thought it was okay)

Primary Care ElementsMaking the Referral

Community Resource ElementsAvailability

To attempt to bridge to community To attempt to bridge to community resources resources

assumes that there are resources in the assumes that there are resources in the

communitycommunity

Community Resource ElementsAvailability

Methods to link patients to community resources

3.7511.25 8.75

3541.25

47.5

11.25

51.25

0

20

40

60

80

100

Physical activity Healthy eating Smokingcessation

Risky drinking

% p

ract

ices

Coordination andfeedback betweenpractice, patients, andresources

Designated personensures maximumresource use

List of resources available

No systematic approach

Community Resource ElementsAffordability

When resources did exist in communities, When resources did exist in communities, they they

were often not affordable:were often not affordable:

A barrier for counseling is patients affording it - A barrier for counseling is patients affording it - many would be interested in group visits if free, many would be interested in group visits if free, this is the reason for interest in the study… this is the reason for interest in the study…

Currently they will refer to some nutritionists, Currently they will refer to some nutritionists, but this is sporadic and dependant on what but this is sporadic and dependant on what insurance will cover, more often for diabetic insurance will cover, more often for diabetic patientspatients

Community Resource ElementsAccessibility

Distance and transportation can be barriersDistance and transportation can be barriers

......He also had another story of how one woman He also had another story of how one woman came to the clinic and explained that she had came to the clinic and explained that she had tried to go to a healthy diet meeting at a different tried to go to a healthy diet meeting at a different clinic, but had problems with transportation.clinic, but had problems with transportation.

Scheduling can be a barrier for people whoScheduling can be a barrier for people whowork:work:

One last concern that has come up is that, One last concern that has come up is that, because some of the nutrition classes are limited because some of the nutrition classes are limited to weekdays, patients' jobs preclude them from to weekdays, patients' jobs preclude them from attending. We'll continue to work on adding a attending. We'll continue to work on adding a web-based option…web-based option…

Community Resource ElementsPerceived Value

Branded, well-known resources were Branded, well-known resources were well-well-

received:received:

State Quit lines or AAState Quit lines or AA

He also said, "I want to send a lot of He also said, "I want to send a lot of my patients to Weight Watchers or my patients to Weight Watchers or something like telephone counseling, something like telephone counseling, but they just cannot afford it. Now they but they just cannot afford it. Now they can!”can!”

Community Resource ElementsPerceived Value

Lesser known resources were more Lesser known resources were more difficult to convincingly sell to difficult to convincingly sell to

patientspatients

For the telephone counseling they For the telephone counseling they needed someone in the practice who needed someone in the practice who was knowledgeable to explain the was knowledgeable to explain the program before the patient connected program before the patient connected to the counselors or patients would get to the counselors or patients would get connected who had no idea what the connected who had no idea what the program was or what to expect.program was or what to expect.

Thematic ImplicationsThematic Implications Bridging to community resourcesBridging to community resources

Structural holes between primary Structural holes between primary care practice and community care practice and community resources need to be filledresources need to be filled

Infrastructure is needed to support Infrastructure is needed to support health risk assessment and brief health risk assessment and brief counseling and referralcounseling and referral

Community resources need to be Community resources need to be accessible, affordable and desirableaccessible, affordable and desirable

Conducting Mixed Methods Conducting Mixed Methods Meta-SynthesisMeta-SynthesisSummary Points – Things to Summary Points – Things to ConsiderConsider Collection of common data across Collection of common data across

projectsprojects

Develop strategies for integrating Develop strategies for integrating qualitative and quantitative dataqualitative and quantitative data

Importance of multi-disciplinary teamsImportance of multi-disciplinary teams

Integrate quantitative and qualitative Integrate quantitative and qualitative methodologistsmethodologists

Conducting Mixed Methods Conducting Mixed Methods Meta-SynthesisMeta-SynthesisSummary Points – Some analytical Summary Points – Some analytical thoughtsthoughts Iterate between cross project and within project Iterate between cross project and within project

analysisanalysis

Identify project specific themes using language Identify project specific themes using language that references cross project experiences or that references cross project experiences or insightsinsights

When analyzing a theme in more depth, When analyzing a theme in more depth, consider all of the data sources that might consider all of the data sources that might contain some evidence or provide insightscontain some evidence or provide insights

Openness to having both quantitative and Openness to having both quantitative and qualitative data ‘lead’ the analysisqualitative data ‘lead’ the analysis

Conducting Mixed Methods Conducting Mixed Methods Meta-SynthesisMeta-Synthesis Summary Points – BenefitsSummary Points – Benefits

The sum of the cross project The sum of the cross project insights insights

can be greater than the insights of can be greater than the insights of any any

individual projectindividual project

Conducting Mixed Methods Conducting Mixed Methods Meta-SynthesisMeta-Synthesis Summary Points – Potential LimitationsSummary Points – Potential Limitations

Variable data quality: data are only as Variable data quality: data are only as good as that collected by each projectgood as that collected by each project

Themes will not manifest themselves Themes will not manifest themselves evenly across projectsevenly across projects

The analyst is telling a cross project The analyst is telling a cross project story -- each project highlights story -- each project highlights complementary insightscomplementary insights

The presence and absence of a theme The presence and absence of a theme across different projects can be across different projects can be analytically importantanalytically important