Rigor AND Relevance

Kurt C. Stange, MD, PhDEditor, Annals of Family Medicine

Gertrude Donnelly Hess, MD Professor of Oncology Research

Professor of Family Medicine, Epidemiology & Biostatistics,

Sociology and OncologyCase Western Reserve University

Overview• Reflections on Ornstein & Ruhe presentations

• The problem

• An integrative epistemology

• A transdisciplinary whole systems approach

• QI “laboratories” - PBRNs

• Participatory approaches to both R & D

• Mixed methods – quantitative & qualitative

• A complexity science underpinning

• Putting it together

Reflections on Presentations by Ornstein & Ruhe

• Responding to common stimuli

• Intervention approaches

• Evaluation Methods

Responding to Common Stimuli• Real-world primary care

– Incompatible reimbursement/business model– Competing demands / opportunities– PBRNs

• Diversity– Undesirable (low rates of effective practices)– Desirable (local adaptation)

• Melding research & quality improvement– Resistance to randomization

• Peer review / categorical funding

Intervention Approaches

• Multifaceted– Multiple processes, tools & outcomes– Moving toward multilevel Chronic Care Model

• Individualization of shared best practices– Outside facilitation / consultation– Shared learning

• Complexity science principles

Evaluation Approaches• Mixed methods designs

– Group RCT → pre/post design– Concurrent qualitative process evaluation

• Tailoring of intervention

• Measures– Quantitative process & outcome

• Inductive approach– Observation, interviews– Discovery of what works locally

The Problem• Rigor vs. relevance

• Internal vs. external validity

• Isolation of a phenomenon from context (so that it can be more rigorously studied) when context matters

• Our methods, theories, world view don’t match the problems or apparent solutions

4 Ways of Knowing

Adapted from:

Wilber, K. Sex, Ecology, Spirituality. 1995/2000, Boston: Shambhala Publications, Inc.

Wilber, K. A Brief History of Everything. 1996, Boston: Shambhala Publications, Inc.

Inner Reality

Outer Reality

Individual “I” “It”

Collective “We” “It”

4 Ways of Knowing About Health & Health Care

Adapted from:Stange KC, Miller WL, McWhinney I. Developing the knowledge base of family practice. Fam Med. 2001; 33(4):286-297.

1Clinician, Patient,

Worker, Policymaker

4Disease,

Treatment

2Family, Practice,

Community

3Systems,

Organization

4 Ways of Knowing About Health & Health Care

Borders 1-2 Relationship2-3 Justice3-4 Prioritization4-1 Information mastery1-3 Collaboration2-4 Illness

1Clinician, patient,

worker, policymaker

4Disease,

Treatment

2Family, Practice,

Community

3Systems,

Organization

Multiple Ways of Knowing

SYNTHESIS of ways of knowingThe craft of policy & general practice is the integration and application of knowledge of biomedicine, health care systems, individuals, families, communities & self.

1-2 Relationship2-3 Justice3-4 Prioritization4-1 Information mastery1-3 Collaboration2-4 Illness

1Clinician, patient,

worker, policymaker

4Disease,

Treatment

2Family, Practice,

Community

3Systems,

Organization

BORDERS among ways of knowing

Multiple Ways of Knowing

1 INFORMATION 4 (Quadrant 1) MASTERY (Quadrant 4)

CLINICIAN Evidence-Based DISEASESelf-Awareness Medicine ScienceReflection, Journaling Learning Epidemiology &

Experimentation

RELATIONSHIP INTEGRATION PRIORITIZATIONHuman Interactions Health Care & Healing ValueParticipant Observation Transdisciplinary, Cost-Effectiveness Analysis

Multi-method,Participatory

(Quadrant 2)

PATIENT, (Quadrant 3)

FAMILY SYSTEMCOMMUNITY OrganizationsPersonal Values Health ServicesIn-Depth Interviews JUSTICE ResearchLiving in Place Social Values

Policy Analysis2 3

For each item, bold capitalized words on the first line signify "FOCUS OF KNOWLEDGE," normal text on thesecond line signifies "Task of Understanding," and italicized words on the third line signify "Mode of Inquiry."

Honoring Different Ways of Knowing

• Acknowledging different perspectives– In planning studies– In interpreting findings

• Multiple groups working separately but communicating

• Paying attention to (or at least considering) other ways of knowing, even if only working on one.

Integrating Different Ways of Knowing

• Sequential studies

• Simultaneous studies– Multiple viewpoints– Multiple methods

• Studies of the border regions

• Studies of the integrative function

Transdisciplinary, Whole Systems Collaboration

can create abundance by sharing scarcity.

Crabtree BF, Miller WL, Adison RB, Gilchrist VJ, Kuzel A. Exploring Collaborative Research in Primary Care. Thousand Oaks, California: Sage Publications; 1994.

Leadership for Collaborative Research, Development

and Quality Improvement

• Whole system– Top down– Bottom-up

Thomas P, Griffiths F, Kai J, O'Dwyer A. Networks for research in primary health care. BMJ. 2001;322:588-590.

Thomas P. Integrative Primary Health Care: Leading, Managing, Facilitation. London: Radcliff Press, 2006.

A Typology of Collaboration

• Multidisciplinary

• Interdisciplinary

• Transdisciplinary

Crabtree BF, Miller WL, Adison RB, Gilchrist VJ, Kuzel A. Exploring Collaborative Research in Primary Care. Thousand Oaks, California: Sage Publications; 1994.

Multidisciplinary Research

• Multiple disciplines

• Each contributes their piece to solving a problem

• Like an edited book or separate presentations by multiple experts

Interdisciplinary Research

• A conversation between and among disciplines

• Working together on solving a common problem

• Like a collaborative health care team

Transdisciplinary Research

• A sustained conversation across and beyond disciplinary boundaries

• Creates a new shared language

• Such as the emergence of family systems medicine

6 Stages of Collaboration

• Acceptance / validation

• Shared expectations

• Declaring group process

• Action consensus

• Common space

• Sustained common action

Barriers to Collaborative Process

• Rhetorical stones– Power heaping– Shaming– Jargon hurling

• Powerful hegemony– Rationality vs. / and – Wonder, confession gratitude, receptivity to gift and mystery

• Tension between pragmatism and reflection

• Tension between individual & systems focus

Tactics for Advancing the Process

• Brainstorming

• Humor

• Storytelling

• Silence and time out for play

The Actual Process

• Non-linear

• Blurring and blending of levels

• Back and forth between levels

• Destabilized by time pressure

• Enhanced by flexibility, tact, patience and persistence

Dangers of Collaborative Research Relationships

• Sloppiness from training down

• Suppression of individuality

• Cultism

• Political nature of groups

• Minimizing these requires self-reflection and challenging

6 Ingredients for Successful Collaborations

• Linkage perspective

• Local context and action

• Problem-focused

• Appropriate methods

• Critical multiplism

• Coordination by a generalist researcher

Leadership for Collaborative Research

• In the face of change and uncertainty– Animation

• Provide initial direction• Encourages updating• Facilitates respectful interaction - trust, trustworthiness

– Improvisation• A hunch held lightly is a direction to be followed, not a

decision to be defended

– Lightness• “I don’t know”

– Authentication• Sensemaking

– Learning

Weick KE. Leadership as the legitimation of doubt. In Bennis W, Spreitzer GM, Cummings TG, (eds). The Future of Leadership. San Francisco: Jossey-Bass, 2001.

R &D &QI “Laboratories”

• Whole systems e.g. HMO Research Network

• NIH Research Center Model e.g. Cancer Centers

• Practice-based research networks (PBRNs)

PBRNs• Affiliated practices devoted to patient care

• Often academic or other partners

• Engage frontlines wisdom to – Develop or frame questions– Gather data– Interpret findings– Implement findings

• More generalizable patient populations, & theoretically, more transportable research

http://ahrq.gov/research/pbrnfact.htm

Primary Care PBRNs• Laboratories for primary care research

• 111 networks in 44 states

• Translate research into practice

• Translate practice into research

Nutting P, Beasley J, Werner J. Practice-based research networks answer primary care questions. JAMA. 1999;281:686-688.

Thomas P, Griffiths F, Kai J, O'Dwyer A. Networks for research in primary health care. BMJ. 2001;322:588-590.

Lanier D. Primary care practice-based research comes of age in the United States. Ann Fam Med. 2005; 3(suppl 1):S2-S4.

Primary Care PBRNs• Answer important questions

• Challenging current environment

• With support, ready for prime-time

• Answer to the call of the NIH Roadmap

Contemporary Challenges for Practice-Based Research Networks. Ann Fam Med; 2005; 3 (suppl 1). http://www.annfammed.org/content/vol3/suppl_1/index.shtml

Nutting PA, Stange KC, eds. Prescription for Health: Changing Primary Care Practice to Foster Healthy Behaviors. Ann Fam Med; 2005; 3 (suppl 2). Full text free at: http://annfammed.org/content/vol3/suppl_2/index.shtml

Green LA, Dovey SM. Practice based primary care research networks. They work and are ready for full development and support. BMJ.2001;322:567-568.

Nutting PA, Beasley J, Werner JJ. Practice-based research networks answer primary care questions. JAMA. 1999;281:686-688.

1000 persons

800 report symptoms

327 consider seeking medical care

217 visit a physician’s office (113 visit a primary care physician’s

office) (PBRN Research)

65 visit a complementary or alternative medical care provider

21 visit a hospital outpatient clinic

14 receive home health care

13 visit an emergency dept

8 are hospitalized

<1 is hospitalized in an academic medical center (Most Research)

Fig. Results of a reanalysis of the monthly prevalence of illness in the community and the roles of various sources of health care. (Green LA et al., N Engl J Med 2001, 344:2021-2024)

The Ecology of Medical Care

Community Participatory Research

• Knowledge, resources, involvement of communities are key to success of research and its transportability & sustainability

• 3 primary features– Collaboration– Mutual education– Acting on results developed from research

questions relevant to the community

Macaulay AC, Commanda LE, Freeman WL, et al. Participatory research maximises community and lay involvement. BMJ. 1999;319:774-778.

Facilitation of Participatory Research

• Continuity

• Timeliness

• Flexibility

• Clear and explicit expectations

• Appropriate & instructive accountability

• A vision for participatory research

• Tailored technical assistance

• Enhancing partner collaboration & support

Mercer SL, MacDonald G, Green LW. Participatory research and evaluation: from best practices for all states to achievable practices within each state in the context of the Master Settlement Agreement. Health Promot Pract. 2004;(3 Suppl):167S-178S.

Multimethod Approaches

• Increasingly accepted

• Efficient for generating new knowledge

Stange KC, Zyzanski SJ. Integrating qualitative and quantitative research methods. Fam Med, 1989; 21:448-451.

Stange KC, Miller WL, Crabtree BF, O’Connor PJ, Zyzanski SJ. Multimethod research: Approaches for integrating qualitative and quantitative methods. J Gen Intern Med. 1994; 9:278-282.

Crabtree BF, Miller WL. Doing Qualitative Research. 2nd Ed. Thousand Oaks, California: Sage Publications, 1999.

Crabtree BF, Miller WL, Stange KC. Understanding practice from the ground up. J Fam Pract, 2001; 50:881-887.

Borkan JM. Mixed methods studies: a foundation for primary care research. Ann Fam Med. 2004;2(1):4-6

NCI Conference on Mixed Methods Research, July, 2004, Denver, CO.

Multimethod Research • Quantitative methods

• Counting descriptions• Testing a priori hypotheses• Seek to isolate phenomenon from context

• Qualitative methods• Rich descriptions• Discovery; testing evolving hypotheses• Seek to understand meaning and context

• Integrated use• Qualitative, then quantitative• Quantitative, then qualitative• Simultaneous

Stange KC, Miller WL, Crabtree BF, O’Connor PJ, Zyzanski SJ. Multimethod research: Approaches for integrating qualitative and quantitative methods. J Gen Intern Med. 1994; 9:278-282.

Mixed methods allow you to have your cake and eat it too.

Additional Theoretical Underpinnings

• Complexity science principles

• Chronic care model

• Re-Aim

• Framework for complex interventions

• Gree nhalgh

Practices as Complex Adaptive Systems

• Complex behavior emerges from relationships among agents

• Simple rules

• Recurrent patterns

• Co-evolution

• Dependence on initial conditions

• Non-linearity

• Strategies for intervention• Joining• Transforming• Learning

Miller WL, Crabtree BF, McDaniel RA, Stange KC. Understanding primary care practice: A complexity model of change. J Fam Pract, 1998; 46:369-376.

Miller WL, McDaniel RA, Crabtree BF, Stange KC. Practice Jazz: Understanding variation in family practices using complexity science. J Fam Pract, 2001; 50:872-878.

Stroebel CK, McDaniel RR Jr, Crabtree BF, Miller WL, Nutting PA, Stange KC. Using complexity science to inform a reflective practice improvement process. Jt Comm J Qual Patient Saf, 2005; 31:438-446.

Three Key InsightsThree Key Insights

• Practices are co-creative participants in dynamic fitness landscapes.

• There are multiple ways to achieve effective health care delivery in practice.

• The best strategies for improving practice pay attention to improving relationships among stakeholders.

McDaniel RA, 2005

Chronic Care Model• Community• Health care system• Resources, policies & organization

– Self-management support– Delivery system design– Decision support– Clinical information systems

• Interaction– Informed, activated patient– Prepared, proactive practice team

Wagner EH. Chronic disease management: What will it take to improve care for chronic illness: Effective Clinical Practice. 1998;1:2-4.

Glasgow RE, Orleans CT, Wagner EH, Curry SJ, Solberg LI. Does the chronic care model serve also as a template for improving prevention? Millbank Q. 2001;79:579-612.

Re-Aim• Reach• Efficacy/effectiveness• Adoption• Implementation• Maintenance

www.re-aim.org

Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44:119-127.

Glasgow R, Magid D, Beck A, Ritzwoller D, Estabrooks P. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care. 2005;43:551-557.

Design & Evaluation of Complex Interventions

Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694-696.

Next Generation of Diffusion of Health Service Innovations

• Theory-driven• Process rather than ‘package’ oriented• Ecological• Addressed with common definitions, measures, tools• Collaborative & coordinated• Multidisciplinary & multimethod• Meticulously detailed• Participatory

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581-629.

An Incremental Approach

• Work on or pay attention to multiple levels

• Pursue research, development & shared learning

• Develop participatory relationships that transcend single projects

• Integrate qualitative & quantitative methods

• Reconsider enabling the current dysfunctional system versus fostering its transformation