TB MAC, IstanbulOctober 2019

Leveraging health systems perspective

& approaches to understand the

impact of UHC

System Dynamics (SD) Simulation in Support of Global Disease Control Strategies

Özge Karanfil, Ph.D, MSc.

Complex systems are often counterintuitive...

• Policy Resistance: The tendency for interventions to be delayed, diluted, or defeated by the response of the system to the intervention itself

• - Meadows, Richardson & Bruckmann

Dynamic problems w/ complex causal structures and feedback push back against interventions

Policy Resistance: Unintended Consequences of Our Actions

Adding more roads

to reduce congestion

Increased development and

ultimately more congestion

Increasing our use of

antibiotics & antibacterial soap

Stronger, more resistant

strains of bacteria

Suppressing forest fires

Even stronger fires due to

build up of dead wood to fuel

fires

“Todays solutions are tomorrows problems”

What is a System? What are Dynamics?

System (Structure) = Stocks + Flows + Feedback Loops +…

• Stocks are accumulations of flows

(of population, resources, changing goals, perceptions, etc.)

• Feedback loops link accumulations back to decisions that

alter the flows: only 2 types (goal-seeking, self-reinforcing)

• Delays complicate things further

• As do non-linearities (need for critical mass, saturation effects)

Dynamics = Behavior over time

• Patterns in time series data (growth, fluctuation, etc.)

• Visible relationships of two or more variables

(move together, move opposite, lead-lag, etc.)

StockFlow

Feedback

influence

Compartmental models resting on a general theory of how systems change (or

resist change) – often in ways we don’t expect

Uniting feedback control theory, behavioral science, and longitudinal evidence

Concerned with understanding dynamic complexity, continuous monitoring, internal structure drives

behavior..

Systems Modeling Health Applications1970s to the Present

• Disease epidemiology– Cardiovascular, diabetes, obesity, HIV/AIDS, cervical

cancer, chlamydia, dengue fever, drug-resistant infections

• Substance abuse epidemiology – Heroin, cocaine, tobacco

• Health care patient flows – Acute care, long-term care

• Health care capacity and delivery– Managed care, dental care, mental health care,

disaster preparedness, community health programs

• Health system economics– Interactions of providers, payers, patients, and

investors

Homer J, Hirsch G. System dynamics modeling for public health: Background and opportunities.

American Journal of Public Health 2006;96(3):452-458.

A Dynamic Feedback Approach to Understand Problems in Population (Routine) Screening

• Policy decision thresholds of Clinical Practice Guidelines (CPG) for screening are contentious, show big variations.

• To extend early detection benefits, breadth indications of screening (and use of med technology)have expanded: Threshold age to start screening, frequency, threshold to biopsy..

• Evidence-based CPGs not followed, with significant over- or under screening.

• Controversy over screening between evidence-based and advocacy groups

• Unintended, non-intuitive consequences:a) Early detection may cause harms exceeding benefits, mainly over diagnosis (CDC, NCI)b) 38.4% of Americans will be diagnosed with #Cancer in their lifetime, based on 2013-2015 data.

8Source: Nat. Cancer Inst-SEER: http://seer.cancer.gov/statfacts/html/all.html

Source: http://zerocancer.org/

Motivation & Research Questions

• Motivation: Develop dynamic hypothesis to explain underlying structure that accounts for: over/ underscreening & why these are not corrected by evidence

• Generic problem: U.S. context, cancer screening motivating, but applies to medical screening in other places, wide range of managerial applications..

• Medical context: Autism Spectrum Disorder, Hypertension, hypercholesterolemia, Alzheimers, neurodegenerative diseasesimplications for aging societies

• Nonmedical context: applies other settings where breadth indications (definition of target popn) for screening are changing, e.g. airport-background checks, tax audits, contentious jurisdiction

Karanfil, Sterman. 2019. “Saving

Lives or Harming the Healthy?

Overuse and Fluctuations in Routine

Medical Screening. Under Review in

System Dynamics Review.

Karanfil, Özge . PhD thesis in

Management Science. “Why clinical

practice guidelines shift over time: a

dynamic model with application to

prostate cancer screening”, MIT,

Cambridge, 2016

Karanfil, Sterman. 2015. A Dynamic

Model for Health Screening:

Misperceptions,

Feedback and Long-Term Trends in

Screening Mammography.

Implementation Science. 10: A50.

Winner of SD Society’s 2017

Lupina Young Researchers

Award (Best paper in Health

Applications)

What are sources of gaps between scientific evidence and practice?

Why screening so common for some diseases when evidence base is uncertain?

Why some screening tests are overused while some are underused?

LIT SEARCH & DATA COLLECTION

DYNAMIC THEORY

METHODS

EXPERT INTERVIEWS

Objective: Build a sound dynamic theory-guideline for guideline formation, grounded in empirical evidence & data

Committee on the Use of Humans as Experimental Subjects (COUHES) Protocol # 1412006813

Study Title: Dynamics of Routine Screening

• Data: Use qualitative and quantitative data to document evidence of gaps.

• Medical lit; NCI, CDC, NHANES, U.S. Mortality files by the NCHS, NCI-SEER database, NHIS• 34 semi-structured expert interviews (Breast & Prostate Screening)

• w/ policy makers, e.g. USPSTF, ACS, CISNET, academics, clinicians, advocacy group members,

media

Amount of

Screening

Diagnosed

Abnormalities

+

Indications of

Screening

Harms to

Benefits Ratio

+

Reported Harms

Benefits Ratio

+

-

Overtreat

ment

+

+

B

harms andbenefits

self-correction

DELAY

DELAY

Expert Opinion Interviews

Participants and Setting

• Expert opinion, semi-structured in-depth interview study with a purposive sample of 34 professionals

• Main focus: Investigate trends, understand sources of variation in screening guidelines, overuse and underuse

• Evidence-based vs. Specialty/ Advocacy Groups

• Emerging themes, Areas of Disagreements

• Perceived Biases and Problems

Committee on the Use of Humans as Experimental Subjects (COUHES) Protocol # 1412006813

Study Title: Dynamics of Routine Screening, MIT Sloan School of Management

Empirical Evidence

23%

27%25%

9%

16%

Total = 34 interviews, by main title

Academic

Clinician

Policy

Patient

Media/Advocacy/Science Writer

Bounded Rationality in Screening Decision: Perceived sources of biases

• Mis(perception) of Risk

• Cancer Survivors/ Feedback Asymmetry in Screening Decision

• Lack of Long-Term Thinking

• Lack of Time

• Understanding Uncertainty

• Risk Aversion

• Anecdotes

• Specialty Perspective

“…. people respond

to….anecdotes. You hear about a

women whose breast cancer was

detected by mammography in her

30s or 40s, or PSA in a younger

man that can seem pretty

compelling. ...anecdotes are

really powerful, ….But they’re

only one small piece of evidence

and not usually the most useful

piece ……”—Matt Gillman,

M.D., S.M. Director,

Environmental Influences on

Child Health Outcomes (ECHO);

Office of the Director, NIH

• This core step involves only the available facts and the analysis of evidence. Potential harms and benefits lead to balanced decision making.

15

Scientific data accumulation, translation and public perception delays are important: Virginia Moyer, Former Chair of USPSTF: “..It takes 18 years to get anything new into practice.” Science, 2012.

First Layer: Core Feedbacks responsible for setting Guidelines, using Classical Approach

Policy Structure for Development of Evidence-Based Guidelines : Evidence Generation & Guideline Development

Amount of

Screening

Cancer

Diagnosed

+Breadth Indications

of Screening

Harms and

Benefits

+

Reported Harms

Benefits Ratio

+

R1

expansion

-

Overtreatment

LegitimateScreening/Treatment

-

+

+

+B1

self-correction

DELAY

DELAY

DELAY

2-Policy Structure for Guidelines in

Use, Dissemination

& Implementation

Model of Actual

Practice:

Dissemination-

Implementation

TheoreticalEvidence-

Based Model Causal

Structure:

1- Evidence Generation• Stylized world: One set

of recommendation followed by public

• Structure shows how potential harms and benefits lead to BALANCED decision making

• B2 loop in the center reflects the influence of harm and benefit evaluations

2-Policy Structure for Guidelines in

Use, Dissemination

& Implementation

Model of Actual

Practice:

Dissemination-

Implementation

TheoreticalEvidence-

Based Model Causal

Structure:

1- Evidence Generation

• Generates the BREADTH OF SELECTION criteria for screening population, or the FRACTION of the population considered to be candidates for screening.

• Implicit delays embedded in policy decisions.

Even in idealized situation, CPGs are suboptimal, overshoot in screening indications

“Breadth selection criteria” or the disease definition gets expanded as we try to

eradicate

Test loses efficiency as criteria broadens, prevalence in target screening population

drops

Prevalence D+ in Screened Population

.4

.3

.2

.1

0

1980 1989 1998 2007 2016 2025 2034

Time (Year)

dm

nl

"Age Specific Prevalence D+" : Current3

From Model to an Interactive Game for Stakeholders:Action Lab for Population Screening: Scenario Maker

Cardiovascular (CVD) Management and Policy Model for Malaysia:Malaysia Heart Disease Prevention Simulation Model-(MHDPSim)

• CVD burden is on the rise –globally, with differences in age, sex and ethnic groups, which necessitates a careful understanding of the disease cascade and population demographics

• Malaysia’s burden of NCDs high, see National Health & Morbidity Survey (NHMS).

• Accounted for 60% of disease burden in 1990, measured by Disability Adjusted Life Years (DALYs) due premature death & morbidity, 72% by 2013

• We developed a dynamic simulation model of CVD, with team of implementation researchers, statisticians and modelers from U.S. & Malaysia, using cascade of care approach

• Optimal management of chronic & CVD in Malaysia along care continuum to ascertain effectiveness of health system responses, to link each stage to simulate cohort of virtual populations to ascertain effects of interventions along care continuum, test policy options

NHMS: National Health and Morbidity Survey, Results reported to the Government of Malaysia- Ministry of Health

Important implications in policy decisions for Malaysian Health System Reform Study (MHSR)

CDC’s PRISM Model :The Prevention Impacts Simulation Model

23

SD Modeling in Support of CVD Prevention Strategy

In 2006, the CDC’s Divn. of Heart Disease & Stroke Prevention was looking for better ways to…• Represent interacting risks and interventions for CVD• Integrate best available info to support planning and priority setting

The division decided to go with SD rather than microsimulation• Previous success at CDC using SD to model diabetes and obesity• Wanted speed and interactivity, even if “compartments” required some

simplifying assumptions regarding comorbidity distributions

PRISM has been in development and use since 2007, funded by CDC (and NHLBI)• Journal publications 2008-2014; awards 2008-2013• Used with more than 60 local health agencies for intervention planning: CPPW

(2009-2011), CTG (2012-2014), CPG (2015-2017) • Supported CDC’s “Million Hearts” national campaign in 2011

SYSTE M DY NAMI C S I N AC T I O N: Mayo C l i n i c An emi a Ma n a gement System (MC AMS) I MP ROV I NG OUTCOMES FOR D I ALYS I S PAT I E NTS

James T. McCarthy,

MD

Mayo Clinic Rochester

Anytown, USA (Scalable)

Local Configurations (N=11)

25

The model has been widely used…

Users• 11 Local Configurations• 100+ Strategy Labs• 5,000+ Leaders • 20+ Universities

The ReThink Health Dynamics Model

Homer J, Milstein B, Hirsch G, Fisher E. Combined regional investments could substantially enhance health system performance and be financially affordable.

Health Affairs. 2016;35(8):1435-43.

Siegel B, Erickson J, Milstein B, Evans Pritchard K. Multisector partnerships need further development to fulfill aspirations for regional system change. Health

Affairs. 2018;37(1):30-7.

26

Need to capture:

1. BROAD BOUNDARY: Systems interrelate. Holistic approach vs. breaking down into elements. Similar to chronic disease, infectious disease issues are often viewed in isolation. Allows testing combined interventions. Feedback loops, causality (loops (e.g., norm diffusion, policy resistance, reactions by various actors). Because system creates behaviour.

2. BOUNDED RATIONALITY : Diverse stakeholders recognize their interdependence, increasing their desire to collaborate. Differences or inconsistencies in goals/values among stakeholders

3. LIMITING FACTORS: Nonlinear causal relations (threshold, saturation effects) & time delays (differences between short- and long-term consequences of an action)

4. M& M (Maps and models): Allow taking advantage of qualitative & quantitative datasets, mental datasets. Stocks (“compartments”; e.g., prevalences, resources) and flows (e.g., incidence, death)

How to model complex, resilient health

systems to defeat policy resistance? Design Principles to Simulate Dynamic Complexity

Preliminary Ideas: Leverage points, places to intervene to support complex TB Control Strategies..

• Aim to integrate data at interface btw. biology, behaviour & environment to explore population level trajectories

• Need to understand how different interventions are likely to play out (alone and in combinations) —identify, simulate combined & multiple interventions.

• Collect qualitative data to map mental models, goals of stakeholders

• Convene diverse stakeholders to participate in model-supported “Action Labs”

• Screening subpopulations for TB infection, e.g. Healthcare providers (HCPs)

• The TB “System”: A Broad Causal Map-> Similar to Obesity System Map by UK Foresight.. Enable deeper understanding of dynamics contributing to TB epidemic.

27

Complex health & disease management problems critically require a systems approach...

Dynamic problems w/ complex causal structures and feedback push back against interventions.

Acknowledgements

• Prof. John D. Sterman, MIT Sloan School of Management

• Prof. Rifat Atun, HSPH Global Health and Population

• Bobby Milstein, ReThink Health

• Dr. Kimberly Thompson, Kids Risk

• Dr. Jack Homer, MIT Sloan & Homer Consulting

• Jim Rogers, Mayo Clinic

• Prof. Matt Gilman, NIH

• Prof. Harry de Koning, Erasmus & CISNET

• Nancy Keating, HMS & Brigham Women

• Natasha Stout, Harvard Medical School

BACKUP

29

TB Care Cascade

2015 Global TB Care Cascade

30

TB care cascade for 30 high burden countries

Empirical…and…Critical

Empirical

Evidence

System

Conceptualization

Model

FormulationRepresentation of

Model Structure

Comparison and

Reconcilation

Perceptions of

System Structure

Mental Models,Experience,Literature

Literature,

Experience

Empirical andInferred Time

Series

Comparison and

Reconciliation.

Deduction Of

Model Behavior

Diagramming and

Description Tools

Computing

Aids

StructureValidatingProcesses

BehaviorValidatingProcesses

Forrester JW, Senge PM. Tests for building confidence in system dynamics models. In: Legasto A, Forrester JW,

Lyneis JM, editors. System Dynamics. New York, NY: North-Holland; 1980. p. 209-228.

Graham AK. Parameter estimation in system dynamics modeling. In: Randers J, editor. Elements of the System

Dynamics Method. Cambridge, MA: MIT Press; 1980. p. 143-161.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin

McGraw-Hill, 2000.

Practical Options in Causal Modeling

Detail (Disaggregation)

Scope

(Breadth)

Low High

Low

High

Simplistic

Impractical

Focused

Expansive

Too hard to verify,

modify, and understand

Model Structure and Level of Detail

Depends on the Intended Uses and Audiences

• Set Better Goals (Planners & Evaluators)

– Identify what is likely and what is possible

– Estimate intervention impact time profiles

– Evaluate resource needs for meeting goals

• Support Better Action (Policymakers)

– Explore ways of combining policies for better results

– Evaluate cost-effectiveness over extended time periods

– Increase policymakers’ motivation to act differently

• Develop Better Theory and Estimates (Researchers)

– Integrate and reconcile diverse data sources

– Identify causal mechanisms driving system behavior

– Improve estimates of hard-to-measure or “hidden” variables

– Identify key uncertainties to address in intervention studies

Forrester JW. Industrial Dynamics (Chapter 11: Aggregation of Variables).

Cambridge, MA: MIT Press, 1961.

Tests for Building Confidence

in Simulation Models

Focusing on

STRUCTURE

Focusing on

BEHAVIOR

ROBUSTNESS

• Dimensional consistency

• Extreme conditions

• Boundary adequacy

• Parameter (in)sensitivity

• Structure (in)sensitivity

REALISM

• Face validity

• Parameter values

• Replication of behavior

• Surprise behavior

• Statistical tests

UTILITY• Appropriateness for

audience and purposes

• Counterintuitive behavior

• Generation of insights

Forrester 1973, Forrester & Senge 1980, Richardson and Pugh 1981

Five Habits of Thought thatBlock Political Change

1. The best approach is to make a good model and then find someone who can use it.

2. Problems have simple causes. 3. Everyone will be happy, if you can point

out the solution to a problem.4. People will use your model, if you show

that it is more valid than existing models.5. Change comes quickly, once the solution is

known.

Five Habits of Thought thatPromote Political Change

1. Find the client before you build the model.

2. If a behavior persists, it is not caused by a random or simple set of conditions.

3. Every problem fulfills the goals of some people and organizations. They will fight your efforts to solve it.

4. Models do not cause change; people do.

5. Change requires sustained effort.