Population-Based Networks for Comparative Effectiveness Research:Promises and Potholes

Tracy Lieu, MD, MPHJanuary 8, 2013

Kaiser Permanente Research

Kaiser Permanente is a resource for comparative effectiveness research

2

3.3 million patients 7,000 physicians 21 hospitals 234 medical offices Regional quality

improvement programs

Our Division of Research has common interests with UCSF

50+ research scientists in: Cancer Cardiovascular and metabolic Health care delivery and policy Infectious disease Behavioral health and aging Women’s and children’s health

3

Our research and funding are largely public-domain

Federal

Foundation

TPMGKP Community

Benefit

Central Re-search Commit-

tee awards

Pharma/ biotech

% of total funding in 2011 ($107M)

4

Population-based research networks can facilitate CER

5

Patients drawn from a defined group, representative of the general population

Multiple geographic sites Sites have:

– Computerized data on exposures and outcomes

– Access to clinicians and patients

Networks have supported safety and epidemiologic research

6

Vaccine Safety Datalink (CDC, 1991) Mini-Sentinel (FDA, 2010) Cancer Research Network (NCI) Cardiovascular Research Network

(NHLBI, 2007) Mental Health Research Network

(NIMH, 2009)

In 2010, AHRQ sponsored 11 new networks for CER

7

Examples: Population-Based Effectiveness in

Asthma and Lung Diseases (PEAL) Surveillance, Prevention, and

Management of Diabetes Mellitus (SUPREME-DM)

Scalable Partnering Network (SPAN)

New approaches have increased the power of these networks

8

Distributed data network approaches Example – asthma network for CER Methodologic potholes and potential

solutions Resources for using distributed data

networks for CER

Distributed data networks are versatile

Standard, multi-purpose, multi-institutional infrastructure

Can support both observational and intervention studies

Local data holder control over access and uses of data

Mitigates need to share or exchange protected health information

9

Example: The Population-Based Effectiveness in Asthma and Lung Diseases (PEAL) Network

6 sites with diverse populations Sponsored by AHRQ, 2010-2013 Purpose: Establish infrastructure and conduct

CER in asthma Lay foundation for research in other lung

diseases and in other fields, e.g. pharmacogenetics

10

Collaborators in the PEAL Network

KP Northwest

KP NorthernCalifornia

HealthPartners

Harvard PilgrimHealth Care

Vanderbilt

KP Georgia

www.pealnetwork.org

PEAL Data Warehouse

PEAL databases with common structure

Local databases, some standard (VDW) and others w/varying structure

Population selection and data

warehouse building using

distributed programs and site-specific translation programs

PEAL Virtual Data Warehouse

HPHC

KPNC

KPSE

HPRF

VAND

HPHC

KPNC

KPSE

HPRF

VAND

KPNW KPNW

12

Study-specific analysis programs based on common

data dictionary

Compatible de-identified

datasets from each site

Comparative effectiveness research and other studies

Research Team

PEAL Data Warehouse

PEAL databases with common structure

HPHC

KPNC

KPSE

HPRF

VAND

KPNW

13

Data confidentiality is a key hurdle for data networks

14

Pooling individual level-data poses risk De-identification doesn’t always work Distributed analysis gives stronger

protection -- only aggregated, count data are shared

Example: Vaccine Safety Datalink Project and Congressman Dan Burton

PEAL builds on standard datasets from the HMO Research Network’s Virtual Data Warehouse

Derived from the HMORN VDW New, from source data

Demographics Specialty PrescribingEnrollment Dispensing BenefitsUtilization tables: Geocode & copayment Encounter Vitals Diagnosis Death Procedure

17

The PEAL Network has succeeded in its basic purpose Established understandings –governance, data

use, IRB Created data dictionaries & datasets Identified the study cohorts; descriptive analyses Completed studies of controller medication

effectiveness and statins in asthma Studies of adherence, methodology, cost-

sharing, and insurance benefit design underway18

19

Biases inherent in observational study

designsComparative Effectiveness

Research

Example of potential confounding: Outcomes after leukotriene inhibitors compared with inhaled corticosteroids

Retrospective cohort analysis of >44,000 children with probable persistent asthma

70% filled an inhaled corticosteroid (ICS); 26% filled a leukotriene inhibitor (and not an ICS)

Proportional hazards models Adjusted for age, sex, insurer, asthma risk (prior

ED visits, hospitalizations, oral steroid bursts), Charlson score, comorbidities, and adherence as a time-varying covariate

20

Preliminary findings – confidential: In TennCare, users of leukotriene inhbitors were

less likely to experience an asthma-related emergency department visit (HR 0.7, CI 0.5-0.8) in the next 12 months

In HMO populations, users of leukotriene inhibitors were less likely to have subsequent oral steroid bursts (HR 0.6, CI 0.4 – 0.9)

21

Example of potential confounding: Outcomes after leukotriene inhibitors compared with inhaled corticosteroids

Wu AC, under review

22

Retrospective cohort designs for CER are prone to selection bias (confounding by indication) Patients who receive a newer treatment often

differ from patients who don’t Or, better clinicians or better health care

systems may adopt better interventions sooner Traditional multivariate regression often cannot

resolve this confounding

23

We’re testing analytic approaches to reducing confounding

In the PEAL cohort analysis, we are comparing: Propensity score weighting High-dimensionality propensity scores Proportional hazards regression with time-

dependent covariates Marginal structural models Adding patient-reported information to

computerized data

24

Stronger designs may better reduce confounding

Instrumental variable – find a covariate that is associated with the exposure and not the outcome, and use this to create “randomized” groups – if you are lucky

Difference-in-difference – change in time between intervention and comparison groups

Interrupted time series (regression discontinuity)

25

Temporal trend or intervention effect?

Intervention group

26

Difference-in-difference design can distinguish between temporal trend . . .

Intervention group

Comparison group

27

and intervention effect

Intervention group

Comparison group

28

Interrupted Time Series Design

Wagner AK Ann Intern Med 2007 (from Soumerai S)

Interrupted time series analysisBenzodiazepine (BZ) use and hip fractures in women in Medicaid before and after NY policy restricting BZ use

Cum

ula

tive

Inc

iden

ce o

f H

ip F

ract

ure

per

100

000

Fem

ale

Use

rs b

efor

e P

olic

y

Bz

Use

am

ong

Fe

mal

e U

sers

bef

ore

Po

licy,

%

0

10

20

30

40

50

New YorkNew Jersey

Policy

0

0.005

0.01

0.015

0.02

0.025

1 11 21 31Month

Policy

60% decrease in bz use in NY

No change in risk of hip fracture

2007

M01

2007

M02

2007

M03

2007

M04

2007

M05

2007

M06

2007

M07

2007

M08

2007

M09

2007

M10

2007

M11

2007

M12

2008

M01

2008

M02

2008

M03

2008

M04

2008

M05

2008

M06

2008

M07

2008

M08

2008

M09

2008

M10

2008

M11

2008

M12

2009

M01

2009

M02

2009

M03

2009

M04

2009

M05

2009

M06

2009

M07

2009

M08

2009

M09

2009

M10

2009

M11

2009

M12

2010

M01

2010

M02

2010

M03

2010

M04

2010

M05

2010

M06

2010

M07

2010

M08

2010

M09

2010

M10

2010

M11

2010

M12

0

20

40

60

80

100

120

140

160

180

Number of albuterol inhalers dispensed before and after an increase in co-payment due to branding

changes – Preliminary data, confidential:

Cases (changed to brand cost-sharing)

Controls (kept generic cost-sharing)

num

ber o

f inh

aler

s pe

r 1,0

00 c

hild

ren

Policy change

31

Comparative effectiveness research:Is there hope for this half-baked cake?

32

• Observational comparative effectiveness research (including quasi-experimental designs)

• Interventional comparative effectiveness research

• Delivery science / implementation research

Population-based networks are useful for:

33

You can also use population-based networks for:

• Epidemiology, including genetic epidemiology

• Safety surveillance• Identifying patients with specific

conditions, especially uncommon ones, for all types of studies

Population-based research data may be useful for clinical system needs

Research Data

Warehouses & Data Marts

Clinical and Operational

Userscollaborative research

direct access

direct distribution

report repositoryresearch

staff

Firewall

reports

35

Electronic Data Methods (EDM) forum is a national resource

• Facilitates learning across AHRQ projects that build infrastructure for comparative effectiveness research

• Led by AcademyHealth with AHRQ support

• Holds stakeholder symposia

• Organizes reports on specific topics, e.g. building cohorts for research, deidentifying data

36

37

38