Propensity Scores

Propensity Scores

October 2014

Alexander M. Walker MD, DrPHExtensive parts of this presentation incorporate the work of John D. Seeger, PharmD, DrPH

“In mathematics, you don't understand things.

You just get used to them.”

- John von Neumann

1903-1957

Research Goal

Compare two treatments with respect to a health or economic outcome

“Counterfactual” ideal If the same people had received B instead of A, how

would their outcomes have differed? What is achievable: “similar” not “same”

Comparable treatment groups … insofar as you can tell!

4

Pictures for Confounding

Comparison of Heterogeneous Groups

5

E1 E2

6

E1 E2

Internal Composition May Differ

7

Affectedindividuals

E1 E2

Risks that Depend on Subgroup Status

8

Affectedindividuals

50%

15%

50%

15%

E1 E2


Note that the proportions of affected individuals are different in the yellow and gray groups …

9

50%

15%

50%

15%

E1 E2


Note that the proportions of affected individuals are different in the yellow and gray groups …

… and that the group-specific risks are identical in the two exposed populations E1 and E2.

10

The differences in risk are due to the covariate

structure of compared

populations, not to the effects of

E1 and E2

E1 E2

Internal Risk Factor Heterogeneity Creates an Differences in Group Risk

11

Propensity Scores to Create Populations with Similar Covariate Structure

12

E1 E2

Covariate Heterogeneity

E1 has more

Yellow

E2 has more

Gray

13

E2

E1

E1

E2

Gray

predicts E2

Yellow predicts

E1

Covariate Status as a Predictor of Treatment

14

Propensity Scores

PS is the predicted probability of treatment, given all the covariates

Matching on the PS creates study populations that have balance on the covariates

Perfect for a single, dichotomous covariate Not perfect, but very good for multiple covariates

15

E1 E2

Propensity for Covariate PatternsThink of orange and green as two distinct covariate patterns that have the same predicted Pr(E1).

Pr(

E1

|Ora

ng

e)=

xP

r(E

1|G

ree

n)

=x

16

E1 E2

Gathering subjects with identical propensity puts all individuals with covariate patterns orange and green into the same stratum.

Conditioning on Propensity Permits Unconfounded Comparisons

At a given propensity level x, there is no association between treatment and covariate patterns.

Pr(

E1

|Ora

ng

e)=

xP

r(E

1|G

ree

n)

=x

Formal Expression

Propensity(x) P(T=1|x) = E(T|x)

The propensity associated with level x of the covariate X is the probability that treatment is present (equivalently, is “B” as opposed to “A”), given level x, and this is in turn equal to the expected value of treatment, given x.

Note that the definition does not specify the parametric form of the Propensity(x) . The examples in this talk use a logistic function; others -- including nonparametric functions -- are also used.Notation. A single capital letter denotes a variable, a single

lower case letter denotes a particular value for that variable.

Probability Calculus

Under propensity matching, how do X (covariate status) and T (treatment status) relate to one another?

1.Pr( x, t | p ) = Pr( x | p ) Pr( t | x, p ) Probability Theory

2.Pr( t | x, p ) = Pr( t | p ) p incorporates all information about t

that is in x

Pr( x, t | p ) = Pr( x | p ) Pr( t | p )

Pr( x, t | p ) = Pr( x | p ) Pr( t | p )

Given a particular value of the propensity score variable, that is at P=p, the covariates X and T are uncorrelated.

At particular levels of P individually and therefore collectively (i.e. “conditionally on P”), the lack of correlation guarantees that X cannot confound the association between T and any outcome.

20

Matching on Propensity Scores

Propensity Matching: Method

Identify candidate predictors of treatment B v A Perform a logistic regression of B v A Obtain from the regression a “predicted” probability of B v

A Sort all members of A and B according to this propensity Match A patients to B patients on the propensity

Duragesic and Long-Acting Opioids

Duragesic LA Opioids

N 504 2,201

65+ years 29% 10%

Male 35% 49%

Periph Vasc Disease 4% 1%

Sx of Abd or Pelvis 18% 10%

> 2 hospitalztns 6 mo 9% 3%

30 days NonRx Costs $1,136 $746

Straightforward Regressionproc logistic data = mother.propensity2 descending; model DuragesicUser = DischCostIndex EncCostIndex RxCostIndex OtherCostIndex RxCostPrior1 OtherCostPrior1 AnyRx OneDisch TwoDisch ThreePlusDisch AnyICD443 AnyICD719 AnyICD724 AnyICD787 AnyICD789 q3_95_new q4_95 q1_96 q2_96 q3_96 q4_96 q1_97 q2_97 q3_97 q4_97 q1_98 q2_98 q3_98 q4_98 hmo men young old /rl; where enrbaseflag = 1 and validindex = 1 and sameday = 0 and medicare = 0 and malignant = 0; output out = mother.propensity3 p = score ; run;

Propensity Output

Obs PATIENT score 1 2909 0.57475 2 3438 0.328993 3841 0.013244 5674 0.484115 5734 0.068926 8573 0.117807 10210 0.346928 13056 0.097379 13376 0.3735010 16026 0.1563511 16865 0.0610612 16949 0.10568

25

E1

Pr(E1)=x

Pr(E1)=x

E2(sample) E2 (residual)

Choose from E2 a sample that matches E1 in size.

Matching on Propensity

26

E1

Pr(E1) = 0.5

Pr(E1) = 0.5

E2At every level of propensity in the constructed cohorts, Pr(E1) = 0.5. Therefore, treatment is uncorrelated with propensity, and you can collapse all the propensity-matched groups together to form a cohort in which all covariate patterns are uncorrelated with treatment, and there will be no confounding bias.

Matching on Propensity

Stratum I II III IV V

Stratum I II III IV V

Duragesic and Long-Acting Opioids


N 504 2,201

65+ years 29% 10%

Male 35% 49%




30 days NonRx Costs $1,136 $746

Propensity-Matched Cohorts


N 478 478

65+ years 26% 25%

Male 36% 33%




30 days NonRx Costs $1,084 $1,043

Pharmacoepidemiol Drug Saf. 2005 Jul;14(7):465-76.

Do Statins Affect Risk of AMI?

The purpose of the study was to assess whether statins affect the risk of risk of acute myocardial infarction (AMI)

Strong predictors for statin use that affect risk of AMI

How to design an observational study?

Note: we would not ordinarily use observational data for efficacy questions, but this serves as a suitable test case because there is a known gold standard

+Risk Factors: age (45M, 55F), diabetes, smoking, HTN, low HDL,family history of premature CHD-Risk Factor: high HDL

Risk Category LDL to initiate drug Tx

LDL Goal of drug Tx

No CHD and <2 Risk Factors

³ 190 <160

No CHD and ³ 2 Risk Factors

³ 160 <130

With CHD >130 £100

NCEP ATP II guidelines (1993)

Good Clinical Practice Creates Confounding

Gold Standard for the Effect of StatinsC

AR

E T

rial R

esu

lts

Sack

s FM

, et

al

N E

ngl J M

ed

. 19

96;3

35:1

001-9

Data Source

• Fallon Community Health Plan• Central Massachusetts HMO• ~200,000 members• Claims Data available on:

– Enrollment (age, sex, date)– Ambulatory care visits– Hospitalization– Pharmacy dispensings (drug & quantity)– Laboratory tests (tests & results)

Patient Entry, Analytic Sequence

1993 1994 1995 1996 1997 1998 1999

1 of 9 Blocks

1) Apply eligibility criteria• FCHP member for at least 1 year• At least one physician visit in last year• LDL, HDL, TG levels in last 6 months• At least one physician visit in cohort accrual block• No PAD diagnosis before index date• Not current statin user

2) Estimate propensity score (statin initiation)3) Match statin initiators with non-initiators4) Repeat for all blocks of time5) Follow matched groups for diagnosis of MI

2nd/94

~35,000Members

All Fallon members with any LDL > 130 mg/dl

Require 1 yearEnrollment

Current Statin Users (1501)

Statin Initiators, Eligible (77)

Statin Initiators, Not Eligible (34)

Non Statin Users, Not Eligible (24,799)

Non Statin Users, Eligible (9,639)

Month of1/1/94

PropensityScoreMatching

Total subjects in cohort (36,050)

“Typical” Statin Initiator and Non-Initiator Variable Init iators Non- Init iators P-value Lipid-related labs 26.04 13.58 <0.0001 Different Prescription Drugs 5.02 2.88 <0.0001 LDL level (mg/dl) 180.25 155.08 <0.0001 Triglyceride level (mg/dl) 202.66 166.91 <0.0001 Cardiovascular-related Prescription Drugs 0.59 0.26 <0.0001 Cardiovascular-related vis its 1.11 0.27 <0.0001 Age (years) 62.04 58.02 <0.0001 Physician Visits 7.69 6.31 <0.0001 Ischemic Heart Disease 20.27% 5.57% <0.0001 HDL level (mg/dl) 43.29 46.60 <0.0001 Cardiovascular-related diagnoses 0.28 0.12 <0.0001 Cardiovascular-related hospitalizations 0.56 0.13 <0.0001 MI 11.58% 2.92% <0.0001 Angina 11.92% 3.14% <0.0001 Unstable Angina 10.59% 2.22% <0.0001 Smoking 25.80% 18.10% <0.0001 Hypertension 19.96% 12.96% <0.0001 Labs 10.48 10.74 0.0074 Hospitalizations 0.22 0.08 <0.0001 Male 53.14% 47.61% <0.0001

111% (46%-204%)Risk Increase

Statin Non-Initiators

Statin Initiators

Months of Follow-Up

Cu

mula

tive Inci

den

ceMI Outcome (Unmatched)

HR=2.11 (1.46-3.04)

Calculate Propensity Score

Predict treatment Statin initiation vs. not In each 6-month period of cohort accrual

Using baseline covariates Obtain fitted values from regression Fitted value for each study subject is the Propensity

Score

Construct Rich Model

More than 8 events per covariate leads to unbiased estimates

Many more persons exposed to drug of interest than study outcomes

In Drug Safety studies, usually the outcome is rare

Therefore can control for more covariates when exposure is dependent variable than when outcome is

Cepeda S, et al. Am J Epidemiol 2003;158:280-287.

Propensity modeling permits the introduction of many, many covariates

*build model for 9501;proc logistic descending data=new1; model statin = male smok obes age9501 ang9501 usa9501 chf9501 isch9501 ath9501 cva9501 usa9501 mi9501 olmi9501 htn9501 tia9501 afib9501 ascv9501 hth9501 ost9501 cvs9501 htdx9501 circ9501 cond9501 rvsc9501 hhd9501 dysr9501 hrt9501 ns9501

ins9501 diab9501 skca9501 depr9501 adj9501 schz9501 deb9501 rheu9501 days9501 lres9501 tres9501 hres9501 hbac9501 cvhp9501 ekg9501 cvrx9501 cvvs9501 llab9501 lab9501 cvdg9501 hosp9501 rx9501 vist9501 diag9501; output out=psmodel pred=PROPSCORE; run;

Propensity Regression Parameter Estimates

Obs ID STATIN PROPSCORE 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...

Output File – Propensity Scores

45

Obs ID STATIN PROPSCORE 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...

Match exposed to non-exposed according to Propensity Score values

Obs ID STATIN PROPSCOR 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...


Obs ID STATIN PROPSCOR 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...


Propensity Score Distribution (After Matching)

0

50

100

150

200

250

300

350

400

450

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Propensity Score

Nu

mb

er o

f P

erso

ns

Statin Initiators Non-Initiators

Balance Achieved by Matching Matched at 0.01 Propensity Score Variable Initiators Non-Initiators N=2901 N=2901 P-value Lipid-related labs 24.90 24.64 0.4987 Different Prescription Drugs 4.57 4.54 0.7639 LDL level (mg/dl) 177.84 177.58 0.7837 Triglyceride level (mg/dl) 200.34 200.50 0.9626 Cardiovascular-related Prescription Drugs 0.51 0.51 0.9367 Cardiovascular-related visits 0.74 0.83 0.1249 Age (years) 61.47 61.68 0.5030 Physician Visits 7.25 7.27 0.8732 Ischemic Heart Disease 15.13% 15.48% 0.7428 HDL level (mg/dl) 43.51 43.55 0.9079 Cardiovascular-related diagnoses 0.21 0.23 0.2145 Cardiovascular-related hospitalizations 0.40 0.39 0.7929 MI 7.89% 8.69% 0.3169 Angina 8.51% 8.72% 0.8151 Unstable Angina 7.14% 7.31% 0.8393 Smoking 23.85% 24.27% 0.7355 Hypertension 16.58% 17.99% 0.1649 Labs 10.45 10.48 0.7874 Hospitalizations 0.16 0.16 0.6915 Male 52.33% 52.09% 0.8747

Only 1 of 52 variables sig. different at P<0.05

31% (7%-48%)Risk Reduction

Statin Non-Initiators

Statin Initiators

Months of Follow-Up

Cu

mu

lati

ve I

ncid

en

ce

MI Outcome (After Matching)

HR=0.69 (0.52-0.93)

Interpreting Propensity Coefficients

53

Odds Ratio for Non-Statin Lipid Lowering Therapy as a Predictor of Statin Initiation

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

9402 9501 9502 9601 9602 9701 9702 9801 9802

Matching Block (Year and Half)

Od

ds

Ra

tio

LDL as a Predictor of Statin Initiation

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

190+175-190160-175145-160130-145<130

LDL Category

Od

ds

Ra

tio

fo

r In

itia

tio

n

Age as a Predictor of Statin Initiation

0.00

2.00

4.00

6.00

8.00

10.00

12.00

75+65-7555-6545-5534-45<35

Age Category

Od

ds

Ra

tio

fo

r In

itia

tio

n

When Is the Model Sufficient?

57

Early Matching Results

New Variables Suggested post hoc for the Propensity Score

Cardiac Disease Cardiovascular

Diagnoses Hospitalizations Outpatient visits Medications

EKGs Number of labs Number of lipid labs

Other Causes of “Medicalization”

Schizophrenia Adjustment Disorder Depression Non-Skin CA Skin CA Debility Rheumatic Disease

Imbalance on Non-Included Variables

NIVs are Predictors of Statin Initiation

New Ranking of Predictors# Variable c-stat OR P-value

Lipid lab tests 0.82 1.14 <0.011 Rxs 0.72 1.20 <0.012 LDL 0.71 1.02 <0.013 Triglycerides 0.61 1.00 <0.01

CVRxs 0.60 1.70 <0.01CVVisits 0.59 1.25 <0.01

4 Age 0.58 1.02 <0.015 Visits 0.57 1.04 <0.016 Ischemic Heart Ds. 0.57 4.20 <0.017 HDL 0.57 0.98 <0.01

CVDiagnoses 0.56 1.94 <0.01CVHospitalizations 0.55 1.23 <0.01

8 MI 0.54 4.63 <0.019 Angina 0.54 3.88 <0.0110 Unstable Angina 0.54 5.13 <0.0111 Smoking 0.54 1.55 <0.0112 Hypertension 0.54 1.77 <0.01

Balance on New Variables Original Matching Revised Matching Variable Statin Init.

(N=3579) Non-Init. (N=3579)

P-value Statin Init. (N=2901)

Non-Init. (N=2901)

P-value

Lipid lab tests 26.3 16.4 <0.01 24.9 24.6 0.50 CVRxs 0.51 0.53 0.09 0.51 0.51 0.94 CVVisits 0.96 0.54 <0.01 0.74 0.83 0.12 CVDiagnoses 0.25 0.17 <0.01 0.21 0.23 0.21 CVHospitalizations 0.46 0.31 <0.01 0.40 0.39 0.79 Lab tests 10.3 11.3 <0.01 10.5 10.5 0.79 EKGs 0.48 0.48 0.77 0.48 0.51 0.35 Schizophrenia 2.2% 3.6% 0.03 2.1% 2.2% 0.99 Depression 2.4% 2.9% 0.14 1.9% 2.7% 0.05 Non-skin cancer 5.3% 5.7% 0.47 5.2% 5.5% 0.60 Adjustment dis. 2.7% 3.6% 0.03 2.8% 3.1% 0.59 Skin cancer 2.5% 2.7% 0.71 2.2% 2.4% 0.73 Debility 1.7% 1.5% 0.78 1.6% 1.3% 0.44 Rheumatic dis. 1.3% 1.8% 0.07 1.2% 1.7% 0.19

Thank You!

Documents

Propensity Scores