Relative Efficacy and Safety of Relative Efficacy and Safety of Aprotinin and Tranexamic Acid Aprotinin and Tranexamic Acid

in Cardiac Surgeryin Cardiac Surgery

Keyvan Karkouti, MD, FRCPC, MScKeyvan Karkouti, MD, FRCPC, MScToronto General HospitalToronto General HospitalUniversity Health NetworkUniversity Health Network

University of TorontoUniversity of Toronto

DisclosureDisclosure

I have received $3,000 from Bayer Inc. for speaking I have received $3,000 from Bayer Inc. for speaking engagements engagements

OverviewOverview

Propensity analysis primerPropensity analysis primer Aprotinin versus Tranexamic Acid - Karkouti et al.Aprotinin versus Tranexamic Acid - Karkouti et al.

Methodology and ResultsMethodology and Results Aprotinin not superior to tranexamic acidAprotinin not superior to tranexamic acid

Major criticisms and putting it all in contextMajor criticisms and putting it all in context Not randomized, therefore “results are irredeemably biased”Not randomized, therefore “results are irredeemably biased” Results not consistent with findings of placebo-controlled RCTs, Results not consistent with findings of placebo-controlled RCTs,

therefore must be wrongtherefore must be wrong My ConclusionsMy Conclusions

No convincing evidence that aprotinin is better, or worse, than No convincing evidence that aprotinin is better, or worse, than alternativesalternatives

Propensity Analysis PrimerPropensity Analysis Primer

Multivariable analyses in observational studiesMultivariable analyses in observational studies

Multiple logistic regression versus propensity analysisMultiple logistic regression versus propensity analysis Newgard et al. Acad Emerg Med 2004;11:953-961Newgard et al. Acad Emerg Med 2004;11:953-961 Shah et al. J Clin Epid 2005;58:550-559Shah et al. J Clin Epid 2005;58:550-559

Logistic Regression vs. Propensity AnalysisLogistic Regression vs. Propensity Analysis

Logistic RegressionLogistic Regression Objective: to reduce bias by Objective: to reduce bias by

adjusting for measured adjusting for measured confoundersconfounders

Models for outcome, covariates Models for outcome, covariates include exposure and include exposure and confoundersconfounders

Get adjusted odds ratio of Get adjusted odds ratio of outcome for exposure outcome for exposure

Propensity AnalysisPropensity Analysis Objective: to reduce bias by Objective: to reduce bias by

adjusting for measured adjusting for measured confoundersconfounders

Models for exposure, covariates Models for exposure, covariates include confoundersinclude confounders

Get a propensity score (PS) for Get a propensity score (PS) for exposure for each patientexposure for each patient

Difference between exposed and Difference between exposed and unexposed with equal PS is an unexposed with equal PS is an unbiased est. of treatment effectunbiased est. of treatment effect

Logistic Regression vs. Propensity AnalysisLogistic Regression vs. Propensity Analysis

Logistic RegressionLogistic Regression Cannot adjust for confounders with Cannot adjust for confounders with

large differences in distribution large differences in distribution between treatment groupsbetween treatment groups

Assumes linear relationship Assumes linear relationship between confounder and outcomebetween confounder and outcome

# of covariates that can be included # of covariates that can be included is limited by # of outcomesis limited by # of outcomes

Rare covariates cannot be includedRare covariates cannot be included

Propensity AnalysisPropensity Analysis Can adjust for confounders with Can adjust for confounders with

large differences in distribution large differences in distribution between treatment groupsbetween treatment groups

No underlying assumptions in No underlying assumptions in modelingmodeling

Unlimited # of covariates can be Unlimited # of covariates can be included because it is used to included because it is used to balance treatment groups, not to balance treatment groups, not to make inferential statements make inferential statements about treatment groupsabout treatment groups

What Makes for a ‘Good’ Propensity Score What Makes for a ‘Good’ Propensity Score Analysis?Analysis?

Exploit its advantages over logistic regressionExploit its advantages over logistic regression Use it appropriatelyUse it appropriately

Use of PS to match exposed/unexposed patients vs. adjusting Use of PS to match exposed/unexposed patients vs. adjusting for PS as part of logistic regression model for outcomefor PS as part of logistic regression model for outcome

Confirm its validityConfirm its validity Must demonstrate if adequately balanced confoundersMust demonstrate if adequately balanced confounders

Realize its limitationsRealize its limitations Does not match for unmeasured confoundersDoes not match for unmeasured confounders

Use of PS as part of logistic regressionUse of PS as part of logistic regression

Newgard et al. Acad Emerg Med 2004;11:953-961Newgard et al. Acad Emerg Med 2004;11:953-961

Aprotinin vs. Tranexamic Acid - Karkouti et al.Aprotinin vs. Tranexamic Acid - Karkouti et al.

Hypothesis: Aprotinin is superior to tranexamic acid; Hypothesis: Aprotinin is superior to tranexamic acid; therefore, it is associated with reduced transfusions and therefore, it is associated with reduced transfusions and improved outcomesimproved outcomes

Site: Tertiary care, aprotinin used in high risk cases onlySite: Tertiary care, aprotinin used in high risk cases only Sample: Consecutive CPB patients from 1999-2004Sample: Consecutive CPB patients from 1999-2004 Data Source: Prospectively collected clinical databaseData Source: Prospectively collected clinical database Analysis: Propensity score matching of aprotinin with Analysis: Propensity score matching of aprotinin with

tranexamic acidtranexamic acid Propensity score derivation model developed for likelihood of Propensity score derivation model developed for likelihood of

aprotinin use using perioperative clinical variablesaprotinin use using perioperative clinical variables

Study ConclusionStudy Conclusion

Study Hypothesis: Aprotinin is superior to tranexamic acid; Study Hypothesis: Aprotinin is superior to tranexamic acid; therefore, it is associated with reduced transfusions and therefore, it is associated with reduced transfusions and improved outcomesimproved outcomes

Study Result: Aprotinin was not associated with reduced Study Result: Aprotinin was not associated with reduced transfusions or improved outcomestransfusions or improved outcomes

Study Conclusion: Reject hypothesisStudy Conclusion: Reject hypothesis

Major criticisms and putting it in contextMajor criticisms and putting it in context

Not randomized, therefore “results are irredeemably biased” Not randomized, therefore “results are irredeemably biased” because of confounding by indicationbecause of confounding by indication Aprotinin group was higher risk, and no statistical analysis can Aprotinin group was higher risk, and no statistical analysis can

fully adjust for this.fully adjust for this. Thus, aprotinin was more effective because transfusion rates were Thus, aprotinin was more effective because transfusion rates were

similar to a lower risk groupsimilar to a lower risk group Thus, aprotinin was not more harmful because a higher renal Thus, aprotinin was not more harmful because a higher renal

dysfunction rate is expected in the higher risk group dysfunction rate is expected in the higher risk group

Are observational studies irredeemably Are observational studies irredeemably biased?biased?

Best evidence on EFFICACY of therapy comes from Best evidence on EFFICACY of therapy comes from randomized trials randomized trials Caveat: Low quality RCTs may overestimate benefits of therapyCaveat: Low quality RCTs may overestimate benefits of therapy

Best evidence on HARM of therapy will often come from large, Best evidence on HARM of therapy will often come from large, properly analyzed nonrandomized trialsproperly analyzed nonrandomized trials RCTs are not ideal for identifying adverse events if:RCTs are not ideal for identifying adverse events if:

Low frequency owing to restrictive inclusion/exclusion criteriaLow frequency owing to restrictive inclusion/exclusion criteria Short follow-up periodsShort follow-up periods Small sample sizesSmall sample sizes

Vandenbroucke (Editorial) CMAJ 2006 174(5)Vandenbroucke (Editorial) CMAJ 2006 174(5)

Randomized versus Nonrandomized StudiesRandomized versus Nonrandomized Studies

Data from observational studies on adverse effects may be as Data from observational studies on adverse effects may be as valid as those from RCTsvalid as those from RCTs Adverse effects often not linked to treatment indicationsAdverse effects often not linked to treatment indications

confounding by indication not an issueconfounding by indication not an issue little need for randomization to quantify themlittle need for randomization to quantify them

The same is not true for beneficial effectsThe same is not true for beneficial effects Beware of observational studies that claim beneficial effects, even Beware of observational studies that claim beneficial effects, even

if unanticipatedif unanticipated Patients taking the drug were sufficiently well to continue taking Patients taking the drug were sufficiently well to continue taking

them over a longer periodthem over a longer period

Vandenbroucke (Editorial) CMAJ 2006 174(5)Vandenbroucke (Editorial) CMAJ 2006 174(5)

Panagiotis CMAJ 2006;175:635-41Panagiotis CMAJ 2006;175:635-41

Comparison of evidence on HARMS of medical interventions in Comparison of evidence on HARMS of medical interventions in randomized and nonrandomized studiesrandomized and nonrandomized studies Targeted medical interventions that had randomized and Targeted medical interventions that had randomized and

nonrandomized studies with > 4000 subjectsnonrandomized studies with > 4000 subjects Compared relative risksCompared relative risks Found 15 harms that could be assessed Found 15 harms that could be assessed

Papanikolaou, P. N. et al. CMAJ 2006;174:635-641

Comparison of relative risk estimates for specific harms of medical interventions from randomized and nonrandomized studies

Randomized versus Nonrandomized StudiesRandomized versus Nonrandomized Studies

Thus: “It may be unfair to invoke bias and confounding to Thus: “It may be unfair to invoke bias and confounding to discredit observational studies as a source of evidence on discredit observational studies as a source of evidence on harms.”harms.”

Caveat: Observational study must be of high qualityCaveat: Observational study must be of high quality Large sample sizeLarge sample size Proper adjustment for baseline differences to reduce Proper adjustment for baseline differences to reduce

confounding by indicationconfounding by indication TransparentTransparent

Example: Risk of platelet transfusion in cardiac surgeryExample: Risk of platelet transfusion in cardiac surgery

Spiess et al. Transfusion 2004Spiess et al. Transfusion 2004

Used data from phase III RCT aprotinin studiesUsed data from phase III RCT aprotinin studies N = 284 platelets; 1436 no plateletsN = 284 platelets; 1436 no platelets Logistic regression and propensity analysisLogistic regression and propensity analysis Adjusted for: Adjusted for: age, sex, race, weight, history of diabetes, history of unstable

angina, previous history of coronary artery disease (CAD), history of hypertension, history of chronic obstructive pulmonary disease (COPD), history of congestive heart failure (CHF), New York Heart Association classification (identified as ≥II [NYHA II], ≥III [NYHA III], ≥IV [NYHA IV]), left ventricular ejection fraction less than 30 percent, left ventricular ejection fraction less than 50 percent, return to surgery for reexploration for surgical bleeding, return to surgery for reexploration for diffuse bleeding, return to surgery for any reason, left ventricular assist device, volume of reinfused shed mediastinal blood, RBC transfusion (yes/no), duration of surgery, total heparin dose, total protamine dose, minimum intraoperative Hct, minimum intraoperative Hb.

Platelets Associated with Adverse OutcomesPlatelets Associated with Adverse Outcomes

Karkouti et al. CJA 2006Karkouti et al. CJA 2006

Used data from our large databaseUsed data from our large database N = 2174 platelets, 9285 no plateletsN = 2174 platelets, 9285 no platelets Logistic regression and propensity analysisLogistic regression and propensity analysis When adjusted for same variables as previous study, platelets When adjusted for same variables as previous study, platelets

were associated with LOS, renal failure, and deathwere associated with LOS, renal failure, and death Then we adjusted for three additional variables: baseline Then we adjusted for three additional variables: baseline

platelet count, difficult wean from CPB, and massive blood platelet count, difficult wean from CPB, and massive blood lossloss

Platelets NOT Associated with Adverse Platelets NOT Associated with Adverse OutcomesOutcomes

Major criticisms and putting it in contextMajor criticisms and putting it in context

Not randomized, therefore “results are irredeemably biased”* Not randomized, therefore “results are irredeemably biased”* because of confounding by indicationbecause of confounding by indication Thus, this assertion is irredeemably invalidThus, this assertion is irredeemably invalid However, we agree that neither our study, nor the study by However, we agree that neither our study, nor the study by

Mangano et al., is conclusive Mangano et al., is conclusive Not yet reproduced Not yet reproduced Cannot rule out the effect of unmeasured confoundersCannot rule out the effect of unmeasured confounders

• The results of our study on 3500 patients who had surgery at seven The results of our study on 3500 patients who had surgery at seven Canadian hospitals in 2004 highlight the importance of the site-effect Canadian hospitals in 2004 highlight the importance of the site-effect in multi-center observational studiesin multi-center observational studies

*Weiskopf RB. J Thromb Haemost 2066;4:2074-8

Major criticisms and putting it in contextMajor criticisms and putting it in context

Results not consistent with findings of placebo-controlled Results not consistent with findings of placebo-controlled RCTs, therefore must be wrongRCTs, therefore must be wrong

Placebo-controlled RCTsPlacebo-controlled RCTs

Levi et al. Lancet 1999;354:1940-47

Placebo-controlled RCTsPlacebo-controlled RCTs

Brown NEJM 2006

What about head-to-head studies?What about head-to-head studies?

Meta-analysis: Carless et al. BMC Cardiovascular Disorders Meta-analysis: Carless et al. BMC Cardiovascular Disorders 2005,5:192005,5:19

20 head-to-head RCTs identified20 head-to-head RCTs identified Total N = 2430Total N = 2430 Median trial arms N = 25; Range = 14-522Median trial arms N = 25; Range = 14-522 Methodological quality = PoorMethodological quality = Poor Very little high-risk data (e.g., primary CABG = 12 studies)Very little high-risk data (e.g., primary CABG = 12 studies)

24-hour blood loss24-hour blood loss

RBC Transfusion RateRBC Transfusion Rate

37.2%37.2% 36.5%36.5%

Other OutcomesOther Outcomes

Uninformative because the data are sparseUninformative because the data are sparse

Major criticisms and putting it in contextMajor criticisms and putting it in context

Results not consistent with findings of placebo-controlled Results not consistent with findings of placebo-controlled RCTs, therefore must be wrongRCTs, therefore must be wrong Our results are consistent with totality of existing evidenceOur results are consistent with totality of existing evidence

ConclusionsConclusions

No conclusive evidence exists that aprotinin is better, or No conclusive evidence exists that aprotinin is better, or worse, than tranexamic acidworse, than tranexamic acid

There is a glaring lack of high-quality data in high-risk patients There is a glaring lack of high-quality data in high-risk patients that the BART trial will only partly resolvethat the BART trial will only partly resolve