Personalised medicine: Endovascular aneurysm repair risk assessment model using 8 preoperative variables Mary Barnes 1 Robert Fitridge 2, Maggi Boult 2

Personalised medicine: Endovascular aneurysm repair risk assessment model using 8 preoperative variables

Mary Barnes1 Robert Fitridge2, Maggi Boult2

1 CSIRO Mathematical & Information Sciences

2 University of Adelaide Department of Surgery

November 2009

CSIRO. Personalised medicine: ERA model

Imagine you visit surgeon

• Age• Gender• Blood results – Creatinine• Pre-existing conditions – how sick-ASA• Preliminary scans - Aneurysm diameter


Ideally you’d get

Personalised predictions

Early Death 1%

Aneurysm Related Death 8%

Mid-term Re-interventions 4%

Initial Endoleak Type I 1%

Mid-term Endoleak Type I 14%

3 year Survival 80%

5 year Survival 68%

Predicted Outcome Rates


Abdominal aortic aneurysm

Aneurysm = localised dilation of a blood vessel.

Aortic aneurysm large artery from the heart

Bulges like an old worn tire.

http://hcd2.bupa.co.uk/images/factsheets/Abdominal_aortic_Aneurysm_427x240.jpg


Endovascular stent graft – over 1800/year in Aust.


Australian Audit of Endovascular aneurysm repair Royal Australasian College of Surgeons

• Mid to long term safety and effectiveness of the new procedure

• 961 cases Nov 1999 - May 2001 Australia

• 98% follow-up (to mid 2006)

• Mortality data obtained from AIHW National Death Index

• My role – Statistical analysis of audit

EVAR- Endovascular aneurysm repair


Eight preoperative variables in model

Definitions in subsequent slideswww.health.adelaide.edu.au/surgery/evar/predictive.html

Size

Next slide- Fitness

Kidneys -Renal function

Mild <40˚ Severe>60˚

Short necks difficult

Aneurysm Dia. Maximum

Age

ASA

Gender

Creatinine

Aortic Neck angle

Infrarenal Neck Diameter

Infrarenal Neck Length

40 - 80mm

55 - 90yrs

1 - 4

60-200 µmoles/L

degrees

17 - 32mm

6 - 45mm

http://www.health.adelaide.edu.au/surgery/evar/predictive.html


ASA & Creatinine

Assess fitness of patients before surgery American Society of Anesthesiologists

I. A normal healthy patient. II. A patient with mild systemic disease. III. A patient with severe systemic disease. IV. A patient with severe systemic disease that is a

constant threat to life. V. A moribund patient who is not expected to survive

without the operation.

Creatinine measures renal/kidney function 60 poor 200 good

http://en.wikipedia.org/wiki/Surgery

http://en.wikipedia.org/wiki/American_Society_of_Anesthesiologists

http://en.wikipedia.org/wiki/American_Society_of_Anesthesiologists

http://en.wikipedia.org/wiki/Patient

http://en.wikipedia.org/wiki/Systemic_disease

http://en.wikipedia.org/wiki/Disease

http://en.wikipedia.org/wiki/Life

http://en.wikipedia.org/wiki/Death

http://en.wikipedia.org/wiki/Surgery


Definition of variables

Infrarenal Neck LengthInfrarenal Neck Diameter

Size -Maximum Aneurysm Diameter

Aortic neck angle α

α

bifurcation


Key Outcome Measures

• Perioperative mortality (Early death within 30-days)

• Aneurysm related death

• Re-intervention during follow-up

• Type I Endoleak - initial (within 30 days)

- mid-term (6 months - 5 yrs)

• Survival - 3 year

- 5 year


Two-stage predictive ERA modelEndovascular aneurysm repair Risk Assessment

Stage I (based on pre-CT data)• Age• Gender• ASA• Creatinine• Aneurysm diameter

Prediction of Survival at 3 + 5 years and early deaths (perioperative mortality)


Outcome: before angiography (CT scan)

Aneurysm Dia. Maximum 80mm

Age 84years

ASA 4

Gender Male

Creatinine 160µmoles/L

Male

At first surgeon visit have first 5 pre-operative variables


Two-stage predictive ERA model

Visit 2 (after CT scan data)• aortic neck angle• aortic neck length• aortic neck diameter

Provides more detailed personalised predictions

Changes endoleak, re-intervention, graft complication and migration likelihoods


Why develop a predictive model?

Some initial reluctance

Assist preoperative decision makingPredicted survival & outcome rates

Assess risk for particular patient Explain variation in outocmes

Perioperative mortailityEarly Deaths-within 30 days

2% Australian audit6.3% ASA IV in Aust. audit - Sicker patients

1.7% in EVAR-1 - UK trial patients fit for open repair 9% in EVAR-2 - UK trial patients UNFIT for open repair

EVAR- Endovascular aneurysm repair


Statistical detail of model

• Model developed in S-Plus Insightful• Stepwise binomial regressions with logit link• Both backwards and forwards stepwise used to be

sure• AIC criteria used as include terms• Confidence intervals were calculated using

covariance matrix• Results were back transformed onto natural scale for

ease of interpretation• Credible limits used based on Australian audit


Statistical detail of model cont.

The binary logit of a number p between 0 and 1 is given by the formula:

eglogit(Survival5yr) = -8.5575 + 0.0219size + 0.0553Age +

0.5810ASA + 0.0065Creat Back transform to the original measurement scale

exp(logit)/(1+exp(logit))

)1

log()(itlogp

pp


Confidence Intervals

Var(logit) = dTCd

Where

d – data in column format

C – covariance matrix regression

Standard Error

se(logit) = sqrt( Var(logit) )

Confidence intervals (CI) on logit scale

CI_logit = logit + 2 se(logit)

Back transform

CI = exp(CI_logit)/(1+exp(CI_logit))


Regression p-values for primary outcomes

Variables included in each model list likelihood ratio p-values p-values displayed but AIC determined term inclusion

Preoperative variable

Aneurysm Diam. Age ASA Gender

Creat-inine

Aortic neck

angle

Infrarenal neck

diam.

Infrarenal neck

lengthOutcome

3 year survival <0.001 <0.001 <0.001 0.002

Aneurysm related death <0.001 0.030

Early death 0.001 0.070

Initial re-interventions 0.057

Mid-term re-interventions 0.045 0.029 0.014

Initial endoleak type I 0.007

Mid-term endoleak type I 0.005 0.130


Credible ranges- preoperative variables

Aneurysm Dia. Maximum

Age

ASA

Gender

Creatinine

Aortic Neck angle

Infrarenal Neck Diameter

Infrarenal Neck Length

40 - 80mm

55 - 90yrs

1 - 4

60-200 µmoles/L

degrees

17 - 32mm

6 - 45mm

If patient measures are beyond the common ranges, the closest bound of the ranges is used to predict the likelihood.

For example the common age range is 55-90 years. Predictions for a 40 year old are made for a 55 year old in the audit.


External validationSt Georges UK data compared to Australian

UK data

Australian data

Male ratio 90% 86%

Mean age 77.4 75

ASA III 48% 59%

ASA IV 27% 6%

Mean aneurysm size 64mm 58mm

Aneurysms <55mm 19% 44%

Mean creatinine (µmoles/L)

118 115

Infrarenal neck length <15mm

28% 10%

Infrarenal neck diameter (mm)

23.7 23.6

Aortic neck angle >45 degrees

30% 16%

St George’s patients generally are sicker (higher ASA), have larger aneurysms, have more difficult anatomy and are more likely to die than the original cohort of Australian patients


External validation St George’s Vascular Unit London 312 patients

Despite data differences, models for deaths, survival & mid-term type I endoleaks performed better than Australian patients (R2)

R-squared

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Earlydeath

Aneur.relatedDeath

Survive3yr

Initialendoleak

type I

Mid termendoleak

type I

Mid termInterv.

Australian

St George's


Area under ROC curve

00.10.20.30.40.50.60.70.80.9

Earlydeath

Aneur.relatedDeath

Survive3yr

Initialendoleak

type I

Mid termendoleak

type I

Mid termInterv.

Australian

St George's


Goodness of fit summary table using val.prob Frank Harrell’s Design library

Area under ROC close to 1 suggests a good model.


Outcome: before angiography (CT scan)


Age 84years

ASA 4

Gender Male


Male

Early Death 7% Ideally 3% 14%

Aneurysm Related Death 15% 0% 7% 29%

Mid-term Re-interventions 14% 9% 22%

Initial Endoleak Type I 5% 3% 10%

Mid-term Endoleak Type I 9% 6% 15%

3 year Survival 38% Ideally 27% 50%

5 year Survival 23% 100% 16% 33%

Predicted Outcome Rates95% Confidence

Interval


Outcome: after CT angiography

Predictions changed

after scans

Aneurysm Dia. Maximum 80

Age 84

ASA 4

Gender Male

Creatinine 160

Aortic Neck angle 70

Infrarenal Neck Diameter 32

Infrarenal Neck Length 6Have you got all 8 above? All 8

Male

All 8

14%

5%

9%

Early Death 7%

Aneurysm Related Death 15%

Mid-term Re-interventions 17%

Initial Endoleak Type I 7%

Mid-term Endoleak Type I 15%

3 year Survival 38%

5 year Survival 23%

Predicted Outcome Rates

Pre


Outcome for healthier female

Early Death <1% Ideally 0% 1%

Aneurysm Related Death <1% 0% 0% 1%

Mid-term Re-interventions 4% 2% 9%

Initial Endoleak Type I 1% 0% 3%

Mid-term Endoleak Type I 1% 1% 3%

3 year Survival 96% Ideally 92% 98%

5 year Survival 90% 100% 82% 95%


Interval


Age 55years

ASA 2

Gender Female


Aortic Neck angle 10degrees

Infrarenal Neck Diameter 17mm

Infrarenal Neck Length 45mm

Female


Summary

• Original 7-year study resulted in development of ERA model

• Generates personalised predictions to informed decision-making and counselling (before and after CT scan)

• Surgeons liked using Excel rather than learning another software• Increasing use

250 downloads of the spreadsheet in about two years • Basic model - room for improvement

• Potential to develop other models using this approach• NHMRC funding provided to evaluate & improve model


Current & future directions

• NHMRC 5-year grant to assess & improve ERA model• 2009-2013• Comprehensive data set, including biomarkers, to evaluate

additional potential success predictors• 1000 elective and non-urgent EVAR patients over 2 years,

with follow-up for 3-5 yearshttp://

www.health.adelaide.edu.au/surgery/evar

• NZ ethics approval most streamlined

• External validation of model• Imperial College London• EVAR trial

• Medtronic trial (application recently submitted)

http://www.health.adelaide.edu.au/surgery/evar

http://www.health.adelaide.edu.au/surgery/evar

Contact UsPhone: 1300 363 400 or +61 3 9545 2176

Email: [email protected] Web: www.csiro.au

Thank you

CSIRO Mathematics, informatics and StatisticsMary Barnes

Phone: +61 8 8303 8765Email: [email protected] reports: www.surgeons.org/asernip-s/audit.htm Model & NHMRC grant: health.adelaide.edu.au/surgery/evar

M B Barnes, M Boult, G Maddern, R Fitridge. A Model to Predict Outcomes for Endovascular Aneurysm Repair Using Preoperative Variables. European Journal of Vascular and Endovascular Surgery. Volume 35, Issue 5, May 2008, Pages 571-579

http://www.surgeons.org/asernip-s/audit.htm

http://health.adelaide.edu.au/surgery/evar

http://www.sciencedirect.com/science?_ob=PublicationURL&_tockey=%23TOC%236785%232008%23999649994%23684578%23FLA%23&_cdi=6785&_pubType=J&_auth=y&_acct=C000056895&_version=1&_urlVersion=0&_userid=2322062&md5=01e55852b8523c99d60f505f3eb18563


Biomarkers – potential markers of AAA progression

• Osteoprotegerin (OPG)• Osteopontin (OPN)• Macrophage derived chemokine (MDC)• Interleukin-6 (IL-6)• Interleukin-10 (L-10 )• Resistin• Also DNA for genotype analysis


Disclaimer hidden text

Likelihoods based on audit of endoluminal repair of Abdominal Aortic Aneurysms Involving 961 patients in Australia. Procedures between 1999 and July 2001. Follow-up collected up to end August 2006.

Enter Patient details in green cells

Early Death 7% Ideally 3% 15%

Aneurysm Dia. Maximum 80mm Aneurysm Related Death 15% 0% 7% 29%

Age 85years Mid-term Re-interventions 7% 3% 17%

ASA 4 Initial Endoleak Type I 1% 0% 3%

Gender Female Mid-term Endoleak Type I 14% 7% 24%

Creatinine 200µmoles/L 3 year Survival 33% Ideally 22% 45%

Aortic Neck angle 75degrees 5 year Survival 18% 100% 12% 27%

Infrarenal Neck Diameter 30mm

Infrarenal Neck Length 40mm Technical Success 89% Ideally 78% 95%

Have you got all 8 above? All 8 Initial Clinical Success 80% 100% 71% 87%

Initial Endoleak Type II 4% Ideally 1% 12%

Mid-term Endoleak Type II 12% 0% 7% 20%Initial Graft Complications 35% 25% 48%

Mid-term Graft Complications 18% 11% 27%

Initial Re-interventions 37% 30% 45%

Migrations 10% 3% 29%

Convert to Open Repair 4% 1% 13%

Ruptures 6% 3% 10%Caution needs to be exercised when using this model, it does not necessarily predict what will happen for any individual patient.


Interval

Female

All 8

CSIRO. Personalised medicine: ERA model Aneurysm Maximum Diameter

Ane

urys

m R

elat

ed D

eath

Rat

es

30 40 50 60 70 80 90

0.0

0.05

0.10

0.15

Predicted ASA IIPredicted ASA IIIPredicted ASA IV

Actual ASA IIActual ASA IIIActual ASA IV

Graphical presentations difficult to interpretAneursym Related Deaths Model

Aust. R2 = 0.11

Break into categoriesPlot 2 variable models


Receiver Operating Characteristic curves

Sensitivity versus 1- specificity

http://www.medcalc.be/manual/roc.php


Final thoughts

Acknowledge•Contributing Vascular Surgeons in Australia•NHMRC •Royal Australasian College of Surgery

Tips in Excel -Disclaimer hidden text -Matrix multiplications functions

Frank Harrell’s library handy for assessing fit of UK data


NHMRC Study procedure*

Visit 1Pre-op

Visit 2Peri-op

Visit 36 week

Visit 46 M

Visit 512 M

Visit 624M

Visit 736M

Informed consent X

Patient demographics X

Inclusion/exclusion criteria X

Medical/surgical history X

Physical examination X

Vital signs X X X X X X

CT scan or ultrasound X X X X X X

Adverse event recording X X X X X

Concomitant medications X X X X X X

Blood biochemistry including creatinine, complete blood count (+/- biomarkers)

X X X X X X

Peri-operative data collection X

*Flow-charts available


Key Outcome rates (Australian data)

Outcome %

Perioperative deaths 1.8%

Aneurysm related deaths 2.5%

Mid-term interventions 13%

3 year Survival 81%

5 year Survival 68%

Endoleak – Type I

Initial 2.5%

Mid-term 4.5%

Endoleak – Type II

Initial 7%

Mid-term 14%


Significance of Predictors

Table shows Chi-squared p-value for terms included ONE AT A TIME with intercept in binomial (logit link) regression model.

Mig

ration

s

Predictor Variable

Initial

Mid-term

Initial

Mid-t

Mid-term

Initial

Mid-term

Initial

Mid-term

Initial

Mid-term

Ea

rly

La

te

Aneurysm Diameter Max 0.020 0.155 0.303 0.035 0.261 0.002 0.006 0.011 0.056 0.032 0.769 0.626 0.147 0.569 0.004 <.001

Graft Type (eg bifurcated) 0.369 0.012 0.145 0.008 0.041 0.612 0.466 0.226 0.213 0.388 0.013 0.023 0.231 0.082 0.438 0.381

ASA 0.034 0.057 0.935 0.016 0.704 0.100 0.092 0.941 0.317 0.808 0.028 0.932 0.161 0.347 0.112 <.001

Device (eg Zenith) 0.054 0.014 0.023 0.051 0.278 0.128 0.142 0.009 0.007 0.301 0.851 0.307 0.289 0.089 0.682 0.701

Number Preexisting condtns 0.239 0.001 0.928 0.020 0.056 0.046 0.530 0.058 0.681 0.211 0.059 0.525 0.742 0.522 0.737 <.001

Site (eg surgical theatre) 0.239 0.042 0.051 0.036 0.146 0.919 0.049 0.024 0.667 0.275 0.423 0.123 0.510 0.224 0.255 0.707

Aortic Neck Angle 0.024 0.380 0.423 0.200 0.359 0.379 0.866 0.678 0.048 0.903 0.372 0.528 0.024 0.389 0.021 0.374

Age 0.426 0.612 0.496 0.078 0.662 0.882 0.659 0.567 0.297 0.706 0.750 0.064 0.018 0.389 0.017 <.001

Open Suitability 0.380 0.034 0.980 0.595 0.606 0.048 0.670 0.240 0.088 0.086 0.438 0.346 0.711 0.929 0.107 <.001

Infrarenal Neck Length 0.006 0.202 0.632 0.228 0.525 0.994 0.829 0.863 0.012 0.104 0.353 0.232 0.953 0.440 0.087 0.988

Whites Grade 0.088 0.061 0.315 0.346 0.793 0.489 0.472 0.562 0.423 0.952 0.813 0.160 0.082 0.810 0.047 0.505

Public Private 0.727 0.945 0.339 0.344 0.065 0.034 0.494 0.491 0.905 0.324 0.159 0.060 0.883 0.752 0.573 0.867

Gender 0.665 0.101 0.211 0.942 0.584 0.221 0.863 0.883 0.559 0.932 0.008 0.266 0.780 0.219 0.282 0.562

Infrarenal Neck Diameter 0.874 0.889 0.564 0.339 0.265 0.283 0.797 0.342 0.924 0.341 0.327 0.368 0.790 0.779 0.076 0.145

Aneurysm Angle 0.179 0.482 0.343 0.738 0.255 0.416 0.434 0.481 0.548 0.625 0.160 0.517 0.176 0.404 0.420 0.313

Smoking 0.510 0.608 0.597 0.741 0.867 0.529 0.904 0.869 0.250 0.373 0.963 0.260 0.826 0.644 0.171 0.516

Red shading denotes 5% statistical significance (p < 0.05) Yellow shading denotes 10% statistical significance (p < 0.10)

Ru

ptu

res

Clin

ical S

uccess

Death

s

Tech

nical S

ucc.

Interv-

entio

ns

En

do

leak T

ype I

Graft

Co

mp

lic.

En

do

leak T

ype II

Co

nv

erte

d to

Op

en


Eight Predictor Variables

• Age• ASA• Gender• Creatinine

Choice somewhat arbitraryShow large table with many pre-op variables from report

• Aneurysm diameter• Aortic neck angle• Infrarenal neck diameter• Infrarenal neck length



Primary outcomesGoodness

of fit (p)

Validation Results

Corrected

Dxy

Corrected

R2

Corrected

Emax

Early death 0.92 0.384 0.058 0.007

Aneurysm related death 0.53 0.497 0.099 0.022

Mid-term re-interventions 0.13 0.170 0.016 0.075

Initial endoleak type 1 0.59 0.310 0.026 0.142

Mid-term endoleak type 1 0.32 0.255 0.038 0.001

3 year survival 0.57 0.405 0.115 0.017

St George’s patients generally sicker, having larger aneurysms, having more difficult anatomy and are more likely to die than the original cohort of Australian patients

Bold shaded indicates relatively ‘good’ models

Documents

Personalised medicine: Endovascular aneurysm repair risk assessment model using 8 preoperative variables Mary Barnes 1 Robert Fitridge 2, Maggi Boult 2