MERIT STUDY

Jack Chen MBBS PhDAnnual Health Service Research Meeting, 26-28 June 2005 Boston

Background

• Hospitals are unsafe places

• Most patients who suffer adverse outcomes have documented deterioration

• Medical Emergency Team system educates and empowers staff to call a skilled team in response to specific criteria or if “worried”

• Team is called by group pager and responds immediately

MEDICAL EMERGENCY TEAM (MET) CONCEPT

• Criteria identifying seriously ill early

• Rapid response to those patients (similar to a cardiac arrest team)

• Resuscitation and triage

MET Calling Criteria

M.E.R.I.T Study

Medical Early Response Intervention AND

Therapy

Terminology• CAT - Cardiac arrest team

• NFR - Not for resuscitation (DNR, DNAR)

• Events - – Deaths without NFR– Cardiac arrests without NFR– Unplanned ICU admissions – MET and CAT calls independent of above

PRIMARY AIM

• The primary aim of this study was to test the hypothesis that the implementation of the hospital-wide MET system will reduce the aggregate incidence of:

– Unplanned ICU admissions (mainly general wards)

– Cardiac Arrests (-NFR) – Unexpected deaths (-NFR)

STUDY SAMPLE & SAMPLE SIZE:

(at design stage)

• 23 hospitals with at least 20,000 estimated admissions per year

• This will provide us with a 90% chance to detect a 30% reduction in the incidence at the significant level of 5%

Kerry & Bland (1998)

CLUSTER RANDOMISED TRIAL

• More complex to design• More participants to obtain equivalent statistical

power• Key determinants are number of individual units;

the intracluster correlation; and cluster size• More complex analysis than ordinary randomised

trial• Randomised at one time, rather than one at a

time

FRAMEWORK FOR DESIGN, ANALYSIS & REPORTING

CONSORT STATEMENT: extension to cluster randomised

trials

BMJ 2004;328:702

Assessed for eligibility (46 hospitals)

Excluded: 9 already had a MET system, 14 declined stating resource limitations

Randomized (23 hospitals)

Two months baseline period (23 hospitals)

Allocated to MET: (12 hospitals) median admission number at the baseline = 6494, range: 958 - 11026

Allocated to control: (11 hospitals) median admission number over the baseline =5856; range: 1937 –7845.

Lost to follow up: noneAnalyzed: 12 hospitals, median admission number over the study period = 18512; range: 2667 - 33115

Lost to follow up: noneAnalyzed: 11 hospitals, median admission number over the study period = 17555; range: 5891 - 22338

Four months implementation of MET with continued data collection

Four months period with continued data collection

Six months study period with MET system operational Six months study period

RANDOMISATION

• Stratified – blocked randomisation (4) based on teaching hospital status

• Independent statistician

DATA COLLECTION

• 18178 EVENT forms

• 2418 corrections (13.3%)

• Final EVENTS - 13142 after third round data consistency and logic checking

• In-patients – 750,000

DATA COLLECTION • Log books• Scannable technology • Photocopy forms kept by hospital• Filing of forms and storage in Simpson

Centre• Web-based tracking data• 4 databases• Separate neutral data repository

DATA CORRECTION LOOP

• 10 step standardised data entry and correction procedure

• Weekly data entry meeting between statistician, data manager, IT manager and research assistants

Types of analyses

Cluster (Hospital) Level Individual / Multilevel

Unadjusted analysis

Weighted t-test

(weighted by hospital admission number)

Rao-Scott Chi-square

Adjusted t-test;

Adjusted analysis: Analytically weighted regression (weighted by the admission number of the hospital) adjusting for teaching status, number of bed and baseline outcome

Multi-level logistic regression (adjusting for teaching status, number of bed, age and gender of the patients)

Statistical methods used at cluster level and individual/multilevel (unadjusted and adjusted analyses)

WEIGHTING AND ADJUSTMENT

• Weighting: by the number of admissions during the study period

• Cluster Adjustment for: teaching hospital status, bed size and baseline outcome variables, with hospitals weighted by the number of admissions during the study period

• Multilevel model adjustment for: teaching hospital status, bed size, age and gender of the patients

BASELINE DATANon-MET MET

Hospitals

Number 11 12

Teaching 8 9

Non-teaching 3 3

Median bed size 315 364

(119-630) (88-641)

BASELINE DATA

Outcomes (incidence rate/ Non-MET MET

1000 admissions)

Primary Outcome 6.775 6.291

Cardiac arrests (- NFR) 2.606 1.597Unplanned ICU admissions 4.132 4.267

Unexpected deaths (- NFR) 1.605 1.648

No significant differences

RESULTS - DIFFERENCE BETWEEN MET & NON-MET HOSPITALS

Incidence Rate/1000 admissions

OUTCOMES NON-MET

MET % AGE CHANGE

P

Primary outcome 5.860 5.306 10% 0.804

Cardiac arrest – NFR 1.640 1.31 25.1% 0.306

Unplanned ICU admission

4.683 4.185 12% 0.899

Unexpected deaths (– NFR)

1.175 1.063 10% 0.564

OUTCOME RATES/1000 ADMISSIONS OVER BASELINE, IMPLEMENTATION AND STUDY PERIODS

Aggregate outcome Cardiac arrests Unplanned ICU admissions Unexpected deaths Control

Hospitals Baseline Implementation Study Baseline Implementation Study Baseline Implementation Study Baseline Implementation Study

1 2.07 2.65 3.05 1.03 1.45 2.03 0.52 0.24 0.68 0.52 2.41 1.53 2 3.77 6.03 6.32 1.74 2.24 1.93 1.45 3.71 4.49 1.45 1.78 1.31 3 5.10 4.47 4.21 2.29 1.94 1.75 3.31 2.74 2.69 1.15 1.14 0.94 4 5.47 3.55 2.64 2.02 1.31 1.15 2.59 1.85 1.50 2.74 1.31 0.95 5 5.72 4.33 3.53 3.25 3.16 1.47 2.32 1.58 2.39 1.70 0.75 0.60 6 5.98 2.82 2.73 3.76 1.32 1.54 2.22 1.50 1.25 2.22 1.15 0.74 7 6.83 6.28 5.07 3.32 3.29 1.69 2.54 3.29 3.25 3.51 2.39 1.50 8 7.86 5.92 4.72 1.97 1.27 1.11 6.46 4.93 3.61 1.40 0.71 1.02 9 9.04 10.57 8.83 2.85 1.76 0.85 6.66 8.30 7.81 0.95 1.26 1.19

10 9.42 7.82 7.63 2.49 3.41 1.88 7.34 4.97 5.99 1.66 1.00 1.45 11 13.29 18.10 13.92 3.95 7.36 2.65 10.06 11.93 12.07 0.36 1.79 1.72

MET Hospitals

12 0.58 0.89 1.31 0.29 0.59 1.11 0.00 0.00 0.10 0.58 0.74 1.01 13 1.60 3.36 4.61 0.37 1.16 0.78 1.23 2.39 4.16 0.37 0.52 0.45 14 1.85 5.80 3.42 0.46 0.67 0.45 1.39 4.46 2.38 0.46 0.67 0.89 15 2.95 3.40 3.22 1.03 2.03 1.04 2.05 1.57 2.22 0.64 1.24 0.68 16 3.87 3.60 2.86 2.35 1.56 1.24 1.88 2.16 1.99 0.82 0.66 0.66 17 4.26 4.19 4.66 0.82 0.75 1.49 2.46 3.60 2.87 1.48 0.92 1.49 18 6.39 7.27 7.08 3.05 2.75 2.34 3.34 4.90 4.84 2.03 1.30 1.27 19 6.39 4.48 4.44 4.35 2.60 1.62 2.04 2.02 2.67 2.18 1.44 1.38 20 7.29 4.98 5.90 1.56 1.70 2.05 4.17 3.01 3.16 2.86 1.05 1.37 21 7.44 6.18 7.07 2.45 1.89 1.78 5.35 4.48 5.74 1.27 1.18 1.27 22 13.04 6.89 5.59 1.40 1.46 1.07 11.64 5.43 4.66 1.86 1.04 0.40 23 19.83 14.43 12.75 1.04 2.96 0.75 15.66 11.10 10.87 5.22 4.07 1.88

* Excludes patients with prior NFR orders

CALLING RATE/HOSPITAL/1,000 ADMISSIONS

CONTROL HOSPITALS MET HOSPITALS p

3.1 (1.5-5.8) 8.7 (3.5-16.5) <0.001

CALLS NOT ASSOCIATED WITH AN EVENT/1,000 ADMISSIONS

CONTROL MET HOSPITALS HOSPITALS p

1.2 (0-3.3) 6.3 (2.5-11.2) <0.001

194/528 (36.7%) 1329/1886 (70.5%) <0.001

NUMBER OF CALLS/EVENT (%)

CONTROL MET

HOSPITALS HOSPITALS p

Cardiac 236/246 (96%) 244/250 (97.6%) 0.359 arrests

Unplanned 54/568 (9.5%) 209/611 (34.2%) 0.001 ICU admissions

Unexpected 5/59 (17.2%) 4/48 (8.3%) 0.420 deaths

EVENTS WHICH HAD MET CRITERIA BEFOREHAND (<15 min)

CONTROL MET

HOSPITALS HOSPITALS p

Cardiac 130/246 (53%) 115/250 (46%) 0.664 arrests

Unplanned ICU 121/568 (21%) 219/611 (36%) 0.090 admissions

Unexpected 10/29 (35%) 12/48 (25%) 0.473 deaths

EVENTS WHICH HAD MET CRITERIA BEFOREHAND (>15 min)

CONTROL MET

HOSPITALS HOSPITALS p

Cardiac 109/246 (44%) 76/250 (30%) 0.031 arrests

Unplanned ICU 314/568 (55%) 313/611 (51%) 0.596 admissions

Unexpected 16/29 (55%) 24/58 (50%) 0.660 deaths

CALLS WHEN MET CRITERIA WERE PRESENT (<15 min before event)

CONTROL MET

HOSPITALS HOSPITALS p

Cardiac 124/130 (95%) 112/115 (97%) 0.545 arrests

Unplanned ICU 28/121 (23%) 112/219 (51%) 0.049 admissions

Unexpected 4/16 (25%) 2/12 (17%) 0.298 deaths

CALLS WHEN MET CRITERIA WERE PRESENT (>15 min before event)

CONTROL MET

HOSPITALS HOSPITALS p

Cardiac 104/109 (95%) 72/76 (95%) 0.874 arrests

Unplanned ICU 27/314 (9%) 95/313 (30%) 0.009 admissions

Unexpected 4/16 (25%) 2/24 (8%) 0.231 deaths

NFR DESIGNATIONNon-MET MET

Prior NFR/1000 admissions 9.404 9.434

Prior NFR/Deaths 1.01 1.05

NFR made at time of event/

1000 admissions 0.274 0.799

NFR made at time of event/

1000 events 17.189 38.424

NFR ORDERS IN CALLS NOT ASSOCIATED WITH AN EVENT

CONTROL MET

HOSPITALS HOSPITALS p

6/197 (3%) 106/1332 (8%) 0.048

DIFFERENCES BETWEEN BASELINE AND STUDY PERIOD/1,000

ADMISSIONS (%)

p

Primary outcome -0.85 (13%) 0.089

Cardiac arrests -0.68 (33%) 0.003

Unplanned ICU -0.23 (5%) 0.577

admission

Unexpected deaths -0.48 (30%) 0.010

IN SUMMARY• Randomisation was successful and

appeared balanced

• Call rate was much higher in MET hospitals mostly due to calls not associated with events

• More of these event-free calls led to NFR orders in MET hospitals, but overall NFR rate was unaffected

IN SUMMARY• There was no STATISTICALLY SIGNIFICANT

decrease in the incidence of the primary outcome in MET hospitals

• There was no STATISTICALLY SIGNIFICANT decrease in the incidence of the secondary outcomes in MET hospitals

• WHEN ALL HOSPITALS CONSIDERED TOGETHER, The incidence of cardiac arrests and unexpected deaths decreased from baseline to study period

IN SUMMARYIf MET criteria were

documented and followed by an event, only a minority of

patients overall had an actual MET call made

IN SUMMARYThere was an increase in calls before ICU admission in MET

hospitals but not before cardiac arrests or unexpected

deaths

IN SUMMARY

Less than half of all events had MET

criteria documented beforehand

IN SUMMARY36.7% of all cardiac

arrest calls were not in response to an event

IN SUMMARY

Extreme variability in event rates amongst

hospitals

IN SUMMARY23 hospitals – needed >100 to

show a difference• Estimated primary outcome incidence

3% - actual rate 0.57%

• Between hospital variability high

• Intra-class correlation co-efficient high

Why no significant improvement ?•The MET may be ineffective;

•The implementation is less optimal;

•The participating hospitals are unrepresentative;

•We studied wrong outcome;

•The documentation of the vital signs is poor;

•The calling rate is low given the existing calling criteria;

•The contamination;

•The low statistical power

CONCLUSIONS• First large hospital system change trial ever

conducted according to rigorous principles of design and statistical analysis

• It encompassed close to 750,000 admissions• Although we did not demonstrate a significant

difference in the primary outcome, the study produced a large body of useful data on patient care, documentation and outcomes, which will hopefully illuminate future studies

MERIT STUDY

CONDUCTED BY:Simpson Centre for Health Services Research

ANZICS Clinical Trials Group

FUNDED BY:NHMRC

Australian COUNCIL FOR Quality and Safety in Health Care (AQSHC)

MERIT STUDY

MANAGEMENT COMMITTEEProf. Ken Hillman (Chair) Prof. Rinaldo Bellomo Mr. Daniel BrownDr. Jack ChenDr. Michelle CretikosDr. Gordon DoigDr. Simon FinferDr. Arthas Flabouris

PARTICIPATING HOSPITALS, INVESTIGATORS & RESEARCH NURSES

• Bendigo – John Edington, Kath Payne• Box Hill – David Ernest, Angela Hamilton• Broken Hill – Coral Bennet, Linda Peel,

Mathew Oliver, Russell Schedlich, Sittampalam Ragavan, Linda Lynott

• Calvery – Marielle Ruigrok, Margaret Willshire,

• Canberra – Imogen Mitchell, John Gowardman, David Elliot, Gillian Turner, Carolyn Pain

• Flinders – Gerard O’Callaghan, Tamara Hunt• Geelong – David Green, Jill Mann, Gary

Prisco• Gosford – Sean Kelly, John Albury• John Hunter – Ken Havill, Jane O’Brien• Mackay – Kathryn Crane, Judy Struik• Monash – Ramesh Nagappan, Laura Lister

• Prince of Wales – Yahya Shahabi, Harriet Adamsion

• Queen Elizabeth – Sandy Peake, Jonathan Foote

• Redcliffe – Neil Widdicombe, Matthys Campher, Sharon Ragou, Raymond Johnson

• Redland – David Miller, Susan Carney• Repatriation General – Gerard O’Callaghan,

Vicki Robb• Royal Adelaide – Marianne Chapman, Peter

Sharley, Deb Herewane, Sandy Jansen• Royal North Shore - Simon Finfer, Simeon Dale• St. Vincent’s – John Santamaria, Jenny Holmes• Townsville – Michael Corkeron, Michelle

Barrett, Sue Walters• Wangaratta – Chris Giles, Deb Hobijn • Wollongong - Sunny Rachakonda, Kathy

Rhodes• Wyong – Sean Kelly, John Albury