Butt 1

Evaluating the Quality of Trauma Care A Literature Review

Western Michigan University and Kalamazoo Area Mathematics and Science Center

By: Kaylie Butt

May 4, 2015

Introduction

This literature review study aims at developing an understanding of the metrics used to

evaluate the performance of trauma care centers based on the Institute of Medicine’s six

dimensions of healthcare delivery quality: effectiveness, efficiency, timeliness, safety, patient

centeredness, and equitability (45). These dimensions were originally identified by the Institute

of Medicine in the paper Crossing the quality chasm: A new health system for the 21st century

published in 2001 (49).This paper served as the source of the guidelines for this study.

To understand which metrics should be used to evaluate trauma care centers we must first

have an understanding of the term trauma. Trauma is defined by Webster’s Medical Dictionary

as “an injury to the body” (91). In medicine, the term refers to the “most severe injuries that

threaten life and limb” (91). The quality of trauma care has been evaluated through different

metrics. The goal of this literature review was to identify which metrics can be used to represent

each healthcare quality dimension in the evaluation of trauma care centers. The questions to be

answered in this study were:

What are the performance metrics used to evaluate trauma centers? What quality dimensions

are mostly and not fully addressed in the literature and what are the corresponding metrics? Is

there any documentation of correlation/conflict of these dimensions? Have these conflicting

relationships been measured? We hypothesized that Mortality rate is the most commonly used

Butt 2

metric, which effectiveness would be the quality dimension better addressed in the literature and

that efficiency is conflicting with most other dimensions of healthcare quality, respectively.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)

Guidelines where followed in this study (65). These guidelines can be used “to improve the

reporting of systematic reviews and meta-analyses.”

This study identified most commonly used trauma evaluation metrics and classified them

according to IOM’s dimensions of healthcare quality. We focused on articles using a trauma

registry similar to Michigan Trauma Quality Improvement Program (MTQIP) and the National

Trauma Data Bank (NTDB). Each data bank consists of all data collected on all trauma patients

of participating institutions in a given amount of time.

The information from this literature review will be used in a future study where the

performance of trauma centers will be quantified in terms of the identified performance metrics.

In this future study the dimensions of quality and the relationship between these dimensions will

be analyzed.

Methods

Databases used in this study were Scopus, Science Direct, and Google Scholar. To search

theses databases, the keywords “Trauma center” was combined with other keywords, including

the Institute of Medicine’s six dimensions of healthcare delivery quality and other keywords

related to trauma care evaluation found in Crossing the quality chasm: A new health system for

the 21st century. For example, “Trauma center” AND “Performance evaluation” or “Trauma

center” AND “Timeliness”. From the three databases 155 articles were extracted in this way and

read for future analysis.

Butt 3

As each paper was collected it was entered into an excel file. Duplicate papers from

different databases were excluded from the study. The papers were then evaluated according to

the following inclusion criteria.

Must be from a peer reviewed journal

Must be published after 2001 which is after publication of “Crossing the

Quality Chasm: A New Health System for the 21st Century” which describes

Institute of Medicine’s six dimensions of healthcare delivery quality

Involves trauma center(s)

An English-language publication

Must have available abstracts

Uses MTQIP or NTDB or analyzes the performance of trauma centers, using

quantitative measures

Explicitly outlines risk adjustment factors and method

Provides a performance metric of center or patients and a numerical value for

such metric

Prediction models were not included in this search

Focus is not on the statistical approach, but on performance evaluation

Papers not meeting all of these criteria were excluded from the study. After applying the

exclusion criteria to the set of articles, 79 papers were left in the study for further analysis. Figure

1 depicts a flow chart demonstrates the flow of the papers through the search process of this

study.

Butt 4

Figure 1 PRISMA article selection and inclusion flow chart

After finding the papers to be used in the study the next step was to extract the important

information from each of the articles. For each paper the objective, patient cohort, performance

metrics used, values of performance metrics used, injury type studied, trauma center size,

adjustment factor used for analysis, analysis method, database used were all extracted and put

into a spreadsheet file.

The quality dimensions that the article either mentioned or were displayed in the

research where then determined. For instance in Stelfox’s Trauma center volume and quality

improvement programs (86) focused on volume of a trauma center decreasing wait time and used

mortality and morbidity to evaluate improvement in the program. Thus this paper was classified

Butt 5

as satisfying the dimensions of timeliness, efficiency, an effectiveness as well as patient

centeredness. All the information was combined for the next analysis.

After the information from the articles was combined the next step was to find the most

common patient cohort to focus on. The patient cohorts and injury types were then ranked

according to the number of studies that analyze them. The top cohorts (representing about 80%

of the studies) were used in the remainder of the analysis and the other articles were no longer

considered.

After the articles with the same patient cohorts were found and combined, the study was

focused on the metrics used in these studies. The metrics where then ranked according to the

number of papers that use them. Then the top metrics (representing 80% of the studies for

significant cohorts) were identified. The papers not containing these top metrics were excluded

from further analysis. This left three metrics most used in studies. Figure 2 depicts flow map

demonstrates this method of narrowing the pool of articles from the top 80% of cohorts to the top

80% of metrics.

Butt 6

Figure 2 PRISMA analysis flow chart

After narrowing down the top metrics, an analysis on the metrics was performed. The

quality dimension that they represent was predetermined after a second read if need. Also the

adjustment factors that the paper used to scale the data of normalizes it were also recorded along

with the distribution of reported values. Any documented comments on potential relationships

between the different performance metrics, and proposed strategies to improve the performance

based on specific metrics were studied in the remaining studies were also taken not of so that

future research could have a basis to work off of based of open ended or unstudied relationships.

Results 

Patient cohorts are the subjects in a study sharing a common factor that links them. In

Figure 3 the Prato chart shows the patient cohorts of the papers vs. the percent of the total papers

they were found in. A noticeably large proportion of the data is in the first two cohorts of general

Butt 7

trauma, described as all traumas that enter the center, and ISS (injury severity score) >15, that is

the score taken on entrance is greater than>15.

Figure 3 Patient Cohorts

Genera

l Trau

ma ISS AIS

Blunt Truam

a

Brain Tr

auma

Fast P

atien

t

Pancre

atic T

ruama

Spine T

ruma0

5

10

15

20

25

30

35

40

45

0102030405060708090100

Patient Cohorts

count

Cumulative %

Patient Cohort Type

Coun

t

*To see all papers with these patient cohorts see Appendix A1

ISS and general trauma make up 80% of all the papers. These two patient cohorts were

used for further study and should be used in further study. The purpose of this was that since

80% of the papers shared these top two cohorts and since that was the case it was reasonable to

take these as the next

After the top patient cohorts were found the metrics used to analysis those cohorts were

found. In Table 2 we see that the top metrics were LOS, morbidity, and mortality. In Table 1 we

Butt 8

see a general description of the metrics. Metrics are the numerical analysis that was used in each

study to look at quality and its dimensions.

As seen in Table 1, mortality is in 60% of the studies, with morbidity and LOS the top

90% of the papers used one of these three metrics. These metrics encompass effectiveness,

safety, and efficiency dimensions. Mortality, the most common, specifically encompasses all

three of those dimensions because safety of life and death is taking in hand, efficiency of the job

equates to life, and effectiveness of getting the job done to preserve life all can be equated by this

metric. Morbidity equates to all three for similar reasons of preventing disease that could have

been prevented (efficiency), how well they prevented it (effectiveness), and whether or not they

did (safety). LOS encompasses timeliness and efficiency as it is how fast they can get a patient

out of the trauma center (timeliness), as well as how well, (efficiency).

Table 1 Quality Metrics by Dimensions

Metric Percent Use

Effectiveness Efficiency Timeliness Patient Centeredness

Safety Equitability

Mortality 61% X X X X

LOS( Length of Stay)

39%X X

Morbidity 34.4% X X X

ISS (injury severity score)

9.8% X X

Number of days on ICU

4.9%X

Post Injury Complications

4.9%X

Butt 9

Metric Percent Use

Effectiveness Efficiency Timeliness Patient Centeredness

Safety Equitability

Short form 36 / Galsgrow outcome scale

2.4%X X X

Wait Time 2.4% X

Patient volume

2.4%X X

Referral Pattern

2.4%X

Ventilator days

1.8%X

MTOS ( Major Trauma Outcome Survey)

1.8%

X

AIS ( Injury Scale)

1.8% X

PCS(physical component

1.8%X

TRAN Score 1.8% X

Mental Summary (MSC)

1.8%X X X X

Travel Time 1.8% X X

*Some papers use more than one metric. To see the papers with these metrics look at Appendix A2

Mortality is described as rate of how many die as opposed to the survivors (86). It encompasses

effectiveness, efficiency, and safety, and timeliness as dimensions. The explanation being that it evaluates

how many people they could save and takes into account time in a trauma center because best care

Butt 10

happens in the golden hour(1). It takes into account how well they the center cared for the patients

(effectiveness and safety), as well as how many they could save. This metric is useful as it helps

demonstrate four dimensions.

Morbidity is the relative incidence rate of disease (86). This metric encompasses effectiveness, efficiency,

and safety as dimensions. This It takes into account how well they the center cared for the patients

(effectiveness and safety), as well as how many they could prevent giving disease too (safety and

efficiency).

LOS (length of stay) is a term to describe the duration of a single episode of hospitalization. Inpatient

days are calculated by subtracting day of admission from day of discharge (45). This metric uses timeliness

and efficiency as dimensions as it measures they efficacy and speed of getting the patient out of the center

well.

Travel time is specified period of time spent in traveling in a trauma center or from the first location to

the place where work is actually done (ex. From the room to the CT Scan) (65). This metric uses timeliness

and efficiency as dimensions as it measured the time from one place to another and how well that process

goes while trying to keep it within the golden hour (1).

ISS (injury severity score) is an established medical score to assess trauma severity. It correlates with

mortality, morbidity and hospitalization time after trauma (60). This is not to be confuses with the ISS

patient cohort which is >15. It is the same scale but used here not as a category and taken after they are

released not before. This uses efficiency and effectiveness as dimensions as it measures the change in well-

being from beginning to end.

Trauma metrics are usually analyzed over time for a single center and across institutions.

Table 3 lists the most commonly used analysis approaches used in the literature. Most of them

used statistical tests to compare the performance of centers.

Butt 11

Once the metrics were determined the adjustment factors for each were determined. An

adjustment factor is what is taken into account as a variable between cohorts buy using the

metric selected. Table 2 demonstrates the factors for all the metrics.

Table 2 Adjustment Factors by Metrics

Adjustment Factor

Demographics Numberof Unexpected Stops

Distance ISS

Mortality XNumber of ICU Days

X

Ventilator Days

X

Travel time X XPCOR XTRAN Score XAIS XPCS(Physical Component Score)

X

Adjustment Factor

Demographics Numberof Unexpected Stops

Distance ISS

ReferralPattern

X

Short Form36-Galsgrow Outcome Scale

X

Wait Time XMorbidity XMTOS(Major Trauma Outcome Survey)

X

Over 80%of the adjustment factors used demographics. This is interesting but

understandable as that is the most common thing separating cohorts, weight, sex, age and so on.

Butt 12

Mortality and morbidity both use demographics as would be expected. LOS uses demographics

when referring to equality.

Conclusions

This literature review study was performed to determine the quality dimensions that have

been most commonly used in the literature and what metrics have been used to measure them. In

(Figure 3) we can see that over 50% of the papers focus on general trauma patients and 30% use

ISS> 15. Thus these cohorts should be used for further study. We also found that over 60% of

the studies within these two cohorts used mortality as a metric; morbidity and LOS (length of

stay) make up another 70% of the studies. (Table 1)

As for metrics Efficiency, timeliness, and effectiveness are the most widely studied

dimensions of quality. Patient centeredness, safety, and equitability are less commonly studied.

These do not have as easily numerical evaluations, thus more work is need to assess these

dimensions of healthcare quality Also almost every metric has an adjustment factor of

demographics. This suggests that demographics should be used as an adjustment factor for new

metrics proposed in this area (Table 1).

In this study relationship between different metrics was also found. Increase in patient

centeredness has an inverse relation with mortality and morbidity (83). This would be an

interesting study.

Less diverse people tend to have a shorter length of stay and wait time in some but not all

trauma centers. This refers to equitability and efficiency (53). Very few studies have been done

on both of these metrics and need further study. Also shows indicators for evaluating triage and

patient flow, (effectiveness of care, and efficiency of care) often clash (83). However, high-

Butt 13

volume centers are more likely to use quality indicators (QI) for evaluating medical errors,

showing most effective change.

Wait time increases the rate of mortality. In theory, if it is in the nurses “golden hour”, which is

the hour that trauma care should be received by a patient in order for the greatest odds of

survival, else they decrease exponentially (1). Some studies have been done on this in correlation

to nurses but few with in trauma centers as a focus and few with suggestions for change. This

would most likely be the best course of study as it takes into account many of the metrics in the

top 80% ( morbidity, mortality, and LOS), as well as taking into account most of the Institute of

Medicine’s six dimensions of healthcare delivery quality. It takes into account effectiveness,

efficiency, timeliness, safety, and patient centeredness

References

1. Ardolino A., Sleat G., WillettK. Outcome measurements in major trauma--results of a

consensus meeting. Injury 43 (10) p. 1662-6

2. Alali, A.S., Fowler, R.A., Mainprize, T.G., Scales, D.C., Kiss, A., De Mestral, C., Ray,

J.G., Nathens, A.B., Intracranial pressure monitoring in severe traumatic brain injury:

Results from the american college of surgeons trauma quality improvement program,

Journal of Neurotrauma

3. Alali, A.S., Scales, D.C., Fowler, R.A., Mainprize, T.G., Ray, J.G., Kiss, A., De

Mestral, C., Nathens, A.B., Tracheostomy timing in traumatic brain injury: A

propensity-matched cohort study, Journal of Trauma and Acute Care Surgery

Butt 14

4. Anand Nair, Mariana Nicolae, Ram Narasimhan Examining the impact of clinical

quality and clinical flexibility on cardiology unit performance—Does experiential

quality act as a specialized complementary asset?Journal of Operations Management

31 (7-8) p. 505-522

5. Andrew J Schoenfeld, Philip J Belmont, Aaron A See, Julia O Bader, Christopher M

Bono Patient demographics, insurance status, race, and ethnicity as predictors of

morbidity and mortality after spine trauma: a study using the National Trauma Data

Bank.The spine journal : official journal of the North American Spine Society 13 (12)

p. 1766-73

6. Avery B Nathens, H Gill Cryer, John Fildes The American College of Surgeons

Trauma Quality Improvement Program.The Surgical clinics of North America 92 (2) p.

441-54, x-xi

7. Avery, L.E., Stahlfeld, K.R., Corcos, A.C., Scifres, A.M., Ziembicki, J.A., Varcelotti,

J., Peitzman, A.B., Billiar, T.R., Sperry, J.L., 2012, Evolving role of endovascular

techniques for traumatic vascular injury: A changing landscape?, Journal of Trauma

and Acute Care Surgery

8. Barbosa, R.R., Rowell, S.E., Fox, E.E., Holcomb, J.B., Bulger, E.M., Phelan, H.A.,

Alarcon, L.H., Myers, J.G., Brasel, K.J., Muskat, P., Del Junco, D.J., Cotton, B.A.,

Wade, C.E., Rahbar, M.H., Cohen, M.J., Schreiber, M.A., Increasing time to operation

is associated with decreased survival in patients with a positive FAST examination

requiring emergent laparotomy, Journal of Trauma and Acute Care Surgery

9. Barnes, G. D., Birkmeyer, N., Flanders, S. A., Froehlich, J. B., Hemmila, M., Henke,

P. K., & Share, D. (2012). Venous thromboembolism: a collaborative quality

Butt 15

improvement model for practitioners, hospitals, and insurers. Journal of thrombosis and

thrombolysis, 33(3), 274-279.

10. Brian E Morrissey, Ruth A Delaney, Alan J Johnstone, Laurie Petrovick, R Malcolm

Smith Do trauma systems work? A comparison of major trauma outcomes between

Aberdeen Royal Infirmary and Massachusetts General Hospital. Injury

11. Bukur, M., Singer, M.B., Chung, R., Ley, E.J., Malinoski, D.J., Margulies, D.R.,

Salim, A., Influence of resident involvement on trauma care outcomes, Archives of

Surgery

12. C. Macarenhas, D. Carvalho, G. Fernandes, C. Dias, E. Gomes Assessment of quality

of care using quality indicators: Study of a trauma centre registry Resuscitation 81 (2)

p. S83

13. Calland, J.F., Xin, W., Stukenborg, G.J., Effects of leading mortality risk factors

among trauma patients vary by age., The journal of trauma and acute care surgery

14. Cameron D Willis, Belinda J Gabbe, Peter A Cameron Measuring quality in trauma

care.Injury 38 (5) p. 527-37

15. Caputo, L. M., Salottolo, K. M., Slone, D. S., Mains, C. W., & Bar-Or, D. (2013). The

relationship between patient volume and mortality in American trauma centres: A

systematic review of the evidence. Injury.

16. Carr, B.G., Reilly, P.M., Schwab, C.W., Branas, C.C., Geiger, J., Wiebe, D.J.,

Weekend and night outcomes in a statewide trauma system, Archives of Surgery

17. Cassandra V. Villegas, Hannah A. Dineen, Mautomn Sadoun, Julie L. Wynne, Randall

S. Friese, Andrew Tang, Terence O'Keeffe, Narong Kulvatunyou, Peter M. Rhee,

Bellal A. Joseph Are all trauma centers equal: Analyzing pediatric outcomes Journal of

Butt 16

the American College of Surgeons 215 (3) p. S100

18. Clark, D.E., Qian, J., Winchell, R.J., Betensky, R.A., Hazard Regression Models of

Early Mortality in Trauma Centers, Journal of the American College of Surgeons

19. Codner, P.A., Brasel, K.J., Management of the Complex Trauma Patient With Limited

Resources, ICU Director

20. Cook, A., Weddle, J., Baker, S., Hosmer, D., Glance, L., Friedman, L., Osler, T., A

comparison of the Injury Severity Score and the Trauma Mortality Prediction Model,

Journal of Trauma and Acute Care Surgery

21. Crandall, M., Zarzaur, B., Tinkoff, G., American association for the surgery of trauma

prevention committee topical overview: National trauma data bank, geographic

information systems, and teaching injury prevention, American Journal of Surgery

22. Cudnik, M. T., Newgard, C. D., Sayre, M. R., & Steinberg, S. M. (2009). Level I

versus Level II trauma centers: an outcomes-based assessment. Journal of Trauma-

Injury, Infection, and Critical Care, 66(5), 1321-1326.

23. Cummins JS, Koval KJ, Cantu RV, & Spratt KF. (2008). Risk of injury associated with

the use of seat belts and air bags in motor vehicle crashes. Bulletin of the NYU

Hospital for Joint Diseases. 66(4), 290-6.

24. Dams-O'Connor, K., Cuthbert, J.P., Whyte, J., Corrigan, J.D., Faul, M., Harrison-Felix,

C., 2013, Traumatic brain injury among older adults at level i and II trauma centers,

Journal of Neurotrauma

25. Darlene B. Murdock Trauma: When There's No Time to Count AORN Journal 87 (2)

p. 322-328

26. DuBose, J.J., Teixeira, P.G.R., Shiflett, A., Trankiem, C., Putty, B., Recinos, G., Inaba,

K., Belzberg, H., 2009, American College of Surgeons trauma centre designation and

Butt 17

mechanical ventilation outcomes, Injury

27. E M Cole, A West, R Davenport, S Naganathar, T Kanzara, M Carey, K Brohi Can

residents be effective trauma team leaders in a major trauma centre? Injury 44 (1) p.

18-22

28. E. Wei, B. Sharp, S. Santen, L. Hopson, D. Somand, B. Bassin , 141 Interdisciplinary

Comparison of Attitudes Towards Patient Safety, Communication, and Teamwork in

the Emergency DepartmentAnnals of Emergency Medicine 60 (4) p. S51

29. Eastridge BJ, Shafi S, Minei JP, Culica D, McConnel C, & Gentilello L. (2006).

Economic impact of motorcycle helmets: from impact to discharge. The Journal of

Trauma. 60(5), 978-83; discussion 983-4.

30. Eileen Murtagh Kurowski, Terri Byczkowski, Nathan Timm Return visit

characteristics among patients who leave without being seen from a pediatric ED.The

American journal of emergency medicine 30 (7) p. 1019-24

31. Fiona L Lawson, Nadine Schuurman, Lisa Oliver, Avery B Nathens Evaluating

potential spatial access to trauma center care by severely injured patients. Health &

place 19 p. 131-7

32. Franklin, G. A., Cannon, R. W., Smith, J. W., Harbrecht, B. G., Miller, F. B., &

Richardson, J. D. (2011). Impact of withdrawal of care and futile care on trauma

mortality. Surgery, 150(4), 854-860.

33. Fujii, T., Faul, M., Sasser, S., 2013, Risk factors for cervical spine injury among

patients with traumatic brain injury, Journal of Emergencies, Trauma and Shock

34. Ghorbani, P., Falkén, M., Riddez, L., Sundelöf, M., Oldner, A., Strömmer, L.,

Clinical review is essential to evaluate 30-day mortality after trauma, Scandinavian

Butt 18

Journal of Trauma, Resuscitation and Emergency Medicine

35. Glance, L.G., Dick, A.W., Mukamel, D.B., Osler, T.M., Association between trauma

quality indicators and outcomes for injured patients, Archives of Surgery

36. Glance, L.G., Li, Y., Osler, T.M., Mukamel, D.B., Dick, A.W., Impact of obesity on

mortality and complications in trauma patients, Annals of Surgery

37. Glenn Rosenbluth, Christopher P Landrigan Sleep science, schedules, and safety in

hospitals: challenges and solutions for pediatric providers. Pediatric clinics of North

America 59 (6) p. 1317-28

38. Gomez, D., Haas, B., Hemmila, M., Pasquale, M., Goble, S., Neal, M., Mann, N.C.,

Meredith, W., Cryer, H.G., Shafi, S., Nathens, A.B., Hips can lie: Impact of excluding

isolated hip fractures on external benchmarking of trauma center performance, Journal

of Trauma - Injury, Infection and Critical Care

39. Haas, B., Gomez, D., Hemmila, M.R., Nathens, A.B., Prevention of complications and

successful rescue of patients with serious complications: Characteristics of high-

performing trauma centers, Journal of Trauma - Injury, Infection and Critical Care

40. Haas, B., Gomez, D., Neal, M., Hoeft, C., Ahmed, N., Nathens, A.B., Good neighbors?

the effect of a level 1 trauma center on the performance of nearby level 2 trauma

centers, Annals of Surgery

41. Harold K. Simon Pediatric Trauma: A Roadmap for Evidence-Based, Patient-Centered

Coordination and Care Clinical Pediatric Emergency Medicine 11 (1) p. 1-3

42. Hashmi, Z.G., Dimick, J.B., Efron, D.T., Haut, E.R., Schneider, E.B., Zafar, S.N.,

Schwartz, D., Cornwell III, E.E., Haider, A.H., Reliability adjustment: A necessity for

Butt 19

trauma center ranking and benchmarking, Journal of Trauma and Acute Care Surgery

43. Hemmila, M. R., Jakubus, J. L., Maggio, P. M., Wahl, W. L., Dimick, J. B., Campbell

Jr, D. A., & Taheri, P. A. (2008). Real money: complications and hospital costs in

trauma patients. Surgery, 144(2), 307-316.

44. Hemmila, M. R., Jakubus, J. L., Wahl, W. L., Arbabi, S., Henderson, W. G., Khuri, S.

F.& Campbell Jr, D. A. (2007). Detecting the blind spot: complications in the trauma

registry and trauma quality improvement. Surgery,142(4), 439-449.

45. Henry T Stelfox, Sharon E Straus Measuring quality of care: considering conceptual

approaches to quality indicator development and evaluation.Journal of clinical

epidemiology 66 (12) p. 1328-37

46. Hurtuk, M., Reed II, R.L., Esposito, T.J., Davis, K.A., Luchette, F.A., 2006, Trauma

surgeons practice what they preach: The NTDB story on solid organ injury

management, Journal of Trauma - Injury, Infection and Critical Care

47. Institute of Medicine (US). Committee on Quality of Health Care in America.

(2001). Crossing the quality chasm: A new health system for the 21st century. National

Academy Press.

48. Brasel, K.J., Fata, P., Flynn, T.C.Impact of the 80-hour resident work week on

mortality and morbidity in trauma patients (Review).

49. J David Richardson, Glen Franklin, Ariel Santos, Brian Harbrecht, Dan Danzl, Royce

Coleman, Jason Smith, Frank Miller, Kelly McMasters Effective triage can ameliorate

Butt 20

the deleterious effects of delayed transfer of trauma patients from the emergency

department to the ICU. Journal of the American College of Surgeons 208 (5) p. 671-8;

discussion 678-81

50. Jennifer Prah Ruger, Lawrence M Lewis, Christopher J Richter Patterns and factors

associated with intensive use of ED services: implications for allocating resources.The

American journal of emergency medicine 30 (9) p. 1884-94

51. Joanne Disch, Marie Sinioris The quality burden.The Nursing clinics of North America

47 (3) p. 395-405

52. Joseph J DuBose, Pedro G R Teixeira, Anthony Shiflett, Christine Trankiem, Bradley

Putty, Gustavo Recinos, Kenji Inaba, Howard Belzberg American College of Surgeons

trauma centre designation and mechanical ventilation outcomes.Injury 40 (7) p. 708-12

53. Kardooni, S., Haut, E.R., Chang, D.C., Pierce, C.A., Efron, D.T., Haider, A.H.,

Pronovost, P.J., Cornwell III, E.E., 2008, Hazards of benchmarking complications with

the national trauma data bank: Numerators in search of denominators, Journal of

Trauma - Injury, Infection and Critical Care

54. Kelly, K.B., Koeppel, M.L., Como, J.J., Carter, J.W., McCoy, A.M., Claridge, J.A.,

Continued rationale of why hospital mortality is not an appropriate measure of trauma

outcomes, American Journal of Surgery

55. Kilgo PD, Weaver AA, Barnard RT, Love TP, & Stitzel JD. (2014). Comparison of

injury mortality risk in motor vehicle crash versus other etiologies. Accident; Analysis

and Prevention. 67, 137-47.

56. Knudson, M.M., Gomez, D., Haas, B., Cohen, M.J., Nathens, A.B., 2011, Three

thousand seven hundred thirty-eight posttraumatic pulmonary emboli: A new look at an

Butt 21

old disease, Annals of Surgery

57. Kohei Hasegawa, Takuyo Chiba, Yusuke Hagiwara, Hiroko Watase, Yusuke Tsugawa,

David F M Brown, Carlos A Camargo Quality of care for acute asthma in emergency

departments in Japan: a multicenter observational study.The journal of allergy and

clinical immunology. In practice 1 (5) p. 509-15.e1-3

58. Lad, S. P., Umeano, O. A., Karikari, I. O., Somasundaram, A., Bagley, C. A.,

Gottfried, O. N., ... & Boakye, M. (2013). Racial disparities in outcomes after spinal

cord injury. Journal of neurotrauma, 30(6), 492-497.

59. Laura V. Alvarado The Golden Hour for Nursing Nurse Leader 11 (4) p. 50-53

60. Maria Jose Santana, Sharon Straus, Russell Gruen, Henry T Stelfox A qualitative study

to identify opportunities for improving trauma quality improvement.Journal of critical

care 27 (6) p. 738.e1-7

61. Martin A Schreiber, Deborah J Neveleff Achieving hemostasis with topical hemostats:

making clinically and economically appropriate decisions in the surgical and trauma

settings. AORN journal 94 (5) p. S1-20

62. Matthew Scott Davenport, James H Ellis, Shoukofeh H Khalatbari, James D Myles,

Katherine A Klein Effect of work hours, caseload, shift type, and experience on

resident call performance. Academic radiology 17 (7) p. 921-7

63. Michael H McCafferty, Hiram C Polk Patient safety and quality in surgery. The

Surgical clinics of North America 87 (4) p. 867-81, vii

64. Michael L Barringer, Michael H Thomason, Patrick Kilgo, Linda Spallone Improving

Butt 22

outcomes in a regional trauma system: impact of a level III trauma center.American

journal of surgery 192 (5) p. 685-9

65. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group

(2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The

PRISMA Statement. Ann Intern Med, 151(4);

66. Morrison, C.A., Wyatt, M.M., Carrick, M.M., 2009, Impact of the 80-Hour Work

Week on Mortality and Morbidity in Trauma Patients: An Analysis of the National

Trauma Data Bank, Journal of Surgical Research

67. Myers, S.R., Salhi, R.A., Lerner, E.B., Gilson, R., Kraus, A., Kelly, J.J., Hargarten, S.,

Carr, B.G., A pilot study describing access to emergency care in two states using a

model emergency care categorization system, Academic Emergency Medicine

68. Natasha Jennings, Emma McKeown, Gerard O'Reilly, Glenn Gardner Australasian

emergency nursing journal Evaluating patient presentations for care delivered by

emergency nurse practitioners: a retrospective analysis of 12 months. AENJ 16 (3) p.

89-95

69. Nathens, A. B., Jurkovich, G. J., Maier, R. V., Grossman, D. C., MacKenzie, E. J.,

Moore, M., & Rivara, F. P. (2001). Relationship between trauma center volume and

outcomes. Jama, 285(9), 1164-1171.

70. Nathens, A.B., Xiong, W., Shafi, S., 2008, Ranking of trauma center performance: The

bare essentials, Journal of Trauma - Injury, Infection and Critical Care

71. Oyetunji T, Crompton JG, Efron DT, Haut ER, Chang DC, Cornwell EE 3rd, Baker

SP, & Haider AH. (2010). Simplifying physiologic injury severity measurement for

Butt 23

predicting trauma outcomes. The Journal of Surgical Research. 159(2), 627-32.

72. Patel, M.S., Malinoski, D.J., Zhou, L., Neal, M.L., Hoyt, D.B., 2013, Penetrating

oesophageal injury: A contemporary analysis of the National Trauma Data Bank,

Injury

73. Rebecca J Mitchell, Kate Curtis, Shanley Chong, Andrew J A Holland, S V S

Soundappan, Kellie L Wilson, Daniel T Cass Comparative analysis of trends in

paediatric trauma outcomes in New South Wales, Australia.Injury 44 (1) p. 97-103

74. Robert J Stephens, Susan E White, Michael Cudnik, Emily S Patterson Factors

associated with longer length of stay for mental health emergency department

patients.The Journal of emergency medicine 47 (4) p. 412-9

75. Roudsari, B., Nathens, A., Koepsell, T., Mock, C., Rivara, F., 2006, Analysis of

clustered data in multicentre trauma studies, Injury

76. S. Zafar, Z.G. Hashmi, E. Schneider, W.R. Greene, A.A. Shah, E.R. Haut, E.E.

Cornwell, A.H. Haider Morbidity or Mortality? Variations in Trauma Centers in the

Rescue of Older Injured Patients Journal of Surgical Research 186 (2) p. 687

77. Samir M Fakhry, Debbie Couillard, Casey T Liddy, David Adams, E Douglass

Norcross Trauma center finances and length of stay: identifying a profitability

inflection point. Journal of the American College of Surgeons 210 (5) p. 817-21, 821-3

78. Santana, M.J., Stelfox, H.T., Asbridge, M., Ball, C.G., Cameron, P., Dyer, D., Fortin,

C.M., Francescutti, L.H., Jaffe, K.M., Kirkpatrick, A.W., Kmetik, K., Kortbeek, J.,

Moore, L., Nathens, A., Noseworthy, T., Phan, N., Rivara, F., Singleton, B., Straus, S.,

Swiontkowski, M., Tallon, J., Travers, A., Zygun, D., Development and evaluation of

Butt 24

evidence-informed quality indicators for adult injury care, Annals of Surgery

79. Santana, M.J., Stelfox, H.T., Quality indicators used by trauma centers for performance

measurement, Journal of Trauma and Acute Care Surgery

80. Sebastian Schneeweiss, John D Seeger, John W Jackson, Scott R Smith Methods for

comparative effectiveness research/patient-centered outcomes research: from efficacy

to effectiveness. Journal of clinical epidemiology 66 (8 Suppl) p. S1-4

81. Shafi, S., Barnes, S., Nicewander, D., Ballard, D., Nathens, A.B., Ingraham, A.M.,

Hemmila, M., Goble, S., Neal, M., Pasquale, M., Fildes, J.J., Gentilello, L.M., Health

care reform at trauma centers-mortality, complications, and length of stay, Journal of

Trauma - Injury, Infection and Critical Care

82. Shafi, S., Parks, J., Ahn, C., Gentilello, L.M., Nathens, A.B., Hemmila, M.R.,

Pasquale, M.D., Meredith, J.W., Cryer, H.G., Goble, S., Neil, M., Price, C., Fildes, J.J.,

Centers for medicare and medicaid services quality indicators do not correlate with

risk-adjusted mortality at trauma centers, Journal of Trauma - Injury, Infection and

Critical Care

83. Sharma, S., De Mestral, C., Hsiao, M., Gomez, D., Haas, B., Rutka, J., Nathens, A.B.,

Benchmarking trauma center performance in traumatic brain injury: The limitations of

mortality outcomes, Journal of Trauma and Acute Care Surgery

84. Silvana Marasco, Geraldine Lee, Robyn Summerhayes, Mark Fitzgerald, Michael

Bailey Quality of life after major trauma with multiple rib fractures. Injury

85. Spaniolas, K., Velmahos, G.C., Alam, H.B., De Moya, M., Tabbara, M., Sailhamer, E.,

2008, Does improved detection of blunt vertebral artery injuries lead to improved

Butt 25

outcomes? Analysis of the National Trauma Data Bank®, World Journal of Surgery

86. Stelfox, H. T., Khandwala, F., Kirkpatrick, A. W., & Santana, M. J. (2012). Trauma

center volume and quality improvement programs. The journal of trauma and acute

care surgery, 72(4), 962-968.

87. Stitzel JD, Kilgo PD, Danelson KA, Geer CP, Pranikoff T, & Meredith JW. (2008).

Age thresholds for increased mortality of three predominant crash induced head

injuries. Annals of Advances in Automotive Medicine / Annual Scientific Conference

Association for the Advancement of Automotive Medicine. Association for the

Advancement of Automotive Medicine. Scientific Conference. 52, 235-44.

88. T H Rainer, J H H Yeung, S K C Cheung, Y K Y Yuen, W S Poon, H F Ho, C W Kam,

G N Cattermole, A Chang, F L So, C A Graham Assessment of quality of life and

functional outcome in patients sustaining moderate and major trauma: a multicentre,

prospective cohort study. Injury 45 (5) p. 902-9

89. Thomas S Helling, Paul W Nelson, B Todd Moore, Denise Kintigh, Kathy Lainhart Is

trauma centre care helpful for less severely injured patients?Injury 36 (11) p. 1293-7

90. Tominaga, G.T., Schaffer, K.B., Dandan, I.S., Kraus, J.F., 2013, Epidemiological and

clinical features of an older high-risk population of skateboarders, Injury

91. Trauma. (2000). In M. Agnes (Ed.)., Webster’s new world college dictionary (4th ed.,

p. 1377). New York: Macmillan.

92. Van Der Wilden, G.M., Yeh, D.D., Hwabejire, J.O., Klein, E.N., Fagenholz, P.J., King,

D.R., De Moya, M.A., Chang, Y., Velmahos, G.C., 2014, Trauma whipple: Do or don't

after severe pancreaticoduodenal injuries? An analysis of the national trauma data bank

(NTDB), World Journal of Surgery

Butt 26

93. Walter L Biffl, David T Harrington, Sarah D Majercik, Jayne Starring, William G

Cioffi The evolution of trauma care at a level I trauma center. Journal of the American

College of Surgeons 200 (6) p. 922-9

94. Willis, C.D., Stoelwinder, J.U., Lecky, F.E., Woodford, M., Jenks, T., Bouamra, O.,

Cameron, P.A., Applying composite performance measures to trauma care, Journal of

Trauma - Injury, Infection and Critical Care

95. Zachary F Meisel, Brendan G Carr, Patrick H Conway From comparative effectiveness

research to patient-centered outcomes research: integrating emergency care goals,

methods, and priorities. Annals of emergency medicine 60 (3) p. 309-16

96. Zain G. Hashmi, Syed Nabeel Zafar, Adil A. Shah, Eric B. Schneider, William R.

Leeper, Elliott R. Haut, Edward E. Cornwell, Adil H. Haider Unequal Burden of Injury

at Level I Trauma Centers: The Case for Efficiency in Resource Allocation for Optimal

Trauma Care Journal of the American College of Surgeons 219 (3) p. S104-S105

Butt 27

Appendix

Table A1 Papers using different cohort types

Cohort Type Papers (used numbers)

General trauma (meaning everyone who enters the trauma center on a regular basis)

Alali 2014, Ardolino 2012, Barringer 2006, Bukur 2012, Caputo 2013, Carr 2011, DuBose 2009, Franklin 2011, Glance 2012, Gomez 2010, Haas 2011, Helling 2005, Jennings 2013,Kilgo 2014, Lawson 2013, Schneeweiss 2013, Shafi 2010, Stelfox 2013

ISS (Injury severity score)(usually greater

than 15)Zafar 2014, Cole 2014, Mitchell 2013, Morrissey 2014 Franklin 2011, Glance 2012, Gomez 2010, Haas 2011, Helling 2005, Jennings 2013,Kilgo 2014, Lawson 2013, Schneeweiss 2013,

AIS (injury score) Macarenhas 2012, Rainer 2014

Blunt trauma Nathens 2001

Brain trauma Sharma 2013

Fast Patients Barbosa 2014

Pancreatic Trauma Van De Walden 2014

Trauma requiring Ventilation DuBose 2010

Spine trauma Schonfeld 2013,

Butt 28

Table A2 Papers containing specified metrics

Metric Papers usedMortality Cole 2014, Franklin 2011, Sharma

2013, Macarenhas 2012, Stelfox 2013, DuBose 2009, Helling 2005, Nathens 2001, Schonfeld 2013, Mitchell 2013, Morrissey 2014, Zafar 2014, Ardolino 2012, Caputo 2013Schneeweiss 2013 Mitchell 2013, Morrissey 2014, Zafar 2014

Morbidity DuBose 2009, Helling 2005, Mitchell 2013, Morrissey 2014, Zafar 2014Ardolino 2012, Caputo 2013

LOS (length of stay) Barringer 2006, Cole 2014, Jennings 2013, Nathens 2001, Schonfeld 2013

ISS (injury severity score) Barringer 2006,DuBose 2009, Helling 2005Kilgo,2014

Post injury complications Ardolino 2012,Schonfeld 2013Kilgo,2014

ICU days or ventilator days Barringer 2006,Cole 2014,Schonfeld 3013,Ardolino 2012,

Patient volume Nathens 2001, Caputo 2013, Stelfox 2012,Davenport 2010

MCS or PCS Rainer 2014

Referral pattern Jennings 2013

Hemorrhage and shock control Cole 2014

PE(pulmonary Embolisms), DVT (deep Venous Trombosis)

Barnes 2012

PCOR, CER Schneeweiss 2013

Butt 29