Advanced Medical Simulation Applications for Emergency Medicine Microsystems Evaluation and Training

WORKSHOP SESSION

Advanced Medical Simulation Applicationsfor Emergency Medicine MicrosystemsEvaluation and TrainingLeo Kobayashi, MD, Frank L. Overly, MD, Rollin J. Fairbanks, MD, MS, Mary Patterson, MD, MEd,Amy H. Kaji, MD, PhD, MPH, Eric C. Bruno, MD, Michael A. Kirchhoff, MD, Christopher G. Strother,MD, Andrew Sucov, MD, Robert L. Wears, MD, MS

AbstractParticipants in the 2008 Academic Emergency Medicine Consensus Conference ‘‘The Science of Simula-tion in Healthcare: Defining and Developing Clinical Expertise’’ morning workshop session on develop-ing systems expertise were tasked with evaluating best applications of simulation techniques andtechnologies to small-scale systems in emergency medicine (EM). We collaborated to achieve severalobjectives: 1) describe relevant theories and terminology for discussion of health care systems and medi-cal simulation, 2) review prior and ongoing efforts employing systems thinking and simulation programsin general medical sectors and acute care medicine, 3) develop a framework for discussion of systemsthinking for EM, and 4) explore the rational application of advanced medical simulation methods to adefined framework of EM microsystems (EMMs) to promote a ‘‘quality-by-design’’ approach. This articledetails the materials compiled and questions raised during the consensus process, and the resultingsimulation application framework, with proposed solutions as well as their limitations for EM systemseducation and improvement.

ACADEMIC EMERGENCY MEDICINE 2008; 15:1–13 ª 2008 by the Society for Academic EmergencyMedicine

Keywords: emergency medicine, health services research, health care evaluation mechanisms,simulation, systems analysis, systems theory

A ctivities of large organizations can be viewedand studied as the product of highly connected,dynamic, and interdependent groups exhibiting

emergent properties, i.e., systems. Application of thisframework as a problem-solving approach to contextual-ize institutions; explore the hierarchies and interactions

ª 2008 by the Society for Academic Emergency Medicine ISSN 1069-6563doi: 10.1111/j.1553-2712.2008.00247.x PII ISSN 1069-6563583 1

From the Department of Emergency Medicine, Alpert Medical School of Brown University (LK, FLO, AS), Providence, RI; theDepartment of Emergency Medicine, University of Rochester Medical Center (RJF), Rochester, NY; the Division of EmergencyMedicine, Cincinnati Children’s Hospital Medical Center (MP), Cincinnati, OH; the Department of Emergency Medicine, Harbor-UCLA Medical Center (AHK), Torrance, CA; the Department of Emergency Medicine, Lehigh Valley Hospital (ECB), Allentown,PA; the Department of Emergency Medicine, University of Medicine and Dentistry of New Jersey (MAK), Newark, NJ; theDepartment of Emergency Medicine, Mount Sinai Medical Center (CGS), New York, NY; Medical Director, Quality Management,Rhode Island Hospital, Providence, RI; the Department of Emergency Medicine, University of Florida (RLW), Jacksonville, FL;and the Clinical Safety Research Unit, Imperial College (RLW), London, UK.Received July 7, 2008; accepted July 8, 2008.Discussion participants, listed alphabetically (33): James Amsterdam, George Benedetto, Eric Brown, Eric Bruno, Michael Bullard,Brian Clauser, Stephen Donahue, Yue Dong, Rollin Fairbanks, Brian Gillett, Larry Gruppen, Amit Gupta, Leon Haley Jr., SheldonJacobson, Amy Kaji, Ravi Kapoor, Yoichi Kato, Rahul Khare, Michael Kirchhoff, Leo Kobayashi, Scott Korvek, Marjorie Lee White,Jacqueline Levesque, Brian Nelson, Yasuharu Okuda, Frank Overly, Mary Patterson, Emil Petrusa, Emily Powell, Nestor Rodriguez,Jenny Rudolph, Christopher Strother, and Robert Wears.This is a proceeding from a workshop session of the 2008 Academic Emergency Medicine Consensus Conference, ‘‘The Science ofSimulation in Healthcare: Defining and Developing Clinical Expertise,’’ Washington, DC, May 28, 2008.Disclaimer: This material is based on work supported by the University Emergency Medicine Foundation (UEMF), Rhode IslandHospital (RIH), Lifespan Risk Management, and Alpert Medical School of Brown University. Any opinions, findings, and conclusionsor recommendations expressed in this material are those of the authors and do not necessarily reflect the views of UEMF, RIH,Lifespan, or Brown University.Address for correspondence and reprints: Leo Kobayashi, MD; e-mail: [email protected].

of constituent personnel, behaviors, and specialized pro-cesses; and holistically study their performance, definessystems thinking.1 There are few settings where the rele-vance and potential utility of systems-based approachesto training and investigation are greater than in the prac-tice settings of acute care medicine. Emergency medicine(EM) in particular has unique work and environmentcharacteristics that make it unsuitable for traditionaleducational approaches of ‘‘see one, do one, teach one’’pedagogy and acceptance of clinical practice as guidedby anecdotal, impractical, or clinically disconnectedevidence. Methodical investigations, focused bedsideinterventions, and ongoing training, all conducted ina coordinated, feedback-driven and systems-mindedmanner, are necessary to optimize acute care practice.

As members of the Academic Emergency MedicineConsensus Conference workshop group tasked withformulating a corroborated opinion on best applicationsof simulation techniques and technologies to small-scalesystems in EM, the authors assisted in organizing andfacilitating the consensus process. Representatives fromgeneral EM, pediatric EM, medical simulation, humanfactors engineering, patient safety, operations research,and quality management communities were invited. Thisarticle and associated Web addenda reflect the collabo-rative efforts directed toward achievement of severalobjectives: 1) description of relevant theories and termi-nology for discussion of health care systems and medi-cal simulation, 2) review of prior and ongoing effortsemploying systems thinking and simulation programs ingeneral medical sectors and acute care medicine, 3)development of a framework for discussion of systemsthinking for EM, and 4) exploration of the rational appli-cation of advanced medical simulation methods to adefined framework of EM microsystems (EMMs) to pro-mote a ‘‘quality-by-design’’ approach.

SYSTEMS THINKING IN MEDICINE AND EM

A review of systems-based approaches to health care,medicine, and EM (106 references) is presented in DataSupplement S1, available as supporting information inthe online version of this paper.

PRIOR AND ONGOING SIMULATIONAPPLICATIONS FOR SYSTEMS THINKINGIN GENERAL HEALTH CARE

A review of prior and ongoing uses of simulation forsystems-based education and improvement in generalhealth care (47 references) is presented in Data Supple-ment S2, available as supporting information in theonline version of this paper.

APPLICATION OF QUALITY MANAGEMENT ANDHUMAN FACTORS PRINCIPLES TO EMSYSTEMS THINKING THROUGH SIMULATION

One of the major impediments to systems thinking inEM has been the difficulty in maintaining a conceptuali-zation of the emergency department (ED) as a ‘‘nonlin-ear, complex, adaptive system’’2 that features recursivesystem structures. As a truly multidisciplinary environ-

ment, where personnel of diverse training backgroundsand roles interact ceaselessly in a multipurpose facility,the ED can be viewed as the dynamic composite of asubstantial collection of interconnected systems. Despitesubstantial dissimilarities often neglected in compari-sons of health care with aviation, nuclear power, andother high-stakes industries,3 the underlying concept ofapplying systems-based thinking for ED performanceanalysis and enhancement remains compelling.

Education and improvement efforts at the organiza-tional level in EM systems, generally not privy toadvantages of top-down management exemplified bythe highly structured chains-of-command for conflict-related military medicine,4–7 are challenging proposi-tions for a myriad of reasons. Taking the example of a‘‘simple’’ adverse event, such as a delayed endotrachealintubation with transient patient hypoxia, a proximatecause may be identified (nonfunctional laryngoscope)and quickly corrected with a definitive-appearing inter-vention (laryngoscope troubleshooting or replacement).However, this maneuver would leave unaddressed theunderlying, entrenched system failures that will permitsimilar adverse events in the future:

— Absence of defined process to test laryngoscopeprior to its being placed in service;

— No established provision for backup laryngoscopicdevice or rescue device in intubation kit;

— Inadequate education and monitoring of patient careteam expertise in knowledge ⁄ skills for adequate air-way and ventilatory management;

— Inadequate standardization of preintubation prep-aration, equipment check, and time-out throughprotocols;

— Underutilized incident reporting and follow-up infra-structure.

Even a cursory attempt to study and overcome theconcerns raised by a seemingly straightforward mal-function of standard intubation equipment revealsnumerous difficulties to be anticipated when applyingsystems thinking and committing to its implications forEM. A proposed framework and questions raised inresponse by the consensus panel, along with corrobo-rated opinions on possible solutions, are presentedbelow.

Proposed Framework for EM Systems SimulationSeveral methods have been formulated with which toconceptualize, frame, and organize systems involved inhealth care (see Data Supplement S1): Reason’s ‘‘Swisscheese’’8 and Helmreich’s threat-and-error9,10 models,the HealthCare Matrix employing Accreditation Councilfor Graduate Medical Education (ACGME) core compe-tencies and Institute of Medicine goals11 and the Sys-tem Engineering Initiative for Patient Safety (SEIPS)model12 are notable. The consensus track members pro-pose employing elements from established models andbuilding on them with a microsystems perspective forapplied systems thinking at the sharp end of EM clinicalpractice.

The proposed approach begins with the followingdefinitions by Nelson, Batalden, et al.:13

2 Kobayashi et al. • MEDICAL SIMULATION FOR EMERGENCY MEDICINE MICROSYSTEMS

‘‘Clinical microsystems are the small, functional,front-line units that provide most health care tomost people. They are the essential building blocksof larger organizations and of the health system.They are the place where patients and providersmeet.’’

‘‘A clinical microsystem is a small group of peoplewho work together on a regular basis to providecare to discrete subpopulations of patients. It hasclinical and business aims, linked processes and ashared information environment, and it producesperformance outcomes. Microsystems evolve overtime and are often embedded in larger organiza-tions. They are complex adaptive systems, and assuch they must do the primary work associatedwith core aims, meet the needs of internal staff, andmaintain themselves over time as clinical units.’’

These units function in a defined, coordinatedmanner:

‘‘The patients and staff work to meet patients’needs by engaging in direct care processes—accessing systems, assessing needs, diagnosingproblems, establishing treatment plans, and follow-ing up over time. These direct care processes areassisted by supporting processes that involvedistinct tools and resources such as medicalrecords, scheduling, diagnostic tests, medications,and billing. The result of the interaction betweenpatients and staff and clinical and supportprocesses is to produce patterns of criticalresults—biological outcomes, functional status andrisk outcomes, patient perceptions of goodness ofcare, and cost outcomes that combine to representthe value of care.’’13

Emergency medicine microsystems have been refer-enced previously14 due to their significance. By envision-ing large-scale systems (macrosystems) as constructedof smaller systems (microsystems) that produce quality,safety, and cost outcomes at the frontline of care,13,15,16

it can be inferred that ‘‘the outcomes of the macro-systems can be no better than the microsystems of whichit is composed.’’13 This insight allows linkage of EMsystems and microsystems to organizational perfor-mance metrics such as the Patient Value Compass.

From these descriptions and concepts, we can definethe elemental care provider structures (ED micro-systems) that in aggregate form mesosystems (e.g.,large-scale ED facilities, acute care services) and inturn compose hospital and health care macrosystems.17

Interacting components of various ED and contiguoussystems can now be visualized in the context of a spe-cific microsystem, e.g., the clinical care providercohorts, diagnostic and treatment algorithms, securityservice and social work personnel, blood banking pro-tocols, and operating room scheduling procedures thatcoalesce to resuscitate a high-acuity penetratingtrauma victim of gang violence. In contrast to reduc-tionist views that isolate system components and severcritical interdependencies, systems analysis of EMMscan reveal larger- and smaller-scale structures and

complex processes without succumbing to holisticovergeneralization.18

In light of the common perception and employmentof medical simulation as a prototypical method for thedevelopment and evaluation of medical knowledge andtechnical competencies, advanced medical simulationinitiatives may risk leaving health care teams withoutthe mechanisms necessary to acquire the broader clin-ical skill sets required when responding to a variety ofsituations. For example, an ED line-sepsis preventionquality improvement program may disproportionatelyallocate available resources toward increasing simula-tion fidelity for better compliance with aseptic tech-nique during ultrasound-guided catheter placement.By focusing in on a narrow spectrum of the processbeing simulated, e.g., catheter equipment characteris-tics and procedural sequence, universal higher-levelcompetencies such as communication, coordination,problem-solving, and situational awareness that areessential to optimal functioning of microsystems (andlarger-scale constructs) may be neglected. As in otherfields, structuring the problem with a method thatextends beyond the limited perspective afforded froma purely technologic exploitation of simulation iscrucial.19 Directing simulation toward microsystemsintroduces the possibility of working on higher-orderED functions and addressing situations that are highlydynamic, unanticipated, incompletely specified, orotherwise problematic.

An overview of the consensus panel’s attempt toframe system component ingredients of EMMs into a[System - Threat - Simulation - Metric ⁄ Outcome] matrixis presented in Table 111,20,21 (see Data Supplement S1and Figure 1 for full details). Substantial work is stillneeded to identify system components critical to themicrosystems they give rise to, find consistent patternsof system interfacing and aggregation, and determinethe best means to monitor and assist each componentfor improved performance. An educational andresearch agenda on the work needed in the realm ofEM systems simulation is presented in the form ofconsensus questions.

Consensus Question 1: How Should Simulation BeApplied to Improve and Study EM (Micro-)systems? IsThere a Rational Way to Propose and Determine WhichTypes of Simulation, in What Setting, at What Time, forWhom, With What Objectives and Outcomes, Will ProveUseful for EMM Improvement?

Prior work has pioneered the application of simula-tion to EM,22 introduced system-based analysis oferrors in EM23 and described the use of simulation forEM error reduction.24,25 Application of ergonomics26–28

and simulation29 for detection and prevention of sys-tem-level errors has been proposed before. Just asmedical and other devices should be tested under actualconditions of expected use, health care systems wouldbe expected to benefit from pilot-testing and trouble-shooting activities. Examples of previous acute caresystems improvement efforts utilizing computationalsimulation (CS)30–65 and physical simulation (PS)66–86 arelisted in Tables 2 and 3.

ACAD EMERG MED • www.aemj.org 3

The specific capabilities afforded by simulations(especially in situ) can enable safe, reproducible obser-vation and testing of systems and processes to generatevaluable data for quality management activities: objec-tive system probing to identify problems for qualityplanning, acquisition of replicable data to analyze forquality control, and reiterative interventions with feed-back loops for quality improvement. One path to desig-nate simulation applications for EM systems positionsthese attributes within plan-do-study-act (PDSA) cycles(aka Schering cycle;87 see Figure 2).

1. System probing with in situ advanced medical sim-ulation to assist with elucidation of causality and toselect high-yield targets [plan];

2. On-site controlled trials with medical simulationvariants to pilot intervention on systems [do];

3. Reproducible, scripted, safe bedside advancedmedical simulation exercises for upstream and down-stream data acquisition for intervention effects analysis[study];

4. Repeated standard and in situ simulation educationand training events as a forcing function to implementand maintain interventions [act].

In situ advanced medical simulation, conductedproperly, is ‘‘field work’’88 to characterize and under-stand distributed acute care work activities; their con-stituent staff; workspace components, structures,

connections, and transferences; flow and its disrup-tion; and timing plus timeliness, along with conflictsand weaknesses necessitating compensatory defenses.To avert indiscriminate use of advanced medicalsimulation for systems improvement, the consensuspanel evaluated the potential utility of common CSand PS methods with respect to individual EM systemcomponents (see Figure 1). The maintenance of theinterconnected and emergent properties of complexmicrosystems by in situ and progressive simulationsfor intra- and intermicrosystem exercises is visible inthe on-site pediatric trauma resuscitation exampleembedded in the [System - Threat - Simulation -Metric ⁄ Outcome] matrix.

Advanced medical simulation not only assists effortsto improve provider knowledge base, clinician skills,team communication behaviors, and care processes in acontrolled manner, but also helps as a feeder mechanismin determining system-level needs for such interventionsand their impact. Retrospective incident run-throughs,proactive systems probing, and trial interventionsthrough pilot programs are made possible withoutrelying on sentinel events involving actual patients. Byexamining the ‘‘ongoing, interconnected streams ofactivity’’ across individuals, devices, and settings as theunit of study,88 simulation can reveal the ‘‘combinationof cumulative effect and critical coincidences’’20 that

Table 1Ongoing Feedback for Improvement


propagate minor errors to cause system failure. Thisapproach to quality by design is made possible throughuse of simulation methods that create an engineeredwindow on the system.89

Consensus Question 2: What Research MethodologiesShould Be Applied to Study the Use of Simulation andEstablish Its Value for EMMs? There is ongoing diffi-culty in establishing a direct linkage between educa-tional and improvement efforts using simulation andtheir effects on patient safety and outcomes in generalhealth care90 and EM.91 The consensus track membersthereby recommend the categorical assessment ofspecific aspects of EM systems simulation validity.Feasibility, utility, verisimilitude, real-world efficacy,and cost-effectiveness were facets determined torequire independent examination. With the probablerole of validation attempts in widespread efforts to per-suade institutional leaders and shareholders of thevalue of simulation for organizational improvement,analyses focusing on returns-on-investment (ROI) wereidentified as likely to have considerable impact.

For all aspects of system simulation validation, con-ference track participants agreed on the need toidentify, select, and characterize relevant outcomemeasures. Whether deployed in simulated settings orlive clinical areas, depending on the EM system ele-ments being evaluated, surrogate markers were felt tobe instrumental in accomplishing measurement of sys-tem performance and outcomes. This was based onappreciation of a necessary selectivity for applyingtime- and effort-intensive simulation to the study of EMsystems featured in high-risk and low-frequency eventsthat are difficult to examine with other methods.

Outcome metrics previously demonstrated torespond appropriately to improved medical care weredeemed preferable, specifically so as to allow quantita-

tive analyses of system failures and deviations frombest care practices; drilling down with qualitative inves-tigations would follow to propel improvement andmaintenance efforts. Quasi-experimental designs werefavored over traditional study designs that would resultin unrealistic logistic demands. Notably, concerns werevoiced to emphasize the complementary roles of quali-tative and quantitative methods in system simulationvalidation and to require these metrics to be nestedwithin a well-defined feedback structure to driveongoing assessment and improvement in a systematicfashion.

Examples of potential surrogate measures to initiatediscussions on simulation ROI for provocative test-ing ⁄ probing and improvement interventions for EMsystems involved in pediatric trauma management areshown in Figure 1. These may represent the potentialvalue of simulation methods to institutional safety andimproved patient outcomes via an ‘‘incident learningsystem,’’92 an approach that has been postulated toovercome the retrospective limitations of standarderror reporting systems.

Consensus Question 3: How Does One Elicit Micro-system Processes in an Integrated Manner to Unify theSimulated Care of Individual Patients (e.g., InteractiveComputer-controlled Manikin) with the Simulation ofLarger-scale Systems (e.g., PC-based Modeling), i.e.,Are Microsystem and Macrosystem Simulations Recon-cilable? The systems expertise consensus track agreed

Table 2CS Applications for Acute Care Systems Improvement

ED census model development and forecasting30,31

ED flow modeling,32–35 flow analysis by triage level,36 andtriage methodology modeling37

ED length-of-stay analysis38–40 and best demonstratedprocess intervention41

ED maximum patient capacity simulation42

ED radiology queuing analysis43 and activity-based-costingmodeling44

ED personnel staffing model simulation45–52

ED process improvement with modeling53 and Six Sigma54

Effect of ED fast track addition on wait times55

Fire ⁄ out-of-hospital service utilization analysis withGeographic Information System data56

General ED flow and operations modeling57–61

Out-of-hospital workflow process modeling62

Simulation-based evaluation of ED facility restructuringoptions63

Stochastic simulation model of inpatient bed occupancyeffect on emergency admissions64

Trauma resuscitation decision-making error monitoring65

Disaster and surge-related materials are not reviewed.CS = computational simulation; ED = emergency department.

Table 3PS Applications for Acute Care Systems Improvement

Manikin simulationsBar-code medication order entry system testing66

Defibrillator usability and safety assessment67

Development of simulation-based curricula for EMresidency68 and EM-specific systems-based practicecompetency training69

ED multiple concurrent patient simulation exercises70

In-hospital medical resuscitation team and systemresponse assessment71–75

Needs analysis and integrated design of future EDresuscitation areas76

New ED facility resuscitation capabilities testing77

New facility in-hospital Code Blue Team performanceevaluation78

Pediatric in-hospital medical resuscitation responseassessment79

Pediatric trauma stabilization statewide assessment80

Usability testing and standardized patient simulationsCritical care medication infusion error detection by smartpumps81

ED medication management system effects on prescribingpatterns82

ED performance in recognition and treatment ofinhalational anthrax SPs83

ED triage system performance evaluation through scriptedscenarios84

Hospital electrical supply quality and effects on medicalequipment85

Hospital pneumatic tube system analysis86

ED = emergency department; EM = emergency medicine;PS = physical simulation; SP = standardized patient.




on the propriety and best-fit of manikin-based andother PS modalities for sharp-end clinical applications,whereas CS was deemed better suited for macrosystemstructures. Given this theoretical divergence arisingfrom both practical limitations and the current state ofbridging technologies, such as virtual reality, a hybridapproach employing PS and CS was discussed.

Microsystem experts state that ‘‘the clinical unit has asemi-permeable boundary that mediates relationshipswith patients and with many support services andexternal microsystems. . . . it is embedded in, influ-ences, and is influenced by a larger organizationthat itself is embedded in a certain environment—apayment environment, a regulatory environment, acultural-social-political environment. Thus, the simpleconcept of a clinical microsystem is in fact a complex,adaptive system that evolves over time.’’13 In this con-text, microsystems-probing PS might serve to generateperformance and outcomes data incorporating EDoperations efficiency, cost, and variances or personnelknowledge, skills, and attitude (KSA) deficiencies,thereby creating or refining CS models of EM mesosys-tems and larger-scale structures for improvement andoptimization. CS models may direct output towardPS-based interventions at the level of the individualhealth care provider or team.

As a core element of systems-based organizationalimprovement, the importance of feedback and feedfor-ward loops is best expressed as follows: ‘‘The quality andvalue of care produced by a large health system can beno better than the services generated by the smallsystems of which it is composed.’’13 System interdepen-dencies at the micro and macro scales in health care havebeen previously explored by Carayon et al.12 and Karshet al.,21 as well as by macroergonomics treatment of out-patient diagnostic laboratory testing by Hallock et al.93

Scalar efforts in other areas of biology and medicinecorroborate the possibilities of similar approaches.94

Question 4: What Are Some Limitations of Usingthe Microsystems Approach with In Situ Simula-tions? Decentralized health care management and frag-mentation of care systems, insufficient oversight andaccountability through quality metrics, inadequate infor-matics and technology infrastructure, along with a cul-ture of practitioner autonomy and insularity, conspire tomake the envisioning of health care institutions as a‘‘fully linked process of care’’95 difficult. Even within thedefined microsystems of EDs, operating characteristicssuch as nurse:patient ratios, bed layout, and patientqueuing for limited resources can be incompatible (oftenby necessity) with elements of systems engineering.96

Not surprisingly, attempts to apply system thinkingto health care have frequently met with limited real-world success;97 recent efforts have been subject to‘‘hype cycles.’’98 Even when put into effect, patientsafety and enhanced care have not been a guaranteedoutcome.99–101 Invoking the cooperative involvement ofindividuals well versed in medicine, human factors,cognitive psychology, patient safety, systems manage-ment, and simulation to continue similar attempts willremain challenging. Insofar as all technologies harborshortcomings, reviews of the limitations of simulationF

igu

re1

.T

he

sim

ula

tio

nap

pli

cati

on

matr

ixfo

rE

Msy

stem

sexp

ert

ise

rep

rese

nts

ap

rop

ose

dh

yb

rid

of

syst

em

sth

inki

ng

inE

Mw

ith

dif

fere

nt

sim

ula

tio

nm

od

ali

ties

an

dth

eir

ap

pli

-ca

tio

ns,

as

lin

ked

tore

pre

sen

tati

ve

exam

ple

so

fevalu

ati

ve

⁄ass

ess

men

t⁄m

easu

rem

en

tele

men

tsfo

rd

evelo

pm

en

tan

dm

ain

ten

an

ceo

fo

rgan

izati

on

al

exp

ert

ise.

Th

ele

ftth

ird

rep

re-

sen

tsth

ep

rop

ose

dco

reE

Mp

ati

en

tca

re–r

ela

ted

do

main

s[p

ati

en

t,p

roce

sses

+ta

sks,

peo

ple

+en

vir

on

men

t,cu

ltu

re⁄o

rgan

izati

on

,an

din

terd

om

ain

lin

kag

es

+in

terf

ace

s],

class

ified

furt

her

into

ED

syst

em

san

dE

Dsy

stem

com

po

nen

ts).

Th

em

idd

leth

ird

refl

ect

scu

rren

tly

acc

ess

ible

sim

ula

tio

nte

chn

olo

gie

s[s

tan

dard

man

ikin

-base

d,

insi

tum

an

ikin

-base

d,

no

nm

a-

nik

inp

hysi

cal

(e.g

.,ta

sktr

ain

ers

,S

Ps)

,u

sab

ilit

yte

stin

g,

an

dco

mp

uta

tio

nal

(e.g

.,fl

ow

mo

deli

ng

)].

Are

lati

ve

uti

lity

⁄ap

pli

cab

ilit

ysc

ale

has

been

devis

ed

toin

dic

ate

the

po

ssib

lero

leo

feach

sim

ula

tio

nm

od

ality

for

ind

ivid

ual

EM

care

syst

em

com

po

nen

ts(f

rom

‘‘–’

’fo

rm

inim

al⁄

no

ap

pli

cab

ilit

yto

‘‘+

++

+’’

for

op

tim

al

uti

lity

un

matc

hed

by

oth

er

meth

od

s).

Th

eri

gh

tmo

stth

ird

sect

ion

hig

hlig

hts

the

evalu

ati

ve,

ed

uca

tio

nal⁄

inte

rven

tio

nal,

an

dm

easu

rem

en

tasp

ect

so

fsy

stem

-base

do

rgan

izati

on

al

imp

rovem

en

tin

the

fram

ew

ork

of

PD

SA

cycl

es

(see

text)

.A

nexam

ple

of

ah

igh

-acu

ity

ped

iatr

icb

lun

ttr

au

ma

pati

en

tin

situ

sim

ula

tio

nis

use

dto

dem

on

stra

tep

ote

nti

al

item

sfo

rexp

lora

tio

nw

ith

inth

esy

stem

fram

ew

ork

;th

eap

pli

cab

ilit

yo

fth

esp

eci

fic

scen

ari

ofo

reach

ED

syst

em

com

po

nen

tis

sho

wn

gra

ph

ically

(fo

llo

wst

ipp

led

lin

eto

vert

ical

bars

for

sim

ula

tio

np

oin

ts-o

f-u

se)

an

dca

teg

ori

cally

by

pati

en

tca

rep

hase

(1=

pre

arr

ival;

2=

tria

ge;

3=

init

ial

ED

resu

scit

ati

on

;4

=E

Dd

iag

no

stic

+th

era

peu

tic

inte

rven

tio

ns;

5=

soci

al

sup

po

rt;

an

d6

=tr

an

siti

on

;fo

llo

wst

raig

ht

lin

eto

dark

gra

yb

lock

s;w

hit

ean

dli

gh

tg

ray

text

mark

go

od

an

dp

oss

ible

uti

lity

resp

ect

ively

for

the

ph

ase

s).

No

teth

at

ele

men

tso

fco

gn

itiv

e(i

nd

ivid

ual)

,te

am

wo

rk(g

rou

p),

an

dp

roce

-d

ura

l(t

ech

nic

al)

exp

ert

ise

have

been

deem

ph

asi

zed

tom

ain

tain

focu

so

nsy

stem

sexp

ert

ise.

Sim

ula

tio

nte

chn

olo

gie

seit

her

wit

hli

mit

ed

dep

loym

en

tin

healt

hca

reo

rst

ill

un

derg

o-

ing

exte

nsi

ve

develo

pm

en

t(e

.g.,

vir

tual

reali

ty,

mu

ltip

layer

on

lin

ero

le-p

layin

gg

am

esy

stem

s)h

ave

no

tb

een

incl

ud

ed

inth

ecu

rren

tvers

ion

of

the

matr

ix.

BA

RS

=B

eh

avio

rally

An

cho

red

Rati

ng

Sca

le;

BiP

AP

=b

ilevel-

po

siti

ve

air

way

pre

ssu

re;

CP

OE

=co

mp

ute

rize

dp

hysi

cian

ord

er

en

try;

CT

=co

mp

ute

dto

mo

gra

ph

y;

ED

=em

erg

en

cyd

ep

art

men

t;E

GD

-T

=earl

yg

oal-

dir

ect

ed

thera

py;

EM

=em

erg

en

cym

ed

icin

e;

EM

R=

ele

ctro

nic

med

ical

reco

rd;

EM

S=

em

erg

en

cym

ed

ical

serv

ices;

GI

=g

ast

roin

test

inal;

MR

I=

mag

neti

cre

son

an

ceim

ag

ing

;M

RS

A=

meth

icil

lin

-resi

stan

tS

tap

hylo

cocc

us

au

reu

s;O

R=

op

era

tin

gro

om

;P

DS

A=

pla

n-d

o-s

tud

y-a

ct;

PE

EP

=p

osi

tive

en

d-e

xp

irato

ryp

ress

ure

;P

ICU

=p

ed

iatr

icin

ten

sive

care

un

it;

RN

=re

gis

tere

dn

urs

e;

SB

AR

=S

itu

ati

on

Back

gro

un

dA

ssess

men

tR

eco

mm

en

dati

on

com

mu

nic

ati

on

too

l;S

IM=

ad

van

ced

med

ical

sim

ula

tio

nu

tili

zin

gm

an

ikin

tech

no

lo-

gie

s(i

ncl

ud

ing

hyb

rid

man

ikin

⁄SP

sim

ula

tio

n);

SP

=st

an

dard

ized

pati

en

t;U

⁄S=

ult

raso

un

d;

ven

t=

ven

tila

tor.


are surfacing.102–104 The innate complexity of healthcare systems, as well as their safety structures, therequisite time and effort for sustained improvementinterventions, and unyielding institutional culture, willadd to the challenge. Consequently, advancing patientsafety and health care quality through high-fidelitysimulation will likely prove difficult without substantialengagement, coordination, and commitment at all levelsof health care.105

The authors acknowledge Anna C. Cousins for her assistance inmanuscript preparation.

References

1. O’Connor J, McDermott I. The Art of SystemsThinking: Essential Skills for Creativity and Prob-lem-solving. San Francisco, CA: Thorsons Publish-ing, 1997.

2. Chinnis A, White KR. Challenging the dominantlogic of emergency departments: guidelines fromchaos theory. J Emerg Med. 1999; 17:1049–54.

3. Gaba DM. Out of this nettle, danger, we pluck thisflower, safety: healthcare vs. aviation and otherhigh-hazard industries. Simul Healthc. 2007; 2:213–7.

4. Holcomb JB, Dumire RD, Crommett JW, et al.Evaluation of trauma team performance usingan advanced human patient simulator for resusci-tation training. J Trauma. 2002; 52:1078–85.

5. Bruce S, Bridges EJ, Holcomb JB. Preparingto respond: Joint Trauma Training Center andUSAF Nursing Warskills Simulation Laboratory.Crit Care Nurs Clin North Am. 2003; 15:149–62.

6. McCarthy M. US military revamps combat medictraining and care. Lancet. 2003; 361:494–5.

Figure 2. Diagram representing potential roles of in situ advanced medical simulation in PDSA cycles for improvement of micro-systems and larger structures. ED = emergency department; ICU = intensive care unit; PDSA = plan-do-study-act.


7. Sohn VY, Miller JP, Koeller CA, et al. From thecombat medic to the forward surgical team: theMadigan model for improving trauma readiness ofbrigade combat teams fighting the global war onterror. J Surg Res. 2007; 138:25–31.

8. Reason J. Human error: models and management.BMJ. 2000; 320:768–70.

9. Helmreich RL. On error management: lessons fromaviation. BMJ. 2000; 320:781–5.

10. Helmreich RL, Musson DM. The University ofTexas threat and error management model: com-ponents and examples. Available at: homepage.psy.utexas.edu ⁄ homepage ⁄ group ⁄ HelmreichLAB ⁄Publications ⁄ pubfiles ⁄ Pub248.pdf. Accessed Jun28, 2008.

11. Bingham JW, Quinn DC, Richardson MG, MilesPV, Gabbe SG. Using a healthcare matrix to assesspatient care in terms of aims for improvement andcore competencies. Jt Comm J Qual Pat Saf. 2005;31:98–105.

12. Carayon P, Schoofs Hundt A, Karsh BT, et al.Work system design for patient safety: the SEIPSmodel. Qual Saf Health Care. 2006; 15(Suppl 1):i50–8.

13. Nelson EC, Batalden PB, Huber TP, et al. Micro-systems in health care: Part 1. Learning from high-performing front-line clinical units. Jt Comm JQual Improv. 2002; 28:472–93.

14. Espinosa JA, Case R, Kosnik LK. Emergencydepartment structure and operations. Emerg MedClin North Am. 2004; 22:73–85.

15. Mohr JJ, Barach P, Cravero JP, et al. Microsystemsin health care: Part 6. Designing patient safety intothe microsystem. Jt Comm J Qual Saf. 2003;29:401–8.

16. Batalden PB, Nelson EC, Edwards WH, GodfreyMM, Mohr JJ. Microsystems in health care: Part 9.Developing small clinical units to attain peakperformance. Jt Comm J Qual Saf. 2003; 29:575–85.

17. Kosnik L, Espinosa JA. Microsystems in healthcare: Part 7. The microsystem as a platform formerging strategic planning and operations. JtComm J Qual Saf. 2003; 29:452–9.

18. Koestler A. Some general properties of self-regu-lating open hierarchic order (SOHO). in KoestlerA, Smythies JR (eds.). Beyond Reductionism: NewPerspectives in the Life Sciences. London, UK:Hutchinson, 1969, pp 210–6.

19. Dekker S, Dahlstrom N, van Winsen R, Nyce JN.Crew resilience and simulator training in aviation.In: Hollnagel E, Nemeth CP, Dekker S (eds.)Remaining Sensitive to the Possibility of Failure.Aldershot, UK: Ashgate, 2008, pp 119–26.

20. Catchpole KR, Giddings AE, de Leval MR, et al.Identification of systems failures in successful pae-diatric cardiac surgery. Ergonomics. 2006; 49:567–88.

21. Karsh BT, Holden RJ, Alper SJ, Or CK. A humanfactors engineering paradigm for patient safety:designing to support the performance of thehealthcare professional. Qual Saf Health Care.2006; 15(Suppl 1):i59–65.

22. Small SD, Wuerz RC, Simon R, Shapiro N, ConnA, Setnik G. Demonstration of high-fidelity simula-tion team training for emergency medicine. AcadEmerg Med. 1999; 6:312–23.

23. Adams JG, Bohan JS. System contributions toerror. Acad Emerg Med. 2000; 7:1189–93.

24. Croskerry P, Wears RL, Binder LS. Setting theeducational agenda and curriculum for error pre-vention in emergency medicine. Acad Emerg Med.2000; 7:1194–200.

25. Cosby KS, Croskerry P. Patient safety: a curricu-lum for teaching patient safety in emergency medi-cine. Acad Emerg Med. 2003; 10:69–78.

26. Schaefer HG, Helmreich RL, Scheidegger D.Human factors and safety in emergency medicine.Resuscitation. 1994; 28:221–5.

27. Wears R, Leape LL. Human error in emergencymedicine. Ann Emerg Med. 1999; 34:370–2.

28. Wears RL, Perry SJ. Human factors and ergonom-ics in the emergency department. Ann EmergMed. 2002; 40:206–12.

29. Schenkel S. Promoting patient safety and prevent-ing medical error in emergency departments. AcadEmerg Med. 2000; 7:1204–22.

30. Flottemesch TJ, Gordon BD, Jones SS. Advancedstatistics: developing a formal model of emergencydepartment census and defining operational effi-ciency. Acad Emerg Med. 2007; 14:799–809.

31. Hoot NR, Leblanc LJ, Jones I, et al. Forecastingemergency department crowding: a discreteevent simulation. Ann Emerg Med. 2008; 52:116–25.

32. Blasak RE, Starks DW, Armel WS, Hayduk MC.The use of simulation to evaluate hospital opera-tions between the emergency department and amedical telemetry unit. Proc 2003 Winter SimulConf. 2003: 1887–93.

33. Wiinamaki A, Dronzek R. Using simulation in thearchitectural concept phase of an emergencydepartment design. Proc 2003 Winter Simul Conf.2003: 1912–6.

34. Codrington-Virtue A, Whittlestone P, Kelly J,Chaussalet T. An interactive framework for devel-oping simulation models of hospital accident andemergency services. Stud Health Technol Inform.2005; 114:277–83.

35. Nielsen AL, Hilwig H, Kissoon N, Teelucksingh S.Discrete event simulation as a tool in optimizationof a professional complex adaptive system. StudHealth Technol Inform. 2008; 136:247–52.

36. Mahapatra S, Koelling C, Patvivatsiri L, FraticelliB. Pairing emergency severity index-level triagedata with computer aided system design toimprove emergency department access andthroughput. Proc 2003 Winter Simul Conf. 2003:1917–25.

37. Connelly LG, Bair AE. Discrete event simulation ofemergency department activity: a platform for sys-tem-level operations research. Acad Emerg Med.2004; 11:1177–85.

38. Chin L, Fleisher G. Planning model of resourceutilization in an academic pediatric emergencydepartment. Pediatr Emerg Care. 1998; 14:4–9.


39. Ruohonen T, Neittaanmaki P, Teittinen J. Simula-tion model for improving the operation of theemergency department of special health care. Proc2006 Winter Simul Conf. 2006: 453–8.

40. Sinreich D, Marmor Y. Ways to reduce patientturnaround time and improve service quality inemergency departments. J Health Organ Manag.2005; 19:88–105.

41. Hoffenberg S, Hill MB, Houry D. Does sharingprocess differences reduce patient length of stay inthe emergency department? Ann Emerg Med.2001; 38:533–40.

42. Baesler FF, Jahnsen HE, DaCosta M. The use ofsimulation and design of experiments for estimat-ing maximum capacity in an emergency room.Proc 2003 Winter Simul Conf. 2003: 1903–6.

43. Rosenquist CJ. Queuing analysis: a useful planningand management technique for radiology. J MedSyst. 1987; 11:413–9.

44. Glick ND, Blackmore CC, Zelman WN. Extendingsimulation modeling to activity-based costing forclinical procedures. J Med Syst. 2000; 24:77–89.

45. Kumar AP, Kapur R. Discrete simulation applica-tion–scheduling staff for the emergency room.Proc 1989 Winter Simul Conf. 1989: 1112–20.

46. Rosetti MD, Trzcinski GF, Syverud SA. Emergencydepartment simulation and determination of opti-mal attending physician staffing schedules. Proc1999 Winter Simul Conf. 1999: 1532–40.

47. Draeger MA. An emergency department simula-tion model used to evaluate alternative nurse staff-ing and patient population scenarios. Proc 1992Winter Simul Conf. 1992: 1057–64.

48. McGuire F. Using simulation to reduce length ofstay in emergency departments. Proc 1994 WinterSimul Conf. 1994: 861–7.

49. Evans GW, Gor TB, Unger E. A simulation modelfor evaluating personnel schedules in a hospitalemergency department. Proc 1996 Winter SimulConf. 1996: 1205–9.

50. Coats TJ, Michalis S. Mathematical modeling ofpatients flow through an accident and emergencydepartment. Emerg Med J. 2001; 18:190–2.

51. Centeno MA, Giachetti R, Linn R. A simulation-ILPbased tool for scheduling ER staff. Proc 2003Winter Simul Conf. 2003:1930–8.

52. Sinreich D, Jabali O. Staggered work shifts: a wayto downsize and restructure an emergency depart-ment workforce yet maintain current operationalperformance. Health Care Manag Sci. 2007; 10:293–308.

53. Martinez-Garcia AI, Mendez-Olague R. Processimprovement with simulation in the health sector,2005. http://www.eu-lat.org/eHealth/Martinez-and-Mendez.pdf. Accessed Jun 28, 2008.

54. Miller MJ, Ferrin DM. Simulating six sigmaimprovement ideas for a hospital emergencydepartment. Proc 2003 Winter Simul Conf.2003:1926–9.

55. Garcia ML, Centeno MA, Rivera C, DeCario N.Reducing time in an emergency room via a fast-track. Proc 1995 Winter Simul Conf. 1995:1048–53.

56. Rafalski E, Zun L. Using GIS to monitor emer-gency room use in a large urban hospital in Chi-cago. J Med Syst. 2004; 28:311–9.

57. Saunders CE, Makens PK, Leblanc LJ. Modelingemergency department operations using advancedcomputer simulation systems. Ann Emerg Med.1989; 18:134–40.

58. Samaha S, Armel WS, Starks DW. The use of sim-ulation to reduce the length of stay in an emer-gency department. Proc 2003 Winter Simul Conf.2003:1907–11.

59. Takakuwa S, Shiozaki H. Functional analysis foroperating emergency department of a general hos-pital. Proc 2004 Winter Simul Conf. 2004:2003–11.

60. Komashie A, Mousavi A. Modeling emergencydepartments using discrete event simulation tech-niques. Proc 2005 Winter Simul Conf. 2005:2681–5.

61. Duguay C, Chetouane F. Modeling and improvingemergency department systems using discreteevent simulation. Simulation. 2007; 83:311–20.

62. Poulymenopoulou M, Malamateniou F, Vassilaco-poulos G. Specifying workflow process require-ments for an emergency medical service. J MedSyst. 2003; 27:325–35.

63. Edmonds MI, O’Connor HM. The use of computersimulation as a strategic decision-making tool: acase study of an emergency department applica-tion. Healthc Manage Forum. 1999; 12:32–8.

64. Bagust A, Place M, Posnett JW. Dynamics of beduse in accommodating emergency admissions: sto-chastic simulation model. BMJ. 1999; 319:155–8.

65. Clarke JR, Spejewski B, Gertner AS, et al. Anobjective analysis of process errors in traumaresuscitations. Acad Emerg Med. 2000; 7:1303–10.

66. Kuwata S, Kushniruk A, Borycki E, Watanabe H.Using simulation methods to analyze and predictchanges in workflow and potential problems in theuse of a bar-coding medication order entry system.AMIA Annu Symp Proc. 2006:994.

67. Fairbanks RJ, Caplan SH, Bishop PA, Marks AM,Shah MN. Usability study of two common defibril-lators reveals hazards. Ann Emerg Med. 2007;50:424–32.

68. McLaughlin SA, Doezema D, Sklar DP. Humansimulation in emergency medicine training: amodel curriculum. Acad Emerg Med. 2002; 9:1310–8.

69. Wang EE, Vozenilek JA. Addressing the systems-based practice core competency: a simulation-based curriculum. Acad Emerg Med. 2005;12:1191–4.

70. Kobayashi L, Shapiro MJ, Gutman DC, Jay G.Multiple encounter simulation for high-acuitymultipatient environment training. Acad EmergMed. 2007; 14:1141–8.

71. Sullivan MJ, Guyatt GH. Simulated cardiac arrestsfor monitoring quality of in-hospital resuscitation.Lancet. 1986; 2:618–20.

72. Wayne DB, Siddall VJ, Butter J, et al. A longitudi-nal study of internal medicine residents’ retentionof advanced cardiac life support skills. Acad Med.2006; 81(10 Suppl):S9–12.


73. Wayne DB, Didwania A, Feinglass J, Fudala MJ,Barsuk JH, McGaghie WC. Simulation-based edu-cation improves quality of care during cardiacarrest team responses at an academic teachinghospital: a case-control study. Chest. 2008; 133:56–61.

74. Kobayashi L, Shapiro MJ, Haze D, et al. Prelimin-ary analysis of in-hospital cardiopulmonary resus-citation simulation (SimCode) training program[abstract]. BMJ International Forum on Qualityand Safety in Healthcare. Qual Saf Health Care2008; 17:235.

75. Dunbar J, Sheahan B, Kobayashi L. Hospital code-cart equipment quality improvement in responseto simulated resuscitation program findings[abstract]. Research Forum, American College ofEmergency Physicians Scientific Assembly 2008.

76. Siegel N, Darula K, O’Rourke E, Bird E, KobayashiL, Overly F. Towards the resuscitation bay of thefuture: identifying design opportunities throughhuman factors observation [abstract]. New Eng-land Regional Meeting, Society for AcademicEmergency Medicine, 2008.

77. Kobayashi L, Shapiro MJ, Sucov A, et al. Portableadvanced medical simulation for new emergencydepartment testing and orientation. Acad EmergMed. 2006; 13:691–5.

78. Pliego J, Villamaria F, Wehbe-Janek H, et al. Simula-tion is an effective strategy for orienting code blueteams and troubleshooting facilities and responsesto critical events in a new hospital facility. 2008:International Meeting on Simulation in Health-care [abstract #45]. Available at: http://www.ssih.org/public/ssh_content/meetings/08IMSH/OK%20Research%20sih200094.pdf. Accessed Jul 26, 2008.

79. Hunt EA, Walker AR, Shaffner DH, Miller MR,Pronovost PJ. Simulation of in-hospital pediatricmedical emergencies and cardiopulmonary arrests:highlighting the importance of the first 5 minutes.Pediatrics. 2008; 121:e34–43.

80. Hunt EA, Hohenhaus SM, Luo X, Frush KS. Simu-lation of pediatric trauma stabilization in 35 NorthCarolina emergency departments: identification oftargets for performance improvement. Pediatrics.2006; 117:641–8.

81. Rothschild JM, Keohane CA, Cook EF, et al. Acontrolled trial of smart infusion pumps toimprove medication safety in critically ill patients.Crit Care Med. 2005; 33:533–40.

82. Conners GP, Hays DP. Emergency departmentdrug orders: does drug storage location make adifference? Ann Emerg Med. 2007; 50:414–8.

83. Leiba A, Goldberg A, Hourvitz A, et al. Lessonslearned from clinical anthrax drills: evaluation ofknowledge and preparedness for a bioterroristthreat in Israeli emergency departments. AnnEmerg Med. 2006; 48:194–9.

84. Fernandes CM, Wuerz R, Clark S, Djurdjev O.How reliable is emergency department triage?Ann Emerg Med. 1999; 34:141–7.

85. Hanada E, Itoga S, Takano K, Kudou T. Investiga-tions of the quality of hospital electric power sup-

ply and the tolerance of medical electric devices tovoltage dips. J Med Syst. 2007; 31:219–23.

86. Isken MW, Littig SJ. Simulation analysis ofpneumatic tube systems. J Med Syst. 2002; 26:9–19.

87. Graff L, Stevens C, Spaite D, Foody J. Measuringand improving quality in emergency medicine.Acad Emerg Med. 2002; 9:1091–107.

88. Nemeth CP, Cook RI, Woods DD. The messydetails: insights from the study of technical workin healthcare. IEEE Transactions on Systems, Man,Cybernetics–Part A: systems and humans. 2004;34:689–92.

89. Vincent CA. Analysis of clinical incidents: a win-dow on the system not a search for root causes.Qual Saf Health Care. 2004; 13:242–3.

90. Bond WF, Lammers RL, Spillane LL, et al. The useof simulation in emergency medicine: a researchagenda. Acad Emerg Med. 2007; 14:353–63.

91. Nishisaki A, Keren R, Nadkarni V. Does simulationimprove patient safety? Self-efficacy, competence,operational performance, and patient safety. Anes-thesiol Clin. 2007; 25:225–36.

92. Cooke DL, Rohleder TR. Learning from incidents:from normal accidents to high reliability. SystDynam Rev. 2006; 22:213–39.

93. Hallock ML, Alper SJ, Karsh B. A macro-ergo-nomic work system analysis of the diagnostic test-ing process in an outpatient health care facility forprocess improvement and patient safety. Ergo-nomics. 2006; 49:544–66.

94. Higgins G, Athey B, Bassingthwaighte J, et al.Final report of the meeting ‘‘modeling & simula-tion in medicine: towards an integrated frame-work.’’ July 20–21, 2000, National Library ofMedicine, National Institutes of Health, Bethesda,MD. Comput Aided Surg. 2001; 6:32–9.

95. Shortell SM, Singer SJ. Improving patient safetyby taking systems seriously. JAMA. 2008; 299:445–7.

96. Ceglowski R, Churilov L, Wasserheil J. Don’tpanic–prepare: towards crisis-aware models ofemergency department operations. Clin InvestMed. 2005; 28:320–2.

97. Wilson JC. Implementation of computer simulationprojects in health care. J Oper Res Soc. 1981;32:825–32.

98. Runyon R, Gabler J, Handler T, et al. Hype Cyclefor Healthcare Provider Technologies. StamfordCT: Gartner Research Inc., 2005.

99. Han YY, Carcillo JA, Venkataraman ST, et al. Unex-pected increased mortality after implementation ofa commercially sold computerized physician orderentry system. Pediatrics. 2005; 116:1506–12.

100. Koppel R, Metlay JP, Cohen A, et al. Role ofcomputerized physician order entry systems infacilitating medication errors. JAMA. 2005; 293:1197–203.

101. Culler SD, Hawley JN, Naylor V, Rask KJ. Is theavailability of hospital IT applications associatedwith a hospital’s risk adjusted incidence rate forpatient safety indicators: results from 66 Georgiahospitals. J Med Syst. 2007; 31:319–27.


102. Sanderson P. Designing and evaluating healthcareICT innovation: a cognitive engineering view. StudHealth Technol Inform. 2007; 130:3–12.

103. Patterson MD, Blike GT, Nadkarni VM. In situsimulation: challenges and results. Adv Pat Saf.2008; in press. Available at: http://www.ahrq.gov/downloads/pub/advances2/vol3/Advances-Patterson_48.pdf. Accessed Sep 10, 2008.

104. Kobayashi L, Patterson MD, Overly FL, ShapiroMJ, Williams KA, Jay GD. Educational andresearch implications of portable human patientsimulation in acute care medicine. Acad EmergMed. 2008; 15:this issue.

105. Gaba DM, Raemer D. The tide is turning: organi-zational structures to embed simulation in the fab-ric of healthcare. Simul Healthc. 2007; 2:1–3.

Supporting Information

The following supporting information is available in theonline version of this paper:

Data Supplement S1. Systems thinking in medicineand emergency medicine.

Data Supplement S2. Prior and ongoing simulationapplications for systems thinking in general healthcare.

Data Supplement S3. Simulation application matrixfor emergency medicine systems expertise.

The documents are in PDF format.Please note: Wiley Periodicals Inc. are not responsi-

ble for the content or functionality of any supportinginformation supplied by the authors. Any queries (otherthan missing material) should be directed to the corre-sponding author for the article.


Documents

Advanced Medical Simulation Applications for Emergency Medicine Microsystems Evaluation and Training