Pediatrics 1999 Escobar 360 73

7/28/2019 Pediatrics 1999 Escobar 360 73

1/16

1999;103;360PediatricsGabriel J. Escobar

Organization

Evidence-Based Approach Toward Newborn Infections in a Managed CareThe Neonatal ''Sepsis Work-up'': Personal Reflections on the Development of an

http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html

located on the World Wide Web at:The online version of this article, along with updated information and services, is

of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.Boulevard, Elk Grove Village, Illinois, 60007. Copyright 1999 by the American Academypublished, and trademarked by the American Academy of Pediatrics, 141 Northwest Point

publication, it has been published continuously since 1948. PEDIATRICS is owned,PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

at Indonesia:AAP Sponsored on May 28, 2013pediatrics.aappublications.orgDownloaded from
http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.htmlhttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.htmlhttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.htmlhttp://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html


2/16

SECTION 3: CASE STUDIES

The Neonatal Sepsis Work-up: Personal Reflections on theDevelopment of an Evidence-Based Approach Toward Newborn

Infections in a Managed Care Organization

Gabriel J. Escobar, MD

ABSTRACT. Rule out sepsis may be the most com-mon discharge diagnosis among infants admitted to theneonatal intensive care unit. Although the frequency ofsepsis, meningitis, and other confirmed bacterial infec-tions has remained constant (between 1 and 5/1000 livebirths) for many years, the number of infants evaluatedand treated is much higher. Each year in the UnitedStates, as many as 600 000 infants experience at least oneevaluation for suspected bacterial infection during thebirth hospitalization. The number treated is estimated at130 000 to 400 000 per year. Despite massive overtreat-ment, delayed diagnosis still occurs.

The Kaiser Permanente Medical Care Program(KPMCP) considers developing and implementing anevidence-based approach to rule out sepsis, a re-search and operational priority. To achieve these goals,it is essential to consider two key aspects of the prob-lem. First, it is important to adopt a phenomenologicapproach that takes clinicians personal experienceinto account. This must include reflection on thoseaspects of experience often considered irrational orsubjective. Second, incorporation of a phenomeno-logic approach needs to be tempered with sound epi-demiologic methods.

If one considers these two aspectsphysician expe-rience and sound epidemiologyit is clear that muchof the existing literature on rule out sepsis is oflimited utility. Consequently, the KPMCP has con-ducted its own studies. These are aimed at character-izing the sepsis work-up, developing electronic data-sets that would permit clinicians to simulate variousstrategies, and developing techniques for ongoingelectronic monitoring.

This article summarizes the approach taken by theKPMCP Division of Research. It describes the results ofa pilot study as well as the development and use of adedicated neonatology outcomes database, the KaiserPermanente Neonatal Minimum Data Set (NMDS). TheNMDS database includes the Score for Neonatal Acute

Physiology and permits ongoing monitoring of sepsiswork-ups as well as confirmed cases of neonatal infec-

tion. The article also describes how the experience fromthe pilot as well as the NMDS was incorporated in thedesign of a much larger study on rule out sepsis.Finally, the article describes some important theoreticissues affecting decision rule development and the use ofcomputer simulations in neonatology. These issues are 1)how one handles possible overanalysis of a dataset; 2)how one handles data points that are unstable (eg, theabsolute neutrophil count, which can vary considerably

depending on age and sampling conditions); and 3) thelimitations of decision rules based on computersimulations. Pediatrics 1999;103:360373; neonatal inten-sive care, sepsis eval uati ons, sepsis, meningit is, neonatal

bacterial infecti ons, antibi oti c therapy, evi dence-based

medicine.

ABBREVIATIONS. KPMCP, Kaiser Permanente Medical CareProgram; NICU, neonatal intensive care unit; SCN, special carenursery; ICDCM, International Classification of Diseases, ClinicalModification; NMDS, Neonatal Minimum Data Set; SNAP, Scorefor Neonatal Acute Physiology; ANC, absolute neutrophil count;CBC, complete blood count; CART, classification and regressiontrees; LOS, length of stay.

Since 1991, at the Kaiser Permanente Medical CareProgram (KPMCP)s Division of Research in Oak-land, CA, I have developed a research program in

neonatology. This program now includes studies onseverity of illness scoring,15 neonatal jaundice,6 severeneonatal dehydration, the effect of maternal substanceabuse on rates of neonatal assisted ventilation, and theinformatics of neonatal outcomes measurement.7

One major area of effort has been, and continues tobe, the development of an evidence-based approachtoward the neonatal sepsis work-up. Evaluating anewborn suspected of bacterial infection is no longer

considered very interesting by science reporters,who invariably find other subjects far more fascinat-ing. Among these are new therapies such as surfac-tant, nitric oxide, high-frequency ventilation, andpartial liquid ventilation. The approach I am takingto the routine (bread and butter) sepsis work-upepitomizes my philosophic orientation toward re-search. This phenomenologic orientation values aclinicians personal experience, not just method-ologic rigor. This approach includes careful use andassessment of focused data collection efforts as wellas reflection on the policy implications of researchfindings.

From the Kaiser Permanente Medical Care Program Division of Research,

Perinatal Research Unit, Oakland, California.

This article is dedicated to the members of the Division of Research Peri-

natal Research Unit: Mary Anne Armstrong, Marla Gardner, Bruce Folck,

Joan Verdi, Veronica Gonzales, Diane Carpenter, and Blong Xiong. Their

dedication to perinatal research has made this work possible.

Received for publication Sep 8, 1998; accepted Sep 8, 1998.

Address correspondence to Dr Escobar, Kaiser Permanente Medical Care

Program, Division of Research, Perinatal Research Unit, 3505 Broadway,

Rm 718, Oakland, CA 94611.

PEDIATRICS (ISSN 0031 4005). Copyright 1999 by the American Acad-

emy of Pediatrics.

360 PEDIATRICS Vol. 103 No. 1 January 1999at Indonesia:AAP Sponsored on May 28, 2013pediatrics.aappublications.orgDownloaded from
http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/


3/16

PHILOSOPHIC ORIENTATION

To the things themselves!

Edmund Husserl, Cartesian Meditations8,9

One of the primary aims of a specifically phenomenologicmethod in philosophy is to make more thematic what is otherwisemerely implicit and taken for granted in human experience.

Moreover, phenomenology places special stress on firsthand ordirect description, thereby minimizing recourse to the highlymediated constructions of metaphysics, natural science, andother theory-saturated disciplines. What is sought in the imple-mentation of such a method is an accurate description of a given

phenomenon as it presents itself in ones own experience, not anexplanation of its genesis through antecedent causal factors. The

phenomenologists basic attitude is: no matter how somethingcame to be in the first place, what is of crucial concern is thedetailed description of the phenomenon as it now appears.

Edward S. Casey, I magining. A Phenomenologic Study10

Edmund Husserl (18591938), who founded theschool known as phenomenology, claimed to have de-fined a radically new approach toward philosophicinquiry. Setting aside the issue as to whether Husserlsucceeded, neonatologists can learn from his methodsand those of his followers. Central to the vision ofphenomenology is the notion that accurate descriptionshould precede active theorizing. The starting point ofphenomenologic inquiry is called epoche, withholding

judgment. Epoche involves trying to step back and see-ing things as they strike us, momentarily setting asidevalue judgments. For example, if one closes ones eyesand has a mental image of a house, one characterizesthis image without necessarily addressing the issue asto whether it is real. Assessing whether the houseone has visualized is real is a second step. Theory

formation follows accurate characterization.10 Moreover, as

Edward S. Casey emphasizes, a phenomenologiststudying subjects such as memory or imagination willfocus on the common manifestations of these subjectsrather than on the more spectacular or dramatic. Notsurprisingly, phenomenologists are fascinated by Mar-cel Prousts attempts at meticulous description of justhow he remembered the taste of a madeleine.11,12

Phenomenologists are less interested in the elaborateapproaches taken in, to give an ancient example, thecosmology of Platos Timaeus.10,11,13,14

Phenomenologys emphasis on description ofthe common has tremendous appeal to me as aresearcher. It is one reason I have chosen to study

what may be the single most common dischargediagnosis in the neonatal intensive care unit(NICU). Walking into a modern NICU, a neonatalphenomenologist could not fail to notice just howcommon the sepsis work-up is. Moreover, a de-tached observer would not just see many of theseinfants. He or she could not fail to notice disagree-ments over diagnoses and treatments, sometimesextremely bitter, sometimes within the same shiftand the same room. One analogy I often use whenI make presentations on this subject is the follow-ing: if a newborn has cyanosis, we do not seecaregivers in fierce arguments as to whether weshould provide helium or nitrogen. The reason for

this is that a large body of evidence as well asreadily verifiable experience strongly supports theuse of oxygen. Before I consider the evidence, Iwant to describe my own personal experience.

REFLECTIONS ON PERSONAL EXPERIENCE

Back in the days of the giants, we seemed tohave one basic approach to rule out sepsis: treat. Ithas been only relatively recently that the acceptedwisdom of our pediatric training programs has be-

gun to be questioned in a systematic manner. Onesubtle semantic manifestation of this involves threeletters. Until 1994 or so, whenever the subject cameup in our medical centers, the discussions used theterm septic work-up. Implicit in this was the pre-sumption that disease was presenta newborn hadto prove his or her microbiologic innocence. Now itis more common to use the term sepsis work-up,which at least has a more neutral connotation!

The time has come to replace the fears of our youthwith fresh reflections tempered by accumulated ex-perience. Whenever I speak about rule out sepsis,I always hear one phrase from the audience: I had a

baby once. . . One word summarizes our experi-ence: fear. Most pediatricians remember havingcared for what I now call the nightmare infant.Initially asymptomatic, this proverbial infant alsohad normal laboratory results (although no oneseems to remember exactly what thresholds wereconsidered normal, but more on this issue below).Sometime later, perhaps after discharge to parents,we all seem to recall that same infant, now purpuric,in shock, covered with oozing petechiae, its ventila-tor settings and dopamine infusion rates perma-nently etched into our memories.

There is another aspect of experience that we mustconsider: it is demoralizing to treat hundreds of in-fants with negative results while still managing tomiss some. In the back of our minds, we suspect thata different approach might be possible. This ap-proach would mandate more aggressive treatmentand longer observation periods for some infants. Atthe same time, it seems likely that another group ofinfants would not need treatment at all. In my owncase, being able to attempt this approach occurred ina group model managed care organization.

RESEARCH IN MANAGED CARE SETTINGS

More and more medical care is shifting to man-

aged care settings, and cost-containment pressureshave become intense. Outside academic institutions,justification of the research enterprise must go be-yond intrinsic intellectual interest and needs to con-sider issues such as cost, volume, and outcomes.Researchers need an active constituency, not just in-teresting findings. From this standpoint, rule outsepsis was a good place to start. Clinicians wel-comed my taking a careful look at an issue that keptthem up at 3 in the morning and that sometimes hadlifetime consequences for infants and their families.Administrators were interested in careful analyses ofa frequent and expensive process. Given the con-straints of starting a research program from scratch,

SUPPLEMENT 361at Indonesia:AAP Sponsored on May 28, 2013pediatrics.aappublications.orgDownloaded from
http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/


4/16

it would have been much harder to begin with aproject that focused on the rarer events that domi-nate the academic literature (eg, infants of very low

birth weight). The time to accumulate adequate num-bers of such infants in a database would have beenprohibitive.

The sepsis work-up was a good place to start. Itis extremely common: somewhere between 150 000and 600 000 infants are evaluated each year, and asmany as 400 000 are treated with systemic antibiotics

for a minimum of 48 to 72 hours, pending receipt ofnegative culture results.1517 In the KPMCP, 6% to 7%of newborns are treated expectantly pending receiptof negative culture results, and the bread-and-but-ter evaluation and treatment combination accountsfor 25% of all our NICU days.18

EPIDEMIOLOGIC REFLECTIONS

Epidemiologists routinely take their objects ofstudy and slice them up. In many respects, strati-fication is often a very good place to begin. This isa step that, until very recently, has not been man-aged properly in neonatal research. Problems have

occurred because of errors in two directions. Onthe one hand, some studies lump all infants to-gether, pooling the 800 grammer with the fourkilo mec baby. A different error is that of exces-sive reliance on case series. Ultimately, these prob-lems revolve on the failure to pay attention to threequestions considered essential by epidemiologists:What is the denominator? What is the numerator?Is there some sort of a control or comparisongroup?

Before tackling these questions from a method-ologic point of view, I want to defend our professionfrom a phenomenologic and historical standpoint. I

believe that the reason that the pediatricians andneonatologists who conducted the first systematicstudies on rule out sepsis did not consider theseissues is rooted in the history of our specialty. Ourspecialty began in an atmosphere of crisis and ur-gency. It was so much more important to act than tomeasure. The reality of our experience is that in theabsence of data from studies conducted with epide-miologic rigor, we cannot perform any sort of strat-ification except that which is analytically the mostuselessmentally separating cohorts of infants intoindividual anecdotes.

WHAT IS THE DENOMINATOR?

The use of denominators to convert counts into proportionsseems almost too simple to mention. However, a proportion isone basic way to describe a group. One of the central concerns ofepidemiology is to find and enumerate appropriate denominatorsto describe and to compare groups in a meaningful and usefulway.

Gary D. Friedman, Primer of Epidemiology19

The major problem with the rule out sepsis lit-erature is its failure to define, quantify, and use twomethodologically correct denominators (all live birthsand all infants ever evaluated) instead of denominatorsof convenience (eg, infants with positive cultures orinfants in a given weight range with a specific type of

infection). For example, commonly cited mono-graphs2022 quote a series of articles that report pre-senting signs in infants with culture-proven sepsisand/or meningitis eg, 14% of such infants pre-sented with lethargy, 29% had trouble feeding, 30%had hepatosplenomegaly, etc. Absent from thesepublications and neonatology texts, however, aredata on 1) just how common these presenting signsare; and 2) how many infants with a given clinicalsign ultimately experienced adverse outcomes. Be-

cause the number of infants ever evaluated and theproportion of these infants with an individual clini-cal sign are seldom reported, it is not possible todetermine the predictive value of a given clinicalsign. Nonetheless, neonatal texts use the presence ofthese signs as a rationale for evaluation and treat-ment.23

Given that the frequency of some presenting signsmay be very high, it is important to note that cohortstudies and randomized trials are not the only op-tion. Casecontrol studies, the utility of which has

been well-established in the epidemiology litera-ture,19 also are an option. For such methods to beuseful, however, we must first consider just whatconstitutes a case, which brings us to the secondissue: the numerator.

WHAT IS THE NUMERATOR?

There is nothing intrinsically wrong with report-ing on a case series of newborns with positive bloodculture results, any more than there is on reporting acase series of infants with pneumonia. What is wrongis generalizing from such case series to all infants, ormaking blanket recommendations for evaluations

based on such case series. Several sound epidemio-logic principles justify this critique.

Most diseases do not manifest as dichotomous

entities. It is far more common that a given clinicalpresentation can be best described as a point on acontinuum. Because the phenomenon of spontane-ous resolution of disease is also common, and needsto be discussed in the context of control and compar-ison groups, merely reporting on the characteristicsof a group of infants with positive cultures is ofextremely limited utility.

Inferences based on case series highlight how oneset of events (eg, an infant developing septic shock)can lead us to ignore other events that may be just ascommon (eg, infants resolving transient bacteremia).Unfortunately, for several years, we did not take the

epidemiologic approach (which recognizes both theforme fruste as well as the spontaneous cure). Instead,our fear of missing an infant led us to considersuch infants from a purely personal perspective (wegot lucky). The problem is compounded further

because there are infants who look great at first butgo on to crash.

The numerator problem can only be resolved bystudies that report more than just which infants hada positive culture. Ideally, studies should 1) clearlydefine what is considered an infection; 2) includecriteria for defining an infection in the absence ofpositive cultures; 3) report the numbers of infantswho may have had spontaneous resolution of their

362 SUPPLEMENTat Indonesia:AAP Sponsored on May 28, 2013pediatrics.aappublications.orgDownloaded from


5/16

disease; and 4) report the numbers of infants with asevere clinical presentation that was not attributableto infection.

IS THERE SOME SORT OF CONTROL GROUP ORCOMPARISON GROUP?

In developed nations, it is unusual for either sepsisor the suspicion of sepsis to be ignored. Cliniciansintervene, perform evaluations, and provide treat-ment. When assessing screening and treatment strat-

egies, epidemiologists always consider the effects ofno screening and no treatment. This importantconsideration is virtually absent from the rule outsepsis literature, which primarily mentions the riskof death when sepsis is not treated. However, anyattempt at developing rational strategies must in-clude some thought about what occurs to infantswith a given risk factor, clinical sign, or laboratorytest result when treatment is not provided. This iscritical if we are to assess the sensitivity, specificity,and predictive value of risk factors, clinical signs, orlaboratory results.

The problem is complicated because, given thecurrent state of knowledge, agreement as to whoshould and should not be treated does not reallyexist. Although some clinicians have called for arandomized clinical trial,24 I believe that some meth-odologic reflection on what would constitute aproper control or comparison group is indicated be-fore we can even begin to design such trials.

If we use the term control as meaning a group ofinfants who do not receive a specific treatment, thedefinition is problematic in either prospective or ret-rospective study designs. In the context of prospec-tive studies, withholding treatment would be uneth-ical. In the context of retrospective studies, infantsmay have received antibiotics at a fairly late stage. In

other words, if one uses the term control, it isnecessary to also specify a time component. In thecontext of a hypothetic clinical trial, for example, onecould have an infant with risk factor X randomlyassigned to either having immediate antibiotic ther-apy (treatment arm) or observation for Y hours orless (ie, infants in the control arm would be allowedto cross-over to the treatment arm for ethical rea-sons).

Because such a study would be very difficult todesign, let alone conduct, a more realistic strategy isto conduct retrospective studies that use a compari-son group. The term comparison group stresses that

the decision to treat or not is not being made byrandom assignment. The difficulty with using com-parison groups is in teasing out the contribution ofspecific predictors. For example, one could hypoth-esize that an absolute neutrophil count (ANC) of10 000 at 4 to 6 hours of age is a marker for sepsis.Testing this hypothesis is very difficult when analyz-ing retrospective data from a heterogeneous group oftreated and untreated infants with varying frequen-cies of other signs, risk factors, and test results. For-tunately, epidemiologists have developed techniquesfor doing this.25 It is worthwhile to remember that inthis situation, the epidemiologic viewpoint coincideswith a clinicians experience and a phenomenolo-

gists fascination with the common: all us can recalldischarging infants without treatment and wonder-ing just what would happen this time.

FIRST ATTEMPTS AT QUANTIFICATION

Our first attempt at developing an evidence ap-proach to rule out sepsis in KPMCP was a studywith limited aims.18 This project served as the pilotfor the larger study described below. We did notattempt to come up with recommendations as to

whether one should start treatment with antibiotics.We focused on 1) whether one could define a low-risk group using readily available predictors, and 2)whether definition of such a group would permit areasonable inference that could guide a clinician indeciding when to stop antibiotic treatment after 24hours in selected infants.

Figure 1 shows results of our first study. It showsinformation that is highly relevant to the develop-ment and implementation of an evidence-based ap-proach to rule out sepsis. Data shown are from 10KPMCP hospitals in Northern California during themonths of September and October 1990. On a popu-lation basis, a number of event types occur withsufficient frequency that they could be captured witha cohort of 5709 births. For example, one can inferthat in the KPMCP, with 2.7 million members, anewborn crashes approximately every 12 to 18hours. During this 2-month time frame, 13% of alllive births spent at least some time in a special carenursery (SCN, meaning any location reserved fornonnormal newborns), and 260 infants 2500 g re-ceived parenteral antibiotics. Of these 260 infants, 41were critically ill within 24 hours of entry into anSCN, and 5 were treated for presumed or provenmaternal syphilis. This leaves 214 infants (3.7% oflive births) who were of relatively low risk.

Figure 2 shows the outcomes of the 214 infantswho were treated presumptively and who were notcritically ill in the first 24 hours in special care. Itdemonstrates that these infants do not constitute ano-risk group: 3 had severe respiratory deterioration.Importantly, 2 of these infants had negative cultureresults.

Additional insights can be gained by examiningthe final discharge diagnoses among the 749 infantswho entered special care (Table 1). There is noquestion that rule out sepsis is common; almosthalf of the SCN admissions had this diagnosis. Threeother diagnoses that are not necessarily distinguish-

able from early sepsis or meningitis also are frequent;transient tachypnea of the newborn2629 was presentin 21% of all admissions , ill-defined feeding difficul-ties in 6.8%, and probable sepsis in 6.3%. Duringthis period, only 28 of the 749 SCN admissions hadculture-proven disease (a rate of5 per 1000 live

births).The frequencies of these diagnoses demonstrate

that although true sepsis is rare, the number of in-fants with conditions that could presage sepsis is infact quite large. Put differently, there is a rational

basis for our fear. Therefore, if one is to conductstudies to define better approaches to this problem, itis critical to consider those diagnoses in which sepsis



6/16

might have occurred, not just those situations whereit actually occurred. Here the epidemiologic view-point coincides with the experiential and phenom-enologic viewpoint. The epidemiologist is willing toreassess definitions of case or control status, or ofoutcomes. A phenomenologic and experiential view-point accepts the importance of considering thosesituations that overlap with the situation of interest,or where it is perceived to be present.

DEVELOPING THE RIGHT TOOLS

At the same time that our first study began, mycolleagues and I also began to address our generalinformation needs for neonatology, not just thoserelated to rule out sepsis. Although KPMCP datasystems are rich, clinical information systems avail-able to us in 1991 to 1992 relied heavily on theInternational Classification of Diseases, ClinicalModification (ICDCM).30 At that time, the ICDCMsystem had a number of important limitations, someof which have since been corrected. For example,there was no code to identify an infant merely eval-uated for sepsis, or one in whom no cause for symp-

toms was ever found (the V29 code is now used).There also was no code for an umbilical artery cath-eter, which virtually defines a newborn in an inten-sive care setting. Moreover, data quality with respectto the use of ICDCM codes was heterogeneous.

We had to address three other specific problems.One was the obsolescence of mainframe-based infor-mation systems, which could not exploit the manyadvances and advantages of distributed networks.

Second, the organizational response to the problemof legacy systems was complicated by the need togrant considerable local autonomy to individualmedical centers. For example, some places usedMacintosh computers, whereas others did not. Fi-nally, the information sources available to us werenot structured for correct aggregation (using live

births as the denominator).We made the strategic decision to build a neonatal

databasecalled the Neonatal Minimum Data Set(NMDS)from scratch, a process we have describedelsewhere.7 Shortly after we began to collect data, DrDouglas Richardson of Harvard University providedus with a then experimental protocol for a neonatal

Fig 1. Distribution of live births and SCN admis-sions in 10 Kaiser Permanente hospitals during Sep-tember and October 1990. SCN was considered anysetting reserved for nonnormal newborns. The studyfocus was on the 214 infants who were of normal

birth weight, were not critically ill during their first24 hours in an SCN, and who received systemicantibiotics during the birth hospitalization. Theseinfants made up the majority (214/260, or 82%) ofinfants weighing 2500 g who received parenteralantibiotics. The other infants in this weight rangewho were treated with antibiotics included 41 who

deteriorated very quickly and 5 who were treatedbecause of presumed or proven maternal syphilis.



7/16

severity of illness scale, the Score for Neonatal AcutePhysiology (SNAP).1 4 The SNAP assigns points

based on the degree of physiologic derangement a

newborn experiences in the first 12 or 24 hours inintensive care. It is clinically intuitivethe higher theSNAP, the sicker the infant.

Use of the SNAP was important for two major rea-sons: one scientific, one phenomenologic. The scientificreason is fairly clear: traditional predictors used for riskadjustment in neonatology (birth weight, gestationalage, sex) only explain a fraction of the variation ob-served between centers. The phenomenologic reasonwas that without some way of addressing illness sever-ity, we could not address an experience-based state-ment made by physicians, my infants are sicker.Using SNAP permitted us to assess a number of as-pects related to practice variation.5,7

The NMDS now functions as a wide area networklinking six level III units and two level II units inCalifornia and one level III unit in Colorado. It has anevolving reporting structure. Figure 3 shows a lengthof stay comparison grid based on mortality risk(SNAP, Perinatal Extension). Figure 4 shows an in-dividual NICUs basic utilization report. As pre-sented in Fig 4, the NMDS database has the capabil-ity to track an individual units rule out sepsis

TABLE 1. Most Frequent Neonatal Discharge Diagnoses*

Diagnosis No. (%)

Rule out sepsis 365 (48.7%)Jaundice 191 (25.5%)Transient tachypnea of the newborn 157 (21.0%)Hypoglycemia (all forms) 91 (12.1%)

In infants of diabetic mother 31 (4.1%)All other forms 60 (8.0%)

Meconium aspiration 70 (9.3%)Apnea bradycardia 68 (9.0%)Respiratory distress syndrome 67 (8.9%)Ill-defined feeding difficulties 51 (6.8%)Infant of diabetic mother 48 (6.4%)Probable sepsis 47 (6.3%)Infant whose mother used a drug of abuse 29 (3.9%)Confirmed infections (cultures positive) 28 (3.7%)Air leaks 21 (2.8%)Pneumonia 20 (2.6%)

* All special care nursery admissions to the 10 Kaiser Perma-nente birth hospitals (9/1/9010/31/90) from a cohort of 5709live births.

Fig 2. Outcomes of the 214 infantsconstituting the principal study group.Note that only 11 (or 5% of the group)infants had a positive culture result.There were four infants whose cultureresults were considered questionable

by treating physicians. Also note thatof the 3 infants who experienced dete-rioration after 24 hours in a special caresetting, 2 had negative culture results.



8/16

utilization patterns. This capability is critical if oneaims to assess the implementation of an evidence-

based approach.

DESIGNING THE NEXT STUDY

The next step in our work was conducting a

study that used our accumulated experience. Thisstudy was funded by The Permanente MedicalGroup, Inc, Kaiser Foundation Health Plan, Inc,the Sidney Garfield Memorial Fund, the PackardFoundations Center for the Future of Children,and the Maternal and Child Health Bureaus Re-search Program. It is called Watchful Waitingversus Antibiotics A.S.A.P. Its results are de-scribed elsewhere.31 In this article, I focus on thetheoretic issues that defined how we structureddata collection and analysis.

Our goal was to create a large electronic datasetthat could be used to define decision rules (evidence-

based treatment guidelines). One portion of the data-set (derivation dataset) would be used to develop therules, whereas another (validation dataset) would beused to test them.

We made a conscious decision to take this ap-proach because we did not feel that given the state ofknowledge available in 19941995, it would be ethi-cal or feasible to conduct a randomized trial. Theresults of a decision rule approach, however, areamenable to prospective tests.

DEFINING THE TARGET POPULATION

Because of limited resources, we were not able toconduct a study using prospective data collection.

Instead, we opted for a compromise: prospectiveidentification and retrospective chart review. Wewere aided by two factors. First, by 1995, the KPMCPhad developed robust database systems that permit-ted us to establish ongoing downloading of key lab-oratory data (eg, complete blood counts (CBCs), ar-

terial blood gases, and culture results). Second,because of the NMDS, we had a cadre of trainedresearch assistants. These two factors permitted us topiggyback the study onto an existing infrastruc-ture that includes a help desk for chart reviewerswith questions about our research protocols.

We decided to use the proper denominator (allinfants ever evaluated), which posed a problem:How does one define evaluation for sepsis? Afterdiscussions with many neonatologists, we definedthe inclusion criteria as follows: an infant was con-sidered to have been evaluated for sepsis if a physi-cian suspected the condition and obtained a CBC

and/or a blood culture. This definition was indepen-dent of either treatment or outcome. Strictly speak-ing, this definition is incorrect because a physicianwho evaluates a newborn may consider the risk to beso low as not to warrant any sort of screening. Usingthe best definition, however, would have been fa-tal for the study because of not being able to identifyeligible subjects electronically or by chart review.

Four additional decisions merit special mention.First, we decided to track infants after neonatal dis-charge. Lack of any follow-up data has been a nota-

ble weakness of the rule out sepsis literature. Sec-ond, we decided to incorporate outcomes other than

just positive culture results (eg, a category defined as

Fig 3. LOS comparisons using analysis of vari-ance. Analyses are based on groups stratified bymortality risk using the SNAPPE.15. Individualnurseries can determine whether their averageLOS is significantly different from another unitwithin a defined birth weight and mortality riskcategory.



9/16

probable infection for infants with obvious septicshock, but with negative culture results). Third, toavoid circular reasoning, we made an explicit deci-sion not to use results of CBC or arterial blood gas

studies to define patient outcomes. Finally, we madea heavy investment in collecting data based on twopoints in time: time of birth and time of onset.

The use of two points in time merits additional

Fig 4. Individual nursery utilization report generated by the NMDS database. Notice use of proper denominators (live births) and useof medians, not just means, for LOS, and length of assisted ventilation distributions. Also note characterization of triage (quantificationof infants who were in intensive care for 24 hours).



10/16

discussion. Time of birth is important with respect torule out sepsis for several reasons. These includethe fact that some commonly used predictors (suchas the ANC) vary as a function of a newborns chro-nologic age. On the other hand, it also is useful toconceptualize time intervals based on when an infantwas first labeled as being at risk for sepsis (ie, thatmoment when the infant was no longer unequivo-cally considered a well baby). One of the results ofour first study was that we found that this time,

which we have referred to as entry time or onsettime, is charted carefully by nurses or physicians. Inthose situations where it is not charted, it can beinferred by examining the collection date and time ofeither the first CBC or blood culture.

Figure 5 shows a portion of the data collectionform we used to abstract neonatal charts. The use ofhighest and lowest values for vital signs showsthe influence of the SNAP and other severity scales.It also highlights our experience with our neonataldatabase, which uses dichotomous outcomes when-ever possible.7

RECONSIDERING SOME KEY PREDICTORS

No discussion of the sepsis work-up would becomplete without consideration of the interpretationof a test whose performance has acquired a ritualquality: the CBC. Virtually all published guidelinessuggest obtaining this test. Unfortunately, the agree-ment ends there. There are myriad recommendationsas to what is considered predictive: the ANC, imma-ture to total ratio (I:T ratio), not to mention a host ofothers including the band count, total white bloodcell count, platelet count, and various other combi-nations.3239

Although some studies have taken the trouble toactually compute sensitivity, specificity, and positive

predictive value, most of the studies that have at-tempted to justify their recommendations on an em-piric basis pay little attention to two important ques-tions: What constitutes a normal result? What is theeffect of sampling variation on the assumption that agiven result is a hard finding?

The most commonly cited study on the neonatalCBC is that of Manroe et al,40 whose graceful curvesand scatterplots can be found in neonatology text-

books, review articles, and treatment protocols. No

one takes the trouble to remind the reader that in theperiod from 0 to 24 hours of agethe critical deci-sion time for most neonatal sepsis work-ups, espe-cially in the era of early newborn dischargeManroeet al based their graphs on a mere 108 infants. Norare most readers aware that the 90th and 10thpercentile envelopes were defined using visual in-spection. One can forgive Manroe et al for theirmethodology, which antedated the personal com-puter; however, it is less comforting to think that somany of us have accepted these norms withoutquestion.

Setting aside the issue of blood sampling from astatistical perspective, it is important to rememberthat the actual physical sampling can lead to dramaticchanges in CBC results. First of all, it has been well-documented that the CBC does vary depending onthe infants age,4042 on whether the sample is arterialor venous,43 and on whether the infant is cryingvigorously.43 This means that a given test resultsuch as the ANCcan be visualized mentally not asa static value but as an extremely time-dependentone.

Laboratory test results also are experienced by phy-sicians in at least three additional ways. The first way,which unfortunately cannot be drawn in a diagram, isthrough the twin filters of fear and fatigue: a total white

Fig 5. Portion of data abstraction form used in oursecond rule out sepsis study. Items 73 and 74 could

be calculated by computer but have been retained as amemory jog for research assistants. Items 75 to 80

are based on our experience with severity scores,which simplify data collection by using highest andlowest values within a time frame. Items 82 to 86were based on experience with the NMDS, the firstrule out sepsis study, and recommendations of a panelof seven neonatologists.



11/16

blood cell count of 14 500 is experienced differently at11 am than at 4 am. Similarly, we will experience agiven test result in one way when an infant is crump-ing in front of us than when we get a call from laborand delivery informing us that the same result wasobtained from an infant who is currently breastfeedingquite well. Finally, we tend to overremember CBCresults that somehow burned usthese CBCs oftenare mentioned to me when I give talks, usually as partof the phrase, I had a baby once . . . Unfortunately,

very few people take the trouble to document theseunique, nightmare infants, and my systematic searchfor case reports is yielding very few studies such as theone by Christensen and colleagues.44 The philosopherHusserl describes these aspects of our experience usingterms such as series of now-points or series of nowswhich possibly will be filled with other Objects.45

Ideally, norms for any test should be establishedusing populations meeting rigorous criteria for beingconsidered normal. In my opinion, the two beststudies on the CBC are those of Schelonka and co-workers, 41,42 who used a tightly defined populationof 193 squeaky clean well infants (term infantswith no risk factors of any kind). They found that 1)that at 4 hours of age, the normal mean ANC is15 600; 2) that the lowest 10th percentile is 9500; and3) that if one applies commonly accepted criteria(such as those of Manroe et al40 and Rodwell et al38)to healthy term infants, one would label huge num-

bers of them as being at extremely high risk forsepsis! In addition, Schelonka et al remind us that thesecond most popular index, the I:T ratio, is so subjectto interobserver variation (no one seems to agreewhat an immature form isone mans band is an-other mans blast) that it is virtually useless foremergency decisions.

Existing studies also do not report the effect of

maternal treatment with antibiotics on the neonatalCBC. Nor do they compare the sensitivity and spec-ificity of the CBC against the other problematic pre-dictor: asymptomatic status.

Some published reports do support the notionthat asymptomatic status correlates strongly with afavorable outcome.24,46 However, much less atten-tion has been devoted to the details that haunt usearly in the morning when the telephone rings: 1)

Just what is asymptomatic?; 2) Assuming thatwe agree on a definition, who applies it? Should it

be a nurse, nurse practitioner, pediatrician, or neo-natologist?; And of great importance to a re-

searcher3) who reports it, who records it, andwhere is this recorded? Lost in the debates as towhether such infants are at X% or Y% risk is thefact that we do not in fact have many data as towhat is normal in a newborn. For example, text-

books typically label a respiratory rate of 60breaths per minute as a sign of illness. However,when I went back to the original studies statingthat a respiratory rate of 60 was in fact dangerous,I discovered that these studies had extremely smallsample sizes (which in fact can only be approxi-mated because they are not clearly reported), were

based on premature infants in the late 1950s, anddid not actually demonstrate any associations be-

tween (supposed) predictors and outcomes.4752 Al-though no comparable study has been performedon newborns, a study on 1007 infants showed norelationship between respiratory rate and illnessseverity or outcome.53 Significantly, in our collab-orative effort to define SNAP-II with our col-leagues at Harvard and Vancouver, the respiratoryrate variable was not predictive of neonatal mor-tality.54,55

Similar problems exist with another controversial

aspect of the neonatal sepsis work-up: when toperform a lumbar puncture. Here the literatureshows an interesting relationship: studies that care-fully stratify patients and clearly delineate predictor-outcome relationships17,5658 tend to be more cautiousin their recommendations. In contrast, studies thatdo not permit actual quantification of risk59,60 tend torecommend more lumbar punctures.

Ignoring clinicians experience probably is also re-sponsible for the poor compliance with various rec-ommendations regarding screening for and manage-ment of group B streptococcus carriage. Althoughthis literature is now quite voluminous and currentrecommendations have some basis on data from ma-ternal randomized trials,61,62 they neglect one keycomponent of clinicians experience. This is that phy-sicians who decide to rule out sepsis are not think-ing about one organism (Streptococcus agalactiae) butabout several overlapping conditions and organismsmeriting antibiotic treatment (eg, Escherichia coli andListeria monocytogenes).

DECISION RULES AND COMPUTERSIMULATIONS. WHAT THE CLINICIAN WANTS

VERSUS WHAT THE METHODOLOGIST WILLATTEST TO

My colleagues and I have arrived at several con-clusions as to what it would take to define andimplement an evidence-based approach. This iswhere the desires of many clinicians inevitably con-flict with the requirements of the methodologists.When I first began discussing our second study, oneof my friends pointed to the 2 3-inch code cardclipped onto his scrubs and said, You mean youregonna collapse rule out sepsis into one of these? Thatwould be great! His comment synthesized whatmany physicians do in fact expect from my researchunit: one or two simple algorithms with less thanthree branches

The concept of a decision rule is deceptively sim-ple. Consider a hypothetic cohort of five newborns,

If baby meets criterion A and mother meets crite-rion B, then treat.

s

Otherwise, if C holds, watch for D hours and thenreassess.

s

Otherwise, send the baby home.



12/16

each with some predictors, and two with the out-come of interest (a positive culture).

In this cohort, two parsimonious decision rules arepossible: treat infants with predictor A or treat in-fants with predictor B. The first rule identifies cor-rectly all positive culture results (cases 1 and 3) andovertreats two infants (cases 2 and 5). The secondalso identifies all positive culture results but onlyovertreats one infant (case 4).

In practice, things get more complex. One seldomdiscussed problem is that during the course of pre-paring a dataset for analysis, it is common that theinvestigators who handle the data begin to subcon-sciously absorb information patterns. It is then easyfor them to come up with decision rules that workbecause of their knowledge of the dataset, rather than

because of some underlying biologic mechanism. Re-lated to this is the problem of multiple statisticalcomparisons, which are hard to avoid during theanalysis phase.

These problems are not trivial, and a large body ofliterature has emerged that addresses these is-sues.6365 One key step is that one commit, a priori, toreporting the results of the first application of adecision rule on separate test or validation datasets(even if the decision rule fails!).

We are using three mechanisms to handle theseproblems. The first is the use of an expert panel.These are clinicians who are blinded to study data

until the last possible moment. They select whichvariables to include in candidate decision rules. Cou-pled with the use of expert panels is the use of splitvalidation and separate validation datasets. Finally,we also are using a completely impersonal approachfor defining 1) which predictors will be considered incandidate decision rules, and 2) what the thresholdvalues for such predictors will be (eg, highest ma-ternal temperature in the 12 hours before deliverywas never 101.4F). This approach is recursivepartitioning, also known as classification and regres-sion trees (CART).66,67 CART software permits rapidgeneration of outcomes trees. Each branch of the

tree is based on a predictor, which can be a dichot-omous, categoric, or continuous variable.One advantage of CART software is that it permits

one to get a different view of the utility of a givenpredictor. For example, in our first study, the expertpanel pointed to the ANC as an important predictor,and the final decision rule used a cutoff of 10 000.18

However, examination of classification and regres-sion trees using these data showed something else:high ANCs (15 000) in the first 12 to 24 hours of lifewere associated with favorable outcomes.

One also must conduct sensitivity analyses. Theseare essential in any computer simulation strategy

because some data points, for example, the ANC, are

unstable. Consider a hypothetic decision rule forrule out sepsis that incorporates the ANC. Becauseit is known that the ANC may vary with crying orwhere the blood was obtained, it is important to testmodels where one 1) systematically elevates or re-duces ANC values, 2) assumes that the data point ismissing, and 3) randomly inserts wrong values. Last

but not least, one needs to incorporate real worldconstraints (for example, decision rules should notuse data that are not ordinarily available or assume

unlikely contingencies such as an infant being dis-charged immediately after the antibiotics are discon-tinued). Performing these additional simulations in-variably results in projections that are far lessoptimistic!

Ultimately, however, this approachshowingwhere decision rules fail, not just where they suc-ceedis more fruitful. It brings out the real value ofdecision rules and computer simulations. Rules andsimulations by themselves will not produce perfectsolutions. Their true value is that they can be used astools to make clinicians test their assumptions explic-itly. In this context, one needs to remember that adecision rule that does not miss any infants is notnecessarily perfect,68 whereas a decision rule thatmisses one infant in fact may be clinically useful.18

FROM DATA TO GUIDELINE

The KPMCPs approach to clinical guidelines alsohas evolved. One thing our organization has learnedis that the unaided diffusion of new knowledge isvery slow and very uneven. In an integrated system,it is important to have mechanisms not only fordissemination, but for standardization. Guidelinesthat are pushed by an individual champion withoutorganizational buy in are unlikely to be imple-mented.

Accordingly, there is now a formal mechanism forthe development, dissemination, and implementa-tion of guidelines. This mechanism includes a formalapproval process at the beginning, middle, and endof a guidelines development process. At the begin-ning, a department chiefs group (in this case, the 12nursery directors for Northern California) formallyproposes a guideline effort to our Department ofQuality and Utilization. Once approved, this resultsin a small grant that pays for physician time to workon the guideline, an important consideration becauseall our physicians are salaried and cannot simplyleave their offices without affecting many others

schedules. Funding also is provided for nonphysi-cians and for staff support (eg, for photocopying).During the development phase of the guideline,drafts are circulated widely so as to ensure that asmany groups involved are aware that change is inthe air and can comment on the drafts. Finally, oncecomplete, it must be approved by a special groupconsisting of all the directors of our medical centersthat reviews all proposed guidelines on a quarterly

basis. Final approval means that it is possible to placethe new guideline on the KPMCP intranet. In somecases, funding for implementation can be obtained asa separate grant.

Shortly after we began to examine data stratified

Predictors Culture results

Case 1 A B _ ()Case 2 A _ C ()Case 3 A B _ ()Case 4 _ B _ ()Case 5 A _ C ()



13/16

by SNAP, we found that much of the practice vari-ation among our NICUs was driven by rule outsepsis. Equally important, however, was that wecould not study rule out sepsis by itself. We alsohad to consider those clinical conditions that over-lapped: respiratory distress syndrome, plain respi-ratory distress, transient tachypnea of the newborn,pulmonary hypertension, and pneumonia. Duringthis period, we began sharing the results of the firststudy with clinicians, and we also began to provide

feedback to nurseries, showing them their rates ofsepsis work-ups as well as other adverse outcomes.We also began to examine rates of severe pulmo-

nary hypertension (that associated with severe ten-sion pneumothorax, use of extracorporeal membraneoxygenation, and/or death) and soon found that noconsensus existed among our neonatologists as to thedefinition, diagnosis, triage, and management of re-spiratory distress in term infants. This in turn led usto develop a guideline called Evaluation and Manage-ment of Persistent Pulmonary Hypertension of the New-born.69 The guideline team included nurses and re-spiratory care technicians. Part of our guidelineprocess consisted of using data from our NICU da-tabase to test quick and dirty clinician hypotheses(eg, what is the association of pulmonary hyperten-sion with cesarean section?). We will be repeatingthis process with the results of the Watchful Wait-ing versus Antibiotics A.S.A.P. project. We alsoare fortunate in being able to use data from anotherstudy on group B streptococcus that has been con-ducted by our KPMCP colleagues in Southern Cali-fornia, who also will participate in the panel.

CONCLUSION

I do not believe that we will banish the tradi-

tional sepsis work-up from the nursery, but I dobelieve we will succeed in implementing new ap-proaches to this problem. I believe in this not just

because of the compelling logic of properly per-formed analyses, but also because physicians andnurses are tired of the current approaches.

If we do succeed, it will be because of the follow-ing factors. First and foremost is the need for soundscience. Neonatologists are sophisticated informa-tion consumers. They will not endorse new guide-lines based on manifestly erroneous sampling strat-egies. However, it is insufficient merely to haveproperly performed studies. The teams who will pre-

pare such guidelines must be provided with the re-sults of intermediate steps and variable scenarios (eg,computer simulations with random insertions ofbad data).

Second, and particularly in integrated managedcare networks, a political consensus must exist. Thisconsensus includes a willingness to work with clini-cians from other medical centers whose protocolsmay be different from our own. It also will have toincorporate changing how we interact with nurses,who will no longer merely transmit CBC results to usat four in the morning. Finally, the consensus mustinclude those individuals who are not clinicians at all

but who play an increasing role in care delivery:

administrators, computer networking specialists,and even financial analysts.

Most importantly, implementing evidence-basedguidelines cannot ignore the role of what we haveconveniently relegated to the category of items weconsider irrational. No guideline effort on ruleout sepsis can succeed if it does not consider thatfeeling of fear in the pits of our stomachs, the tears inparents eyes when we inform them that we areplacing the baby on antibiotics, that vague sense of

unease we feel when we sign our names after thephrase, Discharge to mother.What are some possible consequences of such

shifts in practice patterns? Because the physical ex-amination of a newborn is critical to proper triage, itis reasonable to expect that nurseries that have anincreased capability to offer a second opinion onan infants assessment will be on average more effi-cient and have better outcomes. Similarly, nurserieswith better information distribution systems also arelikely to be safer as well as more efficient. Finally, aswe treat fewer term infants, this will have a signifi-cant impact on the census of many units. How thispotentially de-stabilizing impact is managed by ourprofession remains to be seen.7072

One result that is not so obvious is that the re-search community will learn from this process itself.We may discover that in the same way that using theSNAP permits delineation of practice variation, well-designed studies can help us tease out patterns inhow clinicians change their practice. This includesconfronting apparently simple issues: Should umbil-ical artery catheters be placed high or low? Atwhat level of illness severity does a sick newbornmerit two umbilical catheters rather than one? Howdoes one compare the safety and effectiveness offeeding regimes in nurseries with identical rates of

necrotizing enterocolitis? Under what conditionscould one justify prophylactic assisted ventilationin term infants suspected of having pulmonary hy-pertension? It also must include thornier issues suchas human error and why physicians may reject evi-dence-based guidelines. Perhaps we may discoverthe rational basis for some apparently irrational be-haviors.

And then we may return to where we shouldalways begin: to reflecting on our own experience,not with the ossified hastiness of our youth, but withthe intellectual freshness that comes from involvingother disciplines.

ACKNOWLEDGMENTS

This work has been funded by the Permanente Medical Group,Inc, Kaiser Foundation Health Plan, Inc, the David and LucilePackards Center for the Future of Children, the Sidney GarfieldMemorial Fund, and the Maternal and Child Health BureausResearch Program.

I thank Dr Joseph V. Selby, Dr De-Kun Li, Dr Jeffrey B Gould,Dr Jeffrey D. Horbar, Ms. Mary Anne Armstrong, and Ms. MarlaN. Gardner for reviewing the manuscript. Graphics were prepared

by Ms. Gardner and Ms. Verdi.

REFERENCES

1. Richardson DK, Gray JE, McCormick MC, Workman K, Goldmann DA.

Score for neonatal acute physiology: a physiologic severity index for

neonatal intensive care. Pediatrics 1993;91:617623



14/16

2. Richardson DK, Phibbs CS, Gray JE, McCormick MC, Workman-

Daniels K, Goldmann DA. Birth weight and illness severity: indepen-

dent predictors of neonatal mortality. Pediatrics 1993;91:969975

3. Richardson DK, Tarnow-Mordi WO. Measuring illness severity in new-

born intensive care. J Intensive Care Med. 1994;9:2033

4. Richardson DK, Tarnow-Mordi WO, Escobar GJ. Neonatal risk scoring

systems: can they predict mortality and morbidity? Clin Perinatol. 1998;

25:591611

5. Escobar GJ, Fischer A, Li DK, Kremers R, Armstrong MA. Score for

neonatal acute physiology: validation in three Kaiser Permanente in-

tensive care nurseries. Pediatrics. 1995;96:918 922

6. Newman TB, Escobar GJ, Gonzales VM, Armstrong MA, Gardner MN,

Folck BF. Neonatal bilirubin testing and bilirubin levels in a large health

maintenance organization. JAMA. Submitted

7. Escobar GJ, Fischer A, Kremers R, Usatin M, Macedo AM, Gardner MN.

Rapid retrieval of neonatal outcomes: the Kaiser Permanente Neonatal

Minimum Data Set. Qual Management Health Care. 1997;5:1933

8. Husserl E. Dorion Cairns, trans. Cartesian Meditations. An Introduction to

Phenomenology. Boston, MA: Kluwer Academic Publishers; 1993;1213

9. Ihde D. Experimental Phenomenology. An Introduction. Albany, NY: SUNY

Press; 1986:29

10. Casey ES. Imagining. A Phenomenological Study. Bloomington, IN: Indi-

ana University Press; 1976:89

11. Casey ES. Remembering. A Phenomenological Study. Bloomington, IN:

Indiana University Press; 1987:206

12. Proust M. Remembrance of things past. C.K. Scott Moncrieff, T. Kilmar-

tin, trans. New York, NY: Random House; 1981;I:51

13. Plato. Desmond Lee, trans. Timaeus and Critias. New York, NY: Penguin

Classics; 1979

14. Cornford FM. Platos Cosmology. The Timaeus of Plato Translated With a

Running Commentary. London, UK: Rutledge & Kegan Paul Ltd; 1937

15. Hammerschlag MR, Klein JO, Herschel M, Chen FCJ, Fermin R. Patterns

of use of antibiotics in two newborn nurseries. N Engl J Med. 1977;296:

12681269

16. Townsend TR, Shapiro M, Rosner B, Kass EH. Use of antimicrobial

drugs in general hospitals. IV. Infants and children. Pediatrics. 1979;64:

573578

17. Schwersenski J, McIntyre L, Bauer CR. Lumbar puncture frequency and

cerebrospinal fluid analysis in the neonate. Am J Dis Child. 1991;145:

5458

18. Escobar GJ, Zukin T, Usatin M, et al. Early discontinuation of antibiotic

therapy in newborns admitted to rule out sepsis: a decision rule. Pediatr

Infect Dis J. 1994;13:860866

19. Friedman GD. Primer of Epidemiology. New York, NY: McGraw Hill;

1994:10

20. Wientzen RL, McCracken GH. Pathogenesis and management of neo-

natal sepsis and meningitis. Curr Prob Pediatr. 1977;8

21. Siegel JD, McCracken GH. Sepsis neonatorum. N Engl J Med. 1981;304:

642647

22. Dillon HC Jr, Khare S, Gray BM. Group B streptococcal carriage and

disease: a 6 year prospective study. J Pediatr. 1987;110:3136

23. Avery GB, ed. Neonatology. Pathophysiology and Management of the New-

born. Philadelphia, PA: JB Lippincott Co; 1987:72, 643645, 729747,

917943

24. Wiswell TE, Stoll BJ, Tuggle JM. Management of asymptomatic, term

gestation neonates born to mothers treated with intrapartum antibiotics.

Pediatr Infect Dis J. 1990;9:826831

25. Fletcher RH, Fletcher SW, Wagner EH. Treatment. In: Clinical Epidemi-

ology. The Essentials. Baltimore, MD: Williams & Wilkins; 1996;136164

26. Hjalmarson O. Epidemiology and classification of acute, neonatal re-

spiratory disorders. A prospective study. Acta Paediatr Scand. 1981;70:

77378327. Hjalmarson O, Krantz ME, Jacobsson B, Sorensen SE. The importance of

neonatal asphyxia and Caesarean section as risk factors for neonatal

respiratory disorders in an unselected population. Acta Paediatr Scand.

1982;71:403408

28. Krantz ME. Acute respiratory disturbances in newborn infants. An

epidemiological study. Gothenburg, Sweden: Gothenburg University;

1987. Thesis

29. Field DJ, Milner AD, Hopkin IE, Madeley RJ. Changing pattern in

neonatal respiratory diseases. Pediatr Pulmonol. 1987;3:231235

30. US Department of Health and Human Services. International Classifica-

tion of Diseases. 9th Rev. Clinical Modification. 4th ed. Washington, DC:

US Government Printing Office; 1995. DHHS Publication No. (PHS)

91-1260

31. Escobar GJ, Li DK, Armstrong MA, Gardner MN, Folck BF, Verdi JE for

the Neonatal Infection Study Group. The neonatal sepsis work-up:

natural history in a large managed care organization. N Engl J Med.

Submitted

32. Akenzua GI, Hui YT, Milner R, Zipursky A. Neutrophil and band

counts in the diagnosis of neonatal infections. Pediatrics. 1974;54:3842

33. Spector SA, Ticknor W, Grossman M. Study of the usefulness of clinical

and hematologic findings in the diagnosis of neonatal bacterial infec-

tions. Clin Pediatr. 1981;20:385392

34. Philip AGS. Decreased use of antibiotics using a neonatal sepsis screen-

ing technique. J Pediatr. 1981;98:795799

35. Squire EN Jr, Reich HM, Merenstein GB, Favara BE, Todd JK. Criteria

for the discontinuation of antibiotic therapy during presumptive treat-

ment of suspected neonatal infection. Pediatr Infect Dis J. 1982;1:8590

36. Benuck I, David RJ. Sensitivity of published neutrophil indexes in

identifying newborn infants with sepsis. J Pediatr. 1983;96196337. Rozycki HJ, Stahl GE, Baumgart S. Impaired sensitivity of a single early

leukocyte count in screening for neonatal sepsis. Pediatr Infect Dis J.

1987;6:440442

38. Rodwell RL, Leslie AL, Tudehope DI. Early diagnosis of neonatal sepsis

using a hematologic scoring system. J Pediatr. 1988;112:761767

39. Greenberg DN, Yoder BA. Changes in the differential white blood cell

count in screening for Group B streptococcal sepsis. Pediatr Infect Dis J.

1990;9:886889

40. Manroe BL, Weinberg AG, Rosenfeld CR, Browne R. The neonatal blood

count in health and disease. I. Reference values for neutrophilic cells.

J Pediatr. 1979;95:8998

41. Schelonka RL, Yoder BA, desJardins SE, Hall RB, Butler TJ. Peripheral

leukocyte count and leukocyte indexes in healthy newborn term infants.

J Pediatr. 1994;125:603606

42. Schelonka RL, Yoder BA. The WBC count and differential: its uses and

misuses. Contemp Pediatr. 1996;13:12414143. Christensen RD, Rothstein G. Pitfalls in the interpretation of leukocyte

counts of newborn infants. Am J Clin Pathol. 1978;72:608611

44. Christensen RD, Rothstein G, Hill HR, Hall RT. Fatal early onset Group

B streptococcal sepsis with normal leukocyte counts. Pediatr Infect Dis J.

1985;4:242245

45. Husserl E. Churchill JS. trans. The Phenomenology of Internal Time-

Consciousness. Bloomington, IN: Indiana University Press; 1964;4849

46. Mecredy RL, Wiswell TE, Hume RF. Outcome of term gestation neo-

nates whose mothers received intrapartum antibiotics for suspected

chorioamnionitis. Am J Perinatol. 1993;10:365368

47. Miller HC, Conklin EV. Clinical evaluation of respiratory insufficiency

in newborn infants. Pediatrics. 1955;16:427436

48. Miller HC, Behrle FC. Respiratory patterns in newborn infants as de-

termined by airflow and pneumographic studies and their possible

relationship to a deficient vagal mechanism. Pediatrics. 1952;10:272285

49. Miller HC, Behrle FC. Changing patterns of respiration in newborninfants. Pediatrics. 1953;12:141150

50. Miller HC, Smull NW. Further studies on the effects of hypoxia on the

respiration of newborn infants. Pediatrics. 1954;14:93103

51. Miller HC. Effects of high concentrations of carbon dioxide and oxygen

on the respiration of fullterm infants. Pediatrics. 1954;14:104113

52. Miller HC. The effects of hypoxia on the respiration of newborn infants.

Pediatrics. 1955;16:93103

53. Morley CJ, Thornton AJ, Fowler MA, Cole TJ, Hewson PH. Respiratory

rate and severity of illness in babies under 6 months old. Arch Dis Child.

1990;65:834837

54. Richardson DK, Escobar G. Simplified and revalidated score for neona-

tal acute physiology (SNAP-II) maintains excellent predictive perfor-

mance. Pediatr Res. 1998;43:227A

55. Lee SK, Corcoran JD, Whyte R, Thiessen P, The Canadian NICU Net-

work. Neonatal acute physiology parameters index (NAPPI)an effi-

cient illness severity score for neonates. Pediatr Res. 1998;43:227A

56. Weiss MG, Ionides SP, Anderson CL. Meningitis in premature infants

with respiratory distress: role of admission lumbar puncture. J Pediatr.

1991;119:973975

57. Halliday HL. When to do a lumbar puncture in a neonate. Arch Dis

Child. 1989;64:313316

58. Weisman LE, Merenstein GB, Steenbarger JR. The effect of lumbar

puncture position in sick neonates. Am J Dis Child. 1983;137:10771079

59. Visser VE, Hall RT. Lumbar puncture in the evaluation of suspected

neonatal sepsis. J Pediatr. 1980;96:10631067

60. Wiswell TE, Baumgart S, Gannon CM, Spitzer AR. No lumbar puncture

in the evaluation for early neonatal sepsis: will meningitis be missed?

Pediatrics. 1995;95:803806

61. Schuchat A, Whitney C, Zangwill K. Prevention of perinatal group B

streptococcal disease: a public health perspective. MMWR. 1996;45:124

62. American Academy of Pediatrics, Committee on Infectious Diseases

and Committee on Fetus and Newborn. Revised guidelines for preven-



15/16

tion of early-onset group B streptococcal (GBS) infection. Pediatrics.

1997;99:489496

63. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: appli-

cation and methodological standards. N Engl J Med. 1985;313:793799

64. Lapaucis A, Sekar N, Stiell IG. Clinical prediction rules. A review and

suggested modifications of methodological standards. JAMA. 1997;277:

488494

65. Browner WS, Newman TB, Cummings SR. Designing a new study. III.

Diagnostic tests. In: Hulley SB, Cummings SR, eds. Designing Clinical

Research. Baltimore, MD: Williams & Wilkins; 1988;9:8797

66. Breiman L, Friedman J, Olshen R, Stone C. Classification and Regression

Trees. Pacific Grove, CA: Wadsworth; 1984

67. Steinberg D, Colla P. CART: Tree-Structured Non-Parametric Data Analy-

sis. San Diego, CA: Salford Systems; 1995

68. Hanley JA, Lippman-Hand A. If nothing goes wrong, is everything all

right? Interpreting zero numerators. JAMA. 1983;249:17431745

69. Regional Nursery Directors. The Permanente Medical Group, Inc.

Valuation and management of persistent pulmonary hypertension of

the newborn. Oakland, CA: The Permanente Medical Group, Inc;

1996

70. Kindig D. Strategic issues for managing the future physician workforce.

In: Altman SH, Reinhardt UE. Strategic Choices for a Changing Health Care

System. Chicago, IL: Health Administration Press; 1996;149182

71. Pollack LD, Ratner IM, Lund GC. US Neonatology Practice Survey:

personnel, practice, hospital, and neonatal intensive care unit charac-

teristics. Pediatrics. 1998;101:398405

72. Hall RT. Neonatal manpower needs: the writings on the wallwe

should read it and heed it. J Perinatol. 1997;17:423424



16/16

1999;103;360Pediatrics

Gabriel J. EscobarOrganization

Evidence-Based Approach Toward Newborn Infections in a Managed CareThe Neonatal ''Sepsis Work-up'': Personal Reflections on the Development of an

ServicesUpdated Information &

E1/360.full.htmlhttp://pediatrics.aappublications.org/content/103/Supplement_including high resolution figures, can be found at:

References

E1/360.full.html#ref-list-1http://pediatrics.aappublications.org/content/103/Supplement_at:This article cites 50 articles, 17 of which can be accessed free

Citations

E1/360.full.html#related-urlshttp://pediatrics.aappublications.org/content/103/Supplement_This article has been cited by 13 HighWire-hosted articles:

Subspecialty Collections

and_newbornhttp://pediatrics.aappublications.org/cgi/collection/premature_Premature & Newbornfollowing collection(s):This article, along with others on similar topics, appears in the

Permissions & Licensing

mlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhttables) or in its entirety can be found online at:Information about reproducing this article in parts (figures,

Reprintshttp://pediatrics.aappublications.org/site/misc/reprints.xhtml

Information about ordering reprints can be found online:

reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.Village, Illinois, 60007. Copyright 1999 by the American Academy of Pediatrics. All rightstrademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove

andpublication, it has been published continuously since 1948. PEDIATRICS is owned, published,PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly
http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.htmlhttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.htmlhttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.htmlhttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#ref-list-1http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#ref-list-1http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#ref-list-1http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#related-urlshttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#related-urlshttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#related-urlshttp://pediatrics.aappublications.org/cgi/collection/premature_and_newbornhttp://pediatrics.aappublications.org/cgi/collection/premature_and_newbornhttp://pediatrics.aappublications.org/cgi/collection/premature_and_newbornhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/cgi/collection/premature_and_newbornhttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#related-urlshttp://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html#ref-list-1http://pediatrics.aappublications.org/content/103/Supplement_E1/360.full.html

Documents

Pediatrics 1999 Escobar 360 73