A computerized audit of 15,009 emergency department records

ORIGINAL CONTRIBUTION computer; risk management

A Computerized Audit of 15,009 Emergency Department Records

The text of 15,009 emergency department medical records was reviewed with the use of a computer program that detected the presence or absence of key words and phrases. The search focused on "trigger diagnoses," that is, any diagnoses associated with an above-average risk for an undetected but more serious condition. Included were the trigger diagnoses from the five high-risk areas of extremity laceration, epiglottitis, abdominal pain, meningitis, and myocardial infarction. The three kinds of medical records that were compared were handwritten records, records dictated and transcribed, and records created by a voice-activated word processor. From a risk management perspective, inclusion of critical pertinent positives and negatives was taken as an index of quality from a risk management perspective, and records created by a voice-activated word processor using real-time risk management prompts were superior to handwritten and dictated records. The computer holds promise as a vehicle to reduce the cost and frequency of malpractice risk in the ED and as a teaching tool to improve the quality of care. [Holbrook J, Aghababian R: A computerized audit of 15,009 emergency department records. Ann Emerg Med February 1990;19:139-144.]

INTRODUCTION The emergency department record often serves as the sole means by

which the emergency physician communicates with staff physicians con- cerning the treatment of their patients. In addition, the ED record serves as the primary documentation of the patient-physician encounter. For these reasons, scrutiny of the emergency record is crucial for ED risk management and quality assurance. In fact, the written ED record typically stands as the central piece of evidence for the defense in a malpractice proceeding.

Ambiguous, overly abbreviated, or illegible records are difficult to interpret, defend, and include in any large-scale analysis. The general practice of using narrative or "free" text rather than a multiple-choice format in creat- ing medical records has contributed to the difficulty of any large-scale audit. Because emergency physicians generally have resisted the use of any highly structured format for the emergency record, computer review of these documents has not been feasible: 1

In emergency medicine, most malpractice risk concentrates in the "failure to diagnose" a small number of diagnoses. 2 According to one analysis of closed claims in the ED, 3 ten high-risk diagnoses accounted for nearly 90% of the total amount paid in settlements. In that study, for example, "failure to diagnose myocardial infarction" accounted for more than 30% of the total amount paid in closed claims. When the actual records of the visits in which the myocardial infarction has been missed are read, a collection of such misdiagnoses as "left shoulder bursitis," "bronchitis," and "hyperventilation" is found, reflecting some of the symptoms that may present as an occult myocardial infarction. It is from these misdiagnoses that we took our concept of trigger diagnosis.

Initially, the list of trigger diagnoses was drawn from the collection of actual misdiagnoses found in closed claims in which physicians had "failed to diagnose" more serious conditions. Then, this list was broadened to include other logically similar diagnoses and can now focus on any widespread risk management audit of ED records. For example, the list of

John Holbrook, MD, MA, FACEP* Springfield, Massachusetts Richard Aghababian, MD, FACEP¢ Worcester, Massachusetts

From the Emergency Department, Mercy Hospital, Springfield, Massachusetts;* and the Division of Emergency Medicine, Department of Medicine, University of Massachusetts Medical School, Worcester.¢

Received for publication May 8, 1989. Revision received September 11, 1989 Accepted for publication September 28, 1989.

Address for reprints:John Holbrook, MD, MA, FACER Emergency Department, Mercy Hospital, PC Box 9012, Springfield, Massachusetts 01102-9012.

19:2 February 1990 Annals of Emergency Medicine 139/57

COMPUTERIZED AUDIT Holbrook & Aghababian

120 - -

100

80

% 60

i 40

.4

20

100 98

I I I I Nerve Tendon Artery Foreign body

Examination

[ ] Written [ ] Dictated m Computer

2

t 8O

6o

%

t 20 J

o I i

Patent airway

Examina t ion

94

[ ] Written [ ] Dictated [ ] Computer

FIGURE I. Trigger diagnoses for extremity laceration.

FIGURE 2. Trigger diagnoses for epiglottitis.

trigger diagnoses for myocardial infarction includes the diagnoses by which myocard ia l in farc t ion has been missed in the past and, by infer- ence, is likely to be missed in the future. Thus, a trigger diagnosis is associated with above-average risk because it may conceal a more serious condition. As such, a trigger diagnosis should alert the clinician that the presence or absence of certain key findings must be collected and recorded on the chart to exclude the more serious condition. In addition, lists of trigger diagnoses can be used as the basis for specific quality assurance audits.

In this retrospective analysis of 15,009 ED records, we tested whether the type of device (handwriting, dictation, or computer) had any effect on the markers of adequate or inadequate documentation for six lists of high-risk trigger diagnoses, that is, myocardial infarction, meningitis or central nervous system (CNS) bleed, ectopic pregnancy, appendicitis, ex- t remity laceration, and airway ob- struction. The null hypothesis was that the type of device had no effect on markers of adequate or inadequate documentation.

In 1988, greater efficiency in computerized review of free text enabled small personal computers to "read" ED records, find the diagnoses, and then scan the free text to pick out critical pertinent positives and negatives in the history and physical examinations. Such a file search provides the basis for the software to de-

termine, from a risk management perspect ive, the adequacy of the medical database for a given diagnosis. This analysis requires only that the record be transcribed on a word processor instead of a type- writer and that the transcribed file be backed up on a floppy disk. Records typed onto paper can be converted back into computer format with an optical scanning device, but in our pilot study this proved to be nine times more costly than merely retyp- ing the records on a word processor. For an ED that is already using dictation and transcription, the only addi- t ional expense is the cost of the floppy disks; this cost is approximately $1.00 per 250 ED records.

A second major advance in 1988 was the introduction and availability of voice-activated word processors that recognize and directly convert specific verbal phrases into computer text wi thou t human transcribers. Such systems now exist for ED records and have vocabularies of up to 5,000 words and phrases available for use. When a record is created with this technology, a high-risk diagnosis can be recognized as the physician speaks so that the computer can sug- gest critical pertinent positives and negatives to be carried out and recorded to improve the physician's eva lua t ion for a given diagnosis. These suggestions, or "prompts , " were entirely optional for all the critical pertinent positives and negatives in the study and had to be spoken to be included in a particular medical record.

M E T H O D S In our study, 15,009 medical rec-

ords were reviewed retrospectively to examine the relationship between

the quality of records from a risk m a n a g e m e n t perspect ive and the method used to generate the record. The investigation focused on several high-risk areas of diagnostic failure: myocardial infarction, meningitis or CNS bleed, appendici t is , ectopic pregnancy, epiglottitis, and occult nerve or tendon injury or occult foreign body in an extremity laceration. The records generated from February 1987 th rough January 1989 were taken from two community hospitals staffed by the same group of emergency physicians. The records at the smaller hospital were all hand-written; the records at the larger hospital were either dictated and manually transcribed or created by voice-activated word processors. At the larger hospital, nurse practitioners created approximately 25% of the records, whereas at the smaller hospital, only physicians created the records.

Overall, the physicians spent equal rotations at the two hospitals. Eleven physicians and five nurse practitioners participated in the study. All of the physicians practiced emergency medicine on a full-time basis. Four were board-certified in emergency medicine and another primary care specialty; two were board-certified only in emergency medicine; two were board-certified in a primary care specialty other than emergency medicine; one was board-eligible in a pr imary care specialty other than emergency medicine; and two were not board-eligible or board-certified.

Five hundred thir ty-eight handwri t ten charts were selected ran- domly for the study. Ini t ial ly, a medical transcriptionist attempted to type the contents of the charts onto a word processor. However, because of widespread illegibility, we were un-

58/140 Annals of Emergency Medicine 19:2 February 1990

100

80

60 %

40

20

0 Loc Neuro Neck Rash Fundi Hydration

Examination

• Written [ ] Dictated [ ] Computer

120

100

80

% 60

40

20

0 4 6

Orthopedic

100

5 7 ~ f ~ 37

I Abdomen Rectal

Examination


30 29 30

" 011o h 0 I I

Pelvic Pregnancy Examination

• Written [ ] Dictated [ ] Cemputer

100

80

60 %

40

20

0

-- 95

I I I Chest pain Lung

Examination

91

Hea~


able to get any meaningful data from these records. Therefore, each physician was asked to recreate these records by reading his written charts into a dictating machine. They were asked to interpret the more arcane abbreviations into understandable language without introducing new thematic content into the charts. Then, these records were transcribed onto a word processor and analyzed.

At the larger hospital, 13,007 transcribed ED records were taken from floppy disks, which represented approximately 25% of all the records transcribed during the period from February 1988 through January 1989. These records disproportionately represented pat ients who were dis- charged because virtually all the patients who were admitted had handwritten records. Apart from this bias, these charts had no particular ob - vious trait other than being sent to a particular typing service that backed up all work on floppy disk. We also studied 964 records created on voice- activated word processors, which represented all ED records created using this technology during a nine-month

trial period at this hospital. To generate medical records more

rapidly, the voice-activated word processor recognizes "trigger phrases," which are single words or small word groups tha t t rans la te to words, phrases, sentences, or paragraphs of text. The contents of the trigger phrases have been edited carefully to maximize the risk management as- pect of the charts, while at the same time accurately reflecting the medical encounter. The content of these trigger phrases changed over time as we learned what texts worked well and what texts were too cumbersome tO r i s e .

We used a computer program to search these 15,009 charts and list the diagnoses. We then decided which of these 15,009 diagnoses represented a trigger diagnosis with respect to the high-r isk diagnoses under s tudy: for example, " l e f t shoulder bursitis" in a 45-year-old man would be considered a trigger diagnosis with respect to failure to diagnose myocardial infarction. For each high-risk diagnosis, we developed a corresponding list of trigger

A n n a l s of E m e r g e n c y M e d i c i n e

FIGURE 3. Trigger diagnoses for meningitis and CNS bleed.

FIGURE 4. Trigger diagnoses for appendicitis.

FIGURE 5. Trigger diagnoses for ectopic pregnancy

FIGURE 6. Trigger diagnoses for myocardial infarction.

diagnoses. In addition, for each high-risk diag-

nosis, we developed a list of critical pertinent positive and negative findings that would help exclude that high-risk diagnosis. For meningitis, the list included some ment ion of level of consciousness, aspects of a neurologic examination, meningismus, petecchial rash, fundoscopic examination, and state of hydration. For extremity lacerations, the list included a determination of nerve, artery, and tendon status and of the possibil i ty of foreign body in the wound. For epiglottitis, the list included only an examination for patency of the airway.

19:2 February 1990 141/59


For abdomina l pain, the list included a review of orthostatic symptoms, an abdominal examination, a rectal examination, and, for females between 15 and 40 years old, a pelvic examinat ion and a considerat ion of pregnancy. For myocardial infarction, the list included asking about chest pain, a lung examination, and a heart examinat ion. The test for the presence of a particular examination was empirically derived by choosing from the various ways as recorded in the 15,009 charts that the practi t ioners actually described that particular examination. We then used this list of words and ph ra ses to d e t e r m i n e whether a pract i t ioner had actually documen ted a par t icular examination in a given record. This method m a k e s the search m o r e sens i t ive than specific in detecting a particular examination.

We have modeled the adequacy of documentat ion with one or two predictive factors. With one factor, a signif icant X 2 test indicates that some categories of the one factor produce s ignif icant ly different rates of adequacy; these categories usually stand out in a graph of results. However, wi th two factors, graphs of results need not clarify the dominant categories. Thus, we have used the method of logqinear analysis to assess the overall importance of factors.

A separate analysis for each diagnostic category directly addresses the practical issue of specific medical sit- uations where better documentat ion may or may not help.

Th i s ana lys i s m e t h o d p r o t e c t s against overinterpretation of the data but (as the reference noted) includes hypotheses of no medical importance such as the hypothesis of equal pro- por t ions of wr i t ten , d ic ta ted , and compute r i zed records. To pick out more meaningful significant effects, we reanalyzed the data using step- wise logistic regression. This enabled us to specify categories within a factor as variables of interest. To avoid type 1 errors, we used a cutoff of .005 for a significant P value instead of .05.

For each diagnostic ca tegory (eg, laceration, airway patency), three factors were considered: the type of examination, the type of device (handwriting, dictation, or computer), and the marker of adequate or inadequate documentation. We analyzed each di- agnost ic category separately using

log-linear analyses to detect which of the three factors contributed significant variation to the three-way con- t ingency table. As part of this analysis, within each table, X 2 analysis of two-way tables was done to confirm the impor tan t findings. The log-linear analysis was performed using the BMDP s ta t i s t i ca l package, op t ion BMDP4F, and logistic regression was pe r fo rmed us ing op t ion BMDPLR, both on the Harris 1000 computer at t he U n i v e r s i t y of M a s s a c h u s e t t s Medical Comput ing Facility.

RESULTS Laceration Diagnostic Category

The device accounted for most of the variation in the three-way table (log-linear, P < .0001), the marker was significant (logqinear, P < .002), and the examinat ion was not. Sec- ond-order interact ions were significant , but th i rd -order i n t e r ac t i ons were not. In other words, some com- binations of a specific examinat ion and a specific marker differed signifi- c an t l y f r o m expec ted values. The compu te r device produced signifi- c a n t l y h i g h e r r a t e s of a d e q u a t e m a r k e r s (Figure 1). T h e l o g i s t i c model selected the variables "computer" and "nerve examinat ion" and the in teract ion term "dictated rec- o r d " for " t e n d o n e x a m i n a t i o n " (P < .0001).

Epiglottitis Diagnostic Category This analysis had only one exam-

ination category, so it was reduced to a two-way table. The device (handwriting, dictation, or computer) was h igh ly s ign i f i can t (log-linear, P < .0001) in accounting for differences in the p resence of adequa te ma rke r s (Figure 2). The computer generated adequate markers in 94% of cases, whereas the other methods did so in 21% or fewer cases.

Meningitis-CNS Bleed Diagnostic Category

The device accounted for most of the variation in the three-way table (log-linear X 2 40,045 with P < .0001), the marker was significant (log-linear X ~ 6,679 wi th P < .0001), but the type of examination was not. The second- order factor of device and examina- t ion was s ign i f i can t ( log-l inear x 2 7,706 with P < .0001). As shown (Fig- ure 3), the logistic analysis selected the variables "computer, . . . . neck examination," and "fundoscopic exam-

inat ion" as the m o s t significant fac. tors (P < .0001) and the "examina. t ion for rash" and the "dic ta ted neck examina t ion" as less-significant factors (P < .005).

Appendicitis Diagnostic Category

The device accounted for m o s t of the variat ion (log-linear X 2 5,055, P < .0001), the m a r k e r was s ignif icant (log-linear X 2 572, P < .001), and the device-examinat ion effect was significant (log-linear X 2 768, P < .0001). The logistic analysis provides a better i n t e r p r e t a t i o n of Figure 4, in which the or thostat ic examinat ion is highly significant (P < .0001) and the abdominal examinat ion and the computer results of the abdominal examination are less significant (P < .05). The devices had similar rates for the orthostat ic examinat ion (X 2, P = .31) and differed significantly for the abdominal examination, wi th the computer rate higher (P < .0001) and the dictated rate higher (P < .0001). The d i c t a t e d ra te was ba re ly (s ignif i - cantly) higher for the rectal examination (P < .05).

Ectopic Diagnostic Category The device accounted for mos t of

the variation (log-linear ×2 1,330, P < .0001), the marker type was significant (log-linear X 2 211, P < .0001), and the m a r k e r - e x a m i n a t i o n combina~ tion was significant (log-linear X 2 16, P < .0001). The log i s t i c ana lys i s found only one significant effect, the d ic ta ted rate w i th p r egnancy (P < .001). The devices were s imilar for the pe lv ic e x a m i n a t i o n , w i t h the writ ten device giving the highest rate (X 2, P = .20). The dictated device was highest for the pregnancy examination (P = .24).

Myocardial Infarction Diagnostic Category

The device accounted for mos t of the variation (log-linear X 2 6,804, P < .0001), the marker type was significant (logqinear X 2 165, P < .0001), and the largest second-order factor, marker-examination, was significant (log-linear x 2 553, P < .0001). The logistic analysis assigned significance to the variable " c o m p u t e r " and to the chest examination (P < .001).

D I S C U S S I O N Overview of Analytic Results

The log-linear analyses focused on


f

overall variation within factors, ig- noring that the adequacy of markers was the outcome of interest. For all diagnoses, the factor device domi- nated, and in all but the diagnostic c~tegory of ectopic pregnancy, the higher rates of adequacy among the computer records stood out as the major source of this variation. The logistic regression analyses treated adequacy as an outcome and assessed the best combination of predictors. Again, the category of computer device d o m i n a t e d the pred ic t ions . These analyses also showed substan- tial variation in the probability of an adequate chart among the types of examinations. Finally, the simple X ~ analysis addressed differences among the adequacy rates of the devices within each examination type: differences apparent in Figures 1 through 5. We ignored P values of more than .005 to avoid type 1 errors.

Our analysis revealed that written ED charts provide inadequate documentation because they are often illegible, incomplete, or both. That is not to say that physicians do less complete examinations when they must wri te the i r records; rather, these results probably highlight the bad habit of poor documentation fail- ing to reflect a generally good job.

For extremity laceration (Figure 1) and epiglottitis (Figure 2), the written, dictated, and computer-assisted charts form three separate groups with respect to adequacy of documentation. The data for meningitis (Figure 3) show that for "level of consciousness," "neurological examination," and "meningismus," the practitioner generally responded to the computer prompt by including those examinations. In the case of "fundoscopic examination," "petecchial rash," and "state of hydration," the practitioner was not as ready to ac- cept the prompt. In fact, our analysis of the frequency of the fundoscopic examination in written and dictated charts led us to observe that in the population observed, the practice of fundoscopic examination is no longer standard in the age of computed to- mography and magnetic resonance imaging scans; on this basis, the prompt for fundoscopic examination was deleted from the voice-activated word processors.

The data from appendicitis (Figure 4) and pregnancy (Figure 5) show that even with computer prompts, if no

rectal or pelvic examination is performed, none will be recorded. Dur- ing our study, we had not developed a prompt for reminding practitioners of the possibility of pregnancy in women between 15 and 40 years old. Prompts to comment on these items are being gradually worked into our voice-activated word processor.

Most of the charts grouped under trigger diagnosis for myocardial infarction (Figure 6) did not mention "chest pain" in the diagnosis. In this group, we found the type of diagnoses often cited when a myocardial infarction is missed: hiatal hernia, peptic ulcer disease, costochondritis, and dyspnea. As we noted above, several researchers have developed ele- gant algorithms for the diagnosis of myocardial infarction. 4 6 These algorithms cite various factors in the history and physical examination that are predictive for the presence or absence of myocardial infarction. But fundamental to availing oneself of these algorithms is being alert to the possibility that the patient complain- ing of "left shoulder pain" actually may be suffering from myocardial is- chemia. Because almost two thirds of the written and dictated charts in this group did not make any mention of chest or substernal pain or its equivalent, we did not look further for c o m m e n t s about smoking or medical history in this study.

CONCLUSION This quantitative risk analysis of

medical or ED records has shown the written record and the record that has been dictated and transcribed to be inferior to records created by the voice-activated word processor. All too often, written records are illegible. In their documentation, emergency physicians who write their own charts do a poor job of risk management. Written records cannot be effectively reviewed on any meaningful scale. While EDs may have per- sisted in this practice for many years, the growing amount and frequency of malprac t ice judgments soon will make handwri t ten charts an eco- nomic liability.

Dictated records reduce such risks. Above all, they are legible. In addition, practitioners document better when they dictate. Now the ability to audit transcribed charts automat- ically by computer opens up new ho- rizons in quality assurance. Educa-

tional programs in risk management can be simply, efficiently, and effectively evaluated by auditing charts before and after the program. The de- cay of the learning can be monitored by an ongoing audit of a department. 7 By making individuals aware of the results of an ongoing audit, new and better habits of documentation can be reinforced and perpetuated. If such an auditing process markedly im- p roves d o c u m e n t a t i o n v is -5-v is trigger diagnoses, then this could decrease the scope and cost of the malpractice problem. There is a pre- cedent for this in Massachusetts: an- esthesiologists who use oximetry throughout surgery may qualify for a 20% discount in their malpractice premiums.

In the same vein, voice-activated word processors offer promise as a quality assurance tool. For the first time, it may be possible and practical for the computer to prompt the physician before the patient has left the ED. Such prompting demonstrably creates a better record. The better record may not in itself produce better medical care; however, it corrects the bad habit in the realm of risk management of not fully document- ing a quality examination. In addition, frequent prompting about critical pertinent positives and negatives tends to have a positive educational effect, s Moreover, some evidence suggests that making better records does lead to better care.9, l° For emergency physicians stressed by unex- pected volume, exceedingly complex complaints , or fatigue, compute r prompting should reduce the number of omission errors.

Finally, in addition to raising the standard of practice, combining the computerized audits from many hospitals would provide a data bank of risk management profiles for discov- ery and refinement of trigger diagnoses. Such a data bank would document items of history and physical examinat ions performed in thou- sands of encounters. While individuals or groups may advance norma- tive guidelines or standards, ulti- mately the legal standard of practice is determined by what most of the physicians practicing good medicine in a community actually do. For the first time, this standard of practice is observable and can be monitored on a central data bank that is accessible to individual hospitals and physi-

19:2 February 1990 Annals of Emergency Medicine 143/61


cians. An empirical set of standards for practice would more clearly de- fine individual cases of malpractice; it also could aid physicians in self- evaluation and in selection of more helpful topics for their continuing education.

By the same token, this set of standards would highlight the fact that our ability to diagnose is finite and that not every misdiagnosis and real- occurrence is malpractice. It might help establish an acceptable misdiagnosis rate specific to a particular set- ting or locale.

The authors thank Robert Lew, PhD, for his invaluable help in research design and statistical analysis; Roger Manahan, MLS,

for his ass i s tance in the research effort; and Patricia U s c h m a n n and Cathie Gavin for their assis tance in the t ranscript ion of medical records.

REFERENCES 1. McDonald CJ, Tierney WM: Computer- stored medical records: Their future role in medical practice. JAMA 1988;259:3437-3439.

2. Trautlein JJ, Lambert RL, Miller J: Malprac- tice in the emergency department - Review of 200 cases. Ann Emerg Med 1984;13:709-711.

3. Rogers JT: Risk Management in E#Rergency Medicine. Dallas, American College of Emer- gency Physicians, 1985.

4. Goldman L, Cook EF, Brand DA, et al: A computer protocol to predict myocardial infarction in emergency department patients with chest pain. N Engl J Med 1988;318:797-802.

5. Tierney WM, Roth B], Psaty B, et al: Predic-

tors of myocardial infarction in emergency room patients. Crit Care Med 1985;13:526-531.

6. Lee TH, Cook EF, Weisberg M, et al: Acute chest pain in the emergency room: Identifica- tion and examination of low-risk patients. Arch Intern Med 1985;145:65-69.

7. Sanazaro PJ, Worth RM: Concurrent quality assurance in hospital care. N Engl J Med 1978; 298:1171-1177.

8. Harchelroad FP, Martin ML, Kremen RM, et al: Emergency department daily record review: A quality assurance system in a teaching hospital. QRB 1988;14:45-49.

9. Murphy JG, Jacobson S: Assessing quality of emergency care: The medical record versus pa k t ient outcome. Ann Emerg Med 1984;13: 158-165.

10. Lyons TF~ Payne BC: The relationship of physicians' medical recording performance to their medical care performance. Med Care 1974;12:463-469.


Documents

A computerized audit of 15,009 emergency department records