. . . . . Quality of Care . . . . .

© 2007 National Rural Health Association 173 Spring 2007

Safety and quality of care are high on the agendas of government, of payers, of providers, and increasingly of patients and their families. 1,2 However, in rural settings, relatively little is known about the factors that

pose the greatest threat to quality. The recent Institute of Medicine report on rural health care 3 draws attention to the fact that rural communities are heterogeneous and diverse, and it calls for health professional competencies that include applying quality improvement and working in teams.

The paradigm of complex adaptive systems suggests that each medical practice can be viewed as a complex adaptive microsystem; to thrive, such a microsystem needs to identify its own unique set of problems and devise solutions that are tailored to the situation, in light of the current quality status, practice costs, and resources available. 4-6 To achieve this, the measurements that are used to identify and prioritize quality and safety problems must be trusted by the members of the practice. The literature refl ects skepticism regarding externally driven measures such as practice profi les and report cards, 7-9 suggesting that, in their current forms, they may not be trusted by many physicians as fair and valid measures.

One approach to identifying problems internally has been to use error reports. The intention is that practice staff members complete these documents voluntarily when they notice mistakes or adverse events and that the information is used internally to drive improvement. Error reporting systems are widely available in hospital and ambulatory settings (commonly under the title “ incident reports ” ), but their usefulness has always been limited by the problem of underreporting. 1

An alternative approach that permits involvement of all team members to identify and prioritize safety and quality problems is Failure Modes and Effects Analysis (FMEA). This has been widely used in other high-risk industries and has been advocated by the Institute of Medicine 1 as a means of analyzing a system to identify its weaknesses (failure modes) and possible consequences of failure (effects) and to prioritize areas

ABSTRACT : Context: Rural primary care is a complex

environment in which multiple patient safety challenges

can arise. To make progress in improving safety with

limited resources, each practice needs to identify those

safety problems that pose the greatest threat to patients

and focus efforts on these. Purpose: To describe and fi eld-

test a novel approach to prioritizing safety problems in

rural primary care based on the method of Failure Modes

and Effects Analysis. Methods: A survey instrument

designed to assess perceptions of medical error frequency,

severity, and cause was administered anonymously to

staff of 2 rural primary care practices in New York State.

Responses were converted to quantitative hazard scores,

which were used to make priority rankings of safety

problems. Concordance analysis was conducted. Results: Response rate was 94% at each site. Analysis yielded a

list of priorities for each site. Comparison between staff

groups (provider vs nursing vs administration), based on

the top 10 priorities perceived by staff, showed 53%

concordance at one site and 30% at the other.

Concordance between sites was lower, at 20%.

Conclusions: Initial fi eld-testing of a Failure Modes and

Effects Analysis approach in rural primary care suggests

that it is feasible and can be used to estimate, based on

staff perceptions, the greatest threats to patient safety in

an individual practice so that limited resources can be

focused appropriately. Higher concordance between staff

within a practice than between practices lends

preliminary support to the validity of the approach.

1 Patient Safety Research Center, Department of Family Medicine,

State University of New York, Buffalo, NY. 2 Niagara Family Medicine Associates, Niagara Falls, NY.

3 Department of Psychology, Canisius College, Buffalo, NY.

For further information, contact: Ranjit Singh, MD, MBA, Patient

Safety Research Center, UB Department of Family Medicine,

UB Clinical Center, 462 Grider Street, Buffalo, NY 14215;

e-mail rs10@buffalo.edu.

Prioritizing Threats to Patient Safety in Rural Primary Care Ranjit Singh , MD, MBA ; 1 Ashok Singh , MD; 2 Timothy J. Servoss , MA ; 1,3 and Gurdev Singh , MSc Eng, PhD 1

. . . . . Quality of Care . . . . .

The Journal of Rural Health 174 Vol. 23, No. 2

for improvement. The Joint Commission on Accreditation of Healthcare Organizations has required since 2002 that all accredited hospitals perform proactive risk assessment each year following a series of steps based on FMEA. 10,11 It is important to point out that this method is based on the perceptions of informed decision makers.

This process, usually made up of 8 steps, 11 is time consuming, costly, and requires considerable expertise and experience. In hospital settings, trained quality-improvement personnel are available, and leadership is mandated by the Joint Commission on Accreditation of Healthcare Organizations to provide the necessary resources for this type of activity. In ambulatory care, however, these factors typically are not present, and the literature is sparse regarding use of FMEA in this setting. In rural ambulatory settings, the scarcity of the necessary resources and expertise is particularly problematic. 3

In an attempt to overcome some of these practical barriers while maintaining the essential thrust of FMEA for the community primary care setting, the authors have developed an adapted FMEA process. This article describes that process and its preliminary fi eld-testing in 2 primary care practices serving rural populations. The adapted FMEA process involves administration of an anonymous safety survey to practice members followed by analysis of the results in order to develop a priority list of safety threats. Ultimately, practices are to use the priority list for developing individualized safety improvements, although that step was not included in the fi eld test reported here. The purpose of the fi eld test is to examine feasibility and begin to address validity issues.

Methods Anonymous FMEA Survey . The fi rst step in the

adapted FMEA process was to begin to understand the system of care in the practice. This was done by fi rst identifying the various entities in the practice (such as the patient, provider, nurse, and chart), listing the main interactions between them, and then portraying them in a diagram (Figure). 12 This diagram was then used as the basis for an anonymous survey. Errors can occur at any point in the practice, including within entities and in the interactions between entities. The survey was designed to examine 12 key areas, with 1 page of the survey dedicated to each (eg, the nurse-provider interaction). Each page consisted of a list of failure modes (errors or causes of error) that can occur in that part of the system. The lists, which include a total of 140 different failure modes, were developed after a review of the literature 1,13-19 and consultation with the

study practice leaders. The lists can be customized to incorporate special circumstances for any given practice if desired. Table 1 shows an example of items on a page from the survey. Participants were asked to consider each of the listed errors in turn and, for each, to respond with their perception of the frequency of the error and the likely severity of the consequences. Explanations of the categorical choices were given at the bottom of each survey page (these appear as footnotes in the table). The diagram of the practice entities was included on each survey page, with the appropriate entities highlighted to orient respondents to the part of the offi ce being assessed.

All members of the 2 practices were given the same survey. They were asked to complete the survey anonymously, stating (if willing) the group to which they belonged (provider, nursing, or administrative).

Hazard Analysis for Prioritization . The goal of the analysis was to rank the failure modes from the survey according to the size of their effects, as perceived by practice staff. We chose to measure effects using the concept of hazard borrowed from engineering. The hazard posed by any given error (or failure mode) is equal to the frequency (ie, probability) with which it occurs multiplied by the severity of the consequences that accrue when it does occur (hazard = frequency × severity).

In the survey, each respondent had rated the frequency and severity of each error according to the categorical scales shown in Table 1 . These categorical responses had to be converted to numerical values so that hazard values could be calculated, summarized, and compared. The fi rst step in this process was to

Practice Diagram. Simplifi ed Diagram of Primary Care Practice Entities (Boxes) and Interactions and Processes (Arrows), as Patient Moves From Assessment Phase to Formulation of Management Plan.

Chart

Provider

Nurse

OfficeStaff

Patient

Third Party

Patient

Nurse

OfficeStaff

Third Party

Assessment Plan

. . . . . Quality of Care . . . . .

Singh, Servoss and Singh 175 Spring 2007

calculate frequency in terms of each practice ’ s patient volume, yielding a rate per 1,000 patient visits. For example, the “ frequent ” probability of occurrence (which was described as 1 or more times in a week) was approximated as 2 times per week. At site 1, 220 patients were seen per week. Hence, “ frequent ” was converted to 2/220 = 0.009, or 9 times per 1,000 visits. Similarly the “ remote ” probability, described as less than once a year, was approximated to 0.5 times per year and calculated as 0.5/11,000 (patients seen per year) = 0.00005, or .05 times per 1,000 visits.

For the severity scores, conversion of the categorical responses to numerical equivalents required decisions regarding the relative weights of the outcomes. For example, how much worse is a “ moderate ” outcome than a “ mild ” outcome? In utility theory, 20 the severity outcome can be seen as a loss of

utility, or disutility. In health care settings, risk aversion is most appropriate as it gives proportionately more weight to the most severe outcomes, that is, those that are most important to avoid. Based on this, the following numerical values were assigned: minimal = 0.01, mild = 0.05, moderate = 0.2, severe = 1.0. Sensitivity analysis using different values, maintaining the risk aversive assumption, showed that hazard rankings remained stable, suggesting that the precise values chosen are not critical to the process.

Hazard scores were then calculated for each failure mode for each respondent by multiplying the numerical conversions for frequency and severity. Mean hazard scores were also calculated for each respondent subgroup (providers, nursing, administrative) and for each practice site as a whole. To allow analysis of concordance between practices, hazard values were scaled to give a range of .01-100.

The hazard scores were also listed in rank order (highest to lowest) for each subgroup and practice.

Setting . The fi eld test was conducted at 2 primary care practices serving rural populations in New York State. Site 1 was a hospital-affi liated clinic with 4 physicians and midlevel providers, 5 nurses and medical offi ce assistants, and 7 administrative offi ce staff. Site 2 had 6 physicians and midlevel providers, 10 nurses and medical offi ce assistants, and 16 offi ce staff. Site 2 also was a rural residency training site.

Results At each practice, 94% of staff voluntarily

participated. Surveys were completed during personal time, with respondents reporting they spent 20-40 minutes on this activity.

Table 2 shows the top 3 hazards for each practice in each of the 12 practice areas included in the survey, based on all respondents. Similar tables can be used to compare the perceptions of the various subgroups within each practice. It is important to note that the absolute values of the hazard scores have no meaning in themselves, it is the rank order of scores that is important since this is what is used to determine priorities.

To compare the priorities identifi ed by each subgroup of respondents within each practice, the rank orders of hazards identifi ed by each subgroup were compared in a pair-wise fashion. That is, separate comparisons were made between providers and nursing, providers and administrative staff, and nursing and administrative staff. Comparison was made by examining the top 10 hazard scores for each subgroup since these are the items that are most likely to be prioritized for remedial action. For site 1, the

Table 1. Survey Example (Nurse Section of 12-Part Survey)

Type/cause of error: Nurse (for each category below, respondents list their perceived frequency/probability of the event ’ s occurrence * and the severity of its consequences † Triage: Does not correctly identify emergency cases Incorrect measurement or recording of vital signs Not making use of available resources for advice/help Nurse fatigued, stressed, ill Nurse in a hurry Errors in handling outside test results Delay in review of results Does not identify results that require urgent attention Errors in carrying out in-offi ce tests Wrong patient Wrong test Incorrect reading of test results Equipment malfunction or miscalibration Incorrect communication of results Errors in calling in prescriptions to pharmacy Wrong patient Wrong medication/treatment Wrong dose Wrong route Wrong times Other

* Four choices for frequency/probability are: frequent (one or more times in a week); occasional (one or more times in a month); uncommon (one or more times in a year); remote (less than once a year).

† Four choices for severity of consequences are: severe (severe or irreversible complications unrelated to natural course of illness [eg, disability, loss of function, hospitalization]); moder-ate (mild or moderate reversible complications unrelated to the natural course of the illness, not requiring hospitalization); mild (increased length or severity of illness, not requiring hospitalization); minimal (no increase in length or severity of illness).

. . . . . Quality of Care . . . . .

The Journal of Rural Health 176 Vol. 23, No. 2

Tab

le 2

. R

an

kin

gs

of

Top

3 P

erc

eiv

ed

Ty

pe

s/C

au

ses

of

Err

ors

in 1

2 P

ract

ice

Are

as

by

Stu

dy

Sit

e

Are

a

Site

1 (N

= 1

5)Si

te 2

(N =

30)

Typ

e/C

ause

of

Erro

rM

ean

Haz

ard

V

alu

e *

Typ

e/C

ause

of

Erro

rM

ean

Haz

ard

V

alu

e *

Rec

epti

on

Mis

fi le

d r

eco

rd2.

33R

ecep

tio

nis

t in

a h

urr

y19

.35

Lon

g w

ait

in o

ffi c

e2.

10M

isfi

led

rec

ord

18.5

6 R

ecep

tio

nis

t fa

tig

ued

, str

esse

d, i

ll1.

71Lo

ng

wai

t in

offi

ce

17.9

3

Nu

rse

Nu

rse

in a

hu

rry

10.9

5N

urs

e in

a h

urr

y18

.14

Nu

rse

fati

gu

ed, s

tres

sed

, ill

3.05

No

t u

sin

g a

vaila

ble

res

ou

rces

fo

r h

elp

7.98

N

ot

usi

ng

ava

ilab

le r

eso

urc

es f

or

hel

p1.

38N

urs

e fa

tig

ued

, str

esse

d, i

ll5.

30

Nu

rse-

pat

ien

t

inte

ract

ion

Mis

un

der

stan

din

g b

ecau

se p

atie

nt

in a

hu

rry

1.81

Mis

un

der

stan

din

g b

ecau

se n

urs

e in

a h

urr

y14

.31

Inad

equ

ate

pat

ien

t ed

uca

tio

n a

bo

ut

trea

tmen

t1.

74M

isu

nd

erst

and

ing

bec

ause

pat

ien

t in

a h

urr

y13

.52

Inad

equ

ate

pat

ien

t ed

uca

tio

n a

bo

ut

dis

ease

1.67

Inad

equ

ate

pat

ien

t ed

uca

tio

n a

bo

ut

trea

tmen

t7.

01

Nu

rse-

char

t

inte

ract

ion

Failu

re t

o u

pd

ate

char

t ad

equ

atel

y3.

90In

com

ple

te/n

ot

up

dat

ed c

har

t22

.69

Inco

mp

lete

/no

t u

pd

ated

ch

art

3.53

Med

icat

ion

list

no

t u

p t

o d

ate

21.4

0 R

ead

ing

fro

m w

ron

g c

har

t2.

04M

issi

ng

ch

art

20.9

9

Pat

ien

t

(ass

essm

ent)

Mas

ks s

ymp

tom

s b

y in

app

rop

riat

e se

lf-t

reat

men

t13

.13

Del

ay in

see

kin

g m

edic

al a

tten

tio

n20

.55

Del

ay in

see

kin

g m

edic

al a

tten

tio

n11

.37

Do

es n

ot

pro

vid

e ac

cura

te in

form

atio

n a

bo

ut

med

ical

pro

ble

ms

15.0

7 D

oes

no

t p

rovi

de

accu

rate

info

rmat

ion

ab

ou

t m

eds

take

n8.

54D

oes

no

t p

rovi

de

accu

rate

info

rmat

ion

ab

ou

t h

abit

s13

.08

Pro

vid

er

(a

sses

smen

t)P

rovi

der

in a

hu

rry

1.50

Pro

vid

er in

a h

urr

y11

.61

Pro

vid

er f

atig

ued

, str

esse

d, i

ll0.

70P

rovi

der

fat

igu

ed, s

tres

sed

, ill

6.10

N

ot

acco

un

tin

g f

or

pat

ien

t ’ s

soci

al/l

ivin

g c

on

dit

ion

s0.

67N

ot

mak

ing

use

of

exte

rnal

res

ou

rces

(co

nsu

ltat

ion

)5.

53

Pro

vid

er-p

atie

nt

in

tera

ctio

n

(a

sses

smen

t)

Inad

equ

ate

his

tory

bec

ause

pat

ien

t ca

nn

ot

exp

ress

cle

arly

2.87

Inad

equ

ate

his

tory

bec

ause

pro

vid

er m

isu

nd

erst

and

s p

atie

nt

7.71

In

adeq

uat

e h

isto

ry b

ecau

se p

atie

nt

in a

hu

rry

1.49

Inad

equ

ate

his

tory

bec

ause

pat

ien

t ca

nn

ot

exp

ress

cle

arly

7.38

In

adeq

uat

e h

isto

ry b

ecau

se m

issi

ng

info

rmat

ion

fro

m f

amily

/car

egiv

er1.

48In

adeq

uat

e h

isto

ry b

ecau

se p

atie

nt

in a

hu

rry

7.23

Pro

vid

er-c

har

t

inte

ract

ion

Inco

mp

lete

/no

t u

pd

ated

ch

art

2.27

Inco

mp

lete

/no

t u

pd

ated

ch

art

22.2

9 Fa

ilure

to

up

dat

e ch

art

adeq

uat

ely

2.27

Mis

sin

g c

har

t21

.23

Illeg

ible

/un

clea

r ch

art

1.83

Med

icat

ion

list

no

t u

p t

o d

ate

20.3

2

Nu

rse-

pro

vid

er

in

tera

ctio

nM

isu

nd

erst

and

ing

bec

ause

pro

vid

er in

a h

urr

y2.

42M

isu

nd

erst

and

ing

bec

ause

han

dw

riti

ng

is u

ncl

ear

13.6

0 M

isu

nd

erst

and

ing

bec

ause

nu

rse

in a

hu

rry

2.35

Imp

ort

ant

info

rmat

ion

is n

ot

com

mu

nic

ated

11.5

6 M

isu

nd

erst

and

ing

bec

ause

han

dw

riti

ng

is u

ncl

ear

1.89

Mis

un

der

stan

din

g b

ecau

se p

rovi

der

in a

hu

rry

9.63

Pro

vid

er (p

lan

)P

lan

no

t co

vere

d b

y p

atie

nt ’

s in

sura

nce

4.20

Un

clea

r co

mm

un

icat

ion

of

pla

n: u

ncl

ear

han

dw

riti

ng

16.6

0 U

ncl

ear

com

mu

nic

atio

n o

f p

lan

: fre

qu

ency

of

do

se u

ncl

ear

1.83

Pro

vid

er is

in a

hu

rry

12.0

0 U

ncl

ear

com

mu

nic

atio

n o

f p

lan

: un

clea

r h

and

wri

tin

g1.

66P

rovi

der

is f

atig

ued

, str

esse

d, i

ll10

.05

Pro

vid

er-p

atie

nt

in

tera

ctio

n (p

lan

)In

adeq

uat

e co

un

selin

g a

bo

ut

dis

ease

an

d t

reat

men

t1.

38M

isu

nd

erst

and

ing

bec

ause

pro

vid

er in

a h

urr

y8.

76

Pat

ien

t d

oes

no

t p

arti

cip

ate

in d

ecis

ion

mak

ing

1.26

Mis

un

der

stan

din

g b

ecau

se p

atie

nt

in a

hu

rry

8.23

N

ot

invo

lvin

g p

atie

nt/

care

giv

er in

dec

isio

n m

akin

g0.

84M

isu

nd

erst

and

ing

bec

ause

jarg

on

use

d b

y p

rovi

der

3.01

Pat

ien

t (P

lan

)D

oes

no

t se

ek c

lari

fi ca

tio

n w

hen

nee

ded

5.67

Do

es n

ot

pro

vid

e ac

cura

te in

form

atio

n a

bo

ut

med

s ta

ken

16.7

7 D

oes

no

t p

rovi

de

accu

rate

info

rmat

ion

ab

ou

t m

ed p

rob

lem

s4.

90D

oes

no

t se

ek c

lari

fi ca

tio

n w

hen

nee

ded

14.7

8 D

oes

no

t p

rovi

de

accu

rate

info

rmat

ion

ab

ou

t m

eds

take

n3.

87D

oes

no

t p

rovi

de

accu

rate

info

rmat

ion

ab

ou

t m

ed p

rob

lem

s12

.67

* H

azar

d v

alu

e is

per

ceiv

ed f

req

uen

cy o

f er

ror

per

1,0

00 p

atie

nt

visi

ts m

ult

iplie

d b

y p

erce

ived

sev

erit

y o

f co

nse

qu

ence

s (0

.01

= m

inim

al, 0

.05

= m

ild, 0

.20

= m

od

erat

e, a

nd

1.0

= s

ever

e).

Val

ues

hav

e b

een

sca

led

to

a r

ang

e o

f .0

1 to

100

fo

r co

nco

rdan

ce a

nal

ysis

bet

wee

n p

ract

ices

.

. . . . . Quality of Care . . . . .

Singh, Servoss and Singh 177 Spring 2007

levels of concordance in the top 10 priorities were: providers versus nursing staff, 40%; providers versus administrative, 50%; and nursing versus administrative, 70%, giving a mean of 53%. For site 2, the concordances were lower (40%, 10%, and 40%, respectively) with a mean of 30%.

Further, hazard rankings were compared across the 2 practices for each of the subgroups using the same methodology as for the within-practice comparisons. Concordance between providers at site 1 and providers at site 2 was 40%, while between nursing staff at each site, it was 10% and between administrative staff, it was 10%, giving a mean value of 20%.

Discussion In this article, we have described an approach that

aims to help individual rural practices to answer the question: “ What are the events or situations that pose the greatest threat to the safety of our patients? ” Practices must fi nd answers to this question so that they can focus their limited resources on solving safety problems that are important rather than following the common and convenient practice of simply solving the problems that are the most visible or those that are most amenable to solution.

The fi eld test in the 2 study practices suggests that the approach is feasible. High response rates were achieved with minimal disruption of practice activities. The analysis required is straightforward and can easily be automated using a spreadsheet.

It is interesting to note that the hazard scores at site 2 were consistently higher than those at site 1. This indicates that staff at site 2 perceived greater frequency and/or severity of the errors in their practice than did those at site 1. This may be a true refl ection of higher error rates in the larger practice, which includes a residency training program, compared to the smaller community clinic, or perhaps it refl ects a heightened perception of hazard due to factors such as the organization ’ s culture.

Within each practice, there was a moderate level of agreement between subgroups of respondents regarding the top 10 sources of hazard. Convergence of opinions within a practice can be helpful since it shows that team members are “ on the same page ” and also lends some credence to the validity of the measurement. On the other hand, variations in opinion are to be expected since different subgroups and different individuals within those subgroups will each have their own perspectives. These similarities and differences of opinion should be embraced and explored by practice members to help in mutual understanding. At site 2, levels of concordance were

lower, suggesting a wider divergence of opinion than at site 1. This might again be due to the nature of it being a larger residency practice site, in which there might be less of a coherent team vision, or it may represent a failure of the proposed approach to capture the prevailing perceptions.

The level of concordance between sites was lower than that seen within the practices, lending support to the notion that each practice is unique.

The proposed approach, based on FMEA, may be viewed as complementary to the externally derived practice profi les and internally derived error reports that are currently in common use. It is based on the anonymously expressed perceptions of individual members of the practice team and can be seen as an attempt to capture the memory of the whole practice and foster teamwork. While this represents an advantage, overcoming some of the problems inherent in the aforementioned approaches, it is also an important weakness; because the hazard scores are derived from subjective judgments, they are subject to various biases that may affect the reliability and validity of the responses. Further work is clearly needed to address this issue.

The sites where the approach was fi eld-tested are larger than most rural practices and administratively atypical (one a hospital-affi liated clinic and one a residency practice site). It remains to be seen whether this kind of approach will prove practical in smaller freestanding offi ces. One concern that may arise is that of maintaining anonymity within a very small group of respondents. Further, although the analytical skills required to perform this analysis are rudimentary, this still may be a challenge in some small practices.

Replication of the methodology at multiple sites will help clarify its potential roles and refi ne the procedures. Finally, since the purpose of the process is to help practices to prioritize quality and safety problems, it is imperative that we evaluate practice members ’ ability to use the priorities generated to focus their efforts and to develop and implement interventions that improve safety.

Conclusion The modifi ed FMEA approach shows promise as a

technique for identifying the most serious threats to patient safety in rural primary care practices.

References 1. Kohn LT , Corrigan J , Donaldson MS , eds . To Err Is Human:

Building a Safer Health System . Washington, DC : National Academy Press ; 2000 .

. . . . . Quality of Care . . . . .

The Journal of Rural Health 178 Vol. 23, No. 2

2. Institute of Medicine . Crossing the Quality Chasm: A New Health

System for the 21st Century . Washington, DC : National Academy of Sciences ; 2001 .

3. Institute of Medicine Board on Health Care Services . Quality

Through Collaboration: The Future of Rural Health Care . Washington, DC : National Academies Press ; 2005 .

4. McDaniel RR Jr . Strategic leadership: a view from quantum and chaos theories . Health Care Manage Rev . 1997 ; 22 : 21 - 37 .

5. Miller WL , Crabtree BF , McDaniel R , Stange KC . Understanding change in primary care practice using complexity theory . J Fam

Pract . 1998 ; 46 : 369 - 376 . 6. Plsek P . Redesigning health care with insights from the science of

complex adaptive systems . In : Committee on Quality of Health Care

in America, Institute of Medicine, ed. Crossing the Quality Chasm: A

New Health System for the 21st Century . Washington, DC : National Academy Press ; 2001 ; Appendix B : 309 - 322 .

7. Rosenstein AH . Hospital report cards: friend or foe? J Clin

Outcomes Manage . 2004 ; 11 : 98 - 102 . 8. Hibbard JH , Stockard J , Tusler M . Does publicizing hospital

performance stimulate quality improvement efforts? Health Aff . 2003 ; 22 : 84 - 94 .

9. Hofer TP , Hayward RA . Are bad outcomes from questionable clinical decisions preventable medical errors? A case of cascade iatrogenesis . Ann Intern Med . 2002 ; 137 : 327 - 333 .

10. JCAHO . Comprehensive Accreditation Manual for Hospitals: The

Offi cial Handbook, Update 1 ; Oak Brook, IL: Joint Commission Resources ; 2001 .

11. Marder R , Sheff RA . The Step-by-Step Guide to Failure Modes and

Effects Analysis . Marblehead, MA: Opus Communications, Inc .; 2002 . 12. Singh R , Singh A , Fox CH , Taylor JS , Rosenthal TC , Singh G .

Computer visualisation of patient safety in primary care: a systems approach adapted from management science and engineering . Inf Prim Care . 2005 ; 13 : 135 - 144 .

13. Dovey SM , Meyers DS , Phillips RL Jr , et al . A preliminary taxonomy of medical errors in family practice . Qual Saf Health

Care . 2002 ; 11 : 233 - 238 . 14. Makeham MAB , Dovey SM , County M , Kidd MR . In

international taxonomy of errors in general practice: a pilot study . Med J Aust . 2002 ; 177 : 68 - 72 .

15. Sheikh A , Hurwitz B . Setting up a database of medical error in general practice: conceptual and methodological considerations . Br J Gen Prac . 2001 ; 51 : 57 - 60 .

16. Bhasale A . The wrong diagnosis: identifying causes of potentially adverse events in general practice using incident monitoring . Fam Pract . 1998 ; 15 : 308 - 318 .

17. Britt H , Miller GC , Steven ID . Collecting data on potentially harmful events: a method of monitoring incidents in general practice . Fam Pract . 1997 ; 14 : 101 - 106 .

18. Fischer G , Fetters MD , Munro AP , Goldman ED . Adverse events in primary care identifi ed from a risk-management database . J Fam Pract . 1997 ; 45 : 40 - 46 .

19. Ely J , Levinson W , Elder NC , Mainous A , Vinson D . Perceived causes of family physicians errors . J Fam Pract . 1995 ; 40 : 337 - 344 .

20. von Neumann J , Morgenstern O . Theory of Games and Economic

Behavior . 1953 ed . Princeton, NJ : Princeton University Press ; 1944 .