Report on the Web-Based Modified Delphi Survey of the ICPS · stage, web-based modified Delphi Survey (Delphi Survey) to test the relevance and acceptability of the ICPS. This report

Project to Develop the International Classification for Patient Safety

Report on the Results of the Web-Based

Modified Delphi Survey of the International Classification for Patient Safety

8 June 2007

Prepared by: Drafting Group of the ICPS

Page 2 of 194

Table of Contents Page Foreword ........................................................................................................................................................... 3

Introduction ....................................................................................................................................................... 4

Key concepts, Definitions and Preferred Terms................................................................................................ 7

The Delphi Process and the Resulting Revisions ............................................................................................ 14

Responses to the Delphi Comments ................................................................................................................ 23

Event Types .............................................................................................................................................. 23

Patient Impact/Outcomes .......................................................................................................................... 37

Patient Characteristics............................................................................................................................... 48

Event Characteristics ................................................................................................................................ 57

Contributing Factors ................................................................................................................................. 64

Actions Taken ........................................................................................................................................... 75

Before Event (Preventive) Factors............................................................................................................ 83

During Event (Recovery) Factors ............................................................................................................. 91

After Event (Mitigating) Factors............................................................................................................. 107

Organizational Outcomes........................................................................................................................ 116

Appendix A – Round One Survey................................................................................................................. 125

Appendix B – Round Two Survey ................................................................................................................ 148

Page 3 of 194

Foreword Between August and November 2006, the Drafting Group set up by the World Health Organization’s World Alliance for Patient Safety to develop an International Classification for Patient Safety (ICPS) carried out a two-stage, web-based modified Delphi Survey (Delphi Survey) to test the relevance and acceptability of the ICPS. This report provides some background information, definitions of key concepts, a description of the iterative process of the Delphi Survey, and the results of the analysis. It is divided into five sections:

• Introduction: An overview of the background and rationale for the ICPS • Key concepts, Definitions and Preferred Terms: A narrative describing the relationships between key

concepts linking preferred terms and definitions, with diagrams showing these links and the conceptual framework underpinning the ICPS

• The Delphi Process: A description of how respondents were recruited, how their responses were collated

with results of the two surveys and a description of the major revisions made to the ICPS conceptual framework and classes, with the rationale for these revisions

• Responses to Delphi Comments: The responses to each comment offered by Delphi participants in both

the first and second rounds of the Delphi Survey The Drafting Group sincerely thanks those who dedicated their valuable time to respond to the Delphi Survey. As shown in this report, the input had a significant impact on the development of the ICPS.

Page 4 of 194

Introduction Although the results of the first large-scale study of adverse events were published over thirty years ago1, the field of patient safety has only gained widespread attention in the last decade.2,3,4 In this time, there has been a rapid increase in the number of publications and reports in this area, but interpretation and comparison have been compromised by a lack of common understanding and language. A need was thus identified for a comprehensive classification, populated by concepts with agreed definitions5,6 which should be described by “preferred terms” from the major languages of the world. The consistent use of such terms and concepts in conjunction with a comprehensive but adaptable classification will pave the way for researchers to understand each others’ work, and will facilitate the systematic collection, aggregation and analysis of relevant information from all available sources, allowing comparisons between facilities and jurisdictions, and over time. The classification should be applicable across the full spectrum of healthcare from primary care to highly specialized areas and should be able to be used in conjunction with existing processes and systems.6 An opportunity to address this need was presented by the launch of the World Alliance for Patient Safety of the World Health Organization (WHO).7 A group was formed under the auspices of the World Alliance and some general principles agreed.8 It was decided that a classification (an arrangement of concepts based on similarities) instead of a taxonomy (a set of rules to name entities based upon their location within a particular structure) was needed. It was decided to use concepts and terms with meanings as close as possible to those in colloquial use and to avoid long definitions with several “qualifiers”, but instead to start with simple, basic definitions, and then “build” by defining the key terms used in these definitions. This makes it necessary to read the terms and their definitions in the sequence provided in the next section. The first requirement was to develop and agree on an underlying conceptual framework. After reviewing the relevant literature and examining some existing classifications6,9,10,11,12,13, an initial version of a framework was developed, with definitions of a few basic concepts. This framework and its accompanying concepts were subjected to a two-stage web-based Delphi review which yielded some 700 responses. These responses led to further discussion amongst the drafting group, and an iterative process was undertaken by which the framework, concepts and definitions were progressively refined and improved. This process and how it influenced and led to revisions of the initial framework, classes, concepts and definitions, together with the responses to the Delphi respondents, form the last two sections of this document.

1 Mills DH, ed. Medical insurance feasibility study. A technical summary. West J Med 1978;128:360-5. 2 Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington DC: National

Academies Press, 2000. 3 Department of Health. An Organisation with a Memory - Report of an Expert Group on Learning from Adverse Events in the NHS

Chaired by the Chief Medical Officer. London: The Stationery Office, 2000. 4 Runciman WB, Moller J. Iatrogenic Injury in Australia. Adelaide: Australian Patient Safety Foundation, 2001. Available at

<http://www.apsf.net.au> 5 World Alliance for Patient Safety. International Classification for Patient Safety. http://www.who-ic4ps.org/ 6 Runciman WB, Williamson JA, Deakin A, et al. An integrated framework for safety, quality and risk management: an information

and incident management system based on a universal patient safety classification. Qual Saf Health Care 2006;15 Suppl 1:i82-90. 7 World Health Organization. World Alliance for Patient Safety: Forward Programme 2005. Geneva: World Health Organization,

2004. Available at: www.who.int/patientsafety/en/brochure_final.pdf 8 Loeb JM, Chang A; Joint Commission on Accreditation of Healthcare Organizations (JCAHO). Towards An International Patient

Safety Taxonomy, 1) A Review of the Literature on Existing Classification Schemes For Adverse Events and Near Misses 2) A Draft Framework to Analyze Patient Safety Classifications. Assignment Report 1&2, 30 June 2003. Geneva: World Health Organization, 2003. Available at: http://www.who.int/patientsafety/taxonomy/JCAHOReport12-30June03.pdf

9 Runciman WB, Helps SC, Sexton EJ, et al. A classification for incidents and accidents in the health-care system. J Qual Clin Pract 1998;18:199-211.

10 National Patient Safety Agency, United Kingdom. National Reporting and Learning System. http://www.npsa.nhs.uk/health/display?contentId=2389

11 Kaplan HS, Callum JL, Rabin Fastman B, Merkley LL. The Medical Event Reporting System for Transfusion Medicine (MERS-TM): Will it help us get the right blood to the right patient? Transfus Med Rev 2002;16:86-102.

12 van der Schaaf TW. PRISMA-Medical. A Brief Description. Eindhoven: Eindhoven University of Technology, Faculty of Technology Management, Patient Safety Systems, 2005. Available at: http://www.who-ic4ps.org/resources/PRISMA-Medical.pdf

13 Chang A, Schyve PM, Croteau RJ, et al. The JCAHO patient safety event taxonomy: a standardized terminology and classification schema for near misses and adverse events. Int J Qual Health Care 2005;17:95-105.

Page 5 of 194

The iterative process was rendered difficult because, at the start, there was no agreement on many commonly used concepts, terms and definitions. Therefore, initial definitions of key terms were used from a previous iterative process, which had taken into consideration many definitions from over 60 sources of information. For example, 16 definitions for “error” and 14 for “adverse event” had been identified and considered before arriving at agreed definitions for these concepts14,15. It was decided to recommend that certain terms not be used in the context of the ICPS, because they have different meanings in different jurisdictions (for example, negligence and liability), because they have discipline-specific meanings (such as accident (aircraft hull loss) in aviation), because they have a range of meanings (complication, misadventure, sequela) or because they overlap with other terms we have decided to use (such as nosocomial and iatrogenic which have been replaced by “healthcare-associated”). The trail of development of the definitions has therefore been complex, as they are adaptations from several sources and from previous Delphi and other consultative processes. The main consideration in coming up with the definitions was that they should convey the appropriate meaning and be brief and clear, without unnecessary or redundant qualifiers. They should also, whenever reasonable, be consistent with concepts from other terminologies and classifications in the Family of International Classifications of the WHO. The resulting conceptual framework for the ICPS is shown in Figure 1. How some key terms and concepts relate to the ICPS is shown in Figure 2. In the next section a narrative, in a logical sequence, incorporates the definitions of the key concepts with the preferred terms and some descriptions and comments. The definitions are listed in Table I in the sequence in which they are discussed in the narrative, and the terms defined are listed in alphabetical order in Table 2.

14 Runciman WB. Shared meanings: preferred terms and definitions for safety and quality concepts. Med J Aust 2006;184(10

Suppl):S41-3. 15 World Health Organization, World Alliance for Patient Safety (2006, March) Comparison Glossary of Patient Safety Terms.

Geneva, Switzerland: H. Sherman, G. Castro, J. Loeb

Page 6 of 194

Figure 1. Conceptual Framework for the International Classification for Patient Safety

Contributing Factors/Hazards

Patient Characteristics

Ameliorating Actions

System Resilience (Proactive & Reactive Risk Assessment)

Clinically meaningful, recognizable categories for incident identification & retrieval

Descriptive information

Conceptual Framework for the International Classification for Patient Safety

April 07

Organizational Outcomes

Detection

Mitigating Factors

Actions Taken to R

educe RiskAct

ions

Tak

en to

Red

uce

Ris

k

Incident Characteristics

Patient Outcomes

IncidentIncident Type

Influences Informs

Influences

Influences

Informs

Informs

Informs Informs

Influences Informs

Legend: The solid lines enclose the 10 major classes of the ICPS and represent the semantic relationships between them. The dotted lines represent the flow of information.

Page 7 of 194

Key Concepts, Definitions and Preferred Terms As foreshadowed in the Introduction, in order to keep the definitions as succinct as possible, concepts are progressively introduced so as to allow understanding to be “built”, starting with the terms in the title of the ICPS (classification, patient, safety). The terms in bold have been deemed ICPS preferred terms; where terms have been highlighted in this way, the agreed definitions follow. A classification is an arrangement of concepts (bearers or embodiments of meaning) into classes (groups or sets of like things, such as “Contributing Factors”, “Incident Types” and “Patient Outcomes”) and their subdivisions to express the semantic relationships between them (the way in which they are associated with each other on the basis of their meanings). For example, Contributing Factors precede and play a role in the generation of any particular Incident Type. Similarly, Mitigating Factors are associated with Incident Type and Outcomes, as steps cannot be taken to interfere with the progression of an incident until its nature has been determined, and the outcomes will not occur until these attempts have reached their conclusion. The ICPS allows assignment of features of incidents on the basis of common characteristics; this facilitates their later extraction for analysis. Each class has hierarchically arranged subdivisions populated by concepts (for example, “fatigue/exhaustion” under the class “Contributing Factors”). Concepts impart the essence of a notion using a term. Concepts may be represented by a number of terms that allow for regional dialects, different languages, clinicians, disciplines and hospital preferences. Preferred terms to describe the concepts have been chosen as “natural categories” (see below), or terms with meanings as close as possible to those in colloquial use.9 Concepts may inherit characteristics from their “parents” (a parent-child or subsumption relationship), or represent selected agreed qualities, properties or features of the concept in question which are not inherited (an “attribute” relationship). For example, “endotracheal tube” and “tracheotomy tube” are children of “artificial airway”, whereas the list of medical devices has an attribute-type relationship to the incident type “medical devices/equipment/property”, and is not one of its children. Concepts have been organized hierarchically in the classification according to natural mapping. Natural mapping is the grouping of the subsets or attributes of a class or concept in a representation reflecting the real world.16

A patient has been defined as a person who is a recipient of healthcare and healthcare has been defined as services received by individuals or communities to promote, maintain, monitor or restore health. For the purposes of the ICPS, patients are referred to rather than clients, tenants or consumers, although it is recognized that a healthy pregnant woman, a child undergoing immunization, the occupant of a halfway house or an adolescent seeking counseling may not be regarded, and may not regard themselves, as patients. Healthcare is not limited to medical care provided by others, and includes self-care. Health has been defined “as a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity”, consistent with the World Health Organization definition. Safety has been defined as freedom from hazard, and hazard as a circumstance, agent or action which can lead to or increase risk. A circumstance has been defined as any factor connected with or influencing an event, agent or person(s); an event as something that happens to or involves a patient; and an agent as a substance, object or system which acts to produce change. For the purposes of the ICPS it is implicit that all these terms (and others, such as quality and outcome) are meant to be interpreted in the context of patient safety. Patient safety is defined as freedom for a patient from unnecessary harm or potential harm associated with healthcare. Healthcare-associated harm is harm arising from or associated with plans or actions taken during the provision of health care rather than an underlying disease or injury.

16 Norman DA. The Psychology of Everyday Things. New York: Basic Books, 1998.

Page 8 of 194

A patient safety incident is an event or circumstance which could have resulted, or did result, in unnecessary harm to a patient, and has a more constrained meaning than the term incident which, when used in a general context, has a wider meaning as an event or circumstance which could have resulted, or did result, in harm to any person and/or a complaint, loss or damage. Please note for purposes of the ICPS, a patient safety incident will be hereafter referred to as an incident. The use of the term “unnecessary” in the definition of patient safety incident is in recognition that errors, violations, patient abuse and deliberately unsafe acts occur in healthcare and are unnecessary incidents, whereas certain forms of harm, such as an incision for a laparotomy, are necessary. The former are incidents, whereas the latter would not be regarded as one. Incidents may arise from both unintended and intended acts. Errors are, by definition, unintentional, whereas violations are intentional, even though they may become routine in certain contexts. An error may be defined as a failure to carry out a planned action as intended or application of an incorrect plan, and may manifest by doing the wrong thing (an error of commission) or by failing to do the right thing (an error of omission), at either the planning or execution phase. Thus, if it is agreed that screening for bowel cancer should be by regular testing for occult blood, then a screening colonoscopy in the absence of prior occult blood testing comprises an error of commission (over servicing), and a failure to arrange testing for occult blood an error of omission (under servicing). A violation implies deliberate deviation from an operating procedure, standard or rule. Both errors and violations increase risk, even if an incident does not actually occur. Risk is the probability that an incident will occur. An adverse event is an incident which results in harm to a patient. Harm implies impairment of structure or function of the body and/or any deleterious effect arising therefrom. Harm includes disease, injury, suffering, disability and death and may thus be physical, social or psychological. Disease is defined as a physiological or psychological dysfunction, injury as damage to tissues caused by an agent or circumstance and suffering as the experience of anything subjectively unpleasant. Suffering includes pain, malaise, nausea, vomiting, depression, agitation, alarm, fear and grief. Disability implies any type of impairment of body structure or function, activity limitation and/or restriction of participation in society, associated with past or present harm. A near miss is an incident that did not cause harm (also known as a close call). A contributing factor is defined as a circumstance, action or influence (such as poor rostering or task allocation) which is thought to have played a part in the origin or development of an incident, or to increase the risk of an incident. Contributing factors may be external (ie not under the control of a facility or organization), organizational (such as unavailability of accepted protocols), be related to a staff factor (a cognitive or behavioral defect in an individual, a lack of supervision, poor team work or inadequate communication) or be related to a patient factor (such as behavior). Incidents are classified into a number of different incident types. An incident type is a descriptive term for a category made up of incidents of a common nature, grouped because of shared agreed features. An incident type is a “parent” natural category under which many concepts (also comprising natural categories) may be grouped; they may be “children” of the parent incident type or attributes (see above under the description of classes, concepts and natural mapping). A natural category is a descriptor (usually a short phrase) which is brief, easily and commonly understood. It captures the essence of an event or circumstance or of associated characteristics or attributes and is not constrained by being restricted to any class or property. To take an example from the workplace – 40 of 100 incident reports describing why staff are late for work might be assigned to the following natural categories: could not find car key; child sick – no other carer; had a puncture or flat tyre; alarm clock did not go off; or usual car park full. These phrases capture the essence of each incident and most observers would categorize them in the same way (e.g., a higher rate of inter-rater reliability). Natural categories constitute an informal classification system, used by a specific professional or cultural group. They reflect a social consensus about what matters, or what is worthy of notice, in a given context. In this case, the context is patient safety.

Page 9 of 194

Incident types are not exclusive categories. An incident may need to be assigned to several incident types, although a “business rule” may be developed to allow a “principal” incident type to be nominated so that all incidents may be counted. One rule is to nominate the incident which led most directly to any harm or potential harm as the principal incident type. Thus, for example, for an incident in which an infusion pump was set up wrongly and delivered an overdose of a sedative, causing respiratory arrest, the drug overdose (the medication incident type) would be selected as the principal incident type rather than the equipment problem. Other examples of incident types are healthcare associated infection, documentation problems and problems with a clinical administration process. Patient characteristics are selected attributes of a patient, such as patient demographics or the reason for presentation to a healthcare service. Attributes are qualities, properties or features of someone or something. Incident characteristics are defined as selected attributes of an incident, such as care setting, hospital treatment status, specialties involved and timing or date of the incident. Terms commonly used in relation to medication incidents include adverse reactions and side effects. An adverse reaction is defined as unexpected harm arising from a justified action where the correct process was followed for the context in which the event occurred. Recurrence of a known adverse reaction may be preventable (such as an allergic reaction to a drug, by avoiding re-exposure). A side effect is a known effect, other than that primarily intended, relating to the pharmacological properties of a medication. An example of an adverse reaction would be unexpectedly getting neutropenia when that particular drug is not known to have this effect. An example of a side effect would be when nausea, pruritis or urinary retention are encountered when morphine has been given to alleviate pain. It is useful to try to identify when an incident is preventable. Preventable has been defined as being accepted by the community as avoidable in the particular set of circumstances. Detection is defined as an action or circumstance that results in the discovery of an incident. This may be by noticing an error via a monitor or alarm, by a change in patient condition or by an audit, review or risk assessment. Detection mechanisms may be part of the system (such as low pressure disconnect alarms in breathing circuits) or may result from a checking process or from vigilance and “situation awareness”. A mitigating factor is defined as an action or circumstance which prevents or moderates the progression of an incident towards harming a patient. The damage mechanism has already started, but has not yet led to the maximum possible harm. The term “error recovery” has been used to describe the combined detection and mitigation sequence. In this context recovery does not refer to clinical recovery (recuperation) but to the process of recovering from an incident that has started. An example of error recovery would be reconnecting a breathing circuit after a disconnect alarm had given warning that there was a disconnection. By collecting information about how and why “saves” are made, system design, training and education can be informed. Patient outcome is defined as the impact upon a patient which is wholly or partially attributable to an incident. Where harm has occurred, the degree of harm is defined as the severity and duration of any harm, and the treatment implications, that result from the incident. It would seem, from first principles, desirable to record the nature, severity and duration of harm separately. However, in practice, there are substantial problems in doing this and nearly all attempts to link degree of harm conflate these parameters. Organizational outcome is defined as the impact upon an organization which is wholly or partially attributable to an incident. Examples would be adverse publicity and additional use of resources. An ameliorating action is an action taken or circumstance altered to make better or compensate any harm after an incident. Patient ameliorating factors are actions taken or circumstances altered to make good any harm to a patient, such as fixing a fracture after a fall, whereas healthcare system ameliorating factors reduce any loss or damage to an organization arising from an incident. For example, good public relations management after a well publicized disaster will ameliorate any long-term effects on the reputation of a facility.

Page 10 of 194

Actions taken to reduce risk are defined as actions taken to reduce, manage or control the harm, or probability of harm, associated with an incident. An action can relate directly to incidents and contributing factors, detection, mitigating factors or ameliorating actions and can be pro-active or reactive. Pro-active actions may be identified by techniques such as failure mode and effects analysis and probabilistic risk analysis, whereas reactive actions are those taken in response to insights gained after an incident (see root cause analysis, below, for an example). Resilience refers to the degree to which a system continuously prevents, detects, mitigates or ameliorates hazards or incidents. Resilience allows an organization to “bounce back” to its original ability to provide core functions as soon as possible after incurring damage. A number of terms are commonly used with respect to organizational management. Accountable is defined as being held responsible. The terms negligence and liability are listed below, but are not considered core definitions as these may vary depending on the jurisdiction. Quality is defined as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. System failure refers to a fault, breakdown or dysfunction within an organization’s operational methods, processes or infrastructure. Factors contributing to system failure can be latent (hidden or apt to elude notice) or apparent, and can be related to the system, the organization or a patient. An example of a latent factor would be a breathing circuit disconnect alarm with no power failure warning or battery backup. System improvement is defined as the result or outcome of the culture, processes and structures that are directed towards the prevention of system failure and the improvement of safety and quality. A process to counter the latent failure just described would be to modify the equipment to alarm when the power supply is compromised, or to always use an additional device such as expired air capnography set up so as to alarm if carbon dioxide is not detected. Finally, root cause analysis is defined as a systematic iterative process whereby the factors which contribute to an incident are identified by reconstructing the sequence of events and repeatedly asking “why?” until the underlying root causes (contributing factors/hazards) have been elucidated. “Why” should be iteratively asked until the investigating team runs out of facts – they should not guess or speculate. The team should also stop the process when the identified contributing factors or hazards require counter measures which are beyond the influence of the organization. These are known as “stopping rules”, and help to determine when a root cause analysis team should stop the investigative process and move on to defining the problems and recommending corrective strategies. Concepts defined and terms chosen so far simply represent a collection of basic building blocks to enhance the study of patient safety. Changes will be necessary as our understanding widens and needs are identified. Translation into other major languages will be necessary and has been started. There are more concepts which will need to be defined, and decisions made with respect to which terms should be preferred and which terms avoided. Certain qualifiers should be regarded as implicit when these terms are used in the context of the ICPS. For example, it should be assumed that the term “incident” refers to a patient safety incident, implying harm or potential harm associated with harm. The same applies to terms such as quality and system failure. Used in this way the concepts defined and terms chosen so far will facilitate understanding and transfer of information relevant to patient safety amongst the increasing number of people with an interest in this area.

Page 11 of 194

Figure 2. The Conceptual Framework of the ICPS Showing Links to Preferred Terms

Contributing Factors/Hazards (26)

Ameliorating Actions (39)

System Resilience (Proactive & Reactive Risk Assessment)

Clinically meaningful, recognizable categories for incident identification & retrieval

Descriptive information

Conceptual Framework for the International Classification for Patient Safety

April 07

Organizational Outcomes (38)

Detection (39)

Mitigating Factors (35)

Actions Taken to R

educe Risk (40)Ac

tions

Tak

en to

Red

uce

Ris

k (4

0)

Incident Characteristics

(30)

Patient Outcomes

(36)

Patient Safety

Incident (15)

Incident Type (27)

Classification (1), Concept (2)Class (3), Semantic Relationship (4)

Patient (5), Healthcare (6)Health (7)

Safety (8)Patient Safety (13)

Error RecoveryHazard (9), Circumstance (10)Event (11), Agent (12)Error (16), Violation (17), Risk (18)

Healthcare Associated Harm (14)Adverse Event (19), Near Miss (25) Adverse Reaction (31), Side Effect (32) Preventable (33), System Failure (44)

Patient Characteristics

(28)

Attributes (29)

Resilience (41)System Improvement (45)Root Cause Analysis (46)

Error Recovery

Harm (20)Disease (21)Injury (22)Suffering (23)Disability (24)Degree of Harm (37)

Accountable (42)Quality (43)

Resilience (41)System Improvement (45)Root Cause Analysis (46)

Attributes (29)

Legend: The solid lines enclose the 10 major classes of the ICPS and represent the semantic relationships between them. The dotted lines link relevant preferred terms which have been defined in the text. The numbers represent the sequence in which they appear in the text and in Tables 1 and 2.

Page 12 of 194

Table 1. List of Preferred Terms and Definitions for Key Concepts 1. Classification: an arrangement of concepts into classes and their subdivisions to express the semantic

relationships between them. 2. Concept: a bearer or embodiment of meaning. 3. Class: a group or set of like things. 4. Semantic relationship: the way in which things (such as classes or concepts) are associated with each

other on the basis of their meaning. 5. Patient: a person who is a recipient of healthcare. 6. Healthcare: services received by individuals or communities to promote, maintain, monitor or restore

health. 7. Health: a state of complete physical, mental and social wellbeing and not merely the absence of disease

or infirmity. 8. Safety: freedom from hazard. 9. Hazard: a circumstance, agent or action that can lead to or increase risk. 10. Circumstance: any factor connected with or influencing an event, agent or person(s). 11. Event: something that happens to or involves a patient. 12. Agent: a substance, object or system which acts to produce change. 13. Patient Safety: freedom, for a patient, from unnecessary harm or potential harm associated with

healthcare. 14. Healthcare-associated harm: harm arising from or associated with plans or actions taken during the

provision of healthcare rather than an underlying disease or injury. 15. Patient safety incident: an event or circumstance which could have resulted, or did result, in

unnecessary harm to a patient. 16. Error: failure to carry out a planned action as intended or application of an incorrect plan. 17. Violation: deliberate deviation from an operating procedure, standard or rules. 18. Risk: the probability that an incident will occur. 19. Adverse event: an incident which results in harm to a patient. 20. Harm: impairment of structure or function of the body and/or any deleterious effect arising there from. 21. Disease: a physiological or psychological dysfunction. 22. Injury: damage to tissues caused by an agent or circumstance. 23. Suffering: the experience of anything subjectively unpleasant. 24. Disability: any type of impairment of body structure or function, activity limitation and/or restriction of

participation in society, associated with past or present harm. 25. Near Miss: an incident that did not cause harm. 26. Contributing Factor: a circumstance, action or influence which is thought to have played a part in the

origin or development of an incident or to increase the risk of an incident. 27. Incident type: a descriptive term for a category made up of incidents of a common nature grouped

because of shared, agreed features. 28. Patient characteristics: selected attributes of a patient. 29. Attributes: qualities, properties or features of someone or something. 30. Incident characteristics: selected attributes of an incident. 31. Adverse reaction: unexpected harm resulting from a justified action where the correct process was

followed for the context in which the event occurred. 32. Side effect: a known effect, other than that primarily intended, related to the pharmacological properties

of a medication. 33. Preventable: accepted by the community as avoidable in the particular set of circumstances. 34. Detection: an action or circumstance that results in the discovery of an incident. 35. Mitigating factor: an action or circumstance which prevents or moderates the progression of an

incident towards harming a patient. 36. Patient outcome: the impact upon a patient which is wholly or partially attributable to an incident. 37. Degree of harm: the severity and duration of harm, and the treatment implications, that result from an

incident.

Page 13 of 194

38. Organizational Outcome: the impact upon an organization which is wholly or partially attributable to an incident.

39. Ameliorating action: an action taken or circumstances altered to make better or compensate any harm after an incident.

40. Actions taken to reduce risk: actions taken to reduce, manage or control the harm, or probability of harm associated with an incident.

41. Resilience: The degree to which a system continuously prevents, detects, mitigates or ameliorates hazards or incidents.

42. Accountable: being held responsible 43. Quality: the degree to which health services for individuals and populations increase the likelihood of

desired health outcomes and are consistent with current professional knowledge. 44. System failure: a fault, breakdown or dysfunction within an organization’s operational methods,

processes or infrastructure. 45. System improvement: the result or outcome of the culture, processes, and structures that are directed

toward the prevention of system failure and the improvement of safety and quality. 46. Root cause analysis: a systematic iterative process whereby the factors which contribute to an incident

are identified by reconstructing the sequence of events and repeatedly asking why? until the underlying root causes have been elucidated.

Table 2. List of Preferred Terms for Key Concepts in Alphabetical Order (the numbers refer to the sequence in which these preferred terms appear in the text)

Accountable (42) Incident characteristics (30) Actions taken (40) Incident type (27) Adverse event (19) Injury (22) Adverse reaction (31) Mitigating factor (35) Agent (12) Near miss (25) Ameliorating action (39) Organizational outcome (38) Attributes (29) Patient (5) Circumstance (10) Patient characteristics (28) Class (3) Patient outcome (36) Classification (1) Patient safety (13) Concept (2) Patient safety incidents (15) Contributing factor (26) Preventable (33) Degree of harm (37) Quality (43) Detection (34) Resilience (41) Disability (24) Risk (18) Disease (21) Root cause analysis (46) Error (16) Safety (8) Event (11) Semantic relationship (4) Harm (20) Side effect (32) Hazard (9) Suffering (23) Health (7) System failure (44) Healthcare (6) System improvement (45)

Page 14 of 194

The Delphi Process and the Resulting Revisions The initial conceptual framework, developed to represent an underlying information model using classes and concepts, was designed to accommodate patient safety data and information obtained from a variety of sources (e.g. reporting systems, incident reports, root cause analyses, medical record reviews, complaints, consumer reporting, sentinel events, coroner’s reports, medico-legal cases). It consisted of the following ten high level classes (shown in the boxes in Figure 3).

Figure 3.

* Includes active and latent factors

Influences

Event types

Patient characteristicsRecovery

factors

Contributing factors*

Event characteristics

Actions taken

Mitigating factors

Lead to

Patient impact/outcomes

Lead to

Organization Impact/Outcomes

Event modifiers

Preventive factors (+)

Influences

Influences

Framework class

Core framework class

Have

The original conceptual framework and its accompanying concepts were subjected to a two-stage, web-based modified Delphi survey.17 Experts in the fields of patient safety, health policy legislation, reporting systems, safety and quality control, classification theory and development, health informatics, consumer advocacy, law and medicine were invited to participate in the Delphi Survey. Each received a personalized invitation to participate in the Delphi process which contained a unique link to the electronic surveys. Input was also sought from a wider stakeholder constituency. Open invitations to take part in the Delphi process were placed in an article published in the International Journal for Quality in Health Care18 and on the websites of the World Health Organization’s World Alliance for Patient Safety, the Australian Patient Safety Foundation, The Joint Commission and the National Patient Safety Agency. Delphi participants were asked to complete two rounds of surveys. Only those who provided input during the first round were asked to participate in the second round. Respondents were assured confidentiality but not anonymity.

17 Brown, B. Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts. Sept. 1968. 18 Donaldson, LJ. In terms of safety. Int. J. Qual. Health Care. 2006;18:325-326

Page 15 of 194

Round One Survey (see Appendix A) The first round of the web-based Delphi survey was from 18 August to 22 September 2006. The Round One survey was comprised of the following sections: 1. Participant demographics – In addition to age, gender, country and native language, questions include

whether the respondent was a health car professional, what expertise he/she had based on the criteria listed above, and whether he/she had practical experience in collecting, classifying and/or analysing patient safety data.

2. The conceptual framework – Questions included:

a. Is the conceptual framework an adequate model for use in describing a patient safety event? Possible responses were “yes”, “yes with modification”, “no” or “unsure”. Respondents who answered “yes with modification” or “no”, were asked to suggest improvements and provide a rationale for the suggested changes.

b. Do you believe any classes are missing from the conceptual framework? Respondents who

answered “yes” were asked to indicate which class(es) were missing, where the class(es) should be incorporated and to provide a rationale for adding the suggested class(es).

c. Do you believe the conceptual framework is a meaningful and useful tool for translating disparate

information into a format conducive to learning and improving patient safety? Possible responses were “strongly agree”, “agree”, “neither agree nor disagree”, “disagree” or “strongly disagree”. Respondents who answered “neither agree nor disagree”, “disagree” or “strongly disagree” were asked to explain why he/she chose the particular answer.

3. Definitions – Respondents were asked to review the terms and definitions for each of the ten classes

contained within the conceptual framework for clarity, precision and accuracy. Respondents who believed a term or definition was unclear, were asked to provide an explanation as to why the term or definition was unclear and to provide an alternative term or definition, the relevant reference(s) to the source(s) and a rationale for replacing the existing term or definition with the one provided.

4. Meaningfulness and usefulness of each of the ten classes within the conceptual framework – Respondents

were asked to react to the following statement for each of the ten classes: The class, [class term (ie Event Types)], is a meaningful and useful class within the International Patient Safety Event Classification. Possible responses were “strongly agree”, “agree”, “neither agree nor disagree”, “disagree” or “strongly disagree”. Respondents who answered “neither agree nor disagree”, “disagree” or “strongly disagree” were asked to explain why he/she chose the particular answer.

5. Overall comments were sought. Round One Survey Results A total of 253 individuals speaking 29 different languages from 43 countries responded to the first round of the Delphi survey. One hundred sixty one (161) responded to the personalized invitation letter sent by the Drafting Group; the remaining 92 were responses to the open invitation. The respondents included health care professionals, health policy experts, developers/manager of patient reporting systems, patient/public representatives, academicians, representatives from professional associations for a variety of health care specialties, representatives from litigation, classification/taxonomy experts, risk managers and representatives from organizations responsible for assessing and monitoring patient safety performance. Eighty one percent (81%) had practical experience in collecting, classifying and/or analysing patient safety data.

Page 16 of 194

Of those who responded:

- 86% believed the conceptual framework was an adequate model for use in describing a patient safety event;

- 81% believed the conceptual framework was a meaningful and useful tool for translating disparate

information into a format conducive to learning and improving patient safety; - 50% responded that the conceptual framework was missing at least one class. However, upon reading

their comments, it became apparent that most of the respondents were referring to concepts they felt should be included within the classes instead of classes that should be included within the conceptual framework.

- 82% felt that the terms and definitions for each of the 10 classes contained within the conceptual

framework were clear, precise and provided an accurate representation of the nature, properties, scope and essential qualities of the concept; and

- 89% felt that all 10 classes within the conceptual framework were meaningful and useful.

All comments were reviewed. The main idea elicited from the comments was the need for clarification, particularly surrounding the:

- Purpose of the classification; - Structure and depth of the conceptual framework; - Intention of the classification to include both adverse events and near misses; - Ability of the conceptual framework to serve as a model to classify a patient safety event; and - Concepts contained within each of the classes.

Modifications made to the Classification as a result of Round One Feedback The members of the classification task forces (Delphi Process, Conceptual Framework and Conceptual Identification) considered the results from Round One and made the following modifications to the Classification. 1. An overview of the classification was developed. The overview a. Provides background on the development of the classification; b. Differentiates and discusses the relationship between a classification and a reporting system; c. Describes the classification, including a detailed explanation of its structure and composition;

d. Explicitly illustrates how the concept of a patient safety event (both adverse and near miss events) were captured by the classification.

e. Delineates each classification tree and the concepts contained therein; and f. Demonstrates how to classify an event using the classification’s conceptual framework as a

model, including two examples.

Page 17 of 194

2. The definitions of the terms for the following classes were clarified: a. Event Type (adverse events and near misses) b. Patient impact/Outcomes c. Contributing Factors d. Actions Taken e. Recovery Factors 3. The relationship between and among Contributing Factors, Preventive Factors, Recovery Factors and

Mitigating Factors was explained. Round Two Survey (see Appendix B) The second round of the Delphi survey took place from 1st November – 27th November, 2006. The Round Two survey consisted of the same sections and questions as the Round One survey, with the one exception and two additions: 1. Exception – Because the Round Two survey was sent only to those who responded in Round One, the

participant demographics section was not included. The information had already been captured. 2. Additions –

a. The overview of the Classification at the beginning of the survey

b. Quantitative and qualitative feedback from Round One for each question, as well as the responses to the comments and the modifications to the Classification based upon the results of the first survey.

Round Two Survey Results 75 out of 253 invited participants responded to the Round Two survey (response rate = 29.6%). Although 75 individuals responded, only 68 completed the entire survey. Again, the respondents included health car professionals, health policy experts, developers/managers of patient reporting systems, patient/public representatives, academicians, representatives from professional associations for a variety of health care specialties, representatives from litigation, classification/taxonomy experts, risk managers and representatives from organizations responsible for assessing and monitoring patient safety performance. Of those who responded:

- 93.3% (69/75 responses) believed the conceptual framework was an adequate model for use in describing a patient safety event;

- 83.1% (59/71 responses) believed the conceptual framework was a meaningful and useful tool for translating disparate information into a format conducive to learning and improving patient safety;

- 13.2% (9/68 responses) thought that the conceptual framework was missing at least one class. Again, upon reading their comments, it became apparent that most of the respondents were referring to concepts they felt should be included within the classes instead of classes that should be included within the conceptual framework.

Page 18 of 194

- 89% (61/68 respondents) felt that the terms and definitions for each of the 10 classes contained within the conceptual framework were clear, precise and provided an accurate representation of the nature, properties, scope and essential qualities of the concept; and

- 82% (61/68 respondents) felt that all 10 classes within the conceptual framework were meaningful and useful.

All comments were reviewed. The number of comments ranged from five (5) to 14 per question. There were 27 general comments made. The main themes to emerge were:

- The need for field testing and an instruction manual - The conceptual framework is too complex for the average, every day user - Confusion regarding the relationship between Recovery Factors, Mitigating Factors and Actions

Taken - Confusion regarding the role of the Preventive Factors class - Structurally, “Patient Procedures” belongs under the Event Characteristics class instead of Patient

Characteristics class - Concern over the term for and purpose of Recovery Factors - Concern over the “Behaviour” concept

Modifications made to the Classifications as a Result of Round Two Feedback The Drafting Group considered the results from Round Two during the 13-14 December 2006 meeting and made the following modifications; 1. The conceptual framework was revised to clarify the purpose of each class and to explicitly show the

relationships among and between them. Specifically;

a. The actions taken as a direct result of the patient safety event were separated out from the Actions Taken class. The Actions Taken class was then renamed “Actions Taken to Reduce Future Risk”. The concepts in this class were narrowed to focus on organizational actions taken as a result of the patient safety event to inform the development of strategies to reduce the risk of the patient safety event reoccurring (eg. learning cycles);

b. The Preventive Factors class was renamed to “Before Event (Preventive) Factors”; c. The Recovery Factors class was renamed to “During Event (Recovery) Factors”. The “Clinical

Actions Taken” concepts were moved from the Actions Taken class to this class to indicate the actions taken as a direct result of the patient safety event;

d. The Mitigating Factors class was renamed to “After Event (Mitigating) Factors”. The “Clinical

Actions Taken” concepts were also included in this class to indicate the actions taken as a direct result of the patient safety event; and

e. “Patient Procedures” was moved from the Patient Characteristics class to the Event Characteristics

class.

Page 19 of 194

2. The concepts contained within the third level concept “Behaviour” (Contributing Factors, Human and Performance Factors, Behaviour) were modified:

a. “Mistakes (includes accidents, slips and lapses}” was moved from “Behaviour” to the third level

concept “Cognitive Related Factors” under Contributing Factors, Human and Performance Factors; b. “Perception” was moved from Behaviour to the third level concept “Cognitive Related Factors” under

Contributing Factors, Human and Performance Factors; c. “At Risk Behaviour” was retained as a fourth level concept; and d. The concepts of “Negligence” and “Recklessness” were made fifth level concepts under “At Risk

Behaviour”

3. The definitions of the following classes were refined: a. Contributing Factors b. Actions Taken – now known as Actions Taken to Reduce Future Risk c. Preventive Factors – now known as Before Event (Preventive) Factors d. Recovery Factors – now known as During Event (Recovery) Factors e. Mitigating Factors – now known as After Event (Mitigating) Factors

Following analysis of the Delphi responses, the conceptual framework was revised as responses revealed an unacceptable level of confusion regarding the relationships between and among the classes; specifically between Contributing Factors, Preventive Factors, Recovery Factors and Mitigating Factors. Many respondents felt the conceptual framework was too complex. The Drafting Group attempted to address these concerns at the December 2006 Drafting Group meeting (see figure 4). Figure 4.


Influences

Incident types

Patient characteristics

During Incident

(recovery)factors



Actions taken to reduce future risk

Post-Incident (mitigating)

factors

Patient OutcomesOrganisationaloutcomes

Pre-Incident(preventive)

factors

Influences

Influences

Framework class


Learning cycles

Modifying factors

Have

Have

Page 20 of 194

Further Modifications Made Following December 2006 Drafting Group Meeting After the December 2006 meeting, it was decided to undertake the further review of the Delphi comments. After developing responses to each Delphi comment, and following the development of definitions for the key patient safety concepts outlined in the Key Concepts, Definitions and Preferred Terms section, the conceptual framework was again revised. The Classes The Delphi analysis revealed the classes were incomplete and, in some instances, inappropriately organized. The classes were refined to ensure that the concepts are organized hierarchically according to natural mapping (the grouping of subsets or attributes of a class or concept in a representation reflecting the real world as it is perceived) and fall into natural categories (descriptors which are brief, easily and commonly understood). To summarize: • Event Types was relabeled Incident Type. • Patient Impact/Outcomes was relabeled Patient Outcomes. • Patient Characteristics remained Patient Characteristics. • Event Characteristics was relabeled Incident Characteristics. • Contributing Factors was relabeled Contributing Factors/Hazards. • Actions Taken was deleted. • Detection was added. To recover, one needs to detect and then mitigate: “to prevent or moderate the

progression of an incident”. Recovery is comprised of detection plus mitigation. The revised conceptual framework is an endeavor to reduce the substantial confusion surrounding the intent and meaning of the classes.

• Before Event (Preventive) Factors was revised and relabeled Actions Taken to Reduce Risk as a result of the

Delphi analysis. This class encompasses concepts previously contained in the deleted Action Taken class. • During Event (Recovery) Factors was deleted. • Mitigating Factors was added in light of the Delphi analysis for the same reason Detection was added. To

recover, one needs to detect and then mitigate. Recovery equals detection plus mitigation. • After Event (Mitigating) Factors was deleted. • Ameliorating Actions was added. The concepts under this class indicate the actions taken or circumstances

altered to make better any harm after an incident • Organizational Outcomes remained Organizational Outcomes. The final conceptual framework (shown in Figure 1) continued to organize high level classes into a transparent, theoretical and logical architecture. The labels changed, but the underlying concepts did not.

Page 21 of 194

The classes Incident Type and Patient Outcomes were designed to group events into recognizable categories in clinically meaningful ways so that new events could be promptly characterized and information of past events easily retrieved. This information is fundamental to the description of a patient safety incident. The classes Contributing Factors/Hazards, Patient Characteristics, Incident Characteristics and Organizational Outcomes provide descriptive information. Finally, the classes Detection, Mitigating Factors, Ameliorating Actions and Actions to Reduce Risk or Harm are concepts associated with system resilience and inform learning and analytical processes. A description of the major revisions to the classes follows: Detection and Mitigating Factors As noted by the subgroup, there was significant confusion regarding the meanings and purposes of the Recovery Factors, Mitigating Factors, and Preventive Factors classes. The concept of error recovery, derived from industrial science and error theory, is particularly important if learning from patient safety incidents is to occur. Recovery factors are “actions or circumstances that follow detection of an incident and prevent or moderate its progression so as not to result in harm;” thus as Tjerk van der Schaaf’s work suggests, error recovery has a detection phase and an action phase. The subgroup decided to create Detection and Mitigating Factors as classes to signify the detection and action phases and to delete Recovery Factors. Detection is defined as “an action or circumstance that results in the discovery of an incident” (i.e., the detection phase). For example, an incident could be detected by noticing an error, via a monitor or alarm, by a change in the patient’s status, or via an audit, review or risk assessment. Detection mechanisms may be built into the system as official barriers or informally developed. Mitigating Factors are defined as “actions or circumstances which prevent or moderate the progression of an incident toward harming the patient” (i.e., the action phase). With mitigating factors, a distinction is made between preventing an initial error and minimizing the harm to the patient after the error has already occurred and the start of the damage mechanism triggered. Recovery factors are consequently detected and then mitigated. Organizational Outcomes The original concepts for this class were adapted from the International Classification of Functioning, Disability, and Health (ICF). These concepts, even as modified, were not fit for purpose for this class. This became clear from the Delphi responses. The revised concepts populating this class have been derived from real world incident reports and are commonly used in the health care field. Ameliorating Actions Ameliorating Actions is a new class. It was borne from the ambiguity created by mitigation/recovery, action taken and preventive factors. To develop this class the subgroup revisited the literature, reviewed the temporal flow of patient safety incidents, and reviewed real world incident reports. The concepts contained in this class are not circumstances but are actions undertaken by the healthcare professional or organization to “right a wrong.” The concepts signify “doing something”, so the word action has been substituted for factor. Actions Taken to Reduce Risk A significant outcome of the Delphi analysis was the need to clarify the meaning of, purpose for, and temporal aspect of, the Preventive Factors class. To redefine the concept of this class the subgroup posed the following questions: (1) could the incident have been prevented, or potentially prevented?; (2) what policies, procedures, etc. were in place to prevent or potentially prevent the incident?; and (3) what was done after the incident to prevent future occurrences?

Page 22 of 194

After much discussion, the subgroup concluded that preventive factors were essentially the inverse of contributing factors. The concepts under Actions Taken to Reduce Risk concentrate on steps taken to reduce the risk of reoccurrence of similar incidents. The concepts are not circumstances but circumstances altered; they focus on learning and system resilience efforts. The concept of resilience in the context of the ICPS is somewhat different than in resilience engineering (Hollnagel, Woods, et al.). In the International Classification for Patient Safety, resilience is defined as the degree to which a system continuously prevents, detects, mitigates or ameliorates hazards or incidents. Ameliorating hazards includes “bouncing back” to the original ability to provide core functions as soon as possible after incurring damage.

Page 23 of 194

Responses to the Delphi Comments EVENT TYPES Original Definition Event types: The characteristics distinguishing a group or class of patient safety events. Revised Definition based on 1st Round of Delphi Results Event types: A group or class of patient safety events distinguished by their characteristics. A patient safety event is a process or act of omission or commission that resulted in harm to the patient or hazardous healthcare conditions that could result in harm to the patient. A patient safety event that harms a patient is an “adverse event”. A patient safety event that results in hazardous healthcare conditions but did not harm the patient is a “near miss” (see Patient Impact/Outcomes). A patient safety event is identified by a generalized high-level, discrete, auditable term or group of terms. The terms have good face validity. Changes from December 2006 Drafting Group Meeting No changes were made to the revised definition. Event types: A group or class of patient safety events distinguished by their characteristics. A patient safety event is a process or act of omission or commission that resulted in harm to the patient or hazardous healthcare conditions that could result in harm to the patient. A patient safety event that harms a patient is an “adverse event”. A patient safety event that results in hazardous healthcare conditions but did not harm the patient is a “near miss” (see Patient Impact/Outcomes). A patient safety event is identified by a generalized high-level, discrete, auditable term or group of terms. The terms have good face validity. Changes after Review of all Delphi Respondents’ Comments The (first) pervasive issue to emerge was the confusion arising from the initial use of the term “event” and the need to specifically add to definitions to point out that hazards and circumstances were also intended to be included. Delphi responses confirmed that the use of the term “event”, on its own, was problematic. It immediately constrains the thought process of the reader or classifier to what is generally understood by the term “event”. Delphi comments confirm that events are generally not perceived to include underlying hazards and dangerous circumstances. The title of the classification was changed from International Patient Safety Event Classification to International Classification for Patient Safety, mainly for this reason. The term “incident” is now widely used in the healthcare safety literature (see, for example, the index to the recent book on “Patient Safety: research into practice”, edited by Kieran Walshe and Ruth Boaden, and our “Shared Meanings” paper which was the result of an extensive Delphi process by a group of international experts in patient safety. Many definitions and qualifying comments had to include additional phrases to encompass hazards and circumstances as well as events. These can be greatly simplified if “incident” is used, as both event and circumstance are then implicit as they are included in the definition of “incident”. The cumulative effect of this is important as short, succinct definitions are needed so as to achieve general consistent usage. The comments of the Delphi respondents provide strong support for the use of the term “incident”. For example, in reviewing the Delphi responses to the questions on “event type”, a substantial number would have been pre-empted or dealt with had “incident” been used rather than “event” (see comments I 13, II 8, II 11, II 15, II 18, II 20, II 27, II 30, II 36 on pages 12 - 18).

Page 24 of 194

Incident type: A descriptive term for a category made up of incidents distinguished by agreed characteristics. Note 1: The key characteristic is usually captured by the term used for the incident type. Incident type is a “super category” or a “parent” natural category under which many concepts (also described by “natural categories”) may be grouped; these may be “children” (of the parent incident type) or attributes of the parent or any child. Note 2: A natural category is a descriptor (usually a short phrase) which is brief, easy and commonly understand which captures the essence of an event or circumstance, or of associated characteristics or attributes, and is not constrained by being restricted to any class or property. To take an example from the workplace, 40 of 100 incident reports describing why staff are late for work might be assigned to the natural categories:

1) could not find car keys, 2) child sick – no other carer, 3) had a puncture (flat tyre), 4) alarm clock did not go off, 5) usual car park full.

These phrases capture the essence of each incident, and most observers would categorize them in the same way. The incident types in ICPS also involve errors and violations on the part of staff and patients, the actions of extrinsic people and/or equipment, medications and organizational and other factors (see ICPS – Incident Types). An incident type is not an exclusive category. An incident may need to be described by several incident types, although a “business rule” may be developed to allow “principal” incident type to be nominated, so that incidents may be counted. One rule is to nominate as principal incident type the incident which led most directly to any harm or potential harm. Thus, for example, for an incident in which an infusion pump was set up wrongly and delivered an overdose of a sedative, causing respiratory arrest, the drug overdose (the medication incident type) would be the principal category, rather than the equipment incident. Note 3: The difference between an incident and a hazard is that a hazard is something that may become or contribute to an incident in the future, whereas an incident has occurred. The same circumstance may be described as a hazard or an incident, depending on whether it is viewed anterospectively or retrospectively. Note 4: An incident type is not an exclusive category. An incident may need to be described by several incident types, although a “business rule” may be developed to allow a “principal” incident type to be nominated, so that incidents may be counted. One rule is to nominate as principal incident type the incident which led most directly to any harm or potential harm. Thus, for example, for an incident in which an infusion pump was set up wrongly and delivered an overdose of a sedative, causing respiratory arrest, the drug overdose (the medication incident type) would be the principal incident category rather than the equipment problem. Note 5: The word “type” does not itself refer to characteristics, but rather to the properties or class itself (see I 7). Hence the new definition has “a descriptive term for a category…” Note6: Some of the comments of Delphi respondents dealt with by the use of the term “incident” rather than event include I 13, II 8, II 11, II 15, II 18, II 20, II 27, II 32, II 36. Note 7: Objections were raised to the use of “process” in this context (I 2, I 3, I 16, 1 26, I 29, I 38) and also to referring to acts of omission or commission (some of the above plus I 30, I 33, I 43, III 1). Note 8: The last two sentences in the “event type” definition are relevant to all classes rather than this particular one, and add little to the definition of this particular class. Note 9: Numerous comments referred to problems with understanding terms or terms not being defined (I 11, I 14,I 18, I 21, I 22, I 23, I 24, I 25, I 26, I 29, I 30, I 32, I 34, I 37, I 39, I 42, II 15, II 18, II 22, II 29, II 41, III 4, III 9, IV 14, V 7).

Page 25 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Event Type, or its associated definition to be unclear.

I 1.

I 2. The event types as described include a mixture of error/harm (e.g. HAI) and factors that contributed to the error/harm (e.g. communication). And I'm unsure as yet why incorporating processes as an event is helpful. If we acknowledge that errors are derived from system failures or weaknesses, the we should be able to identify the outcome of the error/harm (impact on patient, organization, member of staff, family etc) and the contributory factors (grouped by broad type and detailed) Comment: We agree. The use of “process” in this definition is inappropriate. The classification will allow the outcome and the contributing factors to be identified.

I 3. It might turn out that the terms of 'event' and 'process' are not compatible ( a process is a line between events) Comment: We agree; the definition has been changed.

I 4. unclear as to what the basis of the categorization is Comment: We have attempted to deal with this by using definitions of natural category.

I 5. How would Adverse Drug Reactions fit this definition? Unintended harm results from the use of a drug and there is no 'act'. What kind of 'characteristics' do you mean? Comment: We believe adverse drug reaction is now compatible with the definition; adverse reaction and side effect have now been defined.

I 6. test

I 7. I find the associated definition to be unclear (not the term itself). Events can be classified in different ways, according to many different perspectives and consequently characteristics isolated by these perspectives. As such "the characteristics distinguishing a group or class" is too broad. ICPS is only interested in specific characteristics. Besides, ontologically, the Event Type does not correspond to "the characteristics distinguishing a group or class" but to the group or class itself, determined by given pre-selected characteristics. Comment: We agree. Event type does refer to the group or class itself and we believe that the new definition has taken your suggestions onboard.

I 8. As explained in previous box general characteristic of an event and care setting & process is missing : emergency, elective, planned, re-intervention Comment: These concepts will be accommodated in ICPS.

I 9. This is an IT term. You need to pick a Clinical Term that would fit so BOTH and IT and Clinical person can understand it. As qualified in both I think you mean Admission but it could be Episode..or Treatment...but you have the word Action so that should be Treatment..so I am confused Comment: We believe that the new definitions should have meaning both to clinicians and IT people.

I 10. As noted in my first set of comments, I do not think that 'event type' is a useful or achievable entity/classification, as it will inevitably need to be used as a purpose-specific summary of information collected on contributing factors, or demographics, or event characteristics, etc. The current suggestion is not mutually exclusive and would not suit many analysis purposes. It would be better to remove the classification and to develop purpose-specific summary categories at a later stage, in association with specific analysis needs. The definition used here does not define what an event is (although the note does, to some extent). What is needed here is a definition of a patient safety event, and not of the 'characteristics'). The note is not well defined because 'hazardous' is not defined (and all healthcare is hazardous, almost by definition). Also, intended harm (e.g. due to negligence)

Page 26 of 194

needs to be included. The blanket exclusion of all other 'intended' harm is tricky, because the level of harm associated with treatment (e.g. with drug side effects) regarded as 'intended' and 'unintended' is open to judgment. The distinction between acts of omission and commission seems to be unnecessary, and jargon. Comment: To deal with each of your points: we are now using incident type, and believe it can be used for purpose-specific summary information. Incident types are not mutually exclusive as the ‘real world’ that needs to be captured is too complex to force into a single category. The intention is that the classification can be used to store features of deconstructed incidents in such a way that they can be analyzed in various combinations. We have now included a definition of an incident type, as well as of incident and its component terms. Intended harm is now included. We agree that acts of omission and commission should not be mentioned here.

I 11. What do you mean by "distinguishing" a group or class? Comment: See comment 12.

I 12. Examples would be helpful to clarify exactly how this is to be understood. Comment: We have now given a definition of natural category and a list of event types are in the first layer of ICPS.

I 13. The definition of "patient safety event" is unclear. It needs to include those events that are acts of omission or commission that have the potential to cause harm but do not do so, and I don't think the current definition would include those events. Also, the reference to "unintended harm" doesn't seem necessary; the word "unintended" could be omitted. Comment: We believe the new definitions have addressed the points made and “unintended” is not included.

I 14. THE TERM EVENT IN THE CONTEXT USED HAS A NEGATIVE CONNOTATION. EG COMPLAINT Comment: We have stated at the beginning that the term incident implies a negative connotation for the purposes of this classification and that this, amongst some other features, should be regarded as implicit once this point has been made at the beginning.

I 15. should be expand explanation for type of event e.g., mistake identification, maintenance, security, etc. Comment: We believe we have done this by providing a list of event types.

I 16. The definition refers to "A process ...etc [singular]" that had a particular type of result. That result -- hazardous health condition etc -- may well be the result of a series or combination of processes or acts. Definition would be improved if it could reflect this. Comment: The way the framework will be set up will allow this.

I 17. This is obviously very difficult to define and there will never be a perfect definition. However two things seem problematic - Is this always a single event/patient? The inclusion of process (which I think is good) may indicate a class of events - Hazardous healthcare conditions is a good phrase, but sounds like it include disease factors Comment: We hope that we have covered this with the new definitions.

I 18. The definition of a patient safety event in the survey is not the same as that contained in the Executive Summary at p.3. The latter adds the stipulation that the harm shall "not (be) due to the underlying condition of the patient". In my experience (and in several of the vignettes that form part of the survey) the harm, where there is fault, is most often due to the disease and not the doctor, but the concern is that a proper standard of care would probably have prevented the injury, at least to some extent. It is not clear whether such cases would be covered by the ICPS definition of a patient safety event. Another difficulty is that the proposed definition may not make sufficient allowance for the very different standards of care which exist in the world. Comment: In the preamble we have now covered this and it should be taken as “a given” in ICPS that incidents refer to iatrogenic incidents, which are defined. We are not concerned with whether safety is present or absent in an absolute sense, and we will not be trying to quantify safety. The degree of safety (or risk) is clearly context dependant and may be influenced by financial and logistic constraints, and risk may be worthwhile in the context of balancing it against benefit, sometimes in

Page 27 of 194

the context of no alternatives (see Basic Definitions, Notes under ‘Incident’)We believe this issue is now covered by the definition of and notes for “patient safety incident.

I 19. With the definition it is difficult to distinguish between 'Event type' and 'Event characteristics' Comment: We believe we have dealt with this.

I 20. What are hazardous healthcare conditions? This is entirely new concept, the conventional definitions listed below are difficult enough in practice without introducing a new concept that is additionally not patient centered. Comment: We believe this is now covered.

I 21. who is the audience for this information - commission and omission will need to be defined for the average worker - patient safety involves more than the bedside caregivers - so the language may need to be at a lower level in the definition stage Comment: The words commission and omission are now not included.

I 22. 1. The users of this system include social care services, mental health, acute and community services; the words healthcare are not generally accepted in social care environments as they do not treat the patients. 2. The word patient inferrers that we are medicinally treating the patient. Thus we use the word service user as we have patients, tenants, clients. Comment: In the preamble we have included a passage which states that we use “patient” for simplicity, and accept that there are many contexts in which different terms may be more appropriate at the level of the clinical patient-error interface (see Basic Definitions, Notes under ‘Patient’).

I 23. The term relies on other terms which themselves need definition, for instance 'hazardous' and 'harm'. While the framework itself is good, the underlying terms and definitions need more work before they are useable. Comment: This has now been done.

I 24. It does not reference if it includes near misses. What is the level of harm that would be included? Comment: We believe the definitions are easier to understand. However to prevent individual definitions from becoming extremely complicated, we have taken the approach of “building” from basic definitions (see Introduction and Basic Definitions).

I 25. You cannot understand the definition for event type without the associated "note" about what a patient safety event is. The definition needs to stand alone. Comment: This has been included in the definition.

I 26. Patient Safety Event = PSE 1. a PSE is not a process - it is a consequence 2. see before - "resulted" that is a terminal event, whereas proximity beginning at 'potential' describes the temporal line of consequence 3. there is a presumption that an "act" has occurred when it actually effect something from outside the territory may be the effector - e.g. patient harmed when building fell down. 4. the definition further presumes that causality has been determined or can be determined to activate "commission" or "omission". The inclusion of causality without proof is placing the descriptor before the investigation of 'why?' 5. hazardous is a "hard" term. Adverse (implying degrees of severity of consequence) is a much more comfortable and less pejorative term to use. For example, in the NHS risk matrix grid the term "catastrophic" is used. To a Chief Executive such a catastrophe must never occur, yet, such an event (death of a newborn) may be a justifiable consequence of circumstances. The lay press views risk terminology differently to the professionals engaged in such events. Comment: We believe that the points made have been dealt with in the new definitions. The term “hazard” in the definition of safety has now been defined.

I 27. The definition appears to define the "why/how" of event types rather than what the event type is Comment: We agree, and have changed the definition.

Page 28 of 194

I 28. I would have expected a "type" to be a "The set of characteristics distinguishing a group ..." If there are n characteristics associable with an event type. I would expect that in n-space a particular type would be seen as a distinguishable cluster of nearby neighbors. Comment: We think this has been covered.

I 29. An 'event' cannot be a 'process' or an 'act'. Nor is it a 'safety' anything; the term relates to lack of safety, harm or risk Comment: We agree and the definition has been changed.

I 30. Commission / omission is too simple. What about acts that occur too early or too late. Or are too strong or too weak. Or out of order? Comment: We agree and have deleted commission/omission.

I 31. I believe something should be classified as an event only if an adverse outcome occurred. The above definition does not meet this criterion. Comment: The definition of adverse event is now only when harm has occurred.

I 32. The term "hazardous healthcare conditions" seems too broad, does it include the physical environment or does "condition" relate to the persons health, disease or disability? Are "near misses" included in this definition? Comment: The new definition does include further circumstances.

I 33. I have difficulty with the notion of omission or commission Comment: We agree the definition has been changed.

I 34. Event type: don’t necessarily represent characteristic, but it can represent nature/form. And; The associated definition is so complicated Comment: We will continue to use the term “characteristic” but have qualified it.

I 35. "A patient safety event" gives a conception of something positive, not negative result. Comment: We have now stated in the preamble that explicit in the definition of incident is something negative.

I 36. I need more description such as an example - "type" is redundant - "Event" is a clearer term on its own. Comment: We have now given examples.

I 37. I can't figure out where the event type is going - OR events? drug events? near misses? law suit? communication error? is that what your after? Comment: We have now given examples.

I 38. The associated definition describes the particular aspects of the term event type, but not the term itself. In the PDF of the draft it says "a process ... that leads to a patient safety event". As I understood the term: The event type is the patient safety event therefore it cannot lead to it. Comment: We believe that this has now been dealt with in the new definition.

I 39. According to the definition 'sudden event' should be an event type (sudden is a characteristic), so how to classify misplaced drug list? is it an event type or not? Comment: We believe the new definition has rendered this problem redundant.

I 40. Not clear without seeing list of proposed event types Comment: We have now provided a proposed list (see the first layer of ICPS).

Page 29 of 194

I 41. too generic Comment: Hopefully, the list itself does not have the problem of being too “generic”.

I 42. Unclear what characteristics are being requested. Also, why are close calls omitted form the definition? Comment: Close calls have now been included.

I 43. In fact the definition is very clear but I disagree with it Comment: So do we; the definition has been changed.

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Event Type.

II 1. see above

II 2. ?

II 3. I do not have the definition of ADR handy but WHO has a definition and it should be included. Comment: Adverse reaction has now been defined.

II 4. test

II 5. "A group or class of events determined by their underlying causal characteristics." Comment: The new definition is more specific.

II 6. Consider additional item "emergency" for care setting and care phase; "re-admission" for phase of care Comment: Phase of care will be included elsewhere in the classification.

II 7. Perhaps "Admission"

II 8. A patient safety incident is the identification of harm actually occurring to a patient (or of circumstances that resulted in risk of harm occurring to a patient) that was increased compared with usually occurring circumstances. Patient safety incidents include those that arise because an action has been taken, and those that arise because an action has not been taken. They include those that arise because of actions of individuals and those that result from system or organizational arrangements. (I realise that this does not solve the issue of judgment about what harm that occurs or could occur that is 'increased', but it does fit with the notion that there is always a risk of harm, and that patient safety efforts are aimed at reducing that harm.) Comment: We think our new definitions are consistent with the points being made here.

II 9. The characteristics that, together, serve to identify... Comment: We hope our definition is now satisfactory.

II 10. Examples would be helpful to clarify exactly how this is to be understood Comment: Examples of incident types have now been given in the next two layers of ICPS.

II 11. The definition of a medical error is "the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim" (Kohn, Corrigan, Donaldson, 1999). This definition would capture events that did not result in harm, as is your intention. Comment: The definition is intended to capture hazardous circumstances as well as error. We have defined error separately in a manner consistent with the definition cited but excluding outcome. Outcome should have no place in the definition of error. In the definition cited “completed” should be replaced with “carried out”.

Page 30 of 194

II 12. EVENT COULD ALSO SUGGEST WHERE AN ACTIVITY HAS PROVING SUCCESSFUL EG RESEARCH PROJECT AND THEREFORE MAY HAVE A + IMPACT ON THE DELIVERY OF CARE. Comment: This has been dealt with in the Introduction and Basic Definitions.

II 13. Event categorize. Comment: We do not understand this response.

II 14. Suggest changing "resulted in" to "resulted in or contributed to..." Comment: Dealt with by a new definition.

II 15. Needs a statement to clarify that it might be either (a) one particular patient or (b) circumstances that might affect patients. Perhaps part of explanatory material, not core definition - Hazardous conditions for the delivery of healthcare? Comment: This is now in the definition.

II 16. In the circumstances, you might like to think about a definition of a patient safety event along the lines of "an injury or other harm caused or potentially caused when clinical care falls below the standard acceptable for the community in question". Comment: We believe we have dealt with this in the key definition and in notes

II 17. Event type-a category of patient safety events having common characteristics. Comment: We believe this is dealt with in the new definition.

II 18. Two definitions of medical error are commonly quoted: that advocated by Reason and adopted by the IOM: “The failure of a planned action to be completed as intended…or the use of a wrong plan to achieve an aim”; and that used by JCAHO “an unintended act, either of omission or commission, or an act that does not achieve its intended outcome” 88 p5. One includes unplanned acts, the other excludes them 88 p5. The UK National Patient Safety Authority (NPSA) defines a patient safety incident as “Any unintended or unexpected incident that could have or did lead to harm for one or more patients receiving NHS-funded healthcare” 17 p12. Runciman, Merry and Tito argue for error being: “the unintentional use of a wrong plan to achieve an aim, or failure to carry out a planned action as intended”89 Comment: We are using the Runciman, Merry, Tito definition, rephrased in the ‘Shared Meanings’ document (Runciman, W.B. (2006), ‘Shared meanings: preferred terms and definitions for safety and quality concepts’, Medical Journal of Australia 184:10, S41-S43).

II 19. *

II 20. A safety event is a process or act of omission or that resulted in hazardous conditions and/or unintended harm to the client. Comment: We believe our new definition is consistent with this.

II 21. As stated above, this needs more work. I have been involved in the development of several national data collections and, before suggesting alternatives, would want the chance for more discussion and testing with practitioners in Australia. Comment: Testing will be carried out.

II

22.

Near miss definition needed and could likely be found at IHI.org Comment: We have now included this.

II 23. Use specific examples (medication errors, etc.) Comment: We have now done this (see first layer of ICPS).

Page 31 of 194

II 24. A PSE by definition is a consequence of aligned events that culminate in an effect of varying degrees of severity upon a patient either pathologically and/or sociologically that affects the patients expected and proposed clinical outcome of an engaged clinical sequence of events. Comment: We believe our new definition is consistent with this.

II

25.

The distinguishing feature which is characteristic of a group or class of patient safety events Comment: We believe the new definition is consistent with this.

II 26. See above.

II 27. A harm event results from a process or act of omission or commission that resulted in unintended harm to the patient. A risk event results from a process or act of omission or commission that resulted in hazardous healthcare conditions for the patient. Comment: We have not used these terms, but believe our definition encompasses the concepts.

II 28. see above

II 29. Would use instead the IOM definition of an adverse event. The above lumps near-misses and injuries and this is highly problematic. Comment: The classification will not lump near misses and injuries, but will allow each to be

classified.

II 30. XXXXX

II 31. not sure

II 32. Event type could be defined as: The nature/form of the patients’ safety events related group/class. The associated definition, it could be like that: A patient safety event is a process or activity or act that resulted in hazardous healthcare condition and/or unintended/intended harm to the patient.. Comment: Again, we hope our new definition is consistent with the concepts

II 33. Maybe it would be better "patient unsafe event" (James Reason). Comment: We understand this point, but it is not in common usage.

II 34. Event

II 35. I think it depends on what characteristics you follow up with whether event type will make sense or not. How about just "EVENT" Comment: We have now given a list of incidents (see first layer of ICPS. We believe the new definition for patient safety incident and incident type address this issue.

II 36. Why not: a process or act of omission or commission that resulted in hazardous healthcare conditions and/or unintended harm to the patient? Comment: We believe our new definition is consistent with this.

II 37. Event should be the class containing all the events. An event should have an actor a location and so on. Each event should be defined by a combination of these other classes Comment: We believe our new definition is consistent with this.

II 38. Event type such as:

II 39. no

II 40. No suggestion as the entire concept seems to have little utility. It needs to be rethought and extensively revised or started anew. Comment: We believe the new definition will allow the proposed system to have utility.

Page 32 of 194

II 41. Change the Note to incorporate near hits/misses Comment: We now include near misses.

Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. see also statement above Since the terminology of event is confusing since most professionals associate

event with adverse (drug) events, that is a (short-term) health outcome and NOT an act of omission or commission. 4) Suggestion: divide event type into "patient safety incident/error/else", 1) In practice the worsening of health or unexpected non-improvement is mostly the first perception from which questioning starts. (Highlighting this also helps to alert professionals to opportunity for early intervention.) 2) Differentiation of bad health outcome, bad healthcare outcome (either avoidable/error-related/incident-related or unavoidable/healthcare immanent/non-amenable) tends to come second in practice. Comment: We believe these points have now been taken care of.

III 2. See previous remarks on event types

III 3. already did

III 4. Please see my earlier comments on this. It is important to clearly define what this is (and to use that definition as the 'central' definition on which the other elements hang), but it not useful or practical to 'classify' the events in a 'one-size-fits-all' classification. PS. I do not understand what different type of information you are requesting in this part of the survey, compared with the previous part. Comment: We believe we now have clearly defined the core concepts.

III 5. The range of main event types should be listed for it to be useful for those who will use this framework Comment: These are contained in the second level concepts under Incident Type.

III 6. It is unclear to me exactly what it means

III 7. It is useful, but a bit too simplistic - it does not adequately address the fact that an adverse event is often "more than one thing" Comment: The way we intend the classification to be used incidents will be able to be classified using more than one incident type, each of which can have contributing factors.

III 8. This chapter is strikingly similar to the CCS and is the most immediately applicable chapter of ICPS. I would have liked to have rated it higher, except that to do so would have lost the opportunity to comment about the following matter of detail. Section I.A.7 might be better used to identify whether the abuse is patient-to-patient, staff-to-patient or patient-to-staff. Abuse and assaults by patients on staff can have a direct effect on clinical care and are an increasing concern which should be specifically identified in the taxonomy. In our experience, violent patients seldom limit themselves to just one of racial, sexual, verbal or physical abuse and many would score a full house, making the classification difficult to apply to them. Comment: These have been included and will be able to be classified.

III 9. the strength of agreement will depend on the familiarity with the definitions and terms that the user/s have. Comment: We agree. Our experience is that classifiers can use incident types consistently.

III 10. To the extent that it can be conveyed and understood and provides a useful basis for discovering like events - it is fine. But - given the difficulty of doing that, defining or using event type as a group will be very difficult.

Page 33 of 194

III 11. Still need better definition of event types; then may work. Comment: We hope we now have a better definition.

III 12. Same reason. The idea of simplification and Standardization for improved safety should be applied here. I believe we are making this to complex. thanks John

III 13. It is useful, but I wonder whether standardised sub-groups adopted internationally would be useful to enable comparison between systems Comment: We believe this is what we have done with the list of incident types.

III 14. Without an articulation delineating 'event types', I can't really comment on it at this time. Comment: These are contained in the second level concepts under Incident Type.

III 15. See my discussion of the term "recovery factor." I think event has to be redefined to include, at a minimum, events that are immediately antecedent to a "patient safety event" or the current definition of "recovery factor" does not make sense (since one can't prevent something from becoming something that, by definition, it already is). Comment: We believe the inclusion of circumstance in the definition takes care of this.

III 16. As noted earlier, this category lumps a variety of things that I believe should not be lumped including adverse events and near-misses. Comment: The classification will include but discriminate between near miss and adverse event.

III 17. Event Type is a meaningful component of a Patient Safety Dataset or information system. Event types are one of the classifications in the system. Since the system has to use other classifications as parts of data set it should not be called classification. Comment: Incident type is one class in the classification. Each of the classes have hierarchies of concepts (both “children” and “attributes”).

III 18. The type of event is not relevant. The issue is around the nature of the contributing factors. as long as there is a good taxonomy of contributing factors we are then able to allocate event of any type without being specific Comment: We believe that as different types of incident can have different descriptions (e.g. for medication incidents versus aggression), that this is a useful distinction.

III 19. I would have expected these to be coded - and the use of the term to be an event - not a "type". Comment: These will be.

III 20. depends on what kind a sub classification you develop for event type Comment: Each incident type will have specific descriptions.

III 21. It should be "Event". The type of event should be defined by relationships to other classes (eventA has_location surgical_dept and has_cause missed_drug_allergy) Comment: This is the goal of the classification.

III 22. The entire concept for this effort seems to be more of a research type endeavor with little real world use. Comment: Our experience has been that the concept of natural categories for incident types is useful and can be used by classifiers.

Page 34 of 194

Additional Comments from e-mails 1. In my experience (and in several of the vignettes that form part of the survey) the harm is most often due

to the disease and not the doctor, but the concern is that a proper standard of care would probably have prevented the injury, at least to some extent. It is not clear whether such cases would be covered by the ICPS definition of a patient safety event. Another difficulty is that the proposed definition may not make sufficient allowance for the very different standards of care which exist in the world. In the circumstances, you might like to think about a definition of a patient safety event along the lines of "an injury or other harm caused or potentially caused when clinical care falls below the standard acceptable for the community in question".

Comment: We have dealt with this in the preamble in discussing safety (see Notes under Incident and Incident Type).

2. Might be better used to identify whether the abuse is patient-to-patient, staff-to-patient or patient-to-staff.

Abuse and assaults by patients on staff can have a direct effect on clinical care and are an increasing concern which should be specifically identified in the taxonomy. In our experience, violent patients seldom limit themselves to just one of racial, sexual, verbal or physical abuse and many would score a full house, making the classification difficult to apply to them.

Comment: We will address this in the classification.

Delphi Task Force Comments The feedback concerned (a) the Event Types class appeared to address only adverse events and not near misses; and (b) the type of events included were problematic.

Page 35 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the revised definition for Event Type to be unclear.

IV 1. It does not convey much information and has a flavour of tautology.

Comment: We hope we have fixed this with the new definitions.

IV 2. Object to inclusion of sexual harassment, patient abuse and others on this order that don’t belong in patient safety. Comment: We don’t agree. Being raped or assaulted is not safe.

IV 3. The problem isn't so much that the definition is "unclear" it's that the categories of events are not mutually exclusive. A single event could be coded many ways. Take the example of a wrong lens implanted into a patient. It could be caused by problems including: patient mix-up, lens mix-up (accidental switch), misreading handwriting on diopter or size of lens, wrong eye, etc. Each of these would be coded differently according to the information provided. The result would be that there would be no way to get at the full set of "wrong lens" adverse events and therefore no way to get at all the descriptions of the various problems. A group might address the patient mix-up problem since that's the one they found in the dataset, but leave lens mix-up problems unaddressed, or even make them worse as they try to fix the patient mix-up problem. Comment: Each incident will be able to be described by using several incident types. If people want to count incidents then a principle incident type can be chosen. A useful rule would be to choose the incident type that is most directly and proximally related to the actual or potential negative outcome

IV 4. We need to define what kind of characteristics. Comment: We believe we have dealt with this in the discussion about “natural category”.

V

Please provide an alternative definition, the relevant reference(s) to the sources and your rationale for including the alternative definition instead of that provided for Event Type. V 1. This is an important class, but inevitably pragmatic rather than 'clean'. It is usually a professional's

(sometimes a consumer's) attempted summary of 'what went wrong'. Can the definition reflect this? Comment: We have tried to do this with the discussion of natural categories (see Notes).

V 2. Blameworthy, negligence, reckless events don’t belong in patient safety Comment. This issue has been addressed. Negligence and liability will not be concepts included in the ICPS.

V 3. The event types need to be mutually exclusive. This is not easy to do, and the solution will not be inarguable. But, for example, one event type might be adverse drug events, subtypes might be drug overdose, allergic reaction , etc. Another event type might be surgery adverse events, one subtype could be wrong surgery (wrong patient, wrong side, etc.), another subtype might be operating room fire, and a third might be a retained surgical item (like a sponge or forceps). Some will have contentious borders: e.g. wrong patient anesthesia AEs as a subset of wrong surgery or a separate category under anesthesia? But a categorization along these lines (outcomes not causes) would be more intuitive to new users, patient groups and other stakeholders, and be no less useful to expert users with extensive experience. Comment: Each incident will be able to be classified using several incident types

V 4. a group or class of patient safety events distinguished by their clinical and non-clinical characteristics

Page 36 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? VI VI 1. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or

whether) it can be used. Comment: The concept of incident types can be used and practiced with a little training.

VI 2. Blameworthy, negligence, reckless events don’t belong in patient safety. Both aviation and the VA recognize this and do not allow these things in their safety systems. These events are instead kept behind a firewall of administrative investigations that deal with culpable acts. Also see comments made above. Comment: These concepts will not be in the classification. See preamble and definition notes.

VI 3. The concept of an "event type" is clear, but the types should be mutually exclusive, as they are in taxonomies developed by animals, plants, microorganisms, etc. As they are proposed they are not mutually exclusive. Comment: Each incident will be able to be classified using several incident types

VI 4. I worry about new frameworks that do not fit with other national and international classifications. how does this fit with snomed, read etc? Comment: We intend to interface with and/or cross-map to existing classifications

VI 5. Needs some work - on definitions and structure of the subclassifications if the intention is for them to be unambiguous and IsA relationships Comment: The concepts will be refined.

VI 6. Incident whether actual or near miss is probably more appropriate and meaningful than event Comment: We have addressed this.

VI 7. I feel it is not good taxonomy because 1.Class is a word with multiple meanings and 2.Is does not have any word relevance ( say like error does with mistake ) Also "Event Type" is very general and bi syllabic making it just that bit harder to say hence remember. They do both refer to relevant useful data, however, they are too charged with meaning. We must keep it easy and simple with combinations that are simple 1:1. Kind Regards Dr William Marchione Comment: 1) We have discussed the context in which we are using the term class. 2) We hope we have clarified what we mean. It does make data analysis more difficult but is necessary if we are to try to represent the complexity of what occurs.

Additional Comments from e-mails When reviewing the two examples provided is appears that multiple event types can be selected for a patient safety event. I am concerned if this approach is encouraged it would make data analysis difficult. Would it not be better to encourage selection of a primary event type and then allow a reporter to select additional events that relate to the primary event? Once again, I'm thinking about this from the perspective of a reporter. Delphi Task Force Comments The event types are mutually exclusive. The problem is that one event can have more than one event type (i.e., when there are multiple things that go wrong during what is perceived to be a single event). This should be addressed in the instruction manual.

Page 37 of 194

PATIENT IMPACT/OUTCOMES Original Definition Patient Impact/Outcomes: The results of the performance (or non-performance) of a function or one or more processes, services, or activities carried out in a healthcare context affecting the patient. Revised Definition based on 1st Round of Delphi Results Patient Impact/Outcomes: The effects of the patient safety event upon the patient. Changes from December 2006 Drafting Group Meeting No changes were made to the revised definition. Patient Impact/Outcomes: The effects of the patient safety event upon the patient (physical and/or psychological harm or near miss), and the severity (mild – death) and the duration of the impact (temporary or permanent).

Changes after Review of all Delphi Respondents’ comments Patient outcome: The nature, severity and duration of harm arising from an incident. Note: An incident from which no harm arises is known as a near miss (see definition of Basic Definitions, Near Miss).

Page 38 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Patient Impact/Outcomes, or its associated definition to be unclear.

I 1. see statement above: differentiation of outcome and impact at this level would make sense without making the figure too complex Comment: We believe that impact is embraced within the definition of outcome.

I 2. not sure I 3. I'm missing a reference to structural aspects of the facility where a patients receives care, e.g. no hand

rails, broken tiles on stair cases, no (or blocked) emergency exists in easy walking distance Comment: These concepts are dealt with elsewhere in classification.

I 4. I did find them clear but had to answer no to be able to provide a comment. I 5. This term ignores all the other variables operating at the same time. In a complex organization like a

hospital there will be many of them. It all suggests that the person i.e. patient produces the impact. Comment: Hopefully the revised definition will make this clear.

I 6. The associated definition is not exactly 'unclear' but rather 'insufficient' as it does not seem to account for outcomes resulting from events of the type "patient accident" Comment: These are now included in the definition.

I 7. "Patient Impact/Outcome" is appropriate here. but when you say on the page before.. "The result of the performance (or non-performance) of a function or one or more processes, services, or activities carried out in a healthcare context affecting the patient." A FUNCTION is an IT word (never used in Healthcare). It would translate to "procedure" in Healthcare (in English). In the same way you cannot directly translate from English to French you cannot translate Comment: The revised definition is clearer.

I 8. This definition needs to be focused on the patient impact/outcomes. At present, it is focused on the nature of the breakdown of the process, etc, and words like 'in a healthcare context' which should be part of the 'event' definition, if anything. The definition needs to be described in terms of reduced functioning of the patient (described in terms of aetiology and manifestation - i.e. in both ICD and ICF terms), or other impact on the patient - e.g. financial, emotional. It is quite inappropriate to use only ICD terms and not all the ICF terms, as proposed in the main document. Comment: Agree, the definition has been revised.

I 9. The terms "function", "process", "service" and "activity" are used without definition. Their meaning and the distinction between them is unclear. Comment: We have now defined component terms of definitions.

I 10. What exactly is a healthcare context? Is it a setting, or a procedure or what? Comment: This term is no longer in the definition.

I 11. The definition needs to provide for the possibility that the event occurred but did not harm the patient. If the definition includes the phrase "affecting the patient," does that suggest that there was harm? Comment: See revised definition and corresponding note. This definition allows for the possibility of no harm.

I 12. "healthcare context" is confusing, just state healthcare clearly as below: Comment: Term removed.

Page 39 of 194

I 13. patient's impact should be considered on their status and circumstance in anyway. Comment: Now included.

I 14. Grammar makes meaning unclear; failure to refer to harm or injury to patient appears to overly sanitize the concept. It is important to acknowledge what has actually occurred -- seems overly detached to refer to death, for instance, as the "impact" on the patient. Comment: This has now been addressed with the refined definition.

I 15. The initial part of the definition i.e. 'the result of performance is a function ' is overly wordy and basically adds nothing to the clarity of the definition Comment: This has now been addressed with the refined definition.

I 16. Its perfectly clear, but I think you might want to consider highlighting two things: - Both mental and physical impact - Short term and long term outcomes Mental/emotional effects often neglected. Long term outcomes barely considered at all usually Comment: These are now within the definition of harm.

I 17. Do you mean; "What happened to the patient?" Comment: Yes. The refined definition makes this clear.

I 18. After reading definition - not really sure what you are after. Comment: Agree, the definition was confusing therefore it has been revised.

I 19. We don't all have patients. Comment: This has been dealt with in a note after the definition of patient.

I 20. The definition is quite good. The use of the word 'function' is problematic as it is also used (in human functioning) in patient outcomes. 'Result' may also be problematic as it implies causality. Is this consistent with a systems and a modifiers/factors language and approach? Comment: A revised definition has been provided.

I 21. Need to elaborate on processes, services and activities so that everyone using them is using them in the same way. Comment: The definition has been refined.

I 22. too many words! Comments: Agree. The definition has been refined.

I 23. Impact is hard to translate Outcome is used in many other terminologies Comment: Impact is no longer used; it is contained within the definition of outcome.

I 24. Surely this is the impact on the patient, or possible impact. Also no mention of wider effects i.e. psychological, trust etc. Comment: The definition has been refined.

I 25. too wordy - there are 7 expressions (variables) listed Comment: The definition has been refined.

I 26. Patient impact should describe the result, not describe why it happened. The definition defines what led to a result, but not the result. Comment: The definition has been refined.

Page 40 of 194

I 27. too complex and encompassing Comment: The definition has been refined.

I 28. Too technical in its definition and confusing - need to read it a few times to get the point clearly. Put the patient first Comment: The definition has been refined.

I 29. Inclusion of the family/caregivers with the patient would be a useful model. Much like the healthcare system, the relationship between a patient and the family will have feedback loops and both tight and loose couplings-immediate consequences and those that will manifest later on. Comment: There has been much debate amongst the drafting group about this issue; the decision is not to include impact on family at this time.

I 30. the final phrase affecting the patient is ambiguous: it reads as though it qualifies "healthcare context" when I think it means The result of [an action] that affects the patient It does not specify what type of effect: presumably you are interested in health and well-being effects Comment: The definition has been refined.

I 31. ... affecting the patient. Does this include only medical affects? How about financial? Included? Comment: The definition has been refined.

I 32. It is Ok but could be simplified Comment: Agree. The definition has been refined.

I 33. Not clear if this would include mere exposure to hazard -- e.g., giving mismatched blood but no reaction occurs. Degree of impact is also a problem. Pt with AMI doesn't receive beta-blocker, recovers normally. But, probably has reduced myocardial muscle mass than expected if got beta blockers. Comment: The definition has been revised.

I 34. Patient impact should include something about what happened to the patient. I can't interpret the above. Comment: The definition has been revised.

I 35. It is not clear that the result is an outcome (i.e. vomiting from the procedure can be a result, but not an outcome. I see the outcome as harm (or potential harm), e.g. aspiration from the vomiting Comment: The definition has been revised.

I 36. Again brevity is important - either it is outcome - or it is impact - Not both. AS most of the terminology academically is about outcomes I would stick to that term. Comment: The definition has been revised.

I 37. Unclear if Patient does the impact or whether the impact is on the Patient - intuitively I know what the answer should be but language may create confusion. An impact may be focused on the patient but may or may not create a change in outcome - these should be separate issues. Comment: The definition has been revised.

I 38. The multiple subordinate clauses in the sentence make it choppy to read. Comment: The definition has been refined.

I 39. I understand patient impact or outcome as the effect of the particular event. Then why introduce a new definition of the event ("performance (or non-performance) of a function or one or more processes, services, or activities")? Comment: The definition has been simplified.

I 40. Again I would need to see list

Page 41 of 194

I 41. much too long and complicated Comment: The definition has been simplified.

I 42. It is not clear in the definition that it is the resultant effect on or in the patient Comment: The definition has been refined.

I 43. I think the definition would be clearer if "result" was replaced by "impact" Would also be analogous to the way an organizational outcome has been defined Comment: The definition has been refined.

I 44. definition stressed on personal factors and neglect the role of health/organization system, and economic impact Comment: This is now implicit in the definition of incident

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Patient Impact/Outcomes. II 1. see statement above: divide event type into "patient safety incident/error/risky healthcare act" and

patient outcome. Differentiate in a lower level: patient impact and organizational impact (delete organizational outcome next to patient outcome since the avoidance of patient harm comes first in practice and norms). Comment: The drafting group discussed this and determined that the conceptual framework should include both patient and organizational outcomes.

II 2. patient injury

II 3. suggest to include: aspects of the health facility building ..... Comment: This has been included elsewhere in the classification.

II 4. Why are the physical impacts so detailed? No definitions of how to categorise by severity; my judgement of severity may be very different as a patient to a professional care giver Comment: The concept of severity is addressed within the classification tree.

II 5. Don't know at present

II 6. "The result of the performance (or non-performance) of a function or one or more processes, services, or activities carried out by the staff and/or patient in a healthcare context affecting the patient." Comment: The definition has been revised.

II 7. "Patient Impact/Outcome" is appropriate here.

II 8. I suggest that the ICF definitional material on functioning is used here. That could be supplemented with words to cover other losses that could be incurred by the patient, e.g. financial, emotional. Need to have a note that there will not always be patient impact/outcomes (e.g. with a near miss) and that the impact/outcomes could be measured at different time points (e.g. immediate, after 24 hours, after 12 months). Comment: The drafting group discussed whether each aspect of harm can be recorded in sufficient detail to include the duration of each component. The concepts have been reviewed and refined accordingly.

II 9. Unsure

Page 42 of 194

II 10. Perhaps the clarity is subject to who will be expected to use this model. If regular healthcare workers are to be using it, then terminology needs to be clear, simple and concise. Comment: Agree. The definition has been simplified.

II 11. "The result of the identified event." Severity ratings within this category would allow for identification of patients who were not harmed by the event. Comment: Agree. The concept of severity is included in the classification.

II 12. The effect or result on the patient, from the performance or non-performance of one or more healthcare functions, processes, services or activities affecting the patient. Comment: The definition has been simplified.

II 13. Patient and family Impact / Outcomes.

II 14. Patient Impact/Outcomes - The result of performing a function or of one or more processes, services, or activities carried out in a healthcare context affecting a patient [I'm not sure here whether you mean "the context" is one that affects the patient, or "the result" affects the patient - I can't suggest alternatives without knowing that]. 2. Change "Impact/ Outcomes" to include some reference to harm or injury. Comment: The definition has been simplified and therefore clearer.

II 15. Would rephrase definition of Patient Impact/Outcome as 'the effect on a patient of one or more processes, services, or activities undertaken in a healthcare context.' Comment: The definition has been revised.

II 16. As above

II 17. Do you mean; "What happened to the patient?" Comment: Yes, the definition has been revised to reflect this.

II 18. The term is appropriate, the definition needs to be clearer. Comment: The definition has been simplified.

II 19. Client impact/outcomes Comment: Please refer to the definition for patient and the corresponding note. For purposes of the classification, the term patient encompasses “clients,” tenants,” and “consumers.”

II 20. The former Australian Council on Safety and Quality on Healthcare suggested that 'outcome' was 'the status of an individual, a group of people or a population which is wholly or partly attributable to an action, agent or circumstance'. MJA Vol 184 No. 10, May 2006 While this may not be a perfect definition, it illustrates the theme of my comments, that more work on details is required. Comment: This has been accounted for in the revised definition.

II

21.

Processes - give a few examples Services - as above in many countries these may be called one or the same thing Activities in particular needs further explanation Comment: With the revised definition, this is no longer an issue.

II 22. The result of the performance (or non-performance) of one or more processes, services, or activities carried out in a healthcare context affecting the patient. Comment: The definition has been revised.

II 23. In DK we use the term "consequences" in stead of impact. It works better in danish then impact Comment: We hope the use of the term “outcome” is ok for translators.

Page 43 of 194

II 24. The results, including worse case scenario, of this omission or commission, including not only physical harm but also psychological harm, damage to professional/patient relationship, and social impact. Comment: This is included in the definition of harm.

II 25. The consequence of dysfunction within a healthcare delivery process or system. Comment: We agree, but the definition is about the nature of the consequence.

II 26. The effect(s) on a patient of an event. This may range from no harm, through increasing degrees of harm to death. Comment: The new definition addresses this.

II 27. suggest separation of impact and outcome needs stratifying Comment: Impact is now a subset of outcome.

II 28. The impact or resultant outcome for the patient as a result of..... Comment: Please refer to the revised definition.

II 29. Patient and Family Impact/outcomes Immediate Intermediate Long-term Comment: There has been much debate amongst the drafting group; the decision is not to include impact on family at this time.

II 30. alter final phrase to ...that affects the patient's health or well-being Comment: This is included in the definition of harm.

II 31. Not possible, see my question above.

II 32. The result of the performance (or non-performance) of activities carried out in a healthcare context affecting the patient. Comment: Please refer to the revised definition.

II 33. see above

II 34. Again prefer the term adverse event and the IOM definition of what represents an adverse event

II 35. An additional diagnosis resulting from the performance (or non-performance) of a function or one or more processes, services, or activities carried out in a healthcare context affecting the patient. To me, an adverse outcome is something (i.e. an additional diagnosis)that requires (if possible), further intervention or alters the prognosis. Comment: Please refer to the revised definition.

II 36. Patient Outcome

II 37. Impact on the Patient or the Patient's Outcome

II 38. No suggestion at this time.

II 39. The result of the particularized event type.

II 40. Event type such as

II 41. The effect (positive or negative) of this action on this patient

II 42. The resultant effect upon the patient of the performance...........carried out in a healthcare context.

II 43. It would then read: " The impact upon a patient of the performance (or non performance) of a function or one or more processes, services or activities carried out in a healthcare context"

Page 44 of 194

II 44. Add: i.e. the result or impact of policy measures or health interventions in term of a change in health status or health behaviour or health cost. Comment: Please refer to the revised definition.

Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. Differentiation of outcome and impact makes sense: See statement above: 5) replace patient

impact/outcome by patient outcome 6) and place the box below the current event type. 7) Replace organizational outcome by organizational impact and shift below patient outcome. 8) Add patient impact on the same level. Comment: Nature, duration and severity are included within the definition.

III 2. A classification such as that proposed by Vacante-Vitez should be used to make it easier to use this part of the framework Comment: The reference is being located.

III 3. It is certainly important, in fact vital. However, surely the classification should say something about the effect on the patient ... not just say its a result of various processes. This is critical for drawing attention to the impact of these events on patients .. see earlier comments Comment: Agreed, the definition has been revised.

III 4. The ICD-10 classification system is obviously meaningful but, of the several thousand hospitals or other healthcare providers who use the CCS, not one has ever asked for patient outcomes to be classified in this way. I therefore query whether it is a useful element to include in the first version of the taxonomy. The outcomes our users capture are a great deal more specific - for example, neonatal encephalopathy in the case of obstetric incidents. The proposed severity classification is not sufficiently meaningful because it is too subjective (except in the case of death) and not useful as a component of a taxonomy for a large number of incidents because it does not compute. This means it is not possible to calculate aggregates or averages for a group of incidents. This is not acceptable for a standard taxonomy and it should be replaced with a numeric system. I have developed a good solution for this but there does not seem to be any provision for submitting it for consideration. The duration classes are unnecessary because they should already be accounted for in the assessment of severity. Comment: The drafting group has carefully considered several approaches to classifying severity and harm.

III 5. I've already answered the question once!

III 6. Same as earlier

III 7. Again, inclusion of the harm created on the family unit would be a more wholistic approach, again delineated chronologically as immediate, intermediate, and long term. Comment: There has been much debate amongst the drafting group; the decision is not to include impact on family at this time.

III 8. I discussed my issues with the patient impact/outcome in my prior response. Comment: There has been much debate amongst the drafting group; the decision is not to include impact on family at this time.

III 9. As with event type. Patient impact and outcomes should be either diagnosis coded with ICD (10 or 11) OR ICF. It is unnecessary to create a new classification for this purpose. Comment: Agree, there should be a connection with the ICD and ICF.

III 10. the word impact is meaningless in this context Comment: Impact is now encompassed under outcome.

Page 45 of 194

III 11. Again I would prefer to see coded outcomes here so that useful comparisons can be made. What if someone enters abdominal ache - is that pain or is it bloating etc - codes are important Comment: Codes will be developed.

III 12. impact and outcomes may be separate Comment: Impact is now a subset of outcomes and will come under severity.

III 13. not contain the role of health/organization system or cost effect Comment: This concept is contained under the organizational outcomes class.

III 14. The entire concept for this effort seems to be more of a research type endeavor with little real world use. Comment: We hope that it will have practical uses.

Additional Comments from e-mails The proposed severity classification is not sufficiently meaningful because it is too subjective (except in the case of death) and not useful as a component of a taxonomy for a large number of incidents because it does not compute. This means it is not possible to calculate aggregates or averages for a group of incidents. This is not acceptable for a standard taxonomy and it should be replaced with a numeric system. I have developed a good solution for this but there does not seem to be any provision for submitting it for consideration. The ICD-10 classification system is obviously meaningful but, of the several thousand hospitals or other healthcare providers who use the CCS, not one has ever asked for patient outcomes to be classified in this way. I therefore query whether it is a useful element to include in the first version of the taxonomy. The outcomes our users capture are a great deal more specific - for example, neonatal encephalopathy in the case of obstetric incidents. Comment: Dealt with above. Delphi Task Force Comments The feedback from Round 1 indicated the definition was confusing.

Page 46 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the revised definition for Patient Impact/Outcomes to be unclear. IV 1. Are the 'effects' mentioned restricted to physical ones, or will psychological effects be taken into effect as

well? Comment: This is addressed in the definition of harm.

IV 2. I prefer original definition as it encourages consideration of the context in which events occur Comment: The decision was made to revise the definition based upon the majority of comments provided.

IV 3. I think that the original definition is more clear. Comment: The decision was made to revise the definition based upon the majority of comments provided.

IV 4. It does not lend itself towards any specification either antecedent factors(by which it will have both Prognostic & Policy value)or characterization of consequences flowing from it(by which it will have Preventive & Therapeutic value) Comment: This is dealt with elsewhere in the classification.

IV 5. Use of ‘temporary’ vs. ‘permanent’ impact on patients is vague to coders without precise definitions. Comment: The drafting group carefully considered definitions for the concept of duration.

IV 6. patient safety event could mean many things. Comment: There is now a definition for incident.

IV 7. We need to define what kind of characteristics. Comment: This term is no longer contained in the definition.

IV 8. It seems that the Outcomes will only capture the immediate ones. Would these cover the long term outcomes? Comment: We do intend to capture long term outcomes.

V Please provide an alternative definition, the relevant reference(s) to the sources and your rationale for including the alternative definition instead of that provided for Patient Impact/Outcomes. V 1. Patient Impact/Outcomes are the physical and/or psychological effects of the patient safety event upon

the patient/family. Comment: These are now included in the definition of harm.

V 2. Original definition

V 3. See above

V 4. I tend to agree with the earlier definition for the reason cited above. I strongly feel that "Time" dimension need to be factored in to specify "Impact".

Page 47 of 194

V 5. Precise definitions are needed.

V 6. Adverse event

V 7. Patient Impact/Outcomes could be subdivided into Immediate and Delayed. Comments from the Question – Do you believe this is a meaningful and useful class? VI VI 1. Still unclear if psychological consideration will be given, and still unclear if inclusion of the harm to the

'family unit' will be taken into account. Comment: Please refer to the definition for harm and its corresponding note. Harm includes both physical and psychological harm.

VI 2. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or whether) it can be used. Comment: We hope the new definition will be useful.

VI 3. More precise definitions are needed to make results reproducible across coders. Also see comments in above. Comment: The definitions have been refined and examples will be included for the concepts contained in the classification.

VI 4. NA

VI 5. No - I think they are sibling classes that split into Outcome and Impact - Impact appears to be a measure of severity while Outcome identifies problems Comment: Agree – we hope the revised definition reflects this.

Additional Comments from e-mails None Delphi Task Force Comments None

Page 48 of 194

PATIENT CHARACTERISTICS Original Definition Patient Characteristics: The distinguishing features or traits specific to the patient. Revised Definition based on 1st Round of Delphi Results Not revised. Final Changes from December 2006 Drafting Group Meeting No changes were made to the definition. Patient Characteristics: The distinguishing features or traits specific to the patient. The “Patient Procedures” concept was moved from the Patient Characteristics class to the Event Characteristics class. Changes after Review of all Delphi Respondents’ Comments Patient characteristics: Selected attributes of a patient that support useful classification.

Page 49 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Patient Characteristics, or its associated definition to be unclear.

I 1. see separate text

I 2. acuity

I 3. had to answer no in order to provide a comment

I 4. Patient characteristics in part are properties of patients, in another part the involvement of patients into the process which leads to an event Comment: The characteristics intended here are properties of patients. Those patient characteristics which are actually specific to an incident are included under incident characteristics.

I 5. What is 'functioning'??

I 6. As previously noted, the

I 7. Please consider the proposal made in previous boxes concerning severity of patients state

I 8. The functioning is not defined clearly

I 9. This means nothing in Clinical or English language These are known as Patient Demographics Comment: Yes, these are patient demographics.

I 10. It needs to be made clear that the functioning that is mentioned here is the pre-event' functional status. It could be useful to use the term 'comorbidity' or some other term to indicate that 'pre-event' diagnoses would also be included here, and/or to make it clear that any 'condition' of the patient could be described in both ICD and IDF terms. The ICF 'severity' terms should also be used. I don’t think 'distinguishing' is a useful word here, as it is not clear what type of distinguishing would be relevant in this context. Basically, what is needed here is a simple statement that this is about characteristics of the patient, including demographic characteristics, and pre-existing conditions (whether described in terms of disease or functioning). It is important to recognise that these characteristics will be 'risk factors' for events and patient outcomes, so it could be useful to reconsider that wording in that context. It is quite incorrect to have 'procedures' in this category. Procedures are attributes of the event (and therefore should be in 'event characteristics'. They are not attributes of the patient. Comment: We have addressed this in the definition

I 11. I am not sure about what is meant with "functioning". A specific classification such as "MET" should be used. Comment: We believe this has been done

I 12. Again, I would not use "distinguish" here. Also, I would add more examples. Comment: The definition has been revised accordingly.

I 13. We don't all have patients. Comment: Please refer to the definition of patient and corresponding note. For the purposes of the classification, patients include “clients,” tenants,” and “consumers.”

I 14. While the meaning of the definition is fairly clear it is not really precise. Are age and sex 'distinguishing' features or demographic descriptors? And again 'functioning' is used. If this means functioning should be measured at different times (e.g. before intervention and after, as an outcome measure) is this said somewhere? Comment: Agreed, the definition has been simplified.

Page 50 of 194

I 15. more detail to the definition Comment: The definition has been refined.

I 16. It refers to all patients whereas it should refer to patients involved in an event Comment: This class and its concepts refer to the patient involved in the incident

I 17. doesn’t seem to include enough patient chars. Comment: The classification tree has been expanded.

I 18. Unless intended and simply not articulated, the culture/race of patients, as well as socioeconomic status, would be useful in addition to age/gender. Research has shown that not only are women and the elderly more prone to adverse events, but also visible minorities, and those with health Illiteracy issues are intimately affected by their relative socioeconomic status. Comment: The drafting group discussed inclusion of this concept at length and decided not to include it at this time because it can be a contentious issue in some countries.

I 19. Patient characteristics: does it include illnesses patient is suffering from? Comment: Yes it does.

I 20. What about language difficulties when neither patient nor provider speak same (or dominant at site) language. Comment: This concept has been added to the patient characteristics class.

I 21. the word characteristics is unclear. It is at odds with the notion of human "factors" as the major contributor to adverse events.

I 22. You gave examples, which I think are incomplete (e.g. diagnosis and comorbidity are very important): The distinguishing features or traits specific to the patient (e.g., age, gender, functioning). Also, it should be clear that the features are relevant to the event, e.g. obesity would be relevant to a retained sponge, but not to a medication error. Comment: These concepts are addressed under incident characteristics

I 23. Patients have baseline characteristics but their diseases have varying degrees of severity and/or acuity. Similarly, patient respond physiologically in different ways to the diseases and that depends on course or status of disease. Where an adverse event intervenes can then create different outcomes from the event. Multiple diseases create varying states as well. Comment: Agree

I 24. I think you should specify "individual" patient characteristics

I 25. Functioning - does this include underlying co-morbidities and demographics other than age and sex? Comment: Yes, this has been more clearly elucidated.

I 26. If certain traits are desired they should be explicitly stated. The use of the term 'etc.' is not clear at all and will not encourage uniform reporting of information. Comment: Agreed

Page 51 of 194

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Patient Characteristics. II 1. delete

II 2. include acuity

II 3. is this level of information of value - is WHO expecting to capture information from reporting and learning systems with this degree of detail? Comment: ICPS should be a potential tool with respect to safety. with “business rules” of how the classification will be used and different contexts will determine the level of detail that is to be captured by the reporting system. For example, if a recourse analysis is done, people may want this information, but would be unlikely to want it for a near miss.

II 4. at least mentioning patient related factors in the event modifier category

II 5. Difficult to provide an alternative without knowing what the word means.

II 6. Add "severity of illness" Comment: This concept is included.

II 7. To give an alternative a specific definition would be needed. it is possible that the definition would clarify in itself. Comment: The definition has been revised.

II 8. These are known as Patient Demographics Comment: Agreed

II 9. As noted above, what is needed is a simple statement that this is about characteristics of the patient, including demographic information and information on pre-existing conditions (whether described in terms of disease or functioning). Need to note that patient characteristics will not always be collected - e.g. if the event relates to a 'near miss' spill of a caustic substance on a floor of a ward. Comment: The definition has been simplified.

II 10. There are genetic variances in the population that are only (or mainly) discovered and cause harm when the patient come into contact with healthcare, such as atypical pseudocholinesterase or malignant hyperthermia. Comment: If things are fairly rare and not included in ICD, then they can be added as narrative in the reporting system. Information such as this will be collected and reviewed during field testing. The ICPS will be revised accordingly.

II 11. The identifying or notable... Comment: This concept/class is not intended to be used for identifying patients

II 12. Client Characteristics Comment: We have discussed the use of client under the definition of patient. Please see the definition of patient and its corresponding note.

II 13. Demographic and other features that describe the patient, and may be particularly relevant to the provision of or outcome from healthcare. Comment: These are included as concepts within this class. This is not intended to be particularly relevant to the provision of or outcome but merely to be selected attributes

Page 52 of 194

II 14. Distinguishing features specific to the patient including gender, age, heritage/culture, lifestyle, and/or health status. Comment: The definition has been clarified.

II 15. The distinguishing features or traits specific to a patient involved in a patient safety event. Comment: The definition has been revised.

II 16. what about anatomy? what about underlying health condition or natural history (e.g. degenerative arthritis making spinal levels unclear) what about obesity Comment: Agree, consideration of comorbid conditions is important and are included as concepts within this class.

II 17. Patient/Family Characteristics If we really want to know who is being harmed by the healthcare system, let's know as much as possible about them-not just their age and gender. Comment: Additional demographics could be added

II 18. see question above

II 19. x

II 20. patient factors

II 21. The distinguishing relevant features or traits specific to the patient (e.g., age, gender, diagnosis, comorbidities, functioning). Comment: See new definition

II 22. Patient Demographics, Disease-State Characteristics and Responses to Disease-State Comment: See new definition

II 23. individual patient characteristics

II 24. see above - wider description of definition is suggested

II 25. There needs to be an explicit list of minimum data elements required. Experience has shown that this information when collected in this manner to be of little real value. Comment: This will be available in the classification, and users can choose degree of detail.

Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. While the content of patient characteristics makes sense, a

III 2. unclear to distinguish from patient involvement

III 3. See previous remarks on patients' characteristics & status

III 4. The definition is not clear

III 5. already known as patient demographics Comment: The definition has been revised.

III 6. Information in this class may prove meaningful however corresponding provider information would also be required. For example, if we only know what language the patient spoke it is of no value unless we also understand what language the provider spoke.

Page 53 of 194

Comment: This area is very complex for an international classification because of the variety of languages possibly spoken. These details may be more appropriate in a local context.

III 7. This chapter of the taxonomy is certainly meaningful but thought needs to be given as to the extent to which it is useful to capture this information in relation to all incidents. Navigation through the taxonomy would be easier if these elements were only required when they were relevant. Our experience is that devising pathways to navigate through the taxonomy is at least as challenging a task as compiling it in the first place and I am not sure how much consideration has been given to this. Comment: Agree – that the structure should be sound. Ease of navigation may rely heavily on the reporting system setup.

III 8. patient characteristics can be a limiting factor to the sharing of information for learning and prevention due to the presence of identifiable personal information. Comment: The classification has been designed to collect de-identified information.

III 9. Would "traits" be specific to a "patient" or a patient within a group of diseases/conditions? Comment: This concept/class pertains to the specific patient involved. Aggregated data could be examined to look at specific patient characteristics associated with a particular disease condition. This is the value of the classification.

III 10. I don't see it is necessary as such. If the age or gender or any other patient characteristics are involved in the process of the event I would prefer them to be classified as contributing factors. Otherwise they are not important to know - at least not in our culture. I personally am more interested in factors I can change if needed - as acts and processes, contributing factors and event modifiers. Comment: The pertinence and value of this information is context specific. It should be collected nonetheless.

III 11. Same

III 12. Patient characteristics rarely influence whether an event actually occurs in the first place; rather they may influence Patient Outcomes and Organizational Outcomes. Comment: Agreed

III 13. Happy with class just not terminology. Within the model you refer to factors - would suggest 'patient factors' Comment: The terminology and definition have been refined.

III

14.

As mentioned previously, let's get as much characteristics about the patient as possible, including socioeconomic status (where possible) and culture/race that the person represents. Comment: Agree, however the drafting group has extensively discussed adding these concepts and has decided not to include them at this time.

III 15. needs a better definition Comment: The definition has been revised.

III 16. Unclear relationships with contributing factors and patient characteristics. Comment: We believe we have clarified the relationship. See the conceptual framework.

III 17. As with event types. Patient characteristics may be diagnosis (ICD), features in patient functioning or behaviour (ICF?) or straight measurements. This type of data should be collected with existing classifications or as measurement results. It is both more easy and more accurate than coding of the information with yet another system. Comment: Agreed, this is the intention

Page 54 of 194

III 18. believe the "definition" should include "behaviors" and note that the characteristics are both specific to the patient and pertinent to the event Comment: This is addressed under the contributing factors class.

III 19. see my comments

III 20. The entire concept for this effort seems to be more of a research type endeavor with little real world use. Comment: We hope this will not be the case. Collection of this type of information will be possible with the classification but not necessary in all contexts.

Additional Comments from e-mails This chapter of the taxonomy is certainly meaningful but thought needs to be given as to the extent to which it is useful to capture this information in relation to all incidents. Navigation through the taxonomy would be easier if these elements were only required when they were relevant. For example, the admission diagnosis is obviously relevant when the event type is a delay in diagnosis, less so when the event is an abusive patient in A+E. Patient procedures (III.D) is relevant when the procedure has not been properly carried out, but less so when the event is a delay in diagnosis. Our experience is that devising pathways to navigate through the taxonomy is at least as challenging a task as compiling it in the first place and I am not sure how much consideration has been given to this. Comment: We agree and this will be a challenge for those devising the actual software for collecting using the classification. Delphi Task Force Comments The feedback from Round 1 suggested concepts that should be included in the Patient Characteristics class. Therefore, the definition was not modified.

Page 55 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the term, Patient Characteristics, or its associated definition to be unclear. IV 1. In the ICF, functioning is NOT considered to be a characteristic of the person. It is the outcome of

interaction between a person's health condition and environment. 'Ability' or 'capacity' may be measured in a standard environment but functioning is still not a characteristic of the person. Functioning should be measured before and after the event. The difference in functioning will be a key indicator of the patient outcome. Comment: Functioning should ideally be collected both before as a patient characteristic and after as an outcome. This has been addressed.

IV 2. I think it should be clear that the term patient characteristics includes co-morbid diagnoses. Comment: The definition has been clarified.

IV 3. You didn't rather change many things as a result of Round 1. I think the active role of patients in medical encounter and care should be recognized, not only handicaps or impairments

IV 4. The issue isn't lack of clarity it's the likelihood that all the fields will be "fillable" with a reasonable effort. Much of the data will be irrelevant to understanding a large majority of the cases. Comment: Agree, but this is a reporting system issue subject to local business rules

IV 5. Patients procedure by the definition(the procedure employed by a healthcare professional which resulted in the patient safety event)should not be a "Patient characteristic" It should be either a Event characteristic in the same vein as "agent involved" or could equally fit the definition for "Event type" Clinical management --* treatment & procedures" Comment: The concept now has a slightly different focus. It is the procedure/diagnosis for which the patient is seeking medical attention.

IV 6. I prefere to define patients characteristics in details. V Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Patient Characteristics. V 1. definition modified to clarify above comment

V 2. Co-morbid diagnoses are missing from the patient characteristic diagram Comment: These have now been included as concepts in the classification tree.

V 3. NA

V 4. Simply remodel the patients procedure to Event characteristic & define it "Procedure involved" OR alternatively the definition for Event with direct impact on the patient, Clinical management, Treatments and procedures could equally refer to the concept "Patients Procedures" & therefore indicates it is incorrectly modeled or you are duplicating the collection of what is essentially the same information. Comment: Agreed

Page 56 of 194

V 5. The distinguishing features or traits specific to the patient demographics, functioning, admission/diagnosis, procedures and other characteristics Comment: The definition has been revised.


whether) it can be used. Comment: It is important to include these concepts in the classification. The reporting system can be designed such that completing these fields is optional.

VI 2. No, I would remodel patient procedure to event characteristic or event type, direct impact on patient, clinical management, treatments and procedures. Comment: This issue has been addressed. Patient procedures are now included under incident characteristics.

VI 3. Patient characteristics have a use of enrichment value to an academic or manager. However, again, front line staff want the basic message and as such patient characteristics (supreme variables in data collection) may obfuscate the message that requires to be delivered. It is second class information. Comment: It is important to include these concepts in the classification. The reporting system can be designed such that completing these fields is optional or required under specific circumstances..

VI 4. Again, too much information will make the classification too complex.

Additional Comments from e-mails None Delphi Task Force Comments Structurally, the concept “Patient Procedures” fits better under the Event Characteristics class than the Patient Characteristics class.

Page 57 of 194

EVENT CHARACTERISTICS Original Definition Event characteristics: The distinguishing features or traits specific to the event and the circumstances or facts that surround the event (e.g., care setting, phase of care, persons involved, time and date). Revised Definition based on 1st Round of Delphi Results Deleted (e.g., care setting, phase of care, persons involved, time and date). Event characteristics: The distinguishing features or traits specific to the event and the circumstances or facts that surround the event. Final Changes from December 2006 Drafting Group Meeting No changes were made to the revised definition. Event characteristics: The distinguishing features or traits specific to the event and the circumstances or facts that surround the event. The concept, “Patient Procedures”, was incorporated into the Event Characteristics class. Changes after Review of all Delphi Respondents’ Comments Incident characteristics: Selected attributes of an incident that support useful classification.

Page 58 of 194

1st Round Delphi Comments I Please provide an explanation for why you find the term, Event Characteristics, or its associated definition to be unclear.

I 1. see separate text

I 2. Is the 'phase of care' the phase where the event was first detected or the phase where the first step in the event occurred, or is it a multi-select field to reflect multiple successive areas where the event was perpetuated? Comment: This is the phase of care when the event occurred. Detection processes are contained in the class “Detection”.

I 3. Given that I think you mean "Event Type" to be "Admission Type" (**you are either an outpatient or in-patient in a clinical world***) "Event Type" would be Admission Details Comment: The definition has been clarified.

I 4. I think that it would be important to include (in the list of examples), the type of healthcare that was being delivered at the time of the event (e.g. surgery type). Again, I feel that the term 'distinguishing' is not useful, e.g. because this information will not usually be able to be used to distinguish between individuals. I do not think that the word 'facts' is useful here - not sure what it adds. It currently seems to imply that the circumstances are not 'facts'. And 'facts' and 'surrounding' could be taken to mean 'contributing factors. Better would be to state a requirement that the circumstances of the event are to be recorded. Comment: The revised definition should now address these issues.

I 5. Again, replace "distinguishing" Comment: The definition has been revised.

I 6. Quite long-winded; alternatives could be confusing. However, it is comprehensive. Comment: The definition has been revised.

I 7. I think it is a mistake to include provider characteristics in this concept, for the reasons noted previously. Secondly, will this term be broad enough to include the effects of constrained financial resources? This is often omitted from root cause analysis, but in my view that is inappropriate -- see e.g. the Bundaberg Hospital Inquiry in Queensland, Australia, in which the Commission of Inquiry concluded that this was one of the underlying factors causing the unsafe situation / review of provider qualifications and oversight of practice. Comment: These concepts and others related to external factors are contained within the Contributing Factors class.

I 8. Every experience by the patient could be an event. The term "event" is used in the framework to describe negative experiences by the patient - I think the use of the word "incident" might be more appropriate. The use of "patient safety incident" would also be appropriate rather than just event Comment: The term “Incident” is now being used.

I 9. I cannot find the definition of 'event' and so find it hard to make sense of 'event characteristics'. Comment: This issue has been addressed by adding the list of basic definitions at the beginning of the classification.

I 10. An explanation may be required for 'traits' as this is not a commonly used word. Phase of care is more commonly called stage of care. Comment: The definition has been revised.

Page 59 of 194

I 11. this is more about the demography of the event rather than the specific characteristics Comment: The definition has been revised to reflect this.

I 12. Events don't have traits. Complicated Comment: The definition has been clarified.

I 13. Factors that are longstanding may not appear to be 'associated with the event' but still may play a role -- especially if they develop slowly Comment: Agree, these should be included in the contributing factors class.

I 14. as per patient should use the term factors Comment: The definition has been revised.

I 15. Again, incomplete examples (need Who, What, When, Where, How) and need to indicate relevance. Event Characteristics – The distinguishing features or traits specific to the event and the circumstances or facts that surround the event (e.g., care setting, phase of care, persons involved, time and date). Comment: The revised definition addresses this issue.

I 16. The term' care setting ' is capable of a variety of interpretations some of which would not generate useful information. Does this mean ' primary care' 'the ambulance station' the floor next to the bed'? Similarly what does the term 'phase of care' mean? Comment: These concepts are contained under the concept “Phase of Care”.

I 17. see, my previous comment.

I 18. the definition is good in general, but important clue should be added.

I 19. Needs to be an explicit list not a definition that begins 'e.g.' Comment: The definition has been revised and the concepts are included in the classification tree.

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Event Characteristics. II 1. 'Phase of care where the event was initiated'

II 2. Please note that all these terms already - they just need to be applied here to the correct position

II 3. As above, important to include the type of healthcare that was being delivered at the time of the event. This could be surgery type, but could be just receipt of drug treatment, or receipt of 'bed-rest' care. Need to note that 'contributing factors' are not captured here. Nor are 'event modifiers'. Comment: The concepts are included within the classification tree for this class..

II 4. The identified or notable

II 5. Not sure

II 6. I suggest expanding the list of examples in the definition to include specific reference to financial factors, and dividing provider characteristics into a separate category. Comment: Specific financial information is out of scope. Provider information will be provided

Page 60 of 194

II 7. as above "incident" rather than "event". Comment: The term and definition have been revised.

II 8. See for instance the definition of 'incident' in the Australian medical indemnity collection: 'An event or circumstance resulting from healthcare that may have led or did lead to unintended and/or unnecessary harm to a person, and/or a complaint or loss'. See aihw.gov.au website and seek report on medical indemnity national collection. Also former Council (previously cited: an event is 'something that happens to or with a person'. Comment: The term and definition have been revised.

II 9. Alternatively leave trait out and just have features. Phase of care may need further explanation or be called stage of care for example: At what stage in the process of the patients care "preadmission", admission, pre-operative/pre-procedure" etc did the event occur. Where there other Comment: The definition has been revised.

II 10. event demographics

II 11. The factors and circumstances that surround the event (e.g., care setting, phase of care, persons involved, time and date).

II 12. x

II 13. event factors

II 14. Event Characteristics – The distinguishing relevant features or traits specific to the event and the circumstances or facts that surround the event and explain the fundamental who, what, when, where, and how of the event. Comment: The definition has been revised. The specific concepts are contained within the classification tree for this class.

II 15. Perhaps the terms 'location' or 'procedure' might work better

II 16. Each of these characteristics should be a class and the full definition of the event should be constructed by assigning values to each of these classes Comment: This is a valid argument, but would add complexity and confusion to the basic framework.

II 17. Add to the examples:-nosocomial infection

II 18. Needs to be built upon information gleaned form a thorough RCA. Without an standardized approach to RCA this is a low yield endeavor. Secondly, if standardized processes are employed for RCA then the extraction of information needs to be done by the group that seeks to roll up the data not by those submitting it. Comment: Reporting systems are “out there” and can be rendered more useful if the data can be collected in a structured manner. Root causes analyses are highly desirable, but may not be affordable for incidents that should nevertheless be reported.

Page 61 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. already gave reason in previous answer.

III 2. appears a bit to vague

III 3. Seems redundant when these are identified in the other fields.

III 4. Same

III 5. I don't feel it's that important. Should be covered by Contributing Factors?

III 6. Same as for patient characteristics - no problem with grouping but would suggest 'events factors' as referred to in other areas of the module.

III 7. I am unclear if this category would include such human factors considerations as 'ambient noise', lighting level, etc. If so, it is fine as is. If not, explorations and understanding about the environmental conditions existing at the time of an event can help to quantify and find trends in environmental conditions that seem to contribute to errors of either omission or commission. Comment: This is addressed under the contributing factor class.

III 8. Not sure how it differs from event type. There is potential for confusion with fine distinctions that are perhaps unnecessary, particularly when the emphasis we want is on the preventable characteristics of events, and that feeds back into contributory factors, which is potentially a black box at present. Comment: The definition has been revised. Further, the difference between these concepts has been addressed in the introduction.

III 9. Here might bee need for terminological systems or new classifications that would help registration of this type of data. Their registration as a part of a separate classification developed for Patient safety purposes does not sound practical.


III 11. The entire concept for this effort seems to be more of a research type endeavor with little real world use. Comment: The intention is for this classification to be able to be used both for capturing information for “real world use” as well as for research.


Page 62 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the term, Event Characteristics, or its associated definition to be unclear. IV 1. Not clear if this framework accommodates multiple/overlapping characteristics, for example there can be

many locations associated with an event. Even if it does, it is difficult to capture, and analyze by coders, and to interpret by users of the synthesized results. Comment: Agree. As long as the classification contains these concepts, the reporting system and business rules can be designed to address this.

IV 2. The event characteristic is clear however, if the event characteristic can involve an agent, it can equally involve a procedure if by definition patient procedure is the procedure involved. Comment: The definition has been clarified and the concepts contained in this class have been expanded.

IV 3. Incident is a more appropriate word than event for unforeseen occurrences to patients whilst receiving healthcare. Comment: The definition has been revised.

V Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Event Characteristics. V 1. In long run I don’t believe tick boxes to characterize events are necessary or useful.

Comment: The intention is to deconstruct the incident from narratives that are collected via various means (e.g. reporting system, root cause analysis, etc.)

V 2. don’t need to change definition - need to remodel patients procedure from Patient characteristic to either Event characteristic or my preference Event type, event with direct impact on patient, clinical management, treatments and procedures (my preference) Comment: Procedure has changed context so it relates to the procedure that the patient presents to the health service with.

V 3. Incident characteristics.

Page 63 of 194


whether) it can be used. Comment: The intention is for the classification to be able to be used both for capturing information for “real world use” and for research.

VI 2. In the long run I don’t believe tick boxes to characterize events are necessary or useful, and do not allow an accurate recreation of the event after time has passed. Also see comments above. Comment: The intention is to deconstruct the incident from narratives that are collected via various means (e.g. reporting system, root cause analysis, etc.)

VI 3. NA

VI 4. Still some work on subclasses and their definitions Comment: The classification tree for this class has been reviewed and refined.

VI 5. Incident characteristics would be more meaning full and easier for people to relate to Comment: The term and definition have been revised.

VI 6. It is meaningful and useful, but not for ICPS.


Page 64 of 194

CONTRIBUTING FACTORS Original Definition Contributing factor: A circumstance, action or influence which plays a part in the genesis of a patient safety event. Revised Definition based on 1st Round of Delphi Results Contributing factor: A circumstance, action or influence which plays a part in the origin or development of a patient safety event. Final Changes from December 2006 Drafting Group Meeting The definition was revised to include an element of risk - Contributing factor: a circumstance, action or influence which is thought to have played a part in the origin or development of an incident or to increase the risk of an incident. Changes after Review of all Delphi Respondents’ Comments Contributing Factor/Hazard: A circumstance, action or influence which is thought to have played a part in the origin or development of an incident or to increase the risk of an incident. Note: The qualifier “is thought to” was suggested by a respondent (see I 7) and deals with any implication that a course has been established. The use of the word “risk” (on its own, without “occurrence”) imparts the message that it is not only the chance of the “occurrence” that may be reduced, but also nature and/or severity of the harm. The fact that contributing factor is considered negative is implicit in the definitions of both incident and risk. The issue of the distinction of both active and latent factors is dealt with by circumstances or influences implying latent factors and actions implied by active factors. We have addressed the issue of the inclusion of the terms of “negligence” and recklessness” in the classification.

Page 65 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Contributing Factors, or its associated definition to be unclear.

I 1. clear, but had to say no to provide a comment

I 2. Difference to event modifiers Comment: The revised definition should clarify this.

I 3. "Genesis" of the event is too limiting. Comment: The definition has been revised.

I 4. I think that the use of the word "genesis" is not helpful in this definition Comment: The definition has been revised.

I 5. There's somehow an (apparent) overlap and/or dependency between 'contributing factors' and 'event type' (e.g. 'equipment factors' and 'resources-equipment' or 'infrastructure-failure malfunction'). these should be made more explicit. If my statement above does not apply, then better guidelines to distinguish between the two categories should be provided. Comment: Agreed.

I 6. not unclear but maybe incomplete? Comment: The definition has been revised.

I 7. I feel that the term 'influence' is not clear, and that use of the term 'circumstance' makes this definition overlap too much with that proposed for 'event characteristics'. Also, the wording 'which plays a part in the genesis' seems too rigid. Without properly constructed epidemiological studies, it is not easy to say that any factor plays a part, so it would be better to say 'thought to play a part' or similar. Comment: The definition has been revised accordingly.

I 8. words too complicated Comment: The definition has been clarified.

I 9. V.A. Human and Performance Factors V.A.1. The distinction between these three are subtle and frontline healthcare providers may find it difficult to discern amongst them. V.A.2. The distinction between detection and monitoring are again very subtle and the reliability of the data may prove problematic. V.A.3. Behaviour- I understand the need for this type of information however if the focus of a patient safety taxonomy is intended to assist in the identification of hazards in the healthcare setting the language we use will be very important in setting a tone. Currently the terms defined in this classification are not such that they would assist CPSI in our continued efforts to help shift the culture away from blame towards system issues. V.A.3.a. At-Risk Behaviour- consider revising as the term is somewhat inflammatory. V.A.3.b. Negligence- remove from the taxonomy as this is a finding for a courtroom. V.A.3.d. Recklessness- consider revising as the term is somewhat inflammatory. Section V.A. is extensive and broken down into a number of categories whereas Section V.B. is lacking in detail and explanation by comparison. Arguably, patient safety advocates would suggest that Section V.B. is as rich or richer in terms of identifying systemic issues requiring change in healthcare. Comment: These issues have been carefully considered. See the Basic Definitions and Introduction.

I 10. Contributing factors are very unspecific Comment: The concepts will provide the necessary details.

Page 66 of 194

I 11. Genesis may be confusing, if this is a document to be read by lay people or people with low literacy levels. Comment: The definition has been revised.

I 12. I appreciate that this cannot be a tight definition. However, one area that does seem to be excluded (by implication) is the patient themselves, who can also play a part in the genesis of an event. But perhaps patients are included under action

I 13. genesis -

I 14. Would be helpful to list examples as done with other definitions. Comment: This will be done for the concepts within the classification tree.

I 15. you are describing a contributing factor as a cause. Comment: Definition has now been changed not to imply this.

I 16. Genesis is commonly referred to as a biblical term and may offend cross cultures using this classification Comment: Agreed, the definition has been revised.

I 17. "genesis"...don't use that word Comment: Agreed, the definition has been revised.

I 18. "Factors" is pleural. The definition given is singular. Comment: This has been fixed.

I 19. Definition refers to only the "genesis" of the event Comment: The definition has been revised.

I 20. Factors is plural, definition is singular. Comment: This has been fixed.

I 21. Examples would be useful. And some 'hard questions' need to start to be answered. Is the capitalist economic model, with its emphasis on competition versus collaboration, a significant part of the problem? If so, what now? Comment: The example given has been included under “external factors”.

I 22. the problem of Hindsight bias is it raises its head here and can best be controlled by formal processes like RCA Comment: Agreed, and we thoroughly endorse RCA. However, an RCA can not be conducted in all circumstances for all incidents, and, even with training, hindsight bias can probably not be eliminated completely.

I 23. take out genesis, leave in just plays a part, use plain language wherever possible Comment: The definition has been revised.

I 24. We are not sure that the word "genesis" will be understood by all users, although we cannot think of a better word that captures the precise meaning. Causation would be a possible synonym but it is not quite right either Comment: The definition has been revised.

I 25. Unclear what is reakltion with this and patient characteristics. Is e.g. patient’s lowered intellectual level a circumstance? Comment: The revised definitions for both patient characteristics and contributing factors should clarify this issue.

Page 67 of 194

I 26. Contributing factors not only affect the genesis but also the development of the event and outcome Comment: The definition has been revised.

I 27. No stopping rule. How far back should the causal chain be run for contributing factors? Comment: This is outside the scope of a classification. Stopping rules have to be part of the “business rules” for acquiring the information before putting it into the classification.

I 28. This covers a vast array of factors. May be over-broad as a concept. Comment: We do not believe this concept is over-broad.

I 29. This is clear! The difference to preventive and recovery factors is clear from the wording "circumstance, action or influence" versus "effort or action" Comment: The definition has been clarified to address this.

I 30. Seems to preclude reference to equipment failure unless this is meant to be included in the term 'circumstance'. Is the degree of overlap between ' circumstance and influence intentional. Comment: Equipment failure is included as a concept within the classification tree in this class.

I 31. One person's contributing factor may be ignored by another. Comment: Agreed.

I 32. numerous factors can contribute - perhaps the listing of examples would be helpful. Human factors, systems factors, medication factors, allergic factors, response times, disease states, patient factors, family factors, etc.... Comment: Agreed, examples will be included.

I 33. See my previous comments

I 34. "... part in the genesis of a patient safety event" Since we often have a chain reaction and we cannot distinguish cause and circumstances modifying the course of the chain reaction we would prefer a definition with a broader scope including modifying circumstances. Comment: The definition has been revised to be consistent with this thought.

I 35. is a recovery factor also a contributing one? what is the difference between a contributing factor and an event characteristic? Comment: We hope the revised definitions clarify this.

I 36. Again give examples Comment: We intend to provide examples.

I 37. Consider expanding this definition to include factors that may not have contributed to initiation of the event, but may have influenced the severity of the patient outcome. For instance, an overdose of one medication may provoke a mild reaction while another drug causes a severe one. Comment: The revised definition hopefully addresses this issue.

I 38. Need a structured approach to RCA with standardized methodology Comment: We agree, but this is outside the scope of the classification and is at the level of the acquisition of information from the real world.

I 39. I'm happy with the word 'genesis' but we think it probably requires a higher standard of english that all potential users of this taxonomy. Comment: The definition has been revised.

Page 68 of 194

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Contributing Factors. II 1. this requires a degree of sophistication on behalf of a reporter in order to appropriately understand the

factors and ascribe them to the event. Again, if these are going to be recommended as part of a reporting and learning system, this will result in highly complex and time consuming reporting what do the small numbers and letters refer to? Comment: We agree that a granular comprehensive classification system is being constructed, but this is necessary if it is going to be useful. Alpha numeric codes should be a part of the software and should not be visible.

II 2. ?

II 3. Please consider something like "genesis or increased severity". "Mitigating factors" are covered independently, but there may be contributory aspects that worse a case, too. Comment: The definition has been revised to address this.

II 4. Suggest replacing with the word with "development" to read - A circumstance, action or influence which plays a part in the development of a patient safety event Comment: The definition has been revised.

II 5. no answer

II 6. maybe include lack of action, omission; often the policies are there but there is a huge people's component (healthcare staff)influencing the implementation of these policies. Comment: The concept for failure to comply with policies is included as a concept under this class.

II 7. As noted above, need to delete the word 'influence' and add words to indicate that the contributing factors are those 'thought' to have contributed - i.e. acknowledging the uncertainty in processes such as RCA. Comment: The definition has been revised.

II 8. a more commonly used word than genesis Comment: The definition has been revised.

II 9. Consider: Rather than At-Risk Behaviour- Utilize the phrase "departure from a standard practice". Comment: The terminology for this concept is being reconsidered based upon this and other comments.

II 10. I suggest to include contributing factors associated with patient, healthcare organization, health personnel. Comment: The contributing factor class includes these concepts.

II 11. A circumstance, action or influence which leads to/initiates . Comment: The definition has been revised to reflect this.

II 12.

II 13. A circumstance, action or influence which was identified to have directly contributed to the patient safety event. Comment: The definition has been revised.

II 14. See above

Page 69 of 194

II 15. either call the cause the cause or re-define contributing factors; if a factor contributed to an event, did it cause it? Comment: The revised definition clarifies what we mean by “contributing factor”

II 16. Genesis should be replaced with development or unfolding of a patient safety event Comment: The definition has been revised.

II 17. use beginning, start or other synonym Comment: The definition has been revised.

II 18. The circumstances, actions, influences which MAY lead to this /these patient safety events Comment: The definition has been revised to reflect this.

II 19. A circumstance, action or influence which plays a part in the genesis and/or development of a patient safety event Comment: The definition has been revised.

II 20. Contributing Factor - A circumstance, action... Comment: The definition has been revised.

II 21. It's important to look at societal characteristics, and not simply the immediate geographic area around a patient safety event, to determine what really is contributing to iatrogenic harm. Comment: We anticipate that the classification will handle this.

II 22. don't have one - just the caution

II 23. as above

II 24. Alter genesis to another word (? causation) Comment: The definition has been revised.

II 25. see question

II 26. Any situation or influence which plays a part in the initiation, development or outcome of a patient safety event. Comment: We believe that the revised definition deals with this suggestion.

II 27. x

II 28. I would suggest possibly a sub classification here that would guide thinking. Comment: We believe the classification will accomplish this.

II 29. Contributing Factors

II 30. would it help to add 'practical' circumstance?

II 31. Are there areas where one must look for contributing factors? Comment: This is an issue to be addressed in the business rules, rather than the classification itself.

II 32. Issues seems to me to reflect the nuance better than factors - Contributing Issues. Perhaps this is just stylistic Comment: Contributing factors is a commonly used phrase.

II 33. See my previous comments

Page 70 of 194

II 34. A circumstance, action or influence which plays a part in the genesis of a patient safety event or has an impact of its course. Comment: The definition has been revised.

II 35. see my previous comment

II 36. Contributing factors such as

II 37. None

II 38. Need a structured approach to RCA with standardized methodology Comment: Agree, but believe this is outside the realm of the classification and is rather to do with the business rules describing how information is collected.

II 39. ?use contributing Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. unclear to distinguish from event modifiers

Comment: The revised definition clarifies this.

III 2. already gave this reason - a term already exists it could be co-morbidity or Clinical feature

III 3. As noted above, I am not sure what type of information you are after here. However, in general terms, the identification and assignment of 'contributing factors' is an inexact science and there are multiple variations on what is considered to be a useful categorisation with sufficient detail to be of use, but still able to replace full text information. The proposed classification in the main document is not mutually exclusive and would be very difficult to use. The inclusion of 'patient factors' should be reconsidered. As listed, they are 'pre-existing conditions' and could be better regarded as 'patient characteristics'. (Wherever they are located, they should be described using both ICD and ICF.) This issue of patient factors really brings to the fore the notion of what are considered to be 'risk factors' for the events/outcomes. To some extent, it does not matter whether they are called 'contributing factors' or 'patient characteristics' provided that they are recorded somewhere in the framework (but only once) and the data are available for analysis. Comment: We agree that patient factors should be able to be included as contributing factors to a particular incident. The classification discriminates between pre-existing co-morbidities and conditions and patient factors which have a direct bearing on the incident in question.

III 4. There can be confusion about what is a "contributing factor" or an "event characteristic". Comment: The revised definition clarifies this.

III 5. This section of the taxonomy would benefit from further development. If the aim is to point the way to changes in the healthcare system that might reduce the number of patient safety incidents, it follows that ICPS must identify "system" causes as well as idiosyncratic causes. At present, this section is populated mainly by patient factors and human or performance factors on the part of the clinician and the balance should be redressed in favour of "system" causes. I have recently analysed a typical cross section of 230 claims and shortcomings in clinical governance processes were implicated in more than half of the serious patient safety incidents that give rise to them. However, most of the relevant processes are not mentioned at all in ICPS, despite the fact that many of them could be remedied in quite obvious ways, albeit at some expense. This may be done by including some of the clinical governance processes which are apt to cause patient safety incidents when they fail. There are about two dozen of them and they relate to such matters as making adequate arrangements for known higher risk patients, someone taking overall responsibility for the patient, the credentialing and induction of new staff. So far as the Human Factors are concerned, I am not sure that the distinction between knowledge based, rule based and skill based factors will be easily learned or applied to an initial classification of the data. The other human and performance factors identified are either too broad (e.g. communication, documentation) to point the way to feasible reforms of the healthcare system, in which case they need to be broken down into more

Page 71 of 194

specific processes, or too cryptic, in which case they should be clarified or abandoned. V.B Work environment factors are all right. V.C. Patient factors - of the several thousand healthcare providers who use our classification system, not one has ever asked for patient factors to be classified in this way for the purpose of incident reporting and I query whether this is a useful initial category. Perhaps it could be deferred to some subsequent release. V.D. Organizational factors - These need to be defined with greater precision and this section of the taxonomy should be enlarged to encompass the organizational processes which are apt to cause patient safety incidents when they fail. The same is true of V.E. - External factors. Comment: We agree that the system and organizational issues need to be accommodated in this section of the classification. We hope we will have categories relevant to the issues that you raise about overall responsibility, credentialing and induction of new staff. We agree that it is unlikely that most reports will be able to discriminate between the various types of errors and that an overall problem with a classification like this is achieving some balance between ease of use and granularity. Although you question the validity of having patient factors here, we believe it is important and this has been emphasized by other respondents to the Delphi process. We agree that organizational and external factors need to be able to be classified in some detail.

III 6. Same as earlier

III 7. If the plan is to include considerations around teamwork, conflict styles, profit or not-for-profit status of a healthcare system, this will be more than adequate. Contributory factors are far more insidious than 'inadequate documentation'. The question why a person chose to document in a certain way that may or may not have contributed to the event should be understood: there can be a perfectly 'good' reason why 'adequate documentation' was not completed-maybe none of the staff in question document in the 'adequate' manner. And who is defining 'adequacy'? The patient? The nurse? The risk-manager? The malpractice lawyer for patients? Comment: We agree with your comments and reservations and agree that documentation should be able to be described with considerable granularity. Who decides what should be a contributing factor and what constitutes “adequate” is a major philosophical question for who should be acquiring information and who should be classifying it.

III 8. Unclear relationship with contributing factors and patient characteristics Comment: We believe the revised definition has clarified this.

III 9. This class is crucial. My worry is that everybody will expend their energies on fine distinctions to do with patient and event characteristics, and forget that the crucial outcome out of all this is how to prevent recurrence. Comment: We agree with your point and although it is often the case too much time is spent on gathering information rather than on fixing problems, data collection, and analysation are essential for learning.

III 10. It is too broad a category. Comment: Since this is a “high level” or “parent class” it is intended to be broad and stable.

III 11. As event characteristics

III 12. perceptions of contributing factors will vary from individual to individual and from organization to organization Comment: We agree, and this is central to the balance between validity and liability.

III 13. See my previous comments


III 15. Without standardized approaches and methodology this is of little use. The entire concept for this effort seems to be more of a research type endeavor with little real world use. Comment: We agree but to have a useful body of information, there should be standardized approaches to eliciting information from the real world and putting it into a form where it can be stored, aggregated, compared, and analysed. The issue is the balance between validity and reliability in deconstructing real world incidents into what we hope are useful components or concepts.

Page 72 of 194

Additional Comments from e-mails None Delphi Task Force Comments The following concerns were raised by the feedback from Round 1: 1. The word “genesis” in the definition: Respondents noted the word “genesis” may have different connotations

and meanings in different countries. Respondents also indicated that contributing factors can influence a patient safety event at any time.

2. The interplay between Contributing Factors and the Event Modifying classes (Preventive Factors, Recovery

Factors And Mitigating Factors) 3. Causation versus Contributing Factors 4. Inclusion of particular concepts within the Contributing Factors classification tree As a result, the word “genesis” was removed from the definition of Contributing Factors to be sensitive to cultural and linguistic issues and to indicate that a contributing factor can play a part in a patient safety event at any time. The drafting group discussed the concepts of “causal” versus “contributing” factors at length while designing the ICPS. The decision was made to focus on “contributing” factors because it was more encompassing. Further, it is not possible to determine causation at the actual time of the event; additional investigation (e.g., root cause analysis) is required. Comment: We believe that these have been dealt with in the new definitions.

Page 73 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the revised definition for contributing factor to be unclear. IV 1. I actually do find it clear, but think that all the "negative" factors need to convey the concept of

"increased risk" or increased probability of having an unintended outcome. Comment: This has been addressed in the new definition.

IV 2. A stronger connection to the analysis models/tools by J.Reason or Charles Vincent (as used in root-cause-analysis) would be useful. Comment: We agree a connection is valuable. An examination of the underlying information model will reveal that it is actually based on the complex system of failure model of Reason, Vincent and Rasmussen. Root-cause-analysis, is more to do with a discipline of a collecting information than it is to the discipline of classifying it.

IV 3. It is too vague and not truly descriptive.

IV 4. Not convinced that Eindhoven root cause method has demonstrated its effectiveness. Eindhoven includes objectionable categories such as ‘beyond control of organization’, ‘recklessness’, and ‘negligence’. Comment: The Eindhoven classification is characterized by having stopping rules and being kept simple for use in many industries at the coal face. Our classification will have more granularity but this does not preclude an Eindhoven type of approach being used at the operational level has demonstrated its effectiveness in a variety of settings including healthcare.

IV 5. plays a part is not a scientific phrase Comment: The definition has been revised.

V Please provide an alternative definition, the relevant reference(s) to the sources and your rationale for including the alternative definition instead of that provided for contributing factor. V 1. "... which increases the risk or probability of a patient safety event."

Comment: The revised definition includes risk. The definition of “risk” includes probability.

V 2. s. above

V 3. A contributing factor is an underlying circumstance or condition that permits or promotes the occurrence of a patient safety event. (This definition is clearer and allows the inclusion of a broad range of factors.) Comment: The definition has been revised to be consistent with these thoughts.

V 4. Operationally test to show evidence of relevance for this task. Comment: The system will be field tested

V 5. A contributing factor is a circumstance, action or influence which is positively associated with the origin or development of a patient safety event Comment: The definition has been revised to be consistent with this.

Page 74 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? VI VI 1. This class is meaningful however the terms remain concerning. Specifically the third tier of information

collected through the framework such as the distinction between knowledge, rule and skill based factors and the use of the terms negligent and reckless. I understand the need to collect patient safety information from a broad spectrum however the temptation to designate behaviour as negligent or reckless in the absence of evidence may prove harmful to individuals and/ or organizations. Comment: We acknowledge the limitations of observers discriminating between different types of errors. These are included for the sake of completeness but we understand the problems in terms of classifying behaviour in this way. We agree with the need to try to use non-judgmental terms but the fact is, there is some behaviour that is clearly reckless. We wish to avoid the term “negligence” as it has a particular legal meaning and your comments are relevant to the whole debate between accountability and learning systems and the need for information of this type being gathered in a manner which is legally protected from discovery or dissemination.

VI 2. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or whether) it can be used. Comment: We agree. Some aspects will be hard to use and will have a degree of uncertainty associated with them. However, we believe collecting large volumes of information is still valuable.

VI 3. Eindhoven needs to have more extensive operational testing to show evidence of relevance for this task. Also see comments above. Comment: The Eindhoven Classification Model has been thoroughly tested in a number of fields including healthcare.

VI 4. Still some work to be done on the subclasses and their definitions Comment: The concepts contained within this class have been reviewed and refined.

VI 5. as previous answer Comment: The concepts contained within this class have been reviewed and refined.

Additional Comments from e-mails None Delphi Task Force Comments Since Contributing Factors are considered to be negative, the task force discussed revising the definition to include an element of risk. Although only one respondent commented on the issue including Negligence and Recklessness in the classification, this has been a recurring topic of concern in the US. The fear is that including these concepts will deter reporting and perpetuate the “blame and shame” culture. However, to have a complete classification, these concepts should be included. Comment: Risk is now in the definition. The issue of the use of the terms of “negligence” and “recklessness” has been addressed in the comments above.

Page 75 of 194

ACTIONS TAKEN NOTE: After review of the Delphi comments below, the Actions Taken class has been deleted. As shown in the responses to comments below, these important comments are now addressed or have been integrated into one of the other nine high level classes. See the Introduction and Basic Definitions. Original Definition Actions taken: Something done by an individual or the health are organization to prevent, remedy or mitigate the occurrence or recurrence of a real or potential patient safety event. Revised Definition based on 1st Round of Delphi Results Actions taken: Something done by an individual or the healthcare organization to prevent, remedy or mitigate the occurrence or reoccurrence of a patient safety event. Final Changes from December 2006 Drafting Group Meeting The definition was revised - Actions taken: Steps taken to reduce the risk of a similar incident. The conceptual framework was also revised to clarify the relationship between Actions Taken, Recovery Factors and Mitigating Factors. The actions taken as a direct result of the patient safety event were separated out from the Actions Taken class. The Actions Taken class was renamed to “Actions Taken to Reduce Future Risk”. The concepts in this class were narrowed to focus on organizational actions taken as a result of the patient safety event to inform the development of strategies to reduce the risk of the patient safety event from reoccurring (e.g., the learning cycles). Changes after Review of all Delphi Respondents’ Comments Comment: We agree there was confusion between the “factors” classes and substantially changed their structure, definition and concepts. The term and definition have been revised accordingly. Detection: an action or circumstance that results in the discovery of an incident.

Page 76 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Actions Taken, or its associated definition to be unclear.

I 1. not the term itself, comment on the diagram

I 2. This is OK as a definition, except that 'a real or potential' should be deleted. They should be incorporated into the definition of 'patient safety event', and can therefore be regarded as applying to all components of the framework. Comment: The definition has been revised to address this.

I 3. VI.B. Organizational Actions Consider utilizing the VHA NCPS categories for this section as a large number of organizations use the VA NCPS root cause analysis model or an adaptation of it for reviewing incidents (including Canadian organizations). Additionally consider adding categories related to governance, credentialing and competency. VI.B.2. CPSI would suggest that Organizational Culture is not an action taken, rather that it is the result of actions taken or not taken. Comment: Thanks – we will consider. Agreed that Organizational Culture is not an action.

I 4. Definition is clear but see below for suggestion

I 5. remedy

I 6. From the flow chart, actions are only taken after the occurrence of a real patient safety event. Comment: for the revised definition accounts for this.

I 7. Would this include disclosure of the event? Done poorly, the arrow might likely be pointing at another event type in creating more harm for the patient and family by a person ill-equipped for such difficult conversations. Comment: The concepts contained within this class have been refined and expanded to include this.

I 8. Depends on the 'safety event' definition: see above

I 9. Time is missing. Actions taken in the moment to prevent / reduce immediate risk/harm? Or sometime after the incident, to prevent the next incident. Comment: We have included two new clauses which are contained within – one to make good the harm and another to prevent further incident. Reducing harm falls under mitigation factors. Temporal relationships have been clarified by refining the terms and definitions for this class as well as for mitigating factors. Please refer to the definitions at the beginning of this document.

I 10. "Preventing occurrence" and "Preventing re-occurrence" are very different forms of action. I would not lump them. Comment: Agreed, these concepts have been separated.

I 11. seems to preclude actions needed across organizational barriers Comment: The concepts contained within this class have been reviewed and revised to address this and other issues..


Page 77 of 194

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Actions Taken. II 1. Again, if this infers that such data will be collected through a reporting & learning system, then the time

of reporting will impact on the nature of the response to actions taken - immediately after an adverse event, there may have been no investigation nor actions taken to prevent future events. Clinical procedures to prevent a future adverse event - what might these be? Improved protocols, better training & supervision, use of better equipment? Where might these issues be described? Comment: These incidents may be reported via one of several data sources including reporting systems and RCA’s. We acknowledge that there may not be enough information in all incidents to complete the information. The concepts in this class have been reviewed and expanded to account for this.

II 2. Delete 'a real or potential'. Comment: The definition has been revised.

II 3. VHA NCPS categories include the following: Communication Training Fatigue/scheduling Rules/Policies/Procedures Barriers Equipment/environment Comment: Thanks, we will review.

II 4. Might be worth making this `individual, team or organization' as many remedial actions are at team level

II 5. detect, prevent, mitigate

II 6. Thing(s) done by an individual or a healthcare organization to prevent, remedy or mitigate the likelihood of recurrence of a patient safety event.

II 7. Term is fine, just 'flesh it out' a bit.

II 8. See above

II 9. x

II 10. Maybe separate the two actions: Preventing occurrence and preventing reoccurrence. Comment: The conceptual framework class definitions and key concepts have been reviewed and revised accordingly.

II 11. add an 's' so that it reads organization(s)


Page 78 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. Perhaps the items in this class could/should be incorporated into Preventive, recovery and mitigating

factors? there seem to be at some points an unclear overlap between them. Comment: Agree, the conceptual framework and concepts have been reviewed and revised accordingly.

III 2. Making the classes for the actions will be difficult III 3. this seems ok and may be a phrase in a clinical care pathway but each words should stem from the first -

like in a schema Comment: We are trying to make the concepts within the classes hierarchical which is similar to your suggestion. The structure of this classification has been revised to hierarchical.

III 4. Actions instituted instead of actions "taken", in order to change or prevent something you don't "take" you institute or perform. Comment: The definition has been revised.

III 5. Chapters VI-X are too big and every effort should be made to reduce them VI. Actions taken - The clinical action section is probably unnecessary as in most cases it will be no different from the procedures which were wrongly carried out in the first place. The organizational actions in V.1.B. may be difficult for people who report incidents to complete because they will not normally know of them until some considerable time after the event. The definition offered of Actions Taken does not aptly describe the corresponding part (Section VI) of the taxonomy and it appears in large part to duplicate Parts VII, VIII and IX relating to Preventive, Recovery and Mitigating factors, which also appear in the next three definitions below. A better definition, bearing in mind the actual content of Section VI, might be "A clinical, organizational or other action taken in consequence of a patient safety event".

III 6. Same III 7. Listing a 'risk management review' as an action taken is completely 'organizational-centric'. I would prefer

the use of a safety review as the moniker. Risk management frameworks look to protect the assets of a given organization or group of organizations. They have a fundamentally different outlook than patient safety investigations. Also, 'changing document formats' is another 'soft' way of appearing to do something while accomplishing nothing-oh look we killed someone with an overdose of chemotherapy, I guess we better redesign the dosing form. Sorry, but that's just not good enough. Comment: The concepts contained within this class have been revised and expanded to be more comprehensive and patient centered.

III 8. in some cases may be very organization dependent and so not useful in other contexts Comment: The concepts within this class have been revised to be more comprehensive.

III 9. I think the data obtained by the reporting system will be difficult to interpret. It may not be available if the reporting system takes reports early in the process between the event and the longer term management. Comment: These incidents may be reported via one of several data sources including reporting systems and RCA’s. We acknowledge that there may not be enough information in all incidents to complete the information. The concepts in this class have been reviewed and expanded to account for this.

III 10. "Prevent," "remedy," and "mitigate" are different enough actions that they ought not to be lumped. Also, some organization actions are generic approaches to safety improvement, and need a place in the framework. Comment: The conceptual framework and key concepts have been revised.

Page 79 of 194

III 11. Actions taken must be interventions that should be recorded with (existing) interventions classifications. If suitable codes are not available they should be developed as part of the development process of the interventions classifications. Comment: Agree, the definitions for the concepts contained within this class have been revised accordingly.

III 12. Actions may also be taken by teams and at the system / sector level. Comment: Agreed. The definitions for the concepts contained within this class have been revised and additional concepts have been added to address this.

III 13. actions should be directly related to the taxonomy of contributing factors. it should not be different Comment: Agreed. We have revised this section using the Contributing Factors as the basis of a starting point and determining the factors that would deal with the contributing factors.

III 14. See my previous comments. Again, macro level is relevant. Additional Comments from e-mails The definition offered here does not aptly describe the corresponding part (Section VI) of the taxonomy and it appears in large part to duplicate Parts VII, VIII and IX relating to Preventive, Recovery and Mitigating factors, which also appear in the next three definitions below. A better definition, bearing in mind the actual content of Section VI, might be "A clinical, organizational or other action taken in consequence of a patient safety event". Actions taken - The clinical action section is probably unnecessary as in most cases it will be no different from the procedures which were wrongly carried out in the first place. The organizational actions may be difficult for people who report incidents to complete because they will not normally know of them until some considerable time after the event. Delphi Task Force Comments The feedback from Round 1 suggested clarifying what concepts are contained within the Actions Taken class. The definition was modified by deleting the extraneous words “real or potential”. Comment: We agree there was confusion between the “factors” classes and substantially changed their structure, definition and concepts.

Page 80 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the revised definition for Actions Taken to be unclear. IV 1. I sense that the actions are really two separate definitions: #1 is the patient intervention to attempt to

prevent, recover from, or mitigate harm, which could be part of those three classes. #2 is the organization action, which is really after the fact and the true "action" taken to redesign the system to minimize the chances for the error to happen again. Comment: Agree, the concepts and structure have been reviewed and revised.

IV 2. The inclusion of potential or actual reinforces the necessity to be alert to possible hazards. Comment: The definition has been revised.

IV 3. Actions are not measured for impact in this framework or definition. Comment: True, this will be assessed/evaluated during field testing.

IV 4. I think original definition was clearer than the new one. Comment: The definition has been revised.

IV 5. The definition is clear but the analyses and reviews and decisions that need to be done and made before the action are missing from the flowchart that connects the "classes." Comment: These processes will be addressed in the business rules.

IV 6. same as earlier... I don't like "patient safety event" Comment: The definition has been revised.

IV 7. Definition is clear enough - it might be useful to state recover from to accommodate the 3 moderating factors Comment: Agree, the conceptual framework and concepts have been reviewed and revised.

IV 8. Is it "belt and braces" to specify the context of the action? E.g., a corrective or recovery action? Comment: Agree, the conceptual framework and concepts have been reviewed and revised.

Page 81 of 194

V Please provide an alternative definition, the relevant reference(s) to the sources and your rationale for including the alternative definition instead of that provided for Actions Taken. V 1. I recommend putting the patient actions in the "response" classes of prevention, recovery, and mitigation,

and making it clear that the definition of action refers to something done by the organization to prevent, remedy or mitigate the reoccurrence of a patient safety event *by improving the system to make it more highly reliable.* Comment: Agree, the conceptual framework and concepts have been reviewed and revised.

V 2. something done by an individual or the healthcare organization to prevent, remedy or mitigate the occurrence or reoccurrence of a potential or actual patient safety event.

V 3. Need to close the framework’s loop with the determination of action effectiveness. "done and measured for effectiveness" Comment: Evaluation is not part of the classification although its merit is indisputable.

V 4. The original definition was correct V 5. NA V 6. "adverse event" is better V 7. An action taken is something done by an individual or the healthcare organization to prevent, remedy,

recover from or mitigate the occurrence or reoccurrence of a patient safety event. Comments from the Question – Do you believe this is a meaningful and useful class? VI VI 1. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or

whether) it can be used. Comment: The conceptual framework and concepts have been revised for clarity and usability.

VI 2. Preceding steps (analysis, review of previous actions and policies, etc) are not shown. Comment: These are business rules, or are organization specific or data source specific.

VI 3. Still work to do on the subclasses and definitions Comment: The concepts in this class have been reviewed and revised.

VI 4. too much info,Hx,Exam,tests,then treatment?? Too complicated. Its not Harrissons? Comment: The conceptual framework has been revised for simplicity.

Page 82 of 194

Additional Comments from e-mails None Delphi Task Force Comments The questions discussed were:

1. What is the difference between recovery and mitigating factors and actions taken? 2. Aren’t recovery and mitigating factors actually actions taken as a result of the event? If so, should the

Actions Taken class be retained? 3. Should there be a concept for patient management type actions taken (e.g., communication)?

Page 83 of 194

BEFORE EVENT (PREVENTIVE) FACTORS NOTE: After review of the Delphi comments below, this class was revised and retermed to Actions Taken to Reduce Risk. The revised conceptual framework is an endeavor to reduce the substantial confusion surrounding the intent and meaning of this class. See the Introduction and Basic Definitions at the beginning of this document. Original Definition Preventive factors: An effort or action taken in advance to pre-empt an occurrence of a patient safety event. Revised Definition based on 1st Round of Delphi Results Not revised. Final Changes from December 2006 Drafting Group Meeting The term and definition were revised – Before Event (Preventive) Factors: An effort or action taken by an individual or organization in advance to pre-empt an occurrence of a patient safety event.

The conceptual framework was revised to clarify the role of Preventive Factors and to illustrate its place in the learning cycle. A Key Definitions page was added to the ICPS. Included is a paragraph describing Preventive Factors: Before Event (Preventive) Factors: efforts or actions taken by an individual or organization in advance to pre-empt an occurrence of a patient safety event. Preventive factors are the defenses or barriers which are built into the system to counteract or avert the occurrence of a patient safety event. In the ICPS, preventive factors are inversely related to contributing factors (e.g., the management of a particular contributing factor or assessment of risk vulnerabilities can prevent a patient safety event from occurring). Changes after Review of all Delphi Respondents’ Comments Comment: We agree there was confusion between the “factors” classes and substantially changed their structure, definition and concepts. The term and definition have been revised accordingly. Actions Taken to Reduce Risk: An action taken to manage or control the probability of, or harm associated with an incident.

Page 84 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Preventive Factors, or its associated definition to be unclear.

I 1. I don't know what you mean. Preventative Factors I think could be Co-morbidity (e.g smokes) or Significant Clinical Feature (e.g history of MI) Comment: We now have tried to express Preventive Factors in terms of trying to reduce the risk of a similar incident. Most of the factors relate directly or indirectly to the contributing factors.

I 2. I think it would be better to describe this in terms of 'increasing patient safety', rather than in terms of 'in advance, to pre-empt an occurrence', to cover factors such as hospital policies on use of acronyms - which apply to all sorts of actual and potential patient safety events. The wording should be more closely linked to the concept of 'contributing factors' and 'primary' prevention. Comment: Agreed. We have endeavored in the latest draft to link to contributing factors.

I 3. Preventive factors might be underlying structural factors or processes in addition to efforts or actions. Comment: Correct, and a new set of concepts reflects this.

I 4. Word preventative factor is clear it is the breakdown of the additional titles i.e. recovery - do not understand this. Comment: We have taken out recovery and its intended meaning is defective and minimizing factors.

I 5. Since many of the causes of injury to patients are systemic, I suggest this definition be expanded to include more specific reference to systemic responses; I don't think these are completely or clearly encompassed by either "effort" or [the singular] "action". Comment: Agreed. We have changed the definition.

I 6. By definition, preventive factors are antecedents and not sequelae of the event and might therefore be more happily located elsewhere in the taxonomy. Most of them should be re-drafted as omissions and re-positioned as Contributory Factors (see below) Comment: Agreed. We have re-positioned and re-defined them.

I 7. This is an inappropriate level of retrospective analysis, possible preventative factors would be a better term if it must be included. Comment: We have changed the core meaning of the class. We considered Preventative but agreed on Preventive.

I 8. Are you trying to understand if a preventative strategy is in place however, that strategy has not produced the desired result and therefore the occurrence of the same patient safety event or a series of other patient safety events ? Comment: We have re-designed the definition as it was unclear.

I 9. Why is the word 'pre-empt' used? This word has several meanings and most appear not relevant to this situation. Comment: Agreed, pre-empt is deleted now.

I 10. I did not see a full enough description of organizational culture such as safe for reporting, method to report, learning organization, etc. Comment: These concepts are within various parts of the classification.

I 11. I’m not sure it is actually unclear

Page 85 of 194

I 12. no acknowledgement of "recognition" Comment: We now have Detection which may be a synonym.

I 13. It doesn't pre-empt an occurrence, it should prevent it. It is not necessarily an effort or action, it may be a condition. Comment: Correct, new definition

I 14. language is extremely 'technical' Comment: We have changed and hopefully simplified it (and the translators will also advise us if we have been successful in simplifying it).

I 15. Should this be attached to the particular event? as it reads a patient safety program might be the answer to all events Comment: Re-defined

I 16. I am confused about whether this refers to a) What did you do to try and prevent the event? b) What could you have done to prevent the event? The suggested instructions for reading the conceptual framework note 'review the preventative, recovery and mitigating factors (e.g. currently existing policies and procedures) that could have stopped the event from occurring or lessened the severity. This implies that this classification is a theoretical exercise for the coder to determine 'what could have been applied to stop the event occurring'. Comment: We have re-defined it and made it more direct to an incident. The business rules will state that coders should only include known facts.

I 17. This is far too simplistic. It assumes that there are directly modifiable preventive factors, when in most cases there are aspects of the system of care that cause a preventable patient AE. Comment: We acknowledge that there are some things which are difficult to prevent or unpreventable.

I 18. Same issue over 'safety event'

I 19. There is a problem here in thinking about preventing "an event." Prevention may be a broader concept -- making a system safer without specifying the events to be prevented. For example... improving a culture of openness and honesty is preventive, but it does not "prevent" any specific, designated event. Comment: See comment 1 & 16

I 20. If preventive factors are only efforts or actions they should not be called factors. Comment: Definition changed

I 21. It should be clear that such an action is not part of the standard of care, i.e. two people checking the match of patient and blood and preventing a transfusion reaction is standard of care. Comment: Definition changed.

I 22. See my previous comments Comment: It was unclear, definition changed.

I 23. see previous

I 24. could be interpreted as what could be done next time rather than things done prior to this event

Page 86 of 194

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Preventive Factors. II 1. Preventative Factors I think could be Co-morbidity (e.g smokes) or Significant Clinical Feature (e.g

history of MI) Comment: See previous comments

II 2. An effort or action taken or in place before the patient safety event and designed to reduce the occurrence of circumstances that could result in patient safety events. Comment: See new definition

II 3. Structures, processes, or actions taken in advance to prevent the occurrence of a patient safety event. Comment: See new definition

II 4. Unable to comment as do not understand.

II 5. Suggest including reference to systems or frameworks or structures , implemented in advance...(etc) Comment: See new definition

II 6. For example, the omission to follow established guidelines, protocols or policies is a cause or contributory factor and a common one at that. I found the Preventive Factors chapter of ICPS to be an area where more work might be handsomely repaid (see below). Comment: We agree that more work was required. See new concepts structure.

II 7. This is an inappropriate level of retrospective analysis, possible preventative factors would be a better term if it must be included. Comment: See new structure

II 8. It is the wording 'pre-empt' that is troublesome. Could you simply use the wording 'prevent'. Comment: Agreed – has been deleted

II 9. see above

II 10. Organizational culture should be expanded to include some of above characteristics. Comment: Agreed

II 11. But using the phrase safety-barrier or just barrier is more within the theoretical framework

II 12. The recognition of an effort or action taken in advance to pre-empt an occurrence of a patient safety event.

II 13. An action taken or condition in place to prevent the occurrence of a patient safety event.

II 14. pre empt - change to prevent

II 15. above

II 16. A better definition may be 'An effort or action that could have been taken, based on the availability of already existing procedures/protocols, to stop the patient safety event occurring in the first place.' Comment: See new definition

II 17. I don't really have the time to think this through, but you could break down preventive factors into classes that would provide a prompt allowing coders to identify systemic issues. Comment: With new framework that has been our intention.

Page 87 of 194

II 18. See above

II 19. Maybe something that deals with aspects of a safe system -- such as Karl Weick's HRO factors -- belongs here.

II 20. Preventive interventions

II 21. Preventive Factors – An effort or action taken in advance, but not part of standard protocol, to pre-empt an occurrence of a patient safety event.


II 23. see previous

II 24. as above Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. Already gave this reason. This is not a widely used clinical term but I am sure a term already exists for

what you mean

III 2. Preventive steps seems to explain more than factors....it shows a process of achieving an outcome not "facts" Comment: Agreed, new definition deals with this.

III 3. Whislt I believe it is meaningful, in practical terms, most staff will not be fully aware of the existing range of preventive factors in place. Therefore, it is only useful at the highest of management levels, and therefore, may be difficult to create tangible and meaningful links between these factors and the actual events. Comment: The classification is multi-purpose and the different classes do have different meanings.

III 4. These are antecedents and not sequelae of the event and there is an argument for putting failures in the Preventive Factors in the Contributory Factors section of the taxonomy. For example, the omission to follow established guidelines, protocols or policies is a common cause of patient safety incidents. VII.A.1. Cognitive factors - These distinctions may be rather too fine. Many incidents would not fit neatly into one or the other and some users might find the distinctions difficult to grasp without lengthy instruction. Conversely, the Performance Related and Social Factors seem to have potential, but are presently too cryptic and may need to be more precisely defined. Communication is too broad and should be broken down into its constituent. About 20% of the PSIs that we see are attributable to clinician error without any “system” error apparent on that occasion. Typical examples would be unintended damage to neighbouring tissues in the course of a surgical operation, a missed diagnosis that could have been made or other technical error. The Cognitive Factors identified in Chapter VII are perhaps one approach to reducing this type of error but, in a taxonomy intended to be a world standard, there should also be some means of recording whether there has been any failure or shortcoming in the Human Resources processes by which any skills deficit or risk of repetition could have been detected and managed. These processes relate to the Recruitment or Engagement of Staff, Induction, Mentoring, Appraisal, Revalidation and Discipline. Where there is a skills deficit, which is quite a common cause of incidents, it is often worth looking to see whether any improvement in these processes might reduce the likelihood of a recurrence. This is a major omission from ICPS and these processes could usefully be included in the Human and Performance Factors or the Organizational Factors sections. Protocols, policies and procedures are important and an omission to follow them is a common cause of incidents. For reasons of granularity and better learning the causes of such omissions should be more precisely identified. The distinctions in VII.C.1.b (titled Behaviour, but seeming to relate to Policies) are too fine and not as relevant to learning as a breakdown of the various reasons why policies were not followed. I therefore recommend the deletion of the whole of “VII.C.1.b Behaviour” and replace it with “VII.C.1.b Policies and Procedures” with the sub-classes: No relevant policy in force Policy in force but not available Policy available but not followed Policy available was incorrect or inadequate Similar considerations apply to the category of Supervision. In general, Chapter VII is an important Chapter and has the potential to be one of the most powerful sections of ICPS with some extra work.

Page 88 of 194

Comment: We agree that there needs to be linking of contributing and preventive factors. We also agree on the common causes of incidents as you have described. We are reviewing the concepts in the classification to reflect these types of factors.

III 5. as answered earlier this is a confusing term

III 6. While preventive factors represent information that needs to be known, the combination of 'preventive and 'contributing' factors leaves a gap, I think, for factors that contributed to some aspects of the event (perhaps not the genesis, nor preventive). Where do these fit in the model? It seems, as I progress through the questionnaire, I am backing away from a wholly positive assessment of the model. Comment: We have defined the relevant classes.

III 7. Again, seems redundant when Event modifiers is similar.

III 8. Same

III 9. too broad

III 10. Training and education have a popular audience. The fact is that they are the least useful interventions to help prevent an event. Forcing functions, constraints, and systemic redesigns taking into account the intrinsic limitations of human being would be much more effective-so let's list them as examples. Comment: We agree – see new concepts.

Additional Comments from e-mails None Delphi Task Force Comments Feedback from Round 1 indicated a significant amount of confusion about the definition and role of the Preventive Factors class, as well as its relationship to Contributing Factors, Recovery Factors and Mitigating Factors. The following was included in the 2nd Round survey to help explain the role of Preventive Factors and its relationship to the other classes:

Preventive Factors stop a patient safety event before it even starts by controlling for known or unknown contributing factors through identification using risk assessment models, human factors engineering or organizational policies. Examples include guidelines for hand hygiene in healthcare, use of Failure Modes and Effects Analysis to identify potential process failures, procedures to prevent wrong=site surgery, and the provision of educational and training programs.

Page 89 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the term, Preventive Factors, or its associated definition to be unclear. IV 1. As per my comments about actions, patient specific actions could be more appropriately part of

prevention than organizational actions. Comment: We have re-defined the Classes and hence their foci.

IV 2. We believe that your intent is to describe what factors are at play that would prevent an occurrence and we agree that this is very valuable information however the most recent definition creates greater confusion. If preventive factors "pre-empt an occurrence of a patient safety event." then by virtue of the definition, a patient safety event did not occur and there can be no expectation that one would be identified, reported or extrapolated from other data sources. Comment: See new definition and classes.

IV 3. I continue to think about this from the reporters perspective - If a patient safety event has occurred, would a reporter/coder be required to list preventative factors that would prevent the same event occurring again, or does this relate only to prevention of the event in the 1st place? If the latter, wouldn't all preventative factors have failed if the event had occurred, therefore there'd be nothing to comment on!? Comment: We acknowledge that it was unclear and tried to make it clearer.

V Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Preventive Factors. V 1. It should specifically include clinical actions.

Comment: Agreed

V 2. Consider further revising this class further. Comment: Agreed and done

V 3. See query above

Page 90 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? VI VI 1. Please see earlier description and recommendation under the contributory factors. VI 2. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or

whether) it can be used. VI 3. Use definitions that are understandable and thus reproducible across coders. Also see comments above.

Comment: See refined definition

VI 4. NA

VI 5. Still work to do on the subclasses and their definitions VI 6. See previous comment VI 7. as previous answer Additional Comments from e-mails None Delphi Task Force Comments The question regarding this class was “Should this class refer to preventive factors in place prior to the event occurring or should it refer to actions taken as a direct result of the event?” Confusion existed with regard to the temporal aspects of preventive factors Comment: The definition was not clear and we have tried to change them accordingly.

Page 91 of 194

DURING EVENT (RECOVERY) FACTORS NOTE: After review of the Delphi comments below, this class was revised and retermed to Mitigating Factors. To recover, one needs to detect and then mitigate. The revised conceptual framework is an endeavor to reduce the substantial confusion surrounding the intent and meaning of this class. See the Introduction and Basic Definitions at the beginning of this document. Original Definition Recovery factors: Actions taken to stop circumstances or influences from turning into a patient safety event. Revised Definition based on 1st Round of Delphi Results Recovery factors: Actions taken to stop the progression of a patient safety event from harming the patient. Final Changes from December 2006 Drafting Group Meeting The term and definition were revised – During Event (Recovery) Factors: Planned or unplanned actions taken during the progression of an event to stop the patient safety event from reaching or harming the patient (i.e., a patient safety event results in a near miss).

The conceptual framework was revised to clarify the role of Recovery Factors and to illustrate its place in the learning cycle. In addition, the “Clinical Actions Taken” concepts were moved from the Actions Taken class and incorporated into this class to indicate the clinical actions taken as a direct result of the patient safety event. A Key Definitions page was added to the ICPS. Included is a paragraph describing Recovery Factors: During Event (Recovery) Factors: planned or unplanned actions taken during the progression of an event to stop the patient safety event from reaching or harming the patient. Recovery factors are not equivalent to clinical recovery factors (e.g., recuperation). These factors refer to regaining control over a situation to stop the patient event from resulting in an adverse event. Examples include use of a circuit disconnect alarm or checking medication administration procedures and ad-hoc creative problem solving. Changes after Review of all Delphi Respondents’ Comments Comment: To recover, one needs to Detect and then Mitigate: “to prevent or moderate the progression of an incident”. The revised conceptual framework terms and definition are an endeavour to reduce the substantial confusion surrounding the intent and meaning of this class. Mitigating Factor: An action or circumstance which prevents or moderates the progression of an incident toward harming the patient.

Page 92 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Recovery Factors, or its associated definition to be unclear.

I 1. Recovery is usually understood as improvement from a bad health state, not as something that shall specifically reverse or stop an action potentially leading to a bad health outcome - as suggested here. Comment: Agreed. We have revised the conceptual framework and concepts to reflect what we mean by “safety recovery” in detection and mitigation.

I 2. Is there a difference to organizational defenses? Comment: Yes, Reason’s “Swiss Cheese” model illustrates this difference.

I 3. 'Recovery' to me means the event happened and something was done correct the situation after an event occurs. This says the Recovery Factors prevent circumstances from turning into an event. Comment: The conceptual framework and concepts have been revised for clarity.

I 4. recovery would imply that the event has already taken place Comment: Agreed that an incident has taken place. Per the new definitions (see beginning of this document) this does not mean the incident has resulted in an adverse event.

I 5. Need to link this more closely to the concept of contributing factors and the concept of 'secondary' prevention. Comment: The conceptual framework has been revised to elucidate this relationship.

I 6. In my opinion the term should be as most simple as possible to avoid confusion and different interpretations and as most universal as possible. Comment: Agreed, the term and definition have been revised.

I 7. The definition does not seem to fit with the title--I would think that recovery means after a patient safety event has occurred, to try to recover what was lost (e.g. previous level of function, etc.) Comment: We now have the term ameliorating. See Introduction and Basic Definitions.

I 8. There is a subtle difference for recovery and mitigating factors that is likely to lead to difficulty for frontline care providers who may be unable to provide reliable data as a result of the potential to confuse the terms. Comment: The conceptual framework and concepts have been refined for clarity.

I 9. The definition is clear, however the word "recovery" would not be my first choice Comment: The term has been revised.

I 10. The framework becomes unclear because "Recovery Factors" occur after an event. They do not stop circumstances from becoming an event. Prevention actions "stop" circumstances... from turning into a pt safety event. Comment: The conceptual framework and concepts have been reviewed and revised to address this.

I 11. Recovery suggests that the circumstances or influences have already occurred and that these steps are now being taken to address this rather than stopping/preventing them. Comment: The term and definition have been revised. Hopefully this will clarify what was originally meant.

Page 93 of 194

I 12. This in effect is a near miss. Comment: Agree.

I 13. I am afraid I cannot, at the moment, come up with a definitive term but am unsure that recovery is the appropriate one as from a health perspective would have a different connotation (post injury) and therefore not as clear as it could be. Comment: The term and definition have been refined.

I 14. Attentitive or Preventative Factors would be better terms to indicate preventative actions/measures. recovery would seem to be a post-event factor. Comment: The conceptual framework and concepts have been reviewed and revised to alleviate this confusion.

I 15. 1. I'm not sure how an "influence" could "turn into" a patient safety event, although it might trigger or cause or contribute to one. Both terms are unclear. 2. Same comment as previously about the choice of "an action" -- not broad enough. Comment: The term and definition have been revised.

I 16. definition not sufficiently exacting Comment: The term and definition have been revised.

I 17. This is a bit ambiguous, as recovery might be after a patient safety event had occurred - e.g. bleeding in surgery caused by an error. Recovery would occur, but from the event ... and this is different from mitigating the bad outcome Comment: The conceptual framework and concepts have been reviewed and revised to alleviate this confusion.

I 18. I doubt whether the distinction between Recovery and Mitigating Factors would be thought relevant or would be remembered by the majority of patient safety managers who will have to use and administer ICPS. Comment: The conceptual framework and concepts have been revised to elucidate the importance of each class and their relationship to each other.

I 19. Is this related to an action taken for the specific patient safety event or are these actions taken previously to stop circumstances like the specific patient safety event? Comment: The definition has been revised.

I 20. It is too similar to preventative factors and mitigating factors Comment: The conceptual framework and concepts have been reviewed. Revisions were made to clarify the meaning of each class.

I 21. Recovery is term I would equate with action taken after the event. Comment: The term and definition have been revised.

I 22. Associate the term itself as more linked with the area of activities after pse has already happened more than a hold back. But I'm not English native speaker Comment: The conceptual framework, terms, and concepts have been reorganized to show a more temporal relationship.

I 23. Examples would be helpful - to differentiate from preventive factors. Comment: We agree – they have been included.

Page 94 of 194

I 24. Is this a near miss situation? As in, the patient safety event was identified prior to reaching the patient and action taken to prevent the occurrence of the event. In contrast to the preventative factors, is this an action taken at the time-of-care/point-of-care to prevent the occurrence of a patient safety event. Comment: Yes, the conceptual framework has been revise to illustrate this.

I 25. I didn't understand what you meant from this without a description.

I 26. I cannot find the definition of 'patient safety' or 'patient safety event'. Comment: These have now been clearly defined. See introduction to this document.

I 27. I would associate the term recovery with the time period following an adverse event.

I 28. It sounds like the preventive factors.

I 29. Organizational factors are weak. I think more detail or description is needed here.

I 30. Trying to stop circumstances has nothing to do with recovery. Comment: The term and definition have been revised to clarify intent.

I 31. recovery is in the time after the event has occurred, either the term "recovery" should be replaced by "remedy" or the definition needs to be amended. Further, the plural against factors is actions, not action. Comment: The term and definition have been revised.

I 32. Not necessarily an action taken - may be a condition in place. Recovery denotes that, e.g., an error has happened, but there is something in place to rescue and thus prevent it reaching the patient or harming the patient. Comment: We agree, the definition has been simplified.

I 33. uncertain of timing within event/classification here is it singular/plural/individual/org all?? Comment: The conceptual framework has been revised to illustrate the temporal relationship between the classes.

I 34. In my mind, a recovery factor comes into play once a patient safety event has occurred. In the analysis of the patient safety event, the goal is to implement a preventive factor for the future. For example, a patient is given an overdose of morphine, the error is recognized and a reversal agent is given. The patient safety event has already occurred but the recognition of it has led to the ability to recover from the patient safety event. Once I read your definition of Mitigating factors, realize I cannot distinguish between the two. Comment: The conceptual framework and concepts have been revised to alleviate this confusion and show the temporal relationship between the classes.

I 35. The term "recovery factor" implies a response to a situation that has already occurred and resulted in a consequence needed to be recovered from. The given definition suggests recovery from an emergent event. Comment: The term and definition have been revised to clarify intent.

I 36. When I think 'recovery', I'm thinking administration of Narcan after an overdose of morphine, for example. Comment: Agree. The detection of the overdose and the action of giving of Narcan together compromise recovery. The term and definition for this class has been clarified.

I 37. if there has been an event then your definition does not make sense Comment: The term and definition has been revised.

I 38. The definition is reasonably clear but does not modify the term recovery Comment: The term and definition has been revised.

Page 95 of 194

I 39. Not sure how this differs from preventive factors Comment: The conceptual framework and concepts have been revised to clarify intent.

I 40. Not sure when this needs to be applied. e.g. Only if the event is a near miss? Difficult to see the difference between this and mitigating factors. Comment: The conceptual framework and concepts have been revised to clarify intent.

I 41. The word "recovery " will mean very different things to many healthcare practitioners. The definition is fine perhaps use "preventative" factors Comment: The term and definition have been revised to alleviate confusion.

I 42. I am having a difficult time understanding the difference between recovery factors and Comment: The conceptual framework and concepts have been revised to clarify intent.

I 43.

It would be nice if this could be placed in the public health hierarchy of prevention - that is, primary, secondary, tertiary. I take this to be secondary prevention. Tertiary is where the event has occurred, but swift action is taken to ensure no further harm is done. Primary prevention means that the circumstances of care are such as to prevent occurrence of events. Comment: Interesting analysis and we hope that the new framework encompasses this thinking.

I 44. The lack of clarity for me with this term relates to the reference point, i.e., from what is the recovery occurring. "Event" is defined as being a "patient safety event," but "recovery factors" seem to contemplate a type of event that is not yet a "patient safety event." On the other hand, "preventive factors" and "mitigating factors" both in fact are reference to a "patient safety event." I find this dichotomy confusing, and I suspect others might as well. Comment: The conceptual framework and concepts have been revised to alleviate this confusion.

I 45. As before

I 46. Need an example similar to the contributing factors.

I 47. It lumps two types of recovery: identifying hazards and intercepting them before harm, and mitigating harm. This definition also have a odd boundary with the other types in this column. Comment: The conceptual framework and classes have been revised we have separated out identifying, “detection,” response (i.e. what you do about it).

I 48. If the recovery factors are only action they should be called actions.

I 49. I think more of recoveries more as keeping an error, that has already occurred, from harming the patient, e.g. you draw blood from a patient, urine from another, accidentally switch the labels, see the lab has taken the urine, but not the blood, notice the wrong label is on the blood, and call the lab to tell them the urine is mislabeled. Comment: The term and definition have been revised in line with this thought process.

I 50. Believe recovery is generally interpreted to mean stop from turning into, retard the impact of, or mitigate impact of an event Comment: The term and definition have been revised.

I 51. Recovery factors: It’s not clear and precious, because you can recover the act before or after the event to cure/correct the deficient process. Comment: The term and definition have been revised to alleviate confusion.

I 52. I think that the recovery factor refers to what is done after a patient safety event in order to reduce the impact of the harm done. Comment: The term and definition have been revised.

Page 96 of 194

I 53. Recovery implies that something has already occurred and a correction is needed. It seems like there should be a different term used in advance of a patient safety event. Comment: The term and definition have been revised to reflect this.

I 54. If it is an action it is not a factor.

I 55. Is the difference between recovery factors and preventative factors whether the factor was in place a priori? Does a recovery factor from a previous event then become a preventative factor if a similar event occurs again? Comment: Recovery factors can be unplanned or planned. Over time, unplanned ones would become planned and incorporated into the processes of the organization.

I 56. The term does not reflect the definition. recovery implies the event already happened and that recovery steps are now occurring the definition is stating an action to prevent (i.e. stop) the episode from becoming an event Comment: The conceptual framework has been revised to illustrate the temporal relationship between the classes.


I 58. It sounds like an action taken after a patient safety event - like damage control.

I 59. I feel that there is very little difference between recovery factors and preventative factors and is perhaps a duplication. I find that preventative factors is much clearer and encompasses the concept of actions taken to stop circumstances or influences from turning into a patient safety event Comment: The conceptual framework and concepts have been revised to clarify intent and alleviate confusion.

I 60. I am confused about how recovery is different from prevention in this framework. Is prevention more immediate and recovery more long-term? Comment: The conceptual framework and concepts have been revised to clarify intent and alleviate confusion.

I 61. ?Confusion with definition of preventive factors. Comment: The conceptual framework and concepts have been revised to clarify intent and alleviate confusion.

I 62. I think the differentiation between recovery and mitigating factors is pretty arbitrary and the definitions seem to me very similar. Is one of the factors a more active one regarding the providers and the other one more a passive one (sort of "it just happened")? I will have a tough time to categorise our error reports in accordance to these two categories. Comment: The conceptual framework has been revised to clarify intent.

I 63. see previous

I 64. Not clear whether this refers to a fail-safe issue or an ad hoc action

I 65. find it difficult to distinguish where preventive factor stops and recovery factor begins e.g. in the case of a 'near miss' incident Comment: The conceptual framework has been revised to illustrate the temporal relationships between the classes.

I 66. "recovery" is Not a good term in health as recovery often refers to the period in which a patient is getting over the trauma/procedure/event Comment: The term has been revised.

I 67. suggests potential –

Page 97 of 194

I 68. recovery suggests to me actions to relate to actions post incident to counteract effect on a patient, staff or organization. I'm not sure if its the best term but did suggest the term inhibitory factors for factors that inhibit the progression of active and latent factors to an incident - e.g. as one of the examples in the executive summary paper - the use of patient monitoring Comment: The term and definition, as well as the conceptual framework have been revised for clarity to show the temporal relationships between the classes.

I 69. It is not easy to think of example to that action from the name itself, how can we differentiated from preventive factors? is it done only after the events happens or before?

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Recovery Factors. II 1. Replace recovery by reverse factors or else. The term should be place before the error/risky healthcare

act and the patient outcome (see statements above)

II 2. ?

II 3. 'An action taken to stop circumstances or influences from perpetuating the event.' Comment: The new definition has been revised to incorporate this.

II 4. This would be a kind of "secondary prevention"; a stopping mechanism; not quite sure what the best term would be

II 5. An effort or action taken or in place and designed to prevent the occurrence of patient harm that could result from a patient safety event. Comment: The term and definition have been revised.

II 6. Stop (Stopping) Factors

II 7. Perhaps "interruptive" or impact or influence?

II 8. Consider the use of detection rather than recovery. Comment: The conceptual framework has been revised as has the classes contained therein. We now have (Detection and Mitigating Factor) Together they equal Recovery

II 9. I would use "Barriers" since that is JCAHO language

II 10. Recovery Factors - An action taken to return to normal as much as possible, a patient condition or situation, after a patient safety event has occurred.

II 11. Preventive/Pre-emptive

II 12. Can near miss be used in this section as national term used by colleagues Comment: We believe so. Near miss is defined as an incident that did not cause harm.

II 13. Suggestions: reactive, proactive, responsive

II 14. see above

II 15. An action taken to stop or prevent a patient safety event from occurring Comment: The definition has been revised.

Page 98 of 194

II 16. An action taken to prevent circumstances or influences from developing further and progressing to a patient safety event. Comment: The definition has been revised.

II 17. Recovery may need to be differently focused in the diagram. Recovery to me is after an event or error, whereas mitigation is mitigation of an outcome Comment: The conceptual framework has been revised as has the classes contained within (detection and mitigation factors equal recovery)

II 18. These categories would be clumsy to use and should be amalgamated with each other and perhaps also with Actions taken. Comment: The conceptual framework has been revised for clarity.

II

19.

An action taken in advance to stop circumstances or influences from turning into a patient safety event. Comment: The definition has been revised.

II 20. It is too similar to preventative factors and mitigating factors Comment: The conceptual framework and concepts have been revised to illustrate the differences between the classes.

II 21. /

II 22. Perhaps the term Retrieve Factors would be more clear.

II 23. See above

II 24. Near miss or no harm event.

II 25. ??Preventative Factors

II 26. Define basic terms on which concepts of framework are built.

II 27. I would use the term blocking factors or perhaps preventive factors.

II 28. Latent errors?

II 29. Failure to rescue

II 30. I am not sure. In DK we just talk about actions e.g. steps taken to minimize damage

II 31. see above - either wrong definition or wrong term. Likely wrong term and 'remedy' is better.

II 32. An action taken or condition in place which can intercept a potential patient safety event and prevent it from reaching the patient. Comment: The definition has been revised.

II 33. recovery from what?? turning an ae into a near miss absence of harm to patient...

II 34. A Recovery factor is an action taken to lessen the patient impact / outcome once a patient safety event has occurred...or your definition of a mitigating factor. Comment: The definition has been revised.

II 35. No suggestions.

II 36. System safety resilience factors?

II 37. above

Page 99 of 194

II 38. An action taken to assist the organization to re-establish safe conditions by preventing circumstances or influences turning into a patient safety event Comment: The definition has been revised.

II 39. I would need more clarification regarding the difference between preventive & recovery factors to provide a new term

II 40. As per the descriptor on the illustration: 'The action taken to limit the event to a near miss'. Overall, it might be better to frame the description of each primary classification as a question. e.g. What action was taken to limit the event experienced to a near miss, rather than a patient safety event resulting in harm? Comment: The definition has been revised.

II 41. HCPs understand the word "preventative"

II 42. See previous box.

II 43. I think you have to alter the definition of event so that it encompasses events that are antecedent to, and therefore a broader category than, "patient safety events." Otherwise, the reference point for "recovery factors" will remain unstated, and in my mind at least, unclear. Comment: The definition has been revised.

II 44. As before

II 45. example:

II 46. Separate these concepts.

II 47. Recovery interventions

II 48. Recovery Factors – An action taken to stop circumstances or influences from turning an error into a patient safety event at risk for harming the patient. Comment: The definition has been revised.

II 49. An action taken to stop circumstances or influences from turning into a patient safety event, retard the progress of the event to minimize the event, or mitigate the impact of the event at any point in its progression Comment: The definition has been revised.

II 50. Recovery factors has to divided into two types: Recovery factors to atop circumstances influences from turning into a patient safety event. Recovery factors after the event occurrence: to cure the deficient process, activity or act that led to the event. Comment: The term and definition have been revised as have the concepts contained in this class to reflect this.

II 51. Mitigating factors

II 52. Preventive factor?

II 53. I would use the term Action Taken

II 54. event modifier

II 55. "Response" Factors - not sure this is it either


II 57. Should have a name that implies prevention

II 58. Please see above.

II 59. If prevention is more immediate and recovery more long-term, then the definitions should state that difference.

Page 100 of 194

II 60. It is the principle that I find confusing rather than the definition per se. Both "preventive" and "recovery" seem to relate to actions that will prevent future patient safety events.

II 61. (recovery) = see definition of mitigating factors

II 62. see previous

II 63. Fail safe in operation Recovery possible

II 64. No suggestion

II 65. potential preventative factors

II 66. Recovery Actions

II 67. .

II 68. Restoration factors? I might come with other term at later stage Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. recovery and mitigating factors confusing

Comment: The conceptual framework and concepts have been revised to alleviate confusion.

III 2. Making the list of recovery factors will be a difficult classification to form. Comment: The concepts are listed in the classification tree.

III 3. Already gave this reason. This is not a widely used clinical term but I am sure a term already exists for what you mean Comment: The term and definition have been revised.

III 4. While the previous classes refer to terms in common usage (DK) - this is a new one. However the meaning is clear Comment: The term and definition have been revised.

III 5. See earlier comment on "recovery" factors

III 6. As previously discussed in the CPSI response, the information is likely meaningful however the distinction between this class and mitigating factors is subtle and likely to produce unreliable data.

III 7. If you mean by this that these are the things that the individual did, outside of established process, to prevent the event from happening, then I'm okay with it. Otherwise I think it is very soft and could easily be eliminated

III 8. Basically the same answer used in the previous explanation

III 9. As noted earlier unsure duplication within 3 factors

III 10. Whilst I understand the definition, I don’t see how this class fits into a practical "real world" model of incident management. In hindsight, I think this area needs re-definition, as it seems that it is an action that has occurred in response to one event that is undertaken to prevent occurrences of a similar future event. Comment: The conceptual framework has been revised to illustrate intent. The terms and definitions of the classes as well as the concepts have also been reviewed and revised.

Page 101 of 194

III 11. It is meaningful and important ... but I not quite correctly oriented within the diagram. See earlier comments Comment: The conceptual framework has been revised for clarity.

III 12. Recovery factors need to be consolidated with Chapter IX - Mitigating Factors. Both would benefit from substantial revision for the mix of reasons referred to in relation to Chapter VII. Comment: The conceptual framework and concepts have been reviewed and revised.

III 13. The specifics of what is meant by the recovery factors is unclear and it is also difficult to distinguish recovery factors from preventive factors. Comment: The conceptual framework has been revised to illustrate clarity of intent. The term and definition of this concept have been revised.

III 14. As mentioned earlier this is a confusing term (and for your interest at this point in time your time consuming survey design is becoming unacceptable) Comment: Apologies for the repetition.

III 15. Not really sure how it would be used

III 16. I marked "disagree" category as I am not convinced "Recovery" is clear enough. earlier in the survey I have suggested using the term "Retrieving Factors".

III 17. I am not sure at the moment, would need to put this into practice.

III 18. I find the term confusing. Recover means to "get back, bring back to normal position or condition, make up for, gain or anew." I suppose I would get used to it using this model on a frequent basis.

III 19. The name does not clarify the action.

III 20. Would be difficult to identify difference between Recovery, Actions and Preventive factors.

III 21. seems out of place. If better explanation of event types, may be okay. For example, does event types include "near miss" events?

III 22. I've answered the question once!!!

III 23. Same

III 24. Every event and every organization will determine the consequence of an outcome proportional to an event based upon cultural and professional interpretation and action. As such whilst a useful concept it is unlikely to be measurable between countries and organizations. In other words a solution to an event in Zimbabwe (due to effect and causality etc) is likely to differ from a solution to a similar event in Denmark - so one cannot measurably compare solution 'A' with solution 'B' as a suitable and transferable outcome, and so confuses the issue. Each solution must be proportionate to each problem. Only if the environments that the event occurs in are similar can solution comparisons be made. Comment: The concepts contained within this class have been reviewed in light of this. Some new concepts have been added, some have been deleted, and some have been revised.

III 25. Based on my confusion with the difference between mitigating factors and preventive factors. I do not really understand what a recovery factor is. Comment: See new framework

III 26. This would be covered by preventative factors

III 27. I'm uncomfortable with the definition of Recovery Factors.

III 28. I personally don't understand what is trying to be conveyed here. Ostensibly, the physiologic monitors indicate that, for example, too much of a medication has been delivered to a patient, that means that harm has reached the patient-period. I would see a recovery factor more in-line with the actions taken my a healthcare team to minimize/reverse the harm already created. But as I say, I don't quite understand where you are coming from here. I suppose it makes sense in terms of a patient being in distress, that distress being shown by the monitors, and the healthcare team acting on that-so if recovery factors are

Page 102 of 194

speaking exclusively to situations in which a 'failure to save' is involved, physiologic monitors as an example of recovery factors makes perfect sense. But again, I think recovery in this sense is a bit of a misnomer. Comment: We believe that recovery is the whole process with the monitor being Detection and the action being the Mitigating Factor. The conceptual framework has been revised to reflect this.

III 29. The influencing factors may be too narrow in their definitions - people will not be able to distinguish one from the other. Comment: The concepts in this class have been reviewed. New concepts have been added where appropriate.

III 30. could easily be confused with preventative and mitigating factors, less is more when trying to train people in this methodology

III 31. Remain unsure how this differs to preventive factors

III 32. I am confused between recovery factors and mitigating factors. Apologies, perhaps this is because I am not a clinician and haven't had to code particular events. To me recovery and mitigating factors are very similar. Comment: The conceptual framework has been revised to alleviate confusion and to illustrate the relationships between the classes.

III 33. Seems similar to preventative factor. If different need to be very clear the difference in data collection

III 34. This contributes to the strong clinical emphasis of this classification system. At least that is how I read it. I would prefer that different terminology were used to recognise that this is part of a prevention hierarchy according to public health schema, and that great recognition was made of organizational and system factors. The terminology here is heavily clinical. Comment: The classification contains concepts related to both clinical and organizational factors. The concepts contained within the classification have been reviewed and refined to highlight both of these aspects.

III 35. See my earlier comments. The term assumes an event that is not yet a patient safety event, but the structure defines event as a class of "patient safety event." This is confusing. Comment: The conceptual framework and concepts have been revised to alleviate confusion.

III 36. There are muddy boundaries among "preventive," "recovery," and "mitigating" factors. Comment: The conceptual framework has been revised to clarify the intent of each class and the relationships between and among them.

III 37. As with actions taken

III 38. It is not clear to me what the difference is between mitigating factors and recovery factors Comment: The term and definition have been revised to clarify this.

III 39. distinction between preventive, recovery and mitigating in a real-world application is difficult Comment: The conceptual framework and concepts have been revised to hopefully make the distinction clearer. Business rules will assist with real world applicability. The classification will be field tested; the results analyzed and necessary revisions made.

III 40. as noted earlier, offered additions to the concept definition

III 41. recovery factors can be mixed in with prevention and mitigation Comment: The term and definition have been revised to clarify this.

III 42. See previous comments

III 43. Same as previous explanation

Page 103 of 194

III 44. I feel that there is very little difference between recovery factors and preventative factors and is perhaps a duplication. I find that preventative factors is much clearer and encompasses the concept of actions taken to stop circumstances or influences from turning into a patient safety event. Comment: The conceptual framework and concepts have been revised to illustrate the intent of each class and the relationships between and among them.

III 45. See comments on definition of recovery factors


III 47. Recovery may be passive ('fail safe')as well as active

III 48. potentially useful, but need further clarity in relation to definitions of preventive, recovery and mitigating factors Comment: The conceptual framework and concepts have been revised to clarify the relationships between and among the classes.

III 49. see above

III 50. see previous comments on recovery factors

III 51. we might need to have example for this to be useful

III

52.

Institutions don't typically look at events in this way and without a standardized approach such a question has little help of securing consistent reliable information Comment: Agreed, business rules will be developed to create a standardized approach. This approach will be tested for real world applicability and revised accordingly based on the results.

Additional Comments from e-mails None Delphi Task Force Comments Feedback from Round 1 indicated a significant amount of confusion about the definition and role of the Recovery Factors class, as well as its relationship to Contributing Factors, Preventive Factors and Mitigating Factors. Further, the term “recovery” caused substantial confusion. Many respondents thought this meant clinical recovery (e.g., in the recovery room, recovering from an injury, etc.). The following was included in the 2nd Round survey to help explain the role of Recovery Factors and its relationship to the other classes:

Recovery Factors halt the advancement of the patient safety event from reaching the patient through planned or unplanned actions. Examples include use of circuit disconnect alarms, planned rule-based or skill-based actions in anticipation of a difficult endotracheal intubation, or ad-hoc creative problem solving using knowledge-based factors.

Comment: To recover, one needs to Detect….. and the Mitigate “to prevent or moderate the progression or effects of an incident” The new framework and definition is an endeavor to reduce the substantial confusion arising.

Page 104 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the revised definition for Recovery Factors to be unclear. IV 1. There seems to be some overlap with 'mitigating factors' because of the mention of patient harm.

Comment: The conceptual framework has been revised to clarity this.

IV 2. As per my comments about actions, patient specific actions could be more appropriately part of recovery than organizational actions. Comment: The concepts contained within the classification have been reviewed and reorganized if appropriate.

IV 3. Recovery measures cannot always reverse harm endured at that point but can prevent escalation.

IV 4. I do not find the definition unclear, but I wonder if it should be expanded. In example #2 of this survey, a nurse recognized that the wrong medication was sent from the pharmacy and did not give it to the patient. This was a recovery factor, in which the nurse "stopped the progression of a patient safety event (the dispensing of the wrong drug from the pharmacy) from harming the patient." So, an error/event occurred, but did not reach the patient. Should the definition be expanded to say that recovery factors "stop the progression of an event from reaching and harming the patient"? (If the event reaches the patient but does not harm him/her, it is not because of a recovery factor, it is because of the patient's underlying ability to withstand the event.)

IV 5. Something about the time (during the operation or 2 weeks later) should be in the definition.

IV 6. recovery...is a word used a lot in mental healthcare with a different meaning than the one suggested here. Comment: The conceptual framework has been reorganized. The term and definition have been revised.

IV 7. The definition is clear enough it is a comment that the conceptual model implies that these are all actions that flow from Organizational outcomes as well as Patient Outcomes - so should not the definitions indicate reference to both

IV 8. prefer the original definition

IV 9. recovery may be understood to be totally curative not just stopping the progression Comment: The definition has been revised to reflect this.

IV 10. “actions taken to stop the progression of a patient safety event from harming the patient.” There is a presumption that harm WILL occur - supposition. The event may not unfold as expected. Comment: The definition has been revised.

IV 11. for many people , recovery connected to healing from the event outcomes or effects. Comment: The term and definition have been revised.

IV 12. I think the revised definition does indeed help to clarify what is intended by the drafting group here. Where I still find a difficulty with this is the distinction between (1) inhibition of progression of an event ( which ? is part of mitigation) . This could alternatively be viewed as factors that stop an event form escalating in severity. The other concept that I think is included in recovery factors are things that actually regress severity of an event - for example the provision of prompt and effective treatment to a physically or psychologically injured patient that improves the final patient outcome I recognise that I may be confusing recovery and mitigation here. Comment: The concepts contained within this class have been reviewed in light of these and other comments and have been revised where appropriate to address this issue.

Page 105 of 194

IV 13. Stopping the progression of a pateint safety event is so as not to harm the patient suggests it would be OK to let it continue if the judgement was that it would do no harm. Comment: The definition has been revised.

IV

14.

the definition is good and the intent well constructed - not sure the actual word "recovery" is the correct word as it may have different connotations in different settings or cultures Comment: The term has been revised.

V Respondents’ comments below are addressed through the revisions made to the conceptual framework, terms, and concepts. Please provide an alternative definition, the relevant reference(s) to the sources and your rationale for including the alternative definition instead of that provided for Recovery Factors.

V 1. See above and previous comments about needing to review not only the definitions but the categories in these three 'factor' classes, to think again about the definitions.

V 2. It should specifically include clinical actions.

V 3. actions taken to stop the progression of a patient safety event from harming or further harming the patient

V 4. Recovery factors are actions taken to stop the progression of a patient safety event from reaching and harming the patient.

V 5. Recovery Factors are actions taken during the same short-term episode of care to stop the progression of a patient safety event from harming the patient.

V 6. I am not sure...damage control...?

V 7. Recovery Factors are actions taken to stop the progression of a patient safety event from harming the patient and the organization??

V 8. original

V 9. Perhaps the term recovery should be replaced

V 10. should read “actions taken to stop the progression of a patient safety event from harming, or possibly harming, the patient.”

V 11. We could develop two recovery factors sub categories. One sub-category before the event occurrence and one after the effect occurrence to replace the mitigation factors

V 12. I've added a few words that I think encompass the argument above "actions taken to stop and to reverse the progression of a patient safety event from harming the patient.”

V 13. Just stop the progression, period.

V 14. not sure - review various thesauruses for alternate words. If recovery is an accepted term in the human factors literature, then a statement regarding this being a standard term would be in order

Page 106 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? VI Respondents’ comments below are addressed through the revisions made to the conceptual framework, terms, and concepts. VI 1. See previous comments re more work on recovery and mitigation factors VI 2. see earlier responding VI 3. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how

(or whether) it can be used. VI 4. NA VI 5. As said - recovery covers another meaning in mental healthcare! VI 6. Still work to do on the subclasses and their definitions VI 7. Confusing but cannot think of another example at present VI 8. I understand the meaning of the class. I'm not sure it will be intuitively understood globally. VI 9. Again it is a poor term (I always thought recovery was about anaesthesia or debt). Further, again, a

solution in country 'A' may be non-applicable or appropriate for country 'B'. Recovery is an answer, but, not necessarily a remedy or cure.

VI 10. As previous VI 11. Because of the accurate definition of the recovery factors isn't clear VI 12. As with many of the comments from round 1 of the survey and my previous comments on this class, it

still seems rather unclear, without clear distinction from mitigating factors. Additional Comments from e-mails None Delphi Task Force Comments The suggestion was made to revise the definition so that it would not be confused with Mitigating Factors. The revised definition states “stop the progression of a patient safety event from harming the patient.” Stopping the progression of a patient safety event from harming a patient really means mitigation rather than recovery. The term “recovery” continues to cause a substantial amount of confusion. This may be an artifact of the medical field.

Page 107 of 194

AFTER EVENT (MITIGATING) FACTORS NOTE: After review of the Delphi comments below, this class was revised and retermed to Ameliorating Factors. To recover, one needs to detect and then mitigate. The revised conceptual framework is an endeavor to reduce the substantial confusion surrounding the intent and meaning of this class. See the Introduction and Basic Definitions at the beginning of this document. Original Definition Mitigating factors: Circumstances, actions or influences that played a role in diminishing the impact of a patient safety event. Revised Definition based on 1st Round of Delphi Results Not revised. Final Changes from December 2006 Drafting Group Meeting The term and definition were revised –

After event (mitigating) factors: Circumstances, actions or influences that play a role in diminishing the harm caused to the patient after a patient safety event has occurred.

The conceptual framework was revised to clarify the role of Mitigating Factors and to illustrate its place in the learning cycle. In addition, the “Clinical Actions Taken” concepts were moved from the Actions Taken class and incorporated into this class to indicate the clinical actions taken as a direct result of the patient safety event. A Key Definitions page was added to the ICPS. Included is a paragraph describing Mitigating Factors:

After event (mitigating) factors: circumstances, actions or influences that play a role in diminishing the harm caused to the patient after a patient safety event has occurred. In the ICPS, mitigating factors can be inversely related to contributing factors (e.g., action was taken to avert the patient safety event from having a more deleterious effect on the patient). An example of a mitigating factor is good crisis management.

Changes after Review of all Delphi Respondents’ Comments The revised conceptual framework, terms, and definitions are an attempt to reduce the substantial confusion surrounding the intent and meaning of this class. Ameliorating Factor: Actions taken or circumstances altered to make better any harm after an incident.

Page 108 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Mitigating Factors, or its associated definition to be unclear.

I 1. 1) The term does not allow for differentiation of (a) early interventions to minimize bad patient outcomes in a short-term and (b) later interventions to alleviate/minimize long-term impact on patients (e.g. physical, loss of trust due to bad risk communication, economic losses). 2) The term neither allows to differentiate factors targeted at patients and at the organizations. Comment: Agreed that the class was not explicit enough. In the new framework we have (a) mitigating factor (b) ameliorating factor We have incorporated in the concept both patient and organizational concepts. The concepts in this class have been made more explicit. We have incorporated both patient and organizational concepts.

I 2. '...in diminishing the impact of a patient safety event.' Impact on whom or what? Comment: The definition has been revised.

I 3. "...diminishing the impact of a patient safety event" does not cover compensation or punitive damages, which may be part of the response. Comment: Correct. These are purposely no included in the classification.

I 4. Phrasing not clear

I 5. What tense are you taking about? recovery Factors does not make sense but Factors affecting recovery does? Or are you talking about Significant Clinical Feature (e.g history of MI) Comment: The definition has been revised.

I 6. As for previous definitions, I think the word 'influence' should be removed. Link better to the point at which the effect occurs Comment: Agree, the definition has been revised.

I 7. As per recovery factors (as above, even the survey has interchanged the terms mitigating and recovery). Comment: Agreed, the conceptual framework and concepts have been revised to clarity intent and alleviate confusion.

I 8. I think of mitigating factors as things that affect the outcome either positively or negatively not just negatively PS - your * description is incorrect Comment: The preamble to the patient safety incident definitions states that we are talking about incidents that have a negative or potential negative affect on a patient. Conversely, ameliorating factors have a positive effect on an incident.

I 9. Could be clearer - term used within RCA post event rather than what may have positively affected the event therefore potential for confusion. ? overkill with 3 factors Preventive, Recovery and mitigating. Comment: The conceptual framework has been revised to address this.

I 10. The last question was about the Recovery Factor term. I believe you mean to ask here what I find unclear about your definition of mitigating factors. I would add ...a role in diminishing or increasing...the impact........ Comment: The terms and definition have been revised to clarify meaning.

I 11. It is clear but ...

Page 109 of 194

I 12. I doubt whether the distinction between Recovery and Mitigating Factors would be thought relevant or would be remembered by the majority of patient safety managers who will have to use and administer ICPS. Comment: The conceptual framework and concepts have been revised. The classification will be tested for relevance in the field and will be revised accordingly based upon the results.

I 13.

Are these circumstances for the specific patient safety event, or are these circumstances previously defined that would mitigate this kind of patient safety event? Recovery factor? Comment: The definition has been revised.

I 14. While this is clear, I am now not sure how opposite factors would be recorded, for instance factors that made outcomes worse. Comment: These concepts are captured in contributing factors.

I 15. "Mitigating factors" the definition seems unclear. Mitigating can mean "making an offence or a crime seem less serious, or partly excusing it" Comment: The term and definition have been revised.

I 16. Difficulty understanding difference between recovery factors and mitigating factors. Comment: The conceptual framework has been revised to clarify intent.

I 17. Influence is a bit hazy

I 18. confusion with recovery factors

I 19. prefer the term controls

I 20. Mitigate is not a very common verb. I would have understood mitigating factors as factors in defense of the event. Comment: The definition has been revised.

I 21. See comment under recovery factors. Difficult to understand the difference between mitigating and recovery factors. Also, is this talking about 'what could have been done' or 'what was actually done' to mitigate? Comment: The conceptual framework and concepts have been revised to clarify intent.

I 22. This is tertiary prevention, in public health terms. The event has occurred, but swift action prevents it having more deleterious impact on the patient. Comment: Agreed, the conceptual meaning of “ameliorating factors” and “tertiary prevention” are the same.

I 23. This is related to my earlier comment about recovery factor. The definition for "recovery factors", and perhaps also for "preventive factors," seems to contemplate that an event of some type, but not yet a "patient safety event" has occurred, while the reference point for "mitigating factors" appears to be the patient safety event. But the definition of "event" suggests that it is a "patient safety event." I think this needs clarification. Comment: The definitions for the concepts have been revised to clarify this.

I 24. As before

I 25. give an example also there is an error on this page for this box, should say Mitigating Factors instead of recovery factors

I 26. Again... this term is muddy in its boundaries compared with others in the same column -"Recovery Factors." Comment: The conceptual framework and concepts have been revised to clarify intent and illustrate relationships between and among classes.

Page 110 of 194

I 27. I think it should be more clear that it decreases the impact for the patient (as against the provider, etc.).

I 28. mitigating can be interchanged with recovery in my mind Comment: The conceptual framework and concepts have been revised to explicitly show the intent of each class and the relationships between and among them.

I 29. see my previous comments

I 30. see previous

I 31.

Similar to previous, am unsure of distinction between recovery and mitigating factors. I think more detail needs to be included in these definitions so that what one person includes as a recovery factor someone else does not call a mitigating factor Comment: The definitions for the concepts have been revised to address this.

I 32. Mitigating not used widely II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Mitigating Factors.

II 1. Differentiate mitigating factors in (a) "early intervention" (arrow between "risky healthcare acts" and patient outcome) (b)and "impact alleviating factors" (arrow between patient outcome and patient impact. For organizational impact: use "reorganization", "organizational amendment" or else Comment: The conceptual framework has been reviewed and refined. See Introduction, Basic Definitions and New Framework – (a) mitigating (b) ameliorating.

II 2. '...in diminishing the impact of an event on the safety of patient.' Comment: The revised definition incorporates these comments.

II 3. A circumstance, action or influence that played a role in diminishing patient harm or in providing compensation for patient harm. Comment: The revised definition incorporates these comments.

II 4. Impact diminishing factor

II 5. What tense are you taking about? recovery Factors does not make sense but Factors affecting recovery does? Or are you talking about Significant Clinical Feature (e.g history of MI) Comment: The definition has been revised.

II 6. An effort or action taken or in place and designed to reduce the patient harm or organizational impact that could result from a patient safety event. Comment: The definition has been revised.

II 7. Mitigating may be acceptable if the term recovery is sufficiently altered to distinguish the categories. Comment: The term and definition have been revised.

II 8. I would not use the word "diminishing". I would change it to decreasing or increasing the level of the outcome. PS - your * description is incorrect Comment: The term and definition have been revised.

II 9. ?

Page 111 of 194

II 10. again I think you mean Mitigating Factor, and I think the term is fine, just need to amend your definition of....see above

II 11. ... I think mitigating factors should point to the outcomes

II 12. These categories would be clumsy to use and should be amalgamated with each other and perhaps also with Actions taken. Comment: The conceptual framework and concepts have been revised to alleviate confusion.

II 13. A circumstance, action or influence previously defined that played a role in diminishing the impact of a patient safety event. Comment: The revised definition incorporates these comments.

II 14. If the broader notion were to be attempted, a new name would be needed.

II 15. Safety factors would mean that something has been introduced to play a role in diminishing the impact of a patient safety event. Comment: The revised definition incorporates these comments.

II 16. None

II 17. An action taken or condition in place that played a role in diminishing the impact of a patient safety event. Comment: The revised definition incorporates these comments.

II 18. are they preexisting? have they previously been put in place again as with recover unsure of timing here within framework....

II 19. controls

II 20. Diminishing factors might explain this better.

II 21. XX


II 23. Modifying my earlier suggestion, I would suggest now that the three categories in that part of the structure be denominated, within the box of each, "event preventive factors," "event recovery factors" and "patient safety event mitigating factors." Comment: The conceptual framework and concepts have been revised to incorporate these comments.

II 24. As before

II 25. example include

II 26. The whole column, "Recover Factors", needs reworking. Comment: Agreed, the class has been reviewed, refined, and revised.

II 27. Mitigating Factors – A circumstance, action or influence that played a role in diminishing the impact of a patient safety event to the patient. Comment: The revised definition incorporates these comments.

II 28. event modifiers

II 29. see my previous comments

II 30. see previous

II 31. No suggestion

II 32. Diminishing effects factors

Page 112 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. see previous answer

III 2. Already gave this reason. This is not a clinical term but I am sure a term already exists for what you mean

III 3. As previously discussed in the CPSI response, the information is likely meaningful however the distinction between this class and recovery factors is subtle and likely to produce unreliable data.

III 4. How specifically is this different than preventive and recovery factors. Seems like sometimes we spend too much time slotting things into the right descriptive bucket instead of just identifying it. The more possibilities you have the more folks become obsessed with the minutia instead of focusing on the real factors which contributed to the event. Comment: The conceptual framework and concepts have been revised for clarity and simplicity.

III 5. see previous answer

III 6. Again, important but should be oriented differently in diagram. See earlier comments

III 7. Mitigating Factors should be consolidated with Recovery Factors. Both would benefit from substantial revision for the mix of reasons referred to in relation to Chapter VII. Comment: The conceptual framework and concepts have been revised in line with these and other concepts.

III 8. confusing term

III 9. see previous comment re the need to expand the concept.

III 10. Mitigating can mean "making an offence or a crime seem less serious, or partly excusing it"

III 11. I understand the term but would everyone else? It is more of a litigation term. Comment: The term and definition have been revised.

III 12. Same

III 13. As described in the examples, I would see such concepts as more appropriately described as recovery factors. Comment: The concepts have been reviewed and reorganized where appropriate.

III 14. See previous statement.

III 15. controls is a better term

III 16. please see previous answer

III 17. See previous comment

III 18. Emphasize tertiary prevention aspect, and bring out organizational and systemic factors as well as specifically clinical aspects that made this possible. Comment: The concepts in this class have been reviewed. Additional concepts have been incorporated; others have been revised.

III 19. There are muddy boundaries among "preventive," "recovery," and "mitigating" factors.

III 20. As with contributing factors

III 21. it is not clear to me what the difference is between these and recovery factors Comment: The conceptual framework has been revised to clarify the distinguishing characteristics of each class and to illustrate the relationship between and among them.

Page 113 of 194

III 22. distinction between recover and mitigating is difficult in a real-world application Comment: The conceptual framework and concepts have been revised to clarify the distinction between the classes. “Real world” applicability will be tested in the field.

III 23. mitigation can be mixed in with prevention and recovery

III 24. See my previous comments on the need to include macro level


III 26. If there are mitigating factors why not enhancing factors

III 27. As with recovery factors

III 28. without a standardized approach such a question has little help of securing consistent reliable information Comment: Agreed, business rules will be developed to create a standardized approach. This approach will be tested for real world applicability and revised accordingly based on the results.

Additional Comments from e-mails None Delphi Task Force Comments Feedback from Round 1 indicated confusion about the definition and role of the Mitigating Factors class, as well as its relationship to Contributing Factors, Preventive Factors and Recovery Factors. The following was included in the 2nd Round survey to help explain the role of Mitigating Factors and its relationship to the other classes:

Mitigating Factors reduce or minimize the consequences of a patient safety event upon the patient and can hinder a contributing factor’s ability to have a more deleterious effect on the patient. The most well known example is administering treatment in response to an adverse drug event.

Page 114 of 194

2nd Round Delphi Comments: IV The comments below are addressed through the revisions made to the conceptual framework, terms, and concepts. Please provide an explanation for why you find the term, Mitigating Factors, or its associated definition to be unclear. IV 1. See previous comments on 'recovery'

IV 2. As per my comments about actions, patient specific actions could be more appropriately part of mitigation than organizational actions.

IV 3. may need to be adjusted to be consistent with recovery factors if that suggested amendment is accepted.

IV 4. The definition is not unclear, but I want to make a statement about example #1 in this survey. Giving the patient some juice and a sandwich should have been classified as a mitigating factor, an action that diminished the impact of a patient safety event (giving the patient too much insulin). (In fact, correction of a medication error is included in your definition as an example of a mitigating factor.)

IV 5. It is still not clear when a mitigating factor starts to act: When the event has reached the patient? Or just before it could reach the patient? Where are the limits between mitigating and recovery in the process of an event?

IV 6. Recovery and mitigating factors will be viewed as a distinction with a difference.

IV 7. Mitigating factors: You say in your comments "The most well known example is administering treatment in response to an adverse drug event": But in the example shown in the second phase of delphi you say giving fruit juice was an "action taken" and not a "mitigating factor". I can't understand the difference.

IV 8. see my comment on preventive factors V The comments below are addressed through the revisions made to the conceptual framework, terms, and concepts. Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Mitigating Factors. V 1. See previous comments on 'recovery'

V 2. It should specifically include clinical actions.

V 3. See above comment

V 4. .

V 5. I would not distinguish between both terms: recovery = detection + mitigating factors

V 6. eliminate

V 7. Mitigating, preventive and recovery factors can be included in "actions taken" or not, depending on the previous occurrence of patient safety event.

V 8. .

V 9. see my comment on preventive factors

Page 115 of 194

Comments from the Question – Do you believe this is a meaningful and useful class? VI The comments below are addressed through the revisions made to the conceptual framework, terms, and concepts. VI 1. See previous comments re more work on recovery and mitigation factors

VI 2. You should not distinguish recovery and mitigating

VI 3. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or whether) it can be used.

VI 4. Difference between recovery and mitigating factors is unclear as is the need for both categories.

VI 5. NA

VI 6. ...

VI 7. Still work to do on the subclasses and definitions

VI 8. Mitigation is an opinion, not necessarily fact. As such the terminology can blur the outcome data. There is sufficient information recoverable already without adding to the data burden.

VI 9. as previous

VI 10. Because of the accurate definition of the mitigation factors isn't clear

Additional Comments from e-mails I doubt whether this distinction would be thought relevant or would be remembered by the majority of patient safety managers who will have to use and administer ICPS. This category would be clumsy to use and should be amalgamated perhaps with Actions taken. Delphi Task Force Comments None All comments can be dealt with using changes in framework, definitions and concepts.

Page 116 of 194

ORGANIZATIONAL OUTCOMES Original Definition Organizational outcomes: The impact upon an organization which is wholly or partially attributable to a patient safety event. Revised Definition based on 1st Round of Delphi Results Not revised. Final Changes from December 2006 Drafting Group Meeting No changes were made to the definition -

Organizational outcomes: The impact upon an organization which is wholly, or partially, attributable to a patient safety event.

Changes after Review of all Delphi Respondents’ Comments Organizational outcome: The impact upon an organization which is wholly or partially attributable to an incident.

Page 117 of 194

1st Round Delphi Comments: I Please provide an explanation for why you find the term, Organizational Outcomes, or its associated definition to be unclear.

I 1. Organizational outcome is now on the same level as patient outcome, though patient outcomes come first in practice and norms. Comment: Agree but we still need a separate class for organizational.

I 2. see below

I 3. See prior comments

I 4. define impact Comment: The term has been revised.

I 5. Unclear about the term. Possibly examples need to be given. Comment: The term has been revised. Examples will be given.

I 6. Not all healthcare is provided through organizations. Comment: Agreed however the term organization is reasonably generic to represent all entities (whether included family physician or organization) that provide healthcare.

I 7. this would not used in a clinical setting or in a patient care pathway Comment: This concept is intended to be used at a different level of the organization (admin/management).

I 8. The emphasis here needs to be in distinguishing between patient and organizational outcomes, and on what range of outcomes is considered in scope. It goes without saying (or could be added to the 'patient safety event' definition) that the outcomes are those attributed to the patient safety event. Need to make a clear distinction between this and 'actions taken'. Comment: Agree, the concepts have been separated and the definitions revised.

I 9. As I'd indicated earlier, the event itself my have organizational outcomes, but in addition, Actions Taken may have organizational outcomes as well, and the schematic doesn't clearly indicate this 2-way interaction. Comment: Agree, the concepts have been separated and the definitions revised.

I 10. needs a fuller definition, does this include: immediate actions? changes in policy, procedure and practice and out to what time frame after the incident? Comment: The definition has been revised.

I 11. When I think of organizational outcomes, I think of things such as loss of public trust, decreased revenue, decreased market share, threat to personal role in organization (lose one's job), staff morale, etc. I just don't see these terms in the model, although you could guess they are there. In some ways, the outcomes see like solutions such as in education and training. Comment: Many of these concepts are contained within the classification tree. The terms and definitions have been reviewed in light of these and other comments. Additional concepts have been included where necessary to obtain pertinent information.

I 12. No talk of potential impact which is surely important if trying to 'nip a problem in the bud' Comment: We are deliberating on whether to include potential outcome although they are difficult to model and inter-rate reliability is problematic.

Page 118 of 194

I 13. May impact upon a tiny part of an organization, e.g. one staff member, or on the whole institution. Need to give examples of the possible nature of this Comment: Examples have been included.

I 14. It should be broken down into 3 chronological sub-categories. At the time of an event, most organizations won't know if they'll be out 10 million or not!

I 15. If take the example of general medical practice in the UK, most staff will associate themselves as being part of the healthcare team rather than the local primary care organization.

I 16. the problem here is that you have not specified the organizational level - micro, meso or macro. We probably want to know all of these Comment: The implication is macro - the classification has been designed to allow for analysis on all levels.

I 17. I am not sure why this is in the system, unless it is there to encourage changes in processes and procedures such as to prevent recurrence. In which case, I would like this outcome class to be broken up in such a way that coders are prompted to classify for preventive action where that is relevant. Comment: The purpose of this class is to capture information about processes/procedures which will inform the development of preventive strategies. The business rules will address how to classify information in this and all the classes.

I 18. Neither the term nor its definition are specific enough to determine whether the information sought relates only to externally imposed outcomes (e.g., insurance costs rose, bad publicity, decrease in patients seeking same sort of care, etc.) or also includes affirmative organizational responses to the patient safety event (e.g., hired more interpreters or nurses or whatever, restructured departmental organizational chart, etc.). Comment: Many of these concepts are contained within the classification tree. The terms and definitions have been reviewed in light of these and other comments. Additional concepts have been included where necessary to obtain pertinent information.

I 19. As before

I 20. Your organizational outcomes is between the event and the action. Therefore, it should be the organizational failures, not the outcomes. Comment: The definition has been revised.

I 21. should be the organizational responses, actions, and outcomes to the event so that recognition is made and recurrence will be minimized Comment: These concepts are captured in other classes of the classification.

I 22. I would prefer to emphasise that organizations are made up of individuals. Should not be mentioned that the organization is any kind of healthcare or financing organization? Comment: Agreed, however the term organization is reasonably generic to represent all entities (individual practitioner through organizations) that provide health care.

I 23. Examples would be helpful Comment: Examples will be included.

I 24. the definition is good generally, but more explanation could be added . Comment: The definition has been revised.

Page 119 of 194

II Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Organizational Outcomes.

II 1. replace by organizational impact and shift the box below patient outcome next to patient impact(see statements above) Comment: The conceptual framework and concepts have been revised.

II 2. I need to understand what the impact headings mean e.g. social attitudes - need some examples of what this might include Comment: The term has been revised. Examples will be provided.

II 3. See prior comments: "Systemic Impact/Outcomes"

II 4. Define 'impact' as you intend it. For example, 'impact' in terms of time, resources, cost, etc. Comment: The term has been revised.

II 5. Unable to define.

II 6. Replace "an organization" by "a healthcare provider" Comment: Organization and healthcare professional are separate concepts. The definition for both of these are provided at the beginning of this document.

II 7. this would not used in a clinical setting or in a patient care pathway

II 8. The impact of the patient safety event on the organization, for example financial, operational, organization. Comment: These concepts are contained within the classification tree.

II 9. Subject to the comment above, the definition proposed is clear.

II 10. needs a fuller definition, does this include: immediate actions? changes in policy, procedure and practice and out to what time frame after the incident? Comment: These concepts are captured in other classes of the classification.

II 11. See above concepts that should be included.

II 12. not sure

II 13. The impact upon an organization or part thereof which is wholly or partially attributable to a patient safety event. Such an impact may include financial impact, business impact, impact on staff... Comment: The concepts are contained within the classification tree. Examples will be included.

II 14. Include 3 sub-categories of: Immediate Intermediate Long-term

II 15. The impact upon an organization or healthcare team which is wholly, or partially, attributable to a patient safety event

II 16. above


II 18. I assume that both types of information described above are desired, so I would clarify that, at least in the definition.

II 19. As before

Page 120 of 194

II 20. Organizational Vulnerabilities– The weaknesses or deficiencies of an organization which are wholly, or partially, identified by a patient safety event. Comment: These concepts are captured in other classes of the classification.

II 21. Organizational Response and Outcomes

II 22. The impact upon an organization or individuals in an organization which is wholly, or partially, attributable to a patient safety event.

II 23. e.g. financial redress for patient unit strengthened or downgraded

II 24. Add to examples:-socio/economic burden Comments from the Question – Do you believe this is a meaningful and useful class? III III 1. Not sure what you mean by this so difficult to determine its usefulness.

Comment: The term and definition have been revised.

III 2. Not clear on what organizational outcomes encompasses.

III 3. This is often an intermediate step, which follows from "actions taken" by others, including (a) courts, (b) government administrators, (c) insurance companies, third-party payers, and (d) accreditors. Shouldn't this box (and relable it "provider outcomes," too) be under a line of boxes labelled "patient impact/outcomes," "judicial response," "government administrative response," "accreditors' response," and "insurance company/third party payers' response"? If you want to acknowledge that organizations sometimes responded to their own events, maybe the box on this level should be labelled "internal organizational review," or something like that. Comment: Agreed. The temporal aspect is difficult to capture and that some of the information will be unknown depending on the data source.

III 4. The word Outcome is ok. If you find one word, and it is flexible then I think you should use just that word.

III 5. the type of impact should be specified. Is it financial, related to external image, self-image or what? Comment: These concepts are contained within the classification tree. Examples will be included.

III 6. I think this is very soft unless you were thinking financial impact. Most of the impact is borne by the patient and the staff directly involved. Comment: The Drafting Group considered this issue. A patient safety incident has wide implications. The concepts contained within the classification tree have been reviewed and revised as necessary.

III 7. These are not the outcomes which concern most of the healthcare organizations with which we deal. They are concerned mainly with adverse publicity, damage to reputation, inquiries, litigation or the suspension of services. This ICPS Chapter might usefully be deferred to some later version of the taxonomy, pending further review. Comment: See revised concepts. These concepts are contained within the classification tree. Examples will be included.

III 8. confusing term

III 9. Outcome is used in many other settings - risk for confusion

III 10. Same

Page 121 of 194

III 11. The complexity of healthcare delivery between countries and within countries with differing systems means the organizational outcomes are unique to the provider and not necessarily transferable as an import to another organization, unless such an organization is synchronous and in harmony with the solving organization. Comment: Agreed, thus the reason for embarking on the endeavor to develop the classification. There is a defined scope of key patient safety terms that can be standardized. We are attempting to achieve this.

III 12. Its difficult to understand what it means in practical work Comment: The definition has been revised. The concepts contained within this classification tree have been reviewed for pragmatic application.

III 13. You've chosen to give only 'negative consequences' as examples for organizational outcomes. What about the positive effects, such as safety reviews that lead to meaningful system change, stronger system resiliency in terms of safety, a stronger safety culture, or even, an honest admission of the lack of a safety culture. "A stumble may prevent a fall". Comment: These concepts are captured in other classes of the classification.

III 14. As per previous answer re. org/healthcare team

III 15. without information on the context this is not generalizable

III 16. This might be OK as long as what we are talking here is not about litigation and staff shock etc alone, but that the predominant emphasis it on managerial, professional and organizational remedial actions. Comment: The intent of this class is to learn from patient safety incidents and to improve preventive strategies. The concepts for negligence and liability have been removed from the classification.

III 17. See my earlier comment, but I think this phrase could easily be confused with "actions taken."

III 18. This is new type of information that the current classifications systems concentrated on person characters classifications do not cover. There is need for a new (hopefully) relatively short classification for this purpose. Comment: Agreed

III 19. It has meaning within an organization, but not within a microenvironment such as an individual's practice

III 20. Same comment as previously

Additional Comments from e-mails None Delphi Task Force Comments Feedback from Round 1 suggested there was a lack of clarity about the concepts contained within the Organizational Outcomes classification tree.

Page 122 of 194

2nd Round Delphi Comments: IV Please provide an explanation for why you find the term, Organizational Outcomes, or its associated definition to be unclear. IV 1. Here is an opportunity for two secondary classes that are not included in your diagram: #1 is the cost of

caring for the complication and whether the cost is paid by the insurer or the organization. In the U.S., there is a movement that hospitals will have to eat the cost of certain complications, separatge from their claims liability. #2 is the counseling toll on the provider, mentioned by you earlier in the survey. Good workers get lost to the system because of bad errors Comment: 1.) The implication is on the macro level, however the classification has been designed to allow for analysis on a micro level if reporting systems are designed to capture this information. 2.) Psychological harm is a problem. These concepts are contained in other classes of the ICPS.

IV 2. It is clear, but disgusting, that you are finding it more relevant to measure and accumulate "risk management" indicators at the organizational level than to measure and respond to the psychological harm experienced by healthcare providers involved in a patient safety event. Comment: We acknowledge that psychological harm is a problem. See the concepts contained in other classes of the classification.

IV 3. This is not patient safety, more closely related to risk management. Comment: A patient safety incident has wide implications. The classification aims to capture as much information relating to patient safety incidents as possible.

IV 4. Clarity is not the problem. The problem is that the input will be speculative and imprecise. A person who works in the legal department and reads nasty letters will generally have a different impression than the person in the patient safety office who may be aware of the many near misses that represent adverse events averted. The person who does the data entry, and even the day that they do the data entry (what their experience has been that day or week - e.g., any recent phone calls with sobbing family members?) will influence this to a large extent. Comment: We agree that different people will have different perspectives on patient safety this illustrates it multi-faceted nature of patient safety incidents compels organizations to seek a myriad of perspectives to get a true and full picture. Inter rate reliability will be tested during the next phase.

IV 5. The word social may have different connotations in different countries Comment: The terms and definitions of the concepts within this classification tree have been reviewed and revised.

IV 6. This is not a comment about the definition of organizational outcomes, but about economic services and systems. The 3 categories on the illustration are different from your text summary on the prior page which states: production, development and management of material wealth, and resources utilized within or by the organization. These seem more straight forward. Comment: The terms and definitions for the concepts contained in this classification tree have been reviewed and revised.

IV 7. The Social part of the Organizational Outcome does not contain "ethical" aspect. Comment: The terms and definitions for the concepts contained in this classification have been reviewed and revised.

Page 123 of 194

V Please provide an alternative term or definition, the relevant reference(s) to the sources and your rationale for including the alternative term or definition instead of that provided for Organizational Outcomes. V 1. The primary definition is fine as is.

V 2. N/A

V 3. Eliminate

V 4. What purpose this serves is unclear. A simpler point might be if a compensating payment to the patient or their family has been given or is anticipated. Comment: The conceptual framework and concepts have been refined to clarify intent.

V 5. Perhaps use cultural instead of social

V 6. See comment above.

V 7. I will suggest dividing the Organizational Outcomes into two. The first part will have two sub component namely Social and Ethical. The second part will have two component namely Economic and Legal. Comment: The concepts in this classification tree have been reviewed and refined based on these and other comments.

Comments from the Question – Do you believe this is a meaningful and useful class? VI VI 1. A little bit too formal, without regard of organizational change

VI 2. Are we interested in preventing harm to patients, or the public image/bottom line of organizations? They aren't necessarily incompatible, but to elevate the status of an organization above the people who make up that organization is not justifiable. Comment: There is certainly no intention in the structure of the classification to elevate the status of the organization above the people who make the organization. Please refer to the concepts contained in other classes of the classification.

VI 3. In practice, I think this will be very hard to use. Though it makes sense academically – I’m not sure how (or whether) it can be used. Comment: This will be determined through field testing.

VI 4. Not Patient safety. More closely related to risk management, and not productive for patient safety goals. Comment: A patient safety incident has wide implications. The classification aims to capture as much information relating to patient safety incidents as possible.

VI 5. The Organizational Outcome is irrelevant unless it will serve as a meaningful purpose. If so, what purpose? Comment: The plan is to give managers and staff an idea of the wider implication or impact of individual incidents – the revised concepts may make this clearer. The purpose of this class is to capture information to inform the development of a preventive strategies.

VI 6. These are too subjective and prone to randomness in how they are coded, based on who does the coding and their recent personal experiences. Comment: Business rules will address how to classify information in this and all the classes. This should improve inter-rater reliability.

Page 124 of 194

VI 7. Still work to be done on subclasses and their definitions Comment: Agreed. The concepts have been revised. The concepts in this classification tree have been reviewed and revised.

VI 8. again because of the "social" in the description Comment: The concepts in this classification tree have been reviewed and revised.

VI 9. Meaningful and useful if "organization" is defined. Comment: The term and definition have been revised.

VI

10.

In a world of healthcare delivery flux each organization will differ in its response and remedy. Further, the impact may be mono-effective and not universally applicable. It adds a tint, not colour, to the picture. Comment: Agreed that there will be major differences but standardization of the terms is the goal.

VI 11. as previous


Page 125 of 194

Appendix A – Round One Survey

WHO International Patient Safety Event Classification Delphi Survey To assure confidentiality, security and the integrity of the survey, please provide your e-mail address: E-Mail Address: _______________________________________

Page 126 of 194

PARTICIPANT DEMOGRAPHICS: Age in years: _________________________________________ Gender: Female

Male Country: _____________________________________________ Native Language: ______________________________________ Are you a health care professional (e.g., physician, nurse, pharmacist, dentist)?

Yes No

If you are a health care professional, what is your primary area of healthcare activities?

Anaesthesia Administrative Services Allied Health Dietetics Breast Surgery Cardiothoracic Surgery Cardiology Care of the Elderly (Geriatrics) Clinical Immunology and Allergy Dental Medicine and Surgery Dermatology Diagnostic Services Emergency Medicine Endocrinology Gastroenterology General Medicine General Nursing General Surgery Genetics Gynaecology Haematology Infectious Diseases Intensive Care Maxillo-facial Surgery Medical Oncology

Neonatology Nephrology/Renal Neurology Neurosurgery Obstetrics Ophthalmology Orthopaedics Otolaryngology Pain Services Palliative Medicine Paediatrics Pharmacy Physiotherapy (Physical Therapy) Plastic Surgery Podiatry Primary Care Psychiatry Public Health Rehabilitation Radiation Oncology Rheumatology Speech and Language Therapy Thoracic/Respiratory Medicine Vascular Surgery Urology

Other (please specify): ____________________________________________

Page 127 of 194

Expertise (please check all that apply):

Health policy expert (e.g., health policy decision maker within governmental health agencies or Ministries of Health)

Developer and/or manager of patient safety reporting systems

Patient/public representative (e.g., consumer and/or representative from consumer organizations)

Academician (e.g., expertise in patient safety, health information technology, public health, quality improvement, safety in other industries, etc.)

Representative from professional associations for clinicians

Representative from litigation and/or defense bodies

Classification/taxonomy expert

Risk manager (e.g., those with responsibility for patient safety and quality improvement at an operational level)

Representative from organizations responsible for assessing and monitoring patient safety performance (e.g., governmental or non-governmental organizations, accrediting bodies)

Other (please specify): ____________________________________________

Do you have practical experience in collecting, classifying and/or analyzing patient safety data?

Yes No

Page 128 of 194

The Conceptual Framework Patient safety is defined as the freedom from accidental injury. This includes the establishment of processes to minimize the possibility of adverse events and maximize the ability to detect and recover from or lessen the impact of a potential patient safety hazard. The Conceptual Framework for the International Patient Safety Event Classification (below) is designed to capture the salient details of all events which could have or did lead to unintended and/or unnecessary harm to a person receiving health care. It is comprised of ten framework classes, two of which are highlighted as core. The core classes are mainly associated with grouping events into recognisable categories, whilst the other classes are mainly associated with learning activities. The Conceptual Framework illustrates the logic model developed to classify information about patient safety events.


Influences

Event types

Patient characteristicsRecovery

factors



Actions taken

Mitigating factors

Lead to


Lead to

Organization outcomes

Event modifiers


Conceptual Framework for the International Patient Safety Event Classification

Influences

Influences

Framework class


Have

Page 129 of 194

How to read the Conceptual Framework

Page 130 of 194

Page 131 of 194

Is the Conceptual Framework an adequate model for use in describing a patient safety event?

Yes as is Yes with modification No Unsure

If you answered “Yes with Modification” or “No”, please: Suggest improvements and provide a rationale for your suggested changes

Page 132 of 194

Do you believe any classes are missing from the Conceptual Framework?

Yes No

If you answered “Yes”, please answer the following questions: Please indicate which class(es) are missing: Please indicate where the class(es) should be incorporated: Please provide a rationale for adding the suggested class(es):

Page 133 of 194

The Conceptual Framework is a meaningful and useful tool for translating disparate information into a format conducive to learning and improving patient safety.

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement “The Conceptual Framework is a meaningful and useful tool for translating disparate information into a format conducive to learning and improving patient safety”, please explain why you choose your answer:

Page 134 of 194

DEFINITIONS We are interested in determining whether the terms and definitions for each of the classes contained within the Conceptual Framework are clear, precise and provide an accurate representation of the nature, properties, scope and essential qualities of the concept. Please review the terms and associated definitions for clarity, precision and accuracy. Term/Definition Clear Unclear Event Type – The characteristics distinguishing a group or class of patient safety events. Note: A patient safety event is a process or act of omission or commission that resulted in hazardous health care conditions and/or unintended harm to the patient.

Patient Impact/Outcomes – The result of the performance (or non-performance) of a function or one or more processes, services, or activities carried out in a health care context affecting the patient.

Patient Characteristics – The distinguishing features or traits specific to the patient (e.g., age, gender, functioning).

Event Characteristics – The distinguishing features or traits specific to the event and the circumstances or facts that surround the event (e.g., care setting, phase of care, persons involved, time and date).

Contributing Factor – A circumstance, action or influence which plays a part in the genesis of a patient safety event.

Actions Taken – Something done by an individual or a health care organisation to prevent, remedy or mitigate the occurrence or reoccurrence of a real or potential patient safety event.

Preventive Factors – An effort or action taken in advance to pre-empt an occurrence of a patient safety event.

Recovery Factors – An action taken to stop circumstances or influences from turning into a patient safety event.

Mitigating Factors – A circumstance, action or influence that played a role in diminishing the impact of a patient safety event.

Organisational Outcomes – The impact upon an organisation which is wholly, or partially, attributable to a patient safety event.

Page 135 of 194

If you believe any of the terms and/or definitions are unclear please: 1. Indicate the term 2. Please provide an explanation for why you find the term and/or definition

to be unclear 3. Please provide an alternative term or definition, the relevant reference(s)

to the sources and your rationale for including the alternative term/ definition instead of that provided

Please use additional sheets if necessary Term Explanation Alternative

Page 136 of 194

CLASSES We are interested in determining whether the classes within the Conceptual Framework will provide you with useful information to identify and learn from patient safety events. Please review the classes in terms of their meaningfulness and usefulness.

Page 137 of 194

The class, Event Types, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Event Types, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 138 of 194

The class, Patient Impact/Outcomes, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Patient Impact/Outcomes, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 139 of 194

The class, Patient Characteristics, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Patient Characteristics, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 140 of 194

The class, Event Characteristics, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Event Characteristics, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 141 of 194

The class, Contributing Factors, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Contributing Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 142 of 194

The class, Actions Taken, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Actions Taken, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 143 of 194

The class, Preventive Factors, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Preventive Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 144 of 194

The class, Recovery Factors, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Recovery Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 145 of 194

The class, Mitigating Factors, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Mitigating Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 146 of 194

The class, Organisational Outcomes, is a meaningful and useful class within the International Patient Safety Event Classification.


If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Organisational Outcomes, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 147 of 194

OVERALL COMMENTS Please provide us with your comments regarding the Conceptual Framework for the International Patient Safety Event Classification. Thank you for answering the questions regarding participant demographics and the Conceptual Framework. Your input is extremely valuable to us and vital to the development of the International Patient Safety Event Classification.

Page 148 of 194

Appendix B – Round Two Survey

WHO IPSEC - Delphi Survey Round 2 LOGIN: PARTICIPANT LOGIN Please enter your e-mail address to begin the survey. To assure confidentiality, security and the integrity of this survey, please login with your e-mail address. Logging into this survey is particularly important should you decide to complete this survey in more than one session. We thank you for your understanding. E-Mail Address: _______________________________________

Page 149 of 194

WHO IPSEC - Delphi Survey Round 2 Overview of Feedback from Round 1 A total of 253 individuals with 29 different languages responded from 43 countries. The respondents represented health care professionals, health policy experts, developers/managers of patient reporting systems, patient/public representatives, academicians, representatives from professional associations for a variety of health care specialties, representatives from litigation, classification/taxonomy experts, risk managers and representatives from organizations responsible for assessing and monitoring patient safety performance responded. Of those that responded:

• 81% have practical experience in collecting, classifying and/or analysing patient safety data;

• 86% believed the conceptual framework was an adequate model for use in describing a

patient safety event;

• 81% believed the conceptual framework was a meaningful and useful tool for translating disparate information into a format conducive to learning and improving patient safety; and

• 50% thought the conceptual framework was missing at least one class.

On average,

• 82% felt that the terms and definitions for each class contained within the conceptual framework were clear, precise and provided an accurate representation of the nature, properties, scope and essential qualities of the concept; and

• 89% felt that the 10 classes within the conceptual framework were meaningful and useful.

We reviewed and considered all of the comments provided. Based on both the quantitative and the qualitative analysis, the main issue was the need to clarify the: Purpose of the IPSEC;

• Structure and depth of the conceptual framework; • Intention of the IPSEC to include both adverse events and near misses; • Ability of the conceptual framework to serve as a model to classify a patient safety event;

and • Concepts contained within each of the classes.

Page 150 of 194

WHO IPSEC - Delphi Survey Round 2 The IPSEC Many patient safety classifications exist. No one classification, however, is fit for purpose on the global level. As a result, there is little capacity to share information internationally. Better classification of information related to patient safety requires universal agreement and understanding of key terms and concepts as well as a standardized method for examining patient safety data.</br> The IPSEC is based upon:

• The Joint Commission on Accreditation of Healthcare Organization’s Patient Safety Event Taxonomy (PSET) endorsed by the National Quality Forum in the United States

• The National Patient Safety Agency’s National Reporting and Learning System (NRLS) in the

United Kingdom

• The Australian Patient Safety Foundation’s Advanced Information Management System (AIMS)

• The Eindhoven University of Technology’s Eindhoven Classification Model (ECM) from the

Netherlands It was designed to link closely with the WHO – Family of International Classifications as a Derived Classification. Because of this, it is directly linked to the WHO’s:

• International Statistical Classification of Diseases and Related Health Problems (ICD)

• International Classification of Functioning, Disability and Health (ICF)

• International Classification of Health Interventions (ICHI)

• The WHO Drug Dictionary. The purpose of the IPSEC is to:

• Translate information into a common terminology;

• Permit the systematic collection of information about patient safety events (adverse events and near misses);

• Enable data to be aggregated and compared; and

• Allow for international analysis and learning.

The IPSEC aims to define, harmonize and group patient safety concepts into an internationally agreed upon classification in a way that is conducive to learning and improving patient safety across systems.

Page 151 of 194

WHO IPSEC - Delphi Survey Round 2 The IPSEC is a classification not a reporting system The IPSEC is a classification not a reporting system. Reporting systems provide the method to collect patient safety data and work in conjunction with classifications. Classifications provide the underlying concepts and definitions and organize these concepts in a hierarchical structure in order to translate data collected from disparate reporting systems or other data sources into a standardized language. This allows for aggregation, comparison and learning across reporting systems. The IPSEC is a classification which is intended to:

• Be adaptable yet consistent across the spectrum of health care;

• Link closely with the WHO-Family of International Classifications as a Derived Classification;

• Capture adverse event and near miss data;

• Add value to systems already in use;

• Be a tool to organize patient safety information;

• Accommodate data obtained from a variety of sources (incident monitoring, medical record review, root cause analysis, medico-legal cases);

• Be sensitive to cultural and linguistic issues; and

• Strive for maximum comparability of patient information on an international level.

Page 152 of 194

WHO IPSEC - Delphi Survey Round 2 The IPSEC is structured by a Conceptual Framework A conceptual framework for the domain of patient safety was developed for the IPSEC. The conceptual framework is a purposeful construct of patient safety events, rooted in expert practice, and captured by specific concepts and the relationships between these concepts. The conceptual framework for the International Patient Safety Event Classification is shown below.

Page 153 of 194

WHO IPSEC - Delphi Survey Round 2 The Conceptual Framework The conceptual framework of the IPSEC

• Is comprised of 10 high level stable classes with distinct and unambiguous concepts described by a term;

• Is hierarchical (“parent/child” relationships); and

• Has attribute type relationships between classes. This structure demonstrates the relationship between and amongst the concepts contained within all levels of the ten high level classes. The 10 high level classes are comprised of:

• 45 second level concepts • 226 third level concepts

• 159 fourth level concepts

• 42 fifth level concepts

Two high level classes are considered core classes because they are mainly associated with grouping events into recognizable categories. Eight classes are mainly associated with learning. The conceptual framework is intended to be comprehensive and inclusive of a wide range of circumstances when people come into contact with the health system. Information to populate each of the classes may not always be available. It is believed that information on the two core classes (Event Type and Patient Impact/Outcomes) will be generally available and captured because together they provide a description of what happened to the patient.

Page 154 of 194

WHO IPSEC - Delphi Survey Round 2 Three Important Terms and Concepts A Patient is any individual who is the recipient of health care services in any health care setting. Patient safety is the freedom from accidental injury or potential patient safety hazards. As one dimension of quality of care, patient safety is concerned primarily with the prevention and mitigation of or the recovery from adverse events and near misses stemming from the provision of health care. A Patient Safety Event is a process or act of omission or commission that resulted in unintended harm to the patient or hazardous health care conditions that could have resulted in unintended harm to a patient. Anything that could have or did go awry is a patient safety event. The type of patient safety event is captured under the Event Types class. The result of the event, including “near miss”, is captured under the Patient Impact/Outcome class. Together, the concepts contained in these two classes provide a description of what happened to the patient. Therefore a patient safety event is either an adverse event or a near miss. Event types that result in harm to patients are considered “adverse events”. Event types that result in hazardous health care conditions but do not result in harm to patients are considered “near misses”.

Page 155 of 194

WHO IPSEC - Delphi Survey Round 2 The Classes Event Types comprise high level descriptions of patient safety events. An event type is a group or class of patient safety events distinguished by their characteristics. Patient Impact/Outcomes indicate the effects of the patient safety event upon the patient such as a physical and/or psychological outcome or a near miss, and the severity and duration of the impact. Patient Characteristics are the distinguishing features or traits specific to the patient. These include patient demographics, patient functioning (the patient’s physiological or psychological ability to perform duties or fulfill roles prior to entering any phase of care), patient admission diagnosis (the primary disease or injury for which the patient is receiving health care), and patient procedures (the procedure employed by a health care professional which resulted in the patient safety event). Event Characteristics are the distinguishing features or traits specific to the event and the circumstances or facts that surround the event. These include care setting, phase of care, specialties involved, person reporting (including the patient), people involved, date/time of the event, and agent involved (medication, medical devices/equipment, blood and blood products). A Contributing Factor is a circumstance, action or influence which plays a part in the origin or development of a patient safety event. These include human and performance factors, work environment factors, patient factors, organisational factors and external factors. These concepts have been informed by or derived from the theoretical models of complex system errors, human factors engineering and just culture research. Actions Taken are things done by an individual or health care organisation to prevent, remedy or mitigate the occurrence or reoccurrence of the patient safety event. These include clinical and organisational actions. Preventive Factors are efforts or actions taken in advance to pre-empt an occurrence of a patient safety event. These include assessment of risk vulnerabilities and human, social, work environment and organisational factors. Recovery Factors are actions taken to stop the progression of a patient safety event from harming the patient. Recovery factors are not equivalent to clinical recovery factors (e.g., recuperation). Recovery factors are also informed by or derived from the theoretical models of complex system errors, human factors engineering and just culture research. Mitigating Factors are circumstances, actions or influences that play a role in diminishing the harm generated by a patient safety event to the patient (e.g., a patient safety event has occurred but action was taken to avert it from having a more deleterious effect on the patient). Organisational Outcomes describe the impact upon the organisation which is wholly or partially attributable to a patient safety event. These include social outcomes (reputation, support & relationships, or shared set of beliefs, values, norms), economic outcomes (production, development & management of material wealth, or resources utilized within or by the organisation), and legal outcomes (civil protection, political policies or litigation). It should be noted that organisational outcomes may not be known at the time of the patient safety event.

Page 156 of 194

WHO IPSEC - Delphi Survey Round 2 The IPSEC's conceptual framework as a model to classify The conceptual framework can be used as a model to classify a patient safety event as shown below. The numbers correspond to the numbers in the diagram. 1 Identify what happened to the patient 2 Identify the results of the event upon the patient 3 & 4 Identify the pertinent information about the patient and event 5 Identify what factors lead to the event 6 Identify what was done to prevent, mitigate or recover from the event 7, 8 & 9 Identify the defenses or barriers built into the system to counteract, avert or reduce

the occurrence/recurrence of the event 10 Identify the impact of the event on the organisation



Influences

Event types

Patient characteristics Recovery factors



Actions taken

Mitigating factorsLead to


Lead to

Organization Outcomes

Event modifiers


Influences

Influences

Framework classCore framework class

Have

1

2

3

4

5

6

7

8

9

10

Page 157 of 194

WHO IPSEC - Delphi Survey Round 2 The conceptual framework as a model to classify an event - Example 1 Example 1: A 33-year-old man with type 2 diabetes presented to his physician’s office to discuss his diabetes management. The patient admitted not taking his medications or checking his blood sugars regularly. In the office, his blood sugar was 335 ml/dL. The nurse practitioner ordered 6 units of regular insulin to administer to the patient. After the medical assistant brought the insulin and syringe, the nurse practitioner prepared the medication and injected the insulin. Immediately after the injection, the nurse practitioner discovered that a tuberculin syringe was used instead of an insulin one. As a result of the error, the patient inadvertently received 60 units of insulin rather than 6 units. The patient was given orange juice, a sandwich, and his blood sugars were closely monitored for 4 hours with no significant events. The nurse practitioner and the patient reported the event. Please keep in mind how the conceptual framework can be used as a model to classify a patient safety event and the granularity of each high level class. The numbers correspond to the numbers in the diagram. 1 Identify what happened to the patient 2 Identify the results of the event upon the patient 3 & 4 Identify the pertinent information about the patient and event 5 Identify what factors lead to the event 6 Identify what was done to prevent, mitigate or recover from the event 7, 8 & 9 Identify the defenses or barriers built into the system to counteract, avert or reduce

the occurrence/recurrence of the event 10 Identify the impact of the event on the organisation



Influences

Event types

Patient characteristics Recovery factors



Actions taken

Mitigating factorsLead to


Lead to

Organization Outcomes

Event modifiers


Influences

Influences

Framework classCore framework class

Have

1

2

3

4

5

6

7

8

9

10

Page 158 of 194

WHO IPSEC - Delphi Survey Round 2 Example 1 cont. Below is the same example classified using the conceptual framework. Again, the numbers correspond to the numbers in the diagram. 1 Event Type – The patient was administered the wrong dose of insulin because the medication was injected using a tuberculin syringe instead of an insulin syringe. The event type is classified as a direct impact upon the patient involving agents, specifically the preparation/dispensing and administration of medication and the use of a medical device/equipment which was inappropriate for the task. 2 Patient Impact/Outcomes – The patient suffered a mild, temporary physical outcome. The patient’s endocrine system was affected because he received 60 units of insulin instead of 6 units. The patient suffered no significant events. 3 Patient Characteristics – The patient was a 33 year old male. The primary determination of the nature of the disease for which he was receiving health care was Type 2 diabetes. The procedure that resulted in the event was that the patient was injected with 60 units of insulin instead of 6 units because the wrong syringe was used to administer the medication.

Page 159 of 194

WHO IPSEC - Delphi Survey Round 2 Example 1 cont. 4 Event Characteristics – The patient was seen in his physician’s office (primary medical care setting) for treatment of his diabetes. A nurse and a member of the allied health personnel were involved in the event. The nurse and the patient reported the event. The agent involved was medication (insulin).

Page 160 of 194

WHO IPSEC - Delphi Survey Round 2 Example 1 cont. 5 Contributing Factors – The nurse and the member of the allied health personnel failed to use the correct syringe to inject the insulin. Human and performance factors, specifically cognitive (rule based and skill based) and behaviour factors (mistake and perception), contributed to the event. 6 Actions Taken – The patient was given orange juice and a sandwich. In addition, the patient’s blood sugar level was monitored for 4 hours. 7 Preventive Factors – None documented in the example. 8 Recovery Factors – None documented in the example. 9 Mitigating Factors – None documented in the example. 10 Organisational Outcomes – None documented in the example.

Page 161 of 194

WHO IPSEC - Delphi Survey Round 2 The conceptual framework as a model to classify an event - Example 2 Example 2: A 55-year-old man with acute myelogenous leukaemia and several recent hospitalizations for fever and neutropenia presented to the emergency department with fever. After assessment by the emergency physician, administration of empiric broad-spectrum antibiotics, the patient was assessed by his oncologist. The oncologist added an order for Diflucan (fluconazole) 100 mg IV to cover a possible fungal infection. Because intravenous fluconazole was not kept in the emergency department, the nurse phoned the pharmacy to send the medication as soon as possible. She spoke with a pharmacy technician. A 50 ml bottle of Diprivan (propofol, an intravenous sedative-hypnotic commonly used in anaesthesia) that had been mistakenly labeled in the pharmacy as “Diflucan 100 mg/50 ml)” was sent to the emergency department. Because the nurse also worked in the medical intensive care unit, she was quite familiar with both intravenous Diflucan and Diprivan. When a glass bottle containing an opaque liquid arrived instead of the plastic bag containing the clear solution the nurse expected, she realized something was amiss. The nurse did not administer the medication, contacted the pharmacist to obtain the correct medication and reported the event. Below is the same example classified using the conceptual framework. 1 Event Type – The patient was almost administered the wrong medication (Diprivan instead of Diflucan). The event type is classified as a direct impact upon the patient involving agents, specifically the preparation/dispensing of medication. 2 Patient Impact/Outcomes – There was no outcome of the event or impact upon the patient. The outcome is classified as a “near miss – event did not reach the patient” because the event was prevented from developing into an actual event as a result of prevention, mitigation or recovery. The impact, severity, is classified as “none”.

Page 162 of 194

WHO IPSEC - Delphi Survey Round 2 Example 2 cont. 3 Patient Characteristics – The patient was a 55 year old male. The primary determination of the nature of the disease for which he was receiving health care was comorbid conditions resulting from acute myelogenous leukaemia. The pharmacotherapeutic intervention could have resulted in an adverse event.

Page 163 of 194

WHO IPSEC - Delphi Survey Round 2 Example 2 cont. 4 Event Characteristics – The patient was seen in the hospital as an outpatient in the emergency department. He sought health care for fever resulting from a possible fungal infection. Emergency medicine, general nursing, oncology and pharmacy health care staff (nurse, doctors, a pharmacist and a pharmacy technician) were involved in the event. The nurse reported the event. The agent involved was medication (Diprivan versus Diflucan).

Page 164 of 194

WHO IPSEC - Delphi Survey Round 2 Example 2 cont. 5 Contributing Factors – Because the medication was not kept in the emergency department, the nurse ordered it from the pharmacy. The pharmacy technician failed to provide the correct medication. Factors which played a part in the genesis of the event included (a) human and performance factors - cognitive (rule based and skill based) and behaviour factors (at-risk behaviour); (b) communication with staff; and (c) work environment factors – physical environment factors. 6 Actions Taken – The nurse did not administer the medication and contacted the pharmacist. 7 Preventive Factors – None documented in the example. 8 Recovery Factors – The nurse identified that the wrong medication was sent from the pharmacy. The medication was in the wrong package and was of a different consistency. The nurse took the following actions to stop the circumstances from turning into an adverse event: (a) She applied acquired knowledge to a novel situation (knowledge based factors); (b) She monitored the status of the process and thus detected the differences in medication; and (c) She communicated with the pharmacist to obtain the correct medication. 9 Mitigating Factors – None documented in the example. 10 Organisational Outcomes – None documented in the example.

Page 165 of 194

WHO IPSEC - Delphi Survey Round 2 SURVEY Thank you for taking the time to read the preceding information. It has hopefully provided you with useful information to answer the questions in the remainder of the survey.

Page 166 of 194

WHO IPSEC - Delphi Survey Round 2 Is the conceptual framework an adequate model to classify a patient safety event? Feedback from Round 1 Of the respondents 60.7% answered “Yes as is” 25% answered “Yes with modification” 2.4% answered “No” 11.9% answered “Unsure” Feedback from Round 1 (a) indicated there was confusion regarding the point at which the event modifiers influence an event; and (b) suggested reordering the classes in the conceptual framework. Response to Feedback In some systems, contributing factors can have a positive or negative effect. In the IPSEC, contributing factors are considered to be negative factors. Preventive factors are inversely related to contributing factors – the management of a contributing factor can prevent the patient safety event from occurring. Recovery factors enable a care giver to regain control over a situation to stop the patient safety event from resulting in an adverse event. Mitigating factors also are inversely related to contributing factors – the management of a contributing factor can hinder a contributing factor’s ability to make the outcome worse. The IPSEC’s conceptual framework is a logic model designed to capture the salient details of all patient safety events by following pre-established criteria. The model formulates an explanatory construct to fulfill the intended purpose of classifying patient safety events. It enables information to be deconstructed in a way that facilitates aggregation, comparison, analysis and knowledge development. As illustrated in the examples, the conceptual framework is able to serve as a model to classify patient safety events. Modifications Made as a Result of Feedback No modifications have been made to the conceptual framework. Based on the above, is the conceptual framework an adequate model for use in classifying a patient safety event? o Yes as is o Yes with modification o No o Unsure If you selected “Yes with Modification” or “No”, please suggest improvements and provide a rationale for your suggested changes

Page 167 of 194

WHO IPSEC - Delphi Survey Round 2 Is the conceptual framework missing classes? Feedback from Round 1 Of the those that responded 50.7% answered “Yes” 49.3% answered “No” Feedback from Round 1 focused on: (a) concepts respondents felt were missing; and (b) expanding the scope of the IPSEC. Response to Feedback While it may appear that half of the respondents felt the conceptual framework was missing classes, the majority of comments were about “concepts” contained within classes, rather than about particular “classes” that should be included within the conceptual framework. For example, some comments referred to the need to include the patient’s medical condition/ability to function prior to the patient safety event as a class within the conceptual framework. The patient’s medical condition/ability to function is a concept (Patient Functioning) which is included under two classes: Patient Characteristics and Contributing Functions class. Other comments referred to information that is traditionally captured by reporting systems instead of classifications; for example, hospital specific information that only is germane to the individual health care organisation. The comments with respect to expanding the scope of the IPSEC suggested that the classification be able to capture information pertaining to individuals in addition to patients. For example, the IPSEC should include a class to capture information about the impact of the patient safety event upon the health care provider. After significant discussion, the drafting group decided to limit the scope of the IPSEC to patients and patient safety events at this time. Modifications Made as a Result of Feedback No modifications have been made to the conceptual framework.

Page 168 of 194

WHO IPSEC - Delphi Survey Round 2 Is the conceptual framework missing classes? Cont. In light of the above, please review the conceptual framework and then answer the question that follows.

Do you believe any classes are missing from the Conceptual Framework? o Yes o No If you selected “Yes”, please explain what classes you believe are missing from the Conceptual Framework.

Page 169 of 194

WHO IPSEC - Delphi Survey Round 2 Is the IPSEC a meaningful and useful tool for translating disparate information? Feedback from Round 1 Of the those that responded 18% answered “Strongly Agree” 70.1% answered “Agree” 9% answered “Neither Agree nor Disagree” 2% answered “Disagree” 0.8% answered “Strongly Disagree” The feedback from Round 1 centered on (a) the complexity of conceptual framework; and (b) the applicability of the IPSEC to both micro and macro environments. Response to Feedback Eighty-eight point one percent (88.1%) strongly agreed or agreed that the IPSEC was a meaning and useful tool for translating disparate information into a format conductive to learning and improving patient safety. Modifications Made as a Result of Feedback No modifications have been made to the conceptual framework. o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree or (3) strongly disagree, with the statement: The International Patient Safety Event Classification is a meaningful and useful tool for translating disparate information into a format conducive to learning and improving patient safety. Please indicate why you chose your answer:

Page 170 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Event Types clear, precise and accurate? Feedback from Round 1 Of the those that responded 79.7% answered “Clear” 20.3% answered “Unclear” The feedback from Round 1 concerned (a) the Event Types class appeared to address only adverse events and not near misses; and (b) the type of events included. Response to Feedback As explained earlier, the event type is one concept in the equation to determine if the patient safety event was an adverse event or a near miss. The other concept, outcome, is contained in the Patient Impact/Outcome class:

Event Type + Patient Outcome = Type of Patient Safety Event (adverse event or near miss) This class provides high level descriptions of patient safety events. There are two second level concepts, 12 third level concepts, 32 fourth level concepts and 30 fifth level concepts. These cover a wide range of event types. Event types with a direct impact upon the patient resulting from processes or acts of omission or commission include clinical management, documentation and communication, health care associated infection, agents (medication, medical equipment/devices, blood and blood products), patient behaviour, abscondence/missing/left against medical advice, and patient accidents. Event types that result from an existing but not yet developed element within the health care system include organisational factors such as infrastructure and resources. Modifications Made as a Result of Feedback The definition of event type has been clarified based upon the concerns raised. Original definition - “the characteristics distinguishing a group or class of patient safety events.” Revised definition – “a group or class of patient safety events distinguished by their characteristics.” In light of the above information, please indicate whether the revised definition is clear in light of the above information. An event type is a group or class of patient safety events distinguished by their characteristics o Clear o Unclear If you find the revised definition for Event Types to be unclear, please provide an explanation and an alternative definition:

Page 171 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Patient Impact/Outcomes clear, precise and accurate? Feedback from Round 1 Of the those that responded 79.4% answered “Clear” 20.6% answered “Unclear” The feedback from Round 1 indicated the definition for Patient Impact/Outcomes was confusing. Response to Feedback The definition of patient impact/outcomes has been clarified. Modifications Made as a Result of Feedback Original definition - “the results of the performance (or non-performance) of a function or one or more processes, services, or activities carried out in a health care context affecting the patient.” Revised definition – “the effects of the patient safety event upon the patient.” Please indicate whether the revised definition is clear. Patient Impact/Outcomes are the effects of the patient safety event upon the patient. o Clear o Unclear If you find the revised definition for Patient Impact/Outcomes to be unclear, please provide an explanation and an alternative definition:

Page 172 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Patient Characteristics clear, precise and accurate? Feedback from Round 1 Of the those that responded 88.2% answered “Clear” 11.8% answered “Unclear” Round 1 feedback suggested concepts that should be included in the Patient Characteristics class. Response to Feedback This classification tree contains 48 second and third level concepts covering patient demographics (age, gender, native language), patient functioning (the patient’s physiological or psychological ability to perform duties or fulfill roles prior to the patient entering any phase of care), patient admission diagnosis (the primary determination of the nature of a disease or injury for which the patient is receiving health care), and patient procedures (the procedure employed by a health care professional which resulted in the patient safety event). Patient Functioning maps directly to the WHO’s International Classification of Functioning, Disability and Health. Patient Admission Diagnosis links to the 20 chapters contained in the International Statistical Classification of Diseases and Related Health Problems (ICD-10). Patient Procedures are tied to the International Classification of Health Interventions. Modifications Made as a Result of Feedback The definition for Patient Characteristics was not modified. We hope the information above addresses the concerns raised in the feedback. o The definition is still unclear o I have no further comment on this definition If you still find the definition for Patient Characteristics to be unclear, please provide an explanation and an alternative definition:

Page 173 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Event Characteristics clear, precise and accurate? Feedback from Round 1 Of the those that responded 91.3% answered “Clear” 8.7% answered “Unclear” Round 1 feedback suggested the concepts contained in the Event Characteristics class should address the “who, what, where, when, and how” of a patient safety event. Response to Feedback The Event Characteristics classification tree includes concepts addressing the “who, what, where, and when”. Both the Event Characteristics and the Patient Characteristics classification trees include concepts about the “what” (Patient Characteristics - Patient Procedures). The Contributing Factors, Preventive Factors, Recovery Factors and Mitigating Factors classification trees together address the “how”. The Event Characteristics classification tree covers:

• Who: specialties involved, person reporting (including the patient), people involved • What: Agent involved (medication, medical device/equipment and blood and blood

products) • Where: Care setting, hospital treatment status, country • When: Phase of care, timing of event, date of the event

Modifications Made as a Result of Feedback The definition for Event Characteristics was not modified. We hope the information above addresses the concerns raised in the feedback. o The definition is still unclear o I have no further comment on this definition If you still find the definition for Event Characteristics to be unclear, please provide an explanation and an alternative definition:

Page 174 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Contributing Factors clear, precise and accurate? Feedback from Round 1 Of the those that responded 81.6% answered “Clear” 18.4% answered “Unclear” The following concerns were raised by the feedback from Round 1: 5. The word “genesis” in the definition: Respondents noted the word “genesis” may have different

connotations and meanings in different countries. Respondents also indicated that contributing factors can influence a patient safety event at any time.

6. The interplay between Contributing Factors and the Event Modifying classes (Preventive

Factors, Recovery Factors And Mitigating Factors) 7. Causation versus Contributing Factors 8. Inclusion of particular concepts within the Contributing Factors classification tree Response to Feedback The word “genesis” was removed from the definition of Contributing Factors to be sensitive to cultural and linguistic issues and to indicate that a contributing factor can play a part in a patient safety event at any time. The IPSEC considers contributing factors to be negative. Preventive and mitigating factors are positive and thus inversely related to contributing factors. Recovery factors interrupts and stops a contributing factor. The drafting group discussed the concepts of “causal” versus “contributing” factors at length while designing the IPSEC. The decision was made to focus on “contributing” factors because it was more encompassing. Further, it is not possible to determine causation at the actual time of the event; additional investigation (e.g., root cause analysis) is required. The Contributing Factors classification tree includes six (6) second level concepts, 21 third level concepts and 31 fourth level concepts. Modifications Made as a Result of Feedback The definition of contributing factors has been clarified. Original definition - “a circumstance, action or influence which plays a part in the genesis of a patient safety event” Revised definition – “a circumstance, action or influence which plays a part in the origin or development of a patient safety event.”

Page 175 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Contributing Factors clear, precise and accurate? Cont. Please indicate whether the revised definition is clear. Contributing Factor – a circumstance, action or influence which plays a part in the origin or development of a patient safety event. o Clear o Unclear If you find the revised definition for Contributing Factors to be unclear, please provide an explanation and an alternative definition:

Page 176 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Actions Taken clear, precise and accurate? Feedback from Round 1 Of the those that responded 69.2% answered “Clear” 30.8% answered “Unclear” The feedback from Round 1 suggested clarifying what concepts are contained within the Actions Taken classification tree. Response to Feedback The Actions Taken classification tree includes clinical actions (the diagnostic and/or therapeutic actions taken to prevent or remedy the patient safety event) and organisational actions to address a particular function or process (protocols/policies/procedures/processes, organizational culture, resources, work environment). Modifications Made as a Result of Feedback The definition for Actions Taken was modified by deleting the extraneous words “real or potential”. Original definition - “something done by an individual or the health care organisation to prevent, remedy or mitigating the occurrence or recurrence of a real or potential patient safety event.” Revised definition – “something done by an individual or the health care organisation to prevent, remedy or mitigate the occurrence or reoccurrence of a patient safety event.” Please indicate whether the revised definition is clear. An action taken is something done by an individual or the health care organisation to prevent, remedy or mitigate the occurrence or reoccurrence of a patient safety event. o Clear o Unclear If you find the revised definition for Actions Taken to be unclear, please provide an explanation and an alternative definition:

Page 177 of 194

WHO IPSEC - Delphi Survey Round 2 Definitions - Preventive, Recovery, Mitigating Factors Round 1 feedback uncovered confusion about the definitions and roles of Preventive Factors, Recovery Factors and Mitigating Factors (“event modifying classes”), as well as their relationship to Contributing Factors. Considering the three event modifying classes together illustrates and clarifies the differences in definitions and roles. Preventive Factors are efforts or actions taken in advance to pre-empt an occurrence of a patient safety event. Recovery Factors are actions taken to stop the progression of a patient safety event from harming the patient. Mitigating Factors are circumstances, actions or influences that play a role in diminishing the harm caused to the patient by a patient safety event. Preventive Factors stop a patient safety event before it even starts by controlling for known or unknown contributing factors through identification using risk assessment models, human factors engineering or organizational policies. Examples include guidelines for hand hygiene in health care, use Failure Modes and Effects Analysis to identify potential process failures, procedures to prevent wrong-site surgery, and the provision of educational and training programs. Recovery Factors halt the advancement of the patient safety event from reaching the patient through planned or unplanned actions. Examples include use of circuit disconnect alarms, planned rule-based or skill-based actions in anticipation of a difficult endotracheal intubation, or ad-hoc creative problem solving using knowledge-based factors. Mitigating Factors reduce or minimize the consequences of a patient safety event upon the patient and can hinder a contributing factor’s ability to have a more deleterious effect on the patient. The most well known example is administering treatment in response to an adverse drug event. All three event modifying classes are important for learning. Preventive Factors and Mitigating Factors are established in the patient safety field. Recovery from a patient safety event is an emerging concept central to knowledge building to develop new or revise strategies to improve patient safety. Recovery factors are not equivalent to clinical recovery factors (e.g., recuperation). They are premeditated or spontaneous actions that are taken to intervene once the process leading to a patient safety event has begun thus stopping the situation from reaching the patient. The following three pages asks about whether the definitions for Preventive Factors, Recovery Factors and Mitigating Factors are clear, precise and accurate.

Page 178 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Preventive Factors clear, precise and accurate? Feedback from Round 1 Of the those that responded 89.2% answered “Clear” 10.8% answered “Unclear” The feedback from Round 1 indicated the concept of Preventive Factors and its relationship with Contributing Factors, Recovery Factors and Mitigating Factors was unclear. Response to Feedback The concept for Preventive Factors and its relationship with Contributing Factors, Recovery Factors and Mitigating Factors has been clarified explained in previous pages. Modifications Made as a Result of Feedback The definition for Preventive Factors was not modified. We hope the information above addresses the concerns raised in the feedback. o The definition is still unclear o I have no further comment on this definition If you still find the definition for Preventive Factors to be unclear, please provide an explanation and an alternative definition:

Page 179 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Recovery Factors clear, precise and accurate? Feedback from Round 1 Of the those that responded 66.7% answered “Clear” 33.3% answered “Unclear” The feedback from Round 1 indicated the concept of Recovery Factors and its relationship with Contributing Factors, Preventive Factors and Mitigating Factors was unclear. Response to Feedback The concept for Recovery Factors and its relationship with Contributing Factors, Preventive Factors and Mitigating Factors has been clarified explained in previous pages. Modifications Made as a Result of Feedback The definition of recovery factors has been modified slightly to emphasize the important role these factors play in preventing an adverse event. Original definition - “actions taken to stop circumstances or influences from turning into a patient safety event.” Revised definition – “actions taken to stop the progression of a patient safety event from harming the patient.” Please indicate whether the revised definition is clear. Recovery Factors are actions taken to stop the progression of a patient safety event from harming the patient. o Clear o Unclear If you find the revised definition for Recovery Factors to be unclear, please provide an explanation and an alternative definition:

Page 180 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Mitigating Factors clear, precise and accurate? Feedback from Round 1 Of the those that responded 85.2% answered “Clear” 14.8% answered “Unclear” The feedback from Round 1 indicated the concept of Mitigating Factors and its relationship with Contributing Factors, Preventive Factors and Recovery Factors was unclear. Response to Feedback The concept for Mitigating Factors and its relationship with Contributing Factors, Preventive Factors and Recovery Factors has been clarified explained in previous pages. Modifications Made as a Result of Feedback The definition for Mitigating Factors was not modified. We hope the information above addresses the concerns raised in the feedback. o The definition is still unclear o I have no further comment on this definition If you still find the definition for Mitigating Factors to be unclear, please provide an explanation and an alternative definition:

Page 181 of 194

WHO IPSEC - Delphi Survey Round 2 Is the definition of Organisational Outcomes clear, precise and accurate? Feedback from Round 1 Of the those that responded 88.5% answered “Clear” 11.5% answered “Unclear” The feedback from Round 1 suggested there was a lack of clarity about what concepts are contained within the Organisational Outcomes classification tree. Response to Feedback The Organisational Outcomes classification tree consists of the following concepts:

• Social outcomes (reputation, support and relationships, or shared set of beliefs, values and norms)

• Economic outcomes (production, development and management of material wealth, and

resources utilized within or by the organisation)

• Legal outcomes (activities – civil protection, political policies or litigation – permitted under certain conditions dictated by law or by legislation including legal services, systems, and/or legislative policies governing the operation of the organisation).

It should be noted that organisational outcomes may not be known at the exact time the patient safety event occurs or is reported. Modifications Made as a Result of Feedback The definition for Organisational Outcomes was not modified. We hope the information above addresses the concerns raised in the feedback. o The definition is still unclear o I have no further comment on this definition If you still find the definition for Organisational Outcomes to be unclear, please provide an explanation and an alternative definition:

Page 182 of 194

WHO IPSEC - Delphi Survey Round 2 Meaningful and Usefulness of Each Class The conceptual framework is intended to be inclusive of a wide range of circumstances when people come into contact with the health system. We believe the 10 high level classes and the relationships between them comprehensively represent the domain of patient safety. We would like to confirm that you believe each of the 10 high level classes is a meaningful and useful class within the IPSEC’s conceptual framework. Issues identified in the feedback from Round 1 related to this have been raised and responded to in previous questions. Hopefully, the foregoing information, revisions and clarifications adequately addressed the concerns. As a reminder, below is the description of how the conceptual framework can be used as a model to classify a patient safety event. Please keep this information in mind while answer the questions on the following pages. The numbers correspond to the numbers in the diagram. 1 Event Type – Identify what happened to the patient. 2 Patient Impact/Outcomes – Identify the results of the event upon the patient. 3 & 4 Patient Characteristics & Event Characteristics – Identify the pertinent information about the patient and event 5 Contributing Factors – Identify what factors lead to the event. 6 Actions Taken – Identify what was done to prevent, mitigate or recover from the event. 7, 8 & 9 Preventive, Recovery & Mitigating Factors – Identify the defenses or barriers built into the system to counteract, avert or reduce the occurrence/recurrence of the event. 10 Organisational Outcomes – Identify the impact of the event on the organisation.

Page 183 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Event Types Of the those that responded 32% answered “Strongly Agree” 57% answered “Agree” 6.5% answered “Neither Agree nor Disagree” 2.5% answered “Disagree” 2% answered “Strongly Disagree” Do you believe the class, Event Type, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Event Types, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 184 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Patient Impact/Outcomes Of the those that responded 34.3% answered “Strongly Agree” 59.2% answered “Agree” 2% answered “Neither Agree nor Disagree” 2% answered “Disagree” 2.5% answered “Strongly Disagree” Do you believe the class, Patient Impact/Outcomes, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Patient Impact/Outcomes, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 185 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Patient Characteristics Of the those that responded 34% answered “Strongly Agree” 55.8% answered “Agree” 3.6% answered “Neither Agree nor Disagree” 5.6% answered “Disagree” 1% answered “Strongly Disagree” Do you believe the class, Patient Characteristics, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Patient Characteristics, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 186 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Event Characteristics Of the those that responded 33.7% answered “Strongly Agree” 60.7% answered “Agree” 3.6% answered “Neither Agree nor Disagree” 1% answered “Disagree” 1% answered “Strongly Disagree” Do you believe the class, Event Characteristics, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Event Characteristics, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 187 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Contributing Factors Of the those that responded 37.6% answered “Strongly Agree” 54.6% answered “Agree” 5.7% answered “Neither Agree nor Disagree” 1.5% answered “Disagree” 0.5% answered “Strongly Disagree” Do you believe the class, Contributing Factors, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Contributing Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 188 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Actions Taken Of the those that responded 36.8% answered “Strongly Agree” 55.4% answered “Agree” 5.7% answered “Neither Agree nor Disagree” 1% answered “Disagree” 1% answered “Strongly Disagree” Do you believe the class, Actions Taken, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Actions Taken, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 189 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Preventive Factors Of the those that responded 32.5% answered “Strongly Agree” 59.7% answered “Agree” 4.2% answered “Neither Agree nor Disagree” 3.1% answered “Disagree” 0.5% answered “Strongly Disagree” Do you believe the class, Preventive Factors, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Preventive Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 190 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Recovery Factors Of the those that responded 20.4% answered “Strongly Agree” 52.4% answered “Agree” 15.2% answered “Neither Agree nor Disagree” 10.5% answered “Disagree” 1.6% answered “Strongly Disagree” Do you believe the class, Recovery Factors, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Recovery Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 191 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Mitigating Factors Of the those that responded 23.7% answered “Strongly Agree” 61.6% answered “Agree” 8.4% answered “Neither Agree nor Disagree” 5.8% answered “Disagree” 0.5% answered “Strongly Disagree” Do you believe the class, Mitigating Factors, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Mitigating Factors, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 192 of 194

WHO IPSEC - Delphi Survey Round 2 Do you believe the classes are meaningful and useful? Organisational Outcomes Of the those that responded 22.6% answered “Strongly Agree” 66.8% answered “Agree” 5.8% answered “Neither Agree nor Disagree” 3.7% answered “Disagree” 1.1% answered “Strongly Disagree” Do you believe the class, Organisational Outcomes, is a meaningful and useful class within the IPSEC's conceptual framework? o Strongly Agree o Agree o Neither Agree nor Disagree o Disagree o Strongly Disagree If you indicated that you (1) neither agree nor disagree, (2) disagree, (3) or strongly disagree with the statement, “The class, Organisational Outcomes, is a meaningful and useful class within the International Patient Safety Event Classification”, please explain why you choose your answer:

Page 193 of 194

WHO IPSEC - Delphi Survey Round 2 Overall Comments We truly appreciate the time you dedicated to participating in the Delphi survey. Your thoughtful review has directly contributed to improving the International Patient Safety Event Classification. Based upon feedback from Round 1, we have clarified several issues and revised definitions of key concepts. One additional issue deserves mentioning. We are in the process of translating the information provided at the beginning of the survey into the official languages designated by the United Nations. This is an important step in making sure the International Patient Safety Event Classification is applicable globally and sensitive to linguistic and cultural issues. Once these versions are available, we will post them on the WHO World Alliance for Patient Safety website for comment. Please provide us with any further comments regarding the International Patient Safety Event Classification or its conceptual framework.

Page 194 of 194

WHO IPSEC - Delphi Survey Round 2 Thank you

Thank you for your participation. Your input is extremely valuable to us. We will provide you with a report of the responses to this second round survey. Please return this survey via e-mail to [email protected]. If you are unable to return the survey via e-mail, please contact us at [email protected] for an alternative method.

Documents

Report on the Web-Based Modified Delphi Survey of the ICPS · stage, web-based modified Delphi Survey (Delphi Survey) to test the relevance and acceptability of the ICPS. This report