Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Reviewing Methods Used in
Patient Safety Research:
Advantages and Disadvantages
Dr Jeanette Jackson
This SPSRN work is funded by
Introduction
• Effective management of patient safety in
healthcare requires:
1. an understanding of the causes of adverse events and related outcomes
2. a capacity to measure adverse events and their causes as well as related outcomes at different levels (individual, unit, organization,
industry, national, international)
• Measurement of industry safety status is achieved
by a range of methods based on key performance
indicators for risk factors and safety events as well
as leading indicators for safety (including causes
like cultural factors)
Objectives
1. To review possible methods for measuring patient
safety
2. To evaluate available methods for measuring
patient safety
Methods Measuring
Patient SafetyMethods can be classified based on different criteria:
1. Related to the source of data (Runciman et al., 2008): e.g., research
data from previous studies, routine surveillance data, quality
assurance and risk management data
2. Human factor and ergonomic methods used in system design and evaluation (Carayon, 2007): e.g., cognitive work analysis, incident
analysis, work systems and process analysis, usability evaluation,
assessing safety culture and climate
3. WHO report on methods measuring patient safety (Jeffs, Law & Baker, 2008): retrospective, prospective and concurrent states of
patient safety
Methods Measuring
Patient SafetyRetrospective Methods:
• Medical Records, Incident Reporting Systems, Claims and
Complaints, Staff Questionnaires, Interviews
• Exploring adverse event, errors and near misses
• Awareness about patient safety issues
• Inform improvement programmes
Methods Measuring
Patient SafetyProspective Methods:
• Prospective Analysis Tools, Simulations, Safety Culture
Assessment
• Identifying sources of potential error based on knowledge about
error rates
Methods Measuring
Patient SafetyConcurrent Methods:
• Direct Observations, Video Techniques, Checklists and Audits
• Systems that update information once they receive data
• Monitoring and controlling patient care processes
• Estimates of error rates
• Reaction to error as it occurs
Retrospective Methods
Medical Records:
• Indicators of the cause, nature and incidence of harm
• Most frequently used for researching medical error
• Using ‘triggers’, e.g., high risk medication or laboratory values
• Adverse events studies: 3-17%
Medical Records
Advantages:
• Highlighting particular incidents / areas that require further
understanding / training
Disadvantages:
• Missing incidents that are not reported• Not everything documented, e.g., confounding factors or human
factors
• Time consuming, staff costs (training for screening and reviewing)
Retrospective Methods
Incident Reporting Systems:
• Different systems implemented in different health care systems and
settings
• Reporting adverse outcomes for patients, errors and near misses
(e.g., adverse drug events, problems with medical devices, safety of blood products)
• Information on causes of adverse events, provide warnings, point to
important problems, raise awareness and enhance safety
Incident Reporting Systems
Advantages:
• Systems can be implemented across multiple sites
• Minimal costs for volume of data (?)
• Can highlight effectiveness of existing policies / procedures
Disadvantages:
• Depending on coding systems used as well as the ability of using
these systems reliably• What and how to report (high hazard events, willingness to report)
• Number of reported incidents (lack of resources, varying quality)
• Ensuring that feedback (e.g., newsletter, meetings) is given and
actions are taken - effectiveness of learning from incident data
unclear (lack of documentation)• Lack of clarity in roles and responsibilities
• A large amount of time and resources involved
Retrospective Methods
Claims and Complaints:
• Claims reflect a patients’ demand for compensation (general safety
issues) whereas complaints reflect a patients’ subjective
impressions regarding their healthcare (specific incidents)
• Incidence data, experiences with intervention programmes, starting point for reviews and patient safety activities
• Identifying potential problem areas or clinical issues such as
organisational and management issues, teamwork skills, individual
clinicians behaviours
Claims and Complaints
‘When I first started out in practice,
I lied awake at night worrying about my patients.
Now I lie awake worrying about their lawyers.’
(Howard Fischer, MD)
Advantages:
• Themes can be picked up from a large number of claims / complaints
• Looking into 1 case in depth can provide important information for learning
Disadvantages:
• Subjective perspective
• Underreporting by elderly people
• Expensive
• One major event might influence the whole picture
Detecting Adverse Events
(adapted from P. Hebert)
Method AE / 1000 admissions
1. Incident reports 5
2. Retrospective chart reviews 30
3. Stimulated voluntary reports 30
4. Automated flags 55
5. Daily chart reviews 85
6. Automated flag and daily review 130
7. Trigger tool 400Jha et al. (1998). Identifying Adverse Drug Events: Development of a Computer-based Monitor and Comparison with
Chart Review and Stimulated Voluntary Report. Journal of the American Medical Informatics Association, 5, 305-315.
O’Neill et al. (1993). Physician Reporting Compared with Medical -Record Review to Identify Adverse Medical Events.
Annals of Internal Medicine, 119(5), 370-376.
Prospective Methods
Prospective Analysis Tools:
• Proactively identifying, prioritising and mitigating patient safety risk
• Wide range of methods such as Failure Modes Effects and Critical
Analysis, Health Care Failure Mode and Effect Analysis, Hazard
Analysis
• Use at local level, discovered information is not shared throughout
organisations, not identifying combinations of events leading to
incidents
• Probabilistic risk assessment (mixture of process analysis
techniques and decision making processes) to balance and prioritise
between competing goals as well as to identify combinations of
events leading to potential harm
Prospective Analysis Tools
Advantages:
• Uncovering unrecognised system level problems
• Boosting staff communication and morale
• Process of scoring failure modes regarding their probability of
occurrence and severity of consequences (group census)
Disadvantages:
• Logistic challenge
• Complex methodology involved
• Process of scoring failure modes probability of occurrence and
severity of consequences (depending on setting, people)
Prospective Methods
Simulations:
• Training of performance skills in a scenario with the same realistic
problems and demands as in real life settings
• Learning opportunity for healthcare workers and teams about
consequences of their actions
• Identify problem conditions / actions to improve healthcare delivery
• Teach and test specific technical (e.g., performing a new procedure)
and non-technical (e.g., communication) skills
Simulations
Advantages:
• Can identify additional risks that may not have been considered
• Control of the situation and problem area
Disadvantages:
• Challenge to create realistic teams with real life skills mix (by
experience and profession)
• Expensive (staff involvement, equipment, …)• Transfer from simulation to real world
Concurrent Methods
Direct Observations:
• Data collection on errors, adverse events, near misses, team
performance, decision making and organisational culture
• Information related to the analysis of specific types of procedures
(e.g., surgery)
• Structured observations using rating systems, e.g., surgeons’ non-
technical skills, to rate skills and provide feedback during a
postoperative debrief (Yule et al., 2006)
• Range of behaviour rating tools available for individual and team
assessment in acute medicine (see Flin & Mitchell, 2009)
Direct Observations
Advantages:
• Can highlight particular factors of concern
• Can influence local behaviours / policies
Disadvantages:
• Observer as a distraction / interruption
• Observer training
• Time consuming (data collection and analysis)
Concurrent Methods
Checklists and Audits:
• Identifying active errors in the environment such as delay in care,
equipment failure, information transfer, non-compliance with hospital
policy, …
• Checklists are context specific (e.g., ICU versus general ward)
• Safety audits can be performed using checklists, e.g., during and
after morning rounds to identify unlabeled medication at the bedside,
missing ID bands or inappropriate pulse oximeter alarm setting
Checklists and Audits
Advantages:
• Public charts to display ongoing results
• Influence of culture (motivation to get it right)
Disadvantages:
• Outcome measure identification (who decides what to display?)
• Lots of pressure to ward staff
• Lack of compliance
Selection of Methods
• Appropriate selection of methods depends on the question
being addressed, the resources available, and on contextual
factors
Framing Patient Safety Research
Multilevel Framework of Patient Safety Research (Jackson & Flin, in prep):
Organizational
Structure
Unit
Management
Worker
BehavioursOutcomes
Individual
Differences
• Based the causal chain and different levels of analysis (i.e., individual,
team, unit, and organisational) proposed by industrial and patient safety
models
• Applies within an organisation even though external factors such as
government and regulators responsibilities exist outside an organisation
Any Questions?
Dr Jeanette Jackson
This SPSRN work is funded by