Embed Size (px)
Citation preview
i
A REPORT TO HEALTH EDUCATION KENT, SURREY AND SUSSEX
Electronic Prescribing Systems and the
Training Offered to Prescribers
Safe Prescribing Project
Durham University;NHS England;University of Birmingham
August/2015
ii
Contents Authors ...................................................................................................................................... 1
Executive Summary ................................................................................................................... 3
Background ................................................................................................................................ 6
Outcome 1: The electronic prescribing systems currently available in the UK. ........................ 8
Outcome 2: Contact details of the companies providing the systems above. .......................... 8
Outcome 3: .............................................................................................................................. 12
a) Outline of companies above that provide online training for prescribing that allow
staff to become accustomed to the system before starting in practice. ............................ 12
b) Robust inductions to online training for newly qualified professionals to ensure
seamless use of new electronic system .............................................................................. 24
Outcome 4: The common prescribing errors made when using electronic systems. ............. 62
Outcome 5: Describe any variations in error rates associated with specific electronic systems
................................................................................................................................................. 75
Outcome 6: The NHS hospital Trusts in the UK that have implemented electronic prescribing
systems successfully, with examples of success stories, lessons learnt and transferable best
practice. ................................................................................................................................. 129
Outcome 7: Contact details of electronic prescribing leads from a cross-section of Trusts.143
Outcome 8: The training strategies for newly employed prescribers within Trusts. ............ 144
1
Authors Chief investigator: Dr. Sarah Patricia Slight,
School of Medicine, Pharmacy and Health, Wolfson Research Institute, University of Durham,
Queen’s Campus, University Boulevard,
Thornaby, Stockton-on-Tees, TS17 6BH
Phone: +44 (0191) 334 0548 Email: [email protected]
Co-investigators: Miss Clare L. Brown,
School of Medicine, Pharmacy and Health, Wolfson Research Institute, University of Durham,
Queen’s Campus, University Boulevard,
Thornaby, Stockton-on-Tees, TS17 6BH
Email: [email protected]
Dr. Andrew K. Husband, Dean of Pharmacy, School of Medicine, Pharmacy and Health, Wolfson Research Institute, University of Durham,
Queen’s Campus, University Boulevard,
Thornaby, Stockton-on-Tees, TS17 6BH Email: [email protected]
Ann Slee, ePrescribing Lead,
NHS England,
eHealth Research Group, Center for population
health sciences, University of Edinburgh
Professor Jamie Coleman,
2
Professor in Clinical Pharmacology and Medical Education / MBChB Phase 2 Lead and Deputy Programme Director, School of Clinical and Experimental Medicine College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT UK, Telephone +44 (0)121 371 6003 Email [email protected] Sarah Thomas, NIHR Doctoral Research Fellow School of Clinical and Experimental Medicine College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT UK, Telephone +44 (0)121 414 8074 Email [email protected]
Study Coordinating Centre: School of Medicine Pharmacy and Health,
University of Durham
Permissions Obtained: Full NHS ethical approval was not required.
Ethical approval from Durham University Ethics Committee was obtained
Research & Development (R&D) approvals may still be needed at each site.
3
Executive Summary
This project aimed to gain a full understanding of the types of support offered
to newly employed prescribers on electronic prescribing (ePrescribing) systems. The
study objectives included gathering the lessons learnt from Hospital Trusts who have
implemented ePrescribing, as well as drawing on the literature on error rates and
prescriber training. We identified a total of 16 companies that provided ePrescribing
systems in the U.K (Outcome 1); these included suppliers from the UK (n=7), USA
(n=5), Italy (n=1), Portugal (n=1) and Canada (n=1) and their contact details have
been provided within this report (Outcome 2). We searched each company’s website
for information about the training that they offered as part of implementation
(Outcome 3a). Online training was rarely provided; however a arrange of other
training approaches were used such as classroom-based sessions, lectures, workshops,
ward-based training, chat-forums and help desks, patient scenario based examples and
handbooks containing ‘quick-start guides’. A typical training model included a Train-
the-Trainer approach whereby the company would initially deliver training to
designated hospital staff trainers, who would then be responsible for developing,
planning and delivering all end-user training. We also conducted a literature review
(Outcome 3b) to identify the approaches used to train qualified prescribers on
ePrescribing systems in a hospital setting. Three large databases were searched
including: Cumulative Index Nursing and Allied Health Literature (CINAHL),
Embase (OVID), and Medline (OVID), and a total of seven publications met our
inclusion criteria. Online training methods were rarely reported; examples found
included web-based demonstrations and an online training portal developed by The
University of Victoria in Canada, which housed various training versions of electronic
health records to allow end-users to practice prescribing and learn about system
design. The development of expertise-specific scenarios that were relevant to
clinicians from different specialist areas was considered important, as was providing
adequate training opportunities for all clinicians to experience the system prior to
implementation.
A second literature search was performed to identify publications related to
prescribing errors associated with ePrescribing systems (Outcome 4). Over 2,000
papers were retrieved. Studies reported a range of prescribing errors made when using
ePrescribing systems such as wrong patient, wrong dose and frequency, and timing
errors. Factors which contributed to the occurrence of these errors included miss-
selection from drop-down menus, poor screen display, unclear logging in procedures,
autocomplete functions, inflexible ordering and overdependence on the system, and
clinical decision support. A third literature search was performed to identify papers
that reported prescribing error rates from studies carried out in UK hospitals
(Outcome 5). We searched two large databases, reference lists of included
publications, and reference lists of relevant systematic reviews. Titles, abstracts and
full text were reviewed and 14 papers (11 full papers and three abstracts) were
included. Studies were performed in the following clinical settings: Surgical (n=5),
4
Medical (n=1), Mixture (n=2), Critical care or Intensive Care (n=2), Oncology (n=1),
Paediatrics (n=3). Due to the differences in methodology, the clinical setting and the
levels of system customisation, it was not possible to directly compare the prescribing
error rate associated with specific systems. JAC was the only ePrescribing system,
which was evaluated in more than one study. All studies conducted on surgical wards
demonstrated a decrease in prescribing errors following implementation of an
ePrescribing system. The post-implementation error rate ranged from 1.9% to 7.9%.
Two studies took place in a critical care or intensive care setting, one of which
reported mixed results; the introduction of the ePrescribing system was associated
with more complete and legible orders, although the error rate actually increased for
IV fluids and infusions (reduction in percentage of correct entries by 16% and 15.5%
respectively). The second study reported a significant reduction in medication errors
following implementation of an ePrescribing system from 6.7% of all medication
orders to 4.8%. Only one study was conducted in an oncology setting, which reported
a relative risk reduction of 42% when the ePrescribing system was used. All three
studies conducted in a paediatric setting suggested that prescribing errors may be
reduced following implementation of an ePrescribing system.
Information about the experiences of NHS hospital Trusts that have
successfully implemented an ePrescribing system (Outcome 6) was obtained from a
number of different sources, including the literature, conference presentations and the
ePrescribing Toolkit Website. We chose four specific sites (Site A, B, C, D) in
different geographical locations in the UK. Staff at Site A were generally positive
towards the implementation process. However, there were issues surrounding
increases in workload, access to computer terminals and the sub-optimal performance
of software. Furthermore, the existence of co-existing paper and electronic systems
generated difficulties. Users also developed coping mechanisms or workarounds e.g.,
using another staff’s details to deal with “logging-in” problems. While there was
positivity towards the transition to an electronic environment at Site B, there was also
negativity towards the actual system being implemented. End-users felt that there was
a lack of communication and engagement during the implementation process. As the
system was originally developed in the U.S, significant anglicisation was needed to
ensure it was suitable for their U.K. Hospital Trust. At Site C, information was
distributed across a variety of sources following implementation, as the system was
not fully integrated. Further work was needed to learn the full range of functionality
available, refine the decision support system and implement increasing modules of the
wider integrated system. At Site D, the implementation team consisted of both
technical and clinical staff. Anglicisation of the system was also required to make it
more suitable for use in their U.K hospital. The system could not be used in certain
clinical areas due to a lack of functionality. Following initial implementation, there
was also a need for continuous maintenance, including system updates, incorporation
of new advances in technology such as hand-held devices and integration with other
hospital systems. The contact details of ePrescribing leads from a section of trusts
have been provided in the report (Outcome 7).
5
We conducted four semi-structured interviews with members of staff involved
in the training of prescribers across four different hospital Trusts (Outcome 8). We
found that the system supplier provided end-user training at the implementation stage
at Site A. However, the hospital informatics team and IT trainers were responsible for
the development and delivery of all training material and sessions. Super-users were
employed by the Trust to provide ward-based support; this approach was largely
unsuccessful due to a lack of staff engagement. Other difficulties experienced at
induction included: logistical difficulties, the need for updating training to reflect any
new system changes, and providing trainees with too much information, which they
felt unable to retain. At a different site (Site B), a team of internal Trust trainers
delivered the staff training; most of the trainers had a clinical background with
experience in adult education. Much of the original training material was provided by
the company e.g., screen shots. This site recently changed their training approach to
focus more on workflow e.g. ‘admitting a patient’ rather than individual tasks such as
‘finding a patient’. The training has also been customised to focus on the problem
areas of the system. Difficulties surrounding the development and delivery of training
were reported. Prescribing of anticoagulation, insulin and fluid were typically
associated with more issues compared to other medications. At Site C, the core
training was provided by the Trust training team consisting of members of the
prescribing and pharmacy management directorate. Site C used an E-Learning based
training approach, which incorporated 22 modules covering all aspects of how to use
the system, such as patient scenarios and exercises. There was a summative
assessment with a pass mark of 90% for all staff. More specialist training e.g. for a
specialist clinical area such as paediatrics was delivered in person by the lead
pharmacist for that area. Finally, a designated team of informatics trainers from
clinical and non-clinical trainers were responsible for all aspects of the design and
delivery of prescriber training at Site D. The training was tailored towards the
profession of the end-user, using a variety of clinical scenarios and exercises. There
were three specific lessons learnt that were considered important when planning
future foundation doctor training: (1) the training needs to reflect the latest version of
the system, (2) there should be adequate opportunities for staff to attend training
sessions or access training material, and (3) the training provided needs to be
consistent.
6
Background
Health Education Kent, Surrey and Sussex commissioned a piece of research
to investigate the types of support offered to newly employed prescribers to orientate
them to the electronic prescribing (ePrescribing) systems available. This included
lessons learnt from NHS Trusts who have implemented ePrescribing with regards to
error rates and prescriber training. The findings of this report will help feed into the
development of a Diagnostic Prescribing Assessment (DPA) tool suitable for different
ePrescribing systems or an alternative model. This work forms part of the Safe
Prescribing Project, managed by Katie Reygate, Prescribing lead
The UK healthcare system is undergoing great change; traditional paper
records and methods of prescribing are slowly being phased out in favour of
automation and information technology. Electronic prescribing is defined as "the
utilisation of electronic systems to facilitate and enhance the communication of a
prescription, aiding the choice, administration or supply of a medicine through
decision support and providing a robust audit trail for the entire medicines use
process"(NHS Connecting for Health). EPrescribing may also be linked with other
functionality such as clinical decision support, which provides decision-making
support and safety checks at the point of prescribing (e.g., drug-allergy checks) and
electronic medication administration records, allow an electronic record of drug
administration to be kept. A range of potential benefits support the use of ePrescribing
systems such as improved patient safety and potential cost savings.(1-3) Additionally,
financial incentives through NHS schemes such as The Integrated Digital Care
Technology Fund and The Safer Hospitals, Safer Wards Technology Fund have
7
contributed to the adoption of healthcare technology.(4, 5) However, the
implementation of ePrescribing systems raises many challenges, of which training is
just one.
Due to the clear differences between paper-based and ePrescribing systems,
traditional approaches to train and assess safe prescribing practice may not be suitable
and hence a new approach is needed. A team of experienced researchers, comprising
of academics, NHS England ePrescribing lead, clinical pharmacists and researchers
was formed.
A series of team meetings with the funder, Katie Reygate were organised. The
purpose of these meetings was to plan and develop the specific objectives for the
project, provide updates on the work conducted so far, and discuss any issues that
may have been encountered along the way. Specific tasks included:
To conduct a series of internet searches to identify ePrescribing system
suppliers and their contact details;
To correspond with suppliers of ePrescribing systems via email or telephone
to ascertain the approaches used for training prescribers in the use of the
system;
To conduct a series of focused literature searches;
To identify and report on case studies that provided lessons learned from
hospitals that have successfully implemented ePrescribing;
To carry out a number of semi-structured telephone interviews at a number of
different hospital sites to provide an overview of the training strategies for
newly qualified prescribers.
8
1. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, et al. The
impact of computerized physician order entry on medication error prevention.
JAMIA. 1999;6(4):313-21.
2. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al.
Effect of computerized physician order entry and a team intervention on prevention of
serious medication errors. JAMA : the journal of the American Medical Association.
1998;280(15):1311-6.
3. Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, et al. Return on
investment for a computerized physician order entry system. Journal of the American
Medical Informatics Association : JAMIA. 2006;13(3):261-6.
4. NHS England. The Integrated Digital Care Technology Fund 2013
[05/01/2015]. Available from: http://www.england.nhs.uk/ourwork/tsd/sst/tech-fund/.
5. NHS England. Safer Hospitals, Safer Wards: Achieving an Integrated Digital
Care Record. 2013.
Outcome 1: The electronic prescribing systems currently
available in the UK. See Table 1
Outcome 2: Contact details of the companies providing the
systems above. See Table 1
9
Table 1: Name and contact details of companies providing ePrescribing systems in the UK.
Supplier System Date and
country of
origin
Website Address Contact Details
Alert Life
Sciences
Computing
ALERT
Prescription
1999, Portugal http://www.alert-
online.com Head Office: Edifício Lake Towers Rua Daciano Baptista Marques, 245 4400-617 Vila Nova de Gaia Portugal
Tel: +44 07525 262 853
Email: [email protected]
Allscripts Sunrise
clinicals
1995, US http://uk.allscripts.
com/ Battersea Studios 80 Silverthorne Road London, SW8 3HE +44 (0)20 7819 0444 And 15 Oxford Court,Manchester M2 3WQ,+0161 233 4999
Ascribe-
Now emis
health
Ascribe
ePMA
(Emis
ePrescribing)
1984, UK http://www.ascrib
e.com
https://www.emish
ealth.com/product
s/eprescribing/
Ascribe House, Brancker Street,
Westhoughton
Bolton, UK
BL5 3JD
Tel: +44(0)1942 852 400
Email: [email protected]
Emis Health Head Office
Leeds - Rawdon House
Rawdon HouseGreen Lane,
Yeadon
Leeds
LS19 7BY
Tel: 0113 380 3000
Cerner
Corporation
Cerner
ePrescribe
(Millenium)
1979, US http://www.cerner.
com/
Cerner Limited
6th Floor, The Point
37 North Wharf Road
London
W2 1AF
Tel: +44 (0) 20 7432 8100
Email: [email protected]
Civica Paris EPR and
Case
Management
UK https://www.civica
.co.uk/health-and-
social-care
Civica UK Ltd
Station House
Stamford New Road
Altrincham
Cheshire
WA14 1EP
Tel: +44 (0) 161 9415833
Email: [email protected] or
10
CSC Lorenzo http://www.csc.co
m/health_services/
offerings/99982/1
03601-lorenzo
Contact via link:
http://www.csc.com/contact_us/
CSC Medchart http://www.isofthe
alth.com/en-
AU/Solutions/AN
Z%20Hospitals%2
0and%20Clinics/
Medication%20M
anagement.aspx
Brian Hemming
Tel: +44 (0) 129 527 4240
Email: [email protected]
CSE
(servelec-
healthcare)
PICS UK, 1998 http://www.servel
ec-healthcare.com
Servelec Healthcare
The Straddle
Victoria Quays SHEFFIELD
S2 5SY
United Kingdom
Tel: +44 (0) 1246 437500
Email: Sales Email: [email protected] Marketing Email: [email protected] HR - [email protected]
Servelec-
healthcare
RiO ePMA http://www.servel
ec-
healthcare.com/in
dex.html
Servelec Healthcare The Straddle
Victoria Quays SHEFFIELD
S2 5SY
United Kingdom
Tel: +44 (0) 1246 437500
Email: Sales Email: [email protected] Marketing Email: [email protected] HR - [email protected]
Epic EpicCare
EMR
US, 1979 http://www.epic.c
om 1979 Milky Way Verona, Wisconsin 53593
Tel: 608-271-9000
Email: [email protected]
JAC JAC EPMA UK,1983 http://jac.co.uk/co
mplete_and_integr
ated_e_prescribing
_medicines_admin
istration_epma_/
JAC Computer Services
1 Aurum Court Sylvan Way Basildon Essex SS15 6TH United Kingdom
Tel: +44 (0) 1268 416348
Email: [email protected]
MEDITEC
H
Version 6.0 US, 1969 http://home.medite
ch.com/en/d/home
/
One Northumberland Avenue, London, WC2N 5BW
Tel: 0207 872 5583
Noema Life Galileo
Medication
1996, Italy Registered Office:
Monica House,
St Augustines Road, Wisbech
Cambs
PE13 3AD
Head Office:
2-3 St Johns Street
Stamford
Lincolnshire
PE92DA (UK)
Email: [email protected]
Tel: 07875 088 981
11
QuadraMed
Corporation
QCPA 1993, Canada http://www.quadra
med.com
QuadraMed Corporation, 12110 Sunset Hills Road,
Suite 600 Reston, VA 20190
Tel: 703-709-2300
System C Medway UK, 1983 http://www.system
c.com
Medway EPR
System C Healthcare Ltd
+44 (0) 1622 691 616
TPP SystmONE UK, 1997 http://www.tpp-
uk.com
TPP, Mill House
Troy Road
Leeds
LS18 5TN
Tel: +44 (0)113 2050080
Email: [email protected]
12
Outcome 3:
a) Outline of companies above that provide online training for
prescribing that allow staff to become accustomed to the system
before starting in practice.
NB: For completeness, we have provided information on all
methods of training, including online training.
Table 2: Outline of company provided training
Supplier System Communication
Method
Supplier Training
Alert Life Sciences
Computing
ALERT
Prescription
Emailed 19th
April 2015 (no
reply)
Re-emailed 29th
April 2015 (no
reply)
Information
obtained online at
http://www.alert-
online.com/elearn
ing [accessed
28/04/2015]
ALERT eLearning
Alert e-Learning programme for Alert
products can be offered as an
alternative or a complement to
‘traditional teaching’.
Flexible learning is provided with the
ability to access training anytime and
anywhere depending on the availability
of individual staff. Staff can learn at
their own pace and tailor their learning
towards key areas of interest.
The e-learning programme uses a
variety of multimedia to support
learning such as demonstration videos,
trainer instructions and animations. It is
possible to communicate within the
system via chat and forums, which
allows end-users to exchange their
experiences.
The system also supports tutor-trainee
communication through the chat and
forum tools. The e-learning
programme provides continuous
performance evaluation to support end-
users as they learn. The content
continues to be available after
completing individual courses to enable
review of learning material.
A specific course ALERT EDIS
PHYSICIAN® is available and targeted
towards doctors working in the
emergency department. This course
13
uses active and demonstrative methods
to cover a range of areas including:
documenting a chief complaint,
ordering medication and exams, access
results and discharging a patient. A
certificate is awarded to the trainee
once 80% of the course has been
completed, suggested tasks have been
performed and have achieved a pass in
the final evaluation. The course takes
approximately 4 hours. Further courses
are available for example an
Introduction to ALERT ® v2.6 which
allows end-users to learn more about
the functionality of the ALERT
prescribing system.
Courses are available for a US and UK
population.
Allscripts Sunrise
clinicals Emailed 19th April 2015 (no reply) Phonecall 29th April 2015 (no reply) Information obtained online http://uk.allscripts.com/products-services/services/education [accessed 29th April 2015]
Experiential Learning: Scenario-based simulation learning tool designed for staff members. These self-paced courses allow learners to practice workflows using real-world scenarios in a simulation learning environment.
Training Consulting: Training Consultants provide strategy, guidance and recommendations for any size group who needs end-user “best practice” training guidance.
Formal instructor-led classes: These classes are held in Allscripts training facilities, where attention is given to the learning needs of each individual student. The sessions include extensive training materials, hands on exercises and interactive discussions.
Web-based instructor led classes: These smaller web-based classes are for single topics or customised training needs. Students learn from their onsite organisation, while still receiving the individual attention and
14
hands-on time provided in a classroom setting.
eLearning: Budget-friendly, self-paced form of training is scalable for small offices that need to provide training around a busy office schedule. For very large organisations, the company reported having more staff to train clinicians and office personnel.
Custom Solutions: Any combination of services are available for clients who want to design their own learning path.”
Ascribe Ascribe
ePMA
Online material
http://www.ascrib
e.com/solutions-
services/Pages/Tr
aining-
Academy.aspx
[accessed 13th
May 2015]
Phone call 13th
May 2015 (spoke
with member of
sales team)
Training academy
Range of training packages
Training can be delivered on-site or
within Ascribe office in Bolton or an
external venue in London.
Training is typically provided to
approximately 6 members of the
organisation (a multidisciplinary team
is preferred). ‘Train the trainer’
sessions are delivered to give an
overview of the system and features so
that they can then carry out end-user
training at their organisation. Workshop
sessions are also held whereby wider
members of the hospital organisation
can ask questions and provide
comments about features that they
would like to see, thereby having some
influence into system build.
The ‘train the trainers’ then deliver end-
user sessions, which are designed and
customised according to the specific
organisation. For example lecture
sessions, one-to-one training on the
ward to support staff and also provide a
simulation ‘dummy station’ whereby
staff can access and practice using the
15
system even before it has ‘gone live’.
Standard training manuals are available
from the company, however due to the
variations in systems post
customisation; organisations typically
will develop their own training
packages.
E-Learning packages have recently
been developed to train the trainers;
however there is no provision of e-
learning material currently for end-
users. Although experience suggests
Trusts often develop their own e-
learning training packages or outsource
e-learning from external suppliers.
Cerner
Corporation
Cerner
ePrescribe
(Millenium)
Emailed 19th
April 2015 (no
reply)
Material obtained
online
http://www.cerner
.com/uploadedFil
es/Content/Soluti
ons/_Education_a
nd_Training/Lear
ning_Consulting_
Services/UK_lear
ning_servicesflyer
_2012.pdf
[accessed 29th
April 2015]
Phone call 13th
May 2015 (spoke
to Lindsey
Whittaker
02071074413)
Cerner Learning Services:
A range of training options are available
which are delivered by learning consultants
and educators.
Managed Learning Services are available
which offers training across a range of areas
to end-users. This service is available as an
optional extra and is therefore subject to
additional costs.
Managed Learning Services include
implementation education, technical
education, clinical education and leadership
and professional skills education.
The full range of teams include:
1. Learning consultant/coordinator:
Involved in training learning staff
and recommending and planning
end-user learning.
2. Learning Plan Development
Session: A team that works onsite
to identify learning needs, resource
constraints and best practices in
order to develop a tailored learning
strategy for the organisation.
3. Learning Task Analysis: A team
helps develop end-user learning
materials. Critical tasks and
assessment questions that validate
competency are also identified.
16
4. Custom Learning Materials
Development: Examples include
organisation-specific facilitator
guide, performance based
assessment and supporting
materials to assist delivery of
instructor led end-user training
5. Web-Based Training for End
Users: Online learning tools, these
can be standard or customised.
6. Train the Trainer: Trainer-
Advanced training for
organisational trainers.
7. Super-User Training: training of
designated super-users in specific
areas so that they are able to
facilitate system use and support
staff.
8. End-User Training: Typically a
combination of web-based training,
instructor led training; activities are
performed both in a training setting
and as job aids.
9. Advancing Conversion Excellence
(ACE) Programme: A team
provides support with health care
staff during the early stages of
implementation. The ACE team
assist end-users with limited Cerner
experience gain confidence and
expertise.
10. Learning LIVE: An e-learning
program to deliver training and
support continuous learning.
Training is accessible, offering
‘just-in-time learning at the point of
need’
After speaking to Lindsey Whittaker on 13th May 2015, she explained that e-learning is typically not provided to UK organisations unless requested. This is because the UK market tends to want an e-learning package that is exactly customised to the system that the organisation will use and therefore the standard version of e-learning system is seen as less attractive. However e-learning packages can be built and developed with the organisation if
17
needed. Alternative online material such as video clips, which give demos of specific functions, are available and can be accessed at any time.
Civica Paris EPR
and Case
Management
Emailed on
19th April 2015
29th April 2015
Phone called on
29th April 2015
13th May 2015
Unable to obtain
response after
multiple emails
and phone calls.
CSC Lorenzo Information
provided via
telephone call 5th
May 201 (Sarah
Mason, Sales,
07795390018)
Lorenzo and Medchart are systems
provided by CSC and therefore have similar
training available.
Training is delivered through a ‘train the
trainer’ model at the hospital site to selected
individuals. Training is classroom based
and given to small groups of approximately
eight trainees using hands on activities.
Training is delivered on specific modules
within the system depending on local needs.
Sessions will take place on a standard
version of the system. There is a test at the
end of the ‘train the trainer’ sessions to
assess competence, after which the in
house-trainers will deliver sessions to end-
users.
End-user training is supported by CSC
trainers but is delivered by in-house
trainers. Training content and delivery
varies between organisations, and it is up to
the organisation to develop with end-users
what training methods will be used.
e-Learning modules can be provided or
developed in collaboration with the Trust,
however no Trust has used e-learning as a
sole method of training due to the
complexity of the system, typically
classroom based end-user sessions are
delivered. A benefit of the e-learning is that
CSC Medchart
18
it may be accessed off site and at a
convenient time for the end-user.
CSE
(servelec-
healthcare)
PICS Phone call 13th
May 2015
(Lindsay
Dransfield, sales,
07715121244)
There are four strands of learning:
1. Set-up training e.g. setting up a
drug formulary and setting up new
users on the system.
2. Training around the rules which
drive the EPMA and decision
support system and ensure these are
appropriate for the organisation e.g.
Venous thromboembolism
assessments.
3. End-user training; core training
about how to use the system e.g.
how to prescribe, how to
administer.
4. Report training; training on how to
manage alerts and utilise
information that is gathered on the
system.
The NHS Trust will identify a multidisciplinary team who will develop training that is delivered to end-users. Servelec will then train these individuals who will then deliver their own training sessions, typically classroom or ward based face to face teaching. Standard training materials are available however Trusts are encouraged to develop their own customised versions, which are more specific. Test and training environments exist which allow clinicians to work safely through the system. Increasingly Trusts are requesting to use test patients which are in fact anonymised versions of a real patient to ensure the content and scenario is realistic. E-learning or distance based learning is available or can be developed, however is used mainly for teaching specific features or as a refresher for end-users rather than as an alternative to face-to-face sessions. The PICS system is complex and therefore e-learning would perhaps not be a sufficient sole training method. Video tutorials and demos have also be used which would allow trainees to access
19
learning material from their intranet at a convenient time to learn how to perform specific functions.
Servelec-
healthcare
RiO ePMA Phone call 13th
May 2015
(Lindsay Dransfield, sales, 07715121244)
As for PICS system. Both supplied by Servelec.
Epic EpicCare
EMR
Information
obtained online:
http://www.epic.c
om/services-
training.php
[accessed 29th
April 2015]
Followed up with
phone call on 1st
May (Company
called)
Total recall training: Project team members
and key end users from the hospital
organisation receive training at a training
site in Verona, Wisconsin.
Classes are delivered to introduce the
system and discuss how it will impact
workflows. An end-user learning package is
delivered ‘Training Wheels’ which aims to
prepare end-users in usage of the system.
This incorporates e-learning lessons, lesson
plans, hands-on experience; post e-learning
lessons ‘ quick start guides’ and
optimisation materials. Materials are
tailored to the specific roles in which they
are intended to be used and are scenario
based.
e-Learning: Scenario based programmes are
available. The tutorials guide clinicians
through workflows, allowing them to learn
at their own pace in a flexible manner. E-
Learning may be used as an alternative to
or in conjunction with instructor led end-
user training.
JAC JAC EPMA Information
obtained online
http://jac.co.uk/fil
es/JAC%20Syste
m%20Manageme
nt%2020140130.p
df [Accessed 5th
May 2015]
correspondence
5th May 2015 with
Craig Rothwell.
E-mail address:
System Management Training
Refresher training and new training for
system managers
Ensure the system is configured to
specific needs
Optimisation of the system.
Training is typically on-site, and instructor
led.
(No information provided about specific
end-user training or on-line material)
20
MEDITECH Version 6.0 Email 22nd
May
2015
odiaz@meditech.
com
Phone call 28th
May 2015
Training is provided as part of full system
implementation. Meditech trainers from the
US are deployed within the Trust and will
work with the organisation to plan training
according to specific needs (i.e. medical
training will differ to pharmacist or nursing
training content).
Meditech will work with the Trust to
arrange who exactly will be trained and that
decision will be on a case-by-case basis.
The format of training is flexible. Options
include classroom delivered sessions, which
are considered more effective than lectures
and one to one sessions if needed.
The training support is on-going after the
initial implementation. Meditech trainers
will visit the Trust after one year to perform
‘optimisation usage’ to effectively assess
how the system is being used and also carry
out additional training when new versions
are released.
There is currently no provision of e-
learning material that clinicians are able to
access prior to using the electronic
prescribing system. However an online e-
learning module is being developed but
currently only being used internally.
Noema Life Galileo
Medication
Email 29th April
2015
[email protected] (Questions
provided)
1. In what format is training provided to
end-users e.g. class-room based training,
lectures, e-learning?
This is dependent upon the needs of the Trust and users. We can provide class-room based, ward-based, one-to-one and e-learning.
2. Who receives training delivered from
yourselves? Is it only key members of the
hospital team who are trained? Or do you
provide full hospital training programmes?
Depends on what is required. We can do just train the trainers or/ and hospital team through to hospital/Trust wide training.
3. Does training take place onsite or
21
offsite?
Either, though we recommend on-site as easier to get staff to attend.
4. How long does training normally
take?
This depends upon the user roles being trained as well as how IT literate the users are and if they have experience of previous EPMA systems. It can range from 2-3 hours to a whole day, particularly for users that require knowledge of different user role functionalities.
5. If e-learning is provided, what topics
are covered for example prescribing,
general workflow etc.?
The same topics as would be covered in any other training delivery approach.
QuadraMed
Corporation
QCPA Emailed 19th
April 2015,
Reply received
20th April 2015
Information
obtained online
http://www.quadr
amed.com/en/solu
tions_services/cli
nical_solutions/pr
ofessional_service
s/ [accessed 29th
April 2015]
Targeted Customised Training:
A range of classes are offered, including-
new implementation training, database
support training, upgrade service training
and customised training.
Training is offered both on and off site.
(No information was provided regarding
online training)
System C Medway Emails ( 20th
April 2015 and
23rd
April)
Phone call 29th
April (spoke to
member of sales
team)
A dedicated System C Business Education Specialist will be supplied to the Trust, who will work in partnership with the Trust training team to provide guidance, training and support. System C deliver Train the Trainer (TtT) training for the Trust training team, and offer advice on how to deliver end user training. The System C Training Lead will
22
continually assess Trust training staff to ensure that they meet the required competency levels to deliver to end users, and additional training/support can be given to Trust trainers who do not meet the required competency levels. Following completion of TtT the Trust trainers will work on developing the End User Training courses. Once this activity has been completed the Trust Trainers will be asked to deliver their courses to the System C Training Lead to ensure that the system is fully understood. If necessary, the System C Training Lead will provide additional training to supplement any gaps. Aside from the above, it is a Trust responsibility to organise, plan and deliver end user training, and their decision whether to include consolidation type exercises during this training. It is the responsibility of each Trust to deliver end-user training. In their experience, Trusts deliver a mixture of training styles dependant on the content and the type of user attending the sessions. Online learning material is provided for access throughout a project deployment.
TPP SystmONE Emailed 19th
April 2015 (no
reply)
Follow up phone
call 29th April
2015 (spoke to
member of sales
team)
‘Train-the-trainer’ sessions are provided by
TPP to designated staff members within the
hospital who will be given the knowledge
and skills to then train end-users within the
specific organisation.
TPP will also assist hospital trainers to
develop learning materials and tools
specific to the organisation’s needs. Full
end-user training can be provided by TPP,
however this is not the preferred method.
Top-up sessions are available if required to
re-train staff.
Train-the-trainer sessions are delivered
onsite at the hospital and typically last for 5
days, however this will vary by site.
23
Training on the ‘train the trainer’ course is
typically class-room based. There is no
provision for e-learning however the system
is integrated with a question and answer
style communication functionality to allow
queries to be addressed.
24
Outcome 3
b) Robust inductions to online training for newly qualified professionals to
ensure seamless use of new electronic system
BACKGROUND
ePrescribing has been associated with a range of potential benefits over
traditional paper-based systems, including improved patient safety, quality of care and
reduced costs.(1-4) The implementation of these Electronic Prescribing and
Medicines Administration (EPMA) systems with Clinical Decision Support (CDS) in
U.K. hospitals is expected to surge in the coming years due, in part, to the financial
incentives such as the NHS’s Integrated Digital Care Fund and the Safer Hospitals
Safer Wards Fund(5, 6)
A key element of the implementation and on-going use of a ePrescribing
system is ensuring that users are, and remain, sufficiently trained and competent to
use the system efficiently and effectively. The user training should be comprehensive
enough to cover all aspects of how a user may need to interact with a system to
effectively and safely undertake their role, but also any potential pitfalls and
challenges that they may encounter. Organisations face challenges in delivering
effective training including: large numbers of staff; staff resistance/availability to take
time from clinical activities to attend training; rotation between wards and specialties;
and temporary/short term staff. Little evidence has been published on the training
strategies used to familiarize staff with these systems, many of which change
following implementation through local customization and system upgrades.
25
Studies have suggested that insufficient training may be associated with
suboptimal use of a system.(7, 8) Baysari et al. found that a large number of CDS
alerts were generated by the improper use of the system, leading to the production of
‘technically preventable’ alerts.(7) Such studies highlight the importance of training
and education both in facilitating successful implementation of electronic systems and
averting errors.
We conducted a review of the literature to describe the approaches used to
train qualified prescribers on ePrescribing systems in a hospital setting. We were also
interested in knowing whether online training approaches in particular were used and
whether training provided covered the pitfalls and challenges of using these systems.
METHODS
Inclusion and Exclusion Criteria
Inclusion criteria included articles that explored the training of qualified
prescribers (Including medical and non-medical practitioners) on CPOE systems in a
hospital setting. Studies that explored training of undergraduate medical students,
training of clinical skills other than prescribing, or the use of electronic prescribing or
electronic health records in medical education (e.g., to enable students to monitor
patient progress) were excluded (Appendix 1 and 2).
Search Strategy and Study Selection
Three large databases were searched including: Cumulative Index Nursing and
Allied Health Literature (CINAHL), Embase (OVID), and Medline (OVID). The
search terms used are listed in Appendix 1. Sets of search terms employed included
“Electronic Prescribing” OR “Computerized Provider Order Entry” OR “Medical
Order Entry Systems” in Set 1; and “Clinical Decision Support” OR “Decision
26
Support System” in Set 2; and “Electronic Medical Record” OR “Electronic Patient
Record” in Set 3; and “Education Clinical” OR “Medical Education” in Set 4; and
“Education Distance” in Set 5; and “Prescribed” or “Prescribing” in Set 6 (Table 1).
These sets were combined with the Boolean operator “AND”. Only papers published
in English were considered. The search was performed on the 15th
May 2015. The
search terms related to training were kept deliberately broad to capture all relevant
publications. A separate search, which included ‘electronic prescribing’ and ‘online
training’, was also conducted. We did not restrict the timeframe for these searches in
any of the respective databases. In addition, we searched the websites of vendors of
electronic prescribing systems supplied in the U.K for suggested training approaches.
Data Extraction and Synthesis
All duplicate articles were removed. Titles and abstracts were initially
reviewed followed by the full text. Reference lists were also examined for additional
papers. Data were abstracted onto a customized data extraction sheet by the first
author (CLB), which included variables such as: title of the study; country of origin;
decision to include or exclude and justification for the choice. A narrative synthesis of
all eligible studies was undertaken.
27
Figure 1: Search Strategy Diagram: ‘all training’
Records identified through
database searching
(n = 1155)
Additional records identified
through other sources
(n = 1)
Records after duplicates removed and
screening of titles and abstracts
(n =16)
Full-text articles assessed
for eligibility
(n = 16)
Full-text articles
excluded, with reasons
(n =9)
Studies included in
qualitative synthesis
(n =7)
28
Figure 2: Search Strategy Diagram: ‘online training’
Records identified through database
searching
(n =25)
Records after duplicates removed and
screening of titles and abstracts
(n =16)
Full-text articles assessed for
eligibility
(n = 5)
Full-text articles excluded,
with reasons
(n =2)
Studies included in qualitative
synthesis
(n =3)
29
RESULTS
The search for ‘all training’ returned a total of 1,155 publications; after the
review of titles, abstracts and full text, a total of 1,149 were excluded (Figure 1). After
reviewing the reference lists of the remaining publications, one further article was
considered relevant and thus included. A total of seven publications were included,
comprising of five full text publications (9-13) and two conference abstracts. (14, 15)
The authors of the conference abstracts were contacted and asked for additional
information, including (i) the type of training delivered and whether they used any
online training methods (if unclear from the publication), (ii) whether they performed
a competence assessment, and (iii) whether the training was developed internally or
provided by the system developer. We obtained responses from all authors apart from
one.(15) We decided to include the two studies published by Borycki et al. and
Kushniruk et al., as there was potential for these training methods to be used for
practicing prescribers.(11, 12)
The separate search for the use of ‘online’ training methods returned a total of
25 publications. After reviewing the titles and abstracts, three relevant articles were
identified (Table 3), two of which had already been identified and included in the
search of “all training” approaches. The additional article found in this separate
‘online’ search(16) was included in our ‘all training’ search making eight publications
in total.
Traditional training approaches
Typically, a variety of training methods were used such as traditional
classroom-based sessions, which included ‘run through’ system demonstrations and
practical exercises, as well as face-to-face or ward-based training facilitated by
30
‘super-users’ (expert staff members that have received additional training). Super-
users were found to play a valuable role in providing ward-level support and reduce
the need for costly external training.(13) Tools such as e-learning packages, quick
reference guides such as a list for keyboard short cuts and ‘how to’ guides, were also
provided.(9, 14) Three studies used traditional classroom-based learning to train
users, one on a paediatric intensive care unit,(14) another across an integrated
delivery system(9) and the third study across two U.S. hospitals.(13) Users were
given an overview of the specific features of their system, using a combination of
demonstrations, lectures and practical exercises, thus allowing the users to gain
‘hands-on’ experience of using the system.(9, 14) In particular Bredfeldt et al.
encouraged staff to customize their own live version of the Electronic Health Record
(EHR) by, for example, creating preference lists, thus allowing users to immediately
experience the benefits of this functionality immediately.(9) Ensuring clinicians have
ample opportunities to attend training was important, so weekend and out-of-hour
sessions were organized for users in one study.(13)
In terms of user evaluation, formal assessments, quizzes and feedback
methods were utilized in three studies.(9, 14, 15) Bredfeldt et al. evaluated post-
training performance of two skills (covered during the training session) to measure
the effect of training.(9) Classroom-based training and ‘hands-on’ activities were
found to have been associated with improved utility of certain functions.(9) However,
users would have appreciated more opportunities to receive training on the ‘live’
system and felt that the range of topics covered should be broader.(9) Bredfeldt et al.
also sent follow-up e-mails to users to report their usage of specific features and
compared their activity with that of their peers, serving to remind users of the learning
material and track their progress.(9)
31
Online training approaches
Web-based demonstrations were used in only one study.(15) A team at the
University of Victoria in Canada developed an online portal, which housed a range of
simulated versions of different EHRs containing electronic prescribing functionality.
Healthcare professional students, practicing professionals and healthcare
informaticians were all given access to this portal where they could prescribe for
fictitious patients in a safe environment rather than in a real setting.(11, 12, 16) The
portal also provided an opportunity for users to learn about the design of different
systems that influence clinical practice and adoption.(11, 12, 16)
Evaluation of online training methods was limited. Experiences and lessons
learned from the University of Victoria’s EHR electronic portal appeared to be
positive, with users in particular perceiving the experience as valuable and having a
greater understanding of how EHR systems were to be used in practice.(11) Ayoub et
al. did not specify how quizzes were developed or which areas were assessed;
although trainees reportedly scored highly in these quizzes.(15) Jimenez highlighted
the importance of providing timely feedback to users after completing exercises.(10)
Clinical scenarios and exercises
Two studies described using targeted clinical scenarios that focused on
particular problem areas to train staff. Foster et al. developed exercises based on
commonly encountered prescribing errors, such as the prescribing of Tazocin®
(piperacillin-tazobactam, an antibacterial) at non-standard times.(14) Bredfeldt et al.
targeted training to specific clinical areas, such as pre-operative patient visits, where
32
there had been a number of support requests from existing users.(9) It was not clear
whether these areas were also associated with particular system pitfalls. Developing
expertise-specific scenarios relevant to clinicians from different specialist areas was
considered important.(10, 16)
DISCUSSION
The papers identified in this review outlined a number of methods used to
train qualified prescribers, including classroom-based sessions with demonstrations
and ‘hands-on’ exercises. Some studies incorporated a form of assessment, which
allowed users to track their own progress and informed senior staff about those who
may need further assistance.(9, 14, 15) Studies also incorporated clinical scenarios
aimed at addressing commonly encountered prescribing errors or frequent technical
support requests.(9, 14) Although not specified explicitly, such problem areas may be
indicative of system flaws that may contribute to the occurrence of errors or poor
usability.
Using a combination of learning methods is likely to appeal to the learning
styles of different users. McCain et al. found it challenging to get users to attend
classroom-based training sessions due to other clinical commitments. However, users
also felt that these sessions failed to address their learning needs being too simplistic
or too advanced. This resulted in a blended learning strategy being adopted that
included a combination of computer-based learning exercises and a training CD,
which facilitated ‘self-study’ where users could train at a convenient time and
pace.(17) Trainees valued the choice of alternative training methods;(13, 17-19) Ross
and Banchy used a combination of one-to-one and group classroom-training sessions
in order to address the specific needs of end-users and maximize staff attendance .(13)
33
Laramee et al. found that participants preferred written guidance on how to carry out
particular tasks rather than computer ‘help’ functions. Organizations should therefore
consider providing a range of learning tools to meet users’ needs.(19) It is likely that
there may be other training methods employed in practice not discussed in the small
number of articles found in this review.
The use of e-learning as a method of informing and training clinicians on an
electronic prescribing system was considered important.(9, 10) One study, which
delivered educational material primarily to nurses via an e-learning tutorial, was
associated with high completion rates and improvements in the completeness of
documentation within the EHR.(20) Material should be engaging, potentially
including interactive scenarios and quizzes, simplicity over complexity was
emphasized; e-learning should be concise, but informative, learning outcomes should
be clearly specified, and care should be taken to limit the amount of information
presented to trainees.(20)
Training specifically aimed towards educating prescribers about the
challenges and pitfalls of electronic systems was rarely discussed. Studies did include
education and training as a solution to some of “the issues” encountered with such
systems.(7, 21-23) Sittig et al. made specific recommendations, such as, providing
adequate training opportunities for all clinicians to experience the system prior to
implementation, potentially enforcing a minimum level of training before clinicians
are authorized to use the system. He also proposed that organizations deliver multiple
‘walk-throughs’ of the different processes for specific clinical staff.(22) This further
supports the studies by Foster et al. and Bredfeldt et al, which highlight the need to
specifically tailor the clinical scenarios and content of training to the role, expertise
and tasks performed by the user.(9, 14, 24, 25) Training approaches should
34
encompass both procedural tasks (e.g., prescribing a medicine) and cognitive tasks
(e.g., interpreting a CDS alert) so that prescribers may realize the full potential of the
system.(24)
CONCLUSION
Organizations are currently using a range of learning methods to train
qualified prescribers to use electronic systems. Online learning may facilitate the
training of a large number of users, offering them the opportunity to practice and
become familiar with the system at a time and place that is convenient to them.
However, the lack of papers retrieved suggests a need for additional studies to inform
training and assessment methods. Finally, further research should explore the best
way of training users about the pitfalls, challenges and the potential benefits
associated with electronic systems.
35
Table 1: Search Terms
Electronic
Prescribing
Clinical
Decision
Support
Electronic
Health
Record
Training Online
Training
Prescriber
(Included in
Embase
Search Only)
Computerized
prescriber
order entry
Computerized
provider order
entry/
Electronic
physician
order entry
Electronic
order entry
Electronic
prescribing/
Electronic
prescription
Computerized
physician
order entry
CPOE
Computerized
order entry
Medical order
entry systems
Clinical
decision
support
Decision
support
system/
CDS
Drug
therapy,
computer
assisted
Electronic
medical
record/
Electronic
health
record
Electronic
patient
record
Education/
Clinical
education/
Training/
Course
Competence/
Medical
education/
Clinical
competence/
Competence
assessment
Prescriber
training
Prescriber
assessment
Education,
Distance/
Distance
learning
Educational,
non-
traditional
(CINAHL
only)
Prescribed
Prescribing
Prescription
36
Search Strategy
1. Computerized prescriber order entry
2. Computerized provider order entry/
3. Electronic physician order entry
4. Electronic order entry
5. Electronic prescribing/
6. Electronic prescription
7. Computerized physician order entry
8. CPOE
9. Computerized order entry
10. Medical order entry systems
11. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10
12. Clinical decision support
13. Decision support system/
14. CDS
15. Drug therapy, computer assisted
16. 12 or 13 or 14 or 15
17. Electronic medical record/
18. Electronic health record
19. Electronic patient record
20. 17 or 18 or 19
21. Prescribed
22. Prescribing
23. Prescription
24. 21 or 22 or 23
25. Education/
26. Clinical education/
27. Training/
28. Course
29. Competence/
30. Medical education/
31. Clinical competence/
32. Competence assessment
33. Prescriber training
34. Prescriber assessment
35. 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32 or 33 or 34
36. 11 or 16 or 20
37. 35 and 36
38. 24 and 37 for embase to refine search due to high number of returns
39. Limit to English language
40. Education, Distance
41. Distance Learning
42. Education, non-traditional (CINAHL only)
43. 40 or 41 (or 42)
44. 36 and 43
37
Appendix 1: Articles included and excluded following review of abstract: Any Training
Number Paper
(Author,
Year)
Title Database Study Type Country Inclusion
after Review
of Abstract
Justification
for
Exclusion
Inclusion
After
Review of
Full Text
Justification
for
Exclusion
1 Avery AJ,
2014
Research
into practice:
Safe
prescribing.
Embase Report
Summary
UK No Primary care
Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
2 Baysari MT,
2012
Understandin
g doctors'
perceptions
of their
prescribing
competency
and the value
they ascribe
to an
electronic
prescribing
system.
Medline
Embase
Qualitative Australia No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
38
3 English t,
2010
Obstacles to
Rolling Out
an EMR in a
Residency.
Embase No Non-
hospital
setting
Lack of focus
on training
and
competency
on an
electronic
prescribing
system
-
4 Haffey F,
2014
Smartphone
apps to
support
hospital
prescribing
and
pharmacolog
y education:
A review of
current
provision.
Embase UK No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
5 Kamerow D,
2010
What i learnt
from mom.
Embase Viewpoint US No Lack of
relevance
-
39
6 Kaur D, 2015 E learning:
Moving
towards a
technological
ly advanced
and
progressive
psychiatry!.
Embase Conference;
workshop
India No Lack of focus
on training
and
competency
on an
electronic
prescribing
system
-
7 Larson KA,
2004
Reducing
medication
errors in a
surgical
residency
training
program.
Embase
Medline
Qualitative US No Hospital not
using EP, not
relevant to
training/
prescribing
competency
-
8 Miller A S,
2003
The training
process (Part
1).
Embase No Unable to
access
-
9 Ross S, 2012 Prescribing
and the core
curriculum
for
tomorrow's
Embase Review UK No Lack of
relevance for
qualified
prescribers
and
-
40
doctors: BPS
curriculum in
clinical
pharmacolog
y and
prescribing
for medical
students.
electronic
systems
10 Adibe BA,
2010
Electronic
health
records:
potential to
transform
medical
education.
Medline Supplementa
ry piece,
commentary
US No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
11 Bloice MD,
2014
Casebook: a
virtual
patient iPad
application
for teaching
decision-
making
through the
use of
electronic
health
records.
Medline Learning
Tool
Developmen
t
Austria No Lack of focus
on training
and
competency
on an
electronic
prescribing
system
-
12 Chi J, 2014 Clinical Medline Opinion US No Lack of -
41
education
and the
electronic
health
record: the
flipped
patient.
CINAHL piece. focus on
training and
competency
on an
electronic
prescribing
system
13 Elliott K,
2011
A student-
centred
electronic
health record
system for
clinical
education.
Medline Qualitative Australia No Lack of
relevance for
qualified
doctors
Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
14 Han H, 2013 Writing and
reading in
the
electronic
health
record: an
entirely new
world.
Medline Qualitative US No Lack of focus
on training
and
competency
on an
electronic
prescribing
system
-
42
15 Hart J, 2010 University of
Arkansas for
Medical
Sciences
electronic
health
record and
medical
informatics
training for
undergradua
te health
professionals
.
Medline
CINAHL
Report US No Lack of
relevance for
qualified
doctors/
prescribers
-
16 Keenan CR,
2006
Electronic
medical
records and
their impact
on resident
and medical
student
education.
Medline Literature
Review
US No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
17 Knight AM,
2012
The effect of
computerised
provider
order entry
on medical
student’s
Medline Comparative
study
US No Lack of
relevance for
qualified
doctors
-
43
ability to
write orders.
18 Knight AM,
2007
The good
news about
CPOE and
medical
student
ordering
ability.
Medline Comparative
study
US No Lack of
relevance for
qualified
doctors
-
19
Kushniruk
AW, 209
Bringing
electronic
patient
records into
health
professional
education:
towards an
integrative
framework.
Medline Educational
tool
development
Canada Yes - -
20 Morrison F,
2011
Developing
an online
and in-
person HIT
workforce
training
program
using a
team-based
learning
Medline Qualitative US No Lack of focus
on training
and
competency
on an
electronic
prescribing
system
-
44
approach.
21 Moser S E,
2010
Precepting
medical
students in
the era of
EHRs.
Medline Report US No Lack of
relevance for
qualified
prescribers
-
22 Pageler N M,
2013
Refocusing
medical
education in
the EMR era.
Medline Viewpoint US No Lack of focus
on training
and
competency
on an
electronic
prescribing
system
(focus on
EMR)
-
23 Pippitt K,
20113
Medical
student
education in
the EMR era
requires
access to the
EMR.
Medline Comment;
letter
US No Lack of
relevance for
qualified
doctors
Lack of
focus on
training and
competency
-
45
on an
electronic
prescribing
system
24 Reis S, 2013 The impact
of residents'
training in
Electronic
Medical
Record
(EMR) use
on their
competence:
report of a
pragmatic
trial.
Medline
CINAHL
Comparative
study
Israel No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
25 Schenarts PJ,
2012
Educational
impact of the
electronic
medical
record
[Review]
Medline Literature
Review
UK No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
-
26 Schifferdeck
er KE, 2012
Adoption of
computer-
assisted
learning in
medical
education:
the
educators'
Medline Mixed
Methods
UK No Lack of
relevance for
qualified
prescribers.
-
46
perspective.
27 Maxwell S,
2012
e-Learning
initiatives to
support
prescribing.
Embase Review UK Yes
No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
28 Tierney MJ,
2013
Medical
education in
the electronic
medical
record
(EMR) era:
benefits,
challenges,
and future
directions.
Medline
Perspective
piece
US Yes
No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
29 Ellaway RH,
2013
Medical
education in
an electronic
health
record-
mediated
world.
Medline Thematic
analysis
Canada Yes No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
30 Ayoub N,
2014
Developing
competency
through
webinar to
Embase Conference;
training
service
Pakistan;
Tanzania
Yes
47
establish
oncology
pharmacy
services at
the Aga Khan
Hospital Dar-
es-Salaam
Tanzania.
development
31 Foster S,
2011
Competency
based
training
program for
electronic
prescribing
improves
patient
safety.
Embase Evaluation of
training
program
(Conference
Abstract)
UK Yes
32 Borycki EM,
2009
The
University of
Victoria
Interdisciplin
ary
Electronic
Health
Record
Educational
Portal.
Medline Development
of
educational
portal for
EHRs
Canada Yes
33 Bredfeldt
CE; 2013
Training
providers:
beyond the
Medline Mixed
methods
US Yes Training
offered to
users of an
48
basics of
Electronic
health
records.
EHR, which
included
order entry
34 Jimenez, A
2010
E-learning
supports
EHR
implementati
ons. In
addition to
meaningful
use, we need
to define
meaningful
training
Review of
References
Viewpoint US Yes Yes
34 Baillie, L et
al., 2013
A survey of
student
nurses’ and
midwives’
experiences
of learning to
use
electronic
health record
systems in
practice.
CINAHL Quantitative
(questionnair
es) and
Qualitative
(focus group)
UK No Non-
prescribers
Undergradua
te level
35 Pattillo, R Cleveland
Clinic leads
the way in
electronic
medical
record
CINAHL Issue Brief US Yes No Lack of
qualified
doctor/
prescriber
relevance.
49
training
36 Ornes LL
and Gassert
C, 2007
Computer
competencies
in a BSN
program
CINAHL Report of
curriculum
evaluation
US No Lack of
qualified,
prescriber
relevance.
37 Liaw, ST et
al., 2000
Computer
education:
don’t forget
the older
GPs.
CINAHL Quantitative
evaluation
Australia No Primary care
38 Wong, B et
al., 2012
Computerise
d provider
order entry
and
residency
education in
an academic
medical
centre.
CINAHL Qualitative Canada No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
39 Sanchez-
Mendiola, M
et al., 2013
Development
and
implementati
on of a
biomedical
informatics
course for
medical
students:
challenges of
a large-scale
blended-
learning
CINAHL Curriculum
development
Mexico No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
50
program.
40 Warboys, I et
al., 2014
Electronic
Medical
Records in
Clinical
Teaching
CINAHL Evaluation US No Lack of
relevance to
qualified
prescriber
training
41 Shachak, A
et al., 2012
End-user
support for a
primary care
electronic
medical
record; a
qualitative
case study of
a vendor’s
perspective
CINAHL Qualitative Canada No Primary care
setting
42 Hoyt, R et
al., 2013
Evaluating
the Usability
of a Free
Electronic
Health
Record
Training
CINAHL Quantitative
and
Qualitative
US Yes No Lack of
qualified
prescriber
training
relevance.
43 Byrne, M D,
2012
Informatics
Competence
in the EHR
Era…
‘electronic
health
record’.
CINAHL Opinion
piece
US No Lack of
qualified
prescriber
training
relevance
44 Hart MD Informatics
competency
CINAHL Systematic
Review
US No Lack of
relevance for
51
and
development
within the
US nursing
population
workforce: a
systematic
literature
review
training of
qualified
prescribers
45 Edwards, G,
2012
Innovative
health
information
technology
training:
exploring
blended
learning.
CINAHL Mixed
Methods
US Yes No Lack of
prescriber
training
relevance
46 Price. D et
al., 2009
Interprofessi
onal
education in
academic
family
medicine
teaching
units: a
functional
program and
culture
CINAHL Report on
interprofessi
onal practice
experience
Canada No Lack of
prescriber
training/
relevance.
Primary care
setting
47 Laramee. A
S et al., 2011
Learning
from within
to ensure a
successful
CINAHL Qualitative Canada Yes - No Lack of
focus on
training and
competency
52
implementati
on of an
electronic
health
record.
on an
electronic
prescribing
system (Not
clear if EHR
included
electronic
prescribing)
48 Turner. M P.,
2010
Stratifying
computer
literacy; a
competency
measurement
strategy
CINAHL Report US No Lack of
relevance to
prescriber
training
49 Gomes. A
W., 2013
Strengthenin
g Our
Collaboratio
ns: Building
an Electronic
Health
Record
Educational
Module
CINAHL Report of
module
development
US No Lack of
relevance to
qualified
prescribers.
50 Kassum. D
and Peloso.
E., 2009
Targeting
adoption,
training and
device
deployment
strategies.
CINAHL Quantitative
evaluation
US No Lack of
qualified
prescriber
training
relevance
51 Robertson.
M and
Callen. J.,
The
education
needs of
CINAHL Quantitative Australia No Lack of
qualified
prescriber
53
2003. health
information
managers in
an electronic
environment:
what
information
technology
and health
informatics
skills and
knowledge
are required.
training
relevance
52 Janssen. D
G., 2011
The effect of
nursing
leadership
and teaching
methodologi
es on the
level of
adoption on
an electronic
health record
(EHR)
implementati
on
CINAHL Quantitative US No Lack of
qualified
prescriber
training
relevance
53 Schumacher.
D., 2010
The
electronic
medical
record and
clinical
nursing
CINAHL Report of
challenges
when
educating
nursing
student and
US No Lack of
qualified
prescriber
training
relevance.
54
student
instruction:
tips and
tricks for
success.
faculty about
updates to
the EMR.
54 Ross. C and
Banchy. P.,
2007
The key to
CPOE:
thoughtful
planning,
flexible
training and
strong staff
involvement
leads to a
successful
CPOE
implementati
on.
CINAHL Case history
of
implementati
on
US Yes Yes
55 McCain. C
L., 2008
The right
mix to
support
electronic
medical
record
training:
classroom
computer-
based
training and
blended
learning.
CINAHL Lessons
learnt from
training
strategy
US Yes No Lack of
focus on
training and
competency
on an
electronic
prescribing
system (not
clear if EHR
includes
electronic
prescribing)
56 Aleem. S, Translating CINAHL Project US No Lack of
55
2013 10 Lessons
from Lean
Six Sigma
Project in
Paper-Based
Training Site
to Electronic
Health
Record-
Based
Primary Care
Practice:
Challenges
and
Opportunitie
s.
Report qualified
prescriber
training/relev
ance
Primary Care
57 Ulicny. M P.,
2011
Using an
Electronic
Health
Record in an
Introduction
to
Professional
Nursing
Course
CINAHL Abstract of
nurse
training
approach.
US No Lack of
qualified
doctor/
prescriber
training
relevance.
58 Wolf MS,
2013
Shifting
upstream:
Efficacy trial
of a low
literacy,
EMR
medication
Embase Conference US No Lack of
prescriber
training
relevance.
56
education
strategy.
Appendix 2: Articles included and excluded following review of abstract: Online Training
Number Paper
(Author,
Year)
Title Database Study Type Country Inclusion
after Review
of Abstract
Justification
for
exclusion
Inclusion
After
Review of
Full Text
Justification
for
exclusion
1 Borycki EM,
2009
From prototype
to production:
lessons learned
from the
evolution of an
EHR
educational
portal
Medline Development
of
educational
portal for
EHRs
Canada Yes Yes
2 Jimenez, A
2010
E-learning
supports EHR
Medline Viewpoint US Yes Yes
57
implementation
s. In addition to
meaningful use,
we need to
define
meaningful
training
3 McKinney,
M 2012
Docs helping
docs embrace
IT; organization
uses online
tools to promote
value of the
technology
Medline Project
Report
US N/A (no
abstract)
N/A no
abstract)
No Lack of
prescriber
training/
competence
relevance
4 Topaz M,
2013
Educating
clinicians on
new elements
incorporated
into the
electronic
health record;
theories,
evidence and
one educational
project
Medline Training
program
development
US Yes - No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
5 McCullagh
P, 2001
Student-
centered
distance
learning in
health and
medical
informatics
Embase Conference
Poster
UK No Lack of
focus on
training and
competency
on an
electronic
prescribing
58
system
6 Masic I,
2013
The history and
new trends of
medical
informatics
Embase Review Bosnia and
Herzegovina
No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
7 McGuire MJ,
2013
Evolution of an
internet-based
quality focused
medical
education
process in an
ambulatory care
organization
Embase Conference
Abstract
US No Lack of
focus on
training and
competency
on an
electronic
prescribing
system
8 Ayoub N,
2014
Developing
competency
through webinar
to establish
oncology
pharmacy
services at the
Aga Khan
Hospital Dar –
es-Salaam
Tanzania
Embase Conference;
training
service
development
Pakistan;
Tanzania
Yes Yes
9 Welton. N.,
2010
The University
of Washington
electronic
medical record
CINAHL US Report on
development
of
educational
No Lack of
qualified
prescriber
training
59
experience. resources. relevance
60
References
1. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, et al. The
impact of computerized physician order entry on medication error prevention.
JAMIA. 1999;6(4):313-21.
2. Garg A, Adhikari N, McDonald H, Rosas-Arellano M, Devereaux P, Beyene
J, et al. Effects of computerized clinical decision support systems on practitioner
performance and patient outcomes. A systematic review. JAMA : the journal of the
American Medical Association. 2005;293:1223 - 38.
3. Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, et al. Return on
investment for a computerized physician order entry system. Journal of the American
Medical Informatics Association : JAMIA. 2006;13(3):261-6.
4. Nuckols TK, Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson
LJ, et al. The effectiveness of computerized order entry at reducing preventable
adverse drug events and medication errors in hospital settings: A systematic review
and meta-analysis. Systematic Reviews. 2014;3(1).
5. NHS England. The Integrated Digital Care Fund: Achieving integrated health
and care records. 2014.
6. NHS England. Safer Hospitals, Safer Wards: Achieving an Integrated Digital
Care Record. 2013.
7. Baysari MT, Reckmann MH, Li L, Day RO, Westbrook JI. Failure to utilize
functions of an electronic prescribing system and the subsequent generation of
'technically preventable' computerized alerts. JAMIA. 2012;19(6):1003-10.
8. Noblin A, Cortelyou-Ward K, Cantiello J, Breyer T, Oliveira L, Dangiolo M,
et al. EHR implementation in a new clinic: A case study of clinician perceptions.
Journal of Medical Systems. 2013;37(4).
9. Bredfeldt C, Awad EB, Joseph K, Snyder M. Training providers: beyond the
basics of electronic health records. BMC Health Services Research. 2013;13(1):503.
10. Jimenez A. E-learning supports EHR implementations. In addition to
meaningful use, we need to define meaningful training. Health Manag Technol.
2010;31(11):22-3.
11. Borycki EM, Kushniruk AW, Joe R, Armstrong B, Otto T, Ho K, et al. The
University of Victoria Interdisciplinary Electronic Health Record Educational Portal.
Stud Health Technol Inform. 2009;143:49-54.
12. Kushniruk AW, Borycki EM, Armstrong B, Joe R, Otto T. Bringing electronic
patient records into health professional education: towards an integrative framework.
Stud Health Technol Inform. 2009;150:883-7.
13. Ross C, Banchy P. The key to CPOE: thoughtful planning, flexible training
and strong staff involvement leads to a successful CPOE implementation. Health
Management Technology. 2007;28(11):22.
14. Foster S, Murray D, Richardson J, McFadzean J, McCormack J. Competency
based training program for electronic prescribing improves patient safety. Pediatric
Critical Care Medicine. 2011;1):A19.
15. Ayoub N, Sheikh AL, Ahsan S, Zaheer F. Developing competency through
webinar to establish oncology pharmacy services at the Aga Khan Hospital Dar-es-
Salaam Tanzania. Journal of Oncology Pharmacy Practice. 2014;1):10.
61
16. Borycki EM, Armstrong B, Kushniruk AW. From Prototype to Production:
Lessons Learned from the Evolution of an EHR Educational Portal. AMIA Annual
Symposium Proceedings. 2009;2009:55-9.
17. McCain CL. The right mix to support electronic medical record training:
classroom computer-based training and blended learning. Journal for nurses in staff
development : JNSD : official journal of the National Nursing Staff Development
Organization. 2008;24(4):151-4.
18. Edwards G, Kitzmiller R, Breckenridge-Sproat S. Innovative health
information technology training: exploring blended learning. Comput Inform Nurs.
2011;30(2):104 - 9.
19. Laramee AS, Bosek M, Kasprisin CA, Powers-Phaneuf T. Learning from
within to ensure a successful implementation of an electronic health record. Comput
Inform Nurs. 2011;29(8):468-77; quiz 78-9.
20. Topaz M, Rao A, Masterson Creber R, Bowles KH. Educating clinicians on
new elements incorporated into the electronic health record: theories, evidence, and
one educational project. Comput Inform Nurs. 2013;31(8):375-9; quiz 80-1.
21. Campbell EM, Sittig DF, Guappone KP, Dykstra RH, Ash JS.
Overdependence on Technology: An Unintended Adverse Consequence of
Computerized Provider Order Entry. AMIA Annual Symposium Proceedings.
2007;2007:94-8.
22. Sittig DF, Ash JS, Guappone KP, Campbell EM, Dykstra RH. Assessing the
anticipated consequences of Computer-based Provider Order Entry at three
community hospitals using an open-ended, semi-structured survey instrument. Int J
Med Inform. 2008;77(7):440-7.
23. Zhou L, Mahoney LM, Shakurova A, Goss F, Chang FY, Bates DW, et al.
How Many Medication Orders are Entered through Free-text in EHRs? - A Study on
Hypoglycemic Agents. AMIA Annual Symposium Proceedings. 2012;2012:1079-88.
24. van Stiphout F, Zwart-van Rijkom JE, Maggio LA, Aarts JE, Bates DW, van
Gelder T, et al. Task analysis of information technology-mediated medication
management in outpatient care. Br J Clin Pharmacol. 2015; doi: 10.1111/bcp.12625
25. Ross S, Patey R, Flin R. Is it time for a nontechnical skills approach to
prescribing? Br J Clin Pharmacol. 2014;78(4):681-3.
62
Outcome 4: The common prescribing errors made when using
electronic systems.
Aim: To describe the common prescribing errors made when using
ePrescribing systems
METHODS
A literature search was performed in Medline (OVID) and Embase (OVID) to
identify publications related to prescribing errors associated with ePrescribing. This review
considered all types of studies published since 2004. Only papers published in English were
included. MeSH terms and key word related to ‘electronic prescribing’, and errors (See Table
1) were used, using ‘AND’ ‘OR’ Boolean operators. Further publications identified from key
experts in the field were also reviewed. The search returned over 2000 publications. This
review intends to provide a summary of selected papers, which describe the common
prescribing errors encountered with ePrescribing systems and the stages in the prescribing
process at which they occur.
RESULTS
There is strong evidence to support the use of ePrescribing, which has been
associated with reduced medication errors (1, 2) increased prescription legibility and
completeness,(3) improved patient safety, patient care and healthcare costs.(4-6) However
there have also been reports that ePrescribing has contributed to some new types of errors.(7,
8)
Selecting the Wrong Patient
Prescribing medicines for the wrong patient can have significant consequences for
patient safety.(9) ePrescribing has been associated with wrong patient selection errors by five
main mechanisms. Firstly, selection errors from patient lists have been described in the
63
literature,(7, 8, 10, 11) especially with the use of long dense lists of patient names, for
example an entire ward,(8) and use of an alphabetical patient list, which could contribute to
juxtaposition errors if patient names look, sound or indeed are the same. This is further
compounded by the use of busy and fragmented computer ordering displays, where clinicians
must prescribe from or use multiple systems,(12) which in turn can disrupt workflow.(13)
Thirdly, Campbell et al. commented on the possibility of accidentally prescribing medicines
for a test patient, thus withholding drugs from an actual patient; however this is only likely to
occur if the test patient name was not clearly distinguishable from real patients names e.g.
‘ZZZTestJohnSmithZZ.(8) Fourthly, Savage et al. encountered junior doctor reports of
patient selection ‘near-misses’ due to remote prescribing, thus removing the need to visit a
patient’s bedside.(10) Finally, unclear logging in and off processes may result in clinicians
working under a different log in (previous user did not log off) and inadvertently prescribing
inappropriate medicines for a particular patient and possibly fail to prescribe required
treatments for others.(7)
Selecting the Wrong Drug
Erroneous selection
Incidents of prescribers inadvertently picking the wrong drug have been widely reported.(7,
10, 13-22) Certain features of the list are likely to increase the likelihood of these errors
occurring. For example, an alphabetical drug list that does not distinguish between ‘look-
alike, sound-alike’ medicines e.g., ventavis and ventolin(23) led to the selection of Ventavis
(iloprost) instead of Ventolin (salbutamol).(23) Similarly Westbrook et al. revealed that
methylprednisolone acetate (a drug which should not be administered intravenously) was
selected instead of methylprednisolone sodium succinate for intravenous administration; this
error occurred despite the use of a warning alert.(14) Additionally, Koppel et al. highlighted
an issue of ePrescribing design, which may require accessing multiple screens in order to
view all of a patient’s medicines, thus increasing confusion and likelihood of making a
selection error.(7) It is also important to be aware of auto-complete functions, whereby the
computer suggests a drug based on the first few letters entered, which may also lead to
clinicians inadvertently prescribing the wrong drug; this error may not be noticed until a later
date.(24)
Inappropriate drug choice
64
Inappropriate drug errors is a term used here to describe instances where a drug that is not
clinically appropriate for a patient, due to concomitant medications, contraindications or lack
of suitable indication, has been prescribed; and also instances when drugs which may be
clinically appropriate but are not the preferred option have been prescribed. The literature
outlines some examples of how these prescribing errors are facilitated through the use of
ePrescribing. Firstly, the use of order-sets,(13) which include a pre-determined selection of
drugs for an indication, may unintentionally result in contraindicated drugs being prescribed.
Savage et al. describes the example of a non-steroidal analgesic being unintentionally
prescribed to a patient with a history of asthma, as it was ‘hidden’ within an order set.(10)
Walsh et al. also identified a case where an order set for infants, which included vaccinations,
resulted in premature infants receiving a hepatitis B vaccine too early.(11) It is unlikely that
order-sets will be specific to individual patients and may include multiple inappropriate
drugs, therefore it is important that prescribers are aware of system limitations. When using
order sets, clinicians may be given the option to ‘select all’ or ‘deselect all’ which increases
the likelihood of prescribing items that they previously may not have considered.
Furthermore, as with other forms of prescribing error, the prescriber may not be aware of any
issues with the prescription due to gaps in clinical knowledge.(13) Depending on the level of
clinical decision support active within an ePrescribing system, alerts may recommend that
clinicians prescribe certain treatments, which are not appropriate due to stocking problems or
perhaps are not on the hospital formulary.(24) This could delay appropriate treatments, that
are available, reaching the patient if the prescriber is not aware of this issue.
Wrong Dose Errors
Wrong Dose Selection
Studies have described dosage errors occurring with the use of an ePrescribing system.(4, 7,
8, 14, 16) Dosage errors are commonly attributed to selection or scrolling errors from a drop
down menu, which results in an inappropriate dosage, listed close to the correct dose, being
prescribed erroneously.(7, 8, 10, 20, 25) For example, Shulman et al. reported a potentially
fatal error that occurred when the dose of diamorphine was prescribed using a drop down
menu at 7mg/kg instead of 7mg, which could have resulted in a 70 times overdose.(20) It was
unclear whether “7mg” was actually listed as an option for the prescriber to select in this case.
Other studies have also demonstrated the inappropriate use of a ‘standard’ dose when a non-
standard dose was required.(16, 21, 23, 25) It has been posed that clinicians use the pre-
defined dosage list as a prescribing guide, from which a typical dose can be selected.
65
However, in reality the dosage list may be based on an inventory list of available strengths of
a drug formulation, which may be misleading.(7, 21) Similarly, dosage lists which are not
comprehensive, for example a lack of non-standard doses, may require clinicians to complete
a ‘free-text’ order, which are associated with specific errors themselves and likely bypass
clinical decision support checks.(18) Instances of clinicians failing to document a maximum
daily dose have also been reported; this is particularly critical when two forms of the same
drug have been prescribed to aid administration flexibility.(10) Finally, Walsh et al. described
computer typographical errors occurring for example, ibuprofen was prescribed as 5mg rather
than 50mg.(11)
Duplicate Dose
Duplicate dose errors describe occasions where the same drug is prescribed more than once.
This may occur when drugs are prescribed by different routes of administration, two or more
medicines are prescribed which contain the same drug (e.g. combination drug products), and
human oversight. Such errors have been commonly documented.(4, 7, 11, 13, 14, 17) Koppel
et al. explains how such errors may be facilitated by ePrescribing, when a clinician modifies
an existing order or generates a new prescription without discontinuing the original order,
thus duplicate orders remain active and could potentially be given inappropriately.(7) Other
studies have also suggested that clinicians failing to discontinue drugs that are no longer
needed may also contribute to duplicate dose errors.(22) A typical example is when an
intravenous form of a medicine (e.g. with antibiotics and steroids) is initially used and then
stepped down to oral.(23) Duplication errors may also be due to inflexible ordering or
improper use of the system, which results in clinicians generating multiple prescriptions of
the same drug via different routes, if they want to provide nurses with administration
options.(11, 18) Campbell et al. observed an increased likelihood of duplicate doses when
prescribing remotely on an electronic system.(8) Fragmented order screens, which do not
easily allow clinicians to view all active medicines at once, may contribute to the occurrence
of duplicate dose prescribing errors. In particular, a systematic review of thirteen papers
found three studies that identified an increase in duplicate prescriptions following
ePrescribing implementation, citing poor system design such as not being able to display
STAT and PRN orders simultaneously.(25)
Finally, clinicians may also unintentionally prescribe duplicate doses when using the free-text
comment box.(18) Free-text comment boxes within an ePrescribing system are an important
mechanism to allow clinicians to add supporting information and aid communication. For
66
example, dosage lists which lack non-standard doses, may require clinicians to complete a
‘free-text’ order in order to request that dose.(18) This method avoids the inflexibility of
structured orders from drop down menus or pre-defined order sentences. However,
discrepancies have been identified between structured orders and the accompanying free-text
comments.(14, 21, 22, 26) A concern with overusing the free-text comment box is that the
computer is unable to perform safety checks on un-coded data (i.e. not using the drop down
menus) therefore errors may be missed.(21, 27) Use of free-text has been mostly associated
with orders for complex drug regimens such as variable doses or frequencies for example and
possibly also for high risk drugs such as warfarin, insulin and digoxin.(26)
Selecting the Wrong Formulation, Strength and Route
Wrong Strength
Selection of the wrong strength of a formulation (e.g., 50mg) or the wrong units (e.g., mg
instead of mcg) has also been identified.(11, 14) Examples include selecting 50mg
cyclosporine capsules for a dose of 75mg, thus a sub-therapeutic dose may be given and/or, if
picked up, additional clinician time may be needed to rectify the discrepancy.(14) The wrong
strength units of a medicine being selected from a drop down menu, for example selecting
‘900g’ instead of ‘900mg’ of ceftriaxone.(11) Often these faults are intercepted by nurses
when they encounter difficulties administering the dose. However errors encountered by
inexperienced staff or prescriptions for less commonly used drugs may not be so easily
detected.
Wrong Formulation
Selection of the wrong formulation, primarily from drop down menus, has also been
highlighted in the literature.(10, 14, 19, 20, 22, 25, 28) Schulman et al. identified a
prescription for non-liposomal amphotericin 180mg (injection) once daily when the liposomal
formulation was intended. The doses of these two formulations are not considered
interchangeable and this could have led to unnecessary side effects.(20) Errors have also been
made by selecting an available order sentence but failing to correctly change all parameters
such as formulation.(14)
Route Errors
Examples of clinicians prescribing the wrong route of administration on ePrescribing systems
can also be found.(4, 7, 11, 14, 17) This has typically been associated again with selection
67
errors from a drop-down menu.(7) For example ceftriaxone was ordered ‘Intraperitoneal (IP)’
rather than ‘Intravenous (IV)’,(11) salbutamol as an IV injection when the correct route was
inhalation,(14) pantoprazole to be given intra-articularly when the desired route was IV
infusion. (14) The latter two examples were linked to either the erroneous editing or
construction of order sentences. Cho et al. found that the route of administration was
frequently omitted, possibly due to clinicians taking shortcuts when prescribing or assuming
the route was self-explanatory, thus resulting in possible incorrect or default selections.(17)
Missing information can lead to confusion and mistakes occurring (17) and delays in patients
receiving a medicine, while additional explanation is sought. Bates et al. discovered errors
associated with improper use of the multiple routes option. To enable flexibility of
administration choices, clinicians would prescribe multiple routes of the same medicine; this,
in turn, lead to instances of inappropriate doses being prescribed for certain routes or
inappropriate routes for certain drugs were encountered.(4)
Frequency and Timing Errors
Timing errors can lead to unnecessary delays of a medicine, potential overdose and
insufficient administration guidance to patients or clinical staff. Such errors were commonly
encountered in a large study which explored incident reports associated with the use of
computerised order entry system. Missing or incorrect directions/ patient instructions, wrong
time selected and discontinuation issues were among the top 25 most frequent examples of
errors.(13) As with other types of prescribing errors, miss-selection from down drop-down
menus may contribute to the likelihood of clinicians selecting the wrong time or frequency of
a medicine.(19, 22, 25) Further prescribing issues have been attributed to prescribers failing
to adjust default dosage times, which are used within a system; for example systems may
generate specific administration times which clinicians may not be aware of and inadvertently
prescribe medicines at an inappropriate time.(14) Westbrook et al. described how the default
dosing time set for an antibiotic drug was 8am, and an order made at 3pm, unless changed,
would then default to 8am (first dose) the following day resulting in an unnecessary delay in
the administration of a potentially critical medicine.(14) Similar errors were also identified by
Koppel et al, who described ‘late in the day orders’ (e.g., an order made after midnight
prescribed to start ‘tomorrow’), which the clinician is intending the patient to actually receive
later that morning, may actually be delayed for an additional 24 hours.(7) Inflexible ordering
systems also made it difficult to order certain drugs with variable dosing schedules or have
non-standard dosage times for parkinson’s disease treatments(10) or tapering dosage regimes
for prednisolone.(21) This may then lead to a high number of error prone free-text orders as
68
prescribers try to work around the restrictive ordering process,(21, 29) as well as
inconsistencies between the selected administration times and the desired frequency, detailed
in the free-text comment box.
Miscellaneous Errors
These errors are more operational and therefore have been classified separately.
Cancellations of medicines
There have been instances of medicines being automatically cancelled when a patient is
transferred from one clinical area to another.(7, 8) This depends on the interoperability,
specific hospital protocols and extent of ePrescribing in place within an organisation, but can
have clear implications for patient care, particularly if clinicians are not aware that medicines
will be discontinued once they leave a particular clinical area. Koppel et al. also described
failures in the review of antibiotic therapy and gaps in treatment, possibly due in part to the
loss of paper based reminding mechanisms (such as re-approval stickers to indicate an
antibiotic review date).(7)
System Access and Prescribing Rights
Redwood et al. highlighted incidents where clinicians lacking certain access rights and
prevented them from prescribing a medicine for a patient; this ultimately resulted in the
patient missing a dose due to system restrictions.(27) Shulman et al. also found several
instances of orders, which were missing a prescriber’s signature but still administered.
Although the potential of such an error to have an impact on patient safety is relatively low, it
does expose the practice of administering medicines in the absence of a legal signature. It
should however be noted that the computer system did keep a record of the prescriber despite
the signature not being physically present.(20)
Paper Persistence Errors
The use of paper and electronic systems in tandem has been attributed to near misses and the
increased potential for errors.(8, 10, 27) A significant concern is that information is not
documented consistently or that there are delays in entering paper based information into an
electronic form. This could then potentially lead to omission or duplicate doses being
administered if clinicians are not aware of medicines the patient has previously received.(10,
69
27) Cresswell et al. identified instances where clinicians would make notes on paper and then
enter the information onto the system in batches, and as a result information was not always
kept up to date.(18)
Overdependence
A recognised issue of ePrescribing systems is that prescribers may develop an over
dependence on the system when prescribing.(8, 23, 30) Campbell et al. for example highlights
how prolonged use of an ePrescribing system may result in difficulties carrying out processes
manually during inevitable system-downtime. Additionally, clinicians may become de-skilled
in certain areas, for example remembering standard doses or contraindications if this
information is always automatically provided to them.(8) Furthermore, there are risks if the
information within the system such as dosage recommendations or drug interaction warnings
are outdated or incorrect and inadvertently leads to clinicians following erroneous
recommendations,(8) particularly if there are gaps in clinical knowledge.
CONCLUSION
This short review outlines some of the main prescribing errors that occur, which are
particularly associated with the use of ePrescribing systems. It should be noted that many
issues for example, wrong dosage, prescribing for the wrong patient and importantly lack of
clinical knowledge may also occur with handwritten prescriptions, although the mechanisms
by which these errors occur may differ. Prescriber training surrounding proper use and the
vulnerabilities of systems is therefore required to safeguard against prescribing errors and
urge caution during use.
70
Appendix 1: Search Terms
Electronic Prescribing Errors
Electronic Prescribing
EP
Computerized Physician Order Entry
Computerized Provider Order Entry
CPOE
Medical Order Entry Systems
Electronic Health Records Decision Support
Systems
Clinical Decision Support
CDS
Decision Support
Decision Making
Medication Error
Drug Error
Unintended Consequence
71
Appendix 2: Search Strategy
1. Electronic Prescribing
2. EP
3. Computerized Physician Order Entry
4. Computerized Provider Order Entry
5. CPOE
6. Medical Order Entry Systems
7. Electronic Health Records Decision Support Systems
8. Clinical Decision Support
9. CDS
10. Decision Support
11. Decision Making
12. Medication Error
13. Drug Error
14. Unintended Consequence
15. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11
16. 12 or 13 or 14
17. 15 and 16
72
References
1. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al.
Effect of computerized physician order entry and a team intervention on prevention of
serious medication errors. JAMA : the journal of the American Medical Association.
1998;280(15):1311-6.
2. Nuckols TK, Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson
LJ, et al. The effectiveness of computerized order entry at reducing preventable
adverse drug events and medication errors in hospital settings: A systematic review
and meta-analysis. Systematic Reviews. 2014;3(1).
3. Albarrak AI, Al Rashidi EA, Fatani RK, Al Ageel SI, Mohammed R.
Assessment of legibility and completeness of handwritten and electronic
prescriptions. Saudi Pharmaceutical Journal. 2014;22(6):522-7.
4. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, et al. The
impact of computerized physician order entry on medication error prevention.
JAMIA. 1999;6(4):313-21.
5. Garg A, Adhikari N, McDonald H, Rosas-Arellano M, Devereaux P, Beyene
J, et al. Effects of computerized clinical decision support systems on practitioner
performance and patient outcomes. A systematic review. JAMA : the journal of the
American Medical Association. 2005;293:1223 - 38.
6. Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, et al. Return on
investment for a computerized physician order entry system. Journal of the American
Medical Informatics Association : JAMIA. 2006;13(3):261-6.
7. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al.
Role of computerized physician order entry systems in facilitating medication errors.
Journal of the American Medical Association. 2005;293(10):1197-203.
8. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of
unintended consequences related to computerized provider order entry. JAMIA.
2006;13(5):547-56.
9. Adelman JS, Kalkut GE, Schechter CB, Weiss JM, Berger MA, Reissman SH,
et al. Understanding and preventing wrong-patient electronic orders: A randomized
controlled trial. JAMIA. 2013;20(2):305-10.
10. Savage I, Cornford T, Klecun E, Barber N, Clifford S, Franklin BD.
Medication errors with electronic prescribing (eP): Two views of the same picture.
BMC health services research. 2010;10:135.
11. Walsh KE, Adams WG, Bauchner H, Vinci RJ, Chessare JB, Cooper MR, et
al. Medication errors related to computerized order entry for children. Pediatrics.
2006;118(5):1872-9.
12. Sartore ME, Ehman KM, Good CB. The significance of pharmacy
interventions: An updated review in the presence of electronic order entry. American
Journal of Pharmacy Benefits. 2014;6(2):e24-e30.
13. Schiff G, Amato MG, Eguale T, Boehne JJ, Wright A, Koppel R, et al.
Computerised physician order entry-related medication errors: Analysis of reported
errors and vulnerability testing of current systems. BMJ Quality and Safety.
2015;24(4):264-71.
14. Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The
safety of electronic prescribing: manifestations, mechanisms, and rates of system-
73
related errors associated with two commercial systems in hospitals. JAMIA.
2013;20(6):1159-67.
15. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The
extent and importance of unintended consequences related to computerized provider
order entry. JAMIA. 2007;14(4):415-23.
16. Bedouch P, Allenet B, Grass A, Labarere J, Brudieu E, Bosson JL, et al. Drug-
related problems in medical wards with a computerized physician order entry system.
Journal of clinical pharmacy and therapeutics. 2009;34(2):187-95.
17. Cho I, Park H, Choi YJ, Hwang MH, Bates DW. Understanding the nature of
medication errors in an ICU with a computerized physician order entry system. Plos
One. 2014;9(12).
18. Cresswell KM, Bates DW, Williams R, Morrison Z, Slee A, Coleman J, et al.
Evaluation of medium-term consequences of implementing commercial computerized
physician order entry and clinical decision support prescribing systems in two 'early
adopter' hospitals. Journal of the American Medical Informatics Association :
JAMIA. 2014;21(e2):e194-202.
19. Donyai P, O'Grady K, Jacklin A, Barber N, Franklin BD. The effects of
electronic prescribing on the quality of prescribing. British Journal of Clinical
Pharmacology. 2008;65(2):230-7.
20. Shulman R, Singer M, Goldstone J, Bellingan G. Medication errors: a
prospective cohort study of hand-written and computerised physician order entry in
the intensive care unit. Crit Care. 2005;9(5):R516-R21.
21. Singh H, Mani S, Espadas D, Petersen N, Franklin V, Petersen LA.
Prescription errors and outcomes related to inconsistent information transmitted
through computerized order entry: a prospective study. Archives of Internal Medicine.
2009;169(10):982-9.
22. Tully MP. Prescribing errors in hospital practice. British Journal of Clinical
Pharmacology. 2012;74(4):668-75.
23. Villamanan E, Larrubia Y, Ruano M, Velez M, Armada E, Herrero A, et al.
Potential medication errors associated with computer prescriber order entry.
International Journal of Clinical Pharmacy. 2013;35(4):577-83.
24. Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some
unintended consequences of clinical decision support systems. AMIA 2007;Annual
Symposium Proceedings/AMIA Symposium.:26-30.
25. Reckmann MH, Westbrook JI, Koh Y, Lo C, Day RO. Does computerized
provider order entry reduce prescribing errors for hospital inpatients? A systematic
review. JAMIA. 2009;16(5):613-23.
26. Palchuk MB, Fang EA, Cygielnik JM, Labreche M, Shubina M, Ramelson
HZ, et al. An unintended consequence of electronic prescriptions: Prevalence and
impact of internal discrepancies. JAMIA. 2010;17(4):472-6.
27. Redwood S, Rajakumar A, Hodson J, Coleman JJ. Does the implementation of
an electronic prescribing system create unintended medication errors? A study of the
sociotechnical context through the analysis of reported medication incidents. BMC
medical informatics and decision making. 2011;11:29.
28. Joy A, Davis J, Cardona J. Effect of computerized provider order entry on rate
of medication errors in a community hospital setting. Hospital Pharmacy.
2012;47(9):693-9.
29. Turchin A, Shubina M, Goldberg S. Unexpected effects of unintended
consequences: EMR prescription discrepancies and hemorrhage in patients on
74
warfarin. Amia 2011;Annual Symposium proceedings / AMIA Symposium. AMIA
Symposium. 2011:1412-7.
30. Ranji SR, Rennke S, Wachter RM. Computerised provider order entry
combined with clinical decision support systems to improve medication safety: A
narrative review. BMJ Quality and Safety. 2014;23(9):773-80.
75
Outcome 5: Describe any variations in error rates associated
with specific electronic systems
Aim: To describe any variation(s) in error rates associated with
specific ePrescribing systems
METHODS
We conducted a literature search in Medline (OVID) and Embase (OVID) databases
to identify papers that reported prescribing error rates from studies carried out in UK
hospitals.
This review considered all date ranges within the chosen databases. MeSH terms and
key word related to ‘electronic prescribing’, and ‘error rates’ (table 1) were used,
using ‘AND’ ‘OR’ Boolean operators. Titles and abstracts were initially reviewed,
followed by the full text (table 2). We were overly inclusive at each stage of
reviewing titles and abstracts; if it was not clear from the abstract where the study
took place or the system used, the full text was referred to. Reference lists of included
publications were also examined. Relevant systematic reviews were identified from
review of abstracts, their references were also checked.
Inclusion and Exclusion Criteria
Our inclusion criteria were as follows:
● Studies which discussed prescribing error rates; and
● Studies on ePrescribing systems that had been implemented; and
● Studies conducted in the U.K. hospital setting; and
● Studies that provided the brand name of their commercial ePrescribing system;
and
● Studies published in English.
76
Commentaries, viewpoint articles, editorials, letters and papers, were excluded. See
appendix 1 for a list of publications included and included following review of titles.
Figure 1: Search Strategy Diagram
Total articles idetified (N=2424)
(Embase 1419, Medline 1005,
Total articles included (N=14)
Excluded based on title (N=2055)
Excluded based on abstract (N=132)
Removal of duplicates
(N=171)
Excluded based on full text (N=58)
Studies identified from review of systematc
reviews (N=6)
77
RESULTS
A total of 2,424 papers were retrieved from the database search, and a further six from
review of the references of systematic reviews published in this area. There were 11
full papers and three abstracts included in this review.
Five studies were conducted on a surgical ward, one on a medical ward and two
studies across a mixture of clinical areas. Two further studies took place in a critical
or intensive care setting; one study in oncology and three studies explored errors rates
in the paediatric population.
The results of each study, which specifically discuss prescribing error rates, will be
described and compared according to the clinical area in which the study took place.
For details of other findings reported in the included publications that were beyond
the remit of this review, for example administration errors, changes in time taken to
complete tasks and a summary of the study, please see appendix 2.
Surgery
Five studies were conducted in a surgical setting. Three studies describe error rates on
a 28-bed surgical ward in a London teaching hospital.(1-3) The ServeRx V.1:13
MDG Medical, Israel system was in use. A fourth study was conducted on a 36-bed
orthopaedic ward in a 350-bed hospital, using the Pharmakon UK system.(4) The
final study by Mitchell et al. was conducted across surgical wards, theatres and
recovery, in a Bristol teaching hospital, using the Clinical Manager system.(5)
All studies demonstrated a decrease in prescribing errors following implementation of
an ePrescribing system. The ServeRx system was associated with a significant
reduction in the prescribing error rate from between 3.8% to 2.0% (absolute reduction
of 1.8%) in the papers by Dean-Franklin et al. and Donyai et al. Both studies used a
prospective, before and after study design, and report on the same data set.(2, 3) A
second study by Dean-Franklin’s, which used a variety of methods to identify
prescribing errors pre and post implementation, demonstrated a combined reduction
from 10.7% to 7.9% when all methods of data collection were considered.(1) There
was no change to the mean severity of errors following implementation of the
78
ePrescribing system in the papers by Dean-Franklin and Donyai.(2, 3) Donyai et al.
reported that the rate of pharmacist clinical interventions also decreased following
implementation of the ServeRx system from 3% of all orders to 1.9% post-
ePrescribing (RR reduction of 1.1%).(3) Mitchell et al.’s study reported an average
prescribing error rate of 2.9% (143 out of 4927 prescriptions). Only minimal clinical
decision support was in place such as pick lists and order sets, consequently some
prescribing errors for example wrong formulation and wrong dose were recorded
post-implementation.(5) The study by Fowlie et al. is only available in abstract form
and provides minimal information about the methods used. They observed a
significant change in prescribing errors and improved conformity to pre-defined
prescribing standards following implementation. However due to lack of exact figures
and differences in study data collection methods, it would not be appropriate to
directly compare these results.(4) All studies are limited by the lack of a control
group, which would have possibly identified other potential causes of changes in error
rates and questionable generalizability as they were only conducted on one ward in
one hospital.
Medical
One study took place in a medical setting. The MediChain system was implemented
on a 33-bed acute medical ward, with a renal sub-speciality and was associated with
improved prescribing and administration in a prospective controlled before and after
study.(6) Pre- implementation error rates between the control and study ward were
similar. The authors revealed that the pharmacist identified errors in 12% of paper
prescriptions, and that 1% of all prescriptions were illegible at the time of
administration. Post implementation the authors found that all prescriptions passed the
clinical screening, 94% with one or no modifications required. All prescriptions were
legible and included a route of administration at the point of administration. However,
end-users found prescribing medicines with variable dose regimens, intravenous
medicines and fluids difficult.(6)
Mixed Clinical Area
79
Two studies took place in more than one clinical area, however both were only
available in abstract format.(7, 8) A study by Riaz et al. compared the type and
severity of prescribing errors between handwritten and hospital discharge
prescriptions by pharmacists from surgical and medical wards which used the JAC
ePrescribing system. There was no difference in reported errors rate pre and post
implementation (8.2% of all prescribed drugs pre and post-ePrescribing). An increase
in the potential severity of errors made using the electronic prescribing system was
also reported (16.1% of errors with the electronic system were classed as serious,
compared to 6.5% of handwritten prescriptions). The study also found changes in the
types of errors that occurred pre and post implementation. For example the rates of
omission of drug therapy and selection of the incorrect formulation increased from
29.4% to 42% and from 6.5% to 9.7% respectively, whereas the rates of missing or
incorrect drug doses or administration times decreased from 13.5% to 1.1% and 9.4%
to 0% respectively.(7) The second study by Marriott et al. compared pharmacist
clinical interventions between two hospitals, one that used a Medical Information
Technology Inc. electronic prescribing system with a hospital operating a traditional
paper-based prescribing system. The authors found that a larger number of clinical
interventions occurred at the ePrescribing site (0.20 interventions/ finished consultant
episode at the electronic site, compared to 0.05 interventions/ finished consultant
episode at the paper-based site). This study also reported substantial differences in the
types of interventions reported at the two sites. The electronic site was associated with
more interventions relating to pharmaceutical care, whereas the non-ePrescribing site
experienced a greater frequency of interventions related to drug choice and
prescribing appropriate therapy.(8) As both studies are only available in abstract
format, it is not possible to ascertain the exact definition of a prescribing error or a
clinical intervention used in either study, or whether the terms used here are
interchangeable. Therefore, it would be inappropriate to directly compare error rates.
Critical Care
Two studies took place in a critical care setting, the first by Evans et al. on a critical
care unit (CCU), which used the Hewlett Packard CareVue system.(9) Shulman et al.
conducted a study on an intensive care unit (ICU),(10) which used the QS 5.6 Clinical
80
Information System. Both studies used a before-after study design with identification
of errors by a clinical pharmacist. Evans et al. reported mixed results; the
introduction of the ePrescribing system was associated with more complete and
legible orders, although the error rate actually increased for IV fluids and infusions
(reduction in percentage of correct entries by 16% and 15.5% respectively).
Additionally, the use of the ePrescribing system appeared to contribute to the
occurrence of unnecessary drug orders due to an increased failure to discontinue both
drugs that were no longer needed (9.1% of errors with handwritten orders and 57% of
errors with the computerised orders) and duplicate drug orders (no cases pre-EP and
11 cases post-EP).(9) Shulman et al., meanwhile reported a significant reduction in
medication errors following implementation of an ePrescribing system, from 6.7% of
all medication orders to 4.8%. Patient outcome scores improved following
implementation of the system, if intercepted and non-intercepted errors were
combined (intercepted errors were scored on the potential outcome, as if the patient
had received the medication). However, this study found that the rates of minor errors
increased, which required additional patient monitoring, and the only instances of
major errors throughout the study period occurred with the ePrescribing system.(10)
The difference in rates between the two studies may reflect the data collection
methods and definitions used; Evans et al. recorded errors using a pre-determined
criteria and Shulman et al. identified errors based on an established definition.(11)
However whilst both studies provide some support to the use of ePrescribing systems,
they also warn of potential issues associated with their use. As with other studies,
both were limited by the lack of control group and potential for generalizability to
other clinical areas.
Oncology
One study by Small et al. evaluated the difference in the type and rate of prescribing
errors for outpatient chemotherapy prescriptions, when using the VARIS MedOnc
prescribing system or a manual Excel spread sheet prescription (a computer generated
template prescription, which requires manual input of data such as dosage
calculations and adjustments).(12) A clinical pharmacist recorded the rate, type of
error and severity of errors made during their normal clinical practice. The authors
81
reported a relative risk reduction of 42% when the ePrescribing system was used.
Handwritten prescriptions were associated with an error rate of 20.4%, compared to
11.4% for electronic prescriptions. Similar to other studies(7, 8) a difference in the
type of errors that occurred following transition to an electronic system was also
observed.(12) Electronic prescribing was associated with fewer instances of wrong
dosage calculation (6.8% to 1.9% post-implementation), and incomplete prescriptions
(21.1% to 3.2% post-implementation) but increased rates of errors associated with
inputting information such as a patient’s height or weight (3.3% to 8.9% post
implementation). The severity of recorded errors also changed. The ePrescribing
system was associated with fewer minor errors (16.5% of computerised errors and
36.6% of spread sheet errors), but more serious errors (41.8% of computerised errors
and 25.2% of spread sheet errors.(12) An increase in the most serious errors has also
been reported in other studies.(7, 10) When comparing the error results from this
study with others, it is important to consider the very specific setting in which this
study was conducted and also that the comparator was a manual excel spread sheet
which is likely to be more highly structured than a traditional paper drug chart.
Paediatrics
Three studies were conducted in the paediatric population. Two separate studies by
Jani et al. took place in a tertiary care hospital, which was using the JAC ePrescribing
system. One of which compared the incidence and severity of dosing errors for renal
inpatient and outpatients and also patients discharged from the urology and renal
wards.(13) The second study aimed to determine the rate and types of prescribing
errors in a paediatric renal outpatient clinic.(14) In both studies a pharmacist recorded
errors as part of their normal clinical practice. A third study by Warrick et al. was
performed on a paediatric intensive care unit (PICU), which was auditing the effect
the Intellivue Clinical Information Portfolio system on prescribing errors and dose
omissions.(15) A pharmacy student or researcher was responsible for data collection.
All three studies carried out a pre-post study design.
82
The results from these three studies suggest that prescribing errors may be reduced
following implementation of an ePrescribing system. Jani et al’s study, which
concentrated on only dosing errors, revealed an overall significant reduction in the
error rate from 2.2% of all prescriptions to 1.2%.(13) A change in the types of errors
was also seen in this study; handwriting and incorrect unit errors were eliminated,
while new errors such as mis-selection from drop down menus were reported.(13)
Potentially serious errors were eliminated in discharge and outpatient prescriptions,
and dosage errors with potentially minor or moderate outcomes were decreased
following implementation from 0.89% to 0.44% of all errors. (13) The second study
by Jani et al., which included all prescribing errors, reported a reduction in the total
prescribing error rate from 77.4% to 4.8% and the number of error-free patient visits
increased from 21% to 90% following implementation.(14) Warrick et al. assessed the
change in prescribing error rate pre-implementation, 1-week post implementation and
again at 6-months post-implementation. A trend towards a reduction in errors was
observed between the first and third period of data collection from 8.8% to 4.6% of
prescriptions. There were also differences in the types of errors reported pre and post
implementation, for example illegibility errors and orders with insufficient
information were eliminated following implementation, while incomplete
prescriptions and errors made during the prescribing decision making process were
increased.(15)
These three studies all lacked the use of a control group and generalizability is limited
to the area in which the study was conducted. Additionally, the study by Warrick et
al. only collected data over three 96-hour periods in each phase, and therefore is
hindered by the small sample size obtained. (15)
SUMMARY
These 14 studies report a prescribing error rate of between 2% (2) and 11.4% (12) in
hospitals using a ePrescribing system. An even lower error rate was reported in a
study by Jani et al of 1.2%, however this study only focused on dosing errors.(13)
Due to the differences in the data collection methods used, the clinical setting and the
levels of system customisation, it was not possible to directly compare the prescribing
error rate associated with specific systems. JAC was the only ePrescribing system in
83
the publications identified, which was evaluated in more than one study; Jani et al
conducted two studies using this system and reported a reduction in prescribing errors
overall and dosing errors specifically in a paediatric population.(13, 14) However,
Riaz et al. also evaluated the implementation of the JAC system in an adult population
but did not find any difference in prescribing error rates post-implementation.(7)
Dean-Franklin et al. found different error rates for the same data set, depending on the
methods used.(1) It is also important to consider the variety of modifications and level
of active clinical decision support functionalities that a single commercial system has,
as this will also have an impact on the error rate and type.(16) The majority of studies
here also referred to error rates following implementation for example, immediately
after the implementation (6) to 17 months post implementation,(14) only Small et al.
reported on results from a system which had been in place for over two years(12) and
therefore the long-term effects of electronic prescribing in UK hospitals are largely
unknown. Moving forward there is a need for further research, which reports on the
error rates of a range of currently available commercial prescribing systems. In
particular studies should aim to include a range of clinical areas and provide
information about the errors rates associated with established systems.
84
Appendix 1: Inclusion and Exclusion of publications from review of abstract and full text.
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
1. Mitchell, D., et al. (2004). "Evaluation and audit of a pilot of electronic prescribing
and drug administration." Journal on Information Technology in Healthcare 2(1): 19-
29.
Yes Yes
2. Franklin, B. D., et al. (2008). "The impact of an electronic prescribing and
administration system on the safety and quality of medication administration."
International Journal of Pharmacy Practice 16(6): 375-379.
No Lack of relevance
for prescribing
errors
3. Franklin, B. D., et al. (2007). "The impact of a closed-loop electronic prescribing
and administration system on prescribing errors, administration errors and staff time:
a before-and-after study." Quality & safety in health care 16(4): 279-284.
Yes Yes
4. Jani, Y. H., et al. (2010). "Paediatric dosing errors before and after electronic
prescribing." Quality & safety in health care 19(4): 337-340.
Yes Yes
5. Jani, Y. H., et al. (2008). "Electronic prescribing reduced prescribing errors in a
pediatric renal outpatient clinic." Journal of Pediatrics 152(2): 214-218.
Yes Yes
85
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
6. Riaz, I. and S. D. Williams (2010). "Impact of a new electronic discharge system on
the prevalence of prescribing errors." International Journal of Pharmacy Practice 18:
22.
Yes Yes
7. Small, M. D. C., et al. (2008). "The impact of computerized prescribing on error rate
in a department of oncology/hematology." Journal of Oncology Pharmacy Practice
14(4): 181-187.
Yes Yes
8. Warrick, C., et al. (2011). "A clinical information system reduces medication errors
in paediatric intensive care." Intensive Care Medicine 37(4): 691-694.
Yes Yes
9. Went, K., et al. (2010). "Reducing prescribing errors: can a well-designed electronic
system help?" Journal of Evaluation in Clinical Practice 16(3): 556-559.
Yes No Internally developed
ICU electronic
prescribing tool (UK)
10. Ammenwerth, E., et al. (2008). "The effect of electronic prescribing on medication
errors and adverse drug events: a systematic review." Journal of the American
Medical Informatics Association 15(5): 585-600.
Yes To review for
additional papers
only.
No
86
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
11. Eslami, S., et al. (2008). "The impact of computerized physician medication order
entry in hospitalized patients--a systematic review." International Journal of Medical
Informatics 77(6): 365-376.
Yes- To review for
additional papers
only
No
12. Lainer, M., et al. (2013). "Information technology interventions to improve
medication safety in primary care: a systematic review." International Journal for
Quality in Health Care 25(5): 590-598.
Yes To review for
additional papers
only
No
13. Eslami, S., et al. (2007). "Evaluation of outpatient computerized physician
medication order entry systems: a systematic review." Journal of the American
Medical Informatics Association 14(4): 400-406.
Yes To review for
additional papers
only
No
14. Radley, D. C., et al. (2013). "Reduction in medication errors in hospitals due to
adoption of computerized provider order entry systems." Journal of the American
Medical Informatics Association 20(3): 470-476.
Yes To review for
additional papers
only
No
15. Shamliyan, T. A., et al. (2008). "Just what the doctor ordered. Review of the
evidence of the impact of computerized physician order entry system on medication
errors." Health Services Research 43(1 Pt 1): 32-53.
Yes To review for
additional papers
only
No
87
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
16. Kaushal, R., et al. (2003). "Effects of computerized physician order entry and
clinical decision support systems on medication safety: a systematic review."
Archives of Internal Medicine 163(12): 1409-1416.
Yes To review for
additional papers
only
No
17. Reckmann, M. H., et al. (2009). "Does computerized provider order entry reduce
prescribing errors for hospital inpatients? A systematic review." Journal of the
American Medical Informatics Association 16(5): 613-623.
Yes To review for
additional papers
only
No
18. Georgio STUDY OBJECTIVE: We undertake a systematic review of the
quantitative literature related to the effect of computerized provider order entry
systems in the emergency department (ED).
Yes To review for
additional papers
only
No
19. Nuckols, T. K., et al. (2014). "The effectiveness of computerized order entry at
reducing preventable adverse drug events and medication errors in hospital settings:
A systematic review and meta-analysis." Systematic Reviews 3(1).
Yes To review for
additional papers
only
No
20. Franklin, B. D., et al. (2009). "Methodological variability in detecting prescribing
errors and consequences for the evaluation of interventions." Pharmacoepidemiology
& Drug Safety 18(11): 992-999.
Yes Yes
88
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
21. Al-Dorzi, H. M., et al. (2010). "Impact of computerized physician order entry
(CPOE) system on ICU mortality: A before-after study." American Journal of
Respiratory and Critical Care Medicine 181 (1 MeetingAbstracts).
Yes No Non-UK
22. Ali, J., et al. (2010). "The impact of computerised physician order entry on
prescribing practices in a cardiothoracic intensive care unit." Anaesthesia 65(2): 119-
123.
Yes No Non-UK
23. Armada, E. R., et al. (2014). "Computerized physician order entry in the cardiac
intensive care unit: effects on prescription errors and workflow conditions." Journal
of Critical Care 29(2): 188-193.
Yes No Non-UK
24. Bates, D. W., et al. (1998). "Effect of computerized physician order entry and a team
intervention on prevention of serious medication errors." Jama 280(15): 1311-1316.
Yes No Non-UK
25. Bates, D. W., et al. (1999). "The impact of computerized physician order entry on
medication error prevention." Journal of the American Medical Informatics
Association 6(4): 313-321.
Yes No Non-UK
89
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
26. Bedouch, P., et al. (2009). "Drug-related problems in medical wards with a
computerized physician order entry system." J Clin Pharm Ther 34(2): 187-195.
Yes No Non-UK
27. Bedouch, P., et al. (2012). "Computerized physician order entry system combined
with on-ward pharmacist: analysis of pharmacists' interventions." Journal of
Evaluation in Clinical Practice 18(4): 911-918.
Yes No Non-UK
28. Bonnabry, P., et al. (2008). "A risk analysis method to evaluate the impact of a
computerized provider order entry system on patient safety." Journal of the
American Medical Informatics Association 15(4): 453-460.
Yes No Non-UK
29. Bradley, V. M., et al. (2006). "Evaluation of reported medication errors before and
after implementation of computerized practitioner order entry." Journal of
Healthcare Information Management 20(4): 46-53.
Yes No Non-UK
30. Caruba, T., et al. (2010). "Chronology of prescribing error during the hospital stay
and prediction of pharmacist's alerts overriding: a prospective analysis." BMC health
services research 10: 13.
Yes No Non-UK
31. Cho, I., et al. (2014). "Understanding the nature of medication errors in an ICU with
a computerized physician order entry system." Plos One 9(12).
Yes No Non-UK
90
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
32. Choo, J., et al. (2014). "Effectiveness of an electronic inpatient medication record in
reducing medication errors in Singapore." Nursing & Health Sciences 16(2): 245-
254.
Yes No Non-UK
33. Collins, C. M. and K. A. Elsaid (2011). "Using an enhanced oral chemotherapy
computerized provider order entry system to reduce prescribing errors and improve
safety." International Journal for Quality in Health Care 23(1): 36-43.
Yes No Non-UK
34. Colpaert, K., et al. (2006). "Impact of computerized physician order entry on
medication prescription errors in the intensive care unit: a controlled cross-sectional
trial." Critical care (London, England) 10(1): R21.
Yes No Non-UK
35. Condren, M., et al. (2014). "Influence of a systems-based approach to prescribing
errors in a pediatric resident clinic." Academic Pediatrics 14(5): 485-490.
Yes No Non-UK
36. Cordero, L., et al. (2004). "Impact of computerized physician order entry on clinical
practice in a newborn intensive care unit." Journal of Perinatology 24(2): 88-93.
Yes No Non-UK
91
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
37. Cunningham, T. R., et al. (2008). "Impact of electronic prescribing in a hospital
setting: a process-focused evaluation." International Journal of Medical Informatics
77(8): 546-554.
Yes No Non-Uk
38. Dainty, K. N., et al. (2012). "Electronic prescribing in an ambulatory care setting: a
cluster randomized trial." Journal of Evaluation in Clinical Practice 18(4): 761-767.
Yes No Non-UK
39. Dequito, A. B., et al. (2011). "Preventable and non-preventable adverse drug events
in hospitalized patients: a prospective chart review in the Netherlands." Drug Safety
34(11): 1089-1100.
Yes No Non-UK
40. Furuya, H., et al. (2013). "Relationship between the use of an electronic commercial
prescribing system and medical and medication errors in a teaching hospital." Tokai
Journal of Experimental & Clinical Medicine 38(1): 33-36.
Yes No Non-UK
41. Han, Y. Y., et al. (2005). "Unexpected increased mortality after implementation of a
commercially sold computerized physician order entry system." Pediatrics 116(6):
1506-1512.
Yes No Non-UK
92
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
42. Jozefczyk, K. G., et al. (2013). "." Journal of Pharmacy Practice 26(4): 434-437. Yes No Non-UK
43. Kadmon, G., et al. (2009). "Computerized order entry with limited decision support
to prevent prescription errors in a PICU." Pediatrics 124(3): 935-940.
Yes No Non-UK
44. Kazemi, A., et al. (2011). "The effect of Computerized Physician Order Entry and
decision support system on medication errors in the neonatal ward: experiences from
an Iranian teaching hospital." Journal of Medical Systems 35(1): 25-37.
Yes No Non-UK
45. Kilbridge, P. M., et al. (2006). "Automated surveillance for adverse drug events at a
community hospital and an academic medical center." Journal of the American
Medical Informatics Association 13(4): 372-377.
yes No Non-UK
46. Kim, G. R., et al. (2006). "Error reduction in pediatric chemotherapy: computerized
order entry and failure modes and effects analysis." Archives of Pediatrics &
Adolescent Medicine 160(5): 495-498.
Yes No Non-UK
93
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
47. Koppel, R., et al. (2008). "Identifying and quantifying medication errors: evaluation
of rapidly discontinued medication orders submitted to a computerized physician
order entry system." Journal of the American Medical Informatics Association 15(4):
461-465.
yes No Non-UK
48. Kuperman, G. J., et al. (2001). "Patient safety and computerized medication ordering
at Brigham and Women's Hospital." Joint Commission Journal on Quality
Improvement 27(10): 509-521.
Yes No Non-UK
49. Meisenberg, B. R., et al. (2014). "Reduction in chemotherapy order errors with
computerized physician order entry." Journal of oncology practice/American Society
of Clinical Oncology 10(1): e5-9.
Yes No Non-UK
50. Menendez, M. D., et al. (2012). "Impact of computerized physician order entry on
medication errors." Revista de Calidad Asistencial 27(6): 334-340.
Yes No Non-UK
51. Nebeker, J. R., et al. (2005). "High rates of adverse drug events in a highly
computerized hospital." Archives of Internal Medicine 165(10): 1111-1116.
yes No Non-UK
94
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
52. Raimbault-Chupin, M., et al. (2013). "Drug related problems and pharmacist
interventions in a geriatric unit employing electronic prescribing." International
Journal of Clinical Pharmacy 35(5): 847-853.
Yes No Non-UK
53. Roberts, D. L., et al. (2013). "Impact of computerized provider order entry on
hospital medication errors." Journal of Clinical Outcomes Management 20(3): 109-
115.
Yes No Non-UK
54. Shulman, R., et al. (2005). "Medication errors: a prospective cohort study of hand-
written and computerised physician order entry in the intensive care unit." Critical
care (London, England) 9(5): R516-521.
Yes Yes
55. Spencer, D. C., et al. (2005). "Effect of a computerized prescriber-order-entry
system on reported medication errors." American Journal of Health-System
Pharmacy 62(4): 416-419.
yes No Non-UK
56. van Doormaal, J. E., et al. (2009). "The influence that electronic prescribing has on
medication errors and preventable adverse drug events: an interrupted time-series
study." Journal of the American Medical Informatics Association 16(6): 816-825
Yes No Non-UK
57. Voeffray, M., et al. (2006). "Effect of computerisation on the quality and safety of
chemotherapy prescription." Quality & safety in health care 15(6): 418-421.
yes No Non-UK and Internally
developed system
58. Walsh, K. E., et al. (2006). "Medication errors related to computerized order entry Yes
95
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
for children." Pediatrics 118(5): 1872-1879.
59. Walsh, K. E., et al. (2008). "Effect of computer order entry on prevention of serious
medication errors in hospitalized children." Pediatrics 121(3): e421-427.
Yes No Non-UK
60. Westbrook, J. I., et al. (2013). "The safety of electronic prescribing: manifestations,
mechanisms, and rates of system-related errors associated with two commercial
systems in hospitals." Journal of the American Medical Informatics Association
20(6): 1159-1167.
Yes No Non-UK
61. Westbrook, J. I., et al. (2012). "Effects of two commercial electronic prescribing
systems on prescribing error rates in hospital in-patients: a before and after study."
PLoS Medicine / Public Library of Science 9(1): e1001164.
Yes No Non-UK
62. Wolfstadt, J. I., et al. (2008). "The effect of computerized physician order entry with
clinical decision support on the rates of adverse drug events: a systematic review."
Journal of General Internal Medicine 23(4): 451-458.
Yes To review for
additional papers
only
No
63. Zaal, R. J., et al. (2013). "Identification of drug-related problems by a clinical
pharmacist in addition to computerized alerts." International Journal of Clinical
Pharmacy 35(5): 753-762.
Yes No Non-UK
64. Maslove, D. M., et al. (2011). "Computerized physician order entry in the critical
care environment: A review of current literature." Journal of Intensive Care
No Lack of error rate
focus
96
Number Reference Include
(Abstract)
Reason Include
(Full
Text)
Reason
Medicine 26(3): 165-171.
65. Franklin, B. D., et al. (2014). "The effect of the electronic transmission of
prescriptions on dispensing errors and prescription enhancements made in English
community pharmacies: A naturalistic stepped wedge study." BMJ Quality and
Safety 23(8): 629-638.
No Community
setting
66. Franklin, B. D., et al. (2013). "Community pharmacists' interventions with electronic
prescriptions in England: an exploratory study." International Journal of Clinical
Pharmacy 35(6): 1030-1035.
No Community
Setting
97
Study System Clinical
Decision
Support
Locati
on
Aim Methods Outcome Key Points
Donya
i et al,
2007
ServeR
x:
MDG
Medica
l,
Israel,
version
1:13
Drug
dictionary
with
suggested
default doses
28 Bed
Surgica
l ward;
London
teachin
g
hospital
Investigate
effects of
electronic
prescribing
(EP) on
prescribing
quality, and
the number
and types of
pharmacists’
clinical
interventions
.
Before- after study.
Two 4 week periods;
First Phase: 3 months pre-EP
Second Phase: 6 months
post- EP
One pharmacist recorded
any prescribing error,
intervention or both for
inpatient orders and
discharge prescriptions (still
paper written).
A senior clinical pharmacist
checked all medication
orders weekly to identify
overlooked errors.
Five judges assessed the
Retrieved 113 (88%) of 129 patient’s health
care records pre-EP and 126 (86%) of records
post-EP.
Clinical Interventions
Clinical interventions decreased from 73 (3% of
all orders) to 45 (1.9% post-EP). Relative risk
reduction of 1.1%.
Most common intervention pre and post-EP was
elated to need for drug therapy, selection of
dose and specification of instructions for supply
or administration.
Prescribing Interventions
Prescribing errors decreased from 94 (3.8%)
pre-EP to 48 (2.0%) post EP.
Most common errors include the need for drug
therapy and selection of dose.
Pharmacist’s
interventions decreased
significantly post-EP.
Prescribing errors reduced
post-EP
The mean severity of
errors did not differ pre
and post-EP
Only 52% of
interventions related to
prescribing errors pre EP
and 60% post EP.
Similarly only 40 and
44% of prescribing errors
resulted in an
intervention. The lack of
overlap has implications
for studies were the two
are considered
synonymous.
Appendix 2: Summary table of included papers
98
potential clinical
significance of each
prescribing error (0, no harm
to 10, death)
Statistical analysis was
performed.
There were a total of 32 errors post-EP related
to the medication writing process, 10 of which
were thought caused by EP.
Severity
No change in minor errors: 18 errors pre-EP
(19%) and 9 errors (19%) post-EP.
Moderate errors: 73 (78%) pre-EP and 33 (69%)
post-EP
Severe errors: 3 errors (3%) pre-EP and 6 (12%)
post-EP, 2 of which were considered to be due
to default functions.
No difference in mean severity scores pre and
post EP was identified.
Limitations
Variability between
Pharmacists’
interpretation of errors-
but principal investigator
did have overall view
Generalizability, only one
ward and one system
were investigated.
Potential difficulty
identifying some errors
with EP.
Lack of controlled study
design.
Potential conflict of
interest, the study was
part funded r by MDG
Medical, however they
had no input into study
design, data collection,
analysis or interpretation.
Evans Hewlett Automated John Evaluating Pre-post study In total 128 handwritten and 110 computer Computerised
99
et al.
1998
Packar
d
CareVu
e
(Hewle
tt
Packar
d Ltd,
Andove
r,
USA).
W
charting of
physiologica
l and
laboratory
data.
Drug dosage
and route
recommenda
tions
Dose
checking
Automated
discontinuati
on of drugs
as per stop
date.
Radclif
fe
Hospita
l
Oxford,
Critical
Care
Ward
the effect of
an electronic
prescribing
system on
prescribing
errors and
the time
taken to
prescribe
and record
administratio
n.
1st phase: 3 week period pre-
EP
2nd
phase: 3 weeks period, 1
month post-EP
implementation
Each individual drug entry
for both the handwritten and
computer assisted
prescriptions was evaluated
by the ICU pharmacist
according to pre-determined
criteria.
Time taken to prescribe a
single drug was measured,
and the time it took a nurse
to record administration was
measured pre and post-EP.
prescription patient charts were monitored. This
equated to 1184 handwritten and 1225
individual drug entries respectively.
Legible, complete and authorised prescriptions
Signed and dated: 95% of handwritten
prescriptions compared to 100% of
computerised prescriptions fulfilled criteria.
Full patient identification details were
present in 47% of handwritten and 100% of
computerised prescriptions.
IV Fluids and feeds (194 handwritten and
255 computerised entries):
o Percentage of correct individual
entries decreased from 64% to
48% pre and post-EP respectively.
o Entries with missing infusion rate
increased from 18% to 28%.
o Number of drugs remaining on
chart despite having being moved
from treatment regimen more than
24 hours previously also increased
from 1% to 17%
prescriptions were less
accurate for IV fluids and
infusions.
Drugs were less likely to
be discontinued when no
longer needed.
Issue of duplicate drug
entries with EP. Possibly
due to system
configuration, which
separated STAT and PRN
drugs. Not possible to
display and see all drug
groups (prn/ stat/ regular)
simultaneously.
Computerised orders were
more complete, legible,
and fully identified the
patient. Mandatory fields
meant 100% of
prescriptions with CAP
had drug, dose, route and
frequency.
100
IV Infusions (284 handwritten and 247
computerised entries):
o Percentage of correct entries
decreased from 47.5% to 32%.
o Percentage of prescriptions with
incorrect rate increased from 1.4%
to 16%.
Intermittent drugs (706 handwritten and
723 computer assisted entries):
o 90% of prescriptions for both
handwritten and computer
prescriptions complied with
specified criteria.
Increased failure to discontinue no longer
needed drugs with computerised
prescriptions (9.1% of errors with
handwritten to 57% of errors with CAP).
Although represented less than 6% of total
entries.
Eleven cases of duplicate prescriptions
were found in computerised prescriptions,
none with handwritten.
Improved audit-ability of
who has prescribed and
start and stop dates visible
with computerised
prescriptions
Took almost twice as long
to complete computerised
prescriptions.
Administration also took
longer. (Secondary time
savings however may not
be accounted for in this
study)
101
Time taken to prescribe:
It took 20 seconds to prescribe a single
complete handwritten drug entry, and 55
seconds to prescribe the same drug using
computerised methods.
Recording administration took 2 seconds by
hand and 21 seconds with computerised
methods.
Shulm
an et
al,
2005
QS 5.6
Clinical
Inform
ation
System
(GE
healthc
are,
Anapol
is, MD,
USA)
Access to
online
material but
CDS does
not exist.
Univers
ity
College
Hospita
ls
London
, ICU
Compare
handwritten
prescribing
with
computerise
d prescribing
for:
1. Rates
and
types of
medicati
on
errors.
2. Potentia
Before-After study
Identification of medication
errors by an ICU clinical
pharmacist before and for 4
periods after
implementation.
Data collected over a 70
week time period:
Pre-EP: 9 days.
Post-EP: 3 periods,
of 5 days l and 1
period of 2 days.
Total numbers: 134 drug chats with 1036
prescriptions reviewed in the handwritten group
and 253 charts with 2429 prescriptions in the EP
group.
Medication errors pre-EP occurred in 6.7% and
4.8% of prescriptions post-EP CPOE.
Proportion of medication errors varied over time
since it’s initial introduction- declining
proportion over time.
EP was associated with a high number of dosing
errors, omission of required drug and prescriber
Proportion of medication
errors reduced following
implementation of EP.
Evidence of a learning
curve, as proportion of
errors reduced over time.
Benefit of EP on patient
outcome scores when
intercepted combined
with non-intercepted
errors.
Changes in types of errors
seen pre and post EP.
102
l
outcome
of
intercept
ed and
non-
intercept
ed
errors
Medication errors were
assessed by type and patient
outcome.
Patient outcomes for each
error were assigned by the
pharmacist and ICU clinical
director, according to an
adapted scale (Mild,
moderate, severe).
Statistical analysis was
performed.
signature.
Handwritten prescriptions missed dose, units
and frequency.
Most errors pre and post were minor. There
were many cases of minor errors with EP that
increased monitoring, but did not cause patient
harm.
No significant difference in the non-intercepted
error rate was found between groups, however if
intercepted errors are included, a difference in
favour of the EP system was demonstrated, due
to increased error rate with handwritten
prescriptions.
The only 3 major errors were with found with
EP.
EP was associated with more minor errors that
did not cause harm, but did increase monitoring.
Limitations
Only one ward and one
system were evaluated.
Pharmacist attended the
ward round, therefore
potentially lower rate of
errors, as point of care
advice was given.
Patient outcome decided
by clinical director and
pharmacist who were not
blinded, therefore
potential for bias.
103
Warri
ck et
al.
2011
Intelliv
ue
Clinical
Inform
artion
Portfoli
o
(ICIP),
Philips,
UK.
Drug
dictionary,
standard
weight or
surface area
based orders
for most
drugs;
dispensing
instructions
and user
alerts for
nursing staff
when drugs
are due.
St
Mary’s
PICU
at
Imperia
l
College
Healthc
are
NHS
Trust
London
.
Evaluate the
effect of the
system on
prescribing
errors and
dose
omissions.
Prospective audit of
prescribing errors and dose
omissions over a 96-hour
period in three separate
phases.
1st Phase: 2 weeks prior to
implementation of ICIP
2nd
Phase: 1 week after
implementation of ICIP
3rd
Phase: 6 months after
implementation of ICIP.
Prescriptions assessed for
prescribing errors and dose
errors, apart from ‘once only
doses’.
Dose omissions were
recorded for regular items
only.
Discontinued prescriptions
and medications prescribed
Prescribing errors were evaluated in a total of
54 charts. This included 624 prescriptions,
which were assessed for prescribing errors and
1022 regularly scheduled doses, which were
assessed for omissions.
There was no significant change in the
incidence of prescribing errors across the three
periods. However a trend was observed towards
a reduction in the third period. (8.8% to 8.1% to
4.6%)
Differences in errors types were seen pre and
post implementation. Greater incidence of
prescriptions with insufficient information and
illegibility issues with paper prescribing
compared to more errors associated with the
prescribing decision and incomplete
prescriptions using the electronic system.
New errors were identified following
implementation, including incorrect selection of
infusion rates or doses and failure to prescribe a
Introduction of the
electronic prescribing
system reduced omission
errors and possibly
prescribing errors.
Limitations: pre-post
design with no control,
only a small sample size
was used and a short time
period was examined.
104
on the day of audit were
excluded.
Data were collected from
charts of all patients on the
PICU at the time of the
audit, by a pharmacy student
or researcher at 11am on
four randomly selected
weekdays, during a 2 week
period for each phase
Prescribing errors and
missed doses were studied
on different days.
Charts were reviewed for
dose omissions over the
previous 24-hour period up
to an including doses due at
10am on the day of data
collection.
base solution.
Proportion of omissions significantly reduced
from 8.1% to 1.4% of doses, between the first
and third phase.
105
Jani et
al.
2010
JAC
Compu
ter the
system
service
s Ltd.
Alerts
prescriber if
height or
weight
entered was
outside the
96th
centile
range based
on the
child’s age.
Prompts to
update
patient’s
weight if the
date of
previous
entry
exceeded an
age
dependent
time period.
Alerts for
weight
Tertiar
y care
paediat
ric
hospital
.
Renal
inpatie
nts and
outpati
ents
and
patients
dischar
ged
from
the
renal
and
urology
wards.
Compare the
incidence
and severity
rating of
paediatric
dosing errors
before and
after the
implementati
on of a
commerciall
y available
electronic
prescribing
with basic
clinical
decision
support
system.
Before and after study.
Prospectively collected
prescriptions from renal
inpatients and outpatients,
and for patients discharged
from the renal and urology
wards, which were evaluated
for errors at a later date.
Prescriptions initially
reviewed by the ward or
dispensary pharmacist as
part of routine duties. A
member of the research team
reviewed all errors
throughout the data-
collection period to ensure
consistency.
A team of 5 healthcare
professionals scored a
sample of the prescribing
There were a total of 145 dose errors in 8723
prescriptions.
Dose errors occurred in 88/ 3939 (2.2% of
prescriptions) pre EP and 57/4784 post-EP
(1.2%). Thus an absolute reduction of 1%,
which was a significant change.
No apparent change in error rates after
implementation of the system in the inpatient
setting, but a decrease was seen in the outpatient
and discharge prescription setting.
Change in error types also seen. E.g.
handwriting and incorrect unit errors were
eliminated post-EP while knowledge errors
persisted.
New errors found including mis-selection from
drop-down menus.
Severity:
Errors with potentially serious severity were
EP- system reduced rates
of dosing errors in
paediatrics, even in the
absence of CDS.
Errors fell from 2.2% to
1.2% across all
prescriptions during the
study period.
Reductions seen in the
outpatient and discharge
setting with negligible
difference in the inpatient
setting.
Limitations: Lack of
control group and only
small numbers of dose
errors were identified
during the study period.
106
changes of
+/- 10%
compared
with
previous
weight entry.
errors for their potential
severity o a scale of 0 (no
effect) to 10 (death).
Statistical data analysis was
performed.
eliminated in discharge and outpatient
prescriptions post-EP. Not possible to assess
statistical significance as the numbers involved
were too small.
Dose errors with the potential to result in minor
and moderate outcomes decreased after
implementation. 35/3939 (0.89%) pre compared
to 21/4784 (0.44%) post. A similar trend was
seen for dose errors with potential for severe
outcomes.
Jani et
al.
2008
JAC
Compu
ter the
system
service
s Ltd.
Alerts if the
patient’s
weight and
height
entered is
outside the
expected
range, based
on patient’s
age.
CDS
includes
Acute
tertiary
care
paediat
ric
hospital
.
Paediat
ric
nephrol
ogy
outpati
To
determine
the rate and
types of
prescribing
errors in
paediatric
outpatients
and to assess
the effect of
an EP
system on
these errors.
Incidence and type of
prescribing errors.
A pharmacist reviewed all
prescriptions before
dispensing as part of usual
practice. The pharmacist
annotated the prescriptions
with details of any agreed
changes. Any changes to the
original prescription were
considered potential errors.
A total of 520 patients had 2242 items
prescribed on 1141 prescriptions during the
study period. (8 prescriptions, 20 items were
excluded)
The overall prescribing error rate was 77.4% for
handwritten items and 4.8% with EP.
Pre-EP 1153, 73.3% of items missed essential
information and 194, 12.3% items were judged
to be illegible. Post-EP only 9 items were
missing information and none were illegible.
EP significantly reduced
overall prescribing error
rates (77.4% to 4.8%)
Increase in the number of
patient visits that were
error free post EP (21% to
90%
Limitations: Pre-EP and
Post-EP implementation
data were collected
concurrently therefore
prescribers had varying
degrees of familiarity
107
drug
monographs
(including:
indications,
contraindicat
ions, dosage,
interactions
and side
effects)
Drug allergy
and exact
drug
duplication
checks.
ent
clinic
Reviewing pharmacists were
blinded.
Two of the study team
retrospectively reviewed the
nephrology outpatient
prescriptions written during
the study period to identify
errors.
Statistical analysis was
performed.
Number of patient visits that were error free
increased from 21% to 90% after
implementation.
Errors related to drug and dosing schedule were
lower after implementation apart from wrong
drug errors was higher.
with the system.
No control group.
Inclusion of handwriting
and missing information
errors, may be susceptible
to interpretation and a
learning effect.
Dean-
Frankl
in,
2009
(metho
dologi
cal
review
)
ServeR
x:
MDG
Medica
l,
Israel,
version
1:13
Drug
dictionary
and default
doses.
London
teachin
g
hospital
.
28 bed
general
surgery
Compare
four
methods of
detecting
prescribing
errors in the
same patient
cohort, both
before and
Studied all patients on the
study ward during two 4-
week periods.
1st: 3 months prior to
implementation
2nd
: 6 months afterwards.
Prospective data collection
by ward pharmacist:
A total of 93 out of 129 patient records were
reviewed (72%) pre CPOE and 114 out of 147
records (78%) post CPOE.
For those patients reviewed, 1258 medication
orders were written pre-CPOE and 1614 post
CPOE.
Prescribing Errors identified by method:
Using data combined
from all methods the
prescribing error rate
reduced from 10.7% to
7.9% (statistically
significant result) post-
EP.
Each method identified
different prescribing
108
ward.
Closed
loop
system,
incorpo
rating
Compu
terised
physici
an
order
entry
(CPOE
), ward
based
automa
ted
dispens
ing,
barcode
patient
identifi
after
implementati
on of CPOE
Ward pharmacist recorded
prescribing errors as part of
routine practice. In addition
principal investigator
checked for prescribing
errors once a week to help
identify any that had been
missed.
Retrospective health record
review.
A retrospective review (RR)
form was used, included a
checklist of data sources,
patient information,
medication lists and details
of errors identified.
Inpatients during the study
period were identified from
the admission book and
records were retrieved. A
trained clinical pharmacist
completed the RR form, and
(pre:post % of all errors)
Prospective: 36%: 24%
Retrospective 69%: 83%
Trigger tool 0%: 1%
Spontaneous reporting: 1%: 1%
Prescribing Error rate per order written ( pre:
post % of al)
Prospective: 3.8%: 1.9%
Retrospective 7.4%: 6.5%
Trigger tool 0 %: 0.1%
Spontaneous reporting: 0.1%: 0.1%
Comparing Four Methods
Few errors were identified by more than one
method. Most identified either with RR or
prospective review. Using the trigger tool at
least one trigger was positive for 127 (61% of
the 207) patients and PE resulting in harm
identified in two patients.
If the prospective data alone is considered
studied, the reduction in errors would have been
errors, with remarkably
little overlap. Incidence
of errors extremely
dependent on the method
chosen.
Only 5-7% of all
prescribing errors were
recorded as being
identified by both ward
pharmacist and RR,
spontaneous reporting and
trigger tool each
identified less than 1% of
errors.
Limitations: Small pilot
study.
Data from only one
hospital and one ward
therefore questionable
generalizability.
No control group used.
109
cation
and
electro
nic
medicat
ion
adminis
tration.
laboratory data was
examined if relevant. The
research pharmacist was
blinded to the error recorded
by the ward pharmacist but
was able to see any
documentation notes.
Retrospective use of trigger
tool
Each patient record had the
trigger tool applied to it and
the research pharmacist
investigated positive triggers
in more detail, recording any
errors.
Spontaneous reporting
The study team
retrospectively retrieved
details of all incident reports
relating to the study ward for
each period and identified
significant. Absolute reduction from 3.8 to 1.9%
of all medication orders.
However if RR data alone studies, the reduction
would not have been statistically significant.
(7.4 to 6.5%)
Using data from all four methods the overall
reduction from 10.7% to 7.9% was statistically
significant.
Overall most prescribing errors were related to
the ‘need for drug therapy’ and ‘select drug
dose.’
Errors prospectively identified by ward
pharmacist were more likely to have been
rectified prior to administration.
Mean severity scores calculated for prescribing
errors identified by the ward pharmacist, RR
and spontaneous reporting were the same, errors
identified by the trigger tool had higher mean
110
those related to prescribing
error.
Comparison of four methods
Comparator used was the
number of errors per
medication order written
during the study period)
Only compared patients
whose health records were
available for RR.
Comparisons were made pre
and post CPOE. Prescribing
errors classified according to
stage of prescribing process,
stage of patient stay,
whether they were rectified
before they reached the
patient and whether any
harm was caused. Severity
was also assigned.
Appropriate statistical
analysis was performed.
severity scores.
Pre-CPOE no errors were found that resulted in
harm, post CPOE there were 4 cases, RR
identified all cases and the trigger tool identified
two cases. None appeared to be related to the
electronic prescribing system.
111
Dean
Frankl
in,
2007
(Close
d loop
system
)
ServeR
x
V.1:13
MDG
Medica
l,
Israel.
Ward drug
lists, drug
formulary
and drug
dictionary.
Default
doses for
most
products.
No other
CDS. If
allergies
were present
these were
displayed on
the
prescribing
screen.
London
teachin
g
hospital
; 28
Bed
general
surgery
ward
Assess the
effect of the
system on
the
prevalence,
types and
clinical
significance
of
prescribing
errors and
Medication
administratio
n errors
(MAEs),
confirmation
of patient
identificatio
n before
administratio
n and staff
time.
Before- after study,
collecting data on all
outcome measures:
prevalence, types and
clinical significance of
prescribing errors and
MAEs, confirmation of
patient identification before
administration and staff
time.
1st Phase: 3-6 months pre
implementation
2nd
Phase 6-12 months post
implementation.
Prescribing errors:
Ward pharmacist identified
errors during a 4-week
period. Principal investigator
checked for errors once a
Prescribing errors
Prescribing error fell from 93 (3.8%) of 2450, to
48 (2.0%) of 2353 orders. No change in mean
severity of errors was observed. More errors
were resolved before the patient received any
doses (48% pre-intervention, 67% post-
intervention) this however was not statistically
significant.
MAEs and checking patient identity.
Observed 56-drug rounds pre-intervention and
55-post intervention. MAEs fell from 141 (8.6%
of opportunities for error) to 53 (4.4%) after.
This was a significant change.
Due to the high number of errors observed for
IV doses, analysis was conduced for non-IV
doses, to limit bias. The error rate for non-IV
doses fell from 7.0% to 4.3%.
Staff Time:
Prescribing errors almost
halved following
implementation of EP:
Reduced prescribing
errors by 47%. Absolute
reduction in prescribing
errors from 3.8-2.0%
(1.8%).
Administration errors
similarly decreased:
Intervention reduced non-
IV MAEs by 39%,
predominantly through
reductions in wrong dose
and omission errors. The
system increased
percentage of doses for
which the patient’s
identity was checked
before administration.
Increased checking of
patient identity and may
112
week to help identify any
error not previously
documented. The team
recorded whether or not
errors were rectified before
administration. Denominator
was estimate number of
orders written during study
period.
Severity was also assessed
by five judges on a scale of
0 (no harm) to 10 (death).
Medication administration
errors and checking patient
identity
Pharmacists observed a
sample of drug rounds,
during a 2-week period (56).
Denominator was number of
opportunities for error
(doses administered plus
doses omitted/ preparation)
The time to prescribe 32 regular inpatient orders
pre intervention and 15 afterwards was
recorded.
Prescribing took a mean of 15 seconds per
medication order pre-intervention and 39
seconds post intervention (a difference of 24s)
Availability of patient records increased post-
implementation.
Time to provide a pharmacy service to the study
ward increased.
have resulted in more
prescribing errors being
corrected before the
patient received any
doses.
The intervention
increased pharmacy and
medical staff time.
Nursing time spent on
drug rounds decreased.
Limitations: Limited
generalizability, the study
site only took place across
one ward in one hospital.
No control group was
used.
113
Staff Time:
Observed doctors
prescribing inpatient
medication orders and
calculated mean time per
medication order. Then used
activity sampling to evaluate
the proportion of nursing
time spent on medication
related activities, between
drug rounds. The research
pharmacist would shadow
the nurse responsible for one
half of the ward.
Statistical analysis was
performed.
Riaz,
I. and
S. D.
Willia
JAC Information
not
available.
Acute
univers
ity
teachin
Compare the
prevalence,
type and
severity of
Pharmacists from medical
and surgical teams identified
prescribing errors on
prescriptions for patients
A prescribing error rate of 8.2% was found for
both electronic and paper discharges, with an
increase in the potential severity of electronic
prescriptions.
JAC system did not affect
prescribing error rate; and
caused potentially more
serious errors.
114
ms
(2010).
(Abstr
act
Only)
g
hospital
prescribing
errors
between
electronic
and
handwritten
hospital
discharge
prescriptions
.
discharged on four separate
days. They recorded the
number of errors, description
of the error and total number
of prescribed drugs checked.
Errors were also assessed for
severity by two consultant
physicians and a consultant
pharmacist.
Omission of drug therapy: electronic 42%
compared to 29.4% paper.
Selection of incorrect formulation: electronic
9.7% compared to 6.5% paper
Missing or incorrect drug dose strength:
electronic 1.1% compared to 13.5% paper
Missing or incorrect administration time:
electronic 0% compared to 9.4% paper
Fowlie
F et al.
(2000),
(Abstr
act
Only)
Pharma
kon
UK
No
information
36
Bedded
-
orthopa
edic
ward
Assess the
safety of an
electronic
prescribing
administratio
n (EPA)
system
through
assessment
A three-phase observational
study was performed using
disguised observational
methods.
Group A: Paper system
Group B: 1 month post
implementation
Group C: 12 months post
For group A, conformance to standard of
prescription writing (12 criteria) ranged from
1.8% to 99%. Group B had 11 criteria reaching
100%.
Comparison of prescribing errors and
medication administration errors for group A
with B and B with C demonstrated a significant
difference for inpatient (p<0.001) but not
An EPA system can
improve the quality of
prescription writing errors
in an orthopaedic ward.
Limitations: Lack of
control group, exclusion
of intravenous and
controlled drugs from the
MAE study, which were
115
of
medication
errors and
the quality
of
prescription
writing.
implementation.
Prescriptions assessed for
standard of writing, except
for phase 3; and prescribing
errors and administration
errors.
All errors were categorised
and clinical significance
assessed independently by 6
judges (0-10: no harm-
death)
discharge prescriptions.
Clinical significance median of inpatient
prescribing errors was:
Group A: 4
Group B: 3
Group C: 3
Similar distribution for discharge prescriptions.
Comparison of MAEs for group A with B and A
with C demonstrated a significant difference.
Clinical significance of MAEs was:
Group A: 2
Group B: 2
Group C:1
omitted due to non-
standard administration
times. Also restricted to
surgical ward only.
Marri
ott J et
al.,
(2004)
Abstra
ct
Medica
l
Inform
ation
Techno
logy
No
information
Queens
Hospita
l (BH)-
EP site
Good
Compare the
number and
range of
recorded
pharmacist-
led clinical
Pharmacist intervention data
was recorded manually onto
intervention recording
sheets, which were
transcribed for manipulation
and analysis in excel.
Interventions at BH: 2512
Interventions at GHH: 763.
Intervention rates:
BH: 0.20 interventions/ finished consultant
episode
A larger number of
interventions were
reported at the site with
an electronic prescribing
system. It is likely that
this reflects differences in
116
Only Inc.,
Westw
ood
Mass.,
US.
Hope
Hospita
l
(GHH)-
Manual
paper
prescri
bing.
Both
district
general
hospital
with
similar
activity
-
approxi
mately
50,000
finishe
d
consult
interventions
in a hospital
operating a
typical
paper-based
recording
system and
one of
similar
characteristi
cs with an
electronic
patient
management
and
prescribing
system
Interventions were recorded
from each site between
September 2003 and
November 2003.
Events were grouped into 18
categories of interventions.
GHH: 0.05 interventions/ finished consultant
episode.
Main type of interventions:
BH:
Dose regimen (352, 14%);
Drug choice (182, 7%); Length of treatment
(1259, 50%)
GHH:
Dose regimen (266, 35%);
Drug choice (190, 25%);
Length of treatment (45, 6%)
Prescribing transcription error
GHH: 128, 16%
BH: 5, 0.2%
Drug Interaction, use of non formulary agents,
route changes, prescription illegibility
interventions
GHH: 13% of interventions
BH: None.
workload; rather that the
electronic system
facilitates intervention
reporting.
The categories of
interventions differ
between the EP and non-
EP site.
Greater number f
interventions based on
patient focused
pharmaceutical care at the
EP site, whereas non-EP
site interventions were
mostly orientated around
choice and prescribing of
appropriate therapy.
117
ant
episode
s.
Drug information and monitoring patient
clinical markers interventions
BH: 26% interventions
GHH: None
Small,
M. D.
C., et
al.
(2008).
VARIS
MedOn
c
system
in
outpati
ents
Manual
Excel
spreads
heet
prescri
ptions
in
inpatie
nt and
No
information
Norfolk
and
Norwic
h
Univers
ity
Hospita
l
Outpati
ent
1. To
determi
ne
whether
compute
rised
prescribi
ng of
chemoth
erapy
reduces
overall
prescribi
ng error
rates.
2. To
docume
Prospective audit of
chemotherapy prescribing by
a single oncology
pharmacist.
Duration: 4/1/2005-9/9/2005
All oncology chemotherapy
prescriptions (n=1653) were
recorded as Excel spread
sheets (314) or computerised
prescriptions (1339). All
haematological prescriptions
were ordered on excel
spread sheets (n=288)
Only ‘complex’
Error Rates
Spread sheet prescriptions (n=602): 123 errors
(20.4%)
Computerised prescriptions (n=1339): 158
errors (11.8%).
Demonstrating a statistically significant
difference in error rates (8.6%, P<0.0001).
Relative risk reduction of 42%.
Error Type:
Differed significantly according to the
prescription method (P<0.001).
Computerised prescribing: fewer wrong dose or
frequency, incomplete prescriptions and
unnecessary additional agents and more wrong
cycle number or cycle stage, wrong data entered
e.g. height or weight errors.
Computerised prescribing
was associated with a
relative risk reduction of
42% compared to spread
sheet prescriptions.
Dose errors were reduced
significantly.
Handwriting and
transcription errors were
also prevented.
Incomplete prescriptions
were much less common.
Input errors e.g. weight or
BSA occurred frequently
with the computer system.
Minor errors were
reduced. Significant
118
haemat
ology
setting.
nt the
types
and
patterns
of
errors,
and the
potential
for harm
of these
errors.
3. To
examine
variatio
ns in
error
rates
between
individu
al
prescrib
ers
using
prescriptions included.
Errors were detected and
recorded as part of routine
checks by the oncology
pharmacist.
The potential significance of
errors was classified as
either: minor, significant,
serious or life threatening.
Errors were compared
between three staff grade
oncology prescribers.
Haematology prescriptions
were excluded from this part
of analysis.
Statistical analysis was
performed.
In particular dosage errors reduced from 6.8%
to 1.9% (HWP: CPOE). While wrong cycle
number errors increased from 2.5% to 5.6%
Severity:
Computerised prescribing was associated with
fewer minor errors and more serious errors. The
proportion of significant and life-threatening
errors remained the same.
Rate of errors by prescriber:
A wide variation found between prescribers was
seen in both systems.
errors occurred at
approximately the same
rate.
Serious errors were
greater with the
computerised system,
including cycle length
errors. Potentially life-
threatening errors
occurred at approximately
the same rate, however
the nature of errors was
very different.
There was wide variation
if prescriber’s error rates.
119
spread
sheet or
compute
rised
prescript
ions.
Almon
d et
al.,
2002
MediC
hain:
Computerise
d stock
control
Alert
pharmacy
each time a
non-stock
item was
prescribed,
aimed to
reduce the
problem of
ward drug
availability.
33 Bed
Acute
medical
ward,
with a
sub
speciali
ty in
renal
medici
ne.
District
General
Hospita
l.
Prospective, controlled,
before and after study with
external validation
performed by researchers
from the University of
London’s School of
pharmacy.
6-Months evaluation of the
system, 3 months pre-
implementation and 3 after.
External validation team
collected 3 month’s data pre-
implementation on the
intervention ward and a
control ward (respiratory
Pre-implementation:
No significant difference between renal and
control ward in prescribing errors, detected by
the ward pharmacist or external review team.
Prescribing quality
Pre-implementation: 1% of prescriptions
remained illegible at the time of administration.
Post-implementation: All prescriptions passed
the clinical screening, 94% with one or no
modifications. All prescriptions were legible
and contained a route of administration at the
time of administration.
Success rates in administering prescriptions
Pre-implementation:
EPMA can be safely
introduced into a busy
medical ward in a
reasonable time frame.
Improved prescribing
quality was found post-
implementation
120
speciality).
Following implementation,
training and equipping the
ward with necessary
hardware, 3 months post
implementation data
collection took place on
renal ward and control ward.
During the pre-
implementation period paper
charts were reviewed by
external validation team for
evidence of major, moderate
or minor errors in
prescribing.
Medicines administration
rounds were timed, number
of medicines administered
and reasons for non-
administration recorded.
1169 attempted medicine administrations were
observed.
Renal and control wards achieved similar rates
of administration success (90 and 91%)
Significance of errors was classed as major for
25% of occasions on both wards.
Post-implementation:
Control ward- no change.
18357 attempted administrations recorded on
the system.
The system recorded 95.4% as being successful.
This was a significant improvement (p< 0.001).
Where non-administration occurred, the reason
was clearly documented unlike on paper-
records.
Record Availability
Medical, pharmacy and nursing tasks and 1:4
rounds were affected or complicated by at least
one missing administration chart with paper-
system.
Problem eliminated with EPMA.
121
Working practices of the
ward’s clinical pharmacist
were timed.
Data on stock management
and cost of medicines
supplied was routinely kept.
Post- implementation, the
quality of prescribing was
audited against:
Attempts at prescribing and
modifications required
before the prescription was
accepted.
Nurses directly recorded
administration, or non-
administration and reasons
into the system.
The systems ability to
Ward procedure
Time taken to complete an administration round
almost doubled.
Effect on increased of ward length was felt
throughout the ward, with an increased burden
on healthcare assistants.
Due to the in-built security system, the
electronic drug cart was left unattended more
often than the traditional trolley.
System was used as a tool for patient care
beyond simple prescribing and administration.
Users Views
The initial training was associated with high
levels of satisfaction, but somewould have liked
increased access to the training rooms to
practice on an individual basis.
Unanimous view that the suppliers technical
support system was essential during the ‘go-
122
manage ward stock was
assessed by pharmacy.
At the end of the study,
medical, nursing, pharmacy
and dietetic staff were
provided with a
questionnaire.
live’ period.
The majority of staff considered equipment and
software easy to use.
Difficulties were encountered when prescribing
variable dose regiments and the prescribing and
administration of IV medicines and fluid.
Medical staff found it took longer to prescribe o
the EPMA system.
The majority of users believed the system was
safer, although the time taken to achieve this
was considered excessive.
The nurses preferred the EPMA system, a small
majority of medics would like to return to paper.
Mitche
ll D. et
al.,
2004
Clinical
Manag
er
Version
3.03A
None
declared,
however
authors
stated
General
Surgery
ward,
includi
ng high
To examine
the success
of a pilot
introduction
of electronic
A prospective audit of
handwritten and electronic
medication administration
records for accuracy and
completeness.
Nurse audit of administration accuracy
The electronic system was significantly more
accurate than handwritten administration
records. There was a significant improvement in
the accuracy of patient details, prescription
Accuracy of information
within the medication
administration record
improved with EP.
Prescribing error rate
123
(provid
ed by
iSoft
UK
PLC)
system had:
Drug listing,
with
potential to
list most
commonly
used
formulations
first.
Pre-written
pick-lists
depend
ency
unit
and
general
surgery
operati
on
theatres
.
prescribing
and
electronic
drug
administratio
n in a
hospital
A single nurse carried out
manual audits of drug
rounds according to a set
protocol. Information was
either classed as complete or
incomplete. Two cycles
were conducted at 17 and 11
months pre-implementation.
One audit cycle was
conducted eight weeks post
implementation.
The pharmacy department
audited every electronic
drug order for accuracy; this
was reviewed by the project
group on a weekly basis
Pharmacy intervention (need
for enquiry or modification
of a prescription) rates were
recorded throughout the
legibility, dose clarity, start date, 24-hour clock
used, signature recognisable, route of
administration stated, dosage directions and
length of treatment. The study found where
documentation was not mandatory within the
system e.g. allergies, accuracy and
completeness was unaffected.
Pharmacy continuous audit of electron drug
order entry
A total of 4927 prescriptions were written
during the study period (13 weeks).
The error rate during week-1, was 6.4% (35/544
items). The overall error rate throughout the
period was 2.9%. The most common issues
encountered were wrong formulation selected
(41% of prescribing errors), wrong dose and
‘other’ prescribing problem, which included
duplicate drug errors. The study identified
particular system-related errors, which occurred
in 1.2% of the total prescriptions. These errors
were no longer present after week-8 of the pilot
study.
post-implementation was
2.9%.
In terms of clinical
interventions, no
difference was observed
between handwritten and
electronic prescriptions.
Electronic prescribing had
little impact on decision
making for example
selecting the correct drug
and dose, however did
make all information
legible and
understandable.
Nursing staff were
satisfied with the system,
they identified advantages
such as improved
available of drug
administration record and
ability to have feedback
on overdue drugs at the
124
hospital for 5 days in the
middle of the project. The
interventions were graded
according to severity.
A user satisfaction survey
was also conducted for
members of the audit team,
towards the end of the study
period.
Pharmacy intervention audit
A significant difference between handwritten
and electronic intervention rates could not be
demonstrated in this study. Interventions
relating to ‘minor problems’ were significantly
more likely to occur in handwritten
prescriptions.
User-satisfaction Survey
There was a 58% response rate ( 94 end-users)
Nurses
Nearly three quarters (72%) found the laptops
easy to use. The majority (88%) found the
electronic drug administration was considered
easy. Over half of respondents (54%) thought
electronic administration was safer.
Approximately one quarter of respondents felt
drug rounds took as long as with the paper-
methods and 63% of respondents felt that the
system increased the quality of patient
information to support prescribing,
end of a shift.
Medical staff found
reported that the lack of
clinical decision support
was an issue. In particular
staff felt that allergy and
drug interaction check
sand more active clinical
decision support should
have been included.
125
Doctors
The majority (80%) found the system easy to
find drugs to prescribe and over a third (67%)
found it easy to choose the correct dose and
frequency. However ordering complex
prescriptions such as different formulations or
intravenous additives ere associated with lower
ease of use. The laptops were considered
straightforward to use (65%) and there was
approximately an equal split between
respondents regarding the ability of the system
to make prescribing safer.
126
Appendix 3: Search Terms
Electronic Prescribing Clinical Decision
Support
Electronic Health
Record
Error Rates
Computerized
prescriber order entry
Computerized provider
order entry/
Electronic physician
order entry
Electronic order entry
Electronic prescribing/
Electronic prescription
Computerized
physician order entry
CPOE
Computerized order
entry
Medical order entry
systems
Clinical decision
support
Decision support
system/
CDS
Drug therapy,
computer assisted
Electronic medical
record/
Electronic health
record
Electronic patient
record
Medication Errors
Error Rates
127
Appendix 4: Search Strategy
1. Computerized prescriber order entry
2. Computerized provider order entry/
3. Electronic physician order entry
4. Electronic order entry
5. Electronic prescribing/
6. Electronic prescription
7. Computerized physician order entry
8. CPOE
9. Computerized order entry
10. Medical order entry systems
11. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10
12. Clinical decision support
13. Decision support system/
14. CDS
15. Drug therapy, computer assisted
16. 12 or 13 or 14 or 15
17. Electronic medical record/
18. Electronic health record
19. Electronic patient record
20. 17 or 18 or 19
21. Medication Errors
22. Error Rates
23. 21 or 22
24. 11 or 16 or 20
25. 23 AND 24
26. Limit 25 to English language
128
References
1. Franklin BD, Birch S, Savage I, Wong I, Woloshynowych M, Jacklin A, et al.
Methodological variability in detecting prescribing errors and consequences for the
evaluation of interventions. Pharmacoepidemiology & Drug Safety. 2009;18(11):992-9.
2. Franklin BD, O'Grady K, Donyai P, Jacklin A, Barber N. The impact of a closed-loop
electronic prescribing and administration system on prescribing errors, administration errors
and staff time: a before-and-after study. Quality & Safety in Health Care. 2007;16(4):279-84.
3. Donyai P, O'Grady K, Jacklin A, Barber N, Franklin BD. The effects of electronic
prescribing on the quality of prescribing. British Journal of Clinical Pharmacology.
2008;65(2):230-7.
4. Fowlie F, Jardine G, Bicknell S, Toner D, Caldwell M. Evaluation of an electronic
prescribing administration system in a British Hospital. Pharmaceutical Journal
2000;265:7114.
5. Mitchell D, Usher J, Gray S, Gildersleve E, Robinson A, Madden A, et al. Evaluation
and audit of a pilot of electronic prescribing and drug administration. Journal on Information
Technology in Healthcare. 2004;2(1):19-29.
6. Almond M. GK, Kent J., Nice S., Dhillon S., . The effect of the controlled entry of
electronic prescribing and medicines administration on the quality of prescribing safety and
success of administration on an acute medical ward. British Journal of Healthcare Computing
and Information Management, . 2002;19(2):41-6.
7. Riaz I, Williams SD. Impact of a new electronic discharge system on the prevalence
of prescribing errors. International Journal of Pharmacy Practice. 2010;18(SUPPL. 1):22.
8. Marriott J. CC, Carruthers T., Feeley G., Langley C., Tongue R., Wilson K.,. The
influence of electronic prescribing on pharmacist clinical intervention reporting. International
Journal of Pharmacy Practice. 2004;12(S1):R44.
9. Evans KD, Benham SW, Garrard CS. A comparison of handwritten and computer-
assisted prescriptions in an intensive care unit. Crit Care. 1998;2(2):73-8.
10. Shulman R, Singer M, Goldstone J, Bellingan G. Medication errors: a prospective
cohort study of hand-written and computerised physician order entry in the intensive care
unit. Crit Care. 2005;9(5):R516-R21.
11. Dean B, Barber N, Schachter M. What is a prescribing error? Quality in health care :
QHC. 2000;9(4):232-7.
12. Small MD, Barrett A, Price GM. The impact of computerized prescribing on error rate
in a department of Oncology/Hematology. Journal of Oncology Pharmacy Practice.
2008;14(4):181-7.
13. Jani YH, Barber N, Wong IC. Paediatric dosing errors before and after electronic
prescribing. Quality & safety in health care. 2010;19(4):337-40.
14. Jani YH, Ghaleb MA, Marks SD, Cope J, Barber N, Wong IC. Electronic prescribing
reduced prescribing errors in a pediatric renal outpatient clinic. J Pediatr. 2008;152(2):214-8.
15. Warrick C, Naik H, Avis S, Fletcher P, Franklin BD, Inwald D. A clinical information
system reduces medication errors in paediatric intensive care. Intensive Care Medicine.
2011;37(4):691-4.
16. Schiff G, Amato MG, Eguale T, Boehne JJ, Wright A, Koppel R, et al. Computerised
physician order entry-related medication errors: Analysis of reported errors and vulnerability
testing of current systems. BMJ Quality and Safety. 2015;24(4):264-71.
129
Outcome 6: The NHS hospital Trusts in the UK that have implemented
electronic prescribing systems successfully, with examples of success
stories, lessons learnt and transferable best practice.
Aim: To present success stories, lessons learnt and transferable best practice from UK
hospital Trusts that have implemented ePrescribing successfully.
The information detailed below was obtained from a number of different sources, including
the literature, conference presentations and the ePrescribing Toolkit Website.(1-3)This
website is an output of the NIHR funded ePrescribing research programme, of which Dr
Sarah P. Slight, Prof. Jamie Coleman and Ann Slee are all co-investigators on.
Case Study Site A
Background and Methods
Site: This Trust provides acute care for an urban population of approximately 330,000
patients. At the time of data collection, the Trust had a separate Patient Administration
System (PAS), which was used for clinical information and did not integrate with
ePrescribing.
Data collection took place between December 2011 and August 2012. Interviews with
pharmacists, nurses, and doctors (of varying levels of seniority), and with implementers were
conducted; observations of strategic meetings and system use were also documented. Notes
were taken during the recruitment meeting and Trust documents relating to anticipated
changes in processes with the introduction of the new system (including work process maps,
implementation plans, business case) were collected.
The site began implementation of a standalone ePrescribing system in 2010, and already had
a pharmacy stock control system by the same developer. The system included the use of order
sets and had limited decision support functionality for interactions/allergies. The system was
not used for the prescribing of certain types of medications (including infusions and
warfarin).
130
Implementation took place over a period of 12 months (four wards per month), and the Trust
was live in all 36 inpatient wards at the time of data collection. They had not implemented in
outpatients or critical care.
Key findings and challenges
Overall staff were generally positive,
Feeling that workload may have increased in some areas. This was possibly due to
additional hardware and software components introduced within established working
environments, thus changing normal processes,
Concerns relating to limited access to computer terminals and the sub-optimal
performance of software,
Challenges using information from multiple sources including co-existing paper and
electronic systems,
Users employed informal coping mechanisms (e.g., using another staff’s log-in) to
deal with problems with the system, for example, waiting for a computer to become
available or delay inputting data into the system to avoid the repeated need for
perceived lengthy log-in procedures.
Lessons Learnt / Key Messages
Overall, the system was liked by users.
Clinical staff were faced with changes to physical and virtual environments as a result
of implementation, which resulted in them devising strategies to cope with these
challenges.
Implementation teams may not necessarily be aware of the coping strategies as they
are informal, but they need to be tracked over time to avoid potential adverse
consequences for patient care.
131
Case Study Site B
Background and Methods
Pre-implementation case study with lessons learnt: perceptions and attitudes towards
ePrescribing implementation.
This site has around 1,000 beds and 6,000 staff serving a population of over 1 million and
will be implementing an integrated ePrescribing and Medicines Administration system within
the next few months. The initial roll-out will be limited to a pilot area of the hospital in order
to identify and address any issues. It is expected that the system will then be subsequently
rolled out to the rest of the hospital in phases over the course of a four to five month period.
The implementation team is led by a pharmacist and co-chaired by a paediatric consultant and
a pharmacist. Customisation of the system involved working with other Trusts, which were
also implementing the same system and with a team of clinicians. Meetings took place in
order to provide opportunities to discuss plans, demonstrate the system, obtain feedback and
reach decisions.
The views represented in this case study are mostly representative of staff, who had been
involved in the implementation process.
A total of 24 interviews were conducted between 1st April and 31
st July 2013, this included
22 hospital staff (doctors, nurses, pharmacists, allied health professionals, information
technology (IT) staff) and members of the implementation team and two system suppliers.
Observations were also carried out to explore how suppliers, the hospital implementation
team and end-users interacted when the design and customisation of the system.
Key findings and challenges
The research team identified five key areas:
1. Transition to an electronic environment
Respondents were generally in favour of transition to an ePrescribing
system. However, there were mixed views regarding system suitability and
desirability, with considerable negativity towards the system. This
negativity appeared to have been a consequence of previous bad
132
experiences with other systems in use at the hospital, issues regarding the
suitability of the system in a UK hospital (the system had been mainly
implemented in the US), issues regarding the suitability of the system
across all clinical environments and perceived poor usability were also
raised. These issues contributed to delays in the implementation, increased
workloads, disengagement, anger and anxiety towards implementation
among end-users.
2. Communication and Engagement
There were concerns among end-users about a lack of communication
regarding the ‘what’, ‘why’, ‘when’ and ‘how’ of the roll-out process.
Respondents reported feeling that those involved in system customisation
were not representative of all staff members. Additionally there were
concerns that the input of healthcare professionals who were involved was
often ignored. Thus, efforts from the IT team to involve a range of end-
users were not deemed successful. End-users appeared to over-estimate the
level of customisation possible and therefore had unrealistic aims for the
system and implementation process.
3. Leadership
Overall leadership was considered as outstanding by most participants,
however the project appeared to lack visible Trust level endorsement and
leadership. This may have contributed to reduced engagement in the
implementation project.
4. Infrastructure and support
Supplying suitable hardware was perceived as critical to project success by
implementation leaders. Concerns arose however around what levels of
hardware were required and what particular devices were needed. There
were also concerns about connectivity and resilience of the hospitals
network.
5. Training
There was uncertainty about whether the training would meet the needs of
end-users due to concerns about timing, content and ability to encourage
participation from all staff including bank and agency workers. Again
communication issues regarding the lack of information and training plans
133
raised questions from respondents about the attention giving to training in
the project.
Barriers
1. Anglicisation of the system
A high level of anglicisation was required to adapt the system for use in a
UK hospital.
The amount of work required for anglicisation had not been anticipated or
sufficiently considered during the procurement stage.
Decisions had to be made about whether to change work procedures or
adapt the system, based on the differences between the UK and US
healthcare environment (which the system had been designed for).
There were concerns regarding the suitability of the system in a UK setting
despite the successes in the US.
2. Inexperience and lack of knowledge
There was a general lack of experience about how best to implement
ePrescribing in the NHS, as the supplier’s recommendations were mostly
from the US.
A theme was identified around ‘learning as you go’, and adapting plans
based on experience.
In addition to technical lack of experience, a lack of expertise in other
domains was noted. For example, the need for organisation personnel and
the importance of a communications manager.
Potential underestimation of the scale of the change that implementation
would bring.
3. Infrastructure and Integration
Concerns were raised around the robustness and selection of appropriate
hardware to support use of the system across clinical areas,
There was uncertainty due to the lack of experience, lack of consultation
with end-users and need for a large quantity of devices in certain areas and
to meet the demands of a range of clinical staff.
Additional concerns were raised about system network connectivity and
interoperability with other systems in use.
134
Anxiety was reported about downtime due to overreliance on computer
systems.
4. Limited customisation and project scope
Enhancements to the system were not completed because of contractual
arrangements and the supplier’s inexperience.
Due to the narrow scope of the project, which limited the changes that
could be made to the system, the site was not able to benefit from some of
the newer features.
End-users were not aware of the project scope or the systems capabilities
and limits. Therefore, failure to respond to recommendations gave the
perception that the hospital did not listen to end-users or were not driving
the project.
Control over the customisation process was regained in part due to the
hospital implementation team’s thorough knowledge of the system.
Improvements were noted when the supplier’s and implementation team’s
views aligned.
5. Usability and functionality
The system was frequently referred to as ‘clunky’, ‘not-user friendly’ and
with ‘poor design’ by end-users. There were also issues identified
regarding the intuitiveness and display of the system.
There appeared to be disparities between how doctors were taught how to
prescribe and the system’s US-origins.
Inconsistencies in the design between different functionalities e.g.,
administration or prescribing, and suitability for particular clinical areas
were also discussed.
Respondents also felt that the system lacked some functionalities that they
would have expected and that they could not already access in other ways
for example a smartphone British National Formulary (BNF) application,
indicating there is a need to manage end-users expectations.
6. Engagement
Involving a range of healthcare practitioners in the customisation process
was considered to be key to system usability.
Some participants found participation as being beneficial and felt valued.
135
There were some challenges around involving end-users; pharmacists had
been easier to engage with for example than other groups who had opted
out.
Many end-users felt that the project and customisation meetings had not
been widely advertised.
Concerns among end-users included: poor communication about the
progression and timescale of the project, lack of clarity regarding who is
affected by the introduction of the system and who should be involved in
the customisation, lack of information regarding key project milestones.
Overall the project(s) was said to have ‘low visibility’.
The strategy to rely upon optional attendance at meetings may have also
been problematic and may have contributed to members missing meetings.
Support from the wider organisation is also needed as barriers including
insufficient training leave and backfill meant that staff were perhaps
unable to attend meetings.
The need to incorporate the views and perspectives of a range of end-users
was perceived as vital during the customisation process.
Engagement from the Trust board is important, particularly to provide
leadership and viability to the project, which would help engage with staff
throughout the hospital. There was a perceived lack of engagement from
the Trust board, particularly when issues were arising.
A mismatch between healthcare delivery and technology may exist, for
example staff reported being conscious of using technology due to
concerns patients would perceived this as not fitting with their
expectations of what caring for patients looked like.
A lack of agreement was also noted between stakeholders, who interpreted
the implementation process as a success and end-users who had held an
opposing view. This highlights both a lack of communication and
potentially unrealistic expectation. Improvement feedback mechanisms
and greater explanation of various aspects of the implementation process
may facilitate alignment of visions.
There was enthusiasm and positivity towards the move to ePrescribing.
However there appeared to be more negativity towards the system and
136
methods of implementation.
7. Skills and Training
Concerns that some staff lacked basic IT skills,
Low use of smart-cards across staff, especially senior staff,
Poor typing skills,
High usage of bank and agency staff,
Delegation of IT tasks from senior to junior staff,
Doctors failing to attend training sessions,
Use of existing systems by doctors only with assistance from nursing staff,
A solution to these problems requires a comprehensive training
programme, which includes use of the electronic prescribing and
medicines administration (EPMA) itself but also other IT systems in use.
Various references were made to ‘informal local experts’ who showed
other colleagues how to use the IT system, using this method more
formally may be a potential strategy towards the training of staff who are
unable or unwilling to attend formal sessions.
Training is critical for end-users in order to become familiar and
comfortable with using the system.
Lessons Learnt/ Key Messages
Communicate with staff: what is being rolled-out, who will be affected, key
milestones and associated events e.g., training.
Manage expectations: clarify what the system can be expected to do. Regular updates
are important.
Explain in detail the scope of the project and the background to this to end-users.
Ensure a range of expertise within the implementation team to address both technical
and organisational issues.
Ensure the project is focused on improved patient outcomes and not as an ‘IT project’.
Ensure appropriate infrastructure is in place and liaise with end-users to inform the
selection process and ensure they are kept up-to-date with the infrastructure plans.
Ensure the communication strategy has longevity to span the entire implementation
and rollout process.
Tailored training strategies, which have considered the differences between individual
137
end-users and the preferred training approaches.
Provide incentives and support for participation and involvement in the
implementation.
Continuity within the implementation team to retain knowledge and expertise.
Recognition of individual’s efforts is therefore key.
Obtain and maintain Trust board project ownership to maximise engagement across
the hospital.
138
Case Study C
Background and Methods
This Trust provides acute care for ~3,000,000 patients in an urban setting. It started
implementation of a partially integrated ePrescribing module in 2009. Data were collected at
the site between May 2012 and April 2013. A total of 21 interviews were conducted with
users (including pharmacists, nurses, doctors of varying levels of seniority) and
implementers, and four observations (nine hours) of strategic meetings and system use. In
addition, notes were collected from a recruitment meeting and three documents relating to
anticipated/planned changes associated with the implementation (e.g., work process maps,
implementation plan, and business case).
The system was implemented in all inpatient wards except paediatrics. Functionality included
prescribing of all medicines except: variable dose insulin, patient controlled analgesia, and
intravenous hydration fluids. Some other medications such as warfarin were only partially
supported. Decision support consisted of: order sentences, order sets, allergy checking, and
some locally customised pop-up warnings. They had not switched on drug-drug interactions,
duplicates, or contra-indications functionality. Clinical noting was not implemented but users
had the ability to see laboratory and pathology results on the system.
The Trust is planning to continue to implement a fully integrated EHR from the same
supplier across the organisation.
Key findings and challenges
This case study provides information about the use of an ePrescribing system, 3-4years after
the implementation. The study found that information was still distributed amongst many
different sources as the integrated system was not fully implemented, therefore the full range
of benefits had not been realised. Regular maintenance of the system and ongoing
customisation (particularly in relation to tailoring the system, which was developed in the
USA,) for the UK context was needed. Further work was also needed to learn the full range
of functionality available, refine the decision support system and implement increasing
modules of the wider integrated system. This required a committed implementation team,
who was gradually learning the complex skills associated with these activities. This presented
139
a challenge for the implementation team, in order to possess sufficient knowledge and
expertise in order to manage the increasing functionality and data generated by the system.
For example an increasing amount of data became available for secondary uses, resulting in
complex considerations surrounding which data to extract and focus on.
Lessons Learnt / Key Messages
Significant on-going work is required to implement, customise and maintain an
integrated system of great complexity,
On-going work is likely to continue over long time-frames,
Organisations should anticipate and prepare for the additional resources required to
carry out on-going work,
Post-implementation maintenance and customisation is necessary to help hospitals
realise the full potential of ePrescribing systems; including use of more advanced
functionality, clinical decision support and evaluation and application of data
collected by the system.
140
Case Study D
Backgrounds and Methods
The Trust serves a population of approximately 400,000 people across two main sites. In the
late 1980’s an American Patient Administration System (PAS) and clinical modules was
procured and implemented. Additional functionality was subsequently added to include
electronic ordering and resulting for laboratory and radiology test. Electronic prescribing and
medicines administration (EPMA) was implemented in the early 1990’s. The current
electronic prescribing system is live in all areas except the intensive care unit (ITU) and high
dependency unit (HDU) and the neonatal unit, maternity wards and outpatients. The A&E
department uses ePrescribing only for patients who are going to be admitted as inpatients.
Due to issues regarding lack of system support and limited functionality to meet the demands
of 21st Century healthcare, the Trust are now in the process of migrating to a different
integrated Electronic Patient Record (EPR) with EPMA. This case study is adapted from a
presentation given by the Trusts lead informatics pharmacist at a national conference.
Key findings and challenges
Initial roll-out of the EPMA system took place following implementation of a PAS, clinical
modules, and ordering and reporting facilities for laboratory and radiology tests. EPMA was
piloted in a number of wards before extending its use across the majority of hospital wards. A
team of technical and clinical staff were tasked with the challenge of adapting and
customising an American system for use in a UK setting. Hospital-wide use of the EPMA
system was not possible due to the complexity of certain prescribing areas and system
limitations, which resulted in some areas continuing to use paper-based prescribing methods.
At the time of EMPA implementation, the majority of processes in hospitals were paper-
based, therefore the system was designed to work alongside these other methods. However,
with technological advancements such as wireless technology now available, there is less of a
requirement for the EPMA system to depend on paper-processes. The system in use is
continuously being updated and adapted in response to the hospital’s needs, for example,
changes in drug formularies and national initiatives. In response to issues with the existing
system, a new integrated system is being slowly introduced into the Trust, with plans to
implement EPMA. The new system is likely to incur many of the same challenges as the
141
current system, such as (i) a need to anglicise a US system, (ii) respond to new advances in
technology, such as increasing use of handheld and portable devices, and (iii) also work
towards integration with other hospital systems and primary care systems to enable seamless
care.
Lessons Learnt/ Key Messages
System evolution is important for example maintaining drug catalogues, clinical care
pathways and implementing clinical decision support, as is the need to respond to
changes.
A robust PAS system is essential for accurate patient identity and preventing wrong
patient errors.
System characteristics are important; the system should be easy to log-on to and be
fast and reliable for end-users.
It was considered important to instil a strong culture regarding information
governance and the importance of protecting patient data and not sharing log-on
passwords with colleagues. To support this, an audit trail should be available.
Use of a single integrated system for most clinical applications including the PAS,
laboratory results and discharge letters was a major advantage. In particular this was
useful for achieving ‘buy-in’ from end-users across the hospital, including medical,
nursing, pharmacy and allied health professional support, rather than being perceived
as a ‘pharmacy only’ or ‘IT only’ system.
The system must support processes rather than change them, for example the system
can promote formulary management, if there is already a strong drug and therapeutics
team in place.
There is a need for robust down-time procedures to be in place.
Careful consideration of how to use clinical decision support is required, particularly
regarding the issue of overloading clinicians with alerts. The concept of ‘promoting
the path of least resistance’ was also raised, for example using clinical decision
support to encourage appropriate selection in the first instance rather than attempting
to alert clinicians (which we know has limited success) following inappropriate drug
selection.
142
Acknowledge that errors will still happen, therefore systems should be in place to
capture these problems in order to address existing system issues. Importantly any
changes made should also be evaluated in order to judge their effectiveness.
A stable informatics team is needed, which includes members from a range of
disciplines.
When selecting a system, desirable characteristics include flexibility, agility and
ability to perform local customisation.
Organisational leadership and support is important for project success.
Use of the EPMA system should be mandatory.
End-user training should be compulsory.
Use of an electronic data warehouse to enable research and audit of the system and
processes to be performed.
In hindsight some of the pathways in the system were overly complicated and not
user-friendly, these would have benefitted from simplification
The system generated large amounts of paper; any new system should aim to reduce
(or eliminate) the dependence on paper.
Where possible, efforts should be made to implement EPMA across all areas to avoid
the use of hybrid systems.
It was deemed important for future projects to have better involvement of end-users
and not just senior clinicians.
References
1. Cresswell KM, Bates DW, Williams R, Morrison Z, Slee A, Coleman J, et al.
Evaluation of medium-term consequences of implementing commercial computerized
physician order entry and clinical decision support prescribing systems in two 'early adopter'
hospitals. Journal of the American Medical Informatics Association : JAMIA.
2014;21(e2):e194-202.
2. ePrescribing Research Programme Team. ePrescribing Toolkit for NHS Hospitals,
2014 [cited 2015 14th July 2015]. Available from:
http://www.eprescribingtoolkit.com/interact/.
3. Brian Power. Lessons from 20 years of implementation. Electronic Prescribing in
Hospitals: Moving Forward; Manchester Conference Centre2014.
143
Outcome 7: Contact details of electronic prescribing leads from a
cross-section of Trusts.
Afzal Chaudrey
Cambridge University Hospital NHS Foundation Trust
Layla Campbell
Newcastle upon Tyne Hospitals NHS Foundation Trust
Anna Bunch
University Hospital Southampton NHS Foundation Trust
Brian Power
Wirral University Teaching Hospital NHS Foundation Trust
144
Outcome 8: The training strategies for newly employed prescribers
within Trusts. Including;
o Is the training supported by the companies that provide the
systems? If not, who provides the training?
o How is the training facilitated?
o Lessons learnt from trusts training Foundation doctors and
other prescribers on induction. For examples is there
increased technical support when foundation doctors start?
o What common issues arise when training prescribers on
induction?
o The resources and cost associated with the training
145
Site A
Is the training supported by the companies that provide the systems? If not, who
provides the training?
Due to the limited capacity of the existing hospital information technology (IT)
trainers to deliver Trust wide training during the initial roll-out phase, the system vendors
provided very basic and broad training to end-users. Training of the hospital informatics team
during the initial implementation phase was delivered on-site by the company vendor over a
‘couple of days’ and also by an additional partner organisation with extensive experience of
using the system. This organisation deployed a trainer (for periods of two weeks) and also
conducted training via the web (using skype or similar) to provide guidance and support.
Since implementation, end-user training has been delivered entirely by hospital Trust staff
from the IT department; this includes designated IT trainers, an informatics pharmacist and
informatics pharmacy technician, medication safety pharmacist and medical staff involved in
the training of foundation trainees. All ancillary training material for example ‘how-to
guides’, videos and PDFs were developed internally. Typically these are developed by the IT
department and then reviewed by the relevant clinical team for approval prior to being
released to the Trust staff members.
How is the training facilitated?
Implementation
During the implementation period external and internal trainers (see above) delivered
classroom based training sessions to end-users. In addition to the formal Trust trainers,
‘Super-users’ were also recruited. Super-users were clinical staff including matrons, nurses
and some medical staff recruited to provide hands on and in-depth knowledge of the system
to ward based end-users. The super-users shadowed formal trainees; completed classroom
based training sessions and comprehensive training modules to learn about various
components of the system. This approach was considered to be largely unsuccessful, possibly
due to a lack of staff who were engaged and enthusiastic about the system and
implementation process, furthermore not all staff roles were represented for instance a
member of the pharmacy team could not be recruited. Evening sessions were also held in an
attempt to increase training attendance, however these were poorly attended. A large
proportion of implementation training occurred on the ward with ‘hand-holding’ of staff
whilst performing particular tasks.
Junior Doctors
The Trust induction of junior doctors takes place over a one week period, as part of this
they will attend a mandatory session on the ePrescribing system. An initial one hour lecture is
held to introduce end-users to the system, provide them with an overview of the layout,
highlight errors and problems that have occurred during use of the system and demonstrate
appropriate use of the system. Following this, end-users must attend a two hour classroom-
146
based session in a computer suite. During the first hour end-users are granted access to a
training domain, which provides a safe environment for learning how to navigate and
complete tasks on the ePrescribing system on dummy patients and simulated clinical
exercises (e.g. how prescribe a STAT (one off) dose of an antibiotic). The aim of the first
hour of classroom based training is to provide end-users with an in-depth overview of the
system and prescribing processes. During the second hour a ‘prescribing trial’ is performed,
this is not an assessment with a pass or fail mark but is designed to inform trainers about end-
users who may benefit from additional support. The trial includes patient scenarios and
specific tasks that must be completed, such as managing the care of a patient admitted for
elective knee surgery. This would include prescribing the patients regular medicines,
prescribing a STAT dose, amending dosage times, prescribing a treatment dose of Tinzaparin
based on weight, prescribing warfarin and amending the dates of this prescription. The
specific tasks are based on known problem areas that have been associated with previous
errors or involve high-risk drugs. The trials are marked and scored and feedback is provided
to end-users. There is also an overview lecture provided towards the end of the induction
week to summarise the training and enforce key learning points, particularly those related to
safe use of the system. Additional training material is also provided, including a paper and
electronic version of a hand-book outlining how to use the system and perform certain
functions, a top-tips guide, which includes brief ‘how-to’ notes about the system (e.g. how to
customise the computer screen) and access to multiple resources available on the Trust
intranet such as video demonstrations about how to prescribe, administer and order bloods.
There is currently no provision of e-Learning or online training methods. There has been a
conscious decision to move towards a model, which would see the provision of basic training
from the Trust IT department and the more complex understanding of the system to come
from the respective clinical areas in which the newly-employed end-user will work. This is to
allow more specific training to occur, such as knowledge of particular order-sets and
workflows which are beneficial in certain clinical areas.
Newly-employed Prescribers starting outside of main August induction
Non-mandatory training sessions are held on a monthly basis for newly-employed
prescribers who begin working for the Trust outside of the main August junior doctor
induction period. IT trainers hold a 1 ½ hour session called ‘clinician essentials’, which
covers basic system functionality such as how to log onto the system, how to order tests,
bloods and the basics of prescribing. The IT trainers are not clinicians therefore if there are
any queries that they are unable to address, these can be directed to a member of the
pharmacy informatics team for resolution.
Lessons learnt from trusts training Foundation doctors and other prescribers on
induction. For example is there increased technical support when foundation doctors
start?
147
1. Timing of Training: if training takes place too soon before prescribers are expected to
use the system it is possible that they will forget how to use the system. Conversely, if
the ePrescribing training is delivered at the same time as other training, it is possible
that end-users may become overwhelmed and unable to spend the necessary time to
learn the system functionality, therefore the time at which training takes place should
be carefully considered.
2. Training should be mandatory: this will ensure that a minimum level of training is
delivered and that time is protected to allow end-users to practice on the system, even
during a busy induction week.
3. Practical Training Exercises: The practical training element in the classroom based
sessions was considered to be the most effective. Lecture sessions, whilst useful for
delivery of information to large numbers of new starters pose the risk of overloading
end-users with information, particularly if the lecture falls within a longer day of
induction sessions.
4. Feedback: Feedback should be obtained from different clinical areas about their
preferences for system changes and what skills they would like training to focus on.
5. Super-Users: If super-users are to be used they should be engaged and demonstrate
commitment to their continued involvement as a ward-based trainer. It is therefore
important that sufficient information is provided about the system and the
expectations of the super-user role, to allow staff to make a conscious decision about
their participation.
6. Ward-Based Training: Tailored and context specific training is important. The
delivery of basic ePrescribing training, for example how to log onto patient and how
to prescribe is suited for delivery from the Trust IT department, whilst the more
complex training about the system may be better suited to delivery by the ward.
Efforts are being made to try to focus the responsibility of training onto the specific
directorates in which the newly employed prescribers will work, as the staff there
should be best placed to provide specific and tailored training about particular order-
sets, workflows and problem areas.
What common issues arise when training prescribers on induction?
A range of issues arise during the training of prescribers during induction and are also
associated with ongoing use of system. There were reports of logistical difficulties related to
booking rooms and staff availability, particularly as the majority of new-intake training
typically occurs during the school holidays. This may have an effect on the range of support
given, for example in August 2015 it was not possible to hold drop-in sessions due do to
staffing issues. The process of setting up personal accounts for new employees and the
creation of training domains can be problematic, as often the IT training department does not
have the final list of new employees until relatively close to their start date, which creates a
backlog of work. The training material should reflect the system that end-users will actually
use, therefore difficulties may arise if system changes are planned but are not fully
incorporated into the training exercises. Issues were discussed around the amount of training
provided, in particular the lecture sessions may overload end-users and there is the risk that
148
little knowledge is gained during that time. Certain issues with carrying out particular
prescribing tasks were also identified, these include: rescheduling administration times,
ensuring regular review of short duration antibiotics due to a potentially confusing
component of the ePrescribing system display and how to prescribe warfarin.
The resources and cost associated with the training
The participant found it difficult to quantify the resources and cost associated with
training. There is a dedicated IT training team that are responsible for the development and
delivery of all training. The junior doctor yearly induction training on the ePrescribing system
typically takes about two weeks to prepare for a couple of training staff plus extra time for
delivery of the sessions. Since implementation, a large proportion of the training material has
been re-used and recycled with updates, thus maximising staff resources.
149
Site B
Is the training supported by the companies that provide the systems? If not, who
provides the training?
The training sessions are delivered by a dedicated team of internal Trust trainers who
are also involved in the configuration of the system and therefore have an excellent working
knowledge of system functionality and so are able to incorporate this into the training
provided. Most of the trainers will have some form of clinical background and some
experience of adult education as this was a prerequisite for the role. This was a conscious
decision as it was felt that end-users would value training that was being delivered with the
most appropriate skill set. Although the training is delivered by internal staff, much of the
original training material was provided by the company provider, for example screen shots of
how to perform tasks. Overtime this material has been progressively customised and tailored
to the specific needs of the Trust. Furthermore the format of the sessions was decided upon
by the Trust trainers and adjustments were carried out based on feedback. Refresher training
sessions are delivered by members of the Trusts own informatics team, efforts are made to
correspond the trainer’s expertise with that of the end-user, for example a trainer with a
clinical background of medicine would provide refresher training to doctors. The trust is also
in the process of developing some supplementary online training material with an external,
local electronic publishing company. The content is provided by the Trust, however the
publishing company are providing assistance on how to best deliver that.
How is the training facilitated?
The Trust currently employs a mixture of classroom sessions and ancillary material in
order to train end-users on the system. The classroom sessions consist of an initial four hour
period, which begins with a short introduction from the trainer about what the end-users
should expect to learn and a brief demonstration of a projected version of the screen.
Following this, the end-users will work through workbook exercises, which will focus on
caring for a patient and the individual tasks that will need to be performed as part of that. The
exercises and patient scenarios deliberately focus on the entire workflow of caring for a
patient rather than the individual tasks, so that end-users are learning how to perform the
specific skills in the context of a patient and realistic workflow. Specific processes include
basic tasks such as admitting a patient, prescribing their initial drug chart, starting and
stopping medicines and discharging a patient, and also more complex tasks such as
prescribing anticoagulation, insulin and fluids, The content is developed by the informatics
training team in collaboration with the lead pharmacist for education, who is also part of the
committee for safety of medicines to ensure that the training is appropriately focused and
covers areas where there has been previous incidents or known high-risk areas. Each end-user
works at a computer-station in a safe learning environment, which is separate to the main live
version of the system. The ‘playground environment’ is populated with hundreds of dummy
patients to facilitate the training session; once end-users have their password they are able to
access the system at any time from a hospital computer, in order to gain additional
experience. Over the subsequent few weeks the end-users are brought back to receive some
150
additional training in a phased way so that large cohorts of staff are not removed from clinical
service all at once. The classroom sessions are accompanied by a handbook, which is specific
to the staff role, and tip-sheets, which are short documents that focus on specific tasks which
are available on the Trust intranet 24 hours a day. There is also a short assessment at the end
of the classroom based session designed to facilitate learning and identify individuals who
may benefit from additional support, either in the form of some extra guidance at the end of
that day or they may be asked to return on another day to receive some further training. In
addition to the core classroom sessions the Trust has recently implemented refresher sessions,
these take place first thing in the morning so that end-users can attend a session covering a
few quick exercises before work. As above, these refresher sessions are held by trainers with
a similar clinical background to those who being trained, based on the observation that people
respond better to teaching from people that they identify more closely with.
The Trust is also in the process of developing some online learning material. The current plan
is to use the online training as a supplementary session alongside the core classroom based
activities in two possible ways. Possible uses of the e-learning include firstly, an opportunity
for end-users to log in and familiarise themselves with the system before they start working
for the Trust and receive the more formal training sessions, and secondly to serve as a
summary after the classroom training has been completed, in order to reinforce important
points and again identify individuals who would benefit from additional support. One
potential function of the online training is to address the needs of those working on weekends
or those that may be having some difficulties using the system on a day to day basis that was
not identified during the core training.
There are also expert mentors in place for nursing staff, these staff have received
comprehensive training from the Trust informatics team, and are therefore able to deliver
training to other staff on some elements on the system at their local ward meetings, which has
been considered to be a successful approach. There is a similar process, although less
structured currently in place for medical staff whereby a small number of consultants have
volunteered to act as expert mentors and have ran training or drop-in sessions on an ad-hoc
basis. During junior-doctor induction periods the Trust has also operated a help-desk, staffed
by members of the informatics team to provide on-demand support to those newly employed
prescribers, including on weekends and nights when there are typically fewer staff available.
Each Monday a training session is held for newly employed prescribers who begin working
for the Trust outside of the main August intake. The Trust implemented a policy to ensure all
new-employees begin working for the Trust on a Monday so that they can be reliably offered
training. The training delivered in the same 4-hour training session as previously described.
Locum staff are traditionally a problematic area for the Trust training department to manage.
It has been agreed that staff who will be working for one week or longer will attend the
regular training session. However locum or agency staff who will be working for a shorter
period should attend early for work and meet with a trainer where they will take part in a
short 1 ½ hour session to cover the core principles of the system. An attempt has also been
151
made to recruit staff from a limited selection of agencies, to build up a bank of trained staff to
reduce the training that needs to be provided.
Lessons learnt from trusts training Foundation doctors and other prescribers on
induction. For example is there increased technical support when foundation doctors
start?
Workflow Orientated Training: The original training strategy was task orientated, i.e. very
literal training about how to carry out specific processes such as ‘how to prescribe’ or ‘how to
stop a drug’. The Trust decided to move towards a workflow orientated approach to training,
introducing end-users to the individual tasks in the context on an actual patient. This
approach has been more successful, and the training was found to ‘make more sense’ to end-
users as they were able to follow the entire care of a patient during the training sessions.
Focus on Problem Areas: The Trust has also adapted the training to include a greater focus
on known problem areas of the system and more complex prescriptions, such as warfarin,
insulin and fluids. In addition the training also provides information to end-uses about the
challenges and idiosyncrasies of the system, such as the need to click certain buttons or tick a
particular box for items to be processed. The trainers reported telling their trainees “this is the
electronic manifestation of what you used to do in your head, it might come across to you as
an irritation or a quirk of the system but it is actually an important thing for you to do because
if you don’t record it properly later down the line, the pharmacists or you know the
dispensing, will have considerable problems.”
Length of Training: The training content was revised so that a shorter session could be
delivered, which was more agreeable to end-users.
Feedback: Feedback was vitally important for improving and tailoring the training provided.
For example there were parts of the training that all end-users grasped very quickly and other
areas that took far longer for people to understand. Similarly, certain processes had to be
added to the training as end-users experienced difficulties that were not anticipated.
Mandatory Sessions: Training was mandatory and end-users who fail to attend are referred to
their managers.
Planning: Ensure there is a robust plan in place, particularly when training large cohorts such
as during the junior doctor induction period. Excellent communication with medical staffing,
estates and facilities is paramount to ensure the sessions go ahead smoothly.
What common issues arise when training prescribers on induction?
Difficult Prescriptions: Orders for anticoagulation, insulin and fluids require a greater focus
in training to ensure that end-users are competent in these processes.
152
Targeting the training at the right level: It can be difficult to ensure that all relevant areas are
covered during the training sessions. This may be particularly relevant if the system has been
newly implemented and the trainers and existing end-users lack confidence and knowledge of
using the system. In this case the Trust relied upon regular evaluation and modification of the
training content.
The resources and cost associated with the training
There are 15 full time trainers employed by the Trust, which are either on a Band 6 and Band
7 pay scale.
Development of the training material and involvement in the system configuration is an
extensive task and during the implementation phase required all staff to work on this full time
for ‘many months’.
Currently trainers spend approximately 75% of their time on training and 25% of their time
on refining, adjusting and updating the training material.
153
Site C
Is the training supported by the companies that provide the systems? If not, who
provides the training?
The core e-learning training was designed and built internally within the Trust. For those that
are unable, or have difficulties with the e-learning package classroom based sessions can be
provided by an internal member of the training team. The training team consists of members
of the prescribing and pharmacy management directorate. The company provided super-user
training during implementation and training of the informatics team who were involved in the
system build; they have not been involved in the development or delivery of end-user
training.
How is the training facilitated?
The training is entirely e-learning based, this comprises of a 2 ½ hour course, which must be
must completed before end-users are granted access to the system. If problems are identified
end-users can be individually supported. There are 22 modules covering all aspects of how to
use the system, for example finding a patient, adding allergies, adding heights and weights,
generating prescriptions and modifying prescriptions, discharging a patient for one week
leave and a section at the end for troubleshooting about certain tasks that cannot be
performed on the system and how these should be carried out. More complex aspects of the
system are also covered such as prescribing and modifying ‘care bundles’ (groups of drugs
ordered together for a particular condition) and also signposting end-users to parts of the
system such as antibiotic guideline policies. The e-learning module begins with an
introduction of the specific task and what they will be expected to learn and then takes the
end-user through exercises, which incorporates screen shots of the ePrescribing system. Once
all modules have been completed end-users must complete a summative assessment at the
end which carries a pass mark of 90%. The assessment comprises of scenarios related to the
care of a patient; a typical example would begin with identifying a newly admitted patient
and prescribing their regular medicines, then following a few days as an inpatient the end-
user must review their current medicine and will be required to modify the regime, ending
with tasks related to discharging the patient. The scenario is designed to be as realistic as
possible and therefore the end-user is presented with details about the patient’s medication
history, background and a rationale for prescribing certain items. The patient scenarios
represent a patient from a medical background and a surgical. There is currently no provision
for the delivery of more specialist patient scenarios, alternatively the lead specialist
pharmacist for that area would deliver a specialist training package which is outside of the e-
learning domain. For example, end-users who will work in paediatrics must complete a
specialist paediatric package, containing exercises for both adult and paediatric patients so
that end-users are aware of the differences when prescribing. The e-learning packages, would
ideally be updated on a yearly basis, and with every system update and change made.
154
For those who experience difficulties with the e-learning, classroom based training can be
offered, this is essentially e-learning in a classroom accompanied by an internal trainer to
provide additional support. End-users with poor IT literacy are offered one to one training but
this is again e-learning based
During the junior doctor induction week, a speed-date type induction session is held by the
trainers. Trainers spend approximately 10 minutes with small groups of doctors, and advise
them about the e-learning, how to complete it and are also, provided with handy-hints about
the system. During the induction week junior doctors also attend a meeting with the medicine
safety team, where the key safety issues that have occurred with the system (and unrelated to
the system) are discussed.
As part of risk management strategy a handy hints booklet is circulated to end-uses, this
includes information about areas that are not covered in enough detail during the e-learning
modules or are not covered at all and other areas that end-users need to take care of when
they are prescribing. Handy hints are provided as a print out, the Trust originally trialled
emailing it to doctors but this was unsuccessful and the trainers doubted whether the hints
were ever read. It was therefore decided to provide end-users with a booklet with a cautionary
note about the importance of reading the content. Particular topics are selected based on
frequent enquires from staff and e-prescribing related incidents, and are emphasised during
the training, examples include warfarin prescribing and STAT doses. There are
approximately two handy-hints updates each year.
All pharmacy staff are super-users who are able to support the ward staff.
Pre-implementation, staff were required to complete an IT literacy test as part of their
statutory and mandatory training, this informed managers about individuals who may require
additional assistance so that these could be targeted by the informatics team trainers if
needed. Typically a maximum of one or two staff per ward would require additional IT
training. The additional training provided would consist of a classroom based session where
end-users could complete the e-learning supported by the trainers. During the initial stages of
implementation, some individuals were identified with very poor IT skills for example they
were not able to use a computer mouse, such individuals were therefore supported on a one to
one basis. The Trust also paid for external ward based ‘floor-walkers’ during the induction
period to provide support. This approach was not considered necessary or efficient post
induction, as existing-ward staff are now familiar with the system and able to provide
necessary guidance.
A training session is held for training new staff, who begin working for the Trust outside of
the main August induction on the first Wednesday of each month. New employees who begin
after that date will attend the next possible training session and as the e-learning is part of the
statutory and mandatory training the Trust is able to identify individuals who have not
completed the assessment.
155
Lessons learnt from trusts training Foundation doctors and other prescribers on
induction. For example is there increased technical support when foundation doctors
start?
e-Learning accessibility: The e-learning can be accessed 24/7 and is also available externally
to the Trust it is therefore is very easily accessed by staff and there is no constraints on PC
availability or staff time. There may be some issues from staff who may be asked to carry out
training outside of working hours and as a result some local areas may offer to reimburse this
time. For some staff groups such as junior doctors thee-learning is part of their mandatory
education and training and therefore must be completed ideally during designated education
hours, however if they are unable to it is their responsibility to complete this training during
their own time.
Role-Specific Training: The initial training plan comprised of three sets of training depending
on whether the clinician was either a ‘prescriber’ or ‘administrator’, however the Trust
discovered that this did not reflect all end-users, for example Nurse prescribers who would be
expected to both prescribe and administer. Similarly not all prescribers would carry out all
functions covered during the ‘prescriber’s training’, this was a particular issue for some
surgical consultants who would never write a discharge prescription and were therefore
frustrated at the prospect of conducting 2 ½ hours of e-learning. As a result the modules have
now been split into groups depending on the role, for example consultants may only need to
complete modules 1-6, and 8 and 10, which is both more specific to their role and shorter in
duration. Each role-specific training stream has their own dedicated assessment based on the
selection of modules that they were required to complete
Duration: The e-learning should be modified to reduce repetition to reduce the length of the
training. Therefore if the end-user has previously demonstrated that they are able to identify a
patient, they will not be asked to carry out this process again.
Mandatory Training: Access to the e-learning system is only granted following successful
completion of the assessment, which ensures all end-users reach a minimum level of
understanding about the system.
Mirror actual practice: If possible when multiple systems are used in combination for
instance an ePrescribing system and a laboratory results system, the training should
incorporate use of both systems and how they interact and can be accessed. Similarly training
should include realistic patient scenarios and workflows which reflect normal practice on the
ward.
Preparation: During the junior doctor induction week all training staff are involved in the
set-up of end-user accounts, advance planning and preparation with human resources is vital
to ensure that trainers have all the necessary information required.
156
What common issues arise when training prescribers on induction?
System issues: Specific problems related to the ePrescribing system can have an impact on
the delivery of training. The Trust has delayed updating the e-learning material since
implementation in anticipation of a system update, which would fundamentally change the
system. Instead of potentially wasting staff time making changes to the existing e-learning the
handy-hints guides have been issued. The system itself has also been associated with some
problems and issues surrounding usability, as a result the Trust has found it important to
gather feedback and involve end-users in the process, however there a limited amount of
changes that can be made to the actual ePrescribing system. The Trust recommends providing
monthly ‘listening sessions’ which will allow end-users to voice their concerns.
Negotiating training staff-leave during junior doctor induction: Due to the high volume of
work that occurs during this period, training staff should expect to work longer hours and
annual-leave is blocked during peak training periods.
Communicating with Doctors: The Trust has found it particularly difficult to reliably
communicate with Doctors about issues with the system and any changes that will affect
them. Doctors are emailed with such information, however these are not consistently read.
Resistance to training: Some clinical groups were resistant to the e-learning because of the
length of time that it would take and due to a lack of perceived relevance. It became apparent
that due to this resistance there were groups of very senior clinicians who had not completed
the e-Learning material. Subsequent changes were made to the e-learning content to make it
more agreeable to these groups, as described above.
Duration: The training is very detailed, however it has been criticised for being too long and
end-users become frustrated with the system. There was a request to reduce the entire training
to just 20 minutes, and whilst this is highly unlikely efforts are being made to create more
tailored, relevant and therefore shorter training.
e-Learning issues: The e-learning system can be cumbersome and overly specific, for
example end-users must correctly click the exact part of boxes during the assessment and if
the end-user accidentally miss-selects an answer they only have three chances before they
must re-do the entire assessment.
Locum staff: Ensuring locum staff are adequately trained to safely use the system has been a
particular source of difficulty for the Trust. Nurses are now always booked through a select
pool of agencies, which have agreed to only supply nurses who have completed the e-
learning training. Sourcing locum Doctors who have completed the e-learning training is
more challenging as there is no central booking point from which locums may be selected
from. As no central booking point exists the Trust was unable to put a training agreement in
place and subsequently could receive untrained doctors. As a result the Trust encountered
examples of untrained locum doctors receiving log in details for the ePrescribing system from
157
the exiting Trust doctor and then was able to prescribe under another doctors name. A
solution was required, and as the Trusts main objective was to ensure that doctors were
working under their own username a process was set-up whereby locum doctors are now
provided with a username and access to all Trust systems. Locum doctors must now contact
the prescribing team, who will arrange to work through the usual e-learning assessment with
them (rather than the full modules) ensuring that basic skills are covered such as how to
prescribe and how to admit a patient. Locums who are due to work within the Trust for longer
periods, for example over 3 nights, are expected to do the full e-learning.
The resources and cost associated with the training
The training team comprises of five staff members, which is equivalent to having 2 ½ staff on
a band 5 pay scale. The team are responsible for the organisation and development of the
induction material, handy-hints, new-starter set-up and the e-learning material. The e-learning
package was designed internally by an e-learning team in combination with the Trust trainers
and is estimated to have cost approximately £25,000.
158
Site D
Is the training supported by the companies that provide the systems? If not, who
provides the training?
The training has been designed and developed internally by the Trust, although advice from
the system supplier was available if required. The training of doctors, nurses, clerical staff
and the majority of allied health professionals is delivered by a team of informatics trainers
from a range of clinical and non-clinical backgrounds. Pharmacy staff are trained exclusively
by pharmacy members of the informatics team. In addition to training, the informatics team
are also involved with analysing and developing the electronic patient record system.
How is the training facilitated?
All training is mandatory and must be completed before end-users are provided with a login
for the system. The system is a fully integrated patient record, which includes electronic
prescribing functionality and therefore training must include all aspects of the system, such as
information for clinicians about how to prescribe, administer medications, order lab tests and
record basic observations.
Junior doctors and newly-employed doctors receive training as part of their Trust induction.
Non-medical prescribers receive a level of training, which is dependent on their previous
experience with the Trust system. For example, existing staff nurses who receive a
prescribing qualification are required to undertake a top-up training session that specifically
focuses on the prescribing process. Newly employed non-medical prescribers however
receive the same full training as a doctor.
Formal training is provided for all clinical staff (doctors, pharmacists and nurses); this
comprises of hands-on training in a computer suite where all end-users will have access to a
training domain version of the electronic patient record system. The training domain is a copy
of the live system, which allows end-users to become familiar with the system in a safe
environment on ‘dummy patients’. Although efforts are made to ensure that the training
domain exactly reflects the live system, last minute developments may result in some slight
differences being present during the training session. Training is typically delivered in small
groups (up to 8 to 10 end-users). The content of the sessions is tailored according to the staff
member, using specific examples for doctors, nurses or pharmacists, and specialist areas such
as paediatrics. End-users will work through various topics and exercises, for example during
the prescribing component the trainers will cover how to access the drug catalogue, how to
prescribe inpatient medication and discharge medication and for nurses how to administer
drugs under a patient group direction. The training also covers some of the typically more
complex prescribing processes such as: how to include a course length for antibiotics,
prescribing for paediatric patients of a particular age and weight, IV infusions, prescribing a
reducing course of steroids and prescribing warfarin, insulin and other variable dose
159
regimens. The content of the training sessions is informed by trainer experience and incident
reports, which highlight the need to cover certain topics in more detail. For example, since
the Trust changed their electronic system the training team identified a need to educate
doctors about the importance of checking that they are prescribing for the correct patient
encounter to ensure that the prescription is available for nurses to administer ( e.g. prescribing
for a ward based patient under an ‘outpatient’ encounter would not generate an order for the
nurses to administer). The Trust schedules training sessions at various times of the year to
coincide with planned surges in new employees such as at the junior doctor intake in August;
all other training is organised on a need-by-need basis.
Crib-sheets are provided to end-users via the Trust intranet; these cover a variety of functions
on the electronic patient record including how to prescribe and order lab tests. It is not clear
how beneficial the crib-sheets are and what the level of uptake has been; anecdotal
experience suggests that clinicians tend to rely upon their peers for support rather than use the
supplementary material available. There is currently no provision of online or e-learning
material. Although there are training plans and learning objectives, which must be addressed,
there is no formal assessment of end-users prior to obtaining access to the system.
During day to day use of the system there are no designated super-users, however the
pharmacy staff are informally regarded as system experts and provide knowledge and
expertise to the entire ward team on issues beyond the purely ‘pharmacy’ functions. The
pharmacy team have received extra training in order to assist them with this role. During
implementation and major system updates the Trust attempted to provide some super-user
support, however recruitment of such staff was challenging due to staff availability.
Furthermore due to the nature of shift-work ensuring that super-users are available when
needed can also be problematic.
Individuals who may benefit from increased support, particularly those struggling with basic
computer skills, are identified either during the training sessions by the training team or by
ward based colleagues. Additional training sessions or ward based support may be given to
such end-users.
The training of locum doctors is handled differently compared to Trust employed staff. The
current policy stipulates that locums should receive a short run-through of the system and
how to perform certain tasks from the responsible consultant in the clinical area where the
locum will work. Following this, they are issued with a temporary password to the system
that will be suspended when they finish a shift or period of work. The Trust is currently in the
process of reviewing the locum training policy due to issues with management of the
temporary codes and the robustness of training provided.
Lessons learnt from trusts training Foundation doctors and other prescribers on
induction. For example is there increased technical support when foundation doctors
start?
160
Modernisation: The training material should be regularly reviewed and updated according to
end-user feedback and experience and also following any changes to the system. Trainers
should be informed of any changes made to the training with a clear rationale to inform the
delivery of their sessions.
Availability: A robust system for booking training sessions is necessary. Additionally,
training should be held at an appropriate time, to prevent end-users from forgetting the
content covered. Advance planning is also important to ensure that there are sufficient, fully-
equipped training rooms. The informatics team benefited from the support and understanding
of senior management regarding the need for hands-on training, which resulted in designated
informatics-owned training rooms throughout all hospital sites. The training domain is also a
vital component of end-user training and therefore must be available for end-users.
Consistency: The individual training sessions should be reviewed to ensure that sessions are
being delivered consistently. A reliable process should also be in place to inform trainers
about the system and any changes that have been made in to inform their training.
What common issues arise when training prescribers on induction?
Training discrepancies: There have been instances where the training material and training
domain did not reflect the live version of the system, which can be a source of end-user
frustration. Similarly IPads are increasingly been used on the wards by clinicians and
although the ePrescribing system display closely resembles that used in the training session
the trainers are discovering that some members of staff are having initial difficulties with
navigating and using the IPad. It may become important in the future to consider the devices
used during training sessions so that the end-users can familiarise themselves with both the
system and device, and associated problems can be identified.
Prescribing difficulties: Prescribing difficulties include complex prescriptions for variable
doses, IV fluids, paediatrics, variable dosage regimens e.g. insulin, warfarin and steroids and
paediatric prescriptions.
Quantity of content: As the system in use is a fully integrated electronic patient record there
is a large amount of material and training to be provided over a short time period and there is
a high risk of overloading individual end-users.
Organisation of sessions: Organising sessions and trainer availability can be problematic,
particularly during busy training periods such as August. Excellent planning and receiving
early notice from clinical areas about start dates can facilitate organisation.
Staff-involvement: Past attempts to recruit medical and nursing staff as super-users have been
problematic due to staff availability.
161
Trainer expertise: Training of doctors and nurses may be provided by non-clinicians, whilst
this has cost advantages and they are typically very skilled teachers, the trainers may lack the
context and expertise to answer certain questions. The exception to this is the pharmacy team,
which are trained by pharmacy informatics trainers.
Locum doctors: The training of locums is a challenge; issues with the current policy include
management of the temporary codes, for example knowing which individual has used certain
codes and also issues around the quality of the training provided. Consultants are expected to
deliver 45 minutes of informal training to locums in their clinical area; however the
informatics team acknowledge that this quantity of training is unlikely to occur due to the
busy ward environment.
The resources and cost associated with the training
The participant does not have access to the details required in order to answer this question.
However as an estimate, there is approximately 6 staff who act as trainers and have additional
responsibility for the development and analysis of the system. The pharmacy team is trained
by designated informatics pharmacy team members. During implementation and system
update periods there was extensive development of training material, however the day to day
maintenance of the training material is not a major issue for the informatics team providing
changes are managed in a timely fashion.
