WEST MIDLANDS CENTRAL HEALTH INNOVATION AND … · 2019-02-13 · from the data previously referred to, almost all newly qualified doctors have done very few such examinations. Reasons

Simulation topic

Final report

Author:

Jim Parle, Professor of Primary Care

University of Birmingham

August 2013

Contact for correspondence: [email protected]

WEST MIDLANDS CENTRAL HEALTH INNOVATION AND EDUCATION CLUSTER

(WMC HIEC)

ACKNOWLEDGEMENTS

Thanks are due to the following people and organisations for their contribution:

The Simulation project group and Simulation Centres across the West Midlands

The research team were as follows:

Professor Jim Parle, University of Birmingham

Matthew Aldridge, Senior Lecturer, University of Wolverhampton and Birmingham City University

Jackie Beavan, Research Fellow, School of Health and Population Sciences

Karen Reynolds (Barry), Manager of the Interactive Studies Unit, University of Birmingham

Dr Sharon Buckley, Senior Lecturer in Medical Education, University of Birmingham

Dr Ian Davison, Research Fellow, Medical Education, University of Birmingham

Dr Sandra Cooke, Medical Education, University of Birmingham

Mr Edward Davies, Royal Orthopaedic Hospital, Birmingham

This research was funded by:

The West Midlands Central HIEC (April 2010 – March 2013) which was funded by the Department Health

Contents Page

Introduction 1

Supporting the CKD theme 2

Accreditation and certification of existing simulation providers 3

Procedural skills passport 3

Randomised Controlled Trial in Orthopaedics report 4

Survey of newly qualified doctors’ skill acquisition in intimate examinations 4

CIPSET – business plan for network of provision 6

Programme grant application 6

Appendix 1: report on accreditation of simulation centres 7

Appendix 2: development of the procedural skills passport 64

Appendix 3: report on Randomised Controlled Trial in Orthopaedics 70

Appendix 4: Results of intimate examination survey of newly qualified doctors in UK; (199 respondents) 72

Appendix 5: CIPSET report 75

12 WMC HIEC Simulation Report 1

Introduction

Simulation is of course a methodology of teaching and learning, not an end in itself.

However, it can be argued to be a more moral method than the traditional ‘see one, do one,

teach one’ of medical myth (see Draper et ali), and a priori to be safer (since patients are not

put at direct risk). Classifications of simulation include simple/complex; low-tech/high tech

(also sometimes inaccurately called ‘high fidelity’); low fidelity/high fidelity; people

based/technology based/hybrid; individual skills development/team training; and so on. In

the context of the WM HIEC a simulation theme was established to:

• Support the CKD theme as appropriate: dealt with mostly in the industry theme

report, elsewhere in this HIEC report, but see also below

• Develop links between NHS, industry and universities to develop simulations which

could improve patient safety through improved and safer training

• Support and further develop the multi-professional learning system known as CITEC,

to evaluate, disseminate and mainstream this approach: dealt with in the CIPSET

report below

More specifically, simulation work packages were developed as follows (note that the

individual work packages developed partly in response to early development work):

• Accreditation and certification of existing simulation providers

• Procedural skills passport

• RCT Orthopaedic report

• Survey of newly qualified doctors’ skill acquisition in intimate examinations

• CIPTEC – business plan for network of provision

• A programme grant application in intimate examination skills acquisition involving the

development (with industry) of new haptic training devices for performing intimate

examinations.

The HIEC also aimed to develop protocols for further simulation research to engender a

continuing flow of work related to simulation in education (focussed on pre-qualification

education and training, but not exclusively so). This report will also therefore refer:

• A quantitative and qualitative investigation of medical students’ acquisition of intimate

examination skills

• Development of further research for example through a programme grant application


Overall these themes are themselves incorporated into the development of the programme

grant application, focussing on the identification of prostate cancer, building on the data on

intimate examinations referred to above as well as the developing collaboration with the

NHS and industry (which is currently focussed particularly on a renal biopsy trainer).

Supporting the CKD theme

As has been mentioned, simulation is a method and not an end in itself. However, in the

context of such procedures as renal biopsy, the ability to perform a biopsy in simulation (so

long as the simulation is realistic) is a priori safer for patients. East Midlands HIEC with UK

Haptics (now known as Jasmine Media) and WMHIEC are collaborating in developing such

a training programme, and thus far a biopsy needle ‘kit’ has been developed and this will

lead to full simulation using haptic simulation technology and simulated ultrasound (see the

figure below) being available for trialling within the next 6-9 months. Contracts have been

signed and the work is ongoing.

Figure 1:


Accreditation and certification of existing simulation providers

This work was largely performed by a subgroup, led by Matthew Aldridge of the University of

Wolverhampton [UoW], with Karen Barry [UoB] and Sandra Cooke [UoB]) and had

substantial collaboration and input from NHS partners. The aim was to develop a pilot

approach to accreditation of skills centres which was not too burdensome, which when

possible utilised existing data (for example data which may have been collected for other

purposes), which had face validity and yet was sufficiently rigorous to be more than a simple

‘pat on the back’. Methodology was in essence to develop a template from the literature and

from existing tools, pilot it with collaborators, modify, and then hold a Delphi style group to

develop a consensus. The developing work has also been shared nationally, for example

with the major clinical skills groups. The tool is now ready for further development with other

partners across the UK.

The report can be found in Appendix 1

Skills passport

UoB medical school has been aware for some years of the need to develop better quality

control measures of medical student procedural skill competencies: the old ‘see one do one

teach one’ approach to procedural skills is no longer tenable! Over a period of two years or

so UoB developed a skills handbook with key themes: each skill should be core to newly

qualified doctors’ skill sets; a learning trajectory (see figure) would lead to supervised

practice and thence to what Cate has called ‘entrustable skillsii’ (i.e. procedures which

students could competently perform despite not yet being formally qualified)

As part of the WMC HIEC this work has been shared across the West Midlands and

modified versions of the passport are being used at Keele and at Warwick, significantly

facilitating the orientation of newly qualified FY1s into West Midlands Trusts. A draft paper

on the skills passport is at Appendix 2 and an electronic version of the passport, in which

students enter their progress (which the medical school is able to monitor) is in commercial

development.


Figure 2: the skills learning trajectory

RCT Orthopaedic report

The ability to perform an appropriate focussed physical examination is a core skill for a

doctor. In the context of musculoskeletal examinations (for rheumatological, orthopaedic or

trauma conditions) this may involve some pain/discomfort for the patient. In a collaboration

instigated by Mr Ed Davis of the Royal Orthopaedic, an RCT has been run over the past 20

months. The RCT is investigating if training medical students using simulation will improve

their clinical examination skills, so they can perform proficiently whilst minimising patients’

discomfort. We compared two different types of simulation: volunteer simulated and

professional simulated patients vs. Students attending clinics and wards as a control group.

The primary quantitative outcome measure is students’ marks on the objective clinical end-

of-year tests; secondary quantitative outcomes are self-reported numbers of patient

examinations carried out by students in their musculoskeletal attachment. We have also

conducted focus groups in March and April 2013 to explore students’ views on using

simulated patients.

The trial began in October 2011 and we finished collecting data in the summer of 2013.

Motor and Procedure Skills

Manikins and simulated training devices

Theory Indications, contra-indication, anatomy, physiology, equipment

Video, CAL materials

Integratingprocedural and communication skills and sensitivity to the patient

Simulated Patients e.g. GTAs

Transition to the clinical environment

Accompanied by Skills Trainer

Practice inthe clinical environment

Practice on Patients

Professional Development

Performance on Patients

Simulation training for refresher purposes

Learning Trajectory

Motor and Procedure Skills

Manikins and simulated training devices

Theory Indications, contra-indication, anatomy, physiology, equipment

Video, CAL materials

Integratingprocedural and communication skills and sensitivity to the patient

Simulated Patients e.g. GTAs

Transition to the clinical environment

Accompanied by Skills Trainer

Practice inthe clinical environment

Practice on Patients

Professional Development

Performance on Patients

Simulation training for refresher purposes

Learning Trajectory


Some 500 students have entered the trial and results will be available in spring 2014;

analysis is ongoing and a summary of the quantitative and qualitative results is available at

Appendix 3.

Survey of medical students’ skill acquisition in intimate examinations

As the HIEC developed its early thinking it became increasingly clear (anecdotally at least)

that there appeared to be a significant gap in the acquisition by medical students of intimate

examination skills (defined as female breast, male and female rectal, female pelvic, male

genitalia). Concerns had been previously expressed about other clinical skillsiii as well as

DRE skillsv. A simple web-based survey was distributed through the Scottish, Northern and

West Midlands Deaneries. (Note that particular concerns had been expressed by the

Northern Region PG dean that there had been significant local issues with the lack of these

skills in newly qualified doctors).

The results are greatly concerning and are summarised in the figures in Appendix 4; in

essence the vast majority of respondents (n approximately 200) had done less than 10 of

these examinations, and few had done more than 5. Further investigation of the literature

reveals no consensus as to how many such examinations are needed to attain competence

and to retain it, how quickly examination skills attenuate, and what are the best methods to

teach and assess such skills (see further discussion under programme grant developments

below).

A quantitative investigation into final year medical students’ skill acquisition in intimate

examinations (note the survey referred to above is of newly qualified doctors reporting on

their memory of such skills acquisition) has been recently completed; ethics approval has

been granted and the medical schools of Aberdeen, Barts, Birmingham, Brighton, Bristol,

Cambridge, Edinburgh, Glasgow, Imperial, Keele, Kings, Leicester, Liverpool, Norwich,

Nottingham, Oxford, Peninsula, Southampton, St. Georges, UCL and Warwick all took part.

Analysis is ongoing and will provide more accurate assays of the competence of each

medical school’s new doctors, allow comparison between medical schools, and help

determine if this phenomenon is related to patients’ unwillingness, or varies between medical

schools (and perhaps medical school cultures).


CIPSET – business plan for network of provision

For some years the West Midlands SHA has supported the development of inter-

professional learning across the region, and involving numerous Trusts and universities. This

work, led at UoB by Dr. Sharon Buckley, established an excellent ground for further

mainstreaming of IPL through real but complex simulations utilising both people-based

simulations and to a lesser extent technology, plus video and managed feedback by trained

facilitators. Further detail is provided in the CIPSET report in Appendix 5.

Programme grant application

It is becoming increasingly clear that although the technology for simulation is developing

rapidly, the knowledge to underpin its development, introduction and evaluation is sorely

lacking. The WM HIEC is therefore working on a programme grant application on the

Acquisition and Retention of Intimate Exam Skills (ARIES), focussed initially on learning

digital rectal examination (DRE) skills to examine the prostate for cancer. As can be seen

from the data previously referred to, almost all newly qualified doctors have done very few

such examinations. Reasons for this are not clear. The precise role and effectiveness of

haptics in developing these skills is also not clear. The mindmap below shows some of the

issues to be explored, using haptics and DRE as the central themes. If successful such

research will inform the development of learning for other intimate examinations and could

result in major industrial developments (since many countries [e.g. the Middle East,

Pakistan, Bangladesh, China, Japan]) have such strong cultures of modesty it is extremely

difficult for learners to acquire such skills, to the obvious detriment of patients.

Research methods would include:

• Quantitative (questionnaire) and qualitative (focus group and interview) investigations

regarding ethnic and cultural issues surrounding acquiring IE skills

• Measurement of students’ learning styles, 3-dimensional competence and dexterity

and correlation with acquisition of IE skills, particularly determining learning trajectory

and predictors of slow acquisition of IE skills, and remediation if needed

• Measurement of mature doctors’ IE skills and exploration of role of simulation in

maintenance and remediation of such skills if needed


• An RCT comparing haptics vs. manikins vs. simulated real patients vs. ‘usual

learning’ in acquiring DRE skills; primary outcome competence at IEs; secondary

outcomes to include number of IEs performed in clinical environment

Figure of mind map of developing programme grant

Appendix 1: Report on accreditation of simulation centres


REPORT ON SIMULATION THEME,

WM HIEC

Authors: Matthew Aldridge, University of Wolverhampton, Karen Barry and Sandra Cooke, University of Birmingham

(WMCHIEC)


Foreword

Clinical skills centres have increased in number and sophistication over recent years across

the UK and indeed the world, given a particular UK impetus by the Chief Medical Officer’s

report on simulation in 2009. However, development has been unstructured and

uncoordinated and in particular it is not clear that standards are equivalent across the many

different types of institutions running healthcare simulations. Some two years ago the West

Midlands Central Health Innovation Education Cluster identified that the issue of

accreditation of clinical skills centres was of central importance and this report is the fruit of

the work done by my colleagues Aldridge, Barry and Cooke, with admirable co-operation

from the centres involved as well as other interested parties and experts. They have

developed and piloted an accreditation process which is clearly appropriate and of value,

and is ready now for more widespread development. And of course it is applicable outside

the West Midlands. Anyone with an interest in ensuring standards in the skills centres in

which clinicians develop procedural and team-based skills will find this report of value. You

are cordially invited to help develop and validate this approach over the coming years.


Acknowledgments

We would like to thank everyone who was involved and helped in any way with this project,

especially those who attended the Expert Panel day on 30 September 2011 at the PDC,

University of Birmingham (see Appendix III for a list). We would also like to thank ASPiH and

our colleagues in Yorkshire and Humberside who contributed their documents to the

discussion on the day.

We are very grateful for the time and interest shown by the simulation centres who took part

in our pilot accreditation process, and we appreciate the commitment and energy they put it

into the process.


Contents Page

• Section 1: Introduction and context 12

• Section 2: Literature review 13

• Section 3: Design and methods 19

• Section 4: Results 21

• Section 5: Discussion 31

• Section 6: Conclusion and Recommendations 33

• Section 7: References 35

Appendices

Appendix A: i) Literature review search terms 39

ii) Draft benchmarking tool 41

iii) Attendees at the expert panel 2011 52

iv) A Report on a colloquium held at the University of Birmingham Medical School,

30th September 2011 53

v) Post-trial interview schedule 57

vi) Draft benchmarking tool – outcomes 58


1.0 Introduction and context

Literature has demonstrated (1) that simulation, when used as a pedagogical approach in

healthcare education, can have a positive and beneficial outcome upon learners’ self-

confidence and learning. In congruence with this knowledge there has been a rapid growth

in the emergence of simulation facilities and centres, coupled with significant financial

investment in both physical and human resources.

Arguably, there are a number of drivers pushing growth in the area of simulation in

healthcare education namely:

• A concerted drive to improve patient safety and clinical outcomes

• Significant advances in technology related to the provision of simulation

• A decline in the acceptability of using patients as “models” for healthcare

education

• A drive to improve the standardisation of healthcare education

• A desire to make undergraduate healthcare education relevant, grounded in

experiential learning and engaging for the student

• A desire for institutions to have the ‘best’ technology and equipment and an

industry which successfully promotes and supplies this.

Whilst it is evident that students engage with and enjoy simulation as a learning and

teaching approach in healthcare education, it is becoming increasingly apparent that there is

little evidence with which to differentiate what makes an effective simulation centre and

indeed, what constitutes the definition of such a simulation centre. At the beginning of this

project there was also a desire from within NHS West Midlands to explore the possibility of a

quality assurance framework which could be applied to simulation centres and simulation-

based education within the West Midlands region. In order to promote this work a project

team was commissioned by the Health Innovation Education Cluster (HIEC), comprising of a

core of three academic staff on secondment to the HIEC; this core group also drew on the

experience of other healthcare professionals, educators and academics from the HIEC key

stakeholder group. Discussion was also initiated with the Association of Simulated Practice


in Healthcare (ASPiH) which had already begun initial scoping work on the accreditation of

simulation centres.

Aim With the above context in mind the aim of this project was to explore the creation of a

benchmarking and accreditation process for healthcare simulation centres and simulation

providers in the West Midlands region. The aim of the project was:

“To explore the creation of an accreditation process for simulation centres and providers with

the aim of providing measures of objective quality assurance regarding facilities, resources

and methods involved in the delivery of simulation-based education”.

Such investigations were to be based initially in the West Midlands, but with further

exploration beyond the region to enhance validity and generalizability.

2.0 Review

Accreditation of Simulation Training: what is known This is a brief review examining what is already known about accreditation of simulation

training for healthcare professionals. To contextualise this review, we first consider what is

meant by ‘simulation’ and identify some of the leading organisations in this field.

What is simulation in healthcare settings? According to the Society for Simulation in Healthcare (SSH), “Simulation is the imitation or

representation of one act or system by another. Simulations can be said to have four main

purposes: education, assessment, research, and health system integration in facilitating

patient safety. Each of these purposes may be met by some combination of role play, low

and high tech tools, and a variety of settings from tabletop sessions to a realistic full mission

environment” (Society for Simulation in Healthcare, 2011). Advantages for education

include being able to undertake rare procedures and protecting staff from infection and other

risks as well as the obvious safety for patients. It is argued that “Once technology advances

to the point that real tasks can be accurately simulated, truly demonstrating competence”

using simulation must be a better form of assessment than paper or oral exams or, perhaps,

a number of years’ experience (2). Simulation-based research can be focussed on whether a


simulator is useful; in addition, computer modelling is useful regarding community health e.g.

likely spread of an epidemic1. Finally, systems integration includes “quality assessment

mechanisms”.

The Dutch Society for Simulation in Healthcare (DSSH) defines 'Simulation in Healthcare' as

“dynamic and tailored strategies allowing health professionals to grow and develop into

better professionals, without patients being at risk.” (3)

Organisations promoting healthcare simulation The Society for Simulation in Healthcare (US based) seeks to facilitate “excellence in (multi-

specialty) health care education, practice, and research through simulation modalities” (4). It

publishes the journal Simulation in Healthcare (5).

The US based Association of Standardized Patient Educators (ASPE) claims to be “the

international organization for professionals in the field of simulated and standardized patient

methodology” (6). It has created a Core Curriculum “to provide education on the core

fundamental knowledge essential for educators in the field of standardized patient

methodology”; the 8 modules are: Foundations of Methodology; Case and Checklist

Development; Recruitment and Training of standardized patients (SPs); Using SPs for

Instruction; Assessment; Administering an SP Program; Basics of Research/Scholarship;

and Special Topics. It also runs a Scholars’ Certificate Program; to receive the certificate you

need to attend 4 workshops at its annual conferences; and to become an ‘ASPE scholar’,

you must present a work of scholarship within two years of completing the program. It also

has a “Recommended Standardized Patient Case Outline” for people who wish to publish

standardized patient cases (7).

The Australian Society for Simulation in Healthcare is a chapter of Simulation Australia.

“Simulation Australia exists to promote the use of simulation for the benefit of providers,

practitioners and users in order to increase the use of simulation in achieving organisational

goals, for the advancement of Australia's economy and society” (8). Its members include

aeronautical, computer and mining companies.

The Society in Europe for Simulation Applied to Medicine (SESAM) aims “to encourage and

support the use of simulation in health care for the purpose of training and research” (9).

1Note that this report focuses on practical simulations and is not concerned with simulating using mathematical modeling.


There are 19 simulation centres posted on their website; three are in the UK: Bristol, London

and Hertfordshire. They tend to describe their staff, resources and training offered.

The International Nursing Association for Clinical Simulation and Learning (INACSL) mission

is “to promote research and disseminate evidence based practice standards for clinical

simulation methodologies and learning environments” (10). It publishes the Clinical

Simulation in Nursing journal and had recently (2012) published a series of its own

“standards for best practice in simulation education” (11).

These standards outline broad principles which might be adhered to when creating

participant objectives, using standardised terminology and using debrief effectively. They do

not however, venture into the physical aspects of creating a simulation centre or how to

apply a quality assurance framework to it.

The Simulation Innovation Resource Centre (SIRC) is an “online e-learning site where

nursing faculty can learn how to develop and integrate simulation into their curriculum, and

engage in dialogue with experts and peers” (12).

From this review, it is clear that there is global interest and expertise in simulation in medical

education with a number of national organisations taking a range of approaches to promote

and co-ordinate activities. Not all these organisations have a framework for accreditation,

however, and the UK appears to be lacking in this regard. Simulation has largely developed

through “personal interest” and innovation within healthcare delivery and education systems

Accreditation for Healthcare simulation

The American Society of Anesthesiologists (ASA) began the process of approving simulation

centres in 2006. Centres were to be assessed in terms of “Existing educational offerings,

experience and track record, process of curriculum development, process of instructor and

course evaluation, leadership, infrastructure to support CME, and policies and procedures to

address issues such as performance anxiety and confidentiality ” (13). Centres needed to

submit a scenario, so that a library of such scenarios could be developed. Site visits were to

“verify the program's capabilities” (13). However, Steadman went on to note that:

“Standardisation will one day permit performance assessment between centres and foster

evidence-based curricula” (13). Details of this accreditation can be found at

https://ssih.org/accreditation-of-healthcare-simulation-programs Currently, 27 centres are

accredited.

https://ssih.org/accreditation-of-healthcare-simulation-programs


The Australian and New Zealand College of Anaesthetists (ANZCA) accredits the Effective

Management of Anaesthetic Crises (EMAC) course which “consists of five modules run over

two and a half consecutive days at a simulation centre, covering topics such as airway

management, human performance, cardiovascular emergencies, anaesthetic emergencies

and trauma management” (14). From the information required on the ‘EMAC datasheet and

accreditation report’, accreditation of hospitals is based on: the qualifications and courses

attended by the instructors; the suitability of rooms for seminars, skills, debriefing etc.; and,

the simulation area. This area needs to replicate an emergency department bed area and

basic operating room; anaesthetic gases and other equipment must be available; and

monitoring of the simulated patient. The simulated patient (manikin) needs to be able to

demonstrate numerous changes to the respiratory and cardiovascular systems.

The Society for Simulation in Healthcare (SSH) piloted accreditation in 2009 “to recognize

and foster excellence in simulation in healthcare” (15). Applications were required to provide

information on staff, budgets, simulation technologies, floor plan, DVD of a recent task and a

‘program tour’, and details of the activities that relate to the four simulation purposes.

In 2006, the American College of Surgeons (ACS) began accrediting Education Institutes,

which “may use a variety of methods to achieve specific educational outcomes, including the

use of bench models, simulations, simulators, and virtual reality” (16) . About 60 centres are

currently accredited, of which around 10 are outside the US. Accreditation is based on 21

criteria in three areas: the learner groups served; the curricula offered; and the facilities,

resources, and personnel available (17). To be accredited at the basic level, centres must

teach procedural and cognitive skills to surgeons, residents and/ or medical students using

programmes already accredited by LCME, ACGME, ACCME or international equivalents.

Staff must be suitably qualified. The centre must have suitable models, simulations,

standardized patients etc. as well as demonstrating suitable financing and educational

resources. Also they must show the training activities recently offered. At the higher level

(comprehensive) they must teach surgeons and develop educational models containing :

“assessment of educational needs, definition of goals and objectives, selection of

instructional methods, creation of educational materials, delivery of effective education,

assessment of learners, and assessment of the effectiveness of educational programs” (18).

It was reported that in 2010 (17), there were 53 ACS-accredited Education Institutes and that

they have formed a consortium to develop 1) Continuous professional development; 2)

Residency education and training; 3) Research and development; 4) Technology


advancement; 5) Administration and management. This consortium has an annual meeting;

dissemination includes publication of papers in Surgery (17)

Since 2008, the Accreditation Council for Graduate Medical Education (ACGME) has

required US surgical residency programs to include simulations in a skills training lab. The

Fundamentals of Laparoscopic Surgery (FLS) program certification is a prerequisite to

certification by both the American Board of Surgery and the American Board of Colon and

Rectal Surgery. It includes hands-on skills training using the FLS Laparoscopic Trainer Box

and the examination includes a supervised manual skills test (19).

ACGME also regulates competencies in instruction, evaluation, and patient care in US

intensive care units. However, it is argued that teaching methods and assessment have

changed little with these regulations. Bedside case-based teaching and standardized

lectures are the most common teaching methods although there is a strong desire to use

computerized and full body simulations and more web-based learning modules (20). More

optimistically some commentators state that patient simulation (21) has been recognized by

ACGME as an effective means of teaching and evaluating competency. They write that "the

role of simulation appears to be evolving from that of an educational adjunct to use as a tool

in the assurance of clinical competence" and "It is likely only a matter of time before the use

of human patient simulators in the evaluation, certification, and remediation of

anaesthesiologists becomes more widespread".

To consider benchmarking and accreditation for Emergency Medicine, Fernandez et al.

(2010) (22) reviewed existing simulation accreditation programs; analysed EM simulation

program structures from the 100+ US programmes; and proposed a model for EM-based

simulation accreditation. Four programmes were reviewed: the American College of

Surgeons [ACS], American Society of Anesthesiologists [ASA] and the Society for

Simulation in Healthcare [SSH] all had or were developing simulation accreditation

programs, whilst the American College of Obstetrics and Gynecology [ACOG] had “created a

simulation consortium”. The paper considered:

• scope of the accreditation programs. SSH and ACS encompassed all forms of

simulation and were designed to be appropriate for different specialties. The ASA

programme was “more specialty specific in its content, concentrating on curriculum

development and instruction in anesthesia” (22). The ACS requirements have “the

potential to affect faculty recruitment” (22) etc. as they specify minimum time for

different posts.


• accreditation format. The ASA uses one set of criteria for standard accreditation, so

all programs must meet the same benchmarks e.g. regarding faculty expertise. ACS

has Basic and Comprehensive levels. SSH has a modular format where all

accredited institutions must meet ‘core standards’ and one out of ‘assessment’,

‘research’ and ‘education’; additional accreditation is available in ‘system integration

and patient safety’.

• criteria for accreditation focus on curriculum, instructor qualifications, equipment and

technology, and organisation and support. ACS requires minimum space and

equipment whereas ASA and SHH are more flexible and require “adequate space

and hardware to carry out program-specific educational missions” (22). ASA focuses

more on the curriculum requiring CPD courses with evaluations as well as a

simulation-based scenario. SSH also has research requirements.

In considering what Emergency Medicine should do, Fernandez et al (22) regard quality of

instruction and debriefing as crucial, but recognise the constraints of lack of time and

training; therefore they say that training recommendations are needed for the instructors and

high-quality courses must be available for them. Resource demands are high, as simulations

have a maximum of 10 learners unlike traditional large lectures. For assessment,

standardised case development and validation should be built on the current case banks.

Given costs, only the minimum equipment for effective scenarios should be required so

personnel and curricular requirements are also met. They argue for multilevel accreditation,

starting with a basic level.

Reviewing current simulation programmes in EM, many are based in the EM department:

this has advantages of focus but are costly, soak up faculty time and are not

interdisciplinary. Institution-wide centres are better for cost and sharing, but it can be hard to

access the resources and they may not be ideal for EM. A third model is a combination of

these two with an EM satellite from the multidisciplinary simulation centre (22).

In a review of recent advances, Yager, Lok et al. 2011 refer to Fernandez et al. “"Advances

in simulation centre design and function are a vital component of success, especially as

accreditation criteria and programmatic benchmarks are established" (23).

Although positive about the role of simulation in Emergency Medicine, Ten Eyck warns

against attempting “to create a simulation-based curriculum.” Instead, teaching must “start

with defining the targeted learners, assessing their general and specific educational needs,

defining learning objectives, and selecting the best educational strategy for achieving each


objective.” (24). Perhaps this thinking is why the American College of Surgeons (ACS) uses

the term ‘Education Institutes’ instead of ‘Simulation Centres’ (17).

Regarding the accreditation of these Education Institutes by the ACS, several such institutes

have reported on their current and future work. For example, Paige and Chauvin, 2010 (25)

claim The Louisiana State University, Health Sciences Centre will be a state of the art

surgical training centre with approximately 25,000 sqft learning laboratory dedicated to

simulation-based education and training, a 34-station cadaveric/animal demonstration

laboratory and 8 dedicated simulation rooms, including 2 virtual operating rooms. Similarly

accredited is the 13,000 sq. ft. Centre of Excellence for Simulation Education and Innovation

(CESEI) at Vancouver General Hospital. It emphasizes its curriculum and assessment

methodology, a unique online simulator with intelligent tutoring called CyberPatient and

expertise in remote technologies such as tele-presence and tele-robotics (26).

On a more cautious note, Beydon, Dureuil et al report that high fidelity simulation is rapidly

expanding in France; mainly based in universities, it is primarily used for in-site resident

training. They considered the potential for CPD accreditation but concluded that simulation’s

“ability to contribute to continuous medical education is still limited to date” (28).

The development of accreditation programmes for simulation in healthcare education reflects

the growing knowledge, experience and interest in the subject. As these have expanded

there has been recognition of the need to agree common standards against which work can

be judged. Yet the literature reveals a tension between recognising standards and isolating

simulation from the broader educational experience. The challenge therefore is to address

the quality assurance needs whilst retaining a commitment to the appropriate development

of the discipline. Therefore, any accreditation programme must retain the principle of sharing

good practice as a primary goal and should see simulation activities within a holistic

educational framework, not simply as a standalone discipline.

3.0 Design and methods This project consisted of two phases: Phase One (April to Dec 2011) included a Literature

Review and an expert panel event to discuss a proposed benchmarking document. Phase

Two (January to July 2012) saw the piloting of the agreed document and subsequent

evaluation process.


The benchmarking document The benchmarking document or tool was created following a meeting of the WMC HIEC

simulation accreditation and benchmarking work stream group (referred to herein as the

“project team”) in conjunction with NHS West Midlands. Broad themes were identified in

which it was desired to gather information regarding provision of simulation activities in the

West Midlands region.

The expert panel The purpose of the colloquium held in the West Midlands in September 2011 was to invite

experts in the field to share their priorities and knowledge to build upon existing practice in

medical simulation and, in order to apply a degree of external validity to the process, to

comment upon draft proposals for benchmarking standards prepared by the West Midlands

Central Health Innovation and Education Cluster (WMC HIEC) team. A senior representative

of the Association for Simulated Practice in Healthcare also attended to offer opinion and

attendees at the colloquium were given, in advance, copies of both the ASPiH and

WMCHIEC documents to compare and contrast. The HIEC proposal was structured under

six themes: governance, organisational management, facilities, learning and teaching,

research, and evaluation (See Appendix II). In addition to considering the ASPiH and the

HIEC proposals, delegates were asked to consider two questions:

• What are the core competencies which should be achieved in accreditation of a

simulation centre?

• How should we measure/record/reward a simulation centre’s achievement? What are

the metrics/rubrics we should use to benchmark a Centre?

The colloquium was attended by a total of 23 experts, drawn from across the UK and

included educators, clinicians and healthcare managers (See Appendix III). In small groups,

a modified nominal group technique was used to facilitate discussion of the documents and

the two questions. Plenary sessions allowed feedback and further discussion. Three

facilitators acted as rapporteurs and scribes. Their notes were then analysed thematically

according to the data and a summary report compiled (Appendix IV).

Piloting the benchmarking document It was deemed by the project team that a pilot study would be useful to test the performance

and validity of the proposed benchmarking process and document; primarily to look at the

range and depth of the data that could be gathered, but also to explore and, if required

modify, the process of completing the document. Seven local centres (three Higher

Education Institutions (HEIs) and four NHS Healthcare Trusts) were identified as potential


participants in the pilot study and were invited to take part in the pilot exercise. Six Centres

agreed to do so. The benchmarking document was sent to the Centres with an explanation

of what was needed and they were asked to return it with appropriate evidence within six

weeks. Centres were asked to return the evidence electronically and were sent encrypted

data sticks on which to store the data. All six Centres responded although all took longer

than originally estimated. Response times ranged from eight to thirteen weeks although most

responded within ten.

Following receipt of the evidence, an evaluation interview was conducted with the lead

contact at each centre to review the process of collecting and collating the evidence and

related issues. (Appendix V: Interview schedule). At these interviews, additional evidence

was discussed where appropriate. Summaries of interviews were returned to participants for

checking for accuracy. Although interviews were recorded, they were not transcribed.

Data analysis An initial assessment of the documentation received was made and the team worked

together to review what centres had provided and to allocate initial grades to the quality of

the evidence, described below. Some Centres provided additional evidence and this was

subsequently considered in the same way. The intention was to provide an indication of how

comprehensive the evidence was in relation to meeting the outcomes identified in the

document. This was entered into an Excel spread sheet (Appendix VI). Following the

interviews the data was revisited and an overall assessment was agreed discussed under

each subject heading (e.g., Governance and Management).

Interview data was analysed thematically according to common concerns and is reported in

the following results section. Participants were assured that data would be presented

thematically and as far as possible anonymously. Each team member analysed a different

aspect of the data, then team discussions confirmed the analysis.

4.0 Results

The expert panel meeting The meeting of expert practitioners confirmed the importance of accreditation of simulation

centres and identified several strong reasons for developing such a mechanism. These

included professional credibility, sharing of good practice, safeguarding the public and to

assist in the development of ideas and knowledge. Any process needed to be affordable,

deliverable and sustainable and come from within the simulation community itself rather than

being imposed from outside.


It was recognised that any process needed to be flexible enough to meet the needs of

different Centres, ranging from large, well established Centres to smaller Outreach teams.

The participants agreed a list of six key principles for any accreditation process:

• the primary purpose of developing accreditation procedures was to ensure better and

safer care for patients;

• all competencies must map onto existing national standards where they exist, to

minimise the levels of bureaucracy and to incorporate already known best practice;

• accreditation should be supportive and formative, not punitive;

• any framework must be flexible enough to meet all simulation situations, from the

individual ‘learning on the job’ to the multi-million pound specialist centre;

• accreditation should be based upon principles of peer review;

• accreditation should be achievable and affordable, particularly in terms of staff

delivery.

The findings from the Expert Panel informed the further development of the pilot exercise.

See Appendix IV for more details of this event.

Analysis of benchmarking data The completed benchmarking documents were collated by the project team and grading

criteria were devised as follows based upon the relevance of the evidence provided in

support of meeting the outcome:

Level 1 – evidence provided is relevant and comprehensively meets the outcome

Level 2- evidence provided, but minimal compliance/relevance to the outcome

Level 3 – evidence provided but does not meet the outcome

Level 4 – no evidence provided in support of meeting the outcome

Each centres’ evidence was then compared against the benchmarking tool, specifically in

relation to the outcomes where each piece of evidence had been provided to support the

response. The pie charts below provide a generalised graphical representation of the

categories of levels of evidence provided for each outcome:


Governance

Organisational management

level 1

level 2

level 3

level 4

level 1

level 2

level 3

level 4


Facilities

Learning and teaching

level 1

level 2

level 3

level 4

level 1

level 2

level 3

level 4


Evaluation

Thematic analysis of centre participant interviews

The process

Gathering evidence

Centre D reported that it was relatively easy to access the evidence. The other centres

spoke of varying degrees of difficulty. Centres C, E and F spoke of needing to get

information from other people (managers, clinical skills educators and others) which made

the process more difficult. Centre A spoke of security issues and where to put the evidence

but did not address the issue of accessibility. Centre B mentioned difficulty collecting

evidence as all their records are still paper based. Centre E reported that involvement in the

process would now help formulate how they collect data on an annual basis.

Time

level 1

level 2

level 3

level 4


Four centres specified the amount of time it took to gather the data. Centres A and D stated

that it took them 3+ hours, Centre C said 2-3 hours and Centre E said about 10 hours.

Centres B and F didn’t state the length of time but referred to it as taking too long or a very

long time.

The majority of centres felt that the time they had spent on the task was appropriate for the

purpose, particularly for the first time. These were centres A, C, D, E and F. Centre B felt

that there are already plenty of reporting structures in place so adding a new one is not

helpful. Centre E also hoped that this document could be used to feed into other reporting

mechanisms.

General comments

There was an overall feeling that this process should not add to the existing reporting

mechanisms in place. If this process is flexible then centres could use data from other

monitoring systems for this tool too.

Clarity of task

Participating centres were asked to comment upon the clarity of the benchmarking tool and

the task in general. Responses were thematically collated.

Instructions

The majority of the centres (five out of six) indicated that the task was clear, although one of

these centres (F) commented that they would have preferred more detailed instructions on

how to complete the document. Centre B stated that the document was “not clear at all”.

Centre F said that whilst they understood the aim of the task and the document, they were

unsure about the type and depth of supporting documentation they needed to provide as

evidence. Centre F also commented that it was not apparent to them until the researchers

had contacted them after the tool had been completed that they had to provide evidence

such as blank templates for job descriptions and evaluation forms. Centre C echoed these

comments by stating that they were not aware that supporting evidence was required when

completing the tool.

Purpose

Centre D, although happy to complete the document and seeing the relevance to quality

assurance of simulation centres, asked the question “what will this all translate to, what is


the end product, who will administer it?” Centre B was unsure of the purpose of the

benchmarking tool.

Language used

One centre (B) expressed concern about the use of “academic language” in the document

which might not be understood by clinicians, and that use of such phrases as “delivery

metrics” and “financial monitoring” would be off-putting and confusing to them. Centre A also

commented that the academic language used in the accompanying McGaghie et al (2010)

article may cause some groups (e.g., technicians) to struggle, although they felt it would not

be a problem for academics or clinicians. (30)

Structure of the document

One centre (D) commented that the design of the document was helpful, in that it was in

table format which made reading easy. Although another centre (C) commented that they

would have preferred clearer delineation between each sub-section in the table, as text had

a tendency to “creep” out of the section it was intended to be in, sometimes making reading

and interpreting difficult.

There was suggestion from centre B that the document needed to be much simpler and less

“academic”. Instead of the existing sub-categories they made the suggestion that the

document should just ask “what do you do, who does it, who do you do it to, why do you do it

and where is it done?”

There was also comment from centres (B, C and D) regarding the need for further

clarification and definition of what constitutes a “simulation centre” and also further

clarification of the levels of fidelity of simulation mentioned under the facilities benchmark of

the benchmarking tool. Centre B stated that they felt that simulation was about what is done,

by whom rather than where it is done i.e. “simulation centre”, and therefore felt that the focus

of the document should be how simulation is used not the facilities it is performed in.

Two centres (B and C) suggested that the “simulation features and best practices” column in

the benchmarking tool was misleading. The intention of this column was to provide


supporting evidence from McGaghie et al (2009) (30) on the best practice around simulation

design and implementation. However centres B and C believed that this was unclear and

distracted from the task in hand.

Centre B expressed concerns about “commercial sensitivities” in sharing data, a theme

which was reiterated by this respondent again later in the interview.

Relevance and scope of evidence requested

Overall, there was agreement that the evidence asked for was appropriate and

comprehensive, with all six centres reporting some agreement on this. The broad headings

were deemed appropriate and were described as helpful. One centre (C) commented:

Quality assurance of simulation-based education is a hot topic at the moment and

everyone is looking for direction and guidance on this.

One centre (B) reported that some of the evidence requested was ‘irrelevant’, commercially

sensitive or publically available elsewhere, with only a proportion being directly appropriate,

but this was a minority view. No centres suggested other evidence which could or should

have been included, and several suggested they would make use of the process as part of

internal management procedures. For example, one centre described the document as ‘quite

comprehensive and useful’ and continued:

I’d like to try to adapt it and look at a Department’s point of view, simplified, so I can

ask teams to feed back for their annual reports. (Centre E)

Themes recorded in the data for improvement in the document included: the conceptual

nature of the document, the perceived lack of specificity in the document and how best to

capture some aspects of activity and evidence.

Conceptual understandings

Three of the six centres were concerned that some of the language in the document was too

conceptual, at the expense of clarity. In particular, the use of the Kirkpatrick model of

learning (30) was criticised, in part because it was unrealistic: Level 4: I don’t think anyone

has reached there (Centre E); or that it was irrelevant (Centre B). Two centres suggested

greater specificity was needed, for example, (Centre D) asked that the document be more

specific with definitions on types of simulation and fidelity (e.g., low, medium, and high).


Capturing the full scale of activity

One centre (F) reported difficulties with capturing the full breadth of simulation activities,

including different professional needs and various purposes for the activities. They reported

that different professions had particular ways of providing feedback and felt this was not

recognised in the documentation. Similarly, students used simulation for a range of purposes

(to practice, to boost confidence, to catch up) and again they felt this was not reflected in the

documentation. Informal and ad hoc sessions were important to some students and these

were hard to record and evidence for.

Collaboration between Centres: opportunities and threats

The dominant theme in response to questions concerning collaboration was the threat posed

by commercial and professional sensitivities. All centres agreed that collaboration per se

was a good thing, but raised a number of barriers to that collaboration happening. One

centre (Centre B) commented that previous attempts at collaboration had been

disappointing, making further initiatives difficult. Three barriers to collaboration were

identified by participants: financial and commercial pressures, time, and territorial concerns.

All centres expressed some concerns about all of these barriers, though the emphasis

placed varied. Four centres discussed positive ways forward in collaboration, albeit against a

background of competition discussed below.

Financial and commercial pressures

Centre C was typical in its response to questions about sharing good practice: competition

hinders progress. All centres reporting seeing themselves in competition with each other and

expressed reservations about sharing commercially sensitive data. Such data included

curriculum materials and mapping, student numbers, financial arrangements and good

practice. Centres described having developed their work and needing to protect their share

of the market. One centre (Centre D) described this as people are guarding money, amidst

an ethos of competition rather than collaboration. Although participants welcomed the idea of

collaboration, they reported having dedicated significant resources to the development of

materials and fearing collaboration would undermine the strength of this work.

Time

One centre (B) reported that staffing limitations and financial pressures restricted the time

available to commit to collaborative activities. Teaching and development had to take priority

at this centre which was entirely dependent on income from such activities.


Territorial concerns

In addition to financial pressures in a competitive market, centres reported an ethos of

professional competition. Several centres used the phrase empire building to describe how

centres were seeking professional supremacy in simulation education. This made

collaboration much less likely as people sought to become discipline leaders.

Fostering collaboration

Broad agreement amongst centres that collaboration would enhance delivery led to

suggestions for how to foster that collaboration. Common to most of the centres was the

idea that a regional network that was objective, structured and organised (Centre A) would

be most effective. Noting that professions often sit in silos collaboration could foster inter-

professional learning as well as between centres. Another centre (E) suggested the most

effective way to foster collaboration was to host workshops to disseminate good practice or

to pair peers in different centres to work on specific issues. A central depository of materials

and models of good practice would be helpful.

Noting the importance of building upon the momentum created by the pilot benchmarking

exercise, Centre F provided a typical response: collaboration was a good thing but it was not

clear how it could be done in a competitive, cash limited environment.

Other issues raised under relevance and scope of evidence

One centre (E) requested that any assurance mechanisms should form one document in an

agreed, standardised format. For example, in the future, whoever asks for information (for

commissioning or assurance purposes) everyone should use the same format.

Another centre (A) questioned who should administer the accreditation process. They

suggested there were already a ‘multitude’ of professional standards and the need now was

for some consistency within a national framework.

Centre F queried what this process of accreditation would add over and above the existing

course validation processes. The additional benefits needed to be made quite explicit. They

also noted a risk that smaller centres might become isolated and called for stronger

information links between centres.


5.0 Discussion

Evidence provided

The data clearly show that for four out of the five outcomes in the benchmarking document

the highest frequency of level of evidence was “level 4” or “no evidence provided in support

of meeting the outcome”. By contrast, three outcomes, “organisational management”,

“facilities” and “learning and teaching”, achieved Level 1 more frequently. Possible

explanations for this may include: that this type of evidence was more readily accessible, to

the respondent’s nature of their roles within their organisation; or that this type of information

was already required for internal and external monitoring purposes. Evaluation in particular

had a high frequency of level 4 evidence, i.e. “no evidence provided” which may reflect the

wider position of research and evaluation in the simulation community.

The level of evidence provided clearly plays a significant role in the ability to grade a centres’

compliance with the stated outcomes in the benchmarking document. A significant number

of centres did not provide evidence to meet every outcome, and as such, if this had been a

real benchmarking exercise it would have been difficult to have graded those centres with

accuracy and validity. It is acknowledged that this is a pilot process and some centres were

unclear of the type or nature of the data, but this is an area of the process that would need to

be adhered to as closely as possible in order to produce an accurate benchmarking result. It

should be noted however, that a number of centres offered to return to their institutions to

complete a further gathering of data to fill in missing gaps; however the project team

declined this offer as the aim of the pilot project was to see what data was provided and how

that data was gathered rather than the final product.

Clarity of task

Although the majority of the respondents indicated they understood the broad requirement of

the benchmarking tool and process there is a need to emphasise this at the outset. The

levels of supporting evidence provided by some respondents and their verbal responses to

questioning in interviews indicate some confusion over both the purpose and process of the

benchmarking exercise. Further detailed explanation should be provided and a fully

completed mock tool which gives examples of how the document should be completed and

the type of evidence required. However, in recognition of the variety of work currently

underway in centres, it is important to maintain some flexibility. There was clearly, in a

minority of cases, some reticence and concern surrounding the completion of this


benchmarking document and process, which may be understood in the context of as it

involves an external agent applying scrutiny to a centre and its processes. These concerns

would need to be addressed by raising awareness to promote and justify the purpose of the

benchmarking tool and process.

The data also suggests that the design of the tool requires some adjustment in structure and

language to make it more accessible to all professional groups. For example, several

respondents felt the language was too academic for the target audience.

Relevance and scope of evidence requested

There was widespread agreement that the evidence requested was broadly appropriate to

the task and allowed centres to show case their work without significant disruption. Only one

centre felt the task to be disproportionate. It was clear that the pilot exercise had been useful

in itself as it encouraged centres to review their work and provided a framework within which

they could take stock of the stage they had reached and what stage they had reached.

Should the accreditation process be adopted, centres reported its potential use as a means

of structuring their accountability and progress. There was, however, agreement that the

process should be meaningful and not descend into a tick box exercise. Requests for

information should be standardised where possible. It is essential that an accreditation

exercise must not add disproportionately to bureaucratic demands. An area of concern that

was raised repeatedly by centres was that of competition and collaboration. It is difficult to

overstate the nervousness that exists amongst centres about their need to protect the work

they do. This was both a financial and professional concern and became most apparent

when discussing sharing of teaching materials, course outlines or financial information.

Clearly centres feel in competition with one another and this limits their ability to collaborate.

In some instances this was reported to be because of Trust or institutional constraints and in

others it was about protecting the professional work they were undertaking. Thus, any

system of accreditation needs to consider how best to share good practice while respecting

centres’ concerns in this regard.

There was an overall feeling that this process should, so far as possible, not add to the

existing reporting mechanisms in place. If this process is flexible then centres could use data

from other monitoring systems for this tool too.


6.0 Final Conclusions and recommendations

Conclusions

This pilot process has demonstrated that a benchmarking and accreditation system for

simulation centres is feasible and desirable. Whilst there were some issues with range and

quality with the data collected, it is apparent that institutions are willing and able to collect

data for the purpose of benchmarking and accreditation. This report concludes with the

presentation of two recommendations for future work as follows:

Recommendation one

The future oversight of benchmarking and accreditation work needs to be governed by an

autonomous, independent organisation able to make objective judgements.

Concerns were expressed by parties involved with the pilot study that if the process is

governed by one organisation, for instance a higher education institution, then the objectivity

of the process may be questioned. Suggestions as to which body might undertake the

process included the commissioning body replacing NHS West Midlands, or an outside not-

for-profit organisation such as the Association for Simulated Practice in Healthcare (ASPiH),

which has both a wide network and considerable experience base within simulation.

Consensus was clear from within the expert panel that sole responsibility for the

accreditation of simulation centres should not be handed to specific professional bodies such

as the General Medical Council (GMC), Nursing and Midwifery Council (NMC) or the Health

Professionals Council (HPC), as this would encourage further “silo mentality” of professional

groups and further fragment progress.

Recommendation two

There should be the development of a rubric by which to accredit simulation centres.

At present, this work has stopped short of applying a scoring rubric to the data gathered

within the pilot study and has looked only at the process of benchmarking. From the data in

the literature search, expert panel and pilot study, it is notable that there is some reluctance

to apply a defined “OFSTED-style” rating of a simulation centre. Indeed, during the expert

panel event and in the data gathered during the pilot study, participants expressed concern

about what a grading score might mean for their simulation centres and whether this might

have a potential effect on its reputation and funding sources. Whilst this concern is

acknowledged by the project team, it is our opinion that if the benchmarking and


accreditation process is to be meaningful and valid, then there must be some kind of

measurement of quality and performance involved. This is clearly a contentious issue which

must be dealt with sensitively but is essential if such a benchmarking and accreditation

process is to have significance.


References

1. Alinier G, Hunt WB, Gordon R. Determining the value of simulation in nurse

education: study design and initial results. Nurse Education in Practice 4 2004; 3: 200-207.

2. Society for Simulation in Healthcare. What is simulation? [Internet]. [cited 2012 Sept 20]. Available from URL: http://ssih.org/about-simulation.

3. Dutch Society for Simulation in Healthcare. Webpage. [cited 2012 Sept 20]. Available from URL: http://www.dssh.nl/en

4. Society for Simulation in Healthcare. http://www.ssih.org/SSIH/ssih

5. http://journals.lww.com/simulationinhealthcare/pages/default.aspx.

6. Association of Standardized/Simulated Patient Educators. Webpage. [cited 2012 Sept 20]. Available from URL: http://www.aspeducators.org/.

7. Recommended Standardized Patient Case Outline [Internet]. 2007 [cited 2012 Sept 20]. Available from URL: http://aspeducators.org/img/mededportal%20submit_criteria(1).pdf

8. http://www.siaa.asn.au/objectives.html.

9. Society in Europe for Simulation Applied to Medicine. Webpage. [cited 2012 Sept 20] Available from URL: http://www.sesam-web.org/

10. The International Nursing Association for Clinical Simulation & Learning. Webpage. [cited 2012 Sept 20] Available from URL: http://www.inacsl.org/

11. The INACSL Board of Directors. Standards of Best Practice: Simulation. Clinical Simulation in Nursing. 2011; 7: S3-S19

12. SIRN – National League for Nursing. Webpage. [cited 2012 Sept 20]. Available from URL: http://sirc.nln.org/

13. Steadman, R.H. The American Society of Anesthesiologists' national endorsement program for simulation centers. Journal of Critical Care 2008; 23: 203-206

14. ANZCA (2011) Effective Management of Anaesthetic Crises [online]. http://www.anzca.edu.au/trainees/courses/emac-and-emst-courses/overview/?searchterm=simulation%20accreditation The Australian and New Zealand College of Anaesthetists [Accessed 13/07/2011]

15. Society for Simulation in Healthcare. Accreditation Council for Accreditation of Healthcare Simulation Programmes. Accreditation Standards and Measurement

http://ssih.org/about-simulation

http://www.dssh.nl/en

http://www.ssih.org/SSIH/ssih

http://journals.lww.com/simulationinhealthcare/pages/default.aspx

http://www.aspeducators.org/

http://aspeducators.org/img/mededportal%20submit_criteria(1).pdf

http://www.siaa.asn.au/objectives.html

http://www.sesam-web.org/

http://www.inacsl.org/

http://sirc.nln.org/

http://www.anzca.edu.au/trainees/courses/emac-and-emst-courses/overview/?searchterm=simulation%20accreditation

http://www.anzca.edu.au/trainees/courses/emac-and-emst-courses/overview/?searchterm=simulation%20accreditation


Criteria. [Internet] 2012 [cited 2012 Sept 20] 1-19. Available from URL: https://ssih.org/uploads/committees/2012%20Accreditation%20Standards39.pdf

16. American College of Surgeons. Program for Accreditation of Education Institutes (ACS AEI) [Internet]. Revised 2012 [cited 2012 Sept 20]. Available from URL: http://www.facs.org/education/accreditationprogram/index.html

17. Sachdeva AK. Efforts to advance simulation-based surgical education through the American College of Surgeons-accredited Education Institutes. Surgery 2010; 147: (5): 612-613.

18. Sachdeva AK, Pellegrini CA, Johnson KA. Support for Simulation-based Surgical Education through American College of Surgeons – Accredited Education Institutes. World Journal of Surgery 2008; 32: 196-207

19. Bashankaev B, Baido S, Wexner SD. Review of available methods of simulation training to facilitate surgical education. Surgical Endoscopy 2011; 25: (1): 28-35

20. Chudgar SM, Cox CE, Que LG, Andolsek K, Knudsen NW, Clay AS. Current teaching and evaluation methods in critical care medicine: has the Accreditation Council for Graduate Medical Education affected how we practice and teach in the intensive care unit? Critical Care Medicine 2009; 37: (1): 49-60

21. DeMaria S Jr, Levine AI and Bryson EO. The use of multi-modality simulation in the retraining of the physician for medical licensure. J Clin Anesth 2010 June;22(4):294-9.

22. Fernandez R, Wang E, Vozenilek JA, Hayden E, McLaughlin S, Godwin SA, Griswold-Theodorson S, Davenport M and Gordon JA. Simulation center accreditation and programmatic benchmarks: a review for emergency medicine. Academic Emergency Medicine 2010; 17: (10): 1093-1103.

23. Yager P, Lok J and Klig J. Advances in simulation for pediatric critical care and emergency medicine. Current Opinion in Pediatrics 2011; 23: (3): 293-297

24. Ten Eyck RP. Simulation in emergency medicine training. Pediatric Emergency Care 2011; 27: (4): 333-341; quiz 342-334.

25. Paige JT and Chauvin S. Louisiana State University, Health Sciences Center, New Orleans (LSUHSC-NO) Learning Center, an American College of Surgeons (ACS) accredited, comprehensive education institute. Journal of Surgical Education 2010; 67: (6): 464-467

26. Parker JH. Canadian Surgical Technologies and Advanced Robotics. Journal of Surgical Education 2010; 67: (4): 258-261

27. Qayumi AK. Centre of Excellence For Simulation Education and Innovation (CESEI). Journal of Surgical Education 2010; 67: (4): 265-269

https://ssih.org/uploads/committees/2012%20Accreditation%20Standards39.pdf

http://www.facs.org/education/accreditationprogram/index.html


28. Beydon L, Dureuil B, Nathan N, Piriou V and Steib A. High fidelity simulation in

Anesthesia and Intensive Care: context and opinion of performing centres--a survey by the French College of Anesthesiologists and Intensivists. 2010 Nov;29(11):782-6. doi: 10.1016/j.annfar.2010.08.013. Epub 2010 Oct 8.

29. Stefaniak J, Schumacher K, Robbins J and Shanley C. Marcia and Eugene Applebaum Surgical Learning Center. Journal of Surgical Education 2010; 67: (4): 251-254

30. McGaghie WC, Issenberg B, Petrusa ER & Scalese RJ. A critical review of simulation-based medical education research: 2003-2009. Medical Education 2010; 44: 50



Appendix A: Literature review search terms

The literature was accessed from links provided by Matt Aldridge and web searches. Matt also provided one of the subsequent papers (Fernandez, Wang et al., 2010). The subsequent literature search used Medline 2007 to 2011 with the search terms shown below; 728 articles were found. As this was too many for our current purpose, titles were scanned from 2009 to 2011 and 36 articles were selected for further scrutiny. Also, included are a paper that cited Fernandez, Wang et al. (2010) and two related to other selected articles. On inspection, 27 of these articles were excluded as they were not to do with simulation; this seems to be because search number 8 includes ‘Human’ as a keyword. Also, note that seven of these were not in English.

Search number Search terms

Number of articles

1 Computer Simulation/ or Patient Simulation/ or simulation.mp. 64144

2

Role Playing/ or Education, Medical/ or Patients/ or Peer Group/ or Clinical Competence/ or Pediatrics/ or Education, Medical, Undergraduate/ or Physician-Patient Relations/ or standardised patient.mp. or Communication/

51169

3

(manikin or mannequin or mannekin or mannikin or dummy).mp. [mp=protocol supplementary concept, rare disease supplementary concept, title, original title, abstract, name of substance word, subject heading word, unique identifier]

1292

4 Telemedicine/ or Computer Simulation/ or virtual reality.mp. 51916 5 Imaging, Three-Dimensional/ or 3-dimensional.mp. 18366

6 Visual Perception/ or Touch/ or Motion Perception/ or haptic.mp. or Computer Simulation/ 61286

7 Physical Stimulation/ or Touch/ or Touch Perception/ or Pattern Recognition, Visual/ or tactile.mp. 13233

8

(clinical skill? center or clinical skill? centre).mp. [mp=protocol supplementary concept, rare disease supplementary concept, title, original title, abstract, name of substance word, subject heading word, unique identifier]

3

9 Models, Biological/ or human model.mp. 70639

10 Computer Simulation/ or Manikins/ or simulator.mp. or Models, Anatomic/ 53115

11 Computer-Assisted Instruction/ or computer-based.mp. 3789 12 Feedback/ or force feedback.mp. 3323 13 exp Education/ 101423

14

Education, Veterinary/ or Health Education/ or Education, Nursing, Graduate/ or "Early Intervention (Education)"/ or Education, Nursing, Associate/ or Competency-Based Education/ or Education, Nursing, Baccalaureate/ or Education, Public Health Professional/ or Education, Nursing/ or Education, Premedical/ or Education,

91808


Pharmacy, Graduate/ or Education, Professional/ or Education, Nursing, Diploma Programs/ or Education, Medical, Undergraduate/ or Education, Pharmacy, Continuing/ or Education, Graduate/ or Education, Special/ or "Mainstreaming (Education)"/ or Education, Medical/ or education.mp. or "Education of Mentally Retarded"/ or Area Health Education Centers/ or Education, Medical, Graduate/ or Education, Pharmacy/ or Education, Professional, Retraining/ or Nursing Education Research/ or Education, Distance/ or Education, Medical, Continuing/ or Education, Nonprofessional/ or Patient Education Handout/ or Education Department, Hospital/ or Education, Continuing/ or Patient Education as Topic/ or Education, Nursing, Continuing/

15 patient education.mp. or Patient Education as Topic/ 16450

16 exp "Joint Commission on Accreditation of Healthcare Organizations"/ or exp Accreditation/ or accreditation.mp. 3483

17 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 218668 18 13 or 14 or 15 132617 19 16 and 17 and 18 728


Appendix II: Draft benchmarking tool


Governance

Outcomes Benchmarks Simulation Features and Best Practices

(McGaghie et al, 2010)

Evidence

Demonstrable outcomes:

There is a structure in place to monitor simulation and clinical skills activities

Simulation and clinical skills activities are evaluated by learners and the information is received and reviewed by senior managers within the organisation

There is evidence of

Audit trail of sessions:

• Groups delivered to • Frequency • Commissioning information • Financial monitoring

Delivery metrics are reviewed by senior managers for cost effectiveness and efficacy

Professional bodies have reviewed and


professional body review and validation of the quality of provision of clinical skills teaching and simulation

Demonstration of staff development

noted the provision of skills and simulation delivery within the organisation

There is evidence of staff preparation and training in the use of skills and simulation learning and teaching methods

Clinical experience is not a proxy for simulation instructor effectiveness.

Instructor and learner need not be from the same profession

Evidence of staff engagement in development activities related to the delivery of skills and simulation


Organisational Management

Outcomes Benchmarks Simulation Features and Best Practices (Mcgaghie et

al, 2010) Evidence


An audit trail is in place to demonstrate usage of the equipment/facilities and a breakdown of the learner/staff group

There is a clinical skills/simulation policy in place that covers aspects such as: staff and student induction, health and safety, risk assessment, asset register of equipment etc

The organisation has a job description and key lead /

Evidence of internal audit on the use of facilities, data gathered on professional groups usage

Evidence of internal management structures in relation to the delivery of skills and simulation activity

Risk management & H&S policy for staff and students

Audit data, with accompanying analysis on the usage of facilities

Policies exist for the strategic use of simulation.

Evidence of risk assessment


nominated person for clinical skills/simulation

The lead for clinical skills/simulation education and training undergoes annual appraisal and has the opportunity for professional development


Facilities

Outcomes Benchmarks Justification

Simulation Features and Best Practices (Mcgaghie et

al, 2010)

Evidence


Organisations can demonstrate their level of clinical skills/simulation education and training and which category it falls into: I.E. high fidelity, medium fidelity and low fidelity

• Categorisation of equipment and facilities provided by a centre – what is provided

1. Roleplay using standardised patients

2. Part-task training, e.g. using limb and other phantom anatomical models

3. Medium to high fidelity using human patient simulator models

• Dedicated space, protected for the delivery of clinical skills and simulation

Do the tools match the goals?

Multi-modal simulation can bring added value

Evidence of asset registers of equipment, with appropriate maintenance schedules.

Strategic overview and lesson plans demonstrate the use of multimodal simulation

Evidence in course documentation that appropriate methods have been selected to deliver identified outcomes

i.e. Human Patient Simulators for team training in cardiac arrest.


Organisation can demonstrate dedicated clinical skills and simulation facilities

• Ability to record learner performance for viewing and debrief

• Opportunity for skills independent skills rehearsal, e.g. open access skills facilities

Deliberate practice; essential role in simulation based education. Learner-centred. Apparent dose-response relationship

Feedback is an essential feature of simulation-based education.

Learning and Teaching


(Mcgaghie et al, 2010) Evidence



Organisation can demonstrate how skills and simulation service delivery is outcome based and driven. Simulation delivery is contextual

Demonstration of standardisation of delivery

Relation to healthcare curricula

• Demonstration of course design methods

• Relation to national guidance, such a NPSA alerts, NICE guidance etc

• Core lesson plans which provide standardised delivery of sessions

• Learning and teaching is linked to delivery of curricula outcomes

• Learning and teaching is embedded into curricula

Evidence of feedback delivery methods

Post-activity and beyond the classroom or skills lab, e.g. the ability for the learner to

Educational and professional context : context authenticity is critical for simulation based education

Mastery learning: all learners master educational goals at a high level with little or no outcome variation.

Curriculum integration, integrate with other learning events

Course design and evaluation shows that most content delivery was contextual to its related clinical practice

Clear relation in lesson plans and course design of inclusion of national guidance and best practice

Inclusion of examples of lesson plans

• Mapping of NMC “Essential Skills Clusters” against provision for nursing students.

• Mapping of GMC “Tomorrows Doctors” against provision for medical


Mechanisms exist for effective feedback on learner performance and behaviour

Measurement of learners progress

self-evaluate performance electronically post-session.

Feedback is an essential element of simulation-based education

Outcome measurement, Mastery learning

students. • Mapping of

Interprofessional education provision against CAIPE “Principles of Interprofessional Education”

Evaluation


(Mcgaghie et al, 2010) Evidence


Organisations demonstrate evidence of


evaluation of service delivery:

• evaluation of staff performance

• evaluation of learners • progress • evaluation of efficacy of

learning and teaching methods employed

• evidence of research and development into the use of skills and simulation

Evidence of review of staff performance in debrief by use of a validated tool, i.e. Harvard’s DASH tool

Evidence of evaluation as highlighted by Kirkpatrick’s hierarchy of evaluation:

• Is there evidence of evaluation of learner’s reaction to simulation and skills activities? (Kirkpatrick level 1) e.g. post-session satisfaction evaluations

• Is there evidence of evaluation of learner’s progress as a result of skills and simulation activities?(Kirkpatrick level 2) e.g. formative/summative assessment

• Is there evidence of a change in learner’s behaviour as a result of

Feedback is an essential element of simulation-based education

Outcome measurement

Transfer to practice (level 4 Kirkpatrick)

Evidence of DASH evaluations

Level 1 – Evidence of 360 degree student-staff evaluation of learning and teaching delivery

Level 2 – Evidence of formative and summative assessments linked to skills and simulation


participating in skills and simulation activities? (Kirkpatrick level 3) e.g. audit or evaluation of how education translates into modified clinical practice

• Is there evidence of organisational impact; results of behaviour change? (Kirkpatrick level 4) e.g. research into reduction in patient harm or enhanced patient outcomes as a result of simulation and skills activities

(online quizzes and MCQS)

Level 3 – Evidence of clinical audit/evaluation of behaviour of learners post-simulation

Level 4 – Clinical practice research into the impact of simulation/skills education on patient outcomes


Appendix III: Attendees at the expert panel 2011 Luke Bracegirdle Keele University Mick Harper Wessex HIEC John Kinnear NECLES HIEC, Anglia Ruskin University Nina Godson Coventry University Alison Hoare Coventry University Peter Jaye Guy’s and St Thomas Joanne Barratt Leeds Metropolitan University Michelle McKenzie Smith Montagu Clinical Simulation Centre (Yorkshire and

Humber region) Pauline Pearson HIEC North East CETL4 Health Victor Ottoway HIEC North East CETL4 Health Anne Taylor UG Hospital Dean, Stafford Philomena Shaughnessy University of Hertfordshire Ken Spearpoint University of Hertfordshire Amanda Royston West Midlands South Carl Hillerman Coventry and Warwickshire Bryn Baxendale Trent Simulation Centre and ASPiH Catherine Baldock University Hospitals Coventry and Warwickshire Alexandra McCurdie University Hospitals Coventry and Warwickshire Lisa Bayliss-Pratt West Midlands Strategic Health Authority Matt Aldridge University of Wolverhampton/WMC HIEC Karen Barry University of Birmingham/WMC HIEC Sandra Cooke University of Birmingham/WMC HIEC Jonathan Steward Hollier Centre Ruth Jackson NECLES HIEC, Anglia Ruskin University Frank Coffey University of Nottingham Alison Pope NHS WM


Appendix IV:

West Midlands Central Health Innovation and Education Cluster (WMCHIEC) Simulation Benchmarking and Accreditation theme

A Report on a colloquium held at the University of Birmingham Medical School, 30th September 2011.

Benchmarking and accreditation of simulation centres: a framework for moving towards excellence. Matt Aldridge, University of Wolverhampton

Karen Barry, University of Birmingham

Sandra Cooke, University of Birmingham

Introduction According to the Society for Simulation in Healthcare (SSH)1, “Simulation is the imitation or representation of one act or system by another. Simulations can be said to have four main purposes: education, assessment, research and health system integration in facilitating patient safety. Each of these purposes may be met by some combination of role play, low and high tech tools, and a variety of settings from table-top sessions to a realistic full mission environment”.

In the UK, the Association for Simulated Practice in Healthcare (ASPiH) was established in 2009 to provide a forum for health educators to share good practice in the use of simulation and they have subsequently developed a draft framework for the accreditation of centres. Similarly, in Yorkshire and Humberside a model for accreditation has been piloted, but a national process remains elusive. Internationally, there are processes for accreditation, mainly in the US, Australia and in mainland Europe. There is a need for a UK-wide approach to quality assurance and standards in the rapidly growing field of simulation-based education.

The purpose of the colloquium held in the West Midlands in September 2011 was to invite experts in the field to share their priorities and knowledge to build upon existing practice in medical simulation and to comment upon draft proposals for benchmarking standards prepared by the West Midlands Central Health Innovation and Education Cluster (WMC HIEC) team and ASPiH. The HIEC proposal was structured under six themes: governance, organisational management, facilities, learning and teaching, research, and evaluation (See


Appendix I). In addition to considering the ASPiH and the HIEC proposals, delegates were asked to consider two questions:

• What are the core competencies which should be achieved in accreditation of a simulation centre?

• How should we measure/record/reward a simulation centre’s achievement? What are the metrics/rubrics we should use to benchmark a Centre?

The event The colloquium was attended by a total of 23 experts, drawn from across the UK and included educators, clinicians and strategic managers (See Appendix II). In small groups, a modified nominal group technique was used to facilitate discussion of the documents and the two questions. Plenary sessions allowed feedback and further discussion. Three facilitators acted as raconteurs and scribes. Their notes were then analysed thematically and are reported here.

Initial thoughts Delegates agreed that the two documents offered particular strengths. The existing ASPiH document was considered very detailed as a working document but offered excellent depth for further understanding. The proposed document from WMC HIEC was more focussed and could serve as a helpful checklist, complimented by depth in a guidance document. The framework from Yorkshire and Humberside also offered a good example of a detailed document presented in a more user-friendly style.

Defining centres: Throughout the discussion, the broad range of simulation opportunities was referred to. Any framework needs to address this range, which may be an individual responding to a scenario in the workplace or a physically separated formal educational course. As they stand, some of the proposed benchmarks (e.g., open access to facilities, dedicated space, ability to record situations) would exclude many smaller centres. Therefore the group proposed the insertion of ‘where appropriate’ throughout the document. There was some discussion about whether ‘outreach simulation’ should be included within a Centre’s accreditation and responsibility for managing the standard of that work would then fall within the Centre’s remit. It was argued that it would be helpful to think in terms of a national framework, delivered regionally which recognised local conditions. So for example, nationally the standards were the same, within regions there may be different Centres of Excellence in a range of specialties, managed locally.

Key principles of the accreditation process There was widespread agreement of six key principles which should inform the design of any accreditation process:

• the primary purpose of developing accreditation procedures was to ensure better and safer care for patients;


• all competencies must map onto existing national standards where they exist, to minimise the levels of bureaucracy and to incorporate already known best practice;

• accreditation should be supportive and formative, not punitive; • any framework must be flexible enough to meet all simulation situations, from the

individual ‘learning on the job’ to the multi-million pound specialist centre; • accreditation should be based upon principles of peer review; • accreditation should be achievable and affordable, particularly in terms of staff

delivery.

Beyond these principles, discussions centred on three issues: the need for impartiality and credibility in the ownership of the scheme (who should do it?); the need to focus on the development of excellence through accreditation (why do it?); and the mechanisms of delivery (how should it work?).

Impartiality and credibility in ownership (who should do it?) All parties agreed the criteria for ownership of the accreditation process should be independence, professional credibility and capacity to manage the process appropriately. Benchmarking or accreditation should come from within the simulation community, rather than be mandated by one particular professional body. Any process needed to have the capacity to run beyond the lifetime of the HIECs. It was argued that gaining Department of Health backing for the scheme would enhance its credibility further. It was suggested that the Resuscitation Council provided a potential model. In Yorkshire and Humberside, the model used is voluntary, audits are published and there has been positive take-up amongst Centres. HIECs could play a useful role in drawing together the multi-disciplinary framework required. High quality training would be required for the accreditation teams.

The development of excellence through accreditation (why do it?) There was widespread agreement that accreditation should be a formative process to allow Centres to enhance their practice as a result of taking part. Accreditation should therefore have intrinsic value, not simply result in a pass or a fail. Perceived benefits of benchmarking or accreditation included:

• ensuring standards; • ‘remedial’ programme (attending to failings); • developmental programme; • demonstrating competence; • justification for funding; • reassurance for the public; • standardisation for future developments; • sharing of good practice.


The mechanisms of delivery (how should it work?) There was unanimous agreement within the group that the accreditation process should be affordable, deliverable and sustainable to avoid it falling into disrepute and becoming irrelevant. Potential barriers to sustainability included budgets and the use of senior staff time in managing the process. Having ring-fenced funding was considered important. However, balancing the seniority of the staff required to ensure the scheme had credibility against the cost of releasing senior clinicians from front-line patient care was problematic. So a model based on ‘champions’ who cascaded expertise to colleagues would in part address this challenge, as would recognition that some individuals would develop expertise in simulation as an educational tool and others would possess the appropriate clinical specialty knowledge. Expert time could be provided through a reciprocal ‘sharing of time’ agreement for assessment, thus negating the need for payment.

There was widespread agreement that at this stage, a voluntary code of practice would be useful, moving towards a two-stage process whereby Centres conducted a self-evaluation before moving to the formal accreditation process.

Staff expertise: a related point concerning staff expertise was recognised. The need for experts in using simulation as an educational tool was recognised. For example, using feedback as a learning tool requires particular skills and understandings that trainers need to be taught. It was suggested that at present very little CPD was available to address this need.

Conclusion and the way forward The day concluded with a plenary discussion to consider the way forward. Delegates agreed there was value in both documents and other work underway elsewhere. There was also agreement that an accreditation process was needed and would contribute to the development of best practice. The next phase of the project would therefore include:

• On-going discussion with ASPiH on a shared benchmarking/accreditation model which had national applicability, possibly through ASPiH;

Trial of the benchmarking tool with pilot sites/Centres to gather data and determine efficiency and efficacy of process and output.

Reference 1 Society for Simulation in Healthcare (2011) What is healthcare simulation? [online]. http://www.ssih.org/SSIH/SSIH/UploadedImages/PAGR%20Docs/What_Is_Healthcare_Simulation.pdf [Accessed 15/07/2011]


Appendix V West Midlands Central HIEC Benchmarking and Accreditation of Skills and Simulation Centres trial

Post-trial interview schedule

Explain the purpose of the interview is to record Centres’ experiences of the process of gathering the data required for accreditation. Data will be kept confidential to the research team and reported anonymously. Ask for permission to record the interview, transcripts will not be made but recordings used as an aide memoire.

1. Explain that questions are intended to provoke discussion and the aim is to have an

open dialogue.

2. How clear were the instructions you received about what you needed to do and

collect?

3. Can you suggest ways to improve the clarity?

4. How readily accessible was the evidence?

5. Can you describe the process of gathering the evidence together (who? how?)

6. How relevant and appropriate was the evidence asked for in your opinion?

7. Can you suggest any other evidence you think should have been requested?

8. How much time did it take to gather all the evidence together?

9. Did you feel this was an appropriate balance of time for the purpose?

10. One aspect of the process we are keen to explore is that of sharing good practice.

How do you envisage this process helping in that? (eg: would you be happy to share

materials with others?)

11. Overall, can you make any suggestions for improvements to the process?

Thank the participant for their involvement. Offer them a copy of the final report when it is due and invite their continued involvement in the dialogue.


Appendix VI: Draft benchmarking tool - outcomes

Category Outcome Evidence GradeGovernance Structure to monitor activi ties Ski l l s laboratory ca lendar, module timetables , ri sk assessment tra i l , separate finance code 1

Governance Learner eva luation and review Module eva luations fed into s tudent rep group 2

Governance Profess ional body review As part of normal PB review 4

Governance Staff development Staff leads + tra ining videos 2

Organisational Management Audit of faci l i ties use Data in preparation 4

Organisational Management Simulation pol icy re heal th and safety Code of conduct, COSHH assessments 1

Organisational Management Simulation pol icy re s taff and s tudent induction Reported tra ining 4

Organisational Management Speci fic job descriptions Ski l l s = Simulated Pratice Leads , Ski l l s Support Technician 1

Organisational Management Simulation leader appra isa l/development Leadershp s tructure 3

Faci l i ties Categorisation from high to low fidel i ty Included in IPL handbook 1

Faci l i ties Dedicated ski l l s and s imulation faci l i ties Faci l i ties webl ink 1

Learning and Teaching Ski l l s and s imulation i s outcome based/contextual Des ign and eva luation demonstrate contextual 4

Learning and Teaching Demonstration of s tandardisation in del ivery Identi fied variations , under review 4

Learning and Teaching Relationship with heal thcare curricula Lesson plans on request 1

Learning and Teaching Mechanisms for feedback incl . beyond classroom Not currently evidenced 4

Learning and Teaching Measurement of learners ' progress Not currently evidenced 4

Evaluation Staff performance 360 degree s tudent-s taff eva luation not provided; post learning eva luation, not provided 4

Evaluation Learners ' progress Onl ine quizes and MCQS, RLOS, OSCES 4

Evaluation Efficacy of learning and teaching methods 360 degree s tudent-s taff eva luation not provided 4

Evaluation Research and development in field Not evidenced separately 4

Centre A


Category Outcome Evidence GradeGovernance Structure to monitor activi ties Facul ty meetings : agenda, Annual report 3

Governance Learner eva luation and review Not evidenced 4

Governance Profess ional body review Meet commis ioning bodies requirements 3

Governance Staff development Tra ined at St Georges , in house 2

Organisational Management Audit of faci l i ties use Timetabl ing data , review mtgs agenda 1

Organisational Management Simulation pol icy re heal th and safety HEFT mandatory requirements 2

Organisational Management Simulation pol icy re s taff and s tudent induction HEFT mandatory requirements 2

Organisational Management Speci fic job descriptions Reported but not provided (and unwi l l ing to do so) 4

Organisational Management Simulation leader appra isa l/development Reported but not provided (and unwi l l ing to do so) 4

Faci l i ties Categorisation from high to low fidel i ty Asset regis ters , lesson plans , s trategic overview document 1

Faci l i ties Dedicated ski l l s and s imulation faci l i ties Reported, not evidenced, but seen 2

Learning and Teaching Ski l l s and s imulation i s outcome based/contextual Reported but not evidenced 4

Learning and Teaching Demonstration of s tandardisation in del ivery Assumed from lesson plans 3

Learning and Teaching Relationship with heal thcare curricula Courses map onto curricula in Lesson plans 1

Learning and Teaching Mechanisms for feedback incl . beyond classroom Not reported here 4

Learning and Teaching Measurement of learners ' progress Not reported here 4

Evaluation Staff performance Peer review through DASH 1

Evaluation Learners ' progress Not reported here, but feedback emphas ised elsewhere 4

Evaluation Efficacy of learning and teaching methods Pre and post eva luations , informal ta lks 4

Evaluation Research and development in field Reported research projects 2

Centre B


Category Outcome Evidence GradeGovernance Structure to monitor activi ties electronic ca lendar used to monitor bookings 1

Governance Learner eva luation and review reported but not provided 4

Governance Profess ional body review reported but not provided

Governance Staff development reported but not provided 4

Organisational Management Audit of faci l i ties use Not reported 4

Organisational Management Simulation pol icy re heal th and safety reported but not provided 4

Organisational Management Simulation pol icy re s taff and s tudent induction reported but not provided 4

Organisational Management Speci fic job descriptions reported but not provided

Organisational Management Simulation leader appra isa l/development Not reported

Faci l i ties Categorisation from high to low fidel i ty partia l ly reported not evidence provided 4

Faci l i ties Dedicated ski l l s and s imulation faci l i ties reported but not provided 4

Learning and Teaching Ski l l s and s imulation i s outcome based/contextual reported but not provided 4

Learning and Teaching Demonstration of s tandardisation in del ivery reported but not provided 4

Learning and Teaching Relationship with heal thcare curricula reported but not provided 4

Learning and Teaching Mechanisms for feedback incl . beyond classroom no evidence provided 4

Learning and Teaching Measurement of learners ' progress no evidence provided 4

Evaluation Staff performance None provided 4

Evaluation Learners ' progress None provided 4

Evaluation Efficacy of learning and teaching methods None provided 4

Evaluation Research and development in field None provided 4

Centre C


Centre DCategory Outcome Evidence GradeGovernance Structure to monitor activi ties reported but not provided

Governance Learner eva luation and review questionnaires provided 1

Governance Profess ional body review Univeris ty QA process evidence provided 2

Governance Staff development Lis t of s taff courses and tra ining but none provided 4

Organisational Management Audit of faci l i ties use Audit tra i l of prof groups involved & speci fic ski l l s taught 1

Organisational Management Simulation pol icy re heal th and safety risk assessments provided 1

Organisational Management Simulation pol icy re s taff and s tudent induction risk assessments provided

Organisational Management Speci fic job descriptions JDs of manager and educators provided 1

Organisational Management Simulation leader appra isa l/development none provided 4

Faci l i ties Categorisation from high to low fidel i ty description of levels of fidel i ty but no evidence provided 4

Faci l i ties Dedicated ski l l s and s imulation faci l i ties description but no evidence provided 4

Learning and Teaching Ski l l s and s imulation i s outcome based/contextual Lesson plans included, related to national guidance 2

Learning and Teaching Demonstration of s tandardisation in del ivery Speci fic webresource for learners with s tudy guides etc 1

Learning and Teaching Relationship with heal thcare curricula Evidence of mapping aga inst national guidance for profess ions 1

Learning and Teaching Mechanisms for feedback incl . beyond classroom Evaluation forms reported 1

Learning and Teaching Measurement of learners ' progress benchmarking tool reported but no evidence provided 1

Evaluation Staff performance description of 360deg feedback provided but not evidenced 4

Evaluation Learners ' progress Description of continuous assessment but not evidenced 4

Evaluation Efficacy of learning and teaching methods learner eva luation forms provided 1

Evaluation Research and development in field description of planned work provided but not evidenced 4

Comment [AM1]: This column is overhanging the margin and I can’t seem to change it?


Governance Structure to monitor activi ties reported but not provided

Governance Learner eva luation and review reported but not provided

Governance Profess ional body review reported but not provided

Governance Staff development reported but not provided

Organisational Management Audit of faci l i ties use Facul ty annual report 2

Organisational Management Simulation pol icy re heal th and safety Risk assessment provided but not pol icy 3

Organisational Management Simulation pol icy re s taff and s tudent induction Tra ining reported but no evidence of being done 4

Organisational Management Speci fic job descriptions Job description provided 1

Organisational Management Simulation leader appra isa l/development Reported but not provided 4

Faci l i ties Categorisation from high to low fidel i ty Inventory provided 1

Faci l i ties Dedicated ski l l s and s imulation faci l i ties Webl ink provided 1

Learning and Teaching Ski l l s and s imulation i s outcome based/contextual Lesson plans provided, related to national guidance 2

Learning and Teaching Demonstration of s tandardisation in del ivery Reported but not provided 4

Learning and Teaching Relationship with heal thcare curricula Evidence of mapping aga inst national guidance for profess ions 1

Learning and Teaching Mechanisms for feedback incl . beyond classroom Partia l ly reported, equipment being purchesed 3

Learning and Teaching Measurement of learners ' progress No evidence provided 4

Evaluation Staff performance Description but no evidence provided 4

Evaluation Learners ' progress Reported but no evidence provided 4

Evaluation Efficacy of learning and teaching methods Module eva luations reported but not provided 3



Category Outcome Evidence GradeGovernance Structure to monitor activi ties Reported but not provided 4

Governance Learner eva luation and review Reported but not provided 4

Governance Profess ional body review Referenced NMC and SHA reports but not provided 4

Governance Staff development Reported but not provided 4

Organisational Management Audit of faci l i ties use Reported but not provided 4

Organisational Management Simulation pol icy re heal th and safety Reported but not provided 4

Organisational Management Simulation pol icy re s taff and s tudent induction Reported but not provided 4

Organisational Management Speci fic job descriptions Not provided 4

Organisational Management Simulation leader appra isa l/development Not reported 4

Faci l i ties Categorisation from high to low fidel i ty Referred to in benchmarking document and gave tour on interview day 3

Faci l i ties Dedicated ski l l s and s imulation faci l i ties Partia l ly provided and gave tour on interview day 3

Learning and Teaching Ski l l s and s imulation i s outcome based/contextual Sample lesson plan provided 1

Learning and Teaching Demonstration of s tandardisation in del ivery Referenced lesson plan and course plan DVD, no other tra ining 3

Learning and Teaching Relationship with heal thcare curricula As before 3

Learning and Teaching Mechanisms for feedback incl . beyond classroom Referenced 3 year ski l l s passport but not provided 4

Learning and Teaching Measurement of learners ' progress Referenced continual assessment but no evidence provided 4

Evaluation Staff performance Referenced peer reviews but no evidence provided 4

Evaluation Learners ' progress Referenced eva luations but no evidence provided 4

Evaluation Efficacy of learning and teaching methods OSCEs but no evidence provided 4


Centre F


Appendix 2: draft paper on the development of the procedural skills passport

The Birmingham Clinical Procedural Skills Passport. J Parle & D Morley, UoB

Background

Clinical skills procedures are a key part of a junior doctor’s portfolio of skills; many are also

‘risky’ in terms of patient safety if not performed proficiently. Concerns about the

deteriorating acquisition of skills by medical students have been reported previouslyiv,.The

Birmingham Clinical Skills Passport has its origins in an earlier programme for 5th year

students designed to ensure they were competent, at the point of graduation, in the range of

procedural skills defined by the GMC’s Tomorrow’s Doctorsv.

Brief description of process

From 2007 onwards, Year 3 medical students at University of Birmingham have been

provided with a clinical procedural skills passport as they begin their first hospital

placements. The passport contains information outlining the procedures in which students

need to develop competence (see Table 1). The passport blueprints skills acquisition to the

various blocks and rotations in the MBChB programme and specifies when students should

be taught how to perform a procedure, initially in simulation, and when they should

undertake observed practice on patients; these skills and observations are undertaken in the

hospitals and general practices. During the 5th year the students receive some

supplementary teaching at the medical school on procedures that are not so easily taught or

undertaken as observed practice by students in a clinical environment. Towards the end of

the 5th year students are once again called to the medical school and are tested in simulation

in the higher risk procedures (highlighted in Table 1) included in the passport. This provides

an extra layer of assurance that students have indeed achieved the required level of

competence in these procedures.

The passport design

The passport is designed to be carried easily by students on clinical placement, in an A5

booklet format. The first part of the passport contains an introduction and information about

how the procedures are blueprinted to the course (e.g. urinary catheterisation is blueprinted

to the 4th year, whilst venepuncture is blueprinted to both the 3rd year and the 5th year to


ensure revision and re-training close to qualification). Thereafter a double page spread is

devoted to each procedure. The left hand page is devoted to teaching in simulation, and the

right hand page is devoted to space for the students to record their observed practice. The

paper-based passport provides an easily accessible way for students to write in data, to

show to clinicians and patients as evidence of the stage they have reached and of the need

to undertake observed practice, and an easy way for clinicians to sign to say a student has

performed a procedure satisfactorily. This paper passport is accompanied by an online

database, into which students are required to input the data from their passports. The

database allows students’ progress to be monitored (and for faculty to intervene if a student

is falling significantly behind, often an early sign of students in difficulties) and also enables

students to be provided with a replacement passport should theirs be lost. The students are

responsible for uploading all data, but hospital-based administrative staff also go online to

validate data about the students’ training in simulation. Students are trusted to complete the

database honestly. The assessment process in year 5 helps ensure that students do try to

undertake reasonable amounts of observed practice. Local administrators also look at

students’ passports from time to time.

Data

The passport has now been part of the MBChB for over 5 years. With a cohort size of

approximately 400 students a year, and with 30 or so procedures to cover, this has been

quite a significant undertaking, and one that has largely been devolved to our NHS teaching

partners to deliver.

How much observed practice should a student be required to do? Clearly, the more practice

a student can have the better, but this needs to be balanced against practical considerations

such as the number of patients requiring procedures to be performed who can provide

students with the opportunity to practice, and the fact that students need to devote time and

energy to other aspects of their clinical learning. In Birmingham the default position is that

students must undertake at least 6 instances of observed competent clinical performance of

the procedure, after having demonstrated competence in a simulation environment. In other

words, although the student may perform the procedure more times than this, only

competent performance (not ‘learning curve’ performances) are recorded. However, for

some procedures which are less complex (for example urine testing), we have reduced this.

It is perhaps worth considering that over the last 5 years, when taking into account the


numbers of procedures, the numbers of students, and the requirements for observed

practice, about 150,000 procedures have been performed by Birmingham medical students

on patients. We clearly owe a debt of gratitude both to the patients who allow medical

students to perform procedures on them and the clinicians who observe the medical

students undertaking the procedures.

Collaboration with Keele and Warwick

More recently the passport has been shared with the other West Midlands Medical Schools,

Keele and Warwick, and has, in adapted form, been adopted by them. This is useful

because, as the majority of our graduates remain in the West Midlands for their Foundation

years, local Trusts have a good understanding of what their new locally qualified recruits can

do.

Materials

It is intended that students should be as well prepared as possible for the teaching they

receive in how to perform a procedural skill. Therefore, students are provided with a story

board depicting the stages of the procedure. The storyboards are accessed via the VLE.

(Virtual Learning Environment) Students are required to be familiar with these before they

attend a teaching session. The storyboards were developed by the clinical skills team at the

University Hospital Birmingham, and are quality assured both by clinical academics at the

medical school and by the wider clinical skills network (a loose affiliation of the clinical skills

centres which UoB medical students attend). More recently a series of videos called

‘Tomorrow’s Clinicians’ has been produced by the education faculty and clinical skills team

at Russells Hall Hospital, one of the local teaching hospitals. These are now available to

students, and are used both for teaching purposes and for student self-study. The next

stage is the development of materials that can be accessed via mobile devices. These

‘apps’ will provide access to the videos, to storyboards and to quizzes so that students can

test themselves. It is anticipated that students will use these mobile resources to refresh

their knowledge, perhaps as just-in-time (re)learning (for example, when just about to do a

procedure which they have not done for a while).


The role of students in developing the passport

Students have played an important part in the development of the clinical skills passport. In

general the students have been very supportive of the concept. Their evaluations of the

process helped identify some areas where difficulties were experienced, for example while

the early versions were more prescriptive about when and where students should acquire

particular skills, the later versions are more flexible. Where there have been issues to do

with materials, technology, or student expectations, the Medical School has been able to

respond. We have also been able to feed this evaluation back to the Trusts where students

are on placement, and this can help resolve issues where, for example, it has been difficult

to undertake observed practice. When the passport was first developed, a student focus

group was convened. This was instrumental in guiding a redesign of the passport, and

online database. The students also identified reps at each hospital who gathered information

from students about the best places to undertake observed practice in the different

procedures in the hospital. This is then shared with other students, and made available to

students coming to the hospital in subsequent rotations.

Issues

Perhaps the major issues beyond those reported so far concern how much responsibility a

student can be given to perform procedures in a relatively loosely supervised setting. The

main driver behind establishing the passport was that it would be a good way to help make

students ready for their Foundation Years. To prepare students for the responsibilities of

being a doctor, it would be beneficial to allow students some responsibility while they are

students. To do this, for each procedure, the passport contains a ‘sign off’ section at the end

of the observed practice page. The wording of the section, where students are signed off is

– ‘On observation this student was able to perform this procedure competently and

independently.’ This has caused some problems as some people have questioned how

competence is interpreted, have worried that if there are future problems with the student

performing this procedure, they will be held to account for having wrongly ‘signed off’ the

student. Students have not been given very much opportunity for more loosely supervised

practice. This may in part be due to the short nature of the rotations on the programme, and

may be resolved by the recent curriculum review which keeps students at one placement for

longer. If staff become more personally familiar with, and confident in, a student’s ability

they may be more willing to delegate some responsibility.


Is it worth it?

Anecdotal reports from the Trusts who take Birmingham graduates indicates that new

Foundation Doctors are showing greater knowledge and skills in the procedures required by

the passport and that this is reducing the training burden at the start of the Foundation Year.

Hopefully this leaves the new doctors more able to concentrate on other aspects of their new

roles, rather than learning the basics of the procedures they need to perform. It is hoped this

will reinforce Trust’s desire to teach and provide opportunities to medical students, as they

will reap the rewards of better prepared graduates.


Table 1: the procedures in the passport

General Aspects of Procedural Skills Therapeutic Procedures

Aseptic Non-Touch Technique Airway Management (Bag and Mask)

Hand-washing Airway Management (Intubation / LMA)

Moving and Handling Airway Management (Oral Airway)

Protective equipment Blood transfusion 3

Scrubbing Up Central Venous Access Devices 1

Sharps and clinical waste disposal Infusions – setting up a fluid infusion

Diagnostic Procedures Injections Intramuscular 2

Arterial Blood Gas Sampling 1 Injections - IV bolus

Blood Culture – Taking 1 Injections Subcutaneous

Blood Glucose Measuring Naso-gastric Tube insertion 1

ECG – Managing an ECG Monitor Nebuliser (Using a) 2

ECG – Performing a 12 lead ECG 2 Oxygen Therapy – administering Oxygen 2

Ophthalmoscopy Parenteral administration: making up drugs for

Oxygen Saturation – Transcutaneous monitoring Suturing (skin) 1

Swabs – taking nose, throat and skin swabs Urinary Catheterisation 2

Urine Analysis (using Multistix) Venous Cannulation and Flushing - Establishing peripheral venous access. 2

Venepuncture and Blood Sample Management Wound care and basic wound dressing

1 These procedures are taught at the medical school as part of the 5th year programme. Therefore there is no expectation that students will undertake observed practice.

2 These procedures are assessed as part of the 5th year programme, but are taught to students while at clinical placement. There is an expectation that students will have had some observed practice in these procedures

3 This procedure is taught in theory and simulation to students, but students are expected to observe the procedure being performed, not to perform it themselves.


Appendix 3: report on RCT of acquisition of musculoskeletal examination skills

Summary of quantitative results of an RCT comparing simulated patients to usual learning in

the acquisition by medical students of musculoskeletal examination skills

Background: Acquisition of the skills to perform musculoskeletal examinations is core to

doctors’ training but health service pressures and the (morally correct) desire not to inflict

more pain on patients appear to be acting to reduce the competence of newly qualified

doctors. We hypothesised that structured, scenario based training (SSBT) using simulated

patients (SPs) would result in better acquisition of musculoskeletal examination skills than

‘usual learning’ i.e. clinical experience and teaching on wards and in outpatients.

Summary of work: We performed a Randomised Controlled Trial in two year cohorts in a

single medical school, randomising students to SSBT with SPs or to usual learning; the

musculoskeletal module is in year 4 of the 5 year course. All students were assessed in a

single Objective Structured Clinical Examination (OSCE) station on their musculoskeletal

examination skills; for cohort 1 (n = 208) the OSCE was in their 5th year; for cohort 2 (n =

379) the OSCE was in 4th year. Delay between finishing the musculoskeletal module and

sitting the OSCE examination was ~8 months (Inter-Quartile Range 4-12) for the intervention

group and ~7 months (4-10) for the control group. OSCE examiners and statistician were

blind to allocations.

Summary of results: There was a difference in mean musculoskeletal OSCE station scores

of 3 percentage points (t=2.50, p=0.013) comparing the intervention to the control arm There

was no attenuation of difference in OSCE scores with time between module and OSCE.

Conclusions: Scenario-based structured musculoskeletal skills training with SPs produces

significant improvement in medical students’ musculoskeletal OSCE scores which does not

appear to reduce over time.

Take-home message: SPs are effective in training medical students in musculoskeletal

examination skills and more effective than traditional learning in clinic


Reality Check: Medical students’ perceptions of using volunteer and professional simulated patients to learn musculoskeletal examinations

ABSTRACT

Context: little is known about the effectiveness of using simulated patients (SPs) to support

the acquisition by medical students of clinical examination skills. We performed a

Randomised Controlled Trial (RCT) comparing professional to volunteer patients to ‘usual

learning’ in clinics and wards. We report here on the results of focus groups exploring

medical student perceptions of learning from SPs.

Methods: 51 students took part in focus groups; data was transcribed and analysed using

standard qualitative methods, identifying themes.

Results: students clearly valued the involvement of SPs; volunteers were valued for being

‘real’ and the professional SPs for their educational expertise in feedback.

Conclusions: SPs contribute to medical students’ acquisition of musculoskeletal

examination skills in various ways and both models have something to offer over and above

traditional clinic and ward-based learning.


Appendix 4: Results of intimate examination survey of newly qualified doctors in UK; (199 respondents)

Showing % of respondents performing 0, 1, 2-3, 4-5, 6-9, 10-19, 20-49 or 50 and over examinations as a student

Female rectal examination

Male rectal examination

Series1, 0, 42

Series1, 1, 24

Series1, 2 to 3, 32

Series1, 4 to 5, 5 Series1, 6 to 9, 3

Series1, 10 to 19, 5

Series1, 20-49, 1

Series1, 0, 13

Series1, 1, 19

Series1, 2 to 3, 31

Series1, 4 to 5, 18


Series1, 20-49, 3


Female breast

Male genitalia

Series1, 0, 6 Series1, 1, 8

Series1, 2 to 3, 30

Series1, 4 to 5, 23

Series1, 6 to 9, 16 Series1, 10 to 19,

15

Series1, 20-49, 4


Series1, 2 to 3, 36

Series1, 4 to 5, 14


Series1, 20-49, 2


Female pelvic examination



Series1, 6 to 9, 20

Series1, 10 to 19, 15

Series1, 20-49, 6


Appendix 5: CIPSET report

Centre for Inter-professional Simulated Education and Training (CIPSET)

The NHS WMSHA was supportive of extending the work of their previously funded CITEC

project and the CIPSET proposal aimed to ensure that this initiative continued to develop

and become self sustainable for the longer term.

Background In the UK, patterns of patient care are changing rapidly, with increasing emphasis on patient

safety and care that is delivered by skilled multi-disciplinary teams. Patient safety is

receiving particular scrutiny (HM Government Health Select Committee 2009):

approximately 10% of hospital admissions experience an adverse event and settlement of

clinical negligence cases costs the NHS approximately £400m/year (Department of Health

2001). It is recognised that poor communication between members of the multi-disciplinary

team is a contributory factor in many untoward incidents (for example Reader, Flin &

Cuthbertson 2007). These factors have led many regulatory and professional bodies to

require educational providers to integrate Inter-Professional Education (IPE) and simulation

into education and training of health professionals.

Interprofessional simulations, which bring together different professional groups to

participate in authentic clinical scenarios, are emerging as an important educational tool both

for meeting the requirements of regulatory bodies at the undergraduate level and for working

with qualified teams of staff based in hospital Trusts. However, developing high quality inter-

professional simulations is a resource and time intensive process, requiring input from a

range of health professional groups, clinically authentic scenarios and tutors skilled in both

simulation and IPL. Whilst there is some evidence to support the educational effectiveness

of inter-professional simulations, more work needs to be done to identify the most

appropriate models for best practice (Zhang et al 2011).

Recent research suggests that influencing the attitudes of qualified staff teams is difficult

(Kenaszchuk, MacMillan and van Soeren 2011). It is important, therefore, that pre-

registration students begin to develop the knowledge and understanding required for

collaborative working and appropriate consideration of patient safety. The collaboration will

therefore focus initially on provision for pre-registration students. However, we anticipate that

the full collaboration, once established, will also support work with qualified staff teams. The


collaboration also has the potential to form part of a wider regional strategy for the

development of simulation-based education and training per se.

It is advantageous for education providers and Trusts to share expertise and resources for

the development of inter-professional simulations and some collaborative work is already

underway. The Centre for Innovation and Training in Elective Care (CITEC) project, which

involves the Universities of Birmingham, Birmingham City and Worcester and local Trusts

has developed both electronic and ‘live’ inter-professional simulations for undergraduate

students from 6 professional groups. The Hollier simulation centre at Heart of England

Foundation Trust has established collaboration with University Hospitals Birmingham (UHB),

the Royal Centre for Defence Medicine (RCDM) and the University of Birmingham; has

developed a simulation training programme for undergraduate medical students and has a

faculty network trained to teach in simulation settings.

The CIPSET proposal aimed to build on these initiatives by establishing collaboration for

the development of inter-professional simulated education and training (CIPSET), which will

maximise the educational and cost effectiveness of inter-professional simulation training

within the region. As full establishment of the collaboration was thought to take 2-3 years the

opportunity to work with the West Midlands Central HIEC, represented an opportunity to

make progress towards this goal.

The CIPSET proposal was considered as an alignment with the simulation theme objectives

to:

o to use the HIEC partnership to build upon existing work that identified the

landscape of simulation across the West Midlands region

o to lead collaboration between local and specialist simulation centres

o to lead the development of multi-professional learning, patient and carer

learning, team learning and the testing of new devices

o to facilitate the development of the expertise of simulation providers

The main work associated with this proposal consisted of six major deliverables. These are

listed below, together with an outline of outcomes achieved.


4.6.2 Completion and launch of a web portal relating to inter-professional simulation, with a particular emphasis on team training

An innovative new web portal, known as Teamwork Matters, has been built and populated

with resources relating to team work training, including a series of case studies in patient

pathways that highlight the role of different professions, a ‘what do different professions do

activity’, a series of podcasts relating to particular team work issues, resources relating to

human factors and materials from the simulations and events completed as part of the

CIPSET collaboration. This is a fully responsive website, allowing for use on all platforms

and mobile devices, with wiki-style page designs to allow for collaborative content creation.

The pilot version of the web portal is known as Teamwork Matters, and can be viewed at:

http://mymds.bham.ac.uk/teamworkMatters. Final formatting and testing is in progress and

the portal will be made available to students and clinical educators at participating

institutions as part of the development of the team training network (see below).

4.6.3 Development and piloting of undergraduate inter-professional (IP)

simulations relating to chronic kidney disease (CKD)

In total, 16 half day inter-professional simulation sessions relating to CKD were held (3 in

November 2011, 6 in February 2012 and 7 in May 2012). Sessions were held in three sites

(Universities of Birmingham, Birmingham City University and Worcester).

As part of the sessions, a team observation tool was piloted. This allowed facilitators and

participants to assess how mixed groups of students undertook the medicines management

aspects of CKD and provided detailed information to inform both feedback and the

development of future scenarios.

Approximately 120 students from medicine (60), nursing (55) and pharmacy (8) attended a

half day session. Students evaluated the sessions and the CKD scenario very positively.

Materials for running these sessions are now available to tutors in the collaboration in hard

copy and electronically through the CIPSET web portal.

Work with the team observation tool was presented at the All Together Better Health VI IPE

conference, Kobe, Japan in October 2012.

http://mymds.bham.ac.uk/citec/testSite/crisp/home.asp


4.6.4 Development and piloting of inter-professional simulation sessions that incorporate human factors training

In total, 16 half day sessions were held, 4 at the Hollier Simulation Centre and 12 at

Russell’s Hall hospital (RHH) (scenarios of chest pain and a day case patient with diabetic

foot respectively). Both sets of sessions included briefing on the importance of human

factors in effective team working.

Approximately 140 students (from medicine, nursing, operating department practice and

physiotherapy) attended a session at RHH. Forty-three students (from medicine (18),

nursing (18) and radiography (7)) attended a session at the Hollier Centre.

Students evaluated the inclusion of human factors in these sessions very positively.

However, it was apparent that students would benefit from earlier introduction to human

factors training. As a result of experience with the sessions, resources relating to human

factors training have been incorporated into the CIPSET web portal to provide preparatory

materials for students and information for tutors.

4.6.5 Hosting a half ‘away’ day for staff engaged in inter-professional simulation training across the West Midlands Central region

A half away day was held on 22nd February 2012 at Birmingham City University, with

external speakers Martin Bromiley (Clinical Human Factors Group) and Dr Ian Curran

(London Deanery).

In total, 27 delegates attended from 7 different institutions. Participants evaluated the half

day very positively. Of the 15 evaluation forms received, 13 rated the event as excellent and

the remaining 2 as good. Attendees particularly valued the contribution of Martin Bromiley

from the Clinical Human Factors Group.

As a result of the success of the event, plans for future events were incorporated into the

business case proposal submitted to the SHA.

4.6.6 Development and piloting of a short CPD course for facilitators of inter-professional simulation sessions

Two half day sessions were held on the 14th and 26th June 2012 with 13 and 20 attendees

respectively. The programme included an introduction to IP simulation, making the most of


your professional group and techniques of debriefing. In total, 33 tutors attended (13 on the

first day and 20 on the second day). Evaluations from the two days were very positive.

Relevant materials and activities from the days are now available on the CIPSET web portal.

Work on the half day training session was presented at a forthcoming International Clinical

Skills/Simulation conference, May 2013

4.6.7 Preparation of an outline business case for the establishment of a long term inter-professional simulation collaboration for consideration by NHS West Midlands or its successor.

A proposal for a ‘regional’ team training network was submitted to NHS West Midlands at the

end of November 2012, accompanied by letters of support from six different institutions.

Funding was requested for an initial 3 years, with further on going funding subject to

satisfactory completion of objectives.

Further funding of £90K was agreed by NHS West Midlands in early March 2013. Further

funding will be discussed with the LETB once it is established. A development plan is being

prepared for discussion with Midlands and East at the end of May 2013.

References:

Department of Health (2001). Building a Safer NHS for Patients: implementing an

organisation with a memory. Retrieved from:

http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/publicationsandstatistics/publ

ications/publicationspolicyandguidance/browsable/DH_4097460

[Accessed 05 09 13]

HM Government Health Select Committee Report on Patient Safety, 2009. Retrieved from:

http://www.publications.parliament.uk/pa/cm200809/cmselect/cmhealth/151/151i.pdf

[Accessed 05 09 13]

Kenaszchuk C, MacMillan K, van Soeren M, Reeves S 2011. Inter-professional simulated

learning: short-term associations between simulation and inter-professional collaboration

BMC Medicine 9 29-39

http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/publicationsandstatistics/publications/publicationspolicyandguidance/browsable/DH_4097460

http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/publicationsandstatistics/publications/publicationspolicyandguidance/browsable/DH_4097460

http://www.publications.parliament.uk/pa/cm200809/cmselect/cmhealth/151/151i.pdf


Reader, T.W., Flin, R., & Cuthbertson, B.H. (2007). Communication skills and error in the

intensive care unit. Current Opinion in Critical Care 13(6) 732-726.

Zhang, C., Thompson, S., & Miller, C. (2011). A review of simulation-based inter-

professional education. Clinical simulation in nursing 7: e117-e126

i Draper H, Ives J, Ross N, Parle J. Medical education and patients’ responsibilities: back to the future? Journal of Medical Ethics, 2008, 34: 116-119 ii ten Cate. Nuts and Bolts of Entrustable Professional Activities. Journal of Graduate Medical Education, 2013; DOI: http://dx.doi.org/10.4300/JGME-D-12-00380.1 iii IC Mcmanus, P Richards, BC Winder. Clinical experience of UK medical students. Lancet, 1998, 351; 802-803. iv How good are newly qualified doctors at digital rectal examination? J. M.-C. Yeung, H. Yeeles, S.-W. Tang, L. L. Hong and S. Amin. Colorectal Disease _2011. 13, 337–340 doi:10.1111/j.1463-1318.2009.02116 v GMC tomorrow’s doctors

Documents

WEST MIDLANDS CENTRAL HEALTH INNOVATION AND … · 2019-02-13 · from the data previously referred to, almost all newly qualified doctors have done very few such examinations. Reasons