The ABCs of EPAs – an overview of Entrustable Professional …medaproc.facmed.unam.mx/wp-content/uploads/2017/11/The... · 2017. 11. 15. · been through the copyediting, typesetting,

This article has been accepted for publication and undergone full peer review but has not been through the copyediting, typesetting, pagination and proofreading process which may lead to differences between this version and the Version of Record. Please cite this article as doi: 10.1111/imj.12914

This article is protected by copyright. All rights reserved.

The ABCs of EPAs – an overview of ‘Entrustable Professional Activities’ in medical

education

Carlos El-Haddad,1,2 Arvin Damodaran,3 H. Patrick McNeil,4 and Wendy Hu1

1 School of Medicine, University of Western Sydney

2 Department of Rheumatology, Liverpool Hospital, New South Wales

3 Prince of Wales Clinical School, University of New South Wales

4 Faculty of Medicine and Health Sciences, Macquarie University, New South Wales, Australia

Word count: 1716 words

(Abstract 232 words)

Correspondence

Carlos El-Haddad, School of Medicine, University of Western Sydney, Locked Bag 1797, Penrith,

NSW 2751, Australia. Email: [email protected] Phone: +61 2 4620 3702 Fax: +61 2 4620

3890

mailto:[email protected]


Author Contributions: Carlos El-Haddad developed the initial manuscript and finalised for

submission. All authors substantially contributed to the design of the work, revising the paper

critically regarding important intellectual content, approving the final version, and agreeing to be

accountable for the integrity of the work.

Abstract

Consultants regularly need to decide whether a trainee can be entrusted to perform a clinical activity

independently. ‘Entrustable Professional Activities’ (EPAs) provide a framework for justifying and

better utilising supervisor entrustment decisions for trainee feedback and assessment in the

workplace. Since being proposed by Olle ten Cate in 2005, EPAs are emerging as an integral part of

many international medical curricula, and are being considered by the Royal Australasian College of

Physicians in the current review of physician training. EPAs are defined as tasks or responsibilities

that can be entrusted to a trainee once sufficient competence is reached to allow for unsupervised

practice. An example might be to entrust a trainee to ‘Initiate and co-ordinate care of the palliative

patient’ with only off-site or indirect supervision. Rather than attempting to directly measure each of

the many separate competencies required to undertake such a complex task, EPAs direct the trainee

and supervisor’s attention to the trainee’s performance in a limited number of selected,

representative, important day-to-day activities. EPA based assessment is gaining momentum,

amongst significant concerns regarding feasibility of implementation. While the optimal process for

designing and implementing EPAs remains to be determined, it is an assessment strategy where the

over-arching goal of optimal patient care remains in clear sight. This review explores the central role

of trust in medical training, the case for EPAs, and potential barriers to implementing EPAs based

assessment.

Key words

Entrustable professional activities, trust, workplace based assessment


During a busy clinic, Simon, your new basic physician trainee calls you. He is concerned about a

deteriorating patient with advanced pulmonary fibrosis. Unfamiliar with his level of experience, you

reluctantly advise him to discuss end-of-life care plans with the patient’s family. “After all,” you

reason – “…Simon has passed his exams, so he must be competent enough to complete this task…”

Introduction

Current approaches to assessment suggest that a coordinated program of multiple types of

complementary assessments may produce a more meaningful result than simply the sum of its

parts. This acknowledges that no one method of assessment can cover all the required knowledge,

skills and attitudes required of medical experts.1 The Royal Australasian College of Physicians (RACP)

training program includes written and clinical examinations with a workplace based assessment

component known as PREP (Physician Readiness for Expert Practice). ‘Entrustable professional

activities’ (EPAs) have emerged internationally as tools for assessing on-the-job performance, and

may play an important role in revising the workplace component of RACP curricula. However given

their recent development, many clinicians and educators are unfamiliar with this educational tool.

The concept of EPAs was introduced by ten Cate in 2005 as a novel method of assessment in medical

education. The aim was to ‘help supervisors in their determination of competence of trainees’.2 EPAs

are essentially units of significant clinical work. They are defined as tasks or responsibilities to be

entrusted to a trainee once sufficient competence is reached to allow for unsupervised practice.3

What does ‘entrustable’ mean?

Trust plays a central role in the daily interactions of supervisors and trainees. Supervisors regularly

need to decide what level of trainee supervision is needed for safe patient care. Assessment using

EPAs formalises these daily clinical entrustment decisions, by providing a framework to collect


evidence and document what clinical supervisors are already doing – using their expert judgement

based on their observations of the trainee’s proficiency.

When dealing with EPAs it is important to highlight the relationship between ‘activities’ and

‘competencies’. Competencies are general attributes of a doctor – for example, ‘The ability to apply

interpersonal and communication skills’.4 In contrast, activities are elements of professional work –

for example, ‘Discuss end of life care with a patient and family’.

Tasks appropriate for an EPA must be: observable, measurable, executable within a given timeframe,

and suitable for entrustment decisions. An EPA grading system has been developed based on the

amount of trust a supervisor has in a trainee. There are five levels of trainee proficiency;4

1. has knowledge

2. may act under full supervision

3. may act under moderate supervision

4. may act independently (with ‘supervision at a distance’)3

5. may act as a supervisor and instructor

So how do supervisors make ‘entrustment decisions’? Unsurprisingly, the literature suggests that

this is a complex multifactorial process. In a survey-based study of supervisors and trainees, four

main domains of ‘entrusting factors’ were identified: trainee factors (e.g. confidence), supervisor

factors, task factors, and systems factors.5 Furthermore in a qualitative study, Dijksterhuis et al

found significant variability in how obstetrics and gynaecology supervisors make entrustment

decisions.6 Reported methods included direct observation, discussions with colleagues, and even

‘blind faith’.


Given the complexity of an entrustment decision, it has been argued that sustained observation of

the trainee performing clinical work over time allows supervisors to make better decisions.7 The

success of using trust as the basis for assessment therefore hinges on effective implementation in

the real world context of clinical training.

The case for EPAs

With the advent of competency-based education (CBE), traditional assessment techniques

attempted to directly measure a trainee’s ‘competence’. However the literature suggests this is

neither a practical nor useful exercise, highlighting the need for a new approach to assessment.

The first problem with measuring competencies, rather than focussing on activities, is that

competence does not necessarily predict performance. A trainee may appear to be ‘competent’ in

an examination setting, but exhibit poor performance in daily clinical work.8 Most clinical supervisors

can give examples of trainees who perform exceptionally in written examinations and simulated

tests of clinical skills, but poorly in daily clinical work, and vice versa. Consider the introductory

scenario - if Simon was deemed able to apply interpersonal and communication skills, it does not

necessarily mean that he can be trusted to discuss end of life care with a patient.

Furthermore, it may not even be possible to actually measure competencies in the workplace

environment. In 2009 Lurie, Mooney and Lyness systematically reviewed published evidence that

the Accreditation Council for Graduate

Medical Education’s (ACGME) six general competencies can be measured in a reliable way.9 They

found that current measurement tools were not able to measure competencies independently of

one another. For example items on global rating forms tended to cluster into one or perhaps two

domains, rather than the six ACGME competencies on which they were based. The authors resolved

these competencies may ‘exist in a realm outside of measurement’. These problems paved the way

for a novel approach to assessment in CBE.


According to Miller’s well known framework for clinical assessment (“Miller’s pyramid”), trainees

typically progresses through four stages of development;10

1. Knows (knowledge)

2. Knows how (competence)

3. Shows how (performance)

4. Does (action)

Instead of attempting to directly measure ‘competence’, EPAs assess trainees at the ‘Does’ level in

the context of daily clinical practice. Descriptors of a typical EPA are shown in Table 1. For a more

detailed sample EPA, refer to the worked example published by ten Cate using the task of

completing a patient handover.3

Developing EPAs for post-graduate training

The rapid international uptake of EPAs into education and training programs has been remarkable

and perhaps reflects an underlying need for better assessment methods in CBE.

EPAs are being incorporated into numerous post-graduate training curricula including paediatrics,

internal medicine, family medicine, psychiatry, obstetrics and gynaecology, and nursing.11-13 Full-

scale implementation has been undertaken in Australia and New Zealand (psychiatry training) and

the Netherlands (obstetrics and gynaecology training).11 Another notable example is the Association

of American Medical Colleges’ list of EPAs describing a core set of behaviours to be expected from all

medical graduates entering residency.14

The suggested number of EPAs for a full postgraduate program is 20-30.13 The selected EPAs should

be ‘critical activities that constitute a specialty’, which can be unique to the practice setting or


context where the assessment occurs.4 By definition, each EPA will require multiple competencies

which can be mapped in a two dimensional grid. EPAs can also specify the stage of training, so as a

trainee progresses through terms, a portfolio of completed EPAs can develop. Indeed, the concept of

using a digital portfolio to track EPAs through a learner’s lifetime was recently proposed.15

There remain, however, important questions on EPA development and implementation. How should

EPAs be selected? How should educators decide on the content, or the ‘Required knowledge and

skills’ of EPAs? What are appropriate ‘Assessment methods’?

ten Cate has suggested “…there is no single correct mode of description and application of EPAs…”.16

Accordingly, multiple approaches to EPA design have emerged recently highlighting both potential

benefits and pitfalls of this educational tool.

The Royal Australian and New Zealand College of Psychiatrists (RANZCP) provide a model example of

how to select EPAs and schedule them over the course of training.17 EPAs were chosen through a

college-wide consultative process using surveys and expert panels. Respondents provided feedback

on whether trainees should be entrusted with a specific list of EPAs by the first stage of training. The

result was the selection of 4 EPAs from a list of 30.

Building on such work, Aylward, Nixon and Gladding proposed a model for EPA assessment

development using ‘resident handoff’ as an example.18 A nine-step process was used to create the

EPA which can be adopted by other institutions. Their rigorous methodology seems appropriate

given that only a limited number of EPAs form the defining activities for a particular specialty. Key

aspects of their approach were wide consultation and an iterative process allowing for multiple

revisions. Sources used for EPA development included literature reviews, curriculum material from


medical boards, and expert opinion. Behavioural descriptors for each level of entrustment were

written.

Other approaches have included using a modified Delphi technique and more recently a sequential

qualitative and quantitative mixed methods approach.19, 20 However such heterogeneity in the

design process has led to significant variation in developed EPAs. One potentially significant pitfall is

producing an exhaustive list of detailed tasks resembling a checklist, losing the holistic value of EPA

based assessment. For example, a published list of 76 EPAs for ambulatory practice in family

medicine carries this risk.21 Further research in the area of EPA design is needed to inform this

process.

Finally, the potential role of EPAs in undergraduate medical education and internship has recently

emerged.22, 23 This could perhaps improve continuity between undergraduate and post-graduate

medical training, as EPAs are benchmarked against different levels of clinician supervision, which can

be applied across clinical settings.

Is the workplace ready for EPAs?

Significant concerns have been raised regarding the application of EPAs into constrained hospital

based training programs. In a feasibility study evaluating implementation of internal medicine EPAs,

Hauer et al identified multiple barriers including limited trainee-supervisor contact and interns

prioritising immediate work duties over learning activities.19

EPA based assessment hinges on trust, and trust takes time - a precious resource for both

supervisors and trainees. Furthermore, trainee rotations may need to be adjusted to accommodate

EPA based training programs. Factors such as this have led some authors to conclude that the

“…flexibility of current workplaces is insufficient for EPAs”.24 Given that evaluation studies of EPA

based curricula are still pending, one would be forgiven for maintaining status quo for the time

being.


Conclusion

Let us return to the opening scenario. Can EPAs help this supervisor? Perhaps the more important

question is whether EPAs can improve the quality and safety of care this patient receives. To quote

Buhyan et al, “Ultimately, the EPAs should be a list of what the public can expect from their family

physicians”.25 In theory, an EPA based training program can equip Simon’s supervisor to make an

informed, safe entrustment decision. However, this assumes rigorous design and application of this

educational tool in a supportive learning environment. We keenly await the progress and evaluation

of EPA implementation to shed light on this question.


References

1 van der Vleuten CP, Schuwirth LW, Driessen EW, Govaerts MJ, Heeneman S. 12 Tips for programmatic assessment. Med Teach. 2014: 1-6. 2 ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005; 39: 1176-7. 3 ten Cate O, Young JQ. The patient handover as an entrustable professional activity: adding meaning in teaching and practice. BMJ Qual Saf. 2012; 21 Suppl 1: i9-12. 4 ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82: 542-7. 5 Choo KJ, Arora VM, Barach P, Johnson JK, Farnan JM. How do supervising physicians decide to entrust residents with unsupervised tasks? A qualitative analysis. J Hosp Med. 2014; 9: 169-75. 6 Dijksterhuis MG, Voorhuis M, Teunissen PW, Schuwirth LW, ten Cate OT, Braat DD, et al. Assessment of competence and progressive independence in postgraduate clinical training. Med Educ. 2009; 43: 1156-65. 7 Hirsh DA, Holmboe ES, ten Cate O. Time to trust: longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014; 89: 201-4. 8 Rethans JJ, Norcini JJ, Baron-Maldonado M, Blackmore D, Jolly BC, LaDuca T, et al. The relationship between competence and performance: implications for assessing practice performance. Med Educ. 2002; 36: 901-9. 9 Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 2009; 84: 301-9. 10 Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990; 65: S63-7. 11 Englander R, Carraccio C. From Theory to Practice: Making Entrustable Professional Activities Come to Life in the Context of Milestones. Acad Med. 2014. 12 Giddens JF, Lauzon-Clabo L, Morton PG, Jeffries P, McQuade-Jones B, Ryan S. Re-envisioning clinical education for nurse practitioner programs: themes from a national leaders' dialogue. J Prof Nurs. 2014; 30: 273-8. 13 Ten Cate O. AM last page: what entrustable professional activities add to a competency-based curriculum. Acad Med. 2014; 89: 691. 14 Association of American Medical Colleges Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide. Vol. 2015. 2014. 15 Mehta NB, Hull AL, Young JB, Stoller JK. Just imagine: new paradigms for medical education. Acad Med. 2013; 88: 1418-23. 16 Ten Cate O. Competency-based education, entrustable professional activities, and the power of language. J Grad Med Educ. 2013; 5: 6-7. 17 Boyce P, Spratt C, Davies M, McEvoy P. Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011; 11: 96. 18 Aylward M, Nixon J, Gladding S. An Entrustable Professional Activity (EPA) for Handoffs as a Model for EPA Assessment Development. Acad Med. 2014. 19 Hauer KE, Soni K, Cornett P, Kohlwes J, Hollander H, Ranji SR, et al. Developing entrustable professional activities as the basis for assessment of competence in an internal medicine residency: a feasibility study. J Gen Intern Med. 2013; 28: 1110-4. 20 Myers J, Krueger P, Webster F, Downar J, Herx L, Jeney C, et al. Development and Validation of a Set of Palliative Medicine Entrustable Professional Activities: Findings from a Mixed Methods Study. J Palliat Med. 2015. 21 Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J, Jr. Entrustable professional activities in family medicine. J Grad Med Educ. 2013; 5: 112-8. 22 Chen HC, van den Broek WE, Cate OT. The Case for Use of Entrustable Professional Activities in Undergraduate Medical Education. Acad Med. 2014.


23 Perron NJ, Secretan F, Vannotti M, Pecoud A, Favrat B. Patient expectations at a multicultural out-patient clinic in Switzerland. Fam Pract. 2003; 20: 428-33. 24 van Loon KA, Driessen EW, Teunissen PW, Scheele F. Experiences with EPAs, potential benefits and pitfalls. Med Teach. 2014; 36: 698-702. 25 Bhuyan N, Miser WF, Dickson GM, Jarvis JW, Maxwell L, Mazzone M, et al. From family medicine milestones to entrustable professional activities (epas). Ann Fam Med. 2014; 12: 380-1.


Table 1: Descriptors of an EPA3, 13

Title

A succinct title clearly

describing the activity

Initiate and facilitate care of the palliative patient

Description Description of the

activity – with

inclusions and

exclusions

The EPA includes (a) recognition of the dying patient,

(b) participation in end-of life discussions with the

patient and family, (c) communicating with

interdisciplinary health professionals, (d) written

documentation and (e) prescribing appropriate

pharmacotherapy

Required

competencies

Linking the EPA with a

competency

framework

Relevant domains of competence include:

Communication Skills, Professionalism, and

Scholarship/Medical knowledge

Required

knowledge, skills,

and attitudes

(KSAs)

A clear explanation of

expectations for

trainees regarding the

required KSAs. May

refer to resources

which reflect standards

The trainee must have satisfactory knowledge to

understand the overall medical condition of the

patient, including: active medical problems, co-

morbidities, required investigations, treatment

options, and prognosis. The Australian Palliative Care

Therapeutic Guidelines provide a framework for the

standards expected of trainees.

A high level of communication skills is required to

sensitively and professionally discuss end-of-life

issues with the patient, family members, and

interdisciplinary health care professionals

Information

sources to assess

Describe the sources of

information used to

Direct observation of trainee interactions with the

patient, family members, and health care


progress make entrustment

decisions.

professionals.

Structured feedback from interdisciplinary health

professionals caring for the patient

Estimated stage of

training when

unsupervised

practice may be

reached (or

supervision at a

distance)

This will vary

depending on the

nature of the EPA and

the training program

Second year of basic physician training

Basis for formal

entrustment

decisions

Define who will assess

the trainee, criteria for

formal entrustment

The trainee must be directly observed in caring for

palliative patients, including reviewing

documentation and prescribed medications.

Feedback must be received from nursing staff and

other relevant health professionals caring for the

patient.

The supervisor must be satisfied the trainee has

provided safe, compassionate care and

communicated effectively as a member of the

treating team

† More recent EPA examples have included behavioural descriptors at each level of entrustment to guide the

observer and learner 18

Documents

The ABCs of EPAs – an overview of Entrustable Professional …medaproc.facmed.unam.mx/wp-content/uploads/2017/11/The... · 2017. 11. 15. · been through the copyediting, typesetting,