The training of doctors:concerns, challenges and developments

Professor Jan Illing

Centre for Medical Education ResearchSchool of Medicine, Pharmacy and Health

Durham University

∂

Overview of the lecture• My introduction to research in medical education and key influences

• The concerns about patient safety• Challenges which research has tried to respond to• Recent developments.

• The subtext is the culture of medical training and practice. Change has occurred, responded to need, but is it like a table cloth limited in size - when you move it, other areas and problems are uncovered?

∂

Introduction to Medical EducationProf Tim van Zwanenberg OBENorthern Deanery

Prof Ed PeileWarwick UniversityAwarded President’s medal ofAcademy of Medical Educatorsin 2009

Prof John Spencer Prof Jill MorrisonNewcastle University Glasgow University

Award in 2008 Award in 2010

∂

Changes in Medical EducationPMETB to GMCRITA to ARCPPRHO to F1SHO to F2SpR to STEWTD to WTRPortfolios to ePortfoliosSummative to formative -Supervised Learning Events (SLE)Teamworking to interprofessional working

New DealMMC – Modernising Medical Careers MTAS –Medical Training Application SystemFoundation ProgrammeCore TrainingCompetenciesAppraisalLicensureRevalidationReflectionWorkplace-based assessments

∂

Changes to training• The end of the ‘firm’• Greater focus on patient safety• End of locum opportunities as final year student• More learning away from the wards in classrooms and simulation• More patients ‘boarding’ on other wards• Diffused and transient teams• Older patients with multiple health problems• More acutely unwell patients• Shorter periods in hospital• Reduction in working hours, from 120 to 48 hour week

∂

Focus on the transition from medical student to junior doctor

∂

Concerns: the junior doctor

∂

Believe it or not, medical school largely keeps you away from properly sick patients. So I'm thinking, am I really up to this? Hello, is there anyone around who can help me? "We're just a bleep away, you won't be alone in a situation for long" they reassure us in our induction week. A few days in, and that's exactly what happens. With zero experience, I'm suddenly dealing with acutely ill people, out of hours and on my own. I'd like to think I could rely on other doctors, but I can't – I often find them arrogant and abrupt. Instead the nurses are my allies.

I wonder if the patients realise I'm new to the job. I think they'd be shocked if they knew; I had to spend almost an hour on my own in a frightening situation with a very ill young patient. I thought she was going to die on me. I begged two other doctors to help but they were busy with other sick patients. It was OK in the end and the patient survived, but it's not an experience I'm ever going to forget. And I know it will happen again, too soon. (21 September 2012)

∂

Jen et al.(2009) Early in-hospital mortality following trainee doctors’ first day at work. PLoS One 4, e7103.

Guardian 1 August 2012

∂

Challenge: Are medical graduates prepared for practice?National survey reported that over 40% of medical graduates did not feel fully prepared to start work as a doctor. (Goldacre, 2003)

The study also reported that the level of preparedness variedbetween medical schools.

GMC funded research to explore whether medical graduates were preparedfor practice.

∂

Our research

Aim: To examine preparedness for practice in three diverse medical schools in order to explore the extent to which threediffering medical schools can prepare graduates for the workplace.

Medical schools: Newcastle, Warwick and Glasgow.

Multi-method: qualitative and quantitative, prospective and cross-sectional.

∂

Quantitative data

Cohort questionnaireQuestionnaire administered to graduates at all three medical schoolsduring shadowing.

Assessment dataLearning portfolio assessment data reviewed at the end of first placement.

Prescribing assessmentNewcastle and Warwick F1s (junior doctors) took part in a prescribing test.

Clinical team questionnaireQuestionnaire distributed to teams who worked with F1s.

∂

Qualitative dataInterviews with c 20 final year students from each medical school at the end of medical school and after 4 and 12 months as an F1 (n=65, 55, 46).

Interviews with undergraduate tutors, educational supervisors and key managers (n=92).

Focus groups with portfolio assessors (n=3).

Findings from 250+ qualitative interviews + focus groups.

∂

Findings: areas of preparedness

• Communication skills• History taking• Clinical and practical skills• Team working

∂

Findings: lack of preparedness

• Prescribing• Managing acutely ill patients• Complex procedures• Complex communication• On-call and working nights• Prioritising patients and managing time• Hospital procedures and paperwork• Knowledge of NHS• Legal and Ethical

∂

Some illustrative quotesWard work“I don’t feel that medical school prepares you at all for any sort of ward work in any sort of way really” (WPS3, follow-up, quartile 1)

Managing acute patients“I’ve had difficulty with being in the acute situation…being the first person to initiate basic management for that patient and recognising what’s wrong”

(NPS26, follow-up, quartile 4)

On nights“In hospital they are very supervised, apart from on nights…that’s the fear, where they are most exposed” (G educational supervisor

5)

Paperwork“You… presume if you write urgent on it , it will happen urgently and then it doesn’t”

(NPS93, follow-up, quartile 3)

Prescribing“There is one area where they aren’t prepared and that’s prescribing”

(W educational supervisor 4)

(NPS93, follow-up, quartile 3

∂

Conclusions• Preparedness for practice was related to exposure to clinical practice.

• Lack of preparedness was found in areas of practice that were learned on the job i.e. prescribing, managing acute patients, working on-call and prioritising work.

• Minor differences between medical schools - maturity and self directed learning.

• Greater knowledge of role, legal and ethical issues and NHS would also be improved by increased exposure through on-the-job training.

∂

Recommendations• More structured placements that involve the student in authentic

workplace practice as part of the team.

• Final year students to have a role in the team.

• Prescribing – there needs to be more applied learning.

• Improvements to shadowing.

∂

Research impact • Discussed at the Parliamentary Health Select Committee for patient safety

• Discussed in The Lancet

• Discussed in The Times

• Discussed on Radio 4

• Discussed by BMA

• Informed Tomorrow’s Doctors 2009

∂

Developments: following the research• GMC introduced student assistantships

• Further research supporting findings: Matherson and Matherson, 2009, Brennan et al., 2010, Tallentire et al. 2011.

• Testing student assistantship in Belfast, Braniff 2012

“The majority of medical students feel well prepared for starting work after completing the student assistantship”

∂

The future?“The reliance upon doctors in training to deliver a 24/7 service has to change.”

“Where clinical need dictates, this may involve 24/7 consultant working.”

“All trainees need to be supported by close, appropriate supervision and this will then increase the learning opportunities and improve the decision-making, diagnosis and treatment pathways, improving patient safety.”

“Trainees can no longer be expected to learn simply through being around the hospital working. Training has to be planned, focused and, as far as possible directed to the needs of the individual trainee.”

Time for Training, A Review of the impact of the European Working Time Directive on the qualityof training, Temple, 2010 MEE

∂

Focus on the transition of overseas qualified doctors to the UK workplace

∂

Concerns: overseas qualified doctors

∂

∂

∂

Challenge: understanding how to better support non-UK qualified doctors• The NHS depends on overseas doctors to run the NHS.

• Of 246,000 doctors on the medical register in the UK in 2011 37% qualified from non-UK medical schools.

• Discrimination and bullying (Hoosen et al, 2004; Louis et al, 2010; Esmail, 1997)

• A higher risk of referral for under performance (NCAS report 2010)

• Over-represented in later stages of GMC Fitness to Practise process (Humphrey et al, BMJ, 2011)

∂

Our research Aims

To compare the transition of non-UK and UK medical graduates to the NHS workplace (F1s)

To identify what helped or hindered the transition in the first 12 months

ESRC funded

∂

MethodsQuantitative data• Cohort questionnaire UK (n=480) EU (n=12) IMG (n=68)

• Clinical teams questionnaire UK (n=64) EU/IMG (n=19)

Qualitative data• Interviews: prior to starting work (F1) and after 4 months and 12

months in post. UK (n=65), EU (n=14), IMG (n=52)

• Interviews with educational supervisors (n=28)

∂

FindingsCohort questionnaire• Overall, non-UK graduates felt more prepared than UK graduates.• Reflecting cultural issues?

Qualitative data• Both UK and overseas graduates shared concerns about clinical work.• Particularly with prescribing, acute management and on-call. • Concerns reduced for both groups after 12 months.

∂

Overseas doctors had additional hurdles compared to UK doctors• Some started work late, missed shadowing and induction• Those without family and friends needed help with accommodation and

opening a bank account etc.

Cultural differences• Communication with patients – NHS has patient centred care• Communication with clinical team – NHS often less hierarchical,

differing team expertise, and relationships• Different way of working in a team – NHS shared decisions, less ‘doctor knows best’• Different illnesses, equipment and tests in the NHS• Different language, colloquial expressions and non-verbal behaviour• Different legal and ethical issues

∂

Legal and ethical issues “I don’t know if a doctor for example is shadowing, at that time is he

actually allowed to, for example, to take blood from a patient or put in a cannula, or does he have to be registered”. (IMG, ID29c)

“I think it’s different between here and other countries, like for example when do you really need a chaperone, and if you need a chaperone, what do you need to document, the name of the chaperone not just put in ‘chaperoned’.” (IMG, ID49c)

∂

What hindered the transition to the workplace for non-UK doctors?

• A long gap before starting work • Lack of exposure to clinical practice in UK • Lack of information about living and working in UK • Lack of support• Lack of feedback about progress

∂

What helped the transition to the workplace for non-UK doctors?• GMC tests for International Medical Graduate doctors (PLAB)• Clinical attachments – shadowing • Induction to NHS system, ward and to roles and responsibilities • Shadowing the post – longer for non-UK doctors• On-going support and feedback when in post

∂

Conclusions• The NHS depends on overseas doctors to fill rota gaps• Overseas doctors are trained in a different system and arrive with a

different training culture and need to adapt.• There is limited recognition of important influence of early medical

training on practice.• The NHS provides little support to help adjust to the UK workplace• Like UK graduates, overseas graduates need experience of clinical

practice in the UK and do this safely.• Lack of support may lead to difficulties later and explain some of

the referrals for underperformance.

∂

RecommendationsSupport before workProvide clear information about the process of starting work in the UKSupport clinical attachments

Support on starting workSupport inductions for overseas doctors Provide shadowing of own job

Support in workplaceSupport buddying and mentoring schemes Increased cultural awareness from other staffProvide on-going feedback

∂

Research impact • GMC to introduce induction pilot in 2013

“Based on research we’ve commissioned and other available evidence, we believe that doctors new to practice could benefit from additional support before they start practising in the UK.” GMC website 2012

• Informed Northern Deanery induction programme for overseas doctors.

• Informed local Trusts who set up induction programme for overseas doctors.

∂

Recent developments “Doctors who qualified outside the UK were proportionally more

likely to be subject to a GMC investigation about issues such as poor clinical skills and knowledge, lack of knowledge of the law or codes, and an inadequate participation in medical education. They were also more likely to be investigated about these issues within the first two years of joining the UK register”

“However, doctors who trained abroad can face difficulties when they start to practise in the UK, including unfamiliarity with health systems, communication and cultural differences, and lack of proficiency in English. We believe that these doctors may need greater support with adapting to UK practice than has previously been recognised.”

(The State of Medical Education and Practice in the UK, GMC report 2012)

∂

Focus on the introduction of revalidation

∂

Concerns: rooting out bad doctors

∂

∂

Number of complaints to UK health professional regulators (2007-11)

GMC

NMC

HPCGOC

GDC

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000

2007 2008 2009 2010 2011

Year

Annu

al nu

mber

of c

ompla

ints r

eceiv

ed b

y re

gulat

or

∂

Revalidation“The purpose of revalidation is to provide greater assurance to patients and the public, employers and other healthcare professionals that licensed doctors are up-to-date and fit to practise.” (GMC website, 2012)

“Licensed doctors will have to revalidate, usually every five years, by having regular appraisals that are based on our core guidance for doctors Good Medical Practice” (GMC website, 2012)

“These regular checks on doctors will be a world leading system that will help improve the quality of care received by patients.” (GMC website, 2012)

Revalidation will start on 3 December 2012.

∂

Supporting information Doctors will be expected to provide six types of information for discussion during appraisal at least once in each five year cycle. They are:

1. Continuing professional development2. Quality improvement activity3. Significant events4. Feedback from colleagues5. Feedback from patients6. Review of complaints and compliments

∂

Our ResearchStudy for RCGP on the feasibility of collecting evidence for revalidation forGPs working peripatetically as locums, out of hours and in remote places

Aim: To explore the potential problems locum, sessional andremote GPs may have collecting the following evidence for revalidation:

• Clinical audit (quality improvement activity)• Significant Event Analysis • Colleague Feedback • Patient Feedback

∂

Method53 GPs recruited, asked to collect evidence required for revalidation• 10 focus groups • 12 telephone interviews

33 GPs followed up to identify any difficulties in collecting evidence• 8 focus groups• 10 telephone interviews

∂

Findings: cultural issues

“Nobody sees you and nobody knows you. I’ve done locums in practices where nobody’s even said anything…they won’t even take the time to put in your details – you’re just ‘Doctor Locum’.” (initial focus group 5)

• Experience of isolation• Perceptions of not being valued • Lack of interest in supporting education • Lack of support to facilitate the collection of evidence for revalidation

∂

Findings: quality improvement activity and significant events• Audits were difficult to achieve without access to a practice computer.• Lack of interest in informing sessional doctors about significant events • Awareness that some significant events could be viewed as threatening

if identified by an “outsider”

∂

Findings: feedback from colleagues and patientsColleague feedback• Lack of direct contact with other colleagues• Fewer colleagues to ask for feedback• Some GPs refused to provide feedback

Patient feedback• Lack of support from practices to facilitate patient feedback• Feedback based on less doctor contact, could be more negative• Some patients failed to understand the purpose of feedback

Colleague and patient feedback was difficult to achieve for locum, OOHs and GPs working in remote places.

∂

Conclusions from the research • Some doctors will have more difficulty collecting evidence for

revalidation. Locums and those in non-standard settings.

• Ensuring equal opportunity for revalidation will require a culture change, to ensure all doctors are supported during this process.

There is an assumption that revalidation will identify poorly performing doctors• Assumes reasonable knowledge of that doctor • Requires high numbers to be reliable

∂

Research impact “General practices, federations and out-of-hours organisations that frequently employ GPs on short-term, sessional contracts must recognise their responsibility to all their employees, including these doctors. They should inform and involve doctors in any significant event orcomplaint that relates to them; they should facilitate access to the clinical records of patients treated by these doctors for the purposes of clinical audit and Quality improvement; and they should support the conduct of patient surveys”

The RCGP Guide to the Revalidation of General Practitioners,v7, June 2012, RCGP.

∂

Validity of workplace-based assessments questioned?

• Assessment of Foundation doctors is considered to be excessive, onerous and not valued (30 per trainee)

• The validity of the tools has been questioned due to the lack of training to use them, lack of time and the substantial demands on trainer time.

• Tools do not discriminate at top end - focus is on competence not excellence. (Prof John Collins, 2010, Foundation for Excellence. An Evaluation of the

Foundation Programme, MEE)

• Research on feedback highlights that change depends on who provides the feedback and on knowledge of the individual (Sargeant et al., 2010)

• Qualitative feedback is preferred to quantitative feedback (Burford et al., 2010)

• Qualitative feedback tells you how to get better (van der Vleuten, 2012)

∂

Revalidation:an equal opportunity?

Revalidation will depend on annual appraisalsCurrently all doctors do not have annual appraisals: those in non-trainingposts, Staff, Associate Specialist and Specialty grade and locums

“There is some evidence that doctors in postgraduate training who gained their primary medical qualification overseas are proportionally more likely to experience challenges in progressing through training.” (The State of Medical Education and Practice in the UK, GMC report 2012)

Overseas qualified doctors were significantly more likely to have an unsatisfactory outcome in 2010–11. (The State of Medical Education and Practice in the UK, GMC report 2012)

∂

The failure rate for international medical graduates taking the CSA component of the MRCGP was 63% compared to 9% of UK graduates. 13 November 2012

Problems in progressing are about culture not ethnicity.

∂

Conclusion • Can revalidation identify underperformers and at the same time drive

up quality?

• Might the evidence from the problems with workplace-based assessments also apply to revalidation?

• Without a culture change is a subgroup of doctors at risk of not being revalidated? What are the consequences for the NHS if these doctors are not supported?

∂

Final conclusions The concerns raised all focus on patient safetyThe challenge and developments are directed at doctors

A culture change is required: to avoid the reliance on junior doctors providing 24/7 care.

to recognise different early trainingand support non-UK qualified doctors.

to support doctors in non-standard practiceand ensure they are supported in the process towards revalidation.

∂

Final comments on progress “Properly planned and carefully conducted medical education is the

foundation of a comprehensive health service” The Training of Doctors, Report by Goodenough Committee 1944

Recommendations:• Overhaul of undergraduate training, attention to social medicine and

health promotion as well as cure• Co-education and equality in hospital appointments• Reform of examination system• Compulsory hospital appointments after qualification• Comprehensive system for the training of specialists• Linking major hospitals with teaching centres• Greatly increased Exchequer grants for medical education and

research!

A few words of thanks to -

∂

From the Northern Deanery to Durham University

∂

Having fun teaming building in 2008

∂

Meet the teamChristmas 2011

GillTracy Neill

MadelinePaul

Bryan Charlotte

∂

My other team!

Thank you!

Professor Jan Illing

Centre for Medical Education ResearchSchool of Medicine, Pharmacy and Health

Durham University