Workplace-based assessment in surgical training: experiences from

the Intercollegiate Surgical Curriculum Programme

Ian Eardley,*† Maria Bussey,* Adrian Woodthorpe,† Chris Munsch† and Jonathan Beard‡*ISCP, Sydney, New South Wales, Australia†Royal College of Surgeons of England, London, UK and‡Sheffield Vascular Institute, Northern General Hospital, Sheffield, UK

Key words

ISCP, surgical training, workplace-based assessment.

Correspondence

Ian Eardley, Department of Urology, St James’sUniversity Hospital, Beckett Street, Leeds LS9 7TF, UK.Email: ian.eardley@btinternet.com

I. Eardley MChir, FRCS (Urol), FEBU; M. Bussey MBA;A. Woodthorpe BEd, MA, MSc, PhD; C. Munsch MD,FRCS (Cardiac); J. Beard MD, FRCS (Gen).

Accepted for publication 17 February 2013.

doi: 10.1111/ans.12187

Abstract

Background: The Intercollegiate Surgical Curriculum Programme was launched inthe United Kingdom in 2007. At its heart was the reliance upon clear, defined cur-ricula, competence-based training and the use of workplace-based assessments toassess the competence. The principle assessments used were Case-based Discussion,Procedure-based Assessments (PBA), Direct Observation of Procedural Skills, andClinical Evaluation Exercise and a Multisource Feedback tool.Methods: We report the initial experience with that system, and most importantly, theexperience with workplace-based assessment.Results: Themes include issues around faculty development, misuse of assessments,inappropriate timing of assessments, concerns about validity and reliability of theassessments and concerns about the actual process of workplace-based assessments.Of the assessments, the PBA performed best.Conclusions: As a consequence, there has been an increased focus upon facultydevelopment, while some of the assessments have been redesigned in line with thePBA. A global rating scale has been introduced that uses clinical anchors. The ratingscales have also been altered with a reduction in the number of ratings while anenhanced description of the complexity of the case has been introduced within theCase-based Discussion and the Clinical Evaluation Exercise. A re-evaluation will takeplace in the near future.

Introduction

The traditional model of surgical training is the apprenticeshipmodel. This model has been the basis of surgical training for manyyears and involves a surgical trainee learning initially by observa-tion, followed by gradual introduction to surgical techniques initiallywith careful and close supervision, but latterly with ‘detached’ super-vision. Feedback from the trainer was typically intermittent andinformal, and the model required extensive operative experience. Itwould be unusual for a trainee to be told that the operation had gonewell, and for many trainees, the only way that they found out thattheir surgical skills were developing appropriately was when thesupervision became less close.

Such a method of learning was never going to be sustainablewithin the context of reducing hours of work, a need for greaterpublic scrutiny and the requirements of patient safety. The progres-sive introduction of the European Working Time Regulations have

progressively reduced the time that any surgical trainee is allowed towork down to a maximum level of 48 h per week. This has resultedin a significant reduction in operations performed by surgical train-ees.1,2 There is evidence that in the context of reduced experience,regular feedback aids learning,3 and the introduction of workplace-based assessments (WPBAs) facilitate this.

The second driver for change has been the need for morerobust assessment of the performance of surgical trainees withinthe workplace.4 The drivers for this are the necessity of publicscrutiny and the patient safety agenda. WPBAs offer the traineeand trainer a number potential advantages. They allow both trainerand trainee to focus upon a particular clinical area or skill;they allow the trainer and trainee to gauge their own progression;they allow the trainer to identify the underperforming surgeon sothat remediation can be effected; and finally, they allow identifi-cation of the failing surgeon who will be unable to progress tocertification.

SURGICAL EDUCATION: ASSESSMENT IN THE WORKPLACEANZJSurg.com

© 2013 The AuthorsANZ Journal of Surgery © 2013 Royal Australasian College of SurgeonsANZ J Surg 83 (2013) 448–453

The third driver for change in the United Kingdom has beenregulatory, in that there has been a change in emphasis from tradi-tional time-based curricula to competency-based curricula.5 Thecurrent curricula in the United Kingdom have 7–8 indicative years oftraining with regular assessment of competencies at regular inter-vals. Theoretically, at least, trainees can progress through training atdifferent speeds depending upon their ability.

The Intercollegiate Surgical CurriculumProgramme (ISCP)

History

In 2003, the United Kingdom Government published a documententitled ‘Modernising Medical Careers (MMC)’6 which provided themandate for the UK Surgical Royal Colleges and specialty associa-tions to start work on changes to postgraduate surgical education.The surgical colleges embarked upon the Intercollegiate SurgicalCurriculum Project, which in turn became the ISCP.

The initial development phase involved the development of acurriculum framework. The nine Surgical Specialist Advisory Com-mittees developed the specialty syllabuses, identifying what traineesshould know and be able to do at each stage of surgical trainingwhile individual assessment tools were developed. At the same time,faculty development began, and a website was developed. In 2005, anational pilot of the syllabus content, assessment strategy and tools,interactive portfolio and website was undertaken. This was supple-mented by nationwide awareness raising activities includingstakeholder meetings, consultation forums with trainers and train-ees, a faculty development programme for key surgical and deaneryeducators and user testing of the web site.

Structure

ISCP was launched in 2007 and provided the framework for system-atic surgical training through to consultant level. There was a

syllabus that defined the knowledge, clinical judgement, technicaland operative skills and professional skills and behaviours that wereneeded in order to progress. The curriculum was accessed throughhttp://www.iscp.ac.uk and contained the most up-to-date versions ofthe nine specialty syllabuses. Many aspects of the early years werecommon to all, but were increasingly singular as training in eachdiscipline advanced.

The aims were to ensure the highest standards of surgical practicein the United Kingdom by delivering high-quality surgical trainingand to provide a programme of training through to the completion ofspecialist surgical training. The curriculum was founded on anumber of key principles including a common format and similarframework across all the specialties, systematic progression throughto the certification, standards that were underpinned by robustassessment and regulation of progression through training by theachievement of outcomes that were competence-based rather thantime-based.

The assessment system

The purpose of the assessment system was to determine whethertrainees were meeting the standards of competence and performancespecified at various stages in the curriculum, to provide comprehen-sive feedback to the trainee and to determine whether trainees hadacquired the common and specialty-based knowledge, clinical judg-ment, operative and technical skills, and generic professional behav-iour and leadership skills required to practice independently.

The individual components of the assessment system were:• WPBAs (Table 1) covering knowledge, clinical judgment, tech-

nical skills and professional behaviour and attitude• The surgical logbook• Examinations held at key stages• The learning agreement and the educational supervisors’ report• An Annual Review of Competence Progression (ARCP).

Table 1 Summary guidance on frequency, timing and use of WPBA methods

Method Main competencesassessed

Training level Assessment standard Appropriate assessors Minimum frequency

CBD Clinical judgement All Standard at completion ofthat stage of training

Educational supervisor 1 every 2 monthsClinical management Clinical supervisorRecord keeping Senior specialty traineeLeadershipReflective practice

SurgicalDOPS

Technical skills Early yearstraining

Standard at completion ofthat stage of training

Educational supervisor 1 every month for eachindicator procedureClinical supervisor

Senior specialty traineeQualified members of the

multidisciplinary teamPBA Technical skills Mainly specialty

trainingCompletion of training Educational supervisor 1 every month for each

indicator procedureTheatre team-working Clinical supervisorLeadership Senior specialty trainee

CEX History taking All Standard at completion ofthat stage of training

Educational supervisor 1 every 2 monthsCommunication Clinical supervisorPhysical examination Senior specialty traineeDiagnosis Qualified members of the

multidisciplinary teamManagement planOrganisation

Mini-PAT Team working All Standard at completion ofthat stage of training

Trainee’s multiprofessionalteam

Every 3 years (Repeatedif necessary)Professional behaviour

CBD, Case-based Discussion; CEX, Clinical Evaluation Exercise; DOPS, Direct Observation of Procedural Skills; Mini-PAT, Peer Assessment Tool; PBA, Procedure-based Assessments; WPBA, workplace-based assessments.

WPBA in surgical training 449

© 2013 The AuthorsANZ Journal of Surgery © 2013 Royal Australasian College of Surgeons

The primary purpose of the WPBAs was to provide feedbackbetween trainers and their trainees. They were designed to be mainlytrainee driven, but inevitably, there were occasions when they weretrainer triggered. The accumulation of WPBAs was one of a range ofindicators that informed the ARCP. As a consequence, a decisioncould be made whether there had been satisfactory progression andconsequently whether the trainee could progress or completetraining. The trainee’s educational supervisor had a key role injudging whether the trainee requires more than the minimumnumber of assessments. In principle, the assessments needed to bestarted early, and continued regularly with the expectation that therewould be evidence of progression throughout the training period. Allthe assessments in the curriculum included a feedback element.Educational supervisors were able to provide further feedback toeach of their trainees through the regular planned educationalreviews and appraisals that occur at the beginning, middle and end ofeach placement, using information contained in the trainee portfolioand feedback from other trainers in the workplace.

The actual assessments were as follows:(1) Case-based Discussion (CBD) was designed to assess clinical

judgment, decision making and the application of medicalknowledge in relation to patient care in cases for which thetrainee had been directly responsible. The process was astructured discussion between the trainee and educationalsupervisor about how a clinical case was managed by thetrainee.

(2) Mini Clinical Evaluation Exercise (Mini-CEX) assesses thetrainees’ clinical and professional skills in a clinical situa-tion.7 The assessment involves observing the trainee interactwith a patient in a clinical encounter.

(3) Procedure-based Assessments (PBA) assess trainees’ techni-cal, operative and professional skills in a range of proceduresduring routine surgical practice. The PBA was originallydeveloped by the Orthopaedic Competence AssessmentProject8 and has been further developed for all the surgicalspecialties. The assessment method uses two principal com-ponents: a series of competencies within six domains and aglobal assessment that is divided into four levels. The highestrating is the ability to perform the procedure to the standardexpected of a specialist in independent practice within theNational Health Service. The assessment is supported bydescriptors outlining desirable and undesirable behavioursthat assist the assessor in deciding whether or not the traineehas reached a satisfactory standard for Certificate of Comple-tion of Training (CCT) on the occasion observed.

(4) Direct Observation of Procedural Skills (DOPS) in surgery isused to assess the trainees’ technical, operative and profes-sional skills in a range of basic diagnostic and interventionalprocedures, or parts of procedures, during routine surgicalpractice. Surgical DOPS is used in simpler environments andprocedures than a PBA.

(5) Peer Assessment Tool (Mini-PAT) is Multisource Feedback(MSF) assessment and was used to assess professionalcompetence within a team-working environment. Mini-PATcomprised both a self-assessment and assessments of a train-ee’s performance from a minimum of eight raters. The com-

petencies assessed mapped across to the standards of GoodMedical Practice.9

ARCP

The ARCP is a formal process that scrutinizes each surgical trainee’ssuitability to progress to the next stage of, or complete the trainingprogramme. It bases its recommendations on the evidence that hasbeen gathered in the trainee’s learning portfolio during the periodbetween ARCP reviews. The ARCP is undertaken on an annualbasis.

Evaluation of WPBAs within ISCP

The Joint Committee for Surgical Training began a review of theoutcomes of the WPBAs in 2009. The purpose of the evaluation wasto assess the way in which the assessments were being used. Thebasic data are described later and tabulated in Table 2, while therange of results achieved is shown in Tables 3–5.

CBD

Within a 12-month period, 2736 trainees undertook 17 904 CBDs.The CBD was rated on six competencies and a global summaryrating. For each competency, scores of between 1 and 6 were pos-sible, with 1 and 2 representing ‘Below expectations’, 3 being bor-derline, 4 being ‘Meets expectations’ and 5 and 6 representing‘Above expectations’. The scores actually achieved in the globalsummary assessment were as shown in Table 3. Only 0.55% ofscores were actually below 4. There was no apparent relationbetween the stage of training and the summary rating given by theassessor (Table 4).

Mini-CEX

Two thousand five hundred four trainees undertook 14 950 Mini-CEX assessments in the 12-month period. The Mini-CEX was ratedon six competencies and a global summary rating. For each compe-tency, scores of between 1 and 6 were possible, with 1 and 2 repre-senting ‘Below expectations’, 3 being borderline, 4 being ‘Meetsexpectations’ and 5 and 6 representing ‘Above expectations’. Mostactual scores were 4, 5 or 6.

PBA

Two thousand seven hundred one trainees undertook 11 132 PBAsduring the 12-month period. The PBAS was rated with a global scorebetween 0 and 4, with 0 representing ‘Insufficient evidence observedto support a summary judgement’, level 1 representing ‘Unable toperform the procedure, or part observed, under supervision’, level 2representing ‘Able to perform the procedure, or part observed, undersupervision’, level 3 representing ‘Able to perform the procedurewith minimum supervision’ and level 4 representing ‘Competent toperform the procedure unsupervised (could deal with complicationsthat arose)’. The ratings actually given are recorded in Table 3. Therewas a correlation between the global score and the stage of training(Table 5).

Surgical DOPS

Eight hundred sixty-four trainees undertook 2249 assessments in a3-month period. The DOPS was rated on a single scale with seven

450 Eardley et al.

© 2013 The AuthorsANZ Journal of Surgery © 2013 Royal Australasian College of Surgeons

possible ratings with 1 and 2 representing ‘below expectations’, 3being borderline, 4 being ‘meets expectations’, 5 and 6 representing‘above expectations’ and 7 representing ‘unable to comment’. Only0.3% of the actual scores were below 4 (Table 3). The mean domainscores were largely the same for the different stages of training.

Mini-PAT

Over a 2-year period, 3755 Mini-PATs were completed. The Mini-PAT contains 16 questions with seven possible ratings with 1 and 2

representing ‘below expectations’, 3 being borderline, 4 being‘meets expectations’, 5 and 6 representing ‘above expectations’ and7 representing ‘unable to comment’. In all the responses, there wereonly 354 ratings less than 4 (<1%).

Summary of the evaluation andemerging themes

This initial evaluation of the performance of the WPBAs was illu-minating. Several themes emerged, some of them from this quanti-tative evaluation but others from more qualitative assessments.10

These included issues around faculty development, misuse of theassessments, inappropriate timing of assessments, concerns aboutvalidity and reliability of the assessments and concerns about theactual process of WPBAs.

Firstly, trainees and trainers, used to a system of summativeassessments, were often using the assessments inappropriately.Trainees were waiting until they felt they would ‘pass’ before under-taking the assessment, such that they were typically being under-taken late in an attachment. Trainers too had some trouble with theprovision of appropriate feedback, both in terms of both quantity andquality.

Second, there was clear evidence of a paucity of ‘low’ scores. Thisreflected several things; a tendency for trainees to delay assessmentsuntil the end of a placement, the failure of trainers to be explicitregarding areas for development and a difficulty with the trainerhaving a clear idea of where a trainee was in relation either to his orher peers, or to the competencies of a CCT holder. This reflectsfindings elsewhere in the literature.11,12

Thirdly, there appeared to be a difference in the performance ofthe various assessments, most clearly demonstrated by the finding

Table 2 Workplace-based assessments in practice

Period Number oftrainees

Number ofassessments

Mean number ofassessments per trainee

(Mode; Range)

Mean time taken forassessment (Range)

Mean time taken forfeedback (Range)

CBD Aug 2008–July 2009 2736 17 904 6.5 (6; 1–79) 20.3 min (1–999) 9.1 min (1–120)Mini-CEX Aug 2008–July 2009 2504 14 950 6.0 (6; 1–46) 17.6 min (1–720) 8.4 min (1–155)PBA Aug 2008–July 2009 2701 11 132 5.4 (1; 1–48) 47.7 min (1–750) 10.2 min (1–160)DOPS May–July 2008 (†) 864 2 249 2.6 (1; 1–30) 20.7 min (2–240) 7.84 min (1–200)Mini-PAT Aug 2007–July 2008 (‡) 3755 3 755 N/A N/A N/A

†Numbers apply to a 3-month period only. ‡Numbers apply to a 2-year period. CEX: Setting, assessor, grades awarded. CBD, Case-based Discussion; DOPS, DirectObservation of Procedural Skills; Mini-CEX, Mini Clinical Evaluation Exercise; Mini-PAT, Peer Assessment Tool; PBA, Procedure-based Assessments.

Table 3 Scores from the rating scales within ISCP

Score Summary rating of CBD Rating within the sevenscales of the Mini-CEX

Global rating within the PBA Rating of the surgical DOPS Global rating (Question 16)of Mini-PAT

0 N/A N/A 17 (0.15%) N/A N/A1 0 (0%) 0% 138 (1.24%) 0% 1 (<0.1%)2 10 (0.06%) 0% 4259 (38.34%) 0% 3 (0.1%)3 87 (0.49%) 0.1%–0.6% 4454 (40.1%) 0.3% 19 (0.5%)4 5510 (31.2%) 24.0%–33.5% 2240 (20.17%) 31.5% 828 (22.1%)5 9186 (52.01%) 47.3%–54.9% N/A 41.5% 1710 (42.5%)6 2869 (16.24%) 13.2%–23.4% N/A 18.0% 1194 (31.8%)7 N/A N/A N/A 7.4% N/A

CBD, Case-based Discussion; DOPS, Direct Observation of Procedural Skills; ISCP, Intercollegiate Surgical Curriculum Programme; Mini-CEX, Mini Clinical EvaluationExercise; Mini-PAT, Peer Assessment Tool; PBA, Procedure-based Assessments.

Table 4 Relation between the overall score and stage of training for CBD

Stage of training Numbers of assessments Summary rating

ST1 6695 4.84ST2 5989 4.89ST3 2968 4.77ST4 1719 4.78ST5-8 219 4.82

CBD, Case-based Discussion.

Table 5 Relation between global score and stage of training for PBA

Stage of training Numbers of assessments Global score

ST1 1470 2.28ST2 2605 2.61ST3 3707 2.81ST4 2765 30.6ST5 457 3.34ST6-8 128 3.69

PBA, Procedure-based Assessments.

WPBA in surgical training 451

© 2013 The AuthorsANZ Journal of Surgery © 2013 Royal Australasian College of Surgeons

that while the global PBA scores correlated with the stage of train-ing, the summary scores for CBD and the mean scores for DOPS didnot.

Finally, while considerable time and expense had gone intofaculty development in the run-up to the launch of the ISCP, therewas widespread concern regarding the new system among bothtrainers and trainees. While all trainees entering surgical training in2007 entered the new system, all trainees already within UK surgicaltraining remained in the old apprenticeship style system, which formany trainers meant that their exposure to WPBAs was limited inthe early years of ISCP. Those who did use them often complainedthat they were ‘tick-box’ exercises and that they took too much timeto fit into a busy schedule. Both trainers and trainees were concernedabout perceived lack validity and reliability.

Lessons learned and futuredevelopments

While many (but not all) of these issues could have been avoided, toa degree, by faculty development, the initial burst of activity prior toAugust 2007 subsided soon after the launch with the consequencethat some trainers did not ‘have’ a trainee within the new systemuntil 2009 or later. As a consequence, there is ongoing work regard-ing faculty development for instance using course-based training(the so called Training and Assessment in Practice course).

It will be important to get across to trainers and trainees theprinciples of good assessment:

(1) Firstly, that performance is context specific and consequentlythat clinicians should be assessed on a sample of cases.13

(2) Secondly, that complex performance cannot be reduced tosimple checklists. While there is often agreement on who isperforming well and who is performing not so well, theremay be differences in these complex judgements betweendifferent assessors. Consequently, there should be a range ofassessors.14

(3) Thirdly, assessment in the workplace is central. Assessmentof performance outside the workplace has value, but does notfully predict performance in the workplace.15

(4) Finally, the assessment should be as frequent as possible toencourage learning and to provide evidence of competence.

However, even if we are able to get these principles across to trainersand trainees, there remains a need to convince them of the reliabilityand validity of the assessments. Evidence has begun to appear basedupon the experience not only of surgeons, but also of other medicalspecialties. Several studies have demonstrated validity,16 and a recentstudy has demonstrated that the PBA has excellent reliability whenmore than three assessments are used for a particular procedure.17

Because the assessment is procedure specific, each index procedureneeds to be assessed separately in the same way. This and otherresearch suggests that PBAs achieve at least level 2 of theKirkpatrick model of assessment evaluation;18 that is, the traineesand trainers in the studies were appreciative and supportive of theassessment, both as an assessment for learning and as the basis of theevidence for the summative ARCP assessment. Much more researchis needed in this area, not only to deliver the evidence that has

hitherto been lacking, but also to be certain that the format of theassessments is optimized.

The evaluation of ISCP is an example of the latter point, and thedata derived have stimulated changes to the assessments with theobjective of making them more valuable. For example, the lack ofuse of the lower ratings within the DOPS, Mini-CEX and CBD, andknowledge gained from the performance of other assessments haveled to a re-design of the instruments within ISCP, and these newinstruments were introduced in the latter half of 2010, so as yet therehas been no formal evaluation of the changes.

There is emerging evidence that ‘construct aligned’ scalesimprove the performance of WPBAs.19 What this means is thatscales that reflect developing clinical sophistication and independ-ence appear to have greater utility as a consequence of greaterreliability and validity. Given these findings and given the relativelygood performance of the PBA, which already embraces thisapproach, the CBD, CEX and DOPS have been redesigned alongthese lines. A global rating scale that has been introduced that usesclinical anchors as follows:

• Level 0 is below that expected for early years training• Level 1 is appropriate for early years training• Level 2 is appropriate for completion of early years training or

early specialty training• Level 3 is appropriate for central period of specialty training• Level 4 is appropriate for the CCT.

The rating scales have also been altered with a reduction from the 6or 7 different rating to four ratings, namely ‘Outstanding’, ‘Satis-factory’, ‘Development required’ or ‘Not observed’. A final changehas been an enhanced description of the complexity of the casewithin the CBD and CEX documentation.

The Mini-PAT has also been re-designed since some of the ques-tions being asked were found to be non-discriminatory, and also sothat the new assessment (now called MSF) more closely aligns to‘Good Medical Practice’.

Whether these changes achieve greater acceptability, utility, reli-ability and validity, only time will tell, but ISCP is committed to anongoing programme of evaluation that remains central to the newcurriculum. We do know, that just as a curriculum is living and neverstops changing, then we will need to continue to re-evaluate andreassess the assessment system so that is fit for purpose, namely thatit has reliability, validity, acceptability and utility.

References1. Chesser S, Bowman K, Phillips H. The European Working Time Direc-

tive and the training of surgeons. BMJ 2002; 325: S69–70.2. Bates T, Cecil E, Greene I. The effect of the EWTD on training in

general surgery: an analysis of electronic logbook records. Ann. R. Coll.Surg. Engl. (Suppl.) 2007; 89: 106–9.

3. Hattie J, Timperley H. The power of feedback. Rev. Educ. Res. 2007; 77:81–112.

4. Smith R. All changed, changed utterly. BMJ 1998; 316: 1917–8.5. General Medical Council. Standards for curricula and assessment

systems. [Cited 2010.] Available from URL: http://www.gmc-uk.org/education/postgraduate/standards_for_curricula_and_assessment_systems.asp

6. Department of Health. Modernising Medical Careers. [Cited 2003.]Available from URL: http://webarchive.nationalarchives.gov.uk/2013

452 Eardley et al.

© 2013 The AuthorsANZ Journal of Surgery © 2013 Royal Australasian College of Surgeons

0107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4054233.pdf

7. Norcini J, Blank L, Duffy D, Fortna G. The mini-CEX: a method forassessing clinical skills. Ann. Intern. Med. 2003; 138: 476–81.

8. Pitts D, Rowley DI, Sher JL. Assessment of performance in orthopaedictraining. J. Bone Joint Surg. 2005; 87: 1187–91.

9. General Medical Council. Good Medical Practice. Guidance for doctors.[Cited 2006.] Available from URL: http://www.ub.edu/medicina_unitateducaciomedica/documentos/Good_Medical_Practice.pdf

10. Eraut M. The Eraut report. [Cited 2008.] Available from URL: http://www.mee.nhs.uk/pdf/FinalReportISCP%20-%20MichaelEraut.pdf.

11. Kogan J, Holmboe E, Hauer K. Tools for direct observation and assess-ment of clinical skills of medical trainees: a systematic review. JAMA2009; 302: 1316–26.

12. Davies H, Archer J, Southgate L, Norcini J. Initial evaluation of the firstyear of the Foundation Programme. Med. Educ. 2009; 43: 74–81.

13. Elstein A, Shulman L. Medical Problem Solving: An Analysis of ClinicalReasoning. Cambridge MA: Harvard University Press, 1978.

14. Crossley J, Humphris G, Jolly B. Assessing health professionals. Med.Educ. 2002; 36: 800–4.

15. Rethans JJ, Sturmans F, Drop R, van der Vleuten C, Hobus P. Doescompetence of general practitioners predict their performance? Com-parison between examination setting and actual practice. BMJ 1991;303: 1377–80.

16. Beard JD, Jolly BC, Newble DI, Thomas WEG, Donnelly J, SouthgateLJ. Assessing the technical skills of surgical trainees. Br. J. Surg. 2005;92: 778–82.

17. Marriot J, Purdie H, Crossley J, Beard J. A prospective observationalstudy to evaluate procedure based assessment (PBA) for assess trainee’sprocedural skills in the operating theatre. Br. J. Surg. 2011; 98: 450–7.

18. Freeth D, Hammick M, Koppel I, Reeves S, Barr H. Critical Reviewof Evaluations of Interprofessional Education. London: HigherEducation Academy Learning and Teaching Support Netwrok for HealthSciences and Practice, 2002. [Cited 2002.] Available from URL: http://www.health.ltsn.ac.uk/publications/occasionalpaper/occasionalpaper02.pdf

19. Crossley J, Johnson G, Booth J, Wade W. Good questions, goodanswers: construct alignment improves the performance of workplace-based assessment scales. Med. Educ. 2011; 45: 560–9.

WPBA in surgical training 453

© 2013 The AuthorsANZ Journal of Surgery © 2013 Royal Australasian College of Surgeons