Upload
cora-shelton
View
216
Download
0
Embed Size (px)
Citation preview
Workplace-based Assessment (WBA) – Beyond the numbers
James KwanAssociate Professor, Emergency Medicine and Medical Education
Acknowledgements
• A/Prof Lukas Kairitis• Dr Karuna Keat• Professor Jenny Reath• Professor Wendy Hu• Members Year 3/5 Committee
Outline
• Competence vs. Performance• Overview WBA at UWS• Assessor judgements on performance• Delivering Feedback
Definitions• Competence
– “Ability to do something successfully” (Oxford Dictionaries, 2012)
• Competence in medicine– Ability to perform a specific clinical task– Requires integration of competencies– Measurable in terms of observable behaviour– Context specific– Process of growth and development
Knows
Shows how
Knows how
Does
Prof
essi
onal
aut
henti
city
Prof
essi
onal
aut
henti
city
StandardisedAssessment(mostly established)
Non- standardised assessment(emerging)
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
Miller’s pyramid
Goals of WBA
• Make a judgement on a student’s performance of a specific task in the clinical setting
• Provide guidance to the student through effective feedback to improve future performance of the task
Examples
• Encounter based– Case Discussion (Long case in the workplace)– Mini-CEX (direct observation clinical/procedural
skill)
• Global– Clinical Attachment Assessment– Multi-Source Feedback (360)– Shift Feedback Form
Year 3 Mini-CEX
• Student (or assessor) selects a patient • Student performs a focused clinical task e.g.
history taking, physical examination, performing a procedural skill, counseling a patient
• Assessor directly observes the encounter• Assessor rates the student’s performance in a
range of domains and provides feedback• No longer than 30 min including feedback
Year 3 Mini-CEX
Challenges• Competence is developmental vs. normative (not
satisfactory, borderline, satisfactory, good and excellent)
• Assessors do not read the back page – use their own internal rubric
• Reluctance to rate a student as “not satisfactory” – “Failure to fail”
• Feedback is often limited and not as helpful as it could be
• A “tick box” exercise resulting in students chasing a number
Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ, 45(6), 560-569.
Construct aligned scales
• Judgement on extent to which a supervisor would trust a trainee (or student) to perform a task independently
• Descriptors reflect the need for close or more distant supervision
• Different contexts will have a different way of construing “independence”
• Greater reliability with fewer assessments– 3 vs. 6 Mini-CEXs required to achieve a
Generalisability coefficient 0.7
Domains of assessment
Construct aligned Mini-CEX
Providing effective feedback
Effective Feedback
• Characteristics• Perceived need for change• Technique• Creating an action plan• Coaching
Delivering Feedback
• Enable learner reaction• Encourage self assessment• Provide an assessment of strengths and
weaknesses• Develop an action plan• Document the feedback
Year 5 Emergency Medicine
Shift Feedback Form
Summary
• Competence is developmental• Construct aligned scales may help align both
assessor judgements on performance and the development of competence by the student
• Delivering effective feedback is an important part of the assessment process to guide student learning– Action plan and coaching
Questions?