Workplace Learning

The Evidence for Teaching,Learning and Assessing in theLearning and Assessing in theWorkplace

Karen V. Mann PhD

1

Overview

1. What are the assumptions of WL?

2. What are the advantages andlimitations?

What is the role of the teacher in WL?3. What is the role of the teacher in WL?

4. What is the role of the learner in WL?

5. Future directions

2

Workplace learning

Broad term, covers a range of topics

Young and old

Evolving our understanding of apprenticeship

3

Workplace Learning

Deliberate learning

Physicians need to “go beyond the mereaccumulation of experiences” (van de Wiel et al.,

2011, p. 83)

Often overlooked as a valuable tool

Essential for the improvement oftrainees’ performance

Life long practice

4

Assumptions

1. Experience

2. Supervision

5

Experience

Empowering and encouraging environment Trainee must feel supported to engage in experiential

opportunities (van der Zwet et al., 2010)

Opportunities to participate (e.g., clerkships) inenvironments that challenge the trainee andenvironments that challenge the trainee andprovide quality encounters(Eraut, 2011; Pearson & Lucas, 2011; van der Zwet et al., 2010)

Participation is motivating and encouragesfeeling like a doctor (Dornan et al., 2007)

6

Motivation in the LearningEnvironment: An Example

Experience-based learning in undergraduate students

• Participation was central to students’ learning

• Supported participationAffective– Affective

– Pedagogic– Organizational

• Participation at all levels:– Boosted confidence– Increased motivation– Enhanced professional identity– Increased confidence

(Dornan, et al, 2007)

7

Experience

’We have definitely learned a lot from itthat will be useful in our final examsand beyond. The downside of beinghere...it would be nice to have a littlehere...it would be nice to have a littlebit more clinical experience on ourown, rather than just sitting in withthe GPs.’ (Fifth-year medical student)

(Pearson & Lucas, 2011)

8

Experience

Learning from the problemsencountered (van de Wiel et al., 2011)

Clinical learning occurs through Clinical learning occurs throughengagement with many professionalgroups and levels of experiencethrough a respectful manner (Pearson & Lucas,

2011)

9

Experience

Several factors influence experience in workplace learning

Enthusiasm and commitment of teacher

Presence of clear objectives and guidance

Patient cases / clinical work

Immediate problems vs. gaps in skills/knowledge Immediate problems vs. gaps in skills/knowledge

Cooperation of colleagues, supervisors and otherspecialists

Recognition and respect when contributing to theteam

10

The quality of supervision (r = 0.73, p < .01),patient mix (r = 0.65, p < .01), and independence

(r = 0.48, p < .01) impact instructional quality(van der Zwet et al., 2010)

Adequate supervision

a. Assessment-Supervisory assessment and self-assessment

b. Observation-Mini-clinical evaluation and direction observation

of skillsof skills

c. Discussion-Clinical cases, case based discussion

d. Feedback-Peers, coworkers & patients; multi-source

feedback; objective tools- mini-PAT, TAB &PSQ

11

Assessment

Evaluation of skills and practices undertakenin day-to-day work

Assessment supports development of Assessment supports development ofcompetencies

Competency does not always predictperformance

Trainees’ concern about meeting expectationsof teachers (Eraut, 2011)

12

Assessment

How does assessment change learning?

Credibility and Relevancy

Multiple sources of feedback for trainees toreflect uponreflect upon

13

‘Certainly the more respect I had for them the morenegative feedback and the more straightforward feedback Icould tolerate from them.’ (P6)

’…after I told the patient the bad news, then the family toldme, “You seem cold or kind of not compassionate.” I tookthat very seriously’ (P3)

(Watling et al., 2012)

Observation

Direct observation provides the data forfeedback (Ryan et al., 2010)

Faculty, Student and System issues caninfluence the ability to observe or beinfluence the ability to observe or beobserved (Kogan et al, 2011)

Faculty need support to develop skill inobservation and feedback (Fromme et al., 2009)

Observation and feedback requires planningand intention

14

Discussion

Consulting with colleagues/peers on acase

Incidental learning

Who to approach? Who to approach?

Coming to a common ground ondifferences of opinions

Opportunities to ‘learn to talk’, and‘learn from talk’ (Lave and Wenger,1991)

15

Discussion

Difficult cases discussed in daily orspecial review meetings help solvemedical problems and providesalternative solutions for traineesalternative solutions for trainees

16

’I see many routine cases. Occasionally you have to lookthings up. Occasionally it’s good to test your own opinionagainst those of others. That’s what the Friday afternoonpatient review meetings are for,…our strength is that we’re ateam; by talking to each other, we improve our level.’(van de Wiel et al., 2011)

Feedback

Providing Feedback Critical in guiding/adjusting the trainee to

desired outcomes and positive impact ondoctors’ learning & performance(Eraut, 2011; Miller, 2010; Norcini & Burch, 2007; van de Wiel(Eraut, 2011; Miller, 2010; Norcini & Burch, 2007; van de Wielet al., 2011)

Consistent feedback from a credible sourcecan change clinical performance(Veloski et al., 2006; Watling et al., 2010)

17

Feedback

Receiving feedback Multisource feedback is the most evaluated form of

feedback, but there is conflicting evidence on itseffectiveness (Miller, 2010)

Effectiveness of feedback for the learner depends oncredibility of advice and frequency of negative feedbackcredibility of advice and frequency of negative feedback(Eraut, 2011)

Integrating/evaluating is dependent on who isproviding the feedback (e.g. Patient vs. Peer)

Feedback that is incongruent with self-assessment maybe rejected and evoke strong emotional response(Sargeant et al, 2009)

18

Patients rated doctors significantly higher on performancethan their colleagues (4.34 vs. 3.69, t(df, 66)=7.7, p < 0.001)

(Archer & McAvoy, 2011)

Feedback

Seeking & incorporating feedback

Learning and change can occur whenfeedback indicates a need for change

Positive interpretation of feedback and Positive interpretation of feedback andbelieving change is possible improveslearning and growth

Most often sought when immediate careneeds to be provided (van de Wiel et al., 2011)

19

Tensions in seeking and usingfeedback

• Between people

• Wishing to learn and improve vs. wishingto appear knowledgeable and confidentamong peers and superiors

• Tensions in learning environment

• Providing genuine assessment feedbackvs. “playing the evaluation game”

• Tensions within self

• Wanting feedback yet fearingdisconfirming information

(Mann et al, 2011)

20

Advantages of Workplace Learning

Contributes to professional and personalgrowth, and increases confidence inability (Lester & Costley, 2010)

Benefits the organization Benefits the organization

Allows for learning at both the individualand the collective level (Billett 2004)

21

Limitations of Work-base learning

Reciprocal relationship betweenenvironment and learner

Often no definitive guide for facilitatingWBLWBL

Assessment and feedback may beinfluenced by interpersonal relationships(Norcini & Burch, 2007; Watling et al., 2010)

Faculty development, and learnersupport are key to success

22

Role of the Teacher

Offer

comfortable and safe working and learningenvironments

multiple opportunities to engage in practice multiple opportunities to engage in practice

meaningful and emotional encounters withpatients

observation

purposeful and relevant feedback

trust and value in trainees’ competenciesand skills

care, commitment and enthusiasm forteaching 23

Role of the teacher

’[The GPs] will say you need to know that, goaway and read that and it is nice becauseyou do not often get that. People just go‘you should know this’. Well there is a lot ofthings that I should know but to actually bethings that I should know but to actually begiven a title and certain key things to goaway and learn is extraordinarily helpful.’(Fifth-year medical student, Pearson &Lucas, 2011)

Role of the Teacher

Encourage: critical thinking

questioning and seeking answers

knowledge sharing knowledge sharing

engagement in cases and learningopportunities

checking/discussing mutualunderstandings (e.g., coming to acommon solution)

25

Motivation in the LearningEnvironment: An Example

How residents learn

Feedback is essential to motivation

Learning and performance goal Learning and performance goalorientation

Instrumental and supportiveleadership

(Teunissen et al., 2009)

26

Preceptors’ Influence on Motivation

Instrumental Leadership Supportive Leadership

Clear Goals Friendly, ApproachableStructured Work ConsiderateClear Goals Friendly, ApproachableStructured Work ConsiderateGuidelines

More Feedback Seeking More Feedback SeekingIncreased Feedback Increased FeedbackProvision and Value Provision and Value

(Teunissen, et al., 2009)

27

Informal Learning and the HiddenCurriculum

• Loss of idealism

• Adoption of a “ritualized” professional identity

• Emotional neutralizationEmotional neutralization

• Change of ethical integrity

• Acceptance of hierarchy

• Learning of less formal aspects of “gooddoctoring”

(Lempp & Seale, 2004)

28

The Hidden Curriculum

• Power and hierarchy

• Patient

dehumanization

Hidden assessments

• Emerging

accountability

• Balance and sacrifice

Faking it• Hidden assessments

• Emotional

suppression

• Limits of medicine

• Faking it

• Human connection

(Gaufberg et al., 2010)

29

How is Caring Learned?

Professional and pedagogical caring

A vehicle for integrating scientific and humanaspects of professional practice

Interactions between teachers and students

Long term effects of caring

(Cavanaugh, 2002; Haidet et al., 2006)

30

Faculty’s Role in TeachingHumanistic Behaviour

Faculty selected by residents as role models &teachers of humanistic care

Observations revealed non-verbal behaviours:• Demonstration of respect, overtly and frequently

• Building a personal connection

• Eliciting patients’ emotional responses to illness

• Demonstration of self-awareness and reflection

(Weissman et al., 2006)

31

The Student-Supervisor Relationship

The student-supervisor relationship functions to:

• Facilitate direct transmission of patient-centredknowledge, skills and attitudes

• Provide social support for the student’s patient-centred• Provide social support for the student’s patient-centredbehaviour

• Provide support of the student as person

• Mirror patient-centered behaviour by being learner-centred

• Address supervisor vulnerability

(Bombeke et al. , 2010)

32

Role of the learner

Engage in WL to learn and to improvepatient care

Unsure of solution and/or gap in skills

Practical experience triggers reflections Practical experience triggers reflectionsand problem solving

Self-assessment (Eraut, 2011; van deWiel et al, 2011)

Seeking feedback is related to trainees’ability to adequately assess their ownperformance

33

Role of the learner

Provide support to peers andencourage discussion andexchange of information

Informal peer learning was strong and Informal peer learning was strong andimportant in practice

34

‘…you get to compare your level of knowledge withyour peers and also.. .you know, you can teach themand they can teach you as well...’ (Pearson & Lucas,

2011,)

Where Is the Learner’s Voice?

Residents in Family Medicine viewed receivingspecific, timely, and frequent feedback as keyto making the CBAS a worthwhile investmentfor them.

Feedback encouraged their own self-assessmentand motivation to fill gaps.

Faculty development is critical

Learner development and orientation are critical

(Ross et al, 2012)

35

Future Direction

Little work has been done to empirically assess theeffectiveness of work-based learning on doctors’performance and learning

Need to understand what environments facilitate andsupport WBL

Future work should look at incorporating mixedmethodologies and focus on the quality of the data

Literature is dominated by theoretical studies (van deWiel et al., 2011)

What they ‘should’ do vs. what they ‘actually’ do

36

Summary

1. What are the assumptions of WL?

2. What are the advantages andlimitations?

What is the role of the teacher in WL?3. What is the role of the teacher in WL?

4. What is the role of the learner in WL?

5. Future directions

37

Thank you!

Your comments and questions?

References

Archer, J. (2010). State of the science in health professional education: effective feedback. Med Teach,29: 855-71.

Archer, J. & McAvoy, P. (2011). Factors that might undermine the validity of patient and multi-sourcefeedback. Med Edu, 45: 856-93.

Billett S. (2004). Workplace participatory practices: conceptualising workplaces as learningenvironments. J Workplace Lear, 16:312–24.

Bombeke K, Symons L, Debaene L, et al. (2010). Help: I’m losing patient centredness! Experiences ofmedical students and their teachers. Medical Education, 44: 662-673.Bombeke K, Symons L, Debaene L, et al. (2010). Help: I’m losing patient centredness! Experiences ofmedical students and their teachers. Medical Education, 44: 662-673.

Cavanaugh SH. (2002). Professional caring in the curriculum. In Norman G, van der Vleuten CPM &Newble D (eds.) International Handbook of Research in Medical Education. Dordecht: Kluwer, pp. 981-996.

Dornan T, Boshuizen H, King N, Scherpier A (2007). Experience-based learning: a model linking theprocessess and outcomes of medical students’ wokplace learning. Med Educ 41:84-91

Eraut, M. (2011). Informal learning in the workplace: evidence on the real value of work-based learning(WBL). Development and learning in Organizations, 25(5): 8 -12.

Fromme, H. B., Karani, R., & Downing, S. (2009). Direct observation in medical education: Review ofthe literature and evidence for validity. Mount Sinai School of Medicine, 76: 365 0 371.

Lave J, Wenger E (1991). Situated Learning. Legitimate Peripheral Participation. Cambridge, UK:Cambridge University Press.

39

References

Gaufberg L, Batalden M, Sands R & Bell SK. (2010). The hidden curriculum: what can we learn fromstudent narrative reflections? Academic Medicine, 85: 1709-1716.

Haidet P, Kelly PA, Bentley S, et al. (2006). Not the same everywhere: patient-centered learningenvironment of nine medical schools. Journal of General Internal Medicine, 21: 405-409.

Lester, S. & Costley, C. (2010). Work-base learning at higher education level: value, practice andcritique. Studies in Higher Education

Lempp H, Seale C. (2004). The hidden curriculum in undergraduate medical education: qualitative Lempp H, Seale C. (2004). The hidden curriculum in undergraduate medical education: qualitativestudy of medical students’ perception of teaching. British Medical Journal , 329: 770-773.

Kogan, J., Conforti, L., Bernabeo, E., Iobst, W & Holmboe, E. (2011). Opening the black box of clinicalskills assessment via observation: a conceptual model. Med Edu, 45L 1048-1060.

Mann K, van der Vleuten C, Eva K, Armson H, Chesluk B et al. (2011). Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med86:1120-1127.

Miller, A. (2010). Impact of workplace based assessment on doctors’ education and performance: asystematic review. BMJ, 341(c5064).

Norcini, J., & Burch, V. (2007). Workplace-based assessment as an educational tool: AMEE Guide No31. Med Teach, 29: 855-71.

Pearson, D., & Lucas, B. (2011). Engagement and opportunity in clinical learning: Findings from a casestudy in primary care. Medical Teacher, 33: e670-77.

40

References

Ross, S., Poth, C., Donoff, M., Papile, C., Humphries, P. et al. (2012). Involving users in the refinement ofthe competency-based achievement system: An innovative approach to competency-based assessment.Med Teach, 34(2): e143-147.

Ryan, J., Barlas, D., & Sharma, M. (2010). Direct observation evaluations by emergency medicine facultydo not provide data that enhance resident assessment when compared to summative quarterlyevaluations. Academic Emergency Medicine, 17: S72 - 77.

Teunissen P, Stapel D, van der Vleuten C, Scherpbier A, Boor K, et al. (2009). Who wants feedback? Aninvestigation of the variables influencing residents’ feedback seeking behaviour in relation to night shifts.Acad Med 84:910-917.Acad Med 84:910-917.

van der Vleuten, C (1996). The assessment of professional competence: development, research andpractical implications. Adv Health Sci Educ, 1: 41-67.

van de Wiel, M., van den Bossche, P., Janssen, S. & Jossberger, H. (2011). Exploring deliberate practicein medicine: how do physicians learn in the workplace? Adv in Heal Sci Educ, 16: 81-95.

van der Zwet, J., Zwietering, P., Teunissen, P., van der Vleuten, C. & Scherpbier, A. (2010). Workplacelearning in general practice: Supervision, patient mix and independence emerge from the black box onceagain. Med Teach, 32: e294-99.

Veloski, J., Boex, J., Grasberger, M., Evans, A., & Wolfson, D. (2006). Systematic review of the literatureon assessment, feedback and physicians’ clinical performance: BEME Guide No 7. Med Teach, 28: 117-28.

Watling, C., Driesen, E., van der Vleuten, C., & Lingard, L. (2012). Learning from clinical work: the rolesof learning cues and credibility judgments. Medical education, 46: 192-200.

Weissman P, Branch W, et al. (2006). Role modeling humanistic behavior: leaning bedside manner fromthe experts. Academic Medicine 81: 661-667.

41