Reflective Feedback in GME

8/2/2019 Reflective Feedback in GME

1/38

Reflective Feedback in GraduateMedical Education

Adah Chung, M.Ed, University of Colorado School ofMedicine Center for Advancing Professional Excellence, USA

5th Annual ASPE Conference

Tucson, AZAugust, 2006


2/38

Agenda

Welcome and Introductions

Objectives

Background and challenges

Brief literature review

What we did Reflective feedback

Discussion

Concluding remarks


3/38

Lets talk

What are your program experiences withworking with residencies?

None an area were interested in Objective Structured Clinical Examinations

Communications training

Small group teaching

Formative assessment with feedback


4/38

Objectives

Participants will understand how using reflectivefeedback with standardized patients in graduatemedical education can help to meet the ACGMErequirements.

Participants will learn about the benefits of usingverbal feedback for residents.

Participants will learn about a subjective writtenpatient perception tool to guide discussion withresidents

Participants will learn how a formativeassessment using immediate reflective feedbackis implemented.


5/38

A little history

ACGME Outcomes Project emphasis on educationaloutcomes assessment based on (6) competencies: Patient care Medical knowledge Practice based learning Communication Professionalism System Based practice

ACGME Toolbox: Checklist evaluation Global rating of live or recorded performance SP exam

OSCEs Patient Surveys


6/38

The challenges faced

Resident competencies (skills,knowledge and attitudes) must betranslated into a set of complexbehaviors that result in the deliveryof high-quality medical care(Whitcomb, 2002)

how to measure this??


7/38


Checklist evaluation: Difficulty with consensus on essential

behaviors/action and sequence Scoring behavior using checklists is more

difficult when checklists assume a fixedsequence of actions because experiencedphysicians use various valid sequences(ACGME Toolbox, 2000)

Checklists unable to capture complex and

hierarchical problem-solving skills ofexperienced clinicians (Hodges et al, 2002)


8/38


SP Exams, OSCEs:

Perception of SP exams by resident

directors and residents Pervasive idea that residents should

have skills and behaviors at a morecomplex level than medical students


9/38


Patient Surveys:

Not enough specific behavior

information for formative evaluationand performance improvement(ACGME Toolbox, 2000)


10/38

A possible solution a formativeassessment (educational experience)

Allows reflection from residents (meetscomplexity/professional competenceconcerns)

Non-threatening environment for learning(meets attitudes towards SPs)

Trained SPs can provide more specificbehavioral feedback than patients (meets

patient survey concerns)


11/38

The purpose of assessment

The outcomes of assessmentshould foster learning, inspireconfidence in the learner, enhancethe learners ability to self-monitorand drive institutional self-assessment and curricular change.

Epstein, R.M. & Hundert, E.M. (2002) Defining andassessing professional competence, JAMA, 287(2):226-235.


12/38

Professional competency

Epstein & Hundert (2002):

Professional competence developmental, impermanent andcontext dependent

More than demonstration of isolatedcompetencies

A competent physician possesses theintegrative ability to think, feel and actlike a physician


13/38

Dimensions of competence

Barnett (1994), Talbot (2004):

Knows that knows how reflective

knowing

Reflective knowing: focus on dialogand argument for better practicalunderstanding


14/38

Definition of reflection

Reflection:

6: A thought, idea or opinion

formed or a remark made as aresult of meditation, 7:consideration of some subjectmatter, idea or purpose

Merriam-Webster Dictionary


15/38

Reflection

Donald Schon (1983):

Reflection-on-action reflecting onexperiences

Adds to professional repertoire

Difference between effective and non-effective practitioners


16/38

Reflection contd

Reflection as a way to help

professionals integrate skills,knowledge, behavior and attitudes

Reflection as a higher levelcognitive and personal process as

one develops professionally


17/38

My question:

If reflection is the key to the difference inlearning & it promotes integration of skillsand behaviors that residents shouldpossess, then can it be facilitated?


18/38

The literature on SPs giving feedback

Howley & Martindale (2004):

Students who responded favorably toSP experience more likely to have

received feedback from SPs

Students agreed strongly that feedbackfrom SPs are clear, non judgmental,specific and provided them with

patients perspective of the encounter


19/38

The literature on SPs giving feedback

Hodder, Rivington, Calcutt & Hood(1989):

Even brief feedback encounters (2minutes) can have significant influenceon clinical competency


20/38

Feedback & Attitudes

Efficacy of feedback:students who receive feedback

regarding clinical performance report

more positive attitudes towards theclinical experience Howley, LD & Martindale, J. (2004). The efficacy of standardized

patient feedback in clinical teaching. A mixed methods analysis.Med Educ Online, 9(18): 1-10.

Attitudes are an important factor inadult learning!


21/38

Questions?

Agree or disagree reflectionimportant for graduate level medicaleducation/professionals?


22/38

What we did

A formative assessment based on 3aspects of the ACGMEcompetencies:

Patient Care

Communication and Interpersonal Skills

Professionalism


23/38

Logistics

4 stations: 20 min. patient encounters

10 min. verbal feedback from SP/TA

Checklists were in the form of patientperception questionnaires helped toguide discussion with resident

Cases: Bad news, Smoking cessation,

Diabetes management, Urological exam(TA)


24/38

Logistics contd

Feedback session: Invite resident to self assess

Provide a positive statement

Provide an area to improve

End with an overall positive statement

Use the PPQ as a tool to guidediscussion

Describing feelings from patientsperspective


25/38

Reflective Feedback

Promotes reflection as part of adialog (both parties involved)

Focuses on observed behavior Descriptive vs. judgemental

Specific vs. general

Sharing information vs. advicegiving


26/38

Training of SPs

Focused on feedback using afeedback manual: Purpose of feedback

Characteristics of feedback Tips for success

Methods for giving feedback (thesandwich, I statements)

Examples and practice scenarios Appendices


27/38

Patient Perception QuestionnaireChecklist items Yes No

1.Would you do what this doctor asks you to do?Patient Care

2. Did this doctor demonstrate compassion?3.Was this doctor respectful?

Communication and Interpersonal Skills

4. Did this doctor communicate effectively?How?

______________________________________________________________________

______________________________________________________________________

_______________________________

5. Did this doctor demonstrate effective listening skills?How?

______________________________________________________________________

______________________________________________________________________

_______________________________

6. Did this doctor demonstrate nonverbal communication skills?How?

______________________________________________________________________

______________________________________________________________________

_______________________________

7. Did this doctor demonstrate the ability to establish a working relationship with

you?

8. Did this doctor elicit and answer your questions?Professionalism

9. Did this doctor demonstrate sensitivity and responsiveness to your needs?10. Did you feel you could trust this doctor to be your advocate?


28/38

PPQ

Terminology and categoriesfollowing (3) ACGME competencies

Yes/No provide more useful,reproducible results (ACGMEToolbox, 2000)

Room for comments of specific

behaviors for performanceimprovement


29/38

PPQ guide

Guide for training focused on:

Understanding items

Examples for things that could be written

under HOW? Ie. Effective communication

Invited me to tell my story

Summarized what I said

Allowed me to ask questions

Ie. Non-effective communication Interrupted me

Used closed ended questions


30/38

An example

Hi thanks for coming in. How did you think things went? It really is uncomfortable to talk to someone when shes

crying I didnt know what to do. When you put your hand on my shoulder, I felt reassured. I

also liked that you gave me space to cry rather than trying totalk through my tears.

Oh Im glad I actually just didnt know what to say Saying nothing at all was very comforting to me What else could I have done? Several times you tapped your foot on the floor and looked

at your notes rather than me that made me feel like I wastaking up too much of your time.

I didnt even notice that I was doing that! Overall, your gentle demeanor and touch helped me to

understand the news you were giving me. Thank you.


31/38

Resident response

Receiving feedback from standardized patients was very helpful Direct feedback Great food Good learning tool. I felt most of the feedback was useful, but there was no criticism The topics are relevant to practice Ability to ask afterward how they thought I did Immediate feedback Good feedback/Good cases. Immediate feedback was very helpful Excellent specific feedback Helpful feedback. It helps me to see my weak points. I think focusing on communication is best Very low pressure atmosphere. Real life 20 min appts


32/38

Benefits of this approach

Uses ACGME recommended toolbox ideas:combining SP encounters with patientsurveys

Provides structured observation Allows for professional self reflection

(reflection-on-action)

Checklist data for residency directors

Direct feedback for residents from trainedstandardized patients (changing attitudes)


33/38

The purpose of assessment

The outcomes of assessmentshould foster learning, inspireconfidence in the learner, enhancethe learners ability to self-monitorand drive institutional self-assessment and curricular change.

Epstein, R.M. & Hundert, E.M. (2002) Defining and

assessing professional competence, JAMA, 287(2):226-235.


34/38

Discussion

What are the advantages anddisadvantages of using a reflective,

narrative, verbal discussion withstandardized patients?

What strengths and/or weaknesses do

standardized patients bring to this kind offormative assessment?


35/38

Conclusion

A direct feedback method with a

subjective written tool can help todemonstrate competencies whileappreciating the complex learninginteraction that happens as one

becomes a professional


36/38

Future considerations

An added written self reflection

exercise? Research on the efficacy of SP

feedback to promote reflection


37/38

Final thoughts

Recognition of SP strengths andweaknesses and how to meet the needs ofthe residents and residency programs

Challenge to think about ways we canassist residency programs in creative ways,other than our usual OSCE and SP exams

Be intentional rather than accidental!


38/38

Thank you!

Please contact Adah Chung, MEd. [email protected] or (303) 724-1209for more information
mailto:[email protected]:[email protected]

Documents

Reflective Feedback in GME