Original Article

Helping International Medical GraduatesEngage in Effective Feedback

Karen E. Broquet, M.D.

Manisha Punwani, M.D.

Objective: Timely, specific, behaviorally-based feedback is acornerstone of medical education. The authors review basictenets of effective feedback delivery in the context of potentialchallenges faced by (non-United States) international medicalgraduates (IMGs).

Method: The authors provide a brief summary of the elementsof effective feedback, with illustrations of potential barriers forIMGs.

Results: Many IMGs were trained in a hierarchical system,where feedback was delivered publicly, in a manner associatedwith shame and embarrassment. These experiences, combinedwith the challenge of functioning in a second language andanxieties over exposing weaknesses, present some barriers thatmake it more difficult for them to participate in feedback inquiry,self-reflection, and reciprocal feedback.

Conclusions: These challenges can bemitigated by acknowledg-ing the anxieties that learners may have, fostering a learning cul-ture that values feedback as an expected and important part of alllearning, ensuring that all (learners and supervisors) are trainedin feedback skills, and clear setting of expectations.

Academic Psychiatry 2012; 36:282–287

The provision of feedback is one of the most profoundlyeffective educational tools we have for both improve-

ment of performance and for guiding learning. This is truefor the junior medical student or the senior physician, regard-less of when or where they trained. Ende defined feedbackas “the formative process by which the teacher provides thelearner with information about his/her performance, for thepurpose of improving that performance” (1).Without specificand focused feedback from teachers, learners tend to deter-mine for themselves the quality of their work, either by self-assessment or by monitoring. Neither of these impartsaccurate information to the learner. Self-assessment is oftenunreliable, and, without reinforcement by teachers, learnersmay extinguish desired behaviors or establish undesirableones. Often, a learner does something especially positivewithout being aware of it. Monitoring involves the interpre-tation of perceived cues or feedback from other people’sbehaviors (2). Monitored feedback is received through theindividual’s own filter or self-perception, and it is prone tomisinterpretation. For example, a supervisor may grimacebecause of the twinge of an arthritic knee during a learner’scase presentation. A learner who is feedback monitoringmay well notice that and incorrectly infer a negative as-sessment of his/her performance.By definition, accurate feedback is going to include cor-

rective (negative) as well as reinforcing (positive) items.Human nature being what it is, teachers are often uncom-fortable giving what feels like critical feedback, and learnersare often reluctant to seek it. This can be addressed in part bytraining and skill-building for feedback-givers and by creat-ing a culture of a reciprocal feedback for all. BecauseLCME-accredited medical schools have specific standardsfor feedback, most U.S. medical graduates enter residencytraining with a certain amount of familiarity, if not comfort,with the feedback process. Many international medicalgraduates (IMGs) enter United States residency trainingwith either limited experience with feedback or negative

Received February 22, 2011; revised May 13, July 20, 2011; accepted July27, 2011. From theDept. of Psychiatry, Southern Illinois University Schoolof Medicine, Springfield, IL. Correspondence: Karen E. Broquet, M.D.;e-mail: kbroquet@siumed.eduCopyright © 2012 Academic Psychiatry

282 http://ap.psychiatryonline.org Academic Psychiatry, 36:4, July-August 2012

perceptions and experiences with feedback. The lack ofcomfort and familiarity with feedback has been postulatedto be an associated factor in IMGs with poor academic per-formance (3). In 2009–2010, IMGs made up 27.4% of U.S.residents (4). The percentage of IMGs in psychiatricresidency/fellowship positions was slightly higher, at33% (5). Of these, about three-quarters are non-U.S. IMGs.In this article, the term “IMG” will refer to non-U.S. IMGs.Schools in India/Nepal and Pakistan educated the largestgroup of new IMGs entering United States residency in2009 (6).

Method

In this article, we will outline basic tenets of good feed-back for all learners and address some of the issues that canpresent barriers to the delivery and receipt of effectivefeedback for IMGs. The authors are a current and formerpsychiatry program director, with 27 combined years of ex-perience working with United States-trained and IMG resi-dents. In preparation for this research, we reviewed pertinentliterature regarding effective feedback, pulled from personalobservations, reviewed narrative accounts of IMGs trainingin the U.S. and Canada (7–9), and interviewed a small groupof internal medicine and/or psychiatry residents (4 men,1 woman) from India, Nepal, and Nigeria about their expe-riences with feedback. Their comments have been usedwith permission to illustrate salient points.

Delivery of Effective FeedbackCompared with North American schools, many interna-

tional medical schools have a much steeper power-differential between teachers and learners (7). Formativefeedback is often not a uniform component of the curric-ulum. For many IMGs, feedback is tantamount to criticismand is interpreted as a serious failure on the part of thelearner. This can make it difficult to actively seek feed-back or engage in self-assessment, as it is seen as ex-posing weaknesses:

We called the doctors “Ma’am” and “Sir” and stood up when theywalked into the room. Mistakes were pointed out publicly–seldomconstructively.Feedback was given during rounds in front of the patients, nurses andpeers . . . It was very embarrassing. The belief is that we will studymore to avoid shame. I still get anxious when I am called in forfeedback.The only way to do it (give feedback) was by making you embar-rassed . . . yelling at you in front of the patient and family. You onlygot feedback if you were in the limelight, presenting or something.

This group also described a culture of extreme competitionand pressure to be the best:

Everyone has to excel; otherwise it reflects badly on the professor. Ifyou weren’t among the top scorers, it was regarded as failure. Self-reflection was not encouraged or even tolerated, as no student wantedto expose weaknesses.

Multiple authors have contemplated and reported onlearner and teacher factors that contribute to an effectivefeedback experience (1, 10–13). Brinko (10) organizedthem into a variation of journalism’s “five ‘W’s:”

Feedback Is Effective When:

Who

l It comes from a number of sources (peers, supervisors,patients, etc.)l It comes from self as well as others.l The source is perceived by a learner as credible andwell-intentioned.

Any well done feedback session should begin with self-reflection. (“How do you think you did?” or “Tell me whatyou did well and what you would like to improve upon.”)This necessary step is often challenging for learners andteachers. An honest self-assessment will often feel like ei-ther bragging or exposing weaknesses, both of which cangenerate anxiety in the learner. For IMGs, the fear of ex-posing weakness and the pressure to excel, both for the sakeof the learner and the teacher, can make this vital step of theprocess more anxiety-provoking. Searight and Gafford (8)and Dorgan et al (9) interviewed family- and community-medicine residents from a variety of international medicalschools regarding their perceptions of doctor–patient com-munication and their own experiences with behavioral-science education. In these interviews, a number of theIMG residents felt that they were being scrutinized by theirpatients and faculty more closely than their U.S.-trainedpeers. This could clearly influence how credible the feed-back is perceived to be. These residents also echoed thefeeling that they needed to prove themselves to faculty. Also,a number described a high level of insecurity and anxietyabout being terminated. This reluctance to self-assess is ofteninterpreted by the supervisor as a lack of interest or lack ofself-reflection. A predictable vicious cycle ensues.

What

l It contains concrete, specific information.l It contains accurate, irrefutable data/observations.l It is focused on behaviors (not personality).l It is limited to actions (not presumed intentions).

Academic Psychiatry, 36:4, July-August 2012 http://ap.psychiatryonline.org 283

BROQUET AND PUNWANI

l It is limited to behaviors that are remediable.l It creates cognitive dissonance (between the learners’self-perception of performance and where he/she would liketo be).l It contains suggestions for improvement (must end withan action plan).l It is reciprocal.

In terms of performance-improvement, immediate feed-back based on direct observation of specific behaviors isthe ideal. However, it is not uncommon in training pro-grams for the lines between feedback and evaluation tobecome blurred. It is very important for both learners andteachers to be clear on the differences between feedbackand evaluation. Feedback can be seen as a form of coachingor continuous performance-monitoring and correction.Evaluation, on the other hand, is a summative assessment,which involves a judgment or rating for a particular per-formance or time-period. It is useful to see these as pointson an educational continuum.Pure formative feedback will often take place in the

context of a specific clinical encounter or direct observa-tion. It’s useful to conceptualize this as an educational giftfor learners. Most feedback will never appear on a summa-tive evaluation unless previously-identified deficiencies arenot corrected. Formative evaluation refers to that blendingof the two. Formative evaluation typically takes the formof a mid-rotation feedback session and often includes in-formation not just from one encounter, but from a variety ofencounters and even second-hand input from differentsources (i.e., nursing, etc.) over the course of the first half ofthe rotation. The summative evaluation is what we typicallyconsider the formal end-of-rotation evaluation, which maybe delivered face-to-face and in written form. If feedback isoccurring on a regular basis, there should be no surprises forthe learner in the formative evaluation/mid-rotation feed-back. Likewise, if formative evaluation is taking place, thereshould be no surprises for the learner in the final or sum-mative evaluation. This is true for all learners. For IMGs,it can be helpful to clearly and specifically identify anencounter as feedback versus mid-rotation formative eval-uation, etc., to temper some of the previously-describedanxieties.In a well-functioning system, feedback is reciprocal.

Although most residents recognize that faculty need feed-back to make their teaching experiences the best, manyIMGs have difficulty with this. In discussion of reciprocalfeedback in our focus group, one senior resident stated:“There is no way we can do that!” However, our group

reported that the fear factor associated with feedback, bothreceiving and giving, did dissipate somewhat with time inU.S. training, and all reported that it was important forfaculty to keep asking.

When

l It is expected.l It comes soon after performance.l It is given as a process.

Where

l It is given privately.l It is given in a psychologically safe environment.

A safe learning climate is of paramount importance for alllearners. Because of the association of feedback with shameand embarrassment that has been reported by many IMGs,this assumes a greater level of importance. One resident inour focus-group described postponing a scheduled feedbacksession with a very warm and caring faculty member threetimes because of anxiety, even though he knew the facultyinvolved would not belittle him in any way. As with self-assessment, the potential exists here for faculty to interpretthis as a resident who is unreceptive to feedback or unin-terested in his/her education.

How

l It is approached with teacher and learner working to-gether as allies with a common goal.l It is relevant to the learner andbasedon clear expectations.l It reduces uncertainty for the learner.l It allows for response and interaction.l It is phrased in descriptive, common, nonjudgmentallanguage.l It blends a moderate amount of positive feedback withselected negative feedback.

Communication and LanguageIn a well done feedback encounter, the learner should

walk out of the experience with a clear understanding ofhis/her supervisors’ expectations to either build on what thelearner is doing well or change what is not being done well.In many cases, the learner and teacher will be in agreementabout what the deficiencies are. There will be some sit-uations where the learner does not perceive the deficiencyor agree with the faculty. In these instances, it is doubly

284 http://ap.psychiatryonline.org Academic Psychiatry, 36:4, July-August 2012

INTERNATIONAL MEDICAL GRADUATES AND EFFECTIVE FEEDBACK

important to be clear on the expectations and what follow-up will be expected. The way feedback is phrased is veryimportant in how it is received. The language should benonjudgmental and descriptive. It is sometimes helpful tosandwich corrective (negative) feedback between reinfor-cing (positive) feedback. This approach can be highly valu-able for IMGs who may have remembered only negativefeedback in their earlier careers. When positive or rein-forcing feedback is given, it is more effective when it isattributed to internal causes (hard work, effort) and when itis given in the second person (i.e., “You handled thatpatient’s angry outburst very calmly and kindly.”). Negativefeedback, on the other hand, tends to be better receivedwhen it is delivered in the first person (i.e., “I had troublefollowing you when you presented the HPI,” as opposed to“You were very disorganized in your HPI.”). The pre-sentation of negative feedback is also better receivedwhen itis self-referenced, comparing the learner’s performance tomeasures of his/her ability and capabilities, rather than an-other learner’s performance (“If you make these changes,I’m confident that you will perform a much more accuratemental status exam.”) (10, 14).

Giving feedback to residents on interpersonal aspects oftheir patient encounters is often harder for faculty thangiving feedback on more neutral aspects of the physical ormental status examination, yet it is a vitally important part ofclinical care and professional development for residents inany specialty. For IMGs, physician–patient communicationis sometimes made more complicated by language diffi-culties. IMGs enter U.S. training with widely varying levelsof English speaking skills and exposure. Some havetrained in and functioned in English for years; others havejust recently mastered the language upon their decisionto train in the United States. Most IMGs have a good-to-excellent command of formal English. However, they mayhave had little exposure to the regional dialects, accents, orthe colloquialisms of their U.S. training sites. Because offactors previously described, IMGs may be reluctant to askpeople for repetition, for fear of exposing weaknesses. Onemember of our focus group reported:

I have difficulty in understanding Dr. ____ (a U.S. native) about halfthe time.When I don’t understandwhat he is saying, I tend to interpretit negatively.

When giving feedback to IMGs, it is doubly importanttherefore, for faculty to have the learner paraphrase whatthey said and make sure that he/she is clear on the behav-iors in question and action plan. A learner who comes topremature diagnostic closure and receives feedback about

“putting the cart before the horse”may well walk out of thesession perplexed about what to do differently next time.There is some evidence that IMGs may underestimate theimpact that their accent or language skills have in physician–patient communication. In their qualitative analysis ofinterviews with IMGs frommultiple countries, Dorgan, et al(9) reported that IMGs readily identified their patients’accents or dialect as a barrier to communication, but gen-erally did not identify their own accent or non-colloquialEnglish as a barrier. When language-related barriers areidentified in clinical encounters, it is tempting for super-visors to avoid addressing this in feedback in the mis-taken belief that language skill is a nonremediablebehavior. This is a misconception, and residents functioningin their second (or third!) language can benefit greatlyfrom accent-reduction classes, education in regional col-loquialisms, slang, etc., or increased opportunities to conversein English in non-medical terminology. When communica-tion factors are the subject of feedback, it is doubly importantto ascertain the resident’s awareness of the language issuesand to address them in an observationally-based, non-judgmental way.

The Role of Program DirectorIn any training program, although feedback should come

from a number of sources, the program director is animportant source of formative feedback, summative eval-uation, and multiple points in-between. There is someevidence that the program director may assume a greaterimportance to IMGs than to their U.S.-trained counterparts.Elliott et al. (15) surveyed 180 psychiatry residents (38%IMGs) and asked them to rate in order of importance thefactors associated with their level of satisfaction with theirtraining. The top five factors were identical for USMGsand IMGs except for one: The “personal qualities of theprogram director” was rated in the “top five” by IMGs, butnot USMGs. Ellencweig et al (16) replicated this withpsychiatry trainees in Israeli programs. Their sample hadabout 40% IMGs. In their survey, IMGs rated the “pro-fessional abilities of the program director” significantlyhigher in importance than did the Israeli residents. Theauthors theorized quite plausibly that the IMGs, as immi-grants, had a higher need for role models. This informationsuggests that when feedback is given to IMGs from the pro-gram director, it may assume a higher level of importance—and have a greater potential for both positive impact andcultural challenges. Although we are aware of no literatureexploring the effect of the supervisors’ country of trainingon the delivery of feedback, residents in our focus group

Academic Psychiatry, 36:4, July-August 2012 http://ap.psychiatryonline.org 285

BROQUET AND PUNWANI

reported no difference in feedback-associated anxiety whenreceiving it from U.S.-trained versus international faculty.Brinko (10) reported, in the non-medical educational liter-ature, that feedback is more effective when the source offeedback is lower or equal in status to the recipient. To ourknowledge, this has not been formally studied in medicaleducation with either USMGs or IMGs.

Feedback and SupervisionFrom the standpoint of performance-improvement, the

most effective feedback comes from direct observation ofresident–patient encounters. There is a large degree ofoverlap between feedback and supervision. A resident’swillingness to actively seek feedback regarding his/herclinical decisions is vital for adequate supervision and pa-tient safety. The July 2011 ACGME Common ProgramRequirements (CPRs) mandate that residents know whenthey can act independently and when they cannot. Eachprogram will set its own guidelines for when residents mustcontact their supervisor and when they can exercise dis-cretion (17). As medical educators, our understanding offeedback is going to have to expand from a technical un-derstanding of effective delivery of feedback to a greaterappreciation of what leads residents to be active seekers offeedback or supervision. Pulling from research in social andorganizational psychology, Teunissen et al. (2) examinedvariables associated with the active seeking of feedbackamong a group of OB/GYN residents on a night shift. Theydivided feedback-seeking behavior into that of inquiry andmonitoring. Feedback-inquiry is the active seeking of orasking for feedback. Feedback-monitoring is taking in in-formation from the environment by observing the behaviorof others. Feedback-monitoring can be problematic becauseit requires people to interpret and derive meaning them-selves from presented information. This information islikely to be interpreted in a manner consistent with theresidents’ self-perception—which may or may not be ac-curate. An individual’s propensity to seek feedback is di-rectly influenced by an assessment of the costs and benefitsassociated with feedback. If an individual feels that askingfor feedback may make him/her look incompetent, he/shethen would tend to refrain from active inquiry and engagemore in feedback-monitoring. All of the residents in thestudy engaged actively in feedback-monitoring. However,the residents who perceived more feedback benefits alsoengaged in feedback-inquiry. The residents who mainlyperceived the costs of feedback relied more exclusively onmonitoring. This group also identified some supervisorcharacteristics that were positively associated with active

feedback-inquiry: approachable, supportive attending physi-cians who were considerate of the residents’ needs were as-sociated with more perceived feedback benefits and fewercosts. A supervisory style with clear expectations and activeefforts to help structure and organize the residents’ workflowwas also associated with more active feedback-inquiry. Itis reasonable to expect that IMGs will have a higher assess-ment of the costs of feedback and will benefit from rou-tinely scheduled feedback, and very clear expectations andinstructions regarding when to call a senior or attending forfeedback/supervision.

Summary

Timely, specific, behaviorally-based feedback is a cor-nerstone of medical education. The ability of any physician(whether trainee or attending) to receive, integrate, andparticipate in feedback is vital. The ability to actively seekout faculty for supervision and feedback is assuming aneven greater importance with the onset of new accreditationrequirements. Many IMGs were trained in a hierarchicalsystem where active interaction with attending physicianswas not encouraged or tolerated, and the delivery of feed-back was often done publicly, in a manner associated withshame and embarrassment. These experiences, combinedwith the challenge of functioning in a second language andanxieties over exposing weaknesses, present some barriersthat make it more difficult for IMGs to participate in feed-back inquiry, self-reflection, and reciprocal feedback. Be-cause it is such a crucial part of training, it is in everyone’sbest interest to address these barriers wherever we can.These challenges can be mitigated by acknowledging theanxieties learners may have, fostering a learning culturethat values feedback as an expected and important part of alllearning, ensuring that all (learners and supervisors) aretrained in feedback skills, and clear setting of expectations.There is still much we have to learn about how culture, pastexperience, developmental stage, and delivery technique in-teract to influence the feedback experience for all learners.

References

1. Ende J: Feedback in clinical medical education. JAMA 1983;250:777–781

2. Teunissen PW, Stapel DA, van der Vleuten C, et al: Whowants feedback? an investigation of the variables influencingresidents’ feedback-seeking behavior in relation to nightshifts. Acad Med 2009; 84:910–917

3. Bates J, Andrew R: Untangling the roots of some IMGs’ pooracademic performance. Acad Med 2001; 76:43–46

286 http://ap.psychiatryonline.org Academic Psychiatry, 36:4, July-August 2012

INTERNATIONAL MEDICAL GRADUATES AND EFFECTIVE FEEDBACK

4. Brotherton SE, Etzel SI: Graduate medical education, 2009–2010. JAMA 2010; 304:1255–1270

5. American Psychiatric Association Resident Census 2009–20106. American Association of Medical Colleges: GME Track 20097. Walsh A.Working with IMGS: Delivering Effective Feedback.

A Faculty Development Program for Teachers of InternationalMedical Graduates. McMaster University, April 2006

8. Searight HR, Gafford J: Behavioral science education andthe international medical graduate. Acad Med 2006; 81:164–170

9. Dorgan KA, Lang F, Floyd M, et al: International medicalgraduate–patient communication: a qualitative analysis ofperceived barriers. Acad Med 2009; 84:1567–1575

10. Brinko K: The practice of giving feedback to improve teach-ing: what is effective? J Higher Ed 1993;64(5):574-593

11. Hewson MG, Little ML: Giving feedback in medical educa-tion: verification of recommended techniques. J Gen InternMed 1998; 13:111–116

12. Bienstock JL, Katz NT, Cox SM; Association of Professors ofGynecology andObstetrics UndergraduateMedical EducationCommittee: To the point: medical education reviews: pro-viding feedback. Am J Obstet Gynecol 2007; 196:508–513

13. Boehler M, Rodgers D, Schwinnd C, et al: An investigation ofmedical student reactions to feedback: a randomized, con-trolled trial. Med Ed 2006; 40:746–749

14. Milan FB, Parish SJ, Reichgott MJ: A model for educationalfeedback based on clinical communication skills strategies: be-yond the “feedback sandwich.”TeachLearnMed2006; 18:42–47

15. Elliott R, Yudkowsky R, Vogel R: Quality in psychiatrictraining: development of a resident satisfaction questionnaire.Acad Psychiatry 2000; 24:41–46

16. Ellencweig N, Weizman A, Fischel T: Factors determiningsatisfaction in psychiatry training in Israel. Acad Psychiatry2009; 33:169–173

17. Accreditation Council for Graduate Medical Education,Common Program Requirements, July 2011

ERRATUM

In the article “Amethod for evaluating competency in assessment andmanagement of suicide risk,”by Hung EK, Binder RL, Fordwood SR, et al., Acad Psychiatry 2012; 36:23–28, the name of an authorcited in the reference list is spelled incorrectly. Reference 9 is Pisani AR, Cross WF, Gould M: Theassessment and management of suicide risk: state of workshop education. Suicide Life ThreatBehav 2011; 4:255–276.

Academic Psychiatry, 36:4, July-August 2012 http://ap.psychiatryonline.org 287

BROQUET AND PUNWANI