ORIGINAL REPORT

The Effect of Standardized Patient Feedbackin Teaching Surgical Residents InformedConsent: Results of a Pilot Study

Kristine Leeper-Majors, MSE,* James R. Veale, PhD,* Thomas S. Westbrook, PhD,† andKendall Reed, DO‡

*Iowa Pain Management Clinic P.C.; †School of Education, Drake University ; and the‡Des Moines University-Osteopathic Medical Center, Department of Surgery, Des Moines, Iowa

PURPOSE: The purpose of this pilot study was to determinethe effectiveness of using feedback from a standardized patient(SP) to teach a surgical resident (SR) informed consent (IC)protocol.

METHODS: Four general case types of increasing difficultywere tested in a longitudinal experimental design format. Thefour types of cases were appendectomy, cholecystectomy, colo-rectal cancer, and breast cancer. Eight SRs of varying years ofcompletion in medical school served as subjects—four in theexperimental group (received performance feedback from anSP) and four in the control group (received no SP feedback).Both the control and experimental groups participated in twopatient encounters per case type. The first patient encounterserved as the pretest, and the second patient encounter was theposttest. In each encounter, an SP rated the resident on 14measures using an open-ended seven-point rating scale adoptedand modified from the Brown University Interpersonal SkillEvaluation (BUISE). Each resident also reviewed a videotape ofan expert giving IC between pretest and the posttest for basicinstructional protocol. Random stratified sampling was used toequally distribute the residents by postgraduate years. A total of16 SPs were used in this study. All patient/SR encounters werevideotaped.

RESULTS: There was a statistically significant overall change—pretest to posttest and across cases (p � 0.001). The groupeffect was statistically significant (p � 0.000), with the experi-mental group averaging about 10 points greater than the con-trol group.

CONCLUSIONS: Standardized patient feedback is an effec-tive modality in teaching surgical residents informed consentprotocol. This conclusion is tentative, due to the limitations ofsample size. The results of this study support continued re-

search on the effects of standardized patient feedback to teachinformed consent to surgical residents. (Curr Surg 60:615-622. © 2003 by the Association of Program Directors inSurgery.)

KEY WORDS: Informed Consent, Standardized Patients,Surgical Residents, Medical Ethics Education

INTRODUCTION

Teaching surgical residents (SR) how to deliver informed con-sent is a complex behavioral and medical/legal subject. Verylittle in the literature informs the surgical educator what in-structional protocol is best for orienting the resident to this areaof content. Other than the apprenticeship model, little evidenceis found for teaching this subject. The apprenticeship modeldoes not address the issues of standardization (does everyone getthe same experience?), nor does it lead the resident into theprocess without directly involving a real patient.

Informed consent (IC) is a complex process that representsan interaction between a physician and a patient. The processmay be a verbal discussion or a written document that allowspatients to make the best possible judgment regarding the treat-ment of their disease process. This interaction between the phy-sician and patient should satisfy the physician’s moral and legalresponsibility while increasing the patient’s satisfaction with,respect for, and trust in the physician. The desired results canminimize the frequency of malpractice suits and at the sametime enhance physician–patient relationships.

The discipline of surgery is especially vulnerable to medico-legal concerns. Surgical intervention can bring about a varietyof risk factors that range from a simple infection to mortality.This opens the door for potential litigation from an unsatisfac-tory patient result. With the advent of important surgical ad-vances in the treatment of any disease, such as breast cancer,rectal cancer, and the use of endoscopic procedures, it is moreimportant than ever to stress appropriate IC.

How then, do surgical residents (SR) learn about giving pa-

Correspondence: Inquiries to Kristine Leeper-Majors, MSE, Iowa Pain Management ClinicP.C., 1235 Eighth Street West, Des Moines, IA 50256; fax: (515) 327-4782; e-mail:kmajors@ipmc.info

CURRENT SURGERY • © 2003 by the Association of Program Directors in Surgery 0149-7944/03/$30.00Published by Elsevier Inc. doi:10.1016/S0149-7944(03)00157-0

615

tients appropriate IC? There is little in the literature regardingthis question. Few articles are published on the actual teachingof informed consent. Coles et al published a clinical brief on aproposed method for teaching IC to residents but have not yetreported their findings.1 A survey conducted in 1997 by Down-ing et al stated that at least 76% of the general surgery residencyprograms offered little or no formal ethics training that in-cluded the topic of informed consent. Yet it is interesting tonote that 94.3% of the surgery program directors that re-sponded to this survey agreed that it was important to teach ICin the surgical curriculum.2 In 1999, Angelos et al sought todevelop and evaluate a medical ethics curriculum designed spe-cifically for surgical residents. The instructional methods usedto address IC education were case debate, formal presentation,and game simulation.3 Although the confidence levels of theresidents improved significantly from pretest to posttest, it didnot appear that the residents had an opportunity to apply theconceptual framework in the practice setting.

In addition, anecdotal evidence reinforces that surgical resi-dents are most often working in the operating room and surgi-cal floors (inpatient setting), rather than evaluating patients inthe preoperative period in the outpatient setting. “As a result,surgical residents may have little opportunity to witness thedifficult discussions and decisions that are made regarding thecare of a patient, because these discussions are frequently occur-ring in the ambulatory setting.”3 At the very most, residentsserve as apprentices during the informed consent process andtend to model behaviors of the attending surgeons regardingthis issue. Thus, few residents get the chance to visit the patientto address IC issues in a supervised environment. Lack of thisexposure naturally leads to inexperience, which subsequentlymay lead to patient dissatisfaction as the resident progresses ona continuum throughout the resident’s career. Thus, there hasbeen little formal or systematic attention given to this instruc-tion.

The adult teaching and learning process is a “delicate humantransaction.”4 To facilitate learning in the theoretical frame-work of adult learning, residents should be given opportunitiesfor direct experiences in giving informed IC in a safe environ-ment. Such an environment would not only shelter the residentfrom any legal ramifications sought by the patient but alsoprovide the SR with feedback from both the SP and their sur-gical trainers.

The role of experience and learning has been a theme in theadult education literature beginning with Dewey’s seminalwork5 (see also Bateson6 and Usher et al7). Merriam and Caf-farella8 describe how context affects learning and how it canserve as an important method in the teaching/learning transac-tion. They describe three teaching modalities that adult educa-tors utilize to organize learning from experiences. They are re-flective practice, cognitive apprenticeships, and anchoredinstruction. While containing aspects of each of the three mo-dalities, the teaching method employed in this research to in-struct surgical residents informed consent most closely resem-bled anchored instruction. The role of anchored instruction is

to “create situations in which learners, through sustained expe-riences, can grapple with the problems and activities that ex-perts encounter.”8

To date, Des Moines University-Osteopathic Medical Cen-ter (DMU) provides no structured intervention for teaching ICto the residents. Again, residents must rely on the apprentice-ship model to incidentally learn the knowledge and skills nec-essary for performing IC. Although incidental experience mayhave historically been sufficient, it is not satisfactory for thefuture, given the increasing complexity of surgical procedures.Therefore, a structured educational experience is necessary.

The purpose of this pilot study was to determine the effec-tiveness of using feedback from a standardized patient (SP) toteach a surgical resident (SR) informed consent protocol. Thepilot functioned to validate the use of SP encounters as a regularfeature of the DMU residency-training program. From thisexperience, a protocol is being developed that can be used as atemplate for adding SP encounters to the general surgery pro-gram as well as to other programs in the postgraduate track.

The specific aims of the project were to (1) develop a protocolto teach SR informed consent using SPs, (2) produce four vid-eotapes of an expert giving IC to an SP, and (3) test the effect ofSP feedback given to the residents immediately after a SP en-counter. The expert videos served as the formal instructionalpiece for the SRs participating in the project.

METHODS

Four general case types of increasing difficulty were tested in alongitudinal experimental design format. The four types ofcases were appendectomy, cholecystectomy, colorectal cancer,and breast cancer. Eight SRs in postgraduate years two throughfive served as subjects—four in the experimental group and fourin the control group. Random stratified sampling was used todistribute the residents equally over the groups by number ofpostgraduate years. A total of 16 SPs were used in this study.

Both the control and experimental groups participated intwo patient encounters per case type. In each encounter, an SPrated the resident on 14 measures using a feedback form foreach case type (see the Appendix for the breast cancer feedbackform). The rating scale used was adopted and modified from theBrown University Interpersonal Skill Evaluation (BUISE).5 Allresident/patient encounters were videotaped. These videotapesserved as a self-assessment tool for each resident between the SPencounters. The videotapes will be evaluated by surgical ex-perts, and a comparative analysis will be completed and re-ported in a follow-up study.

The experimental group received a feedback debriefing bythe SP between the first and second encounter. In addition, theexperimental group viewed their videotape of the first encoun-ter of each case type with the SP and then completed a self-assessment prior to receiving the SP feedback. The self-assess-ment form was identical to the SP feedback form. The controlgroup viewed the videotape of their first encounter and com-pleted the same self-assessment form but did not receive any

616 CURRENT SURGERY • Volume 60/Number 6 • November/December 2003

feedback from the SP. The use of the self-assessment form witheach group reduced the possibility of an instrument effect. Dif-ferent SPs were used in the two encounters for each case type,and the SPs were blinded as to which residents had received SPfeedback after the initial encounter. Both encounters had thesame general type of health problem (eg, appendectomy), andboth SPs had been trained on the specifics of each case and inthe use of the feedback form. The summed scores from the14-item feedback forms served as the measure of effectivenessused in this study.

The SP feedback form contained 14 questions focusing onthree major performance areas. The areas were as follows:

● Establishing patient rapport● Clinical communication skills and technical considerations● Feedback and closing

Each of the 14 questions were formatted on a seven-pointcontinuum, with the first box of the continuum correspondingto little or no amount of the item exhibited by the SR (scored asone), ranging to the seventh box corresponding to a significantamount of the item exhibited by the SR (scored as seven). An SPfeedback rating form was completed for the first and secondencounter of each case type for both the experimental and con-trol groups. Thus, each SR was rated eight times over the courseof the experiment. The first rating corresponded to the pre-evaluative feedback setting for the experimental group, and thesecond rating corresponded to the post-feedback encounter.The control group also received two SP ratings per case type.However, they did not receive any SP feedback. Using a two-group pretest/posttest design, standardized patient feedbackwas given to 1 group of surgical residents, whereas the othergroup received no feedback. The data consisted of the summedscores on the 14-item assessment instruments used by the SPs toevaluate the SRs on the informed consent process in each case.A repeated measures analysis of variance (ANOVA) was used,including F-tests (univariate and multivariate) for (1) groupeffect, (2) time effect (pretest to posttest and over the variouscases), and (3) group � time interaction. The last of these was ofinterest because the pretest to posttest gain (or loss) could bedifferent for the two groups. These analyses were conducted (1)for each of the four general types of cases (appendectomy, cho-lecystectomy, colorectal cancer, and breast cancer) using pretestand posttest data and (2) for all of the four case types togetherusing (a) pretest and posttest data and (b) posttest data only.The latter analysis was considered appropriate because theinitial pretest difference (for the first case type, the appen-dectomy) was not statistically significant (p � 0.05). (See,eg, Winer10 and Girden11.)

In addition to F-tests, profile plots were produced to graph-ically depict the differences that were found to be statisticallysignificant (p � 0.05). Levene’s test was used to test for homo-geneity of variance, one of the assumptions in repeated mea-sures ANOVA. SPSS for Windows (SPSS, Inc., Chicago, Illi-nois) was used to process and analyze the data.

RESULTS

The results were divided into results (1) by each of the four casetypes using pretest/posttest data, (2) over all four case typesusing pretest/posttest data, and (3) over all four case types usingonly posttest data.

Results by Each Case: Pretest and PosttestData

Appendectomy. There was no significant difference between theexperimental group receiving SP feedback (FB) and the controlgroup receiving no SP feedback (NFB) on the pretest com-pleted by the appendectomy SPs (p � 0.05). This result sup-ports the assumption that the groups had been equalized priorto application of the treatment (SP feedback) by the design,which included randomization.

The FB group gained more from pretest to posttest (p �0.043). The intersecting lines in Fig. 1 indicate this group �time interaction. The feedback group had a somewhat lowermean on the pretest (not significantly lower, however), yet theyachieved a higher posttest mean. On the other hand, bothgroups gained from pretest to posttest, a pattern that was re-peated on the subsequent three cases.Cholecystectomy. The group � time interaction for this case wasmarginally, but not statistically, significant (p � 0.083 � 0.05).The NFB group actually gained somewhat more than the FBgroup from pretest to posttest. As this effect was not statisticallysignificant, we tested for main effects using the F-ratios given inthe ANOVA tables. The FB group had greater overall mean,averaged over pretest and posttest scores (p � 0.012), and bothgroups gained from pretest to posttest (p � 0.015).Colorectal Cancer. There were no significant effects (p � 0.05),except for the one showing a greater variance in the posttestscores for the control group (p � 0.014). This is indicated by

FIGURE 1. Appendectomy mean rating scores showing interaction be-tween group and time (greater gain for FB group).

CURRENT SURGERY • Volume 60/Number 6 • November/December 2003 617

the standard deviations for the groups: 4.50 for the FB groupand 16.19 for the NFB group. Unequal variances (as well asdifferences in means) may be considered evidence of a treat-ment effect (eg, Wolins12). The above result was viewed asfavorable to the FB treatment group, because smaller variabilityis usually considered a desirable trait, indicating greater stabilityand predictability in a process.Breast Cancer. There was a marginally significant overall gainfrom pretest to posttest (p � 0.054) and difference betweengroups (p � 0.065), with the FB group scoring higher. On theposttest, the mean for the FB group was 91.50, whereas that ofthe NFB group was 75.00.

Results for All Four Case Types: Pretest andPosttest Data

There was a statistically significant overall change over time—from pretest to posttest and over cases (p � 0.001). Althoughthere was a gain from pretest to posttest in both groups in allfour case types, this effect was averaged over groups and wasprobably due primarily to the low points in both groups on thepretest for the first breast cancer case and, to a lesser extent, thefirst colorectal cancer case.

The group effect was statistically significant (p � 0.003),with the FB group averaging about 10 points greater overallthan the NFB group. This effect was averaged over the 8 pointsin time (pretest and posttest for the four case types) and isillustrated in Fig. 2.

Another point that stands out in Fig. 2 is the very low valuefor the NFB group on the cholecystectomy pretest. There wasan equally low value for the NFB group mean on the breastcancer pretest. (The FB group also had a low value on the breastcancer pretest assessment, but not on the cholecystectomy pre-test.) We can offer no definitive reasons for these “dips” in thegraphs as illustrated in Fig. 2 other than the possibility thatsome of the surgical residents in the NFB group were not get-

ting sufficient positive reinforcement for their efforts or weresimply having “bad days” (problems that can occur with anytype of assessment).

Results for All Four Cases: Posttest DataOnly

Another way to analyze the data is to focus on the posttestscores, ignoring the pretest. This is acceptable in any study inwhich randomization was used to equalize the groups. (Recallthat there was no significant difference between the 2 groups onthe initial pretest, given in connection with the appendectomycase.) Using just the posttest data, there was a statistically sig-nificant difference between the groups, with the FB group scor-ing higher on the average (p � 0.001). This difference is illus-trated in Fig. 3. The lines in Fig. 3 appear to diverge somewhat,with the FB group increasing and the NFB decreasing over thefour cases. This effect is, however, only marginally significant (p� 0.087) using the multivariate analysis approach. This effect isnot even marginally significant using the univariate approach (p� 0.10). The difference in these two results could be, in part,due to differences in power and robustness. For example, thesignificant differences in the colorectal cancer posttest scorevariances could have impacted these results.

DISCUSSION

This was a small pilot study consisting of eight surgical residents(four in each group). The standardized patients were mixed andmatched with the SR to provide the assessments (both groups)and feedback (treatment group). The purpose of the study wasto assess the effectiveness of using SP feedback in teaching theIC protocol to surgical residents. The conclusions are tentative,due to the limitations of sample size. There was a statisticallysignificant group difference over all cases, with the feedbackgroup performing better. This was true whether both pretest

FIGURE 2. Plot of mean rating scores for all 4 cases (pretest and posttestdata).

FIGURE 3. Plot of mean rating scores for all 4 cases (posttest onlydata).

618 CURRENT SURGERY • Volume 60/Number 6 • November/December 2003

and posttest data or just the posttest data were used. The feed-back group gained more than the non-feedback group on theappendectomy case, from pretest to posttest. In the cholecys-tectomy case, the feedback group did better overall (pretest andposttest) and both groups gained from pretest to posttest. In thecolorectal cancer case, there was smaller variability amongscores for the feedback group, indicative of greater stability inthe performance measures for that group. In the breast cancercase, there was a marginally significant difference betweengroups (feedback group again performed better) and a margin-ally significant overall gain from pretest to posttest.

The findings suggest additional longitudinal studies, with alarger sample of surgical residents, and assessments of the vid-eotaped encounters by expert raters (eg, surgeons). In addition,SP feedback for teaching informed consent in medical practiceother than surgery could be similarly assessed.

REFERENCES

1. Coles WH, Wear SE, Bono JJ, Peters AS, Lenkei EJ.Teaching the informed consent process to residents. SouthMed J. 1989;1:64-66.

2. Downing MT, Way DE, Caniano DA. Results of a na-tional survey on ethics education in general surgery resi-dency programs. Am J Surg. 1997;174:364-368.

3. Angelos P, DaRosa DA, Derossis AM, Kim B. Medical

ethics curriculum for surgical residents: results of a pilotproject. Surgery. 1999;126:701-705.

4. Whitman N, Lawrence P. Surgical Teaching: PracticeMakes Perfect. Salt Lake City, UT: University of UtahSchool of Medicine; 1991.

5. Dewey J. Experience and Education. New York: CollierBooks; 1938.

6. Bateson MC. Peripheral Visions: Learning Along the Way.New York: Harper Collins; 1994.

7. Usher R, Bryant I, Johnson R. Adult Education and thePostmodern Challenge: Learning Beyond the Limits. NewYork: Routledge; 1994.

8. Merriam SB, Cafferella RS. Learning in Adulthood: A Com-prehensive Guide, 2nd ed; San Francisco, CA: Jossey-Bass;1999.

9. Burchard KW, Rowland-Morin PA. A new method of as-sessing the interpersonal skills of surgeons. Acad Med.1990;65:274-276.

10. Winer B. Statistical Principles in Experimental Design, 2nded; New York: McGraw-Hill; 1972.

11. Girden E. ANOVA: Repeated Measures. Thousand Oaks,CA: Sage; 1992.

12. Wolins L. Research Mistakes in the Social and BehavioralSciences. Ames, IA: Iowa State University Press; 1982.

See Appendix

CURRENT SURGERY • Volume 60/Number 6 • November/December 2003 619

APPENDIX

Standardized Patient Feedback Form for Breast Cancer Case

620 CURRENT SURGERY • Volume 60/Number 6 • November/December 2003

CURRENT SURGERY • Volume 60/Number 6 • November/December 2003 621

622 CURRENT SURGERY • Volume 60/Number 6 • November/December 2003