7
Does a 3-Day Workshop for Family Medicine Trainees Improve Preventive Care? A Randomized Control Trial JEANETTE WARD, MBBS, MHPED,PHD, FAFPHM,* AND ROB SANSON-FISHER, BPSYCH, MPSYCH,PH*School of Medical Education, University of New South Wales, New South Wales 2052; and ²Hunter Centre for Health Advancement, Wallsend, New South Wales, 2287, Australia Objective. To evaluate the impact on clinical behav- ior of a 3-day workshop designed to increase trainees’ rates of smoking cessation counseling and reminders about Pap smears in routine consultations. Design. Randomized control trial. Setting. Accredited teaching practices of the Royal Australian College of General Practitioners’ Training Program. Subjects. Thirty-four trainees and 1,500 consecutive adult patients ages 16–65 years. Method. Trainees randomly allocated to the experi- mental group participated in a 3-day interactive work- shop on disease prevention during their 13-week fam- ily medicine term. Audiotapes of consultations with adults conducted by trainees at the beginning and end of the rotation were analyzed blind to compare assess- ment of patients’ smoking status and, for women, date of last Pap smear. A questionnaire mailed to each pa- tient after the consultation also allowed identification of smokers and women overdue for a smear. Consulta- tions with these patients at risk were analyzed for pre- ventive counseling. Inter- and intrarater reliability was calculated for audiotape analysis. Results. Preworkshop rates of questions about smoking were low, occurring in 22% of consultations. While trainees allocated to the experimental work- shop were more likely to ask a routine question about smoking at the end of the term than those in the con- trol group (P = 0.01), two-thirds of smokers remained undetected irrespective of trainee group and fewer than one in five were advised to stop smoking. Re- minders about Pap smears did not change as a result of training and remained low in fewer than 20% of con- sultations. k values demonstrated high reliability of audiotape analysis. Conclusion. This direct measurement of clinical be- havior revealed that low levels of preventive care pro- vided by trainees are resistant to skills training with- out reinforcement in clinical practice. In view of the importance of prevention in routine consultations, we recommend continued evaluation of more intensive educational programs. Those withstanding rigorous evaluation could be considered for implementation in similar training contexts seeking to improve the fre- quency and quality of disease prevention in primary medical care. © 1996 Academic Press, Inc. Key Words: family medicine; disease prevention; cer- vical smears; smoking cessation advice; randomized controlled trial. INTRODUCTION Preventive care in family medicine has considerable potential to reduce morbidity and premature mortality [1]. Family physicians themselves and the community they serve perceive preventive care to be an appropri- ate part of their clinical role [2–4]. An opportunistic approach within the context of a routine consultation is likely to ensure the provision of preventive care to those most in need [5,6]. Irrespective of the presenting problem, the opportunity can be taken to identify pa- tients at risk and respond with appropriate advice or screening. When applied unsystematically, however, an opportunistic approach is no better than an ad hoc one [7]. There is evidence that disease prevention remains unsystematic in everyday practice, however. For ex- ample, general practitioners do not identify and coun- sel all smokers in their practice [8,9], despite convinc- ing evidence that smoking cessation advice routinely offered is effective and acceptable [10]. Further, not all women at risk for cervical cancer are screened by Pa- panicolaou’s (Pap) smear [11]. In contrast, a brief re- minder in routine consultations to women overdue for a smear is acceptable and effective in encouraging screening [12,13]. Although reasons for the suboptimal provision of clinical preventive services are multifacto- rial and poorly understood, inadequate skills training at both undergraduate and postgraduate levels likely contributes [14]. Attempts at skills training have had mixed effects, This study was funded by grants from the NSW Cancer Council and the RACGP Training Program. Address reprint requests to Jeanette Ward at the Needs Assessment & Health Outcomes Unit, Central Sydney Area Health Service, P.O. Box 374, Camperdown 2050, Australia. Fax: Int 61 2 9713 9980. PREVENTIVE MEDICINE 25, 741–747 (1996) ARTICLE NO. 0114 741 0091-7435/96 $18.00 Copyright © 1996 by Academic Press, Inc. All rights of reproduction in any form reserved.

Does a 3-Day Workshop for Family Medicine Trainees Improve Preventive Care? A Randomized Control Trial

Embed Size (px)

Citation preview

Does a 3-Day Workshop for Family Medicine Trainees ImprovePreventive Care? A Randomized Control Trial

JEANETTE WARD, MBBS, MHPED, PHD, FAFPHM,* AND ROB SANSON-FISHER, BPSYCH, MPSYCH, PHD†

*School of Medical Education, University of New South Wales, New South Wales 2052; and †Hunter Centre for Health Advancement,Wallsend, New South Wales, 2287, Australia

Objective. To evaluate the impact on clinical behav-ior of a 3-day workshop designed to increase trainees’rates of smoking cessation counseling and remindersabout Pap smears in routine consultations.Design. Randomized control trial.Setting. Accredited teaching practices of the Royal

Australian College of General Practitioners’ TrainingProgram.Subjects. Thirty-four trainees and 1,500 consecutive

adult patients ages 16–65 years.Method. Trainees randomly allocated to the experi-

mental group participated in a 3-day interactive work-shop on disease prevention during their 13-week fam-ily medicine term. Audiotapes of consultations withadults conducted by trainees at the beginning and endof the rotation were analyzed blind to compare assess-ment of patients’ smoking status and, for women, dateof last Pap smear. A questionnaire mailed to each pa-tient after the consultation also allowed identificationof smokers and women overdue for a smear. Consulta-tions with these patients at risk were analyzed for pre-ventive counseling. Inter- and intrarater reliabilitywas calculated for audiotape analysis.Results. Preworkshop rates of questions about

smoking were low, occurring in 22% of consultations.While trainees allocated to the experimental work-shop were more likely to ask a routine question aboutsmoking at the end of the term than those in the con-trol group (P = 0.01), two-thirds of smokers remainedundetected irrespective of trainee group and fewerthan one in five were advised to stop smoking. Re-minders about Pap smears did not change as a resultof training and remained low in fewer than 20% of con-sultations. k values demonstrated high reliability ofaudiotape analysis.Conclusion. This direct measurement of clinical be-

havior revealed that low levels of preventive care pro-vided by trainees are resistant to skills training with-

out reinforcement in clinical practice. In view of theimportance of prevention in routine consultations, werecommend continued evaluation of more intensiveeducational programs. Those withstanding rigorousevaluation could be considered for implementation insimilar training contexts seeking to improve the fre-quency and quality of disease prevention in primarymedical care. © 1996 Academic Press, Inc.

Key Words: family medicine; disease prevention; cer-vical smears; smoking cessation advice; randomizedcontrolled trial.

INTRODUCTION

Preventive care in family medicine has considerablepotential to reduce morbidity and premature mortality[1]. Family physicians themselves and the communitythey serve perceive preventive care to be an appropri-ate part of their clinical role [2–4]. An opportunisticapproach within the context of a routine consultation islikely to ensure the provision of preventive care tothose most in need [5,6]. Irrespective of the presentingproblem, the opportunity can be taken to identify pa-tients at risk and respond with appropriate advice orscreening. When applied unsystematically, however,an opportunistic approach is no better than an ad hocone [7].There is evidence that disease prevention remains

unsystematic in everyday practice, however. For ex-ample, general practitioners do not identify and coun-sel all smokers in their practice [8,9], despite convinc-ing evidence that smoking cessation advice routinelyoffered is effective and acceptable [10]. Further, not allwomen at risk for cervical cancer are screened by Pa-panicolaou’s (Pap) smear [11]. In contrast, a brief re-minder in routine consultations to women overdue fora smear is acceptable and effective in encouragingscreening [12,13]. Although reasons for the suboptimalprovision of clinical preventive services are multifacto-rial and poorly understood, inadequate skills trainingat both undergraduate and postgraduate levels likelycontributes [14].Attempts at skills training have had mixed effects,

This study was funded by grants from the NSW Cancer Counciland the RACGP Training Program. Address reprint requests toJeanette Ward at the Needs Assessment & Health Outcomes Unit,Central Sydney Area Health Service, P.O. Box 374, Camperdown2050, Australia. Fax: Int 61 2 9713 9980.

PREVENTIVE MEDICINE 25, 741–747 (1996)ARTICLE NO. 0114

741

0091-7435/96 $18.00Copyright © 1996 by Academic Press, Inc.

All rights of reproduction in any form reserved.

however. For example, while family medicine resi-dents’ skills in smoking cessation counseling can im-prove significantly immediately after a workshop [15],these skills deteriorate over 12 months [16]. Healthmaintenance guidelines are ineffective [17] althoughan intensive program comprising handbooks, confer-ences, chart audits, and videotape review of consulta-tions with patients was found to increase smoking ces-sation advice [18]. Although tutorials, prompts alone,or a combination of the two also will increase residents’smoking cessation advice compared with a controlgroup [19], none has been found to be differentiallymore effective than another. Teaching sessions with orwithout prompts have been shown to have no effect inincreasing smoking cessation advice [20].Similarly, promising preliminary results from un-

controlled studies using health maintenance ‘‘grids’’[21], reminders on medical records [22], and a compre-hensive teaching and self-audit program [23] to en-courage Pap smear screening by family medicine resi-dents have not been sustained in more rigorous evalu-ations. Specifically, six randomized controlled studiesevaluating strategies as diverse as book readings [24],noncomputerized age- and sex-specific checklists at-tached to the medial record [25–28], record audit [26],and computer-generated individualized cancer screen-ing reminders attached to the medical record [29,30]failed to increase Pap smear screening in postgraduatetraining settings.In Australia, postgraduate training is provided by

the Royal Australian College of General Practitioners(RACGP) Training Program [31]. Within its 3-year pro-gram, the RACGP Training Program provides at leasttwo 13-week ‘‘terms’’ in private community-based fam-ily medicine practices. Trainees see their own patientsbut have access to informal supervision and advicefrom at least one senior general practitioner known asthe GP Supervisor. The first of these terms is offered inthe 2nd year after 1 year of hospital-based trainingafter internship.As the quality of on-site supervision provided by GP

Supervisors varies, compulsory 3-day workshops areorganized once in each term by training program staff.Trainees leave their practices and travel to the capitalcity of New South Wales for the workshop, returningafterward to their term. However, rigorous evaluationof these educational activities has never been under-taken [32,33]. In response, this study was conducted todetermine the impact on preventive care of a 3-dayworkshop in disease prevention, using a randomizedcontrol design and direct assessment of clinical behav-ior by audiotaping.

METHOD

Trainee Recruitment

Full-time trainees undertaking their first term wererecruited for the study. Once trainees were allocated to

their terms by administrative staff, a random numbertable was used to allocate them to workshops eitherabout disease prevention (experimental group) or ra-tional prescribing (control group). Trainees were notallowed to change their workshop dates. The GP Su-pervisor of each practice was contacted to obtain con-sent for that trainee to be approached about the study.If agreed, trainees were asked 2 weeks before the termto participate in a study of quality of care involvingaudiotapes and patient questionnaires although theywere not informed of the specific study objectives.Trainees were unaware of their random allocation toeither of the two workshops and were unlikely to con-nect the workshop directly to the evaluation.

Audiotaping

Given demonstrated shortcomings in medical recordreview and physician self-report as a measure of pre-ventive care [18,34,35], consultations with real pa-tients were audiotaped and analyzed using a 13-itemrating scale available from the first author. As medicalrecords in Australia are poorly organized [36], traineeswould not be able to identify patients at risk other thanby asking, especially as patients were unknown tothem. A question about smoking would be heard ifasked routinely and patients identified as smokerswould be heard to be offered advice to quit if provided.Consultations with women other than those alreadyattending for a Pap smear represented an opportunityfor a discussion about Pap smears to be initiated and,more specifically, for the date of the last Pap smear tobe ascertained for women at risk [13]. Those found tobe at risk and overdue for a smear could be reminded tohave the test.Four coders used the rating scale to code consulta-

tions. A system of code numbers was devised to ensurethat coders were blinded to group allocation andwhether the consultation occurred at the beginning orend of the term.

Patient Recruitment for Audiotaping

Consecutive adult patients ages 16–65 years wererecruited for the study. Patients were ineligible for thestudy if they had insufficient command of English togive informed consent or were too sick to read and com-plete a written questionnaire before the consultation.Receptionists gave a consent form to eligible patients

which explained study requirements and requested thepatient’s written consent for audiotaping. Patients whoconsented to audiotaping also were asked to indicatetheir age and, if female, whether they were attendingfor a Pap smear. They were asked to provide theirname and address in order to be sent a follow-up ques-tionnaire after the consultation.Receptionists completed a tally sheet to record the

results of recruitment. Patients gave their completedform to the trainee at the beginning of the consultation,

WARD AND SANSON-FISHER742

thereby indicating that the supplied tape recordershould be switched on. Each consent form had a uniquecode number which the trainee copied onto the audio-cassette at the end of the consultation. Audiotapingfinished once 40 patients had agreed to participate orwhen 10 days of data collection had been completed,whichever was the sooner. An identical procedure wasfollowed for 10 days at the end of the term.

Patient Follow-Up Questionnaire to Identify Patientsat Risk

Questions about smoking or date of last Pap smearon the waiting room questionnaire would have alertedtrainees and patients to the preventive focus of thestudy. Yet a system was needed to identify the subsetof consultations with patients at risk to measure pre-ventive counseling.To do this, we mailed a self-administered question-

naire to patients after the consultation. Respondentswere asked if they were a smoker at the time of theconsultation. To assist recall, the name of the trainee,day, and date of the relevant consultation were hand-written on each questionnaire at the beginning of thissection. Smokers were defined as cigarette, pipe, or ci-gar smokers, irrespective of amount smoked. In a sec-tion which appeared in the female version only, therespondent was asked to indicate if she ever had hadsexual intercourse with a male partner and if she hadhad a hysterectomy. She was asked to think back to thetime of her consultation with the trainee and indicatethe time lapse since her last Pap smear. Women weredefined as ‘‘at risk’’ for cervical cancer if they ever hadhad sexual intercourse with a male partner and hadnot had a hysterectomy. Women at risk were consid-ered ‘‘unscreened’’ or ‘‘overdue’’ for a smear if, on self-report, they had not had a Pap smear within 3 years ofthe consultation and had not attended for one on theday the consultation was recorded.If a patient questionnaire was not received within 8

days of initial posting, a reminder call was made and asecond questionnaire mailed.

Workshop Details

Trainees assigned to the experimental group partici-pated in a 3-day workshop during Week 5 of the termwhich included didactic sessions about preventive carein general practice as well as small group sessions topractice skills. General techniques to encourage pa-tient behavior change were outlined, followed by a ses-sion about the epidemiology of smoking-related can-cers, scientific basis for smoking cessation, and inter-actional skills known to be effective in smokingcessation. Case vignettes were devised for trainees topractice a routine question about smoking status irre-spective of presenting problem and ways to facilitatecessation among those ready to quit. These role-playswere videotaped and debriefed by medically qualified

staff from the training program, using Pendleton’srules [37]. The session about Pap smears commencedwith the epidemiology of cervical cancer, currentscreening levels in general practice, barriers to screen-ing, and interactional skills known to increase oppor-tunistic screening and reduce women’s embarrassmentwith the procedure. Trainees divided into small groupsfor video role-play to practice opportunistic questionsabout Pap smears, again using case vignettes. Furtherdetails about the workshops are available from the firstauthor.The workshop was conducted in each of five consecu-

tive 3-month terms. Each large group presentation wasvideotaped to ensure standardized didactic content.Trainees allocated to the control group attended a

3-day workshop in Week 4 of the term which empha-sized rational prescribing. No sessions on preventivecare were included in this workshop.

Statistical Analysis

Sample sizes were calculated to detect a 20% differ-ence between experimental and control groups in ratesof questions about risk and the provision of an inter-vention to those at risk with 80% power and a 5% sig-nificance level for a two-tailed test.Using SAS statistical software [38], age and sex for

consenting and nonconsenting patients were comparedusing t tests and x2 for continuous and categorical vari-ables, respectively. Differences in trainee behavior be-tween experimental and control groups at pre- andpostworkshop were compared by using analysis of co-variance with group as a factor and screening rate pertrainee at preworkshop as a covariate. A t test on theadjusted means was used to assess the magnitude ofthe difference. Where patient numbers per traineewere insufficient to allow the trainee to be the unit ofanalysis, continuity-adjusted x2 was used to compareproportions of patient samples pre- and postworkshop.

Reliability of Audiotaping

To determine interrater reliability, one coder re-coded 12% of audiotapes coded by the three other cod-ers, blind to their ratings. Each coder also recoded aminimum 15% sample of their own audiotapes to de-termine intrarater reliability. Reliability was calcu-lated by k statistic [39].

RESULTS

Trainee Recruitment

During the study period, 68 trainees were allocatedto family medicine terms. For these, 51 GP Supervisorsagreed to the research being conducted in their prac-tice (75% consent rate). The most common reason givenfor nonparticipation was that the practice was too busyto comply with data collection requirements. Of the 51trainees approached about the study, 41 (80%) agreed

DOES A WORKSHOP IMPROVE PREVENTIVE CARE? 743

to participate. Of these 41 consenting trainees, 1 sub-sequently failed to participate in data collection due topersonal illness, 1 transferred to another training pro-gram and 1 was dismissed, 1 trainee declined to par-ticipate in data collection at the end of the term, and 2trainees were in practices in which the senior partnersdecided against their continuing. One trainee at-tempted data collection unsuccessfully in a practicewhere the majority of patients could not speak suffi-cient levels of English to give informed consent andmost consultations were conducted in Italian. Data col-lected by these trainees at the beginning of their at-tachments were discarded.In total, 34 trainees agreed to, and completed, all

data collection requirements. Of these, 16 (11 males, 5females) constituted the experimental group and 18 (8males, 10 females) constituted the control group.Trainees’ ages ranged from 25 to 31 years (median 27years; mode 26, also 28 years). All trainees attendedtheir workshop as allocated.

Patient Recruitment for Audiotaping

Of the 2,179 eligible adult patients seen during au-diotaping periods, 108 were not asked. Of the remain-ing 2,071 adults asked to participate, 1,500 consented(72% consent rate). Consenting patients did not differfrom nonconsenting patients on either age (z 4 0.25, P> 0.1) or sex (z 4 0.78, P > 0.1).For those, 1,500 consenting adults, 1,362 matched

audiotapes were available for analysis: 808 (59%) withfemale patients and 554 (41%) with male patients. Thenumber of available audiotaped consultations pertrainee ranged from 4 to 30.

Trainees’ Use of Opportunistic Questionsabout Smoking

Before training, the proportion of consultations inwhich individual trainees were heard to ask aboutsmoking status ranged from 0 to 71% (mean 22%[Table 1]). Only 3 of 34 trainees asked about smokingstatus in more than 50% of their consultations withadults. Two of these (both female) were in the experi-mental group. The third, a male trainee, was in thecontrol group. Postworkshop, all trainees in the experi-mental group were heard in at least one of their con-sultations to ask about smoking status. After adjustingfor individual differences in preworkshop rates, theproportion of consultations in which a question aboutsmoking by trainees was heard in the experimentalgroup before and after the workshop increased by 7.6%(95% CI 0.4–14.8). This difference was significant (t 42.0, P 4 0.047). However, less than one-third of con-sultations at the end of the term conducted by traineesin the experimental group included a question aboutsmoking status. Trainees allocated to the control groupdecreased their provision of opportunistic questions

about smoking by 5.8% (95% CI −12.5, 0.9). This de-crease was not significant (t 4 1.41, P > 0.10). Thepostworkshop difference of 13.3% (95% CI 3.1–23.5) be-tween trainees in experimental and control groups wassignificant (t 4 2.7, p 4 0.01).

Trainees’ Use of Any Opportunistic Discussion aboutPap Smears

Table 2 summarizes rates for any discussion aboutPap smears in audiotaped consultations after exclud-ing those with women already presenting for a Papsmear. After training, trainees in the experimentalgroup increased routine discussion of Pap smears by1.8% (95% CI −4.7, 8.3) (t 4 1.11, P > 0.2). Trainees inthe control group decreased by 3.2% (95% CI −9.2, 2.9)(t 4 0.9, P > 0.3). The postworkshop difference betweenthe two groups of 5% (95% CI −4.0, 14.0) was not sig-nificant (t 4 1.1, P > 0.2).A specific question from the trainee about the date of

the last Pap smear occurred less frequently (Table 2).From pre- to postworkshop, a specific question by

TABLE 1Opportunistic Questions about Smoking

ExperimentalgroupN 4 16trainees

ControlgroupN 4 18trainees

Consultations with adults 16–65 yearsN audiotaped consultations

Pre 314 346Post 325 377

N (%) consultations in which traineeasked about smoking statusPre 81 (26%) 66 (19%)Post 97 (30%)* 57 (15%)**

Consultations with self-reported smokersN audiotaped consultations

Pre 80 100Post 124 147

N (%) consultations in which traineeAsked about smoking statusPre 22 (28%) 20 (20%)Post 35 (28%) 31 (21%)

Advised to stop smokingPre 15 (19%) 4 (4%)Post 17 (14%) 6 (4%)

Advised of specific health risksPre 14 (18%) 4 (4%)Post 10 (8%) 7 (5%)

Set a ‘‘quit’’ datePre 1 (1%) 1 (1%)Post 2 (2%) 0 (0%)

Made a follow-up appointmentPre 1 (1%) 1 (1%)Post 3 (2%) 0 (0%)

Gave hints about quittingPre 2 (2%) 3 (3%)Post 5 (4%) 4 (3%)

Offered written informationPre 0 (0%) 0 (0%)Post 2 (2%) 0 (0%)

* Pre compared to post P 4 0.047.** Experimental compared to control P 4 0.01.

WARD AND SANSON-FISHER744

trainees in the experimental group about date of lastPap smear increased by 0.2% (95% CI −5.8, 6.3), a non-significant increase (t 4 0.2, P > 0.9). Trainees in thecontrol group decreased by 4.7% (95% CI −10.4, 1.0),again a nonsignificant decrease (t 4 1.3, P > 0.1). Thepostworkshop difference between the two groups of4.9% (95% CI −3.3, 13.1) was not significant (t 4 1.1, P> 0.2). This sample size was sufficient to detect a 20%difference in rates with a power in excess of 95%, sug-gesting a Type 2 error is unlikely.

Patient Follow-Up Questionnaire

In total, 1,491 (890 females, 601 males) of 1,500 con-senting adults provided sufficient details on the wait-ing room questionnaire to be mailed a follow-up ques-tionnaire. Of these, 16 were returned to sender. Fromtelephone follow-up, a further 6 were considered ineli-gible due to severe illness (n 4 3) or prolonged absencefrom the home address (n 4 3). Of 1,469 who couldhave completed the questionnaire, 1,209 (750 females,459 males) did so (82% response rate). Those who re-turned questionnaires were significantly older (z 44.8, P < 0.05) and more likely to be female (z 4 2.12, P< 0.05) than those who did not. Fifty percent of respon-dents completed their questionnaire within 8 days ofthe consultation (range 1–63 days after the consulta-tion, mode 6 days).

Trainees’ Preventive Behavior with Smokers

Matching audiotapes were available for 451 patientswho were smokers. As the number of smokers pertrainee ranged from 0 to 22, it was not possible to ana-lyze these data with the trainee as the unit of analysis.As shown in Table 1, the proportion of consultationswith smokers in which a routine question about smok-ing was heard to occur did not change significantly pre-to postworkshop for trainees in the experimental group(x2 4 0.2, df 4 1, P > 0.7). The postworkshop difference

in the proportion of smokers asked about smoking sta-tus by trainees in experimental and control groups alsowas not significant (x2 4 2.1, df 4 1, P > 0.1). Thissample size had a power of 90% to detect a 20% differ-ence between control and experimental groups at post-test, again suggesting a Type 2 error is unlikely.Smoking cessation strategies used by trainees by

group pre- and postworkshop did not change (Table 1).Irrespective of group, trainees did not use effectivestrategies to encourage smoking cessation. For ex-ample, quit dates were set in less than 2% of consulta-tions with smokers.

Trainees’ Preventive Behavior with Women Overduefor a Pap Smear Although Not Presenting for One

Of 750 women in the sample, 209 were not at risk forcervical cancer (97 had never been sexually active, 95had had a hysterectomy, and 17 indicated both ofthese). Of the remaining 541 women at risk, only 72women (15% of those at risk) indicated they were over-due for a Pap smear at the time of the consultation.Audiotapes were available for only 63, precludingmeaningful analysis.

Inter- and Intrarater Reliability

The k values for interrater reliability exceeded 0.8for eight items on the rating scale (including the ques-tion ‘‘did the trainee ask about smoking?’’) and 0.7 fora further four (including ‘‘did the trainee advise thepatient to stop smoking?’’). The k values for intraraterreliability exceeded 0.8 for all items for all coders ex-cept for one item for coder 2 (‘‘did the trainee advise thepatient of specific health risks due to smoking?’’ k 40.49).

DISCUSSION

To our knowledge, this is the first randomized con-trolled trial of a preventive care workshop for familymedicine trainees to use blinded audiotape analysis oftrainees’ consultations with real patients before andafter participation. More than 90% of audiotapes withconsenting adults were available for analysis. The kvalues demonstrated excellent reliability. Whereasless rigorous methods have suggested significant im-provements in preventive care can be achieved by suchworkshops, these results are disappointing and chal-lenging. Issues worthy of specific comment follow.First, the study demonstrates the paucity of preven-

tive care delivered in routine consultations by traineesat the beginning of their first experience of generalpractice. Less than one-quarter of consultations pre-workshop included a routine question about smoking.Despite accumulating evidence over many years thatundergraduate education and internship remain inad-equate in preparing doctors to provide effective preven-

TABLE 2Opportunistic Questions about Cervical Smears in

Consultations with Women 16–65 Years Not Presentingfor One

ExperimentalgroupN 4 16trainees

ControlgroupN 4 18trainees

N audiotaped consultationsPre 170 188Post 180 225

N (%) consultations in which traineeinitiated any discussion re smearPre 29 (17%) 33 (18%)Post 34 (19%) 33 (15%)

N (%) consultations in which traineeasked date of last smearPre 21 (12%) 30 (16%)Post 25 (14%) 23 (10%)

DOES A WORKSHOP IMPROVE PREVENTIVE CARE? 745

tive care [40,41], little appears to have been achievedsubsequently to change this deficiency.Second, participation in a training workshop which

involved information as well as role-plays and video-debriefing did not guarantee a substantial change inbehavior. Opportunities for preventive care remainedunderutilized. While statistically significant, an in-crease of 8% in an opportunistic question about smok-ing status by trainees in the experimental group pre- topostworkshop was of modest educational significance.Similarly, the 13% difference at postworkshop betweengroups resulted in part from an inexplicable decreasein opportunistic questions by trainees in the controlgroup. The provision of smoking cessation advice tosmokers and the opportunistic assessment of womenoverdue for a Pap smear did not improve. The likeli-hood of Type 2 errors to explain these nonsignificantfindings is low. We suggest that these findings illus-trate the resilience of clinical behavior to current post-graduate training in Australia. Workshop approachesappear to be less effective in changing trainees’ clinicalbehavior than educators might hope or assume [20,28].Further, sustainability of even this modest gain be-yond the first term remains to be ascertained.Thus, more effective strategies are required for ap-

plication in postgraduate training settings. Traineesshould be consulted to ascertain the nature of persist-ing barriers to optimal preventive care. Certainly, ascurrently organized, the training program does not re-quire practice-based interventions to reinforce oppor-tunistic disease prevention upon return to community-based training practices. Reinforcement and feedbackare critical if practical skills acquired in workshops areto be transferred successfully to the clinical setting[42]. Self-audit and record prompts to augment work-shop strategies [18] also show promise. As others havesuggested [22], system issues such as trainee workloadalso need to be addressed. Differences between train-ing practices, on-site tuition, and GP Supervisors alsoshould be measured in future studies.There are three potential methodological weak-

nesses of this study deserving of specific comment.First, the Hawthorne effect may have altered clinicalbehavior during audiotaping although this phenom-enon is unlikely to invalidate the nonsignificant find-ings. If trainees were aware that their preventive be-havior was under scrutiny, then their performance isall the more disappointing. Second, while the same ex-perts participated and were videotaped in each work-shop to enhance standardization of didactic content,there was no comparable technique to ensure consis-tent educational messages during small group role-plays and video-debriefing. It is possible although un-likely that the interventions described by resource per-sons were not practiced or debriefed sufficiently insmall groups to be acquired by trainees. Third, lessthan three-quarters of eligible trainees were ap-

proached about the study because of a low trainer con-sent rate. Although subsequent trainee and patientconsent rates for audiotaping were high, systematic bi-ases could limit generalizability. Nonetheless, ourmethod involving audiotaping of routine consultationswas acceptable to many and could be considered byothers involved in educational evaluation.In conclusion, postgraduate training for family medi-

cine represents a important opportunity for remediat-ing deficiencies in undergraduate education and in-ternship in disease prevention. This study demon-strates that trainees’ provision of opportunisticsmoking cessation advice and Pap smears is subopti-mal yet trainees who participated in a 3-day preventiveworkshop gained only a modest improvement in askinga routine question about smoking status. Despite in-tensive training which involved videotaped role-playsand debriefing, trainees failed to apply effective smok-ing cessation techniques even when smokers wereidentified more frequently. Training failed to equiptrainees with interactional skills to introduce a discus-sion about Pap smears in routine consultations. To im-prove the effectiveness of postgraduate training in pre-ventive care in Australia, an educational programwhich is augmented by reinforcement, self-audit, andmedical record prompts upon return to training prac-tices is recommended. Rigorous evaluation using be-havioral outcomes will be essential. Once an effectiveprogram is demonstrated experimentally, it could bereplicated in comparable training programs in NorthAmerica, Europe, and the United Kingdom.

ACKNOWLEDGMENTS

The participation of trainees, their patients, and practice recep-tionists is acknowledged gratefully. We thank Mrs. Nancy Hardingfor diligent research assistance, Mr. Stephen Halpin for statisticalanalysis, and Dr. Jill Gordon, former State Director of the RACGPTraining Program (NSW), for organizational support and personalencouragement.

REFERENCES

1. Sanson-Fisher RW, Webb G, Reid ALA. The role of the medicalpractitioner as an agent for disease prevention. In: Reports tothe Better Health Commission: looking forward to better health.Vol 3. Canberra: AGPS, 1986:201–12.

2. Cockburn J, Killer D, Campbell E, Sanson-Fisher RW. Measur-ing general practitioners’ attitudes towards medical care. FamPract 1987;4:192–9.

3. Slama K, Redman S, Cockburn J, Sanson-Fisher RW. Commu-nity views regarding the role of general practitioners in diseaseprevention. Fam Pract 1989;6:203–9.

4. Bauman A, Mant A, Middleton L, Mackertich M, Jane E. Dogeneral practitioners promote health? A needs assessment. MedJ Aust 1989;151:262–9.

5. Stott NCH, Davis R. The exceptional potential in each primarycare consultation. J R Coll Gen Pract 1979;29:201–5.

6. Yankauer A. Public and private prevention. Am J Public Health1983;73:1032–4.

WARD AND SANSON-FISHER746

7. Stott N. Screening methods in relation to preventive care. BrMed J 1985;291:1277.

8. Dickinson J, Wiggers J, Leeder S, Sanson-Fisher RW. Generalpractitioners’ detection of patients’ smoking status. Med J Aust1989;150:420–6.

9. Heywood A, Sanson-Fisher RW, Ring I, Mudge P. Risk preva-lence and screening for cancer by general practitioners. PrevMed 1994;23:152–9.

10. Richmond R, Heather N. General practitioner interventions forsmoking cessation: past results and future prospects. BehavChange 1990;7:110–9.

11. Dickinson J, Leeder S, Sanson-Fisher RW. Frequency of cervicalsmears–tests among patients of general practitioners. Med JAust 1988;148:128–31.

12. Burack R, Liang J. The early detection of cancer in the primarycare setting: factors associated with the acceptance and comple-tion of recommended procedures. Prev Med 1987;16:739–51.

13. Ward J, Boyle K, Redman S, Sanson-Fisher RW. Increasing wo-mens’ compliance with opportunistic cervical cancer screening ingeneral practice: a randomised control trial. Am J Prev Med1991;7:285–91.

14. Ward J, Gordon J, Sanson-Fisher RW. Strategies to increasepreventive care in general practice. Med J Aust 1991;154:523–31.

15. Ockene J, Quirk M, Goldberg R, et al. A residents’ training pro-gram for the development of smoking intervention skills. ArchIntern Med 1988;148:1039–45.

16. Quirk M, Ockene J, Kristeller J, et al. Training family practiceand internal medicine residents to counsel patients who smoke:improvement and retention of counselling skills. FamMed 1991;23:108–11.

17. Shank J, Powell T, Llewelyn J. A five-year demonstration projectassociated with improvement in physician health maintenancebehaviour. Fam Med 1989;21:273–8.

18. Patterson J, Fried R, Nagle J. Impact of a comprehensive healthpromotion curriculum on physician behaviour and attitudes. AmJ Prev Med 1989;1:44–9.

19. Strecher V, O9Malley M, Villagra V, et al. Can residents betrained to counsel patients about quitting smoking? Resultsfrom a randomised trial. J Gen Intern Med 1991;6:9–17.

20. McIlvain H, Susman J, Manners M, Davis C, Gilbert C. Improv-ing smoking cessation counselling by family practice residents. JFam Pract 1992;34:745–9.

21. Mandel I, Franks P, Dickinson J. Screening guidelines in a fam-ily medicine program: a five-year experience. J Fam Pract 1982;14:901–7.

22. Robie P. Improving and sustaining outpatient cancer screeningby medicine residents. South Med J 1988;81:902–5.

23. Rodney W, Johnson R, Beaber R, Jonokuchi C, Kujubu D. Resi-dency chart review: preventive medicine practice as noted in themedical record. Fam Pract Res J 1982;1:140–51.

24. Cohen S, Weinberger M, Tierney W, McDonald C. The impact of

reading on physicians’ nonadherence to recommended standardsof medical care. Soc Sci Med 1985;21:909–14.

25. Cheney C, Ramsdall J. Effect of medical records checklists onimplementation of periodic health measures. Am J Med 1987;83:129–36.

26. Schreiner D, Petrusa E, Rettie C, Kluge R. Improving compli-ance with preventive medicine procedures in a house staff train-ing program. South Med J 1988;81:1553–7.

27. Becker D, Gomz E, Kaisr D, Yoshihasi A, Hodge R. Improvingpreventive care at a medical clinic: how can the patient help? AmJ Prev Med 1989;5:353–9.

28. Cowan J, Heckerling P, Parker J. Effect of fact sheet reminder onperformance of the periodic health examination: a randomisedcontrolled trial. Am J Prev Med 1992;8:104–9.

29. McPhee S, Bird J, Fordham D, Rodnick J, Osborn E. Promotingcancer screening: a randomised controlled trial of three inter-ventions. Arch Intern Med 1989;149:1866–72.

30. Turner B, Day S, Borenstein B. A controlled trial to improvedelivery of preventive care: physician or patient reminders? JGen Intern Med 1989;4:403–9.

31. RACGP. The scope of general/family practice. Melbourne:RACGP, 1981.

32. Abrahamson S. Report of the review of the Family MedicineProgram for the Royal Australian College of General Practitio-ners. Sydney: RACGP, 1987.

33. Committee of Inquiry into Medical Education andMedical Work-force. Australian medical education and workforce into the 21stcentury. Canberra: AGPS, 1988.

34. Mant D, Phillips A. Can the prevalence of disease risk factors beassessed from general practice records? Br Med J 1986;292:102–6.

35. Hoppe R, Farquar L, Henry R, Stoffelmayr B. Residents’ atti-tudes toward and skills in counselling: using undetected stan-dardised patients. J Gen Intern Med 1990;5:415–20.

36. Del Mar C, Lowe J, Adkins P, Arnold E. What is the quality ofgeneral practitioner records in Australia? Aust Fam Physician1996;25:21–5.

37. Pendleton D, Schofield T, Tate P, Havelock P. The consultation:an approach to teaching and learning. Oxford: Oxford Univ.Press, 1984.

38. SAS Institute, Inc. SAS/STAT users guide. Version 6, 4th ed.Carey (NC): SAS Inst., 1989.

39. Altman D. Practical statistics for medical research. London:Chapman Hall, 1991.

40. Smith W, Tattersal M, Irwig L, Langlands A. Undergraduateeducation about cancer. Eur J Cancer 1991;27:1448–53.

41. Gordon J, Fahey P, Sanson-Fisher RW. Interns’ identification ofpatients’ health risks in a casualty department. Med J Aust1988;148:615–9.

42. Skeff K, Berman J, Stratos G. A review of clinical teaching im-provement methods and a theoretical framework for their evalu-ation. In: Edwards J, Marier R, editors. Clinical teaching formedical residents. New York: Springer, 1988;92–120.

DOES A WORKSHOP IMPROVE PREVENTIVE CARE? 747