BRIEF REPORTS

Accuracy of women’s recall of opportunistic recruitment for cervical cancer _ _ screening in general practice

Jeanette Ward Central Sydney Area Health Service

Rob Sanson-Fisher Hunter Centre for Health Advancement, Newcastle

Abstract: Evaluation of strategies to improve oppor- tunistic recruitment via general practice of women overdue for a cervical smear requires an accurate behavioural measure. As part of an experimental trial to evaluate the effect of a postgraduate work- shop on preventive care, we conducted this method- ological study to determine the accuracy of women’s recall of an opportunistic discussion about cervical screening, by comparing it against audiotapes ( n = 524). Taking the taped evidence of the trainee’s ver- bal behaviour as the gold standard, sensitivity was 85 per cent (95 per cent confidence interval (CI) 73.1 to 92.0 per cent) and specificity was 78 per cent (CI 73.9 to 81.6 per cent). Given the low rate of oppor- tunistic recruitment by trainees in the main study, only one-third of positive recollections by women of an opportunistic discussion about cervical screening were correct. Until other measures have been vali- dated, women’s recall may continue to be used to measure general practitioners’ behaviour, but an appreciation of likely bias is recommended. (Aust N ZJPublic Health 1997; 21: 335-6)

YLES and Sanson-Fisher recently reported the diminishing effect of repeat mass-mailing B campaigns to prompt women’s participation

in cervical screening, recommending instead more intensive efforts, including recruitment by general practitioners.’ Having shown the cost-effectiveness of an opportunistic approach to recruitment by gen- eral practitioners in the same issue of the Journal, Hyndman et al. speculated that ‘an education pro- gram for doctors could . . . nearly double the rate of screening for attending women’.‘ Further, Redman and Barrett have argued that ‘the challenge is to encourage general practitioners to ensure that all women in their practices are adequately screened’, suggesting potentially costly financial incentives.3

Rigorous evaluation of the effectiveness of such strategies in changing clinical practice is imperative, and the need for accurate measures of general prac- titioner behaviour is self-evident. One plausible mea- sure involves asking women themselves to recall any discussion about cervical smears when recently visit- ing a general practitioner, thereby allowing rates of such behaviour to be compared before and after education or other interventions. As no Australian research has been published reporting the accuracy

Correspondence to Associate Professor Jeanette Ward, CSAHS Needs Assessment & Health Outcomes Unit, Locked Bag 8, Newtown, NSW 2042. Fax (02) 9515 3348.

of women’s recall of an opportunistic discussion about cervical screening in general practice, we con- ducted a methodological study within a larger trial to improve preventive care by postgraduate trainees in the training program of the Royal Australian College of General Practitioners.

Method As described fully el~ewhere,~ consultations between consenting adult patients able to speak and write English (72 per cent consent rate) and 34 partici- pating trainees (80 per cent consent rate) were audiotaped during a three-month term in general practice in 1991-92. Trainees audiotaped consulta- tions when patients gave them a completed consent form at the beginning of the consultation. Trainees copied the patient’s unique code number from the consent form onto the audiotape. The proportion of consultations at the beginning of the term in which trainees were heard to ask an opportunistic question about a cervical smear was low, at 13 per cent. It did not change significantly over the trial p e r i ~ d . ~ Completed consent forms were returned by each practice on a daily basis, allowing us to mail a ques- tionnaire almost immediately to women (84 per cent return rate); exactly half were completed by the women within seven days of the consultation.

The questionnaire elicited women’s recall of a question by the trainee during that consultation about the date of her last cervical smear. While three fixed responses were provided (‘yes’, ‘no’, ‘can’t remember’), ‘can’t remember’ responses were con- sidered to be missing data (15 per cent). Information provided on the original patients’ con- sent form allowed us also to exclude from analysis those women who had actually attended for a Pap smear, as their recall of such a question would differ from their recall of an opportunistic approach in an unrelated consultation. Audiotapes were analysed for an audible question from the trainee about the date of the patient’s last smear. Kappa values for intra-rater and inter-rater reliability for this behav- iour were 0.96 and 0.8 respectively.

Results The resulting 524 analysed audiotapes and matched questionnaires permitted construction of a 2x2 table to calculate sensitivity (the proportion of women who were asked the question who recalled the ques- tion, that is, true positive recall) and specificity (the proportion of women who were not asked the ques-

AUSTRALIAN AND NEW ZEAlAND JOURNAL OF PUBLIC HEALTH 1997 VOL. 21 NO. 3 335

WARD AND SANSON-FISHER

Table 1: Accuracy of women’s recall of a general practice trainee’s opportunistic question about

date of last cervical smear

Patient recalled Trainee had askeda being asked Yes No Total

Yes 55 101 156 No 10 358 368

Note: (a) From analysis of an audiotape

tion who recalled that they were not asked it, that is, true negative recall).5 Taking the taped evidence of the trainee’s verbal behaviour as the ‘gold standard’, 85 per cent of women who were asked about the date of their last smear recalled being asked this question (sensitivity 85 per cent, 95 per cent confi- dence interval (CI) 73.1 to 92.0) (Table 1). Of those who were not asked such a question, 78 per cent reported this to be the case (specificity 78 per cent, CI 73.9 to 81.6) while 22 per cent (CI 18.3 to 26.1) incorrectly reported that they had been asked such a question (false positive rate 22 per cent). Given the low ‘true’ prevalence of an opportunistic discussion about cervical smears and this specificity, only one- third of all positive recollections were correct.

Discussion Like patients’ recall of smoking cessation advice,6 women’s recall of an opportunistic discussion by young trainees of her need for cervical screening is prone to overreport. If this finding is generalised

beyond the training context, estimates of an oppor- tunistic approach to recruitment for cervical screen- ing in general practice based on women’s recall will create an overinflated picture of general practition- ers’ behaviour. Effective implementation of cervical screening and evaluation of initiatives designed to enhance the contribution made by general practi- tioners will require more accurate measures of opportunistic approaches in general practice. In their absence, an appreciation of the likely bias of women’s recall, if used to evaluate behaviour of gen- eral practitioners, is recommended.

Acknowledgements This study was funded in part by grants awarded to Professor Sanson-Fisher from the Royal Australian College of General Practitioners Training Program and the NSW Cancer Council. Neil Donnelly pro- vided statistical support.

References 1.

2.

3.

4.

5.

6.

Byles JE, Sanson-Fisher RW. Mass mailing campaigns to pro- mote screening for cervical cancer: do they work and do they continue to work? A w t N ZJPublic Health 1996; 20: 254-60. Hyndman J, Straton J, Pritchard D, Le Sueur H. Cost-effec- tiveness of interventions for cervical screening in general practice. A w t N ZJPublic Health 1996; 20: 272-7. Redman S. Barratt A. Towards a DoDulation-based screening. program for cervical cancer [edtdrial]. A w t J Public Heal2 1995; 19: 115-17. Ward JE, Sanson-Fisher RW. Does a threeday workshop for family medicine trainees improve opportunistic preventive care? A randomised control trial. Prm Med 1996 25: 741-7. Sackett D, Haynes R, Guyatt G, Tugwell P. Clinical epidemiol- om. 2nd edn. Boston: Little Brown, 1991. Ward IE, Sanson-Fisher RW. Patients’ recall of oDDortunistic

I I

smoking cessation advice in general practice. Tobacco Control 1996; 5: 110-13.

336 AUSTRALIAN AND NEW ZEALAND JOURNAL OF PUBLIC HEALTH 1997 vot. 21 NO. 3