British Journal of Obstetrics and Gynaecology December 1998, Vol. 105, pp. 1308-1311

SHORT COMMUNICATIONS

The reliability of the detection of an early diastolic notch with uterine artery Doppler velocimetry

Tom Farrell Specialist Registrar, Patrick F. W. Chien Senior Lecturer, Gary J. Mires Senior Lecturer Department of Obstetrics and Gynaecology, Ninewells Hospital and Medical School, Dundee

This study evaluates the ability of two reviewers to detect independently an early diastolic notch in 1371 uterine artery Doppler velocity waveform recordings. Agreement between the two reviewers for the detection of uterine artery notching was assessed by using the Kappa statistic. The inter-rater reliability for the detection of unilateral notching was 0.75 (95% CI 0.70-0.80), whereas that for the presence or absence of bilateral notching was 0.66 (95% CI 0.60-0.71). The results suggest that there was good reviewer agreement for the presence or absence of a notch on uterine artery Doppler velocimetry.

Introduction Pre-eclampsia remains a major cause of maternal and perinatal morbidity and Since it is uncommon for this disease to present in the first half of pregnancy and the natural progression of the established disease is unpredictable, a predictive test for this condition in early pregnancy would allow the possibility of vigilant antenatal surveillance and preventive measures to be taken to avoid serious sequelae. The lack of trophoblastic invasion of the decidual and myometrial segments of the spiral arter- ial vasculature resulting in an increased flow resistance in the uterine arteries3 has provided the possibility of using Doppler velocity waveform analysis in the second trimester as a screening test for pre- eclampsia4. Furthermore, these abnormal morpho- logical changes have been demonstrated to precede the clinical manifestation of the disease. An abnor- mal test result can be represented either by increased flow resistance or by the presence of an early diasto- lic notch, considered to be the result of a reflected wave of high amplitude returning from a uteropla- cental bed with high vascular resistance5 (Fig. 1). The identification of a notch on the uterine artery veloc- ity waveform appears to confer a higher sensitivity over the various measures of resistance index alonec8.

However, a recently conducted systematic review (unpublished) suggested that uterine artery Doppler

Correspondence: Dr P. F. W. Chien, Ninewells Hospital and Medical School, Dundee DDl 9SY, UK.

Fig. 1. Uterine artery Doppler velocity waveform with an early diastolic notch.

flow velocity, using either abnormal flow velocity ratios as an indirect proxy for increased flow resis- tance, and/or the presence or absence of a pre-diasto- lic notch has limited predictive value for pre-eclamp- sia, intrauterine growth retardation and perinatal death. One possible reason for the lack of diagnostic accuracy is the lack of high reliability of the mea- surements used in this test9. Reliability is defined as the ability of a test to provide the same or similar result in the same individual on different occasions (intra-rater reliability) or by different observers (inter-rater re1iability)'O. The aim of this study, was therefore, to determine the inter-rater reliability for the detection of an early diastolic notch with uterine

1308 0 RCOG 1998 British Journal of Obstetrics and Gynaecology

S H O R T C O M M U N I C A T I O N S 1309

artery Doppler velocity waveform analysis on one or both uterine arteries.

Methods All consecutive antenatal primigravidae attending Ninewells Hospital, Dundee for routine anomaly ultrasound scanning at 18-22 weeks of gestation over one year during 1993-1994 had both uterine arteries examined for evidence of early diastolic notching.

Ultrasonic examination was performed in the obstetric ultrasound department, which was staffed by five experienced ultrasonographers using transab- dominal real-time ultrasonography with a 3.5 MHz curvilinear probe (Acuson 128 XP/lO, Mountain View, California, USA or Diasonics Prisma (VST series), Les Ulis, France). All the women were scanned in the semi-recumbent position. The mater- nal uterine arteries were identified using colour flow mapping at the site of their apparent crossover with the external iliac artery on either side of the uterus. Pulsed wave Doppler assessment was performed with the high-pass filter set at 50 Hz and the angle of insonation was between 40-50%. The velocity wave- forms for both uterine arteries were visualised in real-time. The quality of the imaging was only con- sidered optimal when four to five successive wave- forms with a clear outline from each uterine artery were obtained. The images were then stored as hard copy for future reference. All data were collected prospectively.

The printed images were subsequently indepen- dently reviewed by two of the authors (TF and GJM) blinded to the clinical outcome(s) for the women. The ultrasonographers involved with the study were also blinded to the clinical information of the women and the clinicians caring for them were blinded to the uterine artery Doppler result. Each reviewer classi- fied the Doppler images of each uterine artery as demonstrating the presence or absence of early diastolic notching. A uterine notch was defined as a definite upward change in velocity after the initial deceleration slope of the primary wave. No attempt was made to quantify the degree of notching, and no other measures of vascular resistance were performed.

The Kappa statistic'l, which provides measure- ment of agreement beyond chance, was used to assess the inter-rater reliability since the measure- ment scale used for the classification of the Doppler images was dichotomous (i.e. the presence or absence of early diastolic notching). Reliability was evaluated for the detection of notching in either unilateral or bilateral uterine arteries by the two reviewers.

Reviewers agreement is considered to be substantial when Kappa levels are > 0.6012.

Results During the study period, 1434 pairs of uterine artery Doppler images were recorded. In 63 cases (4*4%), the paired uterine artery Doppler images became detached from each other and non-identifiable. Therefore, the analysis was restricted to the remain- ing 1371 pairs of uterine artery Doppler recordings. The incidence of moderatehevere pre-eclampsia (defined as a diastolic blood pressure 2 100 mmHg and 2 2+ proteinuria) and intrauterine growth retar- dation (defined as corrected birthweight I 3rd cen- tile) in the study population was subsequently found to be 2.5% and 4.6%, respectively.

Review of the Doppler images for unilateral notching produced agreement between the two reviewers for the presence of notching in 307 women (22.4%) and absence of notching in 929 (67.8%). Disagreement was found in 135 women (9.8%) (Table 1). The inter-rater reliability for the presence of unilateral notching was found to be 0.75 (95% CI 0*70-0*80).

Bilateral uterine artery notching was rated to be present by both reviewers in 105 women (7.6Y0) and absent in 1174 (85.6%). Disagreement was present in 92 cases (6.8%). The corresponding Kappa value for inter-rater reliability on bilateral uterine artery notching was 0.66 (95% CI 0.60-0.71) (Table 1).

Discussion Our study employed consecutive patient enrolment and adequate blinding of results between the observers rating whether an early diastolic notch was present or absent on the uterine artery Doppler images. The observers were also blinded as to the pregnancy outcome(s) of the women. The incidence of moderatehevere pre-eclampsia and intrauterine growth retardation in the recruited study population is within the range generally quoted in the literature, indicating that the study population is representative of an unselected obstetric p~pulation'~. All these methodological elements represent high study design, thus providing confidence in the lack of bias in the results (i.e. the internal validity of the study is high)14J5.

The concordance between the two raters on the presence or absence of unilateral and bilateral notch- ing was high (Kappa values were 0.75 and 0.66, respectively). It might appear paradoxical that the inter-rater reliability was higher with unilateral uterine artery notching when the proportion of disagreement

0 RCOG 1998 Br J Obstet GynaecoZlO5, 1308-1311

1310 SHORT C O M M U N I C A T I O N S

Table 1. Inter-rater reliability on the detection of unilateral and bilateral uterine artery notching on Doppler ultrasound scanning.

n (YO) Kappa value (95% CI)

Unilateral notch (n = 1371) 307 (22.4) 0.75 ( 0 . 7 0 4 8 0 ) 929 (67.8)

92 (6.7)

43 (3.1)

105 (7.6) 1174 (85.6)

Notch present by both reviewers Notch absent by both reviewers Notch present by reviewer 1;

Notch absent by reviewer 1; absent by reviewer 2

present by reviewer 2 Bilateral notch (n = 1371)

Notch present by both reviewers Notch absent by both reviewers Notch present by reviewer 1;

Notch absent by reviewer 1; absent by reviewer 2

present by reviewer 2

64 (4.7)

28 (2.1)

0.66 (0.6W.71)

between the two reviewers was higher (9.8%), com- pared with notching on both uterine arteries (6.8%). The reason for this disparity is that the value of Kappa is sensitive to the prevalence of the different categories of test result16 (i.e. if the proportion of sub- jects in the different categories are different between unilateral and bilateral uterine artery notching, then the Kappa value may indicate a difference in the inter- rater agreement which is not due to the performance of the raters). As a consequence, it is misleading to compare values of Kappa from different outcomes or studies where the prevalences of the categories differ. Despite this, the Kappa statistic is still considered the best approach to the assessment of inter-rater reliabil- ity for a binary test result, provided that the raw data is also presented16.

Despite good reviewer agreement for the presence or absence of a notch on uterine artery Doppler velocimetry, there is still debate as to whether the presence of such a finding is associated with subse- quent adverse pregnancy outcome such as pre- eclampsia, intrauterine growth retardation, prematu- rity or perinatal death”. A recently conducted systematic review (unpublished) on uterine artery Doppler velocity waveform analysis failed to support the routine use of this investigation as a screening test for these adverse pregnancy outcomes.

Future research on diagnostic tests should, ideally, be initially targeted on the assessment of measure- ment variability of the test (i.e. reliability and validity) prior to the evaluation of the diagnostic performance of the test (i.e. sensitivity, predictive values and likeli- hood ratios for positive and negative test results) using a pre-determined cut-off level for abnormal test results. It is unlikely that a test with poor reliability and/or validity in its clinical measurement will prove

to be a clinically useful test9. In the case of uterine artery Doppler velocimetry, the diagnostic perfor- mance of the test remains poor, despite sufficiently high reliability in its clinical measurement.

Acknowledgements The authors would like to thank the staff of the obstetric ultrasound department at Ninewells Hospital, Dundee for their help in enrolling patients into the study and in performing the ultrasound and Doppler examinations.

References Report on Confidential Enquiries into Maternal Deaths in the United Kingdom 1991-1993. London: HMSO, 1996 20-31. Montan S , Sjoberg 0-0, Svenningsen N. Hypertension in preg- nancy-fetal and infant outcome. CIin Exp Hypertens-Hyper in Pregnancy 1987; B62: 337-348. Meekins JW, Pijnenborg R, Hannsens M, McFadyen IR, van Asshe A. A study of placental bed spiral arteries and trophoblast invasion in normal and severe pre-eclampsia pregnancies. Br J Obstet Gynaecol1994; 101: 669474. Steel SA, Pearce JM, Chamberlain G. Doppler ultrasound of the uteroplacental circulation as a screening test for severe pre- eclampsia with intrauterine growth retardation. Eur J Obstet Gynecol Reprod Biol1988; 28: 279-287. Mo LY, Bascom PA, McCowan LME, Ritchie K. A transmission line approach to the interpretation of uterine Doppler waveforms. Ultrasound MedBiol1988; 14: 365-376. Harrington KF, Campbell S , Bewley S, Bower S. Doppler velocimetry studies of the uterine artery in the early prediction of pre-eclampsia and intra-uterine growth retardation. Eur J Obsfei Gynecol Reprod Bioll991; 42: S14S20. Harrington K, Cooper D, Lees K, Hecher K, Campbell S. Doppler ultrasound of the uterine arteries: the importance of bilateral notching in the prediction of pre-eclampsia, placental abruption or delivery of a small for gestational age baby. Ultrasound Obstet Gynecol1996; 7 : 182-188. Bower S, Bewley S, Campbell S . Improved prediction of pre- eclampsia by two-stage screening of uterine arteries using the early diastolic notch and color Doppler imaging. Obstet Gyiecol 1993; 82: 78-83.

0 RCOG 1998 Br J Obstet Gynaecol 105, 1308-1311

S H O R T C O M M U N I C A T I O N S 1311

9 Grant JM. Confusion with Doppler, certainty with salt, and more basic science needed in pre- eclampsia [Editor’s Choice]. Br J Obstet Gynaecol1998; 105: v.

10 Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide io Their Development and Use New York Oxford University Press, 1998.

11 Cohen J. Weighted Kappa: nominal scale agreement with provi- sion for scaled disagreement or partial credit. Psycho Bull 1968; 70: 2 13-220.

12 Landis RJ, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174.

13 Smith MA. Pre-eclampsia. Prim Care 1993; 20: 65-64. 14 Dunn G. The Design and Analysis of Reliability Studies: The

Statistical Evaluution of Measurement Errors. London: Edward Arnold, 1989.

15 Khan KS, Chien PFW, Honest MR, Norman GR. Evaluating measurement variability in clinical investigations: the case of ultrasonic estimation of urinary bladder volume. Br J Obstet Gynaecol1997; 104: 1036-1042.

16 Altman DG. Practical Statistics for Medical Research. London: Chapman and Hall, 1991.

17 Chappell L, Bewley S. Pre-eclamptic toxaemia: the role of uterine artery Doppler. Br J Obsfet Gynaecoll998; 105: 379-382.

Received 15 July 1998 Accepted23 September 1998

0 RCOG 1998 Br J Obstet Gynaecol 105, 1308-1311