Ophthalmic Surgery, laSerS & imaging · VOl. 43, nO. 1, 2012 25

■ C L I N I C A L S C I E N C E ■

I M A G I N G

From Doheny Eye Institute (RKK, MGN, SRS) and the Department of Ophthalmology (SRS), Keck School of Medicine of the University of Southern California, Los Angeles, California; and the Casey Eye Institute (OT, DH), Oregon Health & Science University, Portland, Oregon.

Originally submitted June 29, 2011. Accepted for publication October 24, 2011.Supported by National Institutes of Health Grants R01 EY013516 and P30 EY03040 and by a grant from Research to Prevent Blindness.Dr. Tan receives grant support from Optovue, Inc., Fremont, California. Dr. Huang receives royalties from an optical coherence tomography patent licensed to Carl

Zeiss Meditec, Inc., Dublin, California, and stock options, travel support, research grants, speaker honorarium, and patent royalties from Optovue, Inc. Dr. Sadda has previously shared in royalties for intellectual property licensed to Topcon Medical Systems by the Doheny Eye Institute, has served as a consultant to Heidelberg Engineering, and receives research support from Optovue, Inc., Carl Zeiss Meditec, and Optos. The remaining authors have no financial or proprietary interest in the materials presented herein.

Address correspondence to Srinivas R. Sadda, MD, Doheny Eye Institute, 1450 San Pablo Street DEI 3623, Los Angeles, CA 90033. E-mail: ssadda@doheny.orgdoi: 10.3928/15428877-20111129-04

n BACKGROUND AND OBJECTIVE: To evalu-ate reproducibility and intergrader agreement of total retinal blood flow (TRBF) measurements obtained by semi-automated grading of Doppler Fourier-domain optical coherence tomography (FD-OCT) scans.

n PATIENTS AND METHODS: Doppler FD-OCT scans were obtained from 20 eyes of 18 subjects (10 glaucomatous, 10 normal). Scans were obtained using a circumpapillary scan protocol and analyzed using the Doppler OCT of Retinal Circulation software (version 2). Two masked, independent human graders manually refined the scans, adding or deleting vessels, changing vessel boundaries, and classifying vessels as veins or ar-teries. TRBF was calculated automatically by software summing flow in all veins. Agreement between various vessel parameters and TRBF values generated by the graders was analyzed.

n RESULTS: Mean difference and mean absolute difference (± standard deviation, range) for TRBF

were -0.55 (± 5.37, -8.53 to 13.6) and 3.84 (± 3.70, 0 to 13.6) µL/min, respectively, with an intraclass correlation (ICC) of 0.933 and limits of agreement (95% confidence interval [CI]) of -11.1 to +10.0. Venous area measurements showed similar levels of agreement with mean difference and mean absolute difference (± standard deviation, range) of -2.91 (± 4.29, -10.95 to 6.43) and 3.59 (± 3.70, 0 to 10.9) mm2 with an ICC of 0.933 and limits of agreement (95% CI) were -11.3 and +05.5. The agreement for vessel identification between graders was almost per-fect with a weighted kappa of 0.86.

n CONCLUSION: Reproducible measurements of TRBF can be obtained from Doppler OCT data us-ing semi-automated software with manual refinement. These findings should be of value in future studies eval-uating retinal blood flow in various diseases.

[Ophthalmic Surg Lasers Imaging 2012;43:25-31.]

Reproducibility of Retinal Blood Flow Measurements Derived From

Semi-Automated Doppler OCT AnalysisRanjith K. Konduru, MBBS; Ou Tan, PhD; Muneeswar G. Nittala, MPhil Opt;

David Huang, MD, PhD; Srinivas R. Sadda, MD

26 cOpyright © SlacK incOrpOrated

I M A G I N G

INTRODUCTION

Optical coherence tomography (OCT)1 is com-monly used to evaluate various retinal diseases,2 such as vitreomacular interface disorders,3 retinal vascular diseases,4,5 age-related macular degeneration, choroidal neovascularization,6 and glaucoma.7,8 These diseases constitute the major causes of blindness in the United States and in other developed nations.9

Despite its many attributes and advantages, OCT has still not been considered a useful tool among ophthalmic imaging techniques, such as fluorescein angiography, for assessing retinal blood flow. However, retinal blood flow information can potentially be extracted from Doppler signals pres-ent within the OCT data. This technique of Doppler OCT imaging has been used to calculate blood flow in retinal vascular diseases, such as diabetic retinopa-thy, in which retinal blood flow was shown to be reduced in patients with diabetes mellitus compared to normal subjects.10 Although blood flow variations are well recognized in retinal venous occlusive dis-eases, glaucoma, and other vascular and perfusion disorders of the optic nerve head, they have not been quantitatively well demonstrated.11-14 Previous research suggests that quantitative volumetric mea-surement of total retinal blood flow (TRBF) in the eye may provide new insights into the pathogenesis of retinal disease.10,12,14,15

Different approaches to studying retinal blood flow have been suggested. One potential approach is to collect the Doppler flow signal from Fourier-domain OCT (FD-OCT) images, as demonstrated by Wehbe et al.16 and Wang et al.17 Accurate quantification of blood flow for use in research investigations has been previ-ously demonstrated by successful calculation of TRBF with a semi-automated process, in which candidate vessels are identified by computer algorithms and the results are manually refined by human experts.10,17-19 The reproducibility of this approach is of critical im-portance for future research in this area, particularly for assessing the significance of changes in blood flow in longitudinal studies; however, such reproducibility has yet to be established.

We report the intergrader reproducibility of a va-riety of Doppler blood flow measurements, including TRBF, superior hemisphere flow, inferior hemisphere flow, venous area, and venous velocity.

PATIENTS AND METHODS

Patients who presented to the Doheny Eye Insti-tute for ophthalmic examination were recruited for a prospective study of Doppler OCT retinal blood flow analysis in normal and glaucomatous eyes. To be in-cluded in the study, patients had to be willing to have at least one eye scanned and had to have ocular media sufficiently clear to permit OCT imaging. For this re-producibility study, 20 eyes (10 eyes with a diagnosis of glaucoma and 10 normal eyes) from 18 consecutive subjects were randomly selected from the University of Southern California Advanced Imaging for Glau-coma Study (AIGS) dataset [www.aigstudy.net]. The diagnostic criteria for normal and glaucomatous eyes have been described in a previous publication.8 The research protocols were approved by the Institutional Review Board of the University of Southern California and adhered to the tenets set forth in the Declaration of Helsinki.

Doppler Scan Acquisitions Doppler scans were obtained with the RTVue FD-

OCT (Optovue Inc., Fremont, CA) using the previ-ously published dual circular scan protocol.18 The scan pattern consisted of two concentric circles (inner and outer ring diameters are 3.4 and 3.75 mm, respective-ly) around the optic nerve head, transecting all large retinal vessels (arteries and veins) near the optic nerve head in two locations. During a single scan acquisition sequence, a total of six dual circular scan frames were obtained and averaged for each ring. To ensure that high quality scans were available for grading, the ac-quisition sequence was repeated five to six times, thus yielding 30 to 36 frames for each ring for subsequent analysis. The Doppler shift in the vessel and the Dop-pler angle of the vessel were used to compute flow. The Doppler angle was estimated by vessel center depth dif-ference between the two rings of a scan. The TRBF was calculated by adding flows from all branch veins at one time point.18

Preparing the Data and Manual Grading Doppler scans and three-dimensional disk scan

from the RTVue SD-OCT instrument were exported as raw data using the RTVue Doppler transfer output software. The data were then converted to the Doppler OCT of Retinal Circulation (DOCTORC) Grading

Ophthalmic Surgery, laSerS & imaging · VOl. 43, nO. 1, 2012 27

I M A G I N G

Software compatible data format using ReVue software (Optovue Inc.), and then loaded into the DOCTORC software for initial automated assessment and identifi-cation of candidate vessel locations and vessel boundar-ies.

After the initial automatic processing and candi-date vessel identification, Doppler scans were registered with the OCT projection image, infrared scanning la-ser ophthalmoscopy image, and a color disc fundus photograph. This process allows the grader to correlate candidate vessels seen on the Doppler OCT B-scans with vessels visible on the en face images and facili-tates classification of vessels as arteries and veins (Fig. 1). Scans were manually reviewed by certified Doppler OCT reading center graders to verify and refine the blood vessel position and to check the accuracy of ves-sel classification type (artery vs vein). The grading pro-cedure included adjusting vessel locations and vessel size, matching corresponding vessels on the two rings, adding any missing vessels, deleting extraneous vessels, and verifying the vessel type classification. Graders used a variety of clues to identify vessel positions on B-scans, including characteristic hyperreflectivity and posterior shadowing on OCT, and the position of vessels on an-cillary images, such as the registered OCT projection images, color images, and infrared image. Figure 1A shows the DOCTORC software version 2 display of an OCT projection image of the fundus used to facilitate

identification of blood vessel location. Figure 2 shows a cross-sectional image of the vessel on the OCT B-scan.

Two independent trained graders (RK & OT) completed manual grading by verifying the software-detected blood vessel position and its area, changing or adjusting its borders if necessary, and adding or deleting the vessel as needed as described above. The DOCTORC software graphical user interface allows the user to adjust the position of the vessel center and the width or largest horizontal diameter of the vessel. After the vessel position and its area were adjusted in both the outer and inner rings corresponding to that particular vessel, a confidence score was assigned by the grader to each vessel on every scan to reflect the grader’s certainty regarding the accuracy of the vessel grading. Confidence scores ranged from 0 to 5 and were influenced by several factors, including Dop-pler signal strength of the vessel, clarity of the vessel boundary, and vessel size agreement between the two concentric scan rings.

This subjective confidence score, together with the calculated Doppler angle and the variance of Doppler angle, served as the key criteria in an automated qual-ity check that transformed each parameter into a prob-ability and then combined all parameters into a single probability metric using Bayesian rules. This quality check was used to determine whether the flow result of a particular vessel could be considered valid and could

Figure 1. (A) Reconstructed fundus image of eye from spectral-domain optical coherence tomography (SD-OCT). Numbers identify the vessel location and color indicates type of vessel detected: red = artery; blue = vein. (B) Disk fundus photograph used for comparison of reconstructed fundus image from SD-OCT of the same patient to help identify the type of vessel as an artery or a vein.

28 cOpyright © SlacK incOrpOrated

I M A G I N G

be included in the calculation of the total blood flow. The average speed of flow in all veins was computed by summing the flow in all valid veins and dividing by the sum of the cross-sectional area of all valid veins. In cas-es where a retinal vessel did not pass the validity check, the flow in this vessel was estimated from its vessel area using the mean flow speed of all valid veins (ie, flow was effectively interpolated across these invalid vessels). Total retinal blood flow was computed by summing the calculated flow of valid veins and the estimated flow of invalid veins. For a particular eye, if the percentage of area of valid veins was less than a threshold (currently 50% of total vein area), then the result from that eye was considered unreliable. Although this interpolation method has the potential to underestimate or overes-timate flow in certain cases, it is not relevant to the current study. However, because the focus of this study was to assess intergrader reproducibility and not the validity of the interpolation method, all eyes were used

for reproducibility statistic calculations and for com-parison of intergrader differences.

To determine whether vessel identification and grading reproducibility were affected by the size of the vessels, all graded vessels were divided into two groups (smaller vs larger) based on the vessel cross-sectional area. The level of intergrader agreement in vessel iden-tification (artery versus vein) for larger (> 5,000 µm2) and smaller (< 5,000 µm2) vessels was evaluated.

Statistical AnalysisThe TRBF, venous area, superior hemispheric

blood flow, inferior hemispheric blood flow, venous velocity, Doppler shift, and Doppler angle measure-ments for the two graders were analyzed. Intraclass cor-relation coefficients (ICCs) were used to look for the intergrader reproducibility (an ICC approaching 1 is considered good and indicates better reproducibility). Bland–Altman plots were generated to illustrate the

Figure 2. Cross-sectional image of the vessels for grading. The grader marks the vessel circumference using a dotted sizing circle, and the diameter using a horizontal caliper line, used for vessel area calculation and angle estimation between two rings of the scan.

Ophthalmic Surgery, laSerS & imaging · VOl. 43, nO. 1, 2012 29

I M A G I N G

limits of agreement. We also used kw (weighted kap-pa) statistics, a measure of intergrader concordance in categorical scales that adjusts for chance agreement, to measure the correlation between graders. The kw value is interpreted using the following ranges: 0 to 0.2 = slight agreement; 0.21 to 0.40 = mild; 0.41 to 0.60 = moderate; 0.61 to 0.80 = substantial; and 0.81 to 1.0 = almost perfect agreement. SPSS statistical software (IBM Corporation, Somers, NY) and MedCalc soft-ware (MedCalc Software, version 11.3.8, Mariakerke, Belgium) were used for these analyses. A P value of .05 was considered statistically significant.

RESULTS

Randomly selected Doppler OCT scan acquisi-tions from 20 eyes of 18 participants were included in this analysis. The mean age of the study group was 60.7 years (range: 45 to 79 years). Fourteen of the 20 eyes were left eyes, and 15 eyes were from female partici-pants. Ten eyes were clinically diagnosed as glaucoma-tous; the remaining 10 eyes were classified as normal.

The mean TRBF for all participants was 38.49 µL/min for grader 1 and 39.04 µL/min for grader 2. The mean absolute difference in TRBF for the participants

in this series was 3.84 ± 3.70 µL/min (range: 0 to 13.6 µL/min), and there was no statistically significant dif-ference between the two graders for TRBF (P = .872 and ICC of 0.933 [95% CI: 0.831 to 0.974]). Figure 3A shows the Bland–Altman plot for TRBF with lim-its of agreement. Venous area measurements showed similar levels of agreement with a mean venous area for all participants of 47.27 and 50.18 mm2, for grad-ers 1 and 2, respectively. The mean absolute difference in mean venous area between graders was 3.59 ± 3.70 mm2 (range: 0 to 10.9 mm2). There was no statistically significant difference between the two graders for ve-nous area (P = .288 and ICC of 0.933 [95% CI: 0.83 to 0.973]). Figure 3B shows the Bland–Altman plot for venous area with limits of agreement. There was an almost perfect agreement on identification and clas-sification of vessels between the two graders with a kw of 0.86 (kw for large and small vessels was 0.910 and 0.783, respectively). ICC (95% CI) values for all pa-rameters are summarized in the table.

DISCUSSION

In this study, we observed that manual correction of automated retinal blood vessel detections using the

TABLE

Values of Study Parameters

ParameterMean ± SD (Range) Values of Grader 1

Mean ± SD (Range)

Values of Grader 2Mean ± SD (Range) Absolute Difference ICC (95% CI)

TRBF (µL/min) 38.49 ± 10.84 (12.78–57.14)

39.04 ± 10.65 (15.26–57.14)

3.84 ± 3.70 (0–13.6)

0.933 (0.831–0.974)

Venous area (mm2) 47.27 ± 8.36 (30.35–61.01)

50.18 ± 8.74 (31.14–67.84)

3.59 ± 3.70 (0–10.95)

0.933 (0.830–0.973)

Superior flow (µL/min) 20.17 ± 6.6 (6.67–30.45)

20.82 ± 6.75 (7.31–30.45)

3.15 ± 3.65 (0–13.4)

0.849 (0.619–0.940)

Inferior flow (µL/min) 18.32 ± 5.9 (6.10–29.40)

18.15 ± 6.14 (7.31–28.31)

1.79 ± 1.74 (0–4.46)

0.954 (0.884–0.982)

Superior area (mm2) 25.16 ± 6.07 (16.9–40.63)

26.46 ± 5.94 (18.99–44.18)

2.04 ± 1.92 (0–6.71)

0.955 (0.885–0.982)

Inferior area (mm2) 22.12 ± 5.42 (13.29–35.0)

23.72 ± 6.4 (11.08–40.01)

2.60 ± 2.11 (0–6.87)

0.933 (0.831–0.973)

Doppler angle (degree) 5.06 ± 7.0 (-24.34–+28.3)

5.22 ± 7.0 (-23.01–+31.69)

1.26 ± 2.52 (0–28.83)

0.958 (0.953–0.963)

Doppler shift 0.033 ± 2.66 (-7.63–+78.89)

- 0.008 ± 2.58 (-294.95–+78.08)

0.16 ± 0.78 (0–20.71)

0.977 (0.974–0.979)

SD = standard deviation; ICC = intraclass correlation; CI = confidence interval; TRBF = total retinal blood flow.

30 cOpyright © SlacK incOrpOrated

I M A G I N G

DOCTORC software could yield reproducible calcula-tion of retinal blood flow in normal and glaucomatous eyes. Although the TRBF and venous area measure-ments showed excellent reproducibility, some differ-ences in blood vessel parameters were still observed, particularly for small vessels. However, these differ-ences did not translate into a significant difference in TRBF. This is likely because, although only substantial agreement was observed for identification and grading of small vessels, agreement for large vessels was almost perfect. Because large vessels contribute to the bulk of the blood flow, errors in grading of small vessels are clinically less significant.18 In addition, our strategy to use estimated blood flow for vessels with invalid results also likely contributed to the excellent reproducibility between graders.

In addition to vessel size, another important factor in grading reproducibility appears to be Doppler angle. The calculated Doppler angle is based on identification of the axial position of each vessel within each of the two circular B-scans. Previous studies have confirmed the critical importance of the Doppler angle measure-ment for calculation of total retinal blood flow using this approach.19 We have observed that reproducibil-ity of Doppler angle determination between graders in our study was good (ICC of 0.958). An earlier study20 had also shown that high-quality scans optimize the Doppler signal and facilitate easier identification of the vessel boundaries. Hence, we evaluated intergrader cor-relation of Doppler signal in our study and found it to be highly reproducible (ICC of 0.977), thus allowing more reproducible vessel identification. Eccentric posi-

tioning of the scanning beam within the pupil appears to be a useful method for maximizing the Doppler an-gle, and hence the Doppler signal, among the vessels.20

Although several techniques in addition to those described in this study are being used to evaluate reti-nal blood flow, all of these techniques have limitations. For instance, ultrasound color Doppler imaging only has sufficient resolution to measure the larger retrobul-bar vessels.21 It can measure blood velocity but can-not measure vessel diameter, so volumetric blood flow cannot be determined. Several types of laser Doppler techniques can be used to measure flow in individual retinal vessels22,23 or capillary beds.24 Although TRBF can be determined by adding measurements from in-dividual vessels, doing so requires acquiring many measurements over a long session.25 Such specialized instruments are expensive and are generally available only in major research centers. Fluorescein and indo-cyanine green angiographies are widely used to visual-ize retinal and choroidal circulations, but these do not provide quantitative measurements of blood flow and they require the intravenous injection of dyes that have potential side effects.26

A semi-automated grading strategy using the DOCTORC software was able to achieve reasonably reproducible measurements of total retinal blood flow from Doppler SD-OCT data. Refinement of acquisi-tion protocols to maximize the Doppler signal with an optimal Doppler angle and development of fully auto-mated OCT vessel detection algorithms will be criti-cal for widespread application of Doppler OCT tech-niques. Retinal blood flow measurements, however,

Figure 3. Bland–Altman plot of agreement for (A) total retinal blood flow and (B) venous area measurements between graders 1 and 2. Central solid line indicates the mean absolute difference; dotted line indicates the 95% confidence interval (upper and lower) limits.

Ophthalmic Surgery, laSerS & imaging · VOl. 43, nO. 1, 2012 31

I M A G I N G

may provide new insights into disease mechanisms and response to therapies.

REFERENCES

1. Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography. Science. 1991;254:1178-1181.

2. Hee MR, Izatt JA, Swanson EA, et al. Optical coherence tomography of the human retina. Arch Ophthalmol. 1995;113:325-332.

3. Hee MR, Puliafito CA, Wong C, et al. Optical coherence tomography of macular holes. Ophthalmology. 1995;102:748-756.

4. Jonas J, Paques M, Monés J, Glacet-Bernard A. Retinal vein occlu-sions. Dev Ophthalmol. 2010;47:111-135.

5. Horii T, Murakami T, Nishijjma K, Sakamoto A, Ota M, Yoshimura N. Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy. Am J Ophthalmol. 2010;150:840-848.

6. Hee MR, Baumal CR, Puliafito CA, et al. Optical coherence tomog-raphy of age-related macular degeneration and choroidal neovascular-ization. Ophthalmology. 1996;103:1260-1270.

7. Schuman JS, Pedut-Kloizman T, Hertzmark E, et al. Reproducibility of nerve fiber layer thickness measurements using optical coherence tomography. Ophthalmology. 1996;103:1889-1898.

8. Tan O, Chopra V, Lu ATH, et al. Detection of macular ganglion cell loss in glaucoma by Fourier-domain optical coherence tomography. Ophthalmology. 2009;116:2305-2314.

9. Klaver CC, Wolfs RC, Vingerling JR, Hofman A, de Jong PT. Age-specific prevalence and causes of blindness and visual impairment in an older population: the Rotterdam Study. Arch Ophthalmol. 1998;116:653-658.

10. Wang Y, Fawzi AA, Tan O, Gil-Flamer J, Huang D. Retinal blood flow detection in diabetic patients by Doppler Fourier domain optical coherence tomography. Opt Express. 2009;17:4061-4073.

11. Logan JF, Rankin SJ, Jackson AJ. Retinal blood flow measure-ments and neuroretinal rim damage in glaucoma. Br J Ophthalmol. 2004;88:1049-1054.

12. Hafez AS, Bizzarro RL, Lesk MR. Evaluation of optic nerve head and peripapillary retinal blood flow in glaucoma patients, ocular hyper-tensives, and normal subjects. Am J Ophthalmol. 2003;136:1022-1031.

13. Berisha F, Feke GT, Hirose T, McMeel JW, Pasquale LR. Retinal blood flow and nerve fiber layer measurements in early-stage open-

angle glaucoma. Am J Ophthalmol. 2008;146:466-472. 14. Cuypers MH, Kasanardjo JS, Polak BC. Retinal blood flow changes

in diabetic retinopathy measured with the Heidelberg scanning laser Doppler flowmeter. Graefes Arch Clin Exp Ophthalmol. 2000;238:935-941.

15. Deokule S, Vizzeri G, Boehm A, Bowd C, Weinreb RN. Association of visual field severity and parapapillary retinal blood flow in open-angle glaucoma. J Glaucoma. 2010;19:293-298.

16. Wehbe H, Ruggeri M, Jiao S, Gregori G, Puliafito CA, Zhao W. Au-tomatic retinal blood flow calculation using spectral domain optical coherence tomography. Opt Express. 2007;15:15193-15206.

17. Wang Y, Bower BA, Izatt JA, Tan O, Huang D. In vivo total retinal blood flow measurement by Fourier domain Doppler optical coher-ence tomography. J Biomed Opt. 2007;12:041215.

18. Wang Y, Lu A, Gil-Flamer J, Tan O, Izatt JA, Huang D. Measure-ment of total blood flow in the normal human retina using Dop-pler Fourier–domain optical coherence tomography. Br J Ophthalmol. 2009;93:634-637.

19. Wang Y, Bower BA, Izatt JA, Tan O, Huang D. Retinal blood flow measurement by circumpapillary Fourier domain Doppler optical co-herence tomography. J Biomed Opt. 2008;13:064003.

20. Wang Y, Fawzi AA, Varma R, et al. Pilot study of optical coherence tomography measurement of retinal blood flow in retinal and optic nerve diseases. Invest Ophthalmol Vis Sci. 2011;52:840-845.

21. Goebel W, Lieb WE, Ho A, Sergott RC, Farhoumand R, Grehn F. Color Doppler imaging: a new technique to assess orbital blood flow in patients with diabetic retinopathy. Invest Ophthalmol Vis Sci. 1995;36:864-870.

22. Riva CE, Grunwald JE, Sinclair SH, Petrig BL. Blood velocity and volumetric flow rate in human retinal vessels. Invest Ophthalmol Vis Sci. 1985;26:1124-1132.

23. Feke GT, Tagawa H, Deupree DM, Goger DG, Sebag J, Weiter JJ. Blood flow in the normal human retina. Invest Ophthalmol Vis Sci. 1989;30:58-65.

24. Nicolela MT, Hnik P, Schulzer M, Drance SM. Reproducibility of retinal and optic nerve head blood flow measurements with scanning laser Doppler flowmetry. J Glaucoma. 1997;6:157-164.

25. Garcia JP Jr, Garcia PT, Rosen RB. Retinal blood flow in the normal human eye using the canon laser blood flowmeter. Ophthalmic Res. 2002;34:295-299.

26. Yannuzzi LA, Rohrer KT, Tindel LJ, et al. Fluorescein angiography complication survey. Ophthalmology. 1986;93:611-617.