SPIE Proceedings [SPIE SPIE Medical Imaging - Lake Buena Vista, Florida (Saturday 12 February 2011)] Medical Imaging 2011: Image Processing - Gradient-based 3D-2D registration of cerebral

Gradient-based 3D-2D Registration of Cerebral Angiograms

Uroš Mitrović*a, Primož Markelja,b, Boštjan Likara,b, Zoran Miloševičc, Franjo Pernuša,b

aUniversity of Ljubljana, Faculty of Electrical Engineering, Laboratory of Imaging Technologies bSensum Computer Vision Systems, Tehnološki park 21, 1000 Ljubljana, Slovenia

cUniversity of Ljubljana, Faculty of Medicine, Department of Radiology

ABSTRACT

Endovascular treatment of cerebral aneurysms and arteriovenous malformations (AVM) involves navigation of a catheter through the femoral artery and vascular system into the brain and into the aneurysm or AVM. Intra-interventional navigation utilizes digital subtraction angiography (DSA) to visualize vascular structures and X-ray fluoroscopy to localize the endovascular components. Due to the two-dimensional (2D) nature of the intra-interventional images, navigation through a complex three-dimensional (3D) structure is a demanding task. Registration of pre-interventional MRA, CTA, or 3D-DSA images and intra-interventional 2D DSA images can greatly enhance visualization and navigation. As a consequence of better navigation in 3D, the amount of required contrast medium and absorbed dose could be significantly reduced. In the past, development and evaluation of 3D-2D registration methods received considerable attention. Several validation image databases and evaluation criteria were created and made publicly available in the past. However, applications of 3D-2D registration methods to cerebral angiograms and their validation are rather scarce. In this paper, the 3D-2D robust gradient reconstruction-based (RGRB) registration algorithm is applied to CTA and DSA images and analyzed. For the evaluation purposes five image datasets, each comprised of a 3D CTA and several 2D DSA-like digitally reconstructed radiographs (DRRs) generated from the CTA, with accurate gold standard registrations were created. A total of 4000 registrations on these five datasets resulted in mean mTRE values between 0.07 and 0.59 mm, capture ranges between 6 and 11 mm and success rates between 61 and 88% using a failure threshold of 2 mm. Keywords: 3D-2D registration, gold standard, cerebral angiograms, gradient-based, evaluation, CTA

1. INTRODUCTION During the last few decades image-guided minimally invasive interventions have replaced many of the conventional invasive surgical procedures. Advantages of minimally invasive procedures include shorter patient recovery times, greater patient comfort, lower risk of complications, and faster patient throughput. One of the applications which are still increasing in frequency and complexity are image-guided endovascular interventions (IGEIs). In IGEI a small incision is generally made into the femoral artery, through which instruments such as guide wires and catheters are inserted. Navigation of these instruments through the vasculature to the site of the pathology is carried out under fluoroscopic and angiographic image guidance. The pathology to be treated may be a stenosed or totally occluded vessel, a portion of a vessel that is weakened and bulges to form an aneurysm, or a hypervascular region such as a tumor bed or an arteriovenous malformation (AVM). An IGEI may involve delivery of a drug or embolic material through the catheter, or delivery of a device such as a stent or coil that may be used to keep a stenosed vessel open or prevent rupture of an aneurism. Image-guided interventions require that the interventional radiologist navigate thru a (complex) three-dimensional (3D) environment using only the information contained in the current two-dimensional (2D) X-ray images which lack depth information and quality. For a precise navigation, fluoroscopic images have to be frequently acquired by which the patient and physician are exposed to radiation. To aid guidance, high-quality three-dimensional (3D) pre-interventional images like computed tomography angiography (CTA) or magnetic resonance angiography (MRA) images, that were used for diagnostic and planning purposes, can be used actively during the course of intervention. By bringing into spatial alignment the high quality 3D CTA or MRA image and the 2D fluoroscopic or angiographic

* E-mail: [email protected]; telephone: +386 (0) 1 4768 873; web: http://lit.fe.uni-lj.si/

Medical Imaging 2011: Image Processing, edited by Benoit M. Dawant, David R. Haynor, Proc. of SPIE Vol. 7962, 79621P · © 2011 SPIE · CCC code: 1605-7422/11/$18 · doi: 10.1117/12.877541

Proc. of SPIE Vol. 7962 79621P-1

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 11/19/2013 Terms of Use: http://spiedl.org/terms

image(s) through 3D-2D registration, the patient anatomy depicted in the 3D image is fused with the information on the current state of the intervention. In this way, visualization of the current position of the endovascular elements relative to the final target and other important anatomical structures can be obtained in 3D which would help the physician to perform the procedure according to the plan more accurately, safely, and faster, with a reduced amount of absorbed dose and contrast medium. In general, 3D-2D image registration methods can be categorized as calibration-based and intrinsic or extrinsic image-based methods. The intrinsic image-based methods can be further divided into intensity-based, feature-based and gradient-based methods.1 In the last decade, intensity-based2,3,4,5,6,7, feature-based8,9,10,11,12 and calibration-based13,14,15 registration algorithms were proposed for 3D/2D registration of cerebral angiograms. To the best of our knowledge there are no publications on gradient-based methods related to vascular structures. Intensity-based 3D-2D registration methods are based on 2D-2D matching of real fluoroscopic and simulated images, called digitally reconstructed radiographs (DRR), obtained by casting virtual rays through 3D CTA image. These methods vary with respect to the similarity measure and optimization strategy. Most frequently used similarity measures are mutual information4, (normalized) cross correlation2,4, pattern intesity4, gradient correlation4, entropy of difference image4, gradient difference4,5,6 and sum of square differences.7 Hipwell et al.4 analyzed the first six similarity measures and showed that best results are obtained using pattern intensity and gradient difference. Also, there are some other strategies, such as the method proposed by Chung et al.3, which minimizes the discrepancy between expected (a priori known) and estimated (observed) joint intensity distributions by minimizing the Kullback-Leibler distance. The main drawback of intensity-based methods is the computationally demanding calculation of DRRs, although with the introduction of graphical processor units (GPUs) and software-based approaches for speeding up the generation of DRRs this problem is largely overcome.16 The feature-based methods rely on matching corresponding features extracted from 3D and 2D images. Commonly used features for registering angiographic images are vessel skeletons. The advantage of feature-based methods is that they are computationally undemanding because of the large data reduction. Unfortunately, the accuracy directly depends on the quality of automatic, semi-automated or manual segmentation by which features are extracted. McLaughlin et al.6 compared an intensity-based and a feature-based registration method and showed that the intensity-based method was more accurate and reliable but significantly slower than the feature-based method. Gradient-based methods rely on fact that rays emanating from X-ray source that point to the edges in the X-ray images are tangent to surfaces of distinct anatomical structures. These methods combine positive characteristics of both, intensity-based and feature-based approaches. Using only a certain number of significant volume gradients (surface normals), the amount of data is reduced. Besides, segmentation of intra-operatively 2D images is not required. Calibration-based methods are based on the geometry of the (mobile) C-arm imaging device and a calibration procedure which needs to be performed regularly. However computational time in range of µs allows their application in real time.14,15 Recently, much effort was devoted to fusion of multi-modal images.13,14,15 By registration of images acquired using different modalities, relevant information in each modality can be exploited. For instance, rigid and soft tissue can be imaged using CT or MR, vessels by 3D rotational angiography (3DRA) and the catheter by X-ray fluoroscopy. Ruijters et al.14 proposed a method, where in the pre-processing step a CT or MR image is registered to the 3DRA image by a rough manual segmentation followed by a fine registration based on intensities. The vessel tree is obtained by segmentation of the 3DRA image. Finally, the 3DRA and 2D X-ray fluoroscopic images are registered by the calibration-based approach. Main advantages of the method are that the 3D/2D registration is calibration-based, thus registration can be done in real time. Besides, the vessel tree can be projected on the 2D X-ray fluoroscopic image, allowing injection of a contrast medium only once. Several validation image databases and evaluation criteria were created and made publicly available in the past. However, applications of 3D-2D registration methods to cerebral angiograms and their validation are rather scarce. In this paper, the 3D-2D robust gradient reconstruction-based (RGRB) registration algorithm17 is applied to CTA and DSA images and analyzed. For the evaluation purposes five image datasets, each comprised of a 3D CTA and several 2D DSA-like digitally reconstructed radiographs (DRRs) generated from the CTA, with accurate gold standard registrations were created.



2. METHODOLOGY 2.1 Robust gradient reconstruction-based method (RGRB) In this study, a 3D-2D robust gradient reconstruction-based (RGRB) method proposed by Markelj et al.17 was used. The method is based on matching 3D gradient vectors representing surface normals and a coarsely reconstructed 3D gradient field formed by adding back-projected 2D gradient vectors from all available 2D images. In the pre-processing step, the 3D volume and 2D images were blurred with a Gaussian filter (σ = 0.5 mm) and the resulting 3D image was isotropically resampled to the resolution of 0.75 mm. For calculation of volume gradients, the 3D Canny edge detector was applied, while the 2D gradients were calculated using a simple central-differences kernel. This resulted in approximately 30 000 gradients per 3D volume. Using the projection geometry, 2D gradients were back-projected into 3D, and a coarse 3D gradient field was reconstructed. Correspondences in magnitudes and angles between the two 3D fields were used to calculate the similarity measure. To avoid false gradient correspondences, the random sample consensus algorithm (RANSAC) was used. The six parameters, defining the optimal rigid transformation between the volume gradients and the coarsely reconstructed 3D gradients, were searched for by optimizing the similarity measure with the Powell’s method. 2.2 Datasets and evaluation criteria To evaluate the RGRB method, five datasets with accurate gold standard registrations were created. Each dataset consisted of a 3D CTA image and DSA-like lateral (LAT), anterior-posterior (AP) and oblique (axial rotation of 1500) view digitally reconstructed radiographs created by casting rays thru the CTAs. By this approach, the gold standard registration of a CTA and the corresponding synthetic DSAs was inherently determined by the geometry used to generate the DRRs. CTA data were obtained for five patients with aneurysms using the Siemens SOMATOM Sensation Open 40-slice configuration machine with voxel size 0.39 x 0.39 x 0.75 mm. In order to avoid the influence of the scull on the registration and DRR generation, a rectangular VOI containing the tissues inside the skull was determined. Examples of cross-sections of CTA VOI are shown in Fig. 1.

Figure 1. Sagital (left), transversal (middle) and coronal (right) cross-sections of CTA VOI for dataset 1.

As described in Markelj et al.18, DRRs were generated using Beer-Lambert law and conversion function C(µl, µh, o) that overcomes the problem of different effective photon energies used for CT and X-ray imaging, with pixel size 0.25 x 0.25 mm. Parameters of the conversion function were selected in such a way to obtain realistic DSA images (Fig. 2). To avoid the influence of truncated rays, a ROI was selected on 2D images where truncation artifacts can be observed.19

Figure 2. Lateral (left), anterior-posterior (middle) and oblique (right) view DRRs for dataset 1.



To evaluate the performance of the RGRB method, the methodology proposed by van de Kraats16 was adopted. As a figure of merit the mean target registration error (mTRE) was used. mTRE represents the mean distance between target points in the gold standard position and after the registration method is applied to the CTA displaced from the gold standard position. Target points were defined as lattice of points evenly spread with 10 mm spacing over the entire 3D volume. According to van de Kraats et al.20, initial displacements from the gold standard were made in the range of 0-20 mm of mTRE, with 20 registrations per 1 mm subinterval, resulting in 400 different displacements, i.e. starting positions. As a failure criterion 2 mm of mTRE was selected which is approximately 2/3 of the average diameter of cerebral vessels that most frequently undergo endovascular treatment. The registration accuracy, success rate and capture range (first 1 mm subinterval with less than 95% successful registrations) were computed according to the failure criterion which is a drawback of this approach. Recently, Markelj et al.18 proposed a new approach to measure accuracy that is based on the distribution of mTREs. By this approach, accuracy is defined as a certain percentile of the mTRE. The range of displacements was divided into accumulative subintervals 0-4 mm, 0-8 mm, 0-12 mm, 0-16 mm and 0-20 mm, and for each subinterval accuracy was calculated as a certain percentile of the mTRE distribution. Thus, the capture range can be understood as the upper bond of a subinterval for which accuracy is lower than some predefined failure criteria, which was in our case 2 mm. In addition, according to Markelj et al.18, the registration results are also portrayed using box-whiskers diagrams which are an efficient way to present the behavior of evaluated registration method.

3. RESULTS The MEAN and STD mTRE values of successful registrations, success rate (SR), capture range (CR) and average registration time for all five datasets, using two projections (LAT&AP and LAT&Oblique views, respectively) are given in Tab. 1. The accuracy, which was defined as the 50th (P50) and 95th (P95) percentile of the mTRE distributions for the chosen accumulative subintervals, is given in Tab. 2. Box-whiskers diagrams of registration results are shown in Fig. 3. The results were obtained on an Intel Core 2 Quad [email protected] GHz running Windows XP. A C++ implementation was used and no attempts to optimize the code for speed were made. According to the methodology proposed by van de Kraats20, the RGRB algorithm produced mean mTRE values between 0.07 and 0.59 mm, capture ranges between 6 and 11 mm and success rates between 61 and 88% using a failure threshold of 2 mm. Furthermore, using the evaluation methodology proposed by Markelj et al.18 the accuracy defined as 50th (P50) percentile of mTRE distribution was between 0.08 and 0.71 mm, while the accuracy defined as 95th (P95) percentile of mTRE distribution was 0.11 and 0.98 mm (Tab. 2). According to this methodology, the capture range was between 8 and 16 mm. For all datasets, except dataset 1, better results were obtained using orthogonal projections (LAT & AP). This can be explained by that in AP projection for dataset 1, significant overlapping of vessels exists, resulting in the large number of false gradient correspondences.



Table 1. The MEAN and STD mTRE values of successful registrations, success rates (SR), capture range (CR) and average registration time

Dataset View MEAN ± STD (mm) SR [%] CR

(mm) Time (s) 1 LAT&AP 0.20 ± 0.04 74.5 9 17.28 1 LAT&Oblique 0.17 ± 0.09 86.3 11 17.35 2 LAT&AP 0.13 ± 0.15 80.8 9 21.49 2 LAT&Oblique 0.59 ± 0.17 71.0 9 20.62 3 LAT&AP 0.12 ± 0.04 73.0 6 18.76 3 LAT&Oblique 0.35 ± 0.10 65.5 8 18.36 4 LAT&AP 0.07 ± 0.06 88.3 11 19.01 4 LAT&Oblique 0.29 ± 0.16 83.8 9 18.24 5 LAT&AP 0.19 ± 0.03 61.3 6 16.65 5 LAT&Oblique 0.32 ± 0.18 70.0 8 17.51

Table 2. Accuracy of registration method defined as 50th (P50) and the 95th (P95) percentile of the mTRE distribution for

different initial displacement intervals

Percentile View Initial displacements

(mm) Dataset

1 Dataset

2 Dataset

3 Dataset

4 Dataset

5 0-4 0.20 0.10 0.14 0.07 0.20 0-8 0.20 0.10 0.14 0.07 0.20

LAT&AP 0-12 0.20 0.10 0.14 0.07 0.20 0-16 0.21 0.11 0.14 0.07 0.21

P50 (mm) 0-20 0.21 0.12 0.15 0.08 0.21 0-4 0.21 0.59 0.36 0.30 0.24 0-8 0.21 0.59 0.37 0.30 0.25

LAT&Oblique 0-12 0.22 0.61 0.39 0.31 0.28 0-16 0.22 0.66 0.41 0.32 0.32

0-20 0.22 0.71 0.42 0.33 0.39

0-4 0.29 0.20 0.18 0.13 0.24 0-8 0.28 0.20 0.18 0.11 0.27

LAT&AP 0-12 9.35 0.98 7.48 0.12 16.03 0-16 16.35 11.33 18.73 0.19 23.44

P95 (mm) 0-20 20.41 14.34 42.22 15.39 33.46 0-4 0.25 0.88 0.48 0.53 0.64 0-8 0.25 0.86 0.47 0.53 0.65

LAT&Oblique 0-12 0.32 33.09 41.18 0.59 21.33 0-16 18.77 40.04 47.74 24.69 43.23

0-20 27.64 44.61 51.33 36.23 48.77



Figure 3. Box-whiskers diagrams of mTRE

4. DISCUSSION & CONCLUSION

In the last decade, calibration-based and intrinsic image-based methods were used to register 3D and 2D vascular images. The intrinsic-based methods applied to cerebral angiograms were either intensity-based or feature-based. To the best of our knowledge this is the first study where a gradient-based 3D/2D registration method was applied to cerebral angiograms. In this respect, a specially designed dataset with accurate gold standard was created. For evaluation of registration method two evaluation methodologies were adopted. The obtained results do not indicate that the selection of 2D views influences the algorithm performance significantly, although higher accuracy and reliability can be observed for perpendicular views. Accuracy below 1 mm according to both methodologies is acceptable for clinical purposes. The capture range of the RGRB method in case of vessels registration was found to be smaller than in case of vertebra registration.17,18 This can be explained with the fact that vessels are smaller objects, and especially by the vessels overlapping in 2D views which deteriorates algorithm robustness, increasing the number of false correspondences. The capture range and success rate could probably be increased by using 2D views with less vessel overlapping and only larger vessels for registration process. Worse results are also the consequence of a large number of vessels present in generated DRRs, which is not a real clinical situation. The generated DRRs contain large number of vessels than would be presented in DSA images because a CTA depicts all vessels in the brain as the contrast medium is present in both hemispheres. To overcome this problem, only one



hemisphere will be used for generating DSA-like DRRs in the future. Besides the great advantage of simulated 2D images, that gold standard is accurately known, numerous factors that are present in real images, like deformations, DSA background artifacts, movement of articulated structures, processing carrying out by the fluoroscopy set, errors in intrinsic parameters, etc., are not simulated. Therefore, the proposed dataset and evaluation methodology should be regarded as the best-case scenario and can serve only for preliminary evaluation of registration methods.18 A drawback of reconstruction based methods, like the RGRB method, is fact that they require at least 2 images to construct a 3D image or a 3D gradient field, like in RGRB. Obviously the reconstruction of the 3D image or a gradient field is better if more images are used. However, scenarios in which more than 2 images could be taken simultaneously are not feasible, as in the interventional suite there is only one C-arm which is kept in a fixed position or at most 2 C-arms acquiring 2D images at a certain angle apart. As expected, when compared to methods which use a single 2D view for registration, like in Hipwell et al.4, Byrne et al.5 and McLaughlin et al.6, the RGRB method outperforms them in terms of accuracy, capture range and computational time, as 2 views provide out of plane information. Based on the presented registration results, creation of a larger database using phantom and real clinical data is planned and more registration methods will be evaluated and compared using the proposed dataset in the future. The entire dataset including the gold standard, initial displacements from the gold standard and target points are available upon the request from the authors.

ACKNOWLEDGMENTS

This research was supported by the Ministry of Higher Education, Science and Technology, Republic of Slovenia, under grants L2-2023, L2-9758, J2-0716, J2-2246, and P2-0232.

REFERENCES

[1] P. Markelj, D. Tomaževič, B. Likar and F. Pernuš, “A review of 3D/2D registration methods for image-guided interventions,” Medical Image Analysis, in press (2010).

[2] E. Kerrien, M.-O. Berger, E. Maurincomme, L. Launay, R. Vaillant, and L. Picard, “Full automatic 3-D/2-D subtracted angiography registration,” in Proc. Medical Image Computing and Computer-Assisted intervention (MICCAI), 1679, pp. 664–671 (2002).

[3] A. C. S. Chung, I. William, M. Wells, A. Norbash, and W. E. L. Grimson, “Multi-modal image registration by minimising kullback-leibler distance,” in Proc. Int’l Conf. Medical Image Computingand Computer Assisted Intervention (MICCAI). New York: Springer, Lecture Notes in Computer Science, pp. 525–532 (2002).

[4] John H. Hipwell, Graeme P. Penney, Robert A. McLaughlin, Kawal Rhode, Paul Summers, Tim C. Cox, James V. Byrne, J. Alison Noble, and David J. Hawkes, “Intensity-Based 2-D-3-D Registration of Cerebral Angiograms”, IEEE Transactions on Medical Imaging, 22(11), (2003).

[5] J. V. Byrne, C. Colominas, J. Hipwell, T. Cox, J. A. Noble, G. P. Penney, and D. J. Hawkes, “Assessment of a technique for 2D-3D registration of cerebral intra-arterial angiography,” British Journal of Radiology, 77(914), pp. 123–128 (2004).

[6] R. A. McLaughlin, J. Hipwell, D. J. Hawkes, J. A. Noble, J. V. Byrne, and T. C. Cox, “A comparison of a similarity-based and a feature-based 2-D-3-D registration method for neurointerventional use,” IEEE Transactions on Medical Imaging, 24(8), pp. 1058–1066 (2005).

[7] H. M. Chan, A. C. S. Chung, S. C. H. Yu, and W. M. Wells III, “2D-3D Vascular Registration between Digital Subtraction Angiographic (DSA) And Magnetic Resonance Angiographic (MRA) images”, in IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp.708–711 (2004).

[8] Y. Kita, D. L. Wilson, and J. A. Noble, “Real-time registration of 3-D cerebral vessels to X-ray angiograms,” in Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI), 1496, pp. 1125–1133 (1998).

[9] J. Feldmar, N. Ayache, and F. Betting, “3D-2D projective registration of free-form curves and surfaces,” in Proc IEEE . Int. Conf. Comput.Vis., 20(23), pp. 549–556 (1995).

[10] J. Feldmar, N. Ayache, and F. Betting, “3-D–2-D projective registration of free-form curves and surfaces,” Comput. Vis. Image Understand., 65(3), pp. 403–424 (1997).



[11] A. Liu, E. Bullitt, and S. Pizer, “3D/2D registration via skeletal near projective invariance in tubular objects,” in Proceedings International Confernce Medical Image Computing and Computer Assisted Intervention (MICCAI), 1496, pp. 952–963 (1998).

[12] Hari Sundar, Ali Khamene, Chenyang Xu, Frank Sauer, Christos Davatzikos, “A novel 2D-3D registration algorithm for aligning fluoro images with 3D pre-op CT/MR images”, Proc. SPIE 6141, pp. 760-766 (2006).

[13] M. Söderman, D. Babic, R. Homan, and T. Andersson, “3D Roadmap in Neuroangiography: Technique and Clinical Interest,” Neuroradiology, 47(10), pp. 735-740 (2005).

[14] D. Ruijters, D. Babic, R. Homan, P. Mielekamp, B. M. ter Haar Romeny, and P. Suetens, “3D multimodality roadmapping in neuroangiography“ Proc. SPIE 6509, pp. 65091F (2007).

[15] D. Ruijters, “Multi-modal image fusion during minimally invasive treatment”, PhD thesis, Katholieke Universiteit Leuven and the University of Technology Eindhoven (TU/e), (2010).

[16] Gendrin C, Spörk J, Bloch C, Pawiro S, Weber C, Figl M, Markelj P, Pernuš F, Georg D, Bergmann H, and Birkfellner W., “Towards real-time 2D/3D registration for organ motion monitoring in image-guided radiation therapy”, Medical Imaging 2010: Visualization, Image-Guided Procedures, and Modeling (KH Wong and MI Miga, eds.), 7625, pp. 762512 (2010).

[17] P. Markelj, D. Tomaževič, F. Pernuš, and B. Likar, “Robust gradient-based 3-D/2-D registration of CT and MR to X-ray images,” IEEE Trans. Med. Imaging, 27(12), pp. 1704–1714 (2008).

[18] P. Markelj, F. Pernuš and B. Likar, “Standardized evaluation methodology for 3D/2D registration based on the Visible Human data set,” Medical Physics, 37, pp. 4643-4647 (2010).

[19] Graeme Patrick Penney, “Registration of Tomographic Images to X-ray Projections for Use in Image Guided Interventions”, PhD thesis, University College London, CISG, Division of Radiological Sciences, Guy's Hospital, King's College London, London SE1 9RT England, pp. 68-71 (2000).

[20] E. B. van de Kraats, G. P. Penney, D. Tomaževič, T. van Walsum, and W. J. Niessen, “Standardized evaluation methodology for 2-D-3-D registration,” IEEE Transactions on Medical Imaging, 24(9), pp. 1177–1189 (2005).



Documents

SPIE Proceedings [SPIE SPIE Medical Imaging - Lake Buena Vista, Florida (Saturday 12 February 2011)] Medical Imaging 2011: Image Processing - Gradient-based 3D-2D registration of cerebral