Deep Vesselness Measure from Scale-spaceAnalysis of Hessian Matrix Eigenvalues

Ricardo J. Araujo1,2[0000−0001−7222−196X],Jaime S. Cardoso1,3[0000−0002−3760−2473], and

Helder P. Oliveira1,2[0000−0002−6193−8540]

1 INESC TEC, Porto, Portugal2 Faculdade de Ciencias da Universidade do Porto, Porto, Portugal

3 Faculdade de Engenharia da Universidade do Porto, Porto, Portugal

Abstract. The enhancement of tubular structures such as vessels inmedical images has been addressed in the past, aiming for easier ex-traction and or visualization of such structures by professionals. Someliterature methodologies propose vesselness measures whose design is mo-tivated by local properties of vascular networks and how these influencethe eigenvalues of the Hessian matrix. However, past work fails to com-bine properly the scale-space and neighborhood information, thus lead-ing to the proposal of suboptimal vesselness measures. In this paper,we show that a shallow convolutional neural network is able to learnmore optimal embedding spaces from the eigenvalue analysis at differentscales, thus leading to a stronger vessel enhancement. Additionally, wealso show that such a system maintains one of the biggest advantages ofHessian-based vesselness measures, which is the robustness to data withvarying statistics.

Keywords: Blood Vessel Enhancement · Computer Vision ·Deep Learn-ing.

1 Introduction

Blood vessels have high relevance in many clinical practices and diseases diag-noses. A few examples are the diagnosis of diabetic retinopathy, eligibility forliver transplant, and detection of aneurysms. Imaging data is quickly generated,especially in screening programs, and practitioners often become overwhelmedby the volume of data to analyze. Thus, some computer vision researchers nat-urally started targeting the automation of blood vessel analysis, and the firstadvances were already published by the end of the past century.

The first methodologies to emerge encoded prior knowledge of blood vascularnetworks in different ways, either in the design of probing filters [1], model fit-ting [2], or tracing algorithms that iteratively estimate a new point in the centeraxis of the vessel [3]. When datasets containing retinal fundus images started tobecome publicly available [4, 5], an important branch of algorithms emerged, theones using supervision in order to find more complex mappings from the original

2 R. J. Araujo et al.

data to a vessel probability map [6, 7]. Recently, deep learning methodologieshave also been applied to this scenario, raising the performance of automatedblood vessel analysis to a new level [8, 9]. Nonetheless, good generalization ofsuch methodologies to data coming from different distributions is still hard toachieve, a scenario where unsupervised methods, which are based on strong butintuitive priors, frequently work better.

Vessel enhancement based on Eigen decomposition of the Hessian matrix isone of the most widely used enhancement processes, due to natural formulationfor both 2D and 3D scenarios, good generalization to different data distributions,and also high noise suppression capabilities. Several hand-designed metrics com-bining the eigenvalue information were proposed in the past. Frangi’s vesselnessmeasure [10] is one of the most used. Recently, Jerman et al. [11] proposed avesselness measure that tries to correct some deficiencies of past ones, such aslow responses at aneurysms due to suppression of rounded structures, and poorenhancement of bifurcations due to local deviation from a piecewise linear struc-ture, which is the profile being targeted in other hand-designed metrics. Eventhough these metrics rely on strong prior knowledge of the problem, they usuallyend missing much information due to how they combine multiscale information,thus being suboptimal.

In this work, we aim at finding a more complex and optimal vesselness mea-sure mapping the eigenvalue information at different scales into the final vesselenhanced image, by using a Deep Neural Network (DNN). By using supervision,our goal is to obtain a deep vesselness measure that combines the advantagesof both deep learning methodologies (finding deep complex functions) and usingprior knowledge (increased robustness to data coming from different distribu-tions). Recent research considered the implementation of Frangi’s algorithm asa neural network, by carefully initialization of its weights [12]. The authors thenused supervision to update weights responsible for the computation of the Hes-sian, and coefficients controlling the relevance of the different eigenvalue ratiosused in Frangi’s vesselness. Note, however, that the first option strongly relaxesthe use of prior knowledge, as the network is able to learn features completelydifferent from the Hessian, thus regularization may be lost. Additionally, theauthors do not consider exploring other functions mapping the Eigen maps tothe final vesselness, being restricted to the use of the maximum operator acrossthe responses obtained at different scales, which is suboptimal.

The paper is structured as follows: this Section introduced the relevanceof vessel enhancement and our contribution to this problem; Section 2 brieflydescribes traditional frameworks relying on the multiscale eigen-analysis of theHessian matrix, and how we propose to obtain a deep vesselness measure throughsupervision; in Section 3 we explain the conducted experiments, and present anddiscuss results; finally, Section 4 concludes the paper and points some directionsfor future work.

Deep vesselness from scale-space analysis of the Hessian 3

2 Methodology

Vessel enhancement methodologies based on the Eigen decomposition of the Hes-sian explore the local intensity curvature of images. Such analysis is performedat different scales in order to find structures of different sizes. We start by de-scribing in detail the pipeline these approaches follow. Then, we point out itsmain limitations and propose a novel design by switching its final part by aDNN.

2.1 Eigen decomposition of the Hessian for Vessel Enhancement

Given an D-dimensional image I, the type of structure present at a given locationx = (x1, x2, . . . , xD) may be inferred through the analysis of the Hessian matrixat x, a D ×D matrix encoding the second order derivatives of the intensities:

Hij(x, σ) = σγI(x) ∗ ∂2G(x, σ)

∂xi∂xj, i, j = 1, . . . , D (1)

where G is a D-variate Gaussian, σ denotes its standard deviation, dictatingthe scale at which the image is being analyzed, γ is a constant that normalizesresponses obtained at different scales, allowing a fair comparison [13], and ∗represents the convolution operation.

The Eigen analysis of H(x, σ) produces D eigenvectors representing the prin-cipal directions that decompose the second order structure of the image at x.Each of them has an eigenvalue associated, a scalar whose magnitude and signalallow to characterize the intensity curvature along the corresponding eigenvec-tor. From now on, let us consider that the Eigen decomposition of the Hessianat a location x:

L(x, σ) = eig(H(x, σ)) (2)

produces a set of eigenvalues λ1, λ2, . . . , λD, such that, |λ1| ≤ |λ2| ≤ . . . ≤ |λD|.These provide a concise description of the local geometry at x, allowing thedesign of functions that respond to particular geometries.

In this context, a vesselness measure is any function f of the eigenvalues thatis suited for the enhancement of blood vessels:

V(x, σ) = f(L(x, σ)) (3)

A common assumption on vessel geometry is it being piecewise linear, that is,locally, it resembles a cylinder. As in this work we deal only with 2D images, werestrict the following discussion to this scenario. Nonetheless, extension to 3D isstraightforward and addressed in the aforementioned works. The most commonlyused vesselness measure is Frangi’s [10], which for the 2D case is given by:

VF =

0 if λ2 ≤ 0,

exp(− R2

B

2β2

)·(

1− exp(− S2

2c2

))otherwise

(4)

4 R. J. Araujo et al.

where RB = |λ1|/|λ2| is a ratio measuring local similarity to a blob througheccentricity of the second order ellipsis, S =

√λ21 + λ22 is the amount of local

structure, and β and c, control the relevance of those quantities, respectively.This measure assumes that vessels are darker than the background, but invertingthe conditions of Eq. (4) is enough to detect brighter vessels instead.

In the case of Jerman’s vesselness [11], assumptions are slightly relaxed inorder to better model aneurysms and bifurcations:

VJ =

0 if λ2 ≤ 0 ∨ λp ≤ 0,

1 if λ2 ≥ λp/2 > 0,

λ22 · (λp − λ2) ·[

3λ2+λp

]3otherwise

(5)

where λp is a regularized eigenvalue for ensuring that robustness to noise isachieved in regions with uniform intensity.

Regardless of the used vesselness measure, the final enhanced image, V , isobtained by combining the responses obtained at different scales σ, through apixelwise maximum operation:

V (x) = maxσ1,...,σn

V(x, σ) (6)

The traditional pipeline here described is represented in Fig. 1.

I

Eq. (1)

H(σ1) Eq. (2)

H(σ2) Eq. (2)

H(σn) Eq. (2)

L(σ1) Eq. (3)

L(σ2) Eq. (3)

L(σn) Eq. (3)

V(σ1)

V(σ2)

V(σn)

Eq. (6)

V

Fig. 1. Multiscale pipeline of traditional vessel enhancement methodologies, where ndenotes the number of scales. Pixelwise Hessian matrix and corresponding eigenvaluesare represented as feature vectors for convenience. In this paper, we propose a designusing a DNN replacing the functions delimited by dashed lines.

2.2 Deep Vesselness Measure

In this work, we replace hand designed vesselness measures (see region delimitedby dashed lines in Fig. 1) by a DNN. Our motivation is twofold. First, mapping

Deep vesselness from scale-space analysis of the Hessian 5

eigenvalue information into a vessel probability (Eq. (3)) through hand-designedfunctions, despite being based on prior intuition, is likely suboptimal. Second,combining the responses at different scales by a pixel-wise maximum operation(Eq. (6)) discards much of the information encoded at all scales and fails to cap-ture high-level local information that may be useful in challenging regions. Thus,we replace those functions by a neural network having as input a concatenationof the eigenvalue description, and as output a vessel probability map, as repre-sented in Fig. 2. We use label supervision in order to update its weights, aimingto obtain a more optimal deep vesselness measure, which is still regularized aswe only provide the scale-space eigenvalue description.

Eq. (2)

L(σ1) L(σ2) L(σn)

merge

DNN

V

Fig. 2. Proposed pipeline for vessel enhancement. A DNN learns a more complex ves-selness measure from the eigenvalue information.

Neural network considerations We model our DNN as a Fully ConvolutionalNetwork (FCN) [14], such that images seen at train and test phases may havedifferent sizes. This also allows us to train our model in small patches of bloodvessel images, and still later obtain predictions for entire images at a singlepass. This may be relevant due to memory issues and to avoid training withunnecessary data, such as the black regions in retinal fundus images. In this work,we consider patch-based training, which is not expected to affect the performanceof a FCN.

Batch normalization [15] is especially useful in very deep networks, which isnot the case we seek here. Additionally, care must be taken when the statisticsof the data are not the same in the train and test sets. Such difference may be aresult of performing patch-based training, where, for example, entire images ofthe retina have different statistics than small patches that were just taken fromthe retinal fundus area. It may also naturally arise from training and testing indifferent datasets. This last scenario is very relevant as one of the main advan-tages of traditional Hessian-based methods is their good generalization to otherdistributions of data. Thus, we did not consider batch normalization.

Recent findings [8] seem to support that reducing space resolution via maxpooling or strided convolution does not improve the performance of networkstrying to capture small details, as is the case of blood vessels. Preliminary ex-periments that we conducted support this, such that slightly increasing the kerneldimension and keeping spatial resolution equal across the entire network proved

6 R. J. Araujo et al.

to be more effective. Having features already encoding neighborhood informationas the input of our neural network may also contribute to learn interesting deepvesselness measures by only looking at a relatively small neighborhood. Eventhen, we found that using dilated convolution [16] in the intermediate layersimproved the performance of the system.

An ideal vessel enhancement algorithm would output probability of 1 forevery pixel belonging to a vessel and probability of 0 otherwise. However, notethat, for an adequate scale σ, and when analyzing pixels over the cross sectionof a vessel, it is expected that the Eigen decomposition of the center pixel isthe one matching better the Eigen description of an ideal vessel (|λ1| << |λ2|,in the 2D case). This is the reason why vesselness measures such as Frangi’senhance more the central regions of vessels. Even though a DNN is able tofind complex relations between eigenvalues and thus learn effectively when hardlabels are provided, in preliminary experiments, we found that using soft labels(obtained by blurring the hard labels with a standard normal distribution) washelpful. Nevertheless, as will be shown in Section 3, our design is still capable ofenhancing the peripheral regions of vessels extremely well.

The Rectified Linear Unit function was used as an activation function through-out the network and the last non-linearity was a Sigmoid function. Regardingthe loss function, we considered the binary cross entropy, which is also adequatewhen soft labels are given. The Adam optimizer [17] was used to update theweights. Other state-of-the-art considerations such as Dropout were also tested.The final FCN design is represented in Fig. 3. More information on the tuningprocedure is given in Section 3.

512 256 128 64 32

1-dilated conv., 5× 5 kernel

ReLU activation

conv., 5× 5 kernel

Sigmoid activation

Fig. 3. Fully Convolutional design used in the experiments after tuning the network ar-chitecture. The first set of features is obtained by doing convolution over the eigenvaluepile of features with 5× 5 kernels and no dilation.

3 Experiments and Results

In this Section, we start by briefly describing the datasets we use in our ex-periments. Afterwards, we present the setting we followed to tune the networkarchitecture and its hyperparameters, finally reaching the design presented in

Deep vesselness from scale-space analysis of the Hessian 7

Fig. 3. Finally, we detail the different experiments taken into account to showthe properties of the proposed methodology and present their results.

3.1 Datasets

To the best of our knowledge, blood vessel 2D imaging datasets containing theground truth vessel masks only exist for the retinal case. This means that su-pervision may only be done recurring to this type of vascular network. To con-duct the experiments, we resorted to 4 complete retinal datasets, DRIVE [5],STARE [4], CHASEDB1 [18], and IOSTAR [19].

DRIVE is a result of a diabetic retinopathy screening program conducted inThe Netherlands comprising 40 images of size 584×565 split in the same pro-portion into train and test sets, with 7 of them showing signs of early diabeticretinopathy. A Canon CR5 non-mydriatic 3CCD camera with a 45◦ field of viewwas used. STARE is a dataset containing 20 images of size 605×700, half ofwhich showing pathology. They were obtained with a TopCon TRV-50 funduscamera. CHASEDB1 was compiled after the Child Heart and Health Study inEngland. It comprises 28 images of size 960×999, where central reflex is particu-larly abundant. IOSTAR includes 30 images of size 1024×1024 and a field of viewof 45◦ acquired with the EasyScan camera from i-Optics B.V., The Netherlands.

3.2 Implementation Details

Having in mind the considerations discussed in Section 2.2, we tuned the archi-tecture and hyper-parameters using the DRIVE dataset. We randomly set asidethree images from DRIVE’s training set for validation purposes and used the re-maining ones for training different model configurations. In this work we did notconduct any preprocessing step, we just selected the green channel informationand normalized it to the range [0, 1]. We considered σ ∈ [1, 11] with steps of 2. Ateach training epoch, we give the models 300 batches of 8 patches of size 64× 64.A total of 100 epochs were conducted. These values were empirically found to beappropriate in preliminary experiments but their variation do not yield signifi-cant performance alterations. Patches were randomly extracted from the field ofview region of images. We considered data augmentation via random vertical orhorizontal flipping, and rotations in the range [−π/2, π/2]. The parameters ofthe Adam optimizer were initialized as described in [17]. The best performancein the validation set was obtained when using the design represented in Fig. 3.

With the exception of DRIVE, the datasets do not have a prior split intotrain and test sets. Thus, we consider the first 10 images of each for trainingpurposes and the remaining are set aside for testing. According to the experimentwe conduct, different sets are used for training and testing but details will beprovided as necessary. The training procedure is conducted as described beforefor network and hyperparameter tuning, but this time all the available trainingdata is used. Frangi’s and Jerman’s enhancement responses were obtained usingtheir Matlab implementations.

8 R. J. Araujo et al.

3.3 Results and Discussion

We start by considering the scenario where we train our model using a givendataset and test on the images set aside from the same dataset. With this ex-periment, we aim to show that for a specific distribution of data, it is possibleto use deep learning to obtain more complex and optimal vesselness measuresfor such data. The ROC curves of our method, are shown, along with the onesfrom the baselines, in Fig. 4.

Fig. 4. ROC curve of the proposed methodology when trained and tested in the samedataset. The ROC curves of the baseline methods are presented for comparison.

This shows that, when we target specific data distributions, we are able toobtain more optimal vesselness functions than the traditional ones. This wasexpected, but note that traditional methods do not target specific dataset dis-tributions, but instead a representation that generalizes well. Obviously, themore interesting scenario is to analyze what occurs when we use the proposedmethodology to enhance blood vessels in images coming from datasets that werenot accessed during training. Thus, we now consider the scenario where we setthe test dataset aside and train using the remaining ones. The ROC curves of thesystem in such conditions are again compared against the baselines in Fig. 5. Itis possible to conclude that our system is indeed capable of generalizing well todata coming from other distributions than the ones available during training. Forvery high false positive ratios, Jerman’s vesselness occasionally achieves highertrue positive ratio, however note that such region is not ideal for enhancementfunctions as it already comprises a large portion of noise. This is clearly seen in

Deep vesselness from scale-space analysis of the Hessian 9

Fig. 5. ROC curve of the proposed methodology when trained and tested in differentdatasets. The ROC curves of the baseline methods are presented for comparison.

Fig. 7, where our method proves to be much more robust to noise than Jerman’sone.

Finally, we compare the generalization capability of the proposed methodagainst the well-established Unet [20] for biomedical image segmentation. Suchnetwork has much more capacity and has increased flexibility as it is not re-stricted to use a given set of features, as we do in the proposed methodol-ogy. Instead, it creates representations from the image itself. Fig. 6 shows theROC curves of both methods. Both methodologies have similar performance inSTARE and IOSTAR but the proposed approach generalized better for DRIVEand CHASEDB1. This shows that regularizing deep neural networks, as we doin this work, may be a proper way of achieving designs that generalize better,even using less parameters. We provide visual results of our method and thebaselines in Fig. 7.

4 Conclusion and Future work

In this work we extended traditional Hessian-based methodologies for the en-hancement of blood vessels in medical images. By replacing hand-design func-tions mapping eigenvalue descriptions to the final output with a DNN, we wereable to learn more optimal functions than traditional algorithms. At the sametime, when comparing with DNNs which are fed the images instead of an eigen-value description, our methodology generalized better to data coming from distri-butions other than the ones used at training. This shows that our methodology

10 R. J. Araujo et al.

Fig. 6. ROC curves of the proposed methodology and a regular Unet, when trainedand tested in different datasets.

continues to embed significant prior knowledge, thus helping to achieve goodabstraction of what a blood vessel is.

Regarding future work, we would like to extend the framework to 3D data,possibly exploring synthetic datasets during the training procedure, as it be-comes trivial to obtain the ground truth.

Acknowledgments

This work was financed by National Funds through the Portuguese fundingagency, FCT - Fundacao para a Ciencia e a Tecnologia within PhD grant numberSFRH/BD/126224/2016 and within project UID/EEA/50014/2019.

References

1. Chaudhuri, S., Chatterjee, S., Katz, N., Goldbaum, M.: Detection of blood vessels inretinal images using two-dimensional matched filters. IEEE Transactions on MedicalImaging 8(3), 263–269 (1989)

2. Niessen, W.J., van Bemmel, C.M., Frangi, A.E., Siers, M.J.A., Wink, O.: Model-based segmentation of cardiac and vascular images. In: Proceedings IEEE Interna-tional Symposium on Biomedical Imaging, pp. 22–25. IEEE, Washington, DC, USA(2002)

3. Sun, Y.: Automated identification of vessel contours in coronary arteriograms byan adaptive tracking algorithm. IEEE Transactions on Medical Imaging 8(1), 78–88(1989)

Deep vesselness from scale-space analysis of the Hessian 11

Fig. 7. From top to bottom, example retinal fundus images, ground truth vessel masks,and corresponding enhancements by, respectively, Frangi’s, Jerman’s, Unet, and pro-posed vesselness. From left to right, image from DRIVE, STARE, CHASEDB1, andIOSTAR. Concerning the Unet and proposed method, training was conducted in alldatasets, except the testing set. Frangi’s vesselness was rescaled for visualization pur-poses, since the signal at narrow vessels is usually small.

12 R. J. Araujo et al.

4. Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal im-ages by piece-wise threshold probing of a matched filter response. IEEE Transactionson Medical Imaging 19(3), 203–210 (2000)

5. Staal, J., Abramoff, M.D., Niemeijer, M.: Ridge-based vessel segmentation in colorimages of the retina. IEEE Transactions on Medical Imaging 23(4), 501–509 (2004)

6. Lupascu, C.A., Tegolo, D., Trucco, E.: FABC: retinal vessel segmentation usingAdaBoost. IEEE Transactions on Information Technology in Biomedicine 14(5),1267–1274 (2010)

7. Fraz, M.M., Remagnino, P., Hoppe, A., Uyyanonvara, B., Rudnicka, A.R., Owen,C.G., Barman, S.A.: An ensemble classification-based approach applied to retinalblood vessel segmentation. IEEE Transactions on Biomedical Engineering 59(9),2538–2548 (2012)

8. Liskowski, P., Krawiec, K.: Segmenting retinal blood vessels with deep neural net-works. IEEE Transactions on Medical Imaging 35(11), 2369–2380 (2016)

9. Hu, K., Zhang, Z., Niu, X., Zhang, Y., Cao, C., Xiao, F., Gao, X.: Retinal vesselsegmentation of color fundus images using multiscale convolutional neural networkwith an improved cross-entropy loss function. Neurocomputing 309(2), 179–191(2018)

10. Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel en-hancement filtering. In: Wells, W.M., Colchester, A., Delp, S. (eds.) Medical ImageComputing and Computer-Assisted Intervention 1998, LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056195

11. Jerman, T., Pernus, F., Likar, B., Spiclin, Z.: Enhancement of vascular structuresin 3D and 2D angiographic images. IEEE Transactions on Medical Imaging 35(9),2107–2118 (2016)

12. Fu, W., Breininger, K., Wurfl, T., Ravikumar, N., Schaffert, R., Maier, A.:Frangi-net: a neural network approach to vessel segmentation. arXiv preprintarXiv:1711.03345 (2017)

13. Lindeberg, T.: Edge detection and ridge detection with automatic scale selection.International journal of computer vision 30(2), 117–156 (1998)

14. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semanticsegmentation. In: Proceedings of the IEEE conference on computer vision, pp. 3431–3440. IEEE, Chile (2015)

15. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training byreducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

16. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXivpreprint arXiv:1511.07122 (2015)

17. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprintarXiv:1412.6980 (2014)

18. Owen, C.G., Rudnicka, A.R., Mullen, R., Barman, S.A., Monekosse, D., Whincup,P.H., Ng, J., Paterson, C.: Measuring retinal vessel tortuosity in 10-year-old children:validation of the computer-assisted image analysis of the retina (caiar) program.Investigative ophthalmology & visual science 50(5), 2004–2010 (2009)

19. Abbasi-Sureshjani, S., Smit-Ockeloen, I., Zhang, J., Romeny, B.T.H.: Biologically-inspired supervised vasculature segmentation in SLO retinal fundus images. In:International Conference Image Analysis and Recognition, pp. 325–334. Springer,Cham, Canada (2015)

20. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomed-ical image segmentation. In International Conference on Medical image computingand computer-assisted intervention, pp. 234-241. Springer, Cham (2015)