1

Sparse representation and variational methods in retinal imageprocessing

J. Dobrosotskaya,1, M. Ehler1,2, E. King1,2 , R. Bonner2, W. Czaja1.

1Norbert Wiener Center for Harmonic Analysis and ApplicationsDepartment of Mathematics

University of MarylandCollege Park, MD 20742

2 National Institutes of Health,Eunice Kennedy Shriver National Institute of Child Health and Human Development,

PPB/LIMB/SMB, Bethesda, MD, 20892

Abstract - Relations between different types ofcameras used for retinal imaging were studied withthe purpose of improving the quantitative precisionof the imaging data (used for diagnostics and medicalresearch). Based on the differences in visual qual-ity and quantitative parameters, we designed analyt-ical models of the effects that cameras introduce intothe retinal data and described possible ways of dig-ital post-processing. Some processing tasks involvedetection and separation of features (such as the reti-nal microvessels) prior to subsequent analysis of un-derlying retinal pathology. Mathematical techniquesfor feature detection and inpainting are variational,implemented via numerically stable gradient descentschemes. Other tasks involve the estimates of trans-lation - invariant sparse image coefficients allowingto separate the background and significant scales ofthe image from the texture-like auxiliary information.The above techniques are based on the recent workon the wavelet Ginzburg-Landau energy and meth-ods of adaptive thresholding of the stationary wavelettransform coefficients. We consider algorithms withpartial specialist supervision and deliberate choice ofprocessing methods for different eye areas as well asseparate processing of healthy vs. pathological eyedata.

Keywords - Retinal imaging, variational method,edge detection, wavelet.

I Introduction

Ophthalmologists often rely on retinal imaging to diag-nose, detect, and follow disease progression. Classifyingearly stages of age-related macular degeneration (AMD),for instance, relies on qualitative and quantitative anal-yses of the data from the confocal scanning laser oph-thalmoscope (cSLO) and standard fundus camera images[13, 7, 9]. Decrease in macular pigment has been identi-fied as a risk factor for AMD, and observing its distribu-tion over time would allow to make further conclusionsabout natural and pathological dynamics of macular pig-ment changes. However, due to inter and cross modal-ity variations, better quantitative measurements are stillneeded [12].

Macular pigment measurements based on two-wavelength autofluorescence images have been introduced

by Delori et al. [8]. To compute the macular pigmentmap, we either pair a blue cSLO image (488nm excita-tion, > 500nm emission) with a yellow standard fundusimage (520-600nm excitation, > 600nm emission) or, ifavailable, we pair blue (460− 500nm excitation) and yel-low standard fundus images [9]. While image artifactsand background components cancel out (to some extent)in inter-modality pairings, they are emphasized in cross-modality pairings and introduce significant errors in themacular pigment maps. To trace the dynamics of macularpigment and other chromophore changes in retrospectivestudies, one must compare cSLO autofluorescence withstandard fundus autofluorescence, and quantitative mea-surements require image pre-processing to reduce imageartifacts, non-uniform illumination profiles, and contrastdifferences between modalities.

This paper addresses two retinal image analysis prob-lems. First, we correct autofluorescence images fromcross-modalities (cSLO, standard fundus camera) to beused in the same two-wavelength computations of macu-lar pigment maps. Blood vessels are detected and maskedto facilitate quantitative analysis. Secondly, we extracta binary map of the retinal vascular system from cSLOimages. Those two techniques share the common featureof using the stationary wavelet transform for translation-invariant operations. Since the microvascular system inthe image is relatively contrasting, some special featuresof the wavelet decomposition of almost-binary imagesprovide the apparatus for the detection of the blood ves-sels.

II Adaptive translation invariant wavelet thresholding

First, we will introduce the adaptive thresholding tech-nique that was designed in the attempt to perform min-imal changes needed to obtain a more reliable pigmentmap using images from different digital sources. The needfor non-uniform, “relative” thresholding arises from thenecessity to automatize the procedure, as well as the needto use different thresholds for edges in separate directions.We will use the image decomposition via the translationinvariant (stationary) wavelet transform to perform theadaptive corrections. The involved wavelet function ψis assumed to be sufficiently regular and compactly sup-ported. The respective 2D wavelet basis is assumed to be

2

separable, so that the wavelet coefficients can be catego-rized into directions H,V and D (horizontal, vertical anddiagonal respectively).

Relative thresholding in Besov spaces

If we consider a characteristic function of some mea-surable set u = χE ∈ L2(R2), then, due to ψ having acompact support and χE being locally homogeneous ofdegree 0, we get:

〈u, ψj,k〉 2p〈u, ψj+p,2pk〉.

The decrease in the coefficient values as the scale in-creases and the respective increase of the integration do-main for the translation parameter k makes the the stan-dard deviation change as O(2−2j) while the scale j in-creases. If the boundary of the set E is piecewise smoothone can expect the coefficients of the wavelet modes sup-ported within the same distance form the boundary tobe of the same order of magnitude.

Let us define the thresholding rule for each scale j ofthe wavelet decomposition. In order to do that, considerthe expected value(the mean) Mj and the Dj for theset of all wavelet coefficients within one fixed directioncj,k = 〈ψj,k, u〉 at a fixed scale j ∈ N:

Mj = 2−Nj2J−1∑ki=0

cj,k

D2j = 2−Nj

2J−1∑ki=0

(cj,k −Mj)2 = 2−Nj2J−1∑ki=0

c2j,k −M2j

We define the relative significance threshold at scalej as

τj = C22jDj , C = 2−2Jmax

where Jmax, is the the thresholding scale, which is definedas the maximum level of wavelet decomposition J (i.e.the image resolution ) or as the scale that stores the mostsignificant information (visually significant, or defined bya specific application or the given data quality).

In this manner, we define the following unified cri-terium S for the relative wavelet thresholding.

A mode ψj,~k is chosen to be relatively significant fora function u within a chosen direction (H,V, or D), i.e.bu(j, k) = 1 if and only if it differs from the mean co-efficient value at the scale j by more than the standarddeviation times the dyadic scaling multiple:

|〈u, ψj,k〉 −Mj | ≥ C22jDj .

Thus, the relative thresholding leaves intact those coeffi-cients that differ sufficiently (as much as in the binary im-age case) from the mean of all coefficients at this waveletscale.

Numerical tests

Fig. 1-2 show the results of the numerical tests that wereperformed using a cSLO and a yellow standard fundusimage. The cSLO image was modified via the adaptive

wavelet thresholding procedure defined above in order tocompensate for the contrast differences specifically nearthe blood vessels (Fig. 1(c)). The result of the MPMcomputation without any additional correction is shownin Fig.2(a), the result of the MPM computation usingthresholded cSLO image is shown in Fig.2(b). One cansee that while unevenness of the MPM originating fromsources other than blood vessels was retained, the vessel-related artifacts that are visible Fig.2(a) are almost com-pletely eliminated.

(a) (b) (c)

Figure 1: Images: (a) blue cSLO; (b) yellow fundus; (c) corrected blue cSLO.

(a)

Figure 2: (a) MPM computed without correction, (b) MPM computed usingthresholded blue cSLO image.

III Semi-supervised contour detection using the wavelet

Ginzburg-Landau energy

The wavelet Ginzburg-Landau energy(WGL), intro-duced in [3, 4] was shown effective in variational methodsfor various imaging problems. Here we describe one ofits applications which is essentially a variational methodfor detecting of the microvascular elements within reti-nal images. As was mentioned before, blood vessels are acompletely different structure for the purposes of the reti-nal image analysis, and, in particular, should be excludedfrom the computation of the macular pigment map in or-der to improve the precision of the latter. Here, onceagain, we will use the assumption that those are of suf-ficient contrast with respect to the background, i.e. canbe treated as almost-binary elements of the image. Semi-supervision required here involves the manual choice ofseveral pixel areas that belong to the blood vessels priorto automatized computations that detect the rest of themicrovascular system.

Wavelet Ginzburg-Landau energy

Fourier analysis provides many elegant approaches to dif-ferential operators and related tools in PDE-based im-age processing. In the design of the wavelet Ginzburg-Landau energy, a more localized basis than the Fourierone was used in the context of variational methods based

3

on diffuse interfaces([5] and more). Such construction isnaturally consonant with image processing applicationsinvolving binary images and treat(recover) respective bi-nary values as two equilibria of some system [1],[6].

WGL originated from the idea of designing new typesof pseudo-differential energy functionals that inherit im-portant properties of the ones involving derivatives, butleave out the computational drawbacks associated withthe discrete differentiation. The key idea in [3] com-bined the basic geometric framework of diffuse interfacemethods with advantages of the well-localized and inher-ently multiscale wavelet operators. The Total Variation(TV) seminorm was proven to be a natural and efficientmeasure of image regularity [2],[11]. To avoid computa-tional challenges related to equations for minimizers ofthis norm, one can reformulate the problem using thephase-field method and approximate the TV functional(in the Γ sense). The Ginzburg-Landau (GL) functional,

GL(u) =ε

2

∫|∇u(x)|2dx+

14ε

∫W (u)dx,

W (u) = u2(u− 1)2

is a diffuse interface approximation to the Total Varia-tion functional

∫|∇u(x)|dx in the case of binary images

[10]. GL energy is used in modeling of a vast variety ofphenomena including the second-order phase transitions.However, if used in signal processing applications, diffuseinterface models tend to produce results that are over-smoothed comparing to the optimal output. In the newmodel the H1 seminorm

∫|∇u(x)|2dx is replaced with a

Besov seminorm (or Besov-type seminorm defined using0-regular wavelets). This allows to construct a methodwith properties similar to those of the PDE-based meth-ods but without as much diffuse interface scale blur.

The “wavelet Laplace operator ” was defined by hav-ing the wavelet basis functions as eigenfunctions, and act-ing on those in the same “scale - proportional” manneras the Laplace operator acts on the Fourier basis. Givenan orthonormal wavelet ψ the “wavelet Laplacian” of anyu ∈ L2(R) is formally defned as

∆wu = −+∞∑j=0

22j

∫〈f, ψj,κ〉ψj,κdκ, ψj,κ = 2jψ(2jx− κ).

Then the “wavelet Allen-Cahn” equation ut = ε∆wu −1εW (u) describes the gradient descent in the problemof minimizing the Wavelet Ginzburg-Landau (WGL) en-ergy:

WGL(u) :=ε

2|u|2B +

14ε

∫W (u)dx,

|u|2B =ε

2

+∞∑j=0

22j

∫|〈f, ψj,κ〉|2dκ (3)

is the square of the Besov 1-2-2 (translation-invariant)semi-norm if the wavelet ψ is r-regular, r ≥ 2.

WGL functionals are inherently multiscale and takeadvantage of simultaneous space and frequency localiza-tion, thus allowing much sharper minimizer transitions

for the same values of the interface parameters compar-ing to the classical GL energy.

WGL minimization can be performed via the gradientdescent method. The latter problem is equivalent to solv-ing the following ODE with a sufficiently regular initialcondition u(x, 0) = u0(x):

ut = ε∆wu−1εW ′(u) (GD)

The above problem is well-posed: it has a unique solutionthat exists globally in time and converges to a steadystate as t → ∞. The steady state solution is infinitelysmooth provided that wavelet ψ used in the constructionof the energy has sufficient regularity.

Modified WGL in the variational formulation of the seg-

mentation problem.

Our segmentation model involves minimizing the sum ofWGL (as a regularizer) and the L2 spatial and the B1

2,2

edge-preserving forcing terms.

E(u) = WGL(u)+µs2‖(u−f)χΩ‖2L2

+µw2|PrΛ(u−uorig)|2B ,

(MWGL)χΩ and χΛ are masks in the spatial and wavelet domainsrespectively, and µs and µw are corresponding weights.Ω is assumed to be the manually preclassified part ofthe image, Λ - the set of wavelet modes that need tobe preserved close to the original image. Function f as-sumes value 1 at the non-vessel pixels and 0 at the pixelswithin the blood vessels in the image, uorig denotes theoriginal image rescaled to the range [0, 1]. The set Λ of“relatively” significant modes is obtained by the adaptivethresholding method described earlier. The gradient de-scent equation for this modified WGL energy takes theform

ut = ε∆wu−1εW ′(u)−µs(u−f)χΩ−µw∆wPrΛ(u−uorig)

The initial guess used for numerical simulations may bechosen to be equal to the given image except for blackand white values at the preclassified areas:

u(x, 0) = u0(x) = uorigχΩc(x) + fχΩc(x)

Numerical simulations were performed using discretegradient-stable semi-implicit schemes. Indeed, ∆w is adiagonal operator in the wavelet basis, but the presenceof nonlinearity does not allow to make it fully implicit.The gradient stability is achieved by the convexity split-ting method described in [3]. The computational speed ofWGL-based algorithms is mostly defined by the choice ofthe translation-invariant discrete wavelet transform. Thestationary wavelet transform (SWT) matches the modelperfectly, however, it requires more operations than FFTthat is used within related PDE-based methods. The factthat the SWT is relatively slow in comparison with theFFT is compensated by WGL-based methods requiringfewer iterations to converge. Thus, the pseudo-differentialmethod is comparable to or outperforms the PDE meth-ods in terms of the CPU time.

4 REFERENCES

Numerical tests

The test was performed on an average of several cSLOimages - Fig. 3. Depending on the combination of pa-rameters ε and µ which define the importance of theoutput being binary and having the same set of edgesrespectively, the results vary in the level of detalizationand sharpness of the vessel/non-vessel classification -Fig.3(c).

(a) (b)

(c) (c)

Figure 3: (a) Initial image, (b) the set of edges that need to be preserved(denoted f in the algorithm), (c) the resulting maps of detected blood vessels.

IV Conclusions

The authors addressed some questions related to the reti-nal image processing, in particular - to the computationof the macular pigment map. A successful method of cor-rection of autofluorescence images from cross-modalities(cSLO, standard fundus camera) allowing to use those inthe same two-wavelength computations of macular pig-ment maps was introduced along with a variational tech-nique for extraction of a binary map of the retinal vas-cular system from cSLO images. The latter can be im-proved by designing an explicit, non-iterative way of find-ing or approximating solutions of the described varia-tional problem and, thus, decreasing the computationaltime. This issue is one of the aspects of the authors’ workin progress.

Acknowledgements

The research was funded by the Intramural ResearchProgram of NICHD/NIH, by NSF (CBET0854233), byNGA (HM15820810009), and by ONR (N000140910144).The authors are grateful to Professors John J. Benedettoand Andrea L. Bertozzi for many insightful discussionsand their long-term support.

References

[1] A. Bertozzi, S. Esedoglu, and A. Gillette. Analysis ofa two-scale Cahn-Hilliard model for image inpaint-ing. Multiscale Modeling and Simulation, 6(3):913–936, 2007.

[2] A. Chambolle and P.-L. Lions. Image recovery viatotal variation minimization and related problems.Numerische Mathematik, 76(2):167–188, April 1997.

[3] J. Dobrosotskaya and A. Bertozzi. A Wavelet-Laplace variational technique for image deconvolu-tion and inpainting. ”IEEE Transactions on ImageProcessing”, 17(5), 2008.

[4] J. Dobrosotskaya and A. Bertozzi. WaveletGinzburg-Landau energy in the edge-preservingvariational techniques of image processing. To besubmitted to SIAM Jour. or Appl. Analysis, March2010.

[5] S. Esedoglu and J. Shen. Digital inpainting based onthe Mumford-Shah-Euler image model. Euro. Jnl ofApplied Mathematics, 13:353–370, 2002.

[6] Selim Esedoglu. ”Blind Deconvolution of Bar CodeSignals”. Inverse Problems, (20):121–135, 2004.

[7] A.C.Bird et al. An international classification andgrading system for age-related maculopathy andage-related macular degeneration. The InternationalARM Epidemiological Study Group. Surv Ophthal-mol, 5(39):367–374, 1995.

[8] F.C.Delori et al. Macular pigment density measuredby autofluorescence spectrometry: comparison withreflectometry and heterochromatic flicker photome-try.

[9] M.Ehler et al. High-resolution autofluorescenceimaging for mapping molecular processes within thehuman retina. UMD, 2010. SBEC.

[10] G. Dal Maso. An introduction to Gamma conver-gence. Progress in nonlinear differential equationsand their applications. Birkhauser Boston, Inc.,Boston, MA, 1993.

[11] L. I. Rudin, S. Osher, and E. Fatemi. ”Nonlinear To-tal Variation based noise removal algorithms”. Phys-ica D., 60:259–268, 1992.

[12] S.Beatty, F.J. Van Kuijk, and U. Chakravarthy.Macular pigment and age-related macular degener-ation: longitudinal data and better techniques ofmeasurement are needed. Invest Ophthalmol Vis Sci,3(49), 2008.

[13] S.M.Meyers, M.A. Ostrovsky, and R.F. Bonner. Amodel of spectral filtering to reduce photochemicaldamage in age-related macular degeneration. TransAm Ophthalmol Soc, (102):83–93, 2004.