Regularization method preserving photometry forRichardson-Lucy restorationEmmanuel Bratsolis and Marc SigelleEcole Nationale Sup�erieure des T�el�ecommunicationsD�epartement de Traitement du Signal et des Images46 rue Barrault Paris F-75013, FranceABSTRACTThe success of Richardson-Lucy (RL) algorithm is that it forces the restored image to be non-negative and to conserveglobal ux at each iteration. The problem with RL algorithm is that it produces solutions that are highly unstable,with high peaks and deep valleys. Our aim is to modify RL algorithm in order to regularize it while preservingpositivity and total photometry as far as possible.Data instances that are not compatible with others can cause singularities in the restoration solution. So, we havean ill-posed problem and a regularization method is needed to replace it to a well-posed problem. The regularizationapproach overcomes this di�culty by choosing among the possible objects one \smooth" that approximate the data.The basic underlying idea in most regularization approaches is the incorporation of \a priori" knowledge into therestoration.In this article we try to give a simple method of spatial regularization deriving from RL algorithm in orderto overcome the problem of noise ampli�cation during the image reconstruction process. It is very important inastronomy and remote sensing to regularize images while having under control their photometric behavior. Wepropose a new reconstruction method preserving both the global photometry and local photometric aspects.Keywords: Image processing, restoration, regularization, photometric behavior1. INTRODUCTIONThe RL algorithm1,2 is the technique most widely used for restoring astronomical images. It can be derived verysimply if we start with the blurred image equation and the equation for Poisson statistics.3There are two types of noise.4,5 The �rst one is not associated with the true signal and comes from backgroundssuch as the read-out noise of a detector, which has a Gaussian distribution, or the sky background. The skybackground noise can be removed before any other data processing, whereas the read-out noise can be modi�ed intoa Poisson distribution by a simple modi�cation of data when its Gaussian distribution is known, provided its varianceis large enough.4 The second type of noise is associated with the signal recording process. A regularization methodhas to be applied to remove this type of noise. We assume in the following that read-out and sky background noisehave been corrected.The problem of highly unstable solutions such as Gibbs oscillations or ringing was solved for the case of point starsources whose locations are assumed to be known. The related algorithm, known as PLUCY,6{8 splits the imagelying in the object plane into two channels, one for the point sources and one for the background, and leads to a verygood photometric �delity. The problem now is to study what happens if we have an extended object without pointsource stars but only noise point sources. The wavelet decomposition9,10 applied to extended objects, like a simulatedelliptical galaxy with Poisson and Gaussian (read-out) noise,11 yields a very good photometric �delity between the\true image" and the regularized RL restoration. The same wavelet decomposition with a FMAPE algorithm12 hasbeen used for an image of Saturn from the non-refurbished Hubble Telescope, preclassi�ed into nine regions to obtaindi�erent degrees of smoothing in di�erent regions. A conditional autoregression (CAR) model has been used toincorporate smoothness constraints.14 The CAR model used in astronomy is a Markovian Gaussian model. After acomposition of the CAR prior model, the blurred image equation and the Poisson statistics in a Bayesian framework,E.B.: E-mail: bratsoli@tsi.enst.fr, M.S.: E-mail: sigelle@tsi.enst.frThis research has been supported by a Marie Curie Fellowship of the European Community programme MCFI under contractnumber HPMF-CT-2000-00532 1

the estimation of maximum a posteriori (MAP) gives a smoothed RL restoration with a variable regularization factorfor every step of restoration. That may produce an oversmoothing for low values of SNR ratio. Here we present anew regularization method for RL restoration, named FPR,13 which preserves the local photometric properties.2. RECALL OF THE RICHARDSON-LUCY RESTORATION ALGORITHMWe consider the case of ground-based optical or infrared imaging by a single aperture. Let O(x; y) be the truetwo-dimensional intensity distribution of the object, where x; y are two orthogonal coordinates in some small regionof the sky. The measured data D(x; y) after \bias" and \ at-�eld" corrections takes the form of a convolution:D(x; y) = Z Z H(x� u; y � v) O(u; v) du dv +N(x; y)= (H �O)(x; y) +N(x; y); (1)where � indicates two-dimensional convolution, H(x; y) is the point spread function (PSF) of the imaging systemand N(x; y) represents the additive noise.We shall from now on adopt a discrete representation of signals, i :e: work on a discrete two-dimensional lattice ofsites, S = fsg. The true scene will then be noted as: O = fOsgs2S or equivalently: O = O(i; j) with (i; j) 2 Z2. Ityields a blurred image after discrete convolution as: I = H �O, where the PSF of the blur response can be written:H = fHsrg with Hsr = Hsr = Hr�s for each pair of sites (s; r) 2 S and corresponds to H(x� u; y � v) in Equation(1). Then the observed noisy data, D = fDsgs2S, are obtained by applying a Poisson distribution to I . Thus thefollowing scheme applies: O ! I = H * O ! D = Poisson (H * O).blur noiseThe foundation of RL is to maximize the likelihood of the original image O, which can be written as:P (D = O) = P (D = I = H � O)= Yr2S expf�(H �Or)g [(H �O)r ]DrDr! :The log-likelihood L = log P (D = O), is easily written as:L =Xr2S f � (H �O)r +Dr log(H �O)r � log(Dr!) g ; (2)where a typical convolution product in the previous formula expands as:(H �O)r = Xs2SHsrOs:The Maximum Likelihood (ML) principle allows us to �nd the set of variables fOsg verifying:@L@Os = �Xr2SHsr +Xr2SDr Hsr(H �O)r = 0 8s 2 S:Assuming that the point spread function H is normalized, i :e:Xs2SHsr = 1 8r 2 Simplies in turn: 1 =Xr2S DrHsr(H �O)r 8s 2 S: (3)2

The Richardson-Lucy restoration method starts from this set of exact equations. Multiplying each of theseequation for site s by the related original data Os yields:Os =Xr2S DrHsrOs(H �O)r 8s 2 Swhich is written in an iterative form justi�ed by the EM algorithm3O(n+1)s =Xr2S DrHsrO(n)s(H �O)(n)r 8s 2 S: (4)We write this updating formula more concisely as:O(n+1) = O(n) � DH �O(n) � �HT 8n � 1and initialization O(1), where HT is the transpose of H .This iterative scheme will also be noted in the following as:O(n+1) = RL(O(n)) ; n � 1:It is easy to show that it preserves the global photometry since:Xs2SO(n+1)s = Xs2S Xr2S DrHsrO(n)s(H �O)(n)r= Xr2S Dr(H �O)(n)r Xs2SHsrO(n)s= Xr2SDr:Usually O(1) is taken as a uniform at image having the same total ux as observation D.3. FLUX-PRESERVING REGULARIZATION (FPR) METHOD FOR RESTORATIONThe main problem with the RL algorithm is that in practice it doesn't converge to the global maximum because ofthe fact that we are dealing with an ill-posed problem and some a priori knowledge, not contained in the maximumlikelihood model, is needed. Data instances that are not compatible with others can cause singularities in therestoration solution. So, a regularization method is needed to replace the ill-posed problem with a well-posedproblem.15,16 The regularization approach overcomes this di�culty by choosing among the possible objects one\smooth" object that approximate the data. The basic underlying idea in most regularization approaches is theincorporation of a priori knowledge into the restoration.Assume now that we modify the iterative scheme of Equation (4) in this sense:O(n+1) = (1� �) RL(O(n)) + � T (O(n)) 8n � 1; (5)where T () is some operator regularizing the pixel intensities, and � is some positive constant lying between 0 and 1.We shall note from now on the total ux of an image O as:F (O) = Xs2SOs:From what precedes, the total ux evolves as:F (O(n+1)) = (1� �) F (D) + � F (T (O(n))):3

When the operator T () is chosen so that it preserves the total ux i :e:F (T (O(n))) = F (T (O(n�1)));then iteratively: F (O(n+1)) = F (O(n)) : : : = F (O(1)) = F (D)i :e: total ux is preserved, provided that the initial guess O(1) has the same total ux as the observation D.We can for example choose for the preserving total ux operator T () any convolution �lter associated with a nor-malized matrix R, for example a Gaussian �lter whose standard deviation describes its spatial extension, or moresimply a nearest-neighbor average �lter, as will be used in the next section. The FPR algorithm takes now the form:O(n+1) = (1� �)O(n) � DH �O(n) � �HT + �R �O(n): (6)Thus at each step the current pixel intensity will depend in a regularizing manner on its neighboring ones, accordingto the magnitude of parameter � (for � = 0 the FPR gives back the RL algorithm). It is also obvious that positivityis preserved when 0 � � � 1. 4. RESULTSWe begin with an initial arti�cial Titan image (Fig. 1) with the same apparent size as our real Titan image. ThePSF used is the real PSF measured by the ADONIS system. Titan image and PSF have a size 64� 64. The initialimage O (Fig. 1) has been convolved by the normalized PSF and corrupted by Poisson noise. The degraded image(Fig. 2) has been restored by RL and FPR methods. For every step of restoration the relative error (RE) at of stepn has been estimated by Equation (7) RE(n) =vuutPs2S jO(n)s �Osj2Ps2S jOsj2 : (7)We have to choose the range of regularization operator T (), namely four or eight-connectivity. Four (or eight)-connectivity means that every pixel is connected with its four (or eight) nearest neighbors. In the case of the smallimage of Titan, a four-connectivity is enough, so that we use the same �lter matrix R for both the real and syntheticimage: R = 266664 0 0:25 00:25 0 0:250 0:25 0 377775 :The parameter � is kept constant throughout all iterations and can be chosen as 0:05, 0:2, 0:35 or 0:5.The SPOT image of Tchernobyl was arti�cially blurred by a normalized Gaussian PSF with size 9 � 9 in pixelscale and spatial standard deviation � = 1:35. The noise was an additive Gaussian with the same standard deviation�n = 1:35. An eight-connectivity was chosen and the �lter matrix R has beenR = 266664 0:05 0:15 0:050:15 0:20 0:150:05 0:15 0:05 377775 :The parameter � was chosen as 0:01. It is evident that FPR method can be used for a Gaussian noise too. Theselection of the best �lter matrix is under research.The real image of Titan (Fig. 6), 64� 64 in pixel scale, was acquired with the adaptive optics system ADONISinstalled at the ESO 3.6 m telescope in La Silla (Chile). The resolution is 0:05 arcsec/pixel. The image was acquired4

Position

Pos

ition

10 20 30 40 50 60

10

20

30

40

50

60

Figure 1. Simulated image with the same apparent diameter of Titan.

Position

Pos

ition

10 20 30 40 50 60

10

20

30

40

50

60

Figure 2. Degraded simulated image of Titan.5

Position

Pos

ition

10 20 30 40 50 60

10

20

30

40

50

60

Figure 3. Restored image with the FPR method and � = 0:05.

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Steps

Rel

ativ

e E

rror

Figure 4. The evolution of Relative Error after 100 steps. The dotted line presents the RL restoration and the solidlines the FPR restoration with � equal to 0:5, 0:35, 0:2 and 0:05 from top to bottom.6

0 100 200 300 400 500 600 700 800 900 10000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Steps

Rel

ativ

e E

rror

Figure 5. The evolution of Relative Error after 1000 steps. The dotted line presents the RL restoration and thesolid line the FPR restoration with � = 0:05.

Position

Pos

ition

10 20 30 40 50 60

10

20

30

40

50

60

Figure 6. Initial image of Titan 64� 64.7

Position

Pos

ition

5 10 15 20 25

5

10

15

20

25

Figure 7. Initial image of Titan 25� 25.

5 10 15 20 25

5

10

15

20

25

Position

Pos

ition

Figure 8. Isophot contours corresponding to Fig. 7.8

Position

Pos

ition

5 10 15 20 25

5

10

15

20

25

Figure 9. Image of Titan after FPR restoration with � = 0:05.

5 10 15 20 25

5

10

15

20

25

Position

Pos

ition

Figure 10. Isophot contours corresponding to Fig. 9.9

Figure 11. Degraded SPOT image of Tchernobyl 479� 479.

Figure 12. Isophot contours corresponding to Fig. 11.10

Figure 13. Isophot contours corresponding to the clear image.

Figure 14. Isophot contours corresponding to the restored image.11

at 2:04 �m, where the methane is transparent, with a narrow-band �lter, so it could be possible to see more detailsof surface. The PSF was measured by the system two minutes before the image acquisition. The image has beencorrected for systematic e�ects.The isophot contours correspond to 0:98, 0:95, 0:9, 0:85, 0:82 and 0:4 of the maximum intensity for the Titanimage and to 0:3 of the maximum intensity for the SPOT image.5. CONCLUSIONSA new method, named FPR, has been proposed. The mathematical presentation has been presented as well as theresults for a simulated image and for a real image of Titan acquired in near infrared with the adaptive optics systemADONIS. The same method has been tested in a SPOT degraded image and yield �ne isophot contour detections.ACKNOWLEDGMENTSThe authors are grateful to J.-P. V�eran, A. Coustenis, M. Combes and all the people who work for the adaptiveoptics system ADONIS for the loan of observational material.REFERENCES1. W.H. Richardson, \Bayesian-based iterative method of image restoration," J. Opt. Soc. Amer. 62, pp. 55-59,1972.2. L.B. Lucy, \An iterative technique for the recti�cation of observed distribution," Astron. J. 79, pp. 745-754, 1974.3. L.A. Shepp and Y. Vardi, \Maximum likelihood reconstruction for emission tomography," IEEE Trans. MedicalImag. 1, pp. 113-122, 1982.4. D.L. Snyder, A.M. Hammoud and R.L. White, \Image recovery from data acquired with a charge-coupled-divicecamera," J. Opt. Soc. Amer. A 10, pp. 1014-1023, 1993.5. D.L. Snyder, C.W. Helstrom, A.D. Lanterman, M. Faisal and R.L. White, \Compensation for readout noise inCCD images," J. Opt. Soc. Amer. A 12, pp. 272-283, 1995.6. L.B. Lucy, \Image restorations of high photometric quality," in The Restoration of HST Images and Spectra II,R.J Hanish and R.L. White, eds., Space Telescope Science Institute, Baltimore, pp. 79-85, 1994.7. R.N. Hook and L.B. Lucy, \Image restorations of high photometric quality. II. Examples," in The Restoration ofHST Images and Spectra II, R.J Hanish and R.L. White, eds., Space Telescope Science Institute, Baltimore, pp.86-94, 1994.8. R.N. Hook and L.B. Lucy, in Astronomical Data Analysis Software and Systems IV, R.A. Shaw, H.E. Payne andJ.J.E. Hayes, eds., ASP Conference Series 77, pp. 293-296, 1995.9. J.-L. Starck, F. Murtagh and A. Bijaoui, \Image restoration with denoising using multi-resolution," in TheRestoration of HST Images and Spectra II, R.J Hanish and R.L. White, eds., Space Telescope Science Institute,Baltimore, pp. 111-117, 1994.10. J.-L. Starck and F. Murtagh, \Image restoration with noise suppression using the wavelet transform," Astron.Astrophys. 288, pp. 342-348, 1994.11. F. Murtagh, J.-L. Starck and A. Bijaoui, \Image restoration with noise suppression using a multiresolutionsupport," Astron. Astrophys. Supplement Series 112, pp. 179-189, 1995.12. J. N�u~nez and J. Llacer, \Bayesian image reconstruction with space-variant noise suppression," Astron. Astrophys.Supplement Series 131, pp. 167-180, 1998.13. E. Bratsolis and M. Sigelle, \A spatial regularization method preserving local photometry for Richardson-Lucyrestoration," Astron. Astrophys., in press, 2001.14. R. Molina, J. Mateos and J. Abad, \Prior models and the Richardson-Lucy restoration method," in The Restora-tion of HST Images and Spectra II, R.J Hanish and R.L. White, eds., Space Telescope Science Institute, Baltimore,pp. 118-122, 1994.15. D.M. Titterington, \General structure of regularization procedures in image reconstruction," Astron. Astrophys.144, pp. 381-387, 1985.16. A.K. Katsaggelos, J. Biemond, R.W. Schafer and R.M. Mersereau, \A regularized iterative image restorationalgorithm," IEEE Trans. Signal Processing 39, pp. 914-929, 1991.12