A reweighted `2 method for image restoration with Poisson and

mixed Poisson-Gaussian noise

Jia Li∗ Zuowei Shen∗ Rujie Yin† Xiaoqun Zhang‡

Abstract

We study weighted `2 fidelity in variational models for Poisson noise related imagerestoration problems. Gaussian approximation to Poisson noise statistic is adopted to de-duce weighted `2 fidelity. Different from usual weighted `2 approximation, we propose areweighted `2 fidelity with sparse regularization by framelets. Based on the split Breg-man algorithm introduced in [22], the proposed numerical scheme is composed of threeeasy subproblems that involve quadratic minimization, soft shrinkage and matrix vectormultiplications. Unlike usual least square approximation of Poisson noise, we dynamicallyupdate the underlying noise variance from previous estimate. The solution of the proposedalgorithm is shown to be the same as the one obtained by minimizing Kullback-Leibler di-vergence fidelity with the same regularization. This reweighted `2 formulation can be easilyextended to mixed Poisson-Gaussian noise case. Finally, the efficiency and quality of theproposed algorithm compared to other Poisson noise removal methods are demonstratedthrough denoising and deblurring examples. Moreover, mixed Poisson-Gaussian noise testsare performed on both simulated and real digital images for further illustration of the per-formance of the proposed method.

Keywords: weighted `2 fidelity, Gaussian approximation, split Bregman algorithm, framelets

1 Introduction

We consider an imaging system whose output data is a vector f ∈ RM and the true underlyingimage is u ∈ RN . The observation model is often described by

f = Au (1)

where A denotes a linear operator from RN to RM . In this paper, we focus on variationalimage restoration from observation f contaminated by Poisson or mixed Poisson-Gaussian noise.Typically, variational models for image restoration are composed of two terms, one is a datafidelity term and the other is a regularization term for modeling a priori knowledge on unknownimages. In general, the data fidelity term keeps the inline true image u close enough to theinput data f , so that the solution is meaningful. According to different noise statistics, thefidelity term takes different forms. It is well known that the least square fidelity is used foradditive white Gaussian noise (AWGN). Such fidelity is mostly considered in literature for itsgood characterization of system noise. On the other hand, non-Gaussian types of noise arealso encountered in real images. An important variant is Poisson noise, which is generally

∗J. Li and Z. Shen are with the department of mathematics, National University of Singapore. Block S17, 10Lower Kent Ridge Road, Singapore 119076; {lijia,matzuows}@nus.edu.sg†R. Yin is with Zhiyuan College, Shanghai Jiao Tong University, 800, Dongchuan Road, Shanghai, China,

200240. ry27@duke.edu‡X. Zhang is with the department of mathematics, MOE-LSC, and institute of natural sciences, Shanghai

Jiao Tong University. 800, Dongchuan Road, Shanghai, China, 200240. xqzhang@sjtu.edu.cn

1

observed in photon-limited images, such as fluorescence microscopy, emission tomography andimages have low exposure region like photos taken at night with high ISO setting. Due toits importance in medical imaging and the wide commercial application, a vast amount ofliterature in image restoration is devoted to problems encountering Poisson noise (see [2] andthe references therein). The poisson noise data, i.e. the probability of receiving y particles isgiven by

P (y) =e−ττy

y!, y = 0, 1, 2, · · · (2)

where τ is the expected value and the variance of random counts. Based on the statistics ofPoisson noise, a generalized Kullback-Leibler (KL)-divergence [11] fidelity can be derived (seesection 2.3.1 below) which is usually used in Poisson image restoration. However, difficulties incomputational stability and efficiency arise due to the logarithm function that appears in theKL-divergence function. Moreover, in the case of image acquired with CCD camera, the mixtureof Poisson and read-out noise (modeled as AWGN) is a more appropriate model, although itis seldom considered in literature. In [25], a Stein’s unbiased risk estimator (SURE) basedon Poisson-Gaussian statistics is constructed in wavelet transform for mixed Poisson-Gaussiannoise denoising, but the construction of estimators is so complicated that it cannot be easilyextended for more general image restoration problem such as deblurring. Recently, Gong etal. proposed in [23] a universal `1 + `2 fidelity term for mixed or unknown noise and achieveencouraging numerical results, however the statistical analysis of it remains unclear.

Besides the fidelity term, we need a regularization term to control noise and artifacts inreconstructed images. A simple but effective idea is to urge a sparse representation in sometransform domain for the recovered image. For a large class of applications [15, 8, 36], penalizingthe `1 norm of the transform coefficients of a signal leads to such a sparse solution, as largelyillustrated in compressed sensing community in recent few years. The choice of transform iscrucial to obtain a reasonable solution, and one popular choice is total variation proposed byRudin-Osher-Fatemi [29]. Another choice are framelets, which have been shown useful andsuperior to total variation for different restoration tasks, see [13, 10] and the references therein.Recently, the relation of sparse framelets representation and total variation has been revealedin [10], and theoretically total variation can be interpreted as a special form of `1 frameletsregularization. For these reasons, we consider framelets regularization in this paper.

The main contribution of this paper is in introducing a least square fidelity term, areweighted `2 fidelity, that well approximates KL-divergence and developing an efficient it-erative algorithm solving the optimization problem with framelets regularization. In fact, theidea of approximating KL-divergence by a fixed weighted least-square has been used for a longtime in medical imaging reconstruction and image deblurring [2]. Recently, such a techniqueis restudied by Stagliano et al. [34], where the KL-divergence fidelity term is approximatedby a weighted least square involving the unknown image. Combining with regularization, ascaled gradient projection (SGP) method with line search technique has been applied to solvethe resulted problem. However, the efficiency of the algorithm involves the decomposition ofa specific smoothed total variation regularization and line searching, which can not be easilygeneralized to other sparse regularization. We have a similar approximation, but viewed it asa Gaussian approximation to the Poisson distribution of the same variance and the variance isdynamically estimated using an iterative algorithm based on split Bregman iteration [22]. Theoriginal split Bregman iteration is ideal for solving `2 fidelity based `1 regularization, where the`1 minimization is executed by a soft shrinkage, promising the efficiency of our modified versionof it. In addition, the solution obtained by our algorithm has the same reweighted `2 energyand framelets sparsity as that of the minimizer of the KL-divergence fidelity using the sameregularization. Furthermore, the model immediately gives a reasonable extension to the mixedPoisson-Gaussian noise case with a small modification on the weight.

2

This paper is organized in three consecutive parts. In the first part, we review the back-ground of variational restoration models, the split Bregman method and the existing Poissonfidelities. Then we present the reweighted `2 fidelity for Poisson noise through Gaussian approx-imation and maximizing likelihood (ML) and then build the corresponding image restorationmodels and algorithms. We also provide an analysis of the proposed models and algorithmswhich reveals the connection to the classical KL-divergence model. The models are furtherextended to the mixed noise case, where both Poisson noise and Gaussian white noise present.The last part is dedicated to numerical simulation where other fidelity terms and models arecompared numerically to ours for Poisson image denoising/deblurring and the mixed Poisson-Gaussian denoising/deblurring cases.

2 Background

2.1 Variational image restoration model

The observation model in consideration takes the following generic form

f = Au+ c+ ε (3)

where c is a fixed background image vector in RM and possibly 0, and ε is the noise pertur-bation. Generally, for additive white Gaussian noise setting, it is assumed that c = 0 and εhas independent normal distribution of mean 0 and variance σ2 for each pixel. The operatorA may come from the imaging system such as the point spread function (PSF), the exteriordisturbance such as motion or a combination of the two sources. In this paper, we mainlyconsider denoising and deblurring, where A is either an identity or a convolution operator.

Image restoration by variational model usually takes the following form:

minuF (u) + λG(u) (4)

where F (u) and G(u) are the fidelity term and the regularization term respectively, and λis a positive scaling factor. The fidelity F (u) is derived using classical maximum a posteriorprobability (MAP) P (u|f) estimation, while the regularization G(u) is designed based on apriori assumptions on u. As introduced previously, it is nowadays standard technique to penal-ize the `1 norm of representation coefficients in a transform domain. Therefore, the followingvariational model is largely studied for image restoration from AWGN:

minu

1

2‖Au− f‖22 + λ‖Du‖1 (5)

where ‖ · ‖1 denotes the usual `1 vector norm and D is a linear transform, such as discretegradient used in total variation [29], Fourier transform, local cosine transforms, wavelet orframelets [9].

2.2 Split Bregman algorithm

Split Bregman method introduced in [22] is designed to solve the variational model taking theform as (5). More generally, we consider the minimization problem

minuF (u) + λ‖Du‖1 (6)

where F (u) is a convex functional representing fidelity, such as F (u) = 12‖Au − f‖

22 as in the

case of AWGN. By introducing an auxiliary variable d = Du, the following alternating split

3

Bregman scheme solves (6) : for µ > 0,uk+1 = argminu F (u) +

µ

2‖Du− dk + bk‖22

dk+1 = argmind λ‖d‖1 +µ

2‖Duk+1 − d+ bk‖22

bk+1 = bk +Duk+1 − dk+1

(7)

The efficiency of this algorithm relies on the closed form solution of the second subproblem. Infact, dk+1 is given by the so-called soft-shrinkage operator

dk+1 = sign(Duk+1 + bk) ·max(|Duk+1 + bk| − λ/µ, 0) (8)

where each operation is componentwisely performed. The nonsmooth optimization problem (6)is thus decomposed into three subproblems; the first subproblem is simple to solve when F (u)is in quadratic form. This method has been demonstrated to be very efficient for `1 type ofminimization in a large variant of applications. In a more general setting, it is shown to beequivalent to classical Douglas-Racheford and alternating direction multiplier method (ADMM)[16, 30], and the convergence analysis is given in this framework. In [9], Cai et al. studied theapplication of such an algorithm for framelets based image restoration and provide a detailedconvergence proof. Further approximation of the quadratic subproblem was considered in [37]in order to maximally decouple the subproblems. In this paper, we intend to take advantage ofthe efficiency of split Bregman and Poisson noise formulation to develop an efficient algorithmfor Poisson noise related image restoration problem. Therefore, a reweighted `2 fidelity termwhich is closely related to the problem (6) is considered.

2.3 Poisson noise related data fidelities

We assume that the observation f is corrupted by Poisson noise (see (2)), i.e.

f ∼ P (Au+ c) (9)

Since both f and u denote photon counts, their elements are nonnegative; the following as-sumptions are made for the linear operator A and c as in [32, 3, 7]:

Assumption 1 • The observation data f > 0, and the underlying true u ≥ 0.

• The linear operator A satisfies the following conditions:

Aij ≥ 0,∑i

Aij > 0, ∀j,∑j

Aij > 0,∀i.

where Aij is the (i, j) element of the imaging matrix A.

• The fixed background image c > 0.

For most image restoration models, A is the convolution or line integral operator, and theseconditions can be easily fulfilled without loss of generality. The first assumption is used to avoidmodel deficiency as in KL-divergence and reweighted `2. The second assumption implies thatfor u ≥ 0, Au = 0 if and only if u = 0.

Given Au and c, we have the likelihood of observing f

P (f |Au+ c) =

M∏i=1

(Au+ c)fii e−(Au+c)i

fi!(10)

4

where (Au+ c)i denotes the ith element of Au+ c. By the property of the Poisson distribution,we have the expectation (mean) and the variance f are

E(f |Au+ c) = Var(f |Au+ c) = Au+ c (11)

Before presenting the proposed reweighted `2 fidelity, we first review two existing fidelitiesfor Poisson statistics for comparison.

2.3.1 KL-divergence

The most popular fidelity for Poisson noise is the generalized Kullback-Leibler (KL)-divergencefidelity [11], which can be derived directly by MAP method. Maximizing a posterior probabilityis equivalent to minimizing the negative log likelihood of (10), i.e.

minu≥0− logP (f |Au+ c) = min

u≥01T (Au+ c− log(f !))− fT log (Au+ c), (12)

where 1 is the vector with all 1 entries.If we neglect the constant term log (f !) which is unrelated to the unknown u, we obtain

the following fidelity term

F (u) = 1T (Au+ c)− fT log (Au+ c) (13)

Combining with sparse regularization and nonnegativity constraint on photon counts u, weget the restoration model as follows,

minu≥0

1T (Au+ c)− fT log(Au+ c) + λ‖Du‖1 (14)

Generally, (14) is a difficult optimization problem because of the nonsmooth regularizationterm and the KL-divergence term. Optimization of KL-divergence fidelity with nonnegativeconstraint are typically solved by Expectation-Maximization (EM) algorithm [24]. It is knownthat the convergence of the EM algorithm is slow and it may introduce so-called ”checkboardeffect” [3, 34, 7]. In [7], the authors proposed a two-step iteration method called EM-TV forsolving (14) when D is a discrete differential operator. We change the TV regularization toframelets and rename it as EM+`1 algorithm for later discussion and numerical comparisonwith our scheme. The algorithms is described as follows,

uk+ 12

= ukA∗(

f

Auk + c) (EM step)

uk+1 = argminu≥0

1

2

∥∥∥∥u− uk+ 12√

uk

∥∥∥∥2

2

+ λ‖Du‖1 (`1 step)

(15)

This algorithm and its variants have been proved to be efficient for Poisson noise removalin PET and nasoscopy image deconvolution [7]. Under adequate conditions, the convergenceof the algorithm with a damped parameter is provided in [6]. However, the conditions are alsohard to verify in practice and we also need a subproblem solver for the second step when D isnot an orthogonal operator.

The other kind of method considered in [31, 18] is to apply directly the split Bregmanmethod (7) by introducing a variable d = Du on the model (14). Here, for the reason ofcomparison, we present the algorithm proposed in [31] for solving (14) by inducing three extra

5

variables to represent d(1) = Au, d(2) = Du and d(3) = u to handle the nonnegativity constraint.The overall scheme can be written as

uk+1 = argminu ‖Au− d(1)k + b

(1)k ‖

22 + ‖Du− d(2)

k + b(2)k ‖

22 + ‖u− d(3)

k + b(3)k ‖

22

d(1)k+1 = argmind(1) 1T (d(1) + c)− fT log(d(1) + c) +

1

2γ‖Auk+1 − d(1) + b

(1)k ‖

22

d(2)k+1 = argmind(2) λ‖d(2)‖1 +

1

2γ‖Duk+1 − d(2) + b

(2)k ‖

22

d(3)k+1 = argmind(3)≥0

1

2γ‖uk+1 − d(3) + b

(3)k ‖

22

b(1)k+1 = b

(1)k +Auk+1 − d

(1)k+1

b(2)k+1 = b

(2)k +Duk+1 − d

(2)k+1

b(3)k+1 = b

(3)k + uk+1 − d

(3)k+1

(16)

Several auxiliary variables have been introduced and we will draw into comparison with ourproposed algorithm in the numerical implementation.

2.3.2 Anscombe transform

Another well-known technique of Poisson denoising is the Anscombe transform [1] used in imagedenoising, when the linear operator A in (9) is the identity. The Anscombe transform is definedas the following non-linear transform

A : x 7→ 2

√x+

3

8(17)

If x is a random variable that obeys the Poisson distribution with mean and variance τ , the trans-formed random variable Ax follows an approximated standard Gaussian distribution N (τ, 1).Apply the Anscombe transform on f ∼ P (u+ c), and let f := Af, u := A(u+ c), then f followsapproximately a normal distribution N (u, 1). Hence, the usual least square fidelity term canbe applied for f and we obtain the following denoising model

u∗ = argminu≥0

1

2‖u− f‖22 + λ‖Du‖1 (18)

and the final image is given by u∗ = A−1u∗−c. The regularization used in the above model (18)forces u to be sparse in the transformed domain. This is roughly equivalent to requiring sparsityon u in the transform domain since the Anscombe transform (17) is monotone increasing andkeeps the order of magnitude of elements in u. We note that this model is generally applied fordenoising model, and the adaption to image deblurring is not easy since it involves inverse ofA, which is non-linear.

3 Weighted least square for Poisson noise image restoration

3.1 Framelets regularization

A countable set X ⊂ L2(R) is called a tight frame of L2(R) if

f =∑h∈X〈f, h〉h ∀f ∈ L2(R), (19)

where 〈·, ·〉 is the inner product of L2(R). Given a finite collection of functions Ψ = {ψ1, ψ2, ..., ψm},define X = {ψn,k,l = 2n/2ψl(2

n · −k), 1 ≤ l ≤ m, n, k ∈ Z}. If X satisfies the condition of tight

6

frame, then X is called a wavelet tight frame and Ψ is called the wavelet. We choose tight framebasis as a representation basis of images due to its multi-resolution property that allows fastalgorithm implementation; its redundancy encourages sparse representation of images, as stud-ied in [12],[13]. Generally, multi-resolution analysis (MRA) based wavelets can be generated bythe unitary extension principle (UEP) in [28]. The wavelet tight frame generated through UEPis based on a refinable function φ = ψ0 and the wavelet functions in Ψ satisfies:

ψl(2·) = hlφ, l ∈ {0, 1, 2, ...,m}

andm∑l=0

|hl(ξ)|2 = 1

andm∑l=0

hl(ξ)hl(ξ + π) = 0

where the sequence hl is called the refinement mask of the wavelet tight frame system. Bycollecting all the refinement masks of wavelet tight frame system, we can generate the fast tightframe transform or decomposition operatorW. The matrixW is consisted of J+1 sub-filteringoperators W0,W1, ...WJ . Among them, W0 is the low-pass filtering operator and the rest arehigh-pass filtering operators. Correspondingly, by unitary extension principle [28], the operatorWT is the fast tight frame reconstruction operator and we have WTW = I, i.e., WTWu = ufor any image u. More details on discrete algorithms of framelet transforms can be found in[13].

3.2 Model and algorithm

We consider Poisson noise for the observation model (9). Let

ε = f −Au− c, (20)

which can be interpreted as an additive perturbation noise. Given Au and c, the conditionalmean of ε is

E(ε|Au+ c) = E(f |Au+ c)−Au− c = 0

and the conditional variance is

Var(ε|Au+ c) = Var(f |Au+ c) = Au+ c.

We approximate ε by additive Gaussian noise, following the normal distribution N (0, Au+cΣ),i.e.

P (ε|Au+ c) ' exp{−1

2(f −Au− c)TΣ−1(f −Au− c)} (21)

s.t. diag(Σ) = Au + c. By further imposing the independence of noise at each pixel, Σ is adiagonal matrix. We take the negative log of the normal distribution (21),

− logP (ε|Au+ c) ∝ 1

2(f −Au− c)TΣ−1(f −Au− c), (22)

and set (22) as the fidelity term, which is equivalent to maximum likelihood.Let ‖x‖2Q = xTQx be the weighted `2 norm of a vector x ∈ RN with respect to a symmetric

positive definite matrix Q. Then (22) can be reformulated as

F (u) = ‖Au+ c− f‖2Σ−1 = ‖Au+ c− f‖2diag(Au+c)−1 =

∥∥∥∥Au+ c− f√Au+ c

∥∥∥∥2

2

(23)

7

Notice that both the division and the square root operator in the `2 norm on the right hand sideof (23) are element wise. The positive definitiveness of Σ−1 is guaranteed by the assumptionsthat c > 0 and Au ≥ 0 for u ≥ 0.

Combining the fidelity F (u) with sparse framelets regularization and nonnegativity con-straint, we have the following restoration model

minu≥0

1

2

∥∥∥∥Au+ c− f√Au+ c

∥∥∥∥2

2

+ λ‖Wu‖1 (24)

This formulation is also considered in [34] and a scale gradient projection method is appliedto (24), directly solving the nonlinear objective function, where a smoothed version of the `1

regularization term was adopted for calculating the gradient.To solve (24), we are interested in taking advantage of the weighted least square structure

and utilizing recently proposed efficient sparse regularization scheme, such as split Bregmanpresented in Section 2.2. However, the fidelity term F (u) is not quadratic, because u appearsboth in the denominator and in the numerator of (23). Therefore, in order to solve the firstsub optimization problem in split Bregman algorithm (7), we need to either approximate ordirectly solve the nonlinear square term.

A reasonable approximation of the unknown weight Au+c, for example, can be the observeddata f , so that (23) is approximated by

F (u) =1

2

∥∥∥∥Au+ c− f√f

∥∥∥∥2

2

(25)

Using this simplification, efficient least square based methods can be applied to solve the sub-problem.

However, such an approximation is not accurate, especially when the observed data f isseverely corrupted by noise. Hence we need a better approximation, which may be derivedadaptively in an iterative algorithm. For any iterative algorithm, whose sequence of solutionuk converges to u∗, when k is big enough, as uk becomes stable, uk serves as a more preciseapproximation of u∗ than f . Hence a better approximation to the formulation (24) is given as

minu≥0

1

2

∥∥∥∥Au+ c− f√Au∗ + c

∥∥∥∥2

2

+ λ‖Wu‖1, (26)

with sufficiently large number of iterations k. We may approximate the weight 1Au∗+c by 1

Auk+cin the algorithm. With such an idea in mind, we combine with the popular Split Bregmaniteration and derives a new algorithm called the reweighted `2 algorithm with split Bregmanfor Poisson noise:

uk+1 = argminu≥0

1

2

∥∥∥∥Au+ c− f√Auk + c

∥∥∥∥2

2

+µ

2‖Wu− dk + bk‖22

dk+1 = argmind λ‖d‖1 +µ

2‖d−Wuk+1 − bk‖22

bk+1 = bk + (Wuk+1 − dk+1)

(27)

We describe the general method (27) in more detail in Algorithm 1. Note that the firststep can be solved by a gradient method with a projection onto the nonnegative orthant. Inpractice, often a few iterations are sufficient to get a reasonable result.

In the following, we justify the proposed algorithm by some intuitive analysis. Let (uk, dk, bk)be the sequence generated by the algorithm (27). If (uk, dk, bk) converges to (u, d, b), then u is

8

Algorithm 1 Reweighted `2 with Spit Bregman for Poisson noise restoration Algorithm

Step0. Set the initial value, u0 = f ; d0 =Wf ; b0 = 0; k = 1 ; Σ0 = Au0 + c.Step1.while ‖uk−2 − uk−1‖2 > δ or k = 1 do

uk = argminu≥0

1

2‖Au+ c− f‖2

Σ−1k−1

+µ

2‖Wu− dk−1 + bk−1‖22

dk = sign(Wuk + bk−1) ·max(|Wuk + bk−1| − λ/µ, 0)bk = bk−1 +Wuk − dkΣk = diag(Auk + c)k = k + 1

end while

a minimizer of (14), shown as follows. If (uk, dk, bk) generated by (27) converges to (u, d, b), weimmediately get

AT Σ−1(Au+ c− f) + µW T b− q = 0

λp− µb = 0 with p ∈ ∂‖d‖1Wu = d

(28)

where q ≥ 0 satisfies 〈q, u〉 = 0, and Σ = diag(Au + c). Therefore, substituting b by λ/µp, wehave

AT Σ−1(Au+ c− f) + λW T p− q = 0

On the other hand, the first order condition of the model (14) gives

0 = AT(Au+ c− fAu+ c

)+ λWT p− q (29)

where p ∈ ∂|d|1 with d =Wu and q ≥ 0 is a lagrangian multiplier such that 〈q, u〉 = 0. Since usatisfies exactly this first order condition, it is a minimizer of (14).

A more rigorous analysis for the convergence of the algorithm, with adequate assumptionson the sequences, is presented in the Appendix. Although these conditions are not easy to checkin applications where ground truth is unavailable, this analysis may bring some insight for acomplete convergence analysis in future.

3.3 Extension to Poisson-Gaussian mixed noise

Previously, we mainly discuss models with Poisson noise only. In real imaging systems, besidesPoisson noise which characterizes the fluctuation in counting number of photons, there is othersystem-inherited noise that can be approximated by AWGN, as considered in [33]. The literatureon Poisson-Gaussian mixed noise are limited despite of its importance. We now explore theextended application of the reweighted `2 fidelity to the mixed Gaussian-Poisson noise case.

With a small modification of (23) , the reweighted `2 fidelity can be adapted to the mixednoise case. Let σ2 be the variance of AWGN and the observed image f has distribution

f ∼ P(Au+ c) +N (0, σ2)

Approximating the probability density function of f by normal distribution (21) with covariancematrix Σ = diag(Au+ c) + σ2I, we obtain the new fidelity term as follows for mixed noise:

F (u) =1

2

∥∥∥∥ Au+ c− f√Au+ c+ σ2I

∥∥∥∥2

2

(30)

9

Combining with the tight frame regularization, we have the following restoration model

minu≥0

1

2

∥∥∥∥ Au+ c− f√Au+ c+ σ2I

∥∥∥∥2

2

+ λ‖Wu‖1 (31)

The algorithm solving this model is the same as Algorithm 1 except adding σ2 in the estimationand updating step of the covariance matrix Σ.

4 Numerical results

In this section, numerical experiments on pure Poisson noise model (24) and Poisson-Gaussianmixed noise models (31) are shown for synthesized and real digital photos for demonstrating theperformance of the proposed Algorithm 1. Numerical results of denoising/deblurring algorithmsdiscussed in section 2 are also presented for comparison. In all implementations where frameletssparse regularization is used, we choose the piecewise linear B-spline wavelets with 1 leveldecomposition as the framelets basis. The corresponding filter masks in discrete version are

h0 = [14 ,

12 ,

14 ], h1 = [−1

4 ,12 ,−

14 ], h2 = [

√2

4 , 0,−√

24 ]. The parameter λ is always set to be 0.01 for

pure image denoising and 0.001 for image deblurring in presence of noise. The parameter µ issimply set to be 1 for all numerical simulations.

In the numerical implementation, we find out that preconditioning on Σk is necessary toensure numerical stability. In particular, we enforce a lower bound Cσ > 0 on the diagonalentries of Σk, i.e. Σk = max{Σk, Cσ} to control its conditional number. To improve the visualquality of the final result, in the postprocessing, we apply a bilateral filter [35] B is defined as

y[i, j] = B(x) =1∑

p,q w[i, j, p, q]

∑p,q

w[i, j, p, q]x[p, q]

where [i, j], [p, q] are indices of pixels of an image x,

w[i, j, p, q] = Gσs(√

(i− p)2 + (j − q)2)Gσr(x[i, j]− x[p, q])

and Gσs , Gσr are Gaussian functions with variance σs and σr respectively.Each denoised result u are evaluated quantitatively by the peak signal-to-noise ratios

(PSNR) value defined by

PSNR(u, u) = 10 log10

MN(umax − umin)2

‖u− u‖22,

using the ground truth image u, where umax and umin are its maximal and minimal pixel valuesrespectively and M , N are the size of the image.

4.1 Poisson denoising

In the following, we compare the visual image quality with the three fidelities through theircorresponding denoising methods: the reweighted `2 method (27) with Algorithm 1, the KL-divergence model (14) with the algorithm (15) as well as algorithm (16) and the Anscombetransform model (18). The noisy images in our test are simulated as follows. The clean imagesare first rescaled to an intensity range from 0 to 120; then the Poisson noise is added in Matlabusing the function poissrnd. The parameters in the first three algorithms are set to the sameinstead of optimized respectively (especially λ = .05), hence the energy function they minimizeare consistent to each other. For the Anscombe transform model, the parameters are tuned(λ = .11 ) such that the best result is obtained, since the scale of the energy function is changed

10

after the Anscombe transform of the initial image. For the denoised results, we do not applythe bilateral filtering postprocessing to remove artifacts, so that the comparison is clear andfair.

In Figure 1, the simulation results of the test image cameraman are shown for all thealgorithms considered previously in Section 2. The denoising results of the reweighted `2 modeland that of the KL-divergence model are visually the same and their PSNR are comparible toeach other, the difference in pixel value of these two results is shown in Figure 1. The Anscombetransform model gives a slightly better result than the other two models, whose parameters arewell tuned, mainly in the low pixel value region. The difference of results in pixel value ofthe Anscombe transform algorithm (18) and that of the KL-divergence based split Bregmanmethod (16) is shown as well, where the biggest difference is about 6.

Table 1 shows a quantitative comparison of PSNR of the other test images, fruit, boat andgoldhill. The Anscombe transform algorithm performs substantially better for the test imagegoldhill than the other algorithms, mainly because the test image has a large low intensityregion where the Anscombe transform provides a better Gaussian approximation of the lowrate Poisson distribution than a weighted `2 formulation. On the other hand, it performs worsefor the test image fruit, which has a small low intensity region but large high intensity region.Figure 2 shows the histogram of absolute residual (error) of reconstruction results in low andhigh intensity regions, which further illustrates the limitation of these two methods in differentregions. Therefore, for pure denoising purpose, the Anscombe transform is preferred for lowintensity images while the other three algorithms are preferred for relatively high intensityimages.

re-weighted `2 KL(SB) EM Anscombe

# of (outer) iterations 50 400 50(with 5 inner iteration) 100cameraman 30.54 30.59 30.57 30.69fruit 30.71 30.71 30.71 30.64boat 29.19 29.18 29.18 29.19goldhill 29.24 29.24 29.24 29.38

Table 1: PSNR value of the denoising result.

The similarity between these algorithms can be explained by the connections between theirconvergent solutions. In the Poisson noise only case, the operator A is identity, therefore theobjective function is strictly convex. According to the theorem 5.1, our algorithm (withoutbilateral filter processing) converges to the unique minimizer u∗ of the KL-divergence fidelitymodel with the same framelet sparsity regularization. Consequently, the two solutions shouldhave the same re-weighted `2 energy and `1 sparsity of the framelets coefficients, where theweight in the `2 norm is fixed to u∗ + c and u∗ is approximated by the split Bregman result.This can be proved numerically as shown in Figure 3; the result of our algorithm with regularizedweight Σk has weighted `2 energy plus `1 sparsity close to a minimizer of the KL-divergencefidelity model calculated by the Split Bregman algorithm (16) as well as the solution given bythe EM algorithm (15). The three solutions are not identical, though, because each algorithmis embedded with different regularization and generates numerical error differently. In Figure3, the KL-divergence plus `1 sparsity energy is also plotted. Also, our proposed scheme and theEM algorithm converge faster than the split Bregman method with respect to the number ofiterations, but the EM algorithm involves a considerable number of inner iterations ( 5 iterationsin our implementation ) in solving the second sub-problem; it converges to a solution of slightlyhigher energy.

11

cameraman noisy KL(SB)+l1, PSNR:30.54dB

re-weighted l2, PSNR: 30.59dB Anscombe+l1, PSNR: 30.69dB EM+l1, PSNR:30.57dB

0

0.1

0.2

0.3

0.4

0.5

0.6

difference of re-weighted l2 and KL(SB)+l1 difference of KL(SB)+l1 and Anscombe

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

difference of KL(SB)+l1 and EM+l1

Figure 1: Denoising results of simulated noisy images with peak intensity 120. From left toright, up to down: noisy image, reweighted `2 method by Algorithm 1, Anscombe transform(18), KL-divergence model using split Bregman (16) and KL-divergence model with EM-`1

algorithm (15). The last image shows the difference in pixel value of Anscombe+`1 result andKL(SB) result

12

0 2 4 6 8 10 12 14 16 180

1000

2000

3000

4000

5000

6000absolute residual in low intensity region ( intensity < 30 )

goldhill

Ascombe

weighted L2

0 5 10 15 20 25 300

1000

2000

3000

4000

5000

6000

7000

8000absolute residual in high intensity region ( intensity > 100 )

fruit

Anscombe

weighted L2

Figure 2: Left: histogram of absolute residual |urecon− utrue| in low intensity region of the testimage goldhill, Right: histogram of absolute residual |urecon − utrue| in high intensity region ofthe test image fruit.

0 5 10 15 20 25 30 35 40 45 50−1.7115

−1.711

−1.7105

−1.71

−1.7095

−1.709

−1.7085

−1.708x 10

7 KL−divergence + l1 sparsity

# of iteration

E(u

)

EM+l1KL(SB)+l1re−weighted l2initial point

0 5 10 15 20 25 30 35 40 45 502.9

2.95

3

3.05

3.1

3.15

3.2

3.25x 10

5

# of iteration

E(u

)

re−weighted l2 ( with fixed weight) + l1 sparsity

EM+l1KL(SB)+l1re−weighted l2initial point

Figure 3: Left: KL−divergence (E(u) = 1T (u+ c)−fT log(u+ c)+λ‖Wu‖1) energy evolution of

Algorithm 1, (15) and (16). Right: reweighted `2-norm energy (E(u) = 12

∥∥∥u+c−f√u∗+c

∥∥∥2

2+λ‖Wu‖1)

evolution of Algorithm 1, (15) and (16).

13

4.2 Deblurring from Poisson data

The deblurring case is in general more difficult than denoising. We compare our reweighted`2 method and the KL-divergence model (14) since Ancombre transform can not be easilyapplied in this case. For this application, we compare our proposed Algorithm 1 with twodifferent algorithms (EM+`1 algorithm (15) and KL-split Bregman algorithm (16)) for theKL-divergence model (14). In this subsection, blurred and noisy images are simulated in thefollowing way. The clean images are first rescaled to an intensity range from 0 to 1200; thenthey are corrupted by a disk blurring kernel of radius 3 with symmetric boundary condition;finally, the Poisson noise is simulated in the same way as in the denoising case.

The main computation cost of deblurring Algorithm 1 is that uk+1 is solved by conjugategradient method instead of a direct inversion in the first step of the iteration. As shown in Figure4, the result of the KL-divergence fidelity algorithm is more blurry in contrast to the result ofweighted `2 fidelity, although they have similar PSNR, see Table 2 for more comparisons. Table2 shows that our re-weighted method (26) and KL-split Bregman algorithm (16) have higherPSNR value than EM+`1 algorithm (15). With the postprocess of bilateral filter, our reweightedmethod (26) can obtain the best PSNR value among all the algorithms.

Figure 4: Deblurring results of simulated noisy-blurred ”Goldhill” image with peak intensity1200. First row from left to right: ground truth image, blurry image, EM+`1 model (15).Second row from left to right: KL-split Bregman algorithm (16), direct reweighted `2 method(26) and reweighted method (26) with bilateral filter post-process.

reweighted `2 with bilateral filter reweighted `2 KL-Split Bregman EM+l1

goldhill 26.82 26.76 26.74 26.62cameraman 26.64 26.33 26.41 25.84boat 25.70 25.59 25.39 25.08

Table 2: PSNR value of the deblurring result.

14

Figure 5: Deblurring results of simulated noisy-blurred ”Cameraman” image with peak intensity1200. First row from left to right: ground truth image, blurry image, EM+`1 model (15).Second row: zoom-in images of the first row. Third row from left to right: KL-split Bregmanalgorithm (16), direct reweighted `2 method (26) and reweighted method (26) with bilateralfilter post-process. Fourth row: zoom-in images of the third row.

15

In Figure 6, we compare the energy evolution of different algorithm. According to ourprevious argument, Algorithm 1, if converge, converges to the solution of the model (14). Wethus compare the KL-divergence model E(u) = 1T (Au+ c)− fT log(Au+ c) + λ‖Wu‖1 of thethree algorithms (EM-`1, KL-split Bregman, and the proposed Algorithm 1). We can see thatthe energy for our proposed Algorithm 1 has the fastest speed of convergence. The evolution

of weighted `2-norm energy E(u) = 12

∥∥∥Au+c−f√Au∗+c

∥∥∥2

2+ λ‖Wu‖1 is also shown in the right figure of

Fig. 6, where u∗ is set as the ground truth image. To reach the same PSNR level of the resultgiven by the KL-Split Bregman algorithm after 50 iterations, the proposed Algorithm 1 needsmuch less iterations (less than 5 iterations), See Fig. 7.

Figure 6: Top: KL−divergence (E(u) = 1T (Au+ c) − fT log(Au+ c) + λ‖Wu‖1) energy evo-lution of ”Goldhill” image for (15), (16) and Algorithm 1. Bottom: reweighted `2-norm energy

(E(u) = 12

∥∥∥Au+c−f√Au∗+c

∥∥∥2

2+λ‖Wu‖1) evolution of ”Goldhill” image for (15), (16) and Algorithm 1.

In terms of computational time of deblurring, the reweighted algorithms (1) needs around100s while the EM-`1 around 240s for the same relative error stopping criteria. As a result,our proposed reweighted method (26) can outperform the EM-`1 in terms of visual quality,PSNR value, convergence speed and computational time. Moreover, compared with the KL-split Bregman algorithm (16), our proposed Algorithm 1 has faster convergence speed in termsof both the KL-energy and reweighted `2-norm energy.

4.3 Mixed Poisson-Gaussian noise restoration

In this section, we test the extension of reweighted `2 model to mixed Poisson-Gaussian noisefor some synthesized data and real photos. The synthesized mixed noised image is simulated by

16

Figure 7: PSNR evolution of ”Goldhill” image for three algorithms: (15), (16) and Algorithm1.

adding first Poisson noise to a rescaled image with peak intensity 120 and the Gaussian noiseof σ = 12. Fig. 8 shows the denoising result of model (31) with A being identity operator.

Figure 8: Left: input noisy image with mixed Poisson-Gaussian noise, peak intensity: 120,PSNR: 18.59dB; right: denoised by model (31), PSNR: 27.42dB.

In addition, we perform the mixed denoising model on a digital photo taken in a low lightenvironment with high ISO setting. Our denoising result is compared with the embedded noisereduction algorithm in Sony DSLR-A700 camera in Figure 9, and we can see that our resulthas less noise and better quality.

For deblurring case, we generate blurred image with peak intensity 1200 and then corruptit with Poisson noise, Gaussian noise of σ = 12 consecutively. See Fig. 10 for the deblurringresult of model (31).

5 Conclusion

In this paper, we studied weighted `2 fidelity derived from Gaussian approximation of Poissonstatistics. In solving denoising and deblurring models, we proposed a reweighted algorithmbased on Split Bregman iteration. The solution of the proposed algorithm is same as theminimizer of KL-divergence fidelity model (14) if the sequence converges. Although we showthe convergence of the sequence can be shown under some conditions, but the validity of these

17

Figure 9: Top: real photo; middle: output of camera using the noise reduction option. bottom:denoised by model (31).

18

disk,3 ; PSNR:22.61dB PSNR:26.00dB

disk,5; PSNR:20.83dB PSNR:24.51dB

Figure 10: left: input blurred image with mixed Poisson-Gaussian noise, σ = 12, peak intensity:1200; right: deblurred by model (31).

19

conditions is not easy to check. We rely on our numerical experiments to further verify theperformance of our algorithm; the proposed algorithm gives competitive result with respect toclassical fidelity terms in denoising and deblurring simulations. In addition, the reweighted `2

model can be easily extended to the mixed Poisson-Gaussian noise case. As shown by numericalresults, the reweighted `2 with split Bregman framework is promising in a wide perspective.

Both the algorithm proposed and the theoretical results shown in this paper are an attemptto study and link different variational models in image processing. In this study, we only focusedon the fidelity term, using a simple sparsity term as regularization, without utilizing the fullimage structure for the analysis of Poisson statistics. One possible extension of our currentwork would be incorporating the image structure and utilizing the spatial correlation of thepixel values to build a more precise statistical model, hence a more precise fidelity term, of thePoisson data. This might substantially improve the result as suggested by the improvementmade of a post bilateral filtering process in our deblurring simulation.

Appendix

For the sake of notation simplicity, we drop the constant c in the observation model (3) inthe following, but instead substitute the nonnegative constrain u ≥ 0 by u ≥ c. Under theassumption that A > 0, A1 = 1, the minimizer u∗ to this modified minimization problem

minu≥c

1TAu− fT log(Au) + λ‖Du‖1 (32)

can be rewritten as u∗ + c, where u∗ is the minimizer of (14). Therefore, (32) is essentially thesame as model (14) up to the constant c that is prefixed. We also consider a variant of the splitBregman algorithm (27) by adding a parameter δb in the third step bk+1 = bk+δb(Wuk+1−dk+1).In the following theorem, we consider 1 ≥ δb > 0, which is set to 1 in our algorithm.

Theorem 5.1 Assume that u∗ is an arbitrary minimizer of (14). Assume that λ > 0, µ >0, 1 ≥ δb > 0, A > 0, A1 = 1 and ∃K0, δ1, δ2 s.t. 2δ1 + δ2 ≤

√c,∥∥∥∥A(uk − u∗)√

Au∗

∥∥∥∥∞≤ δ1 ≤

√c, ∀k > K0 (33)

and ∥∥∥∥f −Au∗√Au∗

∥∥∥∥∞≤ δ2 (34)

Then we have the following properties:

limk→+∞

Auk = Au∗ (35)

limk→+∞

1

2

∥∥∥∥Auk − f√Auk

∥∥∥∥2

2

=1

2

∥∥∥∥Au∗ − f√Au∗

∥∥∥∥2

2

limk→+∞

‖Wuk‖`1 = ‖Wu∗‖`1 (36)

In addition, u∗ is also an minimizer of (26). Furthermore,

limk→+∞

‖uk − u∗‖ = 0 (37)

whenever u∗ is the unique solution of (26).

20

Proof. Let u∗ be an solution of (14). By the first order optimality condition, u∗ must satisfy

0 = AT(1− f

Au∗

)+ λWT p∗ − q∗

= AT(Au∗ − fAu∗

)+ λWT p∗ − q∗

(38)

where p∗ ∈ ∂|d∗| with d∗ =Wu∗ and q∗ ≥ 0 is a Lagrange multiplier satisfying

〈q∗, u∗ − c〉 = 0 (39)

Let

Σ∗ = diag(Au∗) and b∗ =λ

µp∗.

We have 0 = ATΣ−1

∗ (Au∗ − f) + µWT (Wu∗ − d∗ + b∗)− q∗

0 = λp∗ + µ(d∗ −Wu∗ − b∗), with p∗ ∈ ∂|d∗|b∗ = b∗ + δb(Wu∗ − d∗)

(40)

In the rest of this proof, we abuse the notation Σ∗ to either stand for the matrix withdiagonal entries Au∗ or the vector Au∗ when there is no confusion. Therefore, the condition 34can be rewritten as ∥∥∥∥f − Σ∗√

Σ∗

∥∥∥∥∞≤ δ2 (41)

Since u∗ ≥ c and by the assumption that A ≥ 0, A1 = 1, the diagonal elements of Σ∗ areuniformly lower bounded by c, i.e. Au∗ ≥ Ac1 = c1 or equivalently

‖Σ∗‖∞ ≥ c. (42)

Let Σk = Auk and rewrite the algorithm (27) into the following form,0 = ATΣ−1

k (Auk+1 − f) + µWT (Wuk+1 − dk + bk)− qk+1

0 = λpk+1 + µ(dk+1 −Wuk+1 − bk), with pk+1 ∈ ∂|dk+1|bk+1 = bk + δb(Wuk+1 − dk+1)

(43)

〈qk, uk − c〉 = 0 (44)

Denote the errors by

uek = uk − u∗, dek = dk − d∗, bek = bk − b∗.

Then the condition (33) can be rewritten as∥∥∥∥Auek√Σ∗

∥∥∥∥∞≤ δ1 (45)

Subtracting the first equation of (43) by the first equation of (40), we have

0 = AT (Σ−1k − Σ−1

∗ )(Auk+1 − f) +ATΣ−1∗ Auek+1

+ µWT (Wuek+1 − dek + bek)− (qk+1 − q∗)

Take the inner product of the both sides with respect to uek+1, we obtain

0 = 〈uek+1,AT (Σ−1

k − Σ−1∗ )(Auk+1 − f)〉+ ‖Auek+1‖2Σ−1

∗+ µ‖Wuek+1‖22

− µ〈uek+1,WTdek〉+ µ〈uek+1,WT bek〉 − 〈qk − q∗, uk+1 − u∗〉(46)

21

The same manipulation applied to the second equations of (40) and (43) leads to

0 = λ〈pk+1 − p∗, dk+1 − d∗〉+ µ‖dek+1‖22 − µ〈dek+1,Wuek+1〉 − µ〈dek+1, bek〉 (47)

We can also do the similar operation to the third equations of (40) and (43) and obtain

〈bek,Wuek+1 − dek+1〉 =1

2δb(‖bek+1‖22 − ‖bek‖22)− δb

2‖Wuek+1 − dek+1‖22. (48)

Using (39) and (44), the last term in (46) becomes

− 〈qk − q∗, uk+1 − u∗〉 = 〈qk, u∗ − c〉+ 〈q∗, uk+1 − c〉 (49)

Adding (47) to (46) and utilizing (49), we get

0 = 〈uek+1, AT (Σ−1

k − Σ−1∗ )(Auk+1 − f)〉+ ‖Auek+1‖2Σ−1

∗

+ λ〈pk+1 − p∗, dk+1 − d∗〉+ µ(‖Wuek+1‖22 + ‖dek+1‖22− 〈dek+1 + dek,Wuek+1〉+ 〈Wuek+1 − dek+1, b

ek〉) + 〈qk, u∗ − c〉+ 〈q∗, uk+1 − c〉

(50)

Substituting (48) into (50), we have

µ

2δb(‖bek‖22 − ‖bek+1‖22) =

〈uek+1, AT (Σ−1

k − Σ−1∗ )(Auk+1 − f)〉+ ‖Auek+1‖2Σ−1

∗

+ λ〈pk+1 − p∗, dk+1 − d∗〉+ µ(‖Wuek+1‖22 + ‖dek+1‖22

− 〈dek+1 + dek,Wuek+1〉 −δb2‖Wuek+1 − dek+1‖22) + 〈qk, u∗ − c〉+ 〈q∗, uk+1 − c〉.

(51)

Expanding the first term on the right hand side,

〈uek+1, AT (Σ−1

k − Σ−1∗ )(Auk+1 − f)〉

= 〈Auek+1,

(1

Auk− 1

Au∗

)(Auk+1 − f)〉

= 〈Auek+1,1

Au∗Auk+1 − f

AukA(uk − u∗)〉

= 〈Auek+1,Auk+1 − f

AukAuek〉Σ−1

∗

= 〈Auek+1,Auek+1 +Au∗ − f

Auek +Au∗Auek〉Σ−1

∗

= 〈Auek+1,Auek+1

Auek + Σ∗Auek〉Σ−1

∗+ 〈Auek+1,

Σ∗ − fAuk + Σ∗

Auek〉Σ−1∗

(52)

Note that all the operations between vectors in (52) are element wise except the inner product.Using (42), (41) and (45) we have the following estimation,

∥∥∥∥ Auek+1

Auek + Σ∗

∥∥∥∥∞≤

∥∥∥Auek+1√Σ∗

∥∥∥∞

−∥∥∥ Auek√

Σ∗

∥∥∥∞

+√

Σ∗≤ δ1√

Σ∗ − δ1≤ δ1√

c− δ1, k ≥ K0 (53)

and ∥∥∥∥ Σ∗ − fAuek + Σ∗

∥∥∥∥∞≤∥∥∥∥Σ∗ − f√

Σ∗

∥∥∥∥∞

∥∥∥∥∥∥ 1Auek√

Σ∗+√

Σ∗

∥∥∥∥∥∥∞

≤ δ2√c− δ1

(54)

22

Let δ = δ1+δ2√c−δ1

≤ 1, then

‖〈uek+1, AT (Σ−1

k − Σ−1∗ )(Auk+1 − f)〉‖

≤‖〈Auek+1,Auek+1

Auek + Σ∗Auek〉Σ−1

∗‖+ ‖〈Auek+1,

Σ∗ − fAuk + Σ∗

Auek〉Σ−1∗‖

≤∥∥∥∥ Auek+1

Auek + Σ∗

∥∥∥∥∞〈|Auek+1|, |Auek|〉Σ−1

∗+

∥∥∥∥ Σ∗ − fAuek + Σ∗

∥∥∥∥∞〈|Auek+1|, |Auek|〉Σ−1

∗

≤ δ1 + δ2√c− δ1

〈|Auek+1|, |Auek|〉Σ−1∗

≤δ2

(‖Auek+1‖2Σ−1∗

+ ‖Auek‖2Σ−1∗

).

(55)

With u∗ ≥ c, uk ≥ c, q∗ ≥ 0 and qk ≥ 0, we also have

〈qk, u∗ − c〉+ 〈q∗, uk+1 − c〉 ≥ 0 (56)

By substituting (55) and (56) into (51) and summing up from k = 0 to k = K, we get

µ

2δb(‖be0‖22 − ‖beK+1‖22) +

K0−1∑k=0

{〈uek+1, A

T (Σ−1k − Σ−1

∗ )(Auk+1 − f)〉+ ‖Auek+1‖2Σ−1∗

}≥δ

2

(‖AueK+1‖2Σ−1

∗− ‖AueK0

‖2Σ−1∗

)+ (1− δ)

K∑k=K0

‖Auek+1‖2Σ−1∗

+ λK∑k=0

〈pk+1 − p∗, dk+1 − d∗〉 −µ

2‖de0‖22

+µ

2‖deK+1‖22 + µ

(1− δb

2

K∑k=0

‖Wuek+1 − dek+1‖22 +1

2

K∑k=0

‖Wuek+1 − dek‖22

)(57)

By rearranging the terms on both sides of (57) and eliminate, we obtain

µ

2δb‖be0‖22 +

δ

2‖AueK0

‖2Σ−1∗

+µ

2‖de0‖22

≥ (1− δ)K∑

k=K0

‖Auek+1‖2Σ−1∗

+ λ

K∑k=0

〈pk+1 − p∗, dk+1 − d∗〉+µ

2

K∑k=0

‖Wuek+1 − dek‖22

+

K0−1∑k=0

{〈uek+1, A

T (Σ−1k − Σ−1

∗ )(Auk+1 − f)〉+ ‖Auek+1‖2Σ−1∗

}(58)

Since pk+1 ∈ ∂|dk+1| and p∗ ∈ ∂|d∗|, together with the convexity of | · |, we have

〈pk+1 − p∗, dk+1 − d∗〉 ≥ 0

Hence all terms on the right hand side of (58), except the last finite summation, are non-negativewith the assumptions of δ < 1, µ > 0, λ > 0, and our first observation is as follows

∞∑k=0

‖Auek+1‖2Σ−1∗< +∞

which leads tolim

k→+∞‖Auk+1 −Au∗‖2Σ−1

∗= 0 (59)

23

Therefore, (35) holds as the weight Σ−1∗ is positive. This immediately leads to the first part

of (36). With (35), the minimization problem (26) is well-defined. Note that the first orderoptimality condition of (26) is the same as (38) and (39) hence u∗ is an minimizer of (26).Likewise we have

limk→+∞

‖Wuek+1 − dek‖ = 0

With Wu∗ = d∗, we further get

limk→+∞

‖Wuk+1 − dk‖ = 0 (60)

(58) also giveslim

k→+∞〈pk − p∗, dk − d∗〉 = 0 (61)

While notice that Bp∗

|·| (dk, d∗) +Bpk

|·| (d∗, dk) = 〈pk − p∗, dk − d∗〉, where Bp

J(u, v) with p ∈ ∂J(v)is the Bregman distance defined on convex function J . As the Bregman distance is alwayspositive, (61) together with (60) results in

limk→+∞

|Wuk| − |Wu∗| − 〈W(uk − u∗), p∗〉 = 0

Using (59), we have

〈W(uk − u∗), p∗〉 = 〈W(uk − u∗), p∗〉+1

λ〈A(uk − u∗), Au∗ − f〉Σ−1

∗

=1

λ〈uk − u∗, ATΣ−1

∗ (Au∗ − f) + λWT p∗〉 = 0

The last equality comes from (38). Therefore,

limk→+∞

|Wuk| − |Wu∗| = 0 (62)

which is the second equation in (36).Next, assume that u∗ is the unique solution of 26. Let E(u) = 1

2 ‖Au− f‖2Σ−1∗

+ λ‖Wu‖`1 . It

is clear that E(u) is convex, lower semi-continuous. Therefore, the uniqueness of the solutionleads to (37).

Remark 1 In Theorem 5.1, the convergence requires conditions (33) and (34). Although thetwo conditions are in the form of infinity norm, which enforces a global bound, we can actuallyrestrict ourselves in a local image patch, or in a sub-region of the image domain, because all theelement-wise operations and the operator A are local; in other words, these two conditions arelocal uniform conditions. For Poisson noise n = Au− f of mean Au, the standard variation ofthe noise is of order O(

√Au), i.e.

Au− f ∼ O(√Au).

For rate Au bigger than 10, that is where the pixel value is big enough, the Poisson distributioncan be well approximated by a Gaussian distribution with the same mean and variance. SinceP(|x| < 3σ) ≈ 1 if x ∼ N(0, σ2), |n| = |Au − f | < 3

√Au with high probability. This implies

that when c is of the order of 10 or bigger, then Au > c is big as well, condition (34) holds forδ2 = 3 with high probability. The other condition (33) is much harder to check, which involvesuk and we cannot gaurantee that Auk is a better approximation of Au∗ than f pointwise. If c islarge, that is the pixel value is large, then we can choose δ1 to be big enough so that (33) holdswith high probability. But for the low intensity part, (33) and (34) may fail. This matches ourobservation in numerical simulations that the algorithm performs worse in the lower intensityregion than in the higher one.

24

Acknowledgments

Xiaoqun Zhang was partially supported by the NNSFC (Grant nos. 11101277, 91330102) andthe Shanghai Pujiang Talent Program (Grant no. 11PJ1405900). Zuowei Shen was partiallysupported by National University of Singapore (Grant no. MOE2011-T2-1-116).

References

[1] F. J. Anscombe. The Transform of Poisson, Binomial and Negative-Binomial Data.Biometrika, 35 (1948).

[2] M. Bertero, P. Boccacci, G. Desidera and G. Vicidomini. Image deblurring with Poissondata: from cells to galaxies. Inverse Problems, 12(25)(2009).

[3] M. Bertero, P. Boccacci, G. Talenti, R. Zanella and L. Zanni. A discrepancy principle forPoisson data. Inverse Problems, 26(10) (2010).

[4] P. J. Bickel and E. Levina. Regularized estimation of large covariance matrices. Ann.Statistics, 36 (2008),199–227.

[5] C. Brune, A. Sawatzky and M. Burger. Bregman-EM-TV Methods with Application toOptical Nanoscopy. In Proc. SSVM 2009, LNCS, Springer, 5567 (2009).

[6] C. Brune, A. Sawatzky and M. Burger. Primal and Dual Bregman Methods with Applicationto Optical Nanoscopy. Int J Comput Vis, 72 (2011), 211–229.

[7] C. Brune, M. Burger, A. Sawatzky, T. Kosters and Frank Wubberling. Forward-BackwardEM-TV methods for inverse problems with Poisson noise. Prepreint, August 2009.

[8] J.-F. Cai, S. Osher and Z. Shen. Linearized Bregman iterations for compressed sensing.Mathematics of Computation, 78 (2009) , 1515-1536.

[9] J.-F. Cai, S. Osher and Z. Shen. Split Bregman Methods and Frame Based Image Restora-tion. Multiscale Modeling & Simulation, 8(2) (2009) , 337-369.

[10] J.-F. Cai, B. Dong, S. Osher and Z. Shen Image Restoration: Total Variation; WaveletFrames; and beyond J. Amer. Math. Soc., 25(4) (2012), pp. 1033-1089.

[11] Imre Csiszar. Why Least Squares and Maximum Entropy? An Axiomatic Approach toInference for Linear Inverse Problems. Ann. Statist. 19(4) (1991) , 2032-2066.

[12] I. Daubechies, B. Han, A. Ron and Z. Shen. Framelets: MRA-based constructions ofwavelet frames. Applied and Computational Harmonic Analysis, 14(2003), 1-46.

[13] B. Dong and Z. Shen. MRA-based wavelet frames and applications, IAS Lecture NotesSeries. Park City Mathematics Institute, expected to appear in 2012 (manuscript revisedon Sep 28, 2011).

[14] B. Dong and Y. Zhang. An efficient algorithm for l0 minimization in wavelet frame basedimage restoration. Journal of Scientific Computing, to appear.

[15] D. Donoho Compressed sensing. Information Theory, IEEE Transactions on, 52(4) (2006)1289-1306.

[16] E. Esser. Applications of Lagrangian-Based Alternating Direction Methods and Connectionsto Split Bregman. UCLA CAM report 09-31,2009.

25

[17] E. Esser, X. Zhang X and T.-F. Chan. A General Framework for a Class of First OrderPrimal-Dual Algorithms for Convex Optimization in Imaging Science. SIAM J. Imag. Sci.3(4) (2010) 1015-1046.

[18] M. Figueiredo and J. Bioucas-Dias. Restoration of Poissonian images using alternatingdirection optimization. IEEE Transactions on Image Processing, to appear

[19] F. Luisier, T. Blu and M. Unser. Image denoising in mixed Poisson-Gaussian niose. Imagerestoration, IEEE Transcations, 20 (2011), 696-708.

[20] J. Friedman, T. Hastie and R. Tibshirani. Sparse inverse covariance estimation with thegraphical lasso. Biostatistics,9(2008), 432–441.

[21] T. Goldstein, Brendan O’Donoghue, and Simon Setzer, Fast Alternating Direction Opti-mization Methods. UCLA CAM report 12-35. May 2012.

[22] T. Goldstein and S. Osher. The split bregman method for l1 regularized problems. SIAMJournal on Imaging Sciences, 2(2) (2009), 323-343 .

[23] Z. Gong, Z. Shen and K.-C. Toh. Image Restoration with Mixed or Unknown Noises.preprint, private communication.

[24] K. Lange and R. Carson. EM reconstruction algorithms for emission and transmissiontomography. J Comput Assist Tomogr. 8(2), pp.306-16 (1984).

[25] F. Luisier, T. Blu and M. Unser Image denoising in mixed Poisson-Gaussian niose IEEETransactions on Image Processing, vol. 20, no. 3, pp. 696-708, March 2011

[26] Y. Nesterov. A method of solving a convex programming problem with convergence rateo(1/k2). Soviet Math. Dokl., 27 (1983) :372C376.

[27] R. Rockafellar, Augmented lagrangians and applications of the proximal point algorithmin convex programming. Mathematics of Operations Research. 1(2) (1976). 97-116.

[28] A. Ron, Z. Shen. Affine Systems in L2(Rd): The Analysis of the Analysis Operator.Journal of Functional Analysis, 148(2)(1997), 408–447.

[29] L.I. Rudin, S. Osher and E. Fatemi Nonlinear total variation based noise removal algorithmsPhysics D, 60 (1992), 259–268.

[30] S. Setzer Split Bregman Algorithm, Douglas-Rachford Splitting and Frame Shrinkage, inSSVM ’09 Proceedings of the Second International Conference on Scale Space and Varia-tional Methods in Computer Vision, 2008.

[31] S. Setzer, G. Steidl, and T. Teuber. Deblurring Poissonian images by split Bregmantechniques. Journal of Visual Communication and Image Representation, 21 (2010), pp.193 - 199.

[32] L. A. Shepp and Y. Vardi. Maximum Likelihood Reconstruction in Positron EmissionTomography. IEEE Transactions on Medical Imaging, 1 (1982), 113-122.

[33] D. L. Snyder, A. M. Hammoud, and R. L. White. Image recovery from data acquired witha charge-coupled-device camera. J. Opt. Soc. Am. A 10(1983), 1014-1023.

[34] A. Stagliano, P. Boccacci and M. Bertero. Analysis of an approximate model for Pois-son data reconstruction and a related discrepancy principle. Inverse Problems 27 (2011).doi:10.1088/0266-5611/27/12/125003.

26

[35] C. Tomasi and R. Manduchi Bilateral filtering for gray and color images Sixth InternationalConference on Computer Vision (1998).

[36] H. Yu and G. Wang Compressed sensing based interior tomography Physics in medicineand biology 54(9)(2009), 2791.

[37] X. Zhang, M. Burger and S. Osher. A Unified Primal-Dual Algorithm Framework Basedon Bregman Iteration. Journal of Scientific Computing, 46 (1) (2010), 20-46.

27