IEEE SIGNAL PROCESSING LETTERS, VOL. 19, NO. 8, AUGUST 2012 535

A New Weight for Nonlocal MeansDenoising Using Method Noise

Hua Zhong, Member, IEEE, Chen Yang, and Xiaohua Zhang, Member, IEEE

Abstract—Method noise, i.e., the difference between the noisyimage and its denoised version, often contains residual imageinformation due to imperfect denoising methods. In this letter,a novel weight is derived for nonlocal means (NLM) denoising,which tries to exploit the nonlocal similarities of residual image inmethod noise, and estimates directly the similarities between noisefree patches instead of the commonly used similarity measurebased on noisy observations or their pre-denoised versions. Thedenoising scheme is implemented in two stages: the first stage isthe original NLM, and the second stage is the NLM with the newweight that incorporates the role of both the pre-denoised resultand the method noise produced in the previous stage. Experi-mental results demonstrate the robustness of the proposed weightand its potential with respect to classical NLM methods and otherstate-of-the-art methods.

Index Terms—Method noise, nonlocal means (NLM), patch sim-ilarity, weight.

I. INTRODUCTION

N ONLOCAL principle, which exploits the nonlocal simi-larity between patches in one image, has attracted a lot

of attention in recent years. Since the nonlocal means (NLM)was first proposed by Buades et al. [1], numerous methods havebeen developed for better denoising performance, among whichsimilarity measure for patches is one of most studied problems.Different weight functions have been developed for the task

of patch similarity measure, and also the related optimizationproblem of several important parameters. Kervrann et al. [2]proposed a Bayesian NLM framework that measures the noisypatches similarities based on the statistical distribution of noise.Deledalle et al. [3] also presented a probabilistic patch based(PPB) weight function, which however updates the weights in adata-driven way using both the noisy patches and iteratively fil-tered results. The principle neighborhood dictionary (PND) pro-posed by Tasdizen et al. [4], as a transformation domain basedmethod, measures the patch similarity in the domain of prin-ciple component analysis (PCA). To deal with the optimizationproblem of the NLM parameters, Van De Ville et al. [5] usedthe Stein’s unbiased risk estimate (SURE) to control the meansquare error (MSE) for image denoising and thus select the op-

Manuscript received May 01, 2012; revised June 14, 2012; accepted June19, 2012. Date of publication June 21, 2012; date of current version July 03,2012. This work was supported by the National Natural Science Foundationof China under Grants 61173092, 61072106, 60972148, 60971128, 60970066,61003198, 61001206, 61077009, and 61050110144), and by the Fund for For-eign Scholars in University Research and Teaching Programs (the 111 Project)under Grant B07048. The associate editor coordinating the review of this man-uscript and approving it for publication was Prof. Giuseppe Scarpa.The authors are with the Key Laboratory of Intelligent Perception and Image

Understanding of the Ministry of Education of China, Xidian University, Xi’an710071, China (e-mail: hzhong@mail.xidian.edu.cn; yangchenxd@163.com;xh_zhang@mail.xidian.edu.cn).Digital Object Identifier 10.1109/LSP.2012.2205566

timal smoothing factor for the exponential kernel of the weightfunction. Salmon [6] gave the valuable discussion on the choicesof another two parameters including the size of the searchingwindow and the weight given to the central patch.Though different in the form of the weight functions, most

NLM methods obtain the weights by simply comparing thenoisy patches [1], [4]–[6]. To reduce the effect of noise,pre-denoised image can be used to get more accurate similaritymeasure such as in [2], [3], and some more complicated non-local methods such as RIBM [7], BM3D [8] and LSSC [9].However, analysis on the method noise [1] shows that image

details are often blurred due to imperfect denoising methods,whichmeans that pre-denoised image does not contain completeimage detail, and thus may lead to inaccurate weight computa-tion. On the other side, nonlocal similarity of residual imagestructures in method noise can be further exploited to improvethe accuracy of the weight.In this letter, a new weight is derived for nonlocal means

(NLM) denoising, which exploits the joint nonlocal similari-ties between method noise and pre-denoised image. The sim-ilarities between noise free patches can be robustly measuredwith higher accuracy by using the new weights, in contrast tothe weights that measure the similarities between noisy imagepatches or the pre-denoised ones. A two stages NLM scheme isimplemented, which shows better denoising performance in thesecond stage and also the robustness with respect to differentparameters in the first stage.In Section II, original NLM are briefly reviewed with the

weight and the method noise analyzed. The new weight andthe two-stage NLM scheme are presented in Section III. Exper-imental results and conclusions are given in Sections IV and V,respectively.

II. NLM WEIGHTS AND METHOD NOISE

Considering the additive white Gaussian noise (WGN)model

(1)

where is the pixel index, is the zero mean noise of varianceand denote the pixel values in noise free and noisy

image, respectively. In addition, the patches centered at pixelare defined as and , respectively. Then, the NLM esti-mation is defined as the follows [1]:

(2)

where is the search window around pixel . Note that de-notes that the weight is computed using the noisy patchesand , which can be defined as:

(3)

1070-9908/$31.00 © 2012 IEEE

536 IEEE SIGNAL PROCESSING LETTERS, VOL. 19, NO. 8, AUGUST 2012

Fig. 1. NLM denoising result under . (a) Denoised image and(b) method noise.

where is the smoothing factor. denotes the Euclideannorm to measure patch similarities1. For another round of de-noising, one can replacing in (3) with ,which results in another kind of weight that measures thesimilarity of pre-denoised patches and .Fig. 1(a) shows the original NLM denoising result using theunder the noise standard deviation , and Fig. 1(b)

is the corresponding method noise, which appears obviousimage structures that are similar with those in Fig. 1(a). Herethe method noise is denoted as , defined as [1]:

(4)

The fact that some image detail are removed and left in themethod noise reveals not only the shortage of the weight ,but also, to some extent, the unreliability of the weight dueto its bias to the noise free image, which, on the other hand, im-plies that it is necessary and yet feasible to exploit the nonlocalsimilarities in method noise for better weight.

III. THE PROPOSED METHOD

A. New Weight Using Method Noise

In this section, we show how to approximately estimate thesimilarities between noise free patches, i.e., the Euclidean norm

by the use of method noise and pre-denoised, andthus obtain the new weight.Combining (1) and (4), the noise free pixels value can be

written as:

(5)

For simplicity, we assume that the is completely included inthe method noised due to the smoothing effect of the NLMdenoising, that is, the pre-denoised image contains no noise.Then (5) can be rewritten as

(6)

where denotes the residual image signal inmethod noise. Note that we can not get good denoising result bysimply adding back to , since can only be obtainedas an coarse approximation.

1The Gaussian kernel used in [1] is neglected for simplicity

Based on (6), the similarity between noise free patchesand , i.e., the Euclidean norm , can be directlyexpressed as:

(7)

where the bold letters denote the corresponding patches and isthe index in the patches. From (7), we can see that the contribu-tion of the pre-denoised patches, i.e., the first term onthe right side has greater power than the other two terms. How-ever, if the residual signal is not zero, the norm alsodepends on the second term and the third term

, which exploits the nonlocal sim-ilarity of the residual signal and its relationship with the pre-de-noised image.Then the next problem is how to get the estimation of the

residual signal . Since the restoration of from themethod noise is very hard due to the very low Signal-noiseratio (SNR), another choice is to smooth the noise as muchas possible while keeping the completeness of the residualimage, so that the estimation of the residual image is just “lesscorrupted by the noise”. For simplicity, 3 3 mean filter isadopted here, which has the greatest power of smoothing noisewhen compared with other filters with the same filter size, andcan also satisfy the requirement of well preserving the residualimage due to its small size.Finally, replacing in (3) with estimated

using (7), the new weight can be obtained as:

(8)

where denotes the smoothing factor for .

B. Two-Stage NLM Scheme and Parameter Analysis

A two-stage NLM scheme is used for the implementation ofthe proposed method, as shown in Fig. 2. The first stage is justthe original NLM denoising [1], which uses the weight in(3) and outputs the pre-denoised image and the method noise.In the second stage, the NLM denoising is implemented againbut with the weight in (8).In the proposed two-stage NLM scheme, the main parameters

are the two smoothing factors and . Since the optimal choicefor in the first stage has been discussed in many references[1]–[3], [5]2, we only focus on the optimal choice for in thesecond stage. Following the way in [4], the relationship betweenand the noise standard deviation can be formulated as a

linear function, expressed as:

(9)

2In fact, our tests in Section IV show that the final results are robust withrespect to the different values of .

ZHONG et al.: A NEW WEIGHT FOR NONLOCAL MEANS DENOISING USING METHOD NOISE 537

Fig. 2. Flowchart of the proposed two-stage NLM denoising scheme.

Fig. 3. The linear fit of optimal values as the function of . “o” on each bardenotes the mean of the optimal for four test images, and “*” denotes theminimum or maximum value of the optimal .

where the parameters and are estimated empirically by thelinear fit between the optimal and .

IV. EXPERIMENTAL RESULTS

In this section, detailed results on four grayscale images i.e.,Barbara, Lena, Boat and House are presented. And for differentNLM methods including ours, the size of the patch and thesearch window are 7 7 and 21 21, respectively, as given in[1].

A. Effects of the New Weight

For the weight of the original NLM (also used in the firststage of our scheme), the smoothing factor often takes a valueof [1], and equals to 5 in this letter. Then the factor for theweight is estimated using (9) for different noise standarddeviation . Fig. 3 shows the best fit of optimal values as thefunction of , where each bar denotes the range of the optimalvalues for four test images. The estimated factors in (9) are

and .An interesting fact is the robustness of with regard to the

different values of . Fig. 4 plots the PSNR values of second-stage results versus different values of for the factor . We cansee that all the PSNR curves arise as increases but becomesmooth when for different noise. The reason may beexplained using (7), as that, though different values lead todifferent ways of separation between the pre-denoised imageand the residual image , as indicated in (6) and (7), the jointcontribution of and to the norm is almost thesame.To further demonstrate the advantage of the new weight ,

the comparison of the weight distribution maps, which drawsthe weight values of the pixels in a neighborhood, is shown inFig. 5 for different methods under . Fig. 5(b) shows two

Fig. 4. PSNR (dB) of our method versus first stage NLM smoothing parameterfor Lena and Boat with different standard deviation .

Fig. 5. Comparison of the weight distribution maps under . Top rowand bottom row represent search windows 1 and 2, respectively. (a) Noisyimage, (b) Noisy patches, (c) NLM, (d) PPB with 25 iterations, (e) our method.

example search windows from Fig. 5(a), where the edges andtextures are seriously corrupted by the noise. The NLM weightdistribution map [1] shown in Fig. 5(c) is used to denote thekind of weight since they depend only on the noisy patches.The larger the weight value is, the brighter the pixel appears.Fig. 5(d) gives the weight distributionmap of the PPBwith 25 it-erations [3] as the representation of the kind of weight sincethe PPB weight makes a good balance between pre-denoisedimages and the noisy patches. Fig. 5(e) presents our weight dis-tribution map, where the edge and texture appear more contin-uous, and more pixels with similar edge or texture structures areassigned larger weight values than in Fig. 5(c) and (d), whichimplies that better denoising performance can be obtained.

B. Comparison With Other Denoising Methods

We first present both quantitative and qualitative comparisonsof the NLMmethods with different weights, including the NLM[1], PPB [3] and PND [4] filter, as shown in Table I, where thebold values are the best in the line. Since these methods obtaina pixel-wised estimation during denoising process, we denotethem as pixel-wised methods. From Table I we can see that ourmethod always outperforms the others, particularly when islarge. When compared with the NLM, the PSNR gains of ourmethod for four test images range from 0.7 dB to 1.4 dB when

, and range from 1.4 dB to 2.3 dB when .Besides, the patch-wised version of our method, as done in

[2], is also implemented for fair comparison with the methodsBM3D [8], K-SVD [10] and BNL [2]. These methods, denotedas patch-wised methods, estimate a patch one time rather than

538 IEEE SIGNAL PROCESSING LETTERS, VOL. 19, NO. 8, AUGUST 2012

TABLE IPSNR COMPARISON OF PIXEL-WISED METHODS

TABLE IIPSNR COMPARISONS OF PATCH-WISED METHODS

one pixel and then aggregates the multiple estimations of onepixel. Table II shows the PSNR comparison for patch-basedmethods, where the bold values denote the best among the lastthree columns. From Table II we can see that the superiorityof our method exists in most cases with respect to K-SVD andBNL, and becomes more obvious as increases. For example,for Lena image, the PSNR gains of our method over K-SVDand BNL are 0.57 dB and 0.48 dB, respectively, when .When , larger PSNR gains of 1.11 dB and 1.09 dB canbe obtained by our method, respectively. Though the results ofBM3D are always the best, our method is very close to BM3Dwhen for images Lena, Boat and House.Finally, Fig. 6 shows an example of the comparison between

the NLM and our method when . It is obviously that thedenoising result of our method (the top of Fig. 6(b)) containsmuch more details than the NLM (the top of Fig. 6(a)), such asthe edges of the hat and the texture of the hair. Correspondingly,when compared with the method noise of NLM (the bottom ofFig. 6(a)), we can notice that the method noise of our methodhas less image residual in these regions, as shown in the bottomof Fig. 6(b).

Fig. 6. (Top) Denoised image and (bottom) the corresponding method noisefor Lena when . (a) NLM, (b) Our method.

V. CONCLUSION

A new weight is presented for NLM denoising that exploitsthe nonlocal similarity of both the pre-denoised image and themethod noise. A two-stage denoising scheme is designed andits robustness is shown with respect to the smoothing factor. Re-sults are shown to demonstrate the effectiveness of the proposedmethod, especially for large noise variance. In future work, weplan to further exploit the residual image with much weakerenergy by an iterative way rather than the proposed two-stagescheme.

REFERENCES

[1] A. Buads, B. Coll, and J.-M. Morel, “A non-local algorithm for imagedenoisesing,” in Proc. IEEE CVPR, 2005, pp. 60–65.

[2] C. Kervrann, J. Boulanger, and P. Coupe, “Bayesian non-local meansfilter, image redundancy and adaptive dictionaries for noise removal,”in Proc. Int Conf. Scale Space Methods Variational Methods Comput.Vis., 2007, pp. 520–532.

[3] C. A. Deledalle, L. Denis, and F. Tupin, “Iterative weighted maximumlikelihood denoising with probabilistic patch-based weights,” IEEETrans. Image Process., vol. 18, no. 12, pp. 2661–2672, Dec. 2009.

[4] T. Tasdizen, “Principal neighborhood dictionaries for nonlocal meansimage denoising,” IEEE Trans. Image Process., vol. 18, no. 12, pp.2649–2660, Dec. 2009.

[5] D. Van DeVille andM. Kocher, “SURE-based non-local means,” IEEESignal Process. Lett., vol. 16, no. 11, pp. 973–976, Nov. 2009.

[6] J. Salmon, “On two parameters for denoising with non-local means,”IEEE Signal Process Lett., vol. 17, no. 3, pp. 269–272, Mar. 2010.

[7] R. Yan, L. Shao, S. D. Cvetkovic, and J. Klijn, “Improved nonlocalmeans based on pre- classification and invariant block matching,”IEEE/OSA J. Display Technol., vol. 8, no. 4, pp. 212–218, Apr. 2012.

[8] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoisingby sparse 3-d transform-domain collaborative filtering,” IEEE Trans.Image Process., vol. 16, no. 8, pp. 2080–2095, Aug. 2007.

[9] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Non-localsparse models for image restoration,” in Proc. IEEE ICCV, 2009, pp.2272–2279.

[10] M. Elad and M. Aharon, “Image denoising via sparse and redun-dant representations over learned dictionaries,” IEEE Trans. ImageProcess., vol. 15, no. 12, pp. 3736–3745, Dec. 2006.