Noise Reduction Based on Partial-Reference

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013 1859

Noise Reduction Based on Partial-Reference,Dual-Tree Complex Wavelet Transform Shrinkage

Massimo Fierro, Ho-Gun Ha, and Yeong-Ho Ha, Senior Member, IEEE

Abstract This paper presents a novel way to reduce noiseintroduced or exacerbated by image enhancement methods, inparticular algorithms based on the random spray samplingtechnique, but not only. According to the nature of sprays,output images of spray-based methods tend to exhibit noisewith unknown statistical distribution. To avoid inappropriateassumptions on the statistical characteristics of noise, a differentone is made. In fact, the non-enhanced image is considered to beeither free of noise or affected by non-perceivable levels of noise.Taking advantage of the higher sensitivity of the human visualsystem to changes in brightness, the analysis can be limited tothe luma channel of both the non-enhanced and enhanced image.Also, given the importance of directional content in human vision,the analysis is performed through the dual-tree complex wavelettransform (DTWCT). Unlike the discrete wavelet transform,the DTWCT allows for distinction of data directionality in thetransform space. For each level of the transform, the standarddeviation of the non-enhanced image coefficients is computedacross the six orientations of the DTWCT, then it is normalized.The result is a map of the directional structures present inthe non-enhanced image. Said map is then used to shrink thecoefficients of the enhanced image. The shrunk coefficients andthe coefficients from the non-enhanced image are then mixedaccording to data directionality. Finally, a noise-reduced versionof the enhanced image is computed via the inverse transforms.A thorough numerical analysis of the results has been performedin order to confirm the validity of the proposed approach.

Index Terms Dual-tree complex wavelet transform(DTWCT), image enhancement, noise reduction, randomsprays, shrinkage.

I. INTRODUCTION

ALTHOUGH the field of image enhancement has beenactive since before digital imagery achieved a consumerstatus, it has never stopped evolving. The present work intro-duces a novel multi-resolution denoising method, tailored toaddress a specific image quality problem that arises whenusing image enhancement algorithms based on random spraysampling. While inspired by the peculiar problem of suchmethods, the proposed approach also works for other image

Manuscript received April 3, 2012; revised November 28, 2012; acceptedDecember 7, 2012. Date of publication January 9, 2013; date of current versionMarch 14, 2013. This work was supported in part by the Korean Ministryof Culture, Sports and Tourism, and the Korea Creative Content Agencyin the Culture Technology Research and Development Program 2012. Theassociate editor coordinating the review of this manuscript and approving itfor publication was Dr. Debargha Mukherjee.

The authors are with the School of Electronics Engineering, Kyung-pook National University, Daegu 702-701, Korea (e-mail: [email protected]; [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIP.2013.2237918

enhancement methods that either introduce or exacerbatenoise. This work builds and expands on a previous article byFierro et al. [1].

Random sprays are a two-dimensional collection of pointswith a given spatial distribution around the origin. sprayscan be used to sample an image support in place of othertechniques, and have been previously used in works suchas Provenzi et al. [2], [3] and Kols et al. [4]. Randomsprays have been partly inspired by the Human Visual System(HVS). In particular, a random spray is not dissimilar fromthe distribution of photo receptors in the retina, although theunderlying mechanisms are vastly different.

Due to the peaked nature of sprays, a common side-effect of image enhancement methods that utilize spray sam-pling is the introduction of undesired noise in the outputimages. The magnitude and statistical characteristics of saidnoise are not known a-priori, instead they depend on severalfactors, like image content, spray properties and algorithmparameters.

Among image denoising algorithms, multi-resolution meth-ods have a long history. A particular branch is that of transformspace coefficients shrinkage, i.e. the magnitude reduction ofthe transform coefficients according to certain criteria. Someof the most commonly used transforms for shrinkage-basednoise reduction are the Wavelet Transform (WT) [5][7],the Steerable Pyramid Transform [8][10], the ContourletTransform [11][13] and the Shearlet Transform [14][16].With the exception of the WT, all other transforms lead toover-complete data representations. Over-completeness is animportant characteristic, as it is usually associated with theability to distinguish data directionality in the transform space.

Independently of the specific transform used, the gen-eral assumption in multi-resolution shrinkage is that imagedata gives rise to sparse coefficients in the transform space.Thus, denoising can be achieved by compressing (shrink-ing) those coefficients that compromise data sparsity. Suchprocess is usually improved by an elaborate statistical analy-sis of the dependencies between coefficients at differentscales. Yet, while effective, traditional multi-resolution meth-ods are designed to only remove one particular type of noise(e.g. Gaussian noise). Furthermore, only the input image isassumed to be given. Due to the unknown statistical propertiesof the noise introduced by the use of sprays, traditionalapproaches do not find the expected conditions, and thus theiraction becomes much less effective.

The proposed approach still performs noise reduction viacoefficient shrinkage, yet an element of novelty is introduced

1057-7149/$31.00 2013 IEEE

1860 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013

Fig. 1. High-level flow charts for traditional noise-reduction methods and theproposed one. It is evident that the scope of application is different, althoughthe goal is the same.

in the form of partial reference images. Having a referenceallows the shrinkage process to be data-driven. A strongsource of inspiration were the works on the Dual-tree Com-plex Wavelet Transform by Kingsbury [17], the work on theSteerable Pyramid Transform by Simoncelli et al. [8], and thework on Wavelet Coefficient Shrinkage by Donoho and John-stone [18]. Fig. 1 depicts the differences between traditionalnoise-reduction methods and the one proposed.

The remainder of this paper is organized as follows.The Dual-tree Complex Wavelet Transform is introduced inSection II, while Section III outlines the concept of randomspray sampling, and the image enhancement methods RandomSpray Retinex and RACE. Section IV then presents the pro-posed denoising algorithm. The experimental setup and resultsare presented in Section V, and some final conclusions aredrawn in Section VI.

II. DUAL-TREE COMPLEX WAVELET TRANSFORMThe Discrete Wavelet Transform (DWT) has been a found-

ing stone for all applications of digital image processing:from image denoising to pattern recognition, passing throughimage encoding and more. While being a complete and(quasi-)invertible transform of 2D data, the Discrete WaveletTransform gives rise to a phenomenon known as checkerboard pattern, which means that data orientation analysisis impossible. Furthermore, the DWT is not shift-invariant,making it less useful for methods based on the computationof invariant features.

In an attempt to solve these two problems affecting theDWT, Freeman and Adelson first introduced the concept ofSteerable filters [19], which can be used to decompose animage into a Steerable Pyramid, by means of the Steer-able Pyramid Transform (SPT) [8]. While, the SPT is anover-complete representation of data, it grants the ability toappropriately distinguish data orientations as well as beingshift-invariant. Yet, the SPT is not devoid of problems: inparticular, filter design can be messy, perfect reconstructionis not possible and computational efficiency can be a concern.

Thus, a further development of the SPT, involving the useof a Hilbert pair of filters to compute the energy response,has been accomplished with the Complex Wavelet Transform

Fig. 2. Quasi-Hilbert pairs wavelets used in the dual-tree complex wavelettransform. Each pair is shown in a column, with the even part on top and theodd one on bottom.

(CWT) [20]. Similarly to the SPT, in order to retain the wholeFourier spectrum, the transform needs to be over-complete bya factor of 4, i.e. there are 3 complex coefficients for eachreal one. While the CWT is also efficient, since it can becomputed through separable filters, it still lacks the PerfectReconstruction property.

Therefore, Kingsbury also introduced the Dual-tree Com-plex Wavelet Transform (DTCWT), which has the added char-acteristic of Perfect Reconstruction at the cost of approximateshift-invariance [17].

Since the topic is extremely vast, only a brief introduction ofthe 2D DTCWT is given. The reader is referred to the the workby Selesnick et al. [21] for a comprehensive coverage on theDTCWT and the relationship it shares with other transforms.

The 2D Dual Tree Complex Wavelet Transform can beimplemented using two distinct sets of separable 2D waveletbases, as shown below.

1,1(x, y) = h(x)h(y), 2,1(x, y) = g(x)g(y),1,2(x, y) = h(x)h(y), 2,2(x, y) = g(x)g(y),1,3(x, y) = h(x)h(y) 2,3(x, y) = g(x)g(y)

(1)

3,1(x, y) = g(x)h(y), 4,1(x, y) = h(x)g(y),3,2(x, y) = g(x)h(y), 4,2(x, y) = h(x)g(y),3,3(x, y) = g(x)h(y) 4,3(x, y) = h(x)g(y).

(2)

The relationship between wavelet filters h and g is shownbelow

g0(n) h0(n 1), for j = 1 (3)g0(n) h0(n 0.5), for j > 1 (4)

where j is the decomposition level.When combined, the bases give rise to two sets of real,

two-dimensional, oriented wavelets (see Fig. 2) i.e.

i (x, y) = 12

(1,i (x, y) 2,i (x, y)) (5)

i + 3(x, y) = 12

(1,i (x, y) + 2,i (x, y)) (6)

i (x, y) = 12

(3,i (x, y) + 4,i (x, y)) (7)

i + 3(x, y) = 12

(3,i (x, y) 4,i (x, y)). (8)

The most interesting characteristic of such wavelets is thatthey are approximately Hilbert pairs. One can thus interpret

FIERRO et al.: NOISE REDUCTION BASED ON PARTIAL-REFERENCE 1861

the coefficients deriving from one tree as imaginary, and obtainthe desired 2D DTCWT.

In this work, we made use of the freely available imple-mentation by Cai and Li [22].

The next section is dedicated to random sprays and twoimage enhancement algorithms that utilize them as samplingstructures.

III. RSR AND RACEThis Section, describes the process of random spray sam-

pling, then introduces Random Spray Retinex (RSR) andRACE, two algorithms that utilize said sampling method.RACE (crasis of RSR and ACE) is the fusion of RSR and anadapted version of Automatic Color Equalization (ACE) [23].

Random spray sampling was first introduced by Provenziet al. [2] as an elaboration over the physical scanning structuresused by Land and McCann in the original Retinex work. So,in order to properly present them, it is first necessary for usto briefly summarize the history of Retinex itself.

A. Brief History of RetinexAfter the very first work of 1971 [24] which introduced the

Retinex process (crasis of retina and cortex), a later article byLand explained and demonstrated the Retinex in a much moredetailed way [25]. In both these papers all the fundamentalsteps of Retinex had already been established, i.e. the operationon three distinct signals and the ratio-reset mechanism. It isimportant to notice that the reset mechanism described inthese two works is tightly related to the analog nature of theinstrumentations used at the time, as a digital implementationcan compute the response at all sites at the same time, priorto the ratio. Also, the threshold mechanism would be provenless influential than originally believed in a later work byProvenzi et al. [26].

In 1983, Frankle and McCann [27] patented a very efficientimplementation of Retinex. The most interesting aspects of thework are the multi-resolution nature of the algorithm and theuse of a spiralling path as the sampling pattern.

A 1983 article [28], again by Land alone, introduced twodistinct processes known as designators, that he aptly namedVersion 1 and Version 2, and that respectively represent thedynamic and static version of the same process. The termdesignator derives from the fact that both versions of theprocesses designate a point in a 3D space with each set ofstimuli, and each distinct point in such space correspondsto a different color. The static version of the designator issomewhat resemblant of Retinex but lack the fundamentalreset operation.

In particular, Version 1 is characterized by dynamic inter-actions that happen only between adjacent cells of a virtualretina, in the form of additions and subtractions of the log-arithmic response of the photo-receptors. On the other hand,Version 2 uses static paths connecting different areas of animage, establishing a relationship using the well known ratiochain (without reset). One sentence regarding the latter versionis of particular interest: The average is taken over areas fromthe entire visual field and not just those nearby; experiments

indicate there may be nearly as much contribution from distantareas as nearby ones.

In 1986, Land expanded the designator model [29], intro-ducing what will be later known as the center-surroundprocess. In this particular formulation, a photometer collectslightness according to a pattern that resembles a randomspray with areolar density decreasing as the inverse square ofthe radius. Subsequently, the ratio is computed between theresponse of a small, central circle (2 degrees in the originalwork) and the response across the whole pattern. Once more,it is important to notice that the full pattern was designed tocover the test Mondrian almost completely, a characteristicthat is inherited by random sprays. The well known NASARetinex [30] takes its steps from this particular work, althoughthe area of the surround appears very small compared to theone Land used.

In a work published in 2000 [31], Rizzi and Marinideveloped an implementation of the original Retinex byusing Brownian pseudo-random paths. While the underlyingprocess remained unchanged, these method distinguished itselfbecause it clearly exposed the possibility of using multiple anddifferent paths for each pixel of a digital image, thus renderingthe computation unique for each element.

A subsequent thorough mathematical analysis of RetinexVersion 2 [26] allowed the model to be significantly simplified,also leading to the sampling strategy known as random spraysand the RSR algorithm.

For a comprehensive treatise of the Retinex models and thedifferent algorithmic implementations we refer the reader tothe work by McCann and Rizzi [32].

B. Random Spray SamplingA single point of a random spray may be generated using

the following formulation, and the whole spray is obtained byreiterating the process

p = [R f () cos (), R f () sin()] (9)where = rand(0, 1) and = rand(0, 2) and rand indicatesthe uniform random distribution. R is the diagonal of theimage, so that the spray can cover its entirety.

Each spray is then used to sample the image by transformingits points as follows

p = p + i (10)where i = [ix , i y

]are the coordinates of the pixel used as

reference for sampling.The function f () is of extreme importance for any

algorithm utilizing random sprays for sampling because itdetermines the locality of the spray. In the original work onRSR, the authors found out that a naturally localized spray,i.e. f () = with 400 sampling points, tends to work betterthan others. Yet, other applications may require a differentlocalizing function or a different number of points. We haveto stress that our denoising method works independently ofnature of the spray.

A depiction of random sprays with varying parameters isgiven in Fig. 3.


(a) (b) (c) (d)

Fig. 3. Random sprays with different properties. (a) Naturally localized spraywith 400 points. (b) Naturally localized spray with 150 points. (c) Spray withf () = 3 and 400 points. (d) Spray with f () = 3 and 400 points.

(a) (b) (c) (d)

Fig. 4. Three synthetic test images named: (a) one square,(b) multiple shapes, (c) big square. (d) Shows superposition of the threeimages and the areas used for noise estimation: from top left to bottom right,area 1, 2, and 3.

C. Problem of NoiseThe sharp sampling imposed by sprays leads to the intro-

duction of speckle-like noise with an unknown distribution.To be more precise the noise distribution depends on thechosen algorithm, its parameters, the sprays used and, moreimportantly, the image content. It follows that the statisticalproperties of noise are not constant over the image support.A detailed analysis of the problem is given in Sec. III-D. Theinsurgence of noise had already been partially addressed inthe work on RACE [3] by using a form of attachment tothe original data, thereby strongly reducing the appearance ofspeckles in uniform areas. This process can be summarizedby the following equation, computed for each chromaticchannel

O(x, y) = (x, y)O(x, y) + (1 x, y)I (x, y) (11)where I indicates the input image, O is the output imageof non-regularized RSR or RACE, and is computed per-pixel. Given a specific spray k, the local parameter can beautomatically determined as follows, again for each chromaticchannel

k(x, y) = (2k(x, y))minmax (12)

where the quantities min and max are the image-wise standarddeviations, while k(x, y) is the standard deviation for thespray in question.

However, the main problem with the above solution is that toreduce the appearance of noise the image enhancement effectis also reduced.

D. Noise AnalysisIn order to properly analyse the noise introduced by random

spray sampling, we used the same approach employed in [2],which in turn followed the indications of [33].

In particular, we selected a number of elongated, uni-form color areas from the non-enhanced test images. Then

Fig. 5. One square test image. Histograms for the different areas afterenhancement with four different algorithms. Plots have been centered aroundthe mean for sake of comparison.

we computed the histograms of the same areas takenfrom the enhanced ones. Since the non-enhanced his-togram is a delta function for each of such areas, anydeviation from the delta distribution is a good repre-sentative of the noise introduced by the combination ofspray sampling and the action of the chosen enhancementmethod.

The three synthetic test images are shown in Fig. 4,along with the areas used for sampling. For each test image,the histogram associated with each (area, algorithm) cou-ple is shown in Figs. 57. The image content, algorithmdependent nature of the introduced noise can be caught ata glimpse. Also, note that while RSR seems to produceclean output, it is only due to the simple nature of the testimages.

The results clearly indicate how the introduced noise isdependent on each one of the following: image content (thenoise changes with changing content), sampling area (differentareas of the same image exhibit different noise), algorithm(different algorithms give rise to differently distributed noise).There is a further element of change, i.e. the spray structure,but we only used the one kind of spray that was found to workbest in [2].

The particular nature of the noise introduced using randomspray sampling image enhancement methods makes it impossi-ble to set the parameters required by traditional noise reductionmethods, which rely on the estimation of the noise statisticaldistribution.

IV. PROPOSED METHODThe main idea behind this work can be summarized as

follows: directional content is what conveys information tothe Human Visual System. This statement is backed up by past


Fig. 6. Multiple shapes test image. Histograms for the different areas afterenhancement with four different algorithms. Plots have been centered aroundthe mean for sake of comparison.

Fig. 7. Big square test image. Histograms for the different areas afterenhancement with four different algorithms. Plots have been centered aroundthe mean for sake of comparison.

research, such as the Retinex theory as well as the high-ordergray-world assumption (alias gray-edges) [34]. In particular,the local white patch effect described by Retinex comes intoplay when, for a given channel, the scanning structure samplesa positive intensity change. For obvious geometrical reasons,intensity changes of a directional nature are more easilycrossed (or sampled) than point-like structures such as noise.

Following such idea, the proposed method revolves aroundthe shrinkage, according to data directionality, of the wavelet

Algorithm 1 Algorithm for Proposed Noise-ReductionMethod

ERG B enhance(IRG B)IY CbCr rgb2ycbcr(IRG B)EY CbCr rgb2ycbcr(ERG B)YI Y channel of IY CbCr(bI , cI ) dtcwt(YI )YE Y channel of EY CbCrrepeat

(bE , cE ) dtcwt(YE ) YE is iteration dependentfor j = 1 J do

for k = 1 6 doe j,k (bIj,k)2 + (cIj,k)2

end forw j mm(stddev

k(e j,k), median

k(e j,k), j )

for k = 1 6 dobEj,k w j bEj,k + (1 w j ) bIj,kcEj,k w j cEj,k + (1 w j ) cIj,ki Ij,k ord(bIj,k) Rank of bIj,kif i Ij,k {1, 2} then

bOj,k bEj,k Shrunk coefficients from YEcOj,k cEj,k

elsebOj,k bIj,k Coefficients from YIcOj,k cIj,k

end ifend for

end forYE idtcwt(bO, cO ) Inverse DTCWT

until ssim(YI , YE ) < 0.001OY CbCr = concat(YE , ECbCr ) YE as in last iterationORG B = ycbcr2rgb(OY CbCr )

coefficients generated by the Dual Tree Complex WaveletTransform. The DTCWT is chosen for the ability to distinguishdata orientation in transform space, its relative simplicity andother useful properties.

The HVS has been proven to be more sensitive tochanges in in the achromatic plane (brightness), than chro-matic ones [35]. Hence, the proposed method first con-verts the image in a space where the chroma is sepa-rated from the luma (such as YCbCr), and operates onthe wavelet space of the luma channel. The choice to useonly the luma channel does not lead to any visible colorartifact.

Finally, a fundamental assumption is made: the input imageis considered to be either free of noise, or contaminated bynon perceivable noise. If such an assumption holds, the inputimage contains the information needed for successful noisereduction.

The algorithm for the proposed method is given asAlgorithm 1. For ease of reference, a visual description is alsogiven in Fig. 8. The following subsections explain the detailsof the shrinkage process and the tests performed to optimizethe algorithm parameters.


(a)

(b)

(c)

Fig. 8. Proposed method flowchart. (a) Luma channels of both thenonenhanced and the enhanced images are transformed using the DTCWT,and the obtained coefficients are elaborated. The output coefficients aretransformed into the output images luma channel via the inverse DTCWT.(b) and (c) Computation indicated by the box in Fig. 8(a) is performed perlevel of the decomposition. (b) Directional energy map is first computed asthe standard deviation of sum-of-squares of the coefficients. A weight mapis then obtained by using the MichaelisMenten function for normalization.The real (even) coefficients of the enhanced image are also ranked accordingto their magnitude. (c) Weight map is used to scale the coefficients of theenhanced image. The resulting scaled coefficients and the coefficients fromthe nonenhanced image are mixed according to the ranking. The process in(c) is illustrated for even coefficients only, but it is repeated identically forodd coefficients.

A. Wavelet Coefficients ShrinkageAssuming level j of the wavelet pyramid, one can compute

the energy for each direction of the non-enhanced imagek{1, 2, . . . , 6} as the sum of squares of the real coefficientsbIj,k and the complex ones cIj,k

e j,k = (bIj,k)2 + (cIj,k)2. (13)Coefficients associated with non-directional data will have

similar energy in all directions. On the other hand, directionaldata will give rise to high energy in one or two directions,according to its orientation.

The standard deviation of energy across the six directionsk = 1, 2, . . . , 6 is hence computed as a measure of direction-ality.

e j = stddevk

(e j,k). (14)

Since the input coefficients are not normalized, it naturallyfollows that the standard deviation is also non-normalized. TheMichaelis-Menten function [36] is thus applied to normalizedata range. Such function is sigmoid-like and it has been usedto model the cones responses of many species. The equationis as follows:

mm(x, , ) = x

x + (15)

where x is the quantity to be compressed, a real-valuedexponent and the data expected value or its estimate.

Hence, a normalized map of directionally sensitive weightsfor a given level j can be obtained as

w j = mm(e j , mediank

(e j,k), j )

where the choice of depends on j as explained later on.A shrunk version of the enhanced images coefficients,

according to data directionality, is then computed as

bEj,k = w j bEj,k + (1 w j ) bIj,k (16)cEj,k = w j cEj,k + (1 w j ) cIj,k . (17)

Since the main interest is retaining directional information,we obtain a rank for each of the non-enhanced coefficients as

i Ij,k = ord(

bIj,k)

, {1, 2, . . . , 6} (18)where ord is the function that returns the rank according tonatural ordering.

The output coefficients are then computed as follows

bOj,k ={

bEj,k, if iIj,k {1, 2}

bIj,k, if i Ij,k {3, 4, 5, 6}(19)

cOj,k ={

cEj,k, if iIj,k {1, 2}

cIj,k, if iIj,k {3, 4, 5, 6}

(20)

where ord is the function that returns the index of a coefficientin bIk=1,2,...,6 when the set is sorted in a descending order.

The meaning of the whole sequence can be roughlyexpressed as follows: where the enhanced image shows direc-tional content, shrink the two most significant coefficients andreplace the four less significant ones with those from the non-enhanced image.

The reason why only the two most significant coefficientsare taken from the shrunk ones of the enhanced image is tobe found in the nature of directional content. For an contentof an image to be directional, the responses across the sixorientations of the DTCWT need to be highly skewed. Inparticular, any data orientation can be represented by a strongresponse on two adjacent orientations, while the remainingcoefficients will be near zero. This will make it so that thetwo significant coefficients are carried over almost un-shrunk.

To help understanding, in Fig. 9 we show a comparison ofthe energies of the decompositions for one of the test images.


(a) (b)

Fig. 9. These images show the (a) first and (b) second level decomposition energies for the images in Fig. 13(a)(c). For compactness sake, only three filterorientations are shown without the approximation (low frequency) component. The values have been slightly compressed for visibility. It can be observedhow the output image is a combination of the nonenhanced and enhanced ones according to the non-enhanced image data directionality.

(a) (b)

Fig. 10. (a) PSN ratios and (b) SSIM scores for the proposed method (J = 3), BM3D, and FLNM. The overall performance of the proposed denoisingmethod is superior to that of traditional approaches. As expected, better results are achieved when the dynamic range of the nonenhanced image is wide. Thes-axis marks the dynamic range scaling factor for the non-enhanced image I , whereas 2 is the variance of the additive white Gaussian noise.

B. Parameter TuningWhen dealing with functions with free parameters, a funda-

mental problem is finding optimum parameters values. Whilethis can often be attempted with optimization techniques, suchmethods proved unfeasible in the case.

To at least provide a reasonable default value for j and J (the parameter of the Michaelis-Menten func-tion and the depth of the complex wavelet decomposi-tion, respectively) tests were performed on three imagesfrom the USC-SIPI Image Database [37]. Such images,shown in Fig. 14, provide a good mixture of mostlyhigh frequency detail (Mandrill), balanced high and lowfrequency content (Lenna), and mainly low frequency content(Splash).

In different rounds, Gaussian, Poissonian and Speckle noisewas added to the luma channel of said images and the proposednoise reduction method was run with a J = 3 and values for j varying from 5 to 10 in unit steps for the two levels ofthe decomposition. This allowed us to determine that valuesof 1, 2 and 3 equal to 1 represent a reasonable choice,although non-optimal for all inputs. It should be noticed thatby doing so the Michaelis-Menten function is reduced to theNakaRushton formulation [38].

Since J proved to be drastically more dependent on theinput image than j , it was impossible to determine a sin-gle optimum value. Therefore, J was left as the only userset parameter of the method, with reasonable bounds ofJmin = 1, Jmax = 3.


(a) (b) (c)

(d) (e) (f)

(g) (h) (i) (j)

Fig. 11. Proposed denoising method applied to RSR filtered images. Crops are shown to better illustrate the strong reduction in noise while directionalcontent is retained. (a) Book, original. (b) RSR. (c) Denoised with J = 1. (d) Gallery, original. (e) RSR. (f) Denoised with J = 1. (g) Book enhanced, detail.(h) Book denoised, detail. (i) Gallery enhanced, detail. (j) Gallery denoised, detail.

(a) (b)

Fig. 12. Scanline plots. The whole scanline is shown above, while below only a part of it is displayed, so that details become more apparent. The detailsection extrema are indicated by discontinuous vertical lines in the topmost plot.

While tuning the parameters, performance was tested usingPSNR and the SSIM [39] measure, holding the unaltered lumachannel as the absolute reference. Iterations were stoppedusing a SSIM threshold t = 0.001. Scores are givenin Table I.

V. EXPERIMENTSIn order to verify the efficacy of the proposed method, we

devised a set of five different tests. Performance assessmentincludes scanline analysis, quality metrics comparison, subjec-tive evaluation and panel tests.


(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Fig. 13. Proposed denoising method applied to histogram equalized images. In this case, either grain or compression noise is already present in the non-enhanced images and exposed by histogram equalization. The proposed method effectively reduces both, while still maintaining sharp edges. (a) Girl, original.(b) Girl, histogram equalized. (c) Girl, denoised with J = 2. (d) Tiffany, original. (e) Tiffany, histogram equalized. (f) Tiffany, denoised with J = 2. (g) Tree,original. (h) Tree, histogram equalized. (i) Tree, denoised with J = 2.

1) Test on Difficult Images for RSR: We first used theproposed method on two images taken from the work onRSR [2] which happened to trigger the appearance of noise.The images, their enhanced versions and the noise reducedresults are shown in Fig. 11. Scanline data is also given inFig. 12(a)(b). The proposed denoising method respects thedesired behaviour and shows good performance.

2) Test on Images Enhanced With Histogram Equalization:To demonstrate the efficacy of the proposed method even inthe absence of random spray sampling, a set of images takenfrom the SIPI database (Girl, Tiffany, Tree) was enhancedusing histogram equalization. This also broughts out latentgrain and compression noise, which were then reduced usingthe proposed approach. All the images are presented in


(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Fig. 14. Images from the SIPI database modified to have a strong color cast, then enhanced and used to test the proposed noise reduction method. (a) Lenna,with color cast. (b) Lenna, RSR enhanced. (c) Lenna, denoised with J = 3. (d) Splash, with color cast. (e) Splash, spray-ACE enhanced. (f) Splash, denoisedwith J = 2. (g) Mandrill, with color cast. (h) Mandrill, spray-ACE enhanced. (i) Mandrill, denoised with J = 2.

Fig. 13. Performance evaluation is subjective, but we canstate that the proposed noise reduction model performs asexpected.

3) Test on Images With Added Color-Cast Enhanced WithSpray-Based Methods: A set of images taken from the SIPIdatabase (Lenna, Splash, Mandrill) was modified by reducingthe dynamic range of one of the RGB channels, thus introduc-ing a strong color cast. The resulting images were enhancedusing a spray based method, then denoised using the proposedapproach, as illustrated in Fig. 14. Once again, performance

evaluation can be nothing but subjective, yet detail is wellpreserved while noise effectively reduced.

4) Test Versus Traditional Noise Reduction Method: To theauthors knowledge, there is no approach directly comparableto the one proposed. In order to perform tests against existingmethods in literature, it is thus necessary to employ a simpli-fied testing model.

If the assumptions made in Sec. IV hold, and also assumingthat we have a full-quality reference image R at our disposal,it is also safe to assume that the non-enhanced image I will be


(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

Fig. 15. Comparison between the proposed method and Foveated NL-means. Noise is Gaussian with standard deviation as per captions. (a) Lenna, original.(b) Noisy, = 50. (c) Foveated NL-means. (d) Proposed, J = 2. (e) Barbara, original. (f) Noisy, = 70. (g) Foveated NL-means. (h) Proposed, J = 2.(i) Boats, original. (j) Noisy, = 100. (k) Foveated NL-means. (l) Proposed, J = 2.

a degraded version of R where the degradation consists onlyof a reduction in dynamic range, i.e.

I (x, y) = s R(x, y) (21)where, for simplicity, s (0, 1] is a linear scaling factor.

We could make the further assumption that the tested imageenhancement method H , will produce as output the full-qualityimage R affected by a AWGN with known properties, thus theenhanced image E is defined as

E(x, y, 2) = R(x, y) + G( 2) (22)where G( 2) is a Gaussian random variable with zero meanand given variance 2.

It should be pointed out that the devised testing schemeputs traditional methods in their optimal conditions (noise

variance is known instead of estimated, noise is assumed to bea stationary Gaussian process). On the other hand, we stresstest the proposed approach by selecting a range of values fors that goes from 1 all the way to 0.1, resulting in a minimumdynamic range of 25 grey levels or less.

The proposed method was tested against the FoveatedNL-means (FNLM) and BM3D. FNLM by Foi and Borac-chi [40] is a recent development of a quite successful work byBuades et al. [41]. BM3D [42] is a work by Dabov et al. and itis also part of the non-local family of noise reduction methods,although it takes a two-step approach, first computing a roughestimate and then using it to drive Wiener estimation.

Three images were chosen from the the work of Foi andBoracchi, and tests were run as per (21) and (22). The resultsof the comparison are shown in Fig. 10. The proposed noise


(a) (b) (c) (d) (e)

Fig. 16. (a) This image has been altered to have a strong gradient from the lower left corner (no color cast) to the upper right one (dark purple). (b) Resultof RSR processing. (c)(e) illustrate the output of the proposed method with J = {1, 2, 3}, respectively. It can be observed how there is no appreciable changein luminance in the three noise reduced images. (a) Image with lighting gradient. (b) RSR output. (c) Denoised, J = 1. (d) Denoised, J = 2. (e) Denoised,J = 3.

TABLE IPSN RATIOS AND SSIM SCORES FOR TEST IMAGES

FROM THE USC-SIPI IMAGE DATABASE

Noisy DenoisedNoise type Img. name PSNR SSIM PSNR SSIM

GaussianLenna 27.43 0.58 35.78 0.95Splash 27.68 0.49 36.37 0.93Mandrill 27.45 0.73 34.78 0.98

PoissonianLenna 30.33 0.84 35.78 0.95Splash 30.92 0.82 36.37 0.93Mandrill 30.16 0.91 34.78 0.98

SpeckleLenna 27.07 0.55 33.96 0.94Splash 27.55 0.53 33.84 0.94Mandrill 26.96 0.69 33.87 0.97

TABLE IIRESULTS OF PANEL TESTS. THE LAST THREE COLUMNSREPRESENT THE BREAKDOWN OF VOTES FOR DENOISED

IMAGES FOR DIFFERENT VALUES OF J

JQuestion Image # Noisy Denoised 1 2 3

Least noisy

1 0 12 0 8 42 0 12 3 1 83 0 12 0 0 124 0 2 3 4 55 0 12 8 1 3

Sharpest

1 9 3 3 0 02 6 6 0 4 23 6 6 0 2 44 9 3 0 3 05 12 0 0 0 0

Preferred

1 3 9 2 7 02 3 9 0 3 63 3 9 0 3 64 3 9 3 3 35 7 5 2 0 3

reduction method offers overall a better performance, yet, asexpected, when the non-enhanced image I has a very limiteddynamic range it falls behind traditional methods (admittedlyby a limited amount).

5) User Preference Test: Finally, a panel test was run among12 users. These included 6 persons with training related toimage processing and 6 persons with no previous knowledgeof the task at hand. All were given brief instructions on themeaning of noise and sharpness. Five test images wereselected from the SIPI database, the Kodak image database[43] and other available. Users were asked to select the leastnoisy, sharpest and preferred image within a set containing theenhanced version of a test picture (using either spray-ACE,RSR or RACE) and the noise reduced images produced withJ = 1, 2, and 3. Results are shown in Table II. It can beobserved that for all test images users indicated the least noisyamong the ones produced by our algorithm. Sharpness, inaccordance with the known HVS behaviour and the proposedapproach characteristics, tends to favor noisy images. Finally,user preference falls mainly on one of the denoised images,indicating good performance.

6) Effects on Gradient Suppression: Retinex-like algorithmshave the property of suppressing low frequency gradientsmuch like the HVS, and taking content from the non-enhancedimage might negatively affect this behaviour. In Fig. 16 wedemonstrate that, thanks to the very localized action of theproposed method, one should not worry of reintroducingunwanted gradients.

VI. CONCLUSIONThis work presents a noise reduction method based on

Dual Tree Complex Wavelet Transform coefficients shrinkage.The main point of novelty is represented by its applicationin post-processing on the output of an image enhancementmethod (both the non enhanced image and the enhanced oneare required) and the lack of assumptions on the statisticaldistribution of noise. On the other hand, the non-enhancedimage is supposed to be noise-free or affected by nonperceivable noise.

Following well known properties of the Human VisualSystem, the images are first converted to a color spacewith distinct chromatic and achromatic axes, then only theachromatic part becomes object of the noise reduction process.To achieve pleasant denoising, the proposed method exploitsthe data orientation discriminating power of the Dual Tree


Complex Wavelet Transform to shrink coefficients from theenhanced, noisy image. Always according to data direction-ality, the shrunk coefficients are mixed with those from thenon-enhanced, noise-free image. The output image is thencomputed by inverting the Dual Tree Complex Wavelet Trans-form and the color transform.

Since at the time of writing no directly comparable methodwas known to the authors, performance was tested in a numberof ways, both subjective and objective, both quantitative andqualitative. Subjective tests include a user panel test, and closeinspection of image details. Objective tests include scanlineanalysis for images without a known prior, and computationof PSNR and SSIM on images with a full reference. Theproposed method produces good quality output, removingnoise without altering the underlying directional structures inthe image. Also, although designed to tackle a quality prob-lem specific to spray-based image enhancement methods, theproposed approach also proved effective on compression andlatent noise brought to the surface by histogram equalization.

The methods main limitations are the necessity of two inputimages (one non-enhanced and one enhanced) and its iterativenature, which expands computation time considerably withrespect to one-pass algorithms.

ACKNOWLEDGMENTThe authors would like to thank Provenzi and Gatta for

providing useful insight and suggestions.

REFERENCES[1] M. Fierro, W.-J. Kyung, and Y.-H. Ha, Dual-tree complex wavelet

transform based denoising for random spray image enahcement meth-ods, in Proc. 6th Eur. Conf. Colour Graph., Imag. Vis., 2012,pp. 194199.

[2] E. Provenzi, M. Fierro, A. Rizzi, L. De Carli, D. Gadia, and D. Marini,Random spray retinex: A new retinex implementation to investigate thelocal properties of the model, vol. 16, no. 1, pp. 162171, Jan. 2007.

[3] E. Provenzi, C. Gatta, M. Fierro, and A. Rizzi, A spatially variant white-patch and gray-world method for color image enhancement driven bylocal contrast, vol. 30, no. 10, pp. 17571770, 2008.

[4] . Kols, I. Farup, and A. Rizzi, Spatio-temporal retinex-inspiredenvelope with stochastic sampling: A framework for spatial coloralgorithms, J. Imag. Sci. Technol., vol. 55, no. 4, pp. 110, 2011.

[5] H. A. Chipman, E. D. Kolaczyk, and R. E. McCulloch, Adaptivebayesian wavelet shrinkage, J. Amer. Stat. Assoc., vol. 92, no. 440,pp. 14131421, 1997.

[6] A. Chambolle, R. De Vore, N.-Y. Lee, and B. Lucier, Nonlinear waveletimage Processing: Variational problems, compression, and noise removalthrough wavelet shrinkage, IEEE Trans. Image Process., vol. 7, no. 3,pp. 319335, Mar. 1998.

[7] D. Cho, T. D. Bui, and G. Chen, Image denoising based on waveletshrinkage using neighbor and level dependency, Int. J. Wavelets, Mul-tiresolution Inf. Process., vol. 7, no. 3, pp. 299311, May 2009.

[8] E. P. Simoncelli and W. T. Freeman, The steerable pyramid: A flexiblearchitecture for multi-scale derivative computation, in Proc. 2nd Annu.Int. Conf. Image Process., Oct. 1995, pp. 444447.

[9] F. Rooms, W. Philips, and P. Van Oostveldt, Integrated approach forestimation and restoration of photon-limited images based on steerablepyramids, in Proc. 4th EURASIP Conf. Focused Video/Image Process.Multimedia Commun., vol. 1. Jul. 2003, pp. 131136.

[10] H. Rabbani, Image denoising in steerable pyramid domain based on alocal laplace prior, Pattern Recognit., vol. 42, no. 9, pp. 21812193,Sep. 2009.

[11] S. Foucher, G. Farage, and G. Benie, Sar image filtering based onthe stationary contourlet transform, in Proc. IEEE Int. Geosci. RemoteSens. Symp., Jul.Aug. 2006, pp. 40214024.

[12] W. Ni, B. Guo, Y. Yan, and L. Yang, Speckle suppression for sar imagesbased on adaptive shrinkage in contourlet domain, in Proc. 8th WorldCongr. Intell. Control Autom., vol. 2. 2006, pp. 1001710021.

[13] K. Li, J. Gao, and W. Wang, Adaptive shrinkage for image denoisingbased on contourlet transform, in Proc. 2nd Int. Symp. Intell. Inf.Technol. Appl., vol. 2. Dec. 2008, pp. 995999.

[14] Q. Guo, S. Yu, X. Chen, C. Liu, and W. Wei, Shearlet-based imagedenoising using bivariate shrinkage with intra-band and opposite ori-entation dependencies, in Proc. Int. Joint Conf. Comput. Sci. Optim.,vol. 1. Apr. 2009, pp. 863866.

[15] X. Chen, C. Deng, and S. Wang, Shearlet-based adaptive shrinkagethreshold for image denoising, in Proc. Int. Conf. E-Bus. E-Government,Nanchang, China, May 2010, pp. 16161619.

[16] J. Zhao, L. Lu, and H. Sun, Multi-threshold image denoising based onshearlet transform, Appl. Mech. Mater., vols. 2932, pp. 22512255,Aug. 2010.

[17] N. G. Kingsbury, The dual-tree complex wavelet transform: A newtechnique for shift invariance and directional filters, in Proc. 8th IEEEDigit. Signal Process. Workshop, Aug. 1998, no. 86, pp. 14.

[18] D. L. Donoho and I. M. Johnstone, Threshold selection for waveletshrinkage of noisy data, in Proc. 16th Annu. Int. Conf. IEEE Eng.Adv., New Opportunities Biomed. Eng. Eng. Med. Biol. Soc., vol. 1.Nov. 1994, pp. A24A25.

[19] W. Freeman and E. Adelson, The design and use of steerable filters,IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 9, pp. 891906,Sep. 1991.

[20] N. G. Kingsbury, Image Processing with complex wavelets, Philos.Trans. Math. Phys. Eng. Sci., vol. 357, no. 1760, pp. 25432560, 1999.

[21] I. W. Selesnick, R. G. Baraniuk, and N. G. Kingsbury, The dual-treecomplex wavelet transform: A coherent framework for multiscale signaland image Processing, vol. 22, no. 6, pp. 123151, Nov. 2005.

[22] S. Cai and K. Li. Matlab Implementation of Wavelet Trans-forms. Polytechnic Univ., Brooklyn, NY [Online]. Available:http://eeweb.poly.edu/iselesni/WaveletSoftware/index.html

[23] A. Rizzi, C. Gatta, and D. Marini, A new algorithm for unsupervisedglobal and local color correction, Pattern Recognit. Lett., vol. 24, no. 11,pp. 16631677, Jul. 2003.

[24] E. H. Land and J. J. McCann, Lightness and retinex theory, J. Opt.Soc. Amer. A, vol. 61, no. 1, pp. 111, 1971.

[25] E. H. Land, The retinex theory of color vision, Sci. Amer., vol. 237,no. 6, pp. 108128, Dec. 1977.

[26] E. Provenzi, L. D. Carli, A. Rizzi, and D. Marini, Mathematicaldefinition and analysis of the retinex algorithm, J. Opt. Soc. Amer. A,vol. 22, no. 12, pp. 26132621, Dec. 2005.

[27] J. Frankle and J. J. McCann, Method and apparatus of lightnessimaging, U.S. Patent 4 384 336, May 17, 1983.

[28] E. H. Land, Recent advances in retinex theory and some implicationsfor cortical computations: Color Vision and the natural image, Proc.Nat. Acad. Sci. United States Amer., vol. 80, no. 16, pp. pp. 51635169,1983.

[29] E. H. Land, An alternative technique for the computation of thedesignator in the retinex theory of color Vision, Proc. Nat. Acad. Sci.United States Amer., vol. 83. pp. 30783080, May 1986.

[30] D. J. Jobson, Z. Rahman, and G. A. Woodell, Properties and perfor-mance of a center/surround retinex, IEEE Trans. Image Process., vol. 6,no. 3, pp. 451462, Mar. 1997.

[31] D. Marini and A. Rizzi, A computational approach to color adaptationeffects, Image Vis. Comput., vol. 18, no. 13, pp. 10051014, 2000.

[32] J. J. McCann and A. Rizzi, The Art and Science of HDR Imaging,M. A. Kriss, Ed. New York: Wiley, 2012.

[33] R. C. Gonzales and R. E. Woods, Digital Image Processing, 2nd ed.Englewood Cliffs, NJ: Prentice-Hall, 2002.

[34] J. van de Weijer, T. Gevers, and A. Gijsenij, Edge-based color con-stancy, IEEE Trans. Image Process., vol. 16, no. 9, pp. 22072214,Sep. 2007.

[35] M. Livingstone, Vision and Art: The Biology of Seeing. New York: HarryN. Abrams, 2002.

[36] L. Menten and M. I. Michaelis, Die kinetik der invertinwirkung,Biochem, vol. 49, pp. 333369, Feb. 1913.

[37] USC-SIPI Image Database. (2009) [Online]. Available:http://sipi.usc.edu/database/

[38] K. I. Naka and W. A. H. Rushton, S-potential from colour units in theretina of fish (Cyprinidae), J. Physiol., vol. 185, no. 3, pp. 536555,1966.


[39] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image qualityassessment: From error visibility to structural similarity, vol. 13, no. 4,pp. 600612, Apr. 2004.

[40] A. Foi and G. Boracchi, Foveated self-similarity in nonlocal imagefiltering, Proc. SPIE, vol. 8291, p. 829110, Jan. 2012.

[41] A. Buades, B. Coll, and J.-M. Morel, A non-local algorithm for imagedenoising, in Proc. IEEE Comput. Soc. Conf. Comput. Vis. PatternRecognit., vol. 2. Jun. 2005, pp. 6065.

[42] K. Dabov, A. Foi, V. Kartkovnik, and K. Egiazarian, Image denoisingby sparse 3d transform-domain collaborative filtering, IEEE Trans.Image Process., vol. 16, no. 8, pp. 20802095, Aug. 2007.

[43] R. Franzen. (2010). Kodak Lossless True Color Image Suite [Online].Available: http://r0k.us/graphics/kodak/

Massimo Fierro received the M.S. degree in com-puter science from the University of Milano, Milan,Italy, in 2007. He is currently pursuing the Ph.D.degree from the Color and Imaging Laboratory,Kyungpook National University, Daegu, Korea.

His current research interests include HDR imag-ing, panoramic imaging, color correction, and imagedenoising.

Ho-Gun Ha received the B.S. and M.S. degreesin electrical engineering and computer science fromKyungpook National University, Daegu, Korea, in2007 and 2009, respectively, where he has beenpursuing the Ph.D. degree in the Color and ImagingLaboratory.

His current research interests include image regis-tration and color image processing.

Yeong-Ho Ha (SM00) received the B.S. and M.S.degrees in electronic engineering from KyungpookNational University, Daegu, Korea, in 1976 and1978, respectively, and the Ph.D. degree in electricaland computer engineering from the University ofTexas, Austin, in 1985.

He joined the Department of Electronics Engineer-ing, Kyungpook National University, where he iscurrently a Professor. He served as President andVice President with the Korea Society for ImagingScience and Technology, and a Vice President of the

Institute of Electronics Engineering of Korea. His current research interestsinclude color image processing, computer vision, digital signal, and imageprocessing.

Dr. Ha is a member of the Pattern Recognition Society, and fellow of IS&Tand SPIE. He served as a TPC Chair, Committee Member, and OrganizingCommittee Chair of many international conferences held in IEEE, SPIE, andIS&T and domestic conferences.

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 600 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 400 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/Description >>> setdistillerparams> setpagedevice

Documents

Noise Reduction Based on Partial-Reference