Wavelet-Domain Video Denoising Based on Reliability Measures

Vladimir Zlokolica, Aleksandra Piˇzurica and Wilfried Philips

Circuits and Systems for Video Technology2006, Transaction on IEEE Journals

Outline

• Introduction• Proposed Video Denoising Method– Noise Estimation[36]– Reliability of MV Estimates– Motion Estimation– Recursive Temporal Filtering (RTF)– Adaptive Spatial Filtering

• Experimental Results• Conclusions

• Video denoising: spatial-temporal filters.• Filtering:– Nonseparable (fully 3-D) [2]–[6]– Separable (2-D +1-D) [7]–[14]• Combined: weighting for spatial and temporal?• Spatial-first: ringing or blurring at high noise levels.• Temporal-first

• We adopt the temporal-first approach, and develop a robust motion estimation method.

Introduction

Proposed Video Denoising Method

Noise Estimation• Assume that most frequent gradient

amplitude is predominately caused by noise:

• It can be related to the input noise level:

• Finally, we recursively update estimated σs :

[36]V. Zlokolica, A. Pizurica, and W. Philips, “Wavelet domain noise-robust motion estimation and noise estimation for video denoising,”presented at the 1st Int. Workshop Video Process. Quality Metrics Consum. Electron., Scotssdale, AZ, Jan. 2005, Paper no. 200.

• Parameters k1 = 0.001, k2 = 1.069 and k3 =2.213.• value is almost independent of image

context and strongly corresponds to noise level.

Noise Estimation

[36]V. Zlokolica, A. Pizurica, and W. Philips, “Wavelet domain noise-robust motion estimation and noise estimation for video denoising,”presented at the 1st Int. Workshop Video Process. Quality Metrics Consum. Electron., Scotssdale, AZ, Jan. 2005, Paper no. 200.

Proposed Video Denoising Method

Proposed Video Denoising Method

• The proposed method uses a nondecimated wavelet transform[35].

• Denote wavelet bands: WB={LL,HL,LH,HH}• The spatial position as: r = (x,y)• The decomposition level: l = 1,…,N

(1 denotes the finest scale and N the coarsest)• WBn: noisy band; WBtf: temporally filtered;

WBstf: spatio-temporally filtered band.

Reliability of MV Estimates• We define the MAD for each block s in the

wavelet band WB(l)(r,t), as follows:

– Bs : the set of r belonging to the given 8x8 block.– WB: {LL,HL,LH,HH}– N: the maximum decomposition level.

Reliability of MV Estimates• Define the horizontal θH and vertical θ V reliabilities of MV v:

– Where d1 = d2 = … = dN and

• Analogously, we define the “per wavelet band” WB(l) reliability of the estimated MV v:

• MAD→σn then θ→1 , MAD>>σn then θ→0.

Motion Estimation

• Wavelet-Domain Three-Step Method:– Estimates first MV field at the roughest scale and in

the following steps refines the MV field:

• vpi {∈ 0,s,s’,t,t’}; P(0) = 0, P(vpi) = 2.5

• v(1)cx , v(1)

cy {-8,-4,0,4,8}; ∈ v(2)cx , v(2)

cy {-4,-2,0,2,4};∈v(3)

cx , v(3)cy {-2,-1,0,1,2}; ∈

Motion Estimation• The cost function:

– Where the constants C1 and C2 are optimized to obtain a noise robust and smooth MV field: C1= 1, C2 = 1.45

– Assign more weight to the cost function for higher θH and θ V for the tested nonzero correction.

Proposed Video Denoising Method

Recursive Temporal Filtering (RTF)

• Wavelet domain temporal filtering:

• When WB(l)tf(r-vb,t-1) has not all been filtered,

noisy wavelet coefficient will propagate.

Recursive Temporal Filtering (RTF)

• To solve this problem, we update α(l)WB(s,t,σn,vb)

with a correction function:

– When α(l)WB(r-vb,t-1) → 0, α(l)

WB*(r,t) →0.5:

Both frames are noisy, perform simple averaging.– When α(l)

WB(r-vb,t-1) → 1, α(l) WB

*(r,t) → α(l) WB(r,t).

– Furthermore, we apply α = 1 at least two time-recursions with reliable MVs have been applied in the last two frames.

Proposed Video Denoising Method

Adaptive Spatial Filtering

• Let δ(rc) denote the neighborhood surrounding the central pixel rc :

– Where T = MAD(l)WB(s,t,vb) , km = 1.

– The lower MAD the for the corresponding wavelet band WB(l) and block s, the less we will average.

Experimental Results

Fig. 4. Results for the 29th frame of “Bicycle” sequence with added Gaussian noise(σn= 15), processed by (c) WRTF filter and (d) 3RDS filter [16].

(a) Original image frame. (b) Noisy image frame.(c) WRTF(d) 3RDS filter [16]

Experimental Results

Fig. 7. Results for the 75th frame of the processed “Flower Garden” sequence with added Gaussian noise (σn = 15), by (c) the 3DWTF algorithm, and (d) theWRSTF algorithm.

(a) Original image frame. (b) Noisy image frame.(c) 3DWTF(d) WRSTF

Experimental Results(a)

(c)

(b)

(d)

Experimental Results

Experimental Results

Conclusions

• We have proposed a new method for motion estimation and image sequence denoising in the wavelet domain.

• By robustly estimating motion and compensating, we efficiently remove noise without introducing visual artifacts.

• In future work, we intend to refine our motion estimation framework in order to deal with occlusion and moving block edges.