View
9
Download
0
Category
Preview:
Citation preview
CSCNet - The Effect of the Stride
1. High CoherenceNatural images mainly consist of smooth and piecewise smooth regions.Hence, the dictionary must contain smooth or piecewise smooth filters.
Having smooth or piecewise smooth filters in the dictionary leads to highly correlated atoms in the global dictionary that results in high coherence.
Rethinking the Convolutional Sparse Coding (CSC) Model for Natural ImagesDror Simon and Michael Elad
Department of Computer Science, Technion – Israel Institute of Technology
{dror.simon@cs, elad@cs}.technion.ac.il
The CSC Model➢ A signal is a sum of m convolutions
of band limited filters with m sparse feature-maps
➢ By interlacing all the feature maps into , and defining a global dictionarythe CSC problem is defined by
where the dictionary is convolutional:
Main Contribution of this Work➢ Explain why the CSC model fails when used
for simple image restoration tasks➢ Show that local patch-based algorithms are
actually MMSE approximations for the global CSC model
➢ Propose novel MMSE approximation for the CSC model that provide superior results compared to patch based algorithms
➢ CSCNet - a discriminative recurrent model trained in a supervised fashion that is on par with current state of the art methods while using much fewer parameters
The CSC - Successful Applications➢ Cartoon-texture separation
➢ Image fusion
➢ Single Image Super Resolution
The CSC - Natural Image Denoising➢ Classic Patch-Averaging (PA) methods
denoise images very well➢ At first it seems that the CSC is a global
extension to patch-averaging methods➢ Unfortunately, CSC has led to poor natural
image denoising resultsWhy does it denoise so poorly?
1. Natural images lead to dictionary with poor properties, i.e. high coherence
2. A Bayesian explanation
2(a). Patch AveragingPatch averaging (PA) using a local dictionary solves N independent problems:
The final image is then obtained using patch-averaging
2(b). CSC MMSEThe MMSE estimator of the CSC model is
The MMSE estimator is not sparse! It is a sum of the oracle estimators of all possible supports
2(c). Connecting the CSC to PA➢ Assume results with non-overlapping
tangent patches➢ There are n such arrangements
➢ Each such shift can be described using a subsampled dictionary and respective
➢ The solution of each such shift is then:
Or equivalently:
CSC - Additional Definitions
PA is CSC MMSE
Generalizing the MMSE Approx.➢ Use strided convolutions with stride q.➢ In the non- overlapping case the stride size
equals the band of the filters.➢ Decrease the stride to allow overlaps
Contradiction!
q<nEach estimate introduces overlaps
q sufficiently largeLow mutual coherence can be preserved
CSCNet - Denoising Model➢ Use Lista [1] and its convolutional extension
[2] to obtain a sparse coding approximation➢ Introduce our strided concept to obtain an
MMSE approximation
CSCNet - Results➢ Denoising results on the BSD68 dataset
➢ Better than BM3D [3], WNNM [4], TNRD [5]
➢ On par with DnCNN [6], FFDNet [7]
BM3D WNNM TNRD DnCNN
FFDNet CSCNet
15 21.07 31.37 31.42 31.72 31.63 31.5725 28.57 28.83 28.92 29.22 29.19 29.1150 25.62 25.87 25.97 26.23 26.29 26.2475 24.21 24.40 — 24.64 24.79 24.77
References[1] Gregor, K., & LeCun, Y. Learning fast approximations of sparse coding. (ICML 2010).[2] Sreter, H., & Giryes, R. Learned convolutional sparse coding. (ICASSP 2018).[3] Dabov, K. et al. Image denoising with block-matching and 3D filtering. (2006).[4] Gu, S. et al. Weighted nuclear norm minimization with application to image denoising. (CVPR 2014)[5] Chen, Y., & Pock, T. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. (IEEE PAMI 2016).[6] Zhang, K. et al. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. (IEEE TIP 2017)[7] Zhang, K., Zuo, W., & Zhang, L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. (IEEE TIP 2018).
Recommended