Variation Dirichlet Blur Kernel Estimation Dirichlet Blur Kernel...  Variational Dirichlet Blur Kernel

  • View

  • Download

Embed Size (px)

Text of Variation Dirichlet Blur Kernel Estimation Dirichlet Blur Kernel...  Variational Dirichlet Blur...


    Variational Dirichlet Blur Kernel EstimationXu Zhou, Student Member, IEEE, Javier Mateos, Member, IEEE, Fugen Zhou, Member, IEEE,

    Rafael Molina, Member, IEEE, and Aggelos K. Katsaggelos Fellow, IEEE

    AbstractBlind image deconvolution involves two key ob-jectives, latent image and blur estimation. For latent imageestimation, we propose a fast deconvolution algorithm, whichuses an image prior of nondimensional Gaussianity measureto enforce sparsity and an undetermined boundary conditionmethodology to reduce boundary artifacts. For blur estimation,a linear inverse problem with normalization and nonnegativeconstraints must be solved. However, the normalization constraintis ignored in many blind image deblurring methods, mainlybecause it makes the problem less tractable. In this paper, weshow that the normalization constraint can be very naturallyincorporated into the estimation process by using a Dirichletdistribution to approximate the posterior distribution of theblur. Making use of variational Dirichlet approximation, weprovide a blur posterior approximation that takes into accountthe uncertainty of the estimate and removes noise in the estimatedkernel. Experiments with synthetic and real data demonstratethat the proposed method is very competitive to state-of-the-artblind image restoration methods.

    Index TermsBlind Deconvolution, image deblurring, varia-tional distribution approximations, Dirichlet distribution, con-strained optimization, point spread function, inverse problem.


    BLIND image deconvolution (BID) refers to the problemof recovering the source image from a degraded obser-vation when the blur kernel is unknown, using partial infor-mation about the imaging system [1]. Primary applicationsof BID include astronomical imaging, medical imaging, andcomputational photography [2].

    Mathematically, the degraded observation y M can beapproximately modeled as [2]

    y = Hx+ n (1)

    where x N is the original image, H of size M N is theconvolution matrix whose row elements are obtained from theblur kernel h K , and n M is assumed to be additivezero-mean Gaussian white noise. BID is a severely ill-posedinverse problem, since multiple pairs of kernels and original

    This work was sponsored in part by National Natural Science Foundationof China (61233005), Ministerio de Ciencia e Innovacion under ContractTIN2013-43880-R, the European Regional Development Fund (FEDER), theCEI BioTic with the Universidad de Granada and the Department of Energygrant DE-NA0002520.

    X. Zhou and F. Zhou are with the Image Processing Center, Bei-hang University, Beijing 100191, China (e-mail:; X. Zhou performed the work while at Universidad deGranada, as a joint-PhD student sponsored by Chinese Scholarship Council.

    J. Mateos and R. Molina are with Departamento de Ciencias de laComputacion e I.A., Universidad de Granada, Granada 18071, Spain (;

    A. Katsaggelos is with the Department of Electrical Engineering andComputer Science, Northwestern University, Evanston, IL 60208-3118 USA(e-mail:

    images can match the model (1) equally well. To overcome theill-posed nature of BID, additional constraints or assumptionson both x and h must be introduced.

    Many earlier works assume that the blur kernel followsa parametric model [1] [2] or satisfies some additional con-straints, e.g., centrosymmetric, nonnegative and normalizationconstraint [3] [4]. For the latent image, many BID methods[5][23] utilize image sparsity to estimate the image.

    Fergus et al. [5] assume that the gradient of the latentimage x obeys a heavy-tailed distribution, meaning thatmost elements of x are zero or very small but a numberof elements are quite large. Specifically, they use a Mixture ofGaussians (MoG) to approximate a heavy-tailed distribution.However, as pointed out in [5], [24], and [17], using themaximum a posteriori (MAP) approach often yields deltakernels because the cost function favors a blurry explana-tion over sharp reconstructions. To avoid this problem, theVariational Bayesian (VB) approach [25] has been adoptedin [5], see also [26] [27] [17]. Using the VB approach, ageneral framework based on the Super Gaussian priors isproposed by Babacan et al. [17]. Instead of using MAPestimation, Levin et al. [24] [11] suggest a MAPh approach,namely marginalizing over latent images and then estimatingthe kernel alone. The rationale behind MAPh is that estimatingh alone is much better conditioned than estimating h and xtogether. Recently, following the MAPh approach and usinga majorization-minimization approach, Zhang and Wipf [23]propose a parameter free BID method for removing camerashake, in which the gradient image is modeled by an inde-pendent Gaussian distribution with zero mean and spatiallyvarying variance. Interested readers are referred to [28] for areview of of Bayesian blind deconvolution methods.

    An alternative to enforce image sparsity in the transformdomain is to use the shock filter [29], see [7] and [9]. In fact,the shock filter not only promotes image sparsity, but alsopredicts step edges. Following this idea, Xu and Jia [14] pointout that not all predicted step edges are helpful. Therefore, theysuggest an edge selection strategy for removing texture edges,which improves the result significantly. Instead of predictingstep edges, a probably better idea is to model step edgesas local minima of a sparsity measure, e.g., x1/x2,x0,

    i |x(i)|/(|x(i)|+x1/N), see [12], [15] and

    [22] for more details. Sun et al. [19] also propose a novelparameterized patch model to regularize step edges, usinglearned image statistics or synthetic structures. Making use ofdirectional filters, Zhong et al. [20] show that a good kernelcan be estimated from a blurred image with 1% 10% noise.Step edge based methods are robust to noise since noise andsmall gradients are smoothed out, but may fail if the image isdominated by textures. By combining step edge prediction [9]


    with covariance image [11] used in kernel estimation, Wanget al [18] propose a robust BID method.

    In this work, a BID algorithm that alternates between theestimation of image and blur is proposed. For the latent imageestimation, we use the Nondimensional Gaussianity Measure(NGM) proposed in [22], to promote image sparsity. As theoptimization algorithm of [22] does not make use of the FFT,we propose a fast deconvolution algorithm with undeterminedboundary condition [30] to estimate the latent sharp image,which uses the variable splitting [31] and mask decoupling[32] [33] approaches. For the blur estimation part we proposea Variational Dirichlet (VD) method. This changes the regular-ization based approach to a variational Bayes approximationto estimate the posterior distribution of the blur. The benefit ofusing VD is that the resulting optimization problem does nothave any equality constraint but lower bound constraints only,so that it can be efficiently solved by the gradient projectionmethod [34]. The other benefit of VD is a new adaptivesparsity promoting term associated with the image, which ishelpful to remove the kernel noise. Furthermore as we willlater show, the VD approach can be applied to regularizationand variational based BID methods that alternate betweenimage and blur estimation, where the blur energy function(either from regularization or blur prior) is quadratic. This isthe case of many state-of-the-art BID methods. In summary,the main contributions of this paper include a novel efficientoptimization algorithm in the image space for NGM baseddeconvolution and a novel VD approach for kernel estimation.

    This paper is organized as follows. In Section II, weintroduce the framework to be used in BID. In Section III,we propose a fast nonblind image deconvolution algorithmformulated in the image domain for the NGM model in [22].Section IV contains our blur kernel estimation algorithm.We derive the VD approximation and show the connectionbetween VD approximation and MAP estimation. An efficientgradient projection algorithm is proposed to solve the resultingoptimization problem. This section also discusses the appli-cations of the proposed VD in kernel estimation when theblur prior is quadratic. A multiscale implementation of the VDbased BID method is presented in Section IV-D, followed by adiscussion on parameter setting and implementation in SectionIV-E. In Section V, we compare our BID method with state-of-the-art BID methods. Experimental results on synthetic andreal data show that the proposed method is very competitivein comparison with state-of-the-art BID algorithms [15], [17],[19], [20]. Finally, Section VI concludes this paper.


    Regularization based BID methods, [3], [4], [7], [10], [21],[26], [27], just to name a few, alternatively solve the followingtwo optimization problems,

    xk+1 = argminx


    2Hkx y22 + xRx(x), (2)

    hk+1 = argminh


    2Xk+1h y22 +

    h2Rh(h), (3)

    subject to h 0 (4)j

    h(j) = 1 (5)

    where k denotes iteration index, Hk is the convolution matrixformed by the estimated impulse response of the blur at thek-th iteration step hk, Xk+1 is the convolution matrix formedby the restored image xk+1, Rx and Rh are regularizationfunctions, x > 0 and h 0 are penalty weights controllingthe tradeoff between the data fitting term and regularization.

    Recent published papers, see for instance [9], [11], [15],[17] have shown, however, that using gradient images leads tobetter kernel estimation. Furthermore, the final output imageof (2) is often highly smoothed mainly due to the largepenalty we