11
AbstractBlind Image Deconvolution\Deblurring (BID) is a challenging task due to the lack of prior information about the blurring process. Noise and ringing artifacts resulted during the restoration process also deter the fine estimation of the pristine image. These artifacts mainly arise from using a poorly estimated Point Spread Function (PSF) combined with a non-efficient restoration filter. This research presents a steepest descent BID scheme for kurtosis maximization. Assuming that uniform blur can be modeled by a parametric form, the scheme tries to estimate the true blurring parameters for the absolute maximum kurtosis value of the deblurred image. The scheme is devised to handle any type of blur that can be modeled in a parametric form such as Gaussian, motion and out-of-focus blur. Gradients are computed against the parameters of the blurring PSF, which is optimized in the direction of increasing kurtosis value till the pristine image is estimated. The scheme is simple and efficient and does not require any prior knowledge about the image and the blurring process. The algorithms and gradients for a number of blurs are derived, and the performance improvements are corroborated through a set of simulations. Validation has been carried out on various examples, both benchmark and real images, and it shows that the scheme optimizes on the parameters in a close vicinity of the original blurring functions. Full- Reference and non-reference image quality measures were used to estimate the eminence of the deblurring results. The proposed method achieved marked improved results over the existing methods. Index TermsBlind Image Deconvolution (BID), Gradient Descent, Image Quality Measures, Kurtosis maximization. I. INTRODUCTION ince the realization of the blind image deconvolution/deblurring (BID) problem in early 1960’s, it still remains a challenging task to the image processing community to find an efficient, reliable and most importantly diversely applicable scheme. A scheme should not only estimate the actual point spread function (PSF) but also restore the image. The main challenge arises from little or no prior information about the image or the blurring process as well as the lack of optimal restoration filters that can eliminate the blurring effect. The two common side effects of deblurring are noise amplification and ringing artifacts arising in the deblurred image due to unrealizable or imperfect restoration filter. Furthermore, developing a scheme that can handle The authors are with the School of Electrical and Electronic Engineering, The University of Manchester, Manchester, M13 9PL, UK. different types of blur especially for real images is yet to be realized to a satisfactory extent. A variety of BID schemes and restoration filters have been proposed over the years. This collection of methods ranges from the time domain to the frequency domain, simultaneous or separate PSF estimation technique, parametric to non- parametric. Examples include the Richardson-Lucy method, Total Variation, Wiener filter, Maximum Likelihood method, Minimum Entropy Deconvolution, Non-negative and Support Size Recursive Inverse Filter, Simulated Annealing, and Multi-channel Blind Deconvolution [1]-[8]. Even hardware solutions have been formulated to revisit the BID problem by capturing much better images in the first place [9], [10]. More recent BID methods also include those based on kurtosis minimization [11] and image priors [12], though both are not fully automatic, as well as super resolution measures [13]. These methods do provide, to some extent, solutions to the BID problem; however many are not satisfactory in terms of stability, robustness, uniqueness and convergence. Mainly, the robustness of these schemes remains or becomes questionable when real-life blurred images are to be restored. This is because unlike artificial deblurring, real deblurring suffers heavily from noise and ringing effects. The blurred images, esp. taken from low quality cameras, may have been degraded by arbitrarily shaped PSFs that are complex and cannot be easily modeled by usual parametric blurring models [14]-[18]. Handling these cases requires more effort since no prior information about the blurring shape or size is known. Nowadays, most of the BID schemes focus mainly on deblurring images corrupted by motion blurring PSFs of arbitrary shapes. Not only because it has common occurrences in real life image acquisition but also because it is challenging. This type of blur ensues mainly from camera shakes or movements of objects or background in the focal range. The PSFs of such blurs are usually of arbitrary shape and sometimes even space invariant. Restoring such images requires more effort, where complex procedures are applied to approximate the blurring kernel as well as restore the image to its pristine form while deterring the deblurring noise and ringing artifacts. Restoring motion blurred images as compared to Gaussian and out-of-focus blurred ones is less problematic in restoration terms as the deblurring of the latter two blurs produces more ringing artifacts and leaves significant amount of residual blur. The community and researchers over the years have strived to come up with BID schemes that can effectively handle Efficient Blind Image Deblurring with Gradient Descent Based Kurtosis Maximization Aftab Khan and Hujun Yin, Senior Member, IEEE S

Efficient Blind Image Deblurring with Gradient Descent

  • Upload
    others

  • View
    11

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Efficient Blind Image Deblurring with Gradient Descent

Abstract—Blind Image Deconvolution\Deblurring (BID) is a

challenging task due to the lack of prior information about the

blurring process. Noise and ringing artifacts resulted during the

restoration process also deter the fine estimation of the pristine

image. These artifacts mainly arise from using a poorly estimated

Point Spread Function (PSF) combined with a non-efficient

restoration filter. This research presents a steepest descent BID

scheme for kurtosis maximization. Assuming that uniform blur

can be modeled by a parametric form, the scheme tries to

estimate the true blurring parameters for the absolute maximum

kurtosis value of the deblurred image. The scheme is devised to

handle any type of blur that can be modeled in a parametric form

such as Gaussian, motion and out-of-focus blur. Gradients are

computed against the parameters of the blurring PSF, which is

optimized in the direction of increasing kurtosis value till the

pristine image is estimated. The scheme is simple and efficient

and does not require any prior knowledge about the image and

the blurring process. The algorithms and gradients for a number

of blurs are derived, and the performance improvements are

corroborated through a set of simulations. Validation has been

carried out on various examples, both benchmark and real

images, and it shows that the scheme optimizes on the parameters

in a close vicinity of the original blurring functions. Full-

Reference and non-reference image quality measures were used

to estimate the eminence of the deblurring results. The proposed

method achieved marked improved results over the existing

methods.

Index Terms—Blind Image Deconvolution (BID), Gradient

Descent, Image Quality Measures, Kurtosis maximization.

I. INTRODUCTION

ince the realization of the blind image

deconvolution/deblurring (BID) problem in early 1960’s,

it still remains a challenging task to the image processing

community to find an efficient, reliable and most importantly

diversely applicable scheme. A scheme should not only

estimate the actual point spread function (PSF) but also restore

the image. The main challenge arises from little or no prior

information about the image or the blurring process as well as

the lack of optimal restoration filters that can eliminate the

blurring effect. The two common side effects of deblurring are

noise amplification and ringing artifacts arising in the

deblurred image due to unrealizable or imperfect restoration

filter. Furthermore, developing a scheme that can handle

The authors are with the School of Electrical and Electronic Engineering,

The University of Manchester, Manchester, M13 9PL, UK.

different types of blur especially for real images is yet to be

realized to a satisfactory extent.

A variety of BID schemes and restoration filters have been

proposed over the years. This collection of methods ranges

from the time domain to the frequency domain, simultaneous

or separate PSF estimation technique, parametric to non-

parametric. Examples include the Richardson-Lucy method,

Total Variation, Wiener filter, Maximum Likelihood method,

Minimum Entropy Deconvolution, Non-negative and Support

Size Recursive Inverse Filter, Simulated Annealing, and

Multi-channel Blind Deconvolution [1]-[8]. Even hardware

solutions have been formulated to revisit the BID problem by

capturing much better images in the first place [9], [10]. More

recent BID methods also include those based on kurtosis

minimization [11] and image priors [12], though both are not

fully automatic, as well as super resolution measures [13].

These methods do provide, to some extent, solutions to the

BID problem; however many are not satisfactory in terms of

stability, robustness, uniqueness and convergence. Mainly, the

robustness of these schemes remains or becomes questionable

when real-life blurred images are to be restored. This is

because unlike artificial deblurring, real deblurring suffers

heavily from noise and ringing effects. The blurred images,

esp. taken from low quality cameras, may have been degraded

by arbitrarily shaped PSFs that are complex and cannot be

easily modeled by usual parametric blurring models [14]-[18].

Handling these cases requires more effort since no prior

information about the blurring shape or size is known.

Nowadays, most of the BID schemes focus mainly on

deblurring images corrupted by motion blurring PSFs of

arbitrary shapes. Not only because it has common occurrences

in real life image acquisition but also because it is challenging.

This type of blur ensues mainly from camera shakes or

movements of objects or background in the focal range. The

PSFs of such blurs are usually of arbitrary shape and

sometimes even space invariant. Restoring such images

requires more effort, where complex procedures are applied to

approximate the blurring kernel as well as restore the image to

its pristine form while deterring the deblurring noise and

ringing artifacts. Restoring motion blurred images as

compared to Gaussian and out-of-focus blurred ones is less

problematic in restoration terms as the deblurring of the latter

two blurs produces more ringing artifacts and leaves

significant amount of residual blur.

The community and researchers over the years have strived

to come up with BID schemes that can effectively handle

Efficient Blind Image Deblurring with Gradient

Descent Based Kurtosis Maximization

Aftab Khan and Hujun Yin, Senior Member, IEEE

S

Page 2: Efficient Blind Image Deblurring with Gradient Descent

different types of blurs. Our motivation is to design a BID

scheme that is efficient in terms of restoration speed and

effective in terms of restoration quality. The proposed scheme

uses a parametric representation for the uniform Gaussian,

motion or out-of-focus blur. The parameters are optimized by

a steepest descent scheme using a restoration measure to find

the true blurring values. The proposed method provides a

unified base for BID framework.

The parametric model of blur helps simplify the search for

the blurring parameters where the use of a restoration measure

is optimized for desired result. In the quest for simple

deblurring processes, the proposed method relies on higher

order statistics based non-Gaussianity measures to lead in the

direction of increased quality of the deblurred image. The

earlier schemes developed by the authors relied on the use of

non-Gaussianity measures in either spatial or frequency

domain, namely spatial and spectral kurtosis [19], [20]. Apart

from this, re-blurring based image quality measure was also

used but its usage was very limited as it was highly sensitive

to restoration noise and artifacts

II. PARAMETRIC BLURRING MODELS

Let ),( nmf represent the original image without any form

of degradation, ),( nmh be the PSF and ),( nmg the output of

the image acquisition system. Mathematically, for a system of

stationary impulse response across the entire image (i.e. a

spatially invariant stationary PSF), the convolution is given by

[21], [22],

),(),(*),(),( nmvnmhnmfnmg (1)

where * denotes the convolution operator, ),( nmv represents

additive noise; and m and n are spatial coordinates.

The frequency domain representation, with spectral

coordinates i and j , obtained using the Fourier Transform is

given as

),(),(),(),( jiNjiHjiFjiG (2)

The blurring PSFs studied here are spatially invariant, that

is, they are independent of the spatial location of the image

and provide the same or uniform effect at each pixel location.

Modeling, restoration and estimation of the original image for

the uniform PSF is less cumbersome as compared to spatially

variant blurs. The classes of blur that can be easily modeled

include the atmospheric turbulence, motion and out-of-focus.

The PSF for atmospheric turbulence blur for long exposures

can be approximated by Gaussian blur and the degraded image

regarded as a Gaussian blurred image [23]. The Gaussian blur

is the result of filtering an image by a low pass filter estimated

by a Gaussian function. The 2-D Gaussian PSF is given by

2

22

2 2

)(

2

1),(

nmenmh (3)

where is the width of the blurring kernel.

Motion blurs commonly result from relative motions

between the scene and the recording device. They can be in

the form of translation, sudden change in scale, rotation or a

combination of these. The motion blur type dealt with here is

the case of translation. Denoting the length of motion by L ,

the angle by , and the pixel coordinates by i and j , the

motion blurring PSF is given by [23],

elsewhere 0

tan and 2

if 1

),;,(

22 n

mLnm

LLnmh (4)

When a camera captures a 2-D image of a 3-D scene,

certain points in the scene may not be focused. For a camera

with a circular aperture, the PSF of a point not focused looks

like a circular disk with a specific radius. The spatially

continuous out-of-focus blur of radius R is given by [23],

elsewhere 0

if 1

);,(

222

2Rnm

RCRnmh (5)

where C is a constant that must be chosen so that energy

conservation law is satisfied

III. PROPOSED GRADIENT BASED DEBLURRING SCHEME

The derivation of the gradient based BID algorithm scheme

starts with differentiation of the cost function with respect to

the parameter of the blurring function. In the proposed method

a kurtosis based cost function for the deblurred image is

employed. The parameter is update iteratively using the

steepest descent method,

)( 1 YK

ppp (6)

under the constraint,

)()(

1YKYK

pp

(7)

where is the blurring PSF parameter, such as kernel width,

length, or radius in case of Gaussian, motion or out-of-focus

blur respectively, p is the iteration number, )(YKp

is the

kurtosis of the deblurred image for the current parameter,

|)(| YKp

represents the gradient matrix for the absolute

kurtosis value, and is the convergence step size.

Since

))(sgn()()( YKYKYK (8)

Eq. (6) can be rewritten as

Page 3: Efficient Blind Image Deblurring with Gradient Descent

))(sgn()( 1 YKYK

pppp (9)

For , a fixed scalar value can be used, e.g. kp where k is

the iteration number and 10 p . For faster convergence,

more efficient computation schemes can also be employed.

Assume that Y is the deblurred image resulting from the

deconvolution of the blurred image G with the Wiener

restoration filter, given by

2

*

H

H

(10)

where is the noise to signal ratio. For simplicity of future

derivation we rewrite the equation as

HH

H

HHH

HH

H

H

H

H

3

2

22 ||

*

||

*

(11)

The kurtosis of the deblurred image Y is given by

3)2(

)4()(

2

Y

YYK

(12)

Let )0(Y represent the expected value or mean of Y .

M

Qi

N

QjY jiY

QNQM 1 1221 1

),(11

)0(

(13)

The gradient matrix is derived as follows.

2)2(

)4()()(

Y

YYKYK

(14)

2

1 12

2

21

1 12

4

21

1

1

)]0(),([11

)]0(),([11

)(M

Qi

N

QjY

M

Qi

N

QjY

jiYQNQM

jiYQNQM

YK

(15)

M

Qi

N

QjY

Y

Y

M

Qi

N

QjY

Y

jiYQNQM

jiYQNQM

YK

1 12213

1 12

3

212

1

1

)]0(),([11

)2(

)4(4

)]0(),([11

)2(

14

)(

(16)

where

))0(()())0(( YYYY

(17)

)0(Y

is equivalent to the mean value of Y

and

HH

HG

HH

HGY

3

2

3

2

)(

(18)

Through further manipulations, we get

H

HH

HH

HH

HGY

23

22

3

32)( (19)

The term H

needs to be calculated for specific types of

blur. The derivation for the Gaussian blurring PSF is as

follows.

In case of Gaussian blurring, the Optical Transfer Function

(OTF) for the parametric model is given by

2

22

2

22

1),(

f

ji

f

ejiH

(20)

with the f of the OTF being reciprocal to of the PSF. For

simplicity f is simply represented by .

Differentiating the OTF with respect to )( of the OTF,

2

22

2

22

1),(),(

ji

ejiHjiH

(21)

2

22

2

22

2

2

2

3 2

1

2

2),(

jiji

eejiH

2

22

2

22

2

5

222

3 22

2

jiji

eji

e

2

22

2

22

2

5

222

3 22

2

jiji

eji

e (22)

That is,

3

222),(),(

jijiHjiH

(23)

Page 4: Efficient Blind Image Deblurring with Gradient Descent

The derivations of the gradient matrix for the motion and

out-of-focus blur can be found in Appendix.

The parameter of the blurring PSF model is optimized in the

direction of increasing value of the absolute kurtosis till the

gradient shows a flat point or the difference of kurtosis

between successive iterations falls below a specified tolerance

value.

IV. EXPERIMENTAL RESULTS AND DISCUSSIONS

A. Experimental Setup

The proposed scheme has been tested on various images

degraded with different degrading functions. Experiments

were conducted on both artificial and real blurred images for

three different types of blur. Several image quality measures

were employed to evaluate the quality of deblurred images.

The full reference Image Quality Assessment (IQA) measures

include the Mean Structural SIMilarity (MSSIM) index [24]-

[26] and the Universal Quality Index (UQI) [27]. The

blind\no-reference based IQA measures are the

Blind/Reference-less Image Spatial QUality Evaluator

(BRISQUE) [28], [29] and the Natural Image Quality

Evaluator (NIQE) [30], [31]. A brief description of these

quality measures is given as follows.

Fig. 1. Test images used for deblurring of artificially blurred images.

The SSIM is an objective image quality metric that measures

the structural similarity between two images by comparison of

local patterns of pixel intensities that have been normalized for

luminance and contrast. Assuming two images f and g , the

SSIM is a function of the luminance ),( gfl , contrast ),( gfc

and the structural similarity ),( gfs of the images.

)),(),,(),,((),( gfsgfcgfltyxSSIM

(24)

1

12),(

22 C

Cgfl

gf

gf

(25)

2

22),(

22 C

Cgfc

gf

gf

(26)

3

3),(

C

Cgfs

gf

fg

(27)

where is the mean, is the standard deviation of the

image, C1, C2 and C3 are constants and t represents the SSIM

transfer function. SSIM is computed for each pixel location

and generally its mean value is used, named as the Mean

SSIM (MSSIM). SSIM has been successfully used previously

for denoising and classification [32], [33].

The UQI analyses the loss of correlation, luminance and

contrast distortion among the two images as the base for

quality perception. The UQI for two images f and g is

defined as [27],

])[(

4),(

2222gfgf

gffggfQ

(28)

or

2222

2.

2.),(

gf

gf

gf

gf

gf

fggfQ

(29)

The first term is the correlation coefficient of the two images

whereas the second and third terms are measures of mean

luminance and structural similarities.

The BRISQUE measure is a distortion measure based on

spatial domain natural scene statistics. It is also a non-

reference IQA model. Rather than computing distortion

specific features such as ringing, blur or blocking, it uses

scene statistics of locally normalized luminance coefficients to

quantify possible losses of ‘naturalness’ in the image due to

the presence of distortions, thereby leading to a holistic

measure of quality. The BRISQUE model uses a mapping

from feature space to quality scores using a regression module

that yields a measure of image quality. The features comprise

the statistic measures of a generalized Gaussian distribution

fitting of the Mean Subtracted Contrast Normalized (MSCN)

coefficients. The MSCN coefficients for the distorted image

g are obtained by subtracting the local mean alue g and

then dividing by the local contrast function g such that

Cnm

nmnmgnmg

g

g

),(

),(),(),(ˆ

(30)

where

Img-01 Img-02 Img-03 Img-04

Img-05 Img-06 Img-07 Img-08

Img-09 Img-10 Img-11 Img-12

Img-13 Img-14 Img-15 Img-16

Page 5: Efficient Blind Image Deblurring with Gradient Descent

U

Uk

V

Vllklkg nmgwnm ),(),( ,,

(31)

U

Uk

V

Vlglklkg nmnmgwnm 2

,, )),(),((),( (32)

and C is a constant. VVlUUkww lk ,.....,,,.....,|, is

a 2-D circular-symmetric Gaussian weighting function and

3VU was used for calculating the measure.

The fourth IQA measure, the none reference NIQE, is a

completely blind image quality analyzer that only makes use

of measurable deviations from statistical regularities observed

in natural images, without training on human-rated distorted

images and without any exposure to distorted images. The

same computation for image quality as in the BRISQUE is

used with the exception that NIQE uses natural scene statistics

features from a staple of natural images, while the BRISQUE

is trained on features obtained from both natural and distorted

images as well as human judgment of quality of the images.

Images from the Desktop Nexus image wallpaper database

[34] were used in the experiments in the artificial blurring

case. Real blurred natural images, captured by either ourselves

or others, were used in the case of real blind image deblurring.

1) Artificially blurred images

Examples of the Desktop Nexus images are shown in Fig. 1

Fig. 2 shows the restorations of their Gaussian-blurred images

using the proposed scheme. The estimated PSF parameters and

comparisons of different quality measures for the Gaussian

deblurring are given in Table 1.

(a) (c)

(b) (d)

Fig. 2. (a) and (b) Gaussian blurred images, (c) and (d) their respective

estimates using the proposed BID scheme.

Fig. 3 shows the out-of-focus blurred images and their

respective deblurred counterparts using the proposed scheme.

Table 2 summarizes the results for out-of-focus deblurring

along with the comparison of different image quality

measures. Fig.4 shows the deblurring result for motion blurred

images. The results for motion deblurring are summarized in

Table 3.

(a) (c)

(b) (d)

Fig. 3. (a) and (b) out-of-focus blurred images; (c) and (d) their respective estimates using the proposed BID scheme.

(a) (c)

(b) (d)

Fig. 4. (a) and (b) motion blurred images, (c) and (d) their respective estimates

using the proposed BID scheme.

(a) (e)(a) (e)

(d) (h)(d) (h)

(b) (f)(b) (f)

(d) (h)(d) (h)

(a) (e)(a) (e)

(c) (g)(c) (g)

Page 6: Efficient Blind Image Deblurring with Gradient Descent

TABLE 1. PSF PARAMETER ESTIMATION FOR GAUSSIAN BLURRED IMAGES AND QUANTITATIVE COMPARISON OF SSIM, UQI, BRISQUE AND NIQE

QUALITY MEASURE.

MSSIM UQI BRISQUE NIQE

Image Original

Sigma

Estimated

Sigma Blurred

Proposed

Scheme Blurred

Proposed

Scheme Blurred

Proposed

Scheme Blurred

Proposed

Scheme

Img-01 1.63 1.85 0.85 0.90 0.99 0.99 56.69 18.28 7.28 8.40

Img-02 4.98 4.76 0.73 0.81 0.95 0.97 77.15 70.69 10.84 7.97

Img-03 5.13 5.26 0.62 0.68 0.94 0.96 79.32 73.12 11.85 9.05

Img-04 3.88 4.05 0.78 0.86 0.97 0.98 72.87 55.30 10.64 7.65

Img-05 0.71 0.68 0.88 0.95 0.99 1.00 25.45 19.06 3.80 3.88

Img-06 0.94 1.21 0.97 0.88 1.00 1.00 52.71 38.15 7.00 7.21

Img-07 2.07 2.18 0.52 0.74 0.94 0.97 61.24 40.69 7.40 7.76

Img-08 2.63 2.50 0.69 0.80 0.95 0.97 67.06 34.43 9.10 6.99

Img-09 3.45 3.32 0.67 0.76 0.97 0.98 67.54 54.20 10.28 6.86

Img-10 2.44 2.24 0.75 0.86 0.97 0.98 64.76 36.55 8.56 6.45

Img-11 1.64 1.82 0.75 0.88 0.98 0.99 54.33 26.65 6.56 6.59

Img-12 0.98 0.99 0.93 0.94 0.99 1.00 47.88 11.99 5.48 3.85

Img-13 1.38 1.53 0.89 0.96 0.99 0.99 45.66 24.88 6.66 5.94

Img-14 3.13 3.26 0.62 0.80 0.93 0.97 72.01 49.99 9.79 6.85

Img-15 4.18 4.13 0.79 0.81 0.97 0.98 80.98 58.73 10.61 8.11

Img-16 5.47 5.39 0.45 0.55 0.93 0.95 82.44 70.65 10.77 9.02

AVERAGE 0.75 0.82 0.97 0.98 63.00 42.71 8.54 7.04

TABLE 2. PSF PARAMETER ESTIMATION FOR OUT-OF-FOCUS BLURRED IMAGES AND QUANTITATIVE COMPARISON OF SSIM, UQI, BRISQUE AND NIQE

QUALITY MEASURE.

MSSIM UQI BRISQUE NIQE

Image Original

Radius

Estimated

Radius Blurred

Proposed

Scheme Blurred

Proposed

Scheme Blurred

Proposed

Scheme Blurred

Proposed

Scheme

Img-01 7.00 6.84 0.70 0.81 0.96 0.98 69.52 27.22 10.24 4.32

Img-02 8.85 9.20 0.70 0.76 0.94 0.97 76.69 44.17 10.42 5.34

Img-03 10.00 9.40 0.59 0.70 0.93 0.95 77.68 39.09 11.99 4.97

Img-04 11.50 11.47 0.69 0.84 0.93 0.97 76.05 32.21 12.35 5.55

Img-05 1.00 0.74 0.95 0.97 1.00 1.00 22.08 12.82 3.29 2.70

Img-06 2.50 2.46 0.97 0.93 1.00 1.00 60.49 14.96 8.15 5.54

Img-07 4.00 3.91 0.48 0.83 0.93 0.98 57.52 10.78 6.92 3.77

Img-08 5.50 4.72 0.64 0.59 0.94 0.94 68.54 29.01 9.56 5.19

Img-09 13.00 12.58 0.63 0.68 0.96 0.97 67.17 39.76 10.84 4.61

Img-10 14.50 14.14 0.63 0.61 0.93 0.95 73.93 45.10 12.12 5.31

Img-11 16.00 15.27 0.51 0.58 0.93 0.95 76.35 43.96 11.36 5.20

Img-12 17.50 18.74 0.73 0.79 0.96 0.98 77.13 37.51 11.19 4.75

Img-13 19.00 18.82 0.40 0.70 0.86 0.95 80.40 45.59 11.93 5.96

Img-14 20.50 20.03 0.42 0.65 0.82 0.93 83.63 46.97 11.43 5.83

Img-15 22.00 21.58 0.68 0.64 0.93 0.93 86.56 56.11 12.76 7.12

Img-16 23.50 22.89 0.38 0.47 0.89 0.91 84.80 57.47 11.57 7.85

AVERAGE 0.63 0.72 0.93 0.96 71.16 36.42 10.38 5.25

Page 7: Efficient Blind Image Deblurring with Gradient Descent

NIQ

E

Pro

po

sed

Sch

eme

5.9

89

3.9

13

4.2

21

4.3

79

4.3

11

4.4

67

5.6

26

6.2

88

4.7

70

4.8

44

3.8

44

3.9

92

4.4

49

4.3

12

5.4

62

5.6

15

4.7

80

Qi

Shan

5.6

63

5.1

46

5.2

57

5.6

63

5.9

16

6.3

50

7.6

87

8.6

94

5.4

84

4.8

40

6.0

72

5.3

35

6.0

58

5.2

32

6.4

92

5.9

20

5.9

88

Blu

rred

6.3

64

7.0

59

6.4

77

8.9

86

8.3

93

9.3

69

7.1

72

7.2

78

7.1

51

7.2

65

7.9

36

9.0

74

9.2

44

8.0

99

8.3

97

6.0

70

7.7

71

BR

ISQ

UE

Pro

po

sed

Sch

eme

24.0

62

36.5

37

42.7

20

38.1

49

44.8

11

21.6

21

24.5

33

35.9

00

26.9

21

24.4

12

53.3

11

26.1

64

46.4

89

45.8

33

55.4

49

45.2

94

37.0

13

Qi

Shan

34.7

59

64.9

61

56.4

72

65.9

50

52.7

07

67.1

86

39.9

75

39.2

33

27.3

67

26.1

57

13.1

88

27.5

14

26.6

08

25.0

11

58.3

30

49.0

92

42.1

57

Blu

rred

34.7

59

64.9

61

56.4

72

65.9

50

52.7

07

67.1

86

39.9

75

39.2

33

27.3

67

26.1

57

13.1

88

27.5

14

26.6

08

25.0

11

58.3

30

49.0

92

42.1

57

UQ

I

Pro

po

sed

Sch

eme

0.9

92

0.9

92

0.9

87

0.9

93

0.9

87

0.9

96

0.9

72

0.9

78

0.9

80

0.9

74

0.9

78

0.9

87

0.9

76

0.9

67

0.9

62

0.9

63

0.9

80

Qi

Shan

0.9

84

0.9

86

0.9

77

0.9

79

0.9

74

0.9

90

0.9

26

0.9

77

0.9

69

0.9

66

0.9

42

0.9

74

0.9

62

0.9

45

0.9

56

0.9

32

0.9

65

Blu

rred

0.9

90

0.9

85

0.9

75

0.9

82

0.9

83

0.9

91

0.9

28

0.9

65

0.9

72

0.9

58

0.9

42

0.9

75

0.9

19

0.8

94

0.9

69

0.9

34

0.9

60

MS

SIM

Pro

po

sed

Sch

eme

0.9

44

0.9

26

0.8

93

0.9

26

0.8

28

0.9

21

0.8

10

0.8

58

0.8

28

0.8

26

0.8

04

0.8

27

0.8

06

0.7

71

0.5

87

0.6

74

0.8

27

Qi

Shan

0.8

67

0.8

94

0.8

10

0.8

18

0.7

30

0.8

90

0.6

16

0.8

60

0.7

60

0.7

94

0.5

30

0.7

69

0.7

18

0.6

73

0.6

79

0.5

51

0.7

47

Blu

rred

0.9

20

0.8

75

0.7

57

0.8

59

0.7

40

0.9

24

0.5

36

0.7

75

0.7

25

0.7

26

0.5

72

0.7

78

0.5

13

0.5

51

0.7

55

0.5

04

0.7

19

Est

imat

ed

Len

gth

8

8

9

10

12

12

13

13

14

15

17

19

19

21

21

24

Ori

gin

al

Len

gth

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

Imag

e

Img

-01

Img

-02

Img

-03

Img

-04

Img

-05

Img

-06

Img

-07

Img

-08

Img

-09

Img

-10

Img

-11

Img

-12

Img

-13

Img

-14

Img

-15

Img

-16

AV

ER

AG

E

TA

BL

E 3

. PS

F P

AR

AM

ET

ER

ES

TIM

AT

ION

FO

R M

OT

ION

BL

UR

RE

D I

MA

GE

S A

ND

QU

AN

TIT

AT

IVE

CO

MP

AR

ISO

N O

F S

SIM

AN

D U

QI

QU

AL

ITY

ME

AS

UR

E

Page 8: Efficient Blind Image Deblurring with Gradient Descent

2) Real blurred images

The proposed deblurring scheme has also been performed

on real images with motion blur, a typical problem facing

amateur photographers. The deblurred images using the

proposed scheme have been compared with the estimated

images using the blind deconvolution scheme by Shan et al. in

[35].

(a) (b)

(c) (d) (e)

Fig. 5. (a) Real motion blurred LABEL image of 720655. (b) Deblurred

using the proposed BID scheme. (c) Deblurred using Shan et al’s BID scheme

[35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF using Shan et al’s BID scheme.

In the first case, an image was captured by a low quality

camera. The blurred image, depicted in Fig. 5 (a), shows an

approximate vertical motion blur with the small digits at the

bottom of the image becoming unreadable. The image also

contains noise and uneven illumination due to poor lighting –

one of the most difficult cases in deblurring. Fig. 5(b) shows

the deblurred image estimated using the proposed scheme with

the estimated blurring kernel shown in Fig. 5(d). Fig. 5(c)

shows the deblurred image and the corresponding PSF

(rescaled) in Fig. 5(e) estimated using the blind deconvolution

scheme of Shan et al. [35]. The image in Fig. 5(c) seems to

have recovered well, however the digits are still unreadable

since the PSF estimated using this scheme does not completely

follow a uniform motion blur. Using the proposed scheme, a

uniform motion PSF of length 21 pixels and angle zero degree

was estimated. In Fig. 5(b), the digits in the deblurred image

are much clear and easily readable. An edgetaper function was

used to reduce the ringing effect caused by using the discrete

Fourier transform.

Figs. 6 and 7 show the deblurring of motion blurred images

resulting from object movement and camera handshake

respectively. The motion blur is almost linear at a certain

angle.

(a) (b)

(c) (d) (e)

Fig. 6. (a) Real motion blurred MATLAB_BOOK image of 469323. (b)

Deblurred using the proposed BID scheme. (c) Deblurred using Shan et al’s

BID scheme [35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF using Shan et al’s BID scheme.

Another motion blurred image taken by an ordinary digital

camera and its deblurred versions are shown in Fig. 8. Fig.

8(b) shows the estimate using the proposed scheme with the

corresponding estimated PSF in Fig. 8(d). Fig. 8(c) shows the

deblurred image and the corresponding PSF (rescaled) in Fig.

8(e) estimated using the blind deconvolution scheme of Shan

et al [35]. The image seems to have recovered well and the

main and sub title text of the book is more readable in the

proposed scheme.

Table 4 shows the BRISQUE and NIQE results for the

deblurred images.

(a) (b)

(c) (d) (e)

Fig. 7. (a) Real motion blurred BUILDINGS image of 459344. (b) Deblurred using the proposed BID scheme. (c) Deblurred using Shan et al’s BID scheme

[35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF

using the Shan et al’s BID scheme.

(a) (b)

(c) (d)

(e) (f)

(a) (b)

(c) (d)

(e) (f)

35 -3

8 88

Page 9: Efficient Blind Image Deblurring with Gradient Descent

(a) (b)

(c) (d) (e)

Fig. 8. (a) Real motion blurred DIP_BOOK image of 364602. (b) Deblurred

using the proposed BID scheme. (c) Deblurred using Shan et al’s BID scheme [35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF

using the Shan et al’s BID scheme.

V. CONCLUSIONS

An improved BID scheme based on kurtosis and steepest

descent algorithm is proposed that can estimate uniform blur

of a parametric PSF form. A steepest descent optimization

scheme was employed to optimize the blurring parameters in

the direction of maximum absolute kurtosis of the deblurred

image. The proposed method operates on single image and is

simple, efficient and easy to implement. The method is

applicable to various types of blurring such as Gaussian,

motion and out-of-focus. Experiments on both artificial and

real blurred images have shown the capability of the scheme.

The scheme can be further extended to recover images blurred

by arbitrary shape motion/camera handshake PSFs.

APPENDIX

A. Motion Blur

The linear motion modeled here is the result of translation

in the horizontal direction. In order to simplify the search for

true blur parameters using the steepest descent scheme, we

rotate the image until the blur can be assumed horizontal. This

simplifies the case from two parameters i.e. length )(L and

angle )( of blur .Now only the length of blur )(L needs to

be optimized. The OTF for motion blur according to [36] is

given by

j

i

iLe

LjiH Lki

)sin(

))1(sin

1

1),(

(A.1)

Differentiating the OTF with respect to the length parameter

)(L of the OTF,

)sin(

))1(sin

1

1),(

),(

i

iLe

LLjiH

L

jiH

Lki

(A.2)

(a) (b)

(c) (d)

(e) (f)

TABLE 4 PSF PARAMETER ESTIMATION FOR MOTION BLURRED IMAGES AND QUANTITATIVE COMPARISON OF BRISQUE AND NIQE QUALITY

MEASURE.

BRISQUE NIQE

Image Original Manual Shan et al

[32] Our Original Manual Qi Shan Our

LABEL 56.080 27.350 52.000 54.070 7.110 6.230 8.100 7.680

DIP_BOOK 29.770 17.780 40.210 45.680 5.780 4.530 6.310 6.510

MATLAB_BOOK 5.480 15.490 28.840 48.550 5.760 5.530 4.070 6.310

BUILDINGS 39.850 25.960 41.170 39.580 7.010 5.440 6.410 6.850

Page 10: Efficient Blind Image Deblurring with Gradient Descent

)sin(

))1(sin

1

1

1

1

)sin(

))1(sin),(

i

iLe

LL

LLi

iLejiH

L

Lki

Lki

(A.3)

))1(sin1

1

)sin(

1

1

1)1(

)sin(

))1(sin),(

2

iLeLLi

Li

iLejiH

L

Lki

Lki

(A.4)

)sin(1

))1(cos))1(sin

)sin(

))1(sin

1

1),(

2

iL

iLieekiiL

i

iLe

LjiH

L

LkiLki

Lki

(A.5)

),())1(cot)(),(),(

1

1

),(

jiHiLijiiHkjiHL

jiHL

(A.6)

))1(cot)(

1

1),(),( iLiik

LjiHjiH

L (A.7)

B. Out-of-focus Blur

The OTF for out-of-focus blur according to [36] is given by

2221 ,)(

),( jipRp

RpJjiH

(A.8)

where 1J is the first order Bessel function. The Bessel

function can be approximated by an exponential for

sufficiently large data [37].

Rp

ejiH

Rp

2),(

(A.9)

Differentiating the OTF with respect to the radius parameter

)(R of the OTF,

RpRp

Rp

eRRpRpR

e

Rp

e

RjiH

R

2

1

2

1

2),(

(A.10)

RpRp

e

RpepjiH

R

RpRp

22

2/12),(

(A.11)

RppRp

ejiH

R

Rp

2),(

(A.12)

RppjiHjiH

R

),(),(

(A.13)

REFERENCES

[1] R. L. Lagendijk, A. M. Tekalp, and J. Biemond, “Maximum-

likelihood image and blur identification - a unifying approach,”

Optical engineering, vol. 29, no. 5, pp. 422-435, 1990.

[2] D. Kundur, and D. Hatzinakos, “A novel blind deconvolution

scheme for image restoration using recursive filtering,” IEEE

Transactions on Signal Processing, vol. 46, no. 2, pp. 375-390,

1998.

[3] T. F. Chan, and C. K. Wong, “Total variation blind

deconvolution,” IEEE Transactions on Image Processing, vol.

7, no. 3, pp. 370-375, 1998.

[4] W. H. Richardson, “Bayesian-based iterative method of image

restoration,” Journal of the Optical Society of America, vol. 62,

no. 1, pp. 55-59, 1972.

[5] R. A. Wiggins, “Minimum entropy deconvolution,”

Geoexploration, vol. 16, no. 1-2, pp. 21-35, 1978.

[6] M. R. Banham, and A. K. Katsaggelos, “Digital image

restoration,” IEEE Signal Processing Magazine, vol. 14, no. 2,

pp. 24-41, 1997.

[7] D. Kundur, and D. Hatzinakos, “Blind image deconvolution,”

IEEE Signal Processing Magazine, vol. 13, no. 3, pp. 43-64,

1996.

[8] R. L. Lagendijk, J. Biemond, and D. E. Boekee, “Regularized

iterative image-restoration with ringing reduction,” IEEE

Transactions on Acoustics Speech and Signal Processing, vol.

36, no. 12, pp. 1874-1888, 1988.

[9] A. Agrawal, and Y. Xu, “Coded exposure deblurring:

Optimized codes for PSF estimation and invertibility,” Proc.

IEEE Conference on Computer Vision and Pattern Recognition,

New York, 2009, pp. 2066-2073.

[10] C. Y. Zhou, S. Lin, and S. K. Nayar, “Coded aperture pairs for

depth from defocus and defocus deblurring,” International

Journal of Computer Vision, vol. 93, no. 1, pp. 53-72, 2011.

[11] D. Li, and S. Simske, “Atmospheric turbulence degraded-image

restoration by kurtosis minimization,” IEEE Geoscience and

Remote Sensing Letters, vol. 6, no. 2, pp. 244-247, 2009.

[12] M. S. C. Almeida and L. B. Almeida, “Blind and semi-blind

deblurring of natural images” IEEE Transactions on Image

Processing, vol. 22, no. 6, pp. 2101-2114, 2013.

[13] E. Faramarzi, D. Rajan, and M. P. Christensen, “Unified blind

method for multi-image super-resolution and single/multi-

image blur deconvolution,” IEEE Transactions on Image

Processing, vol. 19, no. 1, pp. 36-52, 2009

[14] S. Cho, J. Wang, and S. Lee, “Handling outliers in non-blind

image deconvolution,” Proc. International Conference on

Computer Vision, New York, 2011, pp. 495-502.

[15] O. Whyte, J. Sivic, and A. Zisserman, “Deblurring shaken and

partially saturated images,” Proc. International Conference on

Computer Vision Workshops, New York, 2011.

[16] M. Hirsch, C. J. Schuler, S. Harmeling, and B. Scholkopf, “Fast

removal of non-uniform camera shake,” Proc. International

Conference on Computer Vision, New York, 2011, pp. 463-

470.

[17] X. Yuquan, W. Lu, H. Xiyuan, and P. Silong, "Single-image

blind deblurring for non-uniform camera-shake blur”, Proc.

Page 11: Efficient Blind Image Deblurring with Gradient Descent

11th Asian conference on Computer Vision Daejeon,

Korea,2013, pp. 336-348.

[18] A. Gupta, N. Joshi, C. L. Zitnick, M. Cohen, and B. Curless,

“Single image deblurring using motion density functions,”

Proc. 11th European Conference on Computer Vision,

Heraklion, Crete, Greece, 2010, pp. 171-184.

[19] H. Yin, and I. Hussain, “Independent component analysis and

non-gaussianity for blind image deconvolution and deblurring,”

Integrated Computer-Aided Engineering, vol. 15, no. 3, pp.

219-228, 2008.

[20] A. Khan, and H. Yin, “Efficient blind image deconvolution

using spectral non-gaussianity,” Integrated Computer-Aided

Engineering, vol. 19, no. 4, pp. 331-340, 2012.

[21] A. V. Oppenheim, R. W. Schafer, and T. G. Stockham,

“Nonlinear filtering of multiplied and convolved signals,”

Proceedings of the IEEE, vol. 56, no. 8, pp. 1264-1285, 1968.

[22] M. M. Sondhi, “Image restoration - removal of spatially

invariant degradations,” Proceedings of the IEEE, vol. 60, no.

7, pp. 842-853, 1972.

[23] R. L. Lagendijk, and J. Biemond, "Basic methods for image

restoration and identification," The essential guide to image

processing, pp. 323-348: San Diego, USA, Academic Press,

2009.

[24] D. Brunet, E. R. Vrscay, and Z. Wang, “On the mathematical

properties of the structural similarity index,” IEEE

Transactions on Image Processing, vol. 21, no. 4, pp. 1488-

1499, 2012.

[25] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli,

“Image quality assessment: From error visibility to structural

similarity,” IEEE Transactions on Image Processing, vol. 13,

no. 4, pp. 600-612, 2004.

[26] Z. Wang, and Q. Li, “Information content weighting for

perceptual image quality assessment,” IEEE Transactions on

Image Processing, vol. 20, no. 5, pp. 1185-1198, 2011.

[27] Z. Wang, and A. C. Bovik, “A universal image quality index,”

IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, Mar,

2002.

[28] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference

image quality assessment in the spatial domain,” IEEE

Transactions on Image Processing, vol. 21, no. 12, pp. 4695-

4708, 2012.

[29] A. Mittal, A. K. Moorthy, and A. C. Bovik. "BRISQUE

software release, "http://live.ece.utexas.edu/research/

quality/BRISQUE_release.zip.

[30] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a

"Completely blind" Image quality analyzer,” IEEE Signal

Processing Letters, vol. 20, no. 3, pp. 209-212, 2013.

[31] A. Mittal, A. K. Moorthy, and A. C. Bovik. "NIQE software

release," http://live.ece.utexas.edu/research/quality/niqe.zip.

[32] Y. Gao, A. Rehman, and Z. Wang, “CW-SSIM based image

classification,” Proc. 18th IEEE International Conference on

Image Processing, New York, 2011, pp. 1249-1252.

[33] A. Rehman, and Z. Wang, “SSIM-based non-local means

image denoising,” Proc. 18th IEEE International Conference on

Image Processing, New York, 2011, pp. 217-220.

[34] "Desktop nexus wallpaper image database,"

http://www.desktopnexus.com/.

[35] Q. Shan, J. Jia, and A. Agarwala, “High-quality motion

deblurring from a single image,” ACM Transactions on

Graphics, vol. 27, no. 3, pp. 10, 2008.

[36] J. Biemond, R. L. Lagendijk, and R. M. Mersereau, “Iterative

methods for image deblurring,” Proceedings of the IEEE, vol.

78, no. 5, pp. 856-883, 1990.

[37] W. O. Frank, W. L. Daniel, F. B. Ronald, and W. C. Charles,

NIST handbook of mathematical functions: Cambridge

University Press, 2010.