Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
Abstract—Blind Image Deconvolution\Deblurring (BID) is a
challenging task due to the lack of prior information about the
blurring process. Noise and ringing artifacts resulted during the
restoration process also deter the fine estimation of the pristine
image. These artifacts mainly arise from using a poorly estimated
Point Spread Function (PSF) combined with a non-efficient
restoration filter. This research presents a steepest descent BID
scheme for kurtosis maximization. Assuming that uniform blur
can be modeled by a parametric form, the scheme tries to
estimate the true blurring parameters for the absolute maximum
kurtosis value of the deblurred image. The scheme is devised to
handle any type of blur that can be modeled in a parametric form
such as Gaussian, motion and out-of-focus blur. Gradients are
computed against the parameters of the blurring PSF, which is
optimized in the direction of increasing kurtosis value till the
pristine image is estimated. The scheme is simple and efficient
and does not require any prior knowledge about the image and
the blurring process. The algorithms and gradients for a number
of blurs are derived, and the performance improvements are
corroborated through a set of simulations. Validation has been
carried out on various examples, both benchmark and real
images, and it shows that the scheme optimizes on the parameters
in a close vicinity of the original blurring functions. Full-
Reference and non-reference image quality measures were used
to estimate the eminence of the deblurring results. The proposed
method achieved marked improved results over the existing
methods.
Index Terms—Blind Image Deconvolution (BID), Gradient
Descent, Image Quality Measures, Kurtosis maximization.
I. INTRODUCTION
ince the realization of the blind image
deconvolution/deblurring (BID) problem in early 1960’s,
it still remains a challenging task to the image processing
community to find an efficient, reliable and most importantly
diversely applicable scheme. A scheme should not only
estimate the actual point spread function (PSF) but also restore
the image. The main challenge arises from little or no prior
information about the image or the blurring process as well as
the lack of optimal restoration filters that can eliminate the
blurring effect. The two common side effects of deblurring are
noise amplification and ringing artifacts arising in the
deblurred image due to unrealizable or imperfect restoration
filter. Furthermore, developing a scheme that can handle
The authors are with the School of Electrical and Electronic Engineering,
The University of Manchester, Manchester, M13 9PL, UK.
different types of blur especially for real images is yet to be
realized to a satisfactory extent.
A variety of BID schemes and restoration filters have been
proposed over the years. This collection of methods ranges
from the time domain to the frequency domain, simultaneous
or separate PSF estimation technique, parametric to non-
parametric. Examples include the Richardson-Lucy method,
Total Variation, Wiener filter, Maximum Likelihood method,
Minimum Entropy Deconvolution, Non-negative and Support
Size Recursive Inverse Filter, Simulated Annealing, and
Multi-channel Blind Deconvolution [1]-[8]. Even hardware
solutions have been formulated to revisit the BID problem by
capturing much better images in the first place [9], [10]. More
recent BID methods also include those based on kurtosis
minimization [11] and image priors [12], though both are not
fully automatic, as well as super resolution measures [13].
These methods do provide, to some extent, solutions to the
BID problem; however many are not satisfactory in terms of
stability, robustness, uniqueness and convergence. Mainly, the
robustness of these schemes remains or becomes questionable
when real-life blurred images are to be restored. This is
because unlike artificial deblurring, real deblurring suffers
heavily from noise and ringing effects. The blurred images,
esp. taken from low quality cameras, may have been degraded
by arbitrarily shaped PSFs that are complex and cannot be
easily modeled by usual parametric blurring models [14]-[18].
Handling these cases requires more effort since no prior
information about the blurring shape or size is known.
Nowadays, most of the BID schemes focus mainly on
deblurring images corrupted by motion blurring PSFs of
arbitrary shapes. Not only because it has common occurrences
in real life image acquisition but also because it is challenging.
This type of blur ensues mainly from camera shakes or
movements of objects or background in the focal range. The
PSFs of such blurs are usually of arbitrary shape and
sometimes even space invariant. Restoring such images
requires more effort, where complex procedures are applied to
approximate the blurring kernel as well as restore the image to
its pristine form while deterring the deblurring noise and
ringing artifacts. Restoring motion blurred images as
compared to Gaussian and out-of-focus blurred ones is less
problematic in restoration terms as the deblurring of the latter
two blurs produces more ringing artifacts and leaves
significant amount of residual blur.
The community and researchers over the years have strived
to come up with BID schemes that can effectively handle
Efficient Blind Image Deblurring with Gradient
Descent Based Kurtosis Maximization
Aftab Khan and Hujun Yin, Senior Member, IEEE
S
different types of blurs. Our motivation is to design a BID
scheme that is efficient in terms of restoration speed and
effective in terms of restoration quality. The proposed scheme
uses a parametric representation for the uniform Gaussian,
motion or out-of-focus blur. The parameters are optimized by
a steepest descent scheme using a restoration measure to find
the true blurring values. The proposed method provides a
unified base for BID framework.
The parametric model of blur helps simplify the search for
the blurring parameters where the use of a restoration measure
is optimized for desired result. In the quest for simple
deblurring processes, the proposed method relies on higher
order statistics based non-Gaussianity measures to lead in the
direction of increased quality of the deblurred image. The
earlier schemes developed by the authors relied on the use of
non-Gaussianity measures in either spatial or frequency
domain, namely spatial and spectral kurtosis [19], [20]. Apart
from this, re-blurring based image quality measure was also
used but its usage was very limited as it was highly sensitive
to restoration noise and artifacts
II. PARAMETRIC BLURRING MODELS
Let ),( nmf represent the original image without any form
of degradation, ),( nmh be the PSF and ),( nmg the output of
the image acquisition system. Mathematically, for a system of
stationary impulse response across the entire image (i.e. a
spatially invariant stationary PSF), the convolution is given by
[21], [22],
),(),(*),(),( nmvnmhnmfnmg (1)
where * denotes the convolution operator, ),( nmv represents
additive noise; and m and n are spatial coordinates.
The frequency domain representation, with spectral
coordinates i and j , obtained using the Fourier Transform is
given as
),(),(),(),( jiNjiHjiFjiG (2)
The blurring PSFs studied here are spatially invariant, that
is, they are independent of the spatial location of the image
and provide the same or uniform effect at each pixel location.
Modeling, restoration and estimation of the original image for
the uniform PSF is less cumbersome as compared to spatially
variant blurs. The classes of blur that can be easily modeled
include the atmospheric turbulence, motion and out-of-focus.
The PSF for atmospheric turbulence blur for long exposures
can be approximated by Gaussian blur and the degraded image
regarded as a Gaussian blurred image [23]. The Gaussian blur
is the result of filtering an image by a low pass filter estimated
by a Gaussian function. The 2-D Gaussian PSF is given by
2
22
2 2
)(
2
1),(
nmenmh (3)
where is the width of the blurring kernel.
Motion blurs commonly result from relative motions
between the scene and the recording device. They can be in
the form of translation, sudden change in scale, rotation or a
combination of these. The motion blur type dealt with here is
the case of translation. Denoting the length of motion by L ,
the angle by , and the pixel coordinates by i and j , the
motion blurring PSF is given by [23],
elsewhere 0
tan and 2
if 1
),;,(
22 n
mLnm
LLnmh (4)
When a camera captures a 2-D image of a 3-D scene,
certain points in the scene may not be focused. For a camera
with a circular aperture, the PSF of a point not focused looks
like a circular disk with a specific radius. The spatially
continuous out-of-focus blur of radius R is given by [23],
elsewhere 0
if 1
);,(
222
2Rnm
RCRnmh (5)
where C is a constant that must be chosen so that energy
conservation law is satisfied
III. PROPOSED GRADIENT BASED DEBLURRING SCHEME
The derivation of the gradient based BID algorithm scheme
starts with differentiation of the cost function with respect to
the parameter of the blurring function. In the proposed method
a kurtosis based cost function for the deblurred image is
employed. The parameter is update iteratively using the
steepest descent method,
)( 1 YK
ppp (6)
under the constraint,
)()(
1YKYK
pp
(7)
where is the blurring PSF parameter, such as kernel width,
length, or radius in case of Gaussian, motion or out-of-focus
blur respectively, p is the iteration number, )(YKp
is the
kurtosis of the deblurred image for the current parameter,
|)(| YKp
represents the gradient matrix for the absolute
kurtosis value, and is the convergence step size.
Since
))(sgn()()( YKYKYK (8)
Eq. (6) can be rewritten as
))(sgn()( 1 YKYK
pppp (9)
For , a fixed scalar value can be used, e.g. kp where k is
the iteration number and 10 p . For faster convergence,
more efficient computation schemes can also be employed.
Assume that Y is the deblurred image resulting from the
deconvolution of the blurred image G with the Wiener
restoration filter, given by
2
*
H
H
(10)
where is the noise to signal ratio. For simplicity of future
derivation we rewrite the equation as
HH
H
HHH
HH
H
H
H
H
3
2
22 ||
*
||
*
(11)
The kurtosis of the deblurred image Y is given by
3)2(
)4()(
2
Y
YYK
(12)
Let )0(Y represent the expected value or mean of Y .
M
Qi
N
QjY jiY
QNQM 1 1221 1
),(11
)0(
(13)
The gradient matrix is derived as follows.
2)2(
)4()()(
Y
YYKYK
(14)
2
1 12
2
21
1 12
4
21
1
1
)]0(),([11
)]0(),([11
)(M
Qi
N
QjY
M
Qi
N
QjY
jiYQNQM
jiYQNQM
YK
(15)
M
Qi
N
QjY
Y
Y
M
Qi
N
QjY
Y
jiYQNQM
jiYQNQM
YK
1 12213
1 12
3
212
1
1
)]0(),([11
)2(
)4(4
)]0(),([11
)2(
14
)(
(16)
where
))0(()())0(( YYYY
(17)
)0(Y
is equivalent to the mean value of Y
and
HH
HG
HH
HGY
3
2
3
2
)(
(18)
Through further manipulations, we get
H
HH
HH
HH
HGY
23
22
3
32)( (19)
The term H
needs to be calculated for specific types of
blur. The derivation for the Gaussian blurring PSF is as
follows.
In case of Gaussian blurring, the Optical Transfer Function
(OTF) for the parametric model is given by
2
22
2
22
1),(
f
ji
f
ejiH
(20)
with the f of the OTF being reciprocal to of the PSF. For
simplicity f is simply represented by .
Differentiating the OTF with respect to )( of the OTF,
2
22
2
22
1),(),(
ji
ejiHjiH
(21)
2
22
2
22
2
2
2
3 2
1
2
2),(
jiji
eejiH
2
22
2
22
2
5
222
3 22
2
jiji
eji
e
2
22
2
22
2
5
222
3 22
2
jiji
eji
e (22)
That is,
3
222),(),(
jijiHjiH
(23)
The derivations of the gradient matrix for the motion and
out-of-focus blur can be found in Appendix.
The parameter of the blurring PSF model is optimized in the
direction of increasing value of the absolute kurtosis till the
gradient shows a flat point or the difference of kurtosis
between successive iterations falls below a specified tolerance
value.
IV. EXPERIMENTAL RESULTS AND DISCUSSIONS
A. Experimental Setup
The proposed scheme has been tested on various images
degraded with different degrading functions. Experiments
were conducted on both artificial and real blurred images for
three different types of blur. Several image quality measures
were employed to evaluate the quality of deblurred images.
The full reference Image Quality Assessment (IQA) measures
include the Mean Structural SIMilarity (MSSIM) index [24]-
[26] and the Universal Quality Index (UQI) [27]. The
blind\no-reference based IQA measures are the
Blind/Reference-less Image Spatial QUality Evaluator
(BRISQUE) [28], [29] and the Natural Image Quality
Evaluator (NIQE) [30], [31]. A brief description of these
quality measures is given as follows.
Fig. 1. Test images used for deblurring of artificially blurred images.
The SSIM is an objective image quality metric that measures
the structural similarity between two images by comparison of
local patterns of pixel intensities that have been normalized for
luminance and contrast. Assuming two images f and g , the
SSIM is a function of the luminance ),( gfl , contrast ),( gfc
and the structural similarity ),( gfs of the images.
)),(),,(),,((),( gfsgfcgfltyxSSIM
(24)
1
12),(
22 C
Cgfl
gf
gf
(25)
2
22),(
22 C
Cgfc
gf
gf
(26)
3
3),(
C
Cgfs
gf
fg
(27)
where is the mean, is the standard deviation of the
image, C1, C2 and C3 are constants and t represents the SSIM
transfer function. SSIM is computed for each pixel location
and generally its mean value is used, named as the Mean
SSIM (MSSIM). SSIM has been successfully used previously
for denoising and classification [32], [33].
The UQI analyses the loss of correlation, luminance and
contrast distortion among the two images as the base for
quality perception. The UQI for two images f and g is
defined as [27],
])[(
4),(
2222gfgf
gffggfQ
(28)
or
2222
2.
2.),(
gf
gf
gf
gf
gf
fggfQ
(29)
The first term is the correlation coefficient of the two images
whereas the second and third terms are measures of mean
luminance and structural similarities.
The BRISQUE measure is a distortion measure based on
spatial domain natural scene statistics. It is also a non-
reference IQA model. Rather than computing distortion
specific features such as ringing, blur or blocking, it uses
scene statistics of locally normalized luminance coefficients to
quantify possible losses of ‘naturalness’ in the image due to
the presence of distortions, thereby leading to a holistic
measure of quality. The BRISQUE model uses a mapping
from feature space to quality scores using a regression module
that yields a measure of image quality. The features comprise
the statistic measures of a generalized Gaussian distribution
fitting of the Mean Subtracted Contrast Normalized (MSCN)
coefficients. The MSCN coefficients for the distorted image
g are obtained by subtracting the local mean alue g and
then dividing by the local contrast function g such that
Cnm
nmnmgnmg
g
g
),(
),(),(),(ˆ
(30)
where
Img-01 Img-02 Img-03 Img-04
Img-05 Img-06 Img-07 Img-08
Img-09 Img-10 Img-11 Img-12
Img-13 Img-14 Img-15 Img-16
U
Uk
V
Vllklkg nmgwnm ),(),( ,,
(31)
U
Uk
V
Vlglklkg nmnmgwnm 2
,, )),(),((),( (32)
and C is a constant. VVlUUkww lk ,.....,,,.....,|, is
a 2-D circular-symmetric Gaussian weighting function and
3VU was used for calculating the measure.
The fourth IQA measure, the none reference NIQE, is a
completely blind image quality analyzer that only makes use
of measurable deviations from statistical regularities observed
in natural images, without training on human-rated distorted
images and without any exposure to distorted images. The
same computation for image quality as in the BRISQUE is
used with the exception that NIQE uses natural scene statistics
features from a staple of natural images, while the BRISQUE
is trained on features obtained from both natural and distorted
images as well as human judgment of quality of the images.
Images from the Desktop Nexus image wallpaper database
[34] were used in the experiments in the artificial blurring
case. Real blurred natural images, captured by either ourselves
or others, were used in the case of real blind image deblurring.
1) Artificially blurred images
Examples of the Desktop Nexus images are shown in Fig. 1
Fig. 2 shows the restorations of their Gaussian-blurred images
using the proposed scheme. The estimated PSF parameters and
comparisons of different quality measures for the Gaussian
deblurring are given in Table 1.
(a) (c)
(b) (d)
Fig. 2. (a) and (b) Gaussian blurred images, (c) and (d) their respective
estimates using the proposed BID scheme.
Fig. 3 shows the out-of-focus blurred images and their
respective deblurred counterparts using the proposed scheme.
Table 2 summarizes the results for out-of-focus deblurring
along with the comparison of different image quality
measures. Fig.4 shows the deblurring result for motion blurred
images. The results for motion deblurring are summarized in
Table 3.
(a) (c)
(b) (d)
Fig. 3. (a) and (b) out-of-focus blurred images; (c) and (d) their respective estimates using the proposed BID scheme.
(a) (c)
(b) (d)
Fig. 4. (a) and (b) motion blurred images, (c) and (d) their respective estimates
using the proposed BID scheme.
(a) (e)(a) (e)
(d) (h)(d) (h)
(b) (f)(b) (f)
(d) (h)(d) (h)
(a) (e)(a) (e)
(c) (g)(c) (g)
TABLE 1. PSF PARAMETER ESTIMATION FOR GAUSSIAN BLURRED IMAGES AND QUANTITATIVE COMPARISON OF SSIM, UQI, BRISQUE AND NIQE
QUALITY MEASURE.
MSSIM UQI BRISQUE NIQE
Image Original
Sigma
Estimated
Sigma Blurred
Proposed
Scheme Blurred
Proposed
Scheme Blurred
Proposed
Scheme Blurred
Proposed
Scheme
Img-01 1.63 1.85 0.85 0.90 0.99 0.99 56.69 18.28 7.28 8.40
Img-02 4.98 4.76 0.73 0.81 0.95 0.97 77.15 70.69 10.84 7.97
Img-03 5.13 5.26 0.62 0.68 0.94 0.96 79.32 73.12 11.85 9.05
Img-04 3.88 4.05 0.78 0.86 0.97 0.98 72.87 55.30 10.64 7.65
Img-05 0.71 0.68 0.88 0.95 0.99 1.00 25.45 19.06 3.80 3.88
Img-06 0.94 1.21 0.97 0.88 1.00 1.00 52.71 38.15 7.00 7.21
Img-07 2.07 2.18 0.52 0.74 0.94 0.97 61.24 40.69 7.40 7.76
Img-08 2.63 2.50 0.69 0.80 0.95 0.97 67.06 34.43 9.10 6.99
Img-09 3.45 3.32 0.67 0.76 0.97 0.98 67.54 54.20 10.28 6.86
Img-10 2.44 2.24 0.75 0.86 0.97 0.98 64.76 36.55 8.56 6.45
Img-11 1.64 1.82 0.75 0.88 0.98 0.99 54.33 26.65 6.56 6.59
Img-12 0.98 0.99 0.93 0.94 0.99 1.00 47.88 11.99 5.48 3.85
Img-13 1.38 1.53 0.89 0.96 0.99 0.99 45.66 24.88 6.66 5.94
Img-14 3.13 3.26 0.62 0.80 0.93 0.97 72.01 49.99 9.79 6.85
Img-15 4.18 4.13 0.79 0.81 0.97 0.98 80.98 58.73 10.61 8.11
Img-16 5.47 5.39 0.45 0.55 0.93 0.95 82.44 70.65 10.77 9.02
AVERAGE 0.75 0.82 0.97 0.98 63.00 42.71 8.54 7.04
TABLE 2. PSF PARAMETER ESTIMATION FOR OUT-OF-FOCUS BLURRED IMAGES AND QUANTITATIVE COMPARISON OF SSIM, UQI, BRISQUE AND NIQE
QUALITY MEASURE.
MSSIM UQI BRISQUE NIQE
Image Original
Radius
Estimated
Radius Blurred
Proposed
Scheme Blurred
Proposed
Scheme Blurred
Proposed
Scheme Blurred
Proposed
Scheme
Img-01 7.00 6.84 0.70 0.81 0.96 0.98 69.52 27.22 10.24 4.32
Img-02 8.85 9.20 0.70 0.76 0.94 0.97 76.69 44.17 10.42 5.34
Img-03 10.00 9.40 0.59 0.70 0.93 0.95 77.68 39.09 11.99 4.97
Img-04 11.50 11.47 0.69 0.84 0.93 0.97 76.05 32.21 12.35 5.55
Img-05 1.00 0.74 0.95 0.97 1.00 1.00 22.08 12.82 3.29 2.70
Img-06 2.50 2.46 0.97 0.93 1.00 1.00 60.49 14.96 8.15 5.54
Img-07 4.00 3.91 0.48 0.83 0.93 0.98 57.52 10.78 6.92 3.77
Img-08 5.50 4.72 0.64 0.59 0.94 0.94 68.54 29.01 9.56 5.19
Img-09 13.00 12.58 0.63 0.68 0.96 0.97 67.17 39.76 10.84 4.61
Img-10 14.50 14.14 0.63 0.61 0.93 0.95 73.93 45.10 12.12 5.31
Img-11 16.00 15.27 0.51 0.58 0.93 0.95 76.35 43.96 11.36 5.20
Img-12 17.50 18.74 0.73 0.79 0.96 0.98 77.13 37.51 11.19 4.75
Img-13 19.00 18.82 0.40 0.70 0.86 0.95 80.40 45.59 11.93 5.96
Img-14 20.50 20.03 0.42 0.65 0.82 0.93 83.63 46.97 11.43 5.83
Img-15 22.00 21.58 0.68 0.64 0.93 0.93 86.56 56.11 12.76 7.12
Img-16 23.50 22.89 0.38 0.47 0.89 0.91 84.80 57.47 11.57 7.85
AVERAGE 0.63 0.72 0.93 0.96 71.16 36.42 10.38 5.25
NIQ
E
Pro
po
sed
Sch
eme
5.9
89
3.9
13
4.2
21
4.3
79
4.3
11
4.4
67
5.6
26
6.2
88
4.7
70
4.8
44
3.8
44
3.9
92
4.4
49
4.3
12
5.4
62
5.6
15
4.7
80
Qi
Shan
5.6
63
5.1
46
5.2
57
5.6
63
5.9
16
6.3
50
7.6
87
8.6
94
5.4
84
4.8
40
6.0
72
5.3
35
6.0
58
5.2
32
6.4
92
5.9
20
5.9
88
Blu
rred
6.3
64
7.0
59
6.4
77
8.9
86
8.3
93
9.3
69
7.1
72
7.2
78
7.1
51
7.2
65
7.9
36
9.0
74
9.2
44
8.0
99
8.3
97
6.0
70
7.7
71
BR
ISQ
UE
Pro
po
sed
Sch
eme
24.0
62
36.5
37
42.7
20
38.1
49
44.8
11
21.6
21
24.5
33
35.9
00
26.9
21
24.4
12
53.3
11
26.1
64
46.4
89
45.8
33
55.4
49
45.2
94
37.0
13
Qi
Shan
34.7
59
64.9
61
56.4
72
65.9
50
52.7
07
67.1
86
39.9
75
39.2
33
27.3
67
26.1
57
13.1
88
27.5
14
26.6
08
25.0
11
58.3
30
49.0
92
42.1
57
Blu
rred
34.7
59
64.9
61
56.4
72
65.9
50
52.7
07
67.1
86
39.9
75
39.2
33
27.3
67
26.1
57
13.1
88
27.5
14
26.6
08
25.0
11
58.3
30
49.0
92
42.1
57
UQ
I
Pro
po
sed
Sch
eme
0.9
92
0.9
92
0.9
87
0.9
93
0.9
87
0.9
96
0.9
72
0.9
78
0.9
80
0.9
74
0.9
78
0.9
87
0.9
76
0.9
67
0.9
62
0.9
63
0.9
80
Qi
Shan
0.9
84
0.9
86
0.9
77
0.9
79
0.9
74
0.9
90
0.9
26
0.9
77
0.9
69
0.9
66
0.9
42
0.9
74
0.9
62
0.9
45
0.9
56
0.9
32
0.9
65
Blu
rred
0.9
90
0.9
85
0.9
75
0.9
82
0.9
83
0.9
91
0.9
28
0.9
65
0.9
72
0.9
58
0.9
42
0.9
75
0.9
19
0.8
94
0.9
69
0.9
34
0.9
60
MS
SIM
Pro
po
sed
Sch
eme
0.9
44
0.9
26
0.8
93
0.9
26
0.8
28
0.9
21
0.8
10
0.8
58
0.8
28
0.8
26
0.8
04
0.8
27
0.8
06
0.7
71
0.5
87
0.6
74
0.8
27
Qi
Shan
0.8
67
0.8
94
0.8
10
0.8
18
0.7
30
0.8
90
0.6
16
0.8
60
0.7
60
0.7
94
0.5
30
0.7
69
0.7
18
0.6
73
0.6
79
0.5
51
0.7
47
Blu
rred
0.9
20
0.8
75
0.7
57
0.8
59
0.7
40
0.9
24
0.5
36
0.7
75
0.7
25
0.7
26
0.5
72
0.7
78
0.5
13
0.5
51
0.7
55
0.5
04
0.7
19
Est
imat
ed
Len
gth
8
8
9
10
12
12
13
13
14
15
17
19
19
21
21
24
Ori
gin
al
Len
gth
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Imag
e
Img
-01
Img
-02
Img
-03
Img
-04
Img
-05
Img
-06
Img
-07
Img
-08
Img
-09
Img
-10
Img
-11
Img
-12
Img
-13
Img
-14
Img
-15
Img
-16
AV
ER
AG
E
TA
BL
E 3
. PS
F P
AR
AM
ET
ER
ES
TIM
AT
ION
FO
R M
OT
ION
BL
UR
RE
D I
MA
GE
S A
ND
QU
AN
TIT
AT
IVE
CO
MP
AR
ISO
N O
F S
SIM
AN
D U
QI
QU
AL
ITY
ME
AS
UR
E
2) Real blurred images
The proposed deblurring scheme has also been performed
on real images with motion blur, a typical problem facing
amateur photographers. The deblurred images using the
proposed scheme have been compared with the estimated
images using the blind deconvolution scheme by Shan et al. in
[35].
(a) (b)
(c) (d) (e)
Fig. 5. (a) Real motion blurred LABEL image of 720655. (b) Deblurred
using the proposed BID scheme. (c) Deblurred using Shan et al’s BID scheme
[35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF using Shan et al’s BID scheme.
In the first case, an image was captured by a low quality
camera. The blurred image, depicted in Fig. 5 (a), shows an
approximate vertical motion blur with the small digits at the
bottom of the image becoming unreadable. The image also
contains noise and uneven illumination due to poor lighting –
one of the most difficult cases in deblurring. Fig. 5(b) shows
the deblurred image estimated using the proposed scheme with
the estimated blurring kernel shown in Fig. 5(d). Fig. 5(c)
shows the deblurred image and the corresponding PSF
(rescaled) in Fig. 5(e) estimated using the blind deconvolution
scheme of Shan et al. [35]. The image in Fig. 5(c) seems to
have recovered well, however the digits are still unreadable
since the PSF estimated using this scheme does not completely
follow a uniform motion blur. Using the proposed scheme, a
uniform motion PSF of length 21 pixels and angle zero degree
was estimated. In Fig. 5(b), the digits in the deblurred image
are much clear and easily readable. An edgetaper function was
used to reduce the ringing effect caused by using the discrete
Fourier transform.
Figs. 6 and 7 show the deblurring of motion blurred images
resulting from object movement and camera handshake
respectively. The motion blur is almost linear at a certain
angle.
(a) (b)
(c) (d) (e)
Fig. 6. (a) Real motion blurred MATLAB_BOOK image of 469323. (b)
Deblurred using the proposed BID scheme. (c) Deblurred using Shan et al’s
BID scheme [35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF using Shan et al’s BID scheme.
Another motion blurred image taken by an ordinary digital
camera and its deblurred versions are shown in Fig. 8. Fig.
8(b) shows the estimate using the proposed scheme with the
corresponding estimated PSF in Fig. 8(d). Fig. 8(c) shows the
deblurred image and the corresponding PSF (rescaled) in Fig.
8(e) estimated using the blind deconvolution scheme of Shan
et al [35]. The image seems to have recovered well and the
main and sub title text of the book is more readable in the
proposed scheme.
Table 4 shows the BRISQUE and NIQE results for the
deblurred images.
(a) (b)
(c) (d) (e)
Fig. 7. (a) Real motion blurred BUILDINGS image of 459344. (b) Deblurred using the proposed BID scheme. (c) Deblurred using Shan et al’s BID scheme
[35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF
using the Shan et al’s BID scheme.
(a) (b)
(c) (d)
(e) (f)
(a) (b)
(c) (d)
(e) (f)
35 -3
8 88
(a) (b)
(c) (d) (e)
Fig. 8. (a) Real motion blurred DIP_BOOK image of 364602. (b) Deblurred
using the proposed BID scheme. (c) Deblurred using Shan et al’s BID scheme [35]. (d) Estimated PSF using the proposed BID scheme. (e) Estimated PSF
using the Shan et al’s BID scheme.
V. CONCLUSIONS
An improved BID scheme based on kurtosis and steepest
descent algorithm is proposed that can estimate uniform blur
of a parametric PSF form. A steepest descent optimization
scheme was employed to optimize the blurring parameters in
the direction of maximum absolute kurtosis of the deblurred
image. The proposed method operates on single image and is
simple, efficient and easy to implement. The method is
applicable to various types of blurring such as Gaussian,
motion and out-of-focus. Experiments on both artificial and
real blurred images have shown the capability of the scheme.
The scheme can be further extended to recover images blurred
by arbitrary shape motion/camera handshake PSFs.
APPENDIX
A. Motion Blur
The linear motion modeled here is the result of translation
in the horizontal direction. In order to simplify the search for
true blur parameters using the steepest descent scheme, we
rotate the image until the blur can be assumed horizontal. This
simplifies the case from two parameters i.e. length )(L and
angle )( of blur .Now only the length of blur )(L needs to
be optimized. The OTF for motion blur according to [36] is
given by
j
i
iLe
LjiH Lki
)sin(
))1(sin
1
1),(
(A.1)
Differentiating the OTF with respect to the length parameter
)(L of the OTF,
)sin(
))1(sin
1
1),(
),(
i
iLe
LLjiH
L
jiH
Lki
(A.2)
(a) (b)
(c) (d)
(e) (f)
TABLE 4 PSF PARAMETER ESTIMATION FOR MOTION BLURRED IMAGES AND QUANTITATIVE COMPARISON OF BRISQUE AND NIQE QUALITY
MEASURE.
BRISQUE NIQE
Image Original Manual Shan et al
[32] Our Original Manual Qi Shan Our
LABEL 56.080 27.350 52.000 54.070 7.110 6.230 8.100 7.680
DIP_BOOK 29.770 17.780 40.210 45.680 5.780 4.530 6.310 6.510
MATLAB_BOOK 5.480 15.490 28.840 48.550 5.760 5.530 4.070 6.310
BUILDINGS 39.850 25.960 41.170 39.580 7.010 5.440 6.410 6.850
)sin(
))1(sin
1
1
1
1
)sin(
))1(sin),(
i
iLe
LL
LLi
iLejiH
L
Lki
Lki
(A.3)
))1(sin1
1
)sin(
1
1
1)1(
)sin(
))1(sin),(
2
iLeLLi
Li
iLejiH
L
Lki
Lki
(A.4)
)sin(1
))1(cos))1(sin
)sin(
))1(sin
1
1),(
2
iL
iLieekiiL
i
iLe
LjiH
L
LkiLki
Lki
(A.5)
),())1(cot)(),(),(
1
1
),(
jiHiLijiiHkjiHL
jiHL
(A.6)
))1(cot)(
1
1),(),( iLiik
LjiHjiH
L (A.7)
B. Out-of-focus Blur
The OTF for out-of-focus blur according to [36] is given by
2221 ,)(
),( jipRp
RpJjiH
(A.8)
where 1J is the first order Bessel function. The Bessel
function can be approximated by an exponential for
sufficiently large data [37].
Rp
ejiH
Rp
2),(
(A.9)
Differentiating the OTF with respect to the radius parameter
)(R of the OTF,
RpRp
Rp
eRRpRpR
e
Rp
e
RjiH
R
2
1
2
1
2),(
(A.10)
RpRp
e
RpepjiH
R
RpRp
22
2/12),(
(A.11)
RppRp
ejiH
R
Rp
2),(
(A.12)
RppjiHjiH
R
),(),(
(A.13)
REFERENCES
[1] R. L. Lagendijk, A. M. Tekalp, and J. Biemond, “Maximum-
likelihood image and blur identification - a unifying approach,”
Optical engineering, vol. 29, no. 5, pp. 422-435, 1990.
[2] D. Kundur, and D. Hatzinakos, “A novel blind deconvolution
scheme for image restoration using recursive filtering,” IEEE
Transactions on Signal Processing, vol. 46, no. 2, pp. 375-390,
1998.
[3] T. F. Chan, and C. K. Wong, “Total variation blind
deconvolution,” IEEE Transactions on Image Processing, vol.
7, no. 3, pp. 370-375, 1998.
[4] W. H. Richardson, “Bayesian-based iterative method of image
restoration,” Journal of the Optical Society of America, vol. 62,
no. 1, pp. 55-59, 1972.
[5] R. A. Wiggins, “Minimum entropy deconvolution,”
Geoexploration, vol. 16, no. 1-2, pp. 21-35, 1978.
[6] M. R. Banham, and A. K. Katsaggelos, “Digital image
restoration,” IEEE Signal Processing Magazine, vol. 14, no. 2,
pp. 24-41, 1997.
[7] D. Kundur, and D. Hatzinakos, “Blind image deconvolution,”
IEEE Signal Processing Magazine, vol. 13, no. 3, pp. 43-64,
1996.
[8] R. L. Lagendijk, J. Biemond, and D. E. Boekee, “Regularized
iterative image-restoration with ringing reduction,” IEEE
Transactions on Acoustics Speech and Signal Processing, vol.
36, no. 12, pp. 1874-1888, 1988.
[9] A. Agrawal, and Y. Xu, “Coded exposure deblurring:
Optimized codes for PSF estimation and invertibility,” Proc.
IEEE Conference on Computer Vision and Pattern Recognition,
New York, 2009, pp. 2066-2073.
[10] C. Y. Zhou, S. Lin, and S. K. Nayar, “Coded aperture pairs for
depth from defocus and defocus deblurring,” International
Journal of Computer Vision, vol. 93, no. 1, pp. 53-72, 2011.
[11] D. Li, and S. Simske, “Atmospheric turbulence degraded-image
restoration by kurtosis minimization,” IEEE Geoscience and
Remote Sensing Letters, vol. 6, no. 2, pp. 244-247, 2009.
[12] M. S. C. Almeida and L. B. Almeida, “Blind and semi-blind
deblurring of natural images” IEEE Transactions on Image
Processing, vol. 22, no. 6, pp. 2101-2114, 2013.
[13] E. Faramarzi, D. Rajan, and M. P. Christensen, “Unified blind
method for multi-image super-resolution and single/multi-
image blur deconvolution,” IEEE Transactions on Image
Processing, vol. 19, no. 1, pp. 36-52, 2009
[14] S. Cho, J. Wang, and S. Lee, “Handling outliers in non-blind
image deconvolution,” Proc. International Conference on
Computer Vision, New York, 2011, pp. 495-502.
[15] O. Whyte, J. Sivic, and A. Zisserman, “Deblurring shaken and
partially saturated images,” Proc. International Conference on
Computer Vision Workshops, New York, 2011.
[16] M. Hirsch, C. J. Schuler, S. Harmeling, and B. Scholkopf, “Fast
removal of non-uniform camera shake,” Proc. International
Conference on Computer Vision, New York, 2011, pp. 463-
470.
[17] X. Yuquan, W. Lu, H. Xiyuan, and P. Silong, "Single-image
blind deblurring for non-uniform camera-shake blur”, Proc.
11th Asian conference on Computer Vision Daejeon,
Korea,2013, pp. 336-348.
[18] A. Gupta, N. Joshi, C. L. Zitnick, M. Cohen, and B. Curless,
“Single image deblurring using motion density functions,”
Proc. 11th European Conference on Computer Vision,
Heraklion, Crete, Greece, 2010, pp. 171-184.
[19] H. Yin, and I. Hussain, “Independent component analysis and
non-gaussianity for blind image deconvolution and deblurring,”
Integrated Computer-Aided Engineering, vol. 15, no. 3, pp.
219-228, 2008.
[20] A. Khan, and H. Yin, “Efficient blind image deconvolution
using spectral non-gaussianity,” Integrated Computer-Aided
Engineering, vol. 19, no. 4, pp. 331-340, 2012.
[21] A. V. Oppenheim, R. W. Schafer, and T. G. Stockham,
“Nonlinear filtering of multiplied and convolved signals,”
Proceedings of the IEEE, vol. 56, no. 8, pp. 1264-1285, 1968.
[22] M. M. Sondhi, “Image restoration - removal of spatially
invariant degradations,” Proceedings of the IEEE, vol. 60, no.
7, pp. 842-853, 1972.
[23] R. L. Lagendijk, and J. Biemond, "Basic methods for image
restoration and identification," The essential guide to image
processing, pp. 323-348: San Diego, USA, Academic Press,
2009.
[24] D. Brunet, E. R. Vrscay, and Z. Wang, “On the mathematical
properties of the structural similarity index,” IEEE
Transactions on Image Processing, vol. 21, no. 4, pp. 1488-
1499, 2012.
[25] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli,
“Image quality assessment: From error visibility to structural
similarity,” IEEE Transactions on Image Processing, vol. 13,
no. 4, pp. 600-612, 2004.
[26] Z. Wang, and Q. Li, “Information content weighting for
perceptual image quality assessment,” IEEE Transactions on
Image Processing, vol. 20, no. 5, pp. 1185-1198, 2011.
[27] Z. Wang, and A. C. Bovik, “A universal image quality index,”
IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, Mar,
2002.
[28] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference
image quality assessment in the spatial domain,” IEEE
Transactions on Image Processing, vol. 21, no. 12, pp. 4695-
4708, 2012.
[29] A. Mittal, A. K. Moorthy, and A. C. Bovik. "BRISQUE
software release, "http://live.ece.utexas.edu/research/
quality/BRISQUE_release.zip.
[30] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a
"Completely blind" Image quality analyzer,” IEEE Signal
Processing Letters, vol. 20, no. 3, pp. 209-212, 2013.
[31] A. Mittal, A. K. Moorthy, and A. C. Bovik. "NIQE software
release," http://live.ece.utexas.edu/research/quality/niqe.zip.
[32] Y. Gao, A. Rehman, and Z. Wang, “CW-SSIM based image
classification,” Proc. 18th IEEE International Conference on
Image Processing, New York, 2011, pp. 1249-1252.
[33] A. Rehman, and Z. Wang, “SSIM-based non-local means
image denoising,” Proc. 18th IEEE International Conference on
Image Processing, New York, 2011, pp. 217-220.
[34] "Desktop nexus wallpaper image database,"
http://www.desktopnexus.com/.
[35] Q. Shan, J. Jia, and A. Agarwala, “High-quality motion
deblurring from a single image,” ACM Transactions on
Graphics, vol. 27, no. 3, pp. 10, 2008.
[36] J. Biemond, R. L. Lagendijk, and R. M. Mersereau, “Iterative
methods for image deblurring,” Proceedings of the IEEE, vol.
78, no. 5, pp. 856-883, 1990.
[37] W. O. Frank, W. L. Daniel, F. B. Ronald, and W. C. Charles,
NIST handbook of mathematical functions: Cambridge
University Press, 2010.