Image Restoration 1) Image restoration concerns the removal or reduction
of degradations which have occurred during the
acquisition of the image.
2) Such degradations may include noise, which are errors
in the pixel values, or optical effects such as out of
focus blurring, or blurring due to camera motion.
3) Image restoration is to restore a degraded image back3) Image restoration is to restore a degraded image back
to the original image while image enhancement is to
manipulate the image so that it is suitable for a specific
application.
4) Restoration can be performed successfully;
a) Using neighborhood operations
b) Using frequency domain processes1
DKVishwakarma
A model of Image
Degradation/Restoration Process
Image Degraded Image
22
Degraded image can be expressed by;
),(),(),(),( yxyxhyxfyxg η+∗=
where h(x,y) is a system that causes image distortion , it is also called as point spread
function (psf) and η(x,y) is noise (Random Error).DKVishwakarma
•We can perform some operation frequency domain it
is given by;
Optical transfer function (OTF)
),(),().,(),( vuNvuFvuHvuG +=
),( vuH
• If we knew the value of H, & N then we can recover
the F by equation;
3
),(/)],(),([),( vuHvuNvuGvuF +=
DKVishwakarma
Some Important Noise Models
• Noise cannot be predicted but can be approximately
described in statistical way using the probability
density function (PDF).
• Gaussian noise: The probability density function of
Gaussian Noise is given by;Gaussian Noise is given by;
• 70% of its value lies in the range of and
about 95% lies in the range of .4
22 2/)(
2
1)( σµ
πσ−−= zezp
)](),[( σµσµ +−
)]2(),2[( σµσµ +−
DKVishwakarma
5DKVishwakarma
Rayleigh Noise:
• The PDF of Rayleigh Noise is given by;
• Where the mean and variance is given by
<
≥−=−−
azfor 0
for )(2
)(/)( 2
azeazbzp
baz
6
4/ba πµ +=4
)4(2 πσ
−=b
DKVishwakarma
Erlang (Gamma) noise:
• The pdf of Erlang Noise is given by;
• Where a>0, b is a positive integer and the mean and
variance are given by;
<
≥−−=
−−
0zfor 0
0for )()!1()(
1
zeazb
za
zpaz
bb
variance are given by;
7
a
b=µ
2
2
a
b=σ
DKVishwakarma
Exponential Noise:
• The PDF of exponential noise is given by;
• Where a>0. The mean and Variance is given by;
<
≥=
0z 0
0for ae)(
-az zzp
• This PDF is special case of Erlang PDF with b=1.
8
a
1=µ
2
2 1
a=σ
DKVishwakarma
Uniform Noise:
• The PDF of uniform noise is given by
≤≤=
otherwise 0
afor a-b
1
)(bz
zp
12
)( 22 ab −=σ
2
ba +=µ
9DKVishwakarma
Impulse Noise (Salt and pepper Noise):
• The PDF of (bipolar) impulse noise is given by;
• If b>a, intensity b will appear as light dot in the
=
=
=
otherwise 0
for
for
)( bzP
azP
zp b
a
image. If Pa or Pb is zero, the impulse noise is called
unipolar.
10DKVishwakarma
Image Degradation with Additive Noise:
Original image
Degraded images
),(),(),( yxyxfyxg η+=
11
Histogram
DKVishwakarma
Original image
Degraded images
Image Degradation with Additive Noise (cont.)
),(),(),( yxyxfyxg η+=
12
Histogram
DKVishwakarma
Periodic Noise:
• If the image signal is subject to a periodic, rather
than a random disturbance, we might obtain an
image corrupted by periodic noise. The effect is of
bars over the image.
• This noise is generated from electrical or
electromechanical interference during imageelectromechanical interference during image
acquisition.
13
The
twins
image
corrupt
ed by
periodic
noise
In
freque
ncy
domain
DKVishwakarma
MATLAB CODE ‘salt & pepper’
• ‘salt & pepper’ noise
• Also called as impulse
noise, shot noise, or
binary noise.
x=imread('cameraman.jpg');
y=imnoise(x, 'salt & pepper'
,0.2); % 20% pixels get
corrupted.
subplot(2,1,1),imshow(x);
subplot(2,1,2),imshow(y);
14DKVishwakarma
Gaussian Noise
• Gaussian noise is an
idealized form of white
noise, which is caused by
random fluctuations in the
signal. We can observe
white noise by watching awhite noise by watching a
television which is slightly
mistuned to a particular
channel.
• y=imnoise(x, ‘Gaussian' , μ,
σ2);
15DKVishwakarma
Speckle noise
• Gaussian noise can bemodelled by random valuesadded to an image; specklenoise (or more simply justspeckle) can be modelledby random valuesmultiplied by pixel values,hence it is also calledmultiplied by pixel values,hence it is also calledmultiplicative noise.
• Speckle noise is a majorproblem in some radarapplications.
• y=imnoise(x,'speckle’ V);
• Zero mean and V(Variance)is by default 0.04
16DKVishwakarma
Periodic Noise
• t=imread('cameraman.jpg')
• s=size(t);
• [x,y]=meshgrid(1:s(1),1:s(2));
• p=sin(x/3+y/5)+1;
• z=(im2double(t)+p/2)/2;
• subplot(2,1,1),imshow(t);• subplot(2,1,1),imshow(t);
• subplot(2,1,2),Imshow(z);
17DKVishwakarma
Recovery of corrupted noise
• ‘salt and pepper’ noise, ‘Gaussian’ noise and
‘speckle’ noise can all be cleaned by using spatial
filtering techniques.
• ‘periodic’ noise, however, requires the use of
frequency domain filtering.frequency domain filtering.
• This is because whereas the other forms of noise can
be modelled as local degradations, ‘periodic’ noise is
a global effect.
18DKVishwakarma
Low pass filtering [Mean]
• Low pass filtering can be applied in spatial domain forremoval of noise;
• Average filter : It can be obtained by using fspecialcommand in MATLAB of any size(mxn).
>>fspecial(‘average’, [7 ,7]);
• Arithmetic mean filter(Moving Average filter):
• It removes local variation in the image and reduce theeffect of blurring.
19
∑∈
=xySts
tsgmn
yxf),(
),(1
),(ˆ
DKVishwakarma
• Geometric mean filter:
• Each restored pixel is product of the pixels in sub image
window, raised to the power 1/mn.
• This filter loses less details as compared to mean filter.
mn
Sts xy
tsgyxf
1
),(
),(),(ˆ
= ∏
∈
• This filter loses less details as compared to mean filter.
• Harmonic filter:
20
∑∈
=
xySts tsg
mnyxf
),( ),(
1),(ˆ
Works well for salt noise but fails for pepper noiseDKVishwakarma
Geometric Mean Filter: Example
Original
image
Image
corrupted
by AWGN
21
Image
obtained
using a 3x3
geometric
mean filter
Image
obtained
using a 3x3
arithmetic
mean filter
AWGN: Additive White Gaussian NoiseDKVishwakarma
Contraharmonic mean filter:
• Contraharmonic filter restored the image based on the
expression;
• Positive Q is suitable for eliminating pepper noise.
∑
∑
∈
∈
+
=
xy
xy
Sts
Q
Sts
Q
tsg
tsg
yxf
),(
),(
1
),(
),(
),(ˆQ = the filter order
• Positive Q is suitable for eliminating pepper noise.
• Negative Q is suitable for eliminating salt noise.
• For Q = 0, the filter reduces to an arithmetic mean filter.
• For Q = -1, the filter reduces to a harmonic mean filter.
22DKVishwakarma
Contraharmonic Filters: Example
Image
corrupted
by pepper
noise with
prob. = 0.1
Image
corrupted
by salt
noise with
prob. = 0.1
23
Image
obtained
using a 3x3
contra-
harmonic
mean filter
With Q = 1.5
Image
obtained
using a 3x3
contra-
harmonic
mean filter
With Q=-1.5
DKVishwakarma
Order-Statistic Filters:
subimage
Original image
Statistic parameters
Mean, Median, Mode,
Min, Max, Etc.
24
Moving
window
Output imageDKVishwakarma
Order-Statistics Filters:[contd..]
Median filter
{ }),(median),(ˆ),(
tsgyxfxySts ∈
=
Max filter
{ }),(max),(ˆ),(
tsgyxfxySts ∈
=Useful for finding brightest point in
the image.
Reduce “dark” noise (pepper noise)
Provide excellent noise Filtering with
considerable less Blurring than linear
smoothing Filter of same size. Effective
for both unipolar and bipolar impulse
noise.
25
),( xySts ∈
Min filter
{ }),(min),(ˆ),(
tsgyxfxySts ∈
=
Midpoint filter
{ } { }
+=
∈∈),(min),(max
2
1),(ˆ
),(),(
tsgtsgyxfxyxy StsSts
Reduce “dark” noise (pepper noise)
Useful for finding the darkest point
in the image.
Reduce “bright” noise (salt noise)
Best for removing
randomly distributed
Gaussian noise.
DKVishwakarma
Median Filter : How it works
A median filter is good for removing impulse, isolated noise
Salt noisePepper noise
Median
Median filtering seems almost tailor-made for removal of ‘salt and pepper noise’.
Median is a middle value of set when they are sorted. If the numbers are even then
mean of middle two values will be median.
Degraded image
Salt noise
Pepper noise
Moving
window
Sorted
array
Filter output
Normally, impulse noise has high magnitude and is isolated. When we sort pixels in
the moving window, noise pixels are usually at the ends of the array. it’s rare that
the noise pixel will be a median value.
DKVishwakarma
Median Filter : Example
Image
corrupted
by salt-and-
pepper
noise with
pa=pb= 0.1
1 2
27Images obtained using a 3x3 median filter
43
DKVishwakarma
Max and Min Filters: Example
Image
corrupted
by pepper
noise with
prob. = 0.1
Image
corrupted
by salt
noise with
prob. = 0.1
28
Image
obtained
using a 3x3
max filter
Image
obtained
using a 3x3
min filter
DKVishwakarma
Median Filter [Example]
• noisy values will end up at the top or bottom of thesorted list. Thus the median will in general replace anoisy value with one closer to its surroundings.
• In Matlab, median filtering is implemented by the medfilt2 function:
>>x=imread(‘cameraman’);
>>y=imnoise(x, ‘salt & pepper’, 0.02);
>> z=medfilt2(y,[5 5]);
29DKVishwakarma
Median filtering [Types of Median filter] cont…
• Median filtering is a special case of a more general
process called rank-order filtering. Rather than take
the median of a set, we order the set and take the nth
value, for some predetermined value of n.
3x3 matrix then ‘Rank order filter’ with 5.1 2 3
4 5 6 3x3 matrix then ‘Rank order filter’ with 5.
5x5 matrix then ‘Rank order filter’
with 13.
30
7 8 9
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
DKVishwakarma
Median filtering [Types of Median filter] cont…
Median would be 3rd of these values.
Median would be 5th of these
values.
0 1 0
2 3 4
0 5 0
0 0 1 0 0
0 0 1 0 0
1 1 1 1 1 values.
In MATLAB it can be implemented using ordfilt2
command. ordfilt2(y,3,[0 1 0; 1 1 1;0 1 0])
31
1 1 1 1 1
0 0 1 0 0
0 0 1 0 0
DKVishwakarma
Question
32DKVishwakarma
Alpha-trimmed Mean Filter
∑∈−
=xySts
r tsgdmn
yxf),(
),(1
),(ˆ
Where gr(s,t) represent the remaining mn-d pixels after removing the d/2
highest and d/2 lowest values of g(s,t).
Formula:
33
highest and d/2 lowest values of g(s,t).
This filter is useful in situations involving multiple types of noise such as a
combination of salt-and-pepper and Gaussian noise.
DKVishwakarma
Alpha-trimmed Mean Filter: Example
Image
corrupted
by additive
uniform
noise
Image
additionally
corrupted
by additive
salt-and-
pepper
noise
1 2
34
Image
obtained
using a 5x5
arithmetic
mean filter
2 Image
obtained
using a 5x5
geometric
mean filter
2
DKVishwakarma
Alpha-trimmed Mean Filter: Example (cont.)
Image
obtained
using a 5x5
arithmetic
mean filter
Image
obtained
using a 5x5
geometric
mean filter
35
Image
obtained
using a 5x5
median filter
Image
obtained
using a 5x5
alpha-
trimmed
mean filter
with d = 5
DKVishwakarma
Adaptive Filter:
- Filter behavior depends on statistical characteristics of
local areas inside mxn moving window.
- More complex but superior performance compared with
“fixed” filters
General concept:
- Adaptive filters are a class of filters which change their
36
- Adaptive filters are a class of filters which change their
characteristics according to the values of the grayscales
under the mask; they may act more like median filters, or
more like average filters, depending on their position
within the image.
- These filters can be used to clean ‘Gaussian noise’ by
using local statistical properties of the values under the
mask.DKVishwakarma
Adaptive Filter[CONTD..]
Statistical characteristics:
Local mean:
2
ησ
Noise variance:
37
∑∈
=xySts
L tsgmn
m),(
),(1
Local mean:
Local variance:
∑∈
−=xySts
LL mtsgmn ),(
22 )),((1
σ
DKVishwakarma
Adaptive, Local Noise Reduction Filter:
Purpose: want to preserve edges
1. If σσσσηηηη2 is zero, ���� No noise
the filter should return g(x,y) because g(x,y) = f(x,y)
2. If σσσσL2 is high relative to σσσσηηηη2, ���� Edges (should be preserved),
the filter should return the value close to g(x,y)
Concept:
38
3. If σσσσL2 = σσσσηηηη2, ���� Areas inside objects
the filter should return the arithmetic mean value mL
( )LL
myxgyxgyxf −−= ),(),(),(ˆ2
2
σση
Formula:
The filter returns a value close to either g(x,y) or mL depending upon the local
variance is high or low.DKVishwakarma
Adaptive Noise Reduction Filter: Example
Image
corrupted
by additive
Gaussian
noise with
zero mean
and σσσσ2=1000
Image
obtained
using a 7x7
arithmetic
mean filter
Image
39
Image
obtained
using a 7x7
geometric
mean filter
Image
obtained
using a 7x7
adaptive
noise
reduction
filter
DKVishwakarma
MATLAB Implementation:
• Adaptive filter can be implemented in MATLAB by
using wiener2 function.
• x=imread('cameraman.jpg');
• y=imnoise(x,'gaussian',0,0.005);
• t2=wiener2(y,[5,5]);
• t3=wiener2(y,[7,7]);• t3=wiener2(y,[7,7]);
• t4=wiener2(y,[9,9]);
• subplot(2,2,1),imshow(y);
• subplot(2,2,2),imshow(t2);
• subplot(2,2,3),imshow(t3);
• subplot(2,2,4),imshow(t4);
40DKVishwakarma
Estimation of Degradation Model
Degradation model:
),(),(),(),( yxyxhyxfyxg η+∗=
Purpose: to estimate h(x,y) or H(u,v)
),(),(),(),( vuNvuHvuFvuG +=or
Why? If we know exactly h(x,y), regardless of noise, we can do
DKVishwakarma41
Methods:
1. Estimation by Image Observation
2. Estimation by Experiment
3. Estimation by Modeling
Why? If we know exactly h(x,y), regardless of noise, we can do
deconvolution to get f(x,y) back from g(x,y).
Estimation by Image Observation:
f(x,y) f(x,y)*h(x,y) g(x,y)
Original image (unknown) Degraded image
Observation
DKVishwakarma42
Subimage
Reconstructed
Subimage
),( vuGs ),( yxgs
),(ˆ yxfs
DFT
DFT
),(ˆ vuFs
Restorationprocess byestimation
),(ˆ
),(),(),(
vuF
vuGvuHvuH
s
ss =≈
Estimated Transfer function
This case is used when we
know only g(x,y) and cannot
repeat the experiment!
Estimation by Experiment
Used when we have the same equipment set up and can repeat
the experiment.
Input impulse image
System
H( )
Response image fromthe system
DKVishwakarma43
H( )
),( vuG
),( yxg),( yxAδ
{ } AyxADFT =),(δ
A
vuGvuH
),(),( =
DFTDFT
Estimation by Modeling:
Used when we know physical mechanism underlying the image
formation process that can be expressed mathematically.
Atmospheric
Turbulence model
6/522 )(),( vukevuH +−=
Example:Original image Severe turbulence
DKVishwakarma44
k = 0.00025k = 0.001
k = 0.0025
Low turbulenceMild turbulence
Estimation by Modeling: Motion Blurring
Assume that camera velocity is ))(),(( 00 tytxThe blurred image is obtained by
dttyytxxfyxg
T
))(),((),( 00
0
++= ∫where T = exposure time.
dxdyeyxgvuG vyuxj∫ ∫=∞ ∞
+− )(2),(),( π
DKVishwakarma45
dtdxdyetyytxxf
dxdyedttyytxxf
dxdyeyxgvuG
T
vyuxj
vyuxj
T
vyuxj
∫ ∫ ∫
∫ ∫ ∫
∫ ∫
++=
++=
=
∞
∞−
∞
∞−
+−
∞
∞−
∞
∞−
+−
∞− ∞−
+−
0
)(2
00
)(2
0
00
)(2
))(),((
))(),((
),(),(
π
π
π
Estimation by Modeling: Motion Blurring (cont.)
[ ]
dtevuF
dtevuF
dtdxdyetyytxxfvuG
T
tvytuxj
T
tvytuxj
T
vyuxj
∫
∫
∫ ∫ ∫
+−
+−
∞
∞−
∞
∞−
+−
=
=
++=
0
))()((2
0
))()((2
0
)(2
00
00
00
),(
),(
))(),(( ),(
π
π
π
DKVishwakarma46
0
Then we get, the motion blurring transfer function:
dtevuH
T
tvytuxj∫ +−=0
))()((2 00),(π
For constant motion ),())(),(( 00 btattytx =
)(
0
)(2 ))(sin()(
),( vbuaj
T
vbuaj evbuavbua
TdtevuH +−+− +
+== ∫ ππ ππ
Motion Blurring Example
For constant motion
)())(sin()(
),( vbuajevbuavbua
TvuH +−+
+= πππ
DKVishwakarma47
Original image Motion blurred image
a = b = 0.1, T = 1
Removal of Periodic Noise: using frequency domain filtering
• Periodic noise may occur if the imaging equipment
(the acquisition or networking hardware) is subject to
electronic disturbance of a repeating nature, such as
may be caused by an electric motor.
• We can easily create periodic noise by overlaying an
image with a trigonometric function.image with a trigonometric function.
• Periodic noise appears as concentrated bursts of
energy in the Fourier transform, at locations
corresponding to the frequencies of the periodic
interference.
• The approach is use to filter these periodic noise is
selective filtering.
48DKVishwakarma
Selective Filtering
• Selective filtering can be classified as;
1) Band Reject
2) Band Pass
3) Notch Filter
DKVishwakarma49
Periodic Noise Reduction by Freq. Domain Filtering Degraded image
Periodic noise
can be reduced by
setting frequency
components
corresponding to
noise to zero.
DKVishwakarma50
Band reject filter Restored image
MATLAB Implementation:
DKVishwakarma51