Image Restoration

Resmi N.G.

Reference: Digital Image Processing,2nd Edition

Rafael C. Gonzalez

Richard E. Woods

Image Restoration

A Model of the Image Degradation/ Restoration Process

Noise Models

Important Noise Probability Density Functions

Gaussian Noise

Rayleigh Noise

Erlang or Gamma Noise

Exponential Noise

Uniform Noise

Impulse or Salt-and-Pepper Noise

Periodic Noise

Estimation of Noise Parameters

Spatial Filtering – Restoration in the presence of noise only

Mean Filters

Arithmetic Mean Filter

Geometric Mean Filter

Harmonic Mean Filter

Contraharmonic Mean Filter

3/24/2012 2 CS 04 804B Image Processing Module 2

Order-Statistics Filters

Median Filter

Max and Min Filters

Mid-point Filter

Alpha-trimmed Mean Filter

Adaptive Filters

Adaptive, local noise reduction filter

Adaptive median filter

Frequency Domain Filtering - Periodic Noise Reduction

Bandreject Filters

Bandpass Filters

Notch Filters

Linear Position-Invariant Degradations

Inverse Filtering

Minimum Mean Square Error (MMSE) or Weiner Filtering

Constrained Least Squares Filtering



Image Restoration An objective process where it attempts to reconstruct or

recover an image that has been degraded by using a priori

knowledge of degradation phenomenon.

3/24/2012 CS 04 804B Image Processing Module 2 4

Original Image Restored Image


Original Image Restored Image 3/24/2012 CS 04 804B Image Processing Module 2 6

Image Degradation/Restoration model

f(x,y) : Input Image

η(x,y) : Additive Noise

g(x,y) : Degraded Image

f(x,y) : Estimate of the Original Image


^

The more we know about the degradation function H and

the additive noise η, the closer is the estimate to the

original image.

Degraded image in spatial domain:

If H is a linear, position-invariant process, then the

degraded image is given by

g(x,y) = h(x,y) * f(x,y) + η(x,y)

h(x,y) is the spatial representation of the degradation

function.

* indicates spatial convolution.


Frequency Domain Representation:

G(u,v) = H(u,v)F(u,v) + N(u,v)

(Hint: Convolution in spatial domain is equal to

multiplication in frequency domain.)


Image Restoration


Noise Models


Gaussian Noise

Rayleigh Noise


Exponential Noise

Uniform Noise


Periodic Noise



Mean Filters






The principal sources of noise in digital images arise during image acquisition and transmission.

Most types of noise are modeled as probability density functions (PDFs) represented as p(z) for gray levels z.

Parameters can be estimated based on histogram on small flat area of an image.


Noise Models

1. Gaussian noise Arises in image from factors like electronic circuit noise, sensor

noise due to poor illumination or high temperature


Where z : Gray level μ : Mean average value of z σ : Standard deviation of μ

• 70% of values are in [(μ-σ),(μ+σ)] • 95% of values are in [(μ-2σ),(μ+2σ)]

2. Rayleigh noise


Where

a,b are positive integers.

Mean and variance are

Helpful in range imaging.

3.Erlang (Gamma) noise


Where a>0; b is a positive integer.

If the equation includes Gamma

function then the density is more

appropriately called Erlang

density.

Application in laser imaging.

4.Exponential noise


Where a > 0 and b=1.

It is a special case of Erlang

PDF with b=1.

Mean and Variance are given by

Application in laser imaging.

5.Uniform noise


Basis for random number generators that are used in simulators.

6.Impulse (salt-and-pepper) noise

Found where quick transients take place during imaging (as in faulty switching).


• If b > a, gray-level b will appear as a light dot

in the image.

• Conversely, level a will appear like a dark

dot.

• If either Pa or Pb is zero, impulse noise is

called unipolar.

•If neither is zero and are approx. equal, noise

values will resemble salt-and-pepper granules.

Original Image




Periodic Noise Arises from electrical or electromechanical interference

during image acquisition.

Spatially dependent noise.

Can be reduced significantly by frequency domain

filtering.


Image Restoration


Noise Models


Gaussian Noise

Rayleigh Noise


Exponential Noise

Uniform Noise


Periodic Noise



Mean Filters






Estimation of Noise parameters

Parameters of periodic noise – estimated by inspecting the Fourier spectrum of the image.

Parameters of noise PDFs – known partially from sensor specifications.

When only sensor images are available, the parameters of the PDF can also be estimated from small patches of reasonably constant gray level.

Histogram can also be used to identify the PDF.


Estimation of noise parameters

1. Experimentally we can usually choose a small patch

of an image that is relatively uniform and compute

a histogram of the image over that region.


2. The shape of the histogram identifies the closest PDF match.

Estimation of noise parameters 3. Using the histogram, we can estimate the noise mean and

variance as follows:

where zi s are the gray-level values of pixels in strip S, and p(zi) are the corresponding normalized histogram values.

5. The mean and variance are used to solve for the parameters a and b in the density function.


Image Restoration


Noise Models


Gaussian Noise

Rayleigh Noise


Exponential Noise

Uniform Noise


Periodic Noise



Mean Filters






Let Sxy represent the set of coordinates in a rectangular

sub-image window of size mn, centered at point (x,y).

The arithmetic mean filter computes the average value of

the corrupted image g(x,y) in the area defined by Sxy.

The value of the restored image at any point (x,y) is

given by


1.a Arithmetic Mean Filter

,( , )

1ˆ( , ) ( , )x ys t S

f x y g s tmn

f̂

1.b Geometric Mean Filter


Achieves smoothing comparable to arithmetic mean

filter but tends to lose image detail in the process.

1.c Harmonic Mean Filter

Works well for salt noise but fails for pepper noise.

Works well with other types of noise like Gaussian noise.


1.d Contraharmonic Mean Filter

1

( , )

( , )

( , )

ˆ ( , )( , )

xy

xy

Q

s t S

Q

s t S

g s t

f x yg s t


where Q is called the order of the filter.

• Well-suited for eliminating the effects of salt-and-

pepper noise.

• For positive values of Q, it eliminates Pepper noise.

• For negative values of Q, it eliminates Salt noise.

• Cannot work simultaneously.

• Reduces to Arithmetic Mean filter if Q=0 and

Harmonic mean filter if Q= -1

Arithmetic and Geometric Mean filters – well suited for

random noise like Gaussian or uniform noise.

Contraharmonic filter – well suited for impulse noise.



Median Filter

Max and Min Filters

Mid-point Filter


Adaptive Filters




Bandreject Filters

Bandpass Filters

Notch Filters


Inverse Filtering





2.a Median filter

( , )

ˆ ( , ) ( , )xys t S

f x y median g s t


•Replaces the value of a pixel by the median of

graylevels in the neighborhood of that pixel.

•Median represents the 50th percentile of a ranked set of

numbers.

•For random noise, it provides excellent noise-reduction

with lesser blurring than linear smoothing filters of

similar size.

• Effective for Bipolar and Unipolar impulse noise.

2.b Max and Min Filters

Max filter – Uses 100th percentile.

Used for finding the brightest points in an image.

Reduces pepper noise.

Min filter – Uses 0th percentile.

Used for finding the darkest points in the image.

Reduces salt noise.


( , )

ˆ ( , ) max ( , )xys t S

f x y g s t

( , )

ˆ ( , ) min ( , )xys t S

f x y g s t

2.c Midpoint Filter

Computes midpoint between the maximum and minimum values in the area encompassed by the filter.

Works best for randomly distributed noise (Gaussian or uniform noise).


( , )( , )

1ˆ ( , ) max ( , ) min ( , )2 xyxy s t Ss t S

f x y g s t g s t

2.d Alpha-trimmed Filters

If d/2 lowest and d/2 highest gray-level values of g(s,t) in the neighborhood Sxy are deleted, and if gr(s,t) represents the remaining mn-d pixels, then the alpha-trimmed mean filter formed by averaging the remaining pixels is given by


Where, d ranges from 0 to mn-1.

When d=0, the filter reduces to arithmetic mean filter.

When d= (mn-1)/2, the filter reduces to median filter.

( , )

1ˆ( , ) ( , )xy

r

s t S

f x y g s tmn d


Median Filter

Max and Min Filters

Mid-point Filter


Adaptive Filters




Bandreject Filters

Bandpass Filters

Notch Filters


Inverse Filtering






Adaptive, Local Noise Reduction Filter

Mean – measure of average gray-level in a region.

Variance – measure of average contrast in a region.

Response of a filter at (x,y) operating on a region S is based on:

g(x,y) – the value of noisy image at (x,y)

σ2η – the variance of the noise corrupting f(x,y) to form

g(x,y)

mL – the local mean of the pixels in Sxy.

σ2L – the local variance of pixels in Sxy.

The filter expression is

Zero-noise case: If σ2η is zero, the filter returns the value

of g(x,y) which is equal to f(x,y).

If local variance is high relative to σ2η, the filter should

return a value close to g(x,y).

If the two variances are equal, the filter returns the arithmetic

mean value of pixels in the neighborhood.


2

2ˆ( , ) ( , ) ( , ) L

L

f x y g x y g x y m

Adaptive Median Filter Can handle impulse noise with larger probabilities.

Preserves detail while smoothing non-impulse noise.

Adaptive median filter increases the size of subimage during filter operations.

Output of any filter is a single value used to replace the value of the pixel at (x,y), the point on which the window is centered at a given time.


zmin = minimum gray level value in Sxy.

zmax = maximum gray level value in Sxy.

zmed = median of gray levels in Sxy.

zxy = gray level at coordinates (x,y).

Smax = maximum allowed size of Sxy.


Two-level algorithm

Level A:

A1 = zmed - zmin

A2 = zmed - zmax

If A1>0 AND A2<0, goto level B.

Else increase the window size.

If window size ≤ Smax, repeat level A.

Else output zxy.

Level B:

B1 = zxy - zmin

B2 = zxy - zmax

If B1>0 AND B2<0, output zxy.

Else output zmed.


Three main purposes:

To remove salt-and-pepper noise

To provide smoothing of other noise that may not be

impulsive

To reduce distortion(excessive thinning or thickening of

object boundaries).


Zmin and zmax are considered impulse-like noise

components.

Purpose of level A is to determine if the median filter

output zmed is an impulse (black or white) noise or not.

If the condition zmin < zmed < zmax holds, then zmed cannot

be an impulse. Goto level B and test to see if the point zxy

in the centre of the window is itself an impulse.

If B1>0 AND B2<0, then zmin < zxy < zmax. zxy cannot be an

impulse. Algorithm outputs zxy.


If the condition B1>0 AND B2<0 does not hold, then

either zxy = zmin or zxy = zmax. In either case, the value of

pixel is an extreme value and the algorithm outputs the

median value, zmed.

Suppose, A finds an impulse. Then, it increases the size of

the window and repeats level A. Continues until the

algorithm either finds a median value that is not an

impulse or maximum window size is reached.



Median Filter

Max and Min Filters

Mid-point Filter


Adaptive Filters




Bandreject Filters

Bandpass Filters

Notch Filters


Inverse Filtering






Bandreject Filters Bandreject filters remove (or attenuate) a band of

frequencies, around some frequency, say D0 .

An ideal bandreject filter is given by:

where 22, vuvuD

0

0 0

0

1 ( , )2

( , ) 0 ( , )2 2

1 ( , )2

Wif D u v D

W WH u v if D D u v D

Wif D u v D

W - width of the band

D0 is the radial centre.

D(u,v) is the distance from the origin of the centered frequency

rectangle.


3/24/2012 49

Butterworth Bandreject filters A Butterworth bandreject filter of order n is given by

A Gaussian bandreject filter is given by

2

2 2

0

1( , )

( , )1

( , )

nH u v

D u v W

D u v D

22 2

0( , )1

2 ( , )( , ) 1

D u v D

D u v WH u v e

CS 04 804B Image Processing Module 2

3/24/2012 50

Bandreject Filters


3/24/2012 51

Example: Bandreject Filters


3/24/2012 52

Bandpass Filters

Bandpass filters perform the opposite operation of bandreject

filters. They pass a band of frequencies, around some

frequency, say D0 (rejecting the rest).

The transfer function of a bandpass filter is obtained from a

corresponding bandreject filter as:

Hbp(u,v) = 1 - Hbr(u,v)


Bandpass filter is usually used to isolate components

of an image that correspond to a band of frequencies.

It can also be used to isolate noise pattern, so that a

more detailed analysis of the noise can be

performed, independent of the image.



3/24/2012 55

Notch Filters It is a kind of bandreject/bandpass filter that

rejects/passes a very narrow set of frequencies, around a

center frequency.

Due to symmetry of Fourier transform, the notch filters

must occur in symmetric pairs about the origin of the

frequency plane.


The transfer function of an ideal notch-reject filter of

radius D0 with centers at (u0,v0) and (-u0,-v0) is given by:

Where

And


1 0 2 00 ( , ) ( , )( , )

1

if D u v D or D u v DH u v

otherwise

12 2 2

1 0 0( , )2 2

M ND u v u u v v

1

2 2 2

2 0 0( , )2 2

M ND u v u u v v

The transfer function of a Butterworth notch-reject filter

of order n is given by:

A Gaussian notch reject filter has the form


2

0

1 2

1( , )

1( , ) ( , )

nH u v

D

D u v D u v

1 2

20

( , ) ( , )1

2( , ) 1

D u v D u v

DH u v e


Notch-pass filter passes the frequencies contained in the

notch areas.

Performs exactly the opposite function as notch-reject

filters.

Transfer function is given by:

Hnp(u,v) = 1 - Hnr(u,v)


3/24/2012 60

Optimum Notch Filtering When several interference patterns are present,

filtering may remove much image information.

Solution - first filter out the noise interference by placing a notch pass filter H(u,v) at the location of each spike:

N(u,v) = H(u,v)G(u,v)

G(u,v) is the Fourier transform of the corrupted image.


Corresponding pattern in the spatial domain is obtained as

(x,y) = F-1{N(u,v)} = F-1{H(u,v) G(u,v)}

We can then subtract off a weighted portion of (x,y)

from the image g(x,y) to obtain our restored image:

w(x,y) is called weighting or modulation function. It can

be chosen so as to minimize the variance of the estimate

over a specified neighbourhood of every point

(x,y).

(Refer word doc for derivation)


( , ) ( , ) ( , ) ( , )f x y g x y w x y x y

( , )f x y


Median Filter

Max and Min Filters

Mid-point Filter


Adaptive Filters




Bandreject Filters

Bandpass Filters

Notch Filters


Inverse Filtering





3/24/2012 63

Linear, Position-Invariant Degradations Input-output relationship before restoration stage

g(x,y) = H[f(x,y)] + η(x,y)

Assume η(x,y) = 0.

Therefore, g(x,y) = H[f(x,y)]

H is linear if :

H[af1(x,y)+bf2(x,y)] = aH[f1(x,y)]+bH[f2(x,y)]


If a = b = 1, H[f1(x,y)+f2(x,y)] = H[f1(x,y)]+H[f2(x,y)]

This property is called additivity.

If f2(x,y) = 0, H[af1(x,y)] = aH[f1(x,y)]

This property is called homogeneity.

An operator satisfying g(x,y) = H[f(x,y)] is said to be position

invariant if:

H[ f (x-α, y-β) ] = g( x-α, y-β )

That is, response at any point in the image depends only

on the value of input at that point, not in its position.


In terms of continuous impulse function,


( , ) ( , ) ( , )f x y f x y d d

( , , , )

( , ) [ ( , )]

( , ) ( , )

( , ) ( , )

( , ) ( , )

h x y

g x y H f x y

H f x y d d

H f x y d d

f H x y d d

h (x,α, y,β) is the impulse response of H.

If η(x,y) = 0,

This is called the superposition integral of the first kind.

A linear system h is completely characterized by its

impulse response.

If H is position invariant,


( , ) ( , ) ( , , , )g x y f h x y d d

( , ) ( , )H x y h x y

( , ) ( , ) ( , )g x y f h x y d d

is called the convolution integral.

That is, the response g is the convolution of impulse

response and the input function.


( , ) ( , )f h x y d d

Convolution



Presence of Noise

If H is position invariant,


( , ) ( , ) ( , , , ) ( , )g x y f h x y d d x y

( , ) ( , ) ( , ) ( , )

( , ) * ( , ) ( , )

g x y f h x y d d x y

h x y f x y x y

( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v

Degradation is modeled as convolution.

Restoration is modeled as deconvolution.

Restoration filters are hence called deconvolution filters.


Estimating the degradation function

Estimation by observation

Estimation by experimentation

Estimation by mathematical modeling


Estimation by Observation No knowledge about the degradation function H.

Gather information from image itself.

Let: gs(x,y) be the observed subimage.

be the constructed subimage.

Assume negligible noise (choose strong signal area).

Then,

From the characteristics of Hs, deduce the complete

function H(u,v) assuming position invariance.


( , )f x y

( , )( , )

( , )

ss

s

G u vH u v

F u v

Estimation by Experimentation Accurate estimate of the degradation can be obtained if

device similar to the one used for capturing degraded

image is available.

Obtain the impulse response of degradation by imaging an

impulse using the same system settings.

Linear space invariant system is completely described by

its impulse response.



Assume negligible noise.

Fourier transform of an impulse is a constant which

describes the strength of the impulse.


( , )( , )

G u vH u v

A

Estimation by Modeling A) Takes into account the environmental conditions

Where k is a constant that depends on nature of

atmospheric turbulence.


5

2 2 6

( , )k u v

H u v e

B) Image blurred due to uniform and linear motion

between image and sensor during acquisition.

Let f(x,y) undergo planar motion.

x0(t) and y0(t) be the time varying components of motion

in x and y directions.

Total exposure at any point of recording medium is

obtained by integrating instantaneous exposure over the

time interval during which the shutter is open.

Let T be the duration of exposure.



0 0

0

( , ) ( ( ), ( ))

T

g x y f x x t y y t dt

2 ( )( , ) ( , ) j ux vyG u v g x y e dxdy

2 ( )

0 0

0

( ( ), ( ))

T

j ux vyf x x t y y t dt e dxdy


0 0

0 0

2 ( )

0 0

0

( , )

2 ( ( ) ( ))

0

2 ( ( ) ( ))

0

( ( ), ( ))

( , )

( , )

T

j ux vy

shifted F u v

T

j ux t vy t

T

j ux t vy t

f x x t y y t e dxdy dt

F u v e dt

F u v e dt


0 02 ( ( ) ( ))

0

,

( , )

( , ) ( , ) ( , )

T

j ux t vy t

Define the transfer function

H u v e dt

G u v H u v F u v


Median Filter

Max and Min Filters

Mid-point Filter


Adaptive Filters




Bandreject Filters

Bandpass Filters

Notch Filters


Inverse Filtering





Inverse Filtering The simplest method to restore images degraded by a

degradation function H is direct inverse filtering.

It computes an estimate of the transform of the

original image as


( , )F u v

( , )( , )

( , )

G u vF u v

H u v

Cannot restore the image exactly with knowledge of

H(u,v) because N(u,v) is unknown.

If H(u,v) has values 0, will dominate F(u,v).


( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) ( , )

( , )( , ) ( , )

( , )

G u v H u v F u v N u v

F u v H u v H u v F u v N u v

N u vF u v F u v

H u v

( , )

( , )

N u v

H u v

Inverse filtering is hence very sensitive to noise and has

no provision to handle noise.

One way to avoid values of H(u,v) that tend to zero is to

limit the filter frequencies to values near the origin.

H(0,0), the average value of h(x,y) is the highest value of

H(u,v).

Blurring (degradation) corresponds to lowpass filtering

and inverse filtering corresponds to highpass filtering.



Median Filter

Max and Min Filters

Mid-point Filter


Adaptive Filters




Bandreject Filters

Bandpass Filters

Notch Filters


Inverse Filtering





Wiener (MMSE) Filtering Refer Page: 284-286


Refer Page: 288-291

Laplacian Operator



2 22

2 2

2

2

2

2

2

( 1, ) ( 1, ) 2 ( , )

( , 1) ( , 1) 2 ( , )

( 1, ) ( 1, )

( , 1) ( , 1) 4 ( , )

f ff

x y

ff x y f x y f x y

x

ff x y f x y f x y

y

f f x y f x y

f x y f x y f x y


( 1, 1) ( , 1) ( 1, 1)

( , ) ( 1, ) ( , ) ( 1, )

( 1, 1) ( , 1) ( 1, 1)

0 1 0

1 4 1

0 1 0

coeff f x y coeff f x y coeff f x y

p x y coeff f x y coeff f x y coeff f x y

coeff f x y coeff f x y coeff f x y

Geometric Mean Filter Refer Page: 292


Thank You


Documents

Image Restoration