Digital Image Processing

H.R. Pourreza

Digital Image Processing

بسمه تعالي

H.R. Pourreza

Image Restoration(Chapter 5)

H.R. Pourreza

• Image restoration vs. image enhancement Enhancement:

largely a subjective process Priori knowledge about the degradation is not a must

(sometimes no degradation is involved) Procedures are heuristic and take advantage of the

psychophysical aspects of human visual system Restoration:

more an objective process Images are degraded Tries to recover the images by using the knowledge

about the degradation

Image Restoration

H.R. Pourreza

• Two types of degradation Additive noise

Spatial domain restoration (denoising) techniques are preferred Image blur

Frequency domain methods are preferred• We model the degradation process by a degradation function

h(x,y), an additive noise term, (x,y), as g(x,y)=h(x,y)*f(x,y)+ (x,y)

f(x,y) is the (input) image free from any degradation g(x,y) is the degraded image * is the convolution operator The goal is to obtain an estimate of f(x,y) according to the

knowledge about the degradation function h and the additive noise

In frequency domain: G(u,v)=H(u,v)F(u,v)+N(u,v)• Three cases are considered in this Chapter

g(x,y)=f(x,y)+ (x,y) (5-2~5-4) g(x,y)=h(x,y)*f(x,y) (5-5~5-6) g(x,y)=h(x,y)*f(x,y)+ (x,y) (5-7~5-9)

An Image Degradation Model

H.R. Pourreza

A Model of the Image Degradation/Restoration Process

H.R. Pourreza

• We first consider the degradation due to noise only h is an impulse for now ( H is a constant)

• White noise Autocorrelation function is an impulse function

multiplied by a constant

It means there is no correlation between any two pixels in the noise image

There is no way to predict the next noise value The spectrum of the autocorrelation function is a

constant (white) (the statement in page 222 about white noise is wrong)

),(),(),(),( 0

1

0

1

0

yxNytxstsyxaN

t

M

s

Noise Model

H.R. Pourreza

• Noise (image) can be classified according the distribution of the values of pixels (of the noise image) or its (normalized) histogram

• Gaussian noise is characterized by two parameters, (mean) and σ2 (variance), by

• 70% values of z fall in the range [(-σ),(+σ)]• 95% values of z fall in the range [(-2σ),(+2σ)]

22 2/)(

2

1)(

zezp

Gaussian Noise

H.R. Pourreza

Gaussian Noise

H.R. Pourreza

• Rayleigh noise

The mean and variance of this

density are given by

a and b can be obtained through mean and variance

az

azeazbzp

baz

for 0

for )(2

)(/)( 2

4

)4( and 4/ 2

b

ba

Other Noise Models

H.R. Pourreza

• Erlang (Gamma) noise

The mean and variance of this

density are given by

a and b can be obtained through mean and variance

0for 0

0for )!1()(

1

z

zeb

zazp

azbb

22 and /

a

bab

Other Noise Models

H.R. Pourreza

• Exponential noise

The mean and variance of

this density are given by

Special case pf Erlang PDF with b=1

0for 0

0for )(

z

zaezp

az

22 1

and /1a

a

Other Noise Models

H.R. Pourreza

• Uniform noise

The mean and

variance of this density are given by

otherwise 0

if 1

)( bzaabzp

12

)( and 2/)(

22 ab

ba

Other Noise Models

H.R. Pourreza

• Impulse (salt-and-pepper) noise

If either Pa or Pb is zero, the impulse noise is called unipolar

a and b usually are extreme values because impulse corruption is usually large compared with the strength of the image signal

It is the only type of noise that can be distinguished from others visually

otherwise 0

for

for

)( bzP

azP

zp b

a

Other Noise Models

H.R. Pourreza

A Sample Image

H.R. Pourreza

Effect of Adding Noise to Sample Image

H.R. Pourreza

Effect of Adding Noise to Sample Image

H.R. Pourreza

• Arises typically from electrical or electromechanical interference during image acquisition

• It can be observed by visual inspection both in the spatial domain and frequency domain

• The only spatially dependent noise will be considered

Periodic Noise

H.R. Pourreza

• Periodic noise Parameters can be estimated by inspection of

the spectrum

• Noise PDFs From sensor specifications If imaging sensors are available, capture a set of

images of plain environments If only noisy images are available, parameters of

the PDF involved can be estimated from small patches of constant regions of the noisy images

Estimation of Noise Parameters

H.R. Pourreza

H.R. Pourreza

• In most cases, only mean and variance are to be estimated Others can be obtained from the estimated

mean and variance

• Assume a sub-image with plain scene is available and is denoted by S

Syx

iiS ii

yxzN ),(

),(1̂

2

),(

20 )),((

1 Syx

iiS ii

yxzN

2

),(

21 )ˆ),((

1 Syx

iiS ii

yxzN

2

),(

22 )ˆ),((

1

1

Syx

iiS ii

yxzN

Estimation of Noise Parameters

H.R. Pourreza

• Mean filters Arithmetic mean filter

g(x,y) is the corrupted image Sx,y is the mask

Geometric mean filters Tends to preserve more details

Harmonic mean filter

Works well for salt noise but fails for pepper noise Contraharmonic mean filter

Q: order of the filter Positive Q works for pepper noise Negative Q works for salt noise Q=0arithmetic mean filter Q=-1harmonic mean filter

yxSts

tsgmn

yxf,),(

),(1

),(ˆ

mn

Sts yx

tsgyxf

1

),( ,

),(),(ˆ

yxSts tsg

mnyxf

,),( ),(1

),(ˆ

yx

yx

Sts

Q

Sts

Q

tsg

tsg

yxf

,

,

),(

),(

1

),(

),(

),(ˆ

Restoration in the Presence of Noise Only (De-Noising)

H.R. Pourreza

De-Noising

Corrupted by Gaussian Noise

Mean Filtering Geometric

Mean Filtering

H.R. Pourreza

De-Noising

Corrupted by pepper noise

Corrupted by salt noise

3x3 Contraharmonic Q=1.5

3x3 Contraharmonic Q=-1.5

H.R. Pourreza

De-Noising

H.R. Pourreza

• Median filter Median represents the 50th percentile of a

ranked set of numbers • Max and min filter

Max filter uses the 100th percentile of a ranked set of numbers Good for removing pepper noise

Min filter uses the 1 percentile of a ranked set of numbers Good for removing salt noise

• Midpoint filter

Works best for noise with symmetric PDF like Gaussian or uniform noise

)},({median),(ˆ,

tsgyxfyxS(s,t)

)},({min)},({max

2

1),(ˆ

),(),(tsgtsgyxf

xyxy StsSts

Filters Based on Order Statistics(De-Noising)

H.R. Pourreza

De-Noising

Corrupted by salt & pepper noise

One pass median filtering

Two pass median filtering

Three pass median filtering

H.R. Pourreza

De-Noising

Max Filtering

Min Filtering

Corrupted by pepper noise

Corrupted by salt noise

H.R. Pourreza

• Alpha-trimmed mean filter takes the mean value of the pixels enclosed by an m×n mask after deleting the pixels with the d/2 lowest and the d/2 highest gray-level values

gr(s,t) represent the remaining mn-d pixels It is useful in situations involving multiple

types of noise like a combination of salt-and-pepper and Gaussian

xySts

r tsgdmn

yxf),(

),(1

),(ˆ

Alpha-Trimmed Mean Filter(De-Noising)

H.R. Pourreza

De-Noising

Corrupted by additive Uniform noise

Added salt & pepper noise

5x5 Mean Filtering 5x5 Geo-

Mean Filtering

5x5 Median Filtering

5x5 Alpha-trimmed Mean Filtering

H.R. Pourreza

• Adaptive Local Noise Reduction Filter Assume the variance of the noise is either

known or can be estimated satisfactorily Filtering operation changes at different regions

of an image according to local variance calculated within an M×N region

If , the filtering operation is defined as

If , the output takes the mean value

That is:

At edges, it is assumes that

2

2L

22 L

22 L

]),([),(),(ˆ2

2

LL

myxgyxgyxf

22 L

1 be set to is2

2

L

Adaptive Filters(De-Noising)

H.R. Pourreza

De-Noising

Corrupted by Gaussian

noise

Mean Filtering

Geo-Mean

Filtering

Adaptive Filtering

H.R. Pourreza

• Median filter is effective for removing salt-and-pepper noise The density of the impulse noise can not be too

large

• Adaptive median filter Notation

Zmin: minimum gray value in Sxy

Zmax: maximum gray value in Sxy

Zmed: median of gray levels in Sxy

Zxy: gray value of the image at (x,y)

Smax: maximum allowed size of Sxy

Adaptive Median Filter(De-Noising)

H.R. Pourreza

• Two levels of operations Level A:

A1= Zmed –Zmin

A2= Zmed –Zmax If A1 > 0 AND A2 < 0, Go to level B else increase the window size by 2 If window size <= Smax repeat level A else output Zxy

Level B: B1= Zxy –Zmin

B2= Zxy –Zmax

If B1 > 0 AND B2 < 0, output Zxy

else output Zmed

Used to test whether Zmed is part of s-and-p noise. If yes, window

size is increased

Used to test whether Zxy is part of s-and-p noise. If yes, apply

regular median filtering

Adaptive Median Filter(De-Noising)

H.R. Pourreza

De-Noising Median Filtering Adaptive Median Filtering

H.R. Pourreza

• Lowpass and highpass filters for image enhancement have been studied

• Bandreject, bandpass, and notch filters as tools for periodic noise reduction or removal are to be studied in this section.

Periodic Noise Reduction by Frequency Domain Filtering

H.R. Pourreza

• Bandreject filters remove or attenuate a band of frequencies about the origin of the Fourier transform.

• Similar to those LPFs and HPFs studied, we can construct ideal, Butterworth, and Gaussian bandreject filters

Bandreject Filters

H.R. Pourreza

• Ideal bandreject filter

Bandreject Filters

2),(1

2),(

20

2),(1

),(

0

00

0

WDvuDif

WDvuD

WDif

WDvuDif

vuH

H.R. Pourreza

• Butterworth bandreject filter

Bandreject Filters

n

DvuDWvuD

vuH 2

20

2 ),(),(

1

1),(

n =1

H.R. Pourreza

• Gaussian bandreject filter

Bandreject Filters

WvuD

DvuD

evuH),(

),(

2

1 20

2

1),(

H.R. Pourreza

Bandreject Filters

H.R. Pourreza

Bandbass Filters

),(1),( vuHvuH brbp

Bandpass filter performs the opposite of a bandpass filter

H.R. Pourreza

Notch Filters

• Notch filter ejects frequencies in predefined neighborhoods about a center frequency.

• It appears in symmetric pairs about the origin because the Fourier transform of a real valued image is symmetric.

H.R. Pourreza

• Ideal notch filter

otherwise

DvuDorDvuDifvuH

1

),(),(0),( 0201

Notch Filters

2/120

201 2/2/),( vNvuMuvuD

2/120

202 2/2/),( vNvuMuvuD

H.R. Pourreza

• Butterworth notch filter

Notch Filters

n

vuDvuDD

vuH 2

21

20

),(),(1

1),(

n=2

H.R. Pourreza

• Gaussian notch filter

Notch Filters

20

21 ),(),(

2

1

1),( D

vuDvuD

evuH

H.R. Pourreza

Notch Filters

Notch filters that pass, rather than suppress:

),(1),( vuHvuH nrnp

000 vu• NR filters become highpass filters if

• NP filters become lowpass filters if 000 vu

H.R. Pourreza

Notch Filters

You can see the effect of scan lines

Spectrum of image

Notch pass filter

IFT of NP filtered image

Result of NR filter

H.R. Pourreza

Optimum Notch Filtering

H.R. Pourreza

• In the ideal case, the original image can be restored if the noise can be estimated completely. That is:

• However, the noise can be only partially estimated. This means the restored image is not exact. Which means

),(),(),( yxyxgyxf

),(ˆ),(),(ˆ yxyxgyxf


),(),(),(ˆ vuGvuHIFTyx

H.R. Pourreza

• In this section, we try to improve the restored image by introducing a modulation function

Here the modulation function is a constant within a neighborhood of size (2a+1) by (2b+1) about a point (x,y)

We optimize its performance by minimizing the local variance of the restored image at the position (x,y)

a

as

b

bt

yxftysxfba

yx2

2 ),(ˆ),(ˆ)12)(12(

1),(

),(ˆ),(),(),(ˆ yxyxwyxgyxf


a

as

b

bt

tysxfba

yxf ),(ˆ)12)(12(

1),(ˆ

H.R. Pourreza

btbandasaforyxwtysxw ),(),(

2

2

]},(ˆ),(),([

)],(ˆ),(

),({[)12)(12(

1),(

yxyxwyxg

tysxtysxw

tysxgba

yxa

as

b

bt


Points on or near Edge of the image can be treated by considering partial neighborhoods

Assumption:

),(ˆ),(),(ˆ),( yxyxwyxyxw

H.R. Pourreza


0),(

),(2

yxw

yx

2

2

)},(ˆ),(),([

)],(ˆ),(

),({[)12)(12(

1),(

yxyxwyxg

tysxtysxw

tysxgba

yxa

as

b

bt

),(ˆ),(ˆ

),(ˆ),(),(ˆ),(),(

22 yxyx

yxyxgyxyxgyxw

To minimize ),(2 yx

H.R. Pourreza


),(ˆ),(),(),(ˆ yxyxwyxgyxf

),( yxw),(ˆ yx),( yxg

H.R. Pourreza


H.R. Pourreza


H.R. Pourreza


),( yxg ),(ˆ yxfImage size: 512x512

a=b=15

H.R. Pourreza

• Degradation Model

),()],([),( yxyxfHyxg

Linear, Position-Invariant Degradation

)],([)],([)],(),([ 2121 yxfHbyxfHayxfbyxfaH

),()],([)],([),( yxgyxfHyxfHyxg

In the absence of additive noise:

For scalar values of a and b, H is linear if:

H is Position-Invariant if:

H.R. Pourreza

Linear, Position-Invariant Degradation

In the presence of additive noise:

),(),(),(),( yxddyxhfyxg

),(),(),(),(

),(),(*),(),(

vuNvuFvuHvuG

yxyxfyxhyxg

• Many types of degradation can be approximated by linear, position-invariant processes

• Extensive tools of linear system theory are available

• In this situation, restoration is image deconvolution

H.R. Pourreza

• Principal way to estimate the degradation function for use in image restoration: Observation Experimentation Mathematical modeling

Estimating the Degradation Function

H.R. Pourreza

• We look for a small section of the image that has strong signal content ( ) and then construct an un-degradation of this section by using sample gray levels ( ).

),( yxgs

Estimating by Image Observation

),(ˆ yxf s

),(ˆ),(

),(vuF

vuGvuH

s

ss

Now, we construct a function on a large scale, but having the same shape.

),( vuH

H.R. Pourreza

• We try to obtain impulse response of the degradation by imaging an impulse (small dot of light) using the system. Therefore

Estimating by Experimentation

A

vuGvuH

),(),(

H.R. Pourreza

Estimating by Modeling6/522 )(),( vukevuH Atmospheric turbulence model:

Negligible turbulence

Mid turbulence k=0.001

High turbulence k=0.0025

Low turbulence k=0.00025

H.R. Pourreza

Estimating by Modeling

T

dttyytxxfyxg0

00 )](),([),(

Blurring by linear motion:

uaj

TTuat

euaua

T

dtevuH

)sin(

),(0

/2 0)(/)( 00 tyandTattxif

dtevuFvuGT

tvytuxj 0

)]()([2 00),(),(

dtevuHT

tvytuxj 0

)]()([2 00),(

H.R. Pourreza

Estimating by Modeling

)()](sin[)(

),( vbuajevbuavbua

TvuH

TbttyandTattxif /)(/)( 00

H.R. Pourreza

Inverse Filtering

),(

),(),(ˆ

vuH

vuGvuF

The simplest approach to restoration is direct inverse filtering:

),(

),(),(),(ˆ

vuH

vuNvuFvuF

Even if we know the degradation function, we cannot recover the un-degraded image

If the degradation has zero or very small values, then the ratio N/H could easily dominate our estimation of F .

One approach to get around the zero or small-value problem is to limit the filter frequencies to value near the origin.

H.R. Pourreza

Inverse Filtering

Full inverse Filtering

Filtering with H cut off outside a radius of 70



Degraded Image

H.R. Pourreza

Minimum Mean Square Error Filtering(Wiener Filtering)

])ˆ[( 22 ffEe

This approach incorporate both the degradation function and statistical characteristic of noise into the restoration process.

Image and noise are random process

The objective is to find an estimation for f such that minimized e2

H.R. Pourreza


),(),(/),(),(

),(

),(

1

),(),(/),(),(

),(

),(),(),(),(

),(),(),(ˆ

2

2

2

*

2

*

vuGvuSvuSvuH

vuH

vuH

vuGvuSvuSvuH

vuH

vuGvuSvuHvuS

vuSvuHvuF

f

f

f

f

noisetheofspectrumpowervuNvuS 2),(),(

imageundegradedtheofspectrumpowervuFvuS f 2),(),(

If the noise is zero, then the Wiener Filter reduces to the inverse filter.

H.R. Pourreza


),(),(/),(),(

),(

),(

1),(ˆ

2

2

vuGvuSvuSvuH

vuH

vuHvuF

f

Unknown

Constant

),(),(

),(

),(

1),(ˆ

2

2

vuGKvuH

vuH

vuHvuF

H.R. Pourreza

Wiener Filtering

Full inverse filtering

Radially limited inverse filtering

Wiener filtering

(K was chosen interactively)

H.R. Pourreza

Wiener FilteringInverse filtering

Reduced noise varianc

e

Wiener filtering

H.R. Pourreza

Geometric Transformations

• Unlike the techniques discussed so far, geometric transformations modify the spatial relationships between pixels in an image.

Geometric transformation: RUBBER-SHEET TRANSFORMATION

Basic Operations:

1. Spatial Transformation

2. Gray-level Interpolation

H.R. Pourreza

Spatial Transformations

),(

),(

yxsy

yxrx

8765

4321

),(

),(

cxycycxcyxsy

cxycycxcyxrx

H.R. Pourreza

Spatial Transformations

H.R. Pourreza

Gray-level Interpolation

dyxcybxayxv ),(

H.R. Pourreza


H.R. Pourreza


Documents

Digital Image Processing