Smoothing Techniques in Image Processing Prof. PhD. Vasile Gui Polytechnic University of Timisoara

Smoothing Techniques in Image Processing

Prof. PhD. Vasile Gui

Polytechnic University of Timisoara

Content

Introduction Brief review of linear operators Linear image smoothing techniquesNonlinear image smoothing techniques

Introduction

Why do we need image smoothing?

What is “image” and what is “noise”?

– Frequency spectrum– Statistical properties

Brief Review of Linear Operators[Pratt 1991]

Generalized 2D linear operator

Separable linear operator:

Space invariant operator:

Convolution sum

1

0

1

0

),(),;,(),(M

j

N

k

kjfnmkjOnmg

);();(),;,( nkOmjOnmkjO CR

1

0

1

0

),();();(),(M

j

N

kCR kjfnkOmjOnmg

),(),(),;,( knjmHknjmOnmkjO

1

0

1

0

),(),(),(M

j

N

k

kjfknjmhnmg

Brief Review of Linear Operators

Geometrical interpretation of 2D convolution

h0

0

h01h02

h10

h20h21h22

h11h12

h0

0

h01h02

h10

h20h21h22

h11h12

h0

0

h01h02

h10

h20h21h22

h11h12h0

0

h01h02

h10

h20h21h22

h11h12

N+L-1

N

N-L+1

h0

0

h01h02

h10

h20h21h22

h11h12

L-12

L-12

L-12 L-1

2H

G

F

Matrix formulation

Unitary transform

Separable transform

Transform is invertible


TRCFAAT

IfAfAtAf *T*T

*T1 AA

Aft


Basis vectors are orthogonal

Inner product and energy are conserved

Unitary transform: a rotation in N-dimensional vector space

lk

lklk

,0

,1),(l

T*k aa

2T*T*T*T*

*T*T2

||f||ffIffAfAf

(Af)(Af)tt||t||


2D vector space interpretationof a unitary transform

x1

x2

12


DFT unitary matrix

A

1

0 0 0 0

0 1 2 1

0 2 4 2 1

0 2 1 1 2

N

W W W W

W W W W

W W W W

W W W W

N

N

N N N

( )

( ) ( )

)2

sin()2

cos(}2

exp{N

iNN

iWN


1D DFT

2D DFT

1,...,1,0,}2

exp{)(1

)(1

0

NuunN

inf

Nut

N

n

1,...,1,0,1,...,1,0

)}(2exp{),(1

),(1

0

1

0

NvMu

nN

vm

M

uinmf

MNvut

M

m

N

n


Circular convolution theorem

Periodicity of f,h and g with N are assumed

t u vNt u v t u vg f h( , ) ( , ) ( , )

1

1

0

1

0

),(),(),(N

j

N

k

kjfknjmhnmg

Linear Image smoothing techniques Box filters. Arithmetic mean LL operator

1

1

1

1*111

1

111

1

111

111

12

LLLh


SeparableCan be computed recursively, resulting in

roughly 4 operations per pixel

L pixels

+

m,n

m,n+1


Optimality properties Signal and additive white noise

Noise variance is reduced N times

g f n f n 1 1 1

1 1 1N N Nk kk

N

kk

N

kk

N

( ) .

z n1

1N kk

N

.

2

1 1

22

1 12

1 12

T2

1),(

1

}E{1

}E{1

}E{

N=kl

N

nnN

nnN

zz

N

k

N

l

N

k

N

lkl

N

k

N

lklz


Unknown constant signal plus noiseMinimize MSE of the estimation g:

N

kk gfg

1

22 )()(

0)(2

g

g

N

kkfN

g1

1ˆ


i.i.d. Gaussian signal with unknown mean.

Given the observed samples, maximize

Optimal solution: arithmetic mean

}2

)(exp{.)|(

2

2

f

ctfp

N

k

N

kkk fctfp

1 1

22

})(2

1exp{.)|(


Frequency response:

Non-monotonically decreasing with frequency

1

9

1 1 1

1 1 1

1 1 1

1

3111

1

3

1

1

1

[ ] h hx y

t u h h ni

Nunx x

n N

N

( ) ( ) ( ) exp{ }( )/

( )/

02

1 2

1 2

)2

cos(3

2

3

1)( u

Nut


An example

8 5 8 8 5 8

8 5 8 8 5 8

8 5 8 8 5 8

1

9

1 1 1

1 1 1

1 1 1

7 7 7 7 7 7

7 7 7 7 7 7

7 7 7 7 7 7

8 5 8 5 8 5

8 5 8 5 8 5

8 5 8 5 8 5

1

9

1 1 1

1 1 1

1 1 1

6 7 6 7 6 7

6 7 6 7 6 7

6 7 6 7 6 7


Image smoothed with 33, 55, 99

and 11 11 box filters


Original Lena imageLena image filtered with

5x5 box filter

Linear Image smoothing techniques Binomial filters [Jahne 1995]

Computes a weighted average of pixels in the window

Less blurring, less noise cleaning for the same size

The family of binomial filters can be defined recursively

The coefficients can be found from (1+x)n

Linear Image smoothing techniques Binomial filters. 1D versions

112

11b

11 bbb *1214

12

1111 bbbbb ***1464116

14

111111 bbbbbbb *****161520156164

16

As size increases, the shape of the filter is closer to a Gaussian one

Linear Image smoothing techniques Binomial filters. 2D versions

121

242

121

16

1

1

2

1

4

1*121

4

12b

14641

41624164

62436246

41624164

14641

256

1

1

4

6

4

1

16

1*14641

16

14b

Linear Image smoothing techniques Binomial filters. Frequency response

1214

12

b )

2cos(

2

1

2

1)( u

Nut

1464116

14

b 2)]

2cos(

2

1

2

1[)( u

Nut

Monotonically decreasing

with frequency

Linear Image smoothing techniques Binomial filters. Example

Original Lena image Lena image filtered

with binomial 5x5 kernel

Lena image filtered

with box filter 5x5

Linear Image smoothing techniques Binomial and box filters. Edge blurring comparison

Linear filters have to compromise smoothing with edge blurring

Step edge

Result of size L box filter

L

Size L binomial

Nonlinear image smoothingThe median filter [Pratt 1991]

Block diagram

2

1N

m

f1

f2

fN

.

.

.

Order samples

f(1)

f(2)

f(N)

.

.

.select

f(m)

median

N is odd

)()3()2()1( Nffff

Nonlinear image smoothingThe median filter

Numerical example

6 7

3 7 8

2 3

4 6 7

2, 3, 3, 4, 6, 7, 7, 7, 8

Nonlinearity


median{ f1 + f2} median{ f1} + median{ f2}.However:

median{ c f } = c median{ f },median{ c + f } = c + median{ f }.

• The filter selects a sample from the window, does not average

• Edges are better preserved than with liner filters

• Best suited for “salt and pepper” noise


Noisy image 5x5 median filtered 5x5 box filter


Optimality Grey level plateau plus noise. Minimize sum of

absolute differences:

Result:

If 51% of samples are correct and 49% outliers, the median still finds the right level!

N

kk gfg

1

||)(

)(321 },,,,{ mN fffffmediang


Caution: points, thin lines and corners are erased by the median filter

Test images

Results of 33 pixel median filter


Cross shaped window can correct some of the problems

Results of the 9 pixel cross shaped window median filter


Implementing the median filter Sorting needs O(N2) comparisons

– Bubble sort– Quick sort– Huang algorithm (based on histogram)– VLSI median


VLSI median block diagram

a

b

Min(a,b)

Max(a,b)

x(1)

x(2)

x(3)

x(4)

x(5)

x(6)

x(7)

x1

x2

x3

x4

x5

x6

x7


Color median filter– There is no natural ordering in 3D (RGB) color

space– Separate filtering on R,G and B components does

not guarantee that the median selects a true sample from the input window

– Vector median filter, defined as the sample minimizing the sum of absolute deviations from all the samples

– Computing the vector median is very time consuming, although several fast algorithms exist


Example of color median filtering

5x5 pixels window Up: original image Down: filtered image

Nonlinear image smoothingThe weighted median filter

The basic idea is to give higher weight to some samples, according to their position with respect to the center of the window

Each sample is given a weight according to its spatial position in the window.

Weights are defined by a weighting mask Weighted samples are ordered as usually The weighted median is the sample in the ordered array such that

neither all smaller samples nor all higher samples can cumulate more than 50% of weights.

If weights are integers, they specify how many times a sample is replicated in the ordered array

Nonlinear image smoothingThe weighted median filter

Numerical example for the weighted median filter

6 7

3 7 8

2 3

4 6 7

2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8

3

1 1

1 1

2 2

2

2

Nonlinear image smoothingThe multi-stage median filter

Better detail preservation

mi = median( Ri ), i =1,2,3,4.

Result = median( m1,m2,m3,m4,m5 )

R1

R2

R3

R4

m5

Nonlinear image smoothingRank-order filters (L filters)

Block diagram

sum

y

f(1)f1

f2

fN

.

.

.

ordering

f(2)

f(N)

.

.

.

a1

a2

aN

weights

y = a1f(1) + a2 f(2)+...+ aN f(N)akk

N

1

1


Some examples

a1=1

Min. . .

. . .

aN=1

Max

. . . . . .

am=1

Median

a1=.5 aN=.5

Mid range. . .

Particular cases of grey level

morphological filters

Percentile

filters (rank selection):

0%,

50%

100%


Optimality

( ) | |( )f f fkr

k

N

1

Minkovski distance

Case r = 1: best estimator is median.Case r = 2, best estimator is arithmetic mean.Case r , best estimator is mid-range.


Alpha-trimmed mean filter

otherwise

QmkQmforQak ,0

),12/(1

= (2Q + 1) / N, defines de degree of averaging

= 1 corresponds to arithmetic mean

= 1 / N corresponds to the median filter

Properties: in between mean and median


Median of absolute differences trimmed mean Better smoothing than the median filter and good edge preservation

},,,,{ 3211 NffffmedianM

NiMfmedianM i ,...,2,1|},{| 12 }||:{ 21 MMffaverageoutput ii

M2 is a robust estimator of the variance


k nearest neighbour (kNN) median filterMedian of the k grey values nearest by rank

to the central pixel Aim: same as above

Kuwahara type filtering Form regions Ri

Compute mean, mi and variance, Si for each region.

Result = mi : mi < mj for all j different from i, i,e. the mean of the most homogeneous region

Matsujama & Nagao

Nonlinear image smoothingSelected area filtering [Nagao 1980]

reg 1 reg 2

reg 3

reg 4

reg 5 reg 6

reg 7reg 8

Nonlinear image smoothingConditional mean

Pixels in a neighbourhood are averaged only if they differ from the central pixel by less than a given threshold:

otherwise

thnmflnkmfiflkh

lnkmflkhnmgL

Lk

L

Ll

,0

|),(),(|,1),(

),,(),(),(

L is a space scale parameter and th is a range scale parameter

Nonlinear image smoothingConditional mean

Example with L=3, th=32

Nonlinear image smoothingBilateral filter [Tomasi 1998]

Space and range are treated in a similar way Space and range similarity is required for the averaged pixels Tomasi and Manduchi [1998] introduced soft weights to penalize the

space and range dissimilarity.

)),(),((),(),( nmflnkmfrlkslkh

k l

k l

lkhK

lnkmflkhK

nmg

),(

,),(),(1

),(

s() and r() are space and range similarity functions (Gaussian functions of the Euclidian distance between their arguments).

Nonlinear image smoothingBilateral filter

The filter can be seen as weighted averaging in the joint space-range space (3D for monochromatic images and 5D – x,y,R,G,B - for colour images)

The vector components are supposed to be properly normalized (divide by variance for example)

The weights are given by:

));K(d()(

}||||

exp{)(2

shs

h

c

c

xxx

xxx

Nonlinear image smoothingBilateral filter

Example of Bilateral filtering Low contrast texture has been removed Yet edges are well preserved

Nonlinear image smoothingMean shift filtering [Comaniciu 1999, 2002]

Mean shift filtering replaces each pixel’s value with the most probable local value, found by a nonparametric probability density estimation method.

The multivariate kernel density estimate obtained in the point x with the kernel K(x) and window radius r is:

For the Epanechnikov kernel, the estimated normalized density gradient is proportional to the mean shift:

dinii R xx ...1}{

n

id r

Knr

f1

1)(ˆ ixxx

)(

2 1)(

)(ˆ)(ˆ

2 xxi

x

xxxx

x

ri Sr n

Mf

f

d

r

S is a sphere of radius r, centered on x and nx is the number of samples inside the sphere

Nonlinear image smoothingMean shift filtering

The mean shift procedure is a gradient ascent method to find local modes (maxima) of the probability density and is guaranteed to converge.

Step1: computation of the mean shift vector Mr(x).

Step2: translation of the window Sr(x) by Mr(x).

Iterations start for each pixel (5D point) and tipically converge in 2-3 steps.


Example1.


Detail of a 24x40

window from the

cameraman image

a) Original data

b) Mean shift paths for some points

c) Filtered data

d) Segmented data


Example 2


Comparison to bilateral filtering Both methods based on simultaneous processing of both the

spatial and range domains While the bilateral filtering uses a static window, the mean shift

window is dynamic, moving in the direction of the maximum increase of the density gradient.

REFERENCES

D. Comaniciu, P. Meer: Mean shift analysis and applications. 7th International Conference on Computer Vision, Kerkyra, Greece, Sept. 1999, 1197-1203.

D. Comaniciu, P. Meer: Mean shift: a robust approach toward feature space analysis. IEEE Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18.

M. Elad: On the origin of bilateral filter and ways to improve it. IEEE trans. on Image Processing Vol. 11, No. 10, October 2002, 1141-1151.

B. Jahne: Digital image processing. Springer Verlag, Berlin 1995. M. Nagao, T. Matsuiama: A structural analysis of complex aerial

photographs. Plenum Press, New York, 1980. W.K. Pratt: Digital image processing. John Wiley and sons, New York

1991 C. Tomasi, R. Manduchi: Bilateral filtering for gray and color images.

Proc. Sixth Int’l. Conf. Computer Vision , Bombay, 839-846.

Documents

Smoothing Techniques in Image Processing Prof. PhD. Vasile Gui Polytechnic University of Timisoara