ELE 488 F06

ELE 488 Fall 2006Image Processing and Transmission

09-28-06

Edge Map

Laplacian

Median Filter

Filtering Images in Frequency Domain

Image Restoration

9/28/06

ELE 488 F06

Gradient, 1st-order Derivatives, and Edges

• Edge: where luminance changes abruptly• For binary image

– Take black pixels with immediate white neighbors as edge pixel

• For continuous-tone image– Find luminance gradient

• partial derivatives along two orthogonal directions• gives the direction with highest rate of changes• If gradient is larger than a threshold => edge

– To represent edge• Edge map ~ edge intensity & directions

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

& R

.Liu

© 2

002)

ELE 488 F06

The Gradient of Function f(x,y)

The gradient of a smooth function f(x,y) at (x,y) is

derivative of f in the x-direction, partial derivative of f w.r.t. x

derivative of f in the y-direction, partial derivative of f w.r.t. y

2-vector pointing in the direction of greatest increase of f fr

.rate of change of f in the direction h f

ELE 488 F06

The Discrete Gradient

The gradient of a real-valued discrete function f(m,n) at (j.k) is

The symmetric rate of change of f in the m-direction at (j,k)

The symmetric rate of change of f in the n-direction at (j,k)

(Note: for simplicity, drop the scale factor of ½ - it can always be reinserted again if needed.)

ELE 488 F06

Averaging GVF

• gradient is computed using high pass filters (discrete differentiation) will amplify noise.

• So average neighboring gradient vectors using a spatial filtering on each of two gradient components.

• The averaging filters and gradient filters are LSI and hence can be applied in any order, i.e.,

• Can compute gradient and average, or• Average pixels and then compute gradient.

Should we average uniformly in all directions or along a selected direction? e.g. along the perpendicular to the direction of the derivative? Perpendicular to the gradient?

ELE 488 F06

Averaging the GVF

Spatial averaging filter perpendicular to direction of discrete derivative

discrete derivative filter in horizontal direction

Combination of the two masks gives single “averaged gradient mask” in horizontal direction.

ELE 488 F06

Common GVF Masks

– Apply both masks to image and combine results to determine magnitude and angle (if desired).

– Note: Prewitt and Sobel operators spatially average the horizontal and vertical differences of 3 local pixels to reduce the effect of noise

0 1

-1 0

-1 0

-1 0

1

1

-1 0 1

-1 0

-2 0

1

2

-1 0 1

1 0

0 -1

-1 -1

0 0

-1

0

1 1 1

-1 -2

0 0

-1

0

1 2 1

Roberts Prewitt Sobel

ELE 488 F06

Averaged GVF and Edges

Scaled & averaged gradient of apple image plotted at every 3rd pixel.

Large gradient magnitude

Edge:Normal to direction of gradient

Small gradient magnitude

No edge

Subjective Issue: How to select the threshold on the magnitude of the gradient to “detect” an edge?

ELE 488 F06

Example: Edge Detection from GVF

Threshold magnitude of the GVF:Above threshold: edge pixelBelow threshold: no edge

ELE 488 F06

Sobel GVF

Sobel

Prewitt

ELE 488 F06

Examples of Edge Detectors

– Quantize edge intensity to 0/1:• set a threshold• white pixel denotes

strong edge

Roberts Prewitt Sobel

UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

ELE 488 F06

Edges and Second Derivatives in 1-D

“edges” are at points of local maximum and minimum slope of f(x). Here at x=1 and x=3.

“edges” are at local maxima and minima of df/dx. Here at x=1 and x=3. Threshold the magnitude of the derivative…gives local region of edge.

“edges” are at points where the second derivative passes through zero. Here at x=1 and x=3.

From 1D to 2D

ELE 488 F06

2D: Derivatives of f(x,y)

ELE 488 F06

Example: Gaussian

ELE 488 F06

Derivatives

Red lines indicate zero level sets

ELE 488 F06

Difference Operators (along rows, columns similar)

Many choices:

More generally:

ELE 488 F06

Averaging Across Rows

ELE 488 F06

Common Difference MasksRoberts Prewitt Sobel

Prewitt and Sobel masks are separable. They spatially average across 3 rows (or columns).

ELE 488 F06

ExampleComparison of Sobel operation with actual derivative of Gaussian.Note scale difference of approx. 5.3The error (after adjusting for scale & the edge effect) is shown below.

ELE 488 F06

Second Differences

Examples:

In this case, change the 00 pixel of the second dm mask to make the result symmetric

ELE 488 F06

Example: Second Difference Sobel

Do the same for the Sobel column difference operator.

ELE 488 F06

Example: 2nd Diff SobelComparison of Sobel operation with actual second derivative of Gaussian.Note scale difference of approx. 28.24The error (after adjusting for scale and edge effect) is shown below.

ELE 488 F06

A 3x3 Discrete Laplacians

First difference. This operator is rotationally symmetric for 90 deg rotations.

Roberts. This operator is rotationally symmetric for 90 deg rotations.

Composite. This operator is rotationally symmetric for 90 deg rotations.

ELE 488 F06

The 5x5 Sobel Laplacian

Sobel. This operator is also rotationally symmetric for 90 deg rotations. As expected, sum of all entries is 0.

ELE 488 F06

Zero Contour of 3x3 Roberts Laplacian

ELE 488 F06

Zero Contour of Sobel Laplacian

ELE 488 F06

Zero Contour of a 17x17 Laplacian

ELE 488 F06

Spatial Masks

Spatial Mask Finite, 2-D, real valued, array.

One element is designated as (0,0).

Used to compute the weighted sum of an input image, where weights are specified by the mask.

Usually has small support 2x2, 3x3, 5x5, 7x7

Examples:

These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.

1/4 1/4

1/4 1/4

0 1

0

1

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

ELE 488 F06

Use Averaging to Suppressing Noise

• Image with additive noise x(m,n) + N(m,n)• Averaging over window of Nw pixels:

v(m,n) = (1/Nw) x(m-k, n-j) + (1/Nw) N(m-k, n-j)

• Noise variance reduced by a factor of Nw SNR improved by a factor of Nw if x(m,n) is constant in local window

• Limit window size to avoid blurring

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

& R

.Liu

© 2

002)

1/4 1/4

1/4 1/4

0 1

0

1

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

ELE 488 F06

Coping with Salt-and-Pepper NoiseMatlab Image Toolbox Guide Fig.10-12, 10-13

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

& R

.Liu

© 2

002)

ELE 488 F06

Median Filtering

• Median of pixel values in a window of size Nw

– Median is middle value of all pixel values if Nw is odd, and is the average of two middle values if Nw is even.

– Need to order data

• “Salt-and-Pepper” noise– Isolated white or black pixels spread randomly over the image– Spatial averaging filter can cause blur

• Median filtering– Take median value over a small window as output

Note: Median { x(m) + y(m) } Median {x(m)} + Median {y(m)} nonlinear

– Commonly used window sizes • 3x3, 5x5, 7x7• 5x5 “+”–shaped window

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

& R

.Liu

© 2

002)

ELE 488 F06

Other Ways of Image Sharpening

• Combine sharpening with histogram equalization

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

© 2

002)

ELE 488 F06

Image Processing

• Pointwise processing– Each output pixel depends only on one input pixel– Histogram modification

• Spatial filtering– Each output pixel may depend on more than one pixels– Masking, linear filtering, edge detection,

• Linear filtering can be carried out in frequency domain• Extending 1D processing in frequency domain to 2D

ELE 488 F06

Signal Processing in Time and Frequency Domain

output signal y[n]

input signal x[n]

Processor

LTI system

time domain: y[n] = Σ h[k] x[n – k] , h[n] impulse response

Fourier transform: X(ω) = Σx[n] e–jnω. = FT{x[n]}

Inverse Fourier transform: x[k] = (1/2π)∫ X(ω) ejnω dω = IFT {X(ω)}

Frequency domain: Y(ω) = X(ω) H(ω),

where Y(ω) = FT{ y[n] } , H(ω) = FT{ h[n] }

ELE 488 F06

Discrete Fourier Transform

FT: X(ω) = Σx[n] e–jnω. IFT: x[k] = (1/2π)∫ X(ω) ejnω dω .

DFT: X[k] = Σ0≤n≤M–1 x[n] e–jn2π/M.

IDFT: x[n] = (1/M)Σ0≤k≤M–1 X[k] e jk2π/M.

Fast Fourier Transform (FFT)

ELE 488 F06

2D Signal Processing in Time and Frequency Domain

output signal y[m,n]

input signal x[m,n]

Processor

LTI system

time domain: y[m,n] = Σ h[k,l] x[m – k ,n – l] , h[m,n] impulse response

Fourier transform: X(ω,λ) = Σx[m,n] e–j(mω+nλ) = FT{x[m,n]}

Inverse Fourier transform: x[k,l] = (1/2π)∫ X(ω) ej(mω+nλ) dωdλ = IFT {X(ω)}

Frequency domain: Y(ω ,λ) = X(ω ,λ) H(ω ,λ),

where Y(ω ,λ) = FT{ y[m,n] } , H(ω ,λ) = FT{ h[m,n] }

ELE 488 F06

2D Discrete Fourier Transform

FT: X(ω ,λ) = Σx[n] e–j(nω +nλ) .

IFT: x[k,l] = (1/2π)2∫ ∫ X(ω ,λ) e –j(nω +nλ) dω dλ .

DFT: X[k,l] = Σ0≤m≤M–1 Σ0≤n≤N–1 x[m,n] e–j(m2π/M+n2π/N) .

IDFT: x[m,n] = (1/MN) Σ0≤k≤M–1Σ0≤n≤N–1 X[k,l] ej(k2π/M+l2π/M) .

Fast Fourier Transform (FFT)

Separable

ELE 488 F06

Review of Random Signals

A 1D random signal u[n] is stationary if its average behavior does not depend on n. It can be characterized by probability density function fu( . ) and power spectral density Φu ( . ).

Prob [ a ≤u[n] < b ] = b

a

fu(x)dx

E{ u[n] } = < u[n] > = average value (expectation) of u[n] =

x fu(x) dx (first moment)

E{ u[n]2 } = < u[n]2 > = mean square value or power of u[n] =

x2 fu(x) dx

The expectation E{ u[n] u[n+k] }, called the autocorrelation function.

Depends on k, not on n, because u is stationary. It is often denoted by ruu[k].

ELE 488 F06

Random Signal (cont)

Power of u[n] in the frequency band (ω1 , ω2 ) is (1/2π)

2

1||

Φu(ω) dω

Φu(ω) is called the power spectral density of signal u. It is the Fourier transform of ruu[k].

Total power = (1/2π)

Φu(ω) dω = E{ u[n]2 }

White noise: Φu(ω) = constant , E{ u[n]2 } = Φu(ω) Given a linear system with frequency response H(ω), the power spectral density

of the output v[n] is related to that of the input u[n] via: Φv (ω) = Φu (ω) |H(ω) |2

The output power is therefore E{ v[n]2 } = (1/2π)

Φu (ω) |H(ω) |2 dω

If input {u[n]} is white, E{ v[n]2 } = E{ u[n]2 } (1/2π)

|H(ω) |2 dω

ELE 488 F06

2D Random Signal

A 2D random signal u[m, n] is stationary if its average behavior does not depend on m and n. It can be characterized by probability density function fu( . ) and power spectral density Φu ( . ).

Prob [ a ≤u[m, n] < b ] = b

a

fu(x)dx

E{ u[m, n] } = average value (expectation) of u[m, n] =

x fu(x) dx

E{ u[m, n]2 } = mean square value or power of u[m, n] =

x2 fu(x) dx

The expectation E{ u[m, n] u[m+j, n+k] }, called the autocorrelation function.

depends only on k, not on n, because u is stationary. It is often denoted by ruu[j, k].

ELE 488 F06

2D Random Signal (cont)Power of u[m, n] in the frequency band ω1 < |ω| < ω2 and λ1 < |λ| < λ2 is

(1/2π)2

2

1||

2

1||

Φu(ω, λ) dω dλ

Φu(ω,λ) is the power spectral density of signal u. It is the Fourier transform of ruu[j, k].

Total power = (1/2π)

Φu(ω, λ) dω dλ = E{ u[m, n]2 }

White noise: Φu(ω, λ) = constant , E{ u[m, n]2 } = Φu(ω, λ) Given a linear system with frequency response H(ω, λ), the power spectral density of the output

v[m, n] is related to that of the input u[m, n] via: Φv (ω, λ) = Φu (ω, λ) |H(ω, λ) |2

Total output power is therefore E{ v[m,n]2 } = (1/2π)

Φu (ω, λ) |H(ω, λ) |2 dω

If input {u[n]} is white, E{ v[m,n]2 } = E{ u[m,n]2 } (1/2π)

|H(ω, λ) |2 dω

ELE 488 F06From Matlab ImageToolbox Documentation pp12-4

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)

Image Restoration

ELE 488 F06

Imperfection in Image Capturing

• Blurring ~ linear spatial-invariant filter model w/ additive noise

• Impulse response h(n1, n2) & H(1, 2)– Point Spread Function (PSF) ~ positive I/O

– If h(n1, n2) = (n1, n2), no blur

– Linear translational motion blur caused by local average along motion direction

– Uniform out-of-focus blur caused by local average in a circular neighborhood

– Atomspheric turbulence blur, etc.

Hu(n1, n2 ) v(n1, n2 )

N(n1, n2 )

otherwise 0

tan and

x if L1

),;,( yx

222

Ly

Lyxh

otherwise 0

x if R1

);,(222

2 RyRyxh

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)

ELE 488 F06

Fourier Transform of PSF for Common Distortions

From Bovik’s Handbook Sec.3.5 Fig.2&3

UMCP ENEE631 Slides (created by M.Wu © 2001)