ELE 488 F06

ELE 488 Fall 2006Image Processing and Transmission

09-26-06

Linear 2-D Image Filtering

1-D discrete convolution 2-D discrete convolution 2-D spatial masks Mask filtering Mask filtering and 2-D convolution

Spatial Averaging, Blurring, Image Sharpening

Edge Map - gradient

9/26/06

ELE 488 F06

Signal Processing

output signal y[n]

input signal x[n]

y[n] = Σ h[k] x[n – k] , convolution

Processor

LTI system

Example: h[ ] [ 0.25 0.5 0.25 ],

y[n] = 0.25 x[n–1] + 0.5 x[n] + 0.25 x[n–1]

weighted sum of input

Commutative: A * B = B * A

Associative: (A * B ) * C = A * (B * C)

Distributive: A * (B + C) = A * B + A * C

ELE 488 F06

2D Signal Processing (image)

0.250.25

0.250.25

output signal y[m,n]

input signal x[m,n]

y[m,n] = Σ h[k, j] x[m – k, n – j] , 2D convolution

Processor

LSI system

Example: h[ ]:

y[m,n] = 0.25 x[m,n] + 0.25 x[m–1, n] + 0.25 x[m, n–1]

+ 0.25 x[m–1, n–1]

weighted sum of 4 pixels

ELE 488 F06

-1 0 10 1

-1 0 1 0 -1

0 -1 0

0

1

0 1

-1 0

1 0 -1

1 0 -1

1 0 -1

0 -1 0

0

1

0 1

-1 0

1 0 -1

1 0 -1

1 0 -1

0 -1 0

0

1

0 1

-1 0 1 0 -1

0 -1 0

0

1

0 1

-1 0 1 0 -1

0 -1 0

0

1

1 0 -1

Rotate 180

Do same for next row

xy

h

Example: y[m,n] = Σ h[k, j] x[m – k, n – j]

ELE 488 F06

Spatial Masks

Spatial Mask

Finite, 2-D, real valued, array.

One element is designated as (0,0).

Used to compute the weighted sum of an input image, where weights are specified by the mask.

Usually has small support 2x2, 3x3, 5x5, 7x7

Examples:

These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.

1/4 1/4

1/4 1/4

0 1

0

1

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

ELE 488 F06

Applying a 2X2 Spatial Mask to an Image (1)

1/4 1/4

1/4 1/4

Input Image Output Image

1/4 1/4

1/4 1/4

ELE 488 F06

Applying a 2X2 Spatial Mask to an Image (2)

1/4 1/4

1/4 1/4

Input Image Output Image

Shift mask over input image = Shift input image under mask

1/4 1/4

1/4 1/4

ELE 488 F06

Applying a 2X2 Spatial Mask to an Image (3)

1/4 1/4

1/4 1/4

Input Image Output Image1/4 1/4

1/4 1/4

ELE 488 F06

Applying a 2X2 Spatial Mask to an Image (4)

1/4 1/4

1/4 1/4

Input Image Output Image1/4 1/4

1/4 1/4

Edge effect: what to do?

ELE 488 F06

Applying a 2X2 Spatial Mask to an Image (5)

1/4 1/4

1/4 1/4

Input Image Output Image1/4 1/4

1/4 1/4

ELE 488 F06

Applying a 3X3 Spatial Mask to an Image (1)

Input Image Output Image

-1/4 -1/4

-1/4

-1/4

1

0 0

00

-1/4 -1/4

-1/4

-1/4

1

0 0

00mask

ELE 488 F06

Applying a 3X3 Spatial Mask to an Image (2)

Input Image Output Image

-1/4 -1/4

-1/4

-1/4

1

0 0

00

-1/4 -1/4

-1/4

-1/4

1

0 0

00mask

ELE 488 F06

Applying a 3X3 Spatial Mask to an Image (3)

Input Image Output Image

-1/4 -1/4

-1/4

-1/4

1

0 0

00

-1/4 -1/4

-1/4

-1/4

1

0 0

00mask

ELE 488 F06

Spatial Masks

Spatial Mask

Finite, 2-D, real valued, array.

One element is designated as (0,0).

Used to compute the weighted sum of an input image, where weights are specified by the mask.

Usually has small support 2x2, 3x3, 5x5, 7x7

Examples:

These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.

1/4 1/4

1/4 1/4

0 1

0

1

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

ELE 488 F06

Spatial Averaging Masks

• For softening, noise smoothing, LPF before subsampling(anti-aliasing), etc.

1/4 1/4

1/4 1/4

0 1

0

1

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)

ELE 488 F06

Spatial Averaging Masks

1/4 1/4

1/4 1/4

0 1

0

1

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)

ELE 488 F06

Spatial Averaging Masks

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)

ELE 488 F06

Spatial Averaging Masks

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)

ELE 488 F06

Spatial Averaging Masks

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)

ELE 488 F06

The Equations

Let x(m,n) denote the input image, M(m,n) denote the mask and y(m,n) denote the output image. Then for each pixel (m,n) in the output image:

where the indices i and j range over the extent of the mask. This corresponds to shifting the image under the mask. This equation assumes values are provided in the edge regions.

1/4 1/4

1/4 1/4

0 1

0

1

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

U1

L1i

U2

L2j

j)M(i,j)ni,x(mn)y(m,

ELE 488 F06

The Equations, cont.

U1m

L1mi

U2m

L2nj

n)-jm,-M(ij)x(i,n)y(m,

Alternatively we can shift the mask over the image. In this case, for each pixel (m,n) in the output image:

If we pad the input image and the mask with zeros so that they are both of infinite extent, then we can write:

j)M(i,j)ni,x(mn)jm,M(ij)x(i,n)y(m,-i j-i j

ELE 488 F06

Point Spread Function

• For linear shift invariant (LSI) processing, the impulse response h[m,n] is called the point spread function :

• Delta function (point) in, point spread function out

ELE 488 F06

2-D Masking = Convolution with a PSF

• Mask Filter:

• Convolution:

U1

L1i

U2

L2j

U1

L1i

U2

L2j

n)-jm,-M(ij)x(i,j)M(i,j)ni,x(mn)y(m,

U1

L1i

U2

L2j

U1

L1i

U2

L2j

j)-ni,-h(mj)x(i,j)h(i,j)-ni,-x(mn)y(m,

Applying a mask filter M to an image is equivalent to convolution with the PSF h with: M(I, j) = h(– i, – j)

ELE 488 F06

The transformation is equivalent to a rotation by 180 degrees. Or flip on both indeces.

PSF to Mask

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0

-1 0

-2 0

1

2

-1 0 1

1 0

2 0

-1

-2

1 0 -1

-1 0 1

1 0 -1

0 1

-1 0

0 -1

1 0

Notation:

Hence:

ELE 488 F06

Filtering an Image with a Mask

Filtering an image with a mask is a linear shift-invariant operation. Hence:

Masks can be applied in any order

Masks applied sequentially can first be combined into one mask. Exercise:

-1 0 10 1

-1 0 1 0 -1

0 -1 0

0

1

ELE 488 F06

LPF, HPF, & BPF

• Spatial averaging filters (usually positive entries in mask) are Low Pass Filters (LPF)

• HPF and BPF can be constructed from LPF– High-pass filter

• hHP(m,n) = (m,n) – hLP(m,n)

• Useful in edge extraction and image sharpening

– Band-pass filter• hBP(m,n) = hL2(m,n) – hL1(m,n)

• Useful in edge enhancement• Also good for high-pass tasks in the presence of noise

– avoid amplifying high-frequency noise

ELE 488 F06

Example: Image Sharpening

B(m,n) = A(m,n) +a A(m,n) x M(m,n)

Original circuit image is from Matlab Image Toolbox.

0 -¼

-¼ 1

-1 0 1

-1

0

1

0

-¼

0 -¼ 0

ELE 488 F06

Image Sharpening: Edge Enhancement

Use LPF to generate HPF

– Subtract a low pass filtered image ILP from the original I0

– HPF extracts edges and transitions

Enhance edges

0 20 40 60 80 1000

10

20

30

40

50

0 20 40 60 80 1000

10

20

30

40

50

0 20 40 60 80 100-10

-5

0

5

10

0 20 40 60 80 1000

10

20

30

40

50

Edge

I0

IHP

ILP

IS

ELE 488 F06

Example of Image SharpeningOriginal moon image is from Matlab Image Toolbox.UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

ELE 488 F06

Gradient, 1st-order Derivatives, and Edges

• Edge: where luminance changes abruptly• For binary image

– Take black pixels with immediate white neighbors as edge pixel

• For continuous-tone image– Find luminance gradient

• partial derivatives along two orthogonal directions

• gives the direction with highest rate of changes

• If gradient is larger than a threshold => edge

– To represent edge• Edge map ~ edge intensity & directions

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

ELE 488 F06

The Gradient of Function f(x,y)

The gradient of a smooth function f(x,y) at (x,y) is

derivative of f in the x-direction, partial derivative of f w.r.t. x

derivative of f in the y-direction, partial derivative of f w.r.t. y

2-vector pointing in the direction of greatest increase of f fr

.

rate of change of f in the direction h f

ELE 488 F06

Example

ELE 488 F06

Gradient of Example Function

Gradient is a vector field:a 2-vector associated with each point (x,y)

ELE 488 F06

Gradient of Discrete Function (Symmetric Difference)

]j[nΔ

fΔ

The symmetric difference of a real-valued discrete function f [n] at the point j is

The symmetric rate of change of f around the point j

)1]f[j1][jf(2

1]j[

nΔ

fΔ

ELE 488 F06

The Discrete Gradient

The gradient of a real-valued discrete function f(m,n) at (j.k) is

The symmetric rate of change of f in the m-direction at (j,k)

The symmetric rate of change of f in the n-direction at (j,k)

(Note: for simplicity, drop the scale factor of ½ - it can always be reinserted again if needed.)

ELE 488 F06

The Gradient Vector Field (GVF)

The gradient at pixel is the 2 vector

The gradient assigns a 2-vector at each pixel. This is called the gradient vector field.

Illustration of the gradient vector field showing a scaled version of the gradient at every 3rd pixel. The red lines illustrate the direction and magnitude of the gradient vector at each pixel.

Note: the gradient points in the direction of increasing image values.

ELE 488 F06

The Discrete Gradient

Filtering with the mask[-1 0 1] yields:

Filtering with the mask[-1 0 1]T yields:

The gradient can be computed by applying two LSI filters

mΔ

fΔ

nΔ

fΔ

ELE 488 F06

Averaging GVF

• gradient is computed using high pass filters (discrete differentiation) will amplify noise.

• So average neighboring gradient vectors using a spatial filtering on each of two gradient components.

• The averaging filters and gradient filters are LSI and hence can be applied in any order, i.e.,

• Can compute gradient and average, or• Average pixels and then compute gradient.

Should we average uniformly in all directions or along a selected direction? e.g. along the perpendicular to the direction of the derivative? Perpendicular to the gradient?

ELE 488 F06

Averaging the GVF

Spatial averaging filter perpendicular to direction of discrete derivative

discrete derivative filter in horizontal direction

Combination of the two masks gives single “averaged gradient mask” in horizontal direction.

ELE 488 F06

Common GVF Masks

– Apply both masks to image and combine results to determine magnitude and angle (if desired).

– Note: Prewitt and Sobel operators spatially average the horizontal and vertical differences of 3 local pixels to reduce the effect of noise

0 1

-1 0

-1 0

-1 0

1

1

-1 0 1

-1 0

-2 0

1

2

-1 0 1

1 0

0 -1

-1 -1

0 0

-1

0

1 1 1

-1 -2

0 0

-1

0

1 2 1

Roberts Prewitt Sobel

ELE 488 F06

Averaged GVF and Edges

Scaled & averaged gradient of apple image plotted at every 3rd pixel.

Large gradient magnitude

Edge:Normal to direction of gradient

Small gradient magnitude

No edge

Subjective Issue: How to select the threshold on the magnitude of the gradient to “detect” an edge?

ELE 488 F06

Edge Detection: Summary

• Edge: pixel locations of abrupt intensity change• For binary image – black pixels with immediate white

neighbors • For continuous-tone image, measure gradient vector

– Along two orthogonal directions ~ usually horizontal and verticalgx = L / x, gy= L / y

– Magnitude of gradient vectorg(m,n) 2 = gx(m,n) 2 + gy(m,n) 2

OR g(m,n) = |gx(m,n) | + |gy(m,n)| (preferred in hardware implement.)

– Direction of gradient vector tan –1 [ gy(m,n) / gx(m,n) ]

• Characterizing edges in an image– (binary) Edge map: specify “edge point” locations with g(m,n) > thresh.– Edge intensity map: specify gradient magnitude at each pixel– Edge direction map: specify directions

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

ELE 488 F06

Example: Edge Detection from GVF

Threshold magnitude of the GVF:Above threshold: edge pixelBelow threshold: no edge

ELE 488 F06

Sobel GVF

Sobel

Prewitt

ELE 488 F06

Example: Edge Detectors Based on the GVF

Roberts Prewitt Sobel

Threshold the magnitude of the GVF:Above threshold – pixel is on an “edge”Below threshold – no “edge”

ELE 488 F06

Examples of

Edge Detectors

– Quantize edge intensity to 0/1:• set a threshold• white pixel denotes

strong edge

Roberts Prewitt Sobel

UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

ELE 488 F06

Example Edges in a Grayscale Image

Line 1 Line 2

Line 3 Line 4

Line 1

Line 2

Line 3

Line 4

Close up

ELE 488 F06

Common Gradient Operators for Edge Detection

– Move the operators across the image and take the inner products• Magnitude of gradient vector g(m,n) 2 = gx(m,n) 2 + gy(m,n) 2

• Direction of gradient vector tan –1 [ gy(m,n) / gx(m,n) ]

– Gradient operator is HPF in nature ~ could amplify noise• Prewitt and Sobel operators compute horizontal and vertical differences of

local sum to reduce the effect of noise

0 1

-1 0

-1 0

-1 0

1

1

-1 0 1

-1 0

-2 0

1

2

-1 0 1

1 0

0 -1

-1 -1

0 0

-1

0

1 1 1

-1 -2

0 0

-1

0

1 2 1

Roberts

H1(m,n)

H2(m,n)

Prewitt Sobel

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

ELE 488 F06

Suppressing Noise via Spatial Averaging

• Image with iid noise y(m,n) = x(m,n) + N(m,n)• Averaged version

v(m,n) = (1/Nw) x(m-k, n-l) + (1/Nw) N(m-k, n-l)

• Noise variance reduced by a factor of Nw

– Nw ~ # of pixels in the averaging window

• SNR improved by a factor of Nw if x(m,n) is constant in local window

• Window size is limited to avoid blurringUM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

ELE 488 F06

Coping with Salt-and-Pepper NoiseMatlab Image Toolbox Guide Fig.10-12, 10-13

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

ELE 488 F06

Median Filtering

• Salt-and-Pepper noise– Isolated white/black pixels spread randomly over the image

– Spatial averaging filter may incur blurred output

• Median filtering– Take median value over a small window as output ~ nonlinear

• Median{ x(m) + y(m) } Median{x(m)} + Median{y(m)}

– Odd window size is commonly used• 3x3, 5x5, 7x7

• 5-pixel “+”-shaped window

– Even-sized window ~ take the average of two middle values as output

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

ELE 488 F06

Other Variations of Image Sharpening

• High boost filter (Gonzalez-Woods 2/e pp132 & pp188)

I0 ILP IHP = I0 – ILP I1 = (b-1) I0 + IHP

– Equiv. to high pass filtering for b=1

– Amplify or suppress original image pixel values when b2

• Combine sharpening with histogram equalization

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

2)