Lecture 8: Image Enhancement and SpatialFiltering II

Harvey RhodyChester F. Carlson Center for Imaging Science

Rochester Institute of Technologyrhody@cis.rit.edu

October 4, 2005

AbstractBoth linear and nonlinear spatial filtering techniques are employed

for important processing tasks. The design and performance of varioustechniques are compared by examples.

DIP Lecture 8

Median Filters

Median filters may be used when the objective is to achieve noise reduction with a minimum

amount of blurring.

A median filter replaces the pixel at the center of a mask with the median of the set of

pixels under the mask.

Median filters are in the class of order filters. These are nonlinear filters.

The effect of a median filter on a noisy step edge (right) is compared with the effect of a

smoothing filter (left). Note that the smoothing filter tends to distort the edge transition.

DIP Lecture 8 1

Median Filters

Median filters are most effective against impulse (salt & pepper) noise.

Shown below is an impulse doublet on a background of small random values. The

smoothing filters tend to smear and lower the pulses. The median filter with N=3 is

ineffective on doublets. The median filter with N=5 removes doublets.

DIP Lecture 8 2

Median Filters

A comparison of the performance of median filters on salt and pepper noise is illustrated

in the images below.

Original Original with Noise N=3 Median Filter N=5 Median Filter

DIP Lecture 8 3

Sharpening Filters

Sharpening is used to highlight fine detail or enhance detail that has been blurred.

A sharpening filter seeks to emphasize changes. The classic mask for a sharpening filter is

the mask shown below.

W =1

9

-1 -1 -1

-1 8 -1

-1 -1 -1

When the mask is over a region of uniform brightness it has zero output. It has maximum

output when the center pixel differs significantly from the surrounding pixels.

DIP Lecture 8 4

Sharpening Filter

The effect on the logging camp image is shown below. Note that uniform regions, whether

dark or light, have minimum response.

Original After sharpening filter After rectification

DIP Lecture 8 5

Frequency Response of Sharpening Filters

The frequency response of a sharpening filter depends upon the size of the filter. A larger

filter will generally have a sharper response.

19×

-1 -1 -1

-1 8 -1

-1 -1 -1

149×

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 48 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

125×

-1 -1 -1 -1 -1

-1 -1 -1 -1 -1

-1 -1 24 -1 -1

-1 -1 -1 -1 -1

-1 -1 -1 -1 -1

DIP Lecture 8 6

Frequency Response of Sharpening Filters

M=3 M=5 M=7 M=9

DIP Lecture 8 7

Frequency Response of Sharpening Filters

The frequency response along a slice through the origin in the frequency plane is shown

below for several values of M.

DIP Lecture 8 8

High-Boost Filters

A highpass filtered image can be computed as the difference between the original and a

lowpass filtered version.

Highpass = Original− Lowpass

If the original is amplified the result is an image with enhanced high-frequency detail.

Highboost = (A)(Original)− Lowpass

= (A− 1) (Original) + Original− Lowpass

= (A− 1) (Original) + Highpass

When A > 1 part of the original is added back to the highpass output.

This technique is called unsharp masking. It has been used for many years in the printing

and publishing industry.

DIP Lecture 8 9

Unsharp Masking Filters

The filter masks can be modified to directly produce unsharp masking. In the following

use A ≥ 1

19×

-1 -1 -1

-1 9A-1 -1

-1 -1 -1

149×

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 49A-1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1

125×

-1 -1 -1 -1 -1

-1 -1 -1 -1 -1

-1 -1 25A-1 -1 -1

-1 -1 -1 -1 -1

-1 -1 -1 -1 -1

DIP Lecture 8 10

Unsharp Masking Example

With A = 1 the unsharp mask is a HP filter. More of the original image is included as A

is increased. The effect is to sharpen the edges.

Original A = 1.0 A = 1.3 A = 1.5

DIP Lecture 8 11

Feature Detection

Spatial filtering can be used to detect a feature, which is a pattern of pixel values. Consider

a pattern given by the 3× 3 array

f1 f2 f3

f4 f5 f6

f7 f8 f9

A spatial filter of the same size is given by

w1 w2 w3

w4 w5 w6

w7 w8 w9

We want to choose the weights so that the filter response is significantly greater when it

is over a feature compared with the value when it is not.

DIP Lecture 8 12

Feature Detection

Let the response to a feature be

Rf =

9Xi=1

wifi

and the response to another region be

R =9X

i=1

wizi

We can treat R as a random variable. The power of the filter in distinguishing between a

feature pattern and a random background pattern can be analyzed by assuming that the

background is random white noise with mean value µz and variance σ2z. While this is not

always the case in reality, the analysis provides guidance in filter selection.

DIP Lecture 8 13

Feature Detection

In a region of random background the filter output has mean and variance

µR =

9Xi=1

wiµz = µz

9Xi=1

wi

σ2R =

9Xi=1

w2i (zi − µz)

2= σ

2z

9Xi=1

w2i

Define a quality measure for the filter design as the ratio

Q =(Rf − µR)2

σ2R

The quality measure can be maximized by the correct weight selection.

DIP Lecture 8 14

Selecting the Filter Weights

Maximize Q by weight selection by setting

∂Q

∂wi

= 0 for i = 1, . . . , 9

After simplification, this leads to the system of equations

(fi − µz)

9Xj=1

w2j = wi

9Xj=1

(fj − µz)wj for i = 1 . . . , 9

The equations are satisfied by selecting wi = α(fi − µz), where α is a constant that

determines the filter gain.

A filter whose weights match the pattern is called a matched filter. Matched filters are

used in classical signal detection applications such as radar and communication systems.

DIP Lecture 8 15

Point Detection

A mask to detect isolated points can be constructed by a feature pattern that is large in

the center and small in the surrounding pixels. We can use

F =

f1 f2 f3

f4 f5 f6

f7 f8 f9

=

−1 −1 −1

−1 8 −1

−1 −1 −1

This pattern has zero response in flat regions and maximum response when the center

point differs significantly from the surround. The filter output depends only upon the

change in level and not the absolute level.

A point is detected at the center pixel if |R| > T where T is a threshold setting. The

strength of isolated points that are detected can be controlled with the setting of T .

DIP Lecture 8 16

Line Detection

Horizontal, vertical and slanted lines can be detected with the filters shown below.

−1 −1 −1

2 2 2

−1 −1 −1

−1 −1 2

−1 2 −1

2 −1 −1

−1 2 −1

−1 2 −1

−1 2 −1

2 −1 −1

−1 2 −1

−1 −1 −2

Horizontal 45 degrees Vertical -45 degrees

The detection of horizontal and vertical lines in an image is illustrated in the images on

the following slide.

DIP Lecture 8 17

Line Detection

Horizontal Lines Vertical Lines

DIP Lecture 8 18

Edge Detection

Edge detection uses discrete operations that approximate gradient calculation. For a

continuous function f(x, y) the gradient is a vector

∇f =

264

∂f∂x

∂f∂y

375

The magnitude of the gradient is

|∇f | ="�

∂f

∂x

�2

+

�∂f

∂y

�2#1/2

The mathematical formula for the gradient is the inspiration for a calculation rule for

discrete arrays.

The approximation is done with finite differences.

DIP Lecture 8 19

Edge Detection

Consider a small section of an image represented by the 3× 3 array

z1 z2 z3

z4 z5 z6

z7 z8 z9

The gradient could be approximated by

∇z ≈�

z6 − z4

z8 − z2

�

The magnitude can be computed by

|∇z| ≈h(z6 − z4)

2+ (z8 − z2)

2i1/2

A calculation that provides a similar answer, and is faster to compute, is

|∇z| ≈ |z6 − z4|+ |z8 − z2|

DIP Lecture 8 20

Edge Detection Operators

Edge detection can be carried out by convolving a pair of linear filters with an image.

These filters provide vertical and horizontal gradient detection.

Sy =

−1

0

1

Sx = −1 0 1

The gradient vector is

G =

�Gx

Gy

�and the gradient magnitude is

G = |G| =q

G2x + G2

y

The gradient magnitude is often approximated in calculations with

G ≈ |Gx|+ |Gy|

DIP Lecture 8 21

Edge Detection Operators

It is common to use 3× 3 operators for edge detection. The Prewitt detectors are

Sy =

−1 −1 −1

0 0 0

1 1 1

Sx =

−1 0 1

−1 0 1

−1 0 1

These provide some averaging in the direction perpendicular to the gradient measurment.

A similar set of masks are the Sobel operators:

Sy =

−1 −2 −1

0 0 0

1 2 1

Sx =

−1 0 1

−2 0 2

−1 0 1

The Sobel operators give more weight to the center elements in the gradient calculation.

DIP Lecture 8 22

Gradient Images

A Ag

Ax Ay

DIP Lecture 8 23

Gradient PlotsHorizontal Gradient, row 386 Vertical Gradient, col 396

Top=original, Bottom=Gx Top=original, Bottom=Gy

DIP Lecture 8 24

Laplacian

The Laplacian is an operator that computes an isotropic second derivative of a grayscale

image. If f(x, y) is an image, then the Laplacian is

∇2f =

∂2f

∂x2+

∂2f

∂y2

This can be approximated with finite differences as

∂2f

∂x2= f(x + 1, y)− 2f(x, y) + f(x− 1, y)

∂2f

∂y2= f(x, y + 1)− 2f(x, y) + f(x, y − 1)

∇2f = f(x + 1, y) + f(x− 1, y) + f(x, y + 1) + f(x, y − 1)− 4f(x, y)

DIP Lecture 8 25

Laplacian Masks

The Laplacian is implemented by a number of different filter masks that are inspired by

the analysis above.

0 1 0

1 -4 1

0 1 0

1 1 1

1 -8 1

1 1 1

1 0 1

0 -4 0

1 0 1

It is common to use the negative of any of these masks as well.

DIP Lecture 8 26

Example

DIP Lecture 8 27

Combination Gaussian LPF + Laplacian

The Laplacian is a HPF and therefore emphasizes noise. It is often useful to first pass the

image through a LPF followed by a Laplacian to locate edge detail.

A gaussian LPF is described by the function

h(r) = Ke− r2

2σ2 where r2 = x2 + y2

The Laplacian of this function is

∇2h(r) = K

"r2 − σ2

σ4

#e− r2

2σ2

The surfaces are shown to the right for the

case K = −1 and σ = 0.3.

The LoG filter is often called the “Mexican

hat”.

DIP Lecture 8 28

LoG Filter

The LoG accomplishes both LPF and Laplacian operations in one convolution. A discrete

mask that approximates the LoG is the 5× 5 array

0 0 -1 0 0

0 -1 -2 -1 0

-1 -2 16 -2 -1

0 -1 -2 -1 0

0 0 -1 0 0

DIP Lecture 8 29

Example: Finding Edges and Corners

The LoG is particularly useful in locating edges and corners. Consider the results from

processing the artificial image of shapes.

Shapes After LoG Magnitude of LoG

DIP Lecture 8 30

Example (continued)

The LoG response is greatest at the edges and particularly at the corners of objects.

LoG Magnitude (Image Section) Largest LoG values (whole image)

Detected Corners Edges and highlighted corners

DIP Lecture 8 31

Example: Aerial Image

Highlight prominent points of interest in an aerial image. Useful in image registration.

DIP Lecture 8 32

Step 1: LoG Filter

Original Image Magnitude of LoG image

Note: It is important that the image be in integer or floating point format to avoid overflow

effects when filtering.

DIP Lecture 8 33

Step 2: Select Threshold

Pixel Values Points above threshold in LoG image

The threshold was set at T = 1200. Image points were dilated 5× 5 for visual emphasis.

DIP Lecture 8 34

Example Results

Detected Points Zoom on region in upper center

Detected points tend to highlight features such as building corners.

This is the basic idea behind the point location step in Karl Walli’s registration algorithm

(MS Thesis, 2003).

DIP Lecture 8 35