Faculty Orientation Workshop on BE (E&TC/Elex) Revised Syllabus 2015 Course

Under the aegis of BoS-E&TC, SPPU, Pune

Unit 4: Image Segmentation

Digital Image And Video Processing (Elective-I)

Dr. Ashwini M. Deshpande

Cummins College of Engineering for Women

Dept. of E&Tc Engg.

2

Topics to be covered Topic to be referred

from

Pixel classification, Bi-level thresholding, Multi-level

thresholding, Adaptive thresholding, Otsu’s method

Sec. 10.3

Edge detection-

First order derivative- Prewitt and Sobel,

Second order derivative- LoG, DoG,

Canny

Sec. 10.1

Edge linking, Hough Transform Sec. 10.2

Region growing, Region splitting and merging Sec. 10.4

Morphological operators- Dilation, Erosion, Opening,

Closing, Hit or Miss transform, Boundary extraction

Sec. 9.2, 9.3, 9.4, 9.5.1

Thinning, Thickening, Skeletons Sec. 9.5.5, 9.5.6, 9.5.7

T1. Gonzalez and Woods, "Digital Image Processing", Pearson Education,

(3rd Edition)

Contents (8 Hrs)

33

Introduction

◼ Segmentation is to to partition an image intoits constituent regions or objects that stronglycorrelate

◼ Important step in automatic pictorial patternrecognition and scene analysis

4

Segmentation Examples

55

Principal approaches

◼ Segmentation algorithms generally are based onone of 2 basic properties of intensity values-◼ Discontinuity : to partition an image based on sharp

changes in intensity

◼ Similarity : to partition an image into regions that are similar according to a set of predefined criteria

66

Detection of Discontinuities

◼ To detect the three basic types of gray-level discontinuities◼ points, lines, edges

◼ The common way is to run a mask through the image

=

=+++=9

1

992211 ...i

ii zwzwzwzwR

77

Point Detection

◼ Point (isolated) detection can be achieved simply using the mask above

◼ A point can be detected at the location on which the mask is centered. If, |R| T

◼ where ◼ T is a nonnegative threshold ◼ R is the sum of products of the coefficients with the gray

levels contained in the region encompassed by the mark.

❑ Points are detected at those pixels in the subsequent filtered image that are above a set threshold

8

Point Detection

X-ray image of

a turbine blade

Result of point

detection

Result of

thresholding

99

Line Detection

◼ Horizontal mask will result with max response when a line passed through the middle row of the mask with a constant background. Similar idea is used with other masks.

1010

Line Detection

◼ Apply every mask on the image◼ Let R1, R2, R3, R4 denotes the response of

the horizontal, +45 degree, vertical and-45 degree masks, respectively.

◼ If, at a certain point in the image|Ri| > |Rj|,

◼ For all ji, that point is said to be morelikely associated with a line in thedirection of mask i.

1111

Example

Binary image of a

wire bond mask

After

processing

with -45°line

detector

Result of

thresholding

filtering result

• Figures from Gonzalez

1212

Edge Detection

◼ An edge is a set of connected pixelsthat lie on the boundary between tworegions◼ first-order derivative (Gradient

operator)

◼ second-order derivative (Laplacianoperator)

13

Characterizing edges

◼ An edge is a place of rapid change inthe image intensity function

imageintensity function

(along horizontal scanline) first derivative

edges correspond to

extrema of derivative

1414

Ideal and Ramp Edges

Derivatives of 1-D Digital Functions

15© 1992-2008 R.C. Gonzalez & R.E. Woods

1616

First and Second derivatives

the signs of the derivatives would be reversed for an edge that transitions from light to dark

1717

Second derivatives

◼ Produces 2 values for every edge in animage (an undesirable feature)

◼ An imaginary straight line joining theextreme positive and negative values ofthe second derivative would cross zeronear the midpoint of the edge. (zero-crossing property)

◼ Useful for locating the centers of thickedges

1818

Gradient Operator

◼ first derivatives are implemented using the magnitude of the gradient.

=

=

y

fx

f

G

G

y

xf

21

22

21

22 ][)f(

+

=

+==

y

f

x

f

GGmagf yx

yx GGf +

commonly approx.

19

◼ The direction of this gradient:

**correction (Gy/Gx)

= −

y

x

G

Gyx 1tan),(

Gradient Operator

Gradient and Edge

20© 1992-2008 R.C. Gonzalez & R.E. Woods

21

Roberts cross-gradient operators

Prewitt operators

Sobel operators

Gradient Masks

2222

Diagonal edges with Prewitt and Sobel masks

2323

Example

2424

Example

2525

Example

2626

Second Order Derivative Methods - Laplacian

2

2

2

22 ),(),(

y

yxf

x

yxff

+

=(linear operator)

Laplacian operator

)],(4)1,()1,(

),1(),1([2

yxfyxfyxf

yxfyxff

−−+++

−++=

2727

Laplacian of Gaussian (LoG)

◼ Laplacian combined with smoothing to find edges via zero-crossing.

2

2

2)(

r

erh−

−=

where r2 = x2+y2, and is the standard deviation

2

2

24

222 )(

r

er

rh−

−−=

2828

Mexican hat

2929

Example

a). Original imageb). Sobel Gradientc). Spatial Gaussian smoothing functiond). Laplacian maske). LoGf). Threshold LoGg). Zero crossing

30

Second Order Derivative Methods -Difference of Gaussian - DoG

◼ LoG requires large computation time for a large edge detector mask

◼ To reduce computational requirements, approximate the LoG by the difference of two LoG – the DoG

2

2

)2

22(

2

1

)2

22(

22),(

22

21

yxyx

eeyxDoG

+−

+−

−=

31

Canny Edge Detector

Original image

Smoothing by Gaussian convolution

Differential operators along x and y axis

Non-maximum suppressionfinds peaks in the image gradient

Hysteresis thresholding locates edge strings

Edge map

32

Canny Edge Detector Example

original image vertical edges horizontal edges

norm of the gradient after thresholding after thinning

MATLAB: edge(image, ‘canny’)

33

➢Approaches:-Thresholding

- Region growing technique

- Region splitting and merging technique

Similarity based image Segmentation

34

Thresholding

◼Thresholding is usually the first step in any segmentation approach

◼We have talked about simple single value thresholding already

◼Single value thresholding can be given mathematically as follows:

=

Tyxfif

Tyxfifyxg

),( 0

),( 1),(

3535

Thresholdingimage with dark background and a light object

image with dark background and two light objects

36

Problems With Single Value Thresholding

◼Single value thresholding only works forbimodal histograms

◼Images with other kinds of histogramsneed more than a single threshold

Multi-level thresholding

Bi-level thresholding

3737

Multi-level thresholding◼ A point (x,y) belongs-

◼ to an object class if T1 < f(x,y) T2

◼ to another object class if f(x,y) > T2

◼ to background if f(x,y) T1

◼ T depends on ◼ only f(x,y) : only on gray-level values Global

threshold

◼ both f(x,y) and p(x,y) : on gray-level values and its neighbors Local threshold

◼ f(x,y), p(x,y) and (x,y) : on gray-level values, its neighbors and pixel location Adaptive threshold

38

Basic Global Thresholding

◼Based on the histogram of an image

◼Partition the image histogram using a single global threshold

◼The success of this technique very strongly depends on how well the histogram can be partitioned

3939

Basic Global Thresholding: Example

generate binary image

use T midway between the max and min gray levels-

4040

Basic Global Thresholding Algorithm

◼ Based on visual inspection of histogram1. Select an initial estimate for T.2. Segment the image using T. This will produce two

groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values T

3. Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2

4. Compute a new threshold value5. T = 0.5 (1 + 2)6. Repeat steps 2 through 4 until the difference in T in

successive iterations is smaller than a predefined parameter To.

4141

Basic Adaptive Thresholding

◼ Subdivide original image into small areas.

◼ Utilize a different threshold to segmenteach subimages.

◼ Since the threshold used for each pixeldepends on the location of the pixel interms of the subimages, this type ofthresholding is adaptive.

4242

The Role of Illumination

f(x,y) = i(x,y) r(x,y)

a) computer generated reflectance functionb) histogram of reflectance functionc) computer generated illumination function (poor)d) product of a) and c)e) histogram of product image

4343

Example : Adaptive Thresholding

4444

Further subdivisiona). Properly and improperly segmented subimages from previous example b)-c). corresponding histogramsd). further subdivision of the improperly segmented subimage.e). histogram of small subimage at topf). result of adaptively segmenting d).

45

Automatic thresholding method

◼ Otsu’s method

46

Optimal global thresholding◼ This method treats pixel values as probability density

functions. ◼ The goal of this method is to minimize the probability

of misclassifying pixels as either object or background. ◼ There are two kinds of error:

◼ mislabeling an object pixel as background, and◼ mislabeling a background pixel as object.

47

Otsu’s Thresholding Method

◼ Based on a very simple idea: Find the threshold thatminimizes the weighted within-class variance.

◼ This turns out to be the same as maximizing thebetween-class variance.

◼ Operates directly on the gray level histogram [e.g.256 numbers, P(i)], so it’s fast (once the histogram iscomputed).

◼ Basic idea is that well-thresholded classes should bedistinct w.r.t. intensity values and conversely athreshold giving the best separation between classeswould be the best (optimum) threshold.

(1979)

48

Matlab function for Otsu’s method

◼ function level = graythresh(I)◼ GRAYTHRESH Compute global thresholdusing Otsu's method. Level is a normalized intensity value that lies in the range [0, 1].

49

◼ Edges and thresholds sometimes do not give good results for segmentation.

◼ Region-based segmentation is based on the connectivity of similar pixels in a region.◼ Each region must be uniform.

◼ Connectivity of the pixels within the region is very important.

◼ There are two main approaches to region-based segmentation: region growing and region splitting.

Region based segmentation

Region Growing

◼ Region-growing approaches exploit the fact that pixels which are close together have similar gray values.

◼ Algorithm-

51

◼ Let R represent the entire image region.

◼ Segmentation is a process that partitions R into subregions, R1,R2,…,Rn, such that,

RRi

n

i=

=1 (a)

jijiRR ji = , and allfor (c)

niRi ,...,2,1 region, connected a is (b) =

niRP i ,...,2,1for TRUE)( (d) ==

jiji RRRRP and regionsadjacent any for FALSE)( (e) =

Region Growing- Properties

• Drawbacks: large execution time, selection of homogeneity property, selection of seed.

5252

Region Growing

criteria:1. The absolute gray-

level difference between any pixel and the seed has to be less than 65

2. The pixel has to be 8-connected to at least one pixel in that region (if more, the regions are merged)

53

Region Splitting◼ The opposite approach to region growing is

region splitting.◼ It is a top-down approach and it starts with

the assumption that the entire image ishomogeneous

◼ If not, then the method requires that theregion be split into two regions.

◼ If this is not true, the image is split into foursub images

◼ This splitting procedure is repeatedrecursively until we split the image intohomogeneous (uniform) regions

54

Region Splitting◼ One method to divide a region is to use a quadtree

structure.

◼ Quadtree: a tree in which nodes have exactly fourdescendants.

55

Split

◼ Splitting techniques disadvantage- they create regions that may be adjacent and homogeneous, but not merged.

56

◼ Opposite to region split-bottom-up method

- Remove child nodes if parent node satisfy the desired property.

- Image size must be in integer powers of 2.

- If most of the homogeneous regions are small, then split technique is inferior to merge technique in terms of time requirement and vice versa.

- Split and Merge: If no priori knowledge is available start somewhere at the middle level.

Region Merging

Region Merging

Region Splitting and Merging

59

Applications

◼ 3D – Imaging

◼ Several applications in the field of MRI

60

Edge linking and Boundary detection

◼ Local processing

◼ Global processing

61

◼ Two properties of edge points are useful for edgelinking:◼ the strength (or magnitude) of the detected edge points

◼ their directions (determined from gradient directions)

◼ Adjacent edge points with similar magnitude anddirection are linked.

◼ For example, an edge pixel with coordinates (x0,y0) in apredefined neighborhood of (x,y) is similar to the pixelat (x,y) if,

Local processing

62

Local processing example

63

Global Processing

◼ Hough Transform can be used todetermine whether points lie on a curveof a specified shape

64

Hough Transform – Line Detection

◼ Consider the slope-intercept equationof line:

◼ Rewrite the equation as follows:

(a, b are constants, x is a variable, y is a function of x)

(now, x, y are constants, a is a variable, b is a function of a)

P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959

65

◼ Hough transform: a way of finding edge points in an image that lie along a straight line.

◼ Example: xy-plane v.s. ab-plane (parameter space)baxy ii +=

Hough transform

66

Properties in slope-intercept space

◼ Each point (xi , yi) defines a line in thea − b space (parameter space).

◼ Points lying on the same line in the x − yspace, define lines in the parameterspace which all intersect at the samepoint.

◼ The coordinates of the point ofintersection define the parameters ofthe line in the x − y space.

67

HT Algorithm

1. Quantize parameter space (a,b):

P[amin, . . . , amax][bmin, . . . , bmax] (accumulator array)

amax

68

Algorithm (cont’d)

2. For each edge point (x, y)

For(a = amin; a ≤ amax; a++) {b = − xa + y; /* round off if needed *

(P[a][b])++; /* voting */}

3. Find local maxima in P[a][b]

if P[a j][bk]=M, then M points lie on the liney = a j x + bk

69

Polar Representation of Lines

(no problem with vertical lines, i.e., θ =90)

ρ−θ space

70

Example

long

vertical

lines

71

Using the Hough transform for edge linking

7272

Morphology

◼ Morphological image processing is used toextract image components for representationand description of region shape, such asboundaries, skeletons, convex hull etc.

7373

Z2 and Z3

◼ Set in mathematical morphologyrepresent objects in an image◼ binary image (0 = white, 1 = black) : the

element of the set is the coordinates (x,y)of pixel belong to the object Z2

◼ gray-scaled image : the element of the set is thecoordinates (x,y) of pixel belong to the object andthe gray levels Z3

7474

Basic Set Theory

7575

Reflection and Translation

} ,|{ˆ Bfor bbwwB −=

} ,|{)( Afor azaccA z +=

7676

Example

77

Structuring element (SE)

77

▪ small set to probe the image under study▪ for each SE, define origin▪ shape and size must be adapted to geometricproperties for the objects

78

How to describe SE

◼ many different ways!

◼ information needed:◼ position of origin for SE

◼ positions of elements belonging to SE

78

79

Examples of Structuring element (SE)

80

Basic morphological operations

◼ Erosion

◼ Dilation

◼ combine to◼ Opening object

◼ Closing background

80

keep general shape but smooth with respect to

81

Erosion

◼ Erosion of a set A by structuring element B: all z in A such that B is in A when origin of B=z

shrink the object

81

}{ Az|(B)BA z =−The set of all points such that , translated by , is contained by .z B z A

| ( ) c

ZA B z B A= =

8282

}{ Az|(B)BA z =−

83

84

Dilation

◼ Dilation of a set A by structuring element B: all z in A such that B hits A when origin of B=z

◼ grow the object

84

}ˆ{ ΦA)Bz|(BA z =

8585

Dilation

}ˆ{ ΦA)Bz|(BA z =

B = structuring element

8686

Dilation : Bridging gaps

87

Opening and Closing

◼ Opening generally smoothes the contourof an object and eliminates thinprotrusions

◼ Closing tends to smooth sections ofcontours but it generates fuses narrowbreaks and long thin gulfs, eliminatessmall holes, and fills gaps in the contour

88

Opening

Erosion followed by dilation, denoted ∘

◼ eliminates protrusions

◼ breaks necks

◼ smooths contour

88

BBABA −= )(

8989

Opening

BBABA −= )(})(|){( ABBBA zz =

90

Closing

Dilation followed by erosion, denoted •

◼ smooth contour

◼ fuse narrow breaks and long thin gulfs

◼ eliminate small holes

◼ fill gaps in the contour

90

BBABA −=• )(

9191

Closing

BBABA −=• )(

9292

Properties

Opening(i) AB is a subset (subimage) of A(ii) If C is a subset of D, then C B is a subset of D B(iii) (A B) B = A B

Closing(i) A is a subset (subimage) of A•B(ii) If C is a subset of D, then C •B is a subset of D •B(iii) (A •B) •B = A •B

Note: repeated openings/closings has no effect!

9393

9494

95

Hit-or-Miss Transformation ⊛ (HMT)

◼ find location of one shape among a set of shapes template matching

◼ composite SE: object part (B1) and background part (B2)

◼ does B1 fits the object while, simultaneously, B2 misses the object, i.e., fits the background?

95

9696

Hit-or-Miss Transformation

)]([)( XWAXABA c −−−=

9797

Boundary Extraction

)()( BAAA −−=

9898

Example

9999

Thinning

cBAA

BAABA

)(

)(

=

−=

100100

Thickening

)( BAABA =•

101101

SkeletonsK

kk ASAS

0

)()(=

=

BkBAkBAASk )()()( −−−=

})(|max{ −= kBAkK

))((0

kBASA k

K

k=

=

102

Thank you !