Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Lecture 16 Image SegmentationImage Segmentation
1. The basic concepts of segmentation2 P i t li d d t ti2. Point, line, edge detection3. Thresh holding4 Region based segmentation4. Region-based segmentation5. Segmentation with Matlab
What is segmentation
• What is segmentation– Segmentation subdivides an image into its constituent regions or
objects ntil the objects of interest in an application ha e beenobjects, until the objects of interest in an application have been isolated.
• Segmentation conditions0
0
,...,
( )
n
n
ii
R R
a R R=
=U
( ) is a connected set 1, 2,...,( ) for all and ,
( ) Q( ) for 1,2,...,( ) Q( ) f dj t i d
i
i j
i
b R i nc R R i j i j
d R true i nR R f l R R
=
∩ =∅ ≠
= =
∪
• Segmentation problem: to partition the image into
( ) Q( ) for any adjacent regions and
where ( ) is a logical i j i j
i
e R R false R R
Q R
∪ =
predicate defined over the points in set . kR
Segmentation problem: to partition the image into regions satisfying above conditions
Two principal approaches
• Edge-based segmentation– partition an image based on abrupt changes in intensity (edges)
• Region-based segmentation– partition an image into regions that are similar according to a set
f d fi d it iof predefined criteria.
Detection of Discontinuities
• Detect the three basic types of gray-level discontinuities– points , lines , edges
• Use the image sharpening techniques– The first order derivatives produce thicker edges– The second order derives have a strong response to fine detail,The second order derives have a strong response to fine detail,
such as thin lines and isolated points, and noise– Laplasian operation
• Can be done by running a mask through the image• Can be done by running a mask through the image
1 1 2 2 9 9...R w z w z w z= + + +1 1 2 2 9 99
...
k k
R w z w z w z
w z
+ + +
=∑4
1k=∑
Point Detection
Steps for point dection1. Apply Laplacian filter to the imagepp y p g
to obtain R(x, y)2. Create binary image by threshold
1, if | ( , ) |( , )
0, otherwiseR x y T
g x y≥⎧
= ⎨⎩
where T is a nonnegative threshold
,⎩
5
Example
6
Line Detection• A special mask is needed to detect a special type of line• Examples:
H i t l k h hi h h li d th h– Horizontal mask has high response when a line passed through the middle row of the mask.
7
Multiple Lines Detection• Apply every masks on the image, find the maximum response.
Example:Example:
Let R1, R2, R3, R4 denotes the response of the horizontal, +45 degree, vertical and -45 degree masks, respectively. if, at a certain point in the image |Ri| > |Rj|, for all j≠i, that point is said to be more likely associated with a line in the direction of mask i.
8
Edge Detection
• Edge detection is the approach for segmenting images based on abrupt changes in intensityimages based on abrupt changes in intensity.
• What is an edge– an edge is a set of connected pixels that lie on the boundaryan edge is a set of connected pixels that lie on the boundary
between two regions.– an edge is a “local” concept whereas a region boundary, owing
to the way it is defined, is a more global idea.y , g
• Edge models– Step edge (Idea edge), ramp edge (thick edge), and roof edge
9
Example of edges in an image
• An image may all the three types of edges
First and Second derivatives at the edge
1. The magnitude of the first derivative can be used to detect the presence of an edge at a pointpresence of an edge at a point
2. Second derivative produces two values for every edge in an image. An imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint
11
of the edge. zerozero--crossing point: crossing point: the center of thick edges
Noise Images
• First column: images and gray-level profiles of a ramp edge corrupted by random Gaussian p ynoise of mean 0 and σ = 0.0, 0.1, 1.0 and 10.0, respectively.
• Second column: first-derivativeSecond column: first derivative images and gray-level profiles.
• Third column : second-derivative images and grayderivative images and gray-level profiles.
12
Steps in edge detection
1. Image smoothing for noise reduction2. Detection of edge points. Points on an edge3. Edge localization
13
Image gradient
• Gradient is a vector
⎥⎥⎤
⎢⎢⎡
∂∂∂
=⎥⎤
⎢⎡
=∇ fxf
gxf
• The magnitude of the gradient⎥⎥⎥
⎦⎢⎢⎢
⎣∂∂=⎥
⎦⎢⎣
=∇
yfg y
f
• The magnitude of the gradient22
22)f(),( ⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+⎟⎠⎞
⎜⎝⎛∂∂
=+=∇=yf
xfggmagyxM yx
• The direction of the gradient vector⎠⎝ ∂⎠⎝ ∂ yx
⎤⎡
• The direction of an edge at (x y) is perpendicular to the direction
⎥⎦
⎤⎢⎣
⎡= −
x
y
gg
yx 1tan),(α
14
• The direction of an edge at (x, y) is perpendicular to the direction of the gradient vector at that point
Example
2fx∂⎡ ⎤⎢ ⎥ ⎡ ⎤∂ 2
, ( , ) 2 2,2
xf M x yf
⎢ ⎥ −⎡ ⎤∂⎢ ⎥∇ = = =⎢ ⎥∂⎢ ⎥ ⎣ ⎦⎢ ⎥∂⎣ ⎦
1( , ) tan ( 2 / 2) 45o
y
x yα −
⎢ ⎥∂⎣ ⎦= − = −
Gradient Operators and Masks
( 1, ) ( , )
( 1) ( )
xfg f x y f x yxfg f x y f x y
∂= = + −∂∂
= = +( , 1) ( , )yg f x y f x yy
= = + −∂
Roberts cross-gradient operators( , )f x yg z z∂
9 5
8 6( , )
x
y
g z zx
f x yg z zy
= = −∂
∂= = −
∂
Prewitt operators
7 8 9 1 2 3
3 6 9 1 4 7
Prewitt operators
( ) ( )
( ) ( )
x
y
fg z z z z z zxfg z z z z z zy
∂= = + + − + +∂∂
= = + + − + +∂y y∂
7 8 9 1 2 3
Sobel operators
( 2 ) ( 2 )xfg z z z z z zx∂
= = + + − + +∂
16
3 6 9 1 4 7( 2 ) ( 2 )y
xfg z z z z z zy
∂∂
= = + + − + +∂
Prewitt and Sobel masks for Diagonal edges
Sobel masks have slightly superior noise-suppression characteristics which is an important issueis an important issue when dealing with derivatives.
17
Example
18
Example
Example
20
Example
Laplacian
2
2
2
22 ),(),(
yyxf
xyxff
∂∂
+∂
∂=∇
Laplacian operator
yx ∂∂
),1(),1([2 yxfyxff −++=∇)],(4)1,()1,(
),(),([yxfyxfyxf
yfyff−−+++
22
Laplacian of Gaussian
• Laplacian combined with smoothing as a precursor to find edges via zero-crossing.
),( 2 2
22
eyxGyx
=+
−σ
2
),(
2222
222
2
22
eyx
yG
xGyxG
yx
⎥⎤
⎢⎡ −+
∂∂
+∂∂
=∇
+−
σσ
0,(),(),( 2
24
yxfyxGyxg
ey
∗∇=
⎥⎦
⎢⎣
= σ
σ
0,(),(),( yxfyxGyxg ∇
23
Marr-Hildreth edge detection algorithm
1. Filter the input with an n by an Gaussian lowpass filter2. Compute the Laplacian of the image of step 13. Find the zero crossing of the image from step Approximate the zero crossing from LoG image to threshold the LoG image bysetting all its positive values to white and all negative values to black the zerosetting all its positive values to white and all negative values to black. the zerocrossing occur between positive and negative values of the thresholded LoG.
Zero crossing vs. Gradient
• Attractive– Zero crossing produces thinner edges
Noise reduction– Noise reduction• Drawbacks
– Zero crossing creates closed loops. (spaghetti effect)– sophisticated computation.
• Gradient is more frequently used.
25
Example
a). Original imageb). Sobel Gradientc). Spatial Gaussian smoothing functiond). Laplacian mask) L Ge). LoG
f). Threshold LoGg). Zero crossing
26
Edge Linking and Boundary Detection
• An edge detection algorithm are followed by linking procedures to assemble edge pixels into meaningful dedges.
• Basic approaches– Local ProcessingLocal Processing– Global Processing via the Hough Transform– Global Processing via Graph-Theoretic Techniques
27
Local Processing
• Analyze the characteristics of pixels in a small neighborhood Sxy(say, 3x3, 5x5) about every edge pixels (x, y) in an image.All i t th t i il di t t f d fi d it i• All points that are similar according to a set of predefined criteria are linked, forming an edge of pixels that share those criteria.
• Criteria |),(),(| EyxMtsM ≤−
thresholdangle positive a is where|),(),(|
thresholdpositive a is where
AAyxts
E≤−αα
• Algorithm steps
),(and),(,1),(),,(, compute .1
αα
⎧ ±=>
∇
AM TAyxTyxMyxyxMf
Klength specified a exceednot do that ineach tow gaps all fill and g of rows Scan the .3
otherwise,0),( and),(,1
),(imagebinary a Form .2α
⎩⎨⎧ ±>
= AM TAyxTyxMyxg
28 -by result theRotate 3. Stepin procedure scanning
horizontal apply the and angle by this g rotate ,direction any in gap Detect the 4.θ
θ
Example of local precessing
TM =30%,A =90,TA = 45, Gap =25
Hough Transformation (Line)
yi =axi + b b = - axi + yiyi axi + b b axi + yi
all points (xi ,yi) contained on the same line must have lines in parameter space that intersect at (a’,b’)
30
parameter space that intersect at (a ,b )
ρθ-plane
• problem of using equation y = ax + b is that value of a is infinite for a• problem of using equation y ax + b is that value of a is infinite for a vertical line.
• To avoid the problem, use equation x cos θ+ y sin θ = ρ to represent a line instead.
ti l li h ith l t th iti i t t
31
• vertical line has θ = 90° with ρ equals to the positive y-intercept or θ = -90° with ρ equals to the negative y-intercept
Generalized Hough TransformationThe method can be used for any function of the form g(v, c) = 0, where v is a vector of coordinates, c is a vector of coefficients
2 2 2Example: circles, (x-c1)2 + (y-c2)2 = c32
(c1, c2, c3) cube like cells, accumulators of the form A(i, j, k)increment c and c solve of c that satisfies the equationincrement c1 and c2 , solve of c3 that satisfies the equationupdate the accumulator corresponding to the cell associatedwith triplet (c1, c2, c3)
33
Edge-linking based on Hough Transformation
1. Compute the gradient of an image and threshold it to obtain a binary image.
2. Specify subdivisions in the ρθ-plane.3. Examine the counts of the accumulator cells for high
pixel concentrations.p4. Examine the relationship (principally for continuity)
between pixels in a chosen cell.Continuity:Continuity:1. based on computing the distance between
disconnected pixels identified during traversal of the p gset of pixels corresponding to a given accumulator cell.
2. a gap at any point is significant if the distance between that point and its closet neighbor exceeds a certain
34
that point and its closet neighbor exceeds a certain threshold.
Example
link criteria:1). the pixels belonged to one of the set of pixels linked
di t th hi h t t
36
according to the highest count2). no gaps were longer than 5 pixels
Image Segmentation II
1. Threshold2 R i b d t ti2. Region-based segmentation3. Segmentation using watersheds4 Segmentation with Matlab4. Segmentation with Matlab
Thresholding
• Problem: how to determine the threshold value of segmentation?
• Histogram
1, if ( , )( , )
0 if ( )f x y T
g x yf T
>⎧= ⎨ ≤⎩
2
1 2
, if ( , ) ( , ) , if ( , )
a f x y Tg x y b T f x y T
>⎧⎪= < ≤⎨⎪0, if ( , )f x y T≤⎩
2, if ( , ) c f x y T⎪ ≤⎩
The role of noise in image thresholding
• When noise is small, the method work, otherwise it may not work. Noise reduction has to be done before h i th h ld lchoosing threshold value
The Role of Illumination
Illumination plays an important rolef(x,y) = i(x,y) r(x,y) f( y) ( y) ( y)Approach:1.g(x,y) = ki(x,y)g( ,y) ( ,y)2.h(x,y) = f(x,y)/g(x,y) = r(x,y)/k
40
Basic Global ThresholdingBased on visual inspection of histogram1. Select an initial estimate for T.2 Segment the image using T This will produce two2. Segment the image using T. This will produce two
groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values ≤ Tlevel values ≤ T
3. Compute the average gray level values m1 and m2 for the pixels in regions G1 and G2
4 Compute a new threshold value T = 0 5 (m1 + m2)4. Compute a new threshold value T 0.5 (m1 + m2)5. Repeat steps 2 through 4 until the difference in T in
successive iterations is smaller than a predefined parameter To.parameter To.
41
Basic Global Thresholding
Use T midway between the max and min gray levels generate binary image
T0 = 1 , 3 iterations, result T = 1250
Optimum global thresholdingChoose thresholding values that maximize the between-class varianceOtsu’s method1 Comp te the normali ed histogram of the image 0 1 2 1p i L= −1. Compute the normalized histogram of the image2. Computer the cumulative sums3. Computer the cumulative mean
, 0,1,2,..., 1ip i L= −
1( ) , 0,1,2,..., 1k
iP k p k L= = −∑0
( ) , 0,1,2,..., 1k
ii
m k ip k L=
= = −∑
4. Computer the global intensity mean, 5. Compute the between-class variance
0i= 1
0
, 0,1,2,..., 1L
G ii
m ip k L−
=
= = −∑2
2 1[ ( ) ( )]( ) 0 1 2 1Gm P k m kk k L−
6. Obain the Otsu thereshold
2 1
1 1
[ ( ) ( )]( ) , 0,1, 2, ..., 1( )[1 ( )]
GB k k L
P k P kσ = = −
−
* ' 2 ' 2th e av e ra g e o f a ll su c h th a t ( ) m a x{ ( )}k k k kσ σ7. Compute the separability measure
th e a v e ra g e o f a ll su c h th a t ( ) m a x{ ( )}B Bkk k k kσ σ=
2 **
2
( )= B kση 2G
ησ
Example
Example
Example
Using edges to improve global thresholding
1. Compute the edge image g(x, y) of f(x, y)2. Specify a threshold value T of g(x, y)3. Threshold the image f(x, y) using T, obtain gt(x, y) 4. Compute a histogram of f(x, y) using pixels (x, y) with
( ) 1gt(x, y) =15. Use the above histogram to segment f(x,y) globally.
Example
Multiple thresholdingOtsu’s method can be extended to arbitrary number of thresholds.1. Compute the normalized histogram of the image2 Comp ter the c m lati e s ms
, 0,1,2,..., 1ip i L= −1∑ ∑2. Computer the cumulative sums
3. Computer the cumulative mean1
1, , ,..., are classesk k
k i k i Ki C i Ck
P p m ip C CP∈ ∈
= =∑ ∑
2 2( ) [ ]K K
P k k P∑ ∑
4. Obain theresold
2 21
1 1
, ( , . . . , ) [ ]G i i B K i i Gi i
m P m k k P m mσ= =
= = −∑ ∑
4. Obain theresold
5 Threshold image
* * 2 * * 21 1 1 1 1 1, . . . , su c h th a t ( , ... , ) m a x{ ( , ..., )}K B K B Kk
k k k k k kσ σ− − −=
5. Threshold image*
1 1* *
2 1 2
if ( , ) i f ( , )
( , )
a f x y ka k f x y k
g x y
⎧ ≤⎪ < ≤⎪= ⎨
M*
( , )
if ( , ) K K
g y
a f x y k
⎨⎪⎪ >⎩
M
Example
Basic Adaptive Thresholding
• Subdivide original image into small areas.
• Utilize a different threshold to segment each subimages.
• The threshold used for each pixel depends on the• The threshold used for each pixel depends on the location of the pixel in terms of the subimages.
51
Example : Adaptive Thresholding
52
2. Region-Based Segmentation
• Basic Formulation
RRa i
n=∪)(
, ..., n, i Rb
RRa
i
i
=
∪=
21 region, connected a is )(
)(1i
niTRUE)P(Rd
ji RRc ji
==
≠=∩
21for)(
j, and i allfor )( φ
jfor i FALSE) RP(Re, ..., n, i TRUE) P(Rd
ji
i
≠=∪==
)(21for )(
P(Ri) is a logical predicate property defined over the points in set Ri
ex. P(Ri) = TRUE if all pixel in Ri have the same gray level
53
ex. P(Ri) TRUE if all pixel in Ri have the same gray level
Two basic approaches
• Region Growing– start with a set of “seed” points– growing by appending to each seed those neighbors that have
similar properties such as specific ranges of gray levele.g
TRUE if th b l t diff f th⎧TRUE if the absolute difference of the intensites between the seed and the pixel at (x,y) is
QT
⎧⎪⎪= ⎨ ≤⎪
• Region splitting and merging
FALSE otherwise ⎪⎪⎩
g g g g– Iteratively divide a region into smaller regions until all regions
become TRUE– Merge adjacent regions as along as the resulting region is still
54
Merge adjacent regions as along as the resulting region is still TRUE
Region Growing
Select all seed points with gray level 255criteria:1. the absolute gray-
level difference between any pixelbetween any pixel and the seed has to be less than 65
2. the pixel has to be 8-connected to at least one pixel in that pregion (if more, the regions are merged)
55
Region splitting and merging
Quadtree1. Split into 4 disjoint quadrants any region Ri for which
P(R ) FALSEP(Ri) = FALSE2. Merge any adjacent region Rj and Rk for which
P(Ri ∪ Rk ) = TRUE3. Stop when no further merging or splitting is possible.
Example
TRUE if and 0 FALSE otherwise
a m bQ
σ > < <⎧= ⎨⎩
where and the mean and standard deviationof pixels in a quadregion
m σ
Example
P(Ri) = TRUE if at least 80% of the pixels in Ri have theP(Ri) TRUE if at least 80% of the pixels in Ri have the property |zj-mi| ≤ 2σi, where
z is the gray level of the jth pixel in Rzj is the gray level of the jth pixel in Rimi is the mean gray level of that regionσi is the standard deviation of the gray levels in Ri
58
Segmentation using Watersheds
• The concept of watersheds– View an image as 3-D graphics– Three types of points
• Local minimum• Point at which water will flow to a local minimum• Point at which water will flow to a local minimum,
also called catchment basin, or watershed.• Point at which water can equally flow to more than
one local minimum points, also called divide lines, or watershed lines
• Segmentation by watershed lines• Segmentation by watershed lines
Example
Watershed algorithm
1 2Let , ,..., be sets denoting the coordinates of the reginoal minima of image ( , ). ( ) is the set of pixels of catchment
R
i
M M Mg x y C M
basin of [ ] {( , ) | ( , ) }, min 1 to max 1( ) ( ) [ ]
i
n i i
MT n s t g s t n nC M C M T n
= < = + += ∩( ) ( ) [ ]
[ ]n i i
Q n denotes the set of connected components of For min 1 to max 1 doF h t d t f Q[ ]
iMn = + +
For each connected component q of Q[n] if [ 1] Let [ ] [ 1]
q C nC n C n q
∩ − =∅← − ∪
else if [ 1] contains q C n∩ − one connected component of [ 1] [ ] [ 1]
else [ 1] contains more than one components of [ 1]
C nC n C n qq C n C n
−← − ∪
∩ − − else [ 1] contains more than one components of [ 1] The build dam within q
q C n C n∩
Dam construction
Example