Deformable Contours
1
Dr. E. Ribeiro
Classical Methods
2
Thresholding Edge detectionAn image of blood vessel
Slide by: Chunming Li, Vanderbilt University
An Advanced Method: Active Contour Model
3Slide by: Chunming Li, Vanderbilt University
Edges vs. boundaries
Edges useful signal to indicate occluding boundaries, shape.
Here the raw edge output is not so bad…
…but quite often boundaries of interest are fragmented, and we have extra “clutter” edge points.
Images from D. Jacobs
Deformable contours
Given: initial contour (model) near desired object
a.k.a. active contours, snakes
(Single frame)
Fig: Y. Boykov[Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987]
Deformable contours
Given: initial contour (model) near desired object
a.k.a. active contours, snakes
(Single frame)
Fig: Y. Boykov
Goal: evolve the contour to fit exact object boundary
[Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987]
Why do we want to fit deformable shapes?
• Non-rigid, deformable objects can change their shape over time, e.g. lips, hands.
Figure from Kass et al. 1987
Why do we want to fit deformable shapes?
• Some objects have similar basic form but some variety in the contour shape.
Deformable contours: intuition
Image from http://www.healthline.com/blogs/exercise_fitness/uploaded_images/HandBand2-795868.JPG
Figure from Shapiro & Stockman
Deformable contours
initial intermediate final
a.k.a. active contours, snakesHow is the current contour adjusted to find the new contour at each iteration?
• Define a cost function (“energy” function) that says how good a possible configuration is.
• Seek next configuration that minimizes that cost function.
Snakes energy functionThe total energy (cost) of the current snake is defined as:
externalinternaltotal EEE +=
A good fit between the current deformable contour and the target shape in the image will yield a low value for this cost function.
Internal energy: encourage prior shape preferences: e.g., smoothness, elasticity, particular known shape.
External energy (“image” energy): encourage contour to fit on places where image structures exist, e.g., edges.
10))(),(()( ≤≤= ssysxsν
Parametric curve representation(continuous case)
Fig from Y. Boykov
External energy: intuition• Measure how well the curve matches the image data• “Attract” the curve toward different image features– Edges, lines, etc.
How do edges affect “snap” of rubber band?
Think of external energy from image as gravitational pull towards areas of high contrast
External image energy
Magnitude of gradient- (Magnitude of gradient)
( )22 )()( IGIG yx +−22 )()( IGIG yx +
External image energy
• Image I(x,y)• Gradient images and
• External energy at a point v(s) on the curve is
• External energy for the whole curve:
),( yxGx ),( yxGy
)|))((||))((|())(( 22 sGsGsE yxexternal ννν +−=
∫=1
0
))(( dssEE externalexternal ν
10))(),(()( ≤≤= ssysxsν
Internal energy: intuition
http://www3.imperial.ac.uk/pls/portallive/docs/1/52679.JPG
A priori, we want to favor smooth shapes, contours with low curvature, contours similar to a known shape, etc. to balance what is actually observed (i.e., in the gradient image).
Internal energyFor a continuous curve, a common internal energy term is the “bending energy”. At some point v(s) on the curve, this is:
The more the curve bends the larger this energy value is.
The weights α and β dictate how much influence each component has.
Elasticity,
Tension
Stiffness,
Curvature
sdd
dsd
sEinternal 2
2))((
22
ννβαν +=
∫=1
0
))(( dssEE internalinternal νInternal energy for whole curve:
Dealing with missing data
• The smoothness constraint can deal with missing data:
[Figure from Kass et al. 1987]
Total energy(continuous form)
// bending energy
// total edge strength
under curve
externalinternaltotal EEE γ+=
∫=1
0internal ))(( dssEEinternal ν
∫=1
0
))(( dssEE externalexternal ν
Parametric curve representation(discrete form)
• Represent the curve with a set of n points
10),( −== νiyx iii Kν …
Discrete energy function:external term
21
0
2 |),(||),(| iiy
n
iiixexternal yxGyxGE ∑
−
=
+−=
Discrete image gradients
),( yxGx ),( yxGy
• If the curve is represented by n points
Summary: elastic snake• A simple elastic snake is defined by– A set of n points,– An internal elastic energy term– An external edge based energy term
• To use this to locate the outline of an object– Initialize in the vicinity of the object– Modify the points to minimize the total
energy
Level Set Representation of Curves
24
zero level
zero level
Slide by: Chunming Li, Vanderbilt University
Level Set Method (Osher and Sethian, 1988)
25
• Curve evolution:
where F is the speed function, N is normal vector to the curve C
• Level set formulation:
N
Slide by: Chunming Li, Vanderbilt University
26
Geodesic active contour
27
Geodesic active contour
Geodesic active contour
28
29
Segmentation using statistical models (Rousson and Deriche, 2002)
Energy functional
Probability describing the pixel values inside region i
30
Segmentation using statistical models (Rousson and Deriche, 2002)
Energy functional
Probability describing the pixel values inside region i
The energy functional can be rewritten as:
Two-phase case
31
Two-phase case
32
Energy functional
Level set function
Two-phase case
33
Energy functional
Heaviside step function
Smooth approximation
Length term
Indicator functions (partitions)
34
Indicator functions (partitions)
35
Distance function (level set function)
36
Two-phase case
37
Estimating the Parameters of the Gaussian densities
Two-phase case
Experiments
39
Gray level
40
Color and Texture
41
Results
42Slide by: Chunming Li, Vanderbilt University
3D Segmentation of Corpus Callosum
43Slide by: Chunming Li, Vanderbilt University
Result
44
Synthetic noisy image
Slide by: Chunming Li, Vanderbilt University
2D Segmentation of Real Color Images
45
A real image of potatoes
Slide by: Chunming Li, Vanderbilt University
2D Vessel Segmentation
46Slide by: Chunming Li, Vanderbilt University
Segmentation of White Matter in MR images
47Slide by: Chunming Li, Vanderbilt University
Effect of the Level Set Regularization
48
Final zero level contour Final level set function
Without level set regularization
Slide by: Chunming Li, Vanderbilt University
3D Vessel Segmentation
49
MRA Vessel Segmentation
Slide by: Chunming Li, Vanderbilt University
3-D Ultrasound
50http://mrcas.mpe.ntu.edu.sg/research/imgpro/3d_level_set.htm
3-D Ultrasound
51