1

Variational Approaches and Image Segmentation

Lecture #5Lecture #5Hossam Abdelmunim1 & Aly A. Farag2

1Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt

2Electerical and Computer Engineering Department, University of Louisville, Louisville, KY, USA

ECE 643 – Fall 2010

Variational Formulation in Computer Vision and Image Analysis

Variational Formulation in Computer Vision and Image Analysis

Image/Curve/Surface is required to evolve to minimize a Image/Curve/Surface is required to evolve to minimize a certain energy function.certain energy function.

The change of the Image/Curve/SurfaceThe change of the Image/Curve/Surface is described by a is described by a partial differential equation (PDE).partial differential equation (PDE).

In image segmentation, the evolution continuous until the In image segmentation, the evolution continuous until the curve/surface hits the object (region) boundaries.curve/surface hits the object (region) boundaries.

Advantages of Introducing the Variational Formulation to the Computer Vision FieldAdvantages of Introducing the Variational Formulation to the Computer Vision Field

Modeling images in continuous time domain which simplifies the formalism and makes it grid independent.

Research on numerical analysis makes it possible to achieve high speed, accurate, and stable solutions of the derived PDE’s.

Viscosity solutions provide a framework for dealing with non-smooth signals used with these approaches.

Formal analysis in this area give the possibility of providing successful algorithms and useful theoretical results like existence and uniqueness of the solution.

Euler Lagrange & Gradient Descent (Recall)

Euler Lagrange & Gradient Descent (Recall)

RRm :Given a real function:

dLE ),(

Define the following energy functional:

)(

Ldiv

LE

The Euler Lagrange equation will be:

0

E

E

t

The Gradient Descent will be:

Classical Deformable ModelsClassical Deformable Models

Parametric Planar Curve RepresentationParametric Planar Curve Representation

C(0)

C(0.5)

C(1)

,)](),([)( TpypxpC ]1,0[p

Parametric Form:

Arc Length is:

p

p dpCps0

||)(

Snake/Contour EvolutionSnake/Contour Evolution

ExternalInternal EEE

The contour is required to evolve minimizing the following energy function:

Contour characteristics Image data (force)

Internal EnergyInternal Energy

1

0

22

21 )||)(||)((5.0)( dpCpwCpwCE pppInternal

w1 and w2 are non negative.

The term weighted by w1 reduces curve length and removes ripples.

The second term controls the bending and smoothness of the contour.

External EnergyExternal Energy

1

0

))(*()( dpCIGCEExternal

G is a Gaussian smoothing filter with width .

λ is the magnitude of the external force .

This term derives the contour towards the object boundaries.

Vector Valued PDEVector Valued PDE

Using calculus of variation, the contour evolves according to the following equation:

))*(()()( 22

2

1 IGCwp

Cwp

C Cpppt

))*(())](())(([5.0 2222

222

1 IGyxx

wp

yxx

wp

x xpppppp

ppp

t

))*(())](())(([5.0 2222

222

1 IGyxy

wp

yxy

wp

y ypppppp

ppp

t

uv

Parametric Form:

,)],(),,(),,([),( TvuzvuyvuxvuS

]1,0[]1,0[),( vu

Energy:

Parametric Surfaces Representation and Evolution

Parametric Surfaces Representation and Evolution

1

0

1

02

022

20

211

201

210

]||),(||),(

||),(2||),(||),([)(

dudvSvuwSvuw

SvuwSvuwSvuwSE

vvuu

uvvuInternal

1

0

1

0

))(*()( dudvSIGSEExternal

Complete the derivation.

ExampleExample

Aorta segmentation using active contours

Aorta segmentation using active contours

Problems of Classical Deformable ModelsProblems of Classical Deformable Models

Dependency on the parameterization.

Troubles with topology changes (merging/splitting).

Initialization needs to be near the final solution (a big problem in 3D).

Tuning of the weighting coefficients in the PDE (w’s, λ).

Level SetsLevel Sets

Curve Representation using Level SetsCurve Representation using Level Sets

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

x

y

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

x

y

)1,,( tyxZ

)2,,( tyxZ

Zero Level Set or Z=0 plane

Resulting Curve

(Evolving Front)

Invented by Osher and Sethian 1985, to overcome disadvantages of classical deformable models.

Add extra dimension to the problem

Topology ChangeTopology Change

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

x

y

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

x

y

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

x

y

)0,,( tyxZ )1,,( tyxZ )2,,( tyxZ

Topological changes in the evolving front are handled naturally. No parameterization is needed.

This representation allows merging and breaking of the front as t advances

Level Sets and The Signed Distance Function

Level Sets and The Signed Distance Function

17

outsideifXD

boundarytheon

insideifXD

X

)(

0

)(

)(

• D is the min Euclidean distance between X and the contour.

• Ф is continuous and differentiable around the zero level.

Example of Signed Distance FunctionsExample of Signed Distance Functions

1 2

1 2 NwN

Example of Image RepresentationExample of Image Representation

Image Representation

Applications of the Distance Transform (DT)Applications of the Distance Transform (DT)The DT is a fundamental operator in shape analysis, having numerous applications,some of which are briefly listed bellow.•Separation of overlapping objects through watershed segmentation.•Computation of morphological operators.•Computation of geometrical representations and measures, such as skeletonization.•Shape matching.•Shape measures related to distance.•Other interesting areas in which the DT has been applied are:

• image registration• subpixel contour tracing• multidimensional data analysis (classification, clustering)• image enhancement• ray-tracing optimization• embedding surface for level set-based methods• efficient belief-propagation• multiphase flow simulations• medical image analysis• analysis of interaction between biological structures

Signed Distance Generation ComplexitySigned Distance Generation Complexity

So, to compute the distance function you need to get the closest point on the boundary/curve for each position in the domain.

This process is very time-consuming especially for volumes in 3D and higher.

Some approximation methods are used to expedite this process. See the following slides.

Distance Transform GenerationFaster Algorithm

Distance Transform GenerationFaster Algorithm

∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ 0 0 0 0 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 0 0 0 0 ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞

1. Set boundary points to zeroes while others to ∞.2. Initialize an auxiliary set of pixels (usually a queue), the so-called contour set, with

the boundary pixels and their zero values.

3. While the contour set is not empty do the following:-a) Remove a pixel from the contour set . Let us call it the central pixel (xp,yp) now.b) For every pixel (xq,yq) of the neighbors of the central pixel, do the following:-

I. Compute the distance to the central point as: dd=|xp-xq|+|yp-yq|;II. Compute a temporary distance as : dtq=min(dist(p)+dd,dist(q));III. If dtq < dist(q), add the pixel q to the contour set;IV. Dist(q)=dtq;

Distance Transform GenerationFaster Algorithm (Cont…)

Distance Transform GenerationFaster Algorithm (Cont…)

• Note that computing the distance as the difference is faster than the square root function.

• Also, this methodology updates the distance values around the boundary points/pixels until there are no points in the queue.

• Try this method with the dynamic implementation using linked lists. It will be much faster than the ordinary methods….

Assignment: Implement using C++. Apply it for 2D as well as 3D

shapes.