Snakes, Strings, Balloons and Other Active Contour Models

Snakes, Strings, BalloonsSnakes, Strings, Balloonsand Other Active Contour Modelsand Other Active Contour Models

GoalGoal

• Start with image and initial closed curveStart with image and initial closed curve

• Evolve curve to lie along “important” Evolve curve to lie along “important” featuresfeatures– EdgesEdges

– CornersCorners

– Detected featuresDetected features

– User inputUser input

ApplicationsApplications

• Region selection in PhotoshopRegion selection in Photoshop

• Segmentation of medical imagesSegmentation of medical images

• TrackingTracking

Corpus CallosumCorpus Callosum

[Davatzikos and Prince][Davatzikos and Prince]

Corpus CallosumCorpus Callosum

[Davatzikos and Prince][Davatzikos and Prince]

User-Visible OptionsUser-Visible Options

• Initialization:Initialization: user-specified, automatic user-specified, automatic

• Curve properties:Curve properties: continuity, continuity, smoothnesssmoothness

• Image features:Image features: intensity, edges, intensity, edges, corners, …corners, …

• Other forces:Other forces: hard constraints, springs, hard constraints, springs, attractors, repulsors, …attractors, repulsors, …

• Scale:Scale: local, multiresolution, global local, multiresolution, global

Behind-the-Scenes OptionsBehind-the-Scenes Options

• Framework:Framework: energy minimization, forces energy minimization, forces acting on curveacting on curve

• Curve representation:Curve representation: ideal curve, ideal curve, sampled, spline, implicit functionsampled, spline, implicit function

• Evolution method:Evolution method: calculus of calculus of variations, numerical differential variations, numerical differential equations, local searchequations, local search

Snakes: Active Contour ModelsSnakes: Active Contour Models

• Introduced by Kass, Witkin, and Introduced by Kass, Witkin, and TerzopoulosTerzopoulos

• Framework: energy minimizationFramework: energy minimization– Bending and stretching curve = more Bending and stretching curve = more

energyenergy

– Good features = less energyGood features = less energy

– Curve evolves to minimize energyCurve evolves to minimize energy

• Also “Deformable Contours”Also “Deformable Contours”

Snakes Energy EquationSnakes Energy Equation

• Parametric representation of curveParametric representation of curve

• Energy functional consists of three Energy functional consists of three termsterms

)(),()( sysxs v )(),()( sysxs v

dssss conimg )()()(int vvv dssss conimg )()()(int vvv

Internal EnergyInternal Energy

• First term is “membrane” term – minimum First term is “membrane” term – minimum energy when curve minimizes lengthenergy when curve minimizes length(“soap bubble”)(“soap bubble”)

• Second term is “thin plate” term – Second term is “thin plate” term – minimum energy when curve is smoothminimum energy when curve is smooth

2)()()()()(22

int sssss sss vvv 2)()()()()(22

int sssss sss vvv

Internal EnergyInternal Energy

• Control Control and and to vary between to vary between extremesextremes

• Set Set to 0 at a point to allow corner to 0 at a point to allow corner

• Set Set to 0 everywhere to let curve follow to 0 everywhere to let curve follow sharp creases – “strings”sharp creases – “strings”

2)()()()()(22

int sssss sss vvv 2)()()()()(22

int sssss sss vvv

Image EnergyImage Energy

• Variety of terms give different effectsVariety of terms give different effects

• For example,For example,

minimizes energy at intensity minimizes energy at intensity IIdesireddesired

desiredimg IyxIw ),( desiredimg IyxIw ),(

Edge AttractionEdge Attraction

• Gradient-based:Gradient-based:

• Laplacian-based:Laplacian-based:

• In both cases, can smooth with GaussianIn both cases, can smooth with Gaussian

2),( yxIwimg 2),( yxIwimg

22 ),( yxIwimg 22 ),( yxIwimg

Corner AttractionCorner Attraction

• Can use corner detector we saw last Can use corner detector we saw last timetime

• Alternatively, let Alternatively, let = tan = tan-1-1 I Iyy / I / Ixx

and let and let nnbe a unit vector perpendicular be a unit vector perpendicular

to the gradient. Thento the gradient. Then

n

wimg

n

wimg

Constraint ForcesConstraint Forces

• SpringSpring

• RepulsionRepulsion

2xv kcon 2xv kcon

2xv

k

con 2xv

k

con

Evolving CurveEvolving Curve

• Computing forces on v that locally Computing forces on v that locally minimize energy gives differential equation minimize energy gives differential equation for for vv– Euler-Lagrange formulaEuler-Lagrange formula

• Discretize v: samples (Discretize v: samples (xxii, , yyii))

– Approximate derivatives with finite differencesApproximate derivatives with finite differences

• Iterative numerical solverIterative numerical solver

02

2

vvds

d

vds

d

02

2

vvds

d

vds

d

Other Curve Evolution OptionsOther Curve Evolution Options

• Exact solution: calculus of variationsExact solution: calculus of variations

• Write equations directly in terms of Write equations directly in terms of forces,forces,not energynot energy

• Implicit equation solverImplicit equation solver

• Search neighborhood of each (Search neighborhood of each (xxii, , yyii) for ) for

pixel that minimizes energypixel that minimizes energy– Shah & Williams paperShah & Williams paper

Variants on SnakesVariants on Snakes

• Balloons [Cohen 91]Balloons [Cohen 91]– Add inflation forceAdd inflation force

– Helps avoid getting stuck on small featuresHelps avoid getting stuck on small features

)(skFinfl n )(skFinfl n

BalloonsBalloons

[Cohen 91][Cohen 91]SnakesSnakes

BalloonsBalloons

BalloonsBalloons

[Cohen 91][Cohen 91]

Other Energy or Force TermsOther Energy or Force Terms

• Results of previously-run local Results of previously-run local algorithmsalgorithms– e.g., Canny edge detector output convolvede.g., Canny edge detector output convolved

with Gaussianwith Gaussian

• Automatically-evolved control pointsAutomatically-evolved control points

• Others…Others…

Brain Cortex SegmentationBrain Cortex Segmentation

Add energy term for constant-color regions of a single colorAdd energy term for constant-color regions of a single color

Davatzikos and PrinceDavatzikos and Prince





Find featuresFind featuresand addand addconstraintsconstraints



ScaleScale

• In the simplest snakes algorithm, image In the simplest snakes algorithm, image features only attract locallyfeatures only attract locally

• Greater region of attraction: smooth imageGreater region of attraction: smooth image– Curve might not follow high-frequency detailCurve might not follow high-frequency detail

• Multiresolution processingMultiresolution processing– Start with smoothed image to attract curveStart with smoothed image to attract curve

– Finish with unsmoothed image to get detailsFinish with unsmoothed image to get details

• Heuristic for Heuristic for globalglobal minimum vs. local minimum vs. local minimaminima

Diffusion-Based MethodsDiffusion-Based Methods

• Another way to attract curve to Another way to attract curve to localized features: localized features: vector flowvector flow or or diffusiondiffusion methods methods

• Example:Example:– Find edges using CannyFind edges using Canny

– For each point, compute distance toFor each point, compute distance tonearest edgenearest edge

– Push curve along gradient of distance fieldPush curve along gradient of distance field

Gradient Vector FieldsGradient Vector Fields

Xu and PrinceXu and Prince



Simple SnakeSimple Snake With Gradient Vector FieldWith Gradient Vector Field



Documents

Snakes, Strings, Balloons and Other Active Contour Models