1

Optimal Transport, Conformal Mappings,

and Stochastic Methods for Registration and Surface

Warping

Allen TannenbaumGeorgia Institute of Technology

Emory University

2

This lecture is dedicated to Gregory Randall, citizen of the world. Muchas gracias, Gregory (El Goyo).

3

Collaborators:

Conformal Mappings: Angenent, Haker, Sapiro, Kikinis, Nain, Zhu

Optimal Transport: Haker, Angenent, Kikinis, Zhu

Stochastic Algorithms: Ben-Arous, Zeitouni, Unal, Nain

4

Surface Deformations and Flattening Conformal and Area-Preserving Maps

Optical Flow

Gives Parametrization of SurfaceRegistration

Shows Details Hidden in Surface Folds

Path PlanningFly-Throughs

Medical ResearchBrain, Colon, Bronchial PathologiesFunctional MR and Neural Activity

Computer Graphics and VisualizationTexture Mapping

5

Mathematical Theory of Surface Mapping

Conformal Mapping:One-oneAngle PreservingFundamental Form

Examples of Conformal Mappings:One-one Holomorphic FunctionsSpherical Projection

Uniformization Theorem:Existence of Conformal MappingsUniqueness of Mapping

GFEGFE ,,,,

6

Deriving the Mapping Equation

L e t p b e a p o i n t o n t h e s u r f a c e . L e t

b e a c o n f o r m a l e q u i v a l e n c e s e n d i n g p t o t h e N o r t h P o l e .

I n t r o d u c e C o n f o r m a l C o o r d i n a t e s vu , n e a r p ,

w i t h 0 vu a t p .

I n t h e s e c o o r d i n a t e s ,

22 2,2 dvduvuds

W e c a n e n s u r e t h a t 1p . I n t h e s e c o o r d i n a t e s , t h e L a p l a c e B e l t r a m i o p e r a t o r t a k e s t h e f o r m

2

2

2

2

2,

1

vuvu.

2: Sz

7

Deriving the Equation-Continued

Set ivuw . The mapping wzz has a simple pole at 0w , i.e. at p .

Near p , we have a Laurent series 2DwCBwA

wz

Apply to get

wAz

1.

Taking 21A ,

)2(2

1

log2

1

log2

1

1

2

1

pvi

u

wv

iu

wv

iu

wz

8

The Mapping Equation

pvi

uz

.

S i m p l y a s e c o n d o r d e r l i n e a r P D E . S o l v a b l e b y s t a n d a r d m e t h o d s .

9

Finite Elements-I i s a t r i a n g u l a t e d s u r f a c e . S t a r t w i t h

pvi

uz

M u l t i p l y b y a n a r b i t r a r y s m o o t h f a n d i n t e g r a t e b y

p a r t s . F o r a l l f w e w a n t :

pv

fip

u

f

dSfv

iu

dSfz p

L e t PLfz , , t h e s p a c e o f p i e c e w i s e l i n e a r f u n c t i o n s .

10

Finite Elements-IIF o r e a c h v e r t e x P , l e t p b e t h e

c o n t i n u o u s f u n c t i o n s u c h t h a t :

gle.each trianon linear is

, vertexa ,,0

1

P

QPQQ

P

P

P

T h e s e f u n c t i o n s f o r m a b a s i s f o r t h e f i n i t e d i m e n s i o n a l s p a c e PL . T h e n PPP zz . A n d w e w a n t , f o r a l l Q ,

pvQ

pu

QQdSPPzP

T h i s i s s i m p l y a m a t r i x e q u a t i o n .

11

Finite Elements-III

S e t

PQDD , dSQPPQD .

D e f i n e v e c t o r s

p

uQ

Qaa

,

p

vQ

Qbb

.

O u r e q u a t i o n b e c o m e s s i m p l y ibaDz .

SRPQD cotcot21

,

PQDQPPPD .

N e e d f o r m u l a s f o r ., ba

12

Finite Elements-IV

S u p p o s e t h e p o i n t p l i e s o n a t r i a n g l e w i t h v e r t i c e s ABC .

S i n c e

p

uQ

Qaa

,

a n d

p

vQ

Qbb

,

w e h a v e CBAQQibQa , , if 0 .

13

Finite Elements-V

I f CBAQ , , , t h e n c o n s i d e r i n g t h a t Q i s

l i n e a r o n ABC :

, 1

,1

,11

:

CQEC

i

BQEC

iAB

AQEC

iAB

Qib

Qa

2

,

AB

ABAC

14

Finite Elements-VI If we set iyxz , then our system

ibaDz becomes aDx and bDy .

D is sparse, real, symmetric and positive semi-definite. Its kernel is the space of constant vectors, and it is positive definite on the space orthogonal to its kernel.

These properties of D allow us to use the conjugate gradient method to solve the system.

15

Summary of FlatteningF l a t t e n i n g :

C a l c u l a t e t h e e l e m e n t s o f t h e m a t r i c e s baD and , , .

U s e t h e c o n j u g a t e g r a d i e n t m e t h o d t o s o l v e bDyaDx and . T h e r e s u l t i n g iyxz i s t h e c o n f o r m a l m a p p i n g t o t h e c o m p l e x p l a n e .

C o m p o s e z w i t h i n v e r s e s t e r e o p r o j e c t i o n t o g e t a c o n f o r m a l m a p t o t h e u n i t s p h e r e .

16

Cortical Surface Flattening-Normal Brain

17

White Matter Segmentation and Flattening

18

Conformal Mapping of Neonate Cortex

19

Coordinate System on Cortical Surface

20

Principal Lines of Curvature on Brain Surface-I

21

Principal Lines of Curvatures on the Brain-II

22

Flattening Other Structures

23

Bladder Flattening

24

3D Ultrasound Cardiac Heart Map

25

High Intelligence=Bad Digestion

Low Intelligence=Good Digestion

Basic Principle

26

Flattening a Tube(1) Solve

1 1

0 0

10\ 0

onuonu

onu

(2) Make a cut from 0 to 1 .

Make sure u is increasing along the cut.

27

Flattening a Tube-Continued

( 3 ) C a l c u l a t e v o n t h e b o u n d a r y l o o p

00 1 cutcut

b y i n t e g r a t i o n

dsn

uds

s

vv

( 4 ) S o l v e D i r i c h l e t p r o b l e m u s i n g b o u n d a r y v a l u e s o f v .

I f y o u w a n t , s c a l e s o 2h , t a k e ivue

t o g e t a n a n n u l u s .

v = g ( u ) + h

u = 1u = 0

v = g ( u )

28

Flattening Without Distortion-I

In practice, once the tubular surface has been flattened into a rectangular shape, it will need to be visually inspected for pathologies. We present a simple technique by which the entire colon surface can be presented to the viewer as a sequence of images or cine. In addition, this method allows the viewer to examine each surface point without distortion at some time in the cine. Here, we will say a mapping is without distortion at a point if it preserves the intrinsic distance there. It is well known that a surface cannot in general be flattened onto the plane without some distortion somewhere. However, it may be possible to achieve a surface flattening which is free of distortion along some curve. A simple example of this is the familiar Mercator projection of the earth, in which the equator appears without distortion. In our case, the distortion free curve will be a level set of the harmonic function (essentially a loop around the tubular colon surface), and will correspond to the vertical line through the center of a frame in the cine. This line is orthogonal to the “path of flight” so that every point of the colon surface is exhibited at some time without distortion.

29

Flattening Without Distortion-II

S u p p o s e w e h a v e c o n f o r m a l l y f l a t t e n e d t h e c o l o n s u r f a c e o n t o a r e c t a n g l e

,,0 max uR . L e t F b e t h e i n v e r s e o f t h i s m a p p i n g , a n d l e t vu ,22 b e t h e a m o u n t b y W h i c h F s c a l e s a s m a l l a r e a n e a r vu , , i . e . l e t 0 b e t h e “ c o n f o r m a l f a c t o r ” f o r F . F i x 0 w , a n d f o r e a c h max0 ,0 uu d e f i n e a s u b s e t

RwuwuR ,, 000 w h i c h w i l l c o r r e s p o n d t o t h e c o n t e n t s o f a c i n e f r a m e . W e d e f i n e a m a p p i n g

u

u

v

dvvudvvuGvu0 0

0 ,,,,ˆ,ˆ .

30

Flattening Without Distortion-III

W e h a v e

vu

dvvu

vv

uuvudG

u

u

v

vu

vu

,0

,,ˆˆ

ˆˆ,

0

0

,

10

01,, 00 vuvudG .

This implies that composition of the flattening

map with G sends level set loop 0uu on the surface to the vertical line 0u in the vu plane without distortion. In addition, it follows from the formula for dG that lengths measured in the u direction accurately reflect the lengths of corresponding curves on the surface.

31

Introduction: Colon Cancer

US: 3rd most common diagnosed cancerUS: 3rd most frequent cause of deathUS: 56.000 deaths every year

Most of the colorectal cancers arise from preexistent adenomatous polyps

Landis S, Murray T, Bolden S, Wingo Ph.Cancer Statitics 1999. Ca Cancer J Clin. 1999; 49:8-31.

32

Problems of CT Colonography

Proper preparation of bowelHow to ensure complete inspection

Nondistorting colon flattening program

33

Nondistorting colon flatteningSimulating pathologist’ approachNo Navigation is neededEntire surface is visualized

34

Nondistorting Colon Flattening

Using CT colonography dataStandard protocol for CT colonographyTwenty-Six patients (17 m, 9 f)Mean age 70.2 years (from 50 to 82)

35

Flattened Colon

36

Polyps Rendering

37

Finding Polyps on Original Images

38

Polyp Highlighted

39

Path-Planning Deluxe

40

Coronary Vessels-Rendering

41

Coronary Vessels: Fly-Through

42

Area-Preserving Flows-IL e t M b e a c l o s e d , c o n n e c t e d n - d i m e n s i o n a l m a n i f o l d . V o l u m e f o r m :

0)(

...,)( ,1

xg

dxdxdxdxxg n

T h eo rem (M o ser): M , N com pac t m an ifo ld s w ith vo lu m e fo rm s and . A ssu m e th a t M an d N a re d iffeo m o rp h ic . If

NM ,

then there exists a diffeomorphism of M into N taking into .

43

Area-Preserving Flows-IIThe basic idea of the proof of the

theorem is the contruction of an orientation-preserving automorphism homotopic to the identity.

As a corollary, we get that given M and N any two diffeomorphic surfaces with the same total area, there exists are area-preserving diffeomorphism.This can be constructed explicitly via a PDE.

44

Area-Preserving Flows for the Sphere-I

F i n d a o n e - p a r a m e t e r f a m i l y o f v e c t o r f i e l d s

1,0, tu t a n d s o l v e t h e O D E

ttudt

d t

t o g e t a f a m i l y o f d i f f e o m o r p h i s m s t s u c h t h a t

id0 a n d

)det()det(1det DftDftD t .

fS2

S2 f o

N

45

Area-Preserving Flows for the Sphere-IIT o f i n d tu , s o l v e

)det(1 Df ,

t h e n

tDftu t

)det(1

46

Area-Preserving Flows of Minimal Distortion

L e t M a n d N b e t w o c o m p a c t s u r f a c e s w i t h R i e m a n n i a n m e t r i c s h a n d g r e s p e c t i v e l y , a n d l e t b e a n a r e a p r e s e r v i n g m a p . T h i s

m e a n s i f g a n d h a r e t h e a r e a f o r m s t h e n

.)(*hg

M a n y o t h e r a r e a p r e s e r v i n g m a p s f r o m NM ( j u s t c o m p o s e w i t h a n y o t h e r a r e a p r e s e r v i n g m a p ) . W h i c h o n e h a s t h e s m a l l e s t d i s t o r t i o n ? M i n i m i z e t h e D i r i c h l e t i n t e g r a l w i t h r e s p e c t t o a r e a - p r e s e r v i n g m a p s :

J ( þ ) = 1 = 2R

Mj D þ j 2 Ò h

T h i s l e a d s t o e x p l i c i t g r a d i e n t d e s c e n t e q u a t i o n s . M e t h o d w i l l b e d i s c u s s e d w h e n w e d e s c r i b e M o n g e - K a n t o r o v i c h a l g o r i t h m s .

47

Registration and Mass Transport

Image registration is the process of establishing a common geometric frame of reference from two or more data sets from the same or different imaging modalities taken at different times.

Multimodal registration proceeds in several steps. First, each image or data set to be matched should be individually calibrated, corrected from imaging distortions, cleaned from noise and imaging artifacts. Next, a measure of dissimilarity between the data sets must be established, so we can quantify how close an image is from another after transformations are applied to them. Similarity measures include the proximity of redefined landmarks, the distance between contours, thedifference between pixel intensity values. One can extract individual featuresthat to be matched in each data set. Once features have been extracted from each image, they must be paired to each other. Then, a the similarity measure between the paired features is formulated can be formulated as an optimization problem.

We can use Monge-Kantorovich for the similarity measure in this procedure.

48

Mass Transportation ProblemsOriginal transport problem was proposed

by Gaspar Monge in 1781, and asks to move a pile of soil or rubble to an excavation with the least amount of work.

Modern measure-theoretic formulation given by Kantorovich in 1942. Problem is therefore known as Monge-Kantorovich Problem (MKP).

Many problems in various fields can be formulated in term of MKP: statistical physics, functional analysis, astrophysics, reliability theory, quality control, meteorology, transportation, econometrics, expert systems, queuing theory, hybrid systems, and nonlinear control.

49

Monge-Kantorovich Mass Transfer Problem-I

W e c o n s i d e r t w o d e n s i t y f u n c t i o n s

R

ö 0 ( x ) d x =R

ö T ( x ) d x W e w a n t

M : R d ! R d w h i c h f o r a l l b o u n d e d s u b s e t s A ú R d R

x 2 A ö T ( x ) d x =R

M ( x ) 2 A ö 0 ( x ) d x F o r M s m o o t h a n d 1 - 1 , w e h a v e ( J a c o b i a n e q u a t i o n ) )())(())((det 0 xxMxM T W e c a l l s u c h a m a p M m a s s p r e s e r v i n g ( M P ) .

50

MK Mass Transfer Problem-II

J a c o b i a n p r o b l e m h a s m a n y s o l u t i o n s . W a n t o p t i m a l o n e ( L p -K a n t o r o v i c h - W a s s e r s t e i n m e t r i c )

d p ( ö 0 ; ö 1 ) p : = in f M

Rj M ( x ) à x j p ö 0 ( x ) d x

O p t i m a l m a p ( w h e n i t e x i s t s ) c h o o s e s a m a p w i t h p r e f e r r e d g e o m e t r y ( l i k e t h e R i e m a n n M a p p i n g T h e o r e m ) i n t h e p l a n e .

51

Algorithm for Optimal Transport-I

Ò0;Ò1 ú R d

Subdomains with smooth boundaries and positive densities:

RÒ0

ö0 =R

Ò1ö1

We consider diffeomorphisms which map one density to theother:

öo = det(Duà)ö1 î uà

We call this the mass preservation (MP) property. We let u be ainitial MP mapping.

52

Algorithm for Optimal Transport-II

We consider a one-parameter family of MP maps derived as follows:

uà := u î sà 1; s = s(á;t); ö0 = det(Ds)ö0 î s

Notice that from the MP property of the mapping s, and definition of the family,

uàt = àö0

1Duà áð; ð = ö0st î sà 1

div ð = 0

53

Algorithm for Optimal Transport-III

M(t) =R

Ò0Ð(uà(x; t) à x)ö0(x) dx

=R

Ð(u(y) à s(y; t))ö0(y) dy; x = s(y; t); sã(ö0(x)dx) = ö0(y)dy

M 0(t) = àRhÐ0(u à s); stiö0dy

= àRhÐ0(uà(x;t) à x); ö0st î sà 1i dx

= àR

Ò0hÐ0(uà(x;t) à x); ði dx

We consider a functional of the following form which we infimize with respect tothe maps :

Taking the first variation:

uà

54

Algorithm for Optimal Transport-IV

ð = Ð0(uà à x) + r p

div ð = 0

ðj@Ò0 tangential to @Ò0

É p+ div (Ð0(uà à x)) = 0; on Ò0

@n~@p + n~áÐ0(uà à x) = 0; on @Ò0

First Choice:

This leads to following system of equations:

uàt = à 1=ö0Duà á(Ð0(uà à x) + r p)

55

Algorithm for Optimal Transport-V

Duà á(I à r É à 1r á)Ð0(uà à x)@t@uà = à

ö0

1

This equation can be written in the non-local form:

At optimality, it is known that

Ð0(uà à x) = r ë

where is a function. Notice therefore for an optimalsolution, we have that the non-local equation becomes

@t@uà = 0

56

Solution of L2 M-K and Polar Factorization

uà = à 1=ö0Duà(uà à r É à 1 div(uà))

Ð(x) = 2jxj2

uà = u î sà 1 = r w + ÿ; div(ÿ) = 0 H elmholtz decomp:

For the L2 Monge-Kantorovich problem, we take

This leads to the following “non-local” gradient descent equation:

Notice some of the motivation for this approach. We take:

The idea is to push the fixed initial u around (considered as a vectorfield) using the 1-parameter family of MP maps s(x,t), in such a manneras to remove the divergence free part. Thus we get that at optimality

u = r w î s P olar factorization

57

Example of Mass Transfer-I

We want to map the Lena image to the Tiffany one.

58

Example of Mass Transfer-II

The first image is the initial guess at a mapping. The second isthe Monge-Kantorovich improved mapping.

59

Morphing the Densities-I

V(t;x) = x + t(uopt(x) à x)

60

Morphing the Densities-II (Brain Sag)

61

Deformation Map

Brain deformation sequence. Two 3D MR data sets were used. First is pre-operative, and second during surgery, after craniotomy and opening of the dura. First image shows planar slice while subsequent images show 2D projections of 3D surfaces which constitute path from original slice. Here time t=0, 0.33, 0.67,and 1.0. Arrows indicate areas of greatest deformation.

62

Morphing-II

63

Morphing-III

64

Surface Warping-I

M-K allows one to find area-correctingflattening. After conformally flatteningsurface, define density mu_0 to be determinant ofJacobian of inverse of flattening map, and mu_1 to be constant. MK optimal map is then area-correcting.

65

Surface Warping-II

66

67

68

69

70

71

fMRI and DTI for IGS

72

Data Fusion

73

More Data Fusion

74

Scale in Biological Systems

75

Multiscale / Complex System Modeling (from Kevrekidis)

“Textbook” engineering modeling:macroscopic behavior through macroscopic models(e.g. conservation equations augmented by closures)

Alternative (and increasingly frequent) modeling situation: Models

at a FINE / ATOMISTIC / STOCHASTIC level Desired Behavior

At a COARSER, Macroscopic Level E.g. Conservation equations, flow, reaction-diffusion,

elasticity Seek a bridge

Between Microscopic/Stochastic Simulation And “Traditional, Continuum” Numerical AnalysisWhen closed macroscopic equations are not available in closed

form

76

Micro/Macro Models-Scale I

77

Micro/Macro Models-Scale II

78

How to Move Curves and Surfaces

Parameterized Objects: methods dominate control and visual tracking; ideal for filtering and state space techniques.

Level Sets: implicitly defined curves and surfaces. Several compromises; narrow banding, fast marching.

Minimize Directly Energy Functional: conjugate gradient on triangulated surface (Ken Brakke); dominates medical imaging.

79

Diffusions

Explains a wide range of physical phenomenaHeat flowDiffusive transport: flow of fluids (i.e., water,

air)

Modeling diffusion is important At macroscopic scale by a partial differential

equation (PDE)At microscopic scale, as a collection of

particles undergoing random walks

We are interested in replacing PDE by the associated microscopic system

80

81

Interacting Particle Systems-I

Spitzer (1970): “New types of random walk models with certain interactions between particles”

Defn: Continuous-time Markov processes on certain spaces of particle configurations

Inspired by systems of independent simple random walks on Zd or Brownian motions on Rd

Stochastic hydrodynamics: the study of density profile evolutions for IPS

82

Interacting Particle Systems-II

Exclusion process: a simple interaction, precludes multiple occupancy--a model for diffusion of lattice gas

Voter model: spatial competition--The individual at a site changes opinion at a

rate proportional to the number of neighbors who disagree

Contact process: a model for contagion--Infected sites recover at a rate while healthy

sites are infected at another rate

Our goal: finding underlying processes of curvature flows

83

Motivations

Do not use PDEs

IPS already constructed on a discrete lattice (no discretization)

Increased robustness towards noise and ability to include noise processes in the given system

84

Construction of IPS-I

S : a set of sites, e.g. S= Zd

W: a phase space for each site, W={0,1}

The state space: X=WS

Process X

Local dynamics of the system: transition measures c

85

Construction of IPS-II

Connection between the process and the rate function c:

Connection to the evolution of a profile function:

),(),( xtxtdt

d

)(),0( xmt

86

Curvature Driven Flows

87

Euclidean and Affine Flows

88

Euclidean and Affine Flows

89

Gauss-Minkowki Map

90

Parametrization of Convex Curves

91

Evolution of Densities

92

Curve Shortening Flows

93

Main Convergence Result

94

Birth/Death Zero Range Processes-I

S: discrete torus TN, W=N

Particle configuration space: N TN

Markov generator:

)()()( 12 fLfLNLf o

95

Birth/Death Zero Range Processes-II

Markov generator:

)()()( 12 fLfLNLf o

)](2)()())[((2

1)( 1,1,

0 fffigfL ii

Ti

ii

N

elsej

iijj

iijj

jii

),(

0)(,,1)(

0)(,11)(

)(1,

96

Birth/Death Zero Range Process-III

Markov generator:

)()()( 12 fLfLNLf o

NTi

ii ffidffibfL )]()())[(()]()())[(()( ,,1

elsej

ijjji

)(

1)()(,

elsej

iijjji

)(

,0)(,1)()(,

97

The Tangential Component is Important

98

Curve Shortening as Semilinear Diffusion-I

99

Curve Shortening as Semilinear Diffusion-II

100

Curve Shortening as Semilinear Diffusion-III

101

Nonconvex Curves

102

Stochastic Interpretation-I

103

Stochastic Interpretation-II

104

Stochastic Interpretation-III

105

Stochastic Curve Shortening

106

Example of Stochastic Segmentation

107

Stochastic Tracking

108

ConclusionsStochastic Methods are attractive

alternative to level sets.

No increase in dimensionality.

Intrinsically discrete.

Robustness to noise.

Combination with other methods, e.g. Bayesian.