Towards a Multiscale Figural Geometry Stephen Pizer Andrew Thall, Paul Yushkevich Medical Image Display & Analysis Group

Towards aTowards aMultiscale Figural Geometry Multiscale Figural Geometry

Towards aTowards aMultiscale Figural Geometry Multiscale Figural Geometry

Stephen PizerStephen PizerAndrew Thall, Paul YushkevichAndrew Thall, Paul Yushkevich

www.cs.unc.edu/Research/Imagewww.cs.unc.edu/Research/Image

Medical Image Display & Analysis GroupMedical Image Display & Analysis GroupUniversity of North Carolina, Chapel HillUniversity of North Carolina, Chapel Hill

Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, NIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grantNIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grant

Stephen PizerStephen PizerAndrew Thall, Paul YushkevichAndrew Thall, Paul Yushkevich

www.cs.unc.edu/Research/Imagewww.cs.unc.edu/Research/Image

Medical Image Display & Analysis GroupMedical Image Display & Analysis GroupUniversity of North Carolina, Chapel HillUniversity of North Carolina, Chapel Hill

Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, NIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grantNIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grant

Intrinsic Object-Based Geometry Intrinsic Object-Based Geometry Suitable for Shape DescriptionSuitable for Shape Description

Intrinsic Object-Based Geometry Intrinsic Object-Based Geometry Suitable for Shape DescriptionSuitable for Shape Description

The needThe need: object-based positional, : object-based positional, orientational, and metric correspondence orientational, and metric correspondence among topologically figurally equivalent among topologically figurally equivalent objects or groups of objectsobjects or groups of objects Boundary of objectBoundary of object In interior of objectIn interior of object Exterior to object, Exterior to object,

between objectsbetween objects

Suitability for shape description impliesSuitability for shape description implies Magnification invarianceMagnification invariance At all levels of spatial scale (locality)At all levels of spatial scale (locality)

The needThe need: object-based positional, : object-based positional, orientational, and metric correspondence orientational, and metric correspondence among topologically figurally equivalent among topologically figurally equivalent objects or groups of objectsobjects or groups of objects Boundary of objectBoundary of object In interior of objectIn interior of object Exterior to object, Exterior to object,

between objectsbetween objects

Suitability for shape description impliesSuitability for shape description implies Magnification invarianceMagnification invariance At all levels of spatial scale (locality)At all levels of spatial scale (locality)

Definition of Spatial ScaleDefinition of Spatial ScaleDefinition of Spatial ScaleDefinition of Spatial Scale

Scale: Scale: There are two separate and different notions:There are two separate and different notions: Spatial coverage of each geometric elementSpatial coverage of each geometric element Distance of inter-element communicationDistance of inter-element communication

Scale: Scale: There are two separate and different notions:There are two separate and different notions: Spatial coverage of each geometric elementSpatial coverage of each geometric element Distance of inter-element communicationDistance of inter-element communication

Mesh of voxels Boundary atom mesh Mesh of voxels Boundary atom mesh Medial atom meshMedial atom mesh

Multiple Spatial ScalesMultiple Spatial ScalesMultiple Spatial ScalesMultiple Spatial Scales

Scale aspects Scale aspects Geometric element coverageGeometric element coverage Inter-element communication distanceInter-element communication distance

Thesis: The two measures need to be similarThesis: The two measures need to be similar Multiple scale levelsMultiple scale levels

Scale aspects Scale aspects Geometric element coverageGeometric element coverage Inter-element communication distanceInter-element communication distance

Thesis: The two measures need to be similarThesis: The two measures need to be similar Multiple scale levelsMultiple scale levels

Mesh of voxels Mesh of voxels Medial atom meshMedial atom mesh

Figural Geometry (position, orientation, Figural Geometry (position, orientation, local size) Comes from Medial Atomslocal size) Comes from Medial Atoms

Figural Geometry (position, orientation, Figural Geometry (position, orientation, local size) Comes from Medial Atomslocal size) Comes from Medial Atoms

Medial atomsMedial atoms (1st order medial locus)(1st order medial locus) xx, , FF = (= (bb,,n,bn,b) frame, r, ) frame, r, (object angle) (object angle)

bb in direction of minimum dr/ds (- in direction of minimum dr/ds (-xxr)r)bb in level direction of r [3D] in level direction of r [3D]nn is normal to medial skeleton is normal to medial skeleton

Medial atomsMedial atoms (1st order medial locus)(1st order medial locus) xx, , FF = (= (bb,,n,bn,b) frame, r, ) frame, r, (object angle) (object angle)

bb in direction of minimum dr/ds (- in direction of minimum dr/ds (-xxr)r)bb in level direction of r [3D] in level direction of r [3D]nn is normal to medial skeleton is normal to medial skeleton

Figurally RelevantFigurally RelevantSpatial Scale LevelsSpatial Scale LevelsFigurally RelevantFigurally Relevant

Spatial Scale LevelsSpatial Scale Levels Multiple objectsMultiple objects Individual objectIndividual object

i.e., multiple figuresi.e., multiple figures Individual figureIndividual figure

mesh of medial atomsmesh of medial atoms Figural sectionFigural section

i.e., multiple figural sectionsi.e., multiple figural sections figural section centered at medial atomfigural section centered at medial atom

Figural section more finely spaced, ..Figural section more finely spaced, .. Boundary sectionBoundary section Boundary section more finely spaced, ...Boundary section more finely spaced, ...

Multiple objectsMultiple objects Individual objectIndividual object

i.e., multiple figuresi.e., multiple figures Individual figureIndividual figure

mesh of medial atomsmesh of medial atoms Figural sectionFigural section

i.e., multiple figural sectionsi.e., multiple figural sections figural section centered at medial atomfigural section centered at medial atom

Figural section more finely spaced, ..Figural section more finely spaced, .. Boundary sectionBoundary section Boundary section more finely spaced, ...Boundary section more finely spaced, ...

medial atom

Figural Types and the Manifold Figural Types and the Manifold of Medial Atomsof Medial Atoms

Figural Types and the Manifold Figural Types and the Manifold of Medial Atomsof Medial Atoms

Slab

Tube

M-rep Boundary implied from interpolated continuous manifold of medial atoms

Magnification Invariance at Magnification Invariance at All Spatial Scale LevelsAll Spatial Scale Levels


Inside boundary featuresInside boundary features radius of curvature-radius of curvature-

proportional distancesproportional distances Inside figural sectionsInside figural sections

r-proportional distancesr-proportional distances Inside individual figuresInside individual figures

r-proportional distancesr-proportional distances

Inside boundary featuresInside boundary features radius of curvature-radius of curvature-

proportional distancesproportional distances Inside figural sectionsInside figural sections

r-proportional distancesr-proportional distances Inside individual figuresInside individual figures

r-proportional distancesr-proportional distances



Individual objectIndividual object In interface between figuresIn interface between figures

blended r-proportional distances blended r-proportional distances Multiple objectsMultiple objects

Outside objectsOutside objects blended r-proportional distancesblended r-proportional distances concavities’ effect disappear with concavities’ effect disappear with

distancedistance

Individual objectIndividual object In interface between figuresIn interface between figures

blended r-proportional distances blended r-proportional distances Multiple objectsMultiple objects

Outside objectsOutside objects blended r-proportional distancesblended r-proportional distances concavities’ effect disappear with concavities’ effect disappear with

distancedistance

Figural (Medially based) Geometry Figural (Medially based) Geometry Figural (Medially based) Geometry Figural (Medially based) Geometry

Locally magnification invariant Locally magnification invariant means r-proportional distancesmeans r-proportional distances Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary

normals)normals)

Medially (figurally) based coordinate Medially (figurally) based coordinate system provides intrinsic coordinatessystem provides intrinsic coordinates Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary

normals)normals) Overall metric??Overall metric??

Locally magnification invariant Locally magnification invariant means r-proportional distancesmeans r-proportional distances Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary

normals)normals)

Medially (figurally) based coordinate Medially (figurally) based coordinate system provides intrinsic coordinatessystem provides intrinsic coordinates Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary

normals)normals) Overall metric??Overall metric??

Spatial coordinates capable of Spatial coordinates capable of providing correspondence at any scaleproviding correspondence at any scale

Spatial coordinates capable of Spatial coordinates capable of providing correspondence at any scaleproviding correspondence at any scale

Medial coordinates (u[,v])Medial coordinates (u[,v])continuous, integer multiples of continuous, integer multiples of r r

at samples, where at samples, where is scale level is scale level r-proportional along medial surfacer-proportional along medial surface

Boundary coordinates (u[,v],t)Boundary coordinates (u[,v],t) Spatial coordinates (u[,v],t,d/r)Spatial coordinates (u[,v],t,d/r)

From implied boundary along From implied boundary along geodesic of distance that at geodesic of distance that at boundary is in normal directionboundary is in normal direction

Medial coordinates (u[,v])Medial coordinates (u[,v])continuous, integer multiples of continuous, integer multiples of r r

at samples, where at samples, where is scale level is scale level r-proportional along medial surfacer-proportional along medial surface

Boundary coordinates (u[,v],t)Boundary coordinates (u[,v],t) Spatial coordinates (u[,v],t,d/r)Spatial coordinates (u[,v],t,d/r)

From implied boundary along From implied boundary along geodesic of distance that at geodesic of distance that at boundary is in normal directionboundary is in normal direction

Inside object: (u[,v],t,d/r)Inside object: (u[,v],t,d/r) (u,v) give multiples of r(u,v) give multiples of r distance on medial sheet along geodesics of r-distance on medial sheet along geodesics of r-

proportional distanceproportional distance Outside objectOutside object

Near boundary (inside focal surface): Near boundary (inside focal surface): (u[,v],t,d/r)(u[,v],t,d/r)

Far outside boundary: (u[,v],t,d/r) via distance Far outside boundary: (u[,v],t,d/r) via distance (scale) related figural convexification(scale) related figural convexification

geodesics do not crossgeodesics do not cross

Inside object: (u[,v],t,d/r)Inside object: (u[,v],t,d/r) (u,v) give multiples of r(u,v) give multiples of r distance on medial sheet along geodesics of r-distance on medial sheet along geodesics of r-

proportional distanceproportional distance Outside objectOutside object

Near boundary (inside focal surface): Near boundary (inside focal surface): (u[,v],t,d/r)(u[,v],t,d/r)

Far outside boundary: (u[,v],t,d/r) via distance Far outside boundary: (u[,v],t,d/r) via distance (scale) related figural convexification(scale) related figural convexification

geodesics do not crossgeodesics do not cross

Figural Coordinates for Single FigureFigural Coordinates for Single Figure Figural Coordinates for Single FigureFigural Coordinates for Single Figure

Inside figures not near hingesInside figures not near hinges same as for single figuresame as for single figure

Outside object: see two slides laterOutside object: see two slides later

Inside figures not near hingesInside figures not near hinges same as for single figuresame as for single figure

Outside object: see two slides laterOutside object: see two slides later

Figural Coordinates for Object Figural Coordinates for Object Made From Multiple Attached FiguresMade From Multiple Attached Figures


Blend in hinge regionsBlend in hinge regions w=(dw=(d11/r/r11 - d - d2 2 /r/r22 )/T )/T Blended d/r when |w| <1 and u-uBlended d/r when |w| <1 and u-u00 < T < T Implicit boundary: (u,w, t)Implicit boundary: (u,w, t) Implicit normals and geodesicsImplicit normals and geodesics

Blend in hinge regionsBlend in hinge regions w=(dw=(d11/r/r11 - d - d2 2 /r/r22 )/T )/T Blended d/r when |w| <1 and u-uBlended d/r when |w| <1 and u-u00 < T < T Implicit boundary: (u,w, t)Implicit boundary: (u,w, t) Implicit normals and geodesicsImplicit normals and geodesics



Near boundary: via blendingNear boundary: via blending Far outside boundaryFar outside boundary

same convexification principle as with same convexification principle as with single figuressingle figures

blend geodesics according to dblend geodesics according to dkk/r/rkk

Near boundary: via blendingNear boundary: via blending Far outside boundaryFar outside boundary

same convexification principle as with same convexification principle as with single figuressingle figures

blend geodesics according to dblend geodesics according to dkk/r/rkk

Figural Coordinates between Objects Figural Coordinates between Objects Figural Coordinates between Objects Figural Coordinates between Objects

Uses of CorrespondenceUses of CorrespondenceUses of CorrespondenceUses of Correspondence

Geometric typicality (segment’n prior)Geometric typicality (segment’n prior) by boundary point to boundary point correspondenceby boundary point to boundary point correspondence

Geometric representation to image match measureGeometric representation to image match measure by boundary-relative correspondenceby boundary-relative correspondence

in collar about boundary out to fixed distance via metricin collar about boundary out to fixed distance via metric union of collar and interior of objectunion of collar and interior of object

For homologies used in statistical shape For homologies used in statistical shape characterization: leads to localitycharacterization: leads to locality

For elements in mechanical calculationsFor elements in mechanical calculations For comparison of segmented For comparison of segmented

object to true object object to true object

Geometric typicality (segment’n prior)Geometric typicality (segment’n prior) by boundary point to boundary point correspondenceby boundary point to boundary point correspondence

Geometric representation to image match measureGeometric representation to image match measure by boundary-relative correspondenceby boundary-relative correspondence

in collar about boundary out to fixed distance via metricin collar about boundary out to fixed distance via metric union of collar and interior of objectunion of collar and interior of object

For homologies used in statistical shape For homologies used in statistical shape characterization: leads to localitycharacterization: leads to locality

For elements in mechanical calculationsFor elements in mechanical calculations For comparison of segmented For comparison of segmented

object to true object object to true object

Open Geometric QuestionsOpen Geometric QuestionsOpen Geometric QuestionsOpen Geometric Questions

Full space metricFull space metric Outside figure convexificationOutside figure convexification Reflecting scale levelReflecting scale level

Representing toleranceRepresenting tolerance Controlling IIControlling IImedial locusmedial locus, D, Dxx

22r, r, xxrr Principled means forPrincipled means for

Inter-figural blending of figural metrics for attached Inter-figural blending of figural metrics for attached figuresfigures

Inter-object blending of object metricsInter-object blending of object metrics

Full space metricFull space metric Outside figure convexificationOutside figure convexification Reflecting scale levelReflecting scale level

Representing toleranceRepresenting tolerance Controlling IIControlling IImedial locusmedial locus, D, Dxx

22r, r, xxrr Principled means forPrincipled means for

Inter-figural blending of figural metrics for attached Inter-figural blending of figural metrics for attached figuresfigures

Inter-object blending of object metricsInter-object blending of object metrics

Figural (Medially based) GeometryFigural (Medially based) GeometryInternal points on single figureInternal points on single figure

Figural (Medially based) GeometryFigural (Medially based) GeometryInternal points on single figureInternal points on single figure

Sails are separate (Sails are separate (>0)>0) Both sails move with motion on medial surface Both sails move with motion on medial surface

Sails are separate (Sails are separate (>0)>0) Both sails move with motion on medial surface Both sails move with motion on medial surface

p

p

x

s

b

n

Figural (Medially based) GeometryFigural (Medially based) GeometryBranches and EndsBranches and Ends

Figural (Medially based) GeometryFigural (Medially based) GeometryBranches and EndsBranches and Ends

EndsEnds Sails come together (Sails come together (=0)=0) Boundary is vertex (2D) or crest (3D)Boundary is vertex (2D) or crest (3D) Medial disk or ball osculatesMedial disk or ball osculates

BranchesBranches Medial disk or ball tritangentMedial disk or ball tritangent Swallowtail of medial atomSwallowtail of medial atom

Retrograde motion of one sailRetrograde motion of one sail

EndsEnds Sails come together (Sails come together (=0)=0) Boundary is vertex (2D) or crest (3D)Boundary is vertex (2D) or crest (3D) Medial disk or ball osculatesMedial disk or ball osculates

BranchesBranches Medial disk or ball tritangentMedial disk or ball tritangent Swallowtail of medial atomSwallowtail of medial atom

Retrograde motion of one sailRetrograde motion of one sail

Geometrically Geometrically smaller scale smaller scale Interpolate (1st order) finer spacing of Interpolate (1st order) finer spacing of

atomsatoms Residual atom change, i.e., localResidual atom change, i.e., local

ProbabilityProbability At any scale, relates figurally homologous At any scale, relates figurally homologous

pointspoints Markov random field relating medial Markov random field relating medial

atom withatom with its immediate neighbors at that scaleits immediate neighbors at that scale its parent atom at the next larger scale and the its parent atom at the next larger scale and the

corresponding positioncorresponding position its children atoms its children atoms

Geometrically Geometrically smaller scale smaller scale Interpolate (1st order) finer spacing of Interpolate (1st order) finer spacing of

atomsatoms Residual atom change, i.e., localResidual atom change, i.e., local

ProbabilityProbability At any scale, relates figurally homologous At any scale, relates figurally homologous

pointspoints Markov random field relating medial Markov random field relating medial

atom withatom with its immediate neighbors at that scaleits immediate neighbors at that scale its parent atom at the next larger scale and the its parent atom at the next larger scale and the

corresponding positioncorresponding position its children atoms its children atoms

Multiscale Geometry and Probability Multiscale Geometry and Probability for a Figurefor a Figure

Multiscale Geometry and Probability Multiscale Geometry and Probability for a Figurefor a Figure

coarse, global

coarse resampled

fine, local

Documents

Towards a Multiscale Figural Geometry Stephen Pizer Andrew Thall, Paul Yushkevich Medical Image Display & Analysis Group