Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz,...

Unified Joint Feature Registration for Brain Anatomical Alignment

Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan

Image Processing and Analysis GroupDepartments of Electrical Engineering

Yale University

Brain Anatomical Alignment• Brains are different:

– Shape.– Structure.

• Direct comparison of brains between different subject is not very accurate.

• Statistically and quantitatively more accurate study requires the brain image data to be put in a common “normalized” space through alignment.

• Examples of areas that need brain registration:– Studying structure-function connection.– Tracking temporal changes.– Generating probabilistic atlases.– Creating deformable atlases.

Studying Function-Structure Connection

Brain Function

Image

Alignment of Subjects

Comparison of Subjects After Alignment

Direct Comparison of Subjects Distribution Before Alignment

Distribution After Alignment

Inter-Subject Brain Registration

• Inter-subject brain registration: – Alignment of brain MRI images from different

subjects to remove some of the shape variability.

• Difficulties:– Complexity of the brain structure.– Variability between brains.

• Brain feature registration: – Choose a few salient structural features as a

concise representation of the brain for matching.

– Overcome complexity: only model important structural features.

– Overcome variability: only model consistent features.

Previous Work: 3D Sulcal Point Matching

Feature Extraction Extracted Point Features

Previous Work: 3D Sulcal Point Matching

Overlay of 5 subjects before TPS alignment:

After TPS alignment:

A Unified Feature Registration Method

Outer Cortex Surface

Major Sulcal Ribbons

All FeaturesPoint Feature

Representation

Point Feature Representation

Feature Extraction Feature Fusion

Feature

Matching

Subject I

Subject II

Non-rigid Feature Point Registration

Unification of Different Features

• Ability to incorporate different types of geometrical features.– Points.

– Curves.

– Open surface ribbons.

– Closed surfaces.

• Simultaneously register all features --- utilize the spatial inter-relationship between different features to improve registration.

Joint Clustering-Matching Algorithm (JCM)

Overcome Sub-sampling Problem

• Sub-sampling (e.g. clustering) reduces computational cost for matching.

• In-consistency problem with sub-sampling:

• The in-consistency can be overcome by sub-sampling (clustering) and matching simultaneously.

Joint Clustering-Matching Algorithm (JCM)

• JCM:

• Reduce computational cost using sub-sampled cluster centers.

• Accomplish optimal cluster placement through joint cluster-matching.

• Symmetric: two way matching.

MatchingClusters Center Set V

Clustering

Cluster Center Set U

Clustering

Point Set X Point Set YOriginal RPM

• Diagram:

JCM Energy Function

MatchingClusters Center Set V

Clustering

Cluster Center Set U

Clustering

Point Set X Point Set Y

K

a iai

xai

xN

vxm1 1

2||||

K

a jaj

yaj

yN

uym1 1

2||||

K

axaxa Lfvfu

1

22 ||||||),(||

K

ayaya Lfufv

1

22 ||||||),(||

Annealing: .loglog1 11 1

K

a

N

i

yai

yai

K

a

N

i

xai

xai

yx

mmTmmT

JCM Energy Function

• Fuzzy assignment + least squares energy function:

• Row and column summation constraints.

),( ME

K

a iai

xai

xN

vxm1 1

2||||

K

a jaj

yaj

yN

uym1 1

2||||

K

axaxa Lfvfu

1

22 ||||||),(||

K

ayaya Lfufv

1

22 ||||||),(||

.loglog1 11 1

K

a

N

i

yai

yai

K

a

N

i

xai

xai

yx

mmTmmT

JCM Example

• Matching 2 face patterns with JCM (click to play movie).

Experiments

Comparison of Different Features

• Different features can be used in our approach.

• Two types of features investigated:– Outer cortex surface.

– Major sulcal ribbons.

• Comparison of different methods:

Method I Method II Method III

Synthetic Study Setup

Template True Deformation (GRBF)

Target

Template RecoveryEstimated Deformation

(TPS)

Error Evaluation

Feature Matching

Change the choice of features to

compare method I, II and III

Results: Method I vs. Method III

• Outer cortical surface alone can not provide adequate information for sub-cortical structures.

• Combination of two features works better.

Results: Method II vs. Method III

• Major sulcal ribbons alone are too sparse --- the brain structures that are relatively far away from the ribbons got poorly aligned.

• Combination of two features works better.

Conclusion

• Combination of different features improves registration.

• Unified brain feature registration approach:– Capable of estimating non-rigid transformations without the

correspondence information.

– General + unified framework.

– Symmetric.

– Efficient.

Acknowledgements

• Members of the Image Processing and Analysis Group at Yale University: – Hemant Tagare.– Lawrence Staib. – Xiaolan Zeng. – Xenios Papademetris. – Oskar Skrinjar. – Yongmei Wang.

• Colleagues in the brain registration project:– Joseph Walline.

• Financial support is provided by the grants from the Whitaker Foundation, NSF, NIH.

Future Work

Estimating An Average Shape

• Given multiple sample shape (sample point sets), compute the average shape for which the joint distance between the samples and the average is the shortest.

Average ?

• Difficult if the correspondences between the sample points are unknown.

“Super” Clustering-Matching Algorithm (SCM)

• Diagram:

MatchingMatchable

ClustersOutlier Cluster

Clusters Center Set V

Clustering

Matchable Clusters

Outlier Cluster

Clusters Center Set U

Clustering

Point Set X Point Set Y

Average Point Set Z

Matching and

Estimating

End

• Further Information:– Web site: http://noodle.med.yale.edu/~chui/

End

2D Examples of RPM

Point Matching

Example Application: Face Matching

Example Application: Face Matching