Landmark localization and registration of 3D facial scans

for the evaluation of orthodontic treatments in maxillofacial and oral surgery

School of EECS: Prathap Nair, Dr Andrea CavallaroSchool of Medicine and Dentistry: Dr Lifong Zou

Mid-project update

What is the problem?

• To quantify 3D facial asymmetry

• Clinical diagnosis• Treatment planning• Post-treatment monitoring• Statistical studies on a large population

• What is rigid registration?• Alignment of 2 or more faces

• Classical approach: Iterative Closest Point (ICP) algorithm• Advantage

• no prior info needed• Disadvantage

• random points used for matching can lead to erroneous results

Approach: rigid registration

Example

Our approach

• Rigid registration based on landmarks • Landmark detection via Statistical Shape Analysis

BtG project: Achievement 1

• Improved accuracy

Red – before BtGGreen – after BtG

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

Key Landmar

ks

Coarse registration

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

Fine registration

Approach: overview

Test Scan

Reference scan

Detection of

Landmark Points

Detection of

Landmark Points

Coarse registratio

n using Key

landmarks

Detection of

Stable regions

Fine registration using

the Semantic Regions

Distanceestimatio

n

ICP Proposed approach

Example

BtG project: Achievement 2

• User friendly GUI• To ease burden on clinicians• User-feedback mechanisms

Conclusions

• Achievements• Improved landmark localisation accuracy • More user-friendly GUI with the user feedback

• Current work• Clinical evaluation of the landmark detection accuracy • Validation of 3D facial scan registration accuracy• Further improving the GUI based on clinician feedback

Contact:

prathap.nair@elec.qmul.ac.uk

andrea.cavallaro@elec.qmul.ac.uk