View
221
Download
1
Tags:
Embed Size (px)
Citation preview
Active Appearance Models
master thesis presentation
Mikkel B. Stegmann
IMM – June 20th 2000
Presentation outline
Aim
Method
Metacarpals – a case study
Discussion
Conclusion
Aim
To locate non-rigid objects in digital images
The vision utopia
• Fully automated
• General
• Specific
• Robust
• Accurate
• Holistic
• Non-parametric
• Fast
Active Appearance Models
A model-based approach towards segmentation
A priori knowledge is not programmed into the model, but learned through observation
Relies on statistical analysis of shape and texture variation in a training set
Derives a compact object class description which can be used to rapidly search images for new object instances
Model building
1) Data capture
Shape: point annotationTexture: pixel sampling
3) Statistical analysis
Principal component analysis on shape and texture
3) Combining shape and appearance
Shape and texture PCA is combined into a 3rd PCA
4) Model truncation
Parameters are truncated to satisfy a variance constraint
2) Normalisation
Shape: pose alignment using the Procrustes shape metric
Texture: photometric normalisation
Shape analysis
Shape is represented by a linear spline of landmarks:
X = ( x1, … , xn, y1, … , yn)T
• Assumes point correlation
• Requires point correspondence
Alignment w.r.t. position, scale, orientation
Principalcomponentanalysis
Compact shape representation
10 20 30 40 50 60 70 80 90 100
10
20
30
40
50
60
70
80
90
100
Texture analysis
Texture – the intensities across the object – is sampled inside the shape using a suitable warp function
Warp function: A piece-wise affine warp using the Delaunay triangulation
g = ( x1, … , xn)TPrincipalcomponentanalysis
Compact texturerepresentation
Combined Model
Shape and texture is combined into a compact model representation
This representation is capable of derforming in a similar manner to what is observed in the training set
Thus making the model specific to the class of objects it represents
Generative (self-contained)
Model Optimisation
Deforms the AAM to fit the image being searched
Assumes a linear relationship between model parameters and the observed fit:
C = RX
Solved using multivariate linear regression on alarge set of experiments
Actual dy (pixels)
Pre
dic
ted d
y (
pix
els
)
Implementation
Open source C++ API based on the Windows platform[and partly on VisionSDK, LAPACK, Intel MKL, ImageMagick a.o.]
Well documented [cross-referenced HTML and PDF]
Fast [using Intel BLAS for matrix handling and widely use of dynamic programming]
Suitable for education & research[lots of visual and numerical documentation: *.m *.avi *.bmp]
Example usage included[in the form of a console interface]
Metacarpals – a case study
20 x-ray images of the human hand supplied by Pronosco
Metacarpal 2, 3, 4 annotated using 50 points on each
Difficult segmentation problem due to large shape variability and the ambiguous nature of radiographs
Building the model
• Annotation of set of training images
• Capture of shape & texture
• Statistical analysis on shape & texture
Modes of variation
0 5 10 15 20 250
5
10
15
20
25
0 5 10 15 20 250
5
10
15
20
25
30
35
40
45
0 5 10 15 20 250
2
4
6
8
10
12
14
16
18
Shape Texture Combined
Metacarpal AAM
Image modality: radiographs (x-rays)
20 images/shapes in training set
300 points in shape model
~10.000 pixels in texture model
95% variation explained using 16 model parameters
Search
Metacarpal results
Using automatic initialisation
• Good mean location accuracy 0.98 pixel (point to border)
• Acceptable mean texture fit 6.57 gray levels (byte range)
Difficult to locate the exact bone extents at the proximal and distal end
mean pt. errors
proximal
distal
Discussion
“Hidden” benefits
• Automatic registration
• Variance analysis (group/longitudinal studies)
• Discrimination/interpretationusing the model parameters
Weaknesses
• Requires landmarks (point correspondence)
• Can only deform texture by moving edge points
• Not robust to large-scale texture noise
Discussion - cont’d
Image modalities on which AAMs has been evaluated successfully:
• Radiographs - x-rays of human hands
• Normal gray scale images - hands, pork carcasses
• MRI - human hearts
Initialisation has been added, thus making AAM a fully automated segmentation method
The AAM approach extends to 3D and multivariate imaging
Conclusion
AAM has been implemented and extended as a fully automated and data-driven approach towards image segmentation
AAM performs well on very different segmentation problems and different image modalities
Properties
• General
• Specific
• Captures domain knowledge without the need for technical knowledge
• Robust
• Non-parametric
• Self-contained
• Fast
fin
http://www.imm.dtu.dk/~aam