A Heuristic Image Search Algorithm for Active Shape Model

Research Article – SACJ No. 49, September 2012 35

A Heuristic Image Search Algorithm for Active Shape Model

Segmentation of the Caudate Nucleus and Hippocampus in

Brain MR Images of Children with FASD

A.A. Eicher∗, P. Marais∗, C. Warton†, S.W. Jacobson§†‡, J.L. Jacobson§†‡, C.D. Molteno§, E.M.Meintjes†¶

∗Department of Computer Science, University of Cape Town, Rondebosch 7701, South Africa†Department of Human Biology, University of Cape Town, Observatory 7925, South Africa§Department of Psychiatry and Mental Health, University of Cape Town, Observatory 7925, South Africa¶MRC/UCT Medical Imaging Research Unit, University of Cape Town, Observatory 7925, South Africa‡Department of Psychiatry and Behavioral Neurosciences, Wayne State University, Detroit, MI

ABSTRACT

Magnetic Resonance Imaging provides a non-invasive means to study the neural correlates of Fetal Alcohol SpectrumDisorder (FASD) - the most common form of preventable mental retardation worldwide. One approach aims to detectbrain abnormalities through an assessment of volume and shape of two sub-cortical structures, the caudate nucleus andhippocampus. We present a method for automatically segmenting these structures from high-resolution MR imagescaptured as part of an ongoing study into the neural correlates of FASD.

Our method incorporates an Active Shape Model, which is used to learn shape variation from manually segmentedtraining data. A modified discrete Geometrically Deformable Model is used to generate point correspondence betweentraining models. An ASM is then created from the landmark points. Experiments were conducted on the image searchphase of ASM segmentation, in order to find the technique best suited to segmentation of the hippocampus and caudatenucleus. Various popular image search techniques were tested, including an edge detection method and a method basedon grey profile Mahalanobis distance measurement. A novel heuristic image search method was also developed and tested.This heuristic method improves image segmentation by taking advantage of characteristics specific to the target data,such as a relatively homogeneous tissue colour in target structures.

Results show that ASMs that use the heuristic image search technique produce the most accurate segmentations. An

ASM constructed using this technique will enable researchers to quickly, reliably, and automatically segment test data for

use in the FASD study.

KEYWORDS: Active Shape Model, Geometrically Deformable Model, ASM

1 INTRODUCTION

Fetal Alcohol Spectrum Disorder (FASD) is especiallyprevalent amongst residents of the Western Cape re-gion of South Africa. This disorder affects the embryosof women who ingest alcohol whilst pregnant. Chil-dren can suffer serious central nervous system damageas a result. In order to gain a better understanding ofthis condition, 32 children with FASD1 were scannedusing Magnetic Resonance Imaging (MRI).

This paper forms part of an ongoing study aimedat assessing the neural correlates of FASD. One topicof study is the difference in shape, volume and areabetween brain structures, specifically the caudate nu-cleus and hippocampus, of healthy subjects and thosewith FASD. The primary goal of this work is to find

Email: A.A. Eicher [email protected]

19.7 to 13.7 years of age, mean 11.6 years

the best algorithm to automatically segment out cau-date nuclei and hippocampi from previously unseenbrain volumes that form part of the study into FASD.

Manual segmentation of MR images is time-intensive and prone to inter-observer and intra-observer variability [1, 2]. Automatic segmentationof MR images is generally quick and reproducible.It is therefore desirable to automatically segment theMR images used as part of the study. This will en-able researchers to quickly and objectively comparehealthy specimens to those affected by FASD. In orderto achieve this goal, an effective automatic segmenta-tion algorithm must be found.

We construct a Geometrically Deformable Model(GDM) that effectively and efficiently assigns land-mark points to the training data made available foruse as part of the FASD study. We then use anActive Shape Model (ASM) for automatic segmenta-tion. ASMs overcome spurious shape deformities by

36 Research Article – SACJ No. 49, September 2012

learning shape information from a training data set -thereby limiting model deformation to shapes similarto those found in the training set. ASMs are not a gen-eral solution to the image segmentation problem, butwork for a broad class of segmentation problems, espe-cially in the biomedical imaging field, where volumet-ric images such as MRI scans are segmented [3, 4, 5, 6].ASMs are therefore well suited to our specific needs.

ASMs are generated by performing PrincipalComponent Analysis (PCA) on a set of landmarkpoints. This process creates a statistical Point Dis-tribution Model (PDM) which is used to constrain thedeformation of shape models during the image seg-mentation process. The image segmentation processuses an image search algorithm, in which the modelis expanded and deformed iteratively to fit local im-age information (such as an edge). We implementedvarious image search algorithms, in order to find theone that is most suited to our test data. These in-cluded an edge detection method, a method based ongrey profile Mahalanobis distance measurement, anda heuristic image search method.

After segmentation, results were visualised andanalysed not only to make use of the segmented data,but also to evaluate the effectiveness of the segmenta-tion process. Experimental evaluation was employedto determine the most effective ASM constructionmethod. ASMs built to use the various image searchtechniques were tested on sample data that are char-acteristic of the data found in the FASD study. Sta-tistical analysis was conducted on the results, in orderto draw meaningful conclusions. This task enabled usto choose the best ASM construction technique for usein the future of the FASD study.

Results indicate that the heuristic method gen-erated the most accurate segmentation results. Themethod is not perfect - segmentation did fail on somedata sets. However, this segmentation failure was usu-ally due to extreme noise or low contrast. Thus, wecan conclude that the heuristic method displays ac-ceptable reliability for the segmentation of current andexpected future data to be used in the FASD study.

The paper is structured as follows. Section 2 pro-vides details of previous work that we build upon inthis research, as well as some fundamental techniquesused. Our test data are described in Section 3. Spe-cific details of GDM construction are given in Section4. A detailed description of our ASM constructiontechniques can be found in Section 5. This is followedby a discussion of our Heuristic Image Search algo-rithm in Section 6. Finally, we evaluate the perfor-mance of our algorithm, and present our conclusion.

2 BACKGROUND AND RELATED WORK

Many automatic segmentation methods have beenproposed in recent years - each has its advantages anddisadvantages. Each method is also suitable for cer-tain types of target data. GDMs are well suited to thesegmentation of 3D data where the target shapes areclosed surfaces. The problem with standard GDMs

is that they can deform to arbitrary shapes that arenot representative of the class of shapes that they aredesigned to fit. This problem is especially prevalentwhen segmenting noisy data, which contain many falsepositives. Shape priors, such as surface smoothnessconstraints, are sometimes used to limit deformationto a shape that is geometrically similar to the originalmodel, but it is still possible for models to deform intosuboptimal shapes [3].

ASMs were first proposed by Cootes in 1995 in anattempt to address the shortcomings of deformablemodels, by limiting model deformation to shapesfound within the Allowable Shape Domain (ASD) [7].ASMs achieve this by learning shape information frompre-segmented shape training sets. The first ASMswere limited to 2D, but 3D versions of the techniquefollowed soon after [4].

In order to build an ASM, landmark points mustfirst be assigned to training data. In 3D, it is all butimpossible to assign these landmark points manually.A GDM can be employed to automatically assign 3Dlandmark points to volumetric data. The GDM con-sists of a discrete mesh of vertices, which move in 3Dspace in reaction to forces exerted on them by vari-ous deformation terms. These terms cause the GDMto deform to fit a series of binary training shape vol-umes, thereby assigning a fixed number of vertices orlandmark points to the training data.

2.1 GDM Construction

The first task in creating the GDM is to develop aninitial mesh, which is then deformed to fit the targetstructure. In the original GDM paper, [8] use a regu-lar 20-sided icosahedron as their initial mesh. [9] usetheir gravity centre method to stitch together poly-gons from user-generated slices. [10] create a meshusing manually segmented data. These data are seg-mented on a slice-by-slice basis, based on a methodused by [11]. Lee et al convert their original tracinginto minimal hexahedrons and then smooth the datausing a low-pass filter. It is clipped and converted intoa binary volume. An initial ellipsoid model is then fit-ted to this binary volume, using a GDM method firstproposed by [12]. The method involves deforming aninitial mesh to fit a binary volume. It employs a multi-scale approach to deformation - starting with an initiallow resolution mesh, and tessellating it as deforma-tion approaches convergence. This approach is novelin that it explicitly prevents self-intersection of thedeforming mesh by heavily penalising inter-polygonproximity below a certain threshold.

Deformation of the GDM is controlled by iter-ative minimisation of a cost function. Since thereis a large parameter space to search, a naıve bruteforce approach is intractable. The Conjugate Gradi-ent method, discussed by [13], is therefore employed.This method computes successive line minimisationsof the cost function (also called the objective function).It uses the derivative of the objective function to cal-culate optimal direction vectors for british (called con-jugate gradients). After each line minimisation, mesh


vertices are updated to locally optimal positions. Themesh can therefore be seen to iteratively deform tofit its target. When the difference between succes-sive objective function evaluations is smaller than acertain threshold, the algorithm is considered to haveconverged, and deformation stops.

2.2 ASM Construction

ASMs are generated by performing Principal Com-ponent Analysis (PCA) on a set of delimited, reg-istered training shapes, called a Point DistributionModel (PDM). This process produces shape descrip-tors, which are used to constrain the deformation ofshape models during the image segmentation process.The image segmentation process uses an image searchalgorithm, in which the model is expanded and de-formed iteratively to fit local image information (suchas an edge). After segmentation, the results must bevisualised and analysed not only to make use of thesegmented data, but also to evaluate the effectivenessof the segmentation process.

Typically, training data sets are manually seg-mented into 3D binary volumes and an isosurface iscreated from one of the volumes using an algorithmsuch as the Marching Cubes method [14]. This iso-surface, containing arbitrary 3D landmark points, isthen used in a GDM and deformed to fit the other seg-mented volumes [4]. This creates a PDM; consistingof a set of training shape isosurfaces with correspond-ing landmark points, which are then registered to acommon coordinate frame using an algorithm similarto the Procrustes method [15, 16]. After registration,the PDM is passed to the PCA phase for analysis.

It is not always desirable to create a PDM duringthis stage, since PCA can be performed on a variety ofnon-Euclidean shape descriptors, as long as a vector ofvalues is used to describe each training shape. Exam-ples of these shape descriptors include: the MinimumDescription Length (MDL) approach [17, 18], map-ping to Spherical Harmonics (SPHARM) descriptors[4], and mapping to Spherical Wavelet Basis functions[3]. The general (simplified) algorithm followed whenusing these shape descriptors is as follows:

1. Manually segment the input data into a 3D binaryvolume of voxels

2. Map each surface voxel to parameter space, usingan invertible function

3. Perform parameter space registration

4. Perform PCA on shape parameters

New shapes generated by the ASM in parameter spacecan then be mapped back to Euclidean space using theinverse mapping function of that in step 2. Refer toAppendix B for more detail on PCA in ASM construc-tion.

2.3 Image Search Techniques

In order to use an ASM in image segmentation, itis necessary to deform it to fit target data. Targetdata usually consist of greyscale voxels derived from

some imaging process, such as MRI, and are typi-cally noisy. Pre-processing steps such as thresholdingand binary morphology may be necessary to minimisenoise, thereby allowing for better boundary detectionwhilst fitting the ASM to the target data. Edge de-tectors may be used to detect candidate target bound-aries within target data. Anisotropic data can be ad-justed to be isotropic, however care should be takento ensure that the target data dimensions are in pro-portion to those found in the training images used toconstruct the ASM.

Image search techniques are varied, although theygenerally search for the ideal shape, scale and poseparameters which best fit the ASM to the target data.In classical 2D ASM segmentation, an instance of themodel, X, is defined as follows [7]:

X = M(s, θ)[x] +Xc (1)

M(s, θ) is a rotation by θ and a scaling by s. xis a vector of landmark point coordinates represent-ing the current relative position of each point of theshape, and Xc = (xc, yc, xc, yc..., xc, yc)

T is a vectorrepresenting the uniform translation of the landmarkpoints to a centre point in the target data. s, θ andXc together form the scale and pose parameters, andx the shape parameters.

In order to determine the parameters which bestfit the data, it is necessary to first determine the setof adjustments dX = (dX0, dY0, ..., dXn−1, dYn−1)T

which will translate each landmark point closer tothe target boundary. This can be done in variousways, such as region statistics-based search [3], mutualinformation-based coordinate descent [5], or a searchusing grey value intensity profiles [4]. A simple ap-proach is to find the normal to the model boundaryat each landmark point, and determine where it inter-sects the target boundary. The distance to move thelandmark point along the surface normal is then setproportional to the edge strength at the boundary [7].

Once dX has been determined, appropriatechanges to scale and pose parameters need to befound. This is done by finding the best scaling (ds),translation (dXc,dYc) and rotation (dθ) values whichmap X to (X + dX) [7]. The next step is to finddx, the changes to shape parameters necessary to fitthe boundary. Once the appropriate changes to thescale, pose and shape parameters have been found,the model is deformed and the algorithm is repeated.This carries on until it converges to a steady statewhere no significant change is made between succes-sive iterations. Shape deformation is driven by vary-ing the ASM basis weights, within appropriate prede-fined limits. Because of this, we can be certain thatthe model deformation will be constrained to gener-ate shapes that are within the ASD. We describedthis procedure in 2D for the sake of simplicity, but itis easily and intuitively extended to 3D.


3 DATA

The ASM training data used here consists of MRIbrain scans that were acquired as part of a study ofFetal Alcohol Spectrum Disorder [19, 20].

All scans were acquired using a 3T Siemens Al-legra MRI scanner (Siemens Medical Systems, Erlan-gen, Germany). High-resolution anatomical imageswere acquired in the sagittal plane using a 3D in-version recovery gradient echo sequence (160 slices,TR = 2300ms, TE = 3.93ms, TI = 1100ms, slicethickness 1mm, in-plane resolution 1× 1mm2).

In order to use these training data, the structuresof interest to this study, namely the left and rightcaudate nuclei and hippocampi, were manually seg-mented from each greyscale MRI volume by a neu-roanatomist using MultiTracer software [21]. Train-ing data was sampled at 1mm voxel resolution. Thisinitial segmentation yields binary volumes represent-ing the ROIs (caudate nuclei and hippocampi) in thetraining data.

4 LANDMARK POINT GENERATION

In order to capture the shape differences between thevarious training data, it is necessary to assign 3D land-mark points to specific parts of each binary volume,thus creating mesh representations of the data. Eachvertex of each mesh represents a specific point on thetraining data, thus generating point correspondencebetween training data points. Point correspondenceallows for the measurement of variance in shape be-tween training samples, and is therefore necessary totrain the ASM used in the next stage of the project.

Manual assignment of landmark points is tootime-consuming, and error-prone. Therefore, a dis-crete GDM, with a fixed number of points, was fittedto each binary training volume. Our GDM implemen-tation is based on [12]. Certain changes have beenmade to better suit our target data. Our model isdiscussed in detail in Appendix A.

4.1 Self-proximity Term

MacDonald’s self-proximity term measures the small-est distance between pairs of non-adjacent polygons.If a pair of polygons is within a certain distancethreshold, a cost is assigned based on the distance. Inorder to explicitly prevent self-intersection, this costincreases asymptotically towards infinity as the inter-polygon distance approaches zero.

There are three problems with integrating thisspecific implementation into our model. Firstly, thecomputational cost of evaluating each pair of facesis unnecessarily high. Secondly, determining self-intersection using a distance measure is unnecessary,since there are methods to quickly test for the inter-section of two triangles. Thirdly, assigning a pro-hibitively high cost to self-intersection creates prob-lems with the Conjugate Gradient Method used tominimise the cost function, since the inverse partial

Figure 1: Brute force vs Kd-tree. As the number offaces in the mesh increase, the solution using the bruteforce takes exponentially longer, whilst the solution usingthe kd-tree increases linearly in time.

derivatives used as direction vectors for line minimi-sation result in attempts to minimise in suboptimaldirections. These problems are addressed as follows.

Computational Cost

The computational cost of comparing each model facewith every other model face is O(n2). This amountof computation is unnecessary, for the following rea-son: there is a limit on the distance that any singlevertex can be translated in one iteration of deforma-tion. Therefore, there is a limit to the change of dis-tance between any two pairs of faces in one iteration.Since there is a limit to the change in distance betweenfaces, only pairs of faces that are located within thisdistance threshold of each other stand a chance of in-tersecting as a result of any given iteration. Thus, foreach face, it is only necessary to measure distancesto other faces that are within the distance threshold.We take the distance threshold to be equivalent to thesearch distance used in the image search term, sincevertices can not be moved further than this in one it-eration. A kd-tree [22] is used to quickly identify facesthat are within the defined distance threshold of eachother. The kd-tree is built at the beginning of eachiteration of deformation, at a negligible cost to compu-tation (˜0.0035 seconds on 2.66GHz Intel Core2 DuoE6750). During evaluation of the self-proximity term,the nearest non-adjacent neighbours to each face aretested for intersection with that face. Only neighbourswithin the prescribed distance threshold are tested.Figure 1 shows a comparison between the brute forceapproach and the kd-tree approach. It is clear that, asthe number of faces in the mesh increase, the solutionusing the brute force approach runs in quadratic time,whilst the solution using the kd-tree increases linearlyin time.


Figure 2: Triangle intersection vs Distance measureAs the number of faces in the model increase, the in-tersection test preforms significantly better than the dis-tance measure.

Determining Self-intersection

The self-proximity term is used to prevent the deform-ing model from self-intersecting. Therefore, the actualdistance between any given pair of faces is irrelevant,as long as it is above zero. Thus, using a distancemeasure to determine whether pairs of faces intersectis a computationally costly process that could be re-placed by a simple triangle/triangle intersection test.The method described in [23] is used to test for inter-sections. If two faces intersect, the self-proximity termwill evaluate to 1 for the candidate face. If no two facesintersect, the self-proximity term evaluates to 0. Theterm’s weight parameter is used to assign an actualprohibitive cost to self-intersection. In order to vali-date the use of the triangle intersection test, we com-pare our method to the distance measure method interms of evaluation time. We use the triangle-triangledistance measure, as detailed in [24], for comparison.The results can be seen in Figure 2. It is clear thatas the number of faces in the model increase, the in-tersection test performs significantly better than thedistance measure.

Prohibitively High Cost

In order to explicitly prevent self-intersection, Mac-Donald et al set the self-proximity term to increaseasymptotically towards infinity as the distance be-tween two polygons reaches zero. Assigning such ahigh cost to intersecting faces prevents our model fromiterating past a point where a potential intersectionmight take place. This is due to the fact that the costfunction derivative is approximated numerically, andtherefore uses discrete steps in evaluating the deriva-tive of a continuous function. Because of the fixedstep size used in derivative calculation, derivatives offunctions that have a rapidly increasing gradient (suchas MacDonald’s implementation of the self-proximityterm) can not be accurately approximated. This in-accuracy results in extremely large partial derivatives,which throw off the Conjugate Gradient Method used

Figure 3: Point correspondence. Selected representa-tive vertices (403, 489, 1371, 1657, 1794, 1883, 2729)are highlighted on each of 4 fitted, registered meshes.The meshes differ greatly in shape, but the vertices arein clear point correspondence with one another.

for cost function minimisation (discussed next). Inorder to address this inaccuracy problem, we avoidedthe use of a rapidly increasing self-proximity cost func-tion. Instead, as discussed previously, a discrete valueis assigned - based solely on whether a pair of facesintersects or not. It is important to note that thismethod does not explicitly prevent self-intersection,although it allows for parametrically assigning a pro-hibitively high cost to it.

4.2 Segmentation Quality

Segmentation quality is of particular importance tothis application. It is vital to have well-defined land-mark points in order to build an effective ASM. Cri-teria for success in this measure include a low per-vertex segmentation error, and clear point correspon-dence between landmark points. Figure 3 shows thatpoint correspondence is indeed achieved by our GDMimplementation.

Another measure of model effectiveness is the av-erage segmentation error. The distances between eachvertex of 10 fitted meshes and the nearest point inhigh resolution isosurfaces generated from the targetvolumes that these meshes were fitted to, was mea-sured. The mean per-vertex error (averaged over 10fitted meshes) was 0.395mm, with a standard devi-ation of 0.114. Maximum error was 1.592mm. Anerror histogram is shown in Figure 4. Since the meanper-vertex error is less than the 1mm resolution of thetarget data, we conclude that the fit was exceptionallygood.

Overall, segmentation results were excellent. TheGDM model succeeds in capturing the shape of thetraining data volumes, and is sufficient to provide theASM with reliable landmark points.

5 ACTIVE SHAPE MODEL CONSTRUC-TION

The human brain contains both a left and right cau-date nucleus, and a left and right hippocampus. Wetherefore constructed four ASMs, each focusing on a


Figure 4: Segmentation error. A histogram showingthe frequency of per-vertex segmentation error. Themean per-vertex error is less than the 1mm resolutionof the target data.

Figure 5: Scatter Plot of Left Caudate ASM. Selectedvertices are displayed in blue, showing point correspon-dence between aligned training shapes.

specific structure: the left caudate nucleus, right cau-date nucleus, left hippocampus, and right hippocam-pus. Our training data consisted of 30 left and rightcaudate nucleus volumes, and 26 left and right Hip-pocampal volumes. These volumes were manually seg-mented by an expert neuroanatomist, as described inSection 3. 3D landmark points were generated as de-scribed in the previous chapter, using a GDM to fitmanually segmented training data for each structure.Scatter plots showing point correspondence betweenselected vertices of each training shape, for the leftcaudate nucleus, can be seen in Figure 5.

Each of the 4 ASMs is constructed in the samegeneral fashion. Therefore, from here on, we will dis-cuss specifically the construction of the left caudatenucleus ASM. Any differences between the construc-tion of this ASM and others will be discussed as theyarise.

LC% RC% LH% RH%λ1 71.8 65.4 61.8 43.8λ2 8.6 14.0 6.7 14.0λ3 3.9 5.0 5.0 6.4λ4 2.5 2.1 3.5 5.5λ5 1.6 1.3 3.3 4.3λ6 1.3 1.3 2.7 3.7λ7 1.2 1.3 2.4 2.5λ8 1.0 1.1 1.8 2.3λ9 0.8 0.9 1.5 1.9

Table 1: Eigenvalues as a percentage of total varia-tion. Percentages are calculated as Pi = λi

λT× 100

1 forthe ithe eigenvector of the covariance matrix. λT is thetotal variance. LC, RC, LH and RH represent the fourASMs.

5.1 Training Shape Registration and PDMGeneration

The first step in creating the ASM was to align theinput training shapes, each consisting of 3D landmarkpoints, to common coordinate axes. The next step wasto create shape descriptors from the aligned shapes us-ing PCA. The 9 eigenvectors with the highest eigen-values are retained, thus giving t = 9 modes of vari-ation, a feature vector P = (p1p2..p9), and a vectorof 9 basis weights with which to control shape defor-mation, b = (b1b2..b9). Table 1 shows the variationascribed to the first 9 eigenvalues of the 4 ASMs. Bysumming the eigenvalues of the first 9 eigenvectors ofthe left caudate nucleus ASM, we can see that these9 modes account for 92.79% of the total variation inthe model. Including more modes of variation did notsignificantly improve the accuracy of the model. In-deed, using only t = 3 accounts for 84.31% of the totalvariation.

By varying the basis weights in b, new shapes canbe generated by the ASM. Figure 6 shows the effect ofvarying the first basis weight, b1, of our left caudatenucleus ASM by -3 to 3 standard deviations.

5.2 Standard Segmentation Algorithms

Our algorithm is a variation of the ASM segmentationalgorithm described by [7]. This algorithm consists of3 main stages: initialisation, image search and param-eter adjustment. We consider the algorithm to haveconverged when the sum of adjustments between suc-cessive iterations is below a certain threshold.

In order to find the most effective algorithm forthe image search stage, two of the most commonlyused methods were implemented. These algorithmswere applied to our test data, and their weaknessesanalysed. Based on this analysis, we devised our ownheuristic image search algorithm.

Standard Image Search: Edge Detection

In their original paper on ASMs, Cootes et al use Al-gorithm 1 as their image search method.

Some problems exist with using only edge detec-tion to seek boundaries. One obvious problem is that


Figure 6: Variance represented by basis weight b1.Blue vertices represent the mean shape. Green verticesare displaced by varying the first basis weight by -3 to 3standard deviations (A-F).

Algorithm 1

1. For each vertex in the ASM boundary:

(a) Sample the surrounding grey scale intensitiesalong the local surface normal.

(b) Use an edge detector to detect the strongestedge along the sampled 1D grey profile.

(c) The adjustment for that vertex is calculated tobe in the direction of the strongest edge, and ofa magnitude proportional to the edge strength.

Cootes’ Image Search

Figure 7: Boundary Detection Problem. Step A showsvertices from both the top and bottom of a 2D ASMbeing attracted to the same boundary. This causes themodel to be scaled as shown in step B.

the ASM will be attracted to false positives - detectingboundaries that are not part of the target structure. Ifthere are not too many false positives, the restrictionof ASM deformation would prevent the shape from de-forming outside of the ASD. However, too many falsepositives could result in undesirable deformation. An-other problem is illustrated in Figure 7. When a validtarget boundary is completely to one side of a badlyinitialised ASM, both the internal and external ver-tices of the model will be attracted to the same bound-ary. This target boundary is not a false positive, butit causes the model to scale itself inappropriately toattempt to fit both its internal and external verticesto the same boundary - resulting in a very small ASM.

A similar problem tends to occur in narrow regionsof the target structure, such as the tail of the caudatenucleus. It is difficult to initialise the tail accurately,and in most cases the ASM’s tail will be initialisednext to the target tail. This presents a problem inthat model boundaries will be attracted to the sametarget boundary - resulting in poor tail segmentationresults.

Standard Image Search: Grey Profile Mahalanobis Dis-tance

[4] describe an image search technique incorporatingthe use of statistical matching of grey profiles in targetvolumes to grey profiles sampled from training vol-umes. In order to use this technique, an extensionmust be made to the ASM generation stage. Afterlandmark points have been allocated to a training vol-ume, the local greyscale neighbourhood must be sam-pled at each of these points. In our case, fitted GDMvertices represent landmark points. A 1D greyscaleprofile is sampled in both directions along the localsurface normal at each one of these vertices.

The grey profile Mahalanobis distance imagesearch method relies on the relative homogeneity ofthe local greyscale neighbourhood surrounding each


corresponding vertex in the training volumes. Whenthere is a high variance between grey profiles for corre-sponding vertices, the model will not capture the char-acteristics of the greyscale neighbourhood of those ver-tices very well. In this case, the Mahalanobis distancemeasure is not very effective in matching a vertex thatis positioned in a target volume to its correspondingmodel vertex, as stored in the ASM.

6 HEURISTIC IMAGE SEARCH METHOD

In order to address the limitations of the edge de-tection and grey profile Mahalanobis distance imagesearch methods, and to take advantage of various gen-eral characteristics of our two target structures, wedevised a heuristic method for the image search stageof caudate nucleus and hippocampus segmentation.

6.1 Assumptions

The heuristic image search method is based on thefollowing assumptions about the characteristics of thecaudate nucleus and hippocampus. These assump-tions are based on the nature of the MRI data dis-cussed in Section 3.

• The main body of the caudate nucleus is rela-tively homogeneous in greyscale intensity. Basedon empirical observation of test data, this inten-sity lies between the mean grey matter and whitematter intensities for the entire brain. Specifi-cally, the caudate nucleus is around 1.25 timesthe intensity of the mean grey matter intensity.

• The tail of the caudate nucleus, making uparound 20% of the total structure, is darker thanthe body. Specifically, our test data indicates thatit is around 0.7 times the intensity of the body.

• The tissue surrounding the caudate nucleus ismostly grey matter of a lighter intensity than thecaudate itself.

• The entire hippocampus is relatively homoge-neous in greyscale intensity. Empirical observa-tion again shows that this intensity lies betweenthe mean grey matter and white matter intensi-ties for the entire brain. However, the hippocam-pus is slightly darker, at around 1.17 times theintensity of the mean grey matter intensity.

• The tissue surrounding the hippocampus ismostly grey matter of a lighter intensity, but somesurrounding areas are darker than the hippocam-pus. Thus, the only assumption that can be madeabout colour of the surrounding tissue is that it isgenerally of a different intensity to the hippocam-pus.

• Because both the caudate nucleus and hippocam-pus are relatively homogeneous in intensity, theycontain few internal edges. However, since thesurrounding tissue is of a different intensity, therewill be a pronounced edge along large portions ofthe boundaries of both structures.

Figure 8: Histogram of Typical Brain Tissue Intensi-ties. GM, CC, and HC refer to mean grey matter colour,caudate colour and hippocampus colour, respectively.

Figure 9: MRI Scan of caudate nucleus. The sagittalMRI scan shows a manually-traced left caudate nucleusin red.

• The tissue surrounding both structures is gener-ally not homogeneous in intensity, and containsmany edges.

Figure 8 shows a histogram of brain tissue intensities,with the mean colour of the caudate nucleus and hip-pocampus marked. Figures 9 and 10 show MRI scansof the tissue surrounding the caudate nucleus and hip-pocampus.

6.2 Search Function

Our heuristic image search function was designed totake advantage of the characteristics mentioned in theprevious section. The goal of the function, as with theothers, is to find an appropriate boundary in a givengrey profile. The function works similarly to the naıveedge detection method, in that it evaluates the prob-ability of being a part of the boundary, for each voxelin a given grey profile. The difference is that it notonly takes into account edge strength of a given voxel,but also the characteristics of the (8-connected) sur-rounding voxels. Characteristics taken into accountinclude the difference in colour between surroundingvoxels and the target structure, as well as the numberand strength of edges present in the surrounding vox-els. The Heuristic Edge evaluation function, HE(), is


Figure 10: MRI Scan of hippocampus. The sagittalMRI scan shows a manually-traced left hippocampus inred.

made up of 3 weighted terms, formulated as follows.Figure 11 provides an illustration of how the heuristicEdge evaluation function works.

HE(pv, i, s) = a.ES(pv, i, )−b.SE(pv, i, s)−c.DC(pv, i, s)(2)

ES(), SE(), and DC() refer to Edge Score, SampleEdges, and Difference in Colour, respectively. Theseterms will be discussed in detail in the following sec-tion.

Once the voxel with the highest heuristic Edgescore in pv has been detected, the per-vertex adjust-ment is computed in a similar fashion to the edge de-tection image search method. Vertex v is moved alongsurface normal in the direction of the voxel with thehighest Heuristic Edge score, for a distance propor-tional to the Heuristic Edge score of that voxel.

Edge Score

The Edge Score term, ES(pv, i), uses a simple 1D con-volution kernel ( [0.25 0.5 0.25] ) to find the edgestrength of a given voxel, i, as it appears in a greyprofile, pv. This term operates in a similar fashionto the edge detection image search method mentionedearlier. The term is weighted by the constant a. Thisweighting allows for the adjustment of the contribu-tion that each term has on the final score. This con-cept is discussed in further detail in Section 6.4.

Sample Edges

The Sample Edges term, SE(pv, i, s), sums the totalstrength of all edges that are present in a sample thatappears immediately to the right of voxel i in greyprofile pv. The sample length is specified as s. Thisterm effectively measures inhomogeneity in the sampleto the right of the voxel in question. This measureis based on the assumption that the caudate nucleusand hippocampus are relatively homogeneous, and istherefore negatively weighted by the constant −b.

The sample is taken to the right hand side of voxeli because of the direction in which grey profiles are

sampled along local surface normals. The surface nor-mal orientation is such that the left hand side of greyprofiles always point towards the outside of the tar-get structure, and the right hand side always pointstowards the inside.

This term, when negatively weighted and com-bined with the previous Edge Score term, detectsedges that are bordered to the right by homogeneousregions - as one would expect to find around the bor-ders of the target structures.

Difference in Colour

The Difference in Colour term, DC(pv, i, s), measuresthe difference between the mean colour of a sample -also taken to the right of the ith voxel - and the meancolour of the target structure being segmented. Themean colour of the target structure is determined viaa lengthy process that is explained in detail in Section6.3.

Based on the assumption that the caudate nucleusis generally darker than the surrounding areas, theDifference in Colour term only penalises the HeuristicEdge score if the sample mean is lighter than the meancaudate colour. The caudate tail is also known to bedarker than the rest of the structure, so the meancaudate colour is multiplied by 0.7 when the vertex,v, corresponding to the grey profile pv, is part of thetail.

The area surrounding the hippocampus can beboth lighter and darker than the Hippocampal meancolour. So, instead of only penalising the HeuristicEdge score for the sample mean being lighter, in thecase of the hippocampus, the Difference in Colourterm calculates the absolute value of the differencebetween the sample mean and the hippocampus mean- thus incurring a penalty for the sample mean beingeither darker or lighter.

The Difference in Colour term is also negativelyweighted, using the constant −c.

6.3 Target Colour Determination

The heuristic edge detection method relies largely onthe knowledge of the mean tissue intensity of the tar-get structure. In order to determine this intensity,we employ a lengthy algorithm, based mainly on amethod devised by Worth et al, and to a lesser de-gree on the method employed by Xia et al [25, 26].These methods are discussed next. Following this dis-cussion, we describe each step of our method in detail.We then evaluate the effectiveness of our method.

Target Volume Preprocessing

In order to remove scanning artifacts and other spuri-ous voxels from the target volume, a flood fill is firstperformed on the background area, outside of the skullboundary. The flood fill isolates the volume containedin the skull from the rest of the voxels, allowing for amore accurate global histogram.

The next step is to crop the volume to begin on theskull boundary. This is achieved by casting rays from


Figure 11: Heuristic Edge evaluation. pv is a typicalgrey profile sample, taken from a caudate nucleus targetvolume. CC indicates the mean caudate colour, as de-tected by our algorithm. Using the naıve edge detectionmethod, voxel i = 8 would have been incorrectly chosen(it is on the wrong side of the target structure), since ithas the strongest edge value. However, the heuristic edgedetection method takes into account the homogeneity ofthe voxels i = 3 to i = 7 (to the right of the sample),as well as the fact that the mean colour of these voxelsis close to that of the caudate nucleus in this target vol-ume - resulting in i = 2 being correctly chosen as theboundary of the target structure.

the original target volume borders, vertically and hor-izontally, towards the centre of the volume. The pointin each axis at which the rays first intersect with theskull is taken as the cropping plane for that axis. Thecropped volume allows faster processing of subsequentsteps, as the volume to be processed becomes smaller.

The final preprocessing step is to remove strongedges using a Sobel filter. A 2D Sobel filter is con-volved with each slice of the target volume to identifyedges. Edges with scores above a certain thresholdare then removed, leaving behind areas of relativelyhomogeneous intensity, and containing only weak gra-dients.

These preprocessing steps are illustrated in Figure12.

Global Peak Detection

The first step in peak detection is to calculate theglobal histogram of voxel intensities for the target vol-ume - with the background voxels thresholded out.This histogram is initially quite jagged, and must besmoothed to be useful. Smoothing is achieved by con-volving the histogram with a discrete Gaussian con-volution kernel. This smoothing step is repeated anumber of times, depending on the width of the his-togram. The width of the histogram is proportional tothe range of intensities present in the target volume. Ifthe volume contains only a small range of intensities,then smoothing the histogram too many times willcause too much loss of information, rendering it use-less. Thus, the range of intensities is first determined,and the histogram is then smoothed accordingly.

The next step is identify peaks in the smoothedhistogram. This is done by first convolving the his-togram with a discrete Gaussian first derivative ker-nel. This convolution produces an approximation ofthe first derivative of the histogram. The first deriva-

tive approximation is then smoothed using the orig-inal Gaussian kernel. Peaks can now be identifiedby negative-going zero crossings of the first derivativefunction.

Because of the inherently jagged nature of thehistogram and its approximate first derivative, manypeaks will be identified by looking for zero crossings.In order to eliminate false positives, the area of each ofthese peaks must be determined. Peak area is deter-mined in a similar way to Worth’s method of addingthe absolute values of the heights of the first deriva-tive of the histogram on each side of the negative-goingzero crossing [25]. Instead of adding the heights, wesum the area under the curve at each discrete point inthe histogram, between the two zero crossings. Usingthis method, we can express the area of a peak as aproportion of the total histogram area - simply calcu-lated by summing the area under the curve at eachpoint in the full histogram. Peaks with areas that areproportionally much smaller than the histogram itself,are discarded as false positives. Our empirical testingshowed that valid peaks are usually greater than 1

30 thof the total histogram area.

At this point, there should be 2-3 peaks of interestleft, since the dark cerebrospinal fluid can sometimesbe thresholded out during the process. The grey mat-ter peak is always the second last peak of the remain-ing group. This grey matter value should be savedfor later comparison to the local grey matter colour,since this is the peak of interest in our segmentationalgorithm. Figure 12 illustrates the peak detectionprocess.

Local Peak Detection

Now that the global grey matter intensity has been de-termined, a similar process must be undertaken to findthe grey matter intensity local to the area surroundingthe target structure. Since both the caudate nucleusand hippocampus consist entirely of grey matter, de-termining this intensity will provide the necessary in-formation for the Difference in Colour term mentionedpreviously.

The first step in determining local intensity isto isolate the area surrounding the target structure.Worth’s method involves a lengthy heuristic proce-dure involving locating the Corpus Callosum, and us-ing this as a landmark to find the caudate nucleus.This procedure is unnecessary in our algorithm, aswe already have information regarding the mean loca-tion of target structures, as recorded in our trainingshapes. Indeed, our method already relies on this lo-cation to initialise the ASM prior to segmentation.Using the mean location and orientation of the train-ing shapes we can reliably determine the boundingboxes surrounding both the caudate nuclei and hip-pocampi in a given target volume. This is done basedon the previously mentioned assumption that targetdata will be captured in a similar orientation to thetraining data used to create the ASM.

After the target structure has been isolated, thealgorithm proceeds to calculate peaks using the same


Figure 12: Caudate Colour Determination. CC in-dicates the final caudate colour. CSF, GM and WMindicate cerebrospinal fluid, grey matter and white mat-ter, respectively. The processing steps illustrated hereare discussed in detail in section 6.3.

CN HN 62 52Success W/O GM Guess 47% 98%Success W GM Guess 82% 98%Mean Intensity 88.95 80.96Min Intensity 40 36Max Intensity 130 109False Positives 4 1False Pos Rate 6% 2%Detected False Pos 2 1Undetected False Pos 2 0Undetected False Pos Rate 3% 0%

Table 2: Results of Evaluation of Target Colour De-termination. CN refers to caudate nucleus and H refersto hippocampus.

method as was used to determine global peaks. Again,the second last peak that remains after filtering istaken to be the local grey matter intensity. This peakis checked against the global grey matter peak to en-sure that it is of a slightly higher intensity.

In some cases, the target volume is of such poorcontrast that determination of the local grey matterpeak fails. In these situations, local grey matter in-tensity is estimated based on global grey matter inten-sity. Based on our previous assumptions, grey matterbelonging to the caudate nucleus and hippocampuswould be estimated at 1.25 and 1.17 times the globalgrey matter intensity, respectively.

Evaluation

Caudate nucleus and hippocampus colour determi-nation algorithms were tested in the following fash-ion. The algorithm was run on each unsegmented tar-get volume, and the global grey matter and targetstructure colours were recorded. These colours werethen compared to the mean target structure colour inthe corresponding segmented volume. The results aresummarised in Table 2.

GM Guessing refers to the estimation of targetstructure intensity, based on the detected global greymatter intensity. When the algorithm fails to deter-mine the local grey matter intensity (often due to poorcontrast data), or the algorithm determines an inten-sity that is outside of an acceptable range, the in-tensity is estimated according to the previously men-tioned assumptions about the ratio between targetstructure and global grey matter intensity.

A colour estimate is deemed to be a false positivewhen the algorithm does not fail (i.e. falsely indi-cates success), but provides an estimate that is out-side of the acceptable range of intensities for a par-ticular target structure - based on the mean targetstructure intensity, as determined from manually seg-mented test data. False positives are detectable inmost cases, by measuring whether the ratio betweenglobal grey matter intensity and local grey matter in-tensity is within acceptable limits. In very few cases(2 out of 62, for the caudate nucleus, and 0 out of 52,for the hippocampus), these false positives went unde-tected, and would therefore cause poor segmentationresults.

The results show that the Target Colour Determi-nation algorithm works satisfactorily for caudate nu-cleus target structures, with an 82% success rate. Incases where the algorithm fails, the ASM segmenta-tion method would have to be altered to not rely onthe knowledge of the target structure colour. This canbe achieved by assigning a weighting of 0 to the Dif-ference in Colour term. Fortunately these failures aremostly detectable, and can be compensated for.

The algorithm works exceptionally well in deter-mining the colour of target hippocampi. With a 98%success rate, and only 1 false positive (which was de-tectable), the results show that the algorithm is veryreliable when applied to these structures.

6.4 Parameter finding using a GeneticAlgorithm

The heuristic edge detection algorithm relies on theuse of various parameters to control segmentation.These parameters are detailed below.

• Profile length (l). This parameter determines thelength of sampled grey profiles, pv. Typical valuesrange between 3 and 5.

• Sample length (s). This determines the samplelength, as detailed in Section 6.2. Typical valuesrange between 3 and 7.

• Term weighting (a, b, c). These parameters con-trol relative term weighting, as detailed in Section6.2. Typical values range between 0.5 and 6.0.

• Maximum edge value (m). This parametercontrols the maximum expected boundary edgevalue. This varies according to target structure,and is used to control the proportional adjust-ment of vertices in the search function. Thecaudate nucleus tends to have stronger bound-aries than the hippocampus, and will thus require


higher values of m. Typical values range between50 and 500.

In order to maximise segmentation accuracy, it is nec-essary to find the best set of parameters for each targetstructure. Parameters that work well for caudate nu-cleus segmentation do not necessarily work as well forhippocampus segmentation. In order to test a set ofparameters, a segmentation must be run using them.Since the target volume data vary in brightness, con-trast, and signal-to-noise ratio, it is not enough tomerely test a set of parameters on one target volume.A set of parameters must therefore be used in multiplesegmentation runs to test its effectiveness. This pro-cess is considerably time consuming, and it is not fea-sible to experiment with all possible parameter com-binations, as the search space is fairly large.

Implementation

Since exhaustive exploration of the parameter searchspace was infeasible, we use a Genetic Algorithm (GA)to find the best set of parameters for each target struc-ture. The GA was implemented in MATLAB, anduses Roulette Wheel selection. Individual genes inour GA chromosomes simply represent segmentationparameters. The chromosomes therefore consist of 6genes each. Genes consist of integer indexes rang-ing from 1 to the maximum number of discrete in-tervals in the range of values for their respective pa-rameters. For example, the chromosome [1 2 3 6 5 4]represents the parameter values [l = 3 s = 4 a = 1.5b = 3.0 c = 2.5 m = 200]. The fitness function takesa chromosome, evaluates it, and returns a scalar fit-ness value. Segmentation overlap was used as our fit-ness metric. A separate fitness function was used toevaluate the two target structure types, since caudatenucleus segmentation is slightly different to hippocam-pus evaluation.

Results

Optimum results were achieved with a crossover rateof 70% and a mutation rate of 20%. A population sizeof n = 20 was used, and the algorithm ran for 100generations. The fittest 10 individuals were recordedfor each target structure. Each of the 10 fittest chro-mosomes were then evaluated in a full test run of 30and 26 target volumes, in the case of the caudate nu-cleus and hippocampus, respectively. For each targetstructure, the set of parameters that generated thebest overall results was used in the final evaluation.

7 EVALUATION

7.1 Metrics

In this section, we discuss the background and ourimplementation of the quantitative metrics used forevaluation. These metrics include commonly-used seg-mentation measures such as overlap and segmentationerror. Metrics such as volume difference and surfacearea difference offer an easy-to-calculate, but naıve

measure of segmentation success. They are not widelyused, and are therefore excluded from this evaluation.

Overlap

The overlap metric, ε, also knowns as Dice’s coeffi-cient, is used to measure the fraction of possible voxelsshared by two binary segmentations. It is formulatedas follows [26].

ε =2V12

(V1 + V2)(3)

V1 and V2 are the scalar volume measures of thetwo binary segmentations in question. V12 is the scalarvolume of the intersection of V1and V2. Thus, ε rep-resents the ratio of actual overlap to possible over-lap. For our purposes, the overlap metric is used tomeasure the success of our segmentation algorithm inidentifying a target structure in a target data volume.Thus, V1will represent the scalar volume measure ofthe manually segmented ground truth data volume,G. V2 thus indicates the scalar volume measure of theautomatically segmented data volume, A. V12 is thescalar volume measure of the intersection of A and G,A∩G, and can also be represented as VA∩G. This met-ric is used widely in segmentation literature, and cantherefore be used as an objective comparison of seg-mentation results between similar studies [4, 27, 28].

False positive and false negative ratios can be cal-culated in a similar fashion. False positives, or over-estimated voxels, are those voxels that have been er-roneously identified as being part of the target struc-ture. These voxels can be described as the relativecomplement of A, given G, or A −G. The volume offalse positives, VA−G, and the ratio of false positives

to volume of overlap, VA−G

VA∩G, are also commonly-used

metrics. False negatives, or missed voxels, can be de-scribed as the relative complement of G, given A, orG−A. Similarly, the volume of false negatives, VG−A,and the ratio of false negatives to volume of overlap,VG−A

VA∩G, are frequently used. False positives and nega-

tives can also be expressed somewhat more intuitivelyas a proportion of the target volume, formulated asVA−G

VGand VG−A

VG, respectively.

Segmentation Error

Average segmentation error, is a measure of the meandistance between each vertex of a fitted mesh, MA,and the closest point in a mesh generated from a man-ually segmented ground truth data volume, MG. Themean-squared distance, is used more often than meandistance, as squaring the difference generates more no-ticeable results.

One advantage of using these metrics is that, aswell as measuring the total difference between all ver-tices, it is also possible to measure the difference be-tween subsets of vertices - thereby allowing closerscrutiny of surface overlap in localised regions, suchas the tail of the caudate nucleus, for example.

Hausdorff distance can also be used to measurehow successfully the ASM has been fitted to target


Algorithm 2 Evaluation of ASM Segmentation Abil-ity

1. For i = 1..Ns

(a) Construct ASM from Ns training shapes, leav-ing shape i out

(b) For each segmentation method, M (Edge De-tection, grey profile Mahalanobis distance,heuristic method, heuristic method with mul-tisampling):

i. Initialise ASM

ii. Segment unsegmented target data volume i,using segmentation method M

iii. Evaluate segmentation metrics for target vol-ume i, method M

data [3]. This metric measures the maximum errorbetween the boundaries of the ground truth mesh, MG

and the fitted mesh, MA.It is also useful to measure the standard deviation

of distances between vertices of MA and the target,MG. This reflects the regularity of the segmentationresults. Results with a high standard deviation wouldindicate an erratic segmentation, which is undesirable.

7.2 Results

ASMs based on the four target structures were con-structed for this evaluation. These included ASMsfor left and right caudate nuclei, as well as ASMsfor left and right hippocampi. For each target struc-ture, ASMs were constructed using varying numbersof training shapes. This was done to evaluate the ef-fect that adding more training data to the ASMs hadon the segmentation results. The caudate ASMs wereconstructed with 10, 20 and 30 training shapes. Thehippocampus ASMs were constructed with 10, 20 and26 training shapes. Thus, 12 ASMs were evaluatedin total (4 structures × 3 variations in the number oftraining shapes used). Using specific subsets of train-ing shapes could potentially introduce bias into theexperiment. Subsets were chose at random in order tominimize this. This bias effect could also be minimisedby creating several small subsets of training shapes,and averaging the results obtained from them.

For each of these ASMs, Algorithm 2 was usedto evaluate the ability of the ASM to segment un-seen test data. As mentioned, data were consideredunseen since each ASM was built using leave-one-outconstruction.

Statistical Significance Testing

In order to facilitate certain further statistical anal-ysis, normality testing was performed on the results.This was done to each result set by first finding themean and standard deviation of a Gaussian distribu-tion that most closely matched the histogram of re-sults, and then measuring the correlation coefficientto determine how strongly the results are correlatedto the standard normal distribution.

Normality testing was performed on the overlap

Left RightED 0.7766 0.9420

GPM 0.7410 0.6683H 0.8971 0.9390

(a) Caudate Nuclei

Left RightED 0.5439 0.8930

GPM 0.5811 0.7692H 0.8838 0.9458

(b) Hippocampi

Table 3: Normality Testing. ED, GPM, and H referto edge detection, grey profile Mahalanobis, and heuris-tic method. Correlation coefficient values indicate astrong correlation (0.5 < r < 1.0) between all resultshistograms and the standard normal distribution. Wecan therefore conclude that our results are normally dis-tributed.

metric results of segmentation using each image searchmethod, for each target structure. Correlation resultsare summarised in Table 3.

Chauvenet’s Criterion was used to identify andremove outliers in our data [29]. According to Chau-venet, an experimental result can be rejected as anoutlier if the probability of obtaining the result (basedon the number of standard deviations from the mean)is less than 1

2n , where n is the number of results.

Out of 232 results, 37 outliers where identified -giving an outlier percentage of 37

232 ×1001 = 15.9%. 20

of the 37 outliers occurred due to segmentation failureof target volume 6 - this accounts for 20

232×1001 = 8.6%

of the total number of of results.

We use a paired difference test for statistical signif-icance, aiming to disprove the null hypothesis that themeans of two sets of segmentation results are not sta-tistically different. Again, we used the overlap metricas our basis for comparison of results, since this met-ric is the most representative of the effectiveness of theimage search methods in question. Since outliers havebeen eliminated, not all sets of results could be pairedfor comparison. Thus, we performed the dependentt-test only on samples that had corresponding pairsin all results sets. The test was performed using a sig-nificance level of 5%. A summary of results for ASMswith the largest number of training shapes is presentedin Table 4.

Segmentation Results

Table 5 lists a summary of segmentation results forASMs with the largest number of training shapes (i.e.Ns = 30 for caudate nuclei, and Ns = 26 for hip-pocampi).

Overlap Results Beginning with the overlap met-ric, it is clear, from Table 4 and Table 5, that forall four target structures, the heuristic method sig-nificantly outperformed both the ED and the GPMmethods.


µ σ n t-stat p(T¡=t)ED-H 0.08 0.06 29 7.18 ¡0.001

GPM-H 0.15 0.09 29 9.12 ¡0.001

(a) Left Caudate Nuclei

µ σ n t-stat p(T¡=t)ED-H 0.07 0.07 25 5.13 ¡0.001

GPM-H 0.12 0.12 25 5.13 ¡0.001

(b) Right Caudate Nuclei

µ σ n t-stat p(T¡=t)ED-H 0.19 0.11 26 9.23 ¡0.001

GPM-H 0.26 0.13 26 10.45 ¡0.001

(c) Left Hippocampi

µ σ n t-stat p(T¡=t)ED-H 0.13 0.06 23 10.33 ¡0.001

GPM-H 0.15 0.08 23 8.81 ¡0.001

(d) Right Hippocampi

Table 4: Results of Paired Difference Test. ED, GPM,and H refer to the image search methods: edge detec-tion, grey profile Mahalanobis, and heuristic, e.g. ED-H refers to the difference between edge detection andheuristic image search methods. µ and σ refer to themean difference and standard deviation of differences. nis the sample size, while t-stat refers to the t-test statis-tic value. p(T¡=t) refers to the two-tail p value. Alldifferences are statistically significant.

The GPM method performs worst out of thethree image searches. When compared to initialisa-tion statistics, we noted that the GPM method ac-tually generates a worse segmentation over time. Allother methods tend to improve their mean overlap assegmentation proceeds. This indicates a fundamentalflaw with the use of the GPM method in this context.This is possibly due to boundary inhomogeneities inour training data.

False Positive and False Negative Ratios Aspreviously discussed, False Positive Ratio and FalseNegative Ratio refer to the ratio of false positives andfalse negatives to the volume of overlap, respectively.Our results indicate that although the ED methodgenerated acceptable overlap results, it had a tendencyto over-segment the data. This is apparent from thefact that mean false positive ratios were very high,whilst mean false negative ratios remained low forall target structures. In fact, in all target structuresexcept the left hippocampus, the ED method gener-ated the lowest mean false negative ratios. The samemethod also generated the highest mean false positiveratios in all four target structures.

The GPM method generated poor mean false pos-itive and false negative ratios for all target structures.This indicates that the area segmented by this methodwas more or less uniformly offset from the target struc-ture. This lack of precision once again indicates a flawin the use of the GPM method.

The heuristic method significantly outperformed

O FP FN MSD HD SDDED 0.74 0.49 0.13 0.97 3.76 0.63

GPM 0.67 0.26 0.37 1.79 4.09 0.83H 0.82 0.16 0.2 0.75 2.7 0.49

(a) Left Caudate Nuclei


GPM 0.69 0.28 0.32 1.28 3.72 0.73H 0.81 0.14 0.21 0.76 2.73 0.48

(b) Right Caudate Nuclei


GPM 0.5 0.46 0.51 2.82 4.76 1.04H 0.76 0.3 0.21 0.73 3.1 0.56

(c) Left Hippocampi


GPM 0.64 0.43 0.33 1.59 3.97 0.84H 0.79 0.25 0.19 0.72 2.82 0.52

(d) Right Hippocampi

Table 5: Summary of Mean Segmentation Results.ED, GPM, H, and HM refer to the image search meth-ods: edge detection, grey profile Mahalanobis, andheuristic. Overlap (O), False Positive (FP) Ratio, FalseNegative (FN) Ratio, Mean Squared Distances (MSD),Hausdorff Distance (HD) and Standard Deviation of Dis-tances (SDD) segmentation metric means are displayedin the corresponding columns.


both the ED and GPM methods in terms of meanfalse positive ratios for all structures, indicating a uni-form tendency to avoid over- and under-segmentation.However, the ED method generated slightly lower falsenegative ratios for three of the structures.

Segmentation Error Metrics As expected, meansquared distances are approximately inversely propor-tional to overlap. Results for the GPM method showthe highest mean squared distances in most cases, in-dicating that the model boundary was situated rela-tively far from the target boundary. Again, the EDmethod gives moderate results, whilst the heuristicmethod gives the best results - resulting in an averagemean squared distance of under 1mm2 in all cases.

The Hausdorff distance results are much the same,indicating that the maximum segmentation error was,in most cases, lowest for the heuristic method withoutmultisampling. The ED and GPM methods fared theworst in all cases.

Standard Deviation of Distances indicates a largefluctuation in model-vertex-to-target-boundary dis-tances for the ED and GPM methods. This indicateserratic segmentation results, with some vertices situ-ated close to target boundaries, and some situated faraway. Again, the heuristic method showed reliableconsistency in segmentation, with the lowest meanstandard deviation of distances results in all cases.

FreeSurfer Validity Study

A recent study was done into the validity of using theFreeSurfer software package for measurement of brainvolumes in children with fetal alcohol syndrome [30].This study was done using the same MRI test datathat we used in our experiments.

The study reported inter-observer correlations formanual tracings of target structures ranging from r =0.94 to r = 0.99, with a median of r = 0.98. Thisinter-observer correlation indicates the high reliabilityof the manually delimited target volumes used in ourstudy.

The FreeSurfer package automatically recon-structs an entire cortical surface, and correspond-ing sub-cortical volumes from an MRI brain volume.This process is time-consuming, as target structurescan not be segmented individually. Results from theFreeSurfer study indicate that manual tracings arehighly correlated with automatically segmented vol-umes. Correlation coefficient values ranged from aworst case of rs = 0.74 for the right hippocampus toa best case of rs = 0.89 for the total caudate, with amedian of 0.83.

Since overlap is directly proportional to correla-tion, we can compare the FreeSurfer correlations withour overlap results. Table 6 shows median and maxi-mum results for our heuristic method that are similarto the FreeSurfer correlations, although minimum val-ues were significantly lower for most structures. Theseresults indicate that our heuristic method is a viablealternative to FreeSurfer. Also, when segmenting in-dividual target structures, our method takes around 5

Min Max MedianLC 0.65 0.88 0.83RC 0.65 0.88 0.82LH 0.66 0.82 0.78RH 0.72 0.85 0.79

Table 6: Min, Max and Median Overlap for theHeuristic Method. LC, RC, LH and RH represent thefour target structures: left caudate nucleus, right cau-date nucleus, left hippocampus, right hippocampus.

to 10 minutes, whereas FreeSurfer typically takes 24to 36 hours on equivalent hardware (due to the whole-brain segmentation). Thus, when used for rapid seg-mentation of individual target structures, our methodmay be preferable to the use of FreeSurfer.

GPM Image Search Failure

The GPM image search method failed to detectboundaries effectively. This failure was probably dueto the relative heterogeneity of organ boundaries inour training data. The GPM method relies on astrong correlation between greyscale intensities of cor-responding boundary points in training data. If thiscorrelation is not strong enough (due to noise or con-trast factors, for example) the GPM method fails tocreate a good model of the boundary surrounding atarget shape. Without a decent model of the bound-ary, the GPM image search fails to attract vertices toboundaries, and produces a bad segmentation.

8 CONCLUSION AND FUTURE WORK

We attempt to find the best algorithm for the auto-matic segmentation of the caudate nucleus and hip-pocampus from our MRI test data, within the contextof a study into the neural correlates of FASD.

We use a modified GDM to create point correspon-dence in our data. We then implement an ASM thatincorporates various popular image search methods tofind target structure boundaries. In addition to thesemethods, we develop our own heuristic image searchmethod - tailored to work particularly well with theabovementioned target structures.

We use a wide range of segmentation metrics toevaluate the effectiveness of the various image searchmethods when applied to the data from the study intoFASD. The results follow a standard normal distri-bution. Statistical analysis shows that our heuris-tic method is consistently better at segmenting thetarget data than other tested image search methods.The edge detection method produces average results,although it displays a tendency to over-segment thetarget data. The grey profile Mahalanobis methodfares the worst of the three. This failure is ascribed toheterogeneity in training data. The heuristic methodshows the lowest standard deviation in results, againdisplaying reliable consistency in segmentation. Theother methods produce somewhat more erratic results.Future work may include experimenting with image


search methods not attempted in this research, suchas the method suggested by [31].

Results from the heuristic method are favourablewhen compared to results of automatic segmentationof the same data using the FreeSurfer software pack-age. Although the FreeSurfer results are slightly bet-ter, the ASM approach allows for independent tar-get structure segmentation, and is therefore much lesscomputationally demanding. Thus, individual tar-get segmentation using the heuristic method is muchfaster, and may be preferable to the use of FreeSurfer.Both FreeSurfer and the heuristic method provide alower level of accuracy when compared to manual seg-mentation. Since the FreeSurfer method has been usedin numerous studies, such as the one mentioned in theprevious section, and our results are comparable tothose produced by FreeSurfer, we conclude that theheuristic method provides sufficient accuracy to beused in similar FASD-related studies.

Recent state-of-the-art techniques that make useof multi-atlas segmentation, have demonstrated com-parable, and sometimes better overlap results to ourASM method [32]. Multi-atlas segmentation is gener-ally more computationally demanding, but with ad-vances in the availability of computational power, thisbecomes less of an issue. Future work in FASD seg-mentation could include research into the efficacy ofthese methods on our subject data.

In order to improve segmentation results, whilstmaintaining most of the speed benefit, it may bepossible to use the heuristic ASM method as theinitial stage in a semi-automatic segmentation pro-cess - the fast initial automatic segmentation wouldthen be manually corrected by neuroanatomists. Thiswould result in dramatically improved accuracy, at amarginal cost of a few minutes of segmentation time.

All-in-all, our method will allow for more rapidsegmentation of the caudate nucleus and hippocampusstructures, thereby accelerating research into FASD.This important research will allow us to better under-stand the neural correlates of this debilitating disor-der.

ACKNOWLEDGMENTS

I would like to thank Patrick Marais and ErnestaMeintjes for their excellent work in supervising andco-supervising this project.

I am also thankful to the following people for theircontributions to this endeavour: Chris Warton, forperforming manual tracing of the MRI test data; San-dra Jacobson, Joseph Jacobson and Chris Moltenofor proposing development of an automated algorithmfor segmentation of hippocampi and caudate nucleito streamline morphometric analyses of children withFASD, as well as planning, implementing and coordi-nating the MRI test data collection and manual trac-ing.

This work was funded by NIH grants from NIAAAR01AA016781 and Fogarty R03TW007030, a FocusArea grant (FA2005040800024) from the National Re-

search Foundation of South Africa, the South AfricanResearch Chairs Initiative of the Department of Sci-ence and Technology and National Research Founda-tion of South Africa, and seed money grants fromWayne State University and the Joseph Young, Sr.,Fund from the State of Michigan.

REFERENCES

[1] P. Freeborough, N. Fox and R. Kitney. “Interac-tive algorithms for the segmentation and quanti-tation of 3-D MRI brain scans”. Computer Meth-ods and Programs in Biomedicine, vol. 53, no. 1,pp. 15–25, 1997.

[2] Y. Xia, K. Bettinger, L. Shen and A. Reiss. “Au-tomatic Segmentation of the Caudate NucleusFrom Human Brain MR Images”. Medical Imag-ing, IEEE Transactions on, vol. 26, no. 4, pp.509–517, 2007.

[3] D. Nain, S. Haker, A. Bobick and A. Tannen-baum. “Multiscale 3-d shape representation andsegmentation using spherical wavelets”. MedicalImaging, IEEE Transactions on, vol. 26, no. 4, p.598, 2007.

[4] A. Kelemen, G. Szekely and G. Gerig. “Elasticmodel-based segmentation of 3-D neuroradiolog-ical data sets”. Medical Imaging, IEEE Transac-tions on, vol. 18, no. 10, pp. 828–839, 1999.

[5] A. Tsai, W. Wells, C. Tempany, E. Grimsonand A. Willsky. “Mutual information in coupledmulti-shape model for medical image segmenta-tion”. Medical Image Analysis, vol. 8, no. 4, pp.429–445, 2004.

[6] D. Shen, E. Herskovits and C. Davatzikos. “Anadaptive-focus statistical shape model for seg-mentation and shape modeling of 3-D brain struc-tures”. Medical Imaging, IEEE Transactions on,vol. 20, no. 4, pp. 257–270, 2001.

[7] T. Cootes, C. Taylor, D. Cooper, J. Graham et al.“Active shape models-their training and applica-tion”. Computer Vision and Image Understand-ing, vol. 61, no. 1, pp. 38–59, 1995.

[8] J. Miller, D. Breen, W. Lorensen, R. O’Baraand M. Wozny. “Geometrically deformedmodels: a method for extracting closed ge-ometric models from volume data”. SIG-GRAPH Comput. Graph., vol. 25, no. 4,pp. 217–226, 1991. ISSN 0097-8930. doi:http://doi.acm.org/10.1145/127719.122742.

[9] A. Ghanei, H. Soltanian-Zadeh and J. Windham.“A 3D deformable surface model for segmenta-tion of objects from volumetric data in medicalimages”. Computers in Biology and Medicine,vol. 28, no. 3, pp. 239–253, 1998.

[10] J. Lee, S. Kim, D. Jang, T. Ha, J. Kim, I. Kim,J. Kwon and S. Kim. “Deformable model withsurface registration for hippocampal shape de-formity analysis in schizophrenia”. Neuroimage,vol. 22, no. 2, pp. 831–840, 2004.


[11] J. Pantel, D. O’Leary, K. Cretsinger, H. Bock-holt, H. Keefe, V. Magnotta and N. Andreasen.“A New Method for the In Vivo VolumetricMeasurement of the Human Hippocampus WithHigh Neuroanatomical Accuracy”. Hippocampus,vol. 10, no. 6, 2000.

[12] D. MacDonald, N. Kabani, D. Avis and A. Evans.“Automated 3-D Extraction of Inner and OuterSurfaces of Cerebral Cortex from MRI”. Neu-roimage, vol. 12, no. 3, pp. 340–356, 2000.

[13] H. William, S. Teukolsky, W. Vetterling andB. Flannery. Numerical Recipes in C: The art ofscientific computing. Cambridge university pressNew York, NY, USA, 1988.

[14] W. Lorensen and H. Cline. “Marching cubes:A high resolution 3D surface construction algo-rithm”. Proceedings of the 14th annual conferenceon Computer graphics and interactive techniques,pp. 163–169, 1987.

[15] J. Gower. “Generalized Procrustes Analysis.”Psychometrika, vol. 40, no. 1, pp. 33–51, 1975.

[16] F. Rohlf and D. Slice. “Extensions of the Pro-crustes method for the optimal superimpositionof landmarks”. Syst. Zool, vol. 39, no. 1, pp. 40–59, 1990.

[17] R. Davies, C. Twining, T. Cootes, J. Watertonand C. Taylor. “A minimum description lengthapproach to statistical shape modeling”. MedicalImaging, IEEE Transactions on, vol. 21, no. 5,pp. 525–537, 2002.

[18] R. Davies, C. Twining, P. Allen, T. Cootes andC. Taylor. “Shape discrimination in the hip-pocampus using an MDL model”. InformationProcessing in Medical Imaging, vol. 4, 2003.

[19] E. Meintjes, J. Jacobson, C. Molteno, J. Gatenby,C. Warton, C. Cannistraci, H. Hoyme, L. Robin-son, N. Khaole, J. Gore et al. “An fMRI study ofnumber processing in children with fetal alcoholsyndrome”. Alcoholism: Clinical and Experimen-tal Research, vol. 34, no. 8, pp. 1450–1464, 2010.

[20] S. Jacobson, M. Stanton, N. Dodge, M. Pienaar,D. Fuller, C. Molteno, E. Meintjes, H. Hoyme,L. Robinson, N. Khaole et al. “Impaired Delayand Trace Eyeblink Conditioning in School-AgeChildren With Fetal Alcohol Syndrome”. Al-coholism: Clinical and Experimental Research,2010.

[21] R. Woods. “Multitracer: a Java-based tool foranatomic delineation of grayscale volumetric im-ages”. NeuroImage, vol. 19, no. 4, pp. 1829–1834,2003.

[22] J. L. Bentley. “Multidimensional binarysearch trees used for associative search-ing”. Commun. ACM, vol. 18, pp. 509–517, September 1975. ISSN 0001-0782. doi:http://doi.acm.org/10.1145/361002.361007.URL http://doi.acm.org/10.1145/361002.361007.

[23] T. Moller. “A fast triangle-triangle intersectiontest”. Journal of graphics tools, vol. 2, no. 2, pp.25–30, 1997.

[24] D. Eberly. 3D game engine design: a practicalapproach to real-time computer graphics. MorganKaufmann Pub, 2001.

[25] A. Worth, N. Makris, M. Patti, J. Goodman,E. Hoge, V. Caviness and D. Kennedy. “Precisesegmentation of the lateral ventricles and cau-date nucleus in MR brain images using anatomi-cally driven histograms”. Medical Imaging, IEEETransactions on, vol. 17, no. 2, pp. 303–310, 1998.

[26] Y. Xia, Q. Hu, A. Aziz and W. Nowinski. “Aknowledge-driven algorithm for a rapid and auto-matic extraction of the human cerebral ventricu-lar system from MR neuroimages”. Neuroimage,vol. 21, no. 1, pp. 269–282, 2004.

[27] P. Yushkevich, J. Piven, H. Hazlett, R. Smith,S. Ho, J. Gee and G. Gerig. “User-guided 3Dactive contour segmentation of anatomical struc-tures: Significantly improved efficiency and reli-ability”. Neuroimage, vol. 31, no. 3, pp. 1116–1128, 2006.

[28] M. de Bruijne, B. van Ginneken, W. Niessen,J. Maintz and M. Viergever. “Active shapemodel based segmentation of abdominal aorticaneurysms in CTA images”. In Proc. SPIE, vol.4684, pp. 463–474. 2002.

[29] J. Taylor. An introduction to error analysis: thestudy of uncertainties in physical measurements.Univ Science Books, 1997.

[30] S. Jacobson, C. Warton, N. Dodge, F. de Guio,C. Molteno, J. Jacobson and E. Meintjes. “Va-lidity of FreeSurfer for measurement of brain vol-umes in children with fetal alcohol syndrome”.In Proceedings of 16th Annual Meeting of the Or-ganization for Human Brain Mapping (OHBM).2010.

[31] B. Van Ginneken, A. Frangi, J. Staal, B. terHaar Romeny and M. Viergever. “Active shapemodel segmentation with optimal features”. Med-ical Imaging, IEEE Transactions on, vol. 21,no. 8, pp. 924–933, 2002.

[32] H. Wang, S. Das, J. Suh, M. Altinay, J. Pluta,C. Craige, B. Avants and P. Yushkevich. “Alearning-based wrapper method to correct sys-tematic errors in automatic image segmentation:Consistently improved performance in hippocam-pus, cortex and brain segmentation”. NeuroIm-age, vol. 55, no. 3, pp. 968–985, 2011.


APPENDIX A - LANDMARK POINT GENER-ATION

Mesh Initialisation

The initial mesh is generated from one of the targetdata volumes. The Matlab isosurface routine is usedto convert the binary volume into a mesh of vertices,edges and faces. This mesh is then smoothed twice us-ing the Laplacian smoothing technique described be-low.

vi =1

N

N∑j=1

vj (4)

Each vertex, vi, in the mesh is replaced by themean of its 1-ring neighbouring vertices, vj . N is thenumber of neighbouring vertices to vi.

The next step is to automatically register thesmoothed mesh with the target volume. This ensuresthat the mesh is fitted as closely as possible to thetarget, prior to commencing deformation. Registra-tion is done using the Procrustes technique [15]. Thistechnique iteratively rotates, scales and translates acandidate mesh to fit a given target mesh. An isosur-face is therefore created from the binary target volumeusing the isosurface routine. The initial mesh is thenregistered against this target mesh - effectively thesame as registering against the target volume itself.

After registration, our initial mesh is positionedwell enough relative to the target volume to commencedeformation. MacDonald’s approach is slightly differ-ent since it starts with an initial low resolution mesh,which is tessellated as it deforms to fit the target. Thismulti-resolution approach is good for situations wherethe initial mesh is not well registered. It also providesa computation speedup, as the searchable parameterspace is initially much smaller. The advantage of us-ing a single resolution approach, and starting with aninitial model that is similar in shape to the target vol-ume, is that the model will deform in such a way thatensures the placement of particular vertices in similarpositions on each target. This is essential to our for-mulation, as the generation of point correspondencebetween training data volumes is the purpose of thisstage of the project. MacDonald’s multi-resolution ap-proach is able to generate loose point correspondence,but this is not sufficient for our purposes.

Cost Function

The cost function is similar to MacDonald’s formula-tion [12]. It has the following formulation:

C(M) =

(A∗Timage+B∗Tstretch+C∗Tbending+D∗Tself−proximity)

(5)

C(M) is the cost function evaluating mesh M . Itconsists of 4 terms, each with its own weighting pa-rameter (A,B,C and D). Ideal weightings for each

term proved to be difficult to estimate, as this processrelied on trial-and-error. The output of each term issummed into a total cost value for the mesh.

Image Term

The Image term measures the sum of distances fromeach vertex to its nearest boundary, in both directionsalong the local surface normal. As in MacDonald’s for-mulation, search distances were thresholded in orderto prevent vertices from being attracted to incorrectboundaries. The equation for the Image term follows:

Timage =

nv∑v=1

dB(Mv, Nv, t) (6)

dB(Mv, Nv, t) measures the distance from vertexv, which belongs to deformable meshM , along its localsurface normal, Nv, to the nearest boundary of thetarget volume. This distance measure is thresholdedby t.

The surface normal for a specific vertex is calcu-lated as the mean of the normals of the faces surround-ing that vertex:

Nv =1

nf

nf∑f=1

Nv,f (7)

Nv,f denotes the normal of face f , neighbouringvertex v. nf denotes the number of faces neighbouringvertex v.

Boundaries are detected by sampling voxels inthe target volume at uniform intervals from a vertex,along the surface normal in both directions. When achange in voxel value from zero to non-zero (or viceversa) is detected, a boundary has been reached.

Stretch Term

In MacDonald’s formulation, the stretch term mea-sures the deviation in edge length between the de-forming mesh and an ideal model. This regularisationforce prevents the model from stretching or contract-ing into a shape that is jagged, or very different fromthe ideal model. We found that enforcing a prede-fined ideal edge length for each edge was too restric-tive, since this approach prevented overall scaling ofthe deformable model. We opted instead for an idealproportional edge length. The equation for the stretchterm is formulated as follows:

Tstretch =

ne∑e=1

∣∣∣∣ ||Me|| − ||Ime||Medge

∣∣∣∣ (8)

Me refers to the vector that represents edge e ofmesh M . Ime refers to the vector that represents edgee of the ideal mesh Im. ne is the number of edges inthe mesh. Medge is the average length of an edge inmesh M . It can be represented as follows:

Medge =1

ne

ne∑e=1

||Me|| (9)


The use of a proportional edge length allows ourmodel to be uniformly scaled, but still constricts defor-mation to shapes that are similar to the ideal model.

Bending Term

Our bending term is implemented similarly to Mac-Donald’s. The equation is as follows:

Tbending =

ne∑e=1

|a(M, e)− a(Im, e)| (10)

a(M, e) measures the angle between the faces ad-jacent to edge e, belonging to mesh M . Im againrefers to the ideal mesh. This term also measures de-viation from an ideal mesh. The angles between thenormals of adjacent faces are measured, and the ab-solute difference between each angle in the deformingmesh and the ideal mesh is summed.

We use the initial, smoothed mesh as our idealmesh. The stretch and bending terms therefore pe-nalise deviation from the initial mesh configuration.Since our initial configuration is similar to our tar-get shape, the GDM is prevented from deforming toshapes that are dissimilar to the target.

APPENDIX B - PRINCIPAL COMPONENTANALYSIS IN ASM CONSTRUCTION

The PCA phase in ASM construction involves cap-turing the statistics of the m aligned training shapes.Each shape is described by a vector of values, whichwe will call a shape description vector. If a PDM wasused in the previous stage, each shape description vec-tor will consist of a combination of n 3-D landmarkcoordinates. These coordinates are projected onto 3n-D shape space, thereby giving one 3n-D vector of val-ues for each of the m shapes. If the shape surface wasreparametrised to shape descriptors, then each shapedescription vector will consist of a set of shape de-scriptor parameters. The objective of using PCA inat this stage is to find the principal modes of variationof training shapes within the ASD.

The first step of PCA is to calculate the meanshape description vector x:

x =1

m

m∑i=1

xi (11)

Each shape description vector xi is then describedterms of its difference from the mean, such that:

dxi = xi − x (12)

The next step in PCA is to construct the covari-ance between each dimension across all the adjustedshape description vectors dxi. These covariance val-ues are stored in an NxN covariance matrix, whereN is the total number of dimensions present in eachshape description vector (e.g. N = 3n for PDM-basedvectors). The covariance matrix is constructed as fol-lows:

CNxN = (ci,j , ci,j = cov(Dimi, Dimj)) (13)

It can be shown that the eigenvectors of the co-variance matrix with the highest eigenvalues describethe most significant modes of variation between thevariables used to construct the covariance matrix [7].Thus, the next step in PCA is to find the unit eigen-vectors pkof the covariance matrix. For an NxN co-variance matrix, there exist exactly N eigenvectors.The eigenvectors pk(k = 1, .., N) satisfy the followingequation (where λkis the kth eigenvalue of C):

Cpk = λkpk (14)

Each eigenvalue indicates the amount of varianceexplained by its corresponding eigenvector. It is gen-erally the case that a significant amount of variancecan be explained by a small number of modes, t [7].This enables us to approximate instances in a spaceof N dimensions by using only t dimensions - withoutlosing much information as a result of the approxima-tion. An appropriate value chosen for t should balancevariation with model compactness. If t is too low, theASM will not be able to represent finer variations inshape. Conversely, if t is too large, the ASM will con-tain too many parameters - creating a large parametersearch space. The variance represented by t modes canbe evaluated in proportion to the total variance λT ,calculated by summing the eigenvalues:

λT =

N∑k=1

λk (15)

The feature vector, P = (p1p2...pt), is then createdas a matrix of the first t eigenvectors. Any shapein the ASD can now be approximated by adding themean shape description vector to the product of thefeature vector and a transposed vector of basis weightsb = (b1b2...bt)

T [7]:

x = x+ Pb (16)

By varying the values of the weights in b, we cangenerate new shapes that are not part of our trainingset, but are also within the ASD. This is the funda-mental concept on which ASMs are based. If land-mark points were reparametrised prior to PCA, thesewill need to be mapped back into Euclidean coordi-nates to be of use when fitting the ASM to targetdata.

Documents

A Heuristic Image Search Algorithm for Active Shape Model