Diplom:Bevegelse i ryggrada

Ragnar Kjørstad <ragnarkj@stud.ntnu.no>

2003-06-16

Bevegelse i ryggraden

En rekke lidelser i ryggraden gir opphav til unormale bevegelsesmønster mel-lom ryggvirvler når ryggraden beveges. Dette er et kjent fenomen for eksempel iforbindelse med revmatiske lidelser og slitasjeforandringer. I mange tilfeller måde unormale bevegelsene stabiliseres kirurgisk når sykdommen når et stadium derde unormale bevegelsene gir store smerter for pasienten eller der det er fare fornevrologisk skade. Tidspunktet for slik stabilisering har hittil vært valgt på bak-grunn av kliniske kriterier uten at det alltid har vært mulig å få god nok billedmes-sig kvantisering av graden av utslagene.

Ved Intervensjonssenteret ved Rikshospitalet kan slike pasienter undersøkes i enåpen MR. Den åpne MR-maskinen er den eneste av sitt slag i Nord-Europa ogtillater at pasienter undersøkes sittende. Dermed kan ryggen undersøkes med nor-male belastninger og bevegelsesutslag.

I dette prosjektet vil vi på bakgrunn av bildedata fra Intervensjonssenterets åpneMR, samt ordinære CT-data forsøke å kvantisere ryggvirvlenes bevegelse.

Prosjektet er et etablert prosjekt der Intervensjonssenteret og Reumatologiskavdeling ved Rikshospitalet samarbeider med University of Wisconsin/ Madison.

Acknowledgments

I would like to thank Lars Aurdal at Rikshospitalet and NTNU for guiding methrough this project.

I would also like to thank Per Kristian Hol at Rikshospitalet for helping me withunderstanding the medical side of the project, and providing the medical images.

Finally, I would like to thank Baxter Rogers and Victor Haughton at the Universityof Madison, Wisconsin for providing phantom-images and valuable help duringthe project.

Abstract

A wide range of health-problems are caused by abnormalities in the spinal columnand clinical diagnostics are not always reliable. This report analysis techniquesfor medical-imaging based diagnostics, discusses advantages and disadvantagesand suggests potential enhancements.

Magnetic Resonance Imaging (MRI or MR) is the preferred image-acquisitionsystem because it is believed to be without dangerous side-effects for the patient.Unfortunately MR is not yet able to produce very clear images of bone-structures,so some non-trivial image processing is required.

Most existing systems are based on image registration or traditional segmentationsof MR images. Image registration extracts information without doing traditionalsegmentation by matching intensity- or gradient-information in corresponding im-ages. This is used to find translation, rotation or scaling in one image comparedto another image of the same object.

The technical challenges of direct analysis, such as:

� Choice of energy-function� Interpolation� Minimization

are explored, and solutions are proposed. The different algorithms are comparedusing phantom- and real- datasets, and accuracy-estimates are made.

A complete system is developed, in part based on the solutions proposed in[Rogers]. Some of the implementation-challenges are discussed in some detail.

Alternative solutions based on traditional segmentation are discussed briefly, andpossible uses of model-based segmentation are described.

Contents

1 Introduction 1

1.1 Medical background . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Image acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Document structure . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Algorithms 7

2.1 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Energy functions . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4 Searching for the minimal value . . . . . . . . . . . . . . . . . . 30

3 Implementation 37

3.1 Flexibility and modularity . . . . . . . . . . . . . . . . . . . . . 37

3.2 Visualization and verification . . . . . . . . . . . . . . . . . . . . 41

3.3 KISS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.4 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

x CONTENTS

4 Results 51

4.1 Phantom results . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.2 Repeated subject-tests . . . . . . . . . . . . . . . . . . . . . . . . 55

4.3 Final results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5 Discussion 59

5.1 Phantom results . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.2 Repeated subjects-tests . . . . . . . . . . . . . . . . . . . . . . . 62

5.3 Final results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

6 Conclusion 65

7 Future work 67

7.1 Verification of results . . . . . . . . . . . . . . . . . . . . . . . . 67

7.2 Interface-improvements . . . . . . . . . . . . . . . . . . . . . . . 68

7.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

7.4 Automatic pivot-point selection . . . . . . . . . . . . . . . . . . . 71

7.5 3D registration . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

7.6 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Chapter 1

Introduction

The progress in computer science the last decades has made medical imagingpossible. However, there is still no general procedure to analyze medical images- each problem is handled separately.

This report will only discuss the use of medical imaging to analyze images of thespinal column, although some of the techniques can be used in other fields as well.

1.1 Medical background

Back pain is a very common problem, with a lifetime prevalence low-back painalone as high as 80% [Haughton02].

Disc degeneration is widely considered to be the major cause of back pain, butthe exact effect is not well enough understood. Inter vertebral disc degenerationis found at as much as 50% of asymptomatic individuals as well, so back-pain isclearly not always a direct effect.

Spinal instability secondary to disk degeneration is believed to account for 37%of cases of back pain [Friberg49], but the definition of instability and the criteriafor diagnosing patients with spinal instability are not generally agreed upon.

The spinal cord is supposed to be protected by the column [Bjalie98, Rodts03],but if it moves too much there may be pressure on the spinal cord and this willcause a lot of pain and can in potentially cause paralysis.

2 Introduction

Figure 1.1: Illustration of the vertebral column, from [Gray]. Spinal instability is related tolack of movement-constraints for the lumbar vertebrae.

1.1.1 Diagnostics description

Common methods for diagnosing spinal instability include invasive discogra-phy [Peh02] and non-invasive radiographic flexion-extension tests as describedin [Lowe, Buranapanitkit00].

Haughton [Haughton02] suggests an alternative way of diagnosing spinal instabil-ity. The test is similar to radiographic flexion-extension, but instead of having thepatient flex and extend the back, the patient rotates the upper body and the relativerotation at each vertebrae is examined. The assumption is that instable vertebraewill allow more rotation than other vertebrae.

Two MRI-datasets are taken for each patient. One is taken when the patient isrotating his or her upper body to the left, and the other when the patient is rotating

1.2 Image acquisition 3

Figure 1.2: Illustration of a vertebrae. The vertebrae encapsulates and protects the spinalcord. If a vertebrae is able to move too much, it may cause the nerves to get crunchedbetween to vertebrae, and the patient will experience pain. Copyright SpineUniverse.com

his or her upper body to the right.

Corresponding image-slices in the two datasets are registered, to find the relativerotation at each vertebrae. The difference in relative rotation of two neighboringvertebrae is a measure of the stability between the vertebrae.

1.2 Image acquisition

In this report we will focus on the use of MR and CT for image acquisition. Thedescriptions of the techniques are based on [Lisle01].

CT (Computed tomography) is an imaging technique whereby cross-sectional im-ages are obtained with the use of X-rays. A gantry with a X-ray tube on oneside and a set of X-ray detectors on the other side rotate while the patient passesthrough the gantry. The data from the detectors are analyzed by a computer, anda full 3D X-ray image is produced.

4 Introduction

Figure 1.3: Example CT- and MRI- images. The images show a slice in the lower part of thespinal column.

CT is extremely capable of differentiating between bone-structure and soft tissue.Unfortunately CT involves ionizing radiation, and therefor CT should not be usedmore often than required.

MRI (Magnetic resonance imaging) is an image acquisition technique where mag-netic forces are used to analyze an object. First a strong magnetic field (

���) is

applied, to align all the magnetic components of the object (hydrogen atoms).

The���

field is not fixed. It has a gradient component that causes the strengthof the field to vary linearly over the area of interest. The proton in the hydrogennuclei has spin at the Lamour frequency of the strength of the magnetic field itis subjected to. All protons in a specific slice perpendicular to

���have similar

spin-properties.

RF-coils create pulses with signal-frequency set to match the Lamour-frequencyof the protons in one particular slice in the gradient-field. All the protons in thisslice react to the pulse (

���) and change spins. When

���is turned off they all emit

energy as they dephase. The RF-coils register the signal, identifying the type oftissue in that particular slice.

The process is repeated at different frequencies, to map the tissue in the otherslices. The whole procedure is then repeated with a differently directed gradient-component to

���, so the type of tissue along a different axes can be mapped.

Finally, the tissue-maps of each axes are combined and a 3D image can be created.[Hornak02]

1.2 Image acquisition 5

Figure 1.4: The MR-scanner at Rikshospitalet

The RF pulse can be adjusted to create an image with specific properties, forinstance bone-structures can be differentiated from other types of tissue. Un-fortunately even with special pulse-adjustments to detect bone-tissue, MR is notcapable of producing as clear and good images as CT for this purpose.

As opposed to CT, MR has no ionizing radiation and no other known adverseeffects on patients.

Open MR is a special MR where the patient can sit upright during the image ac-quisition. This allows the patient to move more freely, and images to be acquiredin a more natural position.

6 Introduction

1.3 Algorithms

We will develop/describe a set of algorithms to estimate the difference in rotationbetween two different vertebrae, as suggested in section 1.1.

The algorithms are based on energy-functions that quantifies the differences be-tween two images, and a minimal-search algorithm that searches for the set ofparameters that yields the lowest energy of the energy-function.

We will also outline how the model-based segmentation can be used for a morecomplete and accurate segmentation, and how this can be used to help diagnosespinal instability and other causes of back-pain.

1.4 Document structure

The algorithms used in the segmentation of the spinal column are described inchapter 2. The implementation is described in detail in chapter 3.

Results are presented in chapter 4 and discussed in chapter 5.

In chapter 6 we draw conclusions, and in chapter 7 we suggest possible improve-ments and future work.

Chapter 2

Algorithms

In this chapter we will describe how to use image registration to assist in thediagnostics of medical problems related to spinal instability. The algorithm isbased on the algorithm in [Rogers, Rogers02].

The diagnostics will be based on the rotation-test described in section 1.1.1.

For each vertebrae, the image from the the rotation to the right (the right image)and the image from the rotation to the left (the left image) are registered.

Figure 2.1: Corresponding left and right images, with pivot-points and regions of interesthighlighted

8 Algorithms

The registration is done by comparing different transformations of the right imageto the original left image. The difference is quantified by the use of an energy-function, and the rotation-part of the transformation found to produce the leastenergy (the images are the most similar) is used as a measure of rotation for thatvertebrae. A general description of image registration is found in [Friston95].

The difference of rotation at two vertebrae is used to estimate the rotation overthe inter-vertebral disc. Inter-vertebral rotations of up to 3 degrees are consid-ered normal - rotation of more than 3 degrees may be an indication of instability.[Haughton03].

2.1 Transformations

There are of course an unlimited number of possible transformations to apply toan image, so we need prior knowledge about what kind of transformations arepossible to be able to find the parameters for the transformations.

In this particular problem, we know that we should only rotate and translate theright image to try to match it to the corresponding left image.

The combination of rotation and translation is called a rigid transformation, whichis a subclass of the affine transformation-class.

The rigid transformation has 3 parameters; the rotation, and the x and y coordi-nates of the translated origo.

For performance-reasons, we seed the algorithm with user-supplied x and y co-ordinates, and the rotation from the previous vertebrae (or 0 degrees, for the firstvertebrae).

2.2 Energy functions

In a perfect world the two images would line up perfectly after applying our mod-ifications and we could simply test if they were equal or not.

Unfortunately we can not expect the images to match perfectly. Some error mayhave been introduced in the digitalization of the image and there may be otherdifferences between the two images than the movement/rotation we are lookingfor.

2.2 Energy functions 9

Figure 2.2: Corresponding left and right images, with pivot-points and regions of interesthighlighted. The images on the bottom show only the region of interest, with the right imagerotated to match the left one.

To be able to find the best match out of a set of non-perfect matches we needan energy-function. The energy-function takes two parameters, the original firstimage and the transformed version of the second image, and returns a scalar value.1

The energy-function should return the lowest value when the images match the

1The implementation uses a slightly different form of energy-functions, which returns a matrixrather than a scalar. The matrix represents the energy-contribution from each pixel in the region.This makes visualization and manual intervention easier. The scalar value (used to compare differ-ent energy-levels) is calculated by taking the mean value from the matrix. The scalar is also scaledlinearly to typically fit in the interval [0,1] for better visualization. This is discussed in more detailin section 3.1.1

10 Algorithms

best. We also want the energy-function to return strictly increasing values whenwe move away from the best-fit parameters, because this makes the search for theglobal minimum much easier (it should have as few local minimums as possible).

A large number of alternative energy-functions exists, and they tend to performwell in different situations. A short description of some of them follows. Thecomparison is in part based on [Penney98] and [Maintz].

For each function a plot of the energy is included. The energy is calculated bycomparing a left image with a rotated right image from a real-world dataset. Theenergy varies with the amount of rotation, and the minimum energy should indi-cate the rotation at the vertebrae. As we do not have a golden standard available,it can not be used to compare the accuracy of the energy-functions directly, butit illustrates some of the properties of the function, such as the "smoothness" andthe "reach" of the global minimum and the number of local minimums. All areimportant properties that affect the difficulty of finding the global minimum.

A "difference-image" is also included. This image illustrates the difference be-tween the left image and the right image. The difference is estimated with theenergy-function, and illustrates both the scale of the error and what parts of theimages contribute to the error.

2.2.1 Absolute difference

�� � ��� ��������� � ������� �"!$# ��% ���&�'�(! � (2.1)

The simplest function imaginable is to take the sum of the absolute differenceof intensity-values in the two images as described in equation 2.1. This energy-function will fail in situations with different lighting, because a small differencein intensity-value on all pixels will overshadow the error from incorrect modifica-tions.

2.2.2 Squared difference

�)� �*�+� ������� � � ���,���'�(!$# ��% ����� �"!&! % (2.2)

2.2 Energy functions 11

−200 −150 −100 −50 0 50 100 150 2000.2

0.22

0.24

0.26

0.28

0.3

0.32

0.34

0.36

Rotation

Ene

rgy

Figure 2.3: Absolute differenceThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionThe relative low energy for a very narrow range of rotations is a good sign that the energy-function is able to differentiate matching- and non-matching settings.

−200 −150 −100 −50 0 50 100 150 2000.1

0.15

0.2

0.25

0.3

0.35

Rotation

Ene

rgy

Figure 2.4: Squared differenceThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionThe energy-signature of this function appear to be similar to that of absolute difference.

Using the intensity-difference squared as an energy-measure is very similar to us-ing the absolute difference, but makes a few very different pixels relatively moreexpensive. This can be useful in cases where small intensity-differences are irrel-

12 Algorithms

−200 −150 −100 −50 0 50 100 150 2000.832

0.834

0.836

0.838

0.84

0.842

0.844

0.846

Rotation

Ene

rgy

Figure 2.5: Normalized Cross CorrelationThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionAs with absolute difference and squared difference there is a narrow range of rotations thatproduce a relatively low energy based on normalized cross correlation.

evant to the transformation we’re studying.

2.2.3 Normalized Cross Correlation

�)�-# . �*�+� �����/�10 � �2����� �"!$# � �43 0 ��% �,���'�(!$# �5% 36 . �*�+� �&�7�/� 0 � �2���&�'�(!$# � �83 % . �*�+� ������� 0 ��% �,���'�(!$# �5% 3 % (2.3)

The energy-function based on normalized cross correlation (equation 2.3) worksmuch better in situations with different lighting, because the absolute values ofintensity is not used directly but compared to the average intensity-level. Also,edges and other features with extreme intensity-levels are given higher weight inthe cross correlation function, because of the the intensity-levels are multiplied.

2.2.4 Entropy of the difference Image

��9 �;: � � �<#>= ��% (2.4)?@�A=B!C�D# &EGF �,HI!8J�KML F �,HI!(2.5)

2.2 Energy functions 13

−200 −150 −100 −50 0 50 100 150 2000.19

0.2

0.21

0.22

0.23

0.24

0.25

0.26

Rotation

Ene

rgy

Figure 2.6: Entropy of the difference ImageThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionUnlike the previous functions, the entropy-based energy-function does not have a narrowrange of rotations where the energy is very low. Instead, there are multiple local minima,indicating that the energy is not primarily a function of the distance to the best rotation.

A difference image is calculated by subtracting a scaled version �N% from � � (Equa-tion 2.4). A histogram is formed from �B9 �O: , and the entropy of the histogram iscalculated according to equation 2.5.

The scaling factor, s, should be chosen such that the lowest possible energy-value?P�Q=N!is found. 2

As with the normalized cross correlation the entropy of the difference image alsoworks pretty well on differently lighted images: If the image is otherwise correctlyregistered ��9 �O: will be close to constant and

?@�Q=N!will be a large negative value

(low energy) compared to a scattered ��9 �;: .

2To find the best scaling factor a we could add the scale-variable to our minimalization-procedure. However, as the two images are of the same object and should have the exact samelight-properties in this case, we make the assumption that a scaling factor of 1 will cause thelowest energy-value to be found.

14 Algorithms

−200 −150 −100 −50 0 50 100 150 2000.425

0.43

0.435

0.44

0.445

0.45

0.455

Rotation

Ene

rgy

Figure 2.7: Mutual informationThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionMutual information is the energy-function with the energy-plot that has the clearest andnarrowest energy-minima.

2.2.5 Mutual information

R � E � S F �,HT�VUW!&J,KML F �,HT�VUW!F �,HI! F ��UW! (2.6)

F �,HI!refers to the histogram of � � , F �,UX!

to the histogram of �N% andF �,HT�VUW!

is thejoint histogram.

It’s worth noting that as mutual information does not depend on the intensity-values in the images, only the histogram of intensity-values, it works just as wellwith images with different lighting. As opposed to the energy-function previouslydiscussed it also work with completely different intensity-measures, as long asmutual information is still present. [Viola95] describes how to use mutual infor-mation to register depth-maps to regular images, as well as regular MR imageregistration.

This important feature makes it possible to register MR images to CT images eventhough the intensity-levels for a given object may be completely different in thetwo images [Maes97, Studholme96].

Further information on mutual information can be found in [Pluim00b, Pluim02].

2.2 Energy functions 15

−200 −150 −100 −50 0 50 100 150 2000.838

0.84

0.842

0.844

0.846

0.848

0.85

Rotation

Ene

rgy

Figure 2.8: Gradient correlationThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionThe plot based on gradient correlation has a relatively broad range of rotations that producelow energy. There are also multiple local minima along the rotation-axis. The extremelylow energy-contributions along the border of the circular region of interest is a result of asimplification in the implementation. (Described in chapter 3). The pixels outside the regionof interest are replaced with black pixels, causing a high gradient along the border for allimages. The consistently high gradient causes high gradient correlation. This artifact is notbelieved to affect the final result of the minimization.

2.2.6 Gradient correlation

Gradient correlation based energy functions are calculated by first finding the hor-izontal and vertical gradient image for both input images.

Normalized cross correlation is then performed on the two horizontal and verti-cal gradient images respectively. The final result is the average value of the twonormalized cross correlations.

As this energy-function is based on gradient information rather than intensity-values it performs better in situations where the intensity- value itself may not bereliable because of uneven lighting or other effects.

16 Algorithms

−200 −150 −100 −50 0 50 100 150 2000.2

0.22

0.24

0.26

0.28

0.3

0.32

Rotation

Ene

rgy

Figure 2.9: Pattern intensityThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionAn interesting effect in this particular plot is that there is a very clear local minima of energyat ca -150 degrees rotation, where the rotation is exactly 180 degrees off the global minimum.

2.2.7 Pattern intensity

YTZ � [ �A=N!\� ��� � 98]&^ Z ]

_ %_ %<` � ��9 �O: �,���'�(!$# ��9 �;: �,ab�Vc�!&! % (2.7)

d % �e�,�f#@a"! % ` �g��#@c�! % (2.8)

The theory behind the pattern intensity energy function is that the difference-image, �h9 �O: , will not have any structures of objects if the registration is correct.

A pixel in the difference-image is assumed to be part of an object if it’s intensityvalue is different from it’s neighbors. The neighborhood is defined by equation2.8 with a fixed radius r, and the energy-function is defined in equation 2.7. Theconstant k is used to weight many small differences against a few large ones.

As with gradient-based energy functions the pattern intensity function will alsoallow low-frequency differences (like those produced by uneven lighting) with avery small energy-price.

2.2 Energy functions 17

−200 −150 −100 −50 0 50 100 150 2000.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Rotation

Ene

rgy

Figure 2.10: Gradient differenceThe plot on the left show the energy-level at different rotation-settings, and the image onthe right show the energy-contributions from different parts of the image. The white pixelsindicate high energy-contribution, and the dark pixels indicate low energy-contributionThe energy at the global minimum is not very much lower than other local minima.

2.2.8 Gradient difference

i �Q=N!C� �+� � jlkjlk ` � ��9 �O:m:mn �,���'�(!�! % ` jloj�o ` � ��9 �O:m:mp �,���'�(!�! % (2.9)

��9 �O:m:mn �,���'�(!q� d � �d � #r= d ��%d � (2.10)

��9 �O:m:/p ����� �"!C� d � �d � #r= d ��%d � (2.11)

Gradient difference uses the difference between the gradients of the two images.Two constants, jsk and j�o are used for weighting, and the scale s is chosen to givethe lowest possible energy-value 3. As with pattern intensity the weights can beadjusted to find the correct balance between a lot of small errors and a few biggerones.

Since this function operates on gradient images it’s not very sensitive to low fre-quency differences.

3To find the best scaling factor a we could add the scale-variable to our minimalization-procedure. However, as the two images are of the same object and should have the exact samelight-properties in this case, we make the assumption that a scaling factor of 1 will cause thelowest energy-value to be found.

18 Algorithms

2.2.9 Remarks to energy plots

Although the energy-plots included in the previous section are the results of asingle dataset, and does not necessarily represent typical results, some remarksare in place.

2.2.9.1 Signal to noise ratio

An accurate estimation of signal to noise ratio would require a pre-known "cor-rect" signal to compare to. In this case, that’s not available. Instead, we use theterm signal to refer to a virtual measure of "how well the images fit".

Short of a better indication, we can only compare the differences in the signalstrength related to the distance to the global minimum, to the differences in thesignal strength that appear unrelated to the distance.

Plots with a single relative strong peak in signal and little fluctuation is an indica-tion of strong signal to noise ratio.

From this we can deduce that the energy-functions of "absolute difference","squared difference" "Normalized cross correlation" and "Mutual information"all appear to have high signal to noise ratios.

"Gradient difference" appear to have an particularly low signal to noise ratio.

2.3 Interpolation

A simplification of the process can be described as:

1. The real-world situation represent an analog signal

2. The signal is sampled in time and space into a digital representation, with afixed resolution.

3. We translate or rotate the image, and need to resample the signal

2.3 Interpolation 19

The goal of the interpolation is to resample the signal and give an approximationto the original signal at other spatial locations. According to the Nyquist theorem[Nyquist24, Shannon49, Jain89], the original signal can be reconstructed withouterror if and only if the signal is band limited to a frequency up to half the samplingrate. In our case it’s not, so the approximation can not be guaranteed to be 100%correct - however; different algorithms exist to estimate the original signal.

For a general description of image interpolation, see [Lehmann99].

2.3.1 Interpolation techniques

Three common methods of interpolation include interpolation by conversion tothe frequency domain, using an interpolation-kernel and spline interpolation.

By converting an image to the frequency-domain and back, the resolution can beadjusted. The Fourier transform is commonly used for this purpose. [Eddy96]describes how to use Fourier interpolation to improve image registration, and[Larkina97] describes how the fast Fourier method can to accurately rotate sam-pled images.

The Fourier transform requires a lot of computing resources, so most image pro-cessing tools instead use an interpolation kernel. The image is constructed bylinear combination of the values of neighboring pixels. The sinc-kernel with in-finite reach is known to reproduce the original without error if the sample rate ishigh enough, but it turns out to be a very bad choice of interpolation-kernel whenusing a limited reach [Meijering99].

Instead, linear- and cubic- interpolation [keys81] is found to be the best kernel for2- and 4- pixels reach, respectively [Meijering99].

A third alternative to Fourier and interpolation-kernels are spline- interpolation[Meijering01].

2.3.2 Interpolation errors

A lot of work has been done to study, and to reduce, the errors introduced byinterpolation.

Unfortunately, much of this work, such as [Ting97, Nijenhuis96], is focused onhow humans perceive images, and how they can be modified to look better to us.

20 Algorithms

This doesn’t help where, as in our case, the interpolated images are used for furthermachine-processing.

[Thevenaz00] and [Ostuni97] give a more general description of interpolation,resampling and relevant artifacts.

2.3.3 Bias interpolation

The imperfections of the interpolation causes errors in the translated image, andthus in the energy between the two images.

When searching for the variables that cause the lowest energy, the interpolationwill cause uncertainty in the energy when interpolation is needed. However, whenthere is no rotation, and when translation is done in whole pixel increments, theinterpolation is not needed and the energy-estimates are more accurate.

This would not be a problem if the error introduced by the interpolation was un-bias. If the interpolation generally increases (or decreases) the energy, it can causetwo new problems. It can create more local minima, and it can affect the globalminimum.

2.3.3.1 Adding errors to a perfect solution

If we have found a translation that causes a perfect match, any errors in the rep-resentation of the images can only create more differences and thus increase theenergy.

The interpolation may therefor cause extra energy for sub-pixel translations,where the energy for full-pixel translations, where no interpolation is needed, isnot changed. See figure 2.11 for details.

This effect would cause our optimization to be bias towards solutions that causeless interpolation, such as 0 degrees rotation and full pixel translations.

2.3 Interpolation 21

0 0.5 1 1.5 2 2.5 30

100

200

300

400

500

600

Offset (pixels)

Ene

rgy

Figure 2.11: This plot show how interpolation adds energy in a perfect-match situation Animage is translated to an offset and back, and is then compared to the original. If the inter-polation was perfect the image would be unchanged and the energy at the comparison zero.If the translation is done with a full-pixel offset, no error occures, the energy is still zero andthe match is perfect. If, however, the translation is done with a sub-pixel offset, the opera-tion introduces error and the energy is higher. The result is that if the correct solution to aregistration-problem includes sub-pixel translations the energy at the correct solution willnot be zero, and the minimalization will end up at a point where less interpolation is needed,to the side of the correct solution.

2.3.3.2 Interpolation affecting variance

Many interpolation algorithms have an implicit smoothing. This can easily beseen in bilinear interpolation where extreme-values (min/max) may be reduced,and no new extreme-values are introduced. Clearly this reduced the variance inthe data-set. For other interpolation-algorithms the logic is more complicated, butthe effect can easily be demonstrated.

This is illustrated in figure 2.12 by displacing an image-section by sub-pixel dis-

22 Algorithms

0 0.5 1 1.5 2 2.5 3 3.5 40.035

0.04

0.045

0.05

0.055

0.06

0.065

0.07

0.075

0.08

Offset (pixels)

nearestlinearsplinecubic

Figure 2.12: This plot illustrates how the variation of intensity-values is affected by interpo-lation. All algorithms (except Nearest neighbor, which does not help in achieving sub-pixelresolutions) cause a general reduction of variance in the dataset for sub-pixel translations

tances, and plotting the variance in the section. The original signal is cyclic so thatthe displacement does not affect the data in the active "window", and the varianceis not affected by the translation alone. For full pixel translations we can see thatthe variance is unaffected for all interpolation-algorithms.

For sub-pixel translations, however, the variance is reduced when using linearinterpolation, cubic interpolation or spline interpolation. Nearest neighbor doesnot cause a reduction in variance, because it doesn’t offer true sub-pixel translation- we can see that the variance is fixed at the original level.

It can also be shown that the reduction in variance affects the energy when twosections are compared. For absolute difference, squared difference and normal-ized cross correlation this can easily be deduced from the definition.

[Maes98] and [Pluim00] describe how statistical properties of images are affected

2.3 Interpolation 23

126 126.5 127 127.5 128 128.5 129 129.5 1300

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Translation (pixels)

Ene

rgy

(rel

ativ

e)abssquaredccmutual

Figure 2.13: This plot illustrates how the energy is affected by the interpolation. The energy-levels have been normalized to a 0-1 range for better visualization. The different energy-functions are all affected, but affected differently. For energy based on absolute difference,squared difference and normalized cross correlation the interpolation causes a reduction inenergy for sub-pixel translations. For energy based on mutual information, the energy isgenerally lower for full pixel translations, although the tendency is less obvious.

by linear-interpolation and partial volume interpolation4 and how this affects themutual information energy function.

The end result of this problem is that the energy will be lower(higher) the moreinterpolation we perform, and the minimization will be bias towards(away from)values that causes the most interpolation effect, such as rotation other than 0, 90,180 and 270 degrees and translation of 0.5, 1.5, 2.5 and so on pixels.

Figure 2.13 illustrates how the different energy-functions are affected by the in-

4Partial volume interpolation is not a regular interpolation-algorithm, but an algorithm to findthe histogram of an interpolated image. It is tightly connected with the use of the mutual informa-tion energy-function, and can not be used for other energy-functions such as mean squared erroror normalized cross correlation.

24 Algorithms

−8

−6

−4

−2

0

2−3−2

−10

12

3

−9.2

−9

−8.8

−8.6

−8.4

−8.2

−8

−7.8

x 10−5

Translation (pixels)Rotation (degrees)

Ene

rgy

Figure 2.14: This plot of energy as a function of rotation and translation in one dimen-sion illustrates how interpolation-artifacts affects the energy. The energy is clearly higherat variable-combinations that cause less interpolation, such 0 degrees rotation or full pixeltranslations. In this case the error is large enough to both affect the global minimum, and tocreate artificial local minimums where the optimization-procedure may get trapped

terpolation. Absolute difference, squared difference and cross correlation all havevery clear local minima of energy at full pixel translation, whereas sub-pixel trans-lation cause a reduction in energy. For mutual information the effect is different:full pixel translations have generally a lower energy than sub-pixel translations,but the energy-graph is less smooth, so it is more difficult to identify the energy-contribution of the interpolation.

2.3.4 Real world example

The effects of interpolation is not only a theoretical one. It can easily be shownthat bias interpolation significantly affects the energy of real-world images.

2.3 Interpolation 25

125 126 127 128 129 130 1310.836

0.8362

0.8364

0.8366

0.8368

0.837

Translation (bx)

Ene

rgy

Figure 2.15: Energy as a function of translation. The higher energy for full-pixel intervalsindicate that the interpolation that occurs at all other translation has a bias effect on theenergy.

Figure 2.14 shows how the energy varies with translation and rotation. The energyis clearly significantly affected by the interpolation both with regards to rotationand translation. The energy is much higher for 0 degrees rotation and for full pixeltranslations than other transformation.

The errors introduced by the interpolation is in this case large enough to cause lo-cal minima in the energy-landscape, and could potentially cause the optimization-routine to get trapped.

Although it is not possible to tell by visual inspection, it is likely that errors thislarge may also affect the global minima.

26 Algorithms

−10 −8 −6 −4 −2 0 2 4 6 8 100.877

0.8775

0.878

0.8785

0.879

0.8795

0.88

0.8805

Rotation (degrees)

Rel

ativ

e en

ergy

Energy as function of rotation

Figure 2.16: Energy as a function of rotation. The high energy for 0 degrees rotation indicatethat the interpolation that occurs at all other rotations has a bias effect on the energy.

2.3.5 The test

As the effect of the interpolation is relatively small, we construct an artificialexample were the effect is more evident and countermeasures can be tested.

We create two images with random uniform noise, choose a random pivot-pointin each image and calculate the energy at different translations and rotations.

As the images are just uniform noise, the energy-cost should to be at the same levelfor all possible rotations and translations. However, in addition to some smallrandom changes, figures 2.16 and 2.15 also shows that the energy is generallylower there is rotation and sub-pixel translation.

The effect is strong enough to affect the result of the minimization (when workingwith real images), and must be corrected.

2.3 Interpolation 27

−10 −8 −6 −4 −2 0 2 4 6 8 100.835

0.8355

0.836

0.8365

0.837

0.8375

Rotation

Ene

rgy

OriginalVariance compensated

Figure 2.17: The results indicate that the histogram equalization has removed the bias com-ponent of the interpolation.

2.3.6 Compensation to eliminate bias interpolation

We want to compensate so the statistical properties, like the variance and the stan-dard deviation, of the image are preserved after the interpolation. We do this bydoing a histogram equalization after the interpolation. This effectively sets theaverage intensity-value and the variance in the section fixed, so it does not affectthe energy-cost.

The results in figures 2.17 and 2.18 show that the effect of the interpolation iscompletely eliminated. The energy-variations caused by the differences in theimages (random variations) are mostly preserved. This is best seen in figure 2.17where we can see the new curve following the shape of the old one except around0 degrees rotation. Unfortunately, as seen in figure 2.18 the resulting curve is notperfectly smooth. Tiny imperfections make optimization much harder, and mayalso affect the end result.

28 Algorithms

−3 −2 −1 0 1 2 30.8356

0.8358

0.836

0.8362

0.8364

0.8366

0.8368

0.837

0.8372

0.8374

Translation (bx)

Ene

rgy

OriginalVariance compensated

Figure 2.18: The results indicate that the histogram equalization has removed the bias com-ponent of the interpolation. Unfortunately the plot is still not perfectly smooth.

2.3.7 Alternative compensation-procedure

An alternative compensation-procedure was proposed in [Grootoonk00].

The procedure uses a single image.

A displacement simulation is performed by resampling the image with a tiny (sub-pixel) translation. The image is then registered with the original, to translate itback. The resulting image will be marginally different from the original, becauseof the interpolation-errors.

The procedure is then repeated with larger displacements, creating a series of withtiny interpolation-errors compared to the original.

2.3 Interpolation 29

Principal component analysis is performed on the realigned image-series to char-acterize the functional form of the translation-dependence on the intensity varia-tions.

It turns out that the major form of error at each pixel can be described as a sumof a sinus and a cosinus-function of the translation in each dimension. For twodimensions, the function is:� Z't u�v8wIxmy*t 9 � ��z Z � {8� | v8y ` j ��=h�Q}C�A~B�2HI! ` j %m� K�=5�Q~N�2HI! ` jl� =M�A}C�A~B�fUW! ` j�� � K�=5�Q~B�2UX!where

His the translation in the x-dimension and

Uis the translation in the y-

dimension.

The coefficients j �to jl� are then found for each pixel by examining all the im-

ages in the series, and minimizing the square error, and can later be used to com-pensate for the errors introduced by interpolation.

Since the procedure requires a minimalization for every pixel to find the coeffi-cients, the procedure has been found to be to computational expensive for thisproject, and will not be examined in more detail.

2.3.8 Increase resolution to fix interpolation-problem

An alternative solution is to use a higher resolution in the difference-image, thanthe original images. When the resolution is increased, interpolation is requiredeven if there is no rotation and only fully pixel translations, so there is no longerany reason for the optimization to be bias about these values. This procedure isbased on ideas in [Pluim00] and [Maes98].

As seen from figure 2.19, this eliminates the error introduced by the interpolation.The general shape of the plot is preserved, although it doesn’t match perfectly.However, as opposed to the compensation-procedure presented in the previoussection, the plot is perfectly smooth.

Increasing the resolution is a costly solution, so it is important to increase theresolution enough to handle the interpolation-error properly, without wasting toomuch resources. To find the best compromise, we calculated the energy contin-uously as we increased the resolution, and found that when the resolution wassufficiently large the energy stabilized, and further higher resolution did not sig-nificantly affect the energy. Figure 2.20 shows how the energy is affected by thechoice of resolution.

30 Algorithms

−3 −2 −1 0 1 2 30.835

0.8355

0.836

0.8365

0.837

0.8375

Translation (bx)

Ene

rgy

OriginalIncreased resolution

Figure 2.19: The results indicate that the increase in resolution has removed the bias com-ponent of the interpolation.

This example uses a region of interest with a radius of 50,5 pixels, so the originalresolution is 101x101. We see that the energy-function is "unstable" for resolu-tions lower than the original one - this is also to be expected, as much informationis discarded. We also see that there is a spike in energy for a resolution exactlyequal to 101x101. This is also to be expected, as a result of the interpolation-error.

For higher resolutions than the original one, the energy-function appear to be sta-ble. We therefor recommend using a resolution that is higher than the original,such as 102x102 pixels or higher in this particular example.

2.4 Searching for the minimal value

Once we have defined an energy-function we just need a way to find the parame-ters that produce the lowest energy to find the correct registration. There are many

2.4 Searching for the minimal value 31

0 50 100 150 200 250 300 3500.828

0.83

0.832

0.834

0.836

0.838

Resolution

Ene

rgy

Figure 2.20: The energy is affected by the resolution that is used. Low resolution causes theenergy-function to be unstable, and an resolution that exactly matches the original resolutioncauses the energy to be affected by interpolation-errors. Resolutions higher than the originalappear to cause stable results.

algorithms available, and the choice is a tradeoff between speed and the chance offinding the best solution.

2.4.1 Full search

The simplest solution is to try all combinations of parameters, and calculate theenergy for each combination - the parameter-set with the lowest value wins. Thisalgorithm always produce the optimal solution, and for some small problems it’sthe best solution.

The problem, however, is computing-time. The calculation of the energy- functiontakes � �7} % ! calculations, with a constant factor in the range from 2 and upwards

32 Algorithms

depending on the energy-function.

This means that this method is only possible on relative small images and onlywith a few unknown parameters, depending on the hardware available and thetime-limitations for the calculations.

For example, if we have to search a parameter-space with 3 unknowns, say x-translation in the range -10 to 10 pixels, with a 0.01 pixel resolution, y-translationin the range -10 to 10 pixels, with a 0.01 pixel resolution and a rotation in therange -10 to 10 degrees with 0.01 degrees resolution, we would need to perform~���������~���������~��������������h���

energy-calculations. Each energy-calculation inturn requires at least one operation for each pixel within the region of interest -typically thousands of pixels.

Clearly this is a formidable computing-task.

2.4.2 Independent search

The full search doesn’t work well because it doesn’t scale with the number ofunknown parameters. We’ve seen that with just 3 parameters it requires a hugenumber of calculations and if we were to search for the parameters of a move-ment and a rotation (5 unknown parameters) it is simply not possible on standardcomputer hardware.

However, if the parameters are independent of each other, we can simplify thealgorithm - instead of testing all combinations of x, y and z we would first findthe best value of x, then go on to find the best value of y and finally find the bestvalue for z.

Unfortunately in real life situations the unknown parameters are rarely 100% in-dependent. If we’re doing a movement it’s likely that the optimal x-offset dependson the y-offset and visa versa.

A possible compromise is to search for the parameters one by one, but repeatthe process until it stabilizes. When it does, it means that we have found a localminima with lower energy-values along any parameter axis. For some types ofproblems this is typically also the global minima.

2.4 Searching for the minimal value 33

2.4.3 Hierarchal search

An other alternative to improve the performance of the search is to use an hierar-chal search.

A low resolution search is first performed over the full variable-space. Subsequentsearches are performed with higher resolution, but only in the area of the variable-space indicated to hold the lowest values by the previous such.

This combines high resolution and high performance.

2.4.4 Powell’s algorithm

Like the independent search described in the previous section, powell’s algorithmperforms multi-dimensional minimization-search by successive line minimization- it minimizes the function in a single direction and then moves on to minimizealong a different direction. This is repeated until a local minima is found.

The difference is the way the direction of minimization is found. In the indepen-dent search algorithm we searched along the axes of the coordinate-system, andin some situations this is painfully slow. Imagine a 2-D image with a diagonal"valley". The independent search routine would require a lot of tiny steps to findthe minimal value.

Powell’s algorithm records the direction that has been taken during eachminimization-loop, and subsequently it is able to minimize directly along thatdirection rather than minimizing along the directions of the axis.

This allows Powell’s algorithm to converge towards a minima with less iterationsthan the independent search algorithm.

Powell’s algorithm is described in [Press92].

2.4.5 Hill-climbing

Although the independent search routine and Powell’s algorithm are much fasterthan a full search they still both require a minimization in a single dimension.If the energy-function is expensive to calculate, and the parameter-space in eachdimension is too large, it may still be too slow.

34 Algorithms

An alternative too minimize along the whole dimension is to walk a single stepat the time. If we always move to a point with lower energy than the previouspoint we will always end up at a local minima. This is the theory behind thehill-climbing algorithm.

Hill-climbing easily ends up on non-global minimums, so a wide range of varia-tions exists to fix this problem.

2.4.6 Tabu search

The tabu search is a variation of hill-climbing that attempts to avoid ending up atnon-global minimums.

The algorithm does not terminate when it reach a local minimum, but rather itcontinues to search for lower energy. To avoid immediately revisiting the sameposition, the algorithm maintains a list of positions in the variable-space recentlyvisited, the tabu-list. Only positions not in the tabu-list are considered whensearching for the neighbor with the lowest energy.

This ensures that the area around a local minimum is explored, and the search mayescape the local minimum if the ridge is not too far away.

2.4.7 Simulated annealing

Simulated annealing is another variation of hill-climbing.

Simulated annealing works by taking a step in a random direction from the currentposition. If the energy in the new position is lower than the previous one it willproceed to take a new step. If, however, the energy is higher than at the previousposition it means we’ve taken a step in the wrong direction. However, sometimesone needs to cross a mountain to get to the ocean, so simulated annealing doesn’talways retreat. Instead it uses a random-function to determine if it should go backor not. The chance of proceeding despite the increase in energy is set high at thebeginning, but is lowered after each step. This allow the algorithm to pass greatobstacles in the beginning - to find the general area of the minima, but still settleon a minima at the end.

The idea and name of this process comes from the physics of annealing metal; thecrystals tend to move to positions with lower energy-levels, but sometimes theymove to higher energy-levels. The chance of moving to a higher energy-level is

2.4 Searching for the minimal value 35

high when the temperature of the metal is high, but the crystals slow down at fixedpositions as the metal cools off.

Simulated annealing is described in [Press92].

2.4.8 Nelder-Mead (Simplex) Method

The Nelder Mead method [Nelder65] is a simplex-based method based on theideas presented in [Spendley62]. A simplex of n+1 vertices,

H��to

H |h� �(where n

is the number of dimensions in the search / the number of variables in the equation)is constructed from an initial guess at the solution. For a 2D search, a triangle isconstructed.

The simplex is then iteratively changed in an aim to find the minimum. At eachstep, the list of the vertices, is kept ordered by the functional value at each point,so that � �,Hf��!s� � �7H % !s�e�M�M�f� � �7H |M� ��!

. A centroid of the n first vertices,H��

, iscalculated to help finding the vertices for the simplex in the next iteration.

At each iteration one of the following operations are used:

Reflection The vertex with the highest functional value (energy) is replaced tomirror the simplex about the other vertices. The new vertex will be at

HT�$#�,H |M� �$#�Hb�8!.

Expansion The vertex with the highest functional value (energy) is replaced tomirror the simplex about the other vertices and moved further in the samedirection to expand the simplex. The new vertex will be at

HT�M#�~X�7H |M� �B#�Hb��!.

Contraction The vertex with the highest functional value (energy) is replacedto contract the simplex. The contraction can be based on the reflection, orthe original simplex. For an inner contraction, the new vertex will be atHb� ` �

% �,H |M� ��#�Hb�8!, and for an outer contraction (based on the reflection),

the new vertex will be atHI��#�� �

% �7H |M� �$#PHb�8!.

Shrinkage All vertices except the one with the lowest functional value (H��

) arereplaced to shrink the simplex. They are moved to positions

H � � H2� `�% �,H � #�H2��!

- they are moved half way towardsH � .

The type of operation is chosen by evaluating each possible operation in turn, andchecking if it is successful at coming closer to the goal. The exact evaluations and

36 Algorithms

a more detailed description of the algorithm as well as it’s history can be found in[Wright96].

The procedure iterates until a set of termination criteria are met. The criteria maybe based on the size of the simplex (terminate when the simplex is smaller than afixed tolerance) or on the difference in functional-value at the vertices.

Nelder Mead is usually not the fastest minimalization-procedure, but is popularbecause it is easy to implement and does not depend on first order derivatives.

Both are big advantages to us, as we want to keep the implementation simple, andwe do not have first order derivatives available. They would have to be estimatedby calculating the energy at multiple points, and this is a relatively costly operationand not 100% accurate.

2.4.9 Conclusion

The purpose of this comparison is not to perform an in-depth analysis of all avail-able search-algorithms and their pros and cons.

Rather, we need to select an algorithm that give reasonable performance for inter-active use and a high probability of finding the global minimum. Easy implemen-tation was also a consideration.

A combination of full-search and the Nelder-Mead Simplex Method was chosen.First a low resolution full-search is conducted, and then the Nelder-Mead SimplexMethod is initiated at the minimum found by the full search.

This has been found to avoid most local minima, and to perform reasonably well.

Chapter 3

Implementation

The implementation was written to test out different algorithms and to try to verifythe results. The main design-goals were:

1. Flexibility and modularity

2. Visualization and verification

3. KISS (Keep It Simple, Stupid)

4. Performance

3.1 Flexibility and modularity

The single most important feature of the implementation is the ability to test outalternative algorithms. This calls for high flexibility and modularity.

3.1.1 Energy-functions

The choice of energy-functions is central to the image registration algorithm.We wanted to be able to easily implement and test a wide specter of differentenergy-functions, and test their performance on both phantom-data and real-worlddatasets.

38 Implementation

Figure 3.1: To approximate the calculation of energy in the region of interest we calculatethe energy of a square with all pixels outside the ROI replaced with black pixels

In order to do this, the energy-functions were written as separate plug-ins, with awell-defined interface. All that is needed to add more energy-functions is to add anew file with the implementation-code, and add the name of the energy-functionto the menu-structure in the GUI-code.

We want the implementation of energy-functions to be as easy as possible, so allgeneric code is placed in the outer layers common to all energy-function plug-ins. The energy-function receives two bitmaps as input-arguments, and return thedifference.

Ideally we would like the energy-functions to only consider the pixels within acircular region of interest (ROI), but unfortunately there is no practical way ofpassing only the pixels inside the ROI in the argument.1 We would rather not putthe responsibility of extracting the ROI to the energy-function itself, as that wouldmake it more work to implement new energy-functions.

The chosen solution is to pass a square area to the energy-functions, but have thepixels outside the circular ROI replaced with black pixels, as illustrated in figure3.1. For energy- functions such as absolute error and squared error it can easily beproven that this procedure will result in the exact same result as considering onlythe ROI. For other energy-functions the procedure will approximate the energy ofthe ROI.

1For some energy-functions it would have been sufficient to send a list of the pixels inside theROI, but others need the relative position of the pixels to calculate the energy.

3.1 Flexibility and modularity 39

¡q¢ ��£ �A¤1¥¦L"¢ �s�ALW§ £ �Q¤1¥�L"¢ ¨�}2¢h©NL(U

Figure 3.2: For better visualization the energy-functions return a matrix of energy-contributions from the different positions in the image. This figure illustrates the energy-contributions when trying to register two images. The first to images are the left- and right-images we’re trying to register, and the third image is the energy-map. In the first row theimages are not aligned properly, and there are significant energy-contributions along theedges of the image. In the second row the right image has been rotated and translated to reg-ister to the left image, and there is generally less energy. (the energy-map is darker) Whatenergy is left is mostly concentrated around soft-tissue.

For visualization-purposes, we also need the energy-functions to return a matrixof energy-contributions from individual pixels rather than just a scalar as the tradi-tional definition of energy-functions uses. This will enable the operator to betterunderstand where the energy-contributions are made, and to verify that the reg-istration is correct. For best possible visualization the energy-functions shouldlinearly scale the results so the contribution of each pixel typically ends up in therange [0,1]. The result can be seen in figure 3.2.

3.1.2 ROI radius

At the start of the project we did not know what radius of the ROI (Region OfInterest) would best include enough information to perform a successful registra-

40 Implementation

Figure 3.3: This section has possible ROIs of radius 38 and 50 pixels highlighted. A relativelysmall region of interest will make it possible to only consider areas of the image with stablestructures such as bone. A larger region of interest will typically also include areas withsoft-tissue that may affect the registration-process, as can be seen in the far right side of theimage

tion and still exclude areas with soft-tissue that are irrelevant to the registration asgood as possible.

We needed to be able to test different radius-settings, and test their performance.For convenience, the radius-setting can be modified from the GUI. This makes iteasy to test out different settings on multiple datasets.

3.2 Visualization and verification 41

3.1.3 Histogram normalization

The histogram normalization procedure that we developed needed to be tested,and results with the procedure enabled and disabled needed to be compared.

An option to enable/disable the histogram normalization is included in the GUI.

3.1.4 Resolution

We also needed to experiment with different resolution-settings to test what set-tings worked well without introducing interpolation-errors.

An option to set the resolution is included in the GUI.

3.2 Visualization and verification

The development and testing of the implementation dependent on verification ofresults, but unfortunately no golden standard for results were available.

We needed to visualize the process so the operator could manually verify that theimages were registered correctly, and that the results were plausible.

Much effort has gone into finding ways of visualizing and exploring the results.

The main GUI can be seen in figure 3.4. The user-interface includes the full left-and right- images in the first column. The second column shows the sectionswith the regions of interest around the selected pivot-point. The images will betranslated and rotated during the search, so the operator can visually verify thatthe images do in fact lign up correctly and the right match is found.

3.2.1 Energy distribution

On the upper right the GUI shows the energy-contributions from the currentmatch. It can be of help to the operator to verify the match, and to identify possi-ble problems to the registration-process, such as soft tissue that looks different inthe two images and shows up as energy.

42 Implementation

Figure 3.4: The main GUI

3.2.2 Interactive search

The bottom right part of the GUI show the current translation and rotation of theright image. It can be used both to read out results, but also for the operator tointeractively steer the search by adjusting the translation or rotation.

In addition to the manual search the operator has the choice of doing a full searchor an fast search. The full search searches all rotations and translations withinan operator-given range for the best possible solution. The fast search uses theNelder-Mead algorithm to find the best solution. The later is faster but more likelyto be caught by a local minimum.

Both the right section-image and the energy-distribution illustration are updatedcontinuously.

3.2 Visualization and verification 43

Figure 3.5: When the user select a point on one of the selection-images, both the left- andright- images are updated with markers to show the positions of the corresponding points.In this case the user placed the marked on the edge of the vertebrae, but the mark clearly ismisplaced in the right image - indicating that the images are misaligned.

3.2.3 Cross referencing points

The operator may cross reference specific points in the left- and right- selection-images to verify correct registration. By clicking in either image the point, and thecorresponding point in the other image, are highlighted. This allows the operatorto verify that specific features lign up correctly and support the registration.

The effect is shown in figure 3.5.

3.2.4 2D Energy-plots

Plots of energy as a function of rotation or translation is an important tool to verifycorrect registration.

The plot should ideally have a single unique minimum at the point found by theregistration-routine, as in the example in figure 3.6.

If the registration ended up at a local minimum, the plot may show a lower mini-mum at a different position.

If the left- and right- images are too different, the plot will not not have a singlepoint of very low energy. Similarly, if the registration ended up with the wrong

44 Implementation

Figure 3.6: Energy as a function of rotation, with translation unchanged

translation, the rotation-plot will have no distinct minimums (because no rotationwill make the images match correctly) and visa versa.

These plots have also proven useful to detect interpolation-artifacts.

3.2.5 Surface-plot

The surface-plot is a more complicated illustration of the interaction of translation,rotation and energy.

The foundation for the illustrations is a calculation of energy for all combinationsof translation and rotation within a given range.

3.2 Visualization and verification 45

−1 0 1 2 3 4 50.39

0.395

0.4

0.405

0.41

0.415

Rotation (degrees)

Ene

rgy

Figure 3.7: This plot show energy as a function of rotation. A rotation of ca 3.4 degreesappear to produce the lowest amount of energy.

3.2.5.1 2D energy-plots

This is a variation of the 2D energy-plot described earlier where the energy ofrotation is defined as the lowest possible energy at a given rotation.

This is opposed to the simpler 2D plots where translation is kept fixed when plot-ting energy of rotation. An example of the 2D energy-plot is found in figure 3.7.

3.2.5.2 3D energy-plots

The energy-plots are 3D plots that show energy as a function of two of the follow-ing variables:

46 Implementation

156

158

160−1 0 1 2 3 4 5

0.39

0.395

0.4

0.405

0.41

0.415

0.42

0.425

Rotation (degrees)Translation, bx (pixels)

Ene

rgy

Figure 3.8: This plot show energy as a function of both rotation and translation in on dimen-sion. As in the 2D-plot the energy is lowest when the rotation is 3-4 degrees, but we can alsosee the distinct interpolation-artifacts that appear when rotation is 0 degrees and translationis of full-pixel increments.

� X translation� Y translation� Rotation

This is similar to the 2D energy-plots described in the previous section, but givesthe operator a more complete picture as two variables can be combined in theillustration. An example is included in figure 3.8.

3.2 Visualization and verification 47

156

157

158

159

160

129130

131132

1330

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Translation, bx (pixels)

Translation, by (pixels)

Rot

atio

n (d

egre

es)

Figure 3.9: This plot show how the optimal rotation depends on the translation-parameters.For some settings, the lowest energy is found at rotation 3-4 degrees, as in previous plots.However, we can also see that if the translation is off by at little as 1-2 pixels, the preferredrotation changes dramatically. For example, if we set the translation fixed at translation withpivot-points at coordinates (159,132) we would find that the best rotation (the one producingthe lowest energy) is at 0 degrees. This is an effect of the interpolation-artifact.

3.2.5.3 Variable relation

This set of plots help illustrate how the variables for translation and rotation relateto each other.

The plots answer the question "Given translation x and y, what would be the opti-mal rotation?". Like-ways the optimal translation in one dimension can be foundgiven the translation in the other dimension and the rotation.

The plot is useful to check the stability of the result of the registration. If a smallchange in translation in either dimension would result in a different rotation being

48 Implementation

optimal for registration, that would be a sign that the registration is unstable, andthat better verification is needed.

3.2.6 Result

The result of the registration is presented in a plot where the registration at eachvertebrae is included.

Both relative and absolute rotation at each vertebrae is shown.

By graphically presenting the result the operator may easier spot indications ofmis-registration such as unrealistic results.

3.3 KISS

Keeping the code as simple as possible was an absolute requirement both becauseit was needed to achieve the flexibility and modularity we wanted but also to makethe system easy to extend and change in the future.

At some points, the need for simple code required us to compromise performance.

We wanted to avoid having to link in external C-code, and instead write the wholesystem in matlab. This made some of the tight-loop code slower than it could havebeen, but cut down development-time and enabled us to easily try out differentalternative algorithms and implementations.

Further we wanted to stick to matlab’s internal functionality whenever possible.This meant limiting ourselves to the Nelder Mead search procedure, althoughPowell’s algorithm would most likely have been faster.

3.4 Performance

Even though at this phase the most important task was to test out multiple algo-rithms and implementation, some care was also made to make performance asgood as possible.

In particular two features ensured the best performance:

3.4 Performance 49

3.4.1 Combination of full search and Nelder Mead

By combining a full search and the Nelder Mead procedure we not only avoidedlocal minima, but it turned out it also improved performance compared to justrunning the Nelder Mead procedure by it’s own.

By running a full search with relatively low resolution we were able to initiate theNelder Mead much closer to the destination, and the cost of the full search provedsmaller than the gain.

3.4.2 Function cache

Both the searches and the generation of graphical material to help visualize andverify the results were relatively time-consuming. In particular, the backgroundmaterial for the surface-plots requires a lot of computation, and they needed to beperformed interactively as the operator inspected the result.

We implemented a workaround for this problem, by running a batch-job that per-formed the same calculations. A function-cache then stored the results, so thatwhen we inspected results interactively later, results were read from the cache andperformance was perceived as much higher.

Chapter 4

Results

We will first present the phantom-results that were used to find the optimal com-bination of settings, and to estimate the accuracy of the procedure.

We will then present the results of repeated tests with the same subject to verifythat the results are consistent over multiple runs.

Finally, we present the results for both patients symptomatic of spinal instabilityand a number of asymptomatic volunteers.

The results are discussed in chapter 5.

4.1 Phantom results

We used phantom-images to test different settings and to find out which one per-formed best.

We used six datasets, each with two images of the same phantom. In the right im-age the phantom is rotated 0, 0.5, 1, 1.5, 2 and 2.5 degrees respectively, comparedto the left image.

For every test, the results from the procedure is compared to these expected rota-tions, and a root mean square error is calculated for comparison.

There is a 0.1 degree error-margin in the rotation of the phantom.

52 Results

4.1.1 Energy function comparison

We compared the results when using different energy-functions:

The comparison was done with the following settings:

� Region of interest radius: 50 pixels� Variance-normalization: disabled� Interpolation-resolution: 201x201

Energy-function Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 RMS errorAbsolute error 0.000 0.405 0.935 1.411 2.007 2.374 0.079Squared difference 0.000 0.431 0.933 1.413 1.967 2.396 0.069Normalized cross correlation 0.000 0.428 0.932 1.410 1.969 2.396 0.070Entropy 0.510 1.882 -1.925 4.665 0.494 1.152 2.034Gradient Difference 0.000 0.359 0.868 1.317 1.885 2.475 0.119Gradient Cross Correlation 0.000 0.348 0.820 1.326 1.901 2.458 0.127Mutual Information 0.000 0.181 0.896 1.436 1.998 2.429 0.142Pattern Intensity 0.000 0.394 0.867 1.353 1.925 2.459 0.098

4.1.2 Radius comparison

We compared the results when using regions of interest with radius of 38 and 50pixels.

The comparison was done with the following settings:

� Variance-normalization: disabled� Interpolation-resolution: 201x201

4.1.2.1 Normalized Cross Correlation

Radius Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 RMS error38 0.000 0.431 0.923 1.420 1.935 2.420 0.067964550 0.000 0.428 0.932 1.410 1.969 2.396 0.0703385

4.1 Phantom results 53

4.1.2.2 Squared error

Radius Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 RMS error38 0.000 0.432 0.924 1.422 1.934 2.421 0.067182650 0.000 0.431 0.933 1.413 1.967 2.396 0.0691906

4.1.3 Variation-normalization comparison

We compared results with and without the variation-normalization procedure toreduce interpolation-artifacts described in section 2.3.6.

The comparison was done with the following settings:

� Region of interest radius: 50 pixels� Interpolation-resolution: 102x102

4.1.3.1 Normalized cross correlation

Variation normalization Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 RMS errorDisabled -0.000 0.353 0.882 1.376 1.967 2.410 0.10008Enabled -0.011 0.182 0.904 1.463 1.949 2.556 0.13998

4.1.3.2 Squared error

Variation normalization Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 RMS errorDisabled -0.000 0.356 0.885 1.379 1.964 2.411 0.09817Enabled 0.001 0.230 0.904 1.476 1.917 2.348 0.13704

4.1.4 Resolution comparison

We compared the results when using different interpolation-resolutions, to see ifhigher resolution solved the interpolation-problem, and how high the resolutionneeded to be for accurate results. The comparison was done with a 50 pixel radiusand using the normalized cross correlation energy-function.

54 Results

Resolution Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 RMS error41 0.000 0.505 1.013 1.509 1.916 2.416 0.0489745 0.000 0.440 0.896 1.419 2.059 2.504 0.0638749 0.000 0.435 0.919 1.204 1.805 2.412 0.1550153 0.000 0.418 0.906 1.405 1.905 2.467 0.0760557 0.000 0.421 0.952 1.348 1.977 2.365 0.0916561 0.000 0.453 0.984 1.390 1.867 2.339 0.0984765 0.000 0.389 0.851 1.336 1.951 2.371 0.1158069 0.000 0.456 1.011 1.500 1.991 2.417 0.0387973 0.000 0.392 0.860 1.327 1.923 2.428 0.1097877 0.000 0.468 0.922 1.427 1.949 2.374 0.0717881 0.000 0.430 0.933 1.361 2.003 2.467 0.0704985 0.000 0.404 0.900 1.434 1.924 2.396 0.0818289 0.000 0.454 0.947 1.410 1.970 2.401 0.0628993 0.000 0.445 0.924 1.406 1.950 2.431 0.0644297 0.000 0.390 0.880 1.419 2.013 2.398 0.08528101 0.000 0.632 1.040 1.457 1.954 2.403 0.07348105 0.002 0.401 0.889 1.410 1.973 2.374 0.08835109 0.000 0.441 0.952 1.430 1.963 2.419 0.05570113 0.000 0.439 0.926 1.413 1.978 2.390 0.06994117 0.000 0.436 0.930 1.418 1.951 2.391 0.07071121 0.000 0.425 0.911 1.393 1.982 2.409 0.07483125 0.000 0.433 0.931 1.426 1.959 2.392 0.06840129 0.000 0.439 0.918 1.408 1.971 2.398 0.07089133 0.000 0.429 0.928 1.410 1.969 2.405 0.06869137 0.000 0.430 0.926 1.411 1.962 2.396 0.07136141 0.000 0.433 0.937 1.413 1.966 2.399 0.06756145 0.000 0.430 0.925 1.411 1.967 2.397 0.07088149 0.001 0.434 0.934 1.408 1.965 2.405 0.06761153 0.000 0.430 0.926 1.411 1.971 2.404 0.06874157 0.000 0.434 0.935 1.415 1.963 2.397 0.06805161 0.000 0.431 0.926 1.412 1.971 2.403 0.06859165 0.000 0.434 0.932 1.408 1.966 2.394 0.07052169 0.000 0.431 0.928 1.410 1.965 2.399 0.07009173 0.000 0.433 0.930 1.413 1.967 2.402 0.06789177 0.000 0.433 0.928 1.408 1.969 2.398 0.07012181 0.000 0.433 0.931 1.408 1.968 2.401 0.06897185 0.000 0.435 0.927 1.410 1.969 2.398 0.06954189 0.000 0.431 0.928 1.410 1.968 2.398 0.07009193 0.000 0.432 0.930 1.409 1.968 2.399 0.06956197 0.000 0.433 0.928 1.411 1.968 2.399 0.06930201 0.000 0.428 0.932 1.410 1.969 2.396 0.070341000 0.000 0.433 0.929 1.406 1.969 2.398 0.07039

4.2 Repeated subject-tests 55

4.1.5 Accuracy estimates

We used the phantom-tests to estimate accuracy.

The tests was done with the following settings:

� Region of interest radius: 50 pixels� Variance-normalization: disabled� Normalized cross correlation� Interpolation resolution: 201x201

Dataset Result Expected result ErrorSet 1 0.428 0.500 0.072Set 2 0.932 1.000 0.068Set 3 1.410 1.500 0.090Set 4 1.969 2.000 0.031Set 5 2.396 2.500 0.104

Average error: 0.073 degreesStandard deviation: 0.0276Root mean square error: 0.070 degrees

4.2 Repeated subject-tests

To verify that our registrations were correct, we repeated the test twice on one ofthe subjects, and compared the results for the two runs.

The comparison was done with the following settings:

� Region of interest radius: 50 pixels� Variance-normalization: disabled� Interpolation-resolution 201x201

56 Results

4.2.1 Normalized cross correlation

Dataset S1 - L5 L5 - L4 L4 - L3e1242 0.459 3.618 -0.319e2035 0.617 1.601 -1.189

Correlation between the results in the two tests: 0.87

4.2.2 Squared difference

Dataset S1 - L5 L5 - L4 L4 - L3

e1242 0.474 3.627 -0.620e2035 0.611 1.596 -1.195

Correlation between the results in the two tests: 0.90

4.3 Final results

Finally we used the image registration procedure to find the inter-vertebral rotationfor a number of volunteers and back-pain patients.

The registration was done with the following settings:

� Region of interest radius: 50 pixels� Variance-normalization: disabled� Interpolation-resolution 201x201� Normalized cross correlation energy-function

The results were verified visually, and where they were clearly incorrect they wereremoved from the table, and replaced with the term "Fail".

4.3 Final results 57

Dataset Type S1 - L5 L5 - L4 L4 - L3 Max rotatione978 Volunteer 3.678 4.552 4.552e979 Volunteer 1.752 1.281 1.999 1.999e1015 Volunteer 1.630 2.748 Fail 2.748e1042 Volunteer 1.054 0.389 5.887 5.887e1064 Volunteer 2.441 1.303 3.041 3.041e1068 Volunteer 1.227 2.812 0.250 2.812e1069 Volunteer 3.162 3.259 1.537 3.259e1113 Patient 5.608 2.417 2.968 5.608e1114 Patient Fail Fail Faile1115 Patient 0.144 4.218 -1.853 4.218e1118 Patient 2.959 2.352 1.383 2.959e1119 Patient 3.291 2.731 2.237 3.291e1120 Patient 6.032 Fail Fail 6.032e1156 Patient 1.549 1.967 4.698 4.698e1163 Patient 3.243 -0.521 2.616 3.243e1164 Patient 0.015 0.300 2.471 2.471e1178 Volunteer 2.067 0.630 3.599 3.599e1181 Volunteer 0.407 0.066 0.877 0.877e1183 Volunteer 0.700 1.055 1.870 1.870e1186 Patient 0.978 3.063 -0.325 3.063e1187 Patient 2.291 1.707 7.326 7.326e1188 Patient 1.384 2.803 3.711 3.711e1190 Patient Fail Fail Faile1210 Patient 3.042 3.330 3.330e1214 Patient 0.411 4.849 -1.160 4.849e1225 Patient 0.195 2.733 2.933 2.933e1242 Volunteer 0.459 3.618 -0.319 3.618e2035 Volunteer 0.617 1.601 -1.189 1.601

Average maximal rotation for patients: 4.124 degreesAverage maximal rotation for volunteers: 2.954 degrees

Histogram for the rotation of the maximal inter-vertebral rotation:

58 Results

Rotation Patients Volunteers0-1 0.00% 8.33%1-2 0.00% 25.00%2-3 21.43% 16.67%3-4 35.71% 33.33%4-5 21.43% 8.33%5-6 7.14% 8.33%6-7 7.14% 0.00%7-8 7.14% 0.00%

Chapter 5

Discussion

In this chapter we will discuss the results given in chapter 4 and the effects onchoice of parameters and expected accuracy.

5.1 Phantom results

The tests does not accurately represent the real-world scenario, because some ofthe sources of error in the real world are not present, such as:

� Soft tissue� Motion blur� Small differences in vertical position of left and right image.

All this factors may potentially change the accuracy of the energy-functions. Still,the phantom results can be used as a lower bound to error in the procedure, and tosome extent be used to compare results with different settings.

Many results were accurate within only 0.1 degrees or less. This is as low asthe error-margin in our expected rotations, meaning that the differences could becaused by inaccurate expectations as well. The tests were, however, repeated on 6different datasets, and the inaccuracies should even out over the tests.

Since most of the tests were so accurate, for most settings, it is easy to assume thatthe settings are not very important. We should point out that part of the reason

60 Discussion

the results are so accurate is because the phantom-images are easier to register.For real datasets we would expect the results to be less accurate, and the for thesettings to differentiate more.

5.1.1 Choice of energy-function

The only energy-function that fails completely is the "Entropy" function with aroot mean square error of more than 2 degrees.

The functions based on "Gradient Difference", "Gradient Cross Correlation" and"Mutual Information" all perform reasonable well, with all results within 0.3 de-grees of the expected result and a RMS-error between 0.1 and 0.2 degrees.

The rest of the functions, "Absolute difference", "Squared difference", "Normal-ized cross correlation" and "Pattern Intensity" all perform well, with RMS-errorsless than 0.1 degrees.

Based on these results, and the comparison of the properties of the functions dis-cussed in section 2.2, we choose to use "Squared difference" and "NormalizedCross Correlation" in the rest of our study.

The biggest surprise to us was that "Mutual Information" was not among the bestenergy-functions, both because the energy-function is very frequently used in im-age registration literature, and because the features of the function and the energy-plots indicated that it would be a good candidate. The relatively poor RMS-errorfor mutual information is mostly caused by a mis-registration in the 3rd dataset,while the others are all less than ca 0.1 degree of the expected rotation.

It should be noted that the error-margin in the expected results was too large (0.1degrees) and the number of tests too low to make certain comparisons amongst thebest energy-functions. Neither is it certain that the function that have the lowesterrors for phantom images will have the lowest errors for real world images withmore sources of errors.

Still, we feel that the results are a strong indication that both "Squared error" and"Normalized cross correlation" are two good candidates for energy-functions.

5.1 Phantom results 61

5.1.2 Choice of radius

As described in section 3.1.2 the radius of the region of interest is an importantparameter greatly affecting the registration.

The results in chapter 4 indicate that the of 38 pixels result in less errors than aradius of 50 pixels.

The differences are, however, extremely small - well within the error-margin ofthe expected results. Therefor we can not make conclusive arguments as to thepreferable radius-setting.

5.1.3 Variation compensation

The results indicate that enabling variation compensation to avoid interpolation-problems causes a higher RMS-error.

This is believed to be caused by the loss of "smoothness" that again causes min-imization to be more likely to get stuck in local minima. It appears that this costoutweigh the benefit from removing the interpolation-bias.

Again, the differences in RMS-errors are very small, so the result is not conclu-sive.

5.1.4 Choice of resolution

The results in chapter 4 does not indicate that the RMS-error is reduced by in-creasing the interpolation-resolution, as we argued in section 2.3.8. Neither doesusing an interpolation-resolution exactly equal to the native resolution, 101x101,cause significantly higher errors, as we predicted.

At first look it may seem as though the results are contradicting our theory.

However, results for all resolutions higher than the native one result in a RMS-error lower than 0.1 degree. Given the error-margin in the rotation-expectations,we are unable to confirm or deny the theory by examining the RMS-error.

If we look at the actual rotations at registration, we find that the results for higherresolutions appear more stable. For instance the results for resolutions 201x201pixels and 1000x1000 pixels are almost identical, but the results for resolutions

62 Discussion

41x41 and 45x45 differ a lot more. The result for 101x101 is particularly differentfrom the other results.

We believe this to support the theory that the error is reduced with higherinterpolation-resolution, and that tests with more accurate phantom-data wouldshow this.

5.1.5 Accuracy estimates

We should first stress that the estimates are based on phantom-results only, anddoes not necessarily reflect the expected errors for real datasets.

Still, the results indicate that the root mean square error of the results with thepreferred settings are only 0.070 degrees, and the maximal error is only 0.104degrees.

5.2 Repeated subjects-tests

In an attempt to include all the sources of error, repeated tests were conducted onthe same person.

By verifying the consistency of results over consecutive tests, all sources of errormention in the previous section are included, as well as sources of error in theimage acquisition phase.

The consistency may also be affected by physiological changes in the patient be-tween the tests, so we are effectively over-estimating the error in the registration-phase itself.

This is, however, an advantage. If physiological changes in the patient causesthis test to fail, it is an indication that the whole diagnostic procedure is inappro-priate, even if the image-registration process itself does not necessarily cause theproblem.

Although the results of the consecutive tests were not exactly the same, there isa relative high correlation. We also see that features like having the biggest rota-tion being between vertebrae L5 and L4, and having a negative rotation betweenvertebrae L4 and L3 are preserved over consecutive tests.

5.3 Final results 63

This is a indication that the image registration was successful, but with only datafrom a single subject available the result is not conclusive.

5.3 Final results

Unfortunately, even with the preferred settings and with a high interpolation- res-olution to avoid artifacts, the algorithm fails to register several of the image-sets.

Since multiple the registration of some sets failed badly enough that the errorcould be detected by visual verification, we can not rule out that other registrationsfailed with smaller margins and went undetected.

Six of the datasets registered with negative rotation - that is, the inter-vertebralrotation between two vertebrae had the opposite direction of the rotation of otherinter-vertebral rotations. Intuitively we found this to be very unlikely, and anindication of mis-registration, but the fact that this feature was present in bothruns for the same subject indicates that it may actually be correct. More test-runs,in particular more tests with images from the same subject, would be helpful toresolve this question.

It is not possible to separate the symptomatic patients from the asymptomaticvolunteers by maximum inter-vertebral rotation alone. Some patients have as littleas 2.471 degrees maximum inter-vertebral rotation, and some volunteers have asmuch as 5.887 degrees rotation.

There is still a clear difference in the results for the two categories. The differencesin average maximal rotation, is a firm indication that spinal instability is in factrelated to maximal inter-vertebral rotation, and that image registration has beenaccurate enough to preserve this differentiation.

We also see that as much as 43% of patients have a maximum rotation of 4 degreesor more, while only 16% of volunteers have equally large rotations.

Among the symptomatic patients, the algorithm found a inter-vertebral rotation ofmore than 3 degrees for 11 of the 16 (69%) patients examined.

Chapter 6

Conclusion

A complete system for image registration to aid in the diagnostics of spinal insta-bility has been fully implemented and tested.

The system is not able to clearly differentiate between symptomatic patients andvolunteers, but a correlation between the symptoms and the inter-vertebral rotationhas been shown.

The system is able to find the location of inter-vertebral rotations larger than 3degrees for 69% of the symptomatic patients, and may in it’s current state be ofuse to assist in deciding treatment for pre-selected patients.

We have become aware of a number of potential improvements, to make the pro-gram easier and faster to use, to make the registration more accurate and to makestronger verification of results.

Chapter 7

Future work

This chapter describes potential future work that may help in building an improvedsystem. The first sections describes relatively minor adjustments to the currentsystem, while the later sections describe alternative approaches that should beexplored.

7.1 Verification of results

This work suffers from lack of an easy way to provide certain verification ofboth the image registration itself, and the medical conclusion drawn from theregistration-results.

Although much effort went into both technical and non-technical work to aid inverification, there is still much room for improvements.

Our first suggestion is to perform extended tests, repeated several times for eachsubject. Consistent results from the same patients over time will indicate success-ful registration. Both healthy subjects and patients with suspected spinal instabil-ity should be used for this verification.

The second suggestion is to verify that the registration-results are consistent withmedical results. As the existing diagnostics-systems are not 100% reliable, it’s notsufficient to compare results to those systems. Rather, results should be verifiedby following up patients that have had their vertebrae stabilized. We would hopeto see a stronger rate of recovery among the patients where the vertebrae that were

68 Future work

stabilized are the same as those indicated to be unstable by the system, than amongpatients where different vertebrae were stabilized.

Preferably a relatively large group of subjects should be used for verification.Part of the group could be used as a "training set", to tune parameters for betterperformance, and the rest of the group should be used as a control-group.

7.2 Interface-improvements

The graphical interface used for the interactive image registration should be fur-ther improved.

Once the system goes from being a development-test-bed, to a production-systemthe focus should also be changed from flexibility and modularity to ease of useand integration into a more complete system for diagnostics. This will affect thegraphical user-interface strongly.

7.2.1 Improved cross-reference

The current implementation includes a crude cross-reference feature, where pointsin the corresponding images can be marked to compare the positions of commonfeatures. This could be extended further, to make the operator able to interactivelyverify the results by comparing multiple points, edges or other features.

In the long term one may hope that automatic registration will be reliable enoughthat no interactivity is required, but in the short-term it will be extremely importantto do manual verification to avoid mis-registrations and to find potential for furtherimprovements to the system.

7.2.2 Alternative difference-image

The current system visualizes the registration-result by displaying the energy-contributions from the different parts of the image. The operator can use thisto check the general level of energy, and to see what features in the images arecausing the energy.

Alternative visualizations could be implemented to assist the operator further. Onepossibility is to overlay the left image with the transformed right image. The

7.3 Performance 69

overlay should be done in such a way that it’s easy to spot when there is a perfectmatch. One possibility is to combine a version of the left image represented in100% red and 50% green, with a version of the right image represented by 50%green and 100% blue.

If the images don’t match, the resulting image will have parts that turn towardsthe red colors (where the left image is lighter), and parts that turn towards the bluecolors (where the right image is lighter). If the left- and right- images have theexact same light-intensity, the result will be a tone of gray.

7.2.3 Slice and pivot-point selection

The current implementation expects slices and pivot-points for the registration tobe pre-selected.

This is unfortunate for two reasons.

First, it is less practical for the operator to first have to use a separate program tolook at the dataset, and select slices and pivot-points. If it could be done withinthe program, the operator would save time.

Second, sometimes image registration may fail because the selection of slices andpivot-points was not accurate enough. In such cases, it may be beneficial to beable to easily move the pivot-point, and try again, without leaving the system.

7.2.4 Help and documentation

Although the system is mostly self-explanatory, online and interactive documen-tation will be useful for new users.

7.3 Performance

Although performance was not the main focus for this project, it is an importantproperty for the system at a later stage.

It is important both to avoid keeping the operator waiting, wasting valuable time,and to enable the addition of new time-consuming features while keeping theregistration-time down.

70 Future work

7.3.1 Interpolation

Profiling-results indicate that a full 93.9% of computation-time during the searchis spent doing interpolation. This is clearly an important place to look for possibleimprovements.

The current implementation uses matlab’s implementation of cubic interpolation.Using a simpler interpolation, such as linear interpolation, is not a good alternativebecause of the artifacts discussed in section 2.3.3.2.

7.3.1.1 Faster interpolation

Some improvement may be achieved by implementing a better interpolation-procedure than the general purpose-implementation supplied by matlab. However,we speculate that the matlab-implementation is likely a relatively good one, andthat the potential for improvement is most likely small compared to the work itwould require.

7.3.1.2 Fewer interpolations

An alternative approach is to try to reduce the number of interpolations required.The current implementation interpolates at every possible image-comparisonthroughout the search, which is non-optimal.

It can be seen that when doing a full search, the only change between two subse-quent image comparisons is a small translation or a small rotation of the image.For example, we may compare the images at translations -2, -1.5, -1, -0.5, 0, 0.5,1, 1.5 and 2 pixels. Instead of doing an independent translation at each compar-ison, we could interpolate to double the resolution, and then move the image afull pixel (without interpolation) between each comparison. This could, in theory,reduce the number of interpolations for every rotation in the search from 9x9 toonly 1, improving performance by a staggering factor of 81.

For the subsequent Nelder Mead search, this approach doesn’t work directly.Nelder Mead searches by moving around in the parameter-space in small steps,and those steps may be of any size in the current implementation. To be able touse the same improvement during the Nelder Mead search, we would have to in-terpolate the image up to a much higher resolution, and then change the NelderMead implementation to only use full pixel intervals in the search.

7.4 Automatic pivot-point selection 71

Implementing this is not as trivial as it may sound, as it would require an integra-tion of the search- and interpolation- routines - a contradiction to the design-goalsof flexibility and modularity. However, when the system moves from a test-bed-phase to a production-phase, those priorities will change and the change may beworthwhile.

7.3.2 Improved search

The current implementation uses a two-phase search, where the first phase uses afull search with relatively low resolution and the second phase uses Nelder Meadsearch.

Once the interpolation-scheme has been improved as described in the previoussection, the relative "cost" of these operations will shift. Most likely full searchwill become relatively cheaper compared to the Nelder Mead search, with theconsequence that the resolution of the full search should be increased to make thefull search more accurate and reduce the "distance" the Nelder Mead search needsto cover.

It is possible that it would also be even faster to replace the full search with ahierarchal search, as described in section 2.4.3.

Most likely it would also be beneficial to replace Nelder Mead with Powell’s al-gorithm, to accelerate the second phase of the search. Alternatively some kindof heuristic search, search as Tabu search (section 2.4.6) or Simulated annealing(section 2.4.7) could be used to improve performance.

It should be noted that no absolute comments can be made on which search-algorithm will be the fastest without testing them for the problem in question.This particular problem is very special due to the requirement of high-cost inter-polation, which may influence the choice of search-algorithms.

7.4 Automatic pivot-point selection

The search-routine could very-well find the pivot-points for the right-image auto-matically rather than having the operator initiate them. 1

1The pivot-points of the left image, and thus selecting the region of the image we are compar-ing, can not be computed automatically so easily, as it requires knowledge of what regions of theimage include the relevant bone-structures.

72 Future work

Finding the pivot-point automatically, would make the job easier for the operator,at the cost of a longer search.

Automatic pivot-point selection would not add any new degrees of freedom to thesearch, because translation of the right image is already free to change - the onlydifference would be that the search would be initiated further away from the goal,and thus a wider range needs to be searched.

7.5 3D registration

The registration could potentially be done in 3D instead of in 2D. This basicallymeans selecting the slice for the right-image automatically.

3D registration would have two obvious advantages. First, the operator no longerwould need to select the slice for the right-image - making the job easier on theoperator. Second, the registration may now register the left image to a virtualslice, using interpolation, between two physical slices in the right-dataset.

This could be useful when the slices are misaligned, and the slice from the leftdataset doesn’t correspond exactly to a slice in the right-dataset.

3D registration would add at least 1 new degree of freedom. If the registrationwas also allowed to rotate the datasets around a new axis, it would add 2 newdegrees of freedom. This would have significant performance impacts, but maybe possible once the performance-improvements in previous sections have beenimplemented.

7.6 Segmentation

A completely different alternative to the image registration is to perform a fullimage segmentation.

A segmentation-procedure is in many ways more complicated, but may proveto perform more accurately, and perhaps more importantly, may provide moreinformation than the registration.

Instead of just finding the relative rotation at each vertebrae, a complete segmen-tation would provide complete information about how the vertebrae moves andbehaves during the rotation, or other movements.

7.6 Segmentation 73

This could potentially be a very important tool in understanding how the humananatomy works, and how disorders, such as, but not limited to, spinal instabilityaffect us.

7.6.1 Visualization

A full segmentation of the vertebrae-column will make new forms of visualizationpossible. It will be possible to render images emulating the view of the columnfrom any possible angle, and an operator may interactively change the point ofview to get a better understanding of how a specific subject’s vertebrae looks like.

If given multiple datasets, such as the left- and right- rotation-images used in thisstudy, or even more datasets with incremental movements, the operator may evensee how the vertebrae behaves during motion.

Such visualization will be helpful in gaining a more complete understanding ofspinal instability and other back-pain related problems. Instead of just lookingfor one particular effect from spinal instability, it would be possible to visuallycompare the behavior of the vertebrae at healthy subjects and patients with spinalinstability, and look for differences.

As far as we know, this kind of visualization has not been done before.

7.6.2 Model based segmentation

Performing a segmentation purely based on the information in the image, with noprior information about the shape and look of the vertebrae, is unlikely to succeed.The images are too blurry and the edges between bone and soft- tissue too unclear.Instead, focus should be on using model based segmentation.

Model based segmentation uses apriori information to perform the segmentation.Model based segmentation takes many forms. It has been used to improve theresults of traditional segmentation, such as in [Haris99] where a model of thecoronary arterial tree is used to label the arteries after morphological segmenta-tion and in [Sclaroff01a, Sclaroff01b] to group together regions that, accordingto the model, probably belongs to the same object. We will, however, focus onalgorithms that use the model throughout the segmentation.

74 Future work

7.6.2.1 Preprocessing

MRI-images may suffer from intensity inhomogeneities.

Such variations may cause problems during segmentation, because it may causeedges that are mistaken for object-borders by the segmentation-algorithm.

To avoid problems from intensity inhomogeneities, the images may be pre-processed to remove the variation. Compensation based on "bias fields" aredescribed in [wells96] and [styner00].

7.6.2.2 Active Contour Models (snakes)

Active Contour Models, or snakes, are one of the most common systems for modelbased segmentation. The term was introduced in [Kass88], and combines internal-and external- energy-contributions to control an energy-minimizing spline.

The internal-energy contributes to keeping the snake well-formed, with less tightcorners. The external-energy contributes to the snake’s being attracted to edges orother features in the image.

The internal-energy may easily end up shrinking the snake and make it disappearaltogether. [Lobregt95] suggests alternative functions for internal energy to avoidsuch deformations.

[McInerney95] suggests further improvements to the algorithm to make the snakeadopt to different topologies so it can be used to segment more advanced geomet-rical objects.

Another shortcoming of the original snake-definition is that external forces havea very short reach - the snake needs to be initiated relatively close to the object inorder to succeed. Xu and Prince introduce Gradient Vector Flows in [Xu98] and[Xu00] to solve this problem.

Even with all these improvements snakes still have a major shortcoming; they canonly take advantage of very limited apriori information about the shape of theobject. To take advantage of information about the full shape of the object, wemove on to consider active shape models.

7.6 Segmentation 75

7.6.2.3 Active Shape Models

Active shape models are similar to active contour models, but are trained to onlydeform to fit data consistent with the training set. In our case, we can train a modelto only deform to shapes consistent with a spinal vertebra, and with this constraintthe model is more likely to align with subtle edges and features in the image.

General descriptions of active shape models are found in [cootes92], [cootes94],[cootes95], [cootes98], [cootes01].

Minor variations and improvements to the algorithms are described in [szekely96],[Hill96] and [ginneken02].

The original proposal only uses the shape of the object to form the model, butin [Kelemen98] the use of intensity-information about the object is also used toestimate the energy during shape-deformation.

In [Kelemen99] Kelemen, Szekely and Gerig introduce an alternative to the solu-tion proposed by Cootes at al based on a hierarchical parametric object descriptionrather than a point distribution model.

A full survey of active shape models can be found in [McInerney96b], in a shorterversion in [McInerney96a] and in an updated version in [McInerney00].

7.6.2.4 Description / Parameterization

One of the challenges of active shape models is the description of the model, theparameterization.

Tools such as sinusoidal bases functions (such as Fourier) and principle compo-nent analysis are used to describe the model in a compact and effective way. Thecompact description is not only required to store the model easily, but more im-portantly the reduction in parameters enables the deformation of the model bythe variation of these parameters. By varying the biggest principle components,the model deforms to fit the shapes in the training-set. If the training-set is largeenough, the deformation should generalize to all possible shapes of the object.

Algorithms for parameterization are found in [staib96], [Stokely92] and[Brechbuhler92].

76 Future work

The description can, in theory, easily be extended to describe 3- dimensionalshapes. In practice, the task becomes much more complicate in 3D, and the exten-sions required are described in [staib92b] and [Brechbuhler95], and with furtherimprovements to make the algorithm scale better in [quicken00].

As with snakes, the topology of active shape models is typically fixed, but[McInerney99] includes extensions to make the models topology-adaptable.

In many situations, only the global shape of the object is significant - the localshape may vary from object to object. Global- and local- deformations are sepa-rated in [Sclaroff94] and [terzopoulos91] to allow local individual deformations,while still maintaining the global shape.

7.6.2.5 Probabilities

The constraints of the deformation of the model may not only be an absolute limit.Instead, the constraint may be based on probabilities.

The training build model-information about the probability of any given deforma-tion. (internal probability). This probability can be combined with the probabilityof an image being the projection of the same deformation (external probability) toevaluate the total probability of the deformation. The space of possible deforma-tions are searched for the one which is most probable.

This way the abstract and somewhat unclear and ill-defined "energy"-term, is re-placed with probability-terms.

Example of such probability-use in active shape models are found in [staib92a],[metaxes99] and [wang98b].

7.6.2.6 Anatomical atlases

A greater library of anatomical can be constructed to form an anatomical atlas.Such an atlas may include not only the model for individual anatomical objects,but the relationship between them. [evans94]

Extended anatomical knowledge allow the segmentation to make use of otherstructures in the image in addition to the object of the segmentation. This willhelp the algorithm navigate much the same way that radiologists do when analyz-ing medical images.

7.6 Segmentation 77

Such a massive model of information does of course require more computing-power. [Christensen96] describes how parallel computers can be used to speed upthe individualization of the atlases.

Annexes

Bibliography

[Bjalie98] J. G. Blålie, E. Haug, O. Sand, and . V. Sjaastad, Men-nesekkroppen, Fysiologi og anatomi, Gyldendal forlag,1998.

[Brechbuhler92] C. Brechbuhler, G. Gerig, and O. Kubler, Surfaceparametrization and shape description, Visualization inBiomedical Computing, (1992), pp. 80–89.

[Brechbuhler95] , Parametrization of closed surfaces for 3-d shape de-scription, Computer Vision and Image Understanding: ImageUnderstanding, 61 (1995), pp. 154–170.

[Buranapanitkit00] B. Buranapanitkit, Flexion-extension radiographs for de-tecting lower lumbar spinal instability : Standing vs lateraldecubitus views, Songkla Med J, (2000).

[Christensen96] G. E. Christensen, M. I. Miller, M. W. Vannier, andU. Grenander, Individualizing neuroanatomical atlases us-ing a massively parallel computer, IEEE Computer, 29(1996), pp. 32–38.

[Eddy96] W. F. Eddy, M. Fitzgerald, and D. C. Noll, Improved im-age registration by using fourier interpolation, Magn. Re-son. Med, 36 (1996), pp. 923–931.

[Friberg49] S. Friberg and C. Hirsch, Anatomical and clinical studies onlumbar disc degeneration, 1949.

[Friston95] K. J. Friston, J. Ashburner, C. D. Frith, J. B. Poline, J. D.Heather, and R. S. J. Frackowiak, Spatial registration andnormalization of images, Hum. Brain Mapp, 2 (1995),pp. 165–189.

BIBLIOGRAPHY 81

[Gray] H. Gray, Anatomy of the human body, Lea & Febiger,Philadelphia, 1918.

[Grootoonk00] S. Grootoonk, C. Hutton, J. Ashburner, A. M. Howseman,O. Josephs, G. Rees, K. J. Friston, and R. Turner, Charac-terization and correction of interpolation effects in the re-alignment of fmri time series, neuroimage, Neuroimage, 11(2000), pp. 49–57.

[Haris99] K. Haris, S. N. Efstratiadis, N. Maglaveras, C. Pappas,J. Gourassas, and G. Louridas, Model-based morhpologicalsegmentation and labeling of coronary angiograms, IEEETransactions on image processing, 18 (1999), pp. 1003–1015.

[Haughton02] V. Haughton et al., Dynamic spine imaging in the evaluationof low back pain and sciatic. proposal for NIH, 2002.

[Haughton03] V. Haughton, 2003.

[Hill96] A. Hill, T. f. Cootes, and C. J. Taylor, Active shape mod-els and the shape approximation problem, Image and VisionComputing, 14 (1996), pp. 7601–607.

[Hornak02] J. P. Hornak, The basics of mri. http://www.cis.rit.edu/htbooks/mri/.

[Jain89] A. K. Jain, Fundamentals of Digital Image Processing,Prentice Hall, New York, 1989.

[Kass88] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Activecontour models, International Journal of Computer Vision,(1988), pp. 321–331.

[Kelemen98] A. Kelemen, G. Szekely, and G. Gerig, Three-dimensionalmodel-based segmentation of brain mri, in Workshop onBiomedical Image Analysis, 1998, pp. 4–13.

[Kelemen99] , Elastic model-based segmentation of 3-d neuroradio-logical data sets, IEEE Transactions on image processing, 18(1999), pp. 828–839.

[Larkina97] K. G. Larkina, M. A. Oldfieldb, and H. Klemma, Fast fouriermethod for the accurate rotation of sampled images, OpticsCommunications, 139 (1997), pp. 99–106.

82 BIBLIOGRAPHY

[Lehmann99] T. M. Lehmann, C. G¨onne, and K. Spitzer, Survey: Inter-polation methods in medical image processing, IEEE Trans-actions on Medical Imaging, 18 (1999), pp. 1049–1075.

[Lisle01] D. A. Lisle, Imaging for Students, Oxford University Press,2001.

[Lobregt95] S. Lobregt and M. A. Viergever, A discrete dynamic contourmodel, IEEE Transactions on medical imaging, 14 (1995),pp. 12–24.

[Lowe] T. G. Lowe, Treatment of degenerative disc diseaseand degenerative spondylolisthesis of the lumbarspine - lumbar disc hernations, primarily radicu-lar pain. http://www.spineuniverse.com/displayarticle.php/article342.html.

[Maes97] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, andP. Seutenes, Multimodality image registration by maximiza-tion of mutual, information, IEEE Transactions on medicalimaging, 16 (1997), pp. 187–198.

[Maes98] F. Maes, Segmentation and registration of multimodal med-ical images: From theory, implementation and validationto a useful tool in clinical practice, Ph. D. thesis, CatholicUniversity of Leuven, Leuven, Belgium, 1998.

[Maintz] J. B. A. Maintz and M. A. Viergever, An overview of medi-cal image registration methods, Medical Image Analysis, 2(1998), pp. 1–36.

[McInerney00] T. McInerney and D. Terzopoulos, Deformable Models, Aca-demic Press, 2000, pp. 127–145.

[McInerney95] , Topology adaptive snakes, in Fifth International Con-ference on Computer Vision, 1995, pp. 840–845.

[McInerney96a] , Deformable models in medical image analysis, inWorkshop on Mathematical Methods in Biomedical ImageAnalysis, 1996, pp. 171–180.

[McInerney96b] , Deformable models in medical image analysis: A sur-vey, Medical Image Analysis, (1996), pp. 91–108.

BIBLIOGRAPHY 83

[McInerney99] , Topology adaptive deformable surfaces for medicalimage volume segmentation, IEEE Transactions on medicalimaging, 18 (1999), pp. 840–850.

[Meijering01] E. H. W. Meijering and M. A. V. W. J. Niessen, Quantitativeevaluation of convolution-based methods for medical imageinterpolation, Medical Image Analysis, 5 (2001).

[Meijering99] E. H. W. Meijering, W. J. Niessen, J. P. W. Pluim, and M. A.Viergever, Quantitative comparison of sinc-approximatingkernels for medical image interpolation, in Lecture Notesin Computer Science, C. Taylor and A. Colchester, eds.,vol. 1679, Springer-Verlag, Berlin, 1999, pp. 210–217.

[Nelder65] J. A. Nelder and R. Mead, A simplex method for functionminimization, Comput, (1965), pp. 308–313.

[Nijenhuis96] M. R. M. Nijenhuis and F. J. J. Blommaert, A perceptual er-ror measure for sampled and interpolated complex colourimages, Displays, 17 (1996), pp. 27–36.

[Nyquist24] H. Nyquist, Certain factors affecting telegraph speed., BellSystem Technical Journal, (1924).

[Ostuni97] J. L. Ostuni, A. K. S. Santha, V. S. Mattay, D. R. Weinberger,R. L. Levin, and J. A. Frank, Analysis of interpolation effectsin the reslicing of functional mr images, Journal of Com-puter Assisted Tomography, 21 (1997), pp. 803–810.

[Peh02] W. C. Peh, Lumbar discography, in RSNA Scientific Assem-bly and Annual Meeting, 2002.

[Penney98] G. P. Penney, J. Weese, J. A. Little, P. Desmedt, D. L. G. Hill,and D. J. Hawkes, A comparison of similarity measures foruse in 2-d-3-d medical image registration, IEEE transactionson medical imaging, 17 (1998), pp. 586–595.

[Pluim00] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, Interpo-lation artefacts in mutual information based image registra-tion, Computer Vision and Image Understanding, 77 (2000),pp. 211–232.

84 BIBLIOGRAPHY

[Pluim00b] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever., Im-age registration by maximization of combined mutual in-formation and gradient information., in Lecture Notes inComputer Science, vol. 1935, Springer-Verlag, Berlin, 2000,pp. 452–461.

[Pluim02] J. Pluim, J. Maintz, and M. Viergever, Improved image reg-istration robustness by an extension of mutual information,in International Society Magnetic Resonance in Medicine,2002.

[Press92] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flan-nery, Numerical Recipies in C, Cambridge University Press,1992.

[Rodts03] M. Rodts, Your healthy spine. http://www.spineuniverse.com/displayarticle.php/article1296.html.

[Rogers] B. P. Rogers and V. M. Haughton, An automated registra-tion algorithm for measuring relative rotation of vertebraein spinal mr images, Proc. Int. Soc. for Magn. Reson. in Med.(ISMRM), (2001).

[Rogers02] B. P. Rogers, V. M. Haughton, K. Arfankis, and M. E.Meyerand, Application of image registration to measure-ment of intervertebral rotation in lumbar spine, MagneticResounance in Medicine, (2002).

[Sclaroff01a] S. Sclaroff and L. Lie, Deformable shape detection and de-scription via model-based region grouping, IEEE transac-tions on pattern analysis and machine intelligence, 23 (2001),pp. 475–489.

[Sclaroff01b] , Corrections to "deformable shape detection and de-scription via model-based region grouping", IEEE transac-tions on pattern analysis and machine intelligence, 23 (2001),p. 685.

[Sclaroff94] S. Sclaroff and A. P. Pentland, On modal modelling formedical images: Unconstrained shape description and datacompression, IEEE Workshop on Biomed Image Analysis,(1994), pp. 70–79.

BIBLIOGRAPHY 85

[Sclaroff94b] , Object recognition and categorization using modalmatching, in CAD-Based Vision Workshop, 1994, pp. 258–265.

[Shannon49] C. E. Shannon, Communications in the presence of noise,in Proceedings of the Institute of Radio Engineers, vol. 37,1949, pp. 10–21.

[Spendley62] W. Spendley, G. R. Hext, and F. R. Himsworth, Sequentialapplication of simplex design in optimisation and evolution-ary operation, Technometrics, (1962), pp. 441–461.

[Stokely92] E. M. Stokely and S. Y. Wu, Surface parametrization andcurvature measurement of arbitrary 3-d objects: five practi-cal methods, IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, (1992), pp. 833–840.

[Studholme96] C. Studholme, D. L. G. Hill, and D. J. Hawkes, Automated 3dregistration of mr and ct images of the head, Medical ImageAnalysis, 1 (1996), pp. 163–175.

[Szekely00] G. Szekely and G. Gerig, Model-based segmentation of ra-diological images, Kunstliche Intelligenz, (2000), pp. 18–23.

[Thevenaz00] P. Thévenaz, T. Blu, and M. Unser, Image interpolation andresampling, in Handbook of Medical Imaging, Processingand Analysis, I. Bankman, ed., Academic Press, San DiegoCA, USA, 2000, ch. 25, pp. 393–420.

[Ting97] H.-C. Ting and H.-M. Hang, Edge preserving interpola-tion of digital images using fuzzy inference, Journal of Vi-sual Communication and Image Representation, 8 (1997),pp. 338–355.

[Viola95] P. Viola and W. M. Wells, Alignment by maximization ofmutual information., in Computer Vision (ICCV’95), 1995,pp. 16–23.

[Wright96] M. H. Wright, Direct search methods: once scorned, nowrespectable, in 1995 Dundee Biennial Conference in Numer-ical Analysis, 1996, pp. 191–208.

[Xu00] C. Xu and J. L. Prince, Gradient Vector Flow DeformableModels, Academic Press, 2000, pp. 159–169.

86 BIBLIOGRAPHY

[Xu98] , Snakes, shapes, and gradient vector flow, IEEE Trans-actions on image processing, 7 (1998), pp. 359–369.

[Zana01] F. Zana and J. Klein, Segmentation of vessel-like patternsusing mathematical morpology and curvature evaluation,IEEE Transactions on Image Processing, 7 (2001), pp. 1010–1019.

[cootes01] T. F. Cootes, G. J. Edwards, and C. J. Taylor, Active appear-ance models, IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, 23 (2001), pp. 681–685.

[cootes92] C. J. T. T. F. Cootes, D.H. Cooper and J. Graham, Trainingmodels from sets of examples, British Mach. Vision Conf,(1992), pp. 9–18.

[cootes94] T. F. Cootes, A. Hill, C. J. Taylor, and J. Haslam, The useof active shape models for locating structures in medicalimages, Image and Vision Computing, (1994), pp. 355–366.

[cootes95] D. H. C. T. F. Cootes, C. J. Taylor and J. Graham, Activeshape models - their training and application, Computer Vi-sion and Image Understanding, (1995), pp. 38–59.

[cootes98] T. F. Cootes, G. J. Edwards, and C. J. Taylor, Active appear-ance models, in 5th European Conference on Computer Vi-sion, vol. 2, Springer, 1998, pp. 484–498.

[evans94] A. Evans, M. Kamber, D. I. Collins, and D. MacDonald, Anmri-based probabilistic atlas of neuroanatomy, MagneticResonance Scanning and Epilepsy, (1994), pp. 263–274.

[ginneken02] B. van Ginneken, A. F. Frangi, J. J. Staal, B. M. terHaar Romeny, and M. A. Viergever, Active shape model seg-mentation with optimal features, IEEE Transactions on Med-ical Imaging, 21 (2002).

[keys81] R. G. Keys, Cubic convolution interpolation for digital im-age processing, IEEE Transactions on Acoustics, Speech,and Signal Processing, 29 (1981), pp. 1153–1160.

[lipson90] P. Lipson, A. Yuille, D. O’Keefe, J. Cavenaugh, J. Taaffe, andD. Rosenthal, Deformable templates for feature extractionfrom medical images, in First European Conf. on ComputerVision (ECCV’90), 1990, pp. 477–484.

BIBLIOGRAPHY 87

[metaxes99] D. M. C. Ting, G. Herman, and S. Metaj, The combinationof deformable and probabilistic models for image segmen-tation, in Engineering in Medicine and Biology, 1999. 21stAnnual Conference and the 1999 Annual Fall Meetring of theBiomedical Engineering Society, vol. 2, 1999, p. 1143.

[quicken00] J. H. M. Quicken, C. Brechbuhler, H. Blattmann, andG. Szekely, Parameterization of closed surfaces for para-metric surface description, in Computer Vision and PatternRecognition, vol. 1, 2000, pp. 354–360.

[staib92a] L. H. Staib and J. S. Duncan, Boundary finding with para-metrically deformable models, IEEE Transactions on Pat-tern Analysis and Machine Intelligence, 14 (1992), pp. 1061–1075.

[staib92b] L. H. Staib and J. S. Duncan, Deformable fourier models forsurface finding in 3d images, in Second Conf. on Visualiza-tion in Biomedical computing (VBC’92), 1992, pp. 90–104.

[staib96] L. H. Staib and J. S. Duncan, Model-based deformable sur-face finding for medical images, IEEE Transactions on Med-ical Imaging, 15 (1996), pp. 188–199.

[styner00] M. Styner, C. Brechbuhler, g. Szckely, and G. Gerig, Para-metric estimate of intesity inhomogeneities applied to mri,IEEE Transactions on Medical Imaging, 19 (2000), pp. 153–165.

[szekely96] A. K. G. Szekely, C. Brechbuhler, and G. Gerig, Segmenta-tion of 2-d and 3-d objects from mri volume data using con-strained elastic deformations of flexible fourier contour andsufrace models, Medical Image Analysis, (1996), pp. 19–34.

[terzopoulos91] D. Terzopoulos and D. Metaxas, Dynamic 3d models withlocal and global deformations: Deformable superquadrics,IEEE Trans. on Pattern Analysis and Machine Intelligence,13 (1991), pp. 703–714.

[wang00] Y. Wang and L. H. Staib, Physical model-based non-rigid registration incorporating statistical shape informa-tion, Medical Image Analysis, 4 (2000), pp. 7–20.

88 BIBLIOGRAPHY

[wang98] , Elastic model based non-rigid registration incorporat-ing statistical shape information, in First Int. Conf. on Med-ical Image Comp. and Comp., vol. 1679, 1998, pp. 1162–1173.

[wang98b] , Boundary finding with correspondence using statis-tical shape models, in IEEE Computer Society Conferanceon Computer Vision and Pattern Recognition, 1998, pp. 338–345.

[wells96] W. M. Wells, W. E. I. Grimson, R. Kikinis, and F. A. Jolesz,Adaptive segmentation on of mri data, IEEE Trans. Med.Imageing, 15 (1996), pp. 429–443.