58
IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES Thesis submitted in partial fulfilment of the requirements for the degree of Master of Technology in Electronics and Instrumentation Engineering by Sangeeta Sahu Roll No: 212EC3379 Department of Electronics & Communication Engineering National Institute of Technology Rourkela Rourkela, Odisha-769008 May 2014

IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

  • Upload
    others

  • View
    10

  • Download
    0

Embed Size (px)

Citation preview

Page 1: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

IMAGE REGISTRATION TECHNIQUES

FOR MEDICAL IMAGES

Thesis submitted in partial fulfilment of the requirements for the degree

of

Master of Technology

in

Electronics and Instrumentation Engineering

by

Sangeeta Sahu

Roll No: 212EC3379

Department of Electronics & Communication Engineering

National Institute of Technology Rourkela

Rourkela, Odisha-769008

May 2014

Page 2: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

IMAGE REGISTRATION TECHNIQUES

FOR MEDICAL IMAGES

Thesis submitted in partial fulfilment of the requirements for the degree

of

Master of Technology

in

Electronics and Instrumentation Engineering

by

Sangeeta Sahu

Roll No: 212EC3379

Under the Supervision of

Prof. Umesh Chandra Pati

Department of Electronics & Communication Engineering

National Institute of Technology Rourkela

Rourkela, Odisha-769008

May 2014

Page 3: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

Dedicated

To

My Teachers,

Family and Friends

Page 4: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

Department of Electronics & Communication Engineering

National Institute of Technology, Rourkela

CERTIFICATE

This is to certify that the Thesis Report entitled ― “Image Registration Techniques For

Medical Images” submitted by Miss. SANGEETA SAHU bearing roll no. 212EC3379 in

partial fulfilment of the requirements for the award of Master of Technology in Electronics

and Communication Engineering with specialization in “Electronics and Instrumentation

Engineering” during session 2012-2014 at National Institute of Technology, Rourkela is

authentic work carried out by him under my supervision and guidance.

To the best of my knowledge, the matter embodied in the thesis has not been submitted to any

other University / Institute for the award of any Degree or Diploma.

Prof. Umesh Chandra Pati

Place: Associate Professor

Date: Dept. of Electronics and Comm. Engineering

National Institute of Technology

Rourkela-769008

Page 5: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

v

ACKNOWLEDGEMENT

I would sincerely like to acknowledge the support of my guide Prof. Umesh C.Pati, whose

guidance and support helped me at every step of the research done as well as thesis writing.

He always made himself available for all our doubts and also took every possible care

regarding the facilities in Virtual and Intelligent Instrumentation Laboratory. He encouraged

me all the time and kept me motivated and inspired. He helped me to overcome all hurdles in

my work. It is always my pleasure to interact with him on any topic.

I am grateful to Mr. Sankat and Subhranshu, for their support, guidance and innovative

suggestions. I would also like to thank Achala Madam for her support.

All my lab mates Sandeep, Shishir, Goverdhan and Ranjan made this task more interesting by

discussing various topics and suggesting new ideas. Also, I would thank all my mates of the

specialization of Electronics and Communication Engineering for their constant support and

encouragement.

Most importantly, I would thank my parents and my siblings for always being on my side and

supporting me with their blessings.

DATE: ROLL NO: 212EC3379

PLACE: Dept. of ECE

NIT, Rourkela

Page 6: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

vi

ABSTRACT

Image registration is a primary step in many real time image processing applications.

Registration of images is the bringing of two or more images into a single coordinate system

for its subsequent analysis. It is sometimes called image alignment. It is widely used in

remote sensing, medical imaging, target recognition using multi-sensor fusion, monitoring of

usage of a particular land using satellite images, images alignment obtained from different

medical modalities for diagnosis of diseases. It is an important step in the field of image

fusion and image mosaicing.

In this research work, approaches for image registration are proposed. The image registration

methods can be grouped into two classes. One is intensity based method which is based on

gray values of the pair of images and the second one is based on image feature which is done

by obtaining some features or landmarks in the images like points, lines or surfaces. Edges in

the images can be detected very easily in the images. Thus, using these edges some features

can be obtained by which we can accomplish feature based registration. But, feature based

registration has some limitations as well as advantages. The proposed method employs

feature based registration technique to obtain a coarsely registered image which can be given

as input to intensity based registration technique to get a fine registration result. It helps to

reduce the limitations of intensity based technique. i.e. it takes less time for registration. To

achieve this task, the mutual information is selected as similarity parameter.

Mutual information (MI) is used widely as a similarity measure for registration. In order to

improve the robustness of this similarity measure, spatial information is combined with

normalized mutual information(NMI). MI is multiplied with a gradient term to integrate

spatial information to mutual information and this is taken as similarity measure. The

registration function is less affected if sampling resolution is low. It contains correct global

maxima which are sometimes not found in case of mutual information. For optimization

purpose, Fast Convergence Particle Swarm Optimization technique (FCPSO) is used. In this

optimization method, the diversity of position of single particle is balanced by adding a new

variable, particle mean dimension (pmd) of all particles to the existing position and velocity

equation. It reduces the convergence time by reducing the number of iterations for

optimization.

Page 7: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

vii

TABLE OF CONTENTS

CONTENTS PAGE NO.

Acknowledgement……………………………………………………………………………v

Abstract………………...…………………………………………………………….………vi

List of Figures…………………………………………………………………………..........ix

List of Tables…………………………………………………………………………..…….xi

List of Abbreviations……………………………………………...……………………..…xii

1. Introduction……………………………………………………………………………....2

1.1. Overview………………………………………………………………….……....2

1.2. Image Matching Techniques……………..………………………………...….….3

1.3. Motivation…………………………………………………….…………………..3

1.4. Objective….…………………………………………………….…………….…..4

1.5. Thesis organization…………………………………………………..…………..5

2. Literature Survey………………………………………………………………………...7

3. Registration of Medical images using Contour information and Mutual

information.......................................................................................................................10

3.1. Introduction……………………………………………………………….…….10

3.2. Flow Chart………………………………………………………………….…...10

3.3. Course Registration……………………………………...…………………...….12

3.4. Fine Registration……………………………………………………………..….15

3.5. Results and Discussion…………………………………………………..………15

3.6. Conclusion………………………………………………………………….…....23

4. Intensity based rigid registration of medical images…………………………………25

4.1. Introduction…………………………………………………………………..….25

4.2. Flowchart……………………………………………………………………..….26

4.3. Method…………………………………………………………………….….…27

4.3.1. Mutual Information…………………………………………………….......27

4.3.2. Including gradient information………………………………….……........28

4.3.3. Optimization..................................................................................................29

4.3.4. Fast Convergence Particle Form Optimization..............................…...........31

4.4. Results and Discussion.…….………………………………………………..…..33

4.5. Conclusion…………………………………………………………………….....38

Page 8: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

viii

5. Conclusion…………………………………………………………………………….....40

5.1. Conclusion…………………………………………………………………..…...41

5.2. Suggestions for Future Work………………………………..……………....…..42

References………….…………………..……………………………………………………43

Dissemination…......................................................................................................................46

Page 9: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

ix

LIST OF FIGURES

FIGURES PAGE NO. 3.1. Flow chart for image registration using contour information and mutual information….11

3.2. Registration between CT Images for up-scaling………………………………………...16

3.2(a) Reference image…………………………………………………………….....16

3.2(b) Float image…………………………………………………………………....16

3.3. Coloured images…………………………………………………………………………17

3.3(a) Reference Image(red)……………………………………………...…………..17

3.3(b) Float Image(green)………………………………………………...…………..17

3.4. Edges of Images………………….……………………………………………………...17

3.4(a) Edges of reference image............................................................................…...17

3.4(b) Edges of input image………....…………………...…………………………..17

3.5. Contour of images……………………………………………………………………….18

3.5(a) Contours of reference image…………………...……………………………...18

3.5(b) Contours of input image…………………………...………………………….18

3.6 Rotated input image………………………………………..………...…………………..18

3.7. Boundary of images……………………………………………………………………..19

3.7(a) Reference image……………………………………………………………….19

3.7(b) Rotated input image…….…………………………………..…………………19

3.8 Scaled input image…………………………………………………………...................20

3.9 Coarse registration result after translation of scaled image……………………...………20

3.10 Input Images for fine registration……………………………………………………….21

3.10(a) Reference Image……………………………………………………………..21

3.10(b) Input image after coarse registration………………………………………...21

3.11 Final registered image…………………………………………………………………..21

3.12. Set of two CT Images.……………………………………………………………….…22

3.12(a) Reference Image….……………………………………………………….…22

3.12(b) Float Image……….……………………………………………………….…22

3.13. Registration Results…………………………………………………………………….22

3.13(a) Coarse Registration….………………………………………….……………22

3.13(b) Fine Registration…….………………………………………….……………22

4.1. Flow chart of the method………………………………………………………………..26

Page 10: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

x

4.2. Particle mean dimension for n particles……….……………….………………………..31

4.3. CT images……………………………………………………………………………….33

4.3(a) Reference Image………....…………………………………………………....33

4.3(b). Float Image………..…...……………………………………………………..33

4.4. Coloured CT images……………………………………………………………………..34

4.4(a) Reference image(red)……………….…………………………………………34

4.4(b). Float image(green).………………………...….……………………………...34

4.5. Registered image…………….……………………...…………………………………...34

4.6. MRI images……………………………………………………………………………...35

4.6(a). Reference Image…...………………………………………………………....35

4.6(b). Float Image….………………………………………………………………..35

4.7. Coloured MRI images…………………………………………………………………...35

4.7(a). Reference image(red)….……………………………………………………...35

4.7(b). Float image(green)…….………………….………………………………......35

4.8. Registered image……………………………….……………………………..................36

4.9. CT and MRI image……………………………………………………………………....36

4.9(a). Reference Image….………………………………………………………......36

4.9(b). Float Image…………….…………………………………………………......36

4.10. Coloured CT and MRI images…………………………………………………………37

4.10(a). Reference image(red)….……………………………………………….……37

4.10(b). Float image(green)…….…………….………………………………….…...37

4.11. Registered image….……………………………………………………………….…...37

Page 11: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

xi

LIST OF TABLES

TABLES PAGE NO.

I. Coarse registration parameters…………………………………………………...23

II. Fine registration parameters……………………………………………………...23

III. Parameters with standard PSO…………………………………….……………..37

IV. Parameters with FCPSO…………………………….…………………………...37

Page 12: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

xii

LIST OF ABBREVIATIONS

MI : Mutual Information

NMI : Normalized mutual information

ECC : Entropy Correlation Coefficient

PSO : Particle Swarm Optimization

FCPSO : Fast Convergence Particle Swarm Optimization

Page 13: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

1

CHAPTER 1

INTRODUCTION

Contents

Overview

Image Matching Technique

Motivation

Objective

Thesis organization

Page 14: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

2

INTRODUCTION

Registration of images is the bringing of the images into a single coordinate system for its

further analysis. It is also known as image alignment i.e. the two different images in different

local coordinates have to be aligned in a single coordinate. One of the images is chosen as

reference image and another image or the float image will be registered according to the

coordinates of reference image.

1.1. Overview

In image processing, when we try to combine the details of the images, we are actually search

for the relation between two or more images. The study of this relationship usually becomes

manageable once a resemblance is set up between the images. The task of setting up this

correspondence is known as registration of images.

Image registration is the task of matching two or more partially overlapping images taken at

different time instants or from different observation points. It is a basic image processing

technique which is very important step to integrate information from various sensors. It helps

to find changes in images which are taken at different time instant. It helps to deduce three-

dimensional information from stereo images and to identify model-based objects.

The systems in which image registration is a important constituent includes matching a target

with a real-time image of a scene for target recognition, land utilization observation using

satellite images, stereo image matching to get shape for autonomous navigation, and

alignment of multimodality images for identification of diseases. Image registration is an

initial footstep for various applications such as remote sensing and multi-sensor fusion based

target recognition. It is required earlier to image fusion or image mosaic.

Registration is done manually as well as automatically[1]. In manual registration, images

which are to be registered is taken and human operators manually select corresponding

features. For accurate registration results, operators have to choose a plenty of pairs of feature

over the full images, which is laborious as well as subjected to irregularity and bounded

accuracy. Therefore, it is necessary to find automated techniques which need less or no

operator supervision.

Page 15: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

3

1.2. Image Matching Techniques

Image matching techniques can be categorized as gray-scale based matching or image feature

based matching.

Gray scale-based matching: Gray scale-based matching examines images as two

dimensional signals and make use of statistic approaches to find the correlation functions

among signals [2-4], and then obtain their resembling and homonymy points. Gray-scale

techniques give good results but take more time as compared to feature based techniques.

Feature-based matching: Feature-based matching finds some features within the images

such as points, lines, surfaces and planes [5][6], and then defines properties of those features

and then matches the images according to these characteristics. The similar features which

are used are textures, shapes and spatial positions. It involves only partial pixels and, thus,

reduces matching computation. It gives good registration accuracy because of location

sensitivity of the matching properties. The feature extraction lessens the noise effects and

increases the compliance with changes in intensity values, morphing and occlusion. The

feature based techniques gives less accurate results as compared to intensity based technique.

1.3. Motivation

Edges in the images can be detected very easily in the images. Thus, using these edges some

features can be obtained by which we can accomplish feature based registration. But feature

based registration has some limitations like low accuracy as well as advantages like less time

consuming. Thus, a method was to be searched which employs feature based registration

technique to obtain a coarsely registered image which can be given as input to intensity based

registration technique to get a fine registration result. The problem with intensity based

registration is that it is time consuming. So, the method was required which helps to reduce

the limitations of intensity based technique. i.e. it takes less time for registration.

As mutual information may sometimes leads to incorrect registration in intensity based

technique, normalized mutual information is taken as search parameter in the next technique

as this measure is does not vary with the overlapped area between the images. To make the

measure more informative and to make the measure more robust, spatial information is

included in the similarity measure to incorporate the intensity information of neighboring

pixels. Particle Swarm Optimization (PSO) is a simple and computationally efficient

Page 16: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

4

optimization method. Many modifications have been suggested to the standard particle

swarm optimization to find good and faster solutions than the evolutionary algorithms, but

those modifications may got stucked in poor region or result in divergence to unstable

situations. Thus, modification in the optimization technique was required to avoid this

problem and make the optimization process fast.

1.4. Objective

The objective of this work is to develop efficient method of registration of the medical

images which uses images‟ contour information as well as mutual information i.e. a

registration method which is a combination of feature based and intensity based method.

The next objective is to develop an effective intensity based registration technique which has

sufficient information in the similarity measure and also to reduce convergence time in case

of global optimization of this measure.

1.5. Thesis Organization

Including this introductory chapter, the thesis is divided into six chapters:

Chapter 2: Literature Review

In this chapter, we discuss about the literature survey carried out related to the work.

Chapter 3: Registration of medical images using contour information as well as mutual

information

In this chapter, registration of medical images is discussed. The images are rotated, scaled

and translated with respect to each other. The algorithm uses combination of feature based

and intensity based technique. It discusses the steps involved in the algorithm. Also the

results and discussions have been given in this section.

Chapter 4: Intensity based rigid registration of medical images

This chapter describes a method to register images using modified normalized mutual

information and Fast Convergence PSO optimization. Moreover, it consists of results and

discussions.

Page 17: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

5

Chapter 5: Conclusion

This chapter is concluded with the important points of the research work. Furthermore, some

suggestions for future work are given.

Page 18: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

6

CHAPTER 2

LITERATURE SURVEY

Page 19: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

7

LITERATURE SURVEY

A lot of work has been done in the field of image registration. The following is a brief

introduction of some of the papers.

B.Zitova and J.Flusser presents a basic overview of image registration methods[1].

R.Suganya, Dr.S.Rajaram and K.Priyadharsini used centre of gravity the images for initial

registration of images and final registration is accomplished by maximizing the mutual

information[2]. L.Junli, C.Rijuan, J.Linpeng and W.Ping proposes a weighted mutual

information (WMI) for registration of medical images by which doctors can weightage to the

image according to which registration has to be done[4]. I.Misra, S.M.Moorthi, D.Dhar and

R. Ramakrishnan proposes an automatic registration method for remotely sensed

multispectral images. The method works even if float and reference images are from different

sensors[5]. C.S.Qiao uses an image matching technique based on feature extractor such as

Harris Operator and proposed a new corner point matching method based on the singular

value decomposition[6]. J.Hu, Y.Yang and Z.Su proposes a rapid registration method of the

medical images done by using multi-scale transform and contour line[7]. N.A. Al-Azzawi,

H.A.M. Sakim, W.A.K.W. Abdullah describes the standard PSO algorithm which is used in

the paper to optimize MI. PSO is a global optimization algorithm[8]. F. Maes, A. Collignon,

D. Vandermeulen, G. Marchal, and P. Suetens proposes mutual information of the images as

the matching parameter and used MI to measure the statistical dependency in the images or

redundant information in the image gray values of the corresponding pixels in couple

images[10]. L. Ding, A. Goshtasby, and M. Satter proposes a method of registration by

selecting templates i.e. subimages from an image and locating the same template in another

image of the same view and selecting the centroids of the templates as the control points [11].

J.P.W. Pluim, J.B.A. Maintz and M.A. Viergever gives a survey of various methods

involving mutual information and the various classification of the image registration

methods. They explained various advantages and disadvantages of taking mutual information

as a measure and explains various methods which eliminates the drawback of MI [12]. C.

Studholme, D.L.G. Hill and D.J. Hawkes proposes a normalized measure which is invariant

to overlapping areas between the images. It is the ratio of the addition of the individual

Page 20: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

8

entropies and the joint entropy[13]. C.Studholme, D.L.G.Hill and D.J.Hawkes proposes a

method of registration by computing mutual information of a pair of images by labelling one

of the images i.e. labelling of connected region within the images which helps to maintain or

increase the measure [15]. J. P.W. Pluim, J.B.A. Maintz and M.A. Viergever describes how

gradient information can be included in a similarity measure to increase the information

content in the similarity measure[16]. J. Xie , Z. Chen, G. Xu proposes a method to find

feature points by wavelet multiscale product and mutual information is used to register

feature points and used particle swarm optimization technique to obtain registration

parameters[17]. A. Sahu, S. K. Panigrahi, S. Pattnaik proposes a Fast Convergence Particle

Swarm Optimization (FCPSO) method which balances the diversity in the position of a single

particle by defining a new variable which is the average of the locations of all dimensions of

every particle. It improves the functioning of PSO [18].

Page 21: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

9

CHAPTER 3

REGISTRATION OF MEDICAL IMAGES USING

CONTOUR INFORMATION AS WELL AS

MUTUAL INFORMATION

Contents:

Introduction

Flow Chart

Course Registration

Fine Registration

Results and Discussion

Conclusion

Page 22: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

10

REGISTRATION OF MEDICAL IMAGES USING

CONTOUR INFORMATION AS WELL

AS MUTUAL INFORMATION

3.1. Introduction

The medical image provides the various details of the patient. It helps a doctor in

identification of any disease developed in the patient. In the applications of the medical

images, the medical image registration is an important technique because it helps a doctor to

observe the development of the disease during some time duration and it also helps a doctor

to take an accurate and proper treatment scheme about the disease.

Image registration techniques can be based on intensities or gray-scale of images and features

selected in the images whichever is used in registration. Feature based registration results in

coarse registration. By using some intensity based method, we can get a fine or accurate

registration result. Thus, a method of registration of the medical images based on contour

information as well as mutual information of images is proposed.

Feature based registration gives a coarse result due to involvement of partial pixels. Thus,

using some intensity based method, a fine or accurate registration can be obtained [7]. This is

a registration method for medical images based on contour lines of images and mutual

information. First of all, a coarse registration is obtained using image contour lines obtained

from canny detector and then fine registration is accomplished using mutual information

maximization.

3.2. Flow Chart

Fig 3.1 shows the flow chart for image registration using contour information and mutual

information.

Page 23: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

11

Slave (input image) Target (reference image)

Edges detection

Contour detection

Included angles

Registered Image

Translation of centres

Course registration result

Maximization of MI

Slope of principal axis

Centre detection

Fig 3.1 Flow chart for image registration.

Page 24: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

12

3.3. Coarse Registration

Coarse registration of image is accomplised by using feature based registration method.

A. Contour Line Extraction

The medical images like brain are rigid in nature, and their edge characteristics are very clear.

So extraction of contour of images is easy. Brain images are taken as an example of medical

images. Edges of an image are regions with strong intensity contrasts. Detection of edges

decreases the quantity of data significantly and removes useless information, and preserves

the important structural qualities of an image.

Edge Detection:

Contours of medical images to be registered can be easily extracted by the use of Canny

operator. Canny operator, which is known as optimal edge detector, extracts images‟ edge

distinctly and precisely even in the noisy environment. It provides thin edges.

Contour Extraction:

Now, contours of images can be obtained from the edge images which are obtained by using

Canny operator by using line by line scanning method. Every row of the edge images are

scanned from first to last pixel and only first and last non-zero pixel will be selected. This

gives us the contour information of the rows. Similarly for contour information of the

columns, every column of the edge images are scanned from first to last pixel, and only first

and last non- zero pixel will be selected. Thus, the contour information of a medical image is

obtained.

B. Coarse Registration based on contour information of images

Rotation Correction:

Let the pixel coordinates of an images‟ contour line be {( ,i ix y ) i =1, 2,…n}, where n

denotes total number of pixels of the contour line. Thus, the centre coordinate of an image

can be calculated as given in equation (3.1),

Page 25: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

13

1

1

1

1

n

i

i

n

i

i

x xn

y yn

(3.1)

where,

( x , y ) represent the centre coordinates of the images i.e.

( rx , ry ) represent the centre coordinate of the reference image and

( fx ,fy ) represent the centre coordinate of the float image.

Rotation angle can be obtained by finding the principal axes of the couple medical images.

The inertia matrix of contour line of images can be set as given in equation (3.2),

11 12

21 22

u uI

u u

(3.2)

where,

2

11

1

( )N

i

i

u x x

2

22

1

( )N

i

i

u y y

12

1

( )( )N

i i

i

u x x y y

Based on the inertia matrix calculated by using (2), long and short axes of the images can be

found because these axes are actually the two eigenvectors of the inertia matrices. Now, the

included angles between the two long axes and the two short axes can be found respectively a

as given by equation (3.3),

1

1

1

2

)  ( (1,2)* (1,2) (2,2)* (2,2)

( (1,1)* (1,1) (2,1)* (2,1))

r f r f

r f r f

Cos V V V V

Cos V V V V

(3.3)

Page 26: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

14

where rV represents eigen vectors of the reference image inertia matrix and fV

represents

eigen vectors of the float image inertia matrix. The initial rotation angle is obtained by the

averaging the two included angles as given in equation (3.4).

Based on the initial rotation angle, float image can be rotated.

Scaling Correction:

Again the inertia matrix for the rotated float image is calculated by repeating the above

procedure. Let 'fV be eigen vectors for rotated float image. Then, the slope of the eigen

vectors of newly calculated inertia matrix of rotated float image and eigen vectors of

reference image is calculated by equation (3.5),

1 2,1 / 1,1

2 2,2 / 1,2

m V V

m V V

(3.5)

Principal axes can be plotted for both images as principal axis passes through centre

coordinate ( x , y ) and slope of the axes have been obtained. Shift the principal axes in both

directions, till this axes just touches the object boundary and thus, boundaries of the object is

drawn by the rectangle enclosing the images. By calculating the difference between the

values of those shifts, width and height of the object is obtained. The ratio of width of

reference to rotated float image gives the scaling factor in y-direction and ratio of height of

reference to rotated float image gives the scaling in x-direction. The rotated float image will

be scaled by the obtained scaling factor. Now, scaled image is same of same size as the

reference image.

Translation Correction:

Again the centre coordinates '' ''( , )f fx y of scaled float image can be calculated. Now, the

translation parameters between the images is calculated as given by equation (3.6),

''

''

f r

f r

x x x

y y y

(3.6)

Page 27: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

15

According to this translation value, the coarse registration of the pair of medical images has

been done.

3.4. Fine Registration

Mutual information (MI) is very popular similarity parameter which is based on Shannon

entropy and is widely used in the medical imaging domain. The mutual information is a

statistical measure of the mutual dependence of the two images i.e. it represents the statistics

correlation of two sets of image data. Registration is assumed to be done when mutual

information is maximum i.e. the images should be aligned such that the quantity of

information they have about each other is maximum. The mutual information of two images

A and B is calculated as given in equation (3.7).

I (A, B) = H (A) + H (B) – H (AB) (3.7)

where, ( ) ( ) log ( )A AH A P a P a

H(A) denotes the entropy of image A and the joint entropy H(AB) can be calculated as,

, ,( ) ( , ) log ( , )A B A BH AB P a b P a b .

Larger the value of the mutual information obtained in the registration technique, the more

precise registration results are obtained.

So, varying the input image over a range of angles, a set of translation is checked for rotated

image with respect to input image and the value of translation and rotation for the maximum

value of mutual information is noted. Now, the resultant image of coarse registration is rotated

and translated by that amount to get fine registration result.

3.5. Results and Discussions

The reference and slave image consists of 256 intensity values i.e. 8 bits gray-scale medical

images. Example 1 shows the registration of set of two CT images for up-scaling. Example 2

shows the registration of the set of two CT images for downscaling.

Page 28: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

16

Fig. 3.2 represents the set of two CT images taken for registration. Fig. 3.3 represents the same

images which are coloured just to recognize. Fig. 3.4 represents the edge images. Fig. 3.5

represents the contours of images obtained from edge images. Fig.3.6 shows the rotated input

image. Fig. 3.7 represents the boundary of images. Fig.3.8 shows scaled input image. Fig. 3.9

represents coarse registration result after translation of scaled image. Fig. 3.10 represents the

images after coarse registration. Fig. 3.11 shows the final registered image.

Fig. 3.12 represents the two brain images taken for registration. Fig. 3.13 shows the

registration results.

Example 1:

Registration between CT Images for upscaling

(a) (b)

Fig. 3.2. (a) Reference Image (b) Float Image

Page 29: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

17

(a) (b)

Fig. 3.3.(a) Reference Image coloured as red (b) Float Image coloured as green

(a) (b)

Fig. 3.4 (a)Reference Image (b) Float Image

Page 30: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

18

(a) (b)

Fig. 3.5. (a) Contour of reference Image (b) Contour of float Image

Fig.3.6. Rotated input image

Page 31: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

19

(a) (b)

Fig. 3.7. (a) Reference Image (b) Float Image

Page 32: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

20

Fig.3.8. Scaled input image

Fig.3.9. Coarse registration result after translation of scaled image

Page 33: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

21

(a) (b)

Fig. 3.10 (a) Reference Image (b) Input image after coarse registration

Fig.3.11. Final registered image

Page 34: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

22

Example 2:

Registration between CT Images for downscaling

(a) (b)

Fig. 3.12 (a) Reference Image (b) Float Image

(a) (b)

Fig. 3.13 (a) Coarse Registration (b) Fine Registration

Page 35: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

23

The registration parameters obtained in the two examples of registration are given in Table I

and Table II.

Table I. COARSE REGISTRATION PARAMETERS

Example Rotation

(in degrees)

Scaling Translation

Sx Sy x Y

1 18.78 1.31 1.31 145 249

2 19.34 0.693 0.697 12 34

Table II. FINE REGISTRATION PARAMETERS

Example Rotation

(in degrees)

Translation Maximum MI

x y

1 -4 12 11 1.7

2 -2 14 26 1.14

This method is a combination the feature and intensity information of the images. So, it

involves less mathematical complexity. In feature-based method, partial pixels are selected

and thus, calculations are reduced and in intensity-based method, mutual information is to be

calculated over a small range of angles as the course registration gives us an approximate

result.

3.6. Conclusion

A method of registration of the medical images based on contour information and mutual

information of the images is proposed. The feature as well as intensity information of the

images are used effectively in the proposed work. The result shows that the proposed

approach involves less complexity and is an effective medical image registration method.

Page 36: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

24

CHAPTER 4

INTENSITY BASED RIGID REGISTRATION

OF MEDICAL IMAGES

Contents

Introduction

Flowchart

Method

Mutual information

Including gradient information

Optimization

Fast Convergence Particle Form Optimization

Results and Discussions

Conclusion

Page 37: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

25

INTENSITY BASED RIGID REGISTRATION

OF MEDICAL IMAGES

4.1. Introduction

Mutual information is an gray-scale based similarity parameter used in case of both

monomodal as well as multimodal images[8][9][10][11]. It does not require any features such

as points or surfaces as feature based registration technique which leads to coarse registration.

In spite of the promising results given by mutual information, sometimes it results in

misregistration of images i.e. it fails occasionally. This occurs when the resolution of the

images is low, when little information is present in the images or when there is less overlapping

region between the images. The mutual information measure is influenced by size of the

overlapping part of the images in two ways[12]. Decreasing the overlap area decreases number

of samples. If number of samples reduces, it reduces the statistical power of probability

distribution estimation. Futhermore, if misregistration rises, the mutual information measure

can become high because increasing misregistration matches with reducing overlap. This

happens when the corresponding area of object and background becomes equal and the sum of

the marginal entropies increases rapidly in comparison to joint entropy. A normalized measure

of mutual information is proposed by Studholme [13]. Normalised mutual information(NMI) is

less affected by changes in overlap and is expressed by equation (4.1).

Improved results are obtained if normalized measure is taken for rigid registration of

multimodal images[13].

In mutual information found from Shannon entropy[14], the dependence of the intensity values

of the adjacent pixels is completely neglected. But, the original Shannon entropy definition

includes the dependence of prior signals. However, the definition of Shannon entropy used in

applications is for independent consecutive signals. This idea of independence of signals does

not apply in case of medical images. The dependence of the intensity values of adjacent pixels

is actually termed as spatial information of the images. Thus, including the spatial information

Page 38: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

26

with mutual information[15] can improve registration results. So, to integrate spatial

information, mutual information can be combined with parameter obtained from the gradients

at corresponding points. This parameter aligns gradient vectors of large magnitude and of same

orientation[16].

Particle Swarm Optimization(PSO) is a global optimization technique which is widely

used[17]. But, it takes large time to converge. So, to reduce the number of iterations, Fast

Convergence Particle Swarm Optimization[19] is used in this method.

4.2. Flowchart

Flow chart of the method is given in figure 4.1.

No

Yes

Fig.4.1. Flowchart of method

Start

Random starting locations and velocities

Evaluate fitness function

Fitness output is compared to obtain Pbest

Fitness output is compared to obtain Gbest

k 1 1. 1. –

2. 2. – 3. 3. –

ij ij ij ij

i ij

v wv c rand pbest x

c rand gbestj xij c rand Pmd

k k k

k k k kx

and 1 1ij ij ijx t x t v t

If Gbest remains

constant for 15

iterations

Stop

Page 39: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

27

4.3. Methodology

The method used is the combination of the combined measure of normalized mutual

information as well as the gradient information and the technique used for optimization

purpose.

A. Mutual information

Mutual information of the two images is the addition of the individual and joint entropies of

the two images. Entropy is, how much the probability distribution of the images disperses.

When a probability distribution has acute and dominant peaks, entropy will be minimum.

When every outcome has equal chances of occurring i.e. for a uniform distribution, entropy

will be maximum. By arranging the probability distribution of the images intensity values,

the entropy of the images is calculated. The Shannon entropy for probability distribution is

defined as given in equation (4.2),

,

( , j) logp(i, j)i j

H p i (4.2)

i.e. H(A) denotes the individual entropy of image A and

H(B) denotes the individual entropy of image B and are calculated as given by equation (4.3)

and (4.4),

(A) (a) log (a)A AH P P (4.3)

(B) (b) log (b)B BH P P (4.4)

H(A,B) is the joint entropy of image A and image B, i.e. the entropy of the joint probability

distribution of the intensities of image A and image B.

By computing a normalized joint histogram of the intensities of the two images, joint

probability distribution is estimated. It is expressed as given in equation (4.5).

, ,( , ) ( , ) log ( , )A B A BH A B P a b P a b (4.5)

The mutual information I(A,B) of two images A and B intermingles the individual and joint

entropies of the images and is given by equation (4.6).

Page 40: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

28

I(A,B) = H(A) + H(B) − H(A,B) (4.6)

Images are correctly registered when MI of images is maximum. This suggests, there should

be a balance between joint entropy minimization and marginal entropies maximization. The

joint entropy will be minimized if the joint distribution is minimum dispersed, i.e. when a

distribution has less number of acute and dominant peaks. This coincides with registration. In

case of improper alignment of the images, new pairs of grey values will be introduced which

decreases the probabilities of the „correct‟ combinations resulting in more dispersed joint

probability distribution. But, the mutual information measure is affected by overlapping areas

between the images and normalized MI can overcome this problem[12] and the entropy

correlation coefficient(ECC) is different form of normalised MI given in equation (4.7),

B. Integrating gradient information

Image areas with strong intensity contrasts are the areas of high information value as it

denotes transition of tissues. The gradient is computed on spatial domain. Normalised mutual

information is modified to integrate spatial information existing in the images. i.e. normalised

mutual information is multiplied with a gradient term. This term is based on both the

magnitude and the orientation of the gradient vectors[16]. Directly calculating normalized

mutual information of gradient images can also be done to incorporate spatial information.

But, it can result in narrow attraction range of registration function and a lot of information of

the intensity values is rejected. Thus, a combination of normalised mutual information i.e.

Entropy Correlation Coefficient and spatial information is used. The gradient vector is

calculated for every sample point „a‟ in one image and the corresponding point „b‟ in another

image, which is obtained by geometric transformation of „a‟. The gradient vector is obtained

by calculating two partial derivatives in both x and y direction. To find the gradient vector,

the image is convolved with the first derivatives of a Gaussian kernel of scale σ. σ can be

taken in range of 0.5 to 1. The angle , ( )a b within the gradient vectors can be calculated as

given by equation (4.8),

Page 41: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

29

,

a( ). b( )( ) arccos

( ) b( )a b

a

(4.8)

with ∇a(σ) denotes the gradient vector at point a of scale σ and mode denotes magnitude.

In different modality images, the various tissues have different intensities. So, the gradient of

the images points in dissimilar directions. However, since the different modality images

represent the identical anatomical structures, gradients of the two multimodal images will

have the same orientation; either it can be in same or reverse directions. Weighting function

w is used so as to adjust both very small angles as well as large angles that nearly equals 180

and is given by equation (4.9).

,

,2 ( ) 1 ( )

2

b

a b

acosw

(4.9)

But, because of difference in imaging processes of different modalities, it is not essential that

different modality images represent the same transitions of tissues. Therefore, strong

gradients in a certain modality may not be present or less significant in another modality.

But, we have to include strong gradients of the both images. Thus, minimum of the gradient

magnitudes is multiplied by the angle function. Gradient term is the summation of the

resulting product for all samples which is multiplied by the normalised mutual information

measure. Tissue transitions in both modalities are emphasized. Gradient term may be

mathematically expressed as given by equation (4.10),

(a

,

, ) ( )b

, * (| | | ), |a b

A B

G A B w min a b

(4.10)

The combined measure of normalized MI and spatial information is given by equation (4.11).

ECCnew (A, B) = G(A, B)*ECC (A, B) (4.11)

C. Optimization

Particle Swarm Optimization[18] is a population-based search technique. Particle Swarm

Optimization is both simple and effective. Particles represent a population of possible

solutions. Particles are depicted by a position and a velocity vector both. The location of each

particle represents a solution. The main logic is that individuals or particles gain experience

from the member or particles at the best position to reach the group objective or to reach the

Page 42: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

30

location where there is maximum availability of food. As the population moves towards its

objective, each individual adjusts its position according to its own and the adjacent particles

experiences. A fitness function is used to search for the best position. The fitness function

must be defined by the parameters to be optimized. Every time the loop repeats in simulation,

the fitness function can be computed by taking the location of the particles in the search

space. Every particle stores the best value found by it so far. The location of the highest

fitness value of each particle is called personal best or local best (pbest). The location of the

highest fitness value among the particle swarm i.e. among all particles is called global best

(gbest). In each iteration, there is exactly one gbest and all the particles are pulled in the

direction of gbest location. After finding pbest and gbest values, the particle modify the

velocity and position as given by two equations (4.12) and (4.13).

1 * ( ) 1* 1* ( ) 2*( 2* ( )) ( )ij ij ij ij j ijv k w v k c rand pbest x k c r gbest x k (4.12)

and

1 ( ) 1ij ij ijx k x k v k (4.13)

where,

  1ijv k is the particle‟s velocity i.e. velocity of ith particle at (k + 1)th iteration,

( )ix k is the solution of current particle i.e. position of ith particle,

ipbest is the previous best position of the ith particle,

gbest is the global best position achieved by particle swarm till then,

rand1 and rand2 are random numbers between 0 and 1,

c1 is the cognitive factor or individual learning rate,

c2 is the social constant,

w is the inertia weight,

k is number of iteration i.e. k = 1,2,…,

j is the particle dimension.

To control the velocity of a particle, a maximum velocity maxv is inflicted on particle's

velocity i.e. if any particle moves with a velocity that exceeds the maximum velocity maxv,

then that particle‟s velocity is reduced to maxv. Each velocity vector is compressed within the

limits [ maxv , minv ] to lessen the chances that the particle departs from the position limits.

Page 43: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

31

D. Fast Convergence-Particle Swarm Optimization

In standard PSO, starting particles are evenly dispersed in the search space. The method may

get confined in local minima and ipbest may not get changed for many steps, because of the

mutual limitation of variable of each dimensions. It is difficult for the method to run away

from the local minima, thus, the correct location will not be obtained.

When the swarm gets updated from the k generation to k + 1, along with the trace of ipbest

and gbest, the particles can trace ipmd which is obtained from the swarm. The new variable

ipmd of ith particles are calculated as given by equation (4.14).

Pmd1

Pmd2

Pmd3

PmdN

Fig.4.2. Particle mean dimension for n particles

1 2 .. / i i i iDPmd x x x D (4.14)

where,

D, Particles‟ dimensions in the swarm and the new velocity and position is represented by

equation (4.15) and (4.16) respectively,

k 1

1. 1. –

2. 2. –

3. 3. –

ij ij

ij ij

j

i ij

v wv

c rand pbest x

c rand gbe

k

k k

k kst xij

c rand Pmd k kx

(4.15)

1 1ij ij ijkx k x v k (4.16)

X11 X12 X13………………….X1n……X1D

X21 X22 X23………………….X2n……X2D

X31 X32 X33………………….X3n……X3D

XN1 XN2 XN3………………….XNn……XND

Page 44: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

32

where,

c3 is the mean best learning factor,

rand3 is the random vector between [0, 1].

Here, c1,c2 and c3 are selected satisfying the equation (4.17).

= c1 + c2 + c3 (4.17)

where, >= 4

After adding ipmd in the velocity formula, all ipbest , gbest and ipmd gives information to

the next generation combinely and thus, the information received from previous generation

increases. This method helps to get the favourable solution quickly. Also, the weightage

factor of ipmd , i.e. „c3‟ is small. Thus, this term equals disturbance information and

increases the diversity between the particles. The gbest location improves convergence rate.

However, it decreases the diversity among population which results in local minima.

Simultaneously, parameter ipmd takes the particles to a better location and reduces the

chance of attraction of particles towards local minima.

E. Formulation of FCPSO

First of all, some particles are chosen and the number of dimensions for which float image is

to be corrected is decided. The experiments are performed by taking 25 particles and 3

dimensions are to be searched i.e. rotation in one plane and translation along two axis. Search

space is the limits for the position of particles in which food is to be searched. Search space

for particles position is given as [-48 48; -48 48; -25 25] where first column shows minimum

limit and second column shows maximum limit for particles position in each dimension. The

value of [ maxv , minv ] is selected as twice the limits of position for each dimension.

Cognitive(c1) and social constant(c2) are given as 1.8 and c3 as 0.4. The particles are

allocated random positions at first. The fitness function i.e. ECCnew measure is calculated

for those particle locations. The location for the best availability of food i.e. maximum fitness

function is saved as gbest and for each particle pbest i.e. the location of best food availability

is saved as the initial particle location at first run. The value of pmd is calculated for each

generation of particle. The velocity for the particles is calculated by using equation (4.15). If

the velocity of the particle obtained is greater than maxv , then it is set to maxv and if it is less

than minv then, it is set to minv .The new locations for the particles are calculated by using

Page 45: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

33

equation (4.16). The fitness function output is then checked. If the new locations give high

value then, according to that pbest and gbest are set. The loop was repeated until gbest

remained constant for 15 iterations.

4.4. Results and Discussion

The target and slave image consist of 256 intensity values i.e. 8 bits gray-scale medical

images. The floating image has some translation and rotation with respect to reference image.

Three examples of registration are given. First two examples are for monomodal images and

third example is of multimodal registration. Example 1 gives the registration of two CT

images. Example 2 gives the registration of two MRI images. Example 3 gives the registration

of CT and MRI images.

Fig. 4.3 represents the set of two CT images taken for registration. Fig. 4.4 represents the same

images which are coloured just to recognize. Fig. 4.5 shows the registered image. Fig. 4.6

represents the set of two MRI images taken for registration. Fig. 4.7 represents the same MRI

images which are coloured just to recognize. Fig. 4.8 shows the registered image. Fig. 4.9

represents the CT and MRI images taken for registration. Fig. 4.10 represents the CT and MRI

images which are coloured just to recognize. Fig 4.11 shows the registered image.

Example 1: CT images

(a) (b)

Fig. 4.3. (a) Reference image (b) Float image

Page 46: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

34

(a) (b)

Fig. 4.4. (a) Reference image coloured as red (b) Float image coloured as green

Fig. 4.5

Page 47: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

35

Example 2: MRI images

(a) (b)

Fig. 4.6 (a) Reference Image (b) Float Image

(a) (b)

Fig. 4.7. (a) Reference Image with red (b) Float Image with green

Page 48: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

36

Fig.4.8

Example 3: CT and MRI image

(a) (b)

Fig. 4.9. (a) Reference Image (b) Float Image

Page 49: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

37

(a) (b)

Fig. 4.10. (a) Reference Image with red (b) Float Image with green

Fig. 4.11

Page 50: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

38

Parameters obtained after registration are given in Table III and IV.

Table III. PARAMETERS WITH STANDARD PARTICLE SWARM OPTIMIZATION

Eg. Maximum

ECCnew

No. of

iterations

RMSE

CT-CT 1.39e+05

1275

0.675

MRI-MRI 1.61e+05

1225

1.85

CT-MRI 1.69e+04

1200

8.32

Table IV. PARAMETERS WITH FAST CONVERGENCE PARTICLE SWARM

OPTIMIZATION

Eg. Maximum

ECCnew

No. of

iterations

RMSE

CT-CT 1.29e+05

725

0.973

MRI-MRI 1.21e+05

825

2.78

CT-MRI 1.67e+04

525

8.33

This method is based on intensity values of images. The numbers of iterations get reduced in

case of Fast Convergence Particle Swarm Optimization while the result accuracy is

approximately same. The number of iterations is reduced to almost two third in case of

FCPSO than standard PSO. Thus, time complexity reduces and also, it reduces the chance of

attraction of particles towards local minima. RMSE is high in case of multimodal images as

the same part of images is represented by different intensity values.

4.5. Conclusion

A method of registration of the multimodal medical images based on normalised mutual

information and spatial information is proposed. The intensity information of images is used

effectively in the proposed work. As normalized mutual information is taken along with spatial

information, a measure has been found which is invariant to overlapping region and is more

robust measure than normalised mutual information as spatial information adds to the

information between the images. The combined measure gives a good registration function.

This registration function is less affected by if the sampling resolution is low. There are no

Page 51: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

39

erroneous global maxima which can be obtained in case of mutual information. Also, local

minima caused by interpolations gets decreased.

The method uses a global optimization technique which is better than standard genetic

algorithms. Using Fast Convergence Particle Swarm Optimization, result can be obtained in a

lesser time than standard PSO. The proposed approach is, thus, more robust and is an

effective medical image registration method.

Page 52: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

40

CHAPTER 5

CONCLUSION

Contents:

Conclusion Suggestions for Future Work

Page 53: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

41

CONCLUSION

This chapter focuses on the advantages and limitations of all the methods used for image

registration. The scopes of future research work in this domain are also discussed.

5.1. Conclusion

Canny edge detector is the widely used edge detector as it gives thin edges. It is used in our

method for detection of the edges, which helps us to easily find the contours. Contours are

used for finding the principal axes of the medical rigid images. From principal axis

information, eigen vectors of the inertia matrix can be found. From these two eigen vectors,

the rotation angle can be found. Similarly to find scaling factor, these principal axes can be

shifted in both directions so as to enclose the medical structures within rectangles. From the

ratio of the heights and widths of the rectangle, scaling factors can be found in both

directions. Similarly, from the translation of the centres, translation between the images can

be found. This method takes very less time for computation and is time efficient. But, the

problem is that it does not give accurate result. The MI based fine registration overcame this

problem since it gives accurate result. As the input images to the MI based technique is

already coarsely registered, it takes very less time for computation and gives finely registered

images. This method is efficient and cheap but it can„t handle the images which are not rigid

structures. It works for rigid structures like brain. Also the images should have properly

defined principal axes to find out the scaling factor. Thus, a feature based registration gives a

course result as seen in chapter 3. Thus applying any intensity based can help us to increase

the accuracy of the method. At the same time, doing so would eliminate drawback of

intensity based technique (it takes large time to register the images) by reducing the search

space. Thus images‟ features and intensity information both are useful to find effective

registration techniques for medical images. The proposed approach involves less complexity

and is an effective medical image registration method.

The intensity information of images is used effectively in chapter 4. Mutual information

between the images is widely used as similarity measures in many intensity based registration

technique. But if the overlapping region between the images is less then MI may give an

incorrect measure. The normalised mutual information overcame this problem since it is

Page 54: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

42

invariant to changes in overlapping region of the two images. The measure is made more

robust by including the spatial information by multiplying the gradient term with the

normalised measure. The combined measure helps in the correct object identification with

low probability of mismatch. For optimization, global optimization methods as better as

compared to local optimization methods like powell, simplex, gradient, etc as it does not

converge to into a local minima or maxima. And the method uses a global optimization

technique which is better than standard genetic algorithms. But, standard PSO takes large

time to converge. Number of iterations gets reduced in case of Fast Convergence PSO. The

number of iterations is reduced to almost two third in case of FCPSO than standard PSO. The

proposed approach is more robust, accurate and is an effective medical image registration

technique.

5.2. Suggestions for Future Work

A new method for feature-based registration can be searched for an overall increase in

registration performance. Different image similarity metrics can be used for refinement of

automatic image registration techniques for fine registration. Also, some more techniques can

be searched to further reduce the times of repetition in optimization technique of the

similarity measure. The input images used for the proposed method are the planar 2-D

images; it can be applied for the 3-D images as well.

Page 55: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

43

REFERENCES

[1] B.Zitova and J.Flusser ”Image registration methods: a survey,” Image and Vision

Computing, Vol 21, pp. 977–1000, 2003.

[2] R.Suganya, Dr.S.Rajaram and K.Priyadharsini “Intensity Based Image Registration by

Maximization of Mutual Information,” Int. Jrnl. of Computer Apps., Vol 1, No. 20, pp.

0975 – 8887, 2010.

[3] C. Studholme, D.L.G. Hill and D.J. Hawkes, “Automated three-dimensional registration

of magnetic resonance and positron emission tomography brain images by

multiresolution optimization of voxel similarity measures,” Medical Physics, Vol. 24,

No. 1, pp. 25–35, 1997.

[4] L.Junli, C.Rijuan, J.Linpeng and W.Ping “A Medical Image Registration Method Based

on Weighted Mutual Information”, Proceeding of Int. Conf. on Bioinformatics and

Biomedical Eng., Shanghai, pp 2549-2552, 2008.

[5] I.Misra, S.Moorthi, D.Dhar and R. Ramakrishnan “An Automatic Satellite Image

Registration Technique Based on Harris Corner Detection and Random Sample

Consensus (RANSAC) Outlier Rejection Model,” Proceedings of Int. Conf on Recent

advances in Info. Tech., Dhanbad, pp 68-73, 2012.

[6] C.S.Qiao “A Corner Matching Algorithm based on Harris Operator”, Proceeding of Int.

Conf. on Info. Eng. and Computer Sc., Wuhan, pp 1-2, 2010.

[7] J.Hu, Y.Yang and Z.Su, “A Novel Hierarchical Medical Image Registration Method

Based on Multi-scale and Contour line”, Proceeding of Int. Conf. on Systems and

Informatics, Yantai, pp 1834-1837, 2012.

[8] N. A. Al-Azzawi, H. A. M. Sakim, W. A. K. W. Abdullah, “MR Image Monomodal

Registration Based on the Non-subsampled Contourlet Transform and Mutual

Information,” Proceedings of Int. conf. on Computer Apps. and Ind. Elec., Lumpur, pp.

481-485, Dec. 2010.

Page 56: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

44

[9] T. Butz and J. P. Thiran, “Affine registration with feature space mutual information,”

Med. Image Computing and Computer-Assisted Intervention, Berlin, Lecture notes in

Computer Science, Vol. 2208, pp. 549–556, 2001.

[10] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens “Multimodality

Image Registration by Maximization of Mutual Information,” IEEE Trans. on Med.

Imaging, Vol. 16, No. 2, pp. 187-198, April 1997.

[11] L. Ding, A. Goshtasby, and M. Satter, “Volume image registration by template

matching” Image and Vision Computing, Vol. 19, No. 12, pp. 821–832, 2001.

[12] J.P.W. Pluim, J.B.A. Maintz and M.A. Viergever, “Mutual information based

registration of medical images: a survey,” IEEE Trans. on Medical Imaging, Vol. 22,

No. 8, pp. 986 – 1004, Aug. 2003.

[13] C. Studholme, D.L.G. Hill and D.J. Hawkes “An overlap invariant entropy measure of

3D medical image alignment,” Pattern Recognition, Vol. 32, no. 1, pp. 71–86, Jan.

1999.

[14] C.E. Shannon, “A Mathematical Theory of Communication,” The Bell System

Technical Journal, Vol. 27, pp. 379–423, Oct. 1948.

[15] C.Studholme, D.L.G.Hill and D.J.Hawkes, “Incorporating Connected Region Labelling

into automated image registration using mutual information,” Proceedings of

workshops of Mathematical Methods in Biomedical Image Analysis, San Francisco, pp.

23–31, June 1996.

[16] J. P.W. Pluim, J.B.A. Maintz and M.A. Viergever “Image registration by maximization

of combined mutual information and gradient information,” IEEE Trans. on Medical

Imaging, Vol.19, No.8, pp. 809-814, Aug. 2000.

[17] J. Xie, Z. Chen, G. Xu “A New Method for Medical Image Registration Based on

Wavelet Multi-scale Transform,” Proceedings of Int. Colloquium on Computing,

Comm., Control, and Management, Guangzhou, pp. 355-359, Aug. 2008.

Page 57: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

45

[18] H. Talbi and M.C.Batouche “Particle swarm optimization for image registration,”

Proceedings of Int. Conf. on Information and Comm. Tech., pp. 397-398, April 2004.

[19] A. Sahu, S. K. Panigrahi, S. Pattnaik “Fast Convergence Particle Swarm Optimization

for Functions Optimization,” Proceedings of 2nd

Int. Conf. on Computer. Comm.,

Control and Information Tech., Hooghly, Vol. 4, pp. 319-324, Feb. 2012.

Page 58: IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

46

DISSEMINATION

[1] Sangeeta Sahu and Umesh Chandra Pati “Rigid medical image registration for

monomodal images,” Proceedings of Int. Conf. on Mathematics and Eng. Sciences

(ICMES), Kalka, Vol. 2, No. 1, pp. 87-90, March 2014.

[2] Sangeeta Sahu and Umesh Chandra Pati “Intensity based registration of medical

images," communicated to Int. J. Comput. Vis. and Robotics (inderscience).