72
Medical Image Registration Kumar Rajamani

Medical Image Registration

  • Upload
    lumina

  • View
    95

  • Download
    9

Embed Size (px)

DESCRIPTION

Medical Image Registration. Kumar Rajamani. Registration. Spatial transform that maps points from one image to corresponding points in another image. Time axis : aging , development …. z. Cross-subject, same modality. …. y. x. Anatomy. …. Label. …. Same subject, cross modality. - PowerPoint PPT Presentation

Citation preview

Page 1: Medical Image Registration

Medical Image Registration

Kumar Rajamani

Page 2: Medical Image Registration

Registration

Spatial transform that maps points from one image to corresponding points in another image

Page 3: Medical Image Registration

xy

zTime axis: aging, development…

………

… … …

Anatomy

Label

Function

Pathology,Histology…

Cross-subject, same modality

Sam

e s

ub

ject,

cro

ss m

od

ality

Commoncoordinatesystem

Image Registration

Data Fusion

Atlases for population studies

Motion Correction for image reconstruction

Surgery Planning

Page 4: Medical Image Registration

Affine Scale / Skew

Example Registration Transformations

Rigid Deformable

SimilarityRigid + Scale

Original

Page 5: Medical Image Registration

5

Image comparison

Difference beforeregistration

Difference afterregistration

Registeredmoving image

Page 6: Medical Image Registration

Before Registration

B-Splines based Registration

After Registration

*GE Owned

Page 7: Medical Image Registration

B-Splines based Registration (Hill et al.)

Page 8: Medical Image Registration

Dense NR Registration (Hill et al.)

Page 9: Medical Image Registration

Registration Criteria

Landmark-based– Features selected by the user

Segmentation-based– Rigidly or deformably align binary

structures Intensity-based

– Minimize intensity difference over entire image

Page 10: Medical Image Registration

Spatial Transformation

Rigid– Rotations and translations

Affine– Also, skew and scaling

Deformable– Free-form mapping

Page 11: Medical Image Registration

Registration framework pieces

2 input images, fixed and moving Metric - determines the “fitness” of the

current registration iteration Optimizer - adjusts the transform in an

attempt to improve the metric Interpolator - applies transform to image

and computes sub-pixel values

11

Page 12: Medical Image Registration

Registration — General Problem

Problem: Find transform T such that image f[p] matches m[T(p)]

Domain ofan individual

subject

Commoncoordinatesystem

f m

)( pq T

transformmetricoptimizer

)]([m],[fminarg ppd T

T

Interpolate

Page 13: Medical Image Registration

Registration Framework

Page 14: Medical Image Registration

Transforms

x’=T(x|p)=T(x,y|tx,ty,θ)

Goal: Find parameter values that optimize image similarity metric

y

x

t

t

y

x

y

xx

cossin

sincos

'

''

Page 15: Medical Image Registration

Optimizer

Often require derivative of image similarity metric (S)

j j

j

ji

x

x

MFSMFS

p

p

p

p '

'

),,|(),,|(

m

nn

m

m

p

x

p

x

p

x

p

x

p

x

p

xp

x

p

x

p

x

J

'''

'''

'''

1

1

1

2

2

2

1

2

1

2

1

1

1

Page 16: Medical Image Registration

Understanding the Transform Jacobian

J shows how changing p shifts a transformed point in the moving image space.

This allows efficient use of a pre-computed moving-image gradient to infer changes in corresponding-pixel intensities for changes in p

Now we can update dS/dp by just updating J

16

Page 17: Medical Image Registration

Jacobian and Image Gradient

Page 18: Medical Image Registration

Transforms

Before we discuss specific transforms, let’s discuss the…

Fixed Set = the set of points (i.e. physical coordinates) that are unchanged by the transform

The fixed set is a very important property of a transform

18

Page 19: Medical Image Registration

Identity Transform

Maps every point to itself Only used for testing Fixed set = everything (i.e., the entire

space)

Page 20: Medical Image Registration

Translation Transform Fixed set = empty set Translation can be closely approximated by:

– Small rotation about distant origin, and/or…– Small scale about distant origin– Both of these do have a fixed point

Optimizers will frequently (accidently) do translation by using either rotation or scale– This makes the optimization space harder to use– The final transform may be harder to understand

20

Page 21: Medical Image Registration

Translation Transform

Fixed set is an empty set

Page 22: Medical Image Registration

Scaling Transform

Isotropic vs. anisotropic Fixed set is the origin of the coordinates

C

C

Page 23: Medical Image Registration

2D Rotation Transform

Rotation transforms are typically specific to either 2D or 3D

Fixed set = origin = “center” = C

24

C

C

Page 24: Medical Image Registration

Rotations in Polar Coordinates

y

x

y

xx

cossin

sincos

'

''

iii

i i

erereyx

rrreyx

)()','(

)sin,cos(),(

Page 25: Medical Image Registration

Transforms

Affine TransformS=Id reduces to Rigid case,

(e.g. Head motion correction)

Affine Transforms also encode scale/skew factors

tAx

txRSxT tS

.],,[

Scaling Rotation Translation

Original

Rigid

Page 26: Medical Image Registration

Optimization

Search for value of θ that minimizes cost function S

Gradient descent algorithm– Update of parameter

– G is the variation from the gradient of the cost function

is step length of algorithm

)exp(

'

' Geeee

S

Si

iii

Page 27: Medical Image Registration

Sequence of translations and metric values at each iteration of the optimizer

Page 28: Medical Image Registration

Scaling, Rotation,Translation

P=arbitrary point C=fixed point of transformation D=scaling factor Θ=rotation angle P and C are complex numbers (x+iy) or reiθ

Store derivates of P in Jacobian matrix for optimizer

Rigid if D=1, otherwise similarity transform

CDeCPPTP i)()('

Page 29: Medical Image Registration

Affine Transformation

Collinearity is preserved x’=A x + T A is a complex matrix of coefficients With fixed point

– x’=A (x–C) + C A is optimized similar to the scaling

factor

Page 30: Medical Image Registration

Quaternions

Quotient of two vectors– Q= A / B

Operator that produces second vector– A= Q B

Represents orientation of one vector with respect to another, as well as ratio of their magnitudes– Versor-rotates vector– Tensor-changes vector magnitude

Page 31: Medical Image Registration

Scalars and Versors

Quaternion represented by 4 numbers– Versor

• Direction – parallel to axis of rotation• Rotation angle• Norm – function of rotation angle

– Tensor• Magnitude

Page 32: Medical Image Registration

Rigid Transform in 3D

Use quaternions instead of phasors P’=V*(P-C)+C P’=V*P+T, T=C-V*C P=point, V=Versor, T=Translation, C=fixed

point Transform represented by 6 parameters

– Three numbers representing versor– Three components of fixed coordinate system

Page 33: Medical Image Registration

Image Interpolators

2 functions– Compute interpolated intensity at

requested position– Detect whether or not requested position

lies within moving-image domain

Page 34: Medical Image Registration

Nearest Neighbor

Uses intensity of nearest grid position Computationally cheap Doesn’t require floating point

calculations

Page 35: Medical Image Registration

Linear Interpolation

Computed as the weighted sum of 2n-1 neighbors

n=dimensionality of image Weighting is based on distance

between requested position and neighbors

Page 36: Medical Image Registration

B-spline Interpolation

Intensity calculated by multiplying B-spline coefficients with shifted B-spline kernels

Higher spline orders require more pixels to computer interpolated value

Third-order B-spline kernels typically used because good tradeoff between smoothness and computational burden

Page 37: Medical Image Registration

Metrics

Scalar function of the set of transform parameters for a given fixed image, moving image, and transformation type

Typically samples points within fixed image to compute the measure

Page 38: Medical Image Registration

Mean Squares

Mean squared difference over all the pixels in an image

Intensities are interpolated for the moving image

For gradient-based optimization, derivative of metric is also required

2))],(()([1

),,|( pxMxFN

MFpS i

N

i i TT

Page 39: Medical Image Registration

Mean Squares

Optimal value of zero Interpolator will affect computation time

and smoothness of metric plot Assumes intensity representing the

same homologous point is in both images

Images must be from same modality

Page 40: Medical Image Registration

Mean Squares

Smoothness affected by interpolator

Page 41: Medical Image Registration

Normalized Correlation

Computes pixel-wise cross-correlation between the intensity of the two images, normalized by the square root of the autocorrelation of each image

For two identical images, metric =1 Misalignment, metric <1

Page 42: Medical Image Registration

Normalized Correlation

-1 added for minimum-seeking optimizers

N

i

N

iii

N

iii

xTMxF

xTMxFMFpS

))(()(

))(()(1),,|(

22

T

Page 43: Medical Image Registration

Normalized Correlation

Page 44: Medical Image Registration

Difference Density

Each pixel’s contribution is calculated using bell-shaped function

f(d) has a maximum of 1 at d=0 and minimum of zero at d=+/-infinity

d is difference in intensity b/w F and M

2)/(1

1)(

ddf

Page 45: Medical Image Registration

Multi-modal Volume Registration by Maximization of Mutual Information

Wells W, Viola P, Atsumi H, Nakajima S, Kikinis R

Page 46: Medical Image Registration

Registering Images from Same Modality Typical measure of error is sum of

squared differences between voxels values

This measure is directly proportional to the likelihood that the images are correctly registered

Same measure is NOT effective for images of different modalities

Page 47: Medical Image Registration

Figure 8.9 from the ITK Software Guide v 2.4, by Luis Ibáñez, et al.62

MI Inputs

T1 MRI Proton density MRI

Page 48: Medical Image Registration
Page 49: Medical Image Registration
Page 50: Medical Image Registration

Relationship Between Images of Different Modalities Example: We should be able to construct a

function F() that predicts CT voxel value from corresponding MRI value

Registration could be evaluated by computing F(MR) and comparing it to the CT image– Via sum of squared differences (or correlation)

In practice, this is a difficult and under-determined problem

Page 51: Medical Image Registration

Mutual Information

Theory: Since MR and CT both describe the underlying anatomy, there will be mutual information between the two images

Find the best registration by maximizing the information that one image provides about the other

Requires no a priori model of the relationship Assumes that max. info. is provided when the

images are correctly registered

Page 52: Medical Image Registration

Notation

Reference (fixed) volume: u(x) Test (moving) volume: v(x) x: coordinates of the volume T: transformation from coordinate frame

of reference volume to test volume v(T(x)): test volume voxel associated

with reference volume voxel u(x)

Page 53: Medical Image Registration

Mutual Information

Defined in terms of entropies

If there are any dependencies, H(A,B)<H(A)+H(B)

)BA,()B()A()BA,(

),(log),()BA,(

)(log)()B(

)(log)()A(

HHHI

dabapbapH

dbbpbpH

daapapH

ABAB

BB

AA

Page 54: Medical Image Registration

Maximizing Mutual Information

h(v(T(x))) encourages transformations that project u into complex parts of v

Last term of MI eqn contributes when u and v are functionally related

Together, last two terms of MI eqn identify transforms that find complexity and explain it well

)))((),(()))((())(()))((),((

)))((),((maxargˆ

xTvxuhxTvhxuhxTvxuI

xTvxuITT

Page 55: Medical Image Registration

Parzen Windowing

Used to estimate probability density P*(z)

Entropy estimated based on P*(z)

Page 56: Medical Image Registration

Finding Maximum of I(T)

To find maximum of mutual information, calculate its derivative:

Derivative of reference volume is 0, b/c not a function of T

Entropies depend on covariance of Parzen window functions

)))((),((*)))(((*))((*)( xTvxuhdT

dxTvh

dT

dxuh

dT

dTI

dT

d

Page 57: Medical Image Registration

Stochastic Maximization of Mutual Information

Similar to gradient descent Steps are taken that are proportional to dI/dT Repeat:

– A {sample of size NA drawn from x}

– B {sample of size NB drawn from x}

– T T+λ(dI/dT)

λ is the learning rate Repeated a fixed # of times, or until

convergence

Page 58: Medical Image Registration

Stochastic Approximation

Uses noisy derivative estimates instead of the true derivative for optimizing a function

Authors have found that technique always converges to a transformation estimate that is close to locally optimal– NA=NB=50 has been successful

The noise introduced by the sampling can effectively penetrate small local minima

Page 59: Medical Image Registration

MRI-CT Example

Coarse to fine registration Images were smoothed by convolving with

binomial kernel Rigid transform represented by displacement

vectors and quaternions Images were sampled and tri-linear

interpolation was used 5 levels of resolution

– 10000, 5000(*4) iterations

Page 60: Medical Image Registration

Initial Condition of MR-CT Registration

Page 61: Medical Image Registration

Final Configuration for MR-CT Registration

Page 62: Medical Image Registration

Initial Condition of MR-PET Registration

Page 63: Medical Image Registration

Final Configuration for MR-PET Registration

Page 64: Medical Image Registration

Application Register 2 MRIs of brain (SPGR and

T2-weighted) to visualize anatomy and tumor– Create at 3-D model for surgical planning

and visualization

Page 65: Medical Image Registration

3-D Model

Tumor(green), Vessels(red), Ventricles(blue), Edema (orange)

Page 66: Medical Image Registration

Correlation Conventional correlation aligns two

signals by minimizing a summed quadratic difference between their intensities

If intensity of one signal is negated, then intensities no longer agree, and alignment by correlation will fail

Mutual information is not affected by negation of either signal

Page 67: Medical Image Registration

Occlusion

Correlation is significantly affected by occlusion because intensity is substantially different

Occlusion will reduce mutual information at alignment– But “mutual information measure degrades

gracefully when subject to partially occluded imagery”

Page 68: Medical Image Registration

Comparison to Other Methods

Many researchers use surface-based methods to register MRI and PET imagery– Need for reliable segmentation is a drawback

Others use joint entropy to characterize registration– “not robust”: difficulty describing partial overlap– Mutual Information is better because it has a larger

capture range• Additional influence from term that rewards for complexity

in portion of test volume into which reference volume is transformed

Page 69: Medical Image Registration

Comparison to Other Methods

Woods et al. register MR and PET by minimizing range of PET values associated with a particular MR intensity value– Effective when test volume distribution is

Gaussian– Mutual Information can handle data that is multi-

modal– Woods’ measure is sensitive to noise and outliers

Page 70: Medical Image Registration

Registration Tools/ Softwares

Elastix (ITK) ANTS (ITK) Plastimatch Slicer3D (ITK) MevisLab Fiji

Page 71: Medical Image Registration

Conclusions

Intensity based techniques work directly with volumetric data (vs. segmentation)

Mutual information does not rely on assumptions about nature of imaging modalities

Have also used this technique to register 3D volumetric images to video images of patients

Page 72: Medical Image Registration

April 2011

87

Kumar Rajamani

GE Global Research

[email protected]