Analuse Globalisée des Données d Imagerie Radiologique From Image Registration in Oncology to Complex Workflows on the GRID Xavier Pennec, PhD, INRIA-Sophia,

Analuse Globalisée des Données d ’Imagerie Radiologique

From Image Registration in Oncologyto Complex Workflows on the GRID

Xavier Pennec, PhD, INRIA-Sophia, projet EpidaureJohan Montagnat, PhD, I3S, Rainbow team, Tristan Glatard, I3S, Rainbow + INRIA, Epidaure teamsPierre-Yves Bondiau, MD, PhD, Centre Antoine Lacassagne, Nice

AGIR - Sophia 2

Analyse Globalisée des Données d’Imagerie Radiologique

www.aci-agir.org

Overview

• The Medical application: – Registration for oncology

• The scientific question:– Evaluation / comparison of registration algorithm performances

• The technical challenge:– Running the workflow on the GRID

AGIR - Sophia 3


www.aci-agir.org

Image Registration for Oncology

• Registration / segmentation are basic components of medical image analysis– Registration: finding homologous points / tranformation– Segmentation: give anatomical label to each image point

• Registration for brain radiotherapy– Planning

Fusion of image modalities (multimodal, rigid) Warp atlas to patient image for segmentation

(mono-modal, non-rigid) Definition of Target volumes and Organs at risk: dose optimization

– Follow-up (monomodal rigid)

http://www.healthgrid.org/docs/pdf/WhitePaperdraft_v1.1-3reviewedv2.pdf (ch 3/4)

AGIR - Sophia 4


www.aci-agir.org

Inter-subject registrationAffine transformation

Correct size and position but high remaining variability in cortex and deep structures

MR T1 Images

256x256x120 voxels

Atlas to patient registrationfor radiotherapy planning


AGIR - Sophia 5


www.aci-agir.org

Anatomically meaningful deformationRegistration in 5 min on 15 PCs

Adaptive non-stationary visco-elastic inter-subject registration


AGIR - Sophia 6


www.aci-agir.org

Atlas

Propagate the segmentation of structure of interest from the atlas to the patient image

AGIR - Sophia 7


www.aci-agir.org


• Define target volume and organs at risk thanks to the segmentation• Optimize the irradiation process to

– maximize the dose within the tumor – minimize it within neighboring organs at risk

AGIR - Sophia 8


www.aci-agir.org


• There is no universal registration algorithm– More than 600 references on medical image registration in 1997– More than 100 papers each year… (70 at MICCAI 2004 only)

• Registration algorithms as Grid services

– Use up to date algorithm– Evaluation / comparison of algorithm performances

• Challenges– Inter-operability (coordinate systems, transformation format…)– Ontology describing data, registration problems and algorithms

AGIR - Sophia 9


www.aci-agir.org

Overview




AGIR - Sophia 10


www.aci-agir.org

Variability of a registration algorithm

Registration algorithm

Final transformation

External parameters

• Data (image) 1

• Data (image) 2

• Acquisition noise

• Patient effects

Varying internal parameters

• Initial transformation

• (…)

• Robustness: ability to find the right transformation (success/failure)

• Precision: Repeatability w.r.t. some parameters (e.g. initialization)

• Accuracy: Variability w.r.t. the ground truth for typical data

Fixed internal parameters

• Multiscale resolution

• (Typical variance…)

AGIR - Sophia 12


www.aci-agir.org

• Uncertainty = deviation from the real transformation– Maximum error: bound– Mean Error: covariance matrix, std dev.

On the transformation ( rotation r [rad], translation t [mm])

On test points (TRE x)

Quantifying the registration errors

• Robustness: – size of the basin of attraction– Probability of convergence

AGIR - Sophia 16


www.aci-agir.org

Performance evaluation and validation

• Synthetic data (simulation): – Available ground truth– Difficult to identify and model all sources of variability

• Real data in a controlled environment (Phantom):– Possible gold standard– Performances evaluation in specific conditions

Difficult to test all clinical conditions May hide a bias

• Image database representative of the clinical application– Usually no ground truth– Should span all sources of variability

AGIR - Sophia 18


www.aci-agir.org

• Bronze standard: The exact result is an unknown

variable– Unbiased estimation: use redundant information

use many different registration algorithms(average biases, so that precision ~ accuracy)

Use many different data (redundant information to ensure precision) Average transformations (maximal consistency)

• Data intensive application:– High number of images across different databases– High number of registration algorithms

Performance Evaluation without Gold Std

AGIR - Sophia 19


www.aci-agir.org

Multiple a posteriori registration

• Best explanation of the observations (ML) :– Robust Fréchet mean– Robust initialisation and Newton gradient descent

• Result

2221

2 ),,(min),( TTTTd i

transrotjiT ,,,

AGIR - Sophia 20


www.aci-agir.org

Example bronze std

222/

2 2 USMRUSMRloop

AGIR - Sophia 21


www.aci-agir.org

• Data intensive application:– High number of images across different databases– High number of registration algorithms

• Grid validation protocol (PhD Tristan Glatard)– Find available data that match the problem description– Find the algorithms that can deal with them– Find and organize the resources to do the job

Performance Evaluation without Gold Std

AGIR - Sophia 22


www.aci-agir.org

Bronze Std workflow

CrestMatch

PFMatchICP

PFRegisterYasmina

Baladin

Resultsmanagement

Format conversion

Crest lines extraction

Format conversion

Results management

Formatconversion

Results management

Format conversion

Results management

Target image :- Image1- Image2- ...

Registrationalgorithms

Othercomponents

data links

input

output

Floating image :- Image1- Image2- ...

The bronze standard workflow

AGIR - Sophia 23


www.aci-agir.org

Overview




AGIR - Sophia 24


www.aci-agir.org

Workflow manager

• Workflow description – components / links– Taverna is the most powerful

• Workflow Execution – Use the available parallelism (different notions of grid….)– Taverna has severe limitations

• Control issues

AGIR - Sophia 25


www.aci-agir.org

Workflow description

• Description of processing components (web services)– Interface (e.g. WSDL), independent of their implementation– Example:

<message name="registrateWithCrestMatchRequest">

<part name="reference" type="xsd:string"/>

<part name="floating" type="xsd:string"/>

<part name="crest-ref" type="xsd:string"/>

<part name="crest-float" type="xsd:string"/>

<part name="input-comment" type="xsd:string"/>

</message>

<message name="response">

<part name="result-image" type="xsd:string"/>

<part name="result-voxel-transfo" type="xsd:string"/>

<part name="result-real-transfo" type="xsd:string"/>

<part name="reference-image" type="xsd:string"/>

<part name="floating-image" type="xsd:string"/>

<part name="comment" type="xsd:string"/>

</message>

<SOAP:address location="http://colors.unice.fr:18002"/>

AGIR - Sophia 26


www.aci-agir.org

Workflow description

• Description of processing components (web services)– Interface (e.g. WSDL), independent of their implementation– Description is syntactic, not semantic

• Description of links between components

– Control links (from e-business): BPEL4WS – WSCDL

– Data links (from e-science) Scufl (Taverna)Scufl (Taverna) – MoML (Kepler)

<sequence><flow><switch><while><wait>BPEL tags

<processor><source><sink><link>

Scufl tags

AGIR - Sophia 27


www.aci-agir.org

Taverna

• Chosen workflow management tool: Taverna – Developed in the UK project myGrid (bioinformatique)– Open source : http://taverna.sourceforge.net– Based on web-services– Most powerful workflow manager for description

• Current research (e.g. in myGrid, UK)– Semantic annotation of services through ontologies– Automatic transcription into translating units

Limitation of translating units needed for algorithm compatibility Systematic discovery of available components

AGIR - Sophia 28


www.aci-agir.org

Taverna

• Limitations of the data iteration strategy description– Scufl:

dot and cross products operators

– In our case: register all images of the same patient

the same modality

A different exam date

Set 0 Set 1

I0

J0

I1

J1

I2

J2

Ref Img Flo Img

A0

A0

A1

A1

A2

A2

B0

B0

B1

B1

Set 0 Set 1

I0

J0

I1

J1

I2

J2

AGIR - Sophia 29


www.aci-agir.org

Taverna: Execution

• Interaction of Taverna with the grid (EGEE)

• Exloiting the parallelism of the workflow– Splits and synchronize, e.g.

C1: Initialization C2: Register Algo 1 C3: Register Algo 2 C4: avarage results

– Taverna is OK for one data…

Tavernaworkflowmanager

RegistrationWeb-Service

EGEE User InterfaceSOAP

(over HTTP)ssh

tunnellingcommand line

interface

Grid Resources

C1

C2

C3

C4

D0

AGIR - Sophia 30


www.aci-agir.org

Exploiting parallelism

• Data parallelism: – components are not multithread in Taverna!– Patch with submission/fetching services

Data order is not preserved (send 1/2/3, receive 3/1/2) Need a track record for each result

C1

C2

C3

C4D

0, D

1, D

2

–Asynchronous interactionTaverna Submission

service

Fetchingservice

GridMonitor2Monitor1

query1

query2

Taverna Web-Service Grid

computation1

query1

result1

computation2

query2

result2

result2

result1

com

pu

tati

on1

com

pu

tati

on2

– Synchronous interaction

AGIR - Sophia 31


www.aci-agir.org

Exploiting parallelism

• Data + component parallelism: streaming (Pipelining)

Nw sequential steps, ND Data sets, Mean time T per component

Execution time = ND.Nw.T vs (ND+Nw-1).T

– Example for registration:

nD = 50 ; n

W = 4 ; T = 30min

Execution time = 100h vs 26.5 h

• Streaming is not possible with Taverna

C1

C2

C3

C4D

0, D

1, D

2

C1 D0 D1 D2 - - - -

C2 - - - C1*D0 C1*D1 C1*D2 -

C3 - - - C1*D0 C1*D1 C1*D2 -

C4 - - - - - - Mean

C1 D0 D1 D2 - -

C2 - C1*D0 C1*D1 C1*D2 -

C3 - C1*D0 C1*D1 C1*D2 -

C4 - - - - Mean

AGIR - Sophia 32


www.aci-agir.org

A new workflow execution engine

• Development of a new execution engine

– compatible with Taverna description (Scufl)

– Allowing data and Component parallelism

– Implementing result traceability

– Article submitted, soft to be available at

http://www.i3s.unice.fr/~glatard

AGIR - Sophia 33


www.aci-agir.org

Controlling the execution

• Taverna and the new execution engine handle:– The traceability of results (execution tree for each data)

• Taverna handles:– Re-submissions and delays– Alternative but predefined locations of web-services

• Remaining issues– Nor Taverna nor EGEE handles

Job submission errors Cancelled or lost jobs Timeouts

– How to do that without stopping the workflow execution?– Is it a middleware or a workflow manager issue?

AGIR - Sophia 34


www.aci-agir.org

Conclusion - perspectives

• Prototype of a new execution engine for Taverna – Exploiting streaming parallelism– Control of traceability

• Open questions– Including ontologies– Granularity of jobs on the grid– Reliable interface with the EGEE infrastructure

(timeouts/errors)

• The Bronze standard application– Verification phase (standardization / converters)– Coupling with ontologies– Benchmark for

registration algorithms Compression Workflow execution engines on the grid

AGIR - Sophia 35


www.aci-agir.org

References

• Bronze Standard Granger et al, MICCAI 2001 & ECCV 2002. Nicolau et al, IS4TM 2003.

• Worflows on GRIDS T. Glatard & al. Grid-enabled workflows for data intensive

applications. IEEE Int. Symp. On Computer-based Medical Systems CBMS’05.

T. Glatard & al. An optimized workflow enactor for data-intensive grid applications, Submitted to IEEE/ACM Intern. Work. On Grid Computing 2005 (associated to Supercomputing 2005).

AGIR - Sophia 36


www.aci-agir.org

AGIR - Sophia 37


www.aci-agir.org

• Scenario 1: user accesses to registration services through the grid on his own data

• Scenario 2: the user test his algorithm on standard image databases

User

GRIDRegistration

service

Computer resources

Image dataresources

Grid registration services

AGIR - Sophia 38


www.aci-agir.org

Interoperability challenges

• Image format (input / output) Dicom (communication module ?) Basic 3D image format ?

• Transformation formats Standardized displacement field / resampled image Internal representation + std resampling function

• Algorithm parameters / options Define std param. w.r.t. classes of registration problems

• Interactivity State of advancement (reporting) Interactive corrections


AGIR - Sophia 39


www.aci-agir.org

Ontology of Algorithms (registration service)

• Type of data Images (2D, 3D, time series) Point clouds, landmarks

• Type of spatial transformation Rigid / similarity / affine Non rigid (global / local) (splines, def. Fields, polyrigids…)

• From Data to Transformation Comparison metric (SSD, Correlation coefficient)

takes into account the intensity transformation Optimization procedure Interactivity


AGIR - Sophia 40


www.aci-agir.org

Ontology of Registration Problems (image databases)

• Modality involved (specifies the type of data) Monomodal (CT, MR, US, Video, point measures…) Multimodal (combination of above) Atlas to modality

• Image content (specifies the type of transformation) Anatomical part concerned (head, thorax, abdomen…) Changes expected

• intrasubject / intersubject / atlas• Smooth evolution / pathology


AGIR - Sophia 48


www.aci-agir.org

Interactive volume

reconstuctionA. Osorio

Workflow Management J. Montagnat

MedicalApps.

Les thématiques

Cardiological images

SegmentationI. Magnin

Humanitarian Medical

DevelopmentV. Breton

Image registration in oncologyX. Pennec

Dissem

ination C. G

ermain

Services for InteractivityC. Germain

Middleware evaluation E. Jeannot

Medical data ManagementJ. Montagnat

Medical data access

protocols J-M. Moureaux

CoreGrid

MedicalServices

AlgorithmGridification

Medical applications evaluation P-Y Bondiau

Documents

Analuse Globalisée des Données d Imagerie Radiologique From Image Registration in Oncology to Complex Workflows on the GRID Xavier Pennec, PhD, INRIA-Sophia,