Development of a harmonized protocol for hippocampal tracing

An EADC-ADNI joint effort

5th Meeting, New Orleans, April 24, 2012

Update as of April 2012

Principal Investigators: Giovanni B Frisoni & Clifford R JackProject Coordinator: Marina Boccardi

AAN – New Orleans

HOUSEKEEPING INFORMATION

Webinar:-Screen sharing through GotoMeeting-Audio through Première via regular telephone line. Please mute mike through *6

Questions:-Unmute *6 and ask question

PREMIERE TEL NUMBERS: Passcode: 337487

Australia, Melbourne: +61 (0)3 9001 1134 Australia, Sydney: +61 (0)2 9037 1678 Austria, Vienna: +43 (0)1 2530085419 Belgium, Bruxelles: +32 (0)2 792 0496 Bulgaria, Sofia: +359 (0)2 491 7281 Canada, Montreal: +1 514 315 6013 Canada, Toronto: +1 416 915 9970 Czech Republic, Prague: +420 246 019 521 Denmark, Copenhagen: +45 32 71 15 12 Estonia, Tallin: +372 616 0395 Finland, Helsinki: +358 (0)9 2310 1651 France: 0811 655 288 France, Paris: +33 (0)1 70 37 14 50 Germany: 01801 003 823 Germany, Frankfurt: +49 (0)69 667775735 Germany, Munich: +49 (0)89 710424668 Greece, Athen: +30 210 969 6406 Hong Kong, Hong Kong: +852 3071 5041 Hungary, Budapest: +36 1808 8176 Ireland: 1890 907 117 Ireland, Dublin: +353 (0)1 553 0119 Istrael, Tel Aviv: +972 3721 9261 Italy, Milan: +39 02 3626 9646 Italy, Rome: +39 06 8336 0547 Japan, Tokyo: +81 (0)3 5767 4791 Latvia, Riga: +371 6778 2513

Lituania, Vilnius: +370 5205 4214 Luxembourg, Luxembourg: +352 2088 0404 Netherlands, Amsterdam: +31 (0)20 708 5030 New Zealand, Auckland: +64 (0)9 912 3296 Norway, Oslo: +47 2100 0499 Poland, Warsaw: +48 (0)22 212 0122 Portugal, Lisbon: +351 217 810 252 Romania, Bucharest: +40 (0)31 811 0025 Russia, Moscow: +7 495 646 9191 Singapore, Singapore: +65 3158 1291 Slovacchia, Bratislava: +421 (0)2 3300 6528 Slovenia, Ljubljana: +386 1 600 1435 South Africa, Johannesburg: +27 11 019 7002 South Korea, Seoul: +82 (0)2 3483 7321 Spain, Barcelona: +34 93 545 2524 Spain, Madrid: +34 91 791 7142 Sweden, Stockholm: +46 (0)8 5051 4917 Switzerland, Geneva: +41 (0)22 592 7951 Switzerland, Zurich: +41 (0)44 567 1875 UK, Belfast: +44 (0)28 9595 0006 UK, Edinburgh: +44 (0)13 1460 0034 UK, London: +44 (0)20 3059 5702 UK, Reading: +44 (0)11 8370 0027 UK: 0845 351 0340 USA, New York: +1 646 254 3345 USA, Houston: +1 832 369 1692

OUTLINE

- Summary as of to date

- Update since last meeting: Delphi panel - Landmarks defined - Segmentation modalities defined - Check for axes - Paper “Consensus”

- Publication policy of project products

- Project schedule

- Scientific papers schedule

SUMMARY AS OF TO DATE

BACKGROUND

Standardized hippocampal atrophy quantification needed as:

- Biomarker for early diagnosis of AD- Surrogate marker tracking disease progression in clinical trials- Validation of automated segmentation algorithms

STANDARDIZATION STEPS OF MANUAL HIPPOCAMPAL VOLUMETRY

Acquisition Orientation Segmentation

ADNI

EADC-ADNI Hippocampal Harmonization Project

Funded by Alzheimer’s Association Co-funded by unrestricted grant from Wyeth and Lilly

THE EADC-ADNI WORKING GROUP ON THE Harmonized Protocol for Hippocampal Volumetry

GB Frisoni, M Boccardi, M Bocchetta, R Ganzola, A Redolfi, D Tolomeo, G Corbetta, E Cavedo, M Lanfredi Brescia

B Bartzokis, PM Thompson,

L Apostolova, M Tinley Los Angeles

CR Jack, G Preboske, C WardRochester

S Duchesne, N RobitailleQuebec City

H Soininen, MPihlajamäki, Y Liu Kuopio

B Dubois, S Lehericy, C Boutet Paris

H Hampel Frankfurt

S Taipel, M Grothe

Rostock

L-O Wahlund, B Winbald Stockholm

F Barkhof, P Scheltens, W Henneman, M Pronk,

F van DommelenAmsterdam

N Fox, J Barnes, M Blair

London

A SimmonsLondon

C deCarli, S Hollander

Davis

M Weiner, S Mueller San Francisco

L deToledo-Morrell, D Bennet, T Stoub

Chicago

JG Csernansky, L Wang, A Christensen Evanston

RJ Killiany, C BauerBoston

M AlbertBaltimore

M deLeonNew York

J Kaye, L Silbert, T Swihart Portland

J Pruessner, S Pietrantonio

MontrealR Camicioli, N Malykhin Edmonton

J PantelFrankfurt

C Watson Detroit

J O’Brien, M Firbank, E BurtonNewcastle

P Sachdev, JJ Maller (PATH through life Study)

M Geerlings, L Gerritsen, M Portegies(SMART-Medea Study)

T denHeijer(Rotterdam Scan Study)

P Pasqualetti,Rome

L Collins Montreal

PJ VisserMaastricht

L Froelich Mannheim

G Waldemar Copenhagen

L Launer Bethesda W Jagust

Berkeley

C Hock, H Wolf,Zurich

METHODS

Assessment of measurement

properties of SUs

Break down into segmentation units

Bar

tzok

is,

1998

Con

vit,

199

7K

illia

ny,

1993

Soi

nine

n, 1

994

Lehe

ricy,

199

4

Jack

, 19

94;

deT

oled

o-M

orre

ll, 2

004

Hal

ler,

199

7P

rues

sner

, 20

00

Mal

ykhi

n, 2

007

Pan

tel,

2000

Wat

son,

199

2

Survey and operationalization

METHODS

Assessment of measurement

properties of SUsDelphi panel

5 ROUNDS

HARMONIZED HIPPOCAMPUS

DELPHI PANEL

Hippo experts participated to evidence-based Delphi panel

P.I. City Delphi Panelist

Barkhof/Scheltens Amsterdam Wouter HennemanGeorge Bartzokis Los Angeles George BartzokisRichard Camicioli Edmonton Nikolai MalykhinJohn Csernansky Chicago Lei WangCharles DeCarli Sacramento Charles DeCarliLeyla DeToledo-Morrell Chicago Leyla DeToledo-MorrellNick Fox London Josephine BarnesMirjam Geerlings Stockholm Lotte GerritsenClifford Jack Rochester Clifford Jack/Greg PreboskeRonald Killiany Boston Ronald KillianyJohn O’Brien Newcastle Michael FirbankJens Pruessner Montreal Jens PruessnerHilkka Soininen Kuopio H. Soininen/M. PihlajamäkiPaul Thompson Los Angeles Liana ApostolovaCraig Watson Detroit Craig WatsonChristoph Hock Zurich Henrike Wolf

L ApostolovaLos Angeles

J BarnesLondon

G BartzokisLos Angeles

C deCarli Sacramento

L deToledo- Morrell Chicago

M Firbank

Newcastle

L Gerritsen

Stockholm

W HennemanAmsterdam

CR JackRochester

R KillianyBoston

N Malykhin Edmonton

J PruessnerMontreal

H Soininen Kuopio

L Wang

Chicago

C Watson

Detroit

H Wolf

Zurich

Delphi panel results

Round Median Agree Neutral Disagree P

N (%) N (%) N (%) (Binomial)

Landmarks

Inclusion of alveus/fimbria II 9 13 (81%) 0 (0%) 3 (19%) 0.021

Segmentation of subiculum (morphology - I choice) III 7.5 11 (69%) 4 (25%) 1 (6%) 0.006

Segmentation of subiculum (horizontal - II choice) III - 13 (81%) 0 (0%) 3 (19%) 0.021

Inclusion of tail end III 9 12 (75%) 2 (12.5%) 2 (12.5%) 0.013

Harmonized Hippocampus II 8 14 (88%) 1 (6%) 1 (6%) 0.001

Segmentation Modality

Disambiguating amygdala with 3D navigation II 8 16 (100%) 0 (0%) 0 (0%) <0.0005

Inclusion of vestigial tissue III - 10 (63%) 0 (0%) 6 (37%) n.s.

CSF pools III 8 14 (88%) 0 (0%) 2 (12%) 0.004

Not visible structures (final definition) III 8 16 (100%) 0 (0%) 0 (0%) <0.0005

Separating Alveus/Fimbria from fornix IV 8 12 (82%) 0 (0%) 1 (8%) 0.003

AC-PC Image orientation V 8.5 11 (79%) 2 (14%) 1 (7%) 0.006

LANDMARKS

Delphi consensus - LANDMARKS

“Alveus/Fimbria”: The hyperintense tissue located on the dorso-lateral aspect of the hippocampal head and body.

Significant agreement on Round II p=0.021

Delphi consensus - LANDMARKS

“Subiculum-Morphology”: A line following the visible contour of the medial border of the hippocampus, and relying on the morphological details.

Significant agreement on “Morphology” as first choice criterion on Round III

P=0.006

Delphi consensus - LANDMARKS

“Subiculum- Horizontal”: A horizontal line connecting the highest point of the parahippocampal WM medially, to the ambient cistern.

Significant agreement on “Horizontal” as second choice criterion on Round III

p=0.021

Oblique Horizontal Line as second choice

Delphi consensus - LANDMARKS

“Tail”: all hippocampal tissue visible from the joint visualization of the superior and inferior colliculi, to the most caudal slice where GM mass is visible inferomedially to the trigone of the lateral ventricle.

Significant agreement on Round III p=0.013

Delphi consensus - LANDMARKS

“Harmonized Hippocampus”: Global definition of hippocampus, including all SUs

Significant agreement on Round II p=0.001

Delphi consensus - LANDMARKS

Covers 100% of hippocampus proper

Captures 100% of AD-related atrophy

Intra-rater (BS-Mayo-LONI-Mainz): 0.98Inter-rater (BS-Mayo-LONI-Mainz): 0.94

Delphi Landmark Consensus movie

"Movie 1“

SEGMENTATION MODALITIES

Delphi consensus – Discrimination Amy/Hippo

Significant agreement for discrimination based on 3D navigation

Round II p<0.0005

Not possible Possible

Delphi consensus – Vestigial

Disagreement for discrimination based on 3D navigation in round III

0 1

II_vest

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Freq

uenz

a

Mean = 0,40Std. Dev. = 0,50709N = 15

Include exclude vestigial

Delphi consensus – CSF pools

The CSF located internal to the hippocampus, also in unexpected regions, must be excluded from the segmentation when the tracer is sufficiently sure that the hypointense voxels correspond to CSF, and not to partial volume effect.

Delphi consensus – CSF pools

0 1

II_csF

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Freq

uenz

a

Mean = 0,8667Std. Dev. = 0,35187N = 15

disagree agreewith definition

Round III p=0.004

Delphi consensus – hardly visible structures

Delphi consensus – hardly visible structures

“…a priori knowledge of hippocampal morphology must be used to carry out additional checks to ascertain whether hippocampal tissue can be detected in atrophic patients.”

Round III p<0.0005

Delphi consensus – separate Fimbria/Fornix

“Include the WM as alveus/fimbria whenever it appears attached to the hippocampal GM.

Exclude the above WM as fornix. this part diverges from the plane parallel to the parahippocampal WM, taking a different angle, and is clearly detached from hippocampal GM”

Delphi consensus – separate Fimbria/Fornix

Round IV p=0.003

Delphi consensus – image orientation

Methodological problem: data fed to Delphi were collected on the AC-PC axis. Shift to hippo axis may represent an inconsistency.

We compared the topographic and volumetric stability (TRTR and IRR of master tracers) of segmentation on

hippo and AC-PC axes.

Liana Apostolova

Gregory Preboske

Martina Bocchetta

Design of axis check

segmented (based on Harmonized Prot)

10 ADNI subjects (R+L)

on hippo axis on AC-PC axis

Volume agreement within and between tracers and Dice’s coefficients for spatial overlapping were computed.

Image orientationbetter overlapping of segmentation in AC-PC

Dice’s coefficients for spatial overlapping agreement between MB and GP.

Image orientation: visualization of tracings showsbetter overlapping in AC-PC

"Movie 2"Maximum Dice for AC-PC (0.8908) computed between Tracer 2 and Tracer 3, and corresponding to Subject 3, Right hippo.

The Delphi panel agreed to keep AC-PC, in consideration of the methodological problem and the new evidence:

Median 8.5

Binomial p=0.006

Delphi panel V: agreement for AC-PC

disagreement agreement

Round V p=0.006

PUBLICATION POLICY

Publication policy

http://www.hippocampal-protocol.net/SOPs/LINK_PAGE/Publication_Policy_V3-2.pdf

Latest release: March 28

Steering Committee

Applicability

Availability of deliverables

Authorships

Acknowledgements

Revisions

Availability of deliverables

The Harmonized Protocol (i.e. the text description and illustrative material), the digital master tracer masks, and digital training sets (masks developed by the 5 best naïve tracers) will be available only to beta-test users in the period (about 1 year) between the end of the Delphi Panel and the publication of the full validation results (i.e. the analysis of the whole set of hippocampal tracings by all naïve tracers). To qualify as beta-testers, users will submit proposals to the SC; if accepted, beta-testers will be granted access to deliverables under conditions dictated by a written cooperation agreement that will be signed by the user and the Chair of the SC. Assuming mutual benefit, the cooperation agreement will include provisions related to scientific publications, donations, and feedback that beta users will provide on their exploitation of the Harmonized Protocol.

PROJECT SCHEDULE

FUTURE STEPS

1. Develop a qualification environment and qualification thresholds for naïve tracers and automated algorithms(by June 2012)

"Movie 3“Prototype of qualification environment and thresholds

FUTURE STEPS

2. Validate on:- 1800 ADNI hippocampi segmented by 20 human tracers- 20x2 (R&L) hippocampi with volume on pathology, neuronal density, and ex vivo MR(by January 2013)

Variability Sources Examination:

Effects of: rater, trace-retrace, side, atrophy, time/progression, scanner, strength field

GANTT

PAPERS SCHEDULE

Papers describing the project

1. Survey of protocols (preliminary phase; published, JAD 2011)

2. Operationalization (preliminary phase; submitted to Alzheimer’s & Dementia, MS n. ADJ-D-12-00094)

3. Orientation check (Brescia Team, in progress)

4. Delphi consensus (Brescia Team, in progress)

5. Master tracers’ practice and reliability (Brescia Team, in progress)

6. Development of certification platform (Duchesne & coll)

7. Validation data and Protocol definition (Brescia Team)

8. Validation vs pathology (TBD)

AAN - Session S04: Aging and Dementia: Therapeutic Interventions

“Delphi Consensus on Landmarks for the Manual Segmentation of the Hippocampus on MRI: Preliminary Results from the EADC−ADNI Harmonized Protocol Working Group”

platform presentation number S04.003

Tuesday, April 24, 2012 at 1:30 pm

Results will be presented at

Results will be highlighted at

www.hippocampal-protocol.net

More info about the project at

Geriatric Neurology Section Highlights in the Field Wednesday, April 25th from 5:30-6:30 p.m. Room 356/357

(Drs. Ringman and Apostolova)

Acknowledgements

Maria Carrillo, Meredith McNeil Alzheimer’s Association, Chicago IL

All partners!