Upload
badru
View
39
Download
2
Tags:
Embed Size (px)
DESCRIPTION
May 2012 Update May 10, 2012 Andrew J. Buckler, MS Principal Investigator, QI-Bench. With Funding Support provided by National Institute of Standards and Technology. Resources are needed to address widening gap in imaging capability as practiced vs. capability of modern medicine. 2. 2. 2. - PowerPoint PPT Presentation
Citation preview
May 2012 UpdateMay 10, 2012
Andrew J. Buckler, MSPrincipal Investigator,
QI-Bench
WITH FUNDING SUPPORT
PROVIDED BY NATIONAL
INSTITUTE OF STANDARDS AND
TECHNOLOGY
Resources are needed to address widening gap in imaging capability as practiced vs. capability of modern medicine
2222222
Example: Beyond Anatomy to Palette of Functional Measures
33333333
glucose metabolism
Biologic Target
bone formation
proliferation
hypoxia
amino acid metabolism
angiogenesis
receptor statusapotosis
18F-FDG18F-NaF
18F-FLT18F-
FACBC
18F-FMISO
18F-XXX
DCE-MRI
PET
18F-FES
QI-Bench is a resource that may be used by single sponsors, defined-entity consortia, or true open science programs
4444444
Example: COPD
Either:• To assist individual suppliers
in optimizing their offerings• To assist groups like
COPDgene consortia• To enable open
development such as by QIBA
or flexible mix of these.
55555555
Data warehouse
QI-Bench is composed of building blocks
6666666
Provenance architecture of Iterate
Example: DCE-MRI using Patient, Synthetic, and Phantom Data• Curate, maintain and serve
reference data sets• Execute batch runs over multi-
parameter synthetic data• Characterize performance
77777777
Data warehouse
Provenance architecture of Iterate
MD Andersen receives
phantoms
Parallel Imaging procedure
GE (new) scanner
Phantoms are comparable
Ship phantom A to UPENN
Ship phantom B to U Chicago
Imaging procedureSiemens MR
scanners
Imaging procedurePhilips (new)
scanner
Ship phantom BTo UC Davis
Image procedureGE (old) scanners
Ship phantom A toDuke
Imaging procedurePhilips (old)
Ship phantom A and B back to MD
Anderson
Test bed: CT volumetry method challenge (“3A”)
8888888
1. Median Technologies2. Vital Images, Inc.3. Fraunhofer Mevis4. Siemens5. Moffitt Cancer Center6. Toshiba
Pilot
Pivotal
Investigation 1
Train
Test
Pilot
Pivotal
Investigation
Train
Test
Pilot
Pivotal
Investigation
Train
Test
Pilot
Pivotal
Investigation n
Train
Test
Pr im
ar y
Se co n dar y
• Defined set of data• Defined challenge• Defined test set policy
Some of the Participants7. GE Healthcare8. Icon Medical Imaging9. Columbia University10. INTIO, Inc.11. Vital Images, Inc.
Broader capability: Systematic qualification of CT volumetry
99999999
1. Median Technologies2. Vital Images, Inc.3. Fraunhofer Mevis4. Siemens5. Moffitt Cancer Center6. Toshiba
Some of the Participants7. GE Healthcare8. Icon Medical Imaging9. Columbia University10. INTIO, Inc.11. Vital Images, Inc. PROFILE Authoring and Testing
Inter-analysis technique (algorithm) variability (3A)
Correlation with clinical endpoints and outcomes (3B)
machine view human expert
view
Transformation
Modality Environment
Therapy Decision
Environment
Patient Patient
Transformation Transformation
FeedbackTherapy-Machine Human Observer
0.0
0.2
0.4
0.6
0.8
1.0
% R
each
ing Pa
rtial R
espo
nse
0 42 84 126 168 210 252 294 336 378
Time on Trial (days)
SLDVolumetric
Survival Plot
Intra- and inter-reader variability (1A)
Minimum detectable biological change (1B)
5 readers, 3 reads
each
Expl
ore
figur
es-
of-m
erit
and
QC
proc
edur
es
Update: Iterate progress• Added small workflows to Iterate
demonstration server.• Stored and retrieved data on server
via workflows.• Distinction between workflow
execution on server and local workstation.
1010
Update: Iterate coming attractions• Upgraded base software installation.• New site with workflows for local
workstation.• Server workflows that produce
outputs.
1111
Analyze performs the statistical analyses…Subject Predicate Object
A Is Patient
A isDiagnosedWith DiseaseA
DiseaseA Is NonSmallLCellLunCancer
A hasClinicalObservation
B
B Is TumorShrinkage
C Is Patient
C hasClinicalObservation
B
D hasClinicalObservation
B
Pazopanib Is TyrosoineKinaseInhibitor
A isTreatedWith Pazopanib
A hasOutcome Death
C hasOutcome Survival
Subject Predicate Object
CT images Tumor
Volumetry analyzes CT
<compliant>LongitudinalVolumetry
estimates TumorSizeChange
TumorSizeChange predicts CytotoxicTreatmentResponse
TyrosoineKinaseInhibitor is CytotoxicTreatment
well-controlled Phase II and III efficacy studies
uses CytotoxicTreatmentResponse
CytotoxicTreatment influences NonSmallCellLungCancer
CT images Thorax
Thorax contains NonSmallCellLungCancer
regulatory drug approval dependsOn PrimaryEndpoint
well-controlled Phase II and III efficacy studies
assess PrimaryEndpoint
CT Volumetry is SurrogateEndpoint for CytotoxicTreatment
1
3
2
1212
…and adds the results to the knowledgebase (using W3C “best practices” for “relation strength”).
Subject Predicate Object
45324 biasMethod <r script used>
45324 bias <summary statistic>
45324 variabilityMethod <r script used>
45324 variability <summary statistic>
9956 <correlation>Method <r script used>
9956 correlation <summary statistic>
9956 <ROC>Method <r script used>
9956 ROC <summary statistic>
98234 Effect of treatment on true endpoint <value>
98234 Effect of treatment on surrogate endpoint <value>
98234 Effect of surrogate on true endpoint <value>
98234 Effect of treatment on true endpoint relative to that on surrogate endpoint
<value>
Subject Predicate Object
CT images Tumor
Volumetry analyzes CT
<compliant>LongitudinalVolumetry
estimates TumorSizeChange
TumorSizeChange predicts CytotoxicTreatmentResponse
TyrosoineKinaseInhibitor is CytotoxicTreatment
well-controlled Phase II and III efficacy studies
uses CytotoxicTreatmentResponse
CytotoxicTreatment influences NonSmallCellLungCancer
CT images Thorax
Thorax contains NonSmallCellLungCancer
regulatory drug approval dependsOn PrimaryEndpoint
well-controlled Phase II and III efficacy studies
assess PrimaryEndpoint
CT Volumetry is SurrogateEndpoint for CytotoxicTreatment
1
3
2
URI=45324
URI=9956
URI=98234
1313
1414
Demonstration of statistical analysis using R within Iterate• Obtain data• Produce summary plots and statistics• Perform analyses
Provenance architecture of Iterate
Conceptual Example• Example data based on QIBA 3A layout• Phantom study
– 5 lesions, each read by 3 readers• The study design enables us to answer several questions
– Is there a difference among readers on volume? • If so, which are different?
– Is there a difference among readers on bias? • If so, which are different?
• Appropriate figures flow from the study design and enable us to visually understand the data
151515151515
DatalesionID readerID trueVolume readVolume
2 lstk 524.7 520.24 lstk 527.4 508.3
10 lstk 431.6 435.239 lstk 280.5 299.663 lstk 439.8 457.8
2 reader1 524.7 560.74 reader1 527.4 504.7
10 reader1 431.6 438.139 reader1 280.5 306.963 reader1 439.8 433.5
2 reader2 524.7 507.34 reader2 527.4 495.1
10 reader2 431.6 396.939 reader2 280.5 258.863 reader2 439.8 421.2
171717171717
181818181818
191919191919
202020202020
212121212121
summary1
222222222222
summary2
232323232323
tukey1
242424242424
tukey2
252525252525
262626262626
27272727272727
Up and running now for you to use
2828
Value proposition of QI-Bench• Efficiently collect and exploit evidence establishing
standards for optimized quantitative imaging:– Users want confidence in the read-outs– Pharma wants to use them as endpoints– Device/SW companies want to market products that produce them
without huge costs– Public wants to trust the decisions that they contribute to
• By providing a verification framework to develop precompetitive specifications and support test harnesses to curate and utilize reference data
• Doing so as an accessible and open resource facilitates collaboration among diverse stakeholders
2929
Summary:QI-Bench Contributions• We make it practical to increase the magnitude of data for increased
statistical significance. • We provide practical means to grapple with massive data sets.• We address the problem of efficient use of resources to assess limits of
generalizability. • We make formal specification accessible to diverse groups of experts that are
not skilled or interested in knowledge engineering. • We map both medical as well as technical domain expertise into
representations well suited to emerging capabilities of the semantic web. • We enable a mechanism to assess compliance with standards or
requirements within specific contexts for use.• We take a “toolbox” approach to statistical analysis. • We provide the capability in a manner which is accessible to varying levels of
collaborative models, from individual companies or institutions to larger consortia or public-private partnerships to fully open public access.
3030
QI-BenchStructure / Acknowledgements• Prime: BBMSC (Andrew Buckler, Gary Wernsing, Mike Sperling, Matt Ouellette)
• Co-Investigators– Kitware (Rick Avila, Patrick Reynolds, Julien Jomier, Mike Grauer)– Stanford (David Paik)
• Financial support as well as technical content: NIST (Mary Brady, Alden Dima, John Lu)
• Collaborators / Colleagues / Idea Contributors– Georgetown (Baris Suzek)– FDA (Nick Petrick, Marios Gavrielides) – UMD (Eliot Siegel, Joe Chen, Ganesh Saiprasad, Yelena Yesha)– Northwestern (Pat Mongkolwat)– UCLA (Grace Kim)– VUmc (Otto Hoekstra)
• Industry– Pharma: Novartis (Stefan Baumann), Merck (Richard Baumgartner)– Device/Software: Definiens, Median, Intio, GE, Siemens, Mevis, Claron Technologies, …
• Coordinating Programs– RSNA QIBA (e.g., Dan Sullivan, Binsheng Zhao)– Under consideration: CTMM TraIT (Andre Dekker, Jeroen Belien)
3131