Standards to Enable Efficient

Interpretation of Imaging Data

ACR Center for Research and Innovation

1818 Market Street

Suite 1720

Philadelphia, PA

Panelists

Lindsey Dymond, Director of Data Management LDymond@acr.org

Jim Gimpel, Imaging Core Lab Manager JGimpel@acr.org

Charlie Apgar, Chief Operations Director CApgar@acr.org

Amanda Buck, Regulatory Specialist ABuck@acr.org

Adam Opanowski, Imaging Analyst AOpanowski@acr.org

2

3

Staff and Infrastructure

American College of Radiology

37,000 MembersA

ccre

dit

ati

on

Eco

no

mic

s

Rese

arc

h

Pra

cti

ce G

uid

elin

es

Ed

ucati

on

Core Purpose: Serve patients and society by empowering members

to advance the practice, science, and professions of radiological care.

The ACR Data Value Chain

Primary

Research

Developing

Guidance

• Data Collection

• Discovery

• Validation

• R&D

MonitoringEducation &

Training

• Expert

Consensus

• Algorithms

• Standards &

Guidelines

• Accreditation

• RadPeer

• Outcomes

Research

• Registries

• Quality Programs

• Ed Center

• Distributed

Education

Influence &

Adoption

• Publications

• Advocacy

• Economics

• Marketing

People, Process, and Technology Infrastructure

The ACR Data Value Chain

Primary

Research

Developing

Guidance

• Data Collection

• Discovery

• Validation

• R&D

MonitoringEducation &

Training

• Expert

Consensus

• Algorithms

• Standards &

Guidelines

• Accreditation

• RadPeer

• Outcomes

Research

• Registries

• Quality Programs

• Ed Center

• Distributed

Education

Influence &

Adoption

• Publications

• Advocacy

• Economics

• Marketing

People, Process, and Technology Infrastructure

Research Focus: advancing the science of radiology…

ACR Research and Innovation

Clinical Research

• Spans more than 40 years

• Over 500 clinical trials

• 2 million images processed annually

• Established research infrastructure in Philadelphia

16,318 square feet of office space

>130 researchers on staff

Distributed staff and technology

Remote and central interpretation and analysis environment

Passed FDA audits (not for cause) with no deficiencies

6

Services• Study Design

• Protocol, Charter, Imaging Manuals

• Site selection and Qualification

• Training

• Integrated Diagnostics

• Study execution and operational management

• Statistical analysis

• Clinical Trials, Registries, other projects

• Big Data/Archives

7

Evolution of ACR CRI

…as of 2014

Since 2014…

ACR Enterprise Solutions

Robust Data

Ecosystem

Research

Registries

Accreditation

• LCSR

• IR

• CTC

• DIR

35,000 sites

• NOPR

• IDEAS

• Screening/Detection • Novel Imaging Methods • Imaging Biomarkers • Socio-Economic Research

(Neiman Health Policy Research)

• Software/Hardware Validation

• GRID

• ICE

• NMD

Object Store

QA ProcessesTRIAD

Central Processing

PACSSite Submission

Cloud-Based Analytic

Environment (e.g. Imaging

Pipeline)

Data Ecosystem: TRIAD and DART

Advanced Search

DART Portal

DataMapper

Authentication

Authorization

Data Provenance Portal DB

Users

Genomic DataBiospecimen Data

Clinical Data Image MetaData

Integrated Meta Data

DART Integrated Repository

Review Workflow

Image Interpretation & Reporting

Clinical Data / EDCClinical Data /

EDCEMR/ EDC

Standards to Enable Efficient

Interpretation of Imaging DataSite

Selection

Image Acquisition

Image Collection

Image QC

Image Analysis

Assessment

Standards: critical to interpretationLifecycle Step Key Considerations Recap standards and source of the standard

Site selection Qualified to perform the required imaging

Access to Resources (imaging platform, agents, etc)

Access to eligible subjects and referral relationships

Qualified researchers available Protocol, Imaging Charter, FDA Clinical Trial Imaging

Endpoint Process Standards (Guidance for Industry)

Site

qualification

Accreditation

Training

Test image submission

Image

acquisition

Platform specific for the study

Test submission

Protocol, Imaging charter, Imaging manual,

Manufacturer guidelines, QIBA profiles, FDA Clinical

Trial Imaging Endpoint Process Standards (Guidance

for Industry), 21CFR Part 11

Image

Collection

Electronic image transfer

Submission within defined standard of time

Protocol, Imaging Manual, FDA Clinical Trial Imaging

Endpoint Process Standards (Guidance for Industry),

21CFR Part 11, DICOM standards, Guidance for

Industry: Computerized Systems Used in Clinical

Investigations, Guidance for Industry: Electronic

Source Data in Clinical Investigation, QIN

Image QC Study-specific QC methods

Autocapture of data/DICOM header data

Image Analysis Qualitative vs. Quantitative

Timing requirements

Source of Analysis and collection of results

Standards to Enable Efficient

Interpretation of Imaging Data

3 Areas for discussion

• Central Read

• Validated tools to analyze data

• Data access, transfer, and dissemination

Existing Standards:

Protocol, Imaging Charter, Imaging Manual, 21CFR Part 11, FDA Guidance for Industry: Clinical Trial Imaging Endpoint Process Standards, FDA Guidance for Industry: Computerized Systems Used in Clinical Investigations, FDA Guidance for Industry: Electronic Source Data in Clinical Investigation, QIBA, QIN

(1) Central Read

Hypothesis: More efficient interpretation of

imaging data can be achieved through a

central read paradigm which supports remote

readers.

Standards for Central ReadOperational

Considerations/Imaging

Charter

Access to images to read

Software and Hardware

Reader environment

Monitoring the readers

Integration of other data for

presentation

Capture of reader results

Standards for Remote ReadCharacteristic Remote Reader Approach

Access to images to read Controlled access, validated

systems

Software and Hardware Hosted to ensure standardized;

specifications for viewing and

transfer of data

Reader environment Dictated by sponsor/imaging

manual

Monitoring the readers Deploy staff, monitor via

webcam, shadow operator

Integration of other data for

presentation

Selective presentation to ensure

unbiased read

Capture of reader results Direct data entry to EDC, auto-

capture of data

(2) Validated Tools for Analysis

Hypothesis: Machine learning can lead to

increased efficiency in interpretation of

imaging data by expediting the process,

incorporating quantitative analysis tools, and

reduce human variability involved in the

process

DEEP LEARNING OF MEDICAL IMAGESTRAINING PROCESS

Findings

No Findings Untrained Neural Network

Finding 1

Finding 2

Finding 3

Finding 4

Finding 1

Finding 2

Finding 3

Finding 4Unknown

Trained Neural Network

DEEP LEARNING OF MEDICAL IMAGESAPPLICATION PROCESS

AlgorithmZebra has developed an algorithm that

automatically calculates Coronary Calcium

Scores based on standard, non-contrast

Chest CTs. This tool can provide early

detection of people at high risk of severe

cardiovascular events.

What future can we envision?

Define ROI • Algorithm

AIM

Launch QtvApplication

Generate output data

Incorporate into

database

Standards for more validated tools

• Can we validate algorithms/methods developed by the

community?

• What types of studies/analyses could benefit from

such capabilities?

• Is there a potential impact on reader study designs?

Could sufficient variability be eliminated to adopt

single reader model?

how do we ensure that such algorithms

will be accepted by FDA?

(3) Data Access and Transfer

Hypothesis: Data warehouses can create a

more efficient platform for interpretation of

imaging data for analysis, transfer, and

dissemination

Data access and dissemination

standards• Analysis

• Retrospective central read, or

• Concurrent access and reading?

• Transfer to Sponsor/FDA

• File transfer, or

• Access to reports and integrated source data?

• Future Research

• Proprietary or a community good?

• Project Data Sphere

Project Data Sphere

• Project Data Sphere, LLC (PDS), an independent, not-for-profit initiative of the CEO Roundtable on Cancer's Life Sciences Consortium (LSC), operates the Project Data Sphere platform, a free digital library-laboratory that provides one place where the research community can broadly share, integrate and analyze historical, patient-level data from academic and industry phase III cancer clinical trials.

• The Project Data Sphere platform is available to researchers affiliated with life science companies, hospitals and institutions, as well as independent researchers. Anyone interested in cancer research can apply to become an authorized user.

• A goal of the Project Data Sphere initiative is to spark innovation. The Project Data Sphere initiative can help the cancer community unlock the potential of valuable data by generating new insights and opening up a new world of research possibilities.

• The true power of this platform will come from an engaged, diverse and global community focused on advancing future research to improve the lives of cancer patients and their families around the world.

Our Model….

DART Data Store

AIM Module

AIM

Template(CDE Compliant)

Dart Imaging

Pipeline DART

Analytics

Conduct

Image

Review and

Markup

Test Hypotheses

based on

integrated Clinical,

Imaging, and Other

Data Sets

AIM Markup

Stored in

DART

Load AIM

TemplateIdentify

Hypotheses to

test on new and

existing Data

Acquire Images and

Data According to

Protocol

Upload, QC and

Process Imaging

Data

1

23

6

7

8

10

Markup and Secondary

Analysis

RAVETRIAD Populates

RAVE DB and

Forms

4

TRIAD to DART

Imaging Data, RAVE

to DART Clinical Data

5

RaveRave Integrate

Pathology, Omics,

and other Clinical

Data as available

9