5/17/2018

1

QIBA CT Small Lung Nodule Profile

Profile Conformance Process and Feasibility Test Example

Ricardo S. Avila

rick.avila@accumetra.com

May 16, 2018

Quantitative Imaging Biomarkers Alliance (QIBA) Annual Meeting

Small Lung Nodule Measurement

= 668 mm3 T2 = 661 mm3

DV = No

Change

Time 1 Time 2

5/17/2018

2

Lung Nodule Volumetric Error

Scan Specification1. Detector Rows2. mAs3. kVp4. Slice Thickness5. Recon Kernel6. Iterative Recon7. Dual Source8. Auto mA & kVp9. …

Detection &Measurement

Sem

i-A

uto

mat

ed

Soft

war

e

CT

Acq

uis

itio

n

AcquiredImagesAcquiredImagesAcquiredImages

Issues1. Large & Growing # Parameters2. Constantly Changing3. Operator Preferences/Errors4. Difficult to Fully Reproduce

Patient Characteristics1. Lesion Size2. Lesion Margin3. Patient Size4. Attachment5. Local Conditions6. High HU Objects7. Metal Implants8. Motion9. …

Challenges1. Internal Structures2. Large Complex Borders3. Changing Presentation4. Very Large Patients

Object Scanner SoftwareVolumetric

Performance

1 2 3

JMI 2016

Free CT Image Quality Report (2015-Present)

Accumetra.com

Email

Tech Can Do

The Scan In

< 5 Minutes

Optimize

5/17/2018

3

Spatial Warping

0 mm100 mm200 mm

CT Lung Screening Protocol Guidelines

Detectors

>=

Thickness

<=

Spacing

<=Kernel

16 1.25 1.25Highest

Res.

64 1.25 1.25Highest

Res.

16 1.0 0.7No

Pref.

162.5,

1.0 pref.

No

Pref.

No

Pref.

162.5,

1.0 pref.

2.5,

1.0 pref.

Range,

Not Easy

CT Acquisition

2015 European Society of Radiology

2015 American College of Radiology

(10 Pillars Publication)

2016 RSNA/QIBA Small Nodule Profile (19% to 42%)

2016 AAPM Lung Cancer Screening Protocols

2016 I-ELCAP Guidelines

Our Specification: >= 16 detector rows, <=1.25 thickness , <=1.25 spacing

5/17/2018

4

Detection Slice Thickness & Recon Kernel

Slice

ThicknessSites

Soft

Recon

Medium

Recon

Edge En.

Recon

<= 0.625 4 (15%) 0 3 1

0.8, 1.0, 1.25 12 (46%) 6 2 4

>= 1.5 10 (38%) 6 3 1

3 used 2mm ST &

1mm spacing

QIBA CT Small Lung Nodule Profile

5/17/2018

5

Small Lung Nodule Profile Claims

Overview: The profile addresses accuracy/precision of CT volumetry for solid

lung nodules 6-10 mm in diameter.

Profile Claim 1: For a measured nodule volume of Y, and a CV as specified in

the table below, the true nodule volume is:

Y ± (1.96 Y CV), with 95% confidence.

Profile Claim 2: A measured change in nodule volume of X% indicates that a

true change in nodule volume has occurred if:

X > (2.77 x CV1 x 100), with 95% confidence.

To quantify the amount of change, if Y1 and Y2 are the volume measurements at

the two time points, and CV1 and CV2 are the corresponding values from the

table below, then the 95% confidence interval for the true change is:

(Y2-Y1) ± 1.96 ([Y1 CV1]2 + [Y2 CV2]2).

CV Table

5/17/2018

6

Profile Requirements & Steps

• CT Scanner

– >= 16 Slice

– Model has been verified to

be QIBA Compliant

– ACR CT accreditation

• CT Protocol

– <= 1.25mm slice thickness

– Slice spacing <= slice

thickness

– Pitch < 2.0

• Fundamental Image

Properties

– Edge Enhancement <= 5%

– 3D PSF Ellipsoid Volume

<= 1.5mm3

– PSF Z/X Aspect <= 2.0

– Linearity Bias < 35 HU

– Image Noise <= 50 HU SD

– Spatial Warping <= .3 mm

• Nodule Analysis

Software

– Software has been verified

to be QIBA Compliant

No Bias,

CV limits

Resolution: 3D PSF Sigma Ellipsoid Volume

3D Gaussian PSF along with

3D sampling rate represents the

resolution of the system

2D

Gaussian

1D

Gaussian

3D

Gaussian

5/17/2018

7

We are saying…

If you follow the profile

specifications and requirements

Your volumetric change error

should fall within this 95% CI

Conformance Without Tools Would Be

Extremely Challenging For Many

Clinical Sites

0.625 ST

CT Image Quality Compliance

Maintain Throughout The Full Scanner Field of View

1. Artifacts

2. 3D Distance

3. Linearity

4. Image Noise

5. 3D Resolution

6. 3D Edge Enhancement

7. 3D Spatial Warping

AC

R/G

amm

ex&

M

anu

fact

ure

rP

han

tom

Ch

eck

s

Tech

no

logi

st:

Fast

& L

ow

Co

st P

han

tom

Au

tom

ate

d P

roto

col C

he

ck

Scanner Is GenerallyOperating OK

Rad

iolo

gist

: Lo

oks

Fo

r A

ny

Issu

es

(mai

nly

no

du

le, n

ois

e, a

rtif

acts

)

Scanner/Protocol/ScanIs OK

Scanner/ProtocolAchieves Specifications

5/17/2018

8

A QIBA Small Lung Nodule Phantom

Teflon (~950 HU)

Cylinder

Acrylic (~120 HU)

Cylinder

Delrin (~340 HU)

Concentric Cyl

Air (-1000 HU)

Room For

Other

Compartments

At Iso-Center

200 mm from

Iso-Center

Cost:

$200

CTLX1 Phantom

5/17/2018

9

RSNA/QIBA Conformance Certification Pilot Project

Using Cloud-Based Computing Services

http://quality.rsna.org

Email

Optimize

Guidance

Webpages &

FAQs

Check Each

Time Scanner

or Protocol

Changes and

Once Per Year

~ 100 CTLX1 Phantoms Are

Being Globally Distributed

• Confirms Fundamental CT Image Properties

– 3D Resolution:

• 3D PSF Ellipsoid Volume <= 1.5 mm3

– 3D Resolution Aspect:

• PSF Z/X <= 2.0

– Linearity Bias:

• Air and Acrylic Bias < 35 HU

– Image Noise:

• Acrylic Noise <= 50 HU SD

– Recon Kernel Edge Enhancement:

• Air to Delrin Enhancement <= 5%

– 3D Spatial Warping:

• Delrin Cylinder RMSE <= 0.3 mm

• Lung Nodule Volume Change Performance

– Verifies That Image Quality Meets or Exceeds

The QIBA CT Lung Nodule Profile Volume

Change Measurement Claims

PCF

Funding

5/17/2018

10

International CT Image Quality Monitoring

60 CTLX1 Phantoms Sent Out As Of 5/17/2018

Data Received & Analyzed From:

• ~30 Sites

• ~50 Unique CT Scanners

• > 200 CT Scans

• 4 Manufacturers

• Siemens, GE, Philips, Toshiba

• > 20 Different Scanner Models

Automated CT Scanner/Protocol Image Quality Report

Image Quality

Measurements

Analysis

Status

Protocol

Settings

5/17/2018

11

Report With Issues

Quantitative Conformance Data

5/17/2018

12

QIBA SLN Conformance Certification

Pilot Project

Requested vs Observed Slice Thickness

5/17/2018

13

Research: Dose and Resolution Study

Software Conformance Certification Testing

Perform Volume Measurements on:

1. Clinical Zero Change Datasets

• Evaluate measurement precision in real nodules

2. CTLX1 Phantoms With Embedded Ellipsoids

• Evaluate measurement precision, bias, and linearity in

synthetic objects

• As a bonus we can look at volume measurement prediction

performance, but not needed for software conformance

5/17/2018

14

Clinical Zero Change Testing

• All datasets will be acquired with a conformant CT acquisition protocol

and a medium level of CT image quality characteristics (verified)

• 6.00 to 7.99 mm nodule– 15 solid nodule cases each with 5 repeat CT scans

– wCV in profile is 0.23

– Max allowable wCV by an actor is 0.19 to be 95% confident

• 8.00 to 9.99 mm– 15 solid nodule cases each with 5 repeat CT scans

– wCV in profile is 0.16

– Max allowable wCV by an actor is 0.14 to be 95% confident

• 10.00 to 11.99 mm (since our table goes up to 12 to support growth)– 15 solid nodule cases each with 5 repeat CT scans

– wCV in profile is 0.12

– Max allowable wCV by an actor is 0.10 to be 95% confident

Data

Collected,

About To Be

Measured

Example Zero Change Dataset (6.2mm)

1 2 3

4 5

5/17/2018

15

Bias Assessments

• Claim 1 in the Profile assumes no bias.

• Must verify that actor’s bias is <5%.

• With 3 replicate scans of the CTLX1S phantom,

we can estimate actor’s bias to within +1%.

• So even if an actor’s bias is 4%, we can verify,

with 95% confidence that their bias is <5%.

• Repeat this for each of the 5 ellipsoid sizes.

Measurement Linearity and Slope

• From the CTLX1S phantom data, actors will perform a

regression analysis.

𝑌 = 𝛽𝑜 + 𝛽1𝑋 + 𝛽2𝑋2

• They need to provide an estimate of the quadratic term, 𝛽2 .

This should be <0.5.

• They need to provide a 95% CI for the linear term, 𝛽1 .

• We have assumed that 𝛽1 equals one so the 95% CI for the

slope needs to be completely contained in the interval 0.95 to

1.05.

5/17/2018

16

CTLX1-S

CTLX1 w/Ellipsoid Modules Scan & Measure

16 Ellipsoids 7.0 x 6.0 x 6.0 mm

16 Ellipsoids 6.0 x 3.6 x 3.6 mm

• Will CT scan the enhanced CTLX1 phantom 3 times using a medium

level of CT IQ characteristics (resolution, sampling, noise, etc.)

• Ellipsoids will be volume measured and mean/stdev calculated

• Require 1 site for conformance of software name/version

• We should try to get this data from at least two sites using same

phantom

16 Ellipsoids 10.0 x 6.0 x 6.0 mm

16 Ellipsoids 9.0 x 5.4 x 5.4 mm

16 Ellipsoids 8.0 x 4.8 x 4.8 mm

5/17/2018

17

CTLX1-S

Scanner Conformance Certification

Design of Experiments

• Helps Prevent Issues In The Clinical Environment By Allowing

Vendors To Define Parameter Operating Envelopes

• Focuses On High Performance Risk Parameters Such As mA

and kVp

Slice Thickness

Tu

be

Cu

rre

nt

Slice Thickness

Tu

be

Cu

rre

nt

SingleOperating

Point

WiderOperatingEnvelope

5/17/2018

18

Scanner Conformance Certification

Design of Experiments

• CT Scanner DOE

Provide Recommended

CT Scanning Protocol

Test the Protocol Over

an Operating Envelope

Future Tools

5/17/2018

19

Next Steps

• Finish Up SLN Conformance Pilot Project Resources

– Pilot Project Conformance Certification Website

– Post Datasets & Spreadsheets

• Perform Final Technical Confirmation Testing

– Clinical Conformance : Launch Guidance Pages

– Software Conformance : CTLX1S Scan and Measure

– CT Scanner Conformance : DOE Testing

Thank You