Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Temporal Enhance Ultrasound: A Novel Paradigm to Enable Accurate
TRUS-guided Biopsy and Grading
Shekoofeh Azizi
University of British Columbia, Vancouver, BC, Canada
Outline
TeUS Framework
TeUS Application
In vivo Studies and Technical Developments
Theoretical Derivation of TeUS
Experiments and Simulations − Pathology mimicking simulation
− Tissue Phantoms
− Tissue mimicking phantoms
2
Temporal Enhanced Ultrasound (TeUS)
3
Temporal Enhanced Ultrasound
Cancer
Benign
Feature Learning
Classification
Deep Learning
4
TeUS Applications
5
ex vivo PCa detection [Moradi2009]
Classifying PCa grades [Azizi2016]
Breast cancer diagnosis [Uniyal2013]
In vivo Studies
6
Data Acquisition
Data was acquired at the National Institutes of Health (NIH), Maryland.
MR-Ultrasound fusion (UroNav Invivo Corp.)
Taking biopsy & histological processing
Temporal Enhanced US data
MR Target
197 TRUS-guided biopsy cores from 132 subjects.
7
Method Overview
Training Dataset
20 ROIs
Target
Deep Belief Network (DBN)
Visible Layer Hidden Layers
Feature Space
GS3 GS4
Distribution Learning (F1,F2)
Clustering Model
Trained Deep Network
Clustering
Model
Test Data ?
?
?
?
?
? ?
?
?
?
?
?
?
?
??
?
?
?
?
Feature 1
Feat
ure
2
Cluster of Gleason Pattern 3
Cluster of Gleason Pattern 4
Benign Cluster
S. Azizi et al., “Detection and Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural Networks and Tissue Mimicking Simulation”, MICCAI Special Issue 2017.
8
PCa Grading Results
9
Benign vs. clinically significant PCa: 0.8 Combination Rule:
− Intermediate suspicious level: TeUS (70% of the cores) − High and low suspicious level: MRI (30% of the cores)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Are
a U
nd
er
the
RO
C C
urv
e (
AU
C)
Clustering
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Are
a U
nd
er
the
RO
C C
urv
e (
AU
C)
Clustering
Clustering+mp-MRI
Length of tumor in MR ≥ 2cm
S. Azizi et al., “Detection and Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural Networks and Tissue Mimicking Simulation”, MICCAI Special Issue 2017.
Cancer Likelihood Maps
10
MRI lesion length = 24 mm, GS ≤ 3+4
MRI lesion length = 36 mm, GS ≤ 3+4
Target
MRI lesion length = 17 mm, GS ≥ 4+3
Target
MRI lesion length = 27 mm, Benign Target
Target
Target
(Benign blue, G3 yellow, G4 red) 2nd Sample GS: 4+4
1st Sample GS: 3+3
S. Azizi et al., “Detection and Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural Networks and Tissue Mimicking Simulation”, MICCAI Special Issue 2017.
B-mode TeUS Development
RF-mimicking B-mode TeUS Data
Layer 1
Layer 2
. . .
. . .
Representation Sequence . . .
x1 xT
Benign vs. Cancer
xi = (x1, …, xT)
FC
LSTM cell
FC Fully Connected Layer
Transfer Learning
Bmode TeUS Data RF TeUS Data
We utilize joint information of RF and B-mode data through transfer learning.
S. Azizi et al., “Transfer Learning from RF to B-mode Temporal Enhanced Ultrasound Features for Prostate Cancer Detection”, IPCAI 2017.
11
Classification Results: RF TeUS
12
0
20
40
60
80
100
120
140
TeUS MRI
Nu
mb
er
of
Co
rce
s
Incorrect Prediction
Correct Prediction
Train: RF TeUS data from 84 biopsy cores.
Test: RF TeUS Data from 121 biopsy cores.
S. Azizi et al., “Transfer Learning from RF to B-mode Temporal Enhanced Ultrasound Features for Prostate Cancer Detection”, IPCAI 2017.
Classification Results: RF vs. B-mode TeUS
13
0.79
0.81
0.83
0.85
0.87
0.89
0.91
0.93
0.95
All MR Level Moderate MR High MR
Are
a U
nd
er
the
Cu
rve
RF
B-mode
Train: RF TeUS data from 84 biopsy cores.
Test: RF TeUS Data from 121 biopsy cores.
S. Azizi et al., “Transfer Learning from RF to B-mode Temporal Enhanced Ultrasound Features for Prostate Cancer Detection”, IPCAI 2017.
Classification Results: TeUS RF vs. B-mode
14
For moderate MR level cores (n=86)
Train: RF TeUS data from 84 biopsy cores.
Test: RF TeUS Data from 121 biopsy cores.
S. Azizi et al., “Transfer Learning from RF to B-mode Temporal Enhanced Ultrasound Features for Prostate Cancer Detection”, IPCAI 2017.
Colormaps
S. Azizi et al., “Transfer Learning from RF to B-mode Temporal Enhanced Ultrasound Features for Prostate Cancer Detection”, IPCAI 2017.
15
Why does TeUS carry tissue typing information?
16
17
Cancer
Tissue response = f (Acoustic signal, Tissue microstructure, …)
Cell Nuclei (Scatterers)
Speckle
Benign
Cell Nuclei Speckle
TeUS: Physical Phenomena
Changes in tissue temperature
18
0
0.01
0.02
0.03
0.04
0.05
Trial 1 Trial 2 Trial 3 Trial 4
Temperature increase (°C) in 1 minute Round of eye 1
Round of eye 2
0
0.01
0.02
0.03
0.04
0.05
Trial 1 Trial 2 Trial 3
Temperature increase (°C) in 1 minute Turkey breast 1
Turkey breast 2
Turkey breast 3
TeUS: Changes in tissue temperature
Feature Visualization
Benign Gleason Pattern 3 Gleason Pattern 4
Layer 1: 100 hidden neurons
Layer 2: 50 hidden neurons
Layer 3: 6 hidden neurons
Trai
ne
d D
BN
Bac
k P
rop
agat
ion
Absolute Difference
Low-frequency components
Pulsation?
Visible Layer 50 spectral features
19
TeUS: Physical Phenomena
20
Cancer Benign
Tissue response = f (Acoustic signal, Tissue microstructure,…)
Cell Nuclei (Scatterers)
Speckle Cell Nuclei Speckle
S. Bayat et al., “Investigation of Physical Phenomena Underlying Temporal Enhanced Ultrasound as a New Diagnostic Imaging Technique: Theory and Simulations”, UFFC (submitted)
S. Bayat et al., “Tissue mimicking simulations for temporal enhanced ultrasound-based tissue typing“, SPIE 2017.
• Analytical representation of TeUS:
I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆
𝜕𝑥𝑥0 K/E sin (ωt)+n
* Lateral (mm)
Lateral (mm)
PSF(x)
I(x)
S(x)
21 S. Bayat et al., “Investigation of Physical Phenomena Underlying Temporal Enhanced Ultrasound as a New Diagnostic Imaging Technique: Theory and Simulations”, UFFC (submitted)
S. Bayat et al., “Tissue mimicking simulations for temporal enhanced ultrasound-based tissue typing“, SPIE 2017.
Theoretical Derivation of TeUS
• Analytical representation of TeUS:
I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆
𝜕𝑥𝑥0 K/E sin (ωt)+n
* Lateral (mm)
Lateral (mm)
PSF(x)
I(x)
S(x)
22
Theoretical Derivation of TeUS
Conventional tissue characterization
• Analytical representation of TeUS:
I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆
𝜕𝑥𝑥0 K/E sin (ωt)+n
* Lateral (mm)
Lateral (mm)
PSF(x)
I(x)
S(x)
23
Theoretical Derivation of TeUS
Elastography
• Analytical representation of TeUS:
I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆
𝜕𝑥𝑥0 K/E sin (ωt)+n
* Lateral (mm)
Lateral (mm)
PSF(x)
I(x)
S(x)
24
Theoretical Derivation of TeUS
TeUS
Pathology Mimicking Simulations
• Data 14 H&E whole-mount slides (10 patients) scanned at 20x magnification using a ScanScope XT scanner (Aperio / Leica) obtained at U Colorado
26
Slide annotated by pathologist
Data
27
0
20
1 m
m
328 mm
GS4
GS5
Benign
Pathology Mimicking Simulations
Feature Extraction
Finite Element Simulations
Nuclei Location Extraction
Digital Pathology
Ultrasound Simulations (Field II)
Time
Pathology Mimicking Simulations
28
K. Iczkowski, et al., "Digital quantification of five high-grade PCa patterns, including the cribri-form pattern, and their association with adverse outcome", American Journal of Clinical Pathology (2011). (Colorado University )
S. Bayat, et al., “Tissue mimicking simulations for temporal enhanced US-based tissue typing”, SPIE 2017.
Cancer Normal
• Plot of P*S’ for fixed E
29
GS4
GS5
GS5
GS4
Benign
Benign
0
20
1 m
m
328 mm
GS4
GS5
Benign
Validation of Theoretical Derivation
Finite element modelling + Field II simulation was used
30
0
20
1 m
m
328 mm
GS4
GS5
Benign
In all cases, benign and cancer are separable.
Validation of Theoretical Derivation
Simulation Results
31
Simulation Results 1 Hz
Excitation
Backpropagation of feature from Neuron 1
S. Bayat et al., “Tissue mimicking simulations for temporal enhanced ultrasound-based tissue typing“, SPIE 2017.
S. Azizi et al., “Detection and Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural Networks and Tissue Mimicking Simulation”, MICCAI Special Issue 2017.
Tissue Phantom Analysis
Samples
25
3D printed phantoms
4.5x3.5x0.5 cm phantom containing
aluminum oxide particles
4.5x3.5x0.5 cm phantom containing
viable liver cancer cells
Fabricated using Aspect Biosystems unique Lab-on-a-printerTM
Samples
34
Normal cells - Human aortic smooth muscle cells [T/G HA-VSMC] - 18 micrometer. - cell concentration of 1 million cells/mL of the phantom. - Approximately 8 million cells soon after printing. - Spherical in shape.
Cancer cells - Human hepatocellular carcinoma cell [HEPG2] - 18 micrometer. - cell concentration of 1 million cells/mL of the phantom. - Approximately 8 million cells soon after printing. - Spherical in shape.
Both cells clump in the phantom as they grow. The cancer cells form more clusters at a faster rate. The growth rate of the cancer cells is faster then the other normal cell.
Data Acquisition
35
Imaging Setting:
Depth = 4, 5 , and 6 cm Focal Point = 2 cm Frequency: 5, 6.6, and 10 MHz FPS: 25 and 51 Hz Dynamic Range: 75, 85 and 100 dB Power: 0, 2, and 4
Number of Samples: 6 - Benign, cancer and no-cell. - Two samples from each category. - For each sample, we have acquired RF data at
4 different planes.
Results
36
37
Tissue Mimicking Phantoms: Elasticity vs. Scatterer size
Experiment Design
• Different scatterer size or elasticity
− 60 μm phantom/1x gelatin vs. 32 μm phantom/1x gelatin
− 60 μm phantom/0.5x gelatin vs. 32 μm phantom/1x gelatin
− Simulating heart beats by vibrating a plastic tubing at 1 Hz
38
Vibration phantoms
US transducer
US phantom
Function generator controlled valve
Gas outlet Gas inlet
2 cm
The valve sends compressed gas at 1Hz frequency
The US transducer registers the
vibration
A circular inflatable SilasticTM tubing of 0.94 mm OD × 0.51 ID mm was extended through the phantom
The maximal tubing expansion was estimated by a thin wall approximation to be around 15.5 μm at a vibration level of 65 Psi.
39
Data Acquisition
a) b) c)
d)
40
e)
B-mode frames Scatterer size experiment b) 60 μm phantom/1x gelatin c) 32 μm phantom/1x gelatin Elasticity experiment d) 32 μm phantom/1x gelatin* e) 32 μm phantom/0.5x gelatin
a) The clamp holds the transducer in place and remain still during each TeUS acquisition.
Comparing the 1 Hz peaks across the phantoms
41
0
500
1000
1500
2000
2500
3000
0 psi 25 psi 40 psi 50 psi
FFT
Po
we
r A
mp
litu
de
s at
1 H
z (a
u.)
1x gelatin/60 µm scatterers 1x gelatin/32 µm scatterers
1x gelatin/32 µm scatterers* 0.5x gelatin/32 µm scatterers
The phantoms responded to the increase in the vibration amplitudes. At the baseline, the 1 Hz peaks were not distinguishing the phantoms. At higher vibration levels, the larger response resulted from smaller scatterer size and
less elastic phantoms.
Analytical Derivations: TeUS and Elastography complement each other!
42
Elasticity
50
0 m
icro
ns
?
Analytical Derivations: TeUS and Elastography complement each other!
43
?
Iczkowski, et al. 2011
Elasticity
Implications of Analytical Derivations: TeUS and Elastography complement each other!
44
PSF*S’
I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆
𝜕𝑥𝑥0 K/E sin (ωt)+n
Elasticity
Cancer Type I
Cancer Type II
Summary and Future Works
23
In an in vivo study including 197 TRUS-guided biopsy:
− AUC of 0.8 in separation of clinically significant PCa from normal tissue type.
Tissue micro-structure, induced by mechanical excitation, can be used to distinguish benign and cancer.
TeUS captures a combination of spatial changes in scattering function and elasticity.
Future Works:
− Verify in larger population. − Further investigation of the underlying physical phenomena.
Thank You
Investigator and Collaborators: Dr. Purang Abolmaesumi, UBC Dr. Parvin Mousavi, Queen’s University Dr. Mehdi Moradi, IBM Dr. Amir Tahmasebi, Philips Research Dr. Pingkun Yan, Philips Research Dr. Bradford Wood, NIH Dr. Baris Turkey, NIH Dr. Peter Pinto, NIH Dr. Larry Goldenberg, VGH Dr. Martin Gleave, VGH Dr. Peter Black, VGH Dr. Storey Wilson, Dr. Kenneth A. Iczkowski Dr. Scott Lucia Students/Staff: Dr. Sharareh Bayat Dr. Farhad Imani Dr. Guy Nir Dr. Ajay Rajaram Si Jia Li Samira Sojoudi Nathan Van Woudenberg Siavash Khallaghi Hussam Ashab
46
@azizishekoofeh , UBC, Vancouver, December 2017
48