Highly Accurate Brain Stroke Diagnostic System and

Generative Lesion Model

Junghwan Cho, Ph.D.CAIDE Systems, Inc.

Deep Learning R&D Team

� Established in September, 2016 at 110 Canal st. Lowell, MA 01852, USACEO & Founder: Jacob Kyewook Lee

Employees : 6 Contact: caideinfo@caidesystems.com

� Our Mission Save human lives by developing Cognitive Artificial Intelligence Disease Detection Systems. Provide protection of human life and equal access to health care and treatment through artificial intelligence technology.

� Our Goals Eliminating human errors and reducing delayed diagnosisDeveloping the most reliable AI system for analyzing images (ultra sound, MRI, CT and X-ray), electronic medical records, and genome data.

� Available PositionLooking for Talented Research Scientist or Engineer

With CAIDE, Better and Healthier Life!

http://www.caidesystems.com

Outlines

§CAIDE Diagnostic System for Brain Stroke§Stroke Classification/Stroke Lesion Segmentation §Stroke Lesion Generative Network§Demo- CAIDE m: Studio BSR

With CAIDE, Better and Healthier Life!

§ Brain attack/accident

§ Up to 2 million brain cells die every minute.

§ About 795,000 people suffer from stroke every year in US.

§ More than 137,000 people (17% of all strokes) die from

the stroke, with a cost of approximately $76.3 billion.

*Source image from http://www.stroke.org/understand-stroke/what-stroke/stroke-facts

Brain Stroke

Ischemic Stroke (Blood Blockage) Hemorrhagic Stroke (Bleeding)

CT Findings on Intracranial Hemorrhage Types

Epidural (EDH) Subdural (SDH)

Subarachnoid (SAH)Intraparenchymal (IPH) Intraventricular (IVH)

IPH+IVH

CAIDE Diagnostic SystemDICOM Files--->Gray Scale (on Window Level/Width)

VS

Hemorrhagic(bleeding) No Bleeding

False Negative

NO

NOPositive?(Bleeding)

CT Images

Preprocessing

CNN1- Classifier(Default Window)

CNN2- Classifier(Stroke Window)

Stroke LesionDelineation (FCN) Positive?

YES

YES

END

Cascaded CNN ClassifiersModel 1

Model 2

Sensitivity (Recall) vs Specificity

Cascaded CT window for increasing sensitivity while preserving specificity

>>

Sensitivity Specificity

False Positives Rate (1- Specificity)

0 0.05 0.1 0.15 0.2 0.250.85

0.9

0.95

1ROC for Classification

True

Pos

itive

s R

ate

(Sen

sitiv

ity)

ROC for Classification

(0.991, 0.961)= (Specificity, Sensitivity)

(0.953, 0983)

Thp>=0.5

Thp>=0.2

Default Window vs Stroke Window Setting

50/100 (WL/WW) 40/40

Ground Truth

• Narrow window width (high-contrast)• Increase detection of subtle abnormalities

50/100

Default Brain Window Stroke Window

Turner, P. J., and G. Holdsworth. "CT stroke window settings: an unfortunate misleading misnomer?." The British journal of radiology 84, no. 1008 (2011): 1061-1066.

Training for Cascaded CNN Classifier (Bleeding or not)• Total data sets- 5,647 patients (3,000 no bleeding vs 2,647 bleeding)

o2D axial CT images with 512x512 size • 5-fold cross validation• Trained cascaded CNN model

oTwo different training solvers: Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (ADAM)

oScratch vs fined tuned using pre-trained model• Hardware computer: NVIDIA DGX-1 with 8 Tesla V100

Evaluation- Classification- (top 1- accuracy) – after 15 epoch

1 2 3 490

92

94

96

98

100

SGD-Scratch

SGD-Fine-tuned

ADAM-Scratch

ADAM-Fine-tuned

("#. "" ± &. '%)

("*. *+ ± &. ,%)

("-. ". ± &. .%) ("'. +' ± &. +%)%

Evaluation – Cascade CT Window Increasing Sensitivity

1 20

50

100

150

200

250

300

350

50/100(WL/WW)

50/100+40/40 (Cascaded)

# of

False

Neg

ativ

e CT

Imag

es

%

1 2 3 495

96

97

98

99

100

50/100+40/40

50/100 50/100+40/40 (Cascaded)

Specificity

Sensitivity

50/100(WL/WW)

Outlines

§CAIDE Diagnostic System for Brain Stroke§Stroke Lesion Segmentation§Stroke Lesion Generative Network§Demo- CAIDE m: Studio BSR

Encoder-Decoder Architecture- for sematic image segmentation

• Encoder

§ Feature extraction (Convolution)

§ Dimensional reduction (Pooling)

§ VGG 16 or ResNet

• SegNet, U-Net, and Fully Convolutional Network (FCN)

• Decoder

§ High resolution from low resolution

§ Unpooling/up-sampling with

transposed convolution (deconvolution)

Source image from "Segnet”, IEEE transactions on pattern analysis and machine intelligence 39, no. 12 (2017): 2481-2495.

Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully convolutional networks for semantic segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.

14

Fully Convolutional Network (FCN) for Stroke Lesion Segmentation

Pool4 prediction

2x upsampled

SUM

FCN-8s

Pool3 prediction

2x upsampled

SUM

8x upsampled

Soft

max

1 2 3 4 5

VGG16 Network

FCN Training• Total data sets- 2,647 patients (corresponding 33,391 well labeled images)• 5-fold cross validation• Fully convolutional network with ADAM solver using pre-trained model• NVIDA DGX-1 (8 V100 GPU)

1 2 3 4 50

2000

4000

6000

8000

10000

12000

14000

IPH IVH EDH SDH SAH

Histogram of Hemorrhagic Stroke Type

IVH DCIVH=.86

IPH, SAH DCIPH=.89, DCSAH=.77

Segm

ente

d Re

sults

by

FCN

8s

afte

r 50

Epoc

h Tr

aini

ng

SAH, SDH DCSAH=. 84

EDH, SDH DCEDH=.89, DCSDH=.54

SDH: False Negative

SAH: False Positive

Segm

ente

d Re

sults

by

FCN

8s

afte

r 50

Epoc

h Tr

aini

ng

0 1 2 3 4 5 650

60

70

80

90

100

0 1 2 3 4 5 650

60

70

80

90

100

Performance Evaluation

IPH IVH EDH SDH SAH

% %

IPH IVH EDH SDH SAH

#FP: Number of Pixels Falsely Positive Segmented

#FP>300

#FP>200

#FP>100

Precision Recall, Sensitivity

DC>5%

DC>25%

DC>50%

Precision=TP/(TP+FP) Recall=TP/(TP+FN)

DC: Dice Coefficient

Outlines

§CAIDE Diagnostic System for Brain Stroke§Stroke Classification/Stroke Lesion Segmentation §Stroke Lesion Generative Network§Demo- CAIDE m: Studio BSR

Generative Adversarial Networks (GANs)• Two networks competing against each other in a zero sum game

Source Image from https://twitter.com/ch402/status/793911806494261248

Source Image from https://www.slideshare.net/ckmarkohchang/generative-adversarial-networks

(z)

(x)

The discriminator (D):Distinguish real data from fake created by the generator

The generator (G):Learn distribution of the data from random noise, in an attempt to fool the discriminator

Image to Image Translation- for Generating Stoke Lesion Images

Source image from, Phillip, et al. "Image-to-image translation with conditional adversarial networks." arXiv preprint (2017)

l Apply to map stoke lesion labels to corresponding lesion image.l Stoke lesion masks (segmented regions) - conditional input images to the

Generator (G) as well as Discriminator (D)

Stoke Lesion Labels

Lesion Generated Image

Target (Stroke Lesion)

GAN for Generating Stoke Lesion Images

Training Pix2Pix-Tensorflow

l Trained conditional GAN below conditionsq Total data set : 2,647 patients (corresponding 33,391 well labeled images)

: 80% training, 20% testing

q Learning parameters: Learning rate =0.0002, L1 weight=100, and GAN weight=1.0

q About 16 hour up to 200 epoch on NVIDIA Tesla V100 (1 GPU)

SDH

SAH

SAH

Input Target Output Input Target Output

Examples of Generated Fake CT Image after 200 Epoch Training

EDH

SAH

SAH, EDH

SAH

IVH, IPH

SAH, IVH

SAH, IVH, IPH

Input Target OutputInput Target Output

IPH, SAH

SAH, IVH

Evaluationl In general, evaluating GANs is difficult

q Loss function makes it harder during trainingq FCN /Inception scores and Amazon Mechanical Turk (AMT by human)

l FCN scores : fake/generated images inferred by FCNl Clarity- threshold blurriness (variance of Laplacian)l Second discriminator- choose more realistic images

Dice: .981 Dice: .976 Dice: .981 Dice: .978

10 epochs 50 epochs 100 epochs 200 epochs

IPH

FCN scores (DICE) vs Image Quality

RecallDICE (FCN Scores)

Evaluation – Recall, DICE l Evaluated by varying:

q Percentage of training data (based on patient number): 2.5, 10, 50, and 100%q Number of epochs : 10, 50, 100, and 200 epoch

Evaluation- Data Augmentation

Original(Real Data)

Original+Augment

Original+2xAugment

Original+3xAugment

CAIDE m: Studio BSR (Demo), Booth #726