Abnormality Detection in Carotid Ultrasounds with Convolutional NetworksE. Sabri Eyuboglu , Geoffrey Angus, Pierce Freeman, Bhavik Patel, Mu Zhou, Katie Shpanskaya, Kristen Yeom, Matthew Lungren

Image Localization

Convolutional Layer Filters: 32, Size: 3x3, Stride:1, Padding: 1

Max. Pool Layer

Batch Norm. Layer

x2

Fully Connected Layer Hidden Units: 4

Fully Connected Layer Hidden Units: 128

Abnormality Detection

Aggregator FC & Sigmoid

Encoder (DenseNet, VAE, etc.)

σ

1 Exam ≈ 20 Images

Introduction

Dataset

Variational Auto-Encoder

Results

Conclusion & Future WorkSparsity Loss

[ { “minX”: 15.5, “minY”: 20.3, “maxX”: 300.2, “maxY”: 263.4 }, ...

]

ℓ(Y (i), ̂Y (i)) = CE(Y (i), ̂Y (i)) + α|X (i)|

∑j

CE(0, ̂Y (i)j )

Carotid artery stenosis is a common disease responsible for roughly 25% of all strokes. In carotid stenosis, plaque deposits in the carotid artery narrow the vessel and reduce or block blood flow.

We present two deep learning methods for automating gray-scale carotid ultrasound screening:

1. An object localization model that crops-out extraneous graphical and textual information in gray-scale ultrasounds.

2. A convolutional neural network trained to detect signs of stenosis in gray-scale carotid ultrasounds. We present a robust analysis of current architectures and their shortcomings in the context of this dataset.

21,000 grayscale Ultrasound Screen-captures

Screen-captures include the actual ultrasound image surrounded by extraneous textual and graphical features.

In order to crop the ultrasound image from the 21,000 screen-captures, we trained an image localization model. We built a GUI to label a training dataset for the model.

The dataset was labeled at the exam-level. In an abnormal exam, only around 10% of the images will show signs of stenosis. Thus, our dataset presents a challenging multiple instance learning problem with sparse sets.

We utilized Sparsity Loss to discourage misclassification.

Our activation saliency maps seem to demonstrate a learned focus on the borders around the main artery. This lent credibility to our hypothesis that the model was learning some patterns about plaque deposits.

Further activations are scattered around the image, which seem like noise that the model overfits to.

Ultrasound Angle 1 Ultrasound Angle 2 Ultrasound Angle 2

32

Image Localization Abnormality Detection

92.1% Validation IOU

We achieved a 92.1% validation Intersection Over Union (IOU), which measures overlap of predicted region with ground-truth.

Val. Test

Average Merge 56.0% 48.0%

Maximum Merge 65.0% 60.0%

RNN Merge 69.8% 50.0%

Our models showed a tendency to overfit, regardless of the regularization methods we used (L2, Dropout, Data Augmentation, etc.)

According to the team at AIMI, the localization model we built is capable of completing a task in seconds that previously took months to complete by hand.

We also presented a robust analysis of several CNN architectures for abnormality detection. The success of future abnormality detection models will likely depend on 1) more labeled data or data labeled at the image level and/or 2) new techniques for handling sparse sets in multiple instance learning problems.

Model Architecture

Metrics

Input Image

Input

Decoded ImageEmbedded Image

DecoderEncoder

Saliency Maps

1st Principal Component

2nd P

rinci

pal C

ompo

nent

The VAE embedded our dataset of images into 32-dimensional space. We further reduced the dimensionality using principal component analysis and plotted the dataset. The dataset clearly organizes into five distinct clusters.