The problem State of the art Proposed approach Achieved results

Combined Radiology and Pathology Classification of BrainTumors

Rozpoznanie guza mózgu na podstawie obrazu radiologicznego i patologicznego

Piotr Giedziun

Supervisor:dr hab. inz. Henryk Maciejewski

4 March 2016

Piotr Giedziun Wrocław University of Technology 4 March 2016 1 / 25

The problem State of the art Proposed approach Achieved results

Outline

1 The problem

2 State of the art

3 Proposed approach

4 Achieved results

Piotr Giedziun Wrocław University of Technology 4 March 2016 2 / 25

The problem State of the art Proposed approach Achieved results

Definitions

Cancer

Cancer occurs when abnormal cells grow out of control

Brain tumor

Benign or Malignant

Over time, a low-grade tumor can become a high-grade tumor

Brain tumors are classified as grade I, grade II, or grade III, or grade IV

Piotr Giedziun Wrocław University of Technology 4 March 2016 3 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Survival rate (5 years or more)

FIGURE – Based on data from SEER 18 2005-2011, cancer.gov

Piotr Giedziun Wrocław University of Technology 4 March 2016 4 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Survival by stage

FIGURE – Ovarian cancer, Five-year stage-specific relative survival rates, adults (ages 15-99), Anglia Cancer Net-work, 1987-2008

Piotr Giedziun Wrocław University of Technology 4 March 2016 5 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Diagnosis process

General Practitioner

Neurologist

MR or CT - Radiologists

(tumor confirmed)

Neurosurgeon

Benign

Pathologist

(final diagnosis)

Malignant

2 Pathologists

(final diagnosis)

(clear image)

Piotr Giedziun Wrocław University of Technology 4 March 2016 6 / 25

The problem State of the art Proposed approach Achieved results

Diagnosis problems

Problems

Diverse shapes, sizes andappearances of tumors

Relies on histopathologic examination(biopsy examination)

Waiting for tests and to start treatment

Radiology imaging is used only toestablish location, size and whether itis benign and malignant tumor

Targets in the UK

No more than 2 months wait between thedate the hospital receives an urgent GPreferral for suspected cancer and startingtreatment

FIGURE – Glioblastoma cells

FIGURE – Oligodendroglioma cells

Piotr Giedziun Wrocław University of Technology 4 March 2016 7 / 25

The problem State of the art Proposed approach Achieved results

Diagnosis problems

Problems

Diverse shapes, sizes andappearances of tumors

Relies on histopathologic examination(biopsy examination)

Waiting for tests and to start treatment

Radiology imaging is used only toestablish location, size and whether itis benign and malignant tumor

Targets in the UK

No more than 2 months wait between thedate the hospital receives an urgent GPreferral for suspected cancer and startingtreatment

FIGURE – Glioblastoma cells

FIGURE – Oligodendroglioma cells

Piotr Giedziun Wrocław University of Technology 4 March 2016 7 / 25

The problem State of the art Proposed approach Achieved results

Aims & Limitations

Aims

Research & build a segmentation mechanism for the MRI scans (ROI selection)

Research & build a classifier based on the segmented radiological images

(if possible) Combine the Pathology-based classification with radiology-basedclassifier

Limitations

Limited access to the MRI samples with the diadnosis provided by the doctor

Conservative environment - only non-black box models

Piotr Giedziun Wrocław University of Technology 4 March 2016 8 / 25

The problem State of the art Proposed approach Achieved results

Related work

Brain tumor segmentation

The topic of brain segmentation is relatively popular thanks to BraTS challengeSeveral supervised and unsupervised algorithms were proposed

Random Decision Forest that classifies voxelsFuzzy C-means clusteringMean Shift and K-means clustering

Brain tumor classification

Slightly less popular subject (current diagnosis fully rely on histopathologyimaging)Feature extraction

Extraction of structure informationFeature selection

GLCM (Gray-Level Co-occurrence Matrix)

Piotr Giedziun Wrocław University of Technology 4 March 2016 9 / 25

The problem State of the art Proposed approach Achieved results

Influential articles

Joana Festa and Sérgio Pereira and José António Mariz and Nuno Sousa andCarlos A. SilvaAutomatic Brain Tumor Segmentation of Multi-sequence MR images usingRandom Decision ForestsProceedings of NCI-MICCAI BRATS 2013, Nagoya, Japan, 2013

Nitish Zulpe and Vrushsen PawarGLCM Textural Features for Brain TumorInternational Journal of Computer Science, 2012

Hassan Khotanlou, Olivier Colliot, and Isabelle BlochAutomatic brain tumor segmentation using symmetry analysis and deformablemodelsNationale Superieure des Telecommunications, 2007

Piotr Giedziun Wrocław University of Technology 4 March 2016 10 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Modified diagnosis process

General Practitioner

Neurologist

MR or CT - Radiologists & piece of software

(tumor confirmed & partial diagnosis)

Neurosurgeon

Benign

Pathologist

(final diagnosis)

Malignant

2 Pathologists

(final diagnosis)

(clear image)

Piotr Giedziun Wrocław University of Technology 4 March 2016 11 / 25

The problem State of the art Proposed approach Achieved results

Data set

0 5 10 15 20 25 30 35

Sample No.

100

200

300

400

500

600

700

800

900

Avera

ge inte

nsi

ty

0 5 10 15 20 25 30 35

Sample No.

0

1000

2000

3000

4000

5000

6000

7000

Max inte

nsi

ty

0 5 10 15 20 25 30 35

Sample No.

200

250

300

350

400

450

500

550

Wid

th

FIGURE – Plots of different attributes of the data set

Angle 1 Angle 2 Angle 3

FIGURE – Viewing angles of MRI scan

Piotr Giedziun Wrocław University of Technology 4 March 2016 12 / 25

The problem State of the art Proposed approach Achieved results

Data set

Summary

27 cases with lower grade glioma tumors

13 of them with Oligodendroglioma and 14 with Astrocytoma

Each case has 3 or 4 MRI scans (T1, T1C, FLAIR, and T2)

Provided samples were taken using different hardware

0 5 10 15 20 25 30 35

Sample No.

100

200

300

400

500

600

700

800

900

Avera

ge inte

nsi

ty

0 5 10 15 20 25 30 35

Sample No.

0

1000

2000

3000

4000

5000

6000

7000

Max inte

nsi

ty

0 5 10 15 20 25 30 35

Sample No.

200

250

300

350

400

450

500

550

Wid

th

FIGURE – Plots of different attributes of the data set

Angle 1 Angle 2 Angle 3

FIGURE – Viewing angles of MRI scan

Piotr Giedziun Wrocław University of Technology 4 March 2016 13 / 25

The problem State of the art Proposed approach Achieved results

Pre-processing

0 50 100 150 200 250

Intensity level

0

1000

2000

3000

4000

5000

Num

er

of

pix

els

HistogramTresholded binary map Extracted skull binary map

FIGURE – Process of skull extraction

FLAIR skull figure T2 skull figure

FIGURE – Skulls properties in FLAIR and T2

Piotr Giedziun Wrocław University of Technology 4 March 2016 14 / 25

The problem State of the art Proposed approach Achieved results

Pre-processing

0 20 40 60 80 100 120 140 160 180

Intensity level

0

1000

2000

3000

4000

5000

6000

Num

er

of

pix

els

Histogram before FLAIR before

0 20 40 60 80 100 120 140 160 180

Intensity level

010002000300040005000600070008000

Num

er

of

pix

els

Histogram after FLAIR after

FIGURE – Median filter effect on image histogram

Piotr Giedziun Wrocław University of Technology 4 March 2016 15 / 25

The problem State of the art Proposed approach Achieved results

Segmentation - K-Means

0.1 0.0 0.2 0.4 0.6 0.8 1.0

The silhouette coefficient values

Clu

ster

label

0

1

2

3

The silhouette plot for the various clusters.

Feature space for the X

0.00.2

0.40.6

0.81.0

1.2 Feature space

for th

e Y

0.00.2

0.40.6

0.81.0

1.2 Featu

re s

pace

for

the Inte

nsi

ty

2024

6

8

10

12

14

The visualization of the clustered data.

FIGURE – Silhouette analysis for K-Means(k=5)

Piotr Giedziun Wrocław University of Technology 4 March 2016 16 / 25

The problem State of the art Proposed approach Achieved results

Segmentation - Combined

K-Means segmentation Symetry analysis segmentation Result

K-Means segmentation Symetry analysis segmentation Result

K-Means segmentation Symetry analysis segmentation Result

FIGURE – Segmentation with results

Piotr Giedziun Wrocław University of Technology 4 March 2016 17 / 25

The problem State of the art Proposed approach Achieved results

Segmentation - Alternatives

3 4 5 6 7 8

Number of clusters

0

20

40

60

80

100

120

Sco

res

Modified K-Means vs K-Means

Mini Batch K-Means

K-Means

FIGURE – Mini K-Means

3 4 5 6 7 8

Number of clusters

0

20

40

60

80

100

120

Sco

res

Agglomerative Clustering vs K-Means

Agglomerative

K-Means

FIGURE – Agglomerative cluste-ring

3 4 5 6 7 8

Number of clusters

0

20

40

60

80

100

120

Sco

res

Modified K-Means vs K-Means

Modified K-Means

K-Means

FIGURE – K-Means with position

Piotr Giedziun Wrocław University of Technology 4 March 2016 18 / 25

The problem State of the art Proposed approach Achieved results

Classification

Tested methods

Feature extraction & evaluation

Texture features extraction with Gray-Level Co-Occurrence Matrix

Texture features extraction with Local Binary Pattern

Classification algorithms

SVM (Support vector machine)

Gaussian Naive Bayes

Logistic Regression

Random Forest

Piotr Giedziun Wrocław University of Technology 4 March 2016 19 / 25

The problem State of the art Proposed approach Achieved results

Classification - Feature extraction & evaluation

Selected features (out of 59)

Tumor volume (in mm3)

Tumor position (x,y,z) calculated fromthe middle of the brain

Metrics intensity of tumor area

8 bins of intensity histogram

0 2000 4000 6000 8000 10000 12000 14000 16000

Values of FLAIR 4th histogram bin

0

5000

10000

15000

20000

Valu

es

of

FLA

IR 5

th h

isto

gra

m b

in

Oligodendroglioma

Astrocytoma

FIGURE – Selected features extracted from data set

pos_x

0.60.4

0.20.0

0.20.4

0.6

pos_y

0.6

0.4

0.2

0.00.2

0.40.6

pos_

z

0.4

0.3

0.2

0.1

0.0

0.1

0.2

0.3

0.4

The visualization of tumor position. Temporal lobe

Frontal lobe

Brain

Oligodendroglioma

Astrocytoma

FIGURE – Tumor positional features

Piotr Giedziun Wrocław University of Technology 4 March 2016 20 / 25

The problem State of the art Proposed approach Achieved results

Classification - Texture features extraction with GLCM & LBP

Oligodendroglioma Astrocytoma

O #1 O #2 O #3 O #4 O #5

A #1 A #2 A #3 A #4 A #5

0 10 20 30 40 50 60

GLCM Dissimilarity

0.2

0.0

0.2

0.4

0.6

0.8

1.0

GLV

M C

orr

ela

tion

1

23

4

51

2

3

45 O

A

FIGURE – Co-occurence matrix features for Oligodendroglioma and Astrocytoma

Piotr Giedziun Wrocław University of Technology 4 March 2016 21 / 25

The problem State of the art Proposed approach Achieved results

Radiology imaging

Tumor segmentationMETHOD BEST SCOREMini Batch K-Means (5 clusters) 89.027% (std : 5.408)K-Means (5 clusters) 88.168% (std : 5.264)K-Means with position (5 clusters) 86.026% (std : 5.282)Agglomerative Clustering 88.956% (std : 10.632)

Cancer classificationMETHOD BEST SCORERandom Forest Classifier 87.000% (std : 12.991)Logistic Regression 81.297% (std : 5.744)Logistic Regression (texture) 68.285% (std : 0.082)

Piotr Giedziun Wrocław University of Technology 4 March 2016 22 / 25

The problem State of the art Proposed approach Achieved results

Combined Radiology and Pathology

0.0 0.2 0.4 0.6 0.8 1.0

Radiology Classifier estimate

0.0

0.2

0.4

0.6

0.8

1.0

Path

olo

gy C

lass

ifie

r est

imate

Oligodendroglioma

Astrocytoma

FIGURE – Comparison of Pathology and Radiology results (average estimations of Oligodendroglioma cancer foreach sample)

Piotr Giedziun Wrocław University of Technology 4 March 2016 23 / 25

The problem State of the art Proposed approach Achieved results

Results

Conclusion

Random Forest classifier validated with k-fold cross validation had averageaccuracy of 87.0%

Pre-processing of the input data is a hand-crafted process, that had to beperformed

K-Means had the best score out of Mini Batch K-Means, K-Means with modifiedinput vector (with position), and Agglomerative clustering

Piotr Giedziun Wrocław University of Technology 4 March 2016 24 / 25

The problem State of the art Proposed approach Achieved results

Combined Radiology and Pathology Classification of BrainTumors

Rozpoznanie guza mózgu na podstawie obrazu radiologicznego i patologicznego

Piotr Giedziun

Supervisor:dr hab. inz. Henryk Maciejewski

4 March 2016

Piotr Giedziun Wrocław University of Technology 4 March 2016 25 / 25