Deep Learning: How it Works and How it Fails

Bradley J Erickson, MD PhD, FSIIM, CIIP

Professor and Associate Chair, Radiology

Mayo Clinic, USA

ASNR 2017, April 23, 11:00-11:30

Deep Learning: How it Works and How it Fails

Disclosures

• Relationships with commercial interests:

– Board / Stock Owner:

• FlowSIGMA

• OneMedNet

• VoiceIT

– Research support from NVidia

– Chair, American Board of Imaging Informatics

– Member, RSNA Radiology Informatics Committee

1. Middle Management

2. Commodity Salespeople

3. Report writers, journalists,

Authors & Announcers

4. Accountants & Bookkeepers

5. Medical Doctors

“Radiologists are Easy to Replace!”

•Andrew Ng, renowned Stanford Professor and expert on

machine learning was quoted in The Economist this week as

saying “a highly trained and specialized radiologist may now

be in greater danger of being replaced by a machine than his

own executive assistant: She does so many different things

that I don’t see a machine being able to automate everything

she does any time soon.”

Radiologists Replaced in Four Years??!

• In the past year, articles appeared in the New England Journal “Predicting the

Future – Big Data, Machine Learning, and Clinical Medicine” and the Journal of

the American College of Radiology “The End of Radiology? Three Threats to

the Future Practice of Radiology”.

• Ezekiel Emanuel, MD PhD, suggests that radiologists may be replaced by

computers in the next four to five years

5

Machine Learning History

• Artificial Neural Networks (ANN)

– Starting point of machine learning

– Early versions didn’t work well

• Other Machine Learning Methods

– Naïve Bayes

– Support Vector Machine (SVM)

– Random Forest Classifier (RFC)

How Neural Networks Learn

X

Y

Z

f(Σ)

f(Σ)

f(Σ)

f(Σ)

f(Σ)

f(Σ)

f(Σ)

Input Layer Hidden Layer Output Layer

Tumor

Brain


45

322

128

f(Σ)

f(Σ)

f(Σ)

f(Σ)

f(Σ)

f(Σ)

f(Σ)

T1 Pre

T1 Post

FLAIR

Tumor

Brain



45

322

128

f(Σ)

f(Σ)

34

57

418

-68

312

Tumor

Brain


T1 Pre

T1 Post

FLAIR


45

322

128

1

0

34

57

418

-68

312

Tumor

Brain


T1 Pre

T1 Post

FLAIR


• Output Node Value Computation

– Multiply input node value times a weight

– Activation function on inputs: e.g. threshold the sum

• Weight Update

– Error = expected output – computed output

– New weight = old weight * error * learning rate

Deep Learning: Why the Hype?

Performance in ImageNet Challenge

2010 2011 2012 2013 2014 2015 2016

74% Hand Coded Computer Vision

HumanDeep

Learning

100%

90%

80%

70%

What is “Deep Learning”

• “Deep” because it uses many layers

– ANN typically had 3 or fewer layers

An Example CNN

Convolution Pooling Conv Pooling Conv Pooling Conv Pool Fully Connected

Benefit of DL vs Conventional ML

• Deep Learning Finds Features and Connections vs Just Connections

“Computers Programming Computers”

Hand-Crafted Feature

Extraction

Learning

Feature Extractor

Classifier

Classifier

Deep Learning: Why Now?

• Yes, much more computation power

• Many critical theoretical advances

• Substantial Investment

Theoretical Advances: See arXiv.org

• Pooling (often after C-layer. Example: MaxPool)

• Regularization: Dropout—great performance by eliminating some connections!

• Activation Functions (Leaky Rectified Linear Unit or LeakyReLU)

• Residual Network (force each layer to learn—really a layer not a network, just

as CNN means convolution layers are included)

• Cascaded architectures (if one is good, how about 2…)

• There’s something new every day!

Why the Excitement Now?

1. Deep Neural Network Theory

2. Exponential Compute Power Growth

Moore’s Law

Computing performance doubles approximately every 18 months

General Purpose Computing on GPUs

• Use Graphics Cards for ‘Regular’ (Parallel) Computing

• Special instructions on GPUs for Deep Learning

Now Beating Moore’s Law By A Lot!

FPGA

Qubit?

FPGA

GPU

CPU

Ice Age 2000 2005 2010 2015 2020

1,000,000

100,000

10,000

1000

100

10

How Deep Learning Can Fail

• With millions of parameters, deep learning can actually learn each training

example without learning the principle. E.g. learn specific pixel values/locations

rather than ‘Brain tumors are bright on T2 and Gad’

• This is why Google & Facebook use millions of photographs to train their

systems

• Some say we also need millions of exams to train medical systems

How Deep Learning Can Fail

• With millions of parameters, deep learning can actually learn each training

example without learning the principle. E.g. learn specific pixel values/locations

rather than ‘Brain tumors are bright on T2 and Gad’

• This is why Google & Facebook use millions of photographs to train their

systems

• Some say we also need millions of exams to train medical systems

Ways To Avoid Need For Large Data Sets

• Data Augmentation

– Create variants of data that are different enough that they help learning

– Similar enough that teaching point is kept

– Examples: Mirror/Flip/Rotate/Contrast/Crop/Noise



• Transfer Learning

Image

Conv

Ma

xP

ool

Fully

Conn

ecte

d

So

ftM

ax

Fully

Conn

ecte

d

Fully

Conn

ecte

d

Conv

Conv

Ma

xP

ool

Conv

Conv

Ma

xP

ool

Conv




Image

Conv

Ma

xP

ool

Fully

Conn

ecte

d

So

ftM

ax

Fully

Conn

ecte

d

Fully

Conn

ecte

d

Conv

Conv

Ma

xP

ool

Conv

Conv

Ma

xP

ool

Conv

Freeze These Layers




Image

Conv

Ma

xP

ool

Fully

Conn

ecte

d

So

ftM

ax

Fully

Conn

ecte

d

Fully

Conn

ecte

d

Conv

Conv

Ma

xP

ool

Conv

Conv

Ma

xP

ool

Conv

Freeze These Layers Train this

How to Know: Monitor Training

• There should ALWAYS be a separate test set that is used for reporting final results. These should not be used at all during training.

• A validation test set usually is used to assess training progress.

– Sometimes this is kept separate

– Sometimes cross validation is used, meaning that the validation set changes with each ‘fold’.

– Cross validation is probably better, but must review performance of each fold to see that results are similar.

– Cross validation produces multiple models so:

• If consistent performance of folds, then use all to create network, or

• Use voting scheme of the N-folds to decide

How to Know: Monitor Training

Loss Curve for Good Performance Loss Curve if Overfitting

Why the Excitement Now?

1. Deep Neural Network Theory

2. Exponential Compute Power Growth

3. Huge investment

Investment in Artificial Intelligence in

Healthcare: $1.5 Billion since 2013

https://www.cbinsights.com/blog/artificial-intelligence-startups-healthcare/

The Pace of Change

The Pace of Change

We always overestimate the change that will occur in the

next 2 years and underestimate what will occur in the next

10.

---Bill Gates

Computers and The Radiologist

• (Too) much hype on computers replacing the radiologist

– There is more variability in our task than non-radiologists recognize

– There is much more information available that computers can help extract

(much like text part of medical record)

• Quantitative values that are hard to get today

• Unseen information that is lost

Deep Learning Enables Quantitative Imaging…

*Kline, Neph Diab Trans, 2016

Seeing the Unseen

• Because DL doesn’t require humans to select the things to be measured in an

image, it may find new information in images that we simply can’t see.

Seeing the Unseen

• 155 patients, standard CNN vs Residual Network (100 train/test, 55 validation) :

Korfiatis, CMIMI, 2016

MGMT status PPV Sens Accuracy

Methylated 0.965 0.963 0.964

Unmethylated 0.961 0.920 0.918

MGMT status PPV Sens Accuracy

Methylated 0.661 0.745 0.701

Unmethylated 0.651 0.682 0.667

DL Can Predict 1p19q Status (IDH1 and ATRX coming)

• 159 subjects, 3 slices at ‘equator’ used from Stereotactic planning MRI

• 57 non-deleted & 102 co-deleted

• Mostly OG, but some Astro; Mostly LGG, but a few Gr4

• T2 & Post-Gad available; T2 was best and Post-Gad didn’t add

• Sens: 93.3% Spec: 82.2% Accuracy: 87.7%

• (Handcrafted features with traditional machine learning was better in this group

at >90% for all measures, though unpublished results for DL now surpassing)

*Akkus, arXiv.org, 2016

Deep Learning in Radiology

• Can likely produce reasonable ‘prelim reports’ for high volume exams for

common findings

• Will promote quantitative imaging

– Implicitly (or explicitly) identifies structures humans believe may be

important

– Will lead to large databases of normal values that can be used as

biomarker

– Is able to find biomarkers that are not humanly perceptible

If you want to learn more in “Hands-On” form…

• GPU Technology Conference San Jose, May 8-11

• Conference on Machine Intelligence in Medical Imaging Baltimore, Sep 26-7

• RSNA Chicago, Nov 26-30

Deep Learning in Neuroradiology: Review of Basic Concepts, Current Applications, and Future Opportunities

Luciano M Prevedello, MD, MPHDivision Chief, Medical Imaging InformaticsDirector, 3D and Advanced Visualization LabDivision of NeuroradiologyDepartment of Radiology

Overview

• What is machine learning?

• How does it differ from deep learning?

• How does deep learning work?

‒ Traditional Machine Learning vs Deep learning• Pixel data

• Textual analysis

• Pixel and Textual analysis

• Current Applications

• Future Directions

What is machine Learning?

• Subfield of computer science focused on making machines learn without being explicitly programmed.

• Heuristics vs Machine Learning (rule-based)

Machine Learning vs Deep Learning

Implementing Machine Learning in Radiology Practice and Research

Marc Kohli, Luciano M. Prevedello, Ross W. Filice and J. Raymond Geis. April 2017,

Volume 208, Number 4

http://playground.tensorflow.org/

Text Analysis

Traditional Text Analysis: NLP

Achour et al. A UMLS-based Knowledge Acquisition Tool for Rule-based Clinical Decision Support

System Development. J Am Med Inform Assoc. 2001 Jul-Aug; 8(4): 351–360.


• Associated DX: Severe dizziness [1366950]

• Reason for study: Patient with profound dizziness and emergent hypertension. Unequal pupils and altered mental status. Prior history of cancer.


Matched Term Code Concept Name Semantic Type Annotations

Patient C0030705 Patient Patient or

Disabled Group

Patient/0

profound C0205125 Deep Qualitative

Concept

profound/13

dizziness C0012833 Dizziness Sign or

Symptom

dizziness/22

emergent

hypertension

C0745136 Hypertensive

emergency

Disease or

Syndrome

emergent/36,

hypertension/45

Unequal pupils C0003079 Anisocoria Finding Unequal/59,

pupils/67

altered mental

status

C0278061 Abnormal

mental state

Mental or

Behavioral

Dysfunction

altered/78,

mental/86,

status/93

history CL449097 History of Qualitative

Concept

history/107

cancer C1306459 Neoplasm,

malignant

(primary)

Neoplastic

Process

cancer/118

Deep Learning for Text: Vectors


Sclerosis


SclerosisMultiple


SclerosisTuberous


SclerosisAmyotrophic Lateral


SclerosisMultiple

SclerosisTuberous

SclerosisAmyotrophic Lateral

Deep Learning for Text









https://ronxin.github.io/wevi/

Deep Learning for Images0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 9 37 119 196 218 187 139 140 141 142 188 201 183 98 33 6 1 1 1 1 1 1

0 1 1 1 1 11 49 163 223 164 135 121 121 136 122 115 123 125 171 209 142 41 7 1 1 1 1 1

0 1 1 1 6 50 187 219 150 127 122 121 130 134 104 80 91 122 116 123 203 164 33 3 1 1 1 1

0 1 1 4 28 162 224 148 132 134 118 119 135 127 90 74 84 114 120 118 115 207 137 25 3 1 1 1

0 1 1 22 110 226 159 136 135 120 121 125 133 127 90 79 80 88 105 121 109 125 208 109 20 2 1 1

0 1 7 86 226 182 140 137 132 120 119 128 136 114 83 84 107 103 103 127 118 107 158 219 81 9 1 1

0 2 27 157 217 146 139 126 119 113 116 126 128 101 78 79 115 113 103 117 111 111 123 211 143 29 2 1

0 6 58 195 192 141 133 119 113 114 119 112 103 91 85 87 117 110 87 85 101 101 110 178 188 61 6 1

0 16 111 221 183 146 138 124 115 119 132 112 91 86 85 92 110 111 103 93 94 95 123 189 220 112 15 1

0 30 145 234 177 133 124 122 120 114 132 119 94 88 83 84 87 96 91 87 91 106 141 182 236 155 29 7

0 63 178 234 164 136 125 132 121 116 126 98 73 71 84 86 84 85 81 92 119 138 143 151 231 182 46 20

0 72 196 226 150 137 131 132 124 111 114 113 95 58 81 114 98 100 99 109 122 129 132 136 213 218 91 28

0 52 197 220 147 139 137 130 128 112 109 124 111 84 82 125 112 109 112 122 126 133 136 138 175 222 79 36

0 50 184 206 141 136 136 136 128 115 113 107 90 91 100 116 108 107 115 132 132 131 136 146 165 202 69 52

0 48 151 207 141 133 130 124 116 96 71 67 90 105 104 111 104 109 117 131 134 134 138 144 157 188 54 61

0 40 116 216 144 138 128 110 108 66 53 86 108 124 130 107 97 77 103 123 125 130 134 139 168 187 47 50

0 56 57 210 162 139 136 119 108 80 98 116 116 134 150 119 111 96 97 108 118 122 143 147 205 160 40 34

0 66 21 165 209 142 139 127 113 105 123 127 126 125 136 121 130 121 101 110 125 131 136 167 233 88 47 12

0 42 46 62 222 173 141 141 121 112 130 125 128 127 138 124 131 131 105 110 128 144 144 213 197 37 61 2

0 8 72 26 156 231 151 142 130 114 116 108 128 130 140 133 125 118 110 114 127 133 160 235 106 45 41 1

0 1 25 72 56 191 209 138 134 126 121 112 120 130 138 133 117 115 120 121 136 140 215 165 27 70 12 1

0 1 1 32 88 54 170 203 145 131 131 121 132 138 142 128 117 124 124 131 154 209 174 36 58 35 1 1

0 1 1 1 27 64 49 156 211 169 137 129 134 152 161 130 126 127 146 188 225 160 41 55 31 2 1 1

0 1 1 1 1 14 54 62 89 171 210 193 196 228 234 207 194 208 227 191 96 43 55 23 1 1 1 1

0 1 1 1 1 1 3 25 53 59 77 107 127 128 135 142 139 126 86 59 46 30 6 1 1 1 1 1

0 1 1 1 1 1 1 1 3 17 31 45 45 43 50 49 53 47 38 21 4 1 1 1 1 1 1 1

Deep Learning for Images0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 9 37 119 196 218 187 139 140 141 142 188 201 183 98 33 6 1 1 1 1 1 1

0 1 1 1 1 11 49 163 223 164 135 121 121 136 122 115 123 125 171 209 142 41 7 1 1 1 1 1

0 1 1 1 6 50 187 219 150 127 122 121 130 134 104 80 91 122 116 123 203 164 33 3 1 1 1 1

0 1 1 4 28 162 224 148 132 134 118 119 135 127 90 74 84 114 120 118 115 207 137 25 3 1 1 1

0 1 1 22 110 226 159 136 135 120 121 125 133 127 90 79 80 88 105 121 109 125 208 109 20 2 1 1

0 1 7 86 226 182 140 137 132 120 119 128 136 114 83 84 107 103 103 127 118 107 158 219 81 9 1 1

0 2 27 157 217 146 139 126 119 113 116 126 128 101 78 79 115 113 103 117 111 111 123 211 143 29 2 1

0 6 58 195 192 141 133 119 113 114 119 112 103 91 85 87 117 110 87 85 101 101 110 178 188 61 6 1

0 16 111 221 183 146 138 124 115 119 132 112 91 86 85 92 110 111 103 93 94 95 123 189 220 112 15 1

0 30 145 234 177 133 124 122 120 114 132 119 94 88 83 84 87 96 91 87 91 106 141 182 236 155 29 7

0 63 178 234 164 136 125 132 121 116 126 98 73 71 84 86 84 85 81 92 119 138 143 151 231 182 46 20

0 72 196 226 150 137 131 132 124 111 114 113 95 58 81 114 98 100 99 109 122 129 132 136 213 218 91 28

0 52 197 220 147 139 137 130 128 112 109 124 111 84 82 125 112 109 112 122 126 133 136 138 175 222 79 36

0 50 184 206 141 136 136 136 128 115 113 107 90 91 100 116 108 107 115 132 132 131 136 146 165 202 69 52

0 48 151 207 141 133 130 124 116 96 71 67 90 105 104 111 104 109 117 131 134 134 138 144 157 188 54 61

0 40 116 216 144 138 128 110 108 66 53 86 108 124 130 107 97 77 103 123 125 130 134 139 168 187 47 50

0 56 57 210 162 139 136 119 108 80 98 116 116 134 150 119 111 96 97 108 118 122 143 147 205 160 40 34

0 66 21 165 209 142 139 127 113 105 123 127 126 125 136 121 130 121 101 110 125 131 136 167 233 88 47 12

0 42 46 62 222 173 141 141 121 112 130 125 128 127 138 124 131 131 105 110 128 144 144 213 197 37 61 2

0 8 72 26 156 231 151 142 130 114 116 108 128 130 140 133 125 118 110 114 127 133 160 235 106 45 41 1

0 1 25 72 56 191 209 138 134 126 121 112 120 130 138 133 117 115 120 121 136 140 215 165 27 70 12 1

0 1 1 32 88 54 170 203 145 131 131 121 132 138 142 128 117 124 124 131 154 209 174 36 58 35 1 1

0 1 1 1 27 64 49 156 211 169 137 129 134 152 161 130 126 127 146 188 225 160 41 55 31 2 1 1

0 1 1 1 1 14 54 62 89 171 210 193 196 228 234 207 194 208 227 191 96 43 55 23 1 1 1 1

0 1 1 1 1 1 3 25 53 59 77 107 127 128 135 142 139 126 86 59 46 30 6 1 1 1 1 1

0 1 1 1 1 1 1 1 3 17 31 45 45 43 50 49 53 47 38 21 4 1 1 1 1 1 1 1

1.18 1.78 1.78 1.78 8.61 35.1 119 228 322 349 311 278 250 252 292 319 335 265 171 75.1 22.7 4.8 1.78 1.78 1.78 1.78

1.79 3.11 3.11 11.6 51.8 184 348 454 521 502 467 448 407 407 430 440 508 517 446 282 129 32.8 6.73 3.11 3.11 3.11

3.01 5.31 9.57 54.4 218 419 612 763 855 840 790 752 680 661 683 721 796 771 733 570 346 150 29.6 6.51 5.31 5.31

3.01 7.86 32.1 191 423 643 828 857 797 697 671 672 628 590 547 564 642 737 771 758 582 354 126 21.9 6.51 5.31

3.01 24.9 125 387 626 821 882 791 688 659 669 671 614 545 473 488 532 612 691 753 729 577 313 96.7 18.5 5.91

8.13 94.4 317 594 777 878 822 707 668 658 672 660 594 526 470 486 533 596 605 711 741 728 545 272 71.8 10.8

28.9 215 500 730 866 848 728 666 654 646 666 645 576 510 480 487 535 581 580 624 724 777 687 478 177 29.2

73.6 364 655 824 919 758 687 649 637 631 637 602 547 491 509 519 556 568 555 580 635 807 755 640 326 71

159 509 728 897 878 713 669 631 631 622 611 561 508 471 512 531 569 560 534 540 587 778 830 725 472 148

253 612 799 938 824 692 656 626 639 618 594 534 479 464 496 516 530 510 496 503 581 742 901 811 595 252

365 711 880 966 802 703 661 640 654 608 569 498 450 452 475 495 501 503 516 545 623 749 955 902 691 373

439 775 924 957 768 695 660 640 645 605 573 488 443 450 468 496 483 503 528 575 652 742 956 949 778 495

484 803 945 947 754 710 677 650 634 598 560 484 451 467 489 535 511 536 576 637 692 734 896 943 779 576

491 774 928 920 742 716 693 652 629 599 559 504 484 507 521 585 563 595 624 665 691 718 840 944 759 636

450 728 892 902 735 705 684 631 581 543 523 523 511 550 546 598 590 625 653 691 708 729 797 917 676 633

386 698 830 893 725 689 661 583 523 485 482 520 552 577 566 552 561 603 648 690 706 728 792 893 636 608

290 671 753 904 738 679 641 540 492 454 472 548 613 622 607 554 554 573 621 666 702 728 829 866 622 559

195 623 696 906 790 686 643 538 517 499 541 619 669 651 656 567 556 554 597 648 689 738 888 815 646 457

137 503 645 847 870 736 671 597 596 584 631 664 701 676 696 628 593 583 592 643 703 807 935 756 663 315

130 343 609 769 919 817 702 649 635 612 660 662 700 679 702 659 623 615 611 655 723 886 886 708 587 182

118 206 498 666 880 894 764 683 661 622 654 652 699 691 696 665 631 626 637 678 799 912 777 664 411 137

85.3 123 363 542 742 887 833 740 680 629 654 656 709 704 690 662 632 636 676 751 846 813 664 531 231 130

25.6 96.2 173 387 545 724 842 824 763 687 675 687 732 723 703 666 661 713 802 830 815 686 509 328 121 101

3.01 34.5 110 158 362 506 649 791 826 815 822 844 887 875 851 811 823 845 824 786 682 497 326 141 92 39.3

3.01 5.31 29.8 79.1 126 293 437 598 689 718 755 797 841 849 845 795 771 763 742 638 447 287 130 83.1 34.9 6.28

3.01 5.31 5.31 17.5 59.9 104 191 316 450 572 647 691 732 745 742 718 695 632 489 330 199 103 67.8 26.8 5.31 5.31

Deep Learning for Images

32.2 104 403 1016 2020 3212 4270 4854 4978 4751 4396 4158 4056 4200 4536 4708 4690 4005 2849 1567 657 216 57.3 31.9 21.3

75.6 352 1015 2205 3729 5064 5983 6293 6254 5983 5653 5347 5053 5063 5390 5817 6078 5841 4895 3333 1718 676 192 58.4 31

237 857 2037 3744 5377 6477 6944 6838 6527 6259 6017 5692 5250 5072 5222 5661 6157 6388 6143 5099 3312 1634 579 158 47

650 1744 3383 5236 6547 7088 6945 6412 6062 5916 5771 5430 4889 4562 4574 4998 5492 6056 6286 6134 4952 3110 1416 471 113

1312 2930 4831 6373 7075 7036 6518 6036 5878 5845 5694 5277 4720 4392 4392 4722 5125 5568 5993 6353 6033 4705 2687 1123 302

2184 4201 6070 6996 7155 6644 6118 5808 5753 5714 5540 5093 4642 4415 4500 4727 4982 5275 5553 6200 6453 5923 4068 2067 675

3086 5339 6849 7279 6969 6245 5855 5662 5616 5503 5274 4850 4532 4444 4651 4801 4913 5007 5202 5845 6519 6618 5246 3106 1223

4005 6228 7281 7313 6722 5986 5727 5584 5521 5301 4984 4589 4383 4414 4669 4774 4787 4749 4898 5542 6479 7054 6197 4103 1884

4875 6933 7534 7305 6520 5902 5705 5567 5467 5125 4728 4355 4215 4305 4533 4630 4630 4611 4802 5446 6553 7386 7019 5056 2576

5638 7456 7704 7268 6395 5899 5719 5585 5447 5000 4538 4180 4094 4212 4352 4454 4492 4613 4936 5554 6688 7629 7681 5957 3264

6212 7788 7794 7229 6344 5949 5746 5571 5389 4935 4449 4098 4081 4261 4356 4477 4569 4832 5233 5785 6746 7664 7936 6629 3840

6460 7860 7756 7151 6339 6013 5783 5553 5335 4936 4497 4188 4209 4475 4583 4745 4834 5143 5529 5981 6700 7507 7831 6962 4194

6382 7720 7628 7057 6329 6039 5740 5419 5167 4882 4572 4412 4450 4738 4873 5082 5200 5516 5822 6136 6601 7287 7477 6940 4238

6038 7460 7427 6944 6245 5916 5543 5113 4826 4681 4625 4658 4729 4900 4988 5197 5386 5710 5972 6215 6558 7141 7163 6707 4055

5536 7204 7231 6879 6144 5708 5252 4718 4451 4469 4681 4943 5059 5061 4996 5066 5309 5665 5978 6235 6590 7078 6969 6398 3795

4935 6935 7119 6920 6155 5575 5079 4546 4384 4523 4910 5296 5465 5272 5078 4935 5133 5478 5843 6181 6682 7094 6973 6050 3592

4251 6497 7103 7079 6371 5679 5193 4754 4732 4919 5351 5688 5841 5606 5351 5047 5077 5346 5742 6211 6861 7199 7019 5612 3378

3586 5793 7031 7288 6755 6000 5501 5167 5233 5391 5749 5936 6064 5856 5654 5315 5216 5380 5745 6381 7065 7329 6852 5053 3033

2907 4843 6652 7404 7197 6445 5885 5548 5561 5631 5919 6026 6144 5995 5836 5562 5426 5538 5942 6640 7222 7279 6291 4413 2483

2238 3819 5795 7094 7456 6917 6249 5816 5704 5679 5936 6044 6170 6042 5877 5674 5603 5798 6249 6849 7160 6771 5366 3632 1826

1466 2791 4527 6087 7131 7231 6728 6184 5907 5795 6006 6143 6249 6089 5918 5824 5934 6226 6663 6931 6686 5771 4195 2704 1211

875 1701 3114 4565 5912 6821 7001 6757 6484 6351 6546 6704 6766 6559 6388 6334 6465 6739 6882 6564 5733 4403 2952 1712 744

370 918 1682 2889 4212 5474 6347 6721 6774 6790 7018 7244 7328 7099 6890 6807 6891 6822 6401 5557 4306 2972 1739 939 407

145 343 799 1423 2417 3597 4726 5651 6182 6516 6861 7133 7269 7141 6942 6690 6382 5883 5041 3907 2711 1621 892 433 181

1.18 1.78 1.78 1.78 8.61 35.1 119 228 322 349 311 278 250 252 292 319 335 265 171 75.1 22.7 4.8 1.78 1.78 1.78 1.78

1.79 3.11 3.11 11.6 51.8 184 348 454 521 502 467 448 407 407 430 440 508 517 446 282 129 32.8 6.73 3.11 3.11 3.11

3.01 5.31 9.57 54.4 218 419 612 763 855 840 790 752 680 661 683 721 796 771 733 570 346 150 29.6 6.51 5.31 5.31

3.01 7.86 32.1 191 423 643 828 857 797 697 671 672 628 590 547 564 642 737 771 758 582 354 126 21.9 6.51 5.31

3.01 24.9 125 387 626 821 882 791 688 659 669 671 614 545 473 488 532 612 691 753 729 577 313 96.7 18.5 5.91

8.13 94.4 317 594 777 878 822 707 668 658 672 660 594 526 470 486 533 596 605 711 741 728 545 272 71.8 10.8

28.9 215 500 730 866 848 728 666 654 646 666 645 576 510 480 487 535 581 580 624 724 777 687 478 177 29.2

73.6 364 655 824 919 758 687 649 637 631 637 602 547 491 509 519 556 568 555 580 635 807 755 640 326 71

159 509 728 897 878 713 669 631 631 622 611 561 508 471 512 531 569 560 534 540 587 778 830 725 472 148

253 612 799 938 824 692 656 626 639 618 594 534 479 464 496 516 530 510 496 503 581 742 901 811 595 252

365 711 880 966 802 703 661 640 654 608 569 498 450 452 475 495 501 503 516 545 623 749 955 902 691 373

439 775 924 957 768 695 660 640 645 605 573 488 443 450 468 496 483 503 528 575 652 742 956 949 778 495

484 803 945 947 754 710 677 650 634 598 560 484 451 467 489 535 511 536 576 637 692 734 896 943 779 576

491 774 928 920 742 716 693 652 629 599 559 504 484 507 521 585 563 595 624 665 691 718 840 944 759 636

450 728 892 902 735 705 684 631 581 543 523 523 511 550 546 598 590 625 653 691 708 729 797 917 676 633

386 698 830 893 725 689 661 583 523 485 482 520 552 577 566 552 561 603 648 690 706 728 792 893 636 608

290 671 753 904 738 679 641 540 492 454 472 548 613 622 607 554 554 573 621 666 702 728 829 866 622 559

195 623 696 906 790 686 643 538 517 499 541 619 669 651 656 567 556 554 597 648 689 738 888 815 646 457

137 503 645 847 870 736 671 597 596 584 631 664 701 676 696 628 593 583 592 643 703 807 935 756 663 315

130 343 609 769 919 817 702 649 635 612 660 662 700 679 702 659 623 615 611 655 723 886 886 708 587 182

118 206 498 666 880 894 764 683 661 622 654 652 699 691 696 665 631 626 637 678 799 912 777 664 411 137

85.3 123 363 542 742 887 833 740 680 629 654 656 709 704 690 662 632 636 676 751 846 813 664 531 231 130

25.6 96.2 173 387 545 724 842 824 763 687 675 687 732 723 703 666 661 713 802 830 815 686 509 328 121 101

3.01 34.5 110 158 362 506 649 791 826 815 822 844 887 875 851 811 823 845 824 786 682 497 326 141 92 39.3

3.01 5.31 29.8 79.1 126 293 437 598 689 718 755 797 841 849 845 795 771 763 742 638 447 287 130 83.1 34.9 6.28

3.01 5.31 5.31 17.5 59.9 104 191 316 450 572 647 691 732 745 742 718 695 632 489 330 199 103 67.8 26.8 5.31 5.31


352 2205 5064 6293 6254 5653 5063 5817 6078 4895 1718 192 31

1744 5236 7088 6945 6527 6017 5250 5661 6388 6286 4952 1416 113

4201 6996 7155 6518 5878 5694 4720 4727 5568 6353 6453 4068 675

6228 7313 6969 5855 5616 5274 4532 4801 5007 5845 7054 6197 1884

7456 7704 6520 5719 5467 4728 4305 4630 4630 5554 7629 7681 3264

7860 7794 6344 5783 5389 4497 4475 4745 5143 5981 7664 7936 4194

7720 7628 6329 5740 5167 4658 4900 5197 5710 6215 7287 7477 4238

7204 7231 6155 5252 4523 5296 5465 5078 5665 6235 7094 6973 3795

6497 7288 6755 5501 5391 5936 6064 5654 5380 6381 7329 7019 3378

4843 7404 7456 6249 5704 6044 6170 5877 5798 6849 7279 6291 2483

2791 6087 7231 7001 6484 6704 6766 6388 6739 6931 6686 4195 1211

918 2889 5474 6721 6790 7244 7328 6942 6891 6401 4306 1739 407

32.2 104 403 1016 2020 3212 4270 4854 4978 4751 4396 4158 4056 4200 4536 4708 4690 4005 2849 1567 657 216 57.3 31.9 21.3

75.6 352 1015 2205 3729 5064 5983 6293 6254 5983 5653 5347 5053 5063 5390 5817 6078 5841 4895 3333 1718 676 192 58.4 31

237 857 2037 3744 5377 6477 6944 6838 6527 6259 6017 5692 5250 5072 5222 5661 6157 6388 6143 5099 3312 1634 579 158 47

650 1744 3383 5236 6547 7088 6945 6412 6062 5916 5771 5430 4889 4562 4574 4998 5492 6056 6286 6134 4952 3110 1416 471 113

1312 2930 4831 6373 7075 7036 6518 6036 5878 5845 5694 5277 4720 4392 4392 4722 5125 5568 5993 6353 6033 4705 2687 1123 302

2184 4201 6070 6996 7155 6644 6118 5808 5753 5714 5540 5093 4642 4415 4500 4727 4982 5275 5553 6200 6453 5923 4068 2067 675

3086 5339 6849 7279 6969 6245 5855 5662 5616 5503 5274 4850 4532 4444 4651 4801 4913 5007 5202 5845 6519 6618 5246 3106 1223

4005 6228 7281 7313 6722 5986 5727 5584 5521 5301 4984 4589 4383 4414 4669 4774 4787 4749 4898 5542 6479 7054 6197 4103 1884

4875 6933 7534 7305 6520 5902 5705 5567 5467 5125 4728 4355 4215 4305 4533 4630 4630 4611 4802 5446 6553 7386 7019 5056 2576

5638 7456 7704 7268 6395 5899 5719 5585 5447 5000 4538 4180 4094 4212 4352 4454 4492 4613 4936 5554 6688 7629 7681 5957 3264

6212 7788 7794 7229 6344 5949 5746 5571 5389 4935 4449 4098 4081 4261 4356 4477 4569 4832 5233 5785 6746 7664 7936 6629 3840

6460 7860 7756 7151 6339 6013 5783 5553 5335 4936 4497 4188 4209 4475 4583 4745 4834 5143 5529 5981 6700 7507 7831 6962 4194

6382 7720 7628 7057 6329 6039 5740 5419 5167 4882 4572 4412 4450 4738 4873 5082 5200 5516 5822 6136 6601 7287 7477 6940 4238

6038 7460 7427 6944 6245 5916 5543 5113 4826 4681 4625 4658 4729 4900 4988 5197 5386 5710 5972 6215 6558 7141 7163 6707 4055

5536 7204 7231 6879 6144 5708 5252 4718 4451 4469 4681 4943 5059 5061 4996 5066 5309 5665 5978 6235 6590 7078 6969 6398 3795

4935 6935 7119 6920 6155 5575 5079 4546 4384 4523 4910 5296 5465 5272 5078 4935 5133 5478 5843 6181 6682 7094 6973 6050 3592

4251 6497 7103 7079 6371 5679 5193 4754 4732 4919 5351 5688 5841 5606 5351 5047 5077 5346 5742 6211 6861 7199 7019 5612 3378

3586 5793 7031 7288 6755 6000 5501 5167 5233 5391 5749 5936 6064 5856 5654 5315 5216 5380 5745 6381 7065 7329 6852 5053 3033

2907 4843 6652 7404 7197 6445 5885 5548 5561 5631 5919 6026 6144 5995 5836 5562 5426 5538 5942 6640 7222 7279 6291 4413 2483

2238 3819 5795 7094 7456 6917 6249 5816 5704 5679 5936 6044 6170 6042 5877 5674 5603 5798 6249 6849 7160 6771 5366 3632 1826

1466 2791 4527 6087 7131 7231 6728 6184 5907 5795 6006 6143 6249 6089 5918 5824 5934 6226 6663 6931 6686 5771 4195 2704 1211

875 1701 3114 4565 5912 6821 7001 6757 6484 6351 6546 6704 6766 6559 6388 6334 6465 6739 6882 6564 5733 4403 2952 1712 744

370 918 1682 2889 4212 5474 6347 6721 6774 6790 7018 7244 7328 7099 6890 6807 6891 6822 6401 5557 4306 2972 1739 939 407

145 343 799 1423 2417 3597 4726 5651 6182 6516 6861 7133 7269 7141 6942 6690 6382 5883 5041 3907 2711 1621 892 433 181


0.58 0.59 0.77 0.02 0.19 0.81 0.8 0.93 0.95 0.57 0.28 0.41 0.46

0.11 0.78 0.96 0.42 0.7 0.09 0.74 0.23 0.63 0.93 0.74 0.28 1

0.96 0.51 0.39 0.31 0.28 0.61 0.9 0.13 0.51 0.6 0.32 0.09 0.12

0.14 0.13 0.42 0.03 0.57 0.84 0.86 0.91 0.43 0.8 0.97 0.54 0.3

0.15 0.18 0.59 0.63 0.18 0.93 0.43 0.9 0.53 0.41 0.17 0.5 0.11

0.31 0.49 0.48 0.04 0.78 0.03 0.88 0.02 0.73 0.16 0.19 0.17 0.02

0.32 0.24 0.03 0.2 0.93 0.5 0.01 0.96 0.47 0.39 0.31 0.04 0.25

0.35 0.3 0.78 0.69 0.24 0.49 0.28 0.01 0.58 0.24 0.97 0.93 0.79

0.98 0.33 0.79 0.76 0.68 0.73 0.35 0.02 0.89 0.98 0.45 0.67 0.7

0.82 0.83 0.33 0.16 0.84 0.9 0.81 0.37 0.74 0.95 0.41 0.25 0.14

0.95 0.32 0.16 0.49 0.08 0.63 0.56 0.37 0.45 0.16 0.69 0.38 0.27

0.78 0.32 0.97 0.66 0.61 0.39 0.1 0.01 0 0.19 0.72 0.65 0.41

352 2205 5064 6293 6254 5653 5063 5817 6078 4895 1718 192 31

1744 5236 7088 6945 6527 6017 5250 5661 6388 6286 4952 1416 113

4201 6996 7155 6518 5878 5694 4720 4727 5568 6353 6453 4068 675

6228 7313 6969 5855 5616 5274 4532 4801 5007 5845 7054 6197 1884

7456 7704 6520 5719 5467 4728 4305 4630 4630 5554 7629 7681 3264

7860 7794 6344 5783 5389 4497 4475 4745 5143 5981 7664 7936 4194

7720 7628 6329 5740 5167 4658 4900 5197 5710 6215 7287 7477 4238

7204 7231 6155 5252 4523 5296 5465 5078 5665 6235 7094 6973 3795

6497 7288 6755 5501 5391 5936 6064 5654 5380 6381 7329 7019 3378

4843 7404 7456 6249 5704 6044 6170 5877 5798 6849 7279 6291 2483

2791 6087 7231 7001 6484 6704 6766 6388 6739 6931 6686 4195 1211

918 2889 5474 6721 6790 7244 7328 6942 6891 6401 4306 1739 407


0% 50% 100%

Mass effect

Hemorrhage

Hydrocephalus

Normal0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 9 37 119 196 218 187 139 140 141 142 188 201 183 98 33 6 1 1 1 1 1 1

0 1 1 1 1 11 49 163 223 164 135 121 121 136 122 115 123 125 171 209 142 41 7 1 1 1 1 1

0 1 1 1 6 50 187 219 150 127 122 121 130 134 104 80 91 122 116 123 203 164 33 3 1 1 1 1

0 1 1 4 28 162 224 148 132 134 118 119 135 127 90 74 84 114 120 118 115 207 137 25 3 1 1 1

0 1 1 22 110 226 159 136 135 120 121 125 133 127 90 79 80 88 105 121 109 125 208 109 20 2 1 1

0 1 7 86 226 182 140 137 132 120 119 128 136 114 83 84 107 103 103 127 118 107 158 219 81 9 1 1

0 2 27 157 217 146 139 126 119 113 116 126 128 101 78 79 115 113 103 117 111 111 123 211 143 29 2 1

0 6 58 195 192 141 133 119 113 114 119 112 103 91 85 87 117 110 87 85 101 101 110 178 188 61 6 1

0 16 111 221 183 146 138 124 115 119 132 112 91 86 85 92 110 111 103 93 94 95 123 189 220 112 15 1

0 30 145 234 177 133 124 122 120 114 132 119 94 88 83 84 87 96 91 87 91 106 141 182 236 155 29 7

0 63 178 234 164 136 125 132 121 116 126 98 73 71 84 86 84 85 81 92 119 138 143 151 231 182 46 20

0 72 196 226 150 137 131 132 124 111 114 113 95 58 81 114 98 100 99 109 122 129 132 136 213 218 91 28

0 52 197 220 147 139 137 130 128 112 109 124 111 84 82 125 112 109 112 122 126 133 136 138 175 222 79 36

0 50 184 206 141 136 136 136 128 115 113 107 90 91 100 116 108 107 115 132 132 131 136 146 165 202 69 52

0 48 151 207 141 133 130 124 116 96 71 67 90 105 104 111 104 109 117 131 134 134 138 144 157 188 54 61

0 40 116 216 144 138 128 110 108 66 53 86 108 124 130 107 97 77 103 123 125 130 134 139 168 187 47 50

0 56 57 210 162 139 136 119 108 80 98 116 116 134 150 119 111 96 97 108 118 122 143 147 205 160 40 34

0 66 21 165 209 142 139 127 113 105 123 127 126 125 136 121 130 121 101 110 125 131 136 167 233 88 47 12

0 42 46 62 222 173 141 141 121 112 130 125 128 127 138 124 131 131 105 110 128 144 144 213 197 37 61 2

0 8 72 26 156 231 151 142 130 114 116 108 128 130 140 133 125 118 110 114 127 133 160 235 106 45 41 1

0 1 25 72 56 191 209 138 134 126 121 112 120 130 138 133 117 115 120 121 136 140 215 165 27 70 12 1

0 1 1 32 88 54 170 203 145 131 131 121 132 138 142 128 117 124 124 131 154 209 174 36 58 35 1 1

0 1 1 1 27 64 49 156 211 169 137 129 134 152 161 130 126 127 146 188 225 160 41 55 31 2 1 1

0 1 1 1 1 14 54 62 89 171 210 193 196 228 234 207 194 208 227 191 96 43 55 23 1 1 1 1

0 1 1 1 1 1 3 25 53 59 77 107 127 128 135 142 139 126 86 59 46 30 6 1 1 1 1 1

0 1 1 1 1 1 1 1 3 17 31 45 45 43 50 49 53 47 38 21 4 1 1 1 1 1 1 1

1.18 1.78 1.78 1.78 8.61 35.1 119 228 322 349 311 278 250 252 292 319 335 265 171 75.1 22.7 4.8 1.78 1.78 1.78 1.78

1.79 3.11 3.11 11.6 51.8 184 348 454 521 502 467 448 407 407 430 440 508 517 446 282 129 32.8 6.73 3.11 3.11 3.11

3.01 5.31 9.57 54.4 218 419 612 763 855 840 790 752 680 661 683 721 796 771 733 570 346 150 29.6 6.51 5.31 5.31

3.01 7.86 32.1 191 423 643 828 857 797 697 671 672 628 590 547 564 642 737 771 758 582 354 126 21.9 6.51 5.31

3.01 24.9 125 387 626 821 882 791 688 659 669 671 614 545 473 488 532 612 691 753 729 577 313 96.7 18.5 5.91

8.13 94.4 317 594 777 878 822 707 668 658 672 660 594 526 470 486 533 596 605 711 741 728 545 272 71.8 10.8

28.9 215 500 730 866 848 728 666 654 646 666 645 576 510 480 487 535 581 580 624 724 777 687 478 177 29.2

73.6 364 655 824 919 758 687 649 637 631 637 602 547 491 509 519 556 568 555 580 635 807 755 640 326 71

159 509 728 897 878 713 669 631 631 622 611 561 508 471 512 531 569 560 534 540 587 778 830 725 472 148

253 612 799 938 824 692 656 626 639 618 594 534 479 464 496 516 530 510 496 503 581 742 901 811 595 252

365 711 880 966 802 703 661 640 654 608 569 498 450 452 475 495 501 503 516 545 623 749 955 902 691 373

439 775 924 957 768 695 660 640 645 605 573 488 443 450 468 496 483 503 528 575 652 742 956 949 778 495

484 803 945 947 754 710 677 650 634 598 560 484 451 467 489 535 511 536 576 637 692 734 896 943 779 576

491 774 928 920 742 716 693 652 629 599 559 504 484 507 521 585 563 595 624 665 691 718 840 944 759 636

450 728 892 902 735 705 684 631 581 543 523 523 511 550 546 598 590 625 653 691 708 729 797 917 676 633

386 698 830 893 725 689 661 583 523 485 482 520 552 577 566 552 561 603 648 690 706 728 792 893 636 608

290 671 753 904 738 679 641 540 492 454 472 548 613 622 607 554 554 573 621 666 702 728 829 866 622 559

195 623 696 906 790 686 643 538 517 499 541 619 669 651 656 567 556 554 597 648 689 738 888 815 646 457

137 503 645 847 870 736 671 597 596 584 631 664 701 676 696 628 593 583 592 643 703 807 935 756 663 315

130 343 609 769 919 817 702 649 635 612 660 662 700 679 702 659 623 615 611 655 723 886 886 708 587 182

118 206 498 666 880 894 764 683 661 622 654 652 699 691 696 665 631 626 637 678 799 912 777 664 411 137

85.3 123 363 542 742 887 833 740 680 629 654 656 709 704 690 662 632 636 676 751 846 813 664 531 231 130

25.6 96.2 173 387 545 724 842 824 763 687 675 687 732 723 703 666 661 713 802 830 815 686 509 328 121 101

3.01 34.5 110 158 362 506 649 791 826 815 822 844 887 875 851 811 823 845 824 786 682 497 326 141 92 39.3

3.01 5.31 29.8 79.1 126 293 437 598 689 718 755 797 841 849 845 795 771 763 742 638 447 287 130 83.1 34.9 6.28

3.01 5.31 5.31 17.5 59.9 104 191 316 450 572 647 691 732 745 742 718 695 632 489 330 199 103 67.8 26.8 5.31 5.31

32.2 104 403 1016 2020 3212 4270 4854 4978 4751 4396 4158 4056 4200 4536 4708 4690 4005 2849 1567 657 216 57.3 31.9 21.3

75.6 352 1015 2205 3729 5064 5983 6293 6254 5983 5653 5347 5053 5063 5390 5817 6078 5841 4895 3333 1718 676 192 58.4 31

237 857 2037 3744 5377 6477 6944 6838 6527 6259 6017 5692 5250 5072 5222 5661 6157 6388 6143 5099 3312 1634 579 158 47

650 1744 3383 5236 6547 7088 6945 6412 6062 5916 5771 5430 4889 4562 4574 4998 5492 6056 6286 6134 4952 3110 1416 471 113

1312 2930 4831 6373 7075 7036 6518 6036 5878 5845 5694 5277 4720 4392 4392 4722 5125 5568 5993 6353 6033 4705 2687 1123 302

2184 4201 6070 6996 7155 6644 6118 5808 5753 5714 5540 5093 4642 4415 4500 4727 4982 5275 5553 6200 6453 5923 4068 2067 675

3086 5339 6849 7279 6969 6245 5855 5662 5616 5503 5274 4850 4532 4444 4651 4801 4913 5007 5202 5845 6519 6618 5246 3106 1223

4005 6228 7281 7313 6722 5986 5727 5584 5521 5301 4984 4589 4383 4414 4669 4774 4787 4749 4898 5542 6479 7054 6197 4103 1884

4875 6933 7534 7305 6520 5902 5705 5567 5467 5125 4728 4355 4215 4305 4533 4630 4630 4611 4802 5446 6553 7386 7019 5056 2576

5638 7456 7704 7268 6395 5899 5719 5585 5447 5000 4538 4180 4094 4212 4352 4454 4492 4613 4936 5554 6688 7629 7681 5957 3264

6212 7788 7794 7229 6344 5949 5746 5571 5389 4935 4449 4098 4081 4261 4356 4477 4569 4832 5233 5785 6746 7664 7936 6629 3840

6460 7860 7756 7151 6339 6013 5783 5553 5335 4936 4497 4188 4209 4475 4583 4745 4834 5143 5529 5981 6700 7507 7831 6962 4194

6382 7720 7628 7057 6329 6039 5740 5419 5167 4882 4572 4412 4450 4738 4873 5082 5200 5516 5822 6136 6601 7287 7477 6940 4238

6038 7460 7427 6944 6245 5916 5543 5113 4826 4681 4625 4658 4729 4900 4988 5197 5386 5710 5972 6215 6558 7141 7163 6707 4055

5536 7204 7231 6879 6144 5708 5252 4718 4451 4469 4681 4943 5059 5061 4996 5066 5309 5665 5978 6235 6590 7078 6969 6398 3795

4935 6935 7119 6920 6155 5575 5079 4546 4384 4523 4910 5296 5465 5272 5078 4935 5133 5478 5843 6181 6682 7094 6973 6050 3592

4251 6497 7103 7079 6371 5679 5193 4754 4732 4919 5351 5688 5841 5606 5351 5047 5077 5346 5742 6211 6861 7199 7019 5612 3378

3586 5793 7031 7288 6755 6000 5501 5167 5233 5391 5749 5936 6064 5856 5654 5315 5216 5380 5745 6381 7065 7329 6852 5053 3033

2907 4843 6652 7404 7197 6445 5885 5548 5561 5631 5919 6026 6144 5995 5836 5562 5426 5538 5942 6640 7222 7279 6291 4413 2483

2238 3819 5795 7094 7456 6917 6249 5816 5704 5679 5936 6044 6170 6042 5877 5674 5603 5798 6249 6849 7160 6771 5366 3632 1826

1466 2791 4527 6087 7131 7231 6728 6184 5907 5795 6006 6143 6249 6089 5918 5824 5934 6226 6663 6931 6686 5771 4195 2704 1211

875 1701 3114 4565 5912 6821 7001 6757 6484 6351 6546 6704 6766 6559 6388 6334 6465 6739 6882 6564 5733 4403 2952 1712 744

370 918 1682 2889 4212 5474 6347 6721 6774 6790 7018 7244 7328 7099 6890 6807 6891 6822 6401 5557 4306 2972 1739 939 407

145 343 799 1423 2417 3597 4726 5651 6182 6516 6861 7133 7269 7141 6942 6690 6382 5883 5041 3907 2711 1621 892 433 181

352 2205 5064 6293 6254 5653 5063 5817 6078 4895 1718 192 31

1744 5236 7088 6945 6527 6017 5250 5661 6388 6286 4952 1416 113

4201 6996 7155 6518 5878 5694 4720 4727 5568 6353 6453 4068 675

6228 7313 6969 5855 5616 5274 4532 4801 5007 5845 7054 6197 1884

7456 7704 6520 5719 5467 4728 4305 4630 4630 5554 7629 7681 3264

7860 7794 6344 5783 5389 4497 4475 4745 5143 5981 7664 7936 4194

7720 7628 6329 5740 5167 4658 4900 5197 5710 6215 7287 7477 4238

7204 7231 6155 5252 4523 5296 5465 5078 5665 6235 7094 6973 3795

6497 7288 6755 5501 5391 5936 6064 5654 5380 6381 7329 7019 3378

4843 7404 7456 6249 5704 6044 6170 5877 5798 6849 7279 6291 2483

2791 6087 7231 7001 6484 6704 6766 6388 6739 6931 6686 4195 1211

918 2889 5474 6721 6790 7244 7328 6942 6891 6401 4306 1739 407

0.58 0.59 0.77 0.02 0.19 0.81 0.8 0.93 0.95 0.57 0.28 0.41 0.46

0.11 0.78 0.96 0.42 0.7 0.09 0.74 0.23 0.63 0.93 0.74 0.28 1

0.96 0.51 0.39 0.31 0.28 0.61 0.9 0.13 0.51 0.6 0.32 0.09 0.12

0.14 0.13 0.42 0.03 0.57 0.84 0.86 0.91 0.43 0.8 0.97 0.54 0.3

0.15 0.18 0.59 0.63 0.18 0.93 0.43 0.9 0.53 0.41 0.17 0.5 0.11

0.31 0.49 0.48 0.04 0.78 0.03 0.88 0.02 0.73 0.16 0.19 0.17 0.02

0.32 0.24 0.03 0.2 0.93 0.5 0.01 0.96 0.47 0.39 0.31 0.04 0.25

0.35 0.3 0.78 0.69 0.24 0.49 0.28 0.01 0.58 0.24 0.97 0.93 0.79

0.98 0.33 0.79 0.76 0.68 0.73 0.35 0.02 0.89 0.98 0.45 0.67 0.7

0.82 0.83 0.33 0.16 0.84 0.9 0.81 0.37 0.74 0.95 0.41 0.25 0.14

0.95 0.32 0.16 0.49 0.08 0.63 0.56 0.37 0.45 0.16 0.69 0.38 0.27

0.78 0.32 0.97 0.66 0.61 0.39 0.1 0.01 0 0.19 0.72 0.65 0.41

0.58 0.59 0.77 0.02 0.19 0.81 0.8 0.93 0.95 0.57 0.28 0.41 0.46

0.11 0.78 0.96 0.42 0.7 0.09 0.74 0.23 0.63 0.93 0.74 0.28 1

0.96 0.51 0.39 0.31 0.28 0.61 0.9 0.13 0.51 0.6 0.32 0.09 0.12

0.14 0.13 0.42 0.03 0.57 0.84 0.86 0.91 0.43 0.8 0.97 0.54 0.3

0.15 0.18 0.59 0.63 0.18 0.93 0.43 0.9 0.53 0.41 0.17 0.5 0.11

0.31 0.49 0.48 0.04 0.78 0.03 0.88 0.02 0.73 0.16 0.19 0.17 0.02

0.32 0.24 0.03 0.2 0.93 0.5 0.01 0.96 0.47 0.39 0.31 0.04 0.25

0.35 0.3 0.78 0.69 0.24 0.49 0.28 0.01 0.58 0.24 0.97 0.93 0.79

0.98 0.33 0.79 0.76 0.68 0.73 0.35 0.02 0.89 0.98 0.45 0.67 0.7

0.82 0.83 0.33 0.16 0.84 0.9 0.81 0.37 0.74 0.95 0.41 0.25 0.14

0.95 0.32 0.16 0.49 0.08 0.63 0.56 0.37 0.45 0.16 0.69 0.38 0.27

0.78 0.32 0.97 0.66 0.61 0.39 0.1 0.01 0 0.19 0.72 0.65 0.41

0.58 0.59 0.77 0.02 0.19 0.81 0.8 0.93 0.95 0.57 0.28 0.41 0.46

0.11 0.78 0.96 0.42 0.7 0.09 0.74 0.23 0.63 0.93 0.74 0.28 1

0.96 0.51 0.39 0.31 0.28 0.61 0.9 0.13 0.51 0.6 0.32 0.09 0.12

0.14 0.13 0.42 0.03 0.57 0.84 0.86 0.91 0.43 0.8 0.97 0.54 0.3

0.15 0.18 0.59 0.63 0.18 0.93 0.43 0.9 0.53 0.41 0.17 0.5 0.11

0.31 0.49 0.48 0.04 0.78 0.03 0.88 0.02 0.73 0.16 0.19 0.17 0.02

0.32 0.24 0.03 0.2 0.93 0.5 0.01 0.96 0.47 0.39 0.31 0.04 0.25

0.35 0.3 0.78 0.69 0.24 0.49 0.28 0.01 0.58 0.24 0.97 0.93 0.79

0.98 0.33 0.79 0.76 0.68 0.73 0.35 0.02 0.89 0.98 0.45 0.67 0.7

0.82 0.83 0.33 0.16 0.84 0.9 0.81 0.37 0.74 0.95 0.41 0.25 0.14

0.95 0.32 0.16 0.49 0.08 0.63 0.56 0.37 0.45 0.16 0.69 0.38 0.27

0.78 0.32 0.97 0.66 0.61 0.39 0.1 0.01 0 0.19 0.72 0.65 0.41

0.58 0.59 0.77 0.02 0.19 0.81 0.8 0.93 0.95 0.57 0.28 0.41 0.46

0.11 0.78 0.96 0.42 0.7 0.09 0.74 0.23 0.63 0.93 0.74 0.28 1

0.96 0.51 0.39 0.31 0.28 0.61 0.9 0.13 0.51 0.6 0.32 0.09 0.12

0.14 0.13 0.42 0.03 0.57 0.84 0.86 0.91 0.43 0.8 0.97 0.54 0.3

0.15 0.18 0.59 0.63 0.18 0.93 0.43 0.9 0.53 0.41 0.17 0.5 0.11

0.31 0.49 0.48 0.04 0.78 0.03 0.88 0.02 0.73 0.16 0.19 0.17 0.02

0.32 0.24 0.03 0.2 0.93 0.5 0.01 0.96 0.47 0.39 0.31 0.04 0.25

0.35 0.3 0.78 0.69 0.24 0.49 0.28 0.01 0.58 0.24 0.97 0.93 0.79

0.98 0.33 0.79 0.76 0.68 0.73 0.35 0.02 0.89 0.98 0.45 0.67 0.7

0.82 0.83 0.33 0.16 0.84 0.9 0.81 0.37 0.74 0.95 0.41 0.25 0.14

0.95 0.32 0.16 0.49 0.08 0.63 0.56 0.37 0.45 0.16 0.69 0.38 0.27

0.78 0.32 0.97 0.66 0.61 0.39 0.1 0.01 0 0.19 0.72 0.65 0.41


Worklist

PrioritizationImaging

Acquisition

Normal

Mass

Hemorrhage

A

C

Predictive

Probabilities

Hydro

B

AUC=0.91

AUC=0.81

Deep Learning: Converting Images into Text

Deep learning Yann LeCun, Yoshua Bengio & Geoffrey Hinton. Nature. 521, 436–444

What is the bottleneck?

Future Direction

• Easier Creation

• Gradual development

• Adoption – Still requires validation!

• The app store effect

Thank you!Luciano M Prevedello, MD, MPH

[email protected]

Documents

Deep Learning: How it Works and How it Fails