Automatic detection of pulmonary nodules in lung ct images

Automatic Detection of Pulmonary Nodules in

Lung CT ImagesSchool of Information and Mechatronics

Signal and Image Processing Laboratory

Wook-Jin Choi

2

• Introduction• Lung Volume Segmentation• Genetic Programming based Classi-

fier• Hierarchical Block-image Analysis• Shape-based Feature Descriptor• Experimental Results• Conclusions

Contents

3

INTRODUCTION

4

• Lung cancer is the leading cause of cancer deaths.

• Most patients diagnosed with lung cancer already have advanced disease– 40% are stage IV and 30% are III– The current five-year survival rate is only 16%

• Defective nodules are detected at an early stage– The survival rate can be increased

Introduction

5

Introduction

(a) male (b) female

Trends in death rates for selected cancers, United States, 1930-2008

6

• Early detection of lung nodules is ex-tremely important for the diagnosis and clinical management of lung cancer

• Lung cancer had been commonly de-tected and diagnosed on chest radiogra-phy

• Since the early 1990s CT has been re-ported to improve detection and charac-terization of pulmonary nodules

Introduction

7

• CT was introduced in 1971– Sir Godfrey Hounsfield, United Kingdom

• CT utilize computer-processed X-rays– to produce tomographic images or 'slices' of spe-

cific areas of the body

• The Hounsfield unit (HU) scale is a linear transformation of the original linear attenua-tion coefficient measurement into one in which the radio density of distilled water

Computed Tomography

water

waterx1000

HU

8

Computed Tomography

The HU of common substances

Substance HU

Air −1000

Lung −500

Fat −84

Water 0

Cerebrospinal Fluid 15

Blood +30 to +45

Muscle +40

Soft Tissue, Contrast Agent +100 to +300

Bone +700(cancellous bone)to +3000 (dense bone)

Nodule

9

• Lung cancer screening is currently implemented using low-dose CT examinations

• Advanced in CT technology– Rapid image acquisition with thinner image sections– Reduced motion artifacts and improved spatial reso-

lution

• The typical examination generates large-vol-ume data sets

• These large data sets must be evaluated by a radiologist– A fatiguing process

Computed Tomography

10

• The use of pulmonary nodule detection CAD sys-tem can provide an effective solution

• CAD system can assist radiologists by increasing efficiency and potentially improving nodule de-tection

Pulmonary Nodule Detection CAD system

General structure of pulmonary nodule detection system

11

CAD systems Lung segmentationNodule Candidate De-

tectionFalse Positive Reduction

Suzuki et al.(2003)[26] Thresholding Multiple thresholding MTANN

Rubin et al.(2005)[27] Thresholding Surface normal overlapLantern transform and rule-based classifier

Dehmeshki et al.(2007)[28]

Adaptive thresholding Shape-based GATM Rule-based filtering

Suarez-Cuenca et al.(2009)[29]

Thresholding and 3-D connected component la-beling

3-D iris filteringMultiple rule-based LDA classifier

Golosio et al.(2009)[30] Isosurface-triangulation Multiple thresholding Neural network

Ye et al.(2009)[31]3-D adaptive fuzzy seg-mentation

Shape based detectionRule-based filtering and weighted SVM classifier

Sousa et al.(2010)[32] Region growing Structure extraction SVM classifier

Messay et al.(2010)[33]Thresholding and 3-D connected component la-beling

Multiple thresholding and morphological opening

Fisher linear discriminant and quadratic classifier

Riccardi et al.(2011)[34] Iterative thresholding3-D fast radial filtering and scale space analysis

Zernike MIP classification based on SVM

Cascio et al.(2012)[35] Region growing Mass-spring modelDouble-threshold cut and neural network

Pulmonary Nodule Detection CAD system

12

• To evaluate the performance of the proposed method, Lung Image Database Consortium (LIDC) database is applied

• LIDC database, National Cancer Institute (NCI), United States– The LIDC is developing a publicly available database of

thoracic computed tomography (CT) scans as a medical imaging research resource to promote the development of computer-aided detection or characterization of pul-monary nodules

• The database consists of 84 CT scans– 100-400 Digital Imaging and Communication (DICOM)

images– An XML data file containing the physician annotations of

nodules– 148 nodules– The pixel size in the database ranged from 0.5 to 0.76

mm– The reconstruction interval ranged from 1 to 3mm

Experimental Data Set

13

LUNG VOLUME SEGMEN-TATION

14

Lung Volume Segmentation

• Thresholding– Fixed threshold– Optimal threshold– 3-D adaptive fuzzy thresholding

• Lung region extraction– 3-D connectivity with seed

point– 3-D connected component

labeling

• Contour correction– Morphological dilation– Rolling ball algorithm– Chain code representation

15

Fixed Threshold

• Air has an attenuation of -1000 HU• Most lung tissue is in the range of -910 HU

to -500 HU• The chest wall, blood vessel, and bone are

above -500 HU• The low and high intensities are differen-

tiable around the intensity -500 HU

( , , ) ( , , ) 500i x y z I x y z HUS

16

Fixed Threshold

Input CT images, their intensity histograms, and thresh-olded images

17

• A fixed threshold is applicable to segment lung area– The intensity ranges of images are varied by different

acquisition protocols

• To obtain optimal threshold– Iterative approach continues until the threshold con-

verges– The initial threshold : – is i th threshold and new threshold as

Optimal Threshold

(0) 500T HU

( 1)

2i o bT

( )iT

18

Optimal Threshold

Input CT images, their intensity histograms, and thresh-olded images

19

Lung Region Extraction

• White areas– non-body voxels – including lung cavity

• Black areas– body voxels– excluding lung region

• Lung regions are ex-tracted from the non-body voxels by using 3-D connected com-ponent labeling18-connectivity voxels

20

Lung Region Extraction

Labeled images after applying 3-D connected component la-beling

21

• To extract lung volume– Remove rim attached to boundaries of image– The first and the second largest volumes are

selected as the lung region

• The lung region contains small holes– To remove these holes– Morphological hole filling operations are ap-

plied

Lung Region Extraction

|lung first secondS l l

22

Lung Region Extraction

Binary images of the selected lung region

Lung mask images after hole filling

23

• The contour of the lung volume is needed to correct– To include wall side nodule (juxta-pleural nodule)

Contour Correction

Extracted lung region using 3D connected component labeling and contour corrected lung region (containing wall side nodule)

24

Contour Correction

Contour correction using chain-code representation

25

Segmented Lung Volume

26

Results

27

Results

28

GENETIC PROGRAMMING BASED CLASSIFIER FOR DE-TECTION OF PULMONARY NODULES

29

Genetic Programming based Classifier for Detection of Pulmonary nodules

30

Nodule Candidate Detec-tion

• Detection of nodule candi-dates is important

• The performance of nodule detection system relies on the accuracy of candidate detection

• ROI extraction– Optimal multi-thresholding

• Nodule candidates detec-tion and segmentation– Rule-based pruning

31

• The traditional multi-thresholding method needs many steps of grey levels

• An iterative approach is applied to se-lect the threshold value

• The optimal threshold value is calcu-lated on median slice of lung CT scan

Optimal Multi-thresholding

( 1)

2i o bT

32

• The optimal threshold value– A base threshold for multi-thresholding

• Additional six threshold values are ob-tained– Base threshold + 400,+ 300,+ 200,+ 100, -

100, and - 200

Optimal Multi-thresholding

33

• Rule based classifier removes vessels and noise

• Vessel removing– Volume is extremely bigger than nodule – Elongated object

• Noise removing– Radius of ROI is smaller than 3mm– Bigger than 30mm

• Remaining ROIs are nodule candidates

Rule-based Pruning

34

Rule-based Pruning

Rule

Description

R1 Small noise

R2 Vessel

R3 Large noise

R4 Nodule

Pruning rules for nodule candidate detection

35

Rule-based Pruning

36

Nodule Candidate Detection

(d) (e) (f)The results of nodule candidate detection: (a,d) ROIs, (b,e) vessel, and

(c,f) nodule candidates after rule-based pruning

(a) (b) (c)

37

• The features are useful information that de-scribe characteristics of the nodule candi-dates

• In the proposed CAD system, these features will be used to train the GPC

• The proposed feature extraction process consists of two stages– The variety types of features are extracted from

the nodule candidates– Subsets of features are selected and combined

into sub-groups

Feature Extraction

38

Feature Extraction

Index Feature Index Feature

2-D geometric features Mean inside

Area Mean outside

Diameter Variance inside

Perimeter Skewness inside

Circularity Kurtosis inside

3-D geometric features Eigenvalues

Volume 3-D intensity based statistical fea-tures

Compactness Minimum value inside

Bounding Box Dimensions Mean inside

Principal Axis Length Mean outside

Elongation Variance inside2-D intensity based statistical fea-tures Skewness inside

Minimum value inside Kurtosis inside

1f

2f

3f

4f

5f

6f

97 ~ ff

1210 ~ ff

13f

14f

15f

16f

17f

18f

19f

2720 ~ ff

28f

29f

30f

31f

32f

33f

Features for nodule detection

39

Feature vector Description

2-D geometric features

3-D geometric features

2-D intensity-based statistical features

3-D intensity-based statistical features

2-D features

3-D features

Geometric features

Intensity-based statistical features

All features

Feature Extraction

1 1 4{ ,..., }f ff

2 5 13{ ,..., }f ff

43 71 2{ ,..., }f ff

4 28 33{ ,..., }f ff

5 1 3f f f

6 2 4f f f

7 1 2f f f

8 3 4f f f

1 2 3 4f f f f f

Eight different groups of feature vectors

40

• Genetic Programming (GP) – An evolutionary optimiza-

tion technique

• The basic structure of GP is very similar to Genetic Algorithm(GA)

• The chromosome– GA : variable (binary digit

or string)– GP : program (tree or

graph)

Genetic Programming Based Classifier

A function represented as a tree struc-ture

41

• GP chromosome – The terminal set

• The elements of feature vector extracted from nodule can-didate images

• Randomly generated constants with in the range 0,1

– The function set • Four standard arithmetic operator namely plus, minus, mul-

tiply and division• Additional mathematical operators log, exp, abs, sin and cos• All operators in the function set are protected to avoid ex-

ception

• GP evolves combination of the terminal set and function set

Genetic Programming Based Classifier

42

• Fitness Function– evaluate every individuals in GP generation

• True positive rate (TPR)

• Specificity (SPC)– SPC is the value subtracted from 1 to FPR and also called true negative

rate (TNR)

• Area under the ROC curve (AUC)– ROC curve is plotted between TP and FP for different threshold values– AUC is area under the ROC curve and a good measure of classifier per-

formance in different condition

Genetic Programming Based Classifier

TPTPR

TP FN

1 1TN FP

SPC FPRTN FP FP TN

f TPR FPR AUC

43

Genetic Programming Based Classifier

Objective To evolve a optimum classifier with a maximum TPR, SPC and AUC

Function Set +,-,*,protected division, log, exp, abs, sin and cos

Terminal Set Elements of a feature vector and randomly generated con-stants

Fitness Fit(B)=TPR×SPC×AUC

Selection Generational

Wrapper Positive if , else negative

Population Size 300

Generation Size 80

Initial Tree Depth Limit

6

Initial population Ramped half and half

GP Operators prob. Variable ratio of crossover mutation is used

Sampling Tournament

Survival mechanism Keep the best individuals

Real max. tree level

30

44

Genetic Programming Based Classifier

Flow chart for training the proposed GPC

45

Results

Feature spaces for four types of features

2-D geometric feature 3-D geometric feature

2-D intensity-based statistical feature 3-D intensity-based statistical feature

46

• Examples of GPC expression– log(log(log(times(log(f_{20}),times(abs(log(log(times( ti

mes(f_{5},log(f_{31})),log(abs(log(log(log(times(log(f_{9}),log(f_{31}))))))))))),log(times(times(log(f_{5}),log(log( times(times(f_{5},log(log(f_{5}))),times(times(f_{5}, log(f_{9})),log(f_{9})))))),log(f_{31}))))))))

– plus(plus(plus(plus(plus(plus(f_{4},log(times(f_{11},plus(log(plus(f_{9},plus(log(f_{11}),f_{4}))),f_{4})))),f_{4}),plus(log(plus(sin(log(abs(times(f_{11},plus(log(f_{4}),f_{4}))))),f_{4})),f_{4})),log(log(log(times(f_{4},abs(f_{2})))))),log(plus(log(f_{10}),times(f_{1},abs(log(log(times(f_{10},abs(f_{9}))))))))),log(log(times(log(log(times(f_{11},plus(log(times(log(log(times(f_{11},plus(log(f_{4}), f_{4})))),f_{1})),f_{4})))),f_{1}))))

Results

47

Results

Tree representation of the GPC expression

48

Results

Transformed features and classification threshold generated using a GPC

* Nodule+ Non-nodule

49

Results

Training Performance on the training set

Performance on testing set

Feature set

Fitness Accuracy Sensitiv-ity

Speci-ficity

Accuracy Sensitiv-ity

SPC

0.979 99.1% 98.1% 100.0% 78.0% 70.0% 86.0%

0.954 97.8% 95.6% 100.0% 80.5% 86.5% 74.5%

0.859 94.4% 90.6% 98.1% 74.3% 73.0% 75.7%

0.741 90.9% 91.9% 90.0% 61.3% 64.2% 58.3%

0.972 98.8% 98.1% 99.4% 82.3% 82.3% 82.3%

0.951 98.1% 98.1% 98.1% 84.0% 90.7% 77.3%

0.986 99.4% 98.8% 100.0% 86.3% 87.3% 85.3%

0.858 94.7% 93.1% 96.3% 74.3% 74.2% 74.3%

0.988 99.4% 100.0% 98.8% 83.8% 89.2% 78.5%

0.026 1.3% 0.0% 2.6% 4.8% 6.0% 7.9%

Min 0.938 96.9% 100.0% 93.8% 76.7% 76.7% 66.7%

Max 1.000 100.0% 100.0% 100.0% 89.2% 98.3% 90.0%

1f

2f

3f

4f

5f

6f

7f

8ff

GPC results for different feature vectors using a 20–80 dataset

50

Results

Training Performance on the training set

Performance on the testing set

Feature set

Fitness Accuracy Sensitiv-ity

SpecificityAccuracy Sensitiv-ity

SPC

0.876 94.9% 95.3% 94.5% 80.8% 73.7% 87.9%

0.865 94.5% 90.0% 98.9% 86.1% 85.8% 86.3%

0.764 90.9% 88.4% 93.4% 78.9% 75.5% 82.4%

0.628 85.5% 87.1% 83.9% 70.0% 74.5% 65.5%

0.925 96.8% 96.1% 97.6% 88.9% 89.7% 88.2%

0.907 96.2% 93.9% 98.4% 85.7% 85.5% 85.8%

0.940 97.5% 96.8% 98.2% 85.5% 88.7% 82.4%

0.751 90.1% 88.7% 91.6% 80.8% 81.3% 80.3%

0.919 96.7% 95.1% 98.4% 92.3% 94.0% 90.7%

0.028 1.0% 2.0% 1.1% 5.2% 8.0% 5.6%

Min 0.855 94.3% 90.2% 96.7% 83.3% 80.0% 80.0%

Max 0.943 97.5% 96.7% 100.0% 96.7% 100.0% 100.0%

1f

2f

3f4f

5f

6f

7f

8ff

GPC results for different feature vectors using a 50–50 dataset.

51

Training Performance on the training set

Performance on the testing set

Feature set

Fitness Accuracy Sensitiv-ity

Speci-ficity

Accuracy Sensitiv-ity

SPC

0.874 95.0% 93.3% 96.7% 88.3% 88.0% 88.7%

0.890 95.4% 93.3% 97.5% 87.3% 86.0% 88.7%

0.709 89.1% 85.2% 93.0% 81.0% 81.3% 80.7%

0.557 82.0% 87.7% 76.4% 69.3% 78.7% 60.0%

0.872 94.9% 93.3% 96.6% 90.0% 92.0% 88.0%

0.855 94.2% 92.0% 96.4% 88.7% 87.3% 90.0%

0.923 96.8% 96.1% 97.5% 89.3% 89.3% 89.3%

0.723 89.4% 86.9% 92.0% 83.0% 78.7% 87.3%

0.889 95.5% 93.6% 97.4% 89.0% 96.0% 82.0%

0.049 1.8% 3.7% 2.1% 5.2% 4.7% 11.4%

Min 0.829 93.4% 88.5% 91.8% 80.0% 86.7% 60.0%

Max 0.945 97.5% 98.4% 98.4% 96.7% 100.0% 100.0%

Results

1f

2f

3f4f

5f6f

7f

8ff

GPC results for different feature vectors using a 80–20 dataset.

52

HIERARCHICAL BLOCK-IMAGE ANALYSIS FOR PULMONARY NODULE DETECTION

53

Hierarchical Block-image Analy-sis for Pulmonary Nodule Detec-tion

54

Three-Dimensional Block-image Selec-tion based on Entropy Analysis

• Coarse to fine hierar-chical block-image analysis – Block size : 32, 24, 16,

12, 8

• 3-D CT scan is split into 3-D block-images

• The non-informative block-images are fil-tered out by using en-tropy analysis

55

Three-Dimensional Block-image Selec-tion based on Entropy Analysis

Result images after block splitting with respect to various block sizes

56

• Calculate the entropy H(x) on block image

• Select informative blocks by using entropy

Three-Dimensional Block-image Selec-tion based on Entropy Analysis

12 2

1

1( ) ( ) log ( ) log ( )

( )

n n

i i

H x p i p i p ip i

57

Three-Dimensional Block-image Selec-tion based on Entropy Analysis

The entropy histograms of block-images for five different block sizes (x-axis : entropy value, y-axis : number of blocks, (a) 32, (b) 24, (c) 16, (d)

12, and (e) 8)

58

Nodule Candidate Detection based on Block Analysis

• The selected block-image is enhanced

• The object in the selected block-im-age is segmented

• The location of block image is ad-justed

59

Block-image Enhancement

• Block-image enhancement is presented for more accurate analysis• 3-D coherence-enhancing diffusion (CED) filter

– Hessian matrix based– Preserve small spherical structure (nodule) – Enhance tubular structure (vessel)

(a) Input image and (b) the result image after enhancement

60

• Optimal threshold– Iterative approach– Initial threshold : -500HU

– Threshold converges, and optimal threshold obtained

Block Segmentation

61

• The location of block-image should be ad-justed– The segmented object is not located in the center

of the block

• Block location is iteratively updated by using centroid of the segmented object

• The iteration of the adjustment continues un-til the center position converges

• Or distance between the adjusted location and the original location is larger than half of the block size

Block Location Adjustment

62

Block Location Adjustment

Iterations of automatic block location adjustment, upper: 3-D shapes, lower: the median slices of 3-D block; (a) the first; (b) the fifth; and (c) the last it-

erations of adjustment

63

Feature Extraction

• Three different types of features are extracted from nodule candidate block-images

• Nodule has their own shapes– Important characteristics to distinguish

• 2-D and 3-D geometric features de-scribe the shape of nodule candi-dates

64

Feature Extraction

Features for nodule detection

65

• Support vector machine (SVM)– SVM is a useful technique for data classi-

fication– Supervised learning models with associ-

ated learning algorithms– SVM analyze data and recognize pat-

terns– Classification and regression analysis

SVM Classifier

66

• The basic SVM takes a set of input data and predicts two possible classes for each given input

• Training dataset

• The SVM requires the solution of the following optimiza-tion problem

SVM Classifier

67

• SVM can efficiently perform non-linear classification using the kernel trick

• Kernel function

– Polynomial function

– Radial basis function

– Minkowski distance function

SVM Classifier

68

• k-fold cross-validation is applied to evaluated the proposed classifier

• Performance validation measure– The number of true positives (TPs) and

false positives (FPs)– Accuracy, sensitivity, specificity, and

area under the ROC curve.

SVM Classifier

69

Results

k p AUC AccuracySensitiv-

itySpeci-ficity

5 0.25 0.9738 91.52% 87.16% 95.88%

7 0.25 0.9784 93.97% 91.02% 96.92%

10 0.25 0.9736 92.43% 88.97% 95.88%

The k-fold cross validation results of SVM classifiers with ra-dial basis function kernel for different k values

70

Resultsp AUC Accuracy Sensitivity Specificity

SVM-r 0.1 0.9727 84.72% 69.44% 100.00%0.125 0.9746 88.96% 78.70% 99.23%0.25 0.9784 93.97% 91.02% 96.92%0.5 0.9754 92.82% 91.54% 94.10%1 0.9712 91.79% 91.53% 92.05%2 0.9673 92.30% 93.08% 91.53%

SVM-p 0.1 0.4660 47.40% 0.00% 94.81%0.125 0.4632 44.81% 0.26% 89.35%0.25 0.6876 68.26% 86.13% 50.39%0.5 0.9462 89.85% 91.52% 88.18%1 0.9463 90.74% 92.78% 88.69%2 0.9646 92.29% 91.25% 93.32%

SVM-m 0.1 0.8706 82.55% 86.91% 78.19%0.125 0.7051 69.71% 78.46% 60.95%0.25 0.5706 60.68% 68.69% 52.68%0.5 0.5469 59.02% 66.63% 51.41%1 0.5420 58.11% 66.11% 50.12%2 0.5527 57.60% 65.85% 49.36%

The 7-fold cross validation results of SVM classifiers with three different kernel functions, SVM-r: radial basis function, SVM-p: polynomial function, and SVM-m: Minkowski distance function

71

Results

ROC curves of the SVM classifiers with respect to three different kernel functions, SVM-r: radial basis function, SVM-p: polynomial function, and

SVM-m: Minkowski distance function; (a) p = 0:25 and (b) p = 1.

72

PULMONARY NODULE DETECTION USING THREE-DIMENSIONAL SHAPE-BASED FEATURE DESCRIPTOR

73

Pulmonary Nodule Detection using Three-di-mensional Shape-based Feature Descriptor

74

Nodule Candidate Detection

• Eigenvalue decomposi-tion of Hessian Matrix– Dot enhancement filter – Feature extraction

• Multi-scale dot en-hancement filter– Enhance the nodules – The shape of nodules is

like dot or ball

75

Eigenvalue Decomposition of Hessian Matrix

• Multi-scale dot enhancement filter is based on eigenvalue of Hessian matrix

• Hessian Matrix

• Local structure information is obtained by Hessian matrix

76

• Eigenvalue decomposition of Hessian Matrix

– Structure information : surface-ness, curve-ness, and point-ness

– This information is expressed in the three singular tensors (stick, plate, and ball)

• Tensor based representation

Eigenvalue Decomposition of Hessian Matrix

77

• Stick tensor• Plate tensor• Ball tensor

• Surface-ness : saliency , orientation• Curve-ness : saliency , orientation • Point-ness : saliency , no orientation

Eigenvalue Decomposition of Hessian Matrix

78

Eigenvalue Decomposi-tion of Hessian Matrix

79

• The dot enhancement filter is applied to enhance the spherical object, such as nodule

• For each voxel, the dot value is defined as

• are three eigenval-ues from the Hessian matrix

• Gaussian image smoothing with a variety scales is performed prior to the calculation of the gradient for different size of nodules and reducing noise

Multi-scale Dot Enhancement Filtering

80

• Assuming that the diameter of nodule to be detected are in a range the N discrete smoothing scales ___ in the range of can be calculated as

where and each scale has corre-sponding nodule diameter

• The maximum dot value calculated among the dif-ferent smoothing scales

• Five steps smoothing scales are used in the range of nodule diameter [3mm, 30mm]

Multi-scale Dot Enhancement Filtering

(1/( 1))1 0( / ) Nr d d

81

• The image block is extracted as a potential nodule candidate– The dot values are larger than predefined threshold

• The dimension of the image block is• It is noted that the size of the image block

is considered at the relation to the correspond-ing smoothing scale as follows:

where the braces indicate the ceiling function

Multi-scale Dot Enhancement Filtering

82

• A novel shape-based feature extraction method is proposed

• Angular Histogram of Surface Normal Fea-ture

• The feature extraction has important role in the pulmonary nodule CAD system

• The detected nodule candidates are con-sidered as nodules or non-nodules using the extracted feature information

Feature Extraction

83

• Popular approach in the last decade for 2-D images• The scale invariant feature transform (SIFT)

– It can extract salient points and feature descriptors in the most invariant way with respect to scaling, translation, orientation, affine changes and illumination within images

– The SIFT is designed and tested on 2-D images of 3-D ob-ject.

– Allaire et al. proposed fully orientation invariant 3-D SIFT

• The histograms of oriented gradients (HOG)– Describing salient points on 2-D images of 3-D objects– Scherer et al. proposed the 3-D extension of HOG is pro-

posed for 3-D object retrieval

Feature Extraction

84

• The shape-based feature descriptor is extracted for small 3-D object in image patch

• The AHSN feature extraction method is pro-posed to analyze the shape of the target object

• The eigenvalue decomposition of the Hessian matrix is applied to every voxels for target im-age

• The histograms are obtained on surface-ness in-formation– surface saliency :– surface normal vector :

Angular Histogram of Surface Normal Feature Descriptor

85

• The orientation of surface normal vector is ob-tained prior to calculate AHSN feature based on the eigenvalue decomposition of the Hessian matrix

• The orientation of surface normal vector is rep-resented as two kinds of orientation in spherical coordination

Angular Histogram of Surface Normal Feature Descriptor

86

• Two angular histograms are constructed– The orientation θ histogram with n bins is formed

• Each bin covering 180/n degrees• Each sample in the image block added to a histogram

bin is weighted by its surface-ness saliency and nor-malized by total sum of surface-ness saliency

– The orientation φ is quantized into n bins• Each bin covering 360/n degrees• Each sample in the image block added to a histogram

bin is weighted and normalized

– The dimension of feature descriptor is 2n – The extracted AHSN feature is scale-invariant

Angular Histogram of Surface Normal Feature Descriptor

87

Angular Histogram of Surface Normal Feature Descriptor

The extracted AHSN feature for a sphere (nodule model), left – reconstructed 3-D shape, center - orientation θ histogram, right

- orientation φ histogram

88

Angular Histogram of Surface Normal Feature Descriptor

The extracted AHSN feature for a cylinder (vessel model), left – reconstructed 3-D shape, center - orientation θ histogram, right

- orientation φ histogram

89

Angular Histogram of Surface Normal Feature Descriptor

The extracted AHSN feature for a curved surface (wall model), left – reconstructed 3-D shape, center - orientation θ histogram,

right - orientation φ histogram

90

Angular Histogram of Surface Normal Feature Descriptor

The extracted AHSN feature for a pulmonary nodule, left – re-constructed 3-D shape, center - orientation θ histogram, right -

orientation φ histogram

91

Angular Histogram of Surface Normal Feature Descriptor

The extracted AHSN feature for a pulmonary vessel, left – re-constructed 3-D shape, center - orientation θ histogram, right -

orientation φ histogram

92

• Lung wall influence the detection ac-curacy

• For more accurate nodule detection, walls are eliminated from image blocks of nodule candidates

Refinement of Feature Descriptor

93

Wall Detection and Elimination

94

Wall Detection and Elimination

Comparison of AHSN feature for a juxta-pleural nodule at be-fore (1st row) and after (2nd row) wall elimination, left - recon-

structed 3-D shape, center - orientation θ histogram, right - orientation φ histogram

95

Wall Detection and Elimination

Comparison of AHSN feature for a solid nodule at before (1st row) and after (2nd row) wall elimination, left - reconstructed 3-D shape, center - orientation θ histogram, right - orientation

φ histogram

96

• The extracted AHSN feature vectors are analyzed by SVM classifier

• SVM is a useful technique for data classification

• k-fold cross-validation is applied to evaluated the proposed classifier (k = 10)

SVM Classifier

97

Classfier Accuracy Sensitivity Specificity

Before LWE SVM-p 96.4% 98.4% 94.3%

SVM-r 97.8% 98.7% 96.9%

SVM-m 93.9% 95.9% 92.0%

After LWE SVM-p 97.0% 97.9% 96.1%

SVM-r 97.8% 97.4% 98.2%

SVM-m 94.5% 94.6% 94.3%

Results

The results of 10-fold cross validation on different kernel func-tions using SVM as a classier before and after wall elimination

(LWE)

98

Descriptor Accuracy Sensitivity Specificity

SVM-p Gradient 95.1% 96.4% 93.8%

Hessian Ma-trix 97.0% 97.9% 96.1%

SVM-r Gradient 96.1% 96.4% 95.9%

Hessian Ma-trix 97.8% 97.4% 98.2%

SVM-m Gradient 92.8% 93.0% 92.6%

Hessian Ma-trix 94.5% 94.6% 94.3%

Results

The results of 10-fold cross validation on with four different kernel functions based SVMs for the descriptors using gradient

and Hessian matrix

99

Descriptor Accuracy Sensitivity SpecificitySVM-p AHSN 180 97.0% 97.9% 96.1%

AHSN 90 96.9% 97.4% 96.4%AHSN 72 96.9% 98.5% 95.3%AHSN 36 96.0% 97.4% 94.6%3-D SIFT 128 92.9% 93.3% 92.5%3-D HOG 468 95.2% 96.7% 93.8%3-D HOG 216 94.2% 95.1% 93.3%

SVM-r AHSN 180 97.8% 97.4% 98.2%AHSN 90 97.5% 97.2% 97.9%AHSN 72 97.6% 97.4% 97.7%AHSN 36 96.5% 96.9% 96.1%3-D SIFT 128 36.2% 8.4% 100.0%3-D HOG 468 77.2% 36.5% 100.0%3-D HOG 216 89.3% 78.9% 99.7%

SVM-m AHSN 180 94.5% 94.6% 94.3%AHSN 90 95.2% 95.3% 95.1%AHSN 72 95.8% 96.2% 95.4%AHSN 36 94.9% 95.4% 94.3%3-D SIFT 128 88.9% 87.9% 89.9%3-D HOG 468 94.0% 90.9% 94.0%3-D HOG 216 94.7% 94.3% 95.1%

Results

The results of 10-fold cross validation for the different descriptors on various ker-nel functions of SVM classifier

100

Results

ROC curves of the SVM classifiers with respect to three different kernelfunctions, SVM-r: radial basis function, SVM-p: polynomial function, and SVM-

m:Minkowski distance function; (a) p = 0:25 and (b) p = 1.

101

EXPERIMENTAL RESULTS

102

Nodule and non-nodules

Nodules Non-nodules

103

Pulmonary Nodule Detection

(a) (b)

The result of pulmonary nodule detection: (a) 43rd slice, (b) 3-D representation, the detected nodules are indicated by a red color

and the non-nodules are indicated by a white color

104

Genetic Programming based Classifier

AUC Accuracy Specificity Sensitivity FPs/scan

Nodule Candidates Detection 96.6% 51.25

20-80 0.921 76.6% 75.9% 88.3% 12.32

50-50 0.960 86.7% 86.4% 91.7% 6.99

80-20 0.967 89.6% 89.3% 90.9% 5.45

The results of CAD system using GP based classi-

fier

105

Genetic Programming based Classifier

FROC curves of the GPC with respect to three training and test-ing datasets

106

Hierarchical Block-image Analy-sis

AUC Accuracy Specificity Sensitivity FPs/scan

Nodule Candidates Detection 97.3% 60.21

0.1 0.9931 95.89% 99.62% 92.67% 0.23

0.125 0.9934 96.92% 99.11% 93.95% 0.54

0.25 0.9929 97.61% 96.23% 95.28% 2.27

0.5 0.9835 95.15% 93.93% 92.85% 3.65

1 0.9727 92.98% 92.33% 90.63% 4.62

2 0.9584 92.41% 89.74% 90.45% 6.18

The results of CAD system using Hierarchical Block-image

Analysis

107

Hierarchical Block-image Analy-sis

FROC curves of the proposed CAD system with respect to three different

kernel parameters of SVM-r classifiers

108

Three-dimensional Shape-based Feature Descriptor

The overall performance of CAD system for different parameters p of SVM-r classifiers

AUC Accuracy SpecificitySensitiv-ity FPs/scan

Nodule Candidates Detection 97.9% 135.39

AHSN 180 0.9945 97.8% 98.2% 95.4% 2.43

AHSN 90 0.9923 97.5% 97.9% 95.2% 2.84

AHSN 72 0.9895 97.6% 97.7% 95.4% 3.11

109

Three-dimensional Shape-based Feature Descriptor

FROC curves of the proposed CAD system with respect to three differ-ent dimensions of AHSN features

110

CAD systems Nodule size FPs per case Sensitivity

Suzuki et al.(2003)[26] 8 - 20 mm 16.1 80.3%

Rubin et al.(2005)[27] >3 mm 3 76%

Dehmeshki et al.(2007)[28] 3 - 20 mm 14.6 90%

Suarez-Cuenca et al.(2009)[29]

4 - 27 mm 7.7 80%

Golosio et al.(2009)[30] 3 - 30 mm 4.0 79%

Ye et al.(2009)[31] 3 - 20 mm 8.2 90.2%

Sousa et al.(2010)[32] 3 - 40.93 mm - 84.84%

Messay et al.(2010)[33] 3-30 mm 3 82.66%

Riccardi et al.(2011)[34] >3 mm 6.5 71.%

Cascio et al.(2012)[35] 3-30 mm 6.1 97.66%

Genetic Programming 3-30 mm 5.45 90.9%

Hierarchical Block Analysis3-30 mm 2.27 95.2%

Shape-based Feature 3-30 mm 2.43 95.4%

Comparative Analy-sis

111

• Automated pulmonary nodule detec-tion system is studied

• Pulmonary nodule detection CAD sys-tem is an effective solution for early detection of lung cancer

• The proposed systems are based on – Genetic programming based classifier – Hierarchical block-image analysis– 3-D shape-based feature descriptor

Conclusions

112

• The performance of the proposed CAD systems is evaluated on the LIDC database of NCI

• The GPC based system was shown to significantly reduce the false positives while maintaining a high sensitivity– 5.45 FPs/scan, 90.9% sensitivity

• The hierarchical block-image analysis based system has shown more accurate result with improved local object segmentation– 2.27 FPs/scan, 95.28% sensitivity

• Shape-based feature descriptor was applied the nodule detec-tion CAD system that has shown higher accuracy and robust-ness than conventional descriptor– 2.43 FPs/scan, 95.4% sensitivity

• The proposed methods have significantly reduced the false positives in nodule candidates

Conclusions

113

Q & A

114

THANK YOU