Research ArticleFeature Representation Using Deep Autoencoder forLung Nodule Image Classification

Keming Mao 1 Renjie Tang2 Xinqi Wang1 Weiyi Zhang1 and Haoxiang Wu1

1College of Software Northeastern University Shenyang Liaoning Province 110004 China2China Mobile Group Zhejiang Co Ltd Hanzhou Zhejiang Province 310016 China

Correspondence should be addressed to Keming Mao maokmmailneueducn

Received 16 January 2018 Accepted 27 March 2018 Published 7 May 2018

Academic Editor Dimitrios Vlachakis

Copyright copy 2018 Keming Mao et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

This paper focuses on the problem of lung nodule image classification which plays a key role in lung cancer early diagnosis In thiswork we propose a novel model for lung nodule image feature representation that incorporates both local and global charactersFirst lung nodule images are divided into local patches with Superpixel Then these patches are transformed into fixed-lengthlocal feature vectors using unsupervised deep autoencoder (DAE)The visual vocabulary is constructed based on the local featuresand bag of visual words (BOVW) is used to describe the global feature representation of lung nodule image Finally softmaxalgorithm is employed for lung nodule type classification which can assemble the whole training process as an end-to-end modeComprehensive evaluations are conducted on the widely used public available ELCAP lung image database Experimental resultswith regard to different parameter setting data augmentation model sparsity classifier algorithms and model ensemble validatethe effectiveness of our proposed approach

1 Introduction

Lung cancer is one of the most deadly diseases around theworld with about 20 among all cancers in 2016 The 5-yearcure rate is only 182 in spite of great progress in recentdiagnosis and treatment It is noted that if the patient can beaccurately diagnosed in the early stage and suitable treatmentcan be implemented there will be a greater chance for theirsurvival [1]Therefore it is of great significance to do researchabout early diagnosis of lung cancer Computed Tomography(CT) is currently the most popular method among lung can-cer screening technologies [2] CT can generate high resolu-tion data which enable smalllow-contrast lung noduleseffectively detected compared with conventional radiographymethods According to the report of National Lung Screen-ing low-dose CT scan reduces lung cancer mortality by arate of 20 [3] Due to the fact that traditional lung can-cer diagnosis only relies on professional experts two maindrawbacks will be caused (1) subjectivity different doctorshave different diagnostic results for the same CT scan image(2) huge workload reading CT images consumes much time

and effort This makes the efficiency inevitably weakenedWith the development of computer vision technology somebenefits are brought for medical image process and analysisIts efficiency and stability provide auxiliary help for doctorswith automatically or semiautomatically pattern

During the last two decades a number of researchers havebeen devoted to the development of medical image processand analysis with computer vision and machine learningtechnologies especially for lung disease diagnosis [4] Amongthese studies lung nodule image classification has attractedmuch attentions for it is a key step for lung cancer analysisThe lung nodule is characterized by its appearance and rela-tion between surrounding regions Usually the lung nodulecan be classified into 4 types [5] as shown in Figure 1 To bespecific Figures 1(a)ndash1(d) demonstrate nodule types W V Jand P respectively where

W is well-circumscribed nodule located centrally inthe lung without any connection to other structuresV is vascularized nodule that is also central in the lungbut closely attached to the neighbouring vessels

HindawiComplexityVolume 2018 Article ID 3078374 11 pageshttpsdoiorg10115520183078374

2 Complexity

(a)

(b)

(c)

(d)

Figure 1 Demonstration of four types lung nodule image samples (cropped from images in [6])

J is juxtapleural nodule that has a large portion con-nected to the pleuraP is pleural-tail nodule that is near the pleural surfaceconnected by a thin tail

Lung nodule CT image classification includes two mainsteps First feature extraction and representation use seg-mentation filter and statistical method to describe featureof lung nodule based on shape and texture Second classifier

design constructs classifier based on supervised or unsuper-vised machine learning method However these methodsbelong to the fields of traditional image processing andmachine learning which can only characterize the abstrac-tion of lung nodule image in a shallow layer and make theresearch at low level As a result the complex structure of lungnodule makes the classification still a challenging problemThis paper proposes a novel model for lung nodule featurerepresentation and classification The model considers both

Complexity 3

local feature and global feature Lung nodule CT images arefirst divided into local patches with Superpixel and eachpatch is associated with a relatively intact tissue Then localfeature is extracted from each patch with deep autoencoderVisual vocabulary is constructed with local features Globalrepresentation is constructed by bag of visual word (BOVW)model and classifier is trained using softmax algorithm Themain contributions of our work are as follows (i) a novelfeature representationmodel for lungnodule image classifica-tion is proposed Local and global features are constructed byunsupervised deep autoencoder and BOVW model and (ii)comprehensive evaluations are conducted and performanceanalyses are reported from multiple aspects

The structure of this paper is organized as follows Relatedworks are introduced in Section 2 Section 3 gives the frame-work Local feature representation and global feature repre-sentation are given in Sections 4 and 5 Section 6 presentsthe classifier model Experimental evaluations are shownin Section 7 Section 8 concludes this paper

2 Related Works

Many studies have reported the classification of lung nodulein CT image Some representative works are introduced inthis section Many researches designed feature based ontexture shape and intensity of lung nodule image A featureextraction method based on morphological and shape oflung nodule was designed in [7] A subclass local constraintbased method is proposed in [8] Spectral clustering andapproximate affine matrix were used to construct data sub-class and each subclass was used as reference dictionary Thetesting image was represented by sparse dictionary Finallytwo metrics based on approximation and distribution degreewere merged In [9] spectrum was sampled around centerof lung nodule and feature was constructed by FFT Allfeatures were used to construct the dictionary and thenBOVWmode was used to represent the feature of lung nod-ule The Haralick texture feature based on spatial directiondistribution was proposed in [10] and SVM was used asclassifier finally Ridge direction information was adoptedin [11] Local random comparison method was used toconstruct the feature vector and then random forest wasused as classifier Reference [12] first labeled nodule assolid part-solid and nonsolid Then shape based feature wasextracted and kNN was used train the classifier Reference[13] adopted smoothness and irregularity of lung nodule asfeature representation Texture shape statistics and intensitywere extracted as feature representation and ANN was usedas classifier in [14] An eigenvalue of Hessian matrix basedfeature extraction method is adopted in [15] and AdaBoostwas used as classifier Reference [16] used rotation-invariantsecond-order Markov-Gibbs random field to model theintensity distribution of lung nodule and Gibbs energy wasused to describe the feature vector Finally Bayes classifierwas constructed LDA and 3D lattice were used to constructthe mapping between lung nodule image and feature rep-resentation in [17] Reference [18] used topology histogramto represent feature vector of lung nodule and discrimi-nant and 119870-means were used as classifier These methods

represent the lung nodule image feature in relatively lowlevel and they lack sophisticated extraction On the otherhand thesemethods need heavy participation of professionalexpert and they have less generality

Some well-engineered feature extraction and representa-tion methods widely used in computer vision domain wereadopted in lung nodule image classification Reference [22]proposed a method based on texture and context of lungnodule Lung images are divided into nodule level and contextlevel then SIFT LBP and HOG features were extractedReference [19 23] divided lung nodule as foreground andbackground with graph model and conditional random fieldThen SIFT was used to extract feature and SVM was used asclassifier In [24] SIFT feature was first extracted Then PCAand LDA were used for dimension reduction Finally com-plex Gabor response was used for representation In [25] asupervisedmethodwas used for initial classificationwith 128-length SIFT descriptor and weighted Clique was constructedusing 4-length probability vector against the 4 nodule typesThe overlap that lung nodule belongs to different typeswas used for optimizing the final classification result Thesemethods adopt general designed featuresThey obtain higherperformance compared with traditional low-level featureswhile such methods are considered as mid-level abstractionof lung nodule and with less flexibility

Several methods were concerned with other aspects Anensemble based method was applied in [26] for lung noduleclassification Lung nodule image patch was used as inputand six large scale artificial neural networks were trained forclassification Data imbalance problem was discussed in [27]It used downsampling and SMOTE algorithms to train lungnodule classifier

Due to its breakthrough in the field of image processingand speech recognition deep learning has become one of themost hottest topics inmachine learning research and applica-tion [20 28ndash30] High-level abstraction of image object canbe described using deep learning model Meanwhile featureextraction and representation aremore efficient and effectiveIn [28] curvature hu-moment morphology and shapefeatures were used to detect nodule candidate region Thenconvolutional neural network (CNN) was used to extractfeature for candidate region and multiple classifiers weremerged for final result Some changes were made in [29 30]OverFeat was used for CNN parameter initialization In [20]a deep feature extraction with one hidden layer autoencoderwas adopted and a binary decision tree was used as classifierfor lung cancer detection This paper proposes a lung noduleimage classification method combining both local and globalfeature representation Our proposed work is close but hasessential difference from thework of [20]Method in [20] justapplied one hidden layer autoencoder to lung nodule imageOur proposed method uses Superpixel to generate intactpatches and deep autoencoder to extract local feature More-over method BOVW is incorporated for lung nodule globalfeature representation and method in [20] has no considera-tion

3 Framework

Theprocedure of proposed lung nodule classificationmethodis shown in Figure 2 It contains training and testing stages

4 Complexity

TrainingTraining stage

Testing stage

samples

Superpixel

Superpixel

Autoencoder

Autoencoder

Cluster

Assignment

Bag ofvisual words

Bag ofvisual words

Supervisedlearning

Noduletypes

Local

Local

Local

Local

patch set

patches

feature

features

Visual

Visual

vocabulary

words

Globalrepresentation

Globalrepresentation

Classifiermodel

Classifiermodel

Figure 2 Framework of the proposed method

In training stage lung nodule image samples are used asinput and the output is a trained classifier model Collectedtraining image samples are first divided into local patcheswith Superpixel Local patches are assignedwith no class labeland constitute local patch set With the local patch set localfeatures are extracted by unsupervised learning model deepautoencoder Next visual vocabulary is constructed basedon clustering all local feature vectors A lung nodule imagecan therefore be described by a global feature representationwith bag-of-visual-words model Finally classifier is trainedby supervised learning with nodule type labels In testingstage the input is a lung nodule image with unknown typeand the output is its predicted type label Similar to trainingstage a test image is divided into multiple patches Each localpatch is transformed into local feature and assigned witha visual word Finally global feature representation of testimage is used for classification by the trained model Detailsof the proposed method will be introduced in the followingsections

4 Local Feature Representation

Local feature representation is proposed in this section Theprocess consists of two steps (1) local patch generation and(2) local feature extraction and representation

41 Local Patch Generation Decomposing a lung image intosmall patches is useful and practical and for important tissuescan be picked up and unrelated ones can be get rid of Asshown in (1) a lung nodule image 119909 can be composed of agroup of image patches 119901119894 where 119899 denotes the number oflocal patches

119909 = 1199011 1199012 119901119899 (1)

The location and scale of local patches are determinedthrough generation [22 24] Useless part will be contained forlarge size patch while small part may not cover enough intacttissue Superpixel is a popular method that can partition the

image into small similar regions with better representative-ness and integrity [31] So it is adopted in this work

Figure 3 illustrates the process of the proposed local patchgeneration method For a lung nodule image (Figure 3(a))it is first segmented into local patches using Superpixel anda Superpixel map is obtained (Figure 3(b)) Local patchesessentially indicate the uniform regions Figure 3(c) is anindividual patch sample However the region that Figure 3(c)gives is an irregular shape and it is inconvenient for local fea-ture extraction and representation So we expand local patchwith its minimum enclosing rectangle as shown in Fig-ure 3(d) Finally a lung nodule image is decomposed into a setof local patches as shown in Figure 3(e) Besides there aresome additional criterions to determine whether an imagepatch is qualified for local feature extraction

(i) Let 119901119894 be a local patch it is removed when the area of119901119894 is larger than 119860max or smaller than 119860min

(ii) Let 119901119894 and 119901119895 be two local patches if the ratio betweentheir intersection and their union is larger than 119874119905then the smaller one is removed119860max 119860min and 119874119905 are predefined thresholds

42 Local Feature Extraction and Representation With therapid development of unsupervised learning in recent yearsusing unlabeled data to extract feature with autoencoder hasbecome an appropriate way Autoencodermodel is essentiallya multilayered neural networks Its original version is aforward network with one hidden layer Let 119909119894 be the inputdata 119886119894119895 be the activation of unit 119894 in layer 119895 and 119908119894 be thematrix of weights controlling function mapping from layer 119894to layer 119894 +1 If layer 119894 has 119904119894 units and layer 119894 +1 has 119904119894+1 unitsthen 119908119894 will be a matrix with size of 119904119894 lowast 119904119894+1 The activationcan be formulated as (2) where 11988621 is the 1st unit in the 2ndlayer and 1199090ndash1199093 are 4 input features

11988621 = 119892 (1199081101199090 + 1199081111199091 + 1199081121199092 + 1199081131199093) (2)

The main difference between ordinary forward neuralnetwork and autoencoder is that an autoencoderrsquos output is

Complexity 5

(a) (b) (c) (d) (e)

Figure 3 The process of local patch generation by Superpixel

Input Hidden Hidden Hidden Outputlayer layer layer layer layer

L1 L3 L4 L5L2

Encoding Decoding

Image patch Image patchReconstructed

Figure 4 The process of stacked deep autoencoder model

always the same as or similar to its input The basic formulacan be expressed as follows

119886 = ℎ (119909) = 119891 (119882119864119909 + 119887) 1199091015840 = ℎ1015840 (119909) = 119892 (119882119863119886 + 1198871015840) = 119892 (119882119863ℎ (119909) + 1198871015840)

(3)

An autoencoder can be seen as a combination of encoderand decoder The encoder includes an input layer and ahidden layer which converts an input image 119909 into featurevector 119886 The decoder includes a hidden layer and an outputlayer that transform feature 119886 to output feature 1199091015840 119882119864 and119882119863 are weight matrices of encoder and decoder respectivelyFunctions 119891(sdot) and 119892(sdot) can be either sigmoid or tanh activa-tion functions which is used to activate the unit in each layerWhen 1199091015840 approximates 119909 it is considered that the input fea-ture can be reconstructed from an abstract and compressedoutput feature vector 119886 The cost function can be generallydefined as follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2 + 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2 (4)

A deep autoencoder can be constructed by stacking morehidden layers As shown in Figure 4 there are 5 layers inthe model (including 3 hidden layers) 1198711 to 1198713 are encoding

layers and 1198713 to 1198715 are decoding layers 119871 119894 is used as the inputof the layer 119871 119894+1 and the weights can be gained based on (3)There are 2 stacked autoencodersThe activation of 1st hiddenlayer is the input of the 2nd stacked autoencoderThenetworkcan be trained in a fine-tuning stage by minimizing the equa-tion (4) 1198821 and 1198824 are trained through the encoding anddecoding weights of the 1st stacked autoencoder and1198822 and1198823 are trained through the encoding and decoding weightsof the 2nd stacked autoencoder Finally the whole networkcan be constructed layer by layer in a stacked way MoreoverFigure 4 just shows an example of symmetric encoding anddecoding structures and other variational structures can alsobe adopted

Therefore each local patch of a lung nodule image 119901119894 canbe represented by a fixed-length feature vector 119901119891119894 with deepautoencoder model Then (1) is transformed as follows

119909 = 1199011 1199012 119901119899 = 1199011198911 1199011198912 119901119891119899 (5)

5 Global Feature Representation

For BOVW model visual vocabulary is first constructedbased on clustering all local patch descriptors (local featurerepresentation) generated by a set of training images Theneach lung nodule image can be represented globally by ahistogram of visual words Distance between histograms of

6 Complexity

VisualVocabulary

K-meanscluster

middot middot middot middot middot middot

Figure 5 Procedure of BOVW representation of lung nodule image

visual words can be treated as similarity between lung noduleimage samples

Recall that a lung nodule image is decomposed into agroup of local patches and each patch is represented witha feature vector based on deep autoencoder Assume thereare 119863 local patches generated from all lung nodule train-ing images and each local patch is represented with 119889-dimensional feature vector then all local feature vectorscan be assembled into a feature space with size of 119889 lowast 119863Clustering is performed with 119889 lowast 119863 features and 119896-meansclustering method is adopted since it has relatively low timeand storage complexity irrelevant to data process orderingEach cluster center 119888119894 represents a visual word 119894 and 119896 clustercenters constitute the visual vocabulary A lung nodule imagesample 119909 can be represented by the encoded local patchesas a bag which is the occurrence frequency of visual wordin vocabulary To get the histogram representation ℎ(119909) ofan image 119909 all local patch feature vectors of 119909 are mappedonto the cluster center of the visual vocabulary and eachlocal feature is assigned with the label of its closest clustercenter using Euclidean distance in feature space Then a 119896-bins histogram ℎ(119909) is obtained by counting all the label oflocal patches generated by image 119909 as shown in (6) Figure 5exhibits the procedure of global representation of lung noduleimage

ℎ (119909) = [ℎ (119909)1 ℎ (119909)2 ℎ (119909)119896] (6)

6 Classifier Model

With global representation of lung nodule image softmaxalgorithm is used to train nodule type classifier Let (1199091 1199101)(1199092 1199102) (119909119898 119910119898) denote training data set 119909119894 denotes thelung nodule image sample and 119910119894 isin 0 1 2 3 denotes noduletype label

For an input image sample 119909119894 we want to compute 119901(119910 =119895 | 119909119894) (119895 isin 0 1 2 3) The output a 4-dimensional vectoris estimated to represent the probability of each type that119909119894 belongs to The hypothesis function can be expressed asfollows

ℎ120579 (119909119894) =[[[[[[

119901 (119910119894 = 0 | 119909119894 120579)119901 (119910119894 = 1 | 119909119894 120579)119901 (119910119894 = 2 | 119909119894 120579)119901 (119910119894 = 3 | 119909119894 120579)

]]]]]]= 1sum3119895=0 119890120579119879119895 119909119894

[[[[[[[[

1198901205791198790 1199091198941198901205791198791 1199091198941198901205791198792 1199091198941198901205791198793 119909119894

]]]]]]]]

(7)

where 120579 = 1205790 1205791 1205792 1205793 is model parameter set Thisequation normalizes the result and makes the sum to 1 Fortraining procedure the loss function is given as follows

119869 (120579) = minus 1119898 [

[119898

sum119894=1

3

sum119895=0

1 119910119894 = 119895 log 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

]]

+ 1205822119896

sum119894=1

119899

sum119895=0

1205792119894119895(8)

where 1sdot is an indicative function and stochastic gradientdescent (SGD) is used for function optimization and thecorresponding derivative functions are given as follows

nabla120579119895119869 (120579)

= minus 1119898119898

sum119894=1

[119909119894 (1 119910119894 = 119895 minus 119901 (119910119894 = 119895 | 119909119894 120579))] + 120582120579119895

119901 (119910119894 = 119895 | 119909119894 120579) = 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

(9)

Complexity 7

Figure 6 Demonstration of lung CT images (downloaded from [6])

7 Experimental Evaluations

71 Dataset and Program Implementation In order to eval-uate the performance of the proposed lung nodule imagerepresentation and classification method a widely usedpublic available lung nodule image dataset ELCAP is used fortesting [6] The dataset contains 379 lung CT images whichare collected from 50 distinct low-dose CT lung scans Thecenter position of lung nodule is marked in an extra lowastcsv file

Figure 6 demonstrates the lung nodule CT scan imageswhich are sampled from different slices Table 1 shows theformat of a lowastcvs file Each row denotes a lung nodule The4th column indicates the slice number where the lung noduleexists The 2nd and 3rd columns give the positions that thelung nodule is located in In this section lung nodule imagesare cropped from the raw CT images based on the 119909- and 119910-coordinates of nodule center given inTable 1The raw lungCTscan image is fixedwith 512lowast512pixels and the croppednod-ule images are too small to implement the algorithm There-fore we further resize the cropped lung nodule image into180lowast180 pixels with bicubicmethodThe lung nodule imagesare labeled with one of four types according to the guidanceby an expert Programs are implemented with Matlab 2016aprogramming language and tested on a Pentium i7 CPU 8GRAM NVIDIA GTX 960 GPU Windows OS PC

The experiments include the following aspects (1)parameter setting (2) classification rate with different param-eters (3) classification rate with data augmentation (4)classification rate with model sparsity (5) classification ratewith different classifier algorithms (6) comparing with othermethods (7) classification rate with model ensemble Theperformance of lung nodule image classification is computedwith overall classification rate as shown in the following

Classification rate = 119873correct119873all

(10)

where 119873correct is the number of correctly labeled images and119873all is the number of all testing images Cross validationmodeis adopted The dataset is divided into 8 groups 7 randomlychosen groups are used for training and the left group is used

Table 1 Format of lung nodule position

Type 119909 119910 SliceNodule 98 218 54Nodule 355 153 84Nodule 139 366 130Nodule 436 213 169Nodule 372 163 239Nodule 328 175 229Nodule 54 224 169

for testing This process is repeated 7 times and the result iscomputed by averaging 7 independent tests

72 Parameter Setting The parameters are needed to be setin local patch generation local feature representation andglobal feature representation For local patch generation weneed to set the number of superpixels that each lung noduleimage generates For local feature representation the numberof hidden layers and nodes that each layer contains shouldbe set For global feature representation the size of visualvocabulary should be set

As shown in Table 2 the number of patches that eachlung nodule image generates is set with 15 20 25 and 30The number of hidden layers in deep autoencoder is set with1 2 and 3 The number of nodes in deep autoencoder is setwith 50 75 100 125 and 150 The size of visual vocabulary isset with 200 300 400 and 500 The classification rate is eva-luated on the combination of these parameters For conve-nience parameters are expressed with 1199011 1199012 1199013 and 1199014 re-spectively

73 Classification Rate with Different Parameters The sizeof local patch is set with 30 lowast 30 pixels in our experimentTable 3 gives the average performance of lung nodule imageclassification based on combination of parameters 1199011 1199012 1199013and 1199014 It can be seen that classification model with 1199011 = 251199012 = 2 1199013 = (100 50) and 1199014 = 400 gets the optimal result

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

Complexity 3

local feature and global feature Lung nodule CT images arefirst divided into local patches with Superpixel and eachpatch is associated with a relatively intact tissue Then localfeature is extracted from each patch with deep autoencoderVisual vocabulary is constructed with local features Globalrepresentation is constructed by bag of visual word (BOVW)model and classifier is trained using softmax algorithm Themain contributions of our work are as follows (i) a novelfeature representationmodel for lungnodule image classifica-tion is proposed Local and global features are constructed byunsupervised deep autoencoder and BOVW model and (ii)comprehensive evaluations are conducted and performanceanalyses are reported from multiple aspects

The structure of this paper is organized as follows Relatedworks are introduced in Section 2 Section 3 gives the frame-work Local feature representation and global feature repre-sentation are given in Sections 4 and 5 Section 6 presentsthe classifier model Experimental evaluations are shownin Section 7 Section 8 concludes this paper

2 Related Works

Many studies have reported the classification of lung nodulein CT image Some representative works are introduced inthis section Many researches designed feature based ontexture shape and intensity of lung nodule image A featureextraction method based on morphological and shape oflung nodule was designed in [7] A subclass local constraintbased method is proposed in [8] Spectral clustering andapproximate affine matrix were used to construct data sub-class and each subclass was used as reference dictionary Thetesting image was represented by sparse dictionary Finallytwo metrics based on approximation and distribution degreewere merged In [9] spectrum was sampled around centerof lung nodule and feature was constructed by FFT Allfeatures were used to construct the dictionary and thenBOVWmode was used to represent the feature of lung nod-ule The Haralick texture feature based on spatial directiondistribution was proposed in [10] and SVM was used asclassifier finally Ridge direction information was adoptedin [11] Local random comparison method was used toconstruct the feature vector and then random forest wasused as classifier Reference [12] first labeled nodule assolid part-solid and nonsolid Then shape based feature wasextracted and kNN was used train the classifier Reference[13] adopted smoothness and irregularity of lung nodule asfeature representation Texture shape statistics and intensitywere extracted as feature representation and ANN was usedas classifier in [14] An eigenvalue of Hessian matrix basedfeature extraction method is adopted in [15] and AdaBoostwas used as classifier Reference [16] used rotation-invariantsecond-order Markov-Gibbs random field to model theintensity distribution of lung nodule and Gibbs energy wasused to describe the feature vector Finally Bayes classifierwas constructed LDA and 3D lattice were used to constructthe mapping between lung nodule image and feature rep-resentation in [17] Reference [18] used topology histogramto represent feature vector of lung nodule and discrimi-nant and 119870-means were used as classifier These methods

represent the lung nodule image feature in relatively lowlevel and they lack sophisticated extraction On the otherhand thesemethods need heavy participation of professionalexpert and they have less generality

Some well-engineered feature extraction and representa-tion methods widely used in computer vision domain wereadopted in lung nodule image classification Reference [22]proposed a method based on texture and context of lungnodule Lung images are divided into nodule level and contextlevel then SIFT LBP and HOG features were extractedReference [19 23] divided lung nodule as foreground andbackground with graph model and conditional random fieldThen SIFT was used to extract feature and SVM was used asclassifier In [24] SIFT feature was first extracted Then PCAand LDA were used for dimension reduction Finally com-plex Gabor response was used for representation In [25] asupervisedmethodwas used for initial classificationwith 128-length SIFT descriptor and weighted Clique was constructedusing 4-length probability vector against the 4 nodule typesThe overlap that lung nodule belongs to different typeswas used for optimizing the final classification result Thesemethods adopt general designed featuresThey obtain higherperformance compared with traditional low-level featureswhile such methods are considered as mid-level abstractionof lung nodule and with less flexibility

Several methods were concerned with other aspects Anensemble based method was applied in [26] for lung noduleclassification Lung nodule image patch was used as inputand six large scale artificial neural networks were trained forclassification Data imbalance problem was discussed in [27]It used downsampling and SMOTE algorithms to train lungnodule classifier

Due to its breakthrough in the field of image processingand speech recognition deep learning has become one of themost hottest topics inmachine learning research and applica-tion [20 28ndash30] High-level abstraction of image object canbe described using deep learning model Meanwhile featureextraction and representation aremore efficient and effectiveIn [28] curvature hu-moment morphology and shapefeatures were used to detect nodule candidate region Thenconvolutional neural network (CNN) was used to extractfeature for candidate region and multiple classifiers weremerged for final result Some changes were made in [29 30]OverFeat was used for CNN parameter initialization In [20]a deep feature extraction with one hidden layer autoencoderwas adopted and a binary decision tree was used as classifierfor lung cancer detection This paper proposes a lung noduleimage classification method combining both local and globalfeature representation Our proposed work is close but hasessential difference from thework of [20]Method in [20] justapplied one hidden layer autoencoder to lung nodule imageOur proposed method uses Superpixel to generate intactpatches and deep autoencoder to extract local feature More-over method BOVW is incorporated for lung nodule globalfeature representation and method in [20] has no considera-tion

3 Framework

Theprocedure of proposed lung nodule classificationmethodis shown in Figure 2 It contains training and testing stages

4 Complexity

TrainingTraining stage

Testing stage

samples

Superpixel

Superpixel

Autoencoder

Autoencoder

Cluster

Assignment

Bag ofvisual words

Bag ofvisual words

Supervisedlearning

Noduletypes

Local

Local

Local

Local

patch set

patches

feature

features

Visual

Visual

vocabulary

words

Globalrepresentation

Globalrepresentation

Classifiermodel

Classifiermodel

Figure 2 Framework of the proposed method

In training stage lung nodule image samples are used asinput and the output is a trained classifier model Collectedtraining image samples are first divided into local patcheswith Superpixel Local patches are assignedwith no class labeland constitute local patch set With the local patch set localfeatures are extracted by unsupervised learning model deepautoencoder Next visual vocabulary is constructed basedon clustering all local feature vectors A lung nodule imagecan therefore be described by a global feature representationwith bag-of-visual-words model Finally classifier is trainedby supervised learning with nodule type labels In testingstage the input is a lung nodule image with unknown typeand the output is its predicted type label Similar to trainingstage a test image is divided into multiple patches Each localpatch is transformed into local feature and assigned witha visual word Finally global feature representation of testimage is used for classification by the trained model Detailsof the proposed method will be introduced in the followingsections

4 Local Feature Representation

Local feature representation is proposed in this section Theprocess consists of two steps (1) local patch generation and(2) local feature extraction and representation

41 Local Patch Generation Decomposing a lung image intosmall patches is useful and practical and for important tissuescan be picked up and unrelated ones can be get rid of Asshown in (1) a lung nodule image 119909 can be composed of agroup of image patches 119901119894 where 119899 denotes the number oflocal patches

119909 = 1199011 1199012 119901119899 (1)

The location and scale of local patches are determinedthrough generation [22 24] Useless part will be contained forlarge size patch while small part may not cover enough intacttissue Superpixel is a popular method that can partition the

image into small similar regions with better representative-ness and integrity [31] So it is adopted in this work

Figure 3 illustrates the process of the proposed local patchgeneration method For a lung nodule image (Figure 3(a))it is first segmented into local patches using Superpixel anda Superpixel map is obtained (Figure 3(b)) Local patchesessentially indicate the uniform regions Figure 3(c) is anindividual patch sample However the region that Figure 3(c)gives is an irregular shape and it is inconvenient for local fea-ture extraction and representation So we expand local patchwith its minimum enclosing rectangle as shown in Fig-ure 3(d) Finally a lung nodule image is decomposed into a setof local patches as shown in Figure 3(e) Besides there aresome additional criterions to determine whether an imagepatch is qualified for local feature extraction

(i) Let 119901119894 be a local patch it is removed when the area of119901119894 is larger than 119860max or smaller than 119860min

(ii) Let 119901119894 and 119901119895 be two local patches if the ratio betweentheir intersection and their union is larger than 119874119905then the smaller one is removed119860max 119860min and 119874119905 are predefined thresholds

42 Local Feature Extraction and Representation With therapid development of unsupervised learning in recent yearsusing unlabeled data to extract feature with autoencoder hasbecome an appropriate way Autoencodermodel is essentiallya multilayered neural networks Its original version is aforward network with one hidden layer Let 119909119894 be the inputdata 119886119894119895 be the activation of unit 119894 in layer 119895 and 119908119894 be thematrix of weights controlling function mapping from layer 119894to layer 119894 +1 If layer 119894 has 119904119894 units and layer 119894 +1 has 119904119894+1 unitsthen 119908119894 will be a matrix with size of 119904119894 lowast 119904119894+1 The activationcan be formulated as (2) where 11988621 is the 1st unit in the 2ndlayer and 1199090ndash1199093 are 4 input features

11988621 = 119892 (1199081101199090 + 1199081111199091 + 1199081121199092 + 1199081131199093) (2)

The main difference between ordinary forward neuralnetwork and autoencoder is that an autoencoderrsquos output is

Complexity 5

(a) (b) (c) (d) (e)

Figure 3 The process of local patch generation by Superpixel

Input Hidden Hidden Hidden Outputlayer layer layer layer layer

L1 L3 L4 L5L2

Encoding Decoding

Image patch Image patchReconstructed

Figure 4 The process of stacked deep autoencoder model

always the same as or similar to its input The basic formulacan be expressed as follows

119886 = ℎ (119909) = 119891 (119882119864119909 + 119887) 1199091015840 = ℎ1015840 (119909) = 119892 (119882119863119886 + 1198871015840) = 119892 (119882119863ℎ (119909) + 1198871015840)

(3)

An autoencoder can be seen as a combination of encoderand decoder The encoder includes an input layer and ahidden layer which converts an input image 119909 into featurevector 119886 The decoder includes a hidden layer and an outputlayer that transform feature 119886 to output feature 1199091015840 119882119864 and119882119863 are weight matrices of encoder and decoder respectivelyFunctions 119891(sdot) and 119892(sdot) can be either sigmoid or tanh activa-tion functions which is used to activate the unit in each layerWhen 1199091015840 approximates 119909 it is considered that the input fea-ture can be reconstructed from an abstract and compressedoutput feature vector 119886 The cost function can be generallydefined as follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2 + 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2 (4)

A deep autoencoder can be constructed by stacking morehidden layers As shown in Figure 4 there are 5 layers inthe model (including 3 hidden layers) 1198711 to 1198713 are encoding

layers and 1198713 to 1198715 are decoding layers 119871 119894 is used as the inputof the layer 119871 119894+1 and the weights can be gained based on (3)There are 2 stacked autoencodersThe activation of 1st hiddenlayer is the input of the 2nd stacked autoencoderThenetworkcan be trained in a fine-tuning stage by minimizing the equa-tion (4) 1198821 and 1198824 are trained through the encoding anddecoding weights of the 1st stacked autoencoder and1198822 and1198823 are trained through the encoding and decoding weightsof the 2nd stacked autoencoder Finally the whole networkcan be constructed layer by layer in a stacked way MoreoverFigure 4 just shows an example of symmetric encoding anddecoding structures and other variational structures can alsobe adopted

Therefore each local patch of a lung nodule image 119901119894 canbe represented by a fixed-length feature vector 119901119891119894 with deepautoencoder model Then (1) is transformed as follows

119909 = 1199011 1199012 119901119899 = 1199011198911 1199011198912 119901119891119899 (5)

5 Global Feature Representation

For BOVW model visual vocabulary is first constructedbased on clustering all local patch descriptors (local featurerepresentation) generated by a set of training images Theneach lung nodule image can be represented globally by ahistogram of visual words Distance between histograms of

6 Complexity

VisualVocabulary

K-meanscluster

middot middot middot middot middot middot

Figure 5 Procedure of BOVW representation of lung nodule image

visual words can be treated as similarity between lung noduleimage samples

Recall that a lung nodule image is decomposed into agroup of local patches and each patch is represented witha feature vector based on deep autoencoder Assume thereare 119863 local patches generated from all lung nodule train-ing images and each local patch is represented with 119889-dimensional feature vector then all local feature vectorscan be assembled into a feature space with size of 119889 lowast 119863Clustering is performed with 119889 lowast 119863 features and 119896-meansclustering method is adopted since it has relatively low timeand storage complexity irrelevant to data process orderingEach cluster center 119888119894 represents a visual word 119894 and 119896 clustercenters constitute the visual vocabulary A lung nodule imagesample 119909 can be represented by the encoded local patchesas a bag which is the occurrence frequency of visual wordin vocabulary To get the histogram representation ℎ(119909) ofan image 119909 all local patch feature vectors of 119909 are mappedonto the cluster center of the visual vocabulary and eachlocal feature is assigned with the label of its closest clustercenter using Euclidean distance in feature space Then a 119896-bins histogram ℎ(119909) is obtained by counting all the label oflocal patches generated by image 119909 as shown in (6) Figure 5exhibits the procedure of global representation of lung noduleimage

ℎ (119909) = [ℎ (119909)1 ℎ (119909)2 ℎ (119909)119896] (6)

6 Classifier Model

With global representation of lung nodule image softmaxalgorithm is used to train nodule type classifier Let (1199091 1199101)(1199092 1199102) (119909119898 119910119898) denote training data set 119909119894 denotes thelung nodule image sample and 119910119894 isin 0 1 2 3 denotes noduletype label

For an input image sample 119909119894 we want to compute 119901(119910 =119895 | 119909119894) (119895 isin 0 1 2 3) The output a 4-dimensional vectoris estimated to represent the probability of each type that119909119894 belongs to The hypothesis function can be expressed asfollows

ℎ120579 (119909119894) =[[[[[[

119901 (119910119894 = 0 | 119909119894 120579)119901 (119910119894 = 1 | 119909119894 120579)119901 (119910119894 = 2 | 119909119894 120579)119901 (119910119894 = 3 | 119909119894 120579)

]]]]]]= 1sum3119895=0 119890120579119879119895 119909119894

[[[[[[[[

1198901205791198790 1199091198941198901205791198791 1199091198941198901205791198792 1199091198941198901205791198793 119909119894

]]]]]]]]

(7)

where 120579 = 1205790 1205791 1205792 1205793 is model parameter set Thisequation normalizes the result and makes the sum to 1 Fortraining procedure the loss function is given as follows

119869 (120579) = minus 1119898 [

[119898

sum119894=1

3

sum119895=0

1 119910119894 = 119895 log 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

]]

+ 1205822119896

sum119894=1

119899

sum119895=0

1205792119894119895(8)

where 1sdot is an indicative function and stochastic gradientdescent (SGD) is used for function optimization and thecorresponding derivative functions are given as follows

nabla120579119895119869 (120579)

= minus 1119898119898

sum119894=1

[119909119894 (1 119910119894 = 119895 minus 119901 (119910119894 = 119895 | 119909119894 120579))] + 120582120579119895

119901 (119910119894 = 119895 | 119909119894 120579) = 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

(9)

Complexity 7

Figure 6 Demonstration of lung CT images (downloaded from [6])

7 Experimental Evaluations

71 Dataset and Program Implementation In order to eval-uate the performance of the proposed lung nodule imagerepresentation and classification method a widely usedpublic available lung nodule image dataset ELCAP is used fortesting [6] The dataset contains 379 lung CT images whichare collected from 50 distinct low-dose CT lung scans Thecenter position of lung nodule is marked in an extra lowastcsv file

Figure 6 demonstrates the lung nodule CT scan imageswhich are sampled from different slices Table 1 shows theformat of a lowastcvs file Each row denotes a lung nodule The4th column indicates the slice number where the lung noduleexists The 2nd and 3rd columns give the positions that thelung nodule is located in In this section lung nodule imagesare cropped from the raw CT images based on the 119909- and 119910-coordinates of nodule center given inTable 1The raw lungCTscan image is fixedwith 512lowast512pixels and the croppednod-ule images are too small to implement the algorithm There-fore we further resize the cropped lung nodule image into180lowast180 pixels with bicubicmethodThe lung nodule imagesare labeled with one of four types according to the guidanceby an expert Programs are implemented with Matlab 2016aprogramming language and tested on a Pentium i7 CPU 8GRAM NVIDIA GTX 960 GPU Windows OS PC

The experiments include the following aspects (1)parameter setting (2) classification rate with different param-eters (3) classification rate with data augmentation (4)classification rate with model sparsity (5) classification ratewith different classifier algorithms (6) comparing with othermethods (7) classification rate with model ensemble Theperformance of lung nodule image classification is computedwith overall classification rate as shown in the following

Classification rate = 119873correct119873all

(10)

where 119873correct is the number of correctly labeled images and119873all is the number of all testing images Cross validationmodeis adopted The dataset is divided into 8 groups 7 randomlychosen groups are used for training and the left group is used

Table 1 Format of lung nodule position

Type 119909 119910 SliceNodule 98 218 54Nodule 355 153 84Nodule 139 366 130Nodule 436 213 169Nodule 372 163 239Nodule 328 175 229Nodule 54 224 169

for testing This process is repeated 7 times and the result iscomputed by averaging 7 independent tests

72 Parameter Setting The parameters are needed to be setin local patch generation local feature representation andglobal feature representation For local patch generation weneed to set the number of superpixels that each lung noduleimage generates For local feature representation the numberof hidden layers and nodes that each layer contains shouldbe set For global feature representation the size of visualvocabulary should be set

As shown in Table 2 the number of patches that eachlung nodule image generates is set with 15 20 25 and 30The number of hidden layers in deep autoencoder is set with1 2 and 3 The number of nodes in deep autoencoder is setwith 50 75 100 125 and 150 The size of visual vocabulary isset with 200 300 400 and 500 The classification rate is eva-luated on the combination of these parameters For conve-nience parameters are expressed with 1199011 1199012 1199013 and 1199014 re-spectively

73 Classification Rate with Different Parameters The sizeof local patch is set with 30 lowast 30 pixels in our experimentTable 3 gives the average performance of lung nodule imageclassification based on combination of parameters 1199011 1199012 1199013and 1199014 It can be seen that classification model with 1199011 = 251199012 = 2 1199013 = (100 50) and 1199014 = 400 gets the optimal result

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

4 Complexity

TrainingTraining stage

Testing stage

samples

Superpixel

Superpixel

Autoencoder

Autoencoder

Cluster

Assignment

Bag ofvisual words

Bag ofvisual words

Supervisedlearning

Noduletypes

Local

Local

Local

Local

patch set

patches

feature

features

Visual

Visual

vocabulary

words

Globalrepresentation

Globalrepresentation

Classifiermodel

Classifiermodel

Figure 2 Framework of the proposed method

In training stage lung nodule image samples are used asinput and the output is a trained classifier model Collectedtraining image samples are first divided into local patcheswith Superpixel Local patches are assignedwith no class labeland constitute local patch set With the local patch set localfeatures are extracted by unsupervised learning model deepautoencoder Next visual vocabulary is constructed basedon clustering all local feature vectors A lung nodule imagecan therefore be described by a global feature representationwith bag-of-visual-words model Finally classifier is trainedby supervised learning with nodule type labels In testingstage the input is a lung nodule image with unknown typeand the output is its predicted type label Similar to trainingstage a test image is divided into multiple patches Each localpatch is transformed into local feature and assigned witha visual word Finally global feature representation of testimage is used for classification by the trained model Detailsof the proposed method will be introduced in the followingsections

4 Local Feature Representation

Local feature representation is proposed in this section Theprocess consists of two steps (1) local patch generation and(2) local feature extraction and representation

41 Local Patch Generation Decomposing a lung image intosmall patches is useful and practical and for important tissuescan be picked up and unrelated ones can be get rid of Asshown in (1) a lung nodule image 119909 can be composed of agroup of image patches 119901119894 where 119899 denotes the number oflocal patches

119909 = 1199011 1199012 119901119899 (1)

The location and scale of local patches are determinedthrough generation [22 24] Useless part will be contained forlarge size patch while small part may not cover enough intacttissue Superpixel is a popular method that can partition the

image into small similar regions with better representative-ness and integrity [31] So it is adopted in this work

Figure 3 illustrates the process of the proposed local patchgeneration method For a lung nodule image (Figure 3(a))it is first segmented into local patches using Superpixel anda Superpixel map is obtained (Figure 3(b)) Local patchesessentially indicate the uniform regions Figure 3(c) is anindividual patch sample However the region that Figure 3(c)gives is an irregular shape and it is inconvenient for local fea-ture extraction and representation So we expand local patchwith its minimum enclosing rectangle as shown in Fig-ure 3(d) Finally a lung nodule image is decomposed into a setof local patches as shown in Figure 3(e) Besides there aresome additional criterions to determine whether an imagepatch is qualified for local feature extraction

(i) Let 119901119894 be a local patch it is removed when the area of119901119894 is larger than 119860max or smaller than 119860min

(ii) Let 119901119894 and 119901119895 be two local patches if the ratio betweentheir intersection and their union is larger than 119874119905then the smaller one is removed119860max 119860min and 119874119905 are predefined thresholds

42 Local Feature Extraction and Representation With therapid development of unsupervised learning in recent yearsusing unlabeled data to extract feature with autoencoder hasbecome an appropriate way Autoencodermodel is essentiallya multilayered neural networks Its original version is aforward network with one hidden layer Let 119909119894 be the inputdata 119886119894119895 be the activation of unit 119894 in layer 119895 and 119908119894 be thematrix of weights controlling function mapping from layer 119894to layer 119894 +1 If layer 119894 has 119904119894 units and layer 119894 +1 has 119904119894+1 unitsthen 119908119894 will be a matrix with size of 119904119894 lowast 119904119894+1 The activationcan be formulated as (2) where 11988621 is the 1st unit in the 2ndlayer and 1199090ndash1199093 are 4 input features

11988621 = 119892 (1199081101199090 + 1199081111199091 + 1199081121199092 + 1199081131199093) (2)

The main difference between ordinary forward neuralnetwork and autoencoder is that an autoencoderrsquos output is

Complexity 5

(a) (b) (c) (d) (e)

Figure 3 The process of local patch generation by Superpixel

Input Hidden Hidden Hidden Outputlayer layer layer layer layer

L1 L3 L4 L5L2

Encoding Decoding

Image patch Image patchReconstructed

Figure 4 The process of stacked deep autoencoder model

always the same as or similar to its input The basic formulacan be expressed as follows

119886 = ℎ (119909) = 119891 (119882119864119909 + 119887) 1199091015840 = ℎ1015840 (119909) = 119892 (119882119863119886 + 1198871015840) = 119892 (119882119863ℎ (119909) + 1198871015840)

(3)

An autoencoder can be seen as a combination of encoderand decoder The encoder includes an input layer and ahidden layer which converts an input image 119909 into featurevector 119886 The decoder includes a hidden layer and an outputlayer that transform feature 119886 to output feature 1199091015840 119882119864 and119882119863 are weight matrices of encoder and decoder respectivelyFunctions 119891(sdot) and 119892(sdot) can be either sigmoid or tanh activa-tion functions which is used to activate the unit in each layerWhen 1199091015840 approximates 119909 it is considered that the input fea-ture can be reconstructed from an abstract and compressedoutput feature vector 119886 The cost function can be generallydefined as follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2 + 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2 (4)

A deep autoencoder can be constructed by stacking morehidden layers As shown in Figure 4 there are 5 layers inthe model (including 3 hidden layers) 1198711 to 1198713 are encoding

layers and 1198713 to 1198715 are decoding layers 119871 119894 is used as the inputof the layer 119871 119894+1 and the weights can be gained based on (3)There are 2 stacked autoencodersThe activation of 1st hiddenlayer is the input of the 2nd stacked autoencoderThenetworkcan be trained in a fine-tuning stage by minimizing the equa-tion (4) 1198821 and 1198824 are trained through the encoding anddecoding weights of the 1st stacked autoencoder and1198822 and1198823 are trained through the encoding and decoding weightsof the 2nd stacked autoencoder Finally the whole networkcan be constructed layer by layer in a stacked way MoreoverFigure 4 just shows an example of symmetric encoding anddecoding structures and other variational structures can alsobe adopted

Therefore each local patch of a lung nodule image 119901119894 canbe represented by a fixed-length feature vector 119901119891119894 with deepautoencoder model Then (1) is transformed as follows

119909 = 1199011 1199012 119901119899 = 1199011198911 1199011198912 119901119891119899 (5)

5 Global Feature Representation

For BOVW model visual vocabulary is first constructedbased on clustering all local patch descriptors (local featurerepresentation) generated by a set of training images Theneach lung nodule image can be represented globally by ahistogram of visual words Distance between histograms of

6 Complexity

VisualVocabulary

K-meanscluster

middot middot middot middot middot middot

Figure 5 Procedure of BOVW representation of lung nodule image

visual words can be treated as similarity between lung noduleimage samples

Recall that a lung nodule image is decomposed into agroup of local patches and each patch is represented witha feature vector based on deep autoencoder Assume thereare 119863 local patches generated from all lung nodule train-ing images and each local patch is represented with 119889-dimensional feature vector then all local feature vectorscan be assembled into a feature space with size of 119889 lowast 119863Clustering is performed with 119889 lowast 119863 features and 119896-meansclustering method is adopted since it has relatively low timeand storage complexity irrelevant to data process orderingEach cluster center 119888119894 represents a visual word 119894 and 119896 clustercenters constitute the visual vocabulary A lung nodule imagesample 119909 can be represented by the encoded local patchesas a bag which is the occurrence frequency of visual wordin vocabulary To get the histogram representation ℎ(119909) ofan image 119909 all local patch feature vectors of 119909 are mappedonto the cluster center of the visual vocabulary and eachlocal feature is assigned with the label of its closest clustercenter using Euclidean distance in feature space Then a 119896-bins histogram ℎ(119909) is obtained by counting all the label oflocal patches generated by image 119909 as shown in (6) Figure 5exhibits the procedure of global representation of lung noduleimage

ℎ (119909) = [ℎ (119909)1 ℎ (119909)2 ℎ (119909)119896] (6)

6 Classifier Model

With global representation of lung nodule image softmaxalgorithm is used to train nodule type classifier Let (1199091 1199101)(1199092 1199102) (119909119898 119910119898) denote training data set 119909119894 denotes thelung nodule image sample and 119910119894 isin 0 1 2 3 denotes noduletype label

For an input image sample 119909119894 we want to compute 119901(119910 =119895 | 119909119894) (119895 isin 0 1 2 3) The output a 4-dimensional vectoris estimated to represent the probability of each type that119909119894 belongs to The hypothesis function can be expressed asfollows

ℎ120579 (119909119894) =[[[[[[

119901 (119910119894 = 0 | 119909119894 120579)119901 (119910119894 = 1 | 119909119894 120579)119901 (119910119894 = 2 | 119909119894 120579)119901 (119910119894 = 3 | 119909119894 120579)

]]]]]]= 1sum3119895=0 119890120579119879119895 119909119894

[[[[[[[[

1198901205791198790 1199091198941198901205791198791 1199091198941198901205791198792 1199091198941198901205791198793 119909119894

]]]]]]]]

(7)

where 120579 = 1205790 1205791 1205792 1205793 is model parameter set Thisequation normalizes the result and makes the sum to 1 Fortraining procedure the loss function is given as follows

119869 (120579) = minus 1119898 [

[119898

sum119894=1

3

sum119895=0

1 119910119894 = 119895 log 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

]]

+ 1205822119896

sum119894=1

119899

sum119895=0

1205792119894119895(8)

where 1sdot is an indicative function and stochastic gradientdescent (SGD) is used for function optimization and thecorresponding derivative functions are given as follows

nabla120579119895119869 (120579)

= minus 1119898119898

sum119894=1

[119909119894 (1 119910119894 = 119895 minus 119901 (119910119894 = 119895 | 119909119894 120579))] + 120582120579119895

119901 (119910119894 = 119895 | 119909119894 120579) = 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

(9)

Complexity 7

Figure 6 Demonstration of lung CT images (downloaded from [6])

7 Experimental Evaluations

71 Dataset and Program Implementation In order to eval-uate the performance of the proposed lung nodule imagerepresentation and classification method a widely usedpublic available lung nodule image dataset ELCAP is used fortesting [6] The dataset contains 379 lung CT images whichare collected from 50 distinct low-dose CT lung scans Thecenter position of lung nodule is marked in an extra lowastcsv file

Figure 6 demonstrates the lung nodule CT scan imageswhich are sampled from different slices Table 1 shows theformat of a lowastcvs file Each row denotes a lung nodule The4th column indicates the slice number where the lung noduleexists The 2nd and 3rd columns give the positions that thelung nodule is located in In this section lung nodule imagesare cropped from the raw CT images based on the 119909- and 119910-coordinates of nodule center given inTable 1The raw lungCTscan image is fixedwith 512lowast512pixels and the croppednod-ule images are too small to implement the algorithm There-fore we further resize the cropped lung nodule image into180lowast180 pixels with bicubicmethodThe lung nodule imagesare labeled with one of four types according to the guidanceby an expert Programs are implemented with Matlab 2016aprogramming language and tested on a Pentium i7 CPU 8GRAM NVIDIA GTX 960 GPU Windows OS PC

The experiments include the following aspects (1)parameter setting (2) classification rate with different param-eters (3) classification rate with data augmentation (4)classification rate with model sparsity (5) classification ratewith different classifier algorithms (6) comparing with othermethods (7) classification rate with model ensemble Theperformance of lung nodule image classification is computedwith overall classification rate as shown in the following

Classification rate = 119873correct119873all

(10)

where 119873correct is the number of correctly labeled images and119873all is the number of all testing images Cross validationmodeis adopted The dataset is divided into 8 groups 7 randomlychosen groups are used for training and the left group is used

Table 1 Format of lung nodule position

Type 119909 119910 SliceNodule 98 218 54Nodule 355 153 84Nodule 139 366 130Nodule 436 213 169Nodule 372 163 239Nodule 328 175 229Nodule 54 224 169

for testing This process is repeated 7 times and the result iscomputed by averaging 7 independent tests

72 Parameter Setting The parameters are needed to be setin local patch generation local feature representation andglobal feature representation For local patch generation weneed to set the number of superpixels that each lung noduleimage generates For local feature representation the numberof hidden layers and nodes that each layer contains shouldbe set For global feature representation the size of visualvocabulary should be set

As shown in Table 2 the number of patches that eachlung nodule image generates is set with 15 20 25 and 30The number of hidden layers in deep autoencoder is set with1 2 and 3 The number of nodes in deep autoencoder is setwith 50 75 100 125 and 150 The size of visual vocabulary isset with 200 300 400 and 500 The classification rate is eva-luated on the combination of these parameters For conve-nience parameters are expressed with 1199011 1199012 1199013 and 1199014 re-spectively

73 Classification Rate with Different Parameters The sizeof local patch is set with 30 lowast 30 pixels in our experimentTable 3 gives the average performance of lung nodule imageclassification based on combination of parameters 1199011 1199012 1199013and 1199014 It can be seen that classification model with 1199011 = 251199012 = 2 1199013 = (100 50) and 1199014 = 400 gets the optimal result

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

Complexity 5

(a) (b) (c) (d) (e)

Figure 3 The process of local patch generation by Superpixel

Input Hidden Hidden Hidden Outputlayer layer layer layer layer

L1 L3 L4 L5L2

Encoding Decoding

Image patch Image patchReconstructed

Figure 4 The process of stacked deep autoencoder model

always the same as or similar to its input The basic formulacan be expressed as follows

119886 = ℎ (119909) = 119891 (119882119864119909 + 119887) 1199091015840 = ℎ1015840 (119909) = 119892 (119882119863119886 + 1198871015840) = 119892 (119882119863ℎ (119909) + 1198871015840)

(3)

An autoencoder can be seen as a combination of encoderand decoder The encoder includes an input layer and ahidden layer which converts an input image 119909 into featurevector 119886 The decoder includes a hidden layer and an outputlayer that transform feature 119886 to output feature 1199091015840 119882119864 and119882119863 are weight matrices of encoder and decoder respectivelyFunctions 119891(sdot) and 119892(sdot) can be either sigmoid or tanh activa-tion functions which is used to activate the unit in each layerWhen 1199091015840 approximates 119909 it is considered that the input fea-ture can be reconstructed from an abstract and compressedoutput feature vector 119886 The cost function can be generallydefined as follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2 + 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2 (4)

A deep autoencoder can be constructed by stacking morehidden layers As shown in Figure 4 there are 5 layers inthe model (including 3 hidden layers) 1198711 to 1198713 are encoding

layers and 1198713 to 1198715 are decoding layers 119871 119894 is used as the inputof the layer 119871 119894+1 and the weights can be gained based on (3)There are 2 stacked autoencodersThe activation of 1st hiddenlayer is the input of the 2nd stacked autoencoderThenetworkcan be trained in a fine-tuning stage by minimizing the equa-tion (4) 1198821 and 1198824 are trained through the encoding anddecoding weights of the 1st stacked autoencoder and1198822 and1198823 are trained through the encoding and decoding weightsof the 2nd stacked autoencoder Finally the whole networkcan be constructed layer by layer in a stacked way MoreoverFigure 4 just shows an example of symmetric encoding anddecoding structures and other variational structures can alsobe adopted

Therefore each local patch of a lung nodule image 119901119894 canbe represented by a fixed-length feature vector 119901119891119894 with deepautoencoder model Then (1) is transformed as follows

119909 = 1199011 1199012 119901119899 = 1199011198911 1199011198912 119901119891119899 (5)

5 Global Feature Representation

For BOVW model visual vocabulary is first constructedbased on clustering all local patch descriptors (local featurerepresentation) generated by a set of training images Theneach lung nodule image can be represented globally by ahistogram of visual words Distance between histograms of

6 Complexity

VisualVocabulary

K-meanscluster

middot middot middot middot middot middot

Figure 5 Procedure of BOVW representation of lung nodule image

visual words can be treated as similarity between lung noduleimage samples

Recall that a lung nodule image is decomposed into agroup of local patches and each patch is represented witha feature vector based on deep autoencoder Assume thereare 119863 local patches generated from all lung nodule train-ing images and each local patch is represented with 119889-dimensional feature vector then all local feature vectorscan be assembled into a feature space with size of 119889 lowast 119863Clustering is performed with 119889 lowast 119863 features and 119896-meansclustering method is adopted since it has relatively low timeand storage complexity irrelevant to data process orderingEach cluster center 119888119894 represents a visual word 119894 and 119896 clustercenters constitute the visual vocabulary A lung nodule imagesample 119909 can be represented by the encoded local patchesas a bag which is the occurrence frequency of visual wordin vocabulary To get the histogram representation ℎ(119909) ofan image 119909 all local patch feature vectors of 119909 are mappedonto the cluster center of the visual vocabulary and eachlocal feature is assigned with the label of its closest clustercenter using Euclidean distance in feature space Then a 119896-bins histogram ℎ(119909) is obtained by counting all the label oflocal patches generated by image 119909 as shown in (6) Figure 5exhibits the procedure of global representation of lung noduleimage

ℎ (119909) = [ℎ (119909)1 ℎ (119909)2 ℎ (119909)119896] (6)

6 Classifier Model

With global representation of lung nodule image softmaxalgorithm is used to train nodule type classifier Let (1199091 1199101)(1199092 1199102) (119909119898 119910119898) denote training data set 119909119894 denotes thelung nodule image sample and 119910119894 isin 0 1 2 3 denotes noduletype label

For an input image sample 119909119894 we want to compute 119901(119910 =119895 | 119909119894) (119895 isin 0 1 2 3) The output a 4-dimensional vectoris estimated to represent the probability of each type that119909119894 belongs to The hypothesis function can be expressed asfollows

ℎ120579 (119909119894) =[[[[[[

119901 (119910119894 = 0 | 119909119894 120579)119901 (119910119894 = 1 | 119909119894 120579)119901 (119910119894 = 2 | 119909119894 120579)119901 (119910119894 = 3 | 119909119894 120579)

]]]]]]= 1sum3119895=0 119890120579119879119895 119909119894

[[[[[[[[

1198901205791198790 1199091198941198901205791198791 1199091198941198901205791198792 1199091198941198901205791198793 119909119894

]]]]]]]]

(7)

where 120579 = 1205790 1205791 1205792 1205793 is model parameter set Thisequation normalizes the result and makes the sum to 1 Fortraining procedure the loss function is given as follows

119869 (120579) = minus 1119898 [

[119898

sum119894=1

3

sum119895=0

1 119910119894 = 119895 log 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

]]

+ 1205822119896

sum119894=1

119899

sum119895=0

1205792119894119895(8)

where 1sdot is an indicative function and stochastic gradientdescent (SGD) is used for function optimization and thecorresponding derivative functions are given as follows

nabla120579119895119869 (120579)

= minus 1119898119898

sum119894=1

[119909119894 (1 119910119894 = 119895 minus 119901 (119910119894 = 119895 | 119909119894 120579))] + 120582120579119895

119901 (119910119894 = 119895 | 119909119894 120579) = 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

(9)

Complexity 7

Figure 6 Demonstration of lung CT images (downloaded from [6])

7 Experimental Evaluations

71 Dataset and Program Implementation In order to eval-uate the performance of the proposed lung nodule imagerepresentation and classification method a widely usedpublic available lung nodule image dataset ELCAP is used fortesting [6] The dataset contains 379 lung CT images whichare collected from 50 distinct low-dose CT lung scans Thecenter position of lung nodule is marked in an extra lowastcsv file

Figure 6 demonstrates the lung nodule CT scan imageswhich are sampled from different slices Table 1 shows theformat of a lowastcvs file Each row denotes a lung nodule The4th column indicates the slice number where the lung noduleexists The 2nd and 3rd columns give the positions that thelung nodule is located in In this section lung nodule imagesare cropped from the raw CT images based on the 119909- and 119910-coordinates of nodule center given inTable 1The raw lungCTscan image is fixedwith 512lowast512pixels and the croppednod-ule images are too small to implement the algorithm There-fore we further resize the cropped lung nodule image into180lowast180 pixels with bicubicmethodThe lung nodule imagesare labeled with one of four types according to the guidanceby an expert Programs are implemented with Matlab 2016aprogramming language and tested on a Pentium i7 CPU 8GRAM NVIDIA GTX 960 GPU Windows OS PC

The experiments include the following aspects (1)parameter setting (2) classification rate with different param-eters (3) classification rate with data augmentation (4)classification rate with model sparsity (5) classification ratewith different classifier algorithms (6) comparing with othermethods (7) classification rate with model ensemble Theperformance of lung nodule image classification is computedwith overall classification rate as shown in the following

Classification rate = 119873correct119873all

(10)

where 119873correct is the number of correctly labeled images and119873all is the number of all testing images Cross validationmodeis adopted The dataset is divided into 8 groups 7 randomlychosen groups are used for training and the left group is used

Table 1 Format of lung nodule position

Type 119909 119910 SliceNodule 98 218 54Nodule 355 153 84Nodule 139 366 130Nodule 436 213 169Nodule 372 163 239Nodule 328 175 229Nodule 54 224 169

for testing This process is repeated 7 times and the result iscomputed by averaging 7 independent tests

72 Parameter Setting The parameters are needed to be setin local patch generation local feature representation andglobal feature representation For local patch generation weneed to set the number of superpixels that each lung noduleimage generates For local feature representation the numberof hidden layers and nodes that each layer contains shouldbe set For global feature representation the size of visualvocabulary should be set

As shown in Table 2 the number of patches that eachlung nodule image generates is set with 15 20 25 and 30The number of hidden layers in deep autoencoder is set with1 2 and 3 The number of nodes in deep autoencoder is setwith 50 75 100 125 and 150 The size of visual vocabulary isset with 200 300 400 and 500 The classification rate is eva-luated on the combination of these parameters For conve-nience parameters are expressed with 1199011 1199012 1199013 and 1199014 re-spectively

73 Classification Rate with Different Parameters The sizeof local patch is set with 30 lowast 30 pixels in our experimentTable 3 gives the average performance of lung nodule imageclassification based on combination of parameters 1199011 1199012 1199013and 1199014 It can be seen that classification model with 1199011 = 251199012 = 2 1199013 = (100 50) and 1199014 = 400 gets the optimal result

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

6 Complexity

VisualVocabulary

K-meanscluster

middot middot middot middot middot middot

Figure 5 Procedure of BOVW representation of lung nodule image

visual words can be treated as similarity between lung noduleimage samples

Recall that a lung nodule image is decomposed into agroup of local patches and each patch is represented witha feature vector based on deep autoencoder Assume thereare 119863 local patches generated from all lung nodule train-ing images and each local patch is represented with 119889-dimensional feature vector then all local feature vectorscan be assembled into a feature space with size of 119889 lowast 119863Clustering is performed with 119889 lowast 119863 features and 119896-meansclustering method is adopted since it has relatively low timeand storage complexity irrelevant to data process orderingEach cluster center 119888119894 represents a visual word 119894 and 119896 clustercenters constitute the visual vocabulary A lung nodule imagesample 119909 can be represented by the encoded local patchesas a bag which is the occurrence frequency of visual wordin vocabulary To get the histogram representation ℎ(119909) ofan image 119909 all local patch feature vectors of 119909 are mappedonto the cluster center of the visual vocabulary and eachlocal feature is assigned with the label of its closest clustercenter using Euclidean distance in feature space Then a 119896-bins histogram ℎ(119909) is obtained by counting all the label oflocal patches generated by image 119909 as shown in (6) Figure 5exhibits the procedure of global representation of lung noduleimage

ℎ (119909) = [ℎ (119909)1 ℎ (119909)2 ℎ (119909)119896] (6)

6 Classifier Model

With global representation of lung nodule image softmaxalgorithm is used to train nodule type classifier Let (1199091 1199101)(1199092 1199102) (119909119898 119910119898) denote training data set 119909119894 denotes thelung nodule image sample and 119910119894 isin 0 1 2 3 denotes noduletype label

For an input image sample 119909119894 we want to compute 119901(119910 =119895 | 119909119894) (119895 isin 0 1 2 3) The output a 4-dimensional vectoris estimated to represent the probability of each type that119909119894 belongs to The hypothesis function can be expressed asfollows

ℎ120579 (119909119894) =[[[[[[

119901 (119910119894 = 0 | 119909119894 120579)119901 (119910119894 = 1 | 119909119894 120579)119901 (119910119894 = 2 | 119909119894 120579)119901 (119910119894 = 3 | 119909119894 120579)

]]]]]]= 1sum3119895=0 119890120579119879119895 119909119894

[[[[[[[[

1198901205791198790 1199091198941198901205791198791 1199091198941198901205791198792 1199091198941198901205791198793 119909119894

]]]]]]]]

(7)

where 120579 = 1205790 1205791 1205792 1205793 is model parameter set Thisequation normalizes the result and makes the sum to 1 Fortraining procedure the loss function is given as follows

119869 (120579) = minus 1119898 [

[119898

sum119894=1

3

sum119895=0

1 119910119894 = 119895 log 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

]]

+ 1205822119896

sum119894=1

119899

sum119895=0

1205792119894119895(8)

where 1sdot is an indicative function and stochastic gradientdescent (SGD) is used for function optimization and thecorresponding derivative functions are given as follows

nabla120579119895119869 (120579)

= minus 1119898119898

sum119894=1

[119909119894 (1 119910119894 = 119895 minus 119901 (119910119894 = 119895 | 119909119894 120579))] + 120582120579119895

119901 (119910119894 = 119895 | 119909119894 120579) = 119890120579119879119895 119909119894sum3119897=0 119890120579119879119897 119909119894

(9)

Complexity 7

Figure 6 Demonstration of lung CT images (downloaded from [6])

7 Experimental Evaluations

71 Dataset and Program Implementation In order to eval-uate the performance of the proposed lung nodule imagerepresentation and classification method a widely usedpublic available lung nodule image dataset ELCAP is used fortesting [6] The dataset contains 379 lung CT images whichare collected from 50 distinct low-dose CT lung scans Thecenter position of lung nodule is marked in an extra lowastcsv file

Figure 6 demonstrates the lung nodule CT scan imageswhich are sampled from different slices Table 1 shows theformat of a lowastcvs file Each row denotes a lung nodule The4th column indicates the slice number where the lung noduleexists The 2nd and 3rd columns give the positions that thelung nodule is located in In this section lung nodule imagesare cropped from the raw CT images based on the 119909- and 119910-coordinates of nodule center given inTable 1The raw lungCTscan image is fixedwith 512lowast512pixels and the croppednod-ule images are too small to implement the algorithm There-fore we further resize the cropped lung nodule image into180lowast180 pixels with bicubicmethodThe lung nodule imagesare labeled with one of four types according to the guidanceby an expert Programs are implemented with Matlab 2016aprogramming language and tested on a Pentium i7 CPU 8GRAM NVIDIA GTX 960 GPU Windows OS PC

The experiments include the following aspects (1)parameter setting (2) classification rate with different param-eters (3) classification rate with data augmentation (4)classification rate with model sparsity (5) classification ratewith different classifier algorithms (6) comparing with othermethods (7) classification rate with model ensemble Theperformance of lung nodule image classification is computedwith overall classification rate as shown in the following

Classification rate = 119873correct119873all

(10)

where 119873correct is the number of correctly labeled images and119873all is the number of all testing images Cross validationmodeis adopted The dataset is divided into 8 groups 7 randomlychosen groups are used for training and the left group is used

Table 1 Format of lung nodule position

Type 119909 119910 SliceNodule 98 218 54Nodule 355 153 84Nodule 139 366 130Nodule 436 213 169Nodule 372 163 239Nodule 328 175 229Nodule 54 224 169

for testing This process is repeated 7 times and the result iscomputed by averaging 7 independent tests

72 Parameter Setting The parameters are needed to be setin local patch generation local feature representation andglobal feature representation For local patch generation weneed to set the number of superpixels that each lung noduleimage generates For local feature representation the numberof hidden layers and nodes that each layer contains shouldbe set For global feature representation the size of visualvocabulary should be set

As shown in Table 2 the number of patches that eachlung nodule image generates is set with 15 20 25 and 30The number of hidden layers in deep autoencoder is set with1 2 and 3 The number of nodes in deep autoencoder is setwith 50 75 100 125 and 150 The size of visual vocabulary isset with 200 300 400 and 500 The classification rate is eva-luated on the combination of these parameters For conve-nience parameters are expressed with 1199011 1199012 1199013 and 1199014 re-spectively

73 Classification Rate with Different Parameters The sizeof local patch is set with 30 lowast 30 pixels in our experimentTable 3 gives the average performance of lung nodule imageclassification based on combination of parameters 1199011 1199012 1199013and 1199014 It can be seen that classification model with 1199011 = 251199012 = 2 1199013 = (100 50) and 1199014 = 400 gets the optimal result

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

Complexity 7

Figure 6 Demonstration of lung CT images (downloaded from [6])

7 Experimental Evaluations

71 Dataset and Program Implementation In order to eval-uate the performance of the proposed lung nodule imagerepresentation and classification method a widely usedpublic available lung nodule image dataset ELCAP is used fortesting [6] The dataset contains 379 lung CT images whichare collected from 50 distinct low-dose CT lung scans Thecenter position of lung nodule is marked in an extra lowastcsv file

Figure 6 demonstrates the lung nodule CT scan imageswhich are sampled from different slices Table 1 shows theformat of a lowastcvs file Each row denotes a lung nodule The4th column indicates the slice number where the lung noduleexists The 2nd and 3rd columns give the positions that thelung nodule is located in In this section lung nodule imagesare cropped from the raw CT images based on the 119909- and 119910-coordinates of nodule center given inTable 1The raw lungCTscan image is fixedwith 512lowast512pixels and the croppednod-ule images are too small to implement the algorithm There-fore we further resize the cropped lung nodule image into180lowast180 pixels with bicubicmethodThe lung nodule imagesare labeled with one of four types according to the guidanceby an expert Programs are implemented with Matlab 2016aprogramming language and tested on a Pentium i7 CPU 8GRAM NVIDIA GTX 960 GPU Windows OS PC

The experiments include the following aspects (1)parameter setting (2) classification rate with different param-eters (3) classification rate with data augmentation (4)classification rate with model sparsity (5) classification ratewith different classifier algorithms (6) comparing with othermethods (7) classification rate with model ensemble Theperformance of lung nodule image classification is computedwith overall classification rate as shown in the following

Classification rate = 119873correct119873all

(10)

where 119873correct is the number of correctly labeled images and119873all is the number of all testing images Cross validationmodeis adopted The dataset is divided into 8 groups 7 randomlychosen groups are used for training and the left group is used

Table 1 Format of lung nodule position

Type 119909 119910 SliceNodule 98 218 54Nodule 355 153 84Nodule 139 366 130Nodule 436 213 169Nodule 372 163 239Nodule 328 175 229Nodule 54 224 169

for testing This process is repeated 7 times and the result iscomputed by averaging 7 independent tests

72 Parameter Setting The parameters are needed to be setin local patch generation local feature representation andglobal feature representation For local patch generation weneed to set the number of superpixels that each lung noduleimage generates For local feature representation the numberof hidden layers and nodes that each layer contains shouldbe set For global feature representation the size of visualvocabulary should be set

As shown in Table 2 the number of patches that eachlung nodule image generates is set with 15 20 25 and 30The number of hidden layers in deep autoencoder is set with1 2 and 3 The number of nodes in deep autoencoder is setwith 50 75 100 125 and 150 The size of visual vocabulary isset with 200 300 400 and 500 The classification rate is eva-luated on the combination of these parameters For conve-nience parameters are expressed with 1199011 1199012 1199013 and 1199014 re-spectively

73 Classification Rate with Different Parameters The sizeof local patch is set with 30 lowast 30 pixels in our experimentTable 3 gives the average performance of lung nodule imageclassification based on combination of parameters 1199011 1199012 1199013and 1199014 It can be seen that classification model with 1199011 = 251199012 = 2 1199013 = (100 50) and 1199014 = 400 gets the optimal result

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

8 Complexity

Table 2 Parameter setting

Parameter Parameter explanation Setting1199011 Number of Superpixel generation 15 20 25 301199012 Hidden layers of deep autoencoder 1 2 31199013 Nodes of deep autoencoder 50 75 100 125 1501199014 Size of visual vocabulary 200 300 400 500

Table 3 Performance with different parameters

Parameters setting Performance1199011 1199012 (1199013) 119901415 1 (50) 200 0819915 1 (50) 400 083215 1 (50) 600 082915 2 (100 50) 200 085815 2 (100 50) 400 08615 2 (100 50) 600 084515 3 (150 100 50) 200 083615 3 (150 100 50) 400 084515 3 (150 100 50) 600 08525 1 (50) 200 08325 1 (50) 400 084225 1 (50) 600 084925 2 (100 50) 200 088725 2 (100 50) 400 089525 2 (100 50) 600 089125 3 (150 100 50) 200 083525 3 (150 100 50) 400 083225 3 (150 100 50) 600 08640 1 (50) 200 082440 1 (50) 400 08240 1 (50) 600 081340 2 (100 50) 200 082440 2 (100 50) 400 081540 2 (100 50) 600 083340 3 (150 100 50) 200 08140 3 (150 100 50) 400 08040 3 (150 100 50) 600 0806

reaching 895 We can also see that different parametersettings have great impact on the classification results

74 Classification Rate with Data Augmentation Overfittingis common in machine learning and it is influenced by bothmodel complexity and the size of training data Data aug-mentation scheme is usually adopted to lessen this problem[32] In this section data augmentation is used to enlargethe size of training data Random rotation random croppingand random perturbation (brightness saturation hue andcontrast) are used as basic augment techniques

For original lung nodule image it is sampled withpossibility of 05 for data augmentation The new createdexamples are set with same labels as original As shown inTable 4 data augmentation can increase classification rate

Table 4 Performance with data augmentation

Data augmentation method AverageperformanceRandom

rotationRandomcropping

Randomperturbation

No No No 0895Yes No No 0899No Yes No 0887Yes Yes No 0908No No Yes 0903Yes No Yes 0911No Yes Yes 0912Yes Yes Yes 0924

with 3 This shows that adding more augmented data fortraining can improve the compatibility and generalization ofthe classification model

75 Classification Rate with Model Sparsity In this subsec-tion a sparsity constraint is imposed on the hidden layerSparsity is a recently proposed technique to improve thegeneralization of the model [33] A sparsity regularizationterm is added to (4) and the newobjective functions are givenas follows

119869 (119882 119887) = 1119873119873

sum119894=1

12100381710038171003817100381710038171199091015840119894 minus 11990911989410038171003817100381710038171003817

2

+ 120582119873119897minus1sum119897=1

119872119894sum119894=1

119872119894+1sum119895=1

(119882119897119894119895)2

+ 120573119873119897minus1sum119894=1

KL (120588 120588119894)

KL (120588 120588119894) = 120588 log 120588120588119894 + (1 minus 120588) log (1 minus 120588)

(1 minus 120588119894)

120588119894 = 1119873119873

sum119894=1

ℎ119894 (119909)

(11)

The sparsity regularization term is regulated by Kullback-Leibler divergence KL(120588 120588119894) 120588119894 is the average activation of119894th layer of deep autoencoder and 120588 is the target activation120588 with small value can reduce the mean activation of themodel 120573 is a trade-off parameter Table 5 gives the resultof classification performance with different 120588 (values from01ndash09) It can be seen that 120588 set around 03ndash04 leads to thesuperior performance

76 Classification Rate with Different Classifier Algorithms Inthis subsection we evaluate the performances of 4 commonlyused classifier algorithms Softmax (which is used in thispaper) SVM kNN and decision tree are used The samefeature representation is adopted Table 6 shows that softmaxslightly outperforms SVM kNN and decision tree The

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

Complexity 9

Table 5 The effect of model sparsity

Sparsity (120588) Performance01 09202 092903 093804 093905 09306 0905

Table 6 The effect of different classifier algorithms

Classifier model PerformanceSoftmax 0939SVM 0931kNN 0927Decision tree 0919

results demonstrate that compared with classifier algorithmthe feature representation is the key problem Meanwhile itis easy to combine the softmax algorithm and the proposedfeature representation method into an end-to-end structurewhich can make model training more convenient

77 Comparing with Other Methods In order to evaluate theclassification rate of different methods 5 related algorithmsare used for testing Reference [19] studies the same problemas ours Reference [20] adopts the primitive autoencodermethod References [7 21] use non-deep-learning methodsfor classification Reference [9] employs the BOVW modelThe compared methods are reimplemented and are testedwith diverse parameters Table 7 gives the testing resultAmong all testing methods the proposed one demonstratesthe best performance Comparing with non-deep-learningmethod our method can construct better feature representa-tion while comparing with primitive autoencoder methodthe Superpixel and DAE used in our method can catch moredetailed information

78 Classification Rate with Model Ensemble Model ensem-ble can improve the classification performance by aggregat-ing multiple individual classifiers [34] We evaluate modelensemble based on Majority Rule in this subsection InMajority Rule the class label is assigned with the one thatmost classifier votesThe function to evaluate the class label 119890for image 119868 is given as follows

119901 (119890) =119873

sum119894=1

119878 (119862119894 (119868) = 119890)

119890 = argmax119894

119901 (119890119894) (12)

where 119868 is a testing image 119890 is a class label119873 denotes numberof selected models and 119862119894 means 119894th classifier 119878(119862119894(119868) = 119890) =1 if 119862119894 classifies 119868 as 119890 The label with maximal value of 119901(sdot) isdetermined as the final result If multiple labels have the same

Table 7 Performance comparing with other methods

Classification method PerformanceRef [19] 0877Ref [7] 088Ref [20] 082Ref [21] 0895Ref [9] 0891Our proposed method 0939

Table 8 Performance of model ensemble

Number of models Performance1 09395 09526 09547 0955

votes the arithmetic average of class probabilities predictedby individual model is used as classification result

With different parameters combination models with topperformances are retained for ensemble Table 8 gives thetesting result The 1st row denotes the single model The2nd to 4th rows denote model ensemble with 5 6 and 7individual models respectively The result demonstrates thatmodel ensemble can complement individual ones and theperformance is improved with about 15

8 Conclusion and Future Works

In this paper a novel feature representation method isproposed for lung nodule image classification Superpixel isfirst used to divide lung nodule image into local patchesThen local feature is extracted and represented from localpatcheswith deep autoencoder Bag-of-visual-wordsmodel isused as global feature representation with visual vocabularyconstructed by local feature representation Finally an end-to-end training is implemented with a softmax classifier Theproposed method is evaluated from many aspects includingparameter setting data augmentation model sparsity com-parison among different algorithms and model ensembleWe draw a conclusion that the proposed method achievessuperior performance The merits of our method are thecombination of local and global feature representation andbetter model generalization can be gained by incorporatingunsupervised deep learning model

Our future works will focus on two aspects (1) studynew classification framework and method according to up-to-date convolutional neural network and (2) analysis of ourmethod in large data set formaking further improvement andoptimization

Conflicts of Interest

The authors declare that they have no conflicts of interest

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

10 Complexity

Acknowledgments

This work was supported by Liaoning Doctoral ResearchFoundation of China (no 20170520238) National Natu-ral Science Foundation of China (no 61772125 and no61402097) The authors gratefully acknowledge the supportof NVIDIA Corporation with the donation of GPU used forthis research

References

[1] National Cancer Institute ldquoSEER Stat Fact Sheets Lung andBronchus Cancer NCI online 2016rdquo httpseercancergovstat-factshtmllungbhtml

[2] American Cancer Society httpswwwcancerorg[3] Y Shieh and M Bohnenkamp ldquoLow-Dose CT Scan for

Lung Cancer Screening Clinical and Coding ConsiderationsrdquoCHEST vol 152 no 1 pp 204ndash209 2017

[4] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoContent-Based Image Retrieval System for Pul-monary Nodules Assisting Radiologists in Self-Learning andDiagnosis of Lung Cancerrdquo Journal of Digital Imaging vol 30no 1 pp 63ndash77 2017

[5] S Diciotti G Picozzi M Falchini MMascalchi N Villari andG Valli ldquo3-D segmentation algorithm of small lung nodules inspiral CT imagesrdquo IEEETransactions on Information Technologyin Biomedicine vol 12 no 1 pp 7ndash19 2008

[6] ELCAPpublic lung image database httpwwwviacornelledudatabaseslungdbhtml

[7] J N Stember ldquoThe Normal Mode Analysis Shape DetectionMethod for Automated ShapeDetermination of LungNodulesrdquoJournal of Digital Imaging vol 28 no 2 pp 224ndash230 2015

[8] Y Song W Cai H Huang Y Zhou Y Wang and D D FengldquoLocality-constrained subcluster representation ensemble forlung image classificationrdquo Medical Image Analysis vol 22 no1 pp 102ndash113 2015

[9] F Ciompi C Jacobs E T Scholten et al ldquoBag-of-frequenciesa descriptor of pulmonary nodules in computed tomographyimagesrdquo IEEE Transactions on Medical Imaging vol 34 no 4pp 962ndash973 2015

[10] F Han G Zhang H Wang et al ldquoA texture feature analysis fordiagnosis of pulmonary nodules using LIDC-IDRI databaserdquoin Proceedings of the 2013 IEEE International Conference onMedical Imaging Physics and Engineering ICMIPE 2013 pp 14ndash18 IEEE Shenyang China October 2013

[11] J Bai X Huang S Liu Q Song and R Bhagalia ldquoLearningorientation invariant contextual features for nodule detectionin lung CT scansrdquo in Proceedings of the 12th IEEE InternationalSymposium on Biomedical Imaging ISBI 2015 pp 1135ndash1138IEEE New York NY USA April 2015

[12] C Jacobs E M van Rikxoort T Twellmann et al ldquoAutomaticdetection of subsolid pulmonary nodules in thoracic computedtomography imagesrdquoMedical Image Analysis vol 18 no 2 pp374ndash384 2013

[13] T W Way B Sahiner H-P Chan et al ldquoComputer-aided diag-nosis of pulmonary nodules on CT scans improvement of clas-sification performance with nodule surface featuresrdquo MedicalPhysics vol 36 no 7 pp 3086ndash3098 2009

[14] S Akram M Y Javed and A Hussain ldquoAutomated thresh-olding of lung CT scan for Artificial Neural Network basedclassification of nodulesrdquo in Proceedings of the 14th IEEEACIS

International Conference on Computer and Information ScienceICIS 2015 pp 335ndash340 IEEE Las Vegas NV USA July 2015

[15] A Robert G Jonathan A Fereidoun et al ldquoAutomated classi-fication of lung bronchovascular anatomy in CT using Ada-BoostrdquoMedical Image Analysis vol 11 no 3 pp 315ndash324 2007

[16] A El-Baz G Gimelrsquofarb R Falk and M El-Ghar ldquoAppearanceanalysis for diagnosing malignant lung nodulesrdquo in Proceedingsof the 7th IEEE International Symposium on Biomedical ImagingFrom Nano to Macro ISBI 2010 pp 193ndash196 IEEE RotterdamNetherlands April 2010

[17] H Takizawa S Yamamoto and T Shiina ldquoAccuracy improve-ment of pulmonary nodule detection based on spatial statisticalanalysis of thoracic CT scansrdquo IEICE Transaction on Informa-tion and Systems vol 90 no 8 pp 1168ndash1174 2007

[18] K Yoshiki N Noboru O Hironobu et al ldquoHybrid Classifica-tion Approach of Malignant and Benign Pulmonary NodulesBased on Topological and Histogram Featuresrdquo Medical ImageComputing amp Computer Assisted Intervention pp 297ndash3062000

[19] F Zhang Y Song W Cai et al ldquoA ranking-based lung noduleimage classificationmethod using unlabeled image knowledgerdquoin Proceedings of the IEEE 11th International Symposium onBiomedical Imaging (ISBI rsquo14) pp 1356ndash1359 Beijing ChinaMay 2014

[20] D Kumar A Wong and D A Clausi ldquoLung nodule classifi-cation using deep features in CT imagesrdquo in Proceedings of the12th Conference on Computer and Robot Vision (CRV rsquo15) pp133ndash138 IEEE Halifax Canada June 2015

[21] A K Dhara S Mukhopadhyay A Dutta M Garg and NKhandelwal ldquoA Combination of Shape and Texture Featuresfor Classification of Pulmonary Nodules in Lung CT ImagesrdquoJournal of Digital Imaging vol 29 no 4 pp 466ndash475 2016

[22] F Zhang Y SongW Cai et al ldquoLung nodule classification withmultilevel patch-based context analysisrdquo IEEE Transactions onBiomedical Engineering vol 61 no 4 pp 1155ndash1166 2014

[23] Y Song W Cai Y Wang and D D Feng ldquoLocation classifi-cation of lung nodules with optimized graph constructionrdquo inProceedings of the 9th IEEE International SymposiumonBiomed-ical Imaging (ISBI rsquo12) pp 1439ndash1442 IEEE Barcelona SpainMay 2012

[24] A Farag S Elhabian J Graham A Farag and R Falk ldquoTowardprecise pulmonary nodule descriptors for nodule type clas-sificationrdquo Medical Image Computing and Computer-AssistedIntervention vol 13 no 3 pp 626ndash633 2010

[25] F Zhang W Cai Y Song M-Z Lee S Shan and D DaganldquoOverlapping node discovery for improving classification oflung nodulesrdquo in Proceedings of the 35th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (EMBC rsquo13) pp 5461ndash5464 Osaka Japan July 2013

[26] K Suzuki F Li S Sone and K Doi ldquoComputer-aided diagnos-tic scheme for distinction between benign and malignant nod-ules in thoracic low-dose CT by use ofmassive training artificialneural networkrdquo IEEE Transactions onMedical Imaging vol 24no 9 pp 1138ndash1150 2005

[27] Y Sui Y Wei and D Zhao ldquoComputer-aided lung nodule re-cognition by SVM classifier based on combination of randomundersampling and SMOTErdquoComputational andMathematicalMethods in Medicine Article ID 368674 pp 1ndash13 2015

[28] A A A Setio F Ciompi G Litjens et al ldquoPulmonary NoduleDetection in CT Images False Positive Reduction Using Multi-View Convolutional Networksrdquo IEEE Transactions on MedicalImaging vol 35 no 5 pp 1160ndash1169 2016

Complexity 11

[29] B Van Ginneken A A A Setio C Jacobs and F Ciompi ldquoOff-the-shelf convolutional neural network features for pulmonarynodule detection in computed tomography scansrdquo in Proceed-ings of the 12th IEEE International Symposium on BiomedicalImaging (ISBI rsquo15) pp 286ndash289 Brooklyn NY USA April 2015

[30] W Shen M Zhou F Yang C Yang and J Tian ldquoMulti-scaleconvolutional neural networks for lung nodule classificationrdquoLecture Notes in Computer Science (including subseries LectureNotes in Artificial Intelligence and Lecture Notes in Bioinformat-ics) Preface vol 9123 pp 588ndash599 2015

[31] R Achanta A Shaji K Smith A Lucchi P Fua and SSusstrunk ldquoSLIC superpixels compared to state-of-the-artsuperpixel methodsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 34 no 11 pp 2274ndash2281 2012

[32] G E Hinton N Srivastava A Krizhevsky I Sutskever and RR Salakhutdinov ldquoImproving neural networks by preventingco-adaptation of feature detectorsrdquo httpsarxivorgabs12070580

[33] X Zhang and RWu ldquoFast depth image denoising and enhance-ment using a deep convolutional networkrdquo in Proceedings ofthe 41st IEEE International Conference on Acoustics Speech andSignal Processing ICASSP 2016 pp 2499ndash2503 ShanghaiChina March 2016

[34] L Rokach ldquoEnsemble-based classifiersrdquo Artificial IntelligenceReview vol 33 no 1-2 pp 1ndash39 2010

Hindawiwwwhindawicom Volume 2018

MathematicsJournal of

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Applied MathematicsJournal of

Hindawiwwwhindawicom Volume 2018

Probability and StatisticsHindawiwwwhindawicom Volume 2018

Journal of

Hindawiwwwhindawicom Volume 2018

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawiwwwhindawicom Volume 2018

OptimizationJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

Hindawiwwwhindawicom Volume 2018

Operations ResearchAdvances in

Journal of

Hindawiwwwhindawicom Volume 2018

Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018

International Journal of Mathematics and Mathematical Sciences

Hindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018Volume 2018

Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in

Nature and SocietyHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Dierential EquationsInternational Journal of

Volume 2018

Hindawiwwwhindawicom Volume 2018

Decision SciencesAdvances in

Hindawiwwwhindawicom Volume 2018

AnalysisInternational Journal of

Hindawiwwwhindawicom Volume 2018

Stochastic AnalysisInternational Journal of

Submit your manuscripts atwwwhindawicom