Classification of mice hepatic granuloma microscopic

images based on a deep convolutional neural network

Yu Wang

College of Information and Engineering, Wenzhou Medical University, Wenzhou 325035,

People Republic of China

Yating Chen

College of Information and Engineering, Wenzhou Medical University, Wenzhou 325035,

People Republic of China

Ningning Yang

College of Information and Engineering, Wenzhou Medical University, Wenzhou 325035,

People Republic of China

Longfei Zheng

College of Information and Engineering, Wenzhou Medical University, Wenzhou 325035,

People Republic of China

Nilanjan Dey

Department of Information Technology, Techno India College of Technology, West Bengal,

740000, India

Amira S. Ashour

Department of Electronics and Electrical Communications Engineering, Faculty of

Engineering, Tanta University, Tanta 31111, Egypt

V. Rajinikanth

Department of Electronics and Instrumentation Engineering., St. Josephs College of

Engineering, Chennai 600119, Tamilnadu, India

Joao Manuel R.S. Tavares

Instituto de Ciencia e Inovacao em Engenharia Mecanica e Engenharia Industrial,

Departamento de Engenharia Mecanica, Faculdade de Engenharia, Universidade do Porto,

Porto, Portugal

Fuqian Shi∗

College of Information and Engineering, Wenzhou Medical University, Wenzhou 325035,

People Republic of China

∗Corresponding authorEmail address: sfq@wmu.edu.cn (Fuqian Shi)

Preprint submitted to Applied Soft Computing September 18, 2018

Abstract

Hepatic granuloma develops in the early stage of liver cirrhosis which can

seriously injury liver health. At present, the assessment of medical microscopic

images is necessary for various diseases and the exploiting of artificial intelli-

gence technology to assist pathology doctors in pre-diagnosis is the trend of

future medical development. In this article, we try to classify mice liver mi-

croscopic images of normal, granuloma-fibrosis1 and granuloma-fibrosis2, using

convolutional neural networks (CNNs) and two conventional machine learning

methods: support vector machine (SVM) and random forest (RF). On account

of the included small dataset of 30 mice liver microscopic images, the proposed

work included a preprocessing stage to deal with the problem of insufficient

image number, which included the cropping of the original microscopic images

to small patches, and the disorderly recombination after cropping and labelling

the cropped patches. In addition, recognizable texture features are extracted

and selected using gray the level co-occurrence matrix (GLCM), local binary

pattern (LBP) and Pearson correlation coefficient (PCC), respectively. The

results established a classification accuracy of 82.78% of the proposed CNN

based classifiers to classify 3 types of images. In addition, the confusion ma-

trix figures out that the accuracy of the classification results using the proposed

CNNs based classifiers for the normal class, granuloma-fibrosis1, and granuloma-

fibrosis2 were 92.5%, 76.67%, and 79.17%, respectively. The comparative study

of the proposed CNN based classifier and the SVM and RF proved the superi-

ority of the CNNs showing its promising performance for clinical cases.

Keywords: Hepatic Granuloma, Microscopic imaging, Image Classification,

Deep Learning

This work is supported by Zhejiang Provincial Natural Science Foundation

(Grant no: LY17F030014).

Declarations of interest: none

2

1. Introduction

Hepatic cirrhosis is a degenerative and chronic liver disease, which has as the

main characteristic the fact that normal liver cells been replaced by collagenous

scar (fibrosis). In the early stage of cirrhosis, especially in primary biliary

cirrhosis, the pathologist may uncover one or more granulomas in a biopsy

specimen, diagnosing the stage of underlying liver injury [1]. Granuloma is

commonly defined as a distinct nodular lesion formed by macrophages and their

evolution of localized infiltration and proliferation, which are presented as red-

brown and annular grouped cells arranging disorderly and densely. All these

characteristics are commonly delineated in microscopic images.

Microscopic imaging is as an important medical imaging modality, which

carries rich but complex information, including about the various structures of

biological tissue, the state of different cells and others that pathology doctors

demand. However, the complexity of the acquired images and doctors subjec-

tivity often cause disagreement even among experienced pathologists [2], which

suggests an increasing demand for robust and stable computational approaches

to improve the diagnosis efficiency [3, 4, 5].

In recent years, deep learning, especially convolutional neural networks (C-

NNs) [6], which is considered an efficient machine learning method for image

classification, has been proposed to analyze images. Despite CNNs used in the

ImageNet large scale visual recognition competition (ILSVRC) include images

for natural objects and other macro landscapes, researchers have applied CNNs

in various kinds of images, for example, biomedical images, microscopic images,

and X-ray images. Several advanced techniques, which accompany with CNNs

also improve the classification performance of CNNs, such as arrow detection

for identifying regions of interest (ROI) and fuzzy binarization for segmenting

image regions on biomedical image [7, 8]. There have been many researches in

digital microscopic image analysis employing CNNs as well. Philipp kainz et

al. [9]used CNNs classifier architectures to segment and classify the colorectal

cancer in histopathological images, proposing a new combination of separate-

3

net and object-net. In [10], epithelial and stromal regions in stained H&E

images were automatically extracted features and classified using deep CNNs,

which outperformed the traditional classification methods. Although several ap-

proaches based on CNNs have already proposed with success, the comparations

among them are not available. In [11], a comprehensive tutorial with seven use

cases, such as nuclei segmentation, lymphocyte detection and mitosis detection,

has been presented, outlining a guide for other researchers to study in digital

pathology domain by leveraging CNNs. In addition, for the segmentation and

classification of X-ray images, CNNs has also achieved satisfactory results [12].

In sum, CNNs has prevailed in the field of various medical image processing.

Although CNNs cannot always provide the best performance in the application

of medical image classification, more sophisticated and massive features can be

extracted and analyzed while traditional machine learning methods maybe not

wield too many features due to the limitations of time and computation com-

plexity - only with appreciate features can the traditional classifiers perform

well [10, 13, 14]. Hence, CNNs has become one of the best classifiers at present

in several applications.

A typical microscopic image of hepatic granuloma encompasses four tissue

components: multinuclear giant cells derived from macrophage, cytoplasm, ep-

ithelia and lymphocytes. The multinuclear giant cells are grouped by epithelia

that connect other tissues appearing fibrous, generating the granuloma annular,

in which the numerous nucleuses usually arrange densely, and centre lympho-

cytes. The difference between normal and abnormal microscopic images can

be shown in Figure 1. In the case of granuloma-fibrosis2, the annular area of

granuloma is larger and the background is more fibrotic.

In this article, we describe our CNN-based multi-classification strategy for

the mice hepatic granuloma microscopic images of three classes: normal, granuloma-

fibrosis1 and granuloma-fibrosis2. To our knowledge, there are no previous study

on hepatic granuloma microscopic images based on the application of artificial

intelligence technology. Thus, referring to works on other disease microscopy

images, we also propose a method of data preprocessing to overcome the prob-

4

Figure 1: From the left to the right: mice microscopic images of normal and two degrees of

hepatic granuloma

lem of insufficient number of training images. In order to evaluate the per-

formance of the proposed CNNs model, we compare the classification results

against the results obtained by other two traditional classifiers: support vec-

tor machine with polynomial kernel (SVM) [15] and random forest (RF) [16],

as well as against a famous and classic CNNs model: AlexNet [17]. All the

training processes were based on 4-fold cross validation for assuring the robust-

ness of the final results. In addition, as in many conventional machine learning

approaches, we use gray level co-occurrence matrix (GLCM) and local binary

pattern (LBP) to extract texture features including energy, contrast, correlation

and uniformity. Various features could be calculated through GLCM, the eight

features used in this work are the widespread and most typical features in the

field of image feature extraction. Hepatic granuloma and local annular fibrosis

can be well described by LBP. Therefore, we hypothesized that this two feature

extraction methods can attain the information about image textural character-

istics. After extracting all the features of the input images, we use Pearson

correlation coefficient (PCC) as the feature selection strategy to identify useless

features from the complete feature set, which also benefits to the final classifi-

cation performance. In terms of accuracy and details in the confusion matrix,

our proposed CNNs performed better: the total accuracy was 82.78% (Nor-

mal class, 92.5%; Granuloma-fibrosis1, 76.67%; Granuloma-fibrosis2: 79.17%).

Three postgraduates of the pathology department exchanged their advices with

5

us of what features should be extracted and how to split the datasets, making

our experiment with even more clinical practice significance.

The majority of the previous works regarding hepatic cirrhosis image anal-

ysis are mainly focused on ultrasound images, which usually include tasks of

image segmentation and classification. In [18], a computer-aided cirrhosis diag-

nosis (CAD) system based on ultrasound images was proposed, and CNNs were

employed to extract deep level features from liver capsule, which performed

better than hand-crafted features like histogram of oriented gradient (HOG)

and LBP. In such system, the over-fitting problem was faced during the train-

ing of the CNNs. Suganya and Rajaram [19] achieved accurate categorization

of cirrhosis ultrasound images using the methods: modified Laplacian pyramid

nonlinear diffusion filter as image preprocessing and GLCM, LBP, scale invari-

ant feature transform (SIFT) as image feature extraction. The used algorithm

has the advantage of the CNNs classification ability leading to an image classifi-

cation accuracy of 100%, which overcame considerably the accuracies obtained

by decision tree and SVM. When confronting onerous hand-crafted features,

feature selection is an ideal solution for researchers to improve the classification

performance. Some classic and efficient approaches have been suggested, such

as genetic algorithm [20], particle swarm optimization [21] and singular val-

ue decomposition [22]. In [23], liver cirrhosis, normal liver and hepatocellular

carcinoma were classified through applying the multiresolution wavelet packet

texture descriptors. Extracting ROI patches from the original images contribut-

ed to improve their work and enlightened us as well. With the feature selection

method of Genetic algorithm-SVM, the classification performance did improve

significantly.

Comparing with the current studies towards liver cirrhosis in the field of

medical image classification, the proposed method of data preprocessing avoid-

ed over-fitting as in [23]. It is different from [18, 19] as we achieved to train the

CNNs end to end in the dataset augmented data, which not only took advantage

of the description ability of CNNs, but also the classification of that.Although

numerous differences exist between ultrasound images and microscopic images

6

about liver cirrhosis, we can still refer them to improve our solution. On one

hand, the original ultrasound images of liver cirrhosis should not be cropped,

neither other data augmentation methods used, otherwise that would be unavail-

able for physicians to diagnose liver disease. However, when applying CNNs in

tasks of image classification, a large amount number of data is indispensable to

avoid overfitting. But as to medical image data, it is always difficult to obtain

sufficient and qualified image data. Noticeably, data augmentation has wide-

ly used in various kinds of medical image applications [6], such as skin cancer

[24], lung cancer [25] and cell detection [13]. Based on the data augmentation,

the proposed new method of data preprocessing towards hepatic granuloma mi-

croscopy focuses on how to crop the input raw images in order to satisfy the

demand in training the CNNs model. With the data preprocessing, we overcame

the challenge of insufficient image number and designed a CNNs architecture to

successfully classify 3 kind of mice liver microscopic images. On another hand,

liver cirrhosis in the ultrasound imaging shows the liver capsule as thickness with

jagged, wavy and stepped changes. Comparatively, microscopic images can car-

ry more rich information even after cropped, providing the accurate degree of

fibrosis and the number of granuloma annular. Therefore, applying CNNs to

analysis the mice liver granuloma microscopy was the motivation of this study,

and we anticipated to discover more information in liver granuloma microscopic

images with artificial intelligence.

The rest of this article is structured as followings: The details of the main

used methods are provided in Section 2. Then, the specific explanations of

our experimental setup are given in Section 3. The results and discussion are

presented in Section 4. Concluding remarks and future works are pointed out

in Section 5.

7

2. Methods

2.1. Convolutional neural networks

Convolutional neural networks are from the artificial neural network field

with a strong deep learning ability. Deep learning algorithms are based on the

deep architectures of continuing layers which orderly separate into convolutional

layer, full-connection layer and the output layer. In our CNNs architecture, the

output layer is the Softmax classifier (SMC) [26], which is a common choice for

multi-classification tasks. And, after been cropped, the microscopic images are

fed into every single layer successively. The framework of our proposed CNNs

architecture is shown in Figure 2.

Figure 2: Proposed CNNs architecture

2.1.1. The architecture of the convolutional layers

Typically, the convolutional layers include some alternative convolutional

layers (C layers) and pooling layers (P layers), which can remove image noise, re-

duce the dimensions of the original dataset and normalize the inputted dataset.

After convolution and sampling algorithms, the inputted images will complete

the process of extracting features and the feature maps are analyzed and com-

bined in other C layers.

• C layers Lets assume that Xk(m,n)represents the inputting 2-dimensional

map of the k− th channel, and Hkl(m,n) is the convolution kernel, being

m and n the dimensionality and k and l the number of input and out-

put channels so the size of the convolution kernel be k × l. Each kernel

8

represents a kind of extracting feature and generates a feature map as an

output. Hence, the function we use is:

Y l(m,n) = Xk(m,n) ∗Hkl(m,n) =K−1∑

k=0

I−1∑

i=0

J−1∑

j=0

Xk(m+ i, n+ j)Hkl(i, j)(1)

the range of k,i,j are k = 0, 1, ...,K; i = 0, 1, ..., I; j = 0, 1, ...J . where Y l(m,n)

is the outputting 2-dimensional feature map of the l-th channel andHkl(m,n)

, with size of I×J , means the 2-dimensional kernel of the l-th column and

k-th row. For every patch cropped from the inputted images, represented

by Xk(m,n), all the convolution kernels of every C layers will be operat-

ed with them and finally, the feature maps will be outputted as a result,

Y l(m,n). After attained the Y l(m,n), the results must be multiplied by

the activation function: Rectified Linear Unit (ReLU) [27] before trans-

ferring the data downward to the next layer; the used activation function

is:

ϕ(x) = max(0, x) (2)

The range of ReLU function is [0,+∞).

• P layers The features obtained by the convolution layers can be processed

again in the other layers, thus CNNs setups pooling layers to reduce the

calculated quantity. In this work, we employ the max-pooling method by

computing the maximum value of a given feature in a certain feature map.

Because the unusual features always stand out against the background

in the given image, probably described in a large number, max-pooling

method can efficiently and easily reduce the resolution of the feature maps

which attained in C layers.

2.1.2. The architecture of full-connection layers

In full-connection layers (FC layers), every node is connected to other nodes

from the adjacent layers. Calculated by matrix multiplication with vectors, full-

connection layers transform each input feature into a vector. CNNs still belong

9

to artificial neural networks that create the concept of full-connection layers

used and improved in CNNs.

2.1.3. Pipeline of the proposed CNNs based classifier

Figure 3 presents the whole process of liver microscopic image classification

into normal, hepatic granuloma-fibrosis1 and hepatic granuloma-fibrosis2 tis-

sues. Firstly, before fed into the proposed CNNs model, the raw microscopic

images ( 1536 × 2048) are labelled by pathologists, cropped (256 × 256 ) and

separated into 3 groups that are labelled again. This step is described in more

details in Section 3.1. Secondly, after assigning the dataset to train data and

test data randomly, we train the CNNs model through train data that are split

into four sets to conduct 4-fold cross validation. And finally, we use test data

to evaluate the completed CNNs model and assess the classification accuracy.

Figure 3: The pipeline of the proposed CNNs based classification

2.2. Pipeline of the SVM and RF based classifiers

The developed pipeline of the SVM and RF based classifiers, shown in Figure

4, can be resumed by the following five steps:

10

• Step 1: Labelling of all the raw microscopic images and cropped them to

the size of pixels;

• Step 2: Extraction of the texture features using GLCM and LBP methods;

• Step 3: Division of the input image dataset into training and testing

datasets;

• Step 4: Training of the SVM and RF based classifiers using the training

data;

• Step 5: Evaluation the classification performance of the classifiers.

Figure 4: Pipeline of the SVM and RF based classifiers

SVM with polynomial kernel and RF are two classic and widespread classi-

fiers in the field of image classification. The main concept of the SVM classifiers

is based on determining a hyperplane as a decision surface to maximize the iso-

lation edge between positive and negative samples. Based on the limited sample

information, an SVM classifier can make a compromise between the complexity

of the model and the learning ability. On the other hand, the RF classifier is a

combinatorial learning, which integrates and combines the classification results

11

of several single classifiers to obtain better results [28]. Accordingly, these two

classifiers are considered in the present work as conventional machine learning

methods.

2.2.1. Gray level co-occurrence matrix

Gray level co-occurrence matrix [29] is a statistical method for extracting

texture features from digital images through calculating the spatial relationship

of each image pixel. The implemented method is based on the joint conditional

probability density among image grayscale levels, whose function is:

Q(i, j|d, θ) =L−1∑

i=0

L−1∑

j=0

((x, y)|f(x, y) = i, f(x+ dx, y + dy) = j); (3)

where (i, j) is a pair of image pixels with distance equal to d and an angle θ

equal to: 0◦, 45◦, 90◦ or 135◦, and L is the maximal gray level value presented

in the input image of size N ∗ N . Therefore,Q(i, j|d, θ) can be also defined

taken into account the emergence probability of the given pair pixels, which

are i = f(x, y)&j = f(x + dx, y + dy), where (x, y){x, y = 0, 1, 2, ..., N − 1}represents the pixel location in the input image, with distance d and angle θ,

respectively. Various image features can be extracted from GLCM. Referring the

current research methods of analyzing microscopic images and combining with

the characteristics of granuloma microscopy, we choose eight texture features:

energy, contrast, entropy, correlation, mean, texture variance, dissimilarity and

homogeneity:

• Energy:

Energy =

L−1∑

i=0

L−1∑

j=0

Q2δ(i, j) (4)

• Contrast:

Ct =L−1∑

i=0

L−1∑

j=0

Q∗

δ(i, j)|i− j|2(Q∗

δ(i, j) = normalizeQδ(i, j)) (5)

12

• Entropy:

E = −L−1∑

i=0

L−1∑

j=0

Q∗

δ(i, j) ln(Q∗

δ(i, j)) (6)

• Correlation:

C =

L−1∑

i=0

L−1∑

j=0

(i− υ1)(j − υ2)Q∗

δ(i, j)

√σ1σ2

(7)

where:

υ1 =

L−1∑

i=0

L−1∑

j=0

iQ∗

δ(i, j) (8)

υ2 =

L−1∑

i=0

L−1∑

j=0

jQ∗

δ(i, j) (9)

σ1 =

L−1∑

i=0

L−1∑

j=0

(i − υ1)2Q∗

δ(i, j) (10)

σ2 =

L−1∑

i=0

L−1∑

j=0

(j − υ2)2Q∗

δ(i, j) (11)

• Mean:

Mean =

L−1∑

i=0

L−1∑

j=0

Q∗

δ(i, j)× i (12)

• Texture Variance:

TV =

L−1∑

i=0

L−1∑

j=0

Q∗

δ(i, j)× (i−Mean)2 (13)

• Dissimilarity:

D =

L−1∑

i=0

L−1∑

j=0

Q∗

δ(i, j)× |i − j| (14)

• Homogeneity:

H =

L−1∑

i=0

L−1∑

j=0

Q∗

δ(i, j)×1

1 + (i − j)2 (15)

13

2.2.2. Local binary pattern

The local binary pattern method [30] intends to describe the local features

of an input image and has been improved according to several novel variants,

such as circular the LBP method [31] used in this work. The original LBP

algorithm divides the image into many 3 × 3 cells, where the gray value of a

central pixel determines the neighborhood threshold. In each cell, all the gray

values of other eight pixels neighboring the central pixel are compared against

a threshold value; if the value of the surrounding pixel is greater than of the

central pixel, then the pixel is marked as 1 (one) or 0 (zero).

A formal description of the LBP operator is:

LBP (xc, yc) =

P−1∑

P=0

2P s(iP − ic) (16)

where (xc, yc) is the coordinates of the central pixel with gray value denoted

by ic, and iP means the gray values of the surrounding P pixels, and s is a sign

function defined as:

s(x) =

1

0

if

esle

x ≥ 0

(17)

By computing these two functions, the LBP values can be obtained, which

is represented by a piece of two-valued binary code. The LBP feature values

are described here by a vector consisted of ten characteristics.

2.2.3. Feature selection

Texture feature selection is an efficient method to reduce characteristic di-

mension in medical microscopic image classification. The use of a proper feature

selection technique provides several advantages, including the reduction in the

required execution time in the training process, and the improvement of the

classifier performance. Many practical and common strategies of feature selec-

tion have been applied in image classification, such as textural feature selection

by joint mutual information [32]. PCC is a suitable approach to measure the

14

correlation between feature variables, as well as to assist researchers to under-

stand the relationship between characteristics and response variables [33]. The

function of PCC is:

R =1

n− 1(Xi −X

SX)(Yi − Y

SY) (18)

where is the results of PCC ranged between 1 (one) to -1 (minus one), which

can give the strength and monotonicity of the variables relationship; n represents

the number of samples, 360 in our experiment; X and Y is two variables that

are features and classes; and X and SX represent the average and variance of

two variables.

3. Experimental setup

3.1. Used dataset and preprocessing

The used hepatic cirrhosis microscopic images were labelled as normal, granuloma-

fibrosis1 and granuloma-fibrosis2 without lesions location label by experienced

pathologists. Our labeled dataset contains 30 mice liver histopathological im-

ages of size 1536 × 2048 pixels divided into 10 normal images, 10 granuloma-

fibrosis1 images and 10 granuloma-fibrosis2 images.

Firstly, due to the demand of CNNs training, all the original images were

cropped in patches of size equal to 256× 256 pixels, and then after disordered,

the patches were recombined to new images also of size equal to 1536 × 2048

pixels. In this process, we expanded one image of 1536 × 2048 pixels to 48

(1536× 2048/256× 256) sub-images. Nonetheless, when the size of the original

images was not divised by 256× 256, we used the pixels that the gray value is

1 (one) to complete the original images, or we just selected the nearst point to

make the raw image divisible. In addition, after cropping the recombined images

of size 1536× 2048 pixels to obtain the final patches of size 256× 256 pixels, we

labeled these patches according to the original labels and separated them into 3

sub-datasets: training dataset, validation dataset and testing dataset. Except

15

for the CNNs model, the validation and the training datasets were mixed. The

pseudocode of data preprocessing is:

• Step 1: Input the original images (1536× 2048);

• Step 2: Crop the original images to patches (256× 256);

• Step 3: Disorder the patches and recombine randomly;

• Step 4: Crop the new images again to new patches in same size;

• Step 5: Label the patches according to the original labels, which means

that if a patch is from a raw image labelled normal, granuloma-fibrosis1

or granuloma-fibrosis2, the label of this patch will correspond to it;

• Step 6: Assign the patches into 3 sub-datasets: training dataset, validation

dataset and testing dataset. End.

Figure 5: An original input microscopic image

16

Figure 6: the correspondent image after rearrange and combine the built image patches

With this data preprocessing, we enlarged the image dataset from 30 to 1,440

sub-images (expand 48 times) that were divided into 3 image data sets: one of

720 training images, another one with 360 validation images and a last one of

360 testing images. That is, we augmented our initial dataset of 30 medical

microscopic images to a dataset of 1,440 images. The details of the achieved

dataset are given in Table 1. Additionally, some examples of cropped patches

from the original images are shown in Figure 5 and the correspondent image

after rearrange and combine the built image patches in Figure 6. According

to the mentioned approach of relabeling in Step 5, the 5 patches cropped from

normal images in the first level (Figure 7 (first row)) were relabeled as Normal

regarding to the rest 2 groups.

3.2. Extracting and selecting features from patches

We extracted the texture features from all patches by using GLCM and LBP

methods, obtaining 18 characteristics to train our CNNs based classifier. Then,

17

Table 1: The dataset of the microscopic images after data augmentation

Types Training Validation Testing Total

Normal 240 120 120 480 (10*6*8=480)

Granuloma-fibrosis1 240 120 120 480 (10*6*8=480)

Granuloma-fibrosis2 240 120 120 480 (10*6*8=480)

we selected the optimal features using the PCC based approach.

3.3. Training model with SVM and RF

SVM and RF based classifiers were trained with 1,440 cropped images of

size equal to256 × 256 pixels. For the SVM classifier, we used LIBSVM [34].

All the images were separated using 4-fold cross validation before every exper-

iment, and the polynomial kernel and 5-fold cross-validation were employed to

seek the optimal parameters of the polynomial kernel according to a stride of

optimization of 0.5 and raging from -2 to 4. Two parameters played an im-

portant role in the SVM classifier with polynomial kernel, namely the optimum

parameters of cost (C) and the kernel width parameter (G). For both of them

was used the same strategy to find the optimum parameters, and the obtained

values were (2−2, 2−1.5, 2−1, 2−0.5, 20, 20.5, 21, 21.5, 22, 22.5, 23, 23.5, 24). We also

tried to expand the hunting zone, but all the best results concentrated on the

above zone. For the RF classifier, many studies have concluded that satisfac-

tory results could be achieved with the default parameters [28], except for 2

parameters: the number of decision trees and the number of features in each

single classifier. A quarter of 1,440 image data was randomly selected as testing

data for cross validation (a random generator number is 20); we adopted 180

estimators, the number of decision trees used, and 18 features (8 GLCM features

and 10 LBP features) for every single classifier.

3.4. Details of CNNs layer

The proposed CNNs architecture, which was motivated by the AlexNet ar-

chitecture, consists of 3 convolutional layers, 3 pooling layers, 3 full-connection

18

Figure 7: Examples of five cropped images resultant from normal (first row), granuloma-

fibrosis1 (second row) and granuloma-fibrosis2 (third row) images

layers and the output layer with SMC. Thus, the parameters in the proposed

CNNs model are identical to the AlexNet model. We adjusted the kernel size

due to the relatively small dataset. Table 2 presents details of each layer of the

used CMNs model (Figure 2). The size of the input data is equal to 224× 224

after normalization, and R, G, B are the 3 input color channels. For C1 C3, the

output feature maps of every layer are of size equal to: C1=55×55, P2= 27×27,

C2= 12×12, P2= 6×6 and C3= 6×6, respectively. Correspondingly, the num-

ber of convolutional kernels of every layer are: 64, 64, 32, 32, and 16. A function

that describes the relationship among the other parameters of layers is:

Size(Featuremap) =Size(InputData)− Size(Kernel) + 2× Pad

Stride+ 1 (19)

For example, the size of feature map in P1 layer is 55 × 55, calculated as:

19

Size(C1Featuremap) = 224−8+2×04 + 1 = 55. In addition, by considerning the

characteristic of the used microscopic image datasets in the current study, we

reduced the number of convolutional kernels, which are much more in AlexNet

(from the first convolutional layer to the fifth convolutional layer): 96, 256, 384,

256, 256. In items of FC layers, the first of these 3 layers has 1024 neurons

counting on the size of input image, and the second layer has 512 neurons. The

last FC layer has 3 neurons, which output the 3 results of classification and

input to an 3-way SMC.

Table 2: Output and parameters of the used CNN model

Parameter layers Output Kernel Size Stride Pad Dropout ratio

Input Data (3, 224, 224) - - - -

C1 (64, 55, 55) 8 4 0 -

P1 (64, 27, 27) 3 2 0 -

C2 (32, 12, 12) 8 2 2 -

P2 (32, 6, 6) 3 2 0 -

C3 (16, 6, 6) 3 1 1 -

FC1 (1024) - - - 0.5

FC2 (512) - - - 0.5

FC3 (3) - - -

3.5. Parameter setting for training the CNNs

In the process of training the proposed CNNs model, the learning rate poli-

cy determines how to adjust the learning speed of each layer, and the adopted

parameters were: a base learning rate (base-lr) of 0.01; the learning method (lr-

policy) was step; and gamma and stepsize were 0.5 and 10000, respectively. An-

other critical parameter setup is the approach of seeking optimal solution, which

is AdaDelta in this work [35]. Some of the CNNs parameters are common in ap-

plications employing CNNs, such as the setting of learning rate which is closely

related to lr-policy, base-lr, gamma, and stepsize (the relationship can be shown

20

as a function: learning rate = base lr × gamma(iteration/stepsize)). However,

there is no consensus for other parameters and repeated experiments will attain

the optimal parameters [36]: as batch size, which here was 8; max iteration was

20000; and weight decay had the value of 0.0005.

3.6. Experimental platform

All experiments were conducted on a NVIDIA GeForce GTX 1070 GPU un-

der the Community Enterprise Operating System (CentOS) 7.3 and on a PC

(Intel Core (TM) i7-6700HQ, 2.6 GHz processor with 8 GB of RAM). The soft-

ware implementation was performed using MATLAB 2016A and Caffe frame-

work [27].

4. Results and Discuss

The image features extracted using GLCM method are indicated in Table

3, which includes the mean values of all 8 texture features calculated from the

1,440 microscopic patches built after taking into account the angles of: 0◦, 45◦,

90◦and 135◦. Figure 8 presents the differences found among the three types of

microscopy images in terms of the LBP characteristic vector.

Table 3: Mean values of the 8 texture features extracted using the GLCM method

Texture feature Normal Granuloma-Fibrosis1 Granuloma-Fibrosis2

Contrast 0.1149 0.1018 0.0945

Correlation 0.5011 0.6711 0.6270

Energy 0.8951 0.8835 0.8939

Homogeneity 0.3942 0.4727 0.4596

Entropy 2.6811 2.8128 2.8562

Dissimilarity 0.8189 0.7919 0.7947

Texture Mean 4.1104 3.7889 3.9598

Variance 2.4657 2.9229 3.0670

21

1 2 3 4 5 6 7 8 9 10

LBP Histogram Bins

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8LBP Histograms

Normal-LBPGranuloma-fibrosis1-LBPGranuloma-fibrosis2-LBP

Figure 8: The histogram of the LBP feature values for the three image types under study

After the feature selection step, we are aware of the features that are useful

or useless. We can find that in Table 4, the high positive correlations and high

scores are noted such as for Correlation, Homogeneity, Contrast and Dissimilar-

ity; on other hand, the LBP feature-vector5 and vector9 are negative, meaning

that these features may be weak. These experimental results are consistent with

the actual diagnosis of liver cirrhosis: as in the initial stage, the histopatholog-

ical images of hepatic granuloma ought to have higher Correlation and lower

Contrast because liver granulomas will group cells disorderly and densely ar-

ranged. Actually, in our experiment, the texture features extracted using the

GLCM method are stronger than the LBP based features.

The confusion matrix [37] is a visualization tool commonly used in artifi-

22

Table 4: The Pearson Correlation Coefficient results

Texture PCC (scores, p-value) Texture PCC (scores, p-value)

Contrast (0.318, 8.652e-13) LBP-1 (0.190, 2.747e-05)

Correlation (0.438, 6.397e-24) LBP-2 (0.196, 1.499e-05)

Energy (0.098, 0.031) LBP-3 (0.240, 9.687e-08)

Homogeneity (0.453, 9.650e-26) LBP-4 (0.365, 1.346e-16)

Entropy (0.396, 1.586e-19) LBP-5 (-0.050, 0.270)

Dissimilarity (0.443, 1.331e-24) LBP-6 (0.188, 3.221e-05)

Texture Mean (0.020, 0.646) LBP-7 (0.131, 0.003)

Variance (0.205, 5.908e-06) LBP-8 (0.167, 0.000)

LBP-9 (-0.008, 0.858)

LBP-10 (0.157, 0.000)

cial intelligence technology to assess the performance of a classifier. In image

multi-classification tasks, the confusion matrix displays each class result and

compares the classification results with the actual measured values. With these

elaborated results, we can not only analyze the classification accuracies. but

also calculate other indexes, such as precision, recall and f-score. In this work,

taking in to account the test dataset of 360 sub-images, we decided to explore

the experimental results using the confusion matrix.

Table 5 presents the classification results of every class elaborately in a con-

fusion matrix. 100 images are correctly predicted in 120 images labelled Normal;

there are 5 Normal images predicted to be Granuloma-fibrosis1; and 15 images

are misjudged to Granuloma-fibrosis2 instead of Normal. Table 6 indicates the

accuracy computed for each class under study. Totally, the model of SVM-

Polynomial performed best against the other 3 algorithms. The accuracies of

3 kinds of images are symmetrical and satisfied, suggesting that the SVM al-

gorithm was very competitive in the image classification task of the used small

dataset.

Tables 7 and 8 present the classification results as to the RE based classifier.

23

Table 5: Confusion matrix of the classification results using the SVM-Polynomial based clas-

sifier

Actual class

Predicted classNormal G..-f..1 G..-f..2

Normal 100 5 15

Granuloma-fibrosis1 7 101 12

Granuloma-fibrosis2 10 9 101

Table 6: The accuracy of the classification results using the SVM- Polynomial based classifier

Microscopic image types Accuracy Total / True

Normal 83.33% (120 / 100)

Granuloma-fibrosis1 84.17% (120 / 101)

Granuloma-fibrosis2 84.17% (120 / 101)

As a contrast algorithm to the proposed CNNs method, all the images of the

dataset were used to train the SVM and RF based classifiers; however, the

overall classification accuracy was inferior to the one obtained by the SVM based

classifier. We deem that the reason for this is closely related to the number of

training images used.

Table 7: Confusion matrix of the classification results obtained using the RF based classifier

Actual class

Predicted classNormal G..-f..1 G..-f..2

Normal 92 8 20

Granuloma-fibrosis1 9 96 15

Granuloma-fibrosis2 18 9 93

The correspond results as AlexNet and the proposed CNNs based classifiers

are presented in Tables 9 to 12. Comparing with the results of SVM and RF,

we can find two interesting situations: the accuracy of the Normal class is very

24

Table 8: Accuracy of the classification results obtained using the RF based classifier

Microscopic image types Accuracy Total / True

Normal 76.67 % (120 / 92)

Granuloma-fibrosis1 80 % (120 / 96)

Granuloma-fibrosis2 79.5 % (120 / 93)

high; and the samples misidentified are focused on the Granuloma-fibrosis1 and

Granuloma-fibrosis 2 classes. In fact, the results have more practical and clinical

significance because it is easier for pathologists to identify the normal situation

rather than the different degree of liver granuloma.

Table 9: Confusion matrix of the classification results obtained using the AlexNet classifier

Actual class

Predicted classNormal G..-f..1 G..-f..2

Normal 108 10 2

Granuloma-fibrosis1 12 86 22

Granuloma-fibrosis2 7 15 98

Table 10: Accuracy of the classification results obtained using the AlexNet classifier

Microscopic image types Accuracy Total / True

Normal 90 % (120 / 108)

Granuloma-fibrosis1 71.67 % (120 / 86)

Granuloma-fibrosis2 81.67 % (120 / 98)

Compared with the SVM, RF and AlexNet based classifiers, according to the

confusion matrix, our proposed CNNs based model has many advantages: First-

ly, the Normal class gains the highest accuracy score: 92.5%, and for the cases

of misjudgment: only 7 Normal samples were wrongly classified to Granuloma-

fribrosis1, which can be justified by the fact that the early stage of liver cirrhosis

25

Table 11: Confusion matrix of the classification results obtained using the proposed CNNs

based classifier

Actual class

Predicted classNormal G..-f..1 G..-f..2

Normal 111 7 2

Granuloma-fibrosis1 6 92 22

Granuloma-fibrosis2 5 20 95

Table 12: Accuracy of the classification results using the proposed CNN based classifier

Microscopic image types Accuracy Total / True

Normal 92.5 % (120 / 111)

Granuloma-fibrosis1 76.67 % (120 / 92 )

Granuloma-fibrosis2 79.17 % (120 / 95 )

resembles the normal state. Secondly, for the Granuloma-fibrosis 1 and 2, al-

though the accuracy is inferior than the ones of SVM and RF based classifiers,

the number of Granuloma-fibrosis2misjudged to Normal is too high, which could

nonplus the pathologists due to the evident differences between them. However,

the proposed CNNs and AlexNet based classifiers avoid this case, which also

suggests that the CNNs based classifier is superior to the traditional classifiers

studied in this work for comparison purpose. In addition, our CNNs based clas-

sifier overcome the AlexNet classifier by having more concise architecture: 3 C

and 2 P layers, 3 FC layers and the SMC classifier.

From Figure 9 and Table 13, it can be realized that the misclassification

between granuloma-fibrosis1 and granuloma-fibrosis2 are represented in 4 sub-

images, which are d, e, h, and i. We analyze that there are a few confusions

in the degree of fibrosis in liver granuloma after cropping the original images,

which leads to mistakes. Nonetheless, for other samples in Table 13, we suspect

that these issues are mislabeled after cropping and that a larger dataset could

overcome this problem.

26

Table 13: Comparison of the samples misclassified by the proposed CNNs model relatively to

the other three classification algorithms

Image data Label SVM RF AlexNet proposed CNNs model

a 0 0 0 2 1

b 0 0 2 1 1

c 0 0 2 2 2

d 1 1 1 0 2

e 1 1 1 2 2

f 1 1 1 0 0

g 2 2 2 0 0

h 2 2 2 1 1

i 2 2 2 1 1

Table 14 presents the global accuracy obtained by each of the four classifiers

under comparison.

Table 14: Global accuracy obtained by each classifier

Classifiers Accuracy

Random Forest 78.05 %

SVM-Polynomial 84.17 %

AlexNet 81.11 %

CNNs 82.78 %

Despite our experimental results are not satisfactory as to the RF based

classifier, we cannot deem that the RF classifier is unpromising as the lack

of enough training data can also lead to failure. In [18, 19, 23], very high

classification accuracies and of other evaluation criteria do notarize the success

of this classifier. However, when in multi-classification tasks, it is not enough

only to attain high classification accuracy. We also completed the 2-classification

task in the mice hepatic microscopic images which reached satisfied classification

27

accuracy in our previous works [38].

5. Concluding Remarks and Future works

This research article has presented our designed CNNs architecture that

shown to be competent to classify mice liver microscopic images into normal,

granuloma-fibrosis1 and granuloma-fibrosis2 cases. In order to assess the per-

formance of the proposed CNNs based classifier, we conducted the classification

experiments using other three common classifiers: SVM with polynomial kernel,

RF and AlexNet based classifiers. The experimental results were discussed eval-

uated based on the confusion matrixes built. Moreover, the results of the SVM

and RF based classifiers were also satisfactory, suggesting that our methods of

data preprocessing and feature extraction are promising. After the feature se-

lection step, we figured out that the texture features obtained using the GLCM

method are highly distinguishable.

In future works, we still entail to deal with the problem of insufficient number

of training images. Because when we trained the classifiers by augmenting data

to avoid overfitting, the classification results were not quite desirable which

should be better if we can acquire sufficient images. To assist pathologists

more comprehensively, apart from the final classification results, we should also

try to segment the lesion areas: the annular granulomas and other diagnostic

relevant information employing CNNs. Furthermore, there are other efficient

features extraction methods, such as shape-based features and texture-based

features [39, 40, 41], that can be used to boost the performance of conventional

classifiers. Finally, it is recommended to calculate the error rate along with the

accuracy to evaluate the classifiers performance.

Acknowledgement

To Dalia S. Ashour, Assistant Professor of Medical Parasitology and Dina M.

Aboraya, Lecturer of Medical Parasitology, Medical Parasitology Department,

28

Faculty of Medicine, Tanta University, in Egypt, who performed the microscopic

experiment and preprocessed the acquire image dataset.

References

[1] M. Wehmeyer, J. Lutz, J. S. Z. Wiesch, A. W. Lohse, C. Schram-

m, M. Sebode, Clinical presentation, natural history and treatmen-

t of hepatic sarcoidosis: A case-series of 39 patients with histological-

ly proven hepatic sarcoidosis, Journal of Hepatology 68 (2018) 616–617.

doi:10.1016/S0168-8278(18)31490-9.

[2] M. Pavlides, J. Birks, E. Fryer, D. Delaney, N. Sarania, R. Baner-

jee, S. Neubauer, K. Fleming, L. M. Wang, Interobserver variability

in histologic evaluation of liver fibrosis using categorical and quantita-

tive scores, American Journal of Clinical Pathology 147 (4) (2017) 364.

doi:10.1093/ajcp/aqx011.

[3] S. Chakraborty, S. Chatterjee, N. Dey, A. S. Ashour, Fuqian.Shi, K. Mali,

Modified cuckoo search algorithm in microscopic image segmentation of

hippocampus, Microscopy Research and Technique 80 (10) (2017) 1051–

1072. doi:10.1002/jemt.22900.

[4] Z. Li, N. Dey, A. S. Ashour, L. Cao, Y. Wang, D. Wang, P. McCauley, V. E.

Balas, K. Shi, F. Shi, Convolutional neural network based clustering and

manifold learning method for diabetic plantar pressure imaging dataset,

Journal of Medical Imaging and Health Informatics 7 (3) (2017) 639–652.

doi:10.1166/jmihi.2017.2082.

[5] A. S. Ashour, S. Beagum, N. Dey, A. S. Ashour, D. S. Pistolla, G. N. N-

guyen, D. N. Le, F. Shi, Light microscopy image de-noising using optimized

lpa-ici filter, Neural Computing and Applications 29 (12) (2018) 1517–1533.

doi:10.1007/s00521-016-2678-9.

29

[6] W. Rawat, Z. Wang, Deep convolutional neural networks for image clas-

sification: A comprehensive review, Neural Computation 29 (9) (2017) 1.

doi:10.1162/NECO\_a\_00990.

[7] K. C. Santosh, N. Alam, P. P. Roy, L. Wendling, S. Antani, G. R. Thoma,

A simple and efficient arrowhead detection technique in biomedical im-

ages, International Journal of Pattern Recognition and Artificial Intelli-

gence 30 (5) (2015) 1657002. doi:10.1142/S0218001416570020.

[8] K. C. Santosh, L. Wendling, S. Antani, G. R. Thoma, Overlaid arrow detec-

tion for labeling regions of interest in biomedical images, IEEE Intelligent

Systems 31 (3) (2016) 66–75. doi:10.1109/MIS.2016.24.

[9] P. Kainz, M. Pfeiffer, M. Urschler, Segmentation and classification of colon

glands with deep convolutional neural networks and total variation regu-

larization, PeerJ 5 (3) (2017) e3874. doi:10.7717/peerj.3874.

[10] Jun.Xu, Xiaofei.Luo, G. Wang, H. Gilmore, A. Madabhushi., A deep convo-

lutional neural network for segmenting and classifying epithelial and stro-

mal regions in histopathological images, Neurocomputing 191 (2016) 214–

223. doi:10.1016/j.neucom.2016.01.034.

[11] A. Janowczyk, A. Madabhushi, Deep learning for digital pathology im-

age analysis: A comprehensive tutorial with selected use cases, Journal of

Pathology Informatics 7 (1) (2016) 29. doi:10.4103/2153-3539.186902.

[12] P. Lakhani, Deep convolutional neural networks for endotracheal

tube position and x-ray image classification: Challenges and op-

portunitiess, Journal of Digital Imaging 30 (4) (2017) 460–468.

doi:10.1007/s10278-017-9980-7.

[13] X. Pan, D. Yang, L. Li, Z. Liu, H. Yang, Z. Cao, Y. He,

Z. Ma, Y. Chen, Cell detection in pathology and microscopy im-

ages with multi-scale fully convolutional neural networks, World

30

Wide Web-internet and Web Information Systems 11 (2018) 1–23.

doi:https://doi.org/10.1007/s11280-017-0520-7.

[14] S. Vajda, K. C. Santosh, A fast k-nearest neighbor classifier using unsuper-

vised clustering, Recent Trends in Image Processing and Pattern Recogni-

tion (2017) 185–193doi:10.1007/978-981-10-4859-3-17.

[15] R. S. Majdar, H. Ghassemian, A probabilistic svm approach for hy-

perspectral image classification using spectral and texture features,

International Journal of Remote Sensing 38 (15) (2017) 4265–4284.

doi:10.1080/01431161.2017.1317941.

[16] A. Jog, A. Carass, S. Roy, D. L. Pham, J. L. Prince, Random forest re-

gression for magnetic resonance image synthesis, Medical Image Analysis

35 (2017) 475–488. doi:10.1016/j.media.2016.08.009.

[17] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep

convolutional neural networks, Advances in Neural Information Processing

Systems 25 (2) (2012) 1–9. doi:10.1145/3065386.

[18] X. Liu, J. Song, S. Wang, J. Zhao, Y. Chen, Learning to diagnose cirrhosis

with liver capsule guided ultrasound image classification, Sensors 17 (1)

(2017) 149. doi:10.3390/s17010149.

[19] S. Ramamoorthy, S. Rajaram, An efficient categorization of liver cirrhosis

using convolution neural networks for health informatics, Cluster Comput-

ing 3 (2017) 1–10. doi:10.1007/s10586-017-1629-2.

[20] S. Jiang, K. S. Chin, L. Wang, G. Qu, K. L. Tsui, Modified genetic

algorithm-based feature selection combined with pre-trained deep neural

network for demand forecasting in outpatient department, Expert Systems

with Applications 82 (2017) 216–230. doi:10.1016/j.eswa.2017.04.017.

[21] B. Xue, M. Zhang, W. N. Browne, Particle swarm optimization for feature

selection in classification: a multi-objective approach, IEEE Trans Cybern

43 (6) (2013) 1656–1671. doi:10.1109/TSMCB.2012.2227469.

31

[22] T. H. E. Meuwissen, U. G. Indahl, J. ødegard, Variable selection mod-

els for genomic selection using whole-genome sequence data and singu-

lar value decomposition, Genetics Selection Evolution 49 (1) (2017) 94.

doi:10.1186/s12711-017-0369-3.

[23] J. Virmani, V. Kumar, N. Kalra, N. Khandelwal, Svm-based char-

acterization of liver ultrasound images using wavelet packet tex-

ture descriptors, Journal of Digital Imaging 26 (3) (2013) 530–543.

doi:10.1007/s10278-012-9537-8.

[24] A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau,

S. Thrun, Dermatologist -level classification of skin cancer with deep neural

networks, Nature 542 (7639) (2017) 115–118. doi:10.1038/nature21056.

[25] K.-H. Yu, C. Zhang, G. J. Berry, R. B. Altman, C. Re, D. L. Rubin, M. S-

nyder, Predicting non-small cell lung cancer prognosis by fully automated

microscopic pathology image features, Nature Communications 7 (2016)

12474. doi:10.1038/ncomms12474.

[26] K. Duan, S. S. Keerthi, W. Chu, S. K. Shevade, A. N. Poo, Multi-category

classification by soft-max combination of binary classifiers, multiple clas-

sifier systems, Multiple Classifier Systems, International Workshop 2709

(2003) 125–134. doi:10.1007/3-540-44938-8-13.

[27] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick,

S. Guadarrama, T. Darrell, Caffe: Convolutional architecture for fast fea-

ture embedding, Proceedings of the 22nd ACM international conference on

Multimedia (2015) 675–678doi:10.1145/2647868.2654889.

[28] P. T. Noi, M. Kappas, Comparison of random forest, k-nearest neighbor,

and support vector machine classifiers for land cover classification using

sentinel-2 imagery, Sensors 18 (1) (2018) 18. doi:10.3390/s18010018.

[29] T. Kobayashi, D. Sundaram, K. Nakata, H. Tsurui, Gray-level co-

occurrence matrix analysis of several cell types in mouse brain using

32

resolution-enhanced photothermal microscopy, Journal of Biomedical Op-

tics 22 (3) (2017) 036011. doi:10.1117/1.JBO.22.3.036011.

[30] S. Wan, H. hieh. Lee, X. Huang, T. Xu, T. Xu, X. Zeng, Z. Zhang,

Y. Sheikine, J. L. Connolly, J. G. Fujimoto, C. Zhou, Integrated local

binary pattern texture features for classification of breast tissue imaged by

optical coherence microscopy, Medical image analysis 38 (2017) 104–116.

doi:10.1016/j.media.2017.03.002.

[31] T. Ojala, M. Pietikainen, T. Maenpaa, Multiresolution gray scale and rota-

tion invariant texture classification with local binary patterns, IEEE Trans-

actions on Pattern Analysis andMachine Intelligence 24 (7) (2002) 971–987.

doi:10.1109/TPAMI.2002.1017623.

[32] S. Vajda, A. Karargyris, S. Jaeger, K. C. Santosh, S. Candemir, Z. Xue,

S. Antani, G. Thoma, Feature selection for automatic tuberculosis screening

in frontal chest radiographs, Journal of Medical Systems 42 (8) (2018) 146.

doi:10.1007/s10916-018-0991-9.

[33] E. Reggiani, E. DArnese, A. Purgato, M. D. Santambrogio, Pearson corre-

lation coefficient acceleration for modeling and mapping of neural intercon-

nections, Parallel and Distributed Processing Symposium Workshops,IEEE

(2017) 223–228doi:10.1109/IPDPSW.2017.63.

[34] R.-E. Fan, P.-H. Chen, C.-J. Lin, Working set selection using second or-

der information for training support vector machines, Journal of Machine

Learning Research 6 (4) (2005) 1889–1918.

[35] BAIR, Deep learning framework, http://caffe.berkeleyvision.org/tutorial/solver.htm.

[36] Deep learning for word-level handwritten indic script iden-

tification, Computer Vision and Pattern Recognition, arX-

iv:1801.01627doi:arXiv:1801.01627.

33

[37] J. Ding, X. Hu, V. Gudivada, A machine learning based framework for

verification and validation of massive scale image data, IEEE Transactions

on Big Data 1 (1) (2017) 99. doi:10.1109/TBDATA.2017.2680460.

[38] Y. Wang, L. Cao, N. Dey, A. S. Ashour, F. Shi, Mice liver cirrhosis mi-

croscopic image analysis using gray level co-occurrence matrix and support

vector machines, IOS Press - Frontiers in Artificial Intelligence and Appli-

cations (296) (2017) 509–515. doi:10.3233/978-1-61499-785-6-509.

[39] A. Karargyris, J. Siegelman, D. Tzortzis, S. Jaeger, S. Candemir, Z. Xue,

K. C. Santosh, S. Vajda, S. A. L. Folio, G. R. Thoma, Combination of

texture and shape features to detect pulmonary abnormalities in digital

chest x-rays, International Journal of Computer Assisted Radiology and

Surgery 11 (1) (2015) 1–8. doi:10.1007/s11548-015-1242-x.

[40] K. C. Santosh, S. Antani, Automated chest x-ray screening: Can lung

region symmetry help detect pulmonary abnormalities, IEEE Trans Med

Imaging 37 (5) (2018) 1168–1177. doi:10.1109/TMI.2017.2775636.

[41] K. C. Santosh, S. Vajda, S. Antani, G. R. Thoma, Edge map analysis in

chest x-rays for automatic pulmonary abnormality screening, IEEE Trans-

actions on Pattern Analysis and Machine Intelligence 11 (9) (2016) 1637–

1646. doi:10.1007/s11548-016-1359-6.

34

Figure 9: Samples misclassified by the proposed CNNs model

35