Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

DERMATOLOGICAL DISEASE DIAGNOSIS USING COLOR-SKIN IMAGES

M. SHAMSUL ARIFINI, M. GOLAM KIBRIA2, ADNAN FIROZE" M. ASHRAFUL AMINI, HONG YAN3

1 Computer Vision and Cybernetics Group, SECS, Independent University, Bangladesh 2Department of Computer Science & Engineering, University of Information Technology & Sciences

3 Department of Electronic Engineering, City University of Hong Kong, Hong Kong E-MAIL: arifin_cham@yahoo.com.golam.kibria@uits.edu.bd.adnan.firoze@gmail.com.aminmdashraful@ieee.org.

h.yan@cityu.edu.hk

Abstract: This paper presents an automated dermatological

diagnostic system. Etymologically, dermatology is the medical

discipline of analysis and treatment of skin anomalies. The

system presented is a machine intervention in contrast to human

arbitration into the conventional medical personnel based

ideology of dermatological diagnosis. The system works on two

dependent steps - the first detects skin anomalies and the latter

identifies the diseases. The system operates on visual input i.e.

high resolution color images and patient history. In terms of

machine intervention, the system uses color image processing

techniques, k-means clustering and color gradient techniques to

identify the diseased skin. For disease classification, the system

resorts to feedforward backpropagation artificial neural

networks. The system exhibits a diseased skin detection

accuracy of 95.99% and disease identification accuracy of

94.016% while tested for a total of 2055 diseased areas in 704 skin images for 6 diseases.

Keywords: Skin anomalies, Color Gradient, Clustering, GLCM,

Dermatology.

1. Introduction

Dermatology is the branch of medical science that is concerned with diagnosis and treatment of skin based disorders. The vast spectrum of dermatological disorders varies geographically and also seasonally due to temperature, humidity and other environmental factors. Human skin is one of the most unpredictable and difficult terrains to automatically synthesize and analyze due to its complexity of jaggedness, tone, presence of hair and other mitigating features. Even though there have been several researches conducted to detect and model human skin using Computer Vision techniques, very few have concentrated on the medical paradigm of the problem. Expert diagnostic systems that deal with dermatological disorders are hard to find in the scholarly area of Intelligent and Expert Systems. Therefore, our system

978-1-4673-1487-9/12/$31.00 ©2012 IEEE

addresses such complexities of the school of dermatology of medical science.

Even though a substantial amount of work has undergone in the amalgamation of medicine and computer science, little research has been conducted that connects computer vision and dermatology and none have taken place in the subcontinent of South Asia to our knowledge.

Techniques for segmentation of dermatoscopic images using Stabilized Inverse Diffusion Equations (SIDE) were introduced by Gao et al. [1]. They focused on segmenting legions from regular skin surfaces. In their experiment they used the 6 segmentation procedures to 87 images of Nevi and Melanomas. Their accuracy using algorithms: MMRF, MMRFISIDE, MSIDE, Median cut and Adaptive thresholding in blue channel were 75.29%, 71.76%, 63.5%, 54.77% and 51.5% respectively. A multispectral image processing techniques in dermatology was proposed by Jalil [2]. She exposed several approaches in segmenting and labeling skin anomalies using transformations in the frequency domain. She found an average accuracy of 86.4% in detecting skin abnormalities. Two researches that were close to our very own work presented in this paper were conducted by Antkowiak [3, 5]. His dataset consisted of 215 dermatological images. He used the Fourier transforms of the images and used them as features directly to the Artificial Neural Networks (ANN) and Support Vector Machine (SVM). He presented a comparative analysis on of his classification performances based on SVM and ANN. Using (SVM, ANN), his disease wise classification were as follows: Acne Vulgaris (45.1%, 96.1%), Atopic Dermatitis (42.1%, 93.9%), Granuloma Annulare (42.9%, 94.0%), Keloid (55.0%,95.4%), Melanocytic Nevus (64.1%, 97.4%), Melanoma Maligna (48.4%,96.8%), Nevus Pilosus (66.1%, 97.4%). Thus, ANN showed significantly better classification performance. Similar approaches using ANN were employed by Agatonovic-Kustrin et al. [4] and Bishop and Beresford [6].

A varied imaging dataset were introduced by du Vivier [7] and eCureMe [8]. Gniadecka et al. [9] and Delgado

1675

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

Gomez et al. [10] introduced specialized neural techniques for the detection of Melanoma and Psoriasis. In an extensive study, Handels et al. [11] introduced techniques to extract features from visual skin tumors. Hoffmann et al. [12] focused on diagnosis of skin cancer using neural approaches which was exemplary.

2. The Dermatological Disease Diagnosis System

The complete methodology of our system is represented in Fig. 1 as a flowchart. The individual steps are modularized and are often autonomous and sometimes dependent on each other.

Data Collechon and Image AcqUlS1hon

Color Grad! ent G enerah on

Cl u stenn giThresho 1 ding an d D etechng 0 f th e Re g1 on s of Interest (ROJ)

Fe alure Extrach 0 n

System Trammg, Teshng and Vahdahon

Fig. 1. A flowchart of the methodology of our system

2.1. Skin Image Acquisition and Data Collection

The fIrst and primary task was to collect necessary data of patients in order to develop the system. Besides this, the non-visual data of patient histories had to be collected. Thus, a specialized doctor was present to validate and record the external data of the patients such as disease history, feeling in diseased part of body, elevation of the diseased region etc. Some characteristic details of the data collection steps are given below.

• Camera Build: Pansonic Lumix FZ-35 • Focusing: The camera was focused manually to adjust

the variable nature of natural light • Image Resolution: 12 Megapixels (4000 x 3000 pixels) • Shutter Speed: 1120 to 11125 seconds (based on natural

lighting) • A reference object: (a 50 paisa coin that is standard

Bangladeshi coin of a diameter of 2.2 centimeters) • Source Institution: Sir Salimullah Medical College and

Mitford Hospital, Dhaka, Bangladesh • Department: Department of Dermatology • Patients Participated: 112, details of the data collected is

provided in table I.

TABLE I: DETAILED SAMPLE INFORMATION

Disease Acne Eczema Psoriasis Tinea Scabies Vitiligo Name Corporis

Patients 19 18 18 14 26 26

Number 107 102 105 107 182 101 of

Pictures

Taken

2.2. Image Pre-processing and ROI Detection

In image preprocessing, our task is to segment the diseased skin from healthy skin and making the system prepared for the classification stage.

2.2.1 Cropping the Images

The apparent pictures that were taken had objects, clothing and humans accessories present in the images. We had to remove them manually to recreate and refine the dataset such that the images contain only - "skin" (both healthy and diseased).

2.2.2 Color Gradient Generation

We used the modified Sobel operator based on color instead of gray. Let, the gradient of a 2D function Fe(x, y) is defined as the dependent on co-ordinates x and y. Using this notation, it can be shown that the direction of maximum rate of change of c(x, y) where c is the color image as function of x and y (gx and gy are linear horizontal and vertical gradients) and is given by the angle in following equation.

( ) 1 1 2gxy B X,Y = -tan- ----=--::'---

2 gxx-gyy (1)

The value of the rate of change (i.e. the magnitude of the gradient) in the directions given by the elements of 0, is given by following equation.:

1

Fo (X,y) = H [(gxx + gyy )+ (gxx -gyy )cos2B(x,y) + 2gxy sin 2B(x, Y)ly (2)

2.2.3 Labeling the Regions of Interest (ROI) on the Image After color gradient generation, a threshold was applied

and k-means clustering was performed on the color gradient. Then morphological closing was performed on the clusters to obtain the binary mask and by applying the mask we segmented the diseased part from the healthy skin. All the consequent steps are illustrated in Fig. 2.

1676

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

(d) (e) (I) Fig. 2. (a) Original Image (b) Color Gradient of Original Image (c) Averaging Filter applied over (b) to ignore hair or scar (d) Binary Mask by thresholding and clustering on the image with (c) label (e) Labeled region of interest (t) Isolated ROI (Region of interest or Diseased Skin)

2.3. Feature extraction

The features are of two types: automated visual features (from images) and external features (from patient history forms). The extraction procedures were different for the two.

2.3.1 Automated Visual feature extraction:

The feature collections that were extracted from the images of the ROI and healthy skin are the following:

• Mean and Standard Deviations of Colors of 3 Color Channels (R, G and B) ofROI

• Mean and Standard Deviations of Colors of 3 Color Channels (R, G and B) of healthy skin

• Distribution (scattering of the ROI-s) • Area (in mm2) • Energy, Entropy, Contrast and Homogeneity from GLCM

in each color channel [21]

At this point, we have separated the healthy skin and the individual ROI-s in structures that contain the images of the RGB components. From the RGB components, we have computed the mean, variance (standard deviation) of the channels individually. Energy, entropy, contrast and homogeneity were computed from the regions by computing the gray level co-occurrence matrices (GLCM) in each channel as proposed by Shahbahrami et al. [21]. The separation of ROI and healthy skin was done using masks as shown in Fig. 3.

Thus we can see how the healthy and normal skins were separated to extract colors from them.

Fig. 3. (a) Original Image with ROI-s Labeled (b) Image with the ROI-s taken apart (c) Original image with no healthy skin present

However we have used overlapping concentric circles inside the ROI region to have a more clear idea of the mean colors present inside the Region of Interest (ROI). The circles were used as they provide a measure which is rotation invariant and they were overlapped to attain an attribute of translation invariance.

Five concentric circles were used to extract the colors features. It is illustrated in Fig. 4.

Fig. 4. Mean Color extracted using concentric windows inside the ROI (the left most arrow represents the mean color of healthy skin)

To compute energy, entropy, contrast and homogeneity we first computed GLCM for each color channel in each window. G x G GLCM P d for a displacement vector d = (dx, dy) is given by Shahbahrami et al. [21]. The (i, j) of Pdis the number of occurrences of the pair of gray-level i and j which are a distance "d" apart. The equations for obtaining the 4 features from a GLCM can be found in Shahbahrami et al. [21 ].

The distribution is how the diseased parts are spread out from each other from its manifestations. This was done from calculating the Euclidean distances from one connected component to another and getting their mean. It can be expressed as the following equation (where, every i is a ROI in a single image, dis�j is the distance from ROI i to j and m is the total number of ROI-s in an image). This feature calculates the "spread" of the ROIs in an image.

1677

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

[ m 1 L dist ..

. . . ·-1 Y ixels Dzstrzbutzon = 1- X �

m mm

(3) The area or size of a ROI is very important to classify

the disease. It was done by fIrst using propping regions over the labeled ROI-s over the masks generated to detect the regions of interest. Then we multiplied the area in pixel with the ratio of pixel to the reference object present in the images (a 50 paisa coin with 2.2 c.m. in diameter) to fInally get the area or size of the ROI in milimete?

2.3.2 External features extraction

The features that were collected from the patient history are the following: diseased body part, elevation and frequency of occurrence (in months). The diseased body parts were quantifIed and added manually to the sample descriptions. Since stereo imaging was not feasible in a government hospital to take a 3D images, the elevations of ROIs was taken from the patient history forms. A dermatologist assisted in this regard. Elevation refers to the depth or height of the diseased lesion or region of interest (ROI). Naturally as it is a length that was measured and the unit was in millimeter.

2.4. System Training and Testing

We have used feed-forward back-propagation neural network training to perform this step. We validated and tested our system using the tenfold cross validation process. The virtue of using a cross validation technique was that there were no overlapping of the test data and training data, making the system testing results viable and dependable.

3. Experimental results and discussion

The selection of an optimized threshold of color-gradient and neuron number of the feed-forward back-propagation ANN was tested with different configuration of ANN. We have selected our semi-supervised system as it gives us better response than the unsupervised system and they are discussed in next subsection.

3.1. ROC Curves for Optimal Color Gradient Threshold

The ROC curves in Fig. 5 illustrate the selection of the best threshold. Since the images were cropped to include only skin portions, thus in all cases True Negative=100%. It

should be noted that in the ROC curves: True Positives (TP) = Correctly detected Regions of Interest (diseased skin), False Positives (FP) = Incorrectly detected Regions of Interestlhealthy skin detected as ROI, True Negatives (TN) = Correctly detected Healthy Skin (since the images are cropped to contain skin only, the value of this is constant at 100% at all times for our system), False Negatives (FN) = Incorrectly detected healthy skin (healthy skin contains ROI/the measure of how much the system was unable to detect the diseased parts).

From these results (Fig. 5) we see that we fInd the best "detection of ROI accuracy" at the Color-Gradient threshold of 0.8 and the accuracies are 86.04% for unsupervised clustering (Fig. 5-top) and 92.25% for semi-supervised (user inputted number of ROI) clustering (Fig. 5-bottom). Consequently, we therefore choose the semi-supervised clustering for further validation since it exhibited an improved detection rate over the unsupervised system.

100.00 .. u s::: 80.00 ..

E 60.00 a 1; 40.00

Q. 20.00

0 .00

100.00

..

� 80.00

..

E (; 60.00

1: 8'.. 40.00

20.00

0.00

ROC Curves varying over Color Gradient Threshold

(unsupervised system) --True

Positive % --False

h,.,f--------'''"''''o;;:-----1:-.JII''''-- OO.53 Positive %

0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00

Color Gradient

--True

Negative % --False

Negative % --Accuracy%

ROC Curves varying over Color Gradient Threshold (semi­

supervised system)

0.10 0.20 0.30 0.40 0.50 0.60 0.70 0 .80 0.90 1.00

Color Gradient

-- True

Positive % --False

Positive % -- True

Negative % --False

Negative % -- Accuracy%

Fig. 5. ROC curves of ROI detection for varying Color Gradient using

unsupervised clustering (top) and Sami-supervised clustering (bottom) for Sample size: 2055.

3.2 Classification Performance

Upon creation of the training matrix of dimensions 103 x 2055 (2055 samples with 103 features each) and target matrix of dimension 6 x 2055 (2055 total samples of 6 diseases) we were set for tenfold cross validation. We

1678

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

partitioned each fold with 10% samples for testing and 90% samples for training such that upon 10 folds we get comprehensive accuracy results. We tested our system using 70 neurons growing to 150 neurons in a single hidden layer. From the plot in Fig. 6 we see that the highest classification accuracy was found using 98 neurons in the hidden layer and the accuracy was found to be 94.667%.

Neuron Numbers in Hidden Layervs. Accuracy (Sample Size: 2055)

II o�o�o � o�o�o�o�o�o �� •• ��OO""MMMM •• � """""""""""

Neurons In Hidden Lne,

Fig. 6. Neurous iu hidden layer vs. tenfold cross-validated average accuracy folds vs. classification accuracy in aercentage for 98 Neurons (number of neurons for highest classification accuracy)

3.2.1 Disease-wise classification peiformance using terifold cross-validation:

Among our 2055 samples we had 6 diseases. It is to be noted that every image was not a sample rather every ROI was a sample in our system, making it more versatile so that one image containing multiple diseases can also be classified successfully. The disease-wise accuracy calculation and data info is presented in table II using 98 neurons in the hidden layer of our Artificial Neural Network.

TABLE II: OVERALL CLASSIFICATION PERFORMANCE GENERATION

Accuracy

Disease Sample Size Accuracy X Sample Size

Acne 405 96.66 % 391.347 Eczema 320 88.00 % 281.6

Psoriasis 288 89.33 % 257.2704 T inea 280 88.33 % 247.324

Corporis

Scabies 507 98.67% 500.2569 Vitiligo 255 99.67% 254.1585

Total: 2055 Total: 1932.083 Accuracy using (equation 5) = (1931/2055) x 100% =

94.0146%

3.2.2 Disease Classification Accuracy Evaluation:

We have evaluated our system's performance based on the sample sizes as weights to signify how well it performs. Thus, based on table II, we conclude that our system performs with a classification accuracy of 94.0146% with training from 2055 samples of 6 diseases and an ANN with one hidden layer with 98 neurons.

4. Conclusions

In this paper we presented a robust and automated method for the diagnosis of dermatological diseases. In brief it was a challenging task since the human skin is one of the most difficult surface or terrain to analyze. It is unique and novel since our dataset was not acquired from secondary sources; rather it was a work of months of toil in an actual hospital of Bangladesh making the dataset a standard dataset which is completely new in perspective of Bangladesh. We should point out that it is to replace doctors because no machine can yet replace the human input on analysis and intuition. Nevertheless, we consider our system as a significant leap for machine intervention in medicine in Bangladesh. The system exhibits a diseased skin detection accuracy of 95.99% and disease identification accuracy of 94.016% while tested for a total of 2055 diseased areas in 704 skin images for 6 diseases.

Acknowledgement

This work is supported by a grant from the Citygroup Bangladesh (www.citygroup.com.bd).

References

[1] Jianbo Gao, Jun Zhang, Matthew G. Fleming, IlyaPollak, Armand B. Cognetta. Segmentation of Dermatoscopic Images by Stabilized Inverse Diffusion Equations. Computerized Medical Imaging and Graphics, 22(5):375-389 . 1998.

[2] Bushra Jalil . Multispectral Image Processing Applied to Dermatology. Thesis Submitted for the Degree of MSc. Erasmus Mundus in Vision and Robotics (VIBOT). Universite de Bourgogne .2008 .

[3] Michal Antkowiak . Artificial Neural Networks vs.

1679

Support Vector Machines for Skin Diseases Recognition . Master's Thesis in Computing Science . Umea University, Sweden . 2006.

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

[4] S. Agatonovic-Kustrin and R. Beresford.Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research. Journal of Pharmaceutical and Biomedical Analysis, 22:717-727, 2000.

[5] Michal Antkowiak. Recognition of skin diseases using artificial neural networks. In Proceeding of USCCS'05, pages 313-325, 2005.

[6] Chris M. Bishop. Neural networks and their applications. Review of Scientific Instruments, 65(6):1803-1832, 1994.

[7] Anthony du Vivier. Atlas of Clinical Dermatology. Churchill Livingstone, 2002.

[8] eCureMe. Medical Dictionary. [Online] . Available: http://www.ecureme.com .Retrieved June 16,2011.

[9] Monika Gniadecka, Peter Alshede Philipsen, Sigurdur Sigurdsson, Sonja Wessel, and et al. Melanoma diagnosis by Raman spectroscopy and neural networks: Structure alterations in proteins and lipids in intact cancer tissue. J Invest Dermatol, 122:443--449,2004.

[10] David Delgado Gomez, Toke Koldborg Jensen, SuneDarkner, and Jens Michael Carstensen. Automated visual scoring of psoriasis. [Online] . Available: http://www.imm.dtu.dkivisiondagIVD02/medicinsk/arti culo.pdf, 2002. Retrieved: June 16, 2011.

[11] Handels H. ,RoB T. , Kreusch 1. , Wolff H. H. , and P··oppl S. 1. . Feature selection for optimized skin tumor recognition using genetic algorithms. Artificial Intelligence in Medicine, 16:283-297, 1999.

[12] K. Hoffmann, T. Gambichler, A. Rick, M. Kreutz et al . . Diagnostic and neural analysis of skin cancer (DANAOS).A multicentre study for collection and computer-aided analysis of data from pigmented skin lesions using digital dermoscopy. British Journal of Dermatology, 149:801-809,2003.

[13] Samuel Freire da Silva .Dermatological Atlas [Online]. Available: www.atlasdermatologico.com.br/index.htrnl .Retrieved: July 27,2011.

[14] NIH: National Institute of Health (2011). Medline Plus -Scabies [Online]. Available: http://www.nlm.nih.gov/medlineplus/scabies.htrnl. Retrieved: July 27,2011.

[15] NIH: National Institute of Health (2011). Medline Plus -Eczema [Online]. Available: http://www.nlm.nih.gov/medlineplus/eczema.htrnl. Retrieved: July 27,2011.

[16] NIH: National Institute of Health (2011). Medline Plus -Psoriasis [Online]. Available: http://www.nlm.nih.gov/medlineplus/psoriasis.htrnl. Retrieved: July 27,2011.

[17] NIH: National Institute of Health (2011). Medline Plus -Vitiligo [Online]. Available: http://www.nlm.nih.gov/medlineplus/vitiligo.htrnl. Retrieved: July 27,2011.

[18] NIH: National Institute of Health (2011). Medline Plus­TineaCorporis [Online]. Available: http://www.nlm.nih.gov/medlineplus/tineacorporis.html. Retrieved: July 27,2011.

[19] NIH: National Institute of Health (2011). Medline Plus -Scabies [Online]. Available: http://www.nlm.nih.gov/medlineplus/scabies.htrnl. Retrieved: July 27,2011.

[20] Rafael Gonzalez, Richard Woods . Color Image Processing. Digital Image Processing with MATLAB 2nd Edition. p. 204-207. Prentice Hall Publications , USA . 2003 .

[21] Shahbahrami A. ,Borodin D. ,Juurlink B. . Comparison Between Color and Texture Features for Image Retrieval . In Proceedings of ACM Multimedia Conference, Boston, MA. pp. 361 - 371,2008.

[22] Health Policy of Bangladesh(2011). Ministry of Health and Family Welfare - Government of People's Republic of Bangladesh [Online]. Available: http://nasmis.dghs.gov.bdlmohfw/index.php?option=co m content&task=view&id=388&ltemid=483. Retrieved: Dec 11,2011.

1680