Automatic Image-Based Plant Disease Severity Estimation

Research ArticleAutomatic Image-Based Plant Disease Severity Estimation UsingDeep Learning

GuanWang Yu Sun and JianxinWang

School of Information Science and Technology Beijing Forestry University Beijing 100083 China

Correspondence should be addressed to Yu Sun sunyvbjfueducn and Jianxin Wang wangjxbjfueducn

Received 8 March 2017 Revised 9 May 2017 Accepted 4 June 2017 Published 5 July 2017

Academic Editor Athanasios Voulodimos

Copyright copy 2017 Guan Wang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Automatic and accurate estimation of disease severity is essential for food security disease management and yield loss predictionDeep learning the latest breakthrough in computer vision is promising for fine-grained disease severity classification as themethod avoids the labor-intensive feature engineering and threshold-based segmentation Using the apple black rot images inthe PlantVillage dataset which are further annotated by botanists with four severity stages as ground truth a series of deepconvolutional neural networks are trained to diagnose the severity of the disease The performances of shallow networks trainedfrom scratch and deep models fine-tuned by transfer learning are evaluated systemically in this paper The best model is the deepVGG16model trained with transfer learning which yields an overall accuracy of 904 on the hold-out test setThe proposed deeplearning model may have great potential in disease control for modern agriculture

1 Introduction

The plant diseases are a major thread to losses of modernagricultural production Plant disease severity is an importantparameter to measure disease level and thus can be used topredict yield and recommend treatment The rapid accuratediagnosis of disease severity will help to reduce yield losses[1] Traditionally plant disease severity is scored with visualinspection of plant tissue by trained experts The expensivecost and low efficiency of human disease assessment hinderthe rapid development of modern agriculture [2] With thepopulation of digital cameras and the advances in computervision the automated disease diagnosis models are highlydemanded by precision agriculture high-throughput plantphenotype smart green house and so forth

Inspired by the deep learning breakthrough in image-based plant disease recognition this work proposes deeplearningmodels for image-based automatic diagnosis of plantdisease severity We further annotate the apple healthy andblack rot images in the public PlantVillage dataset [3] withseverity labels To explore the best network architecture andtraining mechanism we train shallow networks of differentdepth from scratch and fine-tune the pretrained state-of-the-art deep networks The modelsrsquo capabilities of correctly

predicting the disease severity stage are compared The bestmodel achieves an accuracy of 904 on the hold-out test setOur results are a first step towards the automatic plant diseaseseverity diagnosis

An overview of the rest of the paper is as follows Section 2reviews the literature in this area Section 3 presents thedeep learning proposal Section 4 describes themethodologySection 5 presents achieved results and related discussionsand finally Section 6 holds our conclusions

2 Related Work

Various studies have found that image-based assessmentapproaches produce more accurate and reproducible resultsthan those obtained by human visual assessments StewartandMcDonald [4] used an automated image analysismethodto analyze disease symptoms of infected wheat leaves causedby Zymoseptoria tritici This method enabled the quantifi-cation of pycnidia size and density along with other traitsand their correlation which provided greater accuracy andprecision compared with human visual estimates of viru-lence Barbedo [5] designed an image segmentation methodtomeasure disease severity inwhiteblack backgroundwhicheliminated the possibility of human error and reduced time

HindawiComputational Intelligence and NeuroscienceVolume 2017 Article ID 2917536 8 pageshttpsdoiorg10115520172917536

2 Computational Intelligence and Neuroscience

taken to measure disease severity Atoum et al [6] proposeda novel computer vision system Cercospora Leaf Spot (CLS)Rater to accurately rate plant images in the real field tothe United States Department of Agriculture (USDA) scaleThe CLS Rater achieved a much higher consistency thanthe rating standard deviation of human experts Many ofthese image-based assessment approaches for plant diseasesshare the same basic procedure [5ndash13] Firstly preprocessingtechniques are employed to remove the background andsegment the lesion tissue of infected plants After that dis-criminative features are extracted for further analysis At lastsupervised classification algorithms or unsupervised clusteralgorithms are used to classify features according to thespecific task Along with advances in computer science manyinteractive tools are developed The Assess [14] is the mostcommonly used and also the discipline-standard program toestimate disease severityThe Leaf Doctor app [15] developedas an interactive smartphone application can be used oncolor images to distinguish lesion areas from healthy tissuesand calculate percentage of disease severity The applicationachieved even higher accuracy than the Assess

But these aforementioned plant disease severity estima-tion approaches are semiautomatic because they dependheavily on series of image-processing technologies such asthe threshold-based segmentation of the lesion area andhand-engineered features extraction There is usually greatvariance in color both between lesions of different diseasesand between lesions from the same disease at different stagesTherefore it is very difficult to determine the appropriate seg-mentation threshold for plant disease images without humanassistance What is more the time consuming hand-craftedfeature extraction should be performed again for new styleimages To the best of our knowledge completely automaticimage-based plant disease severity estimation method usingcomputer vision has not yet been reported

The deep learning approach leads a revolution in speechrecognition [16] visual object recognition [17] object detec-tion [18 19] and many other domains such as drug discov-ery [20] genomics [21] and building reorganization [22]Deep learning is very promising for automatically gradingplant disease severity Recently there have been some worksusing deep learning method for plant species identificationand plant disease identification The recent years of thewell-known annual plant species identification campaignsPlantCLEF [23] were performance-wise dominated by deeplearning methods Choi [24] won the PlantCLEF 2015 byusing the deep learning model GoogleNet [25] to classify1000 species Mehdipour Ghazi et al [26] combined theoutputs of GoogleNet and VGGNet [27] and surpassed theoverall validation accuracy of [24] Hang et al [28] won thePlantCLEF 2016 by the enhanced VGGNet model For plantdisease identification Sladojevic et al [29] created a datasetwith more than 3000 images collected from the Internet andtrained a deep convolutional network to recognize 13 differenttypes of plant diseases out of healthy leaves Mohanty etal [30] used a public dataset PlantVillage [3] consisting of54306 images of diseased and healthy plant leaves collectedunder controlled conditions and trained a deep convolutionalneural network to identify 14 crop species and 26 diseases

In comparison with classification among different diseasesthe fine-grained disease severity classification is much morechallenging as there exist large intraclass similarity andsmall interclass variance [31] Deep learning avoids the labor-intensive feature engineering and threshold-based segmenta-tion [32] which is promising for fine-grained disease severityclassification

3 Deep Learning Proposal

31 Deep Convolutional Neural Network To explore thebest convolutional neural network architecture for the fine-grained disease severity classification problem with fewtraining data we compare two architectures namely buildinga shallow network from scratch and transfer learning by fine-tuning the top layers of a pretrained deep network

The shallow networks consist of only few convolutionallayers with few filters per layer followed by two fully con-nected layers and endwith a softmaxnormalizationWe trainshallow networks of 2 4 6 8 and 10 convolutional layersEach convolutional layer has 32 filters of size 3 times 3 a RectifiedLinear Units (ReLU) activation and all layers are followed bya 2 times 2 max-pooling layer except for the last convolutionallayer which has 64 filters The first fully connected layer has64 units with a ReLU activation and is followed by a dropoutlayer with a dropout ratio of 50 The last fully connectedlayer has 4 outputs corresponding with the 4 classes whichfeed into the softmax layer to calculate the probability output

32 Transfer Learning It is notable that the amount of imageswe can learn from is quite limited Transfer learning is a usefulapproach to build powerful classification network using fewdata by fine-tuning the parameters of a network pretrainedon a large dataset such as ImageNet [26] Although the dis-ease severity classification is targeted for finer grained imagecategory classification problem compared to the ImageNetthe lower layers only encode simple features which can begeneralized to most computer vision tasks For examplethe first layer only represents direction and color and thevisualization of activations in the first layer of VGG16 modelis shown in Figure 1 Though not trained on the plant diseasedataset themodel can be activated against the diseased spotsthe leaf and the background

For transfer learning we compare the VGGNet [27]Inception-v3 [33] and ResNet50 [17] architectures VGGNetand the original Inception architecture GoogleNet yieldedsimilar high performance in the 2014 ImageNet Large ScaleVisual Recognition Challenge (ILSVRC) and ResNet wonthe first place of the challenge in 2016 The VGGNet involves16 (VGG16) and 19 (VGG19) weight layers and shows asignificant improvement on prior configurations by using anarchitecture with very small convolution filters The originalInception architecture GoogleNet combines the network-in-network approach and the strategy of using a series of filtersof different sizes to handle multiple scales The Inception-v3is an improved Inception architecture which can be scaledup with high computational efficiency and low parametercount ResNet is built up by stacking residual building blocksEach building block is composed of several convolutional

Computational Intelligence and Neuroscience 3

(a)

(b)

Figure 1 Visualization of activations for an input image in the firstconvolutional layer of the pretrained VGG16 model (a) originalimage (b) the first convolutional layer output

layers with a skip connection It lets each stacked layer fit aresidual mapping while skip connections carry out identitymapping It is easier to optimize the residual mapping thanto optimize the original mapping The architecture solves thedegeneration problem as stacking more layers the accuracygets saturated and then degrades rapidly ResNet50 is the 50-layer version of the network

4 Material and Experiment

41 Data Material The PlantVillage is an open accessdatabase of more than 50000 images of healthy and diseasedcrops which have a spread of 38 class labels We select theimages of healthy apple leaves and images of apple leaf blackrot caused by the fungus Botryosphaeria obtusa Each imageis assessed into one class by botanists healthy stage earlystage middle stage or end stage The healthy-stage leaves arefree of spots The early-stage leaves have small circular spotswith diameters less than 5mmThe middle-stage leaves havemore than 3 spots with at least one frog-eye spot enlarging to

Table 1 The number of samples in training and test sets

Class Number of images fortraining

Number of imagesfor testing

Healthy stage 110 times 12 27 times 12Early stage 108 29Middle stage 144 36End stage 102 23

irregular or lobed shape The end-stage leaves are so heavilyinfected that will drop from the tree Each image is examinedby agricultural experts and labeled with appropriate diseaseseverity 179 images which are inconsistent among experts areabandoned Figure 2 shows some examples of every stageFinally we get 1644 images of healthy leaves 137 early-stage180 middle-stage and 125 end-stage disease images

As healthy leaves aremuchmore than the diseased leavesthere is much difference in the number of samples per classThenumber of samples per class should be balanced to reducethe bias the networkmay have towards the healthy-stage classwith more samples Our strategy of balancing is as followsfor early stage middle stage and end stage about 80 ofthe images are used as the training set and the left 20 arethe hold-out test set For healthy-stage leaves the imagesare divided into 12 clusters with 110 images in each clusteron average for training 27 images are left for testing Thefinal accuracy is estimated by averaging over 12 runs on theclusters As the PlantVillage dataset hasmultiple images of thesame leaf taken from different orientations all the images ofthe same leaf should be either in the training set or in the testset Table 1 shows the number of images used as training andtest sets for each class

42 Image Preprocessing The samples in the PlantVillagedataset are arbitrarily sized RGB images Thanks to thepowerful end-to-end learning deep learning models onlyneed 4 basic image preprocessing steps Images are processedaccording to the following stages firstly we resize all theimages to 256times 256 pixels for shallow networks 224times 224 forVGG16 VGG19 and ResNet50 and 299 times 299 for Inception-V3 We perform both the model optimization and predictionon these rescaled images Secondly all pixel values are dividedby 255 to be compatible with the networkrsquos initial valuesThirdly sample-wise normalization is performed Normal-ization can significantly improve the efficiency of end-to-end training The normalization is performed as follows foreach input 119909 we calculate the mean value 119898119909 and standarddeviation 119904119909 and then transform the input to 1199091015840 = (119909 minus119898119909)119904119909 so that the individual features more or less look likestandard normally distributed data with zero mean and unitvariance Finally several random augmentations includingrandom rotation shearing zooming and flipping are appliedto the training imagesThe augmentation prevents overfittingand makes the model generalize better

43 Neural Network Training Algorithm The basic architec-ture in the convolutional neural network begins with severalconvolutional layers and pooling layers followed by fully


(a)

(b)

(c)

(d)

Figure 2 Sample leaf images of the four stages of apple black rot (a) healthy stage (b) early stage (c) middle stage and (d) end stage

connected layers For an input119909 of the 119894th convolutional layerit computes

119909119894119888 = ReLU (119882119894 lowast 119909) (1)

where lowast represents the convolution operation and 119882119894 rep-resents the convolution kernels of the layer119882119894 = [119882

1

1198941198822119894

119882119870119894] and 119870 is the number of convolution kernels of the

layer Each kernel119882119870119894is an119872 times119872 times 119873 weight matrix with


119872 being the window size and 119873 being the number of inputchannels

ReLU represents the rectified linear function ReLU(119909) =max(0 119909) which is used as the activation function in ourmodels as deep convolutional neural networks with ReLUstrain several times faster than their equivalents with saturat-ing nonlinearities

A max-pooling layer computes the maximum value overnonoverlapping rectangular regions of the outputs of eachconvolution kernel The pooling operation enables positioninvariance over larger local regions and reduces the outputsize

Fully connected layers are added on top of the finalconvolutional layer Each fully connected layer computesReLU(119882fc119883) where 119883 is the input and 119882fc is the weightmatrix for the fully connected layer

The loss function measures the discrepancy between thepredicted result and the label of the input which is defined asthe sum of cross entropy

119864 (119882) = minus1119899

119899

sum119909119894=1

119870

sum119896=1

[119910119894119896 log119875 (119909119894 = 119896)

+ (1 minus 119910119894119896) log (1 minus 119875 (119909119894 = 119896))]

(2)

where119882 indicates the weight matrixes of convolutional andfully connected layers n indicates the number of trainingsamples i is the index of training samples and 119896 is the indexof classes 119910119894119896 = 1 if the 119894th sample belongs to the kth classelse 119910119894119896 = 0 119875(119909119894 = 119896) is the probability of input 119909119894 belongingto the kth class that the model predicts which is a function ofparameters119882 So the loss function takes119882 as its parameters

Network training aims to find the value of 119882 thatminimizes the loss function 119864 We use gradient descentalgorithm where119882 is iteratively updated as

119882119896 = 119882119896minus1 minus 120572120597119864 (119882)120597119882 (3)

where 120572 is the learning rate which is a very importantparameter that determines the step size of the learning Thevalue of learning rate should be carefully evaluated

We use early stopping as the training stop strategy to stoptraining when the network begins to overfit the data Theperformance of the network is evaluated at the end of eachepoch using the test set If the loss value of the test set stopsimproving the network will stop training

To prevent overfitting the transfer learning is conductedas follows fully connected layers are replaced with a new oneand only fine-tune the top convolutional block for VGG16and VGG19 the top two inception blocks for Inception-v3 and the top residual block for ResNet50 along withthe new fully connected layers To avoid triggering largegradient updates to destroy the pretrained weights the newfully connected network should be initialized with propervalues rather than with random values So firstly we freezeall layers except the new fully connected network The newfully connected network is trained on the output featuresof the final convolutional layer The weights learned from

TrainingTest

10

30

50

70

90

Accu

racy

()

4 6 82 10

Deph (number of convolutional layers)

Figure 3 Accuracies of shallow networks

training are initial values for fine-tuning After that the topconvolutional block for VGG16 and VGG19 the top twoinception blocks for Inception-v3 and the top residual blockfor ResNet50 are unfreezed and then trained along with thenew fully connected network with a small learning rate

The parameters for training shallow networks and fine-tuning pretrained models are presented in Table 2 Besides alearning rate schedule is employedThe initial learning rate isdropped by a factor of 10 every 50 epochs for training shallownetworks with less than 6 convolutional layers and fine-tuning deep networks And it dropped by 10 every 100 epochsfor shallow networks with 6 or more convolutional layersBecause the network goes deeper it needsmore training stepsto converge

44 Implementation The experiment is performed on anUbuntu workstation equipped with one Intel Core i5 6500CPU (16GB RAM) accelerated by one GeForce GTX TITANX GPU (12GB memory) The model implementation ispowered by the Keras deep learning framework with theTheano backend

5 Result and Discussion

Figure 3 shows the training and testing accuracies of shallownetworks trained from scratch Each bar represents theaverage result of 12 runs Both training and test accuraciesimprove slightly with the depth of the model at first The bestperformance that is a test accuracy of 793 is achieved bythe network with 8 convolutional layers But the accuraciesfall when the networkrsquos depth exceeds 8 as there are insuffi-cient training data for models with too many parameters Tocircumvent this problem transfer learning is applied to thestate-of-the-art deep models

The results of fine-tuning the ImageNet pretrained mod-els are reported in Figure 4 Each bar represents the averageresult of 12 runs The overall accuracy on the test set we


Table 2 The hyperparameters of training

Parameters Learning from scratch Transfer learningTraining fully connected layers Fine-tuning

Training algorithm SGD RMSP SGDLearning rate 001 001 00001Batch size 32Early stopping 10 epochs

Table 3 Confusion matrix for the prediction of VGG16 model trained with transfer learning

Predicted

Ground truth

Healthy stage Early stage Middle stage End stageHealthy stage 27 0 0 0Early stage 0 27 2 0Middle stage 0 5 30 1End stage 0 0 3 20

0

20

40

60

80

100

Accu

racy

()

VGG16 VGG19 Inception-V3 ResNet50Pretrained models

TrainingTesting

Figure 4 Accuracies of the state-of-the-art extreme deep modelstrained with transfer learning

obtained varies from 800 to 904 The performance offine-tuned models is superior to that of models trained fromscratch The best result is achieved by the VGG16 modelwith an accuracy of 904 The results indicate that transferlearning alleviates the problem of insufficient training data

For comparison an ANN model is trained by SGDoptimizer end-to-end on the training set A test accuracy of31 is achieved which is basically random guessingWithoutthe convolutional feature extractor the ANN cannot extractlocal correlations and learn discriminative features from theimages

The confusion matrix of the VGG16 model on the hold-out test set is shown in Table 3 The fraction of accuratelypredicted images for each of the four stages is displayed indetail All of the healthy-stage leaves are correctly classified

The accuracies of early stage and end stage are 931 and870 respectively Middle stage is prone to be misclassifiedwith an accuracy of 833 However the misclassified stagesare only confused with their adjacent stages For example theearly stage is only confused with the middle stage and noneof early-stage is classified as end stage

From the results displayed in Figure 4 it is notable thatthe training accuracies of deep networks are close to 100andtrigger the early stopping Since deep learning is data-driventraining on more data will further increase the test accuracyIt is also important to note that the best performance isachieved by the VGGNet The result is consistent with thatof [26 28] where the VGGNet showed better performancein the PlantCLEF plant identification task Though ResNetachieved state-of-the-art result on the ImageNet dataset itperforms poorer than VGGNet on fine-grained classificationtasks The SGD optimizer might put the residual mapping inbuilding blocks of ResNet to zero too early which leads to alocal optimization and results in the poor generalization infine-grained classification

6 Conclusion

This work proposes a deep learning approach to automat-ically discover the discriminative features for fine-grainedclassification which enables the end-to-end pipeline fordiagnosing plant disease severity Based on few trainingsamples we trained small convolutional neural networksof different depth from scratch and fine-tuned four state-of-the-art deep models VGG16 VGG19 Inception-v3 andResNet50 Comparison of these networks reveals that fine-tuning on pretrained deep models can significantly improvethe performance on few data The fine-tuned VGG16 modelperforms best achieving an accuracy of 904 on the test setdemonstrating that deep learning is the new promising tech-nology for fully automatic plant disease severity classification

In future work more data at different stages of differentdiseases will be collected with versatile sensors like infrared


camera and multispectral camera The deep learning modelcan be associated with treatment recommendation yieldprediction and so on

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Authorsrsquo Contributions

Yu Sun and Guan Wang contributed equally to this work

Acknowledgments

This work was supported by the Fundamental ResearchFunds for the Central Universities 2017JC02 and TD2014-01

References

[1] C H Bock G H Poole P E Parker and T R Gottwald ldquoPlantdisease severity estimated visually by digital photography andimage analysis and by hyperspectral imagingrdquo Critical Reviewsin Plant Sciences vol 29 no 2 pp 59ndash107 2010

[2] A MMutka and R S Bart ldquoImage-based phenotyping of plantdisease symptomsrdquo Frontiers in Plant Science vol 5 article no734 2015

[3] ldquoPlant Polyphenols Prenatal Development and Health Out-comesrdquo Biological Systems Open Access vol 03 no 01 2014

[4] E L Stewart and B A McDonald ldquoMeasuring quantitativevirulence in the wheat pathogen zymoseptoria tritici usinghigh-throughput automated image analysisrdquo Phytopathologyvol 104 no 9 pp 985ndash992 2014

[5] J G A Barbedo ldquoAn automatic method to detect and measureleaf disease symptoms using digital image processingrdquo PlantDisease vol 98 no 12 pp 1709ndash1716 2014

[6] Y Atoum M J Afridi X Liu J M McGrath and L EHanson ldquoOn developing and enhancing plant-level diseaserating systems in real fieldsrdquo Pattern Recognition vol 53 pp287ndash299 2016

[7] F Qin D Liu B Sun L Ruan Z Ma and HWang ldquoIdentifica-tion of alfalfa leaf diseases using image recognition technologyrdquoPLoS ONE vol 11 no 12 Article ID e0168274 2016

[8] H AlHiary S Bani AhmadM ReyalatM Braik and Z ALRa-hamneh ldquoFast and Accurate Detection and Classification ofPlant Diseasesrdquo International Journal of Computer Applicationsvol 17 no 1 pp 31ndash38 2011

[9] E Omrani B Khoshnevisan S Shamshirband H SaboohiN B Anuar and M H N M Nasir ldquoPotential of radialbasis function-based support vector regression for apple diseasedetectionrdquo Measurement Journal of the International Measure-ment Confederation vol 55 pp 512ndash519 2014

[10] D L Hernandez-Rabadan F Ramos-Quintana and J GuerreroJuk ldquoIntegrating SOMs and a Bayesian Classifier for Segment-ing Diseased Plants in Uncontrolled Environmentsrdquo ScientificWorld Journal vol 2014 Article ID 214674 2014

[11] F E Correa-Tome ldquoComparison of perceptual color spaces fornatural image segmentation tasksrdquo Optical Engineering vol 50no 11 p 117203 2011

[12] M Schikora A Schikora K H Kogel and D Cremers ldquoProba-bilistic classification of disease symptoms caused by Salmonella

on Arabidopsis plants presented at the GI Jahrestagungrdquo Prob-abilistic classification of disease symptoms caused by Salmonellaon Arabidopsis plants presented at the GI Jahrestagung 2010

[13] J G A Barbedo ldquoAnew automaticmethod for disease symptomsegmentation in digital photographs of plant leavesrdquo EuropeanJournal of Plant Pathology vol 147 no 2 pp 349ndash364 2016

[14] L Lamari Assess Image Analysis software helpdesk Version 2vol 1 APS Press 2008

[15] S J Pethybridge and S C Nelson ldquoLeaf doctor A new portableapplication for quantifying plant disease severityrdquoPlantDiseasevol 99 no 10 pp 1310ndash1316 2015

[16] G Hinton L Deng D Yu et al ldquoDeep neural networks foracoustic modeling in speech recognition the shared views offour research groupsrdquo IEEE Signal Processing Magazine vol 29no 6 pp 82ndash97 2012

[17] K He X Zhang S Ren and J Sun ldquoDeep residual learningfor image recognitionrdquo in Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition (CVPR rsquo16) pp 770ndash778 Las Vegas Nev USA June 2016

[18] S Ren K He R Girshick and J Sun ldquoFaster R-CNN TowardsReal-Time Object Detection with Region Proposal NetworksrdquoIEEE Transactions on Pattern Analysis andMachine Intelligencevol 39 no 6 pp 1137ndash1149 2017

[19] N Doulamis and A Voulodimos ldquoFAST-MDL Fast AdaptiveSupervised Training of multi-layered deep learning models forconsistent object tracking and classificationrdquo in Proceedings ofthe 2016 IEEE International Conference on Imaging Systems andTechniques IST 2016 pp 318ndash323 October 2016

[20] E Gawehn J A Hiss and G Schneider ldquoDeep Learning inDrug DiscoveryrdquoMolecular Informatics vol 35 no 1 pp 3ndash142016

[21] B Alipanahi A Delong M T Weirauch and B J Frey ldquoPre-dicting the sequence specificities of DNA- and RNA-bindingproteins by deep learningrdquo Nature Biotechnology vol 33 no 8pp 831ndash838 2015

[22] K Makantasis N Doulamis and A Voulodimos ldquoRecognizingBuildings through Deep Learning A Case Study on Half-timbered Framed Buildings in Calw Cityrdquo in Proceedings ofthe Special Session on Computer Vision Imaging and ComputerGraphics for Cultural Applications pp 444ndash450 Porto PortugalFeburary 2017

[23] H Goeau P Bonnet and A Joly ldquoLifeCLEF plant identificationtask 2015rdquo in Proceedings of the Conference and Labs of theEvaluation Forum (CLEF rsquo15) 2015

[24] S Choi ldquoPlant identification with deep convolutional neuralnetwork SNUMedinfo at LifeCLEF plant identification task2015rdquo in Proceedings of the 16th Conference and Labs of theEvaluation Forum CLEF 2015 September 2015

[25] C Szegedy W Liu Y Jia et al ldquoGoing deeper with convolu-tionsrdquo inProceedings of the IEEEConference onComputer Visionand Pattern Recognition (CVPR rsquo15) pp 1ndash9 BostonMass USAJune 2015

[26] M Mehdipour Ghazi B Yanikoglu and E Aptoula ldquoPlantidentification using deep neural networks via optimization oftransfer learning parametersrdquo Neurocomputing vol 235 pp228ndash235 2017

[27] K Simonyan and A Zisserman Very Deep Convolutional Net-works for Large-Scale Image Recognition httpsarxivorgabs14091556

[28] S THang andMAono ldquoOpenworld plant image identificationbased on convolutional neural networkrdquo in Proceedings of the


2016 Asia-Pacific Signal and Information Processing AssociationAnnual Summit and Conference (APSIPA) pp 1ndash4 Jeju SouthKorea December 2016

[29] S Sladojevic M Arsenovic A Anderla D Culibrk and DStefanovic ldquoDeep Neural Networks Based Recognition of PlantDiseases by Leaf Image Classificationrdquo Computational Intelli-gence and Neuroscience vol 2016 Article ID 3289801 2016

[30] S P Mohanty D P Hughes and M Salathe ldquoUsing deeplearning for image-based plant disease detectionrdquo Frontiers inPlant Science vol 7 article 1419 2016

[31] S Xie T Yang X Wang and Y Lin ldquoHyper-class augmentedand regularized deep learning for fine-grained image classifica-tionrdquo in Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition CVPR 2015 pp 2645ndash2654 June 2015

[32] Y LeCun Y Bengio and G Hinton ldquoDeep learningrdquo Naturevol 521 no 7553 pp 436ndash444 2015

[33] C Szegedy V Vanhoucke S Ioffe J Shlens and Z WojnaldquoRethinking the inception architecture for computer visionrdquo inProceedings of the 2016 IEEEConference onComputer Vision andPattern Recognition CVPR 2016 pp 2818ndash2826 July 2016

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Distributed Sensor Networks


Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014


ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014


Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of



Computational Intelligence and Neuroscience

Industrial EngineeringJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in



taken to measure disease severity Atoum et al [6] proposeda novel computer vision system Cercospora Leaf Spot (CLS)Rater to accurately rate plant images in the real field tothe United States Department of Agriculture (USDA) scaleThe CLS Rater achieved a much higher consistency thanthe rating standard deviation of human experts Many ofthese image-based assessment approaches for plant diseasesshare the same basic procedure [5ndash13] Firstly preprocessingtechniques are employed to remove the background andsegment the lesion tissue of infected plants After that dis-criminative features are extracted for further analysis At lastsupervised classification algorithms or unsupervised clusteralgorithms are used to classify features according to thespecific task Along with advances in computer science manyinteractive tools are developed The Assess [14] is the mostcommonly used and also the discipline-standard program toestimate disease severityThe Leaf Doctor app [15] developedas an interactive smartphone application can be used oncolor images to distinguish lesion areas from healthy tissuesand calculate percentage of disease severity The applicationachieved even higher accuracy than the Assess

But these aforementioned plant disease severity estima-tion approaches are semiautomatic because they dependheavily on series of image-processing technologies such asthe threshold-based segmentation of the lesion area andhand-engineered features extraction There is usually greatvariance in color both between lesions of different diseasesand between lesions from the same disease at different stagesTherefore it is very difficult to determine the appropriate seg-mentation threshold for plant disease images without humanassistance What is more the time consuming hand-craftedfeature extraction should be performed again for new styleimages To the best of our knowledge completely automaticimage-based plant disease severity estimation method usingcomputer vision has not yet been reported

The deep learning approach leads a revolution in speechrecognition [16] visual object recognition [17] object detec-tion [18 19] and many other domains such as drug discov-ery [20] genomics [21] and building reorganization [22]Deep learning is very promising for automatically gradingplant disease severity Recently there have been some worksusing deep learning method for plant species identificationand plant disease identification The recent years of thewell-known annual plant species identification campaignsPlantCLEF [23] were performance-wise dominated by deeplearning methods Choi [24] won the PlantCLEF 2015 byusing the deep learning model GoogleNet [25] to classify1000 species Mehdipour Ghazi et al [26] combined theoutputs of GoogleNet and VGGNet [27] and surpassed theoverall validation accuracy of [24] Hang et al [28] won thePlantCLEF 2016 by the enhanced VGGNet model For plantdisease identification Sladojevic et al [29] created a datasetwith more than 3000 images collected from the Internet andtrained a deep convolutional network to recognize 13 differenttypes of plant diseases out of healthy leaves Mohanty etal [30] used a public dataset PlantVillage [3] consisting of54306 images of diseased and healthy plant leaves collectedunder controlled conditions and trained a deep convolutionalneural network to identify 14 crop species and 26 diseases

In comparison with classification among different diseasesthe fine-grained disease severity classification is much morechallenging as there exist large intraclass similarity andsmall interclass variance [31] Deep learning avoids the labor-intensive feature engineering and threshold-based segmenta-tion [32] which is promising for fine-grained disease severityclassification

3 Deep Learning Proposal

31 Deep Convolutional Neural Network To explore thebest convolutional neural network architecture for the fine-grained disease severity classification problem with fewtraining data we compare two architectures namely buildinga shallow network from scratch and transfer learning by fine-tuning the top layers of a pretrained deep network

The shallow networks consist of only few convolutionallayers with few filters per layer followed by two fully con-nected layers and endwith a softmaxnormalizationWe trainshallow networks of 2 4 6 8 and 10 convolutional layersEach convolutional layer has 32 filters of size 3 times 3 a RectifiedLinear Units (ReLU) activation and all layers are followed bya 2 times 2 max-pooling layer except for the last convolutionallayer which has 64 filters The first fully connected layer has64 units with a ReLU activation and is followed by a dropoutlayer with a dropout ratio of 50 The last fully connectedlayer has 4 outputs corresponding with the 4 classes whichfeed into the softmax layer to calculate the probability output

32 Transfer Learning It is notable that the amount of imageswe can learn from is quite limited Transfer learning is a usefulapproach to build powerful classification network using fewdata by fine-tuning the parameters of a network pretrainedon a large dataset such as ImageNet [26] Although the dis-ease severity classification is targeted for finer grained imagecategory classification problem compared to the ImageNetthe lower layers only encode simple features which can begeneralized to most computer vision tasks For examplethe first layer only represents direction and color and thevisualization of activations in the first layer of VGG16 modelis shown in Figure 1 Though not trained on the plant diseasedataset themodel can be activated against the diseased spotsthe leaf and the background

For transfer learning we compare the VGGNet [27]Inception-v3 [33] and ResNet50 [17] architectures VGGNetand the original Inception architecture GoogleNet yieldedsimilar high performance in the 2014 ImageNet Large ScaleVisual Recognition Challenge (ILSVRC) and ResNet wonthe first place of the challenge in 2016 The VGGNet involves16 (VGG16) and 19 (VGG19) weight layers and shows asignificant improvement on prior configurations by using anarchitecture with very small convolution filters The originalInception architecture GoogleNet combines the network-in-network approach and the strategy of using a series of filtersof different sizes to handle multiple scales The Inception-v3is an improved Inception architecture which can be scaledup with high computational efficiency and low parametercount ResNet is built up by stacking residual building blocksEach building block is composed of several convolutional


(a)

(b)














(a)

(b)

(c)

(d)



119909119894119888 = ReLU (119882119894 lowast 119909) (1)


1

1198941198822119894









119864 (119882) = minus1119899

119899

sum119909119894=1

119870

sum119896=1

[119910119894119896 log119875 (119909119894 = 119896)

+ (1 minus 119910119894119896) log (1 minus 119875 (119909119894 = 119896))]

(2)



119882119896 = 119882119896minus1 minus 120572120597119864 (119882)120597119882 (3)




TrainingTest

10

30

50

70

90

Accu

racy

()

4 6 82 10














Predicted

Ground truth


0

20

40

60

80

100

Accu

racy

()


TrainingTesting







6 Conclusion









Acknowledgments


References












































Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in




(a)

(b)














(a)

(b)

(c)

(d)



119909119894119888 = ReLU (119882119894 lowast 119909) (1)


1

1198941198822119894









119864 (119882) = minus1119899

119899

sum119909119894=1

119870

sum119896=1

[119910119894119896 log119875 (119909119894 = 119896)

+ (1 minus 119910119894119896) log (1 minus 119875 (119909119894 = 119896))]

(2)



119882119896 = 119882119896minus1 minus 120572120597119864 (119882)120597119882 (3)




TrainingTest

10

30

50

70

90

Accu

racy

()

4 6 82 10














Predicted

Ground truth


0

20

40

60

80

100

Accu

racy

()


TrainingTesting







6 Conclusion









Acknowledgments


References












































Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in




(a)

(b)

(c)

(d)



119909119894119888 = ReLU (119882119894 lowast 119909) (1)


1

1198941198822119894









119864 (119882) = minus1119899

119899

sum119909119894=1

119870

sum119896=1

[119910119894119896 log119875 (119909119894 = 119896)

+ (1 minus 119910119894119896) log (1 minus 119875 (119909119894 = 119896))]

(2)



119882119896 = 119882119896minus1 minus 120572120597119864 (119882)120597119882 (3)




TrainingTest

10

30

50

70

90

Accu

racy

()

4 6 82 10














Predicted

Ground truth


0

20

40

60

80

100

Accu

racy

()


TrainingTesting







6 Conclusion









Acknowledgments


References












































Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in









119864 (119882) = minus1119899

119899

sum119909119894=1

119870

sum119896=1

[119910119894119896 log119875 (119909119894 = 119896)

+ (1 minus 119910119894119896) log (1 minus 119875 (119909119894 = 119896))]

(2)



119882119896 = 119882119896minus1 minus 120572120597119864 (119882)120597119882 (3)




TrainingTest

10

30

50

70

90

Accu

racy

()

4 6 82 10














Predicted

Ground truth


0

20

40

60

80

100

Accu

racy

()


TrainingTesting







6 Conclusion









Acknowledgments


References












































Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in








Predicted

Ground truth


0

20

40

60

80

100

Accu

racy

()


TrainingTesting







6 Conclusion









Acknowledgments


References












































Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in









Acknowledgments


References












































Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in

















Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in










Advances in

FuzzySystems


Volume 2014












Journal of



Advances in

Multimedia


Biomedical Imaging


Advances in


RoboticsJournal of










Advances in



Documents

Automatic Image-Based Plant Disease Severity Estimation