Recent Advances in Crop Classification

Managed by UT-Battellefor the Department of Energy

Recent Advances in Crop Classification

Raju Vatsavai([email protected])

Computational Sciences and Engineering Division

ORNL, Oak Ridge, TN, USA

Collaborators:

B. Bhaduri, V. Chandola, G. Jun, J. Ghosh, S. Shekhar, T. Burk

Remote Sensing – Beyond Images Workshop, Mexico City, Mexico,

14th December, 2013.

2 Managed by UT-Battellefor the Department of Energy

Outline

· Better spectral and spatial resolution– Fine-grained (species) classification– Complex (compound) object recognition

· Challenges– Limited ground-truth: Semi-supervised learning (SSL)– Spatial homogeneity: SSL + Markov Random Fields– Spatial heterogeneity: Gaussian Process (GP) learning– Aggregate vs. Subclasses: Fine-grained classification– Phenology: Multi-view learning

· Conclusions


Challenge 1: Limited Training Data

· Increasing spectral resolution: 4 to 224 Bands

· Challenges– #of training samples ~ (10 to 30) * (number of dimensions)– Costly ~ $500-$800 per plot (depends on geographic

area)– Accessibility – Private/Privacy issues (e.g., USFS may

average 5% denied access)– Real-time – Emergency situations, such as, forest fires,

floods

· Solutions– Reduce number of dimensions– (Artificially) Increase number of samples– By incorporating unlabeled samples

· Naïve semi-supervised (Nigam et al. [JML-2000])– Bagging [Breiman, ML-96]


True Distribution

Estimated Distribution(Small Samples; MLE are good asymptotically)


Initial Estimates +Unlabeled Samples


Iteratively Update Parameters Using Unlabeled Samples






Final parameters after convergence


· E-Step

· M-Step

ithdata vector, jth class

Solution: Semi-supervised Learning

Assume Samples are generated by a Gaussian Mixture Model (GMM)

• Estimate Parameters with Expectation Maximization (EM)


Results

10 Classes, 100 Training Samples(10-30) x No of dimensions / class

Small Subset of 20 Training Samples

20 labeled + 80 unlabeled samples

Supervised (BC) vs. Semi-supervised (BC-EM)

Fixed Unlabeled (85) and Varying (Increasing) Labeled

0 20 40 60 80 100 120

Acc

urac

y

30

40

50

60

70

80

BC - WorstBC - BestBC (EM) - Best

Ranga Raju Vatsavai, Shashi Shekhar, Thomas E. Burk: A Semi-Supervised Learning Method for Remote Sensing Data Mining. ICTAI 2005: 207-211


Prior Distribution Model:· For Markov random field , the conditional

distribution of a point in the field given all other points is only dependent on its neighbors.

s excluding Sin points ofset a denotes

lattice imagean is Where

)}(|)({)}(|)({

sS

S

sspsSsp

sxx

xx

sxx

xxx x

x xsxx

xxx

x x

xx

x

xx

Challenge 2: Spatial Homogeneity

Bayes Theorem: p(c|x) = p(x|c)p(c)/p(x)

For a first - order neighborhood system

p() 1

ze

t c ( )C

e.q.1

t c () is the total number of horizantally

and vertially neighboring points of different

value in in clique c.

e.q.1 is Gibbs distribution and therefore,

an MRF.

is emphirically determined weight.

t c () {0, otherwise. 1 if (i, j ) (k,l )

Spatial Homogeneity


Solution: Spatial Classification

BC (60%) BC-EM (68%)

BC-MRF (65%) BC-EM-MRF (72%)

• Shashi Shekhar, Paul R. Schrater, Ranga Raju Vatsavai, Weili Wu, Sanjay Chawla: Spatial contextual classification and prediction models for mining geospatial data. IEEE Transactions on Multimedia 4(2): 174-188 (2002)

• Baris M. Kazar, Shashi Shekhar, David J. Lilja, Ranga Raju Vatsavai, R. Kelley Pace: Comparing Exact and Approximate Spatial Auto-regression Model Solutions for Spatial Data Analysis. GIScience 2004: 140-161

• Ranga Raju Vatsavai, Shashi Shekhar, Thomas E. Burk: An efficient spatial semi-supervised learning algorithm. IJPEDS 22(6): 427-437 (2007)


Challenge 3: Spatial Heterogeneity· Going From Local to Global

– Signature continuity is a problem in classifying large geographic regions

· Solutions– Assume constant variance structure over space, that is,

train one model, use it on other regions – poor performance

– Train separate model for each region – needs lot of data– Train one model covering samples from all regions –

needs an adaptive model to capture spatial heterogeneity


Solution: Gaussian Process (GP) Classification

· Change of distribution over space is modeled by),(~)|( Nyxp

))(),((~)|)(( ssNysxp

Goo Jun, Ranga Raju Vatsavai, Joydeep Ghosh: Spatially Adaptive Classification and Active Learning of Multispectral Data with Gaussian Processes. SSTDM 2009: 597-603


Challenge 4: Aggregate Vs. Sub-classes

· Spectral Classes vs. Thematic Classes

· Insufficient Ground-truth· Subjective/domain-dependent· Parametric – assumption violations


Solution: Sub-class Classification

· Coarse-to-fine Resolution Information Extraction– Characterizing the nature of the change

· Fallow to Switch grass, Wheat to Corn, or crop damage

Coarse Classes (MODIS)Each class is Gaussian

Sub-Classes (AWiFS)Each class is MoG

Model Selection (BIC,AIC)How many components?Parameter Estimation

Semi-supervised Learning

Characterize Changes


Results: Sub-class Classification

Dataset:LandSat ETM+ Data (Cloquet, Carleton, MN, May 31, 2000)• 6 Bands, 4 Classes, 60 plots• Independent test data: 205 plots• Forest (4 Subclasses; 2 subclasses are

combined into 1)• 2 Labeled plots per sub-class

1. Ranga Raju Vatsavai, Shashi Shekhar, Budhendra L. Bhaduri: A Learning Scheme for Recognizing Sub-classes from Model Trained on Aggregate Classes. SSPR/SPR 2008: 967-976

2. Ranga Raju Vatsavai, Shashi Shekhar, Budhendra L. Bhaduri: A Semi-supervised Learning Algorithm for Recognizing Sub-classes. SSTDM 2008: 458-467


Crop (Opium) Classification

· Helmand accounts for 75% of the world’s opium production

· GeoEye 4-Band Image, 13th May 2011


Ground-truth (Aggregate Classes)

· Ground-truth collected for 4 classes

· 1-Other Crops (Yellow), 2-Poppy (Red), 3-Soils (Cyan), 4-Water (Blue)


Classified (Aggregate) Image

· Maximum Likelihood Classification (Widely used)

· Also did lot of other standard classification schemes – Decision Trees, Random Forest, Neural Nets, …


Classified (Sub-classes) Image

· Sub-class classification – Identifying finer classes from aggregate class – new scheme– 1 -> 11,12,13; 2 -> 21,22,23, 3->31,32, 4->41

· (Overall Accuracy Improved by ~10%)


Challenge 5: Phenology

AWiFS (May 3, 2008; FCC (4,3,2))

AWiFS (July 14, 2008; FCC (4,3,2))

Thematic Classes: C-Corn, S-Soy


More Formally


Solution: Multi-view Learning

· Multi-temporal images are different views of same phenomena– Learn single classifier on different views, chose

the best one through empirical evaluation– Combine different views into a single view, train

classifier on single combined view – stacked vector approach

– Learn classifier on single view and combine predictions of individual classifiers – multiple classifier systems

· Bayesian Model Averaging

– Co-training· Learn a classifier independently on each view· Use predictions of each classifier on unlabeled

data instances to augment training dataset for other classifier

Varun Chandola, Ranga Raju Vatsavai: Multi-temporal remote sensing image classification - A multi-view approach. CIDU 2010: 258-270


Conclusions· We developed several innovative solutions that

address big spatiotemporal data challenges– Semi-supervised learning– Spatial classification (homogeneity and heterogeneity)– Temporal classification– Sub-class classification

· Ongoing– Transfer learning: Adopt model learned in area to the

other with very little additional ground-truth– Compound object classification (multiple instance

learning)– Semantic classification (beyond pixels and objects)– Scaling

· Heterogeneous (OpenMP + MPI + CUDA)· Cloud computing (MapReduce)


Acknowledgements

· Prepared by Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, Tennessee 37831-6285, managed by UT-Battelle, LLC for the U. S. Department of Energy under contract no. DEAC05-00OR22725.

· Collaborators and Sponsors

Education

Recent Advances in Crop Classification