Leaf Classification from Boundary Analysisrvbalan/TEACHING/AMSC663Fall2007/...1 Leaf Classification from Boundary Analysis Anne Jorstad AMSC 663 Midterm Progress Report Fall 2007 Advisor:

11

Leaf Classification from Leaf Classification from Boundary AnalysisBoundary Analysis

Anne JorstadAnne JorstadAMSC 663AMSC 663

Midterm Progress ReportMidterm Progress ReportFall 2007Fall 2007

Advisor: Dr. David Jacobs, Computer ScienceAdvisor: Dr. David Jacobs, Computer Science

22

OutlineOutline

Background, Problem StatementBackground, Problem StatementAlgorithmAlgorithmValidationValidationTest ResultsTest ResultsSchedule, Future WorkSchedule, Future Work

33

BackgroundBackground

Electronic Field Guide for PlantsElectronic Field Guide for Plants

44


Current System:Current System:

–– Inputs photo of leaf on plain backgroundInputs photo of leaf on plain background–– Segments leaf from backgroundSegments leaf from background–– Compares leaf to all leaves in database, Compares leaf to all leaves in database,

using global shape informationusing global shape information–– Returns images of closest matches to Returns images of closest matches to

the userthe user

55


Current System: Current System: –– InnerInner--Distance MeasureDistance Measure

Measures the shortest distance between two points Measures the shortest distance between two points on a path contained entirely within a figureon a path contained entirely within a figure

Good for detecting similarities between deformable Good for detecting similarities between deformable structuresstructures

66

Problem StatementProblem Statement

Current System: Current System: –– All shape information is compared at a All shape information is compared at a

global level, no specific consideration of global level, no specific consideration of edge typesedge types

My Project: My Project: –– Incorporate local boundary information Incorporate local boundary information

to complement existing systemto complement existing system

77

The AlgorithmThe Algorithm

Input:Input:

Capture Capture boundary curve:boundary curve:

88


Each leaf: Each leaf:

–– Vector of ~2000 2Vector of ~2000 2--D pointsD points

–– Treat as 1Treat as 1--D complex pointsD complex points

99

The Algorithm: WaveletsThe Algorithm: Wavelets

Discrete wavelet transformDiscrete wavelet transform–– In: vector of pointsIn: vector of points–– Out: two vectors, eachOut: two vectors, each

half original lengthhalf original length

“approximation coefficients”

= low pass

“detail coefficients”

= high pass

1010


Apply repeatedly Apply repeatedly hierarchy of scaleshierarchy of scales

Original

Scale 1 Scale 3 Scale 5

1111


Goal: classify by local informationGoal: classify by local information

Use only detail coefficientsUse only detail coefficients

Want: nWant: n--D vector for each boundary D vector for each boundary point (where n = # scales)point (where n = # scales)

More work to get all points at all scalesMore work to get all points at all scales

1212

The Algorithm: WaveletsThe Algorithm: WaveletsEach coarser scale has Each coarser scale has ½½as many detail coefficients as many detail coefficients as the previousas the previous

Scale n has (Scale n has (½½))nn as many as many data points as original data points as original boundaryboundary

Must calculate (subMust calculate (sub--sampling) wavelet sampling) wavelet decomposition with boundary vector decomposition with boundary vector starting at each of the first 2starting at each of the first 2nn points for points for complete informationcomplete information

1313


Now forget about leaves:Now forget about leaves:–– Data is ~2000 nData is ~2000 n--D pointsD points

Combine data for all leaves:Combine data for all leaves:–– #leaves x ~2000 n#leaves x ~2000 n--D pointsD points

Classify!Classify!

1414

The Algorithm: ClusteringThe Algorithm: ClusteringGoal: get unique distribution of Goal: get unique distribution of coefficients for each leaf speciescoefficients for each leaf species

Direct method: Direct method: Histograms of each scale separatelyHistograms of each scale separately

–– Sort Sort coeffscoeffs into ascending, equally spaced into ascending, equally spaced intervalsintervals

–– Count number of Count number of coeffscoeffs in each interval, in each interval, divide by # divide by # coeffscoeffs

distributiondistribution

1515

The Algorithm: ClusteringThe Algorithm: Clustering

Considering each scale separately did Considering each scale separately did NOT provide enough information to NOT provide enough information to distinguish between leavesdistinguish between leaves

Use full nUse full n--D vector!D vector!

1616


KK--Means Clustering:Means Clustering:–– Given lots of data points, choose k of Given lots of data points, choose k of

them at randomthem at random–– Assign every point to its nearest Assign every point to its nearest

chosen point chosen point k clustersk clusters–– Find the mean value of each cluster, Find the mean value of each cluster,

these are the k new pointsthese are the k new points–– IterateIterate–– End with k End with k ““cluster centerscluster centers””

1717


KK--Means ClusteringMeans Clustering

Example of final clustering in 2-D

1818


Cluster 8000*2000 nCluster 8000*2000 n--D data points D data points into 36 clustersinto 36 clusters

For individual leaf, can find For individual leaf, can find distribution of its 2000 points over distribution of its 2000 points over the 36 clustersthe 36 clusters

Distribution represents leafDistribution represents leaf

1919


Classification:Classification:

–– Get distribution of new leaf over the 36 Get distribution of new leaf over the 36 cluster centerscluster centers

–– Compare this distribution to that of all Compare this distribution to that of all leaves in the system (chileaves in the system (chi--squared test)squared test)

–– Smallest difference in distribution is Smallest difference in distribution is closest match = the best guess at leaf closest match = the best guess at leaf species!species!

–– In practice, return 10 closest matchesIn practice, return 10 closest matches

2020

Some DetailsSome Details

How to choose the wavelet basis?How to choose the wavelet basis?

–– For our purposes, does not make much For our purposes, does not make much differencedifference

DaubechiesDaubechies 22

2121


How to chose n, the number of How to chose n, the number of scales?scales?

–– Trial cases: n = 5Trial cases: n = 5

–– Update empiricallyUpdate empirically

2222


Requirement: Rotation InvarianceRequirement: Rotation Invariance

Before clustering, for each point:Before clustering, for each point:Rotate coarsest scale to lie strictly on the Rotate coarsest scale to lie strictly on the xx--axisaxisRotate each entry of point vector by this Rotate each entry of point vector by this same anglesame angleEffectively reduces the degrees of freedom by 1Effectively reduces the degrees of freedom by 1

2323

ValidationValidation

Simple test cases:Simple test cases:

circle circle + sine curve circle with sharp points

2424


Histograms:Histograms:

3 runs 3 runs 3 sets of 3 sets of cluster cluster centerscenters

2525


ChiChi--Squared Distances between Squared Distances between distributions over several runsdistributions over several runs

2626

Test ResultsTest Results

Leaf 1

Leaf 2

Leaf 3

2727


Leaf 1

60%

Leaf 2

60%

Leaf 3

60%

2828


Cases the current system classifies poorlyCases the current system classifies poorly

Leaf 1 Leaf 2

2929


leaf 1 leaf 2leaf 1 leaf 2#scales = 1: 33% 50%#scales = 1: 33% 50%#scales = 2: 83% 67%#scales = 2: 83% 67%#scales = 3: 100% 100%#scales = 3: 100% 100%#scales = 4: 67% 87%#scales = 4: 67% 87%#scales = 5: 50% 67%#scales = 5: 50% 67%

3030

Future WorkFuture Work

Wavelets: (January)Wavelets: (January)–– Test on larger data setTest on larger data set–– Experiment with variablesExperiment with variables–– Combine with current systemCombine with current system

Combine wavelet distance with InnerCombine wavelet distance with Inner--DistanceDistanceTest new classificationTest new classification

3131

Future WorkFuture Work

Beyond wavelets: (FebruaryBeyond wavelets: (February--March)March)–– Extend InnerExtend Inner--Distance algorithm over a Distance algorithm over a

hierarchy of scaleshierarchy of scales–– Compare/combine methodsCompare/combine methods

Final report (AprilFinal report (April--May)May)

3232

ReferencesReferencesGauravGaurav AgarwalAgarwal, , HaibinHaibin Ling, David Jacobs, Ling, David Jacobs, SameerSameer ShirdhonkarShirdhonkar, W. John Kress, Rusty Russell, , W. John Kress, Rusty Russell, Peter Peter BelhumeurBelhumeur, , NandanNandan DixitDixit, Steve , Steve FeinerFeiner, , DhruvDhruv MahajanMahajan, , KalyanKalyan SunkavalliSunkavalli, , RaviRaviRamamoorthiRamamoorthi, Sean White. , Sean White. ““First Steps Toward an Electronic Field Guide for PlantsFirst Steps Toward an Electronic Field Guide for Plants””. . TaxonTaxon, vol. , vol. 55, no. 3, Aug. 2006. 55, no. 3, Aug. 2006.

CeneCene C.C.--H. H. ChuangChuang, C., C.--C. Jay C. Jay KuoKuo. . ““Wavelet Descriptor of Planar Curves: Theory and Wavelet Descriptor of Planar Curves: Theory and ApplicationsApplications””. IEEE Transactions of Image Processing, Vol. 5, No. 1, January. IEEE Transactions of Image Processing, Vol. 5, No. 1, January 1996.1996.

Ingrid Ingrid DaubeshiesDaubeshies. . ““Ten Lectures on WaveletsTen Lectures on Wavelets””. Society for Industrial and Applied . Society for Industrial and Applied Mathematicians, Philadelphia, PA, 1992.Mathematicians, Philadelphia, PA, 1992.

Pedro F. Pedro F. FelzenszwalbFelzenszwalb, , JushuaJushua D. Schwartz. D. Schwartz. ““Hierarchical Matching of Deformable ShapesHierarchical Matching of Deformable Shapes””. IEEE . IEEE Conference on Computer Vision and Pattern Recognition, 2007.Conference on Computer Vision and Pattern Recognition, 2007.

JitendraJitendra MalikMalik, Serge , Serge BelongieBelongie, Thomas Leung, , Thomas Leung, JainboJainbo Shi. Shi. ““Contour and Texture Analysis for Contour and Texture Analysis for Image SegmentationImage Segmentation””. International Journal of Computer Vision, vol. 34, no. 1, Jul. International Journal of Computer Vision, vol. 34, no. 1, July 2001.y 2001.

StephaneStephane MallatMallat. . ““A Wavelet Tour of Signal ProcessingA Wavelet Tour of Signal Processing””. Academic Press, Chestnut Hill, . Academic Press, Chestnut Hill, Massachusetts, 1999.Massachusetts, 1999.

Documents

Leaf Classification from Boundary Analysisrvbalan/TEACHING/AMSC663Fall2007/...1 Leaf Classification from Boundary Analysis Anne Jorstad AMSC 663 Midterm Progress Report Fall 2007 Advisor: