Cascaded Classifier for Automatic Crater Detection Henry Z. Lo Advisor: Wei Ding Domain Scientist:...

Cascaded Classifier for Automatic Crater Detection

Henry Z. Lo

Advisor: Wei DingDomain Scientist: Tomasz Stepinski

Knowledge Discovery LabUniversity of Massachusetts Boston

Overview

• Introduction:o Cascading classifier.o Experimental road map.

• Experiments:o Tests on feature sets.o Tests on positive example training set content.o Tests on negative example training set size.o Tests on negative example training set content. 

• Discussion:o Implications of results.o Unresolved issues.o Future directions.

Cascading Classifier

• Architecture:o Layers of Adaboost classifiers.o Each layer trained on the FP of previous layer. o Input must be accepted by all, sequentially, to be

considered a crater.o Rejection can happen at any stage.

Cascading Classifier

• Features:o Exclusively uses Haar-like features.o Can be calculated in constant time.o Contrast based.o Scanned over entire subwindow.

Cascading Classifier

• Implementation:o Used OpenCV implementation.o Free and open source. o Many variables:

Number of layers. "Minimum hit rate" - false positive rate. "Max false alarm" - false negative rate. 3 feature sets.

Experimental Road Map

• Tweak for performance:o OpenCV parameters.o Features.o Training set.

• The following OpenCV parameters improve performance:o Minimum hit rate.o Max false alarm.o Number of layers.

• Still need to tweak features and training sets for:o Training time.o Generalizability.

• L 

Experimental Road Map

• Each of these factors will be tested individually for effect on precision, recall, and F1.

  • We avoid studying interaction effects for simplicity.

 • In the future, we will investigate how to combine different

features and test sets for optimal result.

Experimental Road Map

• We use tile 3-24 for both training and testing.• This tile was chosen for its relatively smooth surface.• Future studies will test on other tiles as well.

Experimental Road Map

Feature Set Variation

Feature Set Variation

• OpenCV offers 3 different feature sets: o CORE:      1a, 1b, 2a, 2c.o BASIC:      CORE + 2b, 2d, 3ao ALL:          all features

 • Since ALL is a superset of CORE and BASIC, it should

perform best.

Feature Set Variation

• In recall, CORE and BASIC outperformed ALL.

• In precision and F1, the exact opposite was true.

Haar Features

Haar Features

• Inclusion of tilted features beneficial to performance. • More features than those given may provide further benefit.

 • It is not obvious how to create Haar features in OpenCV.

 • Postponing creation of specialized Haar features.

Ground Truth Windows

Ground Truth Windows

• Positive examples contained tightly cropped craters. • No crater rims or surrounding area.

 • Experimented with including area around craters. •  • Range: 1x crater radius - 2x crater radius, in steps of .1.

              1.0    1.2     1.4       1.6       1.8         2.0

Ground Truth Windows

• As the subwindow increased, precision and F1 increased.

 • However, recall suffered.

Negative Example Set Size

Negative Example Set Size

• All classifiers tested were trained on 300 negative examples.  • By providing the classifier with more negative examples, we

give it more information. • Performance should increase with more negative examples.

 • Tested classifiers trained on 300, 400, 500, 600, and 700

negative examples.

Negative Example Set Size

• F1 and precision increase with more negative examples.

• Recall decreases.

Negative Example Manipulation

Negative Example Manipulation

• The idea is to put some false positives back into the training set.

  • This will teach the classifier using its own mistakes.

 • However, selecting the false positives is rather difficult, as

we will see later.

Result Implications

• Window scaling has the most noticeable effect on F1, recall, and precision.

 • Next most important is the feature set used.

 • The number of negative training examples is the least

important; however, this may be due to the small range of values being tested.

Future Directions

 • Once optimal features and training sets are found, we can

manipulate OpenCV variables.  • Recall that the classifier may be improved by the following:

o More layers in the classifier. 

o Setting the minimum hit rate (recall).•  

o Setting the max false alarm rate (precision).•  • Time complexity of classifier training requires further study.

Future Directions

• Further exploration of cascaded classification algorithm: 

o Testing classifier on other tiles. • Exploration of other object detection algorithms.

 o Neural networks.

Questions?