Image Classifier Sansoni Davide Trivella Emanuele Digital Image Processing A.A. 2014-2015

Image Classifier

Sansoni DavideTrivella Emanuele

Digital Image ProcessingA.A. 2014-2015

The aim of this project is to develop a software able to classify specific categories of object.

We used Python as programming language and OpenCV, SciKit-Learn as support libraries.

The classification task is reached through Machine Learning techniques.

Introduction

The program rely on a command line interface, which has two main modes of operation:

• Batch Mode

• Interactive Mode

In both cases, the execution outputs the classification result and the probability that the input image belongs to each category.

Introduction (2)

The most common algorithm to detect and compute features from images is SIFT (2004 - patented)

A Speed-up version of the first one is SURF (2006 - patented)

Similar algorithm ORB (2011 - free)

Feature detection and description

The program uses an algorithm to detect keypoints (kp) and compute their descriptors (des).

The information of each kp makes a record of our dataset.

In this case an image is described by a lot of records (one for each kp extracted from the image itself).

Dataset creation

A record is composed by 134 attributes:

• 6 are properties of the kp (coordinate, size, angle, response, octave)

• 128 are descriptor values of the kp

The whole dataset is stored in a csv file that can be read easily by any machine learning application

Dataset creation (2)

We tested a lot of classification algorithms, but the best is DecisionTreeClassifier (similar to C4.5)

Other algorithms used are:• GaussianNB (Naive Bayes)• BernoulliNB (Bernoulli Naive Bayes)• SVC (SVM)• RandomForestClassifier (Random Forest)• ExtraTreesClassifier (Extra Tree Classifier)

Classification algorithms

At first we tested the progam with only three classification categories.

In this case Naive Bayes showed a comparable accuracy with Decision Tree.

But when we decided to add new categories, the performance of NB dropped drastically.

With Random Forest we can obtain discrete results, but lower than DT.

Classification algorithms (2)

SIFT and ORB algorithms take in input some parameters:

• nfeature, the number of best features to retain• nOctaveLayers, the number of layers in each octave.• contrastThreshold, the contrast threshold used to

filter out weak features• edgeThreshold, the threshold used to filter out edge-

like features• sigma, the sigma of the Gaussian applied to the input

image at the octave n. 0

We specified only nfeature because other parameters default values are already good.

Parameters

Otherwise, SURF algorithm takes in input:

• hessianThreshold, threshold for hessian kp detector• nOctaves, number of pyramid octaves the kp detector

will use.• nOctaveLayers, number of octave layers within each

octave.• extended, extended descriptor flag (true: use 128

descriptors; false: use 64 descriptors).• upright – Up-right or rotated features flag (true: do not

compute orientation; false: compute orientation).

As before, we changed only one parameter (hessianThreshold) leaving the others to default.

Parameters (2)

Speaking about Decision Tree Classifier the most important parameters are:

criterion, measure the quality of a split splitter, strategy used to choose the split at each

node max_depth, the maximum depth of the tree min_samples_split, the minimum number of samples

required to split an internal node min_samples_leaf, the minimum number of samples

required to be at a leaf node

Our progam allows the user to enter a value for all of these parameters.

Parameters (3)

If the user wants to use Random Forest or Extra Trees Classifier, the program asks him to insert:

n_estimators, the number of trees in the forest criterion, measure the quality of a split max_depth, the maximum depth of the tree min_samples_split, the minimum number of samples

required to split an internal node min_samples_leaf, the minimum number of samples

required to be at a leaf node bootstrap, whether bootstrap samples are used when

building trees

Parameters (4)

Feature Detection Algorithm• nfeature: 0, meaning that all possible features are

detected (SIFT, ORB)• hessianThreshold: 100 (SURF)

Classification Algorithm• criterion: gini• max_depth: None• min_samples_split: 2• min_samples_leaf: 1• splitter = best (Decision Tree)• n_estimators = 10 (Random Forest, ETC)• bootstrap = True (Random Forest, ETC)

Deafult Parameters

At first we used only three categories: cars, cats and chairs. For each category we downloaded 50 training images and 10 testing images from Google, using the software provided by Damioli’s group.

Performance

Classifier/Detector SIFT SURF ORB

Decision Tree Classifier (default) 96.67% 100.00% 96.67%

Naive Bayes 93.33% 70.00% 93.33%

Bernoulli Naive Bayes 66.67% 60.00% 66.67%

Random Forest (default) 93.33% 100.00% 96.67%

Extra Tree Classifier (default) 73.33% 100.00% 76.67%

Then we added other categories to test similar object recognition: dogs, doors and tables.

Performance (2)

Classifier/Detector SIFT SURF ORB

Decision Tree Classifier (default) 75.00% 86.67% 78.33%

Naive Bayes 55.00% 33.33% 55.00%

Bernoulli Naive Bayes 51.67% 35.00% 51.67%

Random Forest (default) 65.00% 73.33% 61.67%

Extra Tree Classifier (default) 51.67% 65.00% 55.00%

Because the best results we achieved have been got by DT and RF, we decided to modify some parameters of these algorithms.

Performance (3)

Decision Tree [%] (default accuracy = 75%, 88.3%, 76.7%)

Random Forest [%] (default accuracy = 61.7%, 71.7%, 66.7%)n_est crit. max_dep. min_split min_leaf b.strap SIFT SURF ORB

10 entropy None 2 1 True 61.7 70.0 63.3


20 gini None 2 1 True 66.7 75.0 61.7


criterion splitter max_depth min_split min_leaf SIFT SURF ORB

entropy best None 2 1 80.0 90.0 78.3

gini best None 5 2 71.7 86.7 70.0



Added other 4 classes (bottles, shoes, flowers, pineapples).

Performance (7)

criterion splitter max_depth min_split min_leaf SIFT SURF ORB


gini best None 5 2 57.0 81.0 58.0



n_est crit. max_dep. min_split min_leaf b.strap SIFT SURF ORB



20 gini None 2 1 True 51.0 60.0 51.0


Decision Tree [%] (default accuracy = 61%, 87%, 60%)

Random Forest [%] (default accuracy = 57%, 59%, 48%)

To simplify we used default parameters because we didn’t observe a significant performance improvement.

Just to remember…

Decision Tree • SIFT: 58%• SURF: 87% • ORB: 60%

Performance (11)

SIFT (nfeature) SURF (hessianT.) ORB (nfeature)

100 150 200 250 50 200 300 400 100 150 200 250

DT[%] 63 69 68 63 86 86 77 74 68 70 72 71

RF[%] 63 67 62 59 62 61 58 59 72 58 63 62

Random Forest • SIFT: 57%• SURF: 59% • ORB: 48%

Given these results we tried to improve them further more, so we decided to make some preprocessing on our dataset.

• Image resizing

• Low pass filtering

• Image background removal

• Feature standardization

• Feature normalization

Preprocessing

Image resizing

The software of Damioli’s group resizes downloaded images in order to reduce spatial and computational complexity, so we decided to implement it as well.

In this way the whole dataset contains solely images with 500px of width, minding to keep the same aspect ratio.

Low pass filtering

We opted for these two low pass filters in order to remove some noise from dataset images:

Gaussian filter

Bilateral filter

Low pass filtering (2)

In both cases the performances we obtained got worse.

Gaussian• SIFT [nfeature = 150]: 70% 63%• SIFT [nfeature = 200]: 72% 68%

Bilateral• SIFT [nfeature = 200]: 72% 63%• SURF [hessianT.= 200]: 83% 76%

As consequence, we decided to avoid low pass filtering during preprocessing.

Image Background Removal

A further step we thought to introduce in preprocessing phase is background removal.

This step might be useful if the background is well separated from the subject of the image (category).

We observed that performances decrease because the algorithm that provides this functionality depends on a threshold value which is strongly related to the content of the image itself.

Feature Standardization

In general, machine learning estimators might behave badly if the individual feature do not more or less looks like standard normally distributed data (Gaussian with zero mean and unit variance).

However, in our case, we achieved worse performances.

Feature Normalization

Normalization is the process of scaling individual samples to have unit norm. This process can be useful if you plan to use a quadratic form such as the dot-product or any other kernel to quantify the similarity of any pair of samples.

As for standardization, this technique is commonly used to increase performance of classification algorithms. Despite that, we observed a performance decrease also in this case

Feature Selection

Univariate feature selection works by selecting the best features based on univariate statistical tests. It can be seen as a preprocessing step to an estimator.

We used SelectKBest(f_classif,k) to select k highest scoring features from our dataset records.

We tried different k values: 50,100,130 scoring respectively 23, 44, 65 [%] of accuracy against the default 70% with SIFT (nfeature=150).

Conclusions and Results

At the end of the day, the best results we achieved are:

SIFT:• nfeature = 200• Decision Tree• Accuracy: ~72%

SURF:• hessianThreshold= 100• Decision Tree• Accuracy: ~87%

ORB:• nfeature = 200• Decision Tree• Accuracy: ~72%

Results of SIFT

Results of SIFT (2)

Results of SIFT (3)

How SIFT performs in bottles classification:

How SIFT performs in cars classification:

Results of SURF

Results of SURF (2)

Results of SURF (3)

How SURF performs in bottles classification:

How SURF performs in cars classification:

Results of ORB

Results of ORB (2)

Results of ORB (3)

How ORB performs in bottles classification:

How ORB performs in cars classification:

Results of INTERACTIVE MODE

SIFT (nfeature = 200):

SURF (hessianThreshold = 100):

ORB (nfeature = 200):

Input files: dogs_test.jpg, cars_test.jpg

THANKS FOR YOUR ATTENTION

Documents

Image Classifier Sansoni Davide Trivella Emanuele Digital Image Processing A.A. 2014-2015