Embed Size (px)
Citation preview
Where is the bear?
Andy Rosales EliasMentor: Nevena Golubovic
Advisors: Chandra Krintz, Rich WolskiLab: RACELab
Department of Computer Science
Implementing Machine Learning-Based Image Recognition for Animal Detection
Carters New Agency http://www.catersnews.com/
Carters New Agency http://www.catersnews.com/
The Problem of Accuracy, Cost, And Efficiency
= + +
Data Reliability
The Problem of Accuracy, Cost, And Efficiency
= + +
Animal Recognition Using Different Machine Learning Frameworks
Output: “Bear” Output: “None”
Using Convolutional Neural Networks (CNN) to Recognize Animals
-Zhong, Zhuoyao et al. “High Performance Offline Handwritten Chinese Character Recognition Using GoogLeNet and Directional Feature Maps.” -"Multi-column deep neural networks for image classification". 2012 IEEE Conference on Computer Vision and Pattern -Recognition-Ciresan, Dan; Ueli Meier; Jonathan Masci; Luca M. Gambardella; Jurgen Schmidhuber. "Flexible, High Performance Convolutional Neural Networks for Image Classification"
- Almost every highly ranked team used CNN as their basic framework
- Foundation of GoogLeNet and AlexNet
Dog
Cat
How Convolutional Neural Networks Classify Images
Feature extractor black box
Deactivated neuron
Activated neuron
}
}
Input layer
Characteristic layers
Output layer
How Convolutional Neural Networks Classify Images
Feature extractor black box
Dog
Cat
Deactivated neuron
Activated neuron
}
}
Input layer
Characteristic layers
Output layer
How Convolutional Neural Networks Classify Images
Feature extractor black box
Dog
Cat
Deactivated neuron
Activated neuron
}
}
Input layer
Characteristic layers
Output layer
How Convolutional Neural Networks Classify Images
Feature extractor black box
Dog
Cat
Deactivated neuron
Activated neuron
}
}
Input layer
Characteristic layers
Output layer
Re-training an Existing CNN Model
Deactivated neuron
Activated neuron
● Use a pre-trained CNN model
Tree
Deer
Pond
Characteristic layers(pre-trained model)
Re-training an existing CNN Model
Deactivated neuron
Activated neuron
“deer”
Extra filtering layer
● Use a pre-trained CNN model● Add an extra filtering layer
Characteristic layers(pre-trained model)
Results After Classifying 12210 Images Using Pre-trained CNN Model
Gazelle 9%
Mountain bike - 10%
TensorFlow - Top 5 Categories
Barrow17%
MountainTent38%
Other - 26%
Results After Classifying 12210 Images Using TensorFlow Re-trained In Four Categories
Empty/other - 76%
Deer - 19%Coyote 3%
Bear0.5%
Re-training Dataset:● Bear: 170 images● Deer: 489 images● Coyote: 259 images● Empty/other: 21 images
Re-trained CNN: Tensorflow
Accuracy of TensorFlow: Before And After Re-Training
No Re-training Retrained in four categories
Average accuracy: 35% Average accuracy: 73%
Metadata Extraction & Fusion
● Learn more about how animals are affected by the environment.
● Analyze this data to learn more about certain species.
● Time and date can be extracted from the EXIF data● Temperature needs to be recognized using OCR
Extracting metadata from images
Recognizing temperature digits using OCRHave temperature digits recognized automatically
Recognizing temperature digits using OCRHave temperature digits recognized automatically
? ?
Training phase - labeling digitsLabel each digit in the font family in order to train the model
Training phase - labeling digits
0
Manual labeling
Label each digit in the font family in order to train the model
Training phase - labeling digits
0 1
Manual labeling
Label each digit in the font family in order to train the model
Training phase - labeling digits
0 1 2
Manual labeling
Label each digit in the font family in order to train the model
Label each digit in the font family in order to train the model
Training phase - labeling digits
0 1 2 3 4 5 6 7 8 9 -
Manual labeling
Recognizing temperature digits using OCR
● The trained model will recognize any unlabeled character● Which number is the “nearest” in similarity with the labeled data
9 4
Collecting metadata with hidden, motion-triggered cameras at the Sedgwick Reserve
Camera 1 Camera 2 Camera 3
Camera 1 Camera 2 Camera 3
Collecting metadata with hidden, motion-triggered cameras at the Sedgwick Reserve
UnknownClassifier
84
1319
How each OCR algorithm processes temperature images
Previous OCR
UnknownClassifier
84
1319
Camera 1
Camera 2
Camera 3
94
73
79
Previous OCR
New OCR
How each OCR algorithm processes temperature images
Analyzing the accuracy and processing time of both OCR algorithms
Accuracy % Time in seconds
- Re-train TensorFlow with labeled images from Sedgwick Reserve- Collect more data to build a bigger dataset- Move everything to the cloud
The Road Ahead
Manual Labeling
Improve CNN Model
AcknowledgmentsSpecial Thanks To:
Mentor: Nevena GolubovicAdvisors: Chandra Krintz and Rich WolskiProgram Coordinator: Stephanie Mendes
CSEP
Recognizing temperature digits using OCR
● We need to automatically have these numbers recognized
Recognizing temperature digits using OCR
● We need to automatically have these numbers recognized● Find the edges on the image
Recognizing temperature digits using OCR
● We need to automatically have these numbers recognized● Find the edges on the image● With the edges, we know the dimensions of each digit/symbol
Recognizing temperature digits using OCR
● We need to automatically have these numbers recognized● Find the edges on the image● With the dimensions, we ignore characters with small area (degree symbol)
● However, numbers are still unlabeled
? ?
