View
18
Download
0
Category
Preview:
Citation preview
Deep Learning in Computer Vision
Bikash SantraSenior Research Fellow
Electronics and Communication Sciences Unit
Indian Statistical Institute, Kolkata, India
What Computers “See”
Slide Credit: Ava Soleimany, MIT
Images Are Numbers
Slide Credit: Ava Soleimany, MIT
Images Are Numbers
Slide Credit: Ava Soleimany, MIT
Images Are Numbers
What is Image Processing?
1. Photometric Transformations
a) Sharpening
b) Smoothing
c) Contrast enhancement
d) Stretching
2. Geometric Transformations
1. Scaling
2. Rotations
3. Translation
3. Image Compression
4. And many more
Slide Credit: Ava Soleimany, MIT
What is Computer Vision?
1. Emulates human vision
2. Goal is to understand
images and its contents
Slide Credit: Ava Soleimany, MIT
Tasks in Computer Vision
Feature
Extraction
Slide Credit: Ava Soleimany, MIT
High Level Feature Detection
Slide Credit: Ava Soleimany, MIT
Manual Feature Extraction
Slide Credit: Ava Soleimany, MIT
Manual Feature Extraction
Slide Credit: Alexander Amini, MIT
Question 1
Slide Credit: Ava Soleimany, MIT
Question 2
Yes, we can! Using deep learning
Neural NetworksDeep
Neural Networks: Architectures
Slide Credit: Fei-Fei Li et al., Stanford University
Slide Credit: Alexander Amini, MIT
Example Problem
Slide Credit: Alexander Amini, MIT
Example Problem: Will I pass this class?
Slide Credit: Alexander Amini, MIT
Example Problem: Will I pass this class?
Slide Credit: Alexander Amini, MIT
Example Problem: Will I pass this class?
Slide Credit: Alexander Amini, MIT
Example Problem: Will I pass this class?
Slide Credit: Alexander Amini, MIT
Quantifying Loss
Slide Credit: Alexander Amini, MIT
Empirical Loss
Slide Credit: Alexander Amini, MIT
Binary Cross Entropy Loss
Slide Credit: Alexander Amini, MIT
Mean Squared Error Loss
Slide Credit: Alexander Amini, MIT
Mean Squared Error Loss
Slide Credit: Alexander Amini, MIT
Loss Optimization
Solved using
gradient descent
Convolutional Neural Network
Slide Credit: Abin - Roozgard
Introduction
Convolutional neural networks
Signal processing, Image processing
improvement over the multilayer perceptron
performance, accuracy and some degree of
invariance to distortions in the input images
Slide Credit: Ava Soleimany, MIT
FCNN for Image Processing
Slide Credit: Ava Soleimany, MIT
Using Spatial Structure
Slide Credit: Ava Soleimany, MIT
Using Spatial Structure
Slide Credit: Ava Soleimany, MIT
Applying Filters to Extract Features
Slide Credit: Ava Soleimany, MIT
Feature Extraction with Convolution
Slide Credit: Fei-Fei Li et al., Stanford
Convolutional Neural Networks
CNN is evolved basically to deal with images.
Slide Credit: Ava Soleimany, MIT
Components of CNN
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Image Source: http://cs231n.github.io/convolutional-networks/
Convolution
Operation
Demo
Slide Credit: Fei-Fei Li et al., Stanford
Convolution Layer
Slide Credit: Ava Soleimany, MIT
Introducing Non-Linearity
Slide Credit: Fei-Fei Li et al., Stanford
Pooling Layer
Slide Credit: Fei-Fei Li et al., Stanford
Pooling Layer
Slide Credit: Ava Soleimany, MIT
CNNs for Classification: Feature Learning
Slide Credit: Ava Soleimany, MIT
CNNs for Classification: Class Probabilities
Slide Credit: Ava Soleimany, MIT
CNNs: Training with Backpropagation
Deep Learning Resources
Name Language Link Note
Pylearn2 Python http://deeplearning.net/software/pylearn2/ A machine learning library built on Theano
Theano Python http://deeplearning.net/software/theano/ A python deep learning library
Keras Python https://keras.io/ A python deep learning library
Caffe C++ http://caffe.berkeleyvision.org/ A deep learning framework by Berkeley
Torch Lua http://torch.ch/ An open source machine learning framework
Overfeat Lua http://cilvr.nyu.edu/doku.php?id=code:start A convolutional network image processor
Deeplearning4j Java http://deeplearning4j.org/ A commercial grade deep learning library
Word2vec C https://code.google.com/p/word2vec/ Word embedding framework
GloVe C http://nlp.stanford.edu/projects/glove/ Word embedding framework
Doc2vec Chttps://radimrehurek.com/gensim/models/d
oc2vec.htmlLanguage model for paragraphs and documents
StanfordNLP Java http://nlp.stanford.edu/ A deep learning-based NLP package
TensorFlow Python http://www.tensorflow.org A deep learning based python library
PyTorch Python https://pytorch.org/ A deep learning based python library
Thank you
Recommended