View
40
Download
1
Category
Preview:
Citation preview
Classify MNIST using Convolutional Neural NetworkL0444.000300 Introduction of Artificial Intelligence
May 31, 2018
Yu-Jung HeoBioIntelligence Laboratory,
School of Computer Science and Engineering,Seoul National University
http://bi.snu.ac.kr
Contents Future schedule Review Convolutional Neural Network
Convolution Convolution Layer Pooling Layer Feature visualization
Hands on experience
2
Download ipython notebook:https://www.dropbox.com/s/0zg21futo4iwkte/practice8_cnn.ipynb?dl=0
Future Schedule May 31 (today)
Classify MNIST using Convolutional Neural Network June 5 June 7
Regularization technique June 12
Final exam June 17
Due date of final project
important to do final project!
3
Review (1/5) MNIST (Hand-written digits)
Label : 0~9 Image shape : 28x28 Gray scale #data : Training 60,000 (6,000 / class) / Test 10,000 (1,000 / class)
4
Review (2/5) Load data Convert class labels from scalars to one-hot vectors
Load batch Initialize variables
Each epoch (to max_epoch) Each batch (to max_batch)
Load batch Training model by minimizing loss function Calculate accuracy Model save
5
Review (3/5) Model
Define Input using placeholder
Model: MultiLayer Perceptron (MLP)
Output passed multilayer perceptron fed into input
6
Review (4/5) Cross-entropy
[model]
0
0.2
0.1
0.1
0.4
0.5
0.4
0.4
0.2
𝐼𝐼1 𝐼𝐼2 𝐼𝐼3
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆(𝑦𝑦)𝑖𝑖 =exp(𝑦𝑦𝑖𝑖)∑𝑖𝑖 exp(𝑦𝑦𝑖𝑖)
7
Review (5/5) Cross-entropy
Prediction True label Correct?0.3 0.3 0.4 0 0 1 Yes0.3 0.4 0.3 0 1 0 Yes0.1 0.2 0.7 1 0 0 No
Prediction True label Correct?0.1 0.2 0.7 0 0 1 Yes0.1 0.7 0.2 0 1 0 Yes0.3 0.4 0.3 1 0 0 No
𝐻𝐻𝑦𝑦 𝑦𝑦𝑦 = −�𝑖𝑖
𝑦𝑦𝑖𝑖log(𝑦𝑦𝑦𝑖𝑖) Cross entropy loss: 1.38 vs. 0.64
8
Convolutional Neural Networks (CNN)
9
Convolution (1/3) weighted average of the function 𝑆𝑆(𝜏𝜏) at the moment 𝑆𝑆 where the weighting is
given by 𝑔𝑔(−𝜏𝜏) simply shifted by amount 𝑆𝑆 https://en.wikipedia.org/wiki/Convolution
10
Convolution (2/3) Try it by yourself!
-1 0.5 1 -10 1 -2 0
0.4 0 2 0.20.5 1 -1 0
0 1 01 -4 10 1 0
* ?
11
Convolution (3/3) Effects on convolution
https://docs.gimp.org/en/plug-in-convmatrix.html 12
Convolutional Neural Networks(1/4) Convolution neural network
Convolutional layer Pooling layer Fully Connected layer (FC layer)
13
Convolutional Neural Networks(2/4) Classify images using MultiLayer Perceptron Problem?
14
Convolutional Neural Networks(3/4) Too many variations!
15
Convolutional Neural Networks(4/4) Representation learning that generates high-order abstract feature map
by combining the information contained in the input image repeatedly using “convolution” and “subsampling”
Learn Local connection using convolution Topological and geometrical local patterns in image data
Image 1 Image 2
16
Convolutional Layer (1/4)
17
Convolutional Layer (2/4)
Shared weights!
18
Convolutional Layer (3/4)
19
Convolutional Layer (4/4)
20
Pooling layer Pooling
Invariance to translation and rotation Reduce the number of parameters Max-pooling, average pooling, L2-norm pooling, …
21
Feature visualization of CNN
22
Hands on experience!
Open your ipython notebook!
23
Any questions?
24
Recommended