View
1
Download
0
Category
Preview:
Citation preview
Mathematics for Machine LearningSpecial Mathematics LectureNagoya University, Fall 2020
https://www.henrikbachmann.com/mml_2020.html
Lecture 11: Neural Networks I
Machine learning overview
Remaining plan:• Introduce (simple) Neural Networks (Today)• Understand how to train them (Next time)• Consider more complicated NN.• Use finished implementations, e.g. Tensorflow.
Nice to watch
https://www.3blue1brown.com/neural-networks
https://nnfs.io/
3blue1brown
Neural Networks from Scratch
Neural Networks
Recall: Logistic regression
Logistic regression
Hypothesis:
Example: Binary classifier (Pass an exam Yes/No, Spam email Yes/no)
We learned the correct parameters by maximizing the log-likelihood (by using gradient ascent)
Or mizimizing the negative of the log-likelihood (= cost function) (by using gradient descent)
Rewriting logistic regression as a neural network
Motivation: Image classification
Goal: Check if a picture contains a cat
Motivation: Image classification
Goal: Check if a picture contains a cat or dog
Motivation: Image classification. More layers
Goal: Check if a picture contains a cat or dog
(rough) Notation
Content of a layer
Content of a layer
Example of activation functionsThere are several common activiation functions.
https://en.wikipedia.org/wiki/Activation_function
Question: Why not use the identity map as an activation function?
Example of NNLet us consider a NN with 4 layers (2 hidden):
Recommended