32

1 CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman [email protected] Machine Learning: Neural Networks R&N 18.7 Backpropagation

CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

Download PDF Report

Upload
others
View
4
Download
0

Embed Size (px)

Citation preview

Page 1: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

1

CS 4700:Foundations of Artificial Intelligence

Prof. Bart [email protected]

Machine Learning:Neural Networks

R&N 18.7Backpropagation

Page 2: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

2

Page 3: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

3

Page 4: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

4

Page 5: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

5

Page 6: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

6

Page 7: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

7

Page 8: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

8

Ok… details backpropagation NOT on exam.

But do work through gradient descent example on homework.

Page 9: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

9

Page 10: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

10

Page 11: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

11

Page 12: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

12

Page 13: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

13

It’s “just” multivariate (or multivariable) calculus.

Page 14: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

14

Optimizationandgradient descent.

Considerx and y ourtwo weightsand f(x,y)the errorsignal.

Multi-layernetwork:compositionof sigmoids;use chain rule.

Note:For descentwe want togo in oppositedirection ofgradient.

Page 15: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

15

Reminder:Gradient descent optimization.(Wikipedia)

Page 16: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

16

Page 17: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

17

Page 18: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

18

Page 19: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

19

So, first we “backpropagate” the errorsignal to get the gradient, and then we update the weights, layer by layer.

That’s why it’s called “backprop learning.”Now changing speech recognition, imagerecognition etc.!

Page 20: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

20

Page 21: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

21

Trace example!

Page 22: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

22

Page 23: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

23

Page 24: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

24

Page 25: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

25

Page 26: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

26

Page 27: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

27

Page 28: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

28

Page 29: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

29

Etc.

Tedious but“just” gradientdescent in theweight space,after all.

Page 30: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

30

Bla bla

Wall Street Journal, 11/24/16

Good news…

Page 31: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

31

Summary J

A tour of AI:I) AI

--- motivation--- overview of the field

II) Problem-Solving–---- problem formulation using state space– (initial state, operators, goal state(s))–--- search techniques--- adversarial search

White white

Page 32: CS 4700: Foundations of Artificial Intelligence...--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent) The field has grown (and continues to grow)

32

Summary, cont. III) Knowledge Representation and Reasoning

--- logical agents--- Boolean satisfiability (SAT) solvers--- first-order logic and inference

V) Learning–--- decision tree learning (info gain)--- reinforcement learning (intro)--- neural networks / deep learning (gradient descent)

The field has grown (and continues to grow) exponentially but youhave now seen a good part!

Thanks & Have a great winter break!!!!

Gradient Descent Algorithm in Machine Learning · Examples of algorithms with coefficients that can be optimized using gradient descent are: Linear Regression Logistic Regression

Gradient Descent Algorithm in Machine Learning · Examples of algorithms with coefficients that can be optimized using gradient descent are: Linear Regression Logistic Regression

Documents

Dynamics of On-Line Gradient Descent Learning for Multilayer … · 2014. 4. 14. · Dynamics of On-Line Gradient Descent Learning for Multilayer Neural Networks David Saad" Dept

Dynamics of On-Line Gradient Descent Learning for Multilayer … · 2014. 4. 14. · Dynamics of On-Line Gradient Descent Learning for Multilayer Neural Networks David Saad" Dept

Documents

4700 SERIES OWNER'S MANUAL 4700 SERIES

4700 SERIES OWNER'S MANUAL 4700 SERIES

Documents

Learning long-term dependencies with gradient descent is difficult

Learning long-term dependencies with gradient descent is difficult

Documents

Dictionary Learning with Large Step Gradient Descent for ...markp/2012/MailhePlumbley12-lvaica... · Dictionary learning with large step gradient descent for sparse representations

Dictionary Learning with Large Step Gradient Descent for ...markp/2012/MailhePlumbley12-lvaica... · Dictionary learning with large step gradient descent for sparse representations

Documents

Learning to Learn by Gradient Descent with Rebalancing€¦ · that, for instance, are capable of learning to learn without gradient descent by gradient descent. It should be expected

Learning to Learn by Gradient Descent with Rebalancing€¦ · that, for instance, are capable of learning to learn without gradient descent by gradient descent. It should be expected

Documents

Convergence Rates of Accelerated Markov Gradient Descent with … · Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning Thinh T

Convergence Rates of Accelerated Markov Gradient Descent with … · Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning Thinh T

Documents

Gradient Descent Optimization Algorithms for Deep Learning · 31/3/2017 · Gradient Descent Optimization Algorithms for Deep Learning ... vector in Image) ... Lasso q L Ú Û

Gradient Descent Optimization Algorithms for Deep Learning · 31/3/2017 · Gradient Descent Optimization Algorithms for Deep Learning ... vector in Image) ... Lasso q L Ú Û

Documents

Learning to learn by gradient descent by gradient descent · Learning to learn by gradient descent by gradient descent Liyan Jiang July 18, 2019 1 Introduction The general aim of

Learning to learn by gradient descent by gradient descent · Learning to learn by gradient descent by gradient descent Liyan Jiang July 18, 2019 1 Introduction The general aim of

Documents

Learning to learn by gradient descent by gradient descentpapers.nips.cc/...to...descent-by-gradient-descent.pdf · Learning to learn by gradient descent by gradient descent Marcin

Learning to learn by gradient descent by gradient descentpapers.nips.cc/...to...descent-by-gradient-descent.pdf · Learning to learn by gradient descent by gradient descent Marcin

Documents

PILAE: A Non-gradient Descent Learning Scheme for Deep ...pguo/pdf/2018/PILAE A Non... · Most of the learning algorithms for training autoencoders are based on the gradient descent

PILAE: A Non-gradient Descent Learning Scheme for Deep ...pguo/pdf/2018/PILAE A Non... · Most of the learning algorithms for training autoencoders are based on the gradient descent

Documents

Deep Learning I: Gradient Descentgd.pdf · Roadmap Intro,model,cost Gradient descent Gradient descent (GD) Gradient descent is a learning algorithm. Given: a hypothesis space (or

Deep Learning I: Gradient Descentgd.pdf · Roadmap Intro,model,cost Gradient descent Gradient descent (GD) Gradient descent is a learning algorithm. Given: a hypothesis space (or

Documents

Learning curve in modern ML The 'double descent' behavior

Learning curve in modern ML The 'double descent' behavior

Documents

Learning to learn by gradient descent by reinforcement ... › 0b46 › 4e82c52c6e0e63a05d1ba4… · Learning to learn by gradient descent by reinforcement learning Ashish Bora Abstract

Learning to learn by gradient descent by reinforcement ... › 0b46 › 4e82c52c6e0e63a05d1ba4… · Learning to learn by gradient descent by reinforcement learning Ashish Bora Abstract

Documents

Stochastic gradient descent on Riemannian manifolds · 2013-11-22 · Outline 1 Stochastic gradient descent Introduction and examples SGD and machine learning Standard convergence

Stochastic gradient descent on Riemannian manifolds · 2013-11-22 · Outline 1 Stochastic gradient descent Introduction and examples SGD and machine learning Standard convergence

Documents

Stochastic Gradient Descent Tricks › diglib › lsml › bottou-sgd-tricks-2012.pdfTable 1 illustrates stochastic gradient descent algorithms for a number of classic machine learning

Stochastic Gradient Descent Tricks › diglib › lsml › bottou-sgd-tricks-2012.pdfTable 1 illustrates stochastic gradient descent algorithms for a number of classic machine learning

Documents

Intro Logistic Regression Gradient Descent + SGDIntro Logistic Regression Gradient Descent + SGD Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade

Intro Logistic Regression Gradient Descent + SGDIntro Logistic Regression Gradient Descent + SGD Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade

Documents

Documents

Objective Metrics and Gradient Descent Algorithms for … · 2020-06-16 · Objective Metrics and Gradient Descent Algorithms for Adversarial Examples in Machine Learning ... Permission

Objective Metrics and Gradient Descent Algorithms for … · 2020-06-16 · Objective Metrics and Gradient Descent Algorithms for Adversarial Examples in Machine Learning ... Permission

Documents

Deep Learning via Hessian-free Optimizationasamir/cifar/HFO_James.pdfGradient descent is bad at deep learning (cont.) Two hypotheses for why gradient descent fails: increased frequency

Deep Learning via Hessian-free Optimizationasamir/cifar/HFO_James.pdfGradient descent is bad at deep learning (cont.) Two hypotheses for why gradient descent fails: increased frequency

Documents

Gradient Descent - Stephan RobertGradient Descent Learning rate (intuitive) •Practicaltrick: –Debug: J shoulddecrease aftereachiteration –Fixa stop-threshold (0.001?) –Observe

Gradient Descent - Stephan RobertGradient Descent Learning rate (intuitive) •Practicaltrick: –Debug: J shoulddecrease aftereachiteration –Fixa stop-threshold (0.001?) –Observe

Documents

ByRDiE: Byzantine-resilient Distributed Coordinate Descent for ... · ByRDiE: Byzantine-resilient Distributed Coordinate Descent for Decentralized Learning Zhixiong Yang, Graduate

ByRDiE: Byzantine-resilient Distributed Coordinate Descent for ... · ByRDiE: Byzantine-resilient Distributed Coordinate Descent for Decentralized Learning Zhixiong Yang, Graduate

Documents

HW2*posted*today. StochasticGradient* Descent · 1 1 StochasticGradient* Descent Machine*Learning*– CSE546 Sham*Kakade Universityof*Washington October*18,*2016 ©Sham*Kakade*2016

HW2*posted*today. StochasticGradient* Descent · 1 1 StochasticGradient* Descent Machine*Learning*– CSE546 Sham*Kakade Universityof*Washington October*18,*2016 ©Sham*Kakade*2016

Documents

Regression and Gradient Descent - GitHub Pages · 2020. 12. 20. · Linear regression with Batch Gradient Descent Repeat { (for every ) } Learning rate is typically held constant

Regression and Gradient Descent - GitHub Pages · 2020. 12. 20. · Linear regression with Batch Gradient Descent Repeat { (for every ) } Learning rate is typically held constant

Documents

Modeling Learning Trajectories with Batch Gradient Descent

Modeling Learning Trajectories with Batch Gradient Descent

Documents

PathNet: Evolution Channels Gradient Descent in Super ... · PathNet: Evolution Channels Gradient Descent in Super ... and SVHN supervised learning classi ca- ... recti ed linear

PathNet: Evolution Channels Gradient Descent in Super ... · PathNet: Evolution Channels Gradient Descent in Super ... and SVHN supervised learning classi ca- ... recti ed linear

Documents

Dictionary Learning with Large Step Gradient Descent for ... · Dictionary Learning with Large Step Gradient Descent for Sparse Representations Boris Mailhé, Mark Plumbley To cite

Dictionary Learning with Large Step Gradient Descent for ... · Dictionary Learning with Large Step Gradient Descent for Sparse Representations Boris Mailhé, Mark Plumbley To cite

Documents

Gradient Descent Learning and Backpropagationpages.cpsc.ucalgary.ca/~jacob/Courses/Fall2003/CPSC533/Slides/0… · Examples: Perceptron learning, backpropagation. ËReinforcement

Gradient Descent Learning and Backpropagationpages.cpsc.ucalgary.ca/~jacob/Courses/Fall2003/CPSC533/Slides/0… · Examples: Perceptron learning, backpropagation. ËReinforcement

Documents

Machine Learning: Logistic Regression Lecture 04oucsace.cs.ohiou.edu/~razvan/courses/ml4900/lecture04a.pdf · Machine Learning: Logistic Regression Lecture 04. ... gradient descent

Machine Learning: Logistic Regression Lecture 04oucsace.cs.ohiou.edu/~razvan/courses/ml4900/lecture04a.pdf · Machine Learning: Logistic Regression Lecture 04. ... gradient descent

Documents

Probabilistic Line Searches for Stochastic Optimizationthe need to deﬁne a learning rate for stochastic gradient descent. 1 Introduction Stochastic gradient descent (SGD) [1] is

Probabilistic Line Searches for Stochastic Optimizationthe need to deﬁne a learning rate for stochastic gradient descent. 1 Introduction Stochastic gradient descent (SGD) [1] is

Documents