88
쫄쫄쫄쫄 쫄쫄쫄 쫄 쫄 쫄 Seungil Kim 모모모모모모

쫄지말자딥러닝2 - CNN RNN 포함버전

Embed Size (px)

Citation preview

PowerPoint

Seungil Kim

/ Embedded / IoT

(Artificial Intelligence)(Machine Learning)(Deep Learning)Convolutional Neural Network

Recurrent Neural Network :

?

?

.

Dog

Cat .

, . ?

(Functions)

System, Filter .

3??6

xF:x yy

? . ?

?

r

.

(Artificial Intelligence)(Machine Learning)(Deep Learning)Convolutional Neural Network

Recurrent Neural Network :

? ?

?

(Learning) = Adaptation/Update

.

X[n]F[n] F[n-1]Y[n]n : data index

Data (Learning).

SupervisedUnsupervisedReinforcementLearning

3X ???6 Dog CatSupervisedLearning- .- ()

3X ??????UnsupervisedLearning 2, 1 ?

?xy-x-y+1=0,x y .(x,y )

. .

ReinforcementLearning- ( ) reward .

Reward

x???+10 / -10ReinforcementLearning

(Artificial Intelligence)(Machine Learning)(Deep Learning)Convolutional Neural Network

Recurrent Neural Network :

Artistic Style

Image Question and Answering

Image Generation

http://mattya.github.io/chainer-DCGAN/

Deep Reinforcement Learning : Game

Reinforcement Learning : UAV Control

(Machine Learning)(Neural Network)(Deep Learning)? ?

.

. .

x1x1bw 1:Nonlinear() activation function

bw

(Multiple Layer)

2 :Nonlinear function Nonlinear function Nonlinear function

() .Hidden layer 2 NN Deep Learning .

~ ( ) !!!

100 ?

.

(1) Overfitting(2) Vanishing Gradient

Neural Network , .., .

(1) Overfitting

Overfitting ?

(=)

Overfitting ?(1) ,

2 5 ?

Overfitting ?(2) ,

Big Data

.

Overfitting . ???

Dropout

, .

(2) Vanishing Gradient

(Deep) Learning .

= *

0 ,Learning

(1) 1 0.5 , 4 1/16(2) 0 Vanishing Gradient

ReLU

1

Long Short Term Memory

Deep Learning !!

Convolutional Neural NetworkRecurrent Neural Network , ![RNN] http://colah.github.io/posts/2015-08-Understanding-LSTMs/[CNN] http://cs231n.stanford.edu/

.

(Artificial Intelligence)(Machine Learning)(Deep Learning)Convolutional Neural Network

Recurrent Neural Network :

CNN Neural Network .

1 (vanilla) neural network

x1bw 1:Nonlinear() activation function g( )

x1bw

DataSet

Neural Network ! Convolutional Neural Network

Convolution (Operator)?

2+39-77x810/2

[2 3]*[1 1]

Convolution

A Convolution ?B Convolution ?C Convolution ?D Convolution ?E Convolution ? Convolution ? filter ?

AB

C

D

E

Convolution f ?[Ex] f : , g : R/G , h : R (f*g)*h=f*(g*h) .

: feature

Corner Detection Example

feature !

.

3x35x57x7

2. .

3x3

6x6

Convolutional Neural Network!

ConvolutionConvolutionConvolution3x33x33x35x57x7 .

ConvolutionConvolutionConvolution2. .MaxPoolingMaxPooling

Neural NetworkConvolutional Neural Network (CNN)MaxpoolingMaxpoolingFeature extraction

(Artificial Intelligence)(Machine Learning)(Deep Learning)Convolutional Neural Network

Recurrent Neural Network :

? Recurrent Neural Network

(Vanilla)Neural Network .

.

?

Image Caption Generation

.

Sentiment Classification

.

Machine Translation

(Vanilla) Neural Network Hxy

RNN H(t)xyz-1

H(t-1)H(t)=W1x+W2H(t-1)

RNN H(t)xyz-1

H(t)x(t)y(t)H(t-1)H(t-1)x(t-1)y(t-1)H(t-2)x(t-2)y(t-2)

RNN ..

(Artificial Intelligence)(Machine Learning)(Deep Learning)Convolutional Neural Network

Recurrent Neural Network :

?

?

Speech Innovation Consulting Group E-mail : [email protected] Blog : www.whyDSP.org FB:www.facebook.com/lab4all www.facebook.com/groups/modulabs


Dependency Exploitation: A Unified CNN-RNN Approach for ...CNN, which can represent different levels of features from the local view to global view. When training the CNN, we introduce

Dependency Exploitation: A Unified CNN-RNN Approach for ...CNN, which can represent different levels of features from the local view to global view. When training the CNN, we introduce

Documents
Unconstrained Arabic & Urdu Text Recognition using Deep ... · Unconstrained Arabic & Urdu Text Recognition using Deep CNN-RNN Hybrid Networks Thesis submitted in partial fulfillment

Unconstrained Arabic & Urdu Text Recognition using Deep ... · Unconstrained Arabic & Urdu Text Recognition using Deep CNN-RNN Hybrid Networks Thesis submitted in partial fulfillment

Documents
Precise diagnosis of intracranial hemorrhage and subtypes ... · † A 3D joint CNN-RNN deep learning framework was developed for ICH detection and subtype classification, which has

Precise diagnosis of intracranial hemorrhage and subtypes ... · † A 3D joint CNN-RNN deep learning framework was developed for ICH detection and subtype classification, which has

Documents
Dp-Rnn: Type Ii Diabetic Prediction Using Gkfcm And Rnn

Dp-Rnn: Type Ii Diabetic Prediction Using Gkfcm And Rnn

Documents
Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN ...€¦ · 24/08/2020  · Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN Architecture with Transfer Learning

Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN ...€¦ · 24/08/2020  · Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN Architecture with Transfer Learning

Documents
Neural Networks Convolutional and Recurrentai-nlp-ml/course/dnlp/CNN-RNN-LSTM-Attenti… · Convolutional Neural Networks (CNN) A CNN consists of a series (≥ 1) of convolution layer

Neural Networks Convolutional and Recurrentai-nlp-ml/course/dnlp/CNN-RNN-LSTM-Attenti… · Convolutional Neural Networks (CNN) A CNN consists of a series (≥ 1) of convolution layer

Documents
Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -

Detecting Linguistic Characteristics of Alzheimer's ...swetakar/images/Alzheimers_Research_Presentation.pdf-CNN Transcripts 82.8%-RNN Transcripts 83.7%-CNN-RNN Transcripts 84.9% -

Documents
Recurrent Neural Networks (RNN) and Long-Short-Term … · Last Time: CNN Architectures. ... Sequential Processing of Non-Sequence Data Ba, Mnih, and Kavukcuoglu, ... Recurrent Neural

Recurrent Neural Networks (RNN) and Long-Short-Term … · Last Time: CNN Architectures. ... Sequential Processing of Non-Sequence Data Ba, Mnih, and Kavukcuoglu, ... Recurrent Neural

Documents
Workshop Deep Learning - Personal Websitestevenputtemans.github.io/files/DeepLearningWorkshop.pdfnetwerken (ANN, CNN, DNN, RNN, AutoEncoders, GAN, …). •We mogen echter niet vergeten

Workshop Deep Learning - Personal Websitestevenputtemans.github.io/files/DeepLearningWorkshop.pdfnetwerken (ANN, CNN, DNN, RNN, AutoEncoders, GAN, …). •We mogen echter niet vergeten

Documents
JOURNAL OF LA An Investigation of Deep Learning Frameworks ... · this reason, RNN is considered to be a proper model for consistent “spoofing”. CNN+RNN: To further explore the

JOURNAL OF LA An Investigation of Deep Learning Frameworks ... · this reason, RNN is considered to be a proper model for consistent “spoofing”. CNN+RNN: To further explore the

Documents
Multidimensional RNN

Multidimensional RNN

Technology
RNN Explore

RNN Explore

Software
DNPU: An 8.1TOPS/W Reconfigurable CNN-RNN Processor for ... · PDF file14.2: DNPU: An 8.1TOPS/W Reconfigurable CNN-RNN Processor for General-Purpose Deep Neural Networks © 2017 IEEE

DNPU: An 8.1TOPS/W Reconfigurable CNN-RNN Processor for ... · PDF file14.2: DNPU: An 8.1TOPS/W Reconfigurable CNN-RNN Processor for General-Purpose Deep Neural Networks © 2017 IEEE

Documents
The Thirty-Third AAAI Conference on Artificial Intelligence ... · Figure 1: Schematic diagrams of (a) Sketch-RNN, (b) AI-Sketcher, and (c) the CNN-based autoencoder. Third, a CNN-based

The Thirty-Third AAAI Conference on Artificial Intelligence ... · Figure 1: Schematic diagrams of (a) Sketch-RNN, (b) AI-Sketcher, and (c) the CNN-based autoencoder. Third, a CNN-based

Documents
Recurrent Fusion Network for Image Captioning arXiv:1807 ... · encoder is usually a convolutional neural network (CNN), while the decoder is a recurrent neural network (RNN). The

Recurrent Fusion Network for Image Captioning arXiv:1807 ... · encoder is usually a convolutional neural network (CNN), while the decoder is a recurrent neural network (RNN). The

Documents
Interactive Multi-Label CNN Learning with Partial Labels...CNN-RNN LSEP Wsabie Fast0Tag Latent Noise Latent Noise Curriculum IMCL (relevant) (visual) Labeling-0.6 0.1 0.1 0.0 0.9 1.0

Interactive Multi-Label CNN Learning with Partial Labels...CNN-RNN LSEP Wsabie Fast0Tag Latent Noise Latent Noise Curriculum IMCL (relevant) (visual) Labeling-0.6 0.1 0.1 0.0 0.9 1.0

Documents
Sample Fusion Network: An End-to-End Data Augmentation ... · learning methods can be broadly categorized into three groups: CNN-based methods, RNN-based methods and methods based

Sample Fusion Network: An End-to-End Data Augmentation ... · learning methods can be broadly categorized into three groups: CNN-based methods, RNN-based methods and methods based

Documents
Transformers and CNNskanmy/courses/6101_1810/w8-transformer.pdf · • Motivation • CNN is faster since the computations can be more parallelized compared to RNN • RNN needs to

Transformers and CNNskanmy/courses/6101_1810/w8-transformer.pdf · • Motivation • CNN is faster since the computations can be more parallelized compared to RNN • RNN needs to

Documents
Rnn Translation

Rnn Translation

Documents
CNN-RNN: a large-scale hierarchical image classification ... · ing datasets [3, 34], such as ImageNet [6]. Such datasets often organize the large number Such datasets often organize

CNN-RNN: a large-scale hierarchical image classification ... · ing datasets [3, 34], such as ImageNet [6]. Such datasets often organize the large number Such datasets often organize

Documents
RNN Slides

RNN Slides

Documents
DiSAN: Directional Self-Attention Network for RNN/CNN … ·  · 2017-11-22DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding ... e.g., word/character

DiSAN: Directional Self-Attention Network for RNN/CNN … ·  · 2017-11-22DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding ... e.g., word/character

Documents
Question Answering over Freebase via Attentive RNN with ... · Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN Yingqi Qu1, Jie Liu1, Liangyi Kang1,

Question Answering over Freebase via Attentive RNN with ... · Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN Yingqi Qu1, Jie Liu1, Liangyi Kang1,

Documents
Introduction to PyTorch - College of Computing · Introduction to PyTorch. Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples. Introduction to

Introduction to PyTorch - College of Computing · Introduction to PyTorch. Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples. Introduction to

Documents
Comparative Analysis of CNN and RNN for Voice Pathology

Comparative Analysis of CNN and RNN for Voice Pathology

Documents
MAM-RNN: Multi-level Attention Model Based RNN for Video ... · MAM-RNN: Multi-level Attention Model Based RNN for Video Captioning Xuelong Li1, Bin Zhao2, ... video feature is encoded

MAM-RNN: Multi-level Attention Model Based RNN for Video ... · MAM-RNN: Multi-level Attention Model Based RNN for Video Captioning Xuelong Li1, Bin Zhao2, ... video feature is encoded

Documents
Dennis Madsen Fall 2019 Pattern Recognition - unibas.ch · 2019. 11. 1. · •CNN popular architectures •Sequence Models/Recurrent neural networks (RNN) ... To prevent vanishing/exploding

Dennis Madsen Fall 2019 Pattern Recognition - unibas.ch · 2019. 11. 1. · •CNN popular architectures •Sequence Models/Recurrent neural networks (RNN) ... To prevent vanishing/exploding

Documents
RNN & LSTM

RNN & LSTM

Software
Deep Learning: What Makes Neural Networks Great Again?infochim.u-strasbg.fr/CS3_2018/Presentations/... · • Convolutional neural networks (CNN) • Recurrent neural networks (RNN)

Deep Learning: What Makes Neural Networks Great Again?infochim.u-strasbg.fr/CS3_2018/Presentations/... · • Convolutional neural networks (CNN) • Recurrent neural networks (RNN)

Documents
Edge Intelligence: Convergence of Edge Compute …acm-ieee-sec.org/2018/AR_edge.pdf‒Innovation in algorithms: DNN, CNN, DBN, RNN Modern NNs loosely based on neural network function

Edge Intelligence: Convergence of Edge Compute …acm-ieee-sec.org/2018/AR_edge.pdf‒Innovation in algorithms: DNN, CNN, DBN, RNN Modern NNs loosely based on neural network function

Documents