머신러닝/딥러닝 시스템

MEMOML Software Stack

MEMO

High-level Structure

ML Framework

•

•

•

MEMO

High-level Structure

ML Framework

•

MEMO

Training

ML Framework

Forward pass: logits computationMini-batch

Backward pass: gradients computation

Input pipelineCompute

loss

MEMO

Auto-Differentiation

ML Framework

•

•

•

MEMO

Reverse-mode Autodiff

ML Framework

MEMO

Auto-differentiation

ML Framework

MEMOML Framework Categories

Build a graphExecute the graph

Directly execute statements

TensorFlow, Caffe2, MXNet, CNTK

PyTorch, TensorFlow Eager, MXNet Imperative

* Python: De-facto deep learning programming language

MEMOML Framework Categories

Build a graphExecute the graph

Directly execute statements

TensorFlow, Caffe2, MXNet, CNTK

PyTorch, TensorFlow Eager, MXNet Imperative

Easy to optimize and deploy

Hard to program and debug

Easy to program and debug

Little room for optimization

MEMO

Google TensorFlow

ML Framework Example

•

•

•

MEMO

Facebook PyTorch

ML Framework Example

•

•

•

MEMO

Open Neural Network Exchange (ONNX)

Interoperability between Frameworks

MEMO

ML frameworks

Symbolic ML frameworks and imperative ML frameworks

ML framework examples

Summary

텐서플로우

MEMOTensorFlow

TensorFlow 1.x default mode

•

Express numerical computation as a computation graph

•

•

Tensor: N-dimensional array

•

•

•

Tensors flow through the graph → TensorFlow

MEMOGoogle TensorFlow

The graph’s compiled to CPU / GPU / AIPU code

Salient features of TensorFlow graphs

MEMOTensorFlow Programming Model

Graph: model computation

Session: runs a graph

•

MEMOTensorFlow Programming Model

Symbolic Graph Style

•

•

•

•

MEMOTensorFlow Eager Mode

Statements directly executed without the separation of graph definition and execution

Write code that you can easily execute in a REPL

MEMOTensorFlow Programming Example

ReLU

Add

MatMul

MEMOTensorFlow Programming Example

ReLU

Add

MatMul

MEMOTensorFlow Programming Example

ReLU

Add

MatMul

MEMOTensorFlow Programming Example

ReLU

Add

MatMul

MEMOTensorFlow Programming Example

We can deploy the graph with a session

MEMOTensorFlow Eager Mode

Enabling eager execution requires two lines of code

Lets you write code that you can easily execute in a REPL

MEMOTensorFlow Example

Linear Regression

MEMOTensorFlow Example

Linear Regression

MEMOTensorFlow Example

Linear Regression

MEMOInput Pipeline with TensorFlow Dataset

MEMOInput Pipeline with TensorFlow Dataset

Shuffle, repeat, batch your data

Map each element of your dataset to transform it in a specific way to create a new dataset

MEMOSummary

TensorFlow programming model

TensorFlow basic program example

TensorFlow linear regression example

TensorFlow Dataset