Benedict DiederichTensorFlow Tutorial
- Python based (..as well as C++, Haskell, Java and Go APIs)
- Open Source library: machine learning
- by Google - used for research and production at Google products (Translation, Image
Recognition, etc.)
- License Apache 2.0 open source license
- multiple CPUs and GPUs
- Huge Community (https://github.com/tensorflow/tensorflow)
- Codes available (Github)
- „Model-ZOO“ pretrained networks available 2
What is TensorFlow?
1. provides primitives for defining functions on tensors
2. automatically computing their derivativesComputes math on tensors (like numpy but with GPU-support!)
What is a „Tensor“?
- multilinear maps from vector spaces to the real numbers- examples: Scalar, Vector, Matrix
What is TensorFlow?
TensorFlow vs. Numpy
Numpy Tensorflow
TensorFlow requires explicit evaluation!
• Express a numeric computation as a graph.• Graph nodes are operations which have any number of inputs and outputs• Graph edges are tensors which flow between nodes
Tensorflow Computation Graph
• “A Session object encapsulates the environment in which Tensor objects are evaluated”
• “TensorFlow programs are usually structured into a construction phase, that assembles a graph, and an execution phase that uses a session to execute ops in the graph.” - TensorFlow docs
Tensorflow Computation Graph - Session
Graph creation- builds graph sequentially- adds nodes
Graph computation
Graph evaluation (gives result)
never changes its value(s)
input 1 = tf.constant(3.0)input 2 = tf.constant(2.0)input 3 = tf.constant(5.0)intermed = tf.add(input2, input 3)mul = tf.multiply(intermed, input1)
result = mul.eval()
Mathematical operations:computations that will act on tensors• MatMul: Multiply two matrix values• Add: Add elementwise (with broadcasting)• ReLU: Activate with elementwise rectified linear function• etc.
Tensorflow Computation Graph
“When you train a model you use variables to hold and update parameters. Variables are in-memory buffers containing tensors” -
TensorFlow Variables
Was previously a constant
TensorFlow variables must be initialized before they havevalues!
- nodes which output their current value- State is retained across multiple executions
of a graph- e.g. parameters, gradient stores, eligibility
traces, …- value(s) can be updated
tf.placeholder variables - dummy nodes that provide entry points for data to
computational graph- value is fed in at execution time- inputs, variable learning rates, …
feed_dict- python dictionary mapping from tf.placeholder vars to
data (numpy arrays, lists, etc.)
Tensorflow Placeholders and FeedDictionairies
Tensorflow Placeholders and FeedDictionairies
Google Example: Graph Creation
Google Example: Graph Evalution
Example: Linear Regression in TensorFlow
Example: Linear Regression in TensorFlow
Example: Linear Regression in TensorFlow
Example: Linear Regression in TensorFlow
Example: Linear Regression in TensorFlow
Example: Linear Regression in TensorFlow
• Linear regression example computed L2 loss for a linear regression system.
• tf.train.Optimizer creates an optimizer:• tf.train.AdagradOptimizer, tf.train.AdadeltaOptimizer, tf.train.MomentumOptimizer,
tf.train.FtrlOptimizer, etc.
• tf.train.Optimizer.minimize(loss, var_list) adds optimization operation to computation graph.
• Automatic differentiation computes gradients without user input• TensorFlow nodes in computation graph have attached gradient operations
• Use backpropagation (using node-specific gradient ops) to compute required gradients for all variables in graph.
Concept: Auto-Differentiation
1. Build a graph§ Graph contains:
- parameter specifications- model architecture- optimization process, …
2. Initialize a session3. Fetch and feed data with Session.run� Compilation� Optimization� Auto-Differentiation etc.
Workflow for TensorFlow
• Lecture Slides CS224d: TensorFlow Tutorial Bharath Ramsundar• https://en.wikipedia.org/wiki/TensorFlow• Talk: Introduction to TensorFlow -
https://docs.google.com/presentation/d/1oB_U_JagxWQdQJlLD80XlNk6fuv42bHV0hfDAPGAbrc/edit#slide=id.gd2d423435_0_112
• tensorflow.org• Tensorflow Short intro presentation: bit.ly/stanford-tf-tutorial• Great Cours: http://cs231n.github.io/• Great Cours: https://de.udacity.com/course/deep-learning--ud730/
Sources