Upload
hanbit-media
View
38
Download
0
Embed Size (px)
Citation preview
Machine Intelligence at Google Scale
ML APIs, TensorFlow and Cloud ML
+Kazunori Sato @kazunori_279
Kaz Sato
Staff Developer Advocate
Tech Lead for Data & Analytics
Cloud Platform, Google Inc.
What we’ll cover
What is Neural Network and Deep Learning
Machine Learning use cases at Google services
Externalizing the power with ML APIs
TensorFlow: the open source library for ML
TensorFlow in the Wild
Distributed training and prediction with Cloud ML
What is Neural Network
and Deep Learning
Neural Network is a function that can learn
xn
> b?
w1
wn
x2
x1
Inspired by the behavior of biological neurons
How do you classify them?
weights
bias (threshold)
Programmers need to specify the parameters
Let’s see how neural network solves the problem
The computer tries to find the best parameters
A neuron classifies a data point into two kinds
Gradient Descent: adjusting the params gradually to reduce errors
How do you classify them?
What we see What the computer “sees”
28 x 28 gray scale image =
784 numbers
input vector (pixel data)
output vector (probability)
How do you classify them?
More neurons = More features to extract
Hidden Layers:
mapping inputs to
a feature space,
classifying with
a hyperplane From: Neural Networks, Manifolds, and Topology, colah's blog
From: Neural Networks, Manifolds, and Topology, colah's blog
How about this?
More hidden layers = More hierarchies of features
How about this?
We need to go deeper neural network
From: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.
From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.
How CNN works
Machine Learning use cases
at Google services
27
signal for Search ranking,
out of hundreds
improvement to ranking quality
in 2+ years
#3 #1
Search
machine learning for search engines
RankBrain: a deep neural network for search ranking
WaveNet by
DeepMind
30
[glacier]
Google Photos
30
31
Smart reply in Inbox by Gmail
10% of all responses sent on mobile
Google Translate with Neural Machine Translation
Saved Data Center cooling energy for 40% Improved Power Usage Effectiveness (PUE) for 15%
34
Android
Apps
Gmail
Maps
Photos
Speech
Search
Translation
YouTube
and many others ...
Used across products:
2012 2013 2014 2015
Deep Learning usage at Google
Externalizing the power
with ML APIs
TensorFlow Cloud Machine Learning ML API
Easy-to-Use, for non-ML engineers
Customizable, for Data Scientists
Machine Learning products from Google
Image analysis with pre-trained models
No Machine Learning skill required
REST API: receives an image and returns a JSON
$1.50 per 1,000 units
GA - cloud.google.com/vision
Cloud Vision API
Confidential & Proprietary Google Cloud Platform 38
Faces Faces, facial landmarks, emotions
OCR Read and extract text, with support for > 10 languages
Label Detect entities from furniture to transportation
Logos Identify product logos
Landmarks & Image Properties Detect landmarks & dominant color of image
Safe Search Detect explicit content - adult, violent, medical and spoof
39 39
Demo
Pre-trained models. No ML skill required
REST API: receives audio and returns texts
Supports 80+ languages
Streaming or non-streaming
Public Beta - cloud.google.com/speech
Cloud Speech API
Confidential & Proprietary Google Cloud Platform 41
Features
Automatic Speech Recognition
(ASR) powered by deep learning
neural networking to power your
applications like voice search or
speech transcription.
Recognizes over 80
languages and variants
with an extensive
vocabulary.
Returns partial
recognition results
immediately, as they
become available.
Filter inappropriate
content in text results.
Audio input can be captured by an application’s
microphone or sent from a pre-recorded audio
file. Multiple audio file formats are supported,
including FLAC, AMR, PCMU and linear-16.
Handles noisy audio from many
environments without requiring
additional noise cancellation.
Audio files can be uploaded in the
request and, in future releases,
integrated with Google Cloud
Storage.
Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering
Streaming Recognition
Real-time or Buffered Audio Support Noisy Audio Handling Integrated API
42 42
Demo
Pre-trained models. No ML skill required
REST API: receives text and returns analysis results
Supports English, Spanish and Japanese
GA - cloud.google.com/natural-language
Cloud Natural Language API
Confidential & Proprietary Google Cloud Platform 44
Features
Extract sentence, identify parts of
speech and create dependency parse
trees for each sentence.
Identify entities and label by types such
as person, organization, location, events,
products and media.
Understand the overall sentiment of a
block of text.
Syntax Analysis Entity Recognition
Sentiment Analysis
45 45
Demo
Pre-trained models. No ML skill required
REST API: receives text and returns translated text
8 languages: English to Chinese, French, German,
Japanese, Korean, Portuguese, Spanish, Turkish
Public Beta - cloud.google.com/translate
Cloud Translation API Premium
47 47
Demo
TensorFlow:
An open source library for
Machine Intelligence
Google's open source library for
machine intelligence
tensorflow.org launched in Nov 2015
Used by many production ML projects
What is TensorFlow?
50
Sharing our tools with researchers and developers around the world
repository for “machine learning”
category on GitHub
#1
Released in Nov.
2015
From: http://deliprao.com/archives/168
51
Before
Hire Data Scientists ↓
Understand the math model ↓
Impl with programming code ↓
Train with single GPU ↓
Build a GPU cluster ↓
Train with the GPU cluster ↓
Build a prediction server or Impl mobile/IoT prediction
After
Easy network design and impl
↓
Train with single machine
↓
Train on the cloud
↓
Prediction on the cloud
or mobile/IoT devices
many people stuck here
# define the network import tensorflow as tf x = tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) y = tf.nn.softmax(tf.matmul(x, W) + b) # define a training step y_ = tf.placeholder(tf.float32, [None, 10]) xent = -tf.reduce_sum(y_*tf.log(y)) step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)
TensorBoard: visualization tool
Portable and Scalable
Training on:
Mac/Windows
GPU server
GPU cluster / Cloud
Prediction on:
Android and iOS
RasPi and TPU
Distributed Training with TensorFlow
TensorFlow in the Wild (or democratization of deep learning)
TensorFlow powered Cucumber Sorter
From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/
TensorFlow powered Cucumber Sorter
TensorFlow
powered
Fried Chicken
Nugget Server
From: http://www.rt-net.jp/karaage1/
TV popstar face
generator with DCGAN
From: http://memo.sugyan.com/entry/2016/10/12/084751
TensorFlow +
Drones
for counting trucks
From: http://www.brainpad.co.jp/news/2016/09/02/3454
TensorFlow+
RasPi for
sorting garbages
From: https://techcrunch.com/2016/09/13/auto-trash-sorts-garbage-automatically-at-the-techcrunch-disrupt-hackathon/
Autonomous
Driving
of RasPi car
with Inception 3
on TensorFlow
From: https://github.com/zxzhijia/GoPiGo-Driven-by-Tensorflow
Distributed Training and
Prediction with Cloud ML
From: Andrew Ng
The Bigger, The Better
The Challenge: Computing Power
DNN requires large training datasets
Large models doesn't fit into a GPU
Requires try-and-errors to find the best design, configs and params
↓
Need to spend a few days or weeks to finish a training
GPUs run at nanoseconds GPU cluster needs microsec network
Enterprise
Google Cloud is
The Datacenter as a Computer
Jupiter network
10 GbE x 100 K = 1 Pbps
Consolidates servers with
microsec latency
Borg
No VMs, pure containers
10K - 20K nodes per Cell
DC-scale job scheduling
CPUs, mem, disks and IO
Distributed Training
with TensorFlow
by data parallelism
split data,
share model
CPU/GPU scheduling
Communications
Local, RPC, RDMA
32/16/8 bit quantization
Cost-based optimization
Fault tolerance
Distributed Systems for Large Neural Network
Distributed Training with TensorFlow
on Google Cloud
"Large Scale Distributed Systems for Training Neural
Networks", NIPS 2015
Inception / ImageNet: 40x with 50 GPUs
RankBrain: 300x with 500 nodes
Fully managed distributed training and prediction
Supports custom TensorFlow graphs
HyperTune for hyper-parameter tuning automation
Integrated with Cloud Dataflow and Cloud Datalab
Public Beta - cloud.google.com/ml
Cloud Machine Learning (Cloud ML)
Cloud ML at Work: AUCNET
The largest real-time car auction service in Japan
For 30K used car dealers
The auction volume overs $3.7B every year
Problem: auction entry is time consuming task for dealers
Classifying parts of car for thousands of photos
Identifying the exact car model
Solution: Custom Image Classification with TensorFlow/Cloud ML
Used 5,000 training images for 500 car models
Inception v3 + Transfer Learning
Cloud ML: increased training performance for 6x faster
Predicting "large loss" cases in car insurance
Old method (Random Forest): 38% accuracy
New method (TensorFlow): 73% accuracy
A Global Insurance Firm
81 81
Demo
82 82
Ready to use Machine Learning models
Use your own data to train models
Cloud Vision API
Cloud Speech API
Cloud Translation API
Cloud Machine Learning
Develop - Model - Test
Google BigQuery
Cloud Storage
Cloud Datalab
Beta
GA Beta GA
Beta GA
GA GA
Cloud Natural Language API
Tensor Processing Unit
ASIC for TensorFlow
Designed by Google
10x better perf / watt
latency and efficiency
bit quantization
TPU on Production
RankBrain
AlphaGo
Google Photos
Speech
and more
Thank you!