View
543
Download
1
Category
Preview:
Citation preview
#ibmedge© 2016 IBM Corporation
Scalable TensorFlow Deep Learning as a Service with Docker, OpenPOWER, and GPUsAndrei Yurkevich, AltorosIndrajit Poddar, IBMSep 23, 2016
#ibmedge
Please Note:• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice
and at IBM’s sole discretion.
• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.
• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
1
#ibmedge
About Indrajit (a.k.a. I.P)Expertise:
• Accelerated Cloud Data Services, Machine Learning and Deep Learning
• Apache Spark, TensorFlow… with GPUs
• Distributed Computing (scale out and up)• Cloud Foundry, Spectrum Conductor, Mesos,
Kubernetes, Docker, OpenStack, WebSphere
• Cloud computing on High Performance Systems• OpenPOWER, IBM POWER
2
Indrajit Poddar Senior Technical Staff Member,Master Inventor, IBM Systemsipoddar@us.ibm.comTwitter: @ipoddar
#ibmedge
3
#ibmedge 4
Sunnyvale, CA(HQ)
#ibmedge5
“ ”
#ibmedge
We will talk about
6
- Current state of Deep Learning
#ibmedge
We will talk about
7
- Current state of Deep Learning
- Deep Learning for cancer diagnosis (Digital Pathology)
#ibmedge
We will talk about
8
- Current state of Deep Learning
- Deep Learning for cancer diagnosis (Digital Pathology)
- TensorFlow framework for Deep Learning
#ibmedge
We will talk about
9
- Current state of Deep Learning
- Deep Learning for cancer diagnosis (Digital Pathology)
- TensorFlow framework for Deep Learning
- Distributing TensorFlow with Docker
#ibmedge
We will talk about
10
- Current state of Deep Learning
- Deep Learning for cancer diagnosis (Digital Pathology)
- TensorFlow framework for Deep Learning
- Distributing TensorFlow with Docker
- Faster training with TensorFlow on OpenPOWER and GPUs
#ibmedge
We will talk about
11
- Current state of Deep Learning
- Deep Learning for cancer diagnosis (Digital Pathology)
- TensorFlow framework for Deep Learning
- Distributing TensorFlow with Docker
- Faster training with TensorFlow on OpenPOWER and GPUs
- Infrastructure for TensorFlow as a Service
#ibmedge
A picture is worth a thousand words…
12
http://www.wordclouds.com/
#ibmedge
What is Deep Learning?Machine Learning in layers and hierarchies
13
#ibmedge
Face classification example
#ibmedge15
Medical Data Analysis Example: Image classificationComparing classification by humans and by machines
Detected by a Doctor visually
Caught by a Trained model
#ibmedge
Time Scale: Before, Digital Pathology, Deep Learning
16
1980 1990 1997 2005
Video cameras
Progress in functional
telemedicineRobotic
microscopy
First fully functional WSI
Scanner
ANN intro Yann LeCun et al., backpropagation
algorithm
“Deep Learning” for Speech Recognition
#ibmedge
Machines are now learning the way we learn
17
From "Texture of the Nervous System of Man and the Vertebrates" by Santiago Ramón y Cajal.
Artificial Neural Networks
#ibmedge
Deep Learning is improving in accuracy
18
#ibmedge
Time Scale: Advances in Deep Learning
19
2005 2010 2015
Whole Slide Image (WSI)
Scanner
2016
GPUs 12 core/socket 8 thread/core
#ibmedge
Open Source Deep Learning Libraries
20
IBM Machine Learning and Deep Learning distribution for Ubuntu on OpenPOWER:http://openpowerfoundation.org/blogs/openpower-deep-learning-distribution/(does not include TensorFlow and DL4J in the current release)
#ibmedge
Why TensorFlow?• Authored by Google• OpenSource• TensorFlow has a Python API• Use Jupyter notebooks and examples to learn• Distributed training
21
https://www.tensorflow.org/
#ibmedge
Why distribute in clusters and why use GPUs?
• Input data sets are becoming larger
• High resolution images
• Video feeds
• Large number of training features
• Training times are very long (hours, days and weeks)
• Moore’s law is dying
• CPUs are not getting any faster
• Even the largest machine has limited capacity
22
#ibmedge
Distributed Deep Learning using TensorFlow• TensorFlow (version > 0.8.0) can distribute compute intensive tasks on
multiple nodes• Parameter Server for storing parameters (weight matrix)• Performing computations in Clients (Workers)• Once computed, gradients are sent to Parameter Server to update stored parameters
23
SuperVessel Private Network
◼ Worker Task
◼ Parameter Server Task
node #1 node #2 node #10
•••◼ Worker Task ◼ Worker Task
# define Parameter Server jobs:with tf.device('/job:ps/task:%d' % taskID): ...
# define Worker jobswith tf.device('/job:worker/task:%d' %
taskID): ...
TensorFlow cluster
The Problem: automated detection of metastases in whole-slide images of lymph node sections, Source: Camelyon16The Solution: Train Deep Learning Model, and classify whole slide histology image at “Level 0”
Medical Data Analysis Example
#ibmedge
Questions to address
26
- How long does it take to train a model?
- How performance will scale vs the cluster size?
- How scaling the cluster out will affect accuracy?
#ibmedge27
Deep Learning in a TensorFlow clusterGoal: improve the training time for Camelyon16 without losing accuracy significantly.
100K images, ~2GB 4 training epochs
(~5.5k iterations at batch size 72)VGG model
#ibmedge28
Medical Data Analysis Example: applying Deep LearningGoal: improve the training time for Camelyon16 without losing accuracy significantly.
100K images, ~2GB 4 training epochs
(~5.5k iterations at batch size 72)VGG model
#ibmedge29
Medical Data Analysis Example: applying Deep LearningAccuracy metrics: ROC
100K images, ~2Gb 4 training epochs
(~5.5k iterations at batch size 72)VGG model
Zoom
© 2016 IBM Corporation #ibmedge
Infrastructure Components for Deep Learning as a Service
30
#ibmedge
Deep Learning Cluster as a Service
31
#ibmedge
Example Dockerfile to create Deep Learning images
32
FROM ppc64le/ubuntu:14.04MAINTAINER Mike Hollinger <mchollin@us.ibm.com> #bring in some base utilsRUN apt-get -y update && apt-get -y install software-properties-common wget build-essential bash-completion #enable apt-add-repository and wget for the next line and for the cuda installer to work correctlyRUN apt-get -y install dictionaries-common #inexplicably, this needs to be first before vnc-related things will install successfully#install VNC and VNC-related itemsRUN apt-get -y install x11vnc xfce4 xvfb xfce4-artwork xubuntu-icon-theme#install advanced toolchain and Linux SDKRUN wget ftp://public.dhe.ibm.com/software/server/iplsdk/v1.9.0/packages/deb/repo/dists/trusty/B346CA20.gpg.key -O /tmp/B346CA20.gpg.keyRUN wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/precise/6976a827.gpg.key -O /tmp/6976a827.gpg.keyRUN wget http://public.dhe.ibm.com/software/server/POWER/Linux/xl-compiler/eval/ppc64le/ubuntu/public.gpg -O /tmp/xl_public.gpgRUN apt-key add /tmp/B346CA20.gpg.keyRUN apt-key add /tmp/6976a827.gpg.keyRUN apt-key add /tmp/xl_public.gpgRUN add-apt-repository "deb ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at9.0"RUN apt-get -y updateRUN apt-get -y install advance-toolchain-at9.0-runtime \ advance-toolchain-at9.0-devel \ advance-toolchain-at9.0-perf \ advance-toolchain-at9.0-mcore-libs
#install XL C/C++ Community Edition, auto-accepting the license (from Ke Wen Lin)RUN apt-get -y install xlc.13.1.4 xlc-license-community.13.1.4RUN mkdir -p /opt/ibm/xlC/13.1.4/lap/license/ && chmod a+rx /opt/ibm/xlC/13.1.4/lap/licenseRUN echo "Status=9" >/opt/ibm/xlC/13.1.4/lap/license/status.datRUN /opt/ibm/xlC/13.1.4/bin/xlc_configureRUN apt-get -y install ibm-sdk-lop#bring in the ibm mldl PPARUN apt-add-repository -y ppa:ibmpackages/mldl#bring local cuda repo with GPU driver 352.39 and CUDA 7.5RUN wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda-repo-ubuntu1404-7-5-local_7.5-18_ppc64el.deb && \
dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_ppc64el.deb && apt-get update && \
apt-get install -y --no-install-recommends --force-yes cuda gpu-deployment-kit && \
ln -s /usr/lib/nvidia-352/libnvidia-ml.so /usr/lib/libnvidia-ml.so && \rm cuda-repo-ubuntu1404-7-5-local_7.5-18_ppc64el.deb
#bring in and install cudnnCOPY rootfs/cudnn-7.0-linux-ppc64le-v3.0-prod.tgz /tmp/cudnn-7.0-linux-ppc64le-v3.0-prod.tgzRUN tar --no-same-owner -xvf /tmp/cudnn-7.0-linux-ppc64le-v3.0-prod.tgz -C /usr/local #copy then untar to handle ownership problems vs "add" #install the MLDL frameworksRUN apt-get update && apt-get -y install torch caffe theano
continued ..
install deep learning software
install GPU drivers and libraries
#ibmedge
Cluster components to manage compute resources
33
Docker containers and images
OR
#ibmedge
An OpenStack- and Docker-based research cloudSuperVessel
34
https://ny1.ptopenlab.com/bigdata_cluster/
#ibmedge
Mesos with Marathon with Docker and GPUs
#ibmedge
OpenPOWER: GPU support
36
GPU
Credit: Kevin Klaues, Mesosphere
IBM Spectrum Conductor includes
enhanced support for fine grained GPU and CPU scheduling with Apache Spark and
Docker
Mesos supports GPUs
#ibmedge
POWER8 Core: Back bone of big data computing system
• Enhanced Micro-Architecture• Increased Execution Bandwidth• SMT 8• Transactional Memory
• Vector/Scalar Unit• High-performance Integer & FP Vector Processor• Optimized for Data Rich Applications
VSUFXU
IFU
DFU
ISU
PC
PC
LSU
#ibmedge
Combined I/O Bandwidth = 7.6Tb/s
POWER8
Processor
MemoryBuffers
MemoryBuffers
PCI
DMI
PCI
POWER8Processor
POWER8Processor
DMI
DMI
DMI
DMI
DMI
DMI
DMI
NODE-to-NODE
ON-NODE SMP
Putting it all together with the memory links, on- and off-node SMP links as well as PCIe, at 7.6Tb/s of chip I/O bandwidth
#ibmedge
New OpenPOWER Systems with NVLink
39
S822LC-hpc “Minsky”:2 POWER8 CPUs with 4 NVIDIA® Tesla® P100 GPUs GPUs hooked directly to CPUs using Nvidia’s NVLink high-speed interconnecthttp://www-03.ibm.com/systems/power/hardware/s822lc-hpc/index.html
#ibmedge
OpenPOWER: Open Hardware for High Performance
40
#ibmedge
Machine Learning and Deep Learning analytics on OpenPOWERNo code changes needed!!
41
ATLAS Automatically Tuned Linear Algebra Software)
#ibmedge
Challenges and what’s next
42
● Infrastructure issues:
○ Advanced resource scheduling with Platform Conductor and Kubernetes or Mesos
○ More GPUs per system (up to 4-16 cards) for improved power consumption and better density
● TensorFlow issues:
○ Resolve problems with TF-Slim and model convergence
○ Integrate HDFS or another Distributed FS with TensorFlow
○ Try Synchronous training and compare results with Asynchronous
● Improve model training for better accuracy:
○ Train on a 300K samples dataset
○ Increase the number of training iterations to 30 epochs
○ 2 iteration for update False-Positive samples in dataset
○ Use another model (change from VGG16 to Inception-v3)
#ibmedge
More related sessions at Edge•Expo Center Demo•Tue, Sept 20, 1:00-2:00PM, RM 312: Docker on IBM Power Systems: Build, Ship and Run
•Tue, Sept 20, 1:00-2:00PM, RM 313: Docker Containers for High Performance Computing
•Tue, Sept 20, 1:00-2:00PM, RM 317C: Lab: FPGA Virtualization and Operations Environment for Accelerator Application Development on Cloud
•Tue, Sept 20, 2:15-3:15PM, RM 320: Bringing the Deep Learning Revolution into the Enterprise
•Tue, Sept 20, 5:00-6:00PM, RM 308, Thu, Sep 22, 09:45 AM - 10:45 AM : Enabling Cognitive Workloads on the Cloud: GPU Enablement with Mesos, Docker and Marathon on POWER
•Wed, Sept 21, 9:45-10:45AM, RM 317 C: Lab: Fast, Scalable, Easy Machine Learning in the Cloud with OpenPOWER, GPUs and Docker
#ibmedge
Notices and DisclaimersCopyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.
IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law
#ibmedge
Notices and Disclaimers Con’t.
45
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
© 2016 IBM Corporation #ibmedge
Thank You
Recommended