Revolutionizing the Datacenter

Join the Conversation #OpenPOWERSummit

SuperVessel: Cognitive Computing Platformon the OpenStack based OpenPOWER Cloud

Yong Hua Lin

IBM Research China

Join the Conversation #OpenPOWERSummit

Cognitive Computing brings new challenges and opportunities

4/1/2016 2

Machine

Learning

Machine

Reasoning

Knowledge

Engineering

Multilingual NLP,

NLG & Speech

Machine

Vision

Human-Machine

Interaction

Data Mining

& Pattern Reco

Decision /

Optimization

Rule-based

Engines Data

Insights

Decision

MakingResearch & Discovery

Affective Dialog

Artifact Generation

Policy

Assurance

Radiologist Assistant

Personal Concierge

HR / Legal Assistant

Supply Chain

Assistant

Intelligence Analyst

Could we flexibly schedule parallel computation for very

computation intensive workload?

Could we satisfy the work for both training and deployment stage?

Easily program for CPU and GPU/FPGA ?

Data set management and model management?

Cognitive Computing on Cloud ?

How to do resource scheduling for

CPU/GPU/FPGA ?

Cognitive computing refers to systems that learn at scale, reason with purpose and interact with humans naturally.

• Cognitive Computing

• Big data service

• Cloud Data Service

• Accelerator service

• IoT application

development platform

• X86 to POWER porting

service

SuperVessel: The IBM OpenPOWER cloud for Researchers and Developers

3

SuperVessel Cloud

www.ptopenlab.comOpenPOWEROpenStack

SuperVessel Services

4

Computing service

Data store service

Network service

Big Data Service

Cloud Data Service

IoTDevelopmen

t Service

Super Marketplace (Accelerators, Images, Applications)

Infrastructure as Service

Platform layer service

SuperVessel provides multiple layers services.

Accelerator DevOps Service

Cognitive Computing Service

Accelerator service

OpenStack-based cloud infrastructure with FPGA and GPU

User account & authentication

manage

User dashboard

Admin dashboard

Virtual point management

Statistic and analysis

Platform

Management

System

FPGAIBM POWER servers

OpenPOWER server

Container pool for POWER7 LPAR

Distributed file system / shared file system

KVM pool for POWER8 LE/BE

KVM pool for x86Container pool for POWER8 LE/BE

Container pool for x86

Nova Neutron Cinder

LxC/ Docker

Horizon

OpenStack controller (HA)

Nova

NeutronGlance Cinder

HEAT Senlin

Ironic Swift

Keystone

Services layer

System maintenance

System monitoring

Resource usage metering

System analysis

Services for cloud administration

Baremetalmanagement

Image management

Nova Neutron Cinder

KVM

Nova Neutron Cinder

KVM

Nova Neutron Cinder

LxC/ Docker

Nova Neutron Cinder

LxC/ Docker

X86 server

GPU/FPGA scheduler

Auto Provision

GPU

Heterogeneous computing for cognitive cloud with GPU and FPGA

6

Train Data Set DNN Net File

Trained

model

Application Data from User

Training (development) Stage Recognition (deployment) Stage

Big data platform (Hadoop, Spark)

Deep Learning platform(caffe, Torch, Theano,

TensorFlow, etc.)

Model pool

Data Management

CPU + GPU cluster

Data Cleansing

Feature Engineering

Modeling

Deep Learning platformApplication servers, DB service, messaging, etc.

CPU + FPGA cluster

ApplicationRecognition, classification

To support both GPU and FPGA in the cloud infrastructure will be very important for Cognitive Cloud

7

SuperVessel 2.0 : Cognitive Computing System on OpenPOWER Cloud

Try it on : https://dashboard.ptopenlab.com/computing/

• Cognitive Infrastructure

Service

• Cognitive Computing service

• Cognitive Solution and Demo

service

8

GPU service and GPU accelerated Deep Learning

• SuperVessel provides the GPU sharing service by extending OpenStack and dockercapability. It is the first GPU sharing service in the public cloud.

• Users could apply the docker instance on SuperVessel

• Users could apply the deep learning development environment on SuperVessel, e.g. Caffe, Torch, Theano, and TensorFlow.

• All the DL environment will assign the GPU resource for acceleration automatically.

FPGA accelerator sharing service and FPGA maker zone

Xilinx Virtex 7 FPGA

Acceleration Hardware

Key technologies

Accelerator developers : Easily develop and deploy accelerator on cloud

• FPGA virtualization with POWER KVM

• CAPI virtualization with Docker

• OpenStack-based accelerator service

management

Application developers : Easily use accelerator for application

Accelerator Maker Zone

• Upload

accelerator

• Cloudify

Accelerator Service in Cloud

HEAT

orchestrator

POWER8/PowerKVM/Docker

Compute Network StorageFPGA/GPUaccelerator

• Apply VM

with

accelerator

1. Launched the accelerator service in Apr.2015

2. GCG announced SuperVessel to support ISV supporting planwith accelerator service (Jul.2015)

3. GCG announced the Accelerator App Store on SuperVessel on Sep.16 with partners

• SuperVessel developed the leading FPGA accelerator sharing service by extending OpenStack capability.

• IBM SuperVessel team and Xilinx collaboratively developed the first Accelerator DevOps Service on the cloud and it will be launched in the OpenPOWER summit 2016 (Apr.5~8, 2016)

• It leveraged the OpenStack HEAT and Nova-docker capability.

Accelerator DevOps Service on OpenPOWER cloud

• FPGA resource virtualization with Docker and KVM. (2015)

• Accelerator scheduling for FPGA resource, by enabling high level framework

e.g. Mesos, OpenStack, Kubenetes, etc. (2016)

• Debug and fault handling across FPGA+VM and IDE environment (2016)

• CAPI virtualization and sharing (2016)

Online Accelerator project management

Online developmentservice with

Cloud-based IDE

Publish to Accelerator App. Store and deployment for

application on cloud

Key Technologies

Test in VM/Dockerequipped with FPGA(for POWER8 & CAPI)

(Collaboration with Xilinx)

Easily Develop and Build!!!

AccDNN Service : Bridge of Deep Learning on FPGA Without Programming

Our Objectives:• Build up a seamless Bridge between deep learning and FPGA for data scientists without any FPGA experience.

• Needn’t involve any programming effort and keep the RTL level performance.

• Directly use net definition file and trained model parameters (weights) to generate FPGA implementation automatically.

Train Data Set DNN Net File Model Parameters(weights)

AccDNN

Training Process (GPU Accelerator)

Online application with Recognition Process (FPGA Accelerator)

Download

Supervessel user could generate DNN accelerator on FPGA

Automatically on OpenPOWER cloud

CPU + GPU CPU + FPGA

FPGA DNN accelerator

bit file

Cognitive application

Tbl.1 Performance comparison with CIFAR-10 (12 layers with ~384K weights)

Throughput(images/s)

Images / watts

Latency

POWER8 CPU 85

Xilinx xc7vx690tffg1

157-2

6671(2016/3/8)

223.4 150 us

The tested performance of AccDNN is 23x of CPU

Cognitive service (on BlueMix)

Model training (e.g. with Caffe)

Recognition with accelerator

SuperVessel : The Cognitive Computing platform on OpenPOWER Cloud

SuperVessel 2.0 is to build the cognitive computing platform on OpenPOWER cloud. Free for R&D activities on OpenPOWER.

SuperVessel demonstrated the OpenPOWER advantage for Cognitive Computing: • Massive Dockers Support – SMT capability

• High performance with optimized accelerator interface : CAPI and NVLink

• Easy for programming with CAPI

4/1/2016 12

SuperVessel cloud URL: www.ptopenlab.com

Backup

4/1/2016 13

We are entering a new period of computing history -- the cognitive computing era.

Nanotechnology

SupercomputingNeuroscience

Cognitive

Computing

Cognitive systems will require innovation breakthroughs

at every layer of information technology.

Nanotechnology

Programming

Machine Learning

Computer System Design

Information Management

Interface Man & Machine

The Tabulating Era

(1900s – 1940s)The Programming Era

(1950s – present)

The Cognitive Era

(2011 – )

Cognitive computing refers to systems that learn at scale, reason with purpose and interact with humans naturally.