Use Docker to Deliver Cognitive Services in Multi Cloud Environments - The Watson Developer Cloud Use Case

Susan Diamond Senior Software Engineer/Continuous Delivery Leader 6/20/2016

Agenda

• Watson/Watson Services/Watson Developer Cloud (WDC) Introduction

• WDC Transition from VM to Docker • WDC Continuous Delivery Process • Things to Consider to Run Private Docker Registries • Watson Service Demo

What do we do …

Watson Cognitive Services

© 2015 International Business Machines Corporation

Watson Developer Cloud Architecture Goals

• Deliver open platform to host WDC services and provide operational excellence ➢ Cloud Native Enablement ➢ High Availability / Auto Recovery ➢ Microservices Architecture ➢ Continuous Delivery ➢ Elasticity ➢ Operational Visibility ➢ Security / Compliance ➢ Productivity Increase for Service Developers ▪ Quick onboarding, common components, tools, process

improvements

5

6

DevOps [Imaginator, Artifactory,

Asgard]

Data Services [Zookeeper,Cassandra

WS3,Priam]

Remote Log Archival/Analytics

[ELK,Cloudsight]Dynamic Routing [DataPower,Zuul]

Service Registry/ Discovery [Eureka]

DevOps Config Management

[Archaius]

Operational Visibility [Grafana, Graphite]

Testing [Jenkins,Chaos]

Metrics and Event Sources

[Servo,Crawler]

Internal Service Load Balancing

[Ribbon,Litelinks,Eureka]Service Health Checks

[Eureka]

Service Discovery/ Registration

[Eureka, Karyon,]Microservice Architecture Failure Resilient

Doocker Repo

Container Management [Mesos,Marathon]

Image Management Baremetal

VM

Availability Zones/Pods

NetworkingDNS

ObjectStorage

Provisioning [CSB]

Authorization Authentication

[LDAP,Apache Shiro]

Alerting [Seyren,Uptime,

PagerDuty]Metering

Watson Services Patterns

Watson Foundation Services

Softlayer IaaS

Watson Security ID Management

[USAM]Vulnerability Scanning

[AppScan,Nesus]Security Logging

[QRadar]

Watson Developer Cloud Architecture

* Netflix Components

Watson Developer Cloud and Docker

• Started investigate Docker in mid 2014 • Lack of monitoring/management tooling

• Picked up Docker again in beginning of 2015 and use docker in production environment since mid 2015

Some characteristics of Cognitive services:

• Self-learning • Training data/models • Data security • One service instance per customer

Dock Deployment • Deployment time significantly improved • Start up time significantly improved • Enable dynamic service instance creation on the fly • Enable seamless DevOps experience across multi-cloud environments in multi-GEOs

VM deployment Pain Points: • Long deployment time • Slow start up time • Problematic in dynamic deployment

automation

Transitioning from VM to Docker

1. VM/docker image building, bake the image once and use for all environments.

2. Service deployment 3. Service registration 4. Service request routing 5. Full stack in private

network except DataPower

Canary & Red/Black Deployments/Updates

▪Design Philosophy: No destructive or in-place upgrades ▪New ASG deployed alongside existing ASG

▪#instances determine load (point in time) ▪Disable old ASG after sufficient confidence in new – Can flip to old with a couple of clicks in case of an issue

Ubuntu VM

Docker engine

Docker Registry on Docker Container

Ubuntu VM

Docker engine

Docker Registry on Docker Container

Docker

container 1. Java 7 2. Tomcat 7 3. Side-car 4. Eureka 5. Zookeeper 6. Zuul …

Imaginator

Container 1 base image deb files 2. Services debfiles 3. Dreamfiles 4. dockerfiles …

SL object storage

Docker AAS

SL image template repository v3 VM Image template

v1 Deb files + dream file

d1 Deb files + dock file

d2 Store/retrieve deb files

v2 Store/retrieve deb files

d3 Docker image

SL firewall

Jenkins

Git Enterprise

Ubuntu VM

Haproxy

source code

Imaginator server/Ubuntu VM

Docker engine

Imaginator VM AAS

• Design Philosophy: Every change should be done via images

• Base images available to service teams ➢ Ubuntu 14.04 LTS + security hardening + agents (logging, metrics) + JRE + Tomcat + Sidecar for non-Java services

• Service teams build their image on top of base image ➢ “Dream” or Docker file describes dependencies and build targets

➢ Gradle fpm plugin for Debian package generation

• Output is Softlayer CCI image or Docker image for deployment

Continuous Delivery - Image Baking Process

© 2015 International Business Machines Corporation

VM DevOps Console: Asgard

• From Images to Running Servers – Continuous Delivery • Few button clicks to deploy one or more service

instances into Auto Scaling Group (ASG)

11

• Console allows you to: ➢ Provide parameters for HA and instance recovery

➢ Deploy one or more service versions at a time

➢ Canary Testing / Red-Black Deployments

➢ Increase/Decrease number of instances in ASG

➢ Configure auto scaling policies

Docker DevOps: Active Deploy (Investigating)

Service build Server 9.x

Docker deamon

SL object storage

Docker images

Docker Registry 9.x.x.x:5000

SL object storage

Docker images

Dev

Staging

Private Docker Registries in Public Cloud

Docker Registry 9.x.x.x:5001

haproxy

haproxy

Docker deamon

Softlayer IBM intranet

Docker deamon

push

push

pull

pull

dockerrepo-v2-01:5000 dockerrepo-v2-02:5000

dockerregistry1-v2:5000 dockerregistry2-v2:5000

SL object storage

Docker images

Production

haproxy

Docker deamon

pull

dockerregistry1-v2:5000 dockerregistry2-v2:5000

• Security

• Proprietary source code needs to be stored in a private registry • Customer’s security requirement: fully isolated stack

• Network accessible to all docker agent in the environment.

• docker agents are in private network that is not accessible to public network

• Docker registries need to available in each GEO for good performance.

• Maintenance

• Maintaining the multiple private docker registries up to date and operational needs resources.

Things to Consider to Run Private Docker Registries

Watson Services Demo

Reference Linkshttp://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/starter-kits.html

http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/

https://www.ibmchefwatson.com/community