The Software-defined Datacenter, VMs, and Containers: A ......Containers & VMware NSX • Unified operational model for VMs & containers • Programmable, datacenter-wide connectivity

The Software-defined Datacenter, VMs, and Containers:A “Better Together” Story

SDDC3327

Kit Colbert, VMware, Inc

Disclaimer• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

CONFIDENTIAL 2

Agenda

CONFIDENTIAL 3

1 Context

2 Unified Infrastructure Fabric

3 Unified Cloud Management

4 3rd Platform Application Stack

5 Summary

Section 1: Context

HardwareHardware

OS Kernel

OS File system

Use

rspa

ce

ContainerContainer

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

ContainerContainer

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

Linux Containers

55

OS-level Isolation• Isolation at individual kernel subsystem

level (e.g. filesystem, process table, etc)

• User-level process (LXC, libcontainer) orchestrates these subsystems to create a container

Existed for Many Years• Solaris Zones, FreeBSD Jails, OpenVZ

Why?• Process isolation

• Reproducible environment

• Enables management at scale

The Problem in 2014

Static website

Web frontend

User DB Queue Analytics DB

Background workers API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8 hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis‐sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg+ postgresql‐client

Development VM

QA ServerPublic Cloud

Disaster Recovery

Contributor’s LaptopProduction Servers

Multiplicityof Stacks

Multiplicityof hardware

environments

Production VM Cluster

Customer Data Center

Do services and apps interact

appropriately?

Can Imigrate

smoothlyand quickly?

66

Let’s create a shipping container system for applications

Multiplicityof Stacks

Multiplicityof hardware

environments

Do services and apps interact

appropriately?

Can Imigrate

smoothlyand quickly?

Static website Web frontend User DB Queue Analytics DB

Development VM QA Server Public Cloud Contributor’s Laptop

Production VM Cluster

Customer Data Center

An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…

…that can be manipulated using standard operations and run consistently on virtually any hardware platform

77

Container Fits Well with DevOps Lifecycle

8

Development

Package & RepositoryPackage & Repository

Test Automation

Test Automation

Integrated Dev. Env.


Continuous Integration

UAT

Continuous Delivery Platform

ProductionSys. Int. Test

Code Dev & Check-in

Code Dev & Check-in

Build, Integration and Testing


Repository Mgmt

Repository Mgmt

Deployment & Testing


Promotion & GovernancePromotion & Governance

ProductionDeploymentProductionDeployment

Build & Integration

Build & Integration

is a “Shipping Container” for Code

9

Ops ♥ Consistent operations on codeUniform start, stop, logging, monitoring

Devs ♥ Consistent environmentOS, libs, layering on other containers

9

On-premise

Client-server, stateful, scaleup

Tier 1/Converged HW

Classic NAS & SAN

Relies on infrastructure availability

Human-driven

The Rise of Third Platform Applications

10

On/Off premise

Elastic, stateless, scale-out

Commodity/disaggregated HW

DAS, HDFS, Object, Flash, NVM

Built-in application resiliency

API-Driven/DevOps infrastructure

One School of Thought: Containers or VMs?

11

VMsVMs ContainersContainers

Implication: Separate Stacks, Higher CapEx & OpEx

12

ManagementManagement ManagementManagement

InfrastructureInfrastructure InfrastructureInfrastructure


Instead, Containers AND VMs!

13

Unified Cloud ManagementUnified Cloud Management

Unified Infrastructure Fabric Unified Infrastructure Fabric


Unified Infrastructure Fabric(ex. ESX, NSX, SDS)

Unified Cloud Management(ex. vCAC, vCOps, Log Insight)

ContainersVMs

Containers Without Compromise

14

Open Containers API

Single Platform for VMs and Containers

Consistent developer & deployment experience

Common management, monitoring, compliance across all applications

• ‘Better-than-physical’ compute layer• Network & security controls for containers • SDS: data persistence, backup, SLA

management

Enable 2-tier scheduler model; integration with Kubernetes, Pivotal CF, and other schedulers

Section 2: Unified Infrastructure Fabric

15

Unified Cloud Management

Unified Infrastructure Fabric


VM and Container Isolation are Better Together

16

VMs ContainersContainers

• Hardware level isolation• Focused on security and

multi-tenancy• 15 years in production,

battle tested

• OS level isolation• Focused on environmental

consistency• Emerging, still maturing

Great for security Great for reproducibility

VMsVMsrsrs

Best of both worlds

VMs are Lightweight and Efficient

17

Forking

FastSub-second VMprovisioning time

Ready to GoClone a running container in warmed up state

EfficientLower resource usage through sharing

Binaries& Libraries

App A

OS

VM Debunk the Myth• VM overhead < 5%• VM is lightweight• OS tends to be heavierLooking ahead• Thinner OS emerging• Project Fargo

Containers & VMware NSX

• Unified operational model for VMs & containers

• Programmable, datacenter-wide connectivity

• Enterprise-grade security with micro-segmentation.

• Native Open vSwitch support for containers

18

Any Application(without modification)

Virtual Networks

VMware NSX Network Virtualization PlatformVMware NSX Network Virtualization Platform

Logical L2

Any Network Hardware

Any Cloud Management Platform

LogicalFirewall

LogicalLoad Balancer

Logical L3

LogicalVPN

Any Hypervisor

Distributed and Reliable Storage for Container

HOSTHOST

StatelessContainerStatelessContainer

HOSTHOST HOSTHOST HOSTHOST


HOSTHOST

VSAN Distribute, Reliable StorageSnapshots, Clones, QoS, Remote Replication

VSAN Distribute, Reliable StorageSnapshots, Clones, QoS, Remote Replication

BootImage

BootImage

BootImage

Container PROVISIONING AND MANAGEMENT • Simple data persistence

• Easy deployment of containers on cluster

• Reliable, high performance storage

• Tolerant of host/disk failures

• Fast container create leveraging snapshots and clones in VSAN

• Quality of Service Controls


BootImage


BootImage

DBsTraditional Apps

Sharing Infrastructure Efficiently

20

Container cluster Container cluster

• Unified platform to run all your apps• Dynamically allocate resource based on demands and SLA• Strong security and performance isolation

Database clusterDatabase cluster Traditional AppsTraditional Apps

Silo’ed cluster leads to server/cluster sprawling, increases cost

Scenario 1: Multiple workloads Scenario 2: Multiple tenants

Containerized apps Tenant/LOB 1

Tenant 1Tenant 1 Tenant 2Tenant 2 Tenant 3Tenant 3

Data Center VirtualizationSDDC Platform

Hybrid Platform

21

vCloud AirData Centers

On-premisesData Centers

Data

vCloud Plug-in

Security

Apps, Tools, ServicesMulti-tenant

Secure Connectivity

Management

Dedicated

Software-Defined Data Center

Section 3: Unified Cloud Management

22




Container Fits Well with Devops Lifecycle

23

Development

Package & RepositoryPackage & Repository

Test Automation

Test Automation



Continuous Integration

UAT

Continuous Delivery Platform

ProductionSys. Int. Test

Code Dev & Check-in

Code Dev & Check-in



Repository Mgmt

Repository Mgmt



Promotion & GovernancePromotion & Governance

ProductionDeploymentProductionDeployment

Build & Integration

Build & Integration

Manage VMs and Containers at Scale is Key

24

On-premise


Tier 1/Converged HW

Classic NAS & SAN


Human-driven

On/Off premise






Web tier

App tier

DB tier

LoadBalancer

LoadBalancer AuthenticationAuthenticationSession

StoreSession

Store LicensingLicensing

MonitoringMonitoringProvisioningProvisioning

DNSDNSContentContentDatabasex3

Databasex3

WebServer

x3

WebServer

x3

……

Separation of Infrastructure and Apps Concerns

25

Developers“Write code, not tickets”

Infrastructure Team“IT as a service provider”

FocusDeliver IT resources to rest of company

ChallengeAgility for devs, while maintaining control

RoleEnable rapid delivery of dev sandboxes, pre-provision

3rd Platform Services (Kubernetes, Pivotal CF, etc)

FocusFrictionless development, rapid innovation

ChallengeWrite code, without worrying about infrastructure details

RoleSelf-service access to new resources (i.e. new cluster),

comply with company policies and regulations

Separation of Infrastructure and Apps Concerns

Developers“Write code, not tickets”

Infrastructure Team“IT as a service provider”

FocusDeliver IT resources to rest of company

ChallengeAgility for devs, while maintaining control

RoleEnable rapid delivery of dev sandboxes, pre-provision

3rd Platform Services (Kubernetes, Pivotal CF, etc)

FocusFrictionless development, rapid innovation

ChallengeWrite code, without worrying about infrastructure details

RoleSelf-service access to new resources (i.e. new cluster),

comply with company policies and regulations

Architecturally, makes sense to separate infrastructure and app management

Infrastructure Management

Infrequent/no access by developers; devs

shouldn’t care

Application Management

Lightweight, fast; call infrastructure manager

when needed

26

Lifecycle: Self-service, Governance, Automation

27


Benefit: Common portal, catalog, permissions for developers and LOBSelf-Service

Benefit: Compliance consistently enforced across entire datacenterGovernance

Benefit: Same tools for automating traditional and new app lifecyclesAutomation

DBsTraditional Apps Containerized apps Tenant/LOB 1

Operations: Service Availability and Traceability

28

HardwareHardware

vSphere, NSX, vSAN/vVOL

OS

App

Virtual HW

OSVirtual HW

App

Lib

App

Lib

App

Lib

App

Lib

Performance monitoring

Performance monitoring

Capacity management

Capacity management

Log management

Log management…

Instrument all layers of stack

Inputs: Metricsand log data

Delivering better service levels,

availability, root cause analysis, …

Linux OS

App

Lib

App

Lib

App

Lib

Section 4: 3rd Platform Application Stack

29




The Rise of Third Platform Applications

30

On-premise


Tier 1/Converged HW

Classic NAS & SAN


Human-driven

On/Off premise






ManagementvCloud Automation Center,

vCenter Operations, Log Insight



ContainerRepositories


3rd Platform Apps Stack & DevOps Process

31

Container-optimized LinuxContainer-optimized Linux

Container PackagingContainer Packaging

Container Cluster SchedulerContainer Cluster Scheduler

App Definition, Policies, and ProvisioningApp Definition, Policies, and Provisioning

InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service


Developer Production

Dev’s LaptopDev’s Laptop

Optional: Type 2 HypervisorOptional: Type 2 Hypervisor

LinuxLinux

Developer ToolsDeveloper Tools


Open Ecosystem: 3rd Platform Developer Stack

32
















LinuxLinux



E.g. Hashicorp Vagrant, Jenkins, github, etc

E.g. Docker/Docker Hub

E.g. RedHat, Ubuntu, Boot2Docker

E.g. Fusion, Workstation, Player, VirtualBox






LinuxLinux



Open Ecosystem: 3rd Platform Production Stack

E.g. Pivotal CF, FigTerraform, Shipyard

E.g. Kubernetes, libswarm, Mesos, Fleet

E.g. CoreOS, Atomic, Ubuntu











E.g. Docker

33

Craig McluckieGoogle

Containers at Google

• Everything at Google runs in Linux application containers• A decade of production container experience• We start more than 2 billion a week

• Containers have changed the game • Separation of infra and applications ops • Increased efficiency

35

A few lessons learned...1: Declarative trumps imperative

Imperative: run this container on this serverDeclarative: run between 2 and 100 copies; keep latency < 2ms

Pros• Repeatable and eventually consistent deployment and update• Fire-and-forget app management (self scaling, self healing)• Dynamic scheduling yields better efficiency

Cons• Tracing action/reaction can be hard (“is it done?”)• Diagnostics can be tough (“what happened?”)

So• We need a cluster manager• Strong integration with container metrics, logging, etc helps

36

A few things we have learned...2: Prepare for more production services

The system known as Borg made it easier to run production services at scale...so our engineers wrote a lot more

Pros• Strong shift to dev and away from ops• Radically simpler infrastructure operations

But…• Governance gets harder as service number increases• Managing, finding, versioning

So…• We need a cluster manager• It needs mechanism to deal with large numbers of services

37

So we created Kubernetes...

• OSS project created by Google, but owned by the community

• Google style cluster management • Move from static containers to dynamic management

lightweight modular/ extensible portable

:38

And where do VMs fit in?

• Needed to run untrusted and unconstrained workloads • Linux syscall layer is large and difficult to defend• VMs surface can be aggressively defended• VMware has been doing this for 15 years• Critical for multi-tenant cloud use with untrusted tenants• E.g. VMware vCloud Air

• In Google Cloud Platform• VMs create ‘idealized’ infrastructure• Containers package and run applications

• Kubernetes stitched together VMs to create a mini-Google cluster

39

What is next?

• Make it work everywhere• Operationalize• Extend services for distributed systems development

40

James WattersPivotal

42© Copyright 2014 Pivotal. All rights reserved.

Core Application Patterns Are Changing


To Do List Application and Data Services Centric Platform

– Transform human centric data center processes into a software factory

Move towards real time deployment scaling and operations

Focus on ease of deployment, but deliver exceptional operational benefits


Why Containers are Essential Speed: seconds vs. minutes

– Seconds to deployment – Seconds to scaling – Seconds to network configuration – Seconds to health management

Units of currency– Leverage Docker popularity and simplicity for apps and data

services– Push and application artifact (.WAR) or a Docker image


Value of VMware Integration Being deeply integrated into vSphere APIs

allows automated platform set up and scaling

Mixed VM/container model ideal blend of speed and isolation

Enterprise customers leverage existing infrastructure operations process


Containers Alone Aren’t Enough

Enterprises do not want app development groups each writing their own platforms

Value is unlocked when standard common services are built into each managed container


From Data Center to Software FactoryContainers surrounded with services

Application Containerization & Cluster Scheduling

Native and Extended Data

Services

Automatic App Server & OS Configuration with

Buildpacks

Policy, Identity and Roles Management

App Health Management, Load Balancing, Rapid Scaling, Availability Zones

IaaS Provisioning, Scaling & Configuration

Application Network Security

Groups

Application to Services Binding and

Access

Logging as a service,Application metrics & performance,Metric based scaling


Already Strong in the Enterprise


Demo VideoDiego Sneak Peak

Summary


Case Study: ITBM Leveraging Containers on SDDC & vCloud Air

52

fleet & etcdfleet & etcd

ClusterCluster

MesosMesos

HDFS

Registry

ClusterCluster

DNS

Registry

IT Benchmarking Service (ITBM)SaaS application to measure IT process against peers or common recognized patterns• Build and content generation on private cloud (SDDC), Customer-facing modules on vCloud Air• All services running in Dockers on CoreOS VMs

vCloud Air


Case Study: ITBM Leveraging Containers on SDDC & vCloud Air

52

fleet & etcdfleet & etcd

ClusterCluster

MesosMesos

HDFS

Registry

ClusterCluster

DNS

Registry

IT Benchmarking Service (ITBM)SaaS application to measure IT process against peers or common recognized patterns• Build and content generation on private cloud (SDDC), Customer-facing modules on vCloud Air• All services running in Dockers on CoreOS VMs

vCloud Air


• Single platform for running and managing traditional + modern apps

• Enterprise grade: security, performance, operational efficiency

• Ability to extend applications to the hybrid cloud

• Support for community-led projects (Big Data, OpenStack, containers)

53

The Open Platform for Modern Applications

VMware Bridges These Two Worlds

ResilienceSecurity

QoS

ResilienceSecurity

QoS

OpennessPortability

Agility

OpennessPortability

Agility

Traditional Apps Modern Apps

OSOSApp

Virt. HWVirt. HWOSOSApp



Virt. HWVirt. HW

AppApp

Container OSContainer OS

AppApp

OpenStack API Open Container API


ESXi, NSX, VSAN

On-premise Off-premise

IT Faces Conflicting Demands

In Summary• VMware is focused on helping companies run and manage their applications,

whether they are packaged in VMs or containers

• A software-defined datacenter is the best place to run and manage all application types

• Docker, Google, Pivotal, VMware are working together to help companies efficiently run and operationalize containerized applications

55

Q & A

Unified Infrastructure Fabric(ex. ESXi, NSX, Software-Defined Storage)

Unified Infrastructure Fabric(ex. ESXi, NSX, Software-Defined Storage)

Unified Cloud Management(ex. vCloud Automation Center, vCenter Operations, Log Insight)

Unified Cloud Management(ex. vCloud Automation Center, vCenter Operations, Log Insight)

Extends to management of

containers running on physical hosts

2nd Platform App StackInfrastructure Control

• vSphere (VCenter/HA/DRS/…)• SRM (DR)• 3rd-party integrations w.

vSphere

3rd Platform App StackInfrastructure Control

• Kubernetes, Yarn, Mesos, …• Pivotal, BOSH

3rd Platform App StackInfrastructure Control

• Kubernetes, Yarn, Mesos, …• Pivotal, BOSH

2nd vs 3rd Platform Apps: The Value Prop Changes

57

Open Containers APIOpen Containers API2nd Platform App Stack Infrastructure

Control Plane• Scheduling &

placement (DRS)• Resource controls

(SIOC, NIOC)• High availability (HA,

FT) • Mobility (vMotion)• Disaster recovery

(SRM)• Authentication• Logging/Audit• Etc…

3rd Platform App Stack

Infrastructure Control• Scheduling• Resource Controls• Load balancing,

routing• Service registration• Service discovery• Availability• Authentication• Logging/auditing• Data persistence

Thank You

Fill out a surveyEvery completed survey is entered

into a drawing for a $25 VMware company store gift certificate

The Software-defined Datacenter, VMs, and Containers: A “Better Together” Story

SDDC3327

Kit Colbert, VMware, Inc

Documents

The Software-defined Datacenter, VMs, and Containers: A ......Containers & VMware NSX • Unified operational model for VMs & containers • Programmable, datacenter-wide connectivity