The New Stack Container Summit Talk

The EVOLVING Container Ecosystem

1

EVOLVINGthe

CONTAINER ECOSYSTEM

2

Docker & Container eBook Series

www.thenewstack.io/ebookseries

What Now?

4

Who We Interviewed

Respondents inFeb 21 - Mar 14

survey

309

61%of respondents are

end users

49%of end users are

DevOpsEnd users

using and/ortrialing containers

164

What We Found

5

Agreement: Service Discovery, Scheduling, Cluster Management

Configuration Management

CaaS: more consensus than expected

Orchestration Platforms

Products, CaaS & PaaS

Open source tools for specific functionality

MarketDefinition

Product Choices

SelectionCriteria

Not looking for one solution for everything

Integrated tooling more important than

interoperability with systems/ languages

Mega Theme: Automation

Not As Much About Docker Anymore

7

Who is a Network Admin?

Role of the Systems Administrator?

AbstractionsConfusion

IP and Ethernet

Pods

Frameworks

Connected Systems

Flexibility

Identity Constructs Open SourceCNCF

OCI

No way to build scalable systems without open

source ecosystems

2016 Coverage

8

Topics Themes

Container and Container Orchestration Networking, Storage, Security, Monitoring

Serverless /Stateless Architectures AWS, IBM, Microsoft, Google

Languages, Frameworks, IDEs, etc. Go/Node.js, Continuous Delivery, Pipelines

Enterprise Platforms OpenStack, Cloud Foundry

Data Platforms NoSQL, Stream Processing, Machine Learning, AI

API Management Internal Lifecycle Management

Open Source Ecosystems Multi-Clouds, Programmable Infrastructure, IoT

Container Ecosystems

Orchestration is the primary managing method for 45% of those using or trialing containers.

10

11

Scheduling, cluster management & service discovery considered to be container orchestration

12

Tools that can be used by both developers andIT ops are the most important criteria.

13

Kubernetes, Ansible, Mesos/Mesosphere,Amazon ECS & Docker Swarm top users’ plans.

14

Supporting long-running applicationsand load balancing capabilities are important.

Networking and Storage

Project/Product

Company/Sponsor Project Description

Camel Apache Software Foundation

Helps define routing & mediation rules in variety of domain-specific languages, including Java-based fluent API & Scala DSL.

Clocker Apache Software Foundation

Creates & manages Docker cloud infrastructure; part of Apache Brooklyn projects; has plugins for Project Calico & Weave.

Contiv Cisco Open source project defining infrastructure operational policies for container-based application deployment.

Flannel CoreOS Virtual network for hosting containers.

HAProxy N/A Realtime processing pipeline.

Kuryr OpenStack Foundation

Gives OpenStack Neutron a plugin for networking services to Docker containers, and containerized images for Neutron plugins.

libnetwork Docker Native Go implementation for connecting containers.

Project Calico Metaswitch Networks

Networking for OpenStack VMs & containers in Docker environment. Each container gets its own IP and security policy.

Open Source Tools Associated with Networkingand Containers (1 of 2)

16

17

Project/Product


Project Calico Metaswitch Networks

Networking for OpenStack VMs & containers in Docker environment. Each container gets its own IP and security policy.

Tengine Alibaba Group Distribution of NGINX with some advanced features.

Traefik N/A Modern HTTP reverse proxy, load balancer to deploy microservices.

Vulcand N/A A programmatic load balancer backed by Etcd.

Weave Weaveworks Container-specific implementation of SDN across data centers.

Open Source Tools Associated with Networkingand Containers (2 of 2)

Open Source Tools Associated with Storageand Containers (1 of 2)

Project/Product


Crate Crate.io Uses SQL syntax for distributed queries across a cluster.

Elliptics ReverbrainFault tolerant, distributed key-value storage that allows distributed storage of medium & large objects with streaming support.

Hazelcast Hazelcast Open source, in-memory data grid that distributes data & computation across servers, clusters & geographies.

linkerd BuoyantOut-of-process network stack. Functions as transparent RPC proxy. Includes load-balancing, service discovery, instrumentation, routing. It’s built on Finagle.

Manta Joyent HTTP-based object store that uses OS containers to allow running arbitrary compute on data at reset.

Minio Minio Open source object storage server.

Pachyderm Pachyderm Enables storage & analysis of data using containers.

18

Open Source Tools Associated with Storageand Containers (2 of 2)

19

Project/Product


Pachyderm Pachyderm Enables storage & analysis of data using containers.

Reborn N/A Reborn is a next-generation, distributed key-value store.

REX-Ray EMCLayer between storage & container platforms. Admin/orchestration of storage platforms performed w/same set of commands.

Tachyon N/A Memory-centric, distributed storage system, enabling reliable data sharing at memory-speed across cluster frameworks.

Containers as a Service

21

Container orchestration and registries are most associated with Containers as a Service.

22

CaaS along with the likes of Swarm, Kubernetes & Mesos, compete to win over IT Ops.

Comparing Platforms and Services (1 of 2)

Type IaaS CaaS/Container Services PaaS

Description Virtualized compute resources (CPU, RAM, storage, network, etc.) that must be assembled & ready for app deployment.

Provides an abstraction layer over compute resources that allows the deployment and orchestration of containers.

Provides abstraction layer over compute infrastructure that allows deployment and orchestration of applications.

Example Amazon Web Services, Azure, DigitalOcean, Google Cloud Platform, IBM Cloud Infrastructure, VMware

CoreOS Tectonic, Docker Cloud, Google Container Engine (GKE), HashiCorp Nomad, IBM Container Service, Joyent Triton, Mesosphere DCOS

AWS Elastic Beanstalk, Deis, Google App Engine, Heroku, IBM Bluemix, Pivotal Cloud Platform, Red Hat OpenShift

Application Scheduling

Manual or via config mgnt. Resource requirements generally specified by infrastructure selection & config

Container scheduling typically baked-in to platform, and capabilities range from crude to sophisticated. Resource requirements are specified declaratively.

Baked-in to platform. Resource requirements are specified declaratively.

Service Discovery

Manual or via CM tooling like Puppet, Chef or Ansible. CM combined with distributed KV store like Consul or ZooKeeper

Varies from procedural specs (e.g. with Mesos, Registrator & srv_router) to declarative specification (e.g., with services & labels in Kubernetes).

Typically baked-in to the platform, for example, service discovery provided “as a Service.”

23

Comparing Platforms and Services (2 of 2)

24

Type IaaS CaaS/Container Services PaaS

Service Discovery

Manual or via CM tooling like Puppet, Chef or Ansible. CM combined with distributed KV store like Consul or ZooKeeper

Varies from procedural specs (e.g. with Mesos, Registrator & srv_router) to declarative specification (e.g., with services & labels in Kubernetes).

Typically baked-in to the platform, for example, service discovery provided “as a Service.”

Resource allocation (provisioning)

Manual or via CM tooling like Puppet, Chef or Ansible. Operator decides on resource allocation & distribution for fault tolerance & approach to scaling.

Typically baked-in to platform & algorithms like bin-packing or spread available to maximize utilization or fault tolerance. Scaling of container instances implemented via platform.

Hidden from users. Costs typically based on resource allocation/ consumption.

Typical Use Case

Cloud migrations & migrating on-premises VMs & apps like-for-like, or requiring more fine-grained control over resource assembly & allocation.

Already producing cloud-native applications, keen to move away from the operational complexity of IaaS, but not wanting to embrace an opinionated PaaS ecosystem.

Startups looking to cut the costs of maintaining platform infrastructure; large-scale organizations seeking standardization & developer productivity.

Many use the the term CaaS, but not all (1 of 2)

Project/Product


Alauda Alauda Cloud platform that provides CaaS, cloud hosting, image registry.

Azure Container Service Microsoft

Simplifies the creation and configuration of a cluster. The default configuration includes Docker & Docker Swarm for code portability & Marathon, Chronos & Apache Mesos for scalability.

Bluemix IBMPlatform to develop/deploy/manage/run cloud apps. Utilizes Docker containers, VMs & Cloud Foundry w/ open APIs. Catalog of 150+ services w/ for microservices, logging/monitoring & Watson.

Carina Rackspace Zero infrastructure hosted container environment that lets users ignore building, managing & updating container environments.

Docker Cloud DockerNew cloud service by Docker that expands on features of Tutum & brings tighter integration with Docker Hub. It’s a SaaS service for deploying & managing Dockerized applications.

Docker Datacenter Docker

On-premises container management & deployment services for enterprises with production-ready platform supported by Docker & hosted locally behind firewall.

25

Many use the the term CaaS, but not all (2 of 2)

26

Project/Product


Docker Datacenter Docker

On-premises container management & deployment services for enterprises with production-ready platform supported by Docker & hosted locally behind firewall.

EC2 Container Service AWS Amazon EC2 Container Service helps companies manage clusters

of containers on AWS infrastructure.

Heroku Buildpacks Salesforce PaaS providing "buildpacks" for compiling applications.

Customers pay for managed containers called "Dynos."

Joyent Triton Elastic Container Service

Joyent Securely deploy & operate containers with bare metal speed on container-native infrastructure. Provided as part of IaaS offering.

Rancher Rancher Complete infrastructure platform for running containers in production.

Container hosting can be seen as a cousin of IaaS (1 of 2)

27

Project/Product


Apprenda Apprenda PaaS for enterprises that supports hosting of containers.

ContainerShip ContainerShipSelf-hosted container management platform, capable of running on any cloud & used to manage containers from development to production.

DigitalOcean DigitalOcean IaaS provider that targets developers, with "one-click installs" to deploy Docker.

Giant Swarm Giant Swarm Hosted container solution to build, deploy & manage containerized services.

Kyup Cloud Hosting Kyup Scalable cloud-based container hosting on Linux containers.

OpenShift Dedicated Red Hat PaaS built on Docker containers that orchestrates w/

Kubernetes. It also has Atomic and Red Hat Linux components.

Container hosting can be seen as a cousin of IaaS (2 of 2)

28

Project/Product


OpenShift Dedicated Red Hat PaaS built on Docker containers that orchestrates w/

Kubernetes. It also has Atomic and Red Hat Linux components.

Project Atomic Red HatRuns apps in Docker containers w/ components based on RHEL, Fedora & CentOS. The project includes Nulecule, a container- based app spec to use existing containers for new apps.

sloppy sloppy.io Cloud hosting for containers.

SoftLayer IBMIaa including bare metal, virtual servers, networking, turnkey big data solutions & private cloud. It is supported as a provider behind Docker Machine to stand up a cloud-hosted Docker host.

Blurred lines: What is a Service? (1 of 2)

Project/Product


Apcera Platform Apcera

Manages access to compute resources across a cluster of servers. Focuses on managing policies across environments, it secures workloads & containers in enterprise production environments.

Built.io Flow Built.io An Integration Platform-as-a-Service for automating processes, connecting data and devices, and integrating w/ various services.

Cloudsoft App Management Platform

CloudsoftCloudsoft's app management platform, based on the open source Apache Brooklyn project, orchestrates services, platforms and infrastructure, including deployment to containers.

Deis Engine Yard A lightweight, open source PaaS, built on Docker & CoreOS, that makes it easy to deploy & manage apps on your own servers.

Jelastic Jelastic Jelastic provides a PaaS and container-based IaaS on a singular platform that includes container orchestration.

Mantl Cisco Open source platform for deploying globally distributed services. It works w/ tools such as Marathon, Mesos, Docker & Consul.

OpenShift Origin Red Hat The upstream open source version of OpenShift and is meant to

allow for development of cloud-native applications. 29

Blurred lines: What is a Service? (2 of 2)

30

Project/Product


Mantl Cisco Open source platform for deploying globally distributed services. It works w/ tools such as Marathon, Mesos, Docker & Consul.

OpenShift Origin Red Hat The upstream open source version of OpenShift and is meant to

allow for development of cloud-native applications.

Packet Packet Host Packet is a bare metal PaaS that supports Docker, CoreOS, Deis, Mesosphere and Rancher.

Pivotal Cloud Foundry Pivotal A cloud-native enterprise PaaS based on Cloud Foundry. It offers

support for building applications as deployable containers.

PureApplication IBM

Solutions to deploy apps w/ patterns to deploy apps. Containers can be included in patterns w/ non-container components. Support includes a private Docker registry deployable as a shared service.

Scalingo Scalingo A PaaS for containers; users push code to Scalingo, which creates an image & allocates resources to run the application in its cloud.

Takeaways

32

Contact Information

Founder & Editor in ChiefAlex Williams

@[email protected]

503-473-6237

General [email protected]

Technology

The New Stack Container Summit Talk