Tectonic Summit 2016: Brandon Philips, CTO of CoreOS, Keynote

Brandon Philips@brandonphilips | [email protected] | coreos.com

Good Morning!

Experts at Every Layer of the Stack

Linux

Container Engines & Runtime Specs

Image Specs, Build, & Hosting

Clustered Database

Cloud Independence & Lifecycle

Identity & Federation


Linux


Container Image Build, Hosting, & Specs

Clustered Database




Linux



Clustered Database



The shared foundation of thisecosystem is the container

And CoreOS is ensuring that the shared foundation is built on standards

Open Container Initiative

OCI AnnouncedJune 2015

OCI 1.0Q1 2017

rkt OCI supportJuly 2016

OCI Image Spec AddedApril 2016

Quay, Kubernetes, etcQ2 2017

OCI 1.0 RC-1July 2016

Create developer collaboration

Build interoperating products

Confidence in ecosystem stability

Investment in standards

An update aboutthe pod native container engine

rkt community traction

● Laptop Kubernetes, minikube, can use rkt with a single flag

● BlaBlaCar (Series D, $350m) rkt in prod and moving to Kubernetes

● Container Linux services now run under rkt

● Google GKE using rkt for Kubelet mount management

Kubernetes & rkt integration via CRI

Support all OCI standards as they reach 1.0

Continue innovation in design and security

Roadmap for rkt




Roadmap for rkt

Quick Reminder: Pod Basics

cache(pid 5)

asset fetcher(pid 8)

web server(pid 9)

pod sandbox

Quick Reminder: Pod Lifecycle

wor

ker

node

sco

ntro

llers

no

des

EC2 VM EC2 VM EC2 VM




A1

KubernetesScheduler

KubeAPI

Monitoring Service

wor

ker

node

sco

ntro

llers

no

des


A1

KubernetesScheduler

KubeAPI

Monitoring Service

wor

ker

node

sco

ntro

llers

no

des


A1

KubernetesScheduler

KubeAPI

Monitoring Service

J2

wor

ker

node

sco

ntro

llers

no

des


A1

KubernetesScheduler

KubeAPI

Monitoring Service

J2

wor

ker

node

sco

ntro

llers

no

des

Container Runtime Interface

cache(pid 5)


web server(pid 9)

pod sandbox


cache(pid 5)


web server(pid 9)

pod sandbox


cache(pid 5)


web server(pid 9)

pod sandboxHealth Check Fail


cache(pid 5)


pod sandbox


cache(pid 5)


pod sandbox

web server(pid 10)

rkt and CRI will help enable faster innovation in Kubernetes in 2017.




Roadmap for rkt

rkt and runc

cache(pid 5)


web server(pid 8)

runc runc runc

pod sandbox




Roadmap for rkt

rkt is the only container engine with both Linux native and VM isolation.

rkt is the only container engine with both Linux native and VM isolation.

We continue to explore new ideas.

Normal rkt execution

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

pod sandbox

cache(pid 10)

debug agent(pid 38)

web server(pid 20)

pod sandbox

VM rkt execution

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

pod sandbox

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

pod sandbox

bash(uid 1001, pid 8)

Lifecycle of a process


fork()identical perms

su(uid 0, pid 9)

exec() setuid binaryelevate perms

bash(uid 0, pid 9)

exec()identical perms

Normal Execution Path


Lifecycle of a process


fork()identical perms

bash(uid 0, pid 9)

open() kernel exploitelevate perms

Exploit Execution Path

Container Terminated

VM rkt execution

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

kvm virtual machine

Privilege EscalationValidator

pod sandboxCan PID 8

open /proc/9/environ it

is uid 0?

VM rkt execution

Yes, valid elevation to uid 0

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

kvm virtual machine


pod sandbox

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

kvm virtual machine

VM rkt execution

rootkit payload


pod sandbox

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

kvm virtual machine

VM rkt execution

rootkit payload


pod sandboxCan PID 9

open /etc/shadow it is

uid 0?

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

kvm virtual machine

VM rkt execution

rootkit payload

No, invalid transition to uid 0


pod sandbox

cache(pid 5)

debug agent(pid 8)

web server(pid 9)

kvm virtual machine

pod sandbox

VM rkt execution


Container Terminated

Tectonic will support users withDocker Engine or rkt engine.

End-to-end.

Kubernetes scales.And we have worked end-to-end

to make it happen

● Clients talk to Kubernetes API server

● API is stateless and horizontally scales

● State from API persisted to etcd DB

Quick Reminder: Kubernetes Architecture

Brandon Philips

Assigning to [email protected]_Reassigned to Rob Szumski_

Meghan Schofield

Sorry! I just made all the little icon files. You want the penguins replaced with our new CL icon, yes?

Brandon Philips

yes!

Brandon Philips

[email protected] would be great to update these diagrams once the container linux logo is settled._Assigned to Meghan Schofield_

Meghan Schofield

Got it. Done!

● etcd introduced in 2013 by CoreOS

● Persistent database of Kubernetes

● Auto-leader election for availability

etcd Overview

Brandon Philips

[email protected]_Assigned to Rob Szumski_

etcd is the foundation of Kubernetes

CoreOS ensures it isscalable, simple, solid


Scaling Milestones of Kubernetes

100 Nodes300 PodsJune 2015

2,000 Nodes60,000 Pods

November 2016


March 2016

5,000 Nodes150,000 PodsDecember 2016

● Google Chubby

● etcd by CoreOS

● ZooKeeper by Apache

● Consul by Hashicorp

Consistent Key-Value Database

● Google Chubby (closed source)

1.etcd by CoreOS

2.ZooKeeper by Apache

3.Consul by Hashicorp

Consistent Key-Value Database, Benchmark

Memory, key to scalability

Latency, key to reliability

Latency, key to reliability

etcd's delivers consistent latency

Scaling Milestones of Kubernetes


November 2016


March 2016

5,000 Nodes150,000 PodsDecember 2016

20,000 Nodes600,000 Pods

2017



etcd Operator

etcd Operator

etcd Operator

etcd Operator



etcd is Trusted by 100s of OSS Projects

Google. Amazon. Microsoft.

etcd is Trusted by 100s of OSS Projects

Including Projects From Teams At

Self-driving architecture simplifies Kubernetes.

Brandon Philips

[email protected] it would be great on this slide if we could add like a "tron" or underbody to the self-driving car. To tie back to alex.

Brandon Philips

The idea just to tie things together.

$ uname -s minix$ gcc linux.c

$ uname -s minix$ gcc linux.c

$ uname -s linux$ gcc linux.c

$ uname -s linux$ gcc linux.c

Self-Hosted Architecture

wor

ker

node

sco

ntro

llers

no

des





KubernetesScheduler

KubeAPI

Monitoring Service

cont

rolle

rs

node

s

A1

J2

wor

ker

node

s


KubernetesScheduler

KubeAPI MS

cont

rolle

rs

node

s

A1

J2

wor

ker

node

s

KS


KubernetesScheduler

KubeAPI MS

cont

rolle

rs

node

s

A1

J2

wor

ker

node

s

KS

Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows.

Self-Driving Removes Toil

CHECK

But... Failures Still Happen

Self-Driving Removes Toil

Self-Driving Monitoring Architecture

KubernetesScheduler

KubeAPI

Monitoring Service

cont

rolle

rs

node

s

Self-Driving Monitoring Architecture

KubernetesScheduler

KubeAPI

Monitoring Service

cont

rolle

rs

node

s

"Self-hosted" is being adopted in the Kubernetes community.

Kubernetes User Identity




OpenID Connect (OIDC) provider with LDAP plugin.

Integrated into upstream Kubernetes.

No external databases, simply use the Kubernetes API.

Default in Tectonic.

CoreOS is ensuring that the shared foundation is built on standards

rkt will help enable faster innovation in Kubernetes in 2017.

Kubernetes scales.And we have worked end-to-end

to make it happen.

Self-driving architecture simplifies and removes toil.


Linux



Clustered Database



THANK YOU!!

@brandonphilips | [email protected] | coreos.com

Technology

Tectonic Summit 2016: Brandon Philips, CTO of CoreOS, Keynote