Upload
isaac-johnston
View
240
Download
0
Tags:
Embed Size (px)
Citation preview
1st Solution: Ubuntu Server
Internet is full of
servers running years-
old software with
dozens of
vulnerabilities
(Shellshock,
Heartbleed etc)
2nd Solution: Ubuntu + Docker
● Isolation of services
● Rapid service
deployment
● Portable
● Version control
● Reuse
● Lightweight & fast
CoreUpdate
● Make upgrading the default, seamless optiono Regular
o Reliable
o Automatic
● The “Google Chrome” of server operating
systems
CoreUpdate: How does it work ?
● Omaha protcol (check-in/retrieval)o Simple XML-over-HTTP protocol developed by
Google to facilitate polling and pulling updates from
a server
● Active/passive read-only rootfs partitionso One for running live system, one for updates
CoreUpdate: Active/passive rootfs /usr
● /usr is a single image containing most of the
OSo mounted read-only
o rootfs / is mounted read-write on top (persistent
data)
o parts of /etc dynamically generated at boot
o a lot of work moving default config from /etc to /usr
Atomic Updates
● Entire OS is a single read-only imageo e.g. “touch /usr/bin/foo”
● Easy to verify cryptographically
● No chance of inconsistencies due to partial
upgrades
Cloud Config
Automate customizing a CoreOS machine
through user-data.
● YAML
● configure etcd, fleet...
● e.g. EC2 user-data
Etcd
● key-value store
● basis foro shared configuration
o service discovery
o distributed locking
● REST API
● Raft algorithm: raftconsensus.github.io
systemd is great!
● Automatically handles:o Process daemonization
o Service inter-dependencies
o Resource isolation/containment (cgroups)
e.g. MemoryLimit=512M
o Health-checking, restarting failed services
o Logging (journal)
applications can just write to stdout, systemd
adds metadata
o Timers, socket activation...
systemd (and docker) are… not so great
● Problem: poor integration with Dockero Docker is de-facto standard
o Docker and systemd do not always place nice
together
o Both Docker and systemd manage cgroups and
processes, so when the two are trying to manage
the same thing the results are mixed
Example: sending signals to a container
Given a simple container...
[Service]
ExecStart=/usr/bin/docker run busybox /bin/bash -c “while true; do echo Hello
World; sleep 1; done”
● Try to kill it with systemctl kill hello.service
● … Nothing happens
● Kill command sends SIGTERM, but bash in a Docker container has PID1,
which happily ignores the signal...
Example: sending signals to a container
● OK, SIGTERM didn’t work, so escalate to SIGKILL:
systemctl kill -s SIGKILL hello.service
● Now the systemd service is gone:
hello.service main process exited, code=killed, status=9/KILL
● But… the Docker container still exists ?
# docker ps
CONTAINER ID COMMAND STATUS NAMES
feb755848a9a /bin/sh -c ‘whi Up 31 seconds hello
# ps -ef|grep ‘[d]ocker run’
root 24231 1 0 03:49 ? 00:00:00 /usr/bin/docker run -name hello….
Why ?
● Docker client does not run containers itself; it
just sends a command to the Docker
daemon
● systemd expects processes to fork directly
● Since Docker daemon’s cgroup is entirely
seperate systemd cannot keep track of the
forked container
Solution: systemd-docker
● systemd-docker - small app that moves
cgroups of Docker containers back under
systemd’s cgroup
● Uses Docker for images, but systemd-
nspawn for runtime
● (proposed) Docker standalone mode: client
starts container directly rather than via
daemon
Fleet + systemd
● systemd takes care of things so fleet doesn’t
have to
● fleet configuration is just systemd unit files
● fleet extends systemd to the cluster-level,
and adds some features of its own (using [X-
Fleet])o Template units (run n identical copies of a unit)
Fleet Scheduling
● Global (run a unit everywhere in the cluster)
● MachineMetadata (run only on certain
machines)
● Conflicts (prevent a unit from being
collocated)
● MachineOf (run only on a machine that hosts
a specific unit)
Fleet + etcd
● fleet needs a consistent view of the cluster to
make scheduling decisions: etcd provides
this viewo What units exist in the cluster ?
o What machines exist in the cluster ?
o What are their current states ?
● All unit files, unit state, machine state and
scheduling information is stored in etcd
Fleet Reconciler Model
In a loop, run periodically until stopped:
1. Retrieve current state (how the world is) and
desired state (how the world should be) from
etcd
2. Determine necessary actions to transform
current state -> desired state
Flannel
Intra-machine Docker overlay network backed
by etcd.
● one subnet per machine
● get rid of Docker port mapping
● containers talk to each other directly
● pluggable transport backends:o encapsulate packets in UDP
o VXLAN implementation coming
● Everything is open source under Apache 2.0 - join us!
○ https://github.com/joukou
○ https://github.com/coreos
● Any more questions, feel free to
○ email [email protected]
○ or tweet @superstructor