26
LXD Live Migration of Linux Containers Tycho Andersen, Canonical Ltd. [email protected] http://tycho.ws

Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. [email protected] Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

  • Upload
    buidat

  • View
    227

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

LXDLive Migration of Linux Containers

Tycho Andersen, Canonical Ltd.

[email protected]://tycho.ws

Page 2: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Sorry, no demos today :(

Page 3: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Who is this guy?

➔ LXD➔ LXC➔ CRIU➔ Kernel

Page 4: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

LXC 0.9

The history of LXCOver 6 years of Linux system containers

2008 20152009 2010 2011 2012 2013 2014

Beginning ofthe LXC project

LXC 1.0

LXC 1.1LXC 0.8

LXC 0.1

LXC 0.2

LXC 0.5

LXC 0.3LXC 0.4

LXC 0.6

LXC 0.7

Major project rework

Page 5: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

LXD: the container lighter-visorWhat it IS

➔ SimpleClean command line interface, simple REST API and clear terminology.

➔ FastNo virtualization overhead so as fast as bare metal.

➔ SecureSafe by default. Combines all available kernel security features.

➔ ScalableFrom a single container on a developer’s laptop to thousands of containers per host in a datacenter.

Page 6: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

LXD: the container lighter-visorWhat it ISN’T

➔ Another virtualization technologyLXD tries to offer as similar a user experience as that of a virtual machine but it doesn’t itself virtualize anything, you always get access to the real hardware and the real native performance.

➔ A fork of LXCLXD uses LXC’s API to manage the containers behind the scene.

➔ Another application container managerLXD only cares about full system containers and doesn’t care about what runs inside the container.

Page 7: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Hypvervisor-y things

Page 8: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

LXD: the container lighter-visor

Host A

nova-compute-lxd lxc (command line tool)

LXD

Linux kernel

LXC

LXD REST API

your own client/script ?

Host B

LXD

Linux kernel

LXC

Host C

LXD

Linux kernel

LXC

Host D

LXD

Linux kernel

LXC

Host ...

LXD

Linux kernel

LXC

Page 9: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Hypvervisor-y things

Page 10: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

lxc move host1:c1 host2:

Page 11: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

lxc move host1:c1 host2:

➔ host1 negotiates three “channels” with host2◆ control◆ filesystem◆ container process state

➔ Using a tool called CRIU for process state➔ host1 captures memory state using CRIU➔ host2 restores memory state using CRIU

Page 12: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

The history of CRIUFive years of checkpointing!

2006 ... 2011 2012 2013 2014 2015

C/R in OpenVZ Kernel

Attempts to merge OpenVZ in-kernel upstream

First discussion on lkml about C/R from Userspace

First kernel patches merged

CRIU 0.1

Memory page tracking patches merged upstream

LXC gets support for C/R via CRIU

libcontainer gets support for C/R via CRIU

Page 13: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

What’s the catch?

“A note on this: this is a project by various mad Russians to perform c/r mainly from userspace, with various oddball helper code added into the kernel where the need is demonstrated… However I'm less confident than the developers that it will all eventually work!”

- Linus Torvalds (kernel commit 09946950)

Page 14: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

What’s the catch?

“This is not an enterprise feature. It's a promise one cannot keep. We will not add code to systemd that works often but not always, and CRIU is certainly of that kind.”

- Lennart Pottering (systemd-devel, 2015)

Page 15: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Always playing catch-up

Page 16: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Plug-ins needed for custom /dev

Page 17: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Not all kernel features supported

Page 18: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Security!?!!111?1

Page 19: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Security

Secure Migratable

Page 20: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Security

Secure Migratable

Page 21: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Security

Secure & Migratable

Page 22: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Security

➔ cgroups➔ apparmor, selinux, etc.➔ seccomp (STRICT, FILTER)➔ user namespaces

Page 23: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Correct and Fast

Pick two

Page 24: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Making Migration Fast

➔ Three channels◆ control◆ filesystem specific◆ memory state specific

➔ Filesystems:◆ btrfs, LVM, ZFS, (swift, nfs?), etc.◆ rsync between incompatible hosts

➔ Memory state:◆ Stop the world◆ Iterative incremental transfer (via p.haul)

Page 25: Live Migration of Linux Containers Migration of Linux Containers Tycho Andersen, Canonical Ltd. tycho.andersen@canonical.com Sorry, no demos today :(Who is this guy? LXD LXC CRIU Kernel

Administrivia

➔ LXD◆ Current stable release 0.16◆ 1.0 targeted for February 2016◆ Two week release cadence◆ (at least) One more release before Wily◆ https://linuxcontainers.org◆ https://github.com/lxc/lxd

➔ CRIU◆ Current stable release 1.6.2◆ Three month release cycle◆ http://criu.org◆ https://github.com/xemul/criu