Docker Swarm + Machine

Docker - Container ClusteringEueung Mulyana

http://eueung.github.io/docker-stuff/swarmCodeLabs | Attribution-ShareAlike CC BY-SA

1 / 31

Outline

Docker Engine - Revisited

Docker Machine

Docker Swarm

2 / 31

Previously Addressed

Docker Engine

3 / 31

Docker EngineWhen people say "Docker" they typically mean Docker Engine,the client-server application made up of the Docker daemon, aREST API that speci�es interfaces for interacting with thedaemon, and a command line interface (CLI) client that talks tothe daemon (through the REST API wrapper).

Docker Engine accepts docker commands from the CLI, such asdocker run <image>, docker ps to list runningcontainers, docker images to list images, and so on.

Engine is the core of Docker and nothing else will run without it.

Ref: docker.com

4 / 31

Docker Architecture

5 / 31

Docker daemonThe Docker daemon runs on a host machine. The user does notdirectly interact with the daemon, but instead through theDocker client.

Docker clientThe Docker client, in the form of the docker binary, is theprimary user interface to Docker.

It accepts commands from the user and communicates back andforth with a Docker daemon.

Ref: docker.com

6 / 31

Docker ArchitectureDocker uses a client-server architecture. The Docker client talksto the Docker daemon, which does the heavy lifting of building,running, and distributing your Docker containers.

Both the Docker client and the daemon can run on the samesystem, or you can connect a Docker client to a remote Dockerdaemon.

The Docker client and daemon communicate via sockets orthrough a RESTful API.

7 / 31

Docker Machine

8 / 31

Docker Machine is a tool for provisioning and managing your Dockerized hosts (hosts withDocker Engine on them). Typically, you install Docker Machine on your local system. DockerMachine has its own command line client docker-machine and the Docker Engineclient, docker.

You can use Machine to install Docker Engine on one or more virtual systems. These virtualsystems can be local (as when you use Machine to install and run Docker Engine inVirtualBox on Mac or Windows) or remote (as when you use Machine to provisionDockerized hosts on cloud providers).

The Dockerized hosts themselves can be thought of, and are sometimes referred to as,managed "machines".

Using docker-machine commands, you can start, inspect, stop, and restart a managedhost, upgrade the Docker client and daemon, and con�gure a Docker client to talk to yourhost.

Point the Machine CLI at a running, managed host, and you can run docker commandsdirectly on that host. For example, run docker-machine env default to point to ahost called default, follow on-screen instructions to complete env setup, and rundocker ps, docker run hello-world, and so forth.

9 / 31

Docker Machine vs.Docker Engine

Ref: Machine Overview

Docker Machine

10 / 31

Install and run Dockeron Mac or Windows

Provision and managemultiple remote

Docker hosts

Provision Swarmclusters

Let's Try It ...

11 / 31

$ curl -L https://github.com/docker/machine/releases/download/v0.6.0/docker-machine-̀uname $ chmod +x /usr/local/bin/docker-machine

$ docker-machine versiondocker-machine version 0.7.0, build a650a40

$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS

Install

12 / 31

Create aMachine

$ docker-machine create --driver virtualbox defaultCreating CA: /home/em/.docker/machine/certs/ca.pemCreating client certificate: /home/em/.docker/machine/certs/cert.pemRunning pre-create checks...(default) Image cache directory does not exist, creating it at /home/em/.docker/machine/cache...(default) No default Boot2Docker ISO found locally, downloading the latest release...(default) Latest release for github.com/boot2docker/boot2docker is v1.11.1(default) Downloading /home/em/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v1.(default) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%Creating machine...(default) Copying /home/em/.docker/machine/cache/boot2docker.iso to /home/em/.docker/machine/machines/default/boot2docker.iso...(default) Creating VirtualBox VM... (default) Creating SSH key... (default) Starting the VM...(default) Check network to re-create if needed...(default) Found a new host-only adapter: "vboxnet1"(default) Waiting for an IP...Waiting for machine to be running, this may take a few minutes...Detecting operating system of created instance...Waiting for SSH to be available...Detecting the provisioner...Provisioning with boot2docker...Copying certs to the local machine directory...Copying certs to the remote machine...Setting Docker configuration on the remote daemon...Checking connection to Docker...Docker is up and running!To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env default

$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORSdefault - virtualbox Running tcp://192.168.99.100:2376 v1.11.1

13 / 31

Connect

$ docker-machine env defaultexport DOCKER_TLS_VERIFY="1"export DOCKER_HOST="tcp://192.168.99.100:2376"export DOCKER_CERT_PATH="/home/em/.docker/machine/machines/default"export DOCKER_MACHINE_NAME="default"# Run this command to configure your shell: # eval $(docker-machine env default)

$ eval $(docker-machine env default)$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZE

$ env|grep DOCKERDOCKER_HOST=tcp://192.168.99.100:2376DOCKER_MACHINE_NAME=defaultDOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=/home/em/.docker/machine/machines/default

$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORSdefault * virtualbox Running tcp://192.168.99.100:2376 v1.11.1

14 / 31

Work

$ docker run busybox echo hello worldUnable to find image 'busybox:latest' locallylatest: Pulling from library/busybox ... 385e281300cc: Pull complete ... a3ed95caeb02: Pull complete Digest: sha256:4a887a2326ec9e0fa90cce7b4764b0e627b5d6afcb81a3f73c85dc29cea00048Status: Downloaded newer image for busybox:latesthello world

$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEbusybox latest 47bcc53f74dc 6 weeks ago 1.113

$ docker-machine ip default192.168.99.100

$ docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES0fb8afca05ea busybox "echo hello world" 2 minutes ago Exited (

$ docker run -d -p 8000:80 nginxUnable to find image 'nginx:latest' locallylatest: Pulling from library/nginxefd26ecc9548: Pull complete ... 8ddc2d7beb91: Pull complete Digest: sha256:2ca2638e55319b7bc0c7d028209ea69b1368e95b01383e66dfe7e4f43780926dStatus: Downloaded newer image for nginx:latesta8eb8f257cc1a747ff5bd30c056fcf5fe173de8fb0781265eed595ab7cd69b40

$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESa8eb8f257cc1 nginx "nginx -g 'daemon off" 10 seconds ago Up

15 / 31

16 / 31

Work

# or$ curl $(docker-machine ip default):8000

$ docker-machine stopStopping "default"...Machine "default" was stopped.# without explicit name -> default

$ env | grep DOCKERDOCKER_HOST=tcp://192.168.99.100:2376DOCKER_MACHINE_NAME=defaultDOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=/home/em/.docker/machine/machines/default

$ docker imagesAn error occurred trying to connect: Get https://192.168.99.100:2376/v1.23/images/json: dial tcp

$ eval $(docker-machine env -u)$ env | grep DOCKER$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEcomposetest_web latest e7d62ba30c20 4 days ago web latest d6f25a9bf632 4 days ago redis latest 9a450ae418d8 4 days ago

17 / 31

Docker Swarm

18 / 31

Docker Swarm is native clustering for Docker.

It turns a pool of Docker hosts into a single,virtual Docker host.

Because Docker Swarm serves the standardDocker API, any tool that alreadycommunicates with a Docker daemon can useSwarm to transparently scale to multiplehosts.

19 / 31

Docker SwarmRef: Mike Goelzer, Swarm: Docker Native Clustering, Swarm

Overview

20 / 31

21 / 31

Back to Console ...

22 / 31

$ docker-machine create -d virtualbox manager$ docker-machine create -d virtualbox agent1$ docker-machine create -d virtualbox agent2

$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORSagent1 - virtualbox Running tcp://192.168.99.101:2376 v1.11.1agent2 - virtualbox Running tcp://192.168.99.102:2376 v1.11.1default - virtualbox Stopped Unknown manager - virtualbox Running tcp://192.168.99.100:2376 v1.11.1

23 / 31

Create Nodes

$ eval $(docker-machine env manager)

$ docker run --rm swarm createUnable to find image 'swarm:latest' locallylatest: Pulling from library/swarmdd2ca02feeef: Pull complete ... a3ed95caeb02: Pull complete Digest: sha256:69363272443f4080e0623e78ced41bf38273535ee7f422b26bbc555db2b5d4a4Status: Downloaded newer image for swarm:latest323517cd62f3b9697152e09bdcfe8714

$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEswarm latest 47dc182ea74b 32 hours ago 19.32

$ docker run -d -p 3376:3376 -t -v /var/lib/boot2docker:/certs:ro swarm manage -H 0.0.20824649fd03351da2f8b77270332c2d94983664a7a2733a776d48b64b409655

24 / 31

Run SwarmManager

In this case, we are using the discoverybackend hosted on Docker Hub to create aunique discovery token for the cluster. Thisdiscovery backend is only for low-volumedevelopment and testing purposes.

The discovery backend maintains an up-to-date list of cluster members and shares thatlist with the Swarm manager. The Swarmmanager uses this list to assign tasks to thenodes.

25 / 31

Swarm Discovery Token

$ eval $(docker-machine env agent1)$ docker run -d swarm join --addr=$(docker-machine ip agent1):2376 token://323517cd62f3b9697152e09bdcfe8714Unable to find image 'swarm:latest' locallylatest: Pulling from library/swarmdd2ca02feeef: Pull complete ... a3ed95caeb02: Pull complete Digest: sha256:69363272443f4080e0623e78ced41bf38273535ee7f422b26bbc555db2b5d4a4Status: Downloaded newer image for swarm:latest2f929cdff3e9ae6de47b0367140bdc3e0f2d23d8ef809d4bb5f8900ffba82e63

$ eval $(docker-machine env agent2)$ docker run -d swarm join --addr=$(docker-machine ip agent2):2376 token://323517cd62f3b9697152e09bdcfe8714...

$ eval $(docker-machine env -u)

26 / 31

Run SwarmAgents

$ eval $(docker-machine env manager)$ export DOCKER_HOST=$(docker-machine ip manager):3376$ docker infoContainers: 2 Running: 2 Paused: 0 Stopped: 0Images: 2Server Version: swarm/1.2.1Role: primaryStrategy: spreadFilters: health, port, containerslots, dependency, affinity, constraintNodes: 2 agent1: 192.168.99.101:2376 - Containers: 1 - Reserved CPUs: 0 / 1 - Reserved Memory: 0 B / 1.021 GiB agent2: 192.168.99.102:2376 - Containers: 1 - Reserved CPUs: 0 / 1 - Reserved Memory: 0 B / 1.021 GiB...Kernel Version: 4.4.8-boot2dockerOperating System: linuxArchitecture: amd64CPUs: 2Total Memory: 2.042 GiB...WARNING: No kernel memory limit support

27 / 31

Connect toSwarm

$ docker run hello-worldHello from Docker....

$ docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES81ea366b46e5 hello-world "/hello" 18 seconds ago Exited (abbfbcc8de62 swarm "/swarm join --addr=1" 20 minutes ago Up 2f929cdff3e9 swarm "/swarm join --addr=1" 22 minutes ago Up

28 / 31

Use Swarm

Refs

29 / 31

Refs1. Machine Overview2. Get started with a local VM3. Swarm Overview4. Evaluate Swarm in a sandbox5. Mike Goelzer, Swarm: Docker Native Clustering6. Docker Machine - Reference env

30 / 31

ENDEueung Mulyana

http://eueung.github.io/docker-stuff/swarmCodeLabs | Attribution-ShareAlike CC BY-SA

31 / 31