Upload
puppet
View
30
Download
0
Embed Size (px)
Citation preview
The challenges ofcontainer configuration David Lutterkort @lutterkort [email protected]
Overview
● What is configuration ?
● Immutability
● Build vs Run
● Who configures the scheduler ?
● Conclusions
3
What is configuration ?
package/file/service
is only one instance of a more general problem
5
Configuration is any input into infrastructure
It needs to be managed
over time and at scale
6
Core configuration management features:
❏ describe system aspects in isolation
❏ combine aspects into whole
❏ common format for querying
❏ bridge across entire infrastructure
7
$ docker run -d \ -e MYSQL_HOST=mysql.example.com \ -e MYSQL_PORT=3306 \ --health-cmd /usr/bin/check \ webapp
Immutability
$ docker run \ --name example fedora:24 \ /bin/sh -c ‘while true; do \ cat /etc/system-release; \ sleep 1; \ done’
$ docker run …
Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)
$ docker exec example /bin/sh -c \ ‘sed -i -e s/24/25/ /etc/system-release’
Fedora release 24 (Twenty Four)Fedora release 24 (Twenty Four)
Fedora release 25 (Twenty Four)Fedora release 25 (Twenty Four)Fedora release 25 (Twenty Four)Fedora release 25 (Twenty Four)Fedora release 25 (Twenty Four)Fedora release 25 (Twenty Four)
$ docker exec …
$ docker diff exampleC /runA /run/secretsC /etcC /etc/system-release
Containers are not immutable by defaultOnly as immutable as packages
15
$ docker run --read-only \ --name example fedora:24 \ /bin/sh -c ‘while true; do \ cat /etc/system-release; \ sleep 1; \ done’
$ docker exec example /bin/sh -c \ ‘sed -i -e s/24/25/ /etc/system-release’sed: couldn't open temporary file /etc/sed5OCs5t: Read-only file system
$ docker diff exampleC /runA /run/secrets
Suggestion
Enable --read-only whenever possible
19
require 'rubygems'require 'sinatra'require 'haml'
# Handle GET-request (Show the upload form)get "/upload" do haml :uploadend
# Handle POST-request (Receive and save the uploaded file)post "/upload" do File.open('uploads/' + params['myfile'][:filename], "w") do |f|
f.write(params['myfile'][:tempfile].read) end return "The file was successfully uploaded!"end
$ docker run -d --read-only lutter/lolcat
require 'rubygems'require 'sinatra'require 'haml'
# Handle GET-request (Show the upload form)get "/upload" do haml :uploadend
# Handle POST-request (Receive and save the uploaded file)post "/upload" do
File.open('uploads/' + params['myfile'][:filename], "w") do |f|f.write(params['myfile'][:tempfile].read)
end return "The file was successfully uploaded!"end
$ docker run -d --read-only \ -v /srv/lolcat/uploads:/app/uploads \ lutter/lolcat
require 'rubygems'require 'sinatra'require 'haml'
# Handle GET-request (Show the upload form)get "/upload" do haml :uploadend
# Handle POST-request (Receive and save the uploaded file)post "/upload" do File.open('uploads/' + params['myfile'][:filename], "w") do |f|
f.write(params['myfile'][:tempfile].read) end return "The file was successfully uploaded!"end
$ docker run -d --read-only \ -v /srv/lolcat/uploads:/app/uploads \ --tmpfs /tmp \ lutter/lolcat
Suggestion
Use --tmpfs where needed
26
Without technical controls you only have
social guarantees of immutability
27
How do you know the correct
invocation for an image ?
28
Build vs Run
Given an image
❏ What machine built this image ?
❏ How do you run this image ?
❏ Who supports this image ?
❏ Does the image contain malware ?
30
Given a container
❏ Who built it ?
❏ How was it built ?
❏ What software does it contain ?
❏ Is the software up-to-date ?
31
FROM fedora:24
RUN dnf update -y && \ dnf install -y ruby rubygem-bundler && \ dnf clean all
COPY . /app
RUN cd /app && bundle install --path vendor/bundle
WORKDIR /appVOLUME /app/uploadsEXPOSE 9292CMD ["/usr/bin/bundle", "exec", "rackup"]
FROM fedora:24
RUN dnf update -y && \ dnf install -y ruby rubygem-bundler && \ dnf clean all
COPY . /app
RUN cd /app && bundle install --path vendor/bundle
WORKDIR /appVOLUME /app/uploadsEXPOSE 9292CMD ["/usr/bin/bundle", "exec", "rackup"]
Where did the base image come from ?
FROM fedora:24
RUN dnf update -y && \ dnf install -y ruby rubygem-bundler && \ dnf clean all
COPY . /app
RUN cd /app && bundle install --path vendor/bundle
WORKDIR /appVOLUME /app/uploadsEXPOSE 9292CMD ["/usr/bin/bundle", "exec", "rackup"]
What repositories and what package versions ?
FROM fedora:24
RUN dnf update -y && \ dnf install -y ruby rubygem-bundler && \ dnf clean all
COPY . /app
RUN cd /app && bundle install --path vendor/bundle
WORKDIR /appVOLUME /app/uploadsEXPOSE 9292CMD ["/usr/bin/bundle", "exec", "rackup"]
What was in this directory at build time ?
Time is your enemy
36
When do you rebuild images ?
37
Code changes and external factors
should trigger rebuild
38
Explain yourself with metadataDocker labels are a great way to do that
39
Name : glibcVersion : 2.23.1Release : 10.fc24Architecture: x86_64License : LGPLv2+ and LGPLv2+ with exceptions and GPLv2+Signature : RSA/SHA256, Thu 18 Aug 2016 09:27:43 AM PDT, Key ID 73bde98381b46521Source RPM : glibc-2.23.1-10.fc24.src.rpmBuild Date : Thu 18 Aug 2016 06:37:42 AM PDTBuild Host : buildvm-16.phx2.fedoraproject.orgPackager : Fedora ProjectVendor : Fedora ProjectSummary : The GNU libc libraries
$ docker inspect \ -f "{{json .Config.Volumes}}" lutter/lolcat{ "/app/uploads": {}}
$ docker inspect \ -f "{{json .Config.ExposedPorts}}" lutter/lolcat{ "9292/tcp": {}}
LABEL vendor=”ACME Incorporated” \ com.acme.release-status=”beta” \ com.acme.version=”0.1.0-beta” \ com.acme.git.sha=”f260653a”
$ docker inspect \ -f "{{json .Config.Labels}}" lutter/lolcat | jq{ "com.acme.git.sha": "f260653a", "com.acme.release-status": "beta", "com.acme.version": "0.1.0-beta", "vendor": "ACME Incorporated"}
Suggestion
Decide upon and enforcemetadata standards
45
LABEL com.acme.dockerfile=”/Dockerfile”
$ docker inspect \ -f "{{json .Config.Labels}}" lutter/alpine | jq{ "com.example.dockerfile": "/Dockerfile"}
$ docker run -it lutter/alpine cat /DockerfileFROM alpineRUN apk add --update bash && rm -rf /var/cache/apk/*COPY Dockerfile /LABEL com.example.dockerfile="/Dockerfile"
Suggestion
Embed your Dockerfile in the image
49
LABEL com.acme.cmd.packages=”apk info -vv”
$ docker run -it lutter/alpine apk info -vvmusl-1.1.14-r12 - the musl c library (libc)busybox-1.24.2-r11 - Size optimized toolbox of ...alpine-baselayout-3.0.3-r0 - Alpine base dir ...alpine-keys-1.1-r0 - Public keys for Alpine Linux ...zlib-1.2.8-r2 - A compression/decompression Librarybash-4.3.42-r3 - The GNU Bourne Again shell...
Suggestion
Make your images discoverable
52
puppetlabs/puppetlabs-image_build
class { 'nginx': }
nginx::resource::vhost { 'default': www_root => '/var/www/html',}
file { '/var/www/html/index.html': ensure => present, content => 'Hello Puppet and Docker',}
exec { 'Disable Nginx daemon mode': path => '/bin', command => 'echo "daemon off;" >> /etc/nginx/nginx.conf', unless => 'grep "daemon off" /etc/nginx/nginx.conf',}
# metadata.yamlcmd: nginxexpose: 80image_name: puppet/nginx
$ puppet docker build...
$ docker run -d -p 8080:80 acme/nginx-test83d5fbe370e84d424c71c1c038ad1f5892fec579d28b...
$ curl http://127.0.0.1:8080Hello Puppet and Docker
Who configures the scheduler ?
Schedulers/orchestrators isolate you from
❏ where individual containers run
❏ balancing due to new resources
❏ respawning due to failed resources
58
Schedulers operate on constraints
59
Decisions depend on accurate resource
information
60
$ docker daemon \ --label environment=production \ --label storage=ssd
$ docker run -d -P \ --label com.example.environment=production \ -e constraint:storage==ssd --name db mysql
template: metadata: labels: app: guestbook tier: frontend spec: containers: - name: php-redis image: gcr.io/google-samples/gb-frontend:v4 resources: requests: cpu: 100m memory: 100Mi env: - name: GET_HOSTS_FROM value: dns # If your cluster config does not include a dns service, then to # instead access environment variables to find service host # info, comment out the 'value: dns' line above, and uncomment the # line below. # value: env ports: - containerPort: 80
How do you manage properties
for all your hosts ?
64
Suggestion
Compute host properties dynamically
65
$ facter -y | head -n 20aio_agent_version: 1.7.0augeas: version: 1.4.0disks: sda:
model: SanDisk SDSSDA24size: 223.57 GiBsize_bytes: 240057409536vendor: ATA
...dmi: bios: ...memory:...
$ docker daemon \ --label os=$(facter os.family) \ --label kernel=$(facter kernelversion) \ --label memory=$(facter memory.system.total_bytes)
https://forge.puppet.com/puppetlabs/docker_platform
class { 'docker': labels => [ "os=${facts[os][family]", "kernel=${facts[kernelversion]}", "memory=${facts[memory][system][total_bytes]}" ],}
Schedulers introduce higher-level primitives
70
Docker networks
Kubernetes services and replication controllers
Chronos jobs
71
Many interfaces imperative not declarative
72
$ kubectl get pod mypod -o yaml \ | sed -e ‘s/\(image:myimage\):.*$/\1:v4/’ \ | kubectl replace -f -
$ docker network create bobca7b185775966003d38ccbd9bba822fb570766e4bb
$ docker network create bobError response from daemon: network with name bob ...
docker_network { 'bob': ensure => present, driver => 'overlay', subnet => '192.168.1.0/24', gateway => '192.168.1.1', ip_range => '192.168.1.4/32',}
And everything is in YAML
76
“The language to represent the data should be a simple, data-only
format such as JSON or YAML, and programmatic modification of
this data should be done in a real programming language, where
there are well-understood semantics, as well as good tooling.
Borg, Omega, and Kubernetes, ACM Queue, Volume 14 Issue 1 | http://queue.acm.org/detail.cfm?id=2898444
77
Code plus data has advantages
over data alone
78
https://forge.puppet.com/garethr/kubernetes
kubernetes_pod { 'sample-pod': ensure => present, metadata => { namespace => 'default', }, spec => { containers => [{ name => 'container-name', image => 'nginx', }] },}
controller_service_pair { 'redis-master': app => 'redis', role => 'master', tier => 'backend', port => 6379,}
Conclusions
The difference between how you think a
system behaves and how it actually behaves
risks hard-to-debug production issues
83
Container use at scale and over time
requires meaningful abstraction
84
Configuration management as a discipline
provides tools to build those abstractions and
thereby minimize risk
85
86
Project Blueshift boothExhibition Hall
Docker, Mesos, Kubernetes and Puppet? Don't Panic !Deepak Giridharagopal, Thur, 4:45pm
Pulling the strings to containerize your lifeScott Coulton, Fri, 9:50am
Running Puppet software in Docker containersGareth Rushgrove, Fri, 1:30pm