Upload
thach-nguyen
View
192
Download
0
Embed Size (px)
Citation preview
Why?
Contents• The challenge
Pre-[package] dev stack• WAMP, XAMP• Java/Tomcat• Ruby/Rail
It works on MY machine
VM
• closely production env• annoying configure
Provision a server without logging in• Ansible, puppet, salt, Chef• Pre-configure• Pre-setup• Environment-path• by a shell script
Simple, repeatable VMs
Market View: Evolution of IT1995 2015
Running on any available set of
physical resources(public/private/
virtualized)
Assembled by developers using
best available services
Thin app on mobile, tabletThick, client-server app
on thick client
Well-defined stack: - O/S - Runtime - Middleware
MonolithicPhysical
Infrastructure
Challenges2015
How to ensure services interact consistently,
avoid dependency hell
How to migrate & scale quickly, ensure compatibility
How to avoid n X n different configs
Running on any available set of
physical resources(public/private/
virtualized)
Assembled by developers using
best available services
Thin app on mobile, tablet
Static website
Web frontend
User DBQueue Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8
hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Development VM
QA server
Public Cloud
Disaster recoveryContributor’s laptop
Production Servers
The ChallengeM
ultip
licity
of S
tack
sM
ultip
licity
of
hard
war
e en
viro
nmen
ts
Production Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can I migrate
smoothly and quickly?
Results in N X N compatibility nightmareStatic website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM QA Server Single Prod
ServerOnsite Cluster
Public Cloud
Contributor’s laptop
Customer Servers
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
A useful analogy…
Mul
tiplic
ity o
f Goo
dsM
ultip
ilici
ty o
f m
etho
ds fo
r tr
ansp
ortin
g/st
orin
g
Do I worry about
how goods interact
(e.g. coffee beans next to spices)
Can I transport quickly and sm
oothly(e.g. from
boat to train to truck)
Cargo Transport Pre-1960
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Also an NxN Matrix
Contents• The challenge• The solution
Mul
tiplic
ity o
f Goo
dsM
ultip
licity
of
met
hods
for
tran
spor
ting/
stor
ing
Do I worry about
how goods interact
(e.g. coffee beans next to spices)
Can I transport quickly and sm
oothly(e.g. from
boat to train to truck)
Solution: Intermodal Shipping Container
…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another
A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.
This eliminated the NXN problem…
and spawned an Intermodal Shipping Container Ecosystem
• 90% of all cargo now shipped in a standard container• Order of magnitude reduction in cost and time to load and unload ships• Massive reduction in losses due to theft or damage• Huge reduction in freight cost as percent of final goods (from >25% to <3%) massive globalizations • 5000 ships deliver 200M containers per year
Static website Web frontend User DB Queue Analytics DB
Development VM
QA server Public Cloud Contributor’s laptop
Docker is a shipping container system for code M
ultip
licity
of S
tack
sM
ultip
licity
of
hard
war
e en
viro
nmen
ts
Production Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can I migrate
smoothly and quickly
…that can be manipulated using standard operations and run consistently on virtually any hardware platform
An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
Static website Web frontend User DB Queue Analytics DB
Development VM
QA server Public Cloud Contributor’s laptop
Or…put more simplyM
ultip
licity
of S
tack
sM
ultip
licity
of
hard
war
e en
viro
nmen
ts
Production Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can I migrate
smoothly and quickly
Operator: Configure Once, Run Anything
Developer: Build Once, Run Anywhere (Finally)
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM QA Server Single Prod
ServerOnsite Cluster
Public Cloud
Contributor’s laptop
Customer Servers
Docker solves the NXN problem
Why containers matterPhysical Containers Docker
Content Agnostic The same container can hold almost any type of cargo
Can encapsulate any payload and its dependencies
Hardware Agnostic Standard shape and interface allow same container to move from ship to train to semi-truck to warehouse to crane without being modified or opened
Using operating system primitives (e.g. LXC) can run consistently on virtually any hardware—VMs, bare metal, openstack, public IAAS, etc.—without modification
Content Isolation and Interaction
No worry about anvils crushing bananas. Containers can be stacked and shipped together
Resource, network, and content isolation. Avoids dependency hell
Automation Standard interfaces make it easy to automate loading, unloading, moving, etc.
Standard operations to run, start, stop, commit, search, etc. Perfect for devops: CI, CD, autoscaling, hybrid clouds
Highly efficient No opening or modification, quick to move between waypoints
Lightweight, virtually no perf or start-up penalty, quick to move and manipulate
Separation of duties Shipper worries about inside of box, carrier worries about outside of box
Developer worries about code. Ops worries about infrastructure.
Container• A container is a stripped-to-basics (mimic OS kernel)
version of a Linux operating system.
Image• An image is software you load into a container.
• hello-world: is simple app, can do a simple task as echo
• Wordpress: is complex app, run database, wait for data, ...
• or setup Ubuntu bash shell
AppA
Containers vs. VMs
Hypervisor (Type 2)
Host OS
Server
GuestOS
Bins/Libs
AppA’
GuestOS
Bins/Libs
AppB
GuestOS
Bins/Libs
App A’
Docker
Host OS
Server
Bins/Libs
App ABins/Libs
App B
App B’
App B’
App B’VM
Container
Containers are isolated,but share OS and, whereappropriate, bins/libraries
GuestOS
GuestOS
Why are Docker containers lightweight?
Bins/Libs
AppA
Original App(No OS to takeup space, resources,or require restart)
App Δ
Bins/
AppA
Bins/Libs
AppA’
GuestOS
Bins/Libs
Modified App
Union file system allowsus to only save the diffsBetween container A and containerA’
VMsEvery app, every copy of anapp, and every slight modificationof the app requires a new virtual server
AppA
GuestOS
Bins/Libs
Copy ofApp
No OS. CanShare bins/libs
AppA
GuestOS
GuestOS
VMs Containers
Changes and Updates
Docker Engine
DockerContainer
Image Registry
Docker Engine
Push
Update
Bins/Libs
AppA
App Δ
Bins/
Base Container
Image
Host is now running A’’
Container Mod A’’
App Δ
Bins/
Bins/Libs
AppA
Bins/
Bins/Libs
AppA’’
Host running A wants to upgrade to A’’. Requests update. Gets only diffs
Container Mod A’