Transcript
Page 1: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Berlin

Page 2: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Continuous Deployment on ECS

Sébastien Stormacq, Training Manager, AWS

[email protected]

@sebsto

Page 3: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

CONTINUOUS

INTEGRATION

Page 4: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

INFRASTRUCTURE

Page 5: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

INFRASTRUCTURE

Page 6: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

INFRASTRUCTURE

Page 7: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

CONTINUOUS

INTEGRATION SERVERS

PROJECT MANAGEMENT

SERVER

INFRASTRUCTURE

Page 8: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

CONTINUOUS

INTEGRATION SERVERS

PROJECT MANAGEMENT

SERVER

INFRASTRUCTURE

PICK

TASKS

Page 9: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

CONTINUOUS

INTEGRATION SERVERS

PROJECT MANAGEMENT

SERVER

INFRASTRUCTURE

SUBMIT

CODE

Page 10: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

CONTINUOUS

INTEGRATION SERVERS

PROJECT MANAGEMENT

SERVER

INFRASTRUCTURE

CODE

FETCH

Page 11: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

CONTINUOUS

INTEGRATION SERVERS

PROJECT MANAGEMENT

SERVER

DOCS

BINARIES &

PACKAGES

INFRASTRUCTURE

BUILD OUTPUT

Page 12: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

SOURCE CODE

REPOSITORY

CONTINUOUS

INTEGRATION SERVERS

PROJECT MANAGEMENT

SERVER

DOCS

BINARIES &

PACKAGES

INFRASTRUCTURE

Page 13: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

DOCS

BINARIES &

PACKAGES

INFRASTRUCTURE?

Page 14: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

= PROGRAMMABLE PLATFORM

Page 15: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

IF YOU CAN PROGRAM IT,

YOU CAN AUTOMATE IT

Page 16: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry
Page 17: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry
Page 18: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry
Page 19: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Docker on AWS

Sascha Möllering, Senior Software Engineer, zanox AG

[email protected]

sascha242

Page 20: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

What is zanox?

Page 21: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Simplified architecture

Page 22: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Title with free format content section

Page 23: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Shipping with Docker

Amazon Cloud

WorkstationGitHub

Jenkins

Python/Boto

Docker Image

Docker Registry S3 bucket with

Docker Images

Page 24: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Shipping with Docker

EC2 instance with Docker

registry (port 5000)

S3 bucket to store

Docker images

Page 25: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Shipping with Docker

Page 26: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Shipping with Docker

Page 27: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Shipping with Docker

• EC2 User Data

• Data to configure instance during launch– Install Docker

– Start Daemon

– Run Docker container

Page 28: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Immutable Server

Page 29: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Title with free format content section

Page 30: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Immutable Server

• Trash your servers!

• New deployment -> new infrastructure

• Don’t change an existing server

• Problem: snowflake server

Page 31: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Immutable Server

Page 32: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Title with free format content section

Page 33: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Immutable Server

Page 34: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Immutable Server

• Pre-Baking AMIs with Packer

• Security patches

• Configuration

• Additional software

Page 36: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

CONTINUOUS

DEPLOYMENT

Page 37: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEVELOPERS

CONTINUOUS

DEPLOYMENT SERVERS

BINARIES &

PACKAGES

INFRASTRUCTURE

DEPLOY

Page 38: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Elastic Beanstalk Cloudformation Code Deploy OpsWorks

Page 39: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

EC2 Container Services

Page 40: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

+

Page 41: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

CLUSTER MANAGEMENT

Page 42: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

FLEXIBLE CONTAINER

PLACEMENT

Page 43: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

INTEGRATED WITH

OTHER SERVICES

Page 44: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

EXTENSIBLE

Page 45: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

PERFORMANCE AT SCALE

Page 46: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

SECURITY FIRST

Page 47: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

ANATOMY

OF AN ECS

DEPLOYMENT

Page 48: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DOCKER CONTAINERS

Page 49: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

EC2 INSTANCES CLUSTER

Page 50: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

TASK DEFINITIONS

Page 51: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

[

{

"image": "tutum/wordpress-stackable",

"name": "wordpress",

"cpu": 10,

"memory": 500,

"essential": true,

"links": [

"db"

],

"entryPoint": [

"/bin/sh",

"-c"

],

"environment": [

],

"portMappings": [

{

"containerPort": 80,

"hostPort": 80

}

]

},

]

[

{

"image": "mysql",

"name": "db",

"cpu": 10,

"memory": 500,

"essential": true,

"entryPoint": [

"/entrypoint.sh"

],

"environment": [

{

"name": "MYSQL_ROOT_PASSWORD",

"value": "pass"

}

],

"portMappings": []

}

]

Page 52: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

Tasks are defined via Task Definitions[

{

"image": "tutum/wordpress-stackable",

"name": "wordpress",

"cpu": 10,

"memory": 500,

"essential": true,

"links": [

"db"

],

"entryPoint": [

"/bin/sh",

"-c"

],

"environment": [

],

"portMappings": [

{

"containerPort": 80,

"hostPort": 80

}

]

},

]

[

{

"image": "mysql",

"name": "db",

"cpu": 10,

"memory": 500,

"essential": true,

"entryPoint": [

"/entrypoint.sh"

],

"environment": [

{

"name": "MYSQL_ROOT_PASSWORD",

"value": "pass"

}

],

"portMappings": []

}

]

From Dockerhub

10 CPU Units

(1024 is full CPU),

500 Megabytes of Memory

Environment Variables

No external ports exposed

Page 53: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

[

{

"image": "tutum/wordpress-stackable",

"name": "wordpress",

"cpu": 10,

"memory": 500,

"essential": true,

"links": [

"db"

],

"entryPoint": [

"/bin/sh",

"-c"

],

"environment": [

],

"portMappings": [

{

"containerPort": 80,

"hostPort": 80

}

]

},

]

[

{

"image": "mysql",

"name": "db",

"cpu": 10,

"memory": 500,

"essential": true,

"entryPoint": [

"/entrypoint.sh"

],

"environment": [

{

"name": "MYSQL_ROOT_PASSWORD",

"value": "pass"

}

],

"portMappings": []

}

]

Essential to our Task

Docker link to mysql container

Expose port 80 in container

to port 80 on host

Page 54: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEPLOYMENT TO ECS

register-task-definition

update-service

Page 55: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

CONTINUOUS INTEGRATION

or

DEPLOYMENT SERVER

AWS SDK

or

AWS CLI

EC2 CONTAINER SERVICE

Page 56: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

CONTINUOUS INTEGRATION

or

DEPLOYMENT SERVER

AWS SDK

or

AWS CLI

EC2 CONTAINER SERVICE

POST BUILD STEP API CALLS

Page 57: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

DEMO

Page 58: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

AWS CLOUD

JENKINS & AWS CLI

EC2

GITHUBSOURCE

ECS

DOCKER

REGISTRY

EC2 INSTANCES

CLUSTER

Page 59: Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin...Shipping with Docker Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry

[email protected]

@sebsto


Recommended