Building a Scalable CI Platform using Docker, Drone and Rancher

© 2015 Rancher Labs, Inc.© 2015 Rancher Labs, Inc .

Building a Scalable CI Deployment with Docker, Drone and RancherAugust 13, 2015

#ranchermeetup

© 2015 Rancher Labs, Inc.2

Shannon WilliamsCo-Founder/VP Marketing @smw355

Darren ShepherdCo-Founder/Chief Architect @ibuildthecloudFreenode: darren0

#ranchermeetup

© 2015 Rancher Labs, Inc.3

Bill MaxwellDirector of DevOps @CloudNautique

Brad RydzewskiFounder of Drone.io @bradrydzewski @droneiowww.drone.io

#ranchermeetup

© 2015 Rancher Labs, Inc.4

First things first…

This is a not a !#ranchermeetup

#ranchermeetup

© 2015 Rancher Labs, Inc.5

There are rules for a meetup!• Questions are always welcome• Demo, then demo some more• Keep the slides to a minimum• Open to proposals• Give away stuff with your logo

on it

#ranchermeetup

© 2015 Rancher Labs, Inc.6

Tweets = Shirts

First 10 Rancher Screenshots posteds on Twitter about today’s

#RancherMeetup will get a Rancher shirt

Follow @rancher_labs

#ranchermeetup

© 2015 Rancher Labs, Inc.7

Agenda• Why is Docker becoming so popular in DevOps• How we run our CI process at Rancher• Demo: Drone, Docker, Rancher• Drone.io Overview and RoadMap• Latest Rancher News: LibCompose, Rolling Upgrades, Virtual Host

LB, Sidekicks, etc.

#ranchermeetup

© 2015 Rancher Labs, Inc.8

Containers are growing in popularity because they improve the entire DevOps pipeline

• Quick onboarding of developers• Consistent software builds• Accurate test environments• More frequent code releases• Immutable production environments• Orchestrated upgrades

© 2015 Rancher Labs, Inc.9

Many organizations are looking at how to incorporate Docker into their pipelines

Source Code Repo

CI/CD

Build and Test tools

Docker Tools

Develop Build Package Test Deploy/Upgrade Operate

© 2015 Rancher Labs, Inc.10

Open source software for building a Private Container Service

© 2015 Rancher Labs, Inc.11

What is a container service?Central service where users and teams can create container environments for running services and applications

Includes:1. Container Orchestration2. Container Infrastructure3. Container Management

© 2015 Rancher Labs, Inc.12

Environment

What is a container service?

Private Container Service

Resources

Resources (Clouds, VMs, Servers)

Environment

Resources

DevOpsPipeline

User Directory Container Registry

Ops Tools

© 2015 Rancher Labs, Inc.

Build Environment

DevOps builds software in the container service

- DevOps teams create Build Environments- Adds resources and Jenkins slaves- Developer Pushes Code to Repo- Jenkins creates build containers- Build and Unit Tests run in build container- Outputs Artifacts- Jenkins creates Packaging container- Push Docker Image to registry- Build environments scale elastically

BuildDocker File

PackagingDocker File

TestCode Build Docker

File

Develop Build Package Test Deploy/Upgrade Operate

© 2015 Rancher Labs, Inc.

Test Environment

DevOps tests software in the container service

- Create Test Environments- Upload Compose & Test files- Build triggers Jenkins- Jenkins triggers Rancher to deploy compose- Rancher creates containers, networks,

storage, etc.- Test container conducts integration tests

Develop Build Package Test Deploy/Upgrade Operate

Test docker-compose.yml lb:  image: haproxy    links:    - webtomcat:  image: app:buildn  links:    - db    - cachecache:  image: memcachedb:  image: mysql:testn

Hosts

db- mysql:testn

cache- memcache

Lb-haproxy

tomcat-app:buildn

Test

Test

Compose Images

© 2015 Rancher Labs, Inc.

Production Environment

Hosts

DevOps runs production in the container service

- Production Compose- Rancher deploys

environment- Update Scaling- Health Checks & HA- Upgrade- Backup & Recovery

Develop Build Package Test Deploy/Upgrade Operate

docker-compose.yml lb:  image: haproxy    links:    - webtomcat:  image: app:buildn  links:    - db    - cachecache:  image: memcachedb:  image: mysql

lb

tomcat

CacheMySQL

Health Check Interval: 2 secTimeout: 2 secFailure Retry Frequency: 3Success Retry Frequency: 2Recovery Delay: 30 sec

tomcat(build n+1)

Drone, Docker and Rancher

CI here at Rancher Labs

Bill MaxwellDirector of DevOps @CloudNautique

17 #ranchermeetup

© 2015 Rancher Labs, Inc.18

CI / CD @ Rancher Labs

• Primarily Developers

• Distributed• GitHub Focused

The Team

© 2015 Rancher Labs, Inc.19

Solution Space- Low Administrative Overhead- GitHub Integration- Docker Integration

?

© 2015 Rancher Labs, Inc.20

Selection Drone.IO- Built around Docker- State is stored in

Database- Single container

deployment- Simple GitHub

integration

© 2015 Rancher Labs, Inc.

Pipeline View

GitHub Drone

Tests

MasterBuilds

BuildBuild

BuildBuild

refresh

Developer PushesCode

RunTests

WebHook

StatusAPI

© 2015 Rancher Labs, Inc.22

Scaling Drone- Number of

Developers- Build frequency- Multiple projects- Open Source

© 2015 Rancher Labs, Inc.23

RancherWith - Private Networking- Native DockerWe Get- Easily Distributed Drone!

Rancher

DroneWorker

Worker

Worker

Worker

Worker

Worker

© 2015 Rancher Labs, Inc.24

Demo

#ranchermeetup

© 2015 Rancher Labs, Inc.25

Brad RydzewskiFounder of Drone.io @bradrydzewski @droneiowww.drone.io

#ranchermeetup

Latest news about Drone.io

New in 0.4

● Improved Yaml● Improved UX● Matrix Builds● External Plugins

build: image: golang commands: - go test -v

compose: postgres: image: postgres environment: - PG_USER=foo - PG_PASS=bar

Composable Builds

build configuration similar to docker

compose

build: image: golang:$$go_version commands: - go test -v compose: redis: image: redis:$$redis_version

matrix: go_version: - 1.4 - 1.3 redis_version: - 2.8 - latest

Matrix Builds

tests 4 different environment permutations

build: image: golang commands: - go test -v

notify: slack: image: plugins/slack token: ...

External Plugins

docker pull \ plugins/slack

docker run plugins/drone-slack << EOF{ “repo”: { ... }, “build”: { “number”: 1, “status”: “success” } “slack”: { “token”: “...” }}EOF

External Plugins

Plugins accept build details via stdin

Latest Rancher Enhancements

31

Load Balancing• Virtual Host Support for LB Services• Labels/Scheduling for Load Balancers in UI• Internal Load Balancing• Load Balancing External Services• Supporting CNAME for External Services

Service Discovery and Management• Cross Stack Linking• Service Sidekicks• Rancher Compose CLI support for Rolling Upgrades

Other• Support Random IP for DNS• Improved Rancher Server SSL Support

© 2015 Rancher Labs, Inc.32 #ranchermeetup

http://info.rancher.com/beta

© 2015 Rancher Labs, Inc.33

Getting StartedRancher and RancherOS live in GitHub – Get Involved, Join the Beta

#ranchermeetup

http://github.com/rancher

© 2015 Rancher Labs, Inc.© 2015 Rancher Labs, Inc .

Thank yourancher.com