Building a Scalable CI Platform using Docker, Drone and Rancher

© 2015 Rancher Labs, Inc.© 2015 Rancher Labs, Inc .

Building a Scalable CI Deployment with Docker, Drone and RancherAugust 13, 2015

#ranchermeetup

© 2015 Rancher Labs, Inc.2

Shannon WilliamsCo-Founder/VP Marketing @smw355

Darren ShepherdCo-Founder/Chief Architect @ibuildthecloudFreenode: darren0

#ranchermeetup


Bill MaxwellDirector of DevOps @CloudNautique

Brad RydzewskiFounder of Drone.io @bradrydzewski @droneiowww.drone.io

#ranchermeetup


First things first…

This is a not a !#ranchermeetup

#ranchermeetup


There are rules for a meetup!• Questions are always welcome• Demo, then demo some more• Keep the slides to a minimum• Open to proposals• Give away stuff with your logo

on it

#ranchermeetup


Tweets = Shirts

First 10 Rancher Screenshots posteds on Twitter about today’s

#RancherMeetup will get a Rancher shirt

Follow @rancher_labs

#ranchermeetup


Agenda• Why is Docker becoming so popular in DevOps• How we run our CI process at Rancher• Demo: Drone, Docker, Rancher• Drone.io Overview and RoadMap• Latest Rancher News: LibCompose, Rolling Upgrades, Virtual Host

LB, Sidekicks, etc.

#ranchermeetup


Containers are growing in popularity because they improve the entire DevOps pipeline

• Quick onboarding of developers• Consistent software builds• Accurate test environments• More frequent code releases• Immutable production environments• Orchestrated upgrades


Many organizations are looking at how to incorporate Docker into their pipelines

Source Code Repo

CI/CD

Build and Test tools

Docker Tools

Develop Build Package Test Deploy/Upgrade Operate


Open source software for building a Private Container Service


What is a container service?Central service where users and teams can create container environments for running services and applications

Includes:1. Container Orchestration2. Container Infrastructure3. Container Management


Environment

What is a container service?

Private Container Service

Resources

Resources (Clouds, VMs, Servers)

Environment

Resources

DevOpsPipeline

User Directory Container Registry

Ops Tools

© 2015 Rancher Labs, Inc.

Build Environment

DevOps builds software in the container service

- DevOps teams create Build Environments- Adds resources and Jenkins slaves- Developer Pushes Code to Repo- Jenkins creates build containers- Build and Unit Tests run in build container- Outputs Artifacts- Jenkins creates Packaging container- Push Docker Image to registry- Build environments scale elastically

BuildDocker File

PackagingDocker File

TestCode Build Docker

File



Test Environment

DevOps tests software in the container service

- Create Test Environments- Upload Compose & Test files- Build triggers Jenkins- Jenkins triggers Rancher to deploy compose- Rancher creates containers, networks,

storage, etc.- Test container conducts integration tests


Test docker-compose.yml lb: image: haproxy links: - webtomcat: image: app:buildn links: - db - cachecache: image: memcachedb: image: mysql:testn

Hosts

db- mysql:testn

cache- memcache

Lb-haproxy

tomcat-app:buildn

Test

Test

Compose Images


Production Environment

Hosts

DevOps runs production in the container service

- Production Compose- Rancher deploys

environment- Update Scaling- Health Checks & HA- Upgrade- Backup & Recovery


docker-compose.yml lb: image: haproxy links: - webtomcat: image: app:buildn links: - db - cachecache: image: memcachedb: image: mysql

lb

tomcat

CacheMySQL

Health Check Interval: 2 secTimeout: 2 secFailure Retry Frequency: 3Success Retry Frequency: 2Recovery Delay: 30 sec

tomcat(build n+1)

Drone, Docker and Rancher

CI here at Rancher Labs

Bill MaxwellDirector of DevOps @CloudNautique

17 #ranchermeetup


CI / CD @ Rancher Labs

• Primarily Developers

• Distributed• GitHub Focused

The Team


Solution Space- Low Administrative Overhead- GitHub Integration- Docker Integration

?


Selection Drone.IO- Built around Docker- State is stored in

Database- Single container

deployment- Simple GitHub

integration


Pipeline View

GitHub Drone

Tests

MasterBuilds

BuildBuild

BuildBuild

refresh

Developer PushesCode

RunTests

WebHook

StatusAPI


Scaling Drone- Number of

Developers- Build frequency- Multiple projects- Open Source


RancherWith - Private Networking- Native DockerWe Get- Easily Distributed Drone!

Rancher

DroneWorker

Worker

Worker

Worker

Worker

Worker


Demo

#ranchermeetup


Brad RydzewskiFounder of Drone.io @bradrydzewski @droneiowww.drone.io

#ranchermeetup

Latest news about Drone.io

New in 0.4

● Improved Yaml● Improved UX● Matrix Builds● External Plugins

build: image: golang commands: - go test -v

compose: postgres: image: postgres environment: - PG_USER=foo - PG_PASS=bar

Composable Builds

build configuration similar to docker

compose

build: image: golang:$$go_version commands: - go test -v compose: redis: image: redis:$$redis_version

matrix: go_version: - 1.4 - 1.3 redis_version: - 2.8 - latest

Matrix Builds

tests 4 different environment permutations

build: image: golang commands: - go test -v

notify: slack: image: plugins/slack token: ...

External Plugins

docker pull \ plugins/slack

docker run plugins/drone-slack << EOF{ “repo”: { ... }, “build”: { “number”: 1, “status”: “success” } “slack”: { “token”: “...” }}EOF

External Plugins

Plugins accept build details via stdin

Latest Rancher Enhancements

31

Load Balancing• Virtual Host Support for LB Services• Labels/Scheduling for Load Balancers in UI• Internal Load Balancing• Load Balancing External Services• Supporting CNAME for External Services

Service Discovery and Management• Cross Stack Linking• Service Sidekicks• Rancher Compose CLI support for Rolling Upgrades

Other• Support Random IP for DNS• Improved Rancher Server SSL Support

© 2015 Rancher Labs, Inc.32 #ranchermeetup

http://info.rancher.com/beta

http://info.rancher.com/beta


Getting StartedRancher and RancherOS live in GitHub – Get Involved, Join the Beta

#ranchermeetup

http://github.com/rancher

© 2015 Rancher Labs, Inc.© 2015 Rancher Labs, Inc .

Thank yourancher.com

Technology

Building a Scalable CI Platform using Docker, Drone and Rancher