Using Rancher for highly available deployment services with GoCD and TeamCity

Using Rancher for highly available deployment services

with GoCD and TeamCityMatthew Skelton, Skelton Thatcher Consulting

@matthewpskelton / skeltonthatcher.com

Amsterdam Continuous Delivery meetup group08 February 2017, Amsterdam, NL

Rancher + GoCD + TeamCity == 😍

Today'Deployment Services'

Deployment orchestration with GoCDIntroduction to Rancher

Rancher: platform for deployment servicesLive Demo!

Team-first digital transformation30+ organisations

UK, EU, India, Chinaskeltonthatcher.com

Build & Release EngineeringInfrastructure AutomationOrganisational 'Medicine'Training and Mentoring

We're partners with…

DevOpsTopologies.com

About meMatthew Skelton

@matthewpskelton

Co-founder at Skelton Thatcher Consulting

skeltonthatcher.com

Using Rancher for highly available deployment

services with GoCD and TeamCity

Safer, more rapid changes to software systems

(Business Agility)

Deployment services as first-class, core capabilities for modern software teams

use logging as a channel/vector to

make distributed systems more testable

Invest in Build & Release Engineering

'Deployment Services'

Minimum capabilities•Version Control•Build & Unit Test •Packaging / artefact management•Deployment & test orchestration•Logging / metrics / dashboard

Minimum capabilities•Version Control•Build & Unit Test •Packaging / artefact mgt•Deployment & test orchestration•Logging / metrics / dashboard

2017: build & deployment tooling is core/critical

infrastructure – it's the only mechanism we use to make

changes to Production systems

DZone 2017 Guide to DevOps & Continuous Deliveryhttps://dzone.com/guides/devops-continuous-delivery-and-automation

Article: CD Anti-Patterns1. "One-stop shop" tools 2. Hidden logging / errors3. Environment-driven4. Blurred terminology

Authors: Manuel Pais & Matthew Skelton

TSK, TSK.BAD TOOLS!!!

One-stop-shop tools

Compose tooling w/ APIs

Logging exposed

Logging exposed

Environment-driven

Activity-driven

'Partially successful'

Deployment Orchestration with GoCD

My experience•Using GoCD since 2011•(Used CruiseControl 2006-2011)•100s of pipelines•~200 build agents•😍

However…•GoCD is powerful•With great power comesresponsibility some trickiness•Forces you to really understand your build & deployments 👍

Server + Agent model

Server

Build Agents Test Agents

Deployment Agents

Staging

Production

Value stream map

http://jeremie-martinez.com/2016/04/19/gocd-android/

Model the current activities

'Walking Skeleton' deployment pipeline

Helps to transform ways of working

Model the current activities

Model activities, not envs

Secure deployments

Use sensible security boundaries

GoCD encourages good practices and approaches for Continuous Delivery

Introduction to Rancher

"A Complete Platform for Running Containers"

Managing container clusters is tricky

Rancher simplifies the container 'fabric' lifecycle

Rancher Catalog

Custom Catalog !!!

Rancher as a platform forhighly-available

deployment services

Goal: highly-available tools•Survive host failures•Auto-failover to healthy hosts•Auto-scale-out under load•Multi-zone availability

Bad old deployment services

•Single VM•Manual installation•Occasional backups (not tested)•Sitting under a desk (aka randomAWS subscription)

Modern deployment services

•Either SaaSor•Containerised•Built and managed as code•Recreateable in minutes

Platform for HA deployment services

Rancher + Terraform + AWS

Blog post series'Container Clustering with Rancher'https://skeltonthatcher.com/blog/

HA at the Rancher level

HA at the EFS level

HA at the GoCD level (1x AZ)

HA at the GoCD level (2x AZ)

Rancher enables highly-available build &

deployment services using best-of-breed tools like

GoCD

Live Demo!

Demo•Create HA Rancher platform with Terraform

'cdams36'

Demo•Spin up new GoCD agents via Rancher 'scale' setting

Demo•Flexibility and power of GoCD 'value stream map' view

Further material

Links• GoCD: https://www.gocd.io/• Rancher: http://rancher.com/• Continuous Delivery with Windows & .NET by Matthew Skelton & Chris O'Dell

(O'Reilly, 2016): http://cdwithwindows.net/ (free download)• Blog post series on High-Availability Rancher:

https://skeltonthatcher.com/blog/container-clustering-rancher-server-part-3-aws-efs-mounts-using-rancher-nfs/

• DZone 2017 guide to Continuous Delivery: https://dzone.com/guides/devops-continuous-delivery-and-automation

ContainerisedBuilt and managed as code

Recreateable in minutes

Using Rancher for highly available deployment services

with GoCD and TeamCity

thank you

Matthew Skelton & Rich Bosomworth@matthewpskelton & @richbos

skeltonthatcher.com