From Continuous Integration to Continuous Delivery with Jenkins - javaland.de 2014

©2014 CloudBees, Inc. All Rights Reserved

From Continuous Integration To Continuous Delivery With Jenkins Cyrille Le Clerc, Solution Architect, CloudBees


About Me

Cyrille Le Clerc

@cyrilleleclerc

Open Source

CTO

DevOps, Infra as Code, Con9nuous Delivery

Solu9on Architect


About you

Dev? Ops? Maven? CI? CD? Cloud?


Agenda

•  Concepts: CI, Jenkins and CD •  New roles for Jenkins –  Cross team – Deployment – Workflow

•  Great power, great responsibili9es •  Jenkins reference architecture


Concepts

Continuous Integration


Continuous Integration

•  Prac9ce of SoIware Engineering •  Introduced by eXtreme Programming •  Prevent the “Integra9on Hell” •  Itera9ve


Continuous Integration Principles

•  Source code repository •  Automated build •  Self tes9ng build •  Commit to the baseline every day

•  Build every commit

•  Everybody can see the results of the build

•  Latest deliverable easy to download

•  Fast build

Wikipedia


Jenkins


Meet Jenkins… •  #1 Con9nuous Integra9on

and Con9nuous Deployment server

•  Java web applica9on •  “java –jar jenkins.war”

•  MIT-‐licensed, 8 years old •  500+ releases

jenkins-ci.org


#1 Continuous Integration Tool in the Market

Source: Eclipse Community Survey 2012


Massive Adoption, Happy Users

•  61,000+ installa9ons around the world; +60% in last year •  700+ plugins by 500 contributors •  Huge community

87% are “very satisfied” or “satisfied”!


Continuous Delivery


Why Continuous Delivery

•  Limits of Con9nuous Integra9on •  Lean SoIware Manufacturing


Continuous Delivery

•  “Reliable SoAware Releases through Build, Test, and Deployment AutomaJon”

•  DONE = Shippable into Produc9on


Continuous Delivery Steps

Con$

nuou

s Delivery

Source Code

Quality Analysis

Unit Tests

Binaries


DEV

DEV

OPS

QA

Continuous Delivery Steps

Con$

nuou

s Delivery

Source Code

Quality Analysis

Unit Tests

Binaries

Perfs Environment

Func Test Environment

UAT Environment

Production Environment

GO LIVE!

Release!

Staging Environment


Jenkins: pillar of Continuous Delivery

•  Connect the dots

•  It is the process


New Roles for Jenkins

•  Cross team pla_orm •  Deployment automa9on

–  App, dataset … environment provisioning •  Test automa9on: load test, acceptance test … •  Release management •  Workflow

Jenkins as an Applica9on Lifecycle Management pla_orm


Cross Team Platform

•  Not only used by DEV but also QA and OPS •  Various profiles and skills •  Mission Cri9cal


Deployment Automation

•  Same tools and automa9ons on all the environments

xxx Environment

xxx Environment

•  Deploy app, dataset … •  Plugins ecosystem


Test Automation

•  Powered by Jenkins ecosystem •  Requires full environment setup (app, dataset)


Release Management

•  Not only “mvn release:prepare release:perform”

•  Includes QA valida9on, Staging …

•  “Jenkins' Maven job type considered evil” S. Connolly


Workflow

Choreograph ac9vi9es that are … •  complex •  long-‐running •  not sequen9al •  pausable •  retryable


Great power, great responsibilities

Cross team collabora9on pla_orm: •  High Availability and SLAs •  Security


High Availability and SLAs

•  Jumbo Jenkins masters are fragile –  Mul9ple medium sized Jenkins masters

•  Careful version upgrades –  Tes9ng environment + Jenkins & plugins version control

•  Support •  It’s not an open bar, security

Jenkins Opera9ons Center (JOC)

Enterprise Update Center + JOC

Professional support


Security

•  Authen9ca9on •  Audit •  Role Base Access Control


Jenkins Reference Architecture


Multiple Medium Sized Masters Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

JenkinsEnterpriseMaster-N

JenkinsEnterpriseMaster-1

…


Master Slave Setup Jenkins Enterprise Master-1




…

Master-1 BuildNodes

Master-N BuildNodes


High Availability Jenkins Enterprise Master-1


HA Proxy



HA Proxy

NAS Storage

NAS Storage

backup

backup

…


Manage Multiple Masters


Jenkins Enterprise Master-1

Jenkins Operations Center

JenkinsOperations

Center (JOC)

config

configJenkins

EnterpriseMaster-N


…

Master-1 BuildNodes

Master-N BuildNodes


Shared Build Nodes Jenkins Enterprise Master-1


Shared BuildNodes



Master-1 BuildNodes

JenkinsOperations

Center (JOC)


confi

g

config

…


Testing Environment Jenkins Enterprise Master-1



JenkinsEnterpriseMaster-NJenkins

OperationsCenter (JOC)

Jenkins Enterprise Test Environment

JenkinsEnterprise

Test


config

config

config

…


Global Security Jenkins Enterprise Master-1




Role Based AccessControl Plugin

HA Proxy

DEV BuildNodes

HA Proxy

JenkinsOperations

Center (JOC)


JenkinsEnterprise

Test

HA Proxy


Access masters

HA Proxy

Security config pushed to masters

confi

g

config

config


LDAP…

audit

audit

audit

audit


Unified Navigation and SSO


JenkinsEnterprise

TestHA Proxy

Jenkins Enterprise Master-1

HA Proxy JenkinsEnterpriseMaster-1


JenkinsOperations

Center (JOC)HA Proxy

config

config

config


HA Proxy


http://master-1.jenkins.mycie.com

http://test.jenkins.mycie.com

…http://master-n.jenkins.mycie.com

http://joc.jenkins.mycie.com

Acce

ss m

aste

rs (S

SO)


Update Centers Jenkins Enterprise Master-1



Master-1 UpdateCenter

JenkinsEnterpriseMaster-NUpdate

CenterJenkins

OperationsCenter (JOC)


JenkinsEnterprise

Test

Master-1Test UpdateCenter

Test UpdateCenter

Promote plugins

Master specific Update CenterJenkins

EnterpriseMaster-1

confi

g

config

config

Internet

UpdateCenter

http://jenkins-ci.org

…


Jenkins Reference Architecture Jenkins Enterprise Master-1



Shared BuildNodes

UpdateCenter Jenkins

EnterpriseMaster-N

Role Based AccessControl Plugin

Master-1 UpdateCenter

HA Proxy

Master-1 BuildNodes

HA Proxy

JenkinsOperations

Center (JOC)


JenkinsEnterprise

TestHA Proxy

Master-1 Test UpdateCenter

Test UpdateCenter

Promote plugins

Master specific Update CenterJenkins

EnterpriseMaster-1

Access masters (SSO)

HA Proxy


confi

g

config

config

LDAP …

DEV, QA, OPS, admins


Continuous Delivery is

•  People •  Process •  Tools


Technology

From Continuous Integration to Continuous Delivery with Jenkins - javaland.de 2014