From Continuous Integration to Continuous Delivery with Jenkins - javaland.de 2014

©2014 CloudBees, Inc. All Rights Reserved

From Continuous Integration To Continuous Delivery With Jenkins Cyrille Le Clerc, Solution Architect, CloudBees

©2014 CloudBees, Inc. All Rights Reserved

About Me

Cyrille Le Clerc  

@cyrilleleclerc  

Open  Source  

CTO  

DevOps,  Infra  as  Code,  Con9nuous  Delivery  

Solu9on  Architect  

©2014 CloudBees, Inc. All Rights Reserved

About you

Dev? Ops? Maven? CI? CD? Cloud?

©2014 CloudBees, Inc. All Rights Reserved

Agenda

•  Concepts:  CI,  Jenkins  and  CD  •  New  roles  for  Jenkins  –  Cross  team  – Deployment  – Workflow  

•  Great  power,  great  responsibili9es  •  Jenkins  reference  architecture  

©2014 CloudBees, Inc. All Rights Reserved

Concepts

Continuous Integration

©2014 CloudBees, Inc. All Rights Reserved

Continuous Integration

•  Prac9ce  of  SoIware  Engineering  •  Introduced  by  eXtreme  Programming  •  Prevent  the  “Integra9on  Hell”  •  Itera9ve  

©2014 CloudBees, Inc. All Rights Reserved

Continuous Integration Principles

•  Source  code  repository  •  Automated  build  •  Self  tes9ng  build  •  Commit  to  the  baseline  every  day  

•  Build  every  commit  

•  Everybody  can  see  the  results  of  the  build  

•  Latest  deliverable  easy  to  download  

•  Fast  build  

Wikipedia  

©2014 CloudBees, Inc. All Rights Reserved

Jenkins

©2014 CloudBees, Inc. All Rights Reserved

Meet Jenkins… •  #1  Con9nuous  Integra9on  

and  Con9nuous  Deployment  server  

•  Java  web  applica9on    •    “java  –jar  jenkins.war”  

•  MIT-­‐licensed,  8  years  old  •  500+  releases  

jenkins-ci.org

©2014 CloudBees, Inc. All Rights Reserved

#1 Continuous Integration Tool in the Market

Source:  Eclipse  Community  Survey  2012  

©2014 CloudBees, Inc. All Rights Reserved

Massive Adoption, Happy Users

•  61,000+  installa9ons  around  the  world;  +60%  in  last  year  •  700+  plugins  by  500  contributors  •  Huge  community  

87% are “very satisfied” or “satisfied”!

©2014 CloudBees, Inc. All Rights Reserved

Continuous Delivery

©2014 CloudBees, Inc. All Rights Reserved

Why Continuous Delivery

•  Limits  of  Con9nuous  Integra9on  •  Lean  SoIware  Manufacturing  

©2014 CloudBees, Inc. All Rights Reserved

Continuous Delivery

•  “Reliable  SoAware  Releases  through  Build,  Test,  and  Deployment  AutomaJon”  

•  DONE  =  Shippable  into  Produc9on  

©2014 CloudBees, Inc. All Rights Reserved

Continuous Delivery Steps

Con$

nuou

s  Delivery  

Source Code

Quality Analysis

Unit Tests

Binaries

©2014 CloudBees, Inc. All Rights Reserved

DEV

DEV

OPS

QA

Continuous Delivery Steps

Con$

nuou

s  Delivery  

Source Code

Quality Analysis

Unit Tests

Binaries

Perfs Environment

Func Test Environment

UAT Environment

Production Environment

GO LIVE!

Release!

Staging Environment

©2014 CloudBees, Inc. All Rights Reserved

Jenkins: pillar of Continuous Delivery

•  Connect  the  dots  

•  It  is  the  process  

©2014 CloudBees, Inc. All Rights Reserved

New Roles for Jenkins

•  Cross  team  pla_orm  •  Deployment  automa9on  

–  App,  dataset  …  environment  provisioning  •  Test  automa9on:  load  test,  acceptance  test  …  •  Release  management  •  Workflow  

Jenkins  as  an  Applica9on  Lifecycle  Management  pla_orm  

©2014 CloudBees, Inc. All Rights Reserved

Cross Team Platform

•  Not  only  used  by  DEV  but  also  QA  and  OPS  •  Various  profiles  and  skills  •  Mission  Cri9cal  

©2014 CloudBees, Inc. All Rights Reserved

Deployment Automation

•  Same  tools  and  automa9ons  on  all  the  environments  

xxx Environment

xxx Environment

•  Deploy  app,  dataset  …  •  Plugins  ecosystem  

©2014 CloudBees, Inc. All Rights Reserved

Test Automation

•  Powered  by  Jenkins  ecosystem  •  Requires  full  environment  setup  (app,  dataset)  

©2014 CloudBees, Inc. All Rights Reserved

Release Management

•  Not  only  “mvn release:prepare release:perform”  

•  Includes  QA  valida9on,  Staging  …  

•  “Jenkins'  Maven  job  type  considered  evil”  S.  Connolly    

©2014 CloudBees, Inc. All Rights Reserved

Workflow

Choreograph  ac9vi9es  that  are  …  •  complex  •  long-­‐running  •  not  sequen9al  •  pausable  •  retryable  

©2014 CloudBees, Inc. All Rights Reserved

Great power, great responsibilities

Cross  team  collabora9on  pla_orm:  •  High  Availability  and  SLAs  •  Security  

©2014 CloudBees, Inc. All Rights Reserved

High Availability and SLAs

•  Jumbo  Jenkins  masters  are  fragile  –  Mul9ple  medium  sized  Jenkins  masters  

•  Careful  version  upgrades  –  Tes9ng  environment  +  Jenkins  &  plugins  version  control  

•  Support  •  It’s  not  an  open  bar,  security  

Jenkins  Opera9ons  Center  (JOC)  

Enterprise  Update  Center  +  JOC    

Professional  support  

©2014 CloudBees, Inc. All Rights Reserved

Security

•  Authen9ca9on  •  Audit  •  Role  Base  Access  Control  

©2014 CloudBees, Inc. All Rights Reserved

Jenkins Reference Architecture

©2014 CloudBees, Inc. All Rights Reserved

Multiple Medium Sized Masters Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

JenkinsEnterpriseMaster-N

JenkinsEnterpriseMaster-1

…

©2014 CloudBees, Inc. All Rights Reserved

Master Slave Setup Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

JenkinsEnterpriseMaster-N

JenkinsEnterpriseMaster-1

…

Master-1 BuildNodes

Master-N BuildNodes

©2014 CloudBees, Inc. All Rights Reserved

High Availability Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

HA Proxy

JenkinsEnterpriseMaster-N

JenkinsEnterpriseMaster-1

HA Proxy

NAS Storage

NAS Storage

backup

backup

…

©2014 CloudBees, Inc. All Rights Reserved

Manage Multiple Masters

Jenkins Enterprise Master-N

Jenkins Enterprise Master-1

Jenkins Operations Center

JenkinsOperations

Center (JOC)

config

configJenkins

EnterpriseMaster-N

JenkinsEnterpriseMaster-1

…

Master-1 BuildNodes

Master-N BuildNodes

©2014 CloudBees, Inc. All Rights Reserved

Shared Build Nodes Jenkins Enterprise Master-1

Jenkins Operations Center

Shared BuildNodes

Jenkins Enterprise Master-N

JenkinsEnterpriseMaster-N

Master-1 BuildNodes

JenkinsOperations

Center (JOC)

JenkinsEnterpriseMaster-1

confi

g

config

…

©2014 CloudBees, Inc. All Rights Reserved

Testing Environment Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

Jenkins Operations Center

JenkinsEnterpriseMaster-NJenkins

OperationsCenter (JOC)

Jenkins Enterprise Test Environment

JenkinsEnterprise

Test

JenkinsEnterpriseMaster-1

config

config

config

…

©2014 CloudBees, Inc. All Rights Reserved

Global Security Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

Jenkins Operations Center

JenkinsEnterpriseMaster-N

Role Based AccessControl Plugin

HA Proxy

DEV BuildNodes

HA Proxy

JenkinsOperations

Center (JOC)

Jenkins Enterprise Test Environment

JenkinsEnterprise

Test

HA Proxy

JenkinsEnterpriseMaster-1

Access masters

HA Proxy

Security config pushed to masters

confi

g

config

config

Security config pushed to masters

LDAP…

audit

audit

audit

audit

©2014 CloudBees, Inc. All Rights Reserved

Unified Navigation and SSO

Jenkins Enterprise Test Environment

JenkinsEnterprise

TestHA Proxy

Jenkins Enterprise Master-1

HA Proxy JenkinsEnterpriseMaster-1

Jenkins Operations Center

JenkinsOperations

Center (JOC)HA Proxy

config

config

config

Jenkins Enterprise Master-N

HA Proxy

JenkinsEnterpriseMaster-N

http://master-1.jenkins.mycie.com

http://test.jenkins.mycie.com

…http://master-n.jenkins.mycie.com

http://joc.jenkins.mycie.com

Acce

ss m

aste

rs (S

SO)

©2014 CloudBees, Inc. All Rights Reserved

Update Centers Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

Jenkins Operations Center

Master-1 UpdateCenter

JenkinsEnterpriseMaster-NUpdate

CenterJenkins

OperationsCenter (JOC)

Jenkins Enterprise Test Environment

JenkinsEnterprise

Test

Master-1Test UpdateCenter

Test UpdateCenter

Promote plugins

Master specific Update CenterJenkins

EnterpriseMaster-1

confi

g

config

config

Internet

UpdateCenter

http://jenkins-ci.org

…

©2014 CloudBees, Inc. All Rights Reserved

Jenkins Reference Architecture Jenkins Enterprise Master-1

Jenkins Enterprise Master-N

Jenkins Operations Center

Shared BuildNodes

UpdateCenter Jenkins

EnterpriseMaster-N

Role Based AccessControl Plugin

Master-1 UpdateCenter

HA Proxy

Master-1 BuildNodes

HA Proxy

JenkinsOperations

Center (JOC)

Jenkins Enterprise Test Environment

JenkinsEnterprise

TestHA Proxy

Master-1 Test UpdateCenter

Test UpdateCenter

Promote plugins

Master specific Update CenterJenkins

EnterpriseMaster-1

Access masters (SSO)

HA Proxy

Security config pushed to masters

confi

g

config

config

LDAP …

DEV, QA, OPS, admins

©2014 CloudBees, Inc. All Rights Reserved

Continuous Delivery is

•  People  •  Process  •  Tools  

©2014 CloudBees, Inc. All Rights Reserved