How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery Pipeline

1©2015SOASTAInc.-Allrightsreserved.

InThisWebinar

TODAY’SPRESENTERS

TomChavez:SOASTA,Sr.ProductMarkeEngManagerAndyPemberton:CloudBees,SeniorDirector,SoluEonsArchitecture&ConsulEngTWOCOMPANIES,ASINGLESOLUTION

SOASTA:TheleaderinCloudandMobileTesEng

CloudBees:TheEnterpriseJenkinsCompany

Together,weofferacompletecloud-basedservice:

ConEnuousLoadTesEng&ConEnuousDeliverywithJenkins

QuesEons?AskinQuesEonwindow


PollQuesEon

WhatbestdescribesyourstatuswithConEnuousIntegraEon/ConEnuousDelivery(CI/CD)?(singleanswer) a)  DeterminingifCI/CDisagoodopEon

a)  ImplemenEngCI/CD

a)  PracEcingCI

b)  PracEcingCD

c)  WhatisCI/CD?


Agenda•  LoadtesEng:why?•  Whydon’tweloadtestearlier?•  ConEnuousLoadandPerformanceTesEng•  CloudBeesandtheEnterpriseJenkinsPlaYorm•  ConnecEngloadteststoyourCDPipeline•  Q&AQuesEons:SubmitinthequesEonboxduringevent


WhyShouldWeLoadTest?

o  Ensurethatsitewillperformwellatload•  TesthardwareconfiguraEons

•  TestapplicaEon,database,components

•  Testthirdpartypartners

o  Tobereadyforpeakseason•  BlackFriday,CyberMonday,Christmas

•  Taxfiling,Insurancefiling,SuperBowlcommercial


WhyShouldWeLoadTest?

7©2015SOASTA,Inc.-Allrightsreserved.

WhyDon’tWeLoadTestEarlier?

Copyright © 2014 SOASTA, Inc. - All rights reserved.


WhyDON’TWeLoadTestEarlier?

“Because it’s hard.” o  Tools require coding skills we don’t have (C, VB?) o  We don’t have a scalable test environment o  We don’t have hardware for load generation o  It’s a different tool o  There are too many other tests to do




“Because it takes too long.” o  Scripting takes forever o  Results are slow o  Fast setup of environments is impossible o  We’re dependent on IT or other teams o  We’re too agile





“Because no one ever told us to.” o  There are no “performance coverage” requirements o  There isn’t a performance baseline for reference o  There isn’t a defined performance target o  No one is managing to a performance metric





“Because no one ever told us to.” o  There are no “performance coverage” requirements o  There isn’t a performance baseline for reference o  There isn’t a defined performance target o  No one is managing to a performance metric

Until Now.


PollQuesEon

WhatdoyouseeholdingbackamoreiteraEveloadtesEngapproach?(checkallthatapply)

a)  Skillsgaps

b)  Lackoftestenvironment(s)

c)  Slowtestdevelopment

d)  TesttoolslimitaEons

e)  3rdPartyDependencies

f)  UndefinedPerformanceRequirements

g)  LackofManagementPush

h)  Other


LoadTesEngHasBeenanEliteEvent

STheIssue:

• HighexpectaEonsfromtestsrunwithdiminishingEmewindows

• Code-levelperformancebugsfoundlatesetenEreprojectback

• Separateteamandscheduleconflictwithdeliverygoals

• MajorissuesescapetoproducEon

o  pecialtyskillsandtasksarealegacybarrier

Iteration 1

Automated unit tests

Iteration 2

Automated unit & functional tests

Iteration 3

Regression test suite

Iteration n

Regression test suite

Load & Performance Testing

Deploy

Issues?


ConEnuousLoadTesEngValidateperformancepiecebypiece

Iteration 1 Automated unit

tests

Deploy

Production perf tests

Iteration 2 Automated unit & functional tests

Iteration 3 Regression test

suite Incl:

Iteration n Regression test

suite Incl:

End to End Production Scale

Test Small perf test Small perf tests

Perf smoke tests Perf smoke tests

TheSoluEon:•  RunincrementalperformancetestsautomaEcallywitheachiteraEon

•  Establishatrendlineforperformanceearly•  Developersfindcodeandarchitectureissuesearly

•  Performanceteamsfocusonend-to-endinfrastructureanduserrelatedtesEng

•  MinimizeproducEonescapesandimplementproducEontesEng

Development/Test team Performance/Ops team DevOps


ConEnuousPerformance(forWeb&Mobile)ThePathtoaPerformanceCulture I JUST MADE A

CODE CHANGE

CloudTest runs load tests via Jenkins Daily (or more frequent) updates keep

teams “performance focused”

Identify bottlenecks & inform teams,

just like unit and functional

tests in Jenkins

Jenkins launches suitable cloud-based test environment

Manage with a new metric!


PollQuesEon

WhattypesofLoadTesEngareyoudoingtoday?(SingleAnswer)

a)  NotdoinganyloadtesEng

b)  Non-WebProtocols(notmobile)

c)  Primarilywebprotocols

d)  Primarilymobilewebtraffic

e)  Bothwebandmobile

f)  Allweb,mobile&non-webprotocols

17©2015SOASTA,Inc.Allrightsreserved.

EnablingConEnuousPerformanceTesEngwithJenkinsPipeline

Andy Pemberton Twitter: @apemberton

Email: [email protected]


HaveyoumetJenkins?

#1 CI/CD Automation Engine

•  Open Source

•  Highly Extensible

About 8 years old

Easy to install/use

1000+ plugins

Widely adopted

•  100K+ installations


Jenkins is the #1 Automation Platform Used for CI

Continuous Integration

Developer

Source Code

Control System

Compile/Build Code

Scan Tests Code Commit

Results Report

Build Commit Test


Jenkins is Also Your Key to Continuous Delivery

Developer

Source Code

Control System

Compile/Build Code

Scan Tests Code Commit

Results Report

Build Commit Test Stage Deploy

Deploy Prod

Deploy Stage

Complex Delivery Pipelines

Delivery of App and Config

Continuous Delivery

Prod Dev


Over 1000 Jenkins Plugins for Tool Integration

Integrated Development Environments (IDEs), editors/compilers/debuggers, code review and collaboration tools.

Source code control systems and related asset management tools.

Automated API-based testing frameworks.

Release management and release automation software

Automated API-based testing frameworks, static code analysis tools.

Load Testing Tools and Security Scanning technology

Automated and manual UAT tools

Release management, release automation, and change control software.

Environment and Configuration Management Software Develop

Commit

Build & Integrate

Unit Tests

Release (to Test)

Automated Regression Tests

Release (to Prod)

User Acceptance Tests

Load & Security Tests


Jenkins is the Hub of the CD Ecosystem

On-premise or in the cloud

Plug-ins for all your tools

Run Deploy Stage Test Build Commit


AreYouUsingToday?

23

Chances are someone in your

organization is.

24©2015SOASTAInc.-Allrightsreserved.24


How Do You Manage Jenkins and Testing at Enterprise Scale?


About CloudBees

Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO We optimize Jenkins for the enterprise

•  Jenkins Operations Center

•  Jenkins OSS Support

• Enterprise-class plugins

• Commit code and support the Jenkins Community

•  Jenkins training and consulting

•  Jenkins User Conferences (JUC) and newsletters

…we are the Jenkins Enterprise experts!


CloudBees Jenkins Enterprise

27

Professional support

High availability

Large installations

Security

Optimized utilization

Continuous Delivery


Additional Jenkins Services from CloudBees

28

Jenkins Certified Training •  Delivered via CloudBees partners

Jenkins Consulting •  Delivered via CloudBees Professional Services Team

Jenkins Newsletter •  Community news and expert tips

Jenkins User Conferences •  Learn from other community members


HowCanCloudBeesJenkinsEnterpriseHelpwithYourCDStrategy?

29

PipelineStage View Provides deep insight into the execution of your workflow stages so you can detect problems and performance issues Manage complex “level-of-detail” testing pipelines

Jenkins Pipeline Created by CloudBees to provide Jenkins with a way to define flexible delivery pipelines that allow for a rich set of new delivery scenarios Manage and share “testing-as-code”

Dynamic Environment Creation Spin up new test/deployment environments automatically as part of your delivery pipeline

Deployment Tracking and Verfiication Track the history of your deployments – including who kicked off a deployment and what code changes were included in it Create pipelines that leverage verification tests to determine the quality of a new deployment

Jenkins Analytics Monitor and analyze performance across all your Jenkins instances Monitor, measure and manage adoption of testing, CI and CD best practices Restart builds, tests, and deployments where they

break Leverage pipeline checkpoints to restart delivery pipelines without a full (expensive) rebuild

...central source of orchestration to bridge dev, test, and ops alike

And more…


#Jenkins2GreatlyEnhancedUI/UXforeasierstartupandbeOersecurity


#Jenkins2JenkinsPipeline-as-Codefrontandcenter


Jenkins Pipeline and Pipeline Stage View Enable Full Continuous Delivery Pipelines

Continuous Delivery

Prod Dev

Developer

Source Code

Control System

Code Commit


CloudBees Jenkins Pipeline Stage View

Restartable Checkpoints

Artifact Downloads


Monitoring, Visualization and Analytics

Jenkins Pipeline Stage View Visibility into Pipeline status

Jenkins Analytics Cross-Team Build Analytics

… among many other features, tailored for enterprise organizations using Jenkins @ scale



ConEnuousLoadTesEng

o  SOASTACloudTestComponents/Architecture

o  Demo:

•  CloudTestUIOverview

•  JenkinsSetup•  DownloadingandinstallingCloudTestplug-in

•  CreaEngaPerformanceJobinJenkins

•  CreaEngaPipelineinJenkins

•  ReviewingResults

Whatarewegoingtoshow


Architecture:

Jenkins Jill

Sri Raj

DevOps Team 1

CloudTest Manager

Always up at: Cloudtestmanager.soasta.com

CloudTest server

any IP Address/ Domainname Up ‘as needed’ at

“US Grid1”

2 Azure California

2 AWS Virginia

2x Google Iowa

“WW Grid1”

2 Azure California

2 AWS Virginia

2x Google Iowa

CloudTest “Main” server


dev.acme.com

CPU,DISK, Memory,I/O Etc.


Jenkins Enterprise on CloudBees

Jill

Sri Raj

DevOps Team 1

CloudTest Manager

Always up at: Cloudtestmanager.soasta.com

dev.acme.com

“US Grid1”

2 Azure California

2 AWS Virginia

2x Google Iowa

CloudTest “Main” server


DemoPreview:


Demo

QuesEons?AskinQuesEonwindow


Resources

ContactSOASTA:www.soasta.com/cloudtest/[email protected]:

twiOer.com/SoastaIncfacebook.com/cloudtest

KnowledgeCenter•  WhitePapers•  WebinarRecordings•  CaseStudiesCloudLinkCommunity•  Support•  Tutorials•  Video

ContactCloudBeeswww.cloudbees.cominfo@cloudbees.com1.781.404.5100Followus:

twiOer.com/cloudbeesfacebook.com/CloudBees

www.SOASTA.com

[email protected]/get-started

Documenta]on•  CloudBees Jenkins Platform •  Workflow in Jenkins WhitePapers:•  Making the Leap from Continuous Integration to

Continuous Delivery •  The Business Value of Continuous Delivery

HowtoIntroduceCloudBasedLoadTesEngtoyourJenkinsConEnuousDeliveryPipelineApril20,2016TomChavez,SOASTA@TomChavezAndyPemberton,CloudBees@apemberton

Technology

How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery Pipeline