View
212
Download
0
Embed Size (px)
1 CSS Corp | Confidential | www.csscorp.com CSS Corp | Confidential | www.csscorp.com
Integrating effective testing & automation for successful adoption of DevOps
20th May, 2014SCQAA-Orange County, CA
2 CSS Corp | Confidential | www.csscorp.com
Authors
Anil Parambath
Vice President - Solution Engineering
PRabhu Meruga
Director - Solution Engineering
3 CSS Corp | Confidential | www.csscorp.com
Agenda
What is DevOps-History, Evolution and Prominence
Why DevOps now? Todays Application development needs
Deployment statistics of todays modern world
Build Failures impact
Development life cycle survey results
Challenges Collaboration, Release and Integration testing
Drivers for DevOps
Elements of DevOps
Testing framework for DevOps
Integrating continuous testing into DevOps understand the differences
how to integrate testing into DevOps? What are the tools available
Difficulties, Barriers & Benefits
Q&A
4 CSS Corp | Confidential | www.csscorp.com
DevOps history, evolution and prominence
Development
(software engineering)
Technology Operations (IT teams)
Quality Assurance
(testing)DevOps
Started in Belgium as DevOps
days in 2009 by Patrick
Cross departmental integration concept
Idea Culture Practice
5 CSS Corp | Confidential | www.csscorp.com
Build, listen. Mash-up scenarios. Mobile first
Develop Fast. Succeed Fast
Ease of use. Avoid feature overload. Dont replace Simplicity
Speed
Innovation
Quality, Performance, UsabilityCoverage
Why DevOps? Todays dynamic application development needs
6 CSS Corp | Confidential | www.csscorp.com
Deployment challenges of modern world
Social networking sites such as face book on average have minimum 2 builds per day and maximum 7 builds per day
Open source code sharing firms like github.com have 500 builds/day and 100 deploys/day
7 CSS Corp | Confidential | www.csscorp.com
Development methodology survey results from Agile conference
Gartner: By 2015, DevOps will emerge as niche strategy employed by large cloud providers into a mainstream strategy for 20% of the global 2000 Organizations.
8 CSS Corp | Confidential | www.csscorp.com
Collaboration challenges between Dev, QA and Ops
Development completed
Deployment completed
Incorrect version deployed
Checked it again its the right version
Oops we picked the wrong code
Now we got the right one
Scen
ario
1Sc
enar
io 2 Request for new
environment setupPlease wait we are busy
Scen
ario
3
Request for new environment setup
I have daily buildCannot support daily build
Consistent build deployment needed
and many other scenarios.
9 CSS Corp | Confidential | www.csscorp.com
Release flavors
Hourly Daily Weekly Fortnightly Monthly Quarterly Half Yearly Yearly
Business Critical systems-Social Media, Space, Defense, e-Commerce, financial
Some of the changes in business critical systems are pushed on an hourly basis
Release vary based on the criticality up to quarterly releases
Large scale ERP updates, Operating system patches, Anti virus updates, Embedded systems, traditional internal applications
Some of the changes are very slow depending on the nature of business and components involved. Eg Hardware/embedded
10 CSS Corp | Confidential | www.csscorp.com
Integration testing becomes more complex
Dev1
Dev2
Dev3
Continuous Integration server
Submit Code branch
ERP
ch
ange
sPa
tch
u
pd
ates
Mo
bili
ty
enh
ance
me
nts
Deployment
Production Farm
QA Lab
Performance Lab
UAT Lab
Operations Lack of controls in Parallel Dev methodology
Heterogeneous channels PC, laptop, Mobile
Systems Integration Mobile apps, ERP, Web, Client
API testing ( too many bugs escape downstream)
Lack of performance engineering & automation
Multiple environments to setup and maintain Huge number of production applications
monitoring Lack of visibility into customer experience
11 CSS Corp | Confidential | www.csscorp.com
Drivers for DevOps implementation
My enterprise has waterfall modelWe are in agile mode
Need for on the fly multiple environments creation and maintenance
Code synchronization challenges across environments
Business demanding frequent releases to catch up with Users demand
Increased usage of mobile devices and applications
Dri
vers
12 CSS Corp | Confidential | www.csscorp.com
Elements of DevOps
Continuous Integration
Continuous Deployments
Test Driven Development
Continuous Automation testing
13 CSS Corp | Confidential | www.csscorp.com
DevOps can resolve the challenges and ensure quality of applications.
Continuous integration tools- Jenkins, Selenium grid
Automated build deployment process Automation build
qualification(testing)
Developers
IT Operations
QA/Testing team
User Community
Automated monitoring of the deploys
Jump into action in case of deadlocks
Automated daily builds into production Faster release cycles for new features No dependency on IT operations human
resources Alert management systems
14 CSS Corp | Confidential | www.csscorp.com
With and Without DevOps understand the differences
Automated monitoring of the deploys leveraging DevOps tools
Has visibility into application monitoring for better root cause analysis
Reduces dependency on operations
Very focused on development and coding
Line of responsibility does not encourage cross skill set training
Raises tickets to deploy code, monitor application logs, set up environment
Very focused on waiting for the code deploy
Line of responsibility does not encourage creating code snippets for any kind of automation
Raises tickets to deploy code, set up environment
Continuous automation testing for regression
Has visibility into security testing and application monitoring
On the fly environments creation for QA needs
Very focused on operations alone
No perspective on developing deployment scripts
Low focus on validation and purpose of monitoring
Takes time to create environments
Motivated to create or design code for various support activities
Environment creation and monitoring made easy
Roll back, Patch updates no dependency on Development teams
Complements additional skill set
Sharing of efforts and cross pollination DevOps
Without DevOps Silo operations
Dev QA Operations
15 CSS Corp | Confidential | www.csscorp.com
Testing framework for DevOps implementation
Development(agile/iterative)
QA Testing(PC, Laptop & Mobile)
UAT Production
Build Failure
Build Pass
Defect Management Tool & process
QA Pass UAT Pass
Test Cases repository (Automated scripts) leverage Selenium, Test link integration
Continuous integration coupled with continuous automation
16 CSS Corp | Confidential | www.csscorp.com
What will it look like after DevOps implementation?
Sprint 1 Sprint 2 Sprint N
Development
Functional AutomationNon Functional Automation
Automated Unit Testing
Automated Unit Testing
Automated Unit Testing
Functional AutomationNon Functional Automation
Functional AutomationNon Functional Automation
Continuous Automated regression testing
Continuous Automated integration testing
Release
..
..
17 CSS Corp | Confidential | www.csscorp.com
Tools for DevOps implementation
Deployment automation : Fai, KickStart, preseed, Cobbler
Configuration management : Puppet, Chef, cfengine, bcfg2
Build Automation :Jenkins, ant, Cruise control, Hudson
Test Automation : QTP, Autoit, Testlink, Test Complete, CA LISA Test
18 CSS Corp | Confidential | www.csscorp.com
How do I start Dev Ops implementation
Perks and Benefits Starbucks cards etc
Join Developer, QA and Operations team members
Start small with a mini project, assess the success factors
19 CSS Corp | Confidential | www.csscorp.com
Difficulties & Barriers for implementing DevOps
Common Cultural barriers 49 percent of respondents who
had no plans to implement DevOps identified a lack of manager buy-in as a blockerthis was the most frequently identified barrier for this group, followed by "lack of team buy-in" (38%).
48 percent of all respondents indicated that one of the biggest difficulties in implementing DevOps was that the value wasn't understood outside of their group.
Source: Puppetlabs
20 CSS Corp | Confidential | www.csscorp.com
Benefits of DevOps What kind of metrics will help assess DevOps?
Achieve 25 times more frequent code deployments
Reduce build failures/outages by 60 times
Key Metrics Number of Deployments Number of Successful Builds Change Deployment Lead time Validation time per change Quality of deployed change
21 CSS Corp