Integrating effective testing & automation for successful ...· Integrating effective testing & automation

1 CSS Corp | Confidential | www.csscorp.com CSS Corp | Confidential | www.csscorp.com

Integrating effective testing & automation for successful adoption of DevOps

20th May, 2014SCQAA-Orange County, CA

2 CSS Corp | Confidential | www.csscorp.com

Authors

Anil Parambath

Vice President - Solution Engineering

PRabhu Meruga

Director - Solution Engineering

3 CSS Corp | Confidential | www.csscorp.com

Agenda

What is DevOps-History, Evolution and Prominence

Why DevOps now? Todays Application development needs

Deployment statistics of todays modern world

Build Failures impact

Development life cycle survey results

Challenges Collaboration, Release and Integration testing

Drivers for DevOps

Elements of DevOps

Testing framework for DevOps

Integrating continuous testing into DevOps understand the differences

how to integrate testing into DevOps? What are the tools available

Difficulties, Barriers & Benefits

Q&A

4 CSS Corp | Confidential | www.csscorp.com

DevOps history, evolution and prominence

Development

(software engineering)

Technology Operations (IT teams)

Quality Assurance

(testing)DevOps

Started in Belgium as DevOps

days in 2009 by Patrick

Cross departmental integration concept

Idea Culture Practice

5 CSS Corp | Confidential | www.csscorp.com

Build, listen. Mash-up scenarios. Mobile first

Develop Fast. Succeed Fast

Ease of use. Avoid feature overload. Dont replace Simplicity

Speed

Innovation

Quality, Performance, UsabilityCoverage

Why DevOps? Todays dynamic application development needs

6 CSS Corp | Confidential | www.csscorp.com

Deployment challenges of modern world

Social networking sites such as face book on average have minimum 2 builds per day and maximum 7 builds per day

Open source code sharing firms like github.com have 500 builds/day and 100 deploys/day

7 CSS Corp | Confidential | www.csscorp.com

Development methodology survey results from Agile conference

Gartner: By 2015, DevOps will emerge as niche strategy employed by large cloud providers into a mainstream strategy for 20% of the global 2000 Organizations.

8 CSS Corp | Confidential | www.csscorp.com

Collaboration challenges between Dev, QA and Ops

Development completed

Deployment completed

Incorrect version deployed

Checked it again its the right version

Oops we picked the wrong code

Now we got the right one

Scen

ario

1Sc

enar

io 2 Request for new

environment setupPlease wait we are busy

Scen

ario

3

Request for new environment setup

I have daily buildCannot support daily build

Consistent build deployment needed

and many other scenarios.

9 CSS Corp | Confidential | www.csscorp.com

Release flavors

Hourly Daily Weekly Fortnightly Monthly Quarterly Half Yearly Yearly

Business Critical systems-Social Media, Space, Defense, e-Commerce, financial

Some of the changes in business critical systems are pushed on an hourly basis

Release vary based on the criticality up to quarterly releases

Large scale ERP updates, Operating system patches, Anti virus updates, Embedded systems, traditional internal applications

Some of the changes are very slow depending on the nature of business and components involved. Eg Hardware/embedded

10 CSS Corp | Confidential | www.csscorp.com

Integration testing becomes more complex

Dev1

Dev2

Dev3

Continuous Integration server

Submit Code branch

ERP

ch

ange

sPa

tch

u

pd

ates

Mo

bili

ty

enh

ance

me

nts

Deployment

Production Farm

QA Lab

Performance Lab

UAT Lab

Operations Lack of controls in Parallel Dev methodology

Heterogeneous channels PC, laptop, Mobile

Systems Integration Mobile apps, ERP, Web, Client

API testing ( too many bugs escape downstream)

Lack of performance engineering & automation

Multiple environments to setup and maintain Huge number of production applications

monitoring Lack of visibility into customer experience

11 CSS Corp | Confidential | www.csscorp.com

Drivers for DevOps implementation

My enterprise has waterfall modelWe are in agile mode

Need for on the fly multiple environments creation and maintenance

Code synchronization challenges across environments

Business demanding frequent releases to catch up with Users demand

Increased usage of mobile devices and applications

Dri

vers

12 CSS Corp | Confidential | www.csscorp.com

Elements of DevOps

Continuous Integration

Continuous Deployments

Test Driven Development

Continuous Automation testing

13 CSS Corp | Confidential | www.csscorp.com

DevOps can resolve the challenges and ensure quality of applications.

Continuous integration tools- Jenkins, Selenium grid

Automated build deployment process Automation build

qualification(testing)

Developers

IT Operations

QA/Testing team

User Community

Automated monitoring of the deploys

Jump into action in case of deadlocks

Automated daily builds into production Faster release cycles for new features No dependency on IT operations human

resources Alert management systems

14 CSS Corp | Confidential | www.csscorp.com

With and Without DevOps understand the differences

Automated monitoring of the deploys leveraging DevOps tools

Has visibility into application monitoring for better root cause analysis

Reduces dependency on operations

Very focused on development and coding

Line of responsibility does not encourage cross skill set training

Raises tickets to deploy code, monitor application logs, set up environment

Very focused on waiting for the code deploy

Line of responsibility does not encourage creating code snippets for any kind of automation

Raises tickets to deploy code, set up environment

Continuous automation testing for regression

Has visibility into security testing and application monitoring

On the fly environments creation for QA needs

Very focused on operations alone

No perspective on developing deployment scripts

Low focus on validation and purpose of monitoring

Takes time to create environments

Motivated to create or design code for various support activities

Environment creation and monitoring made easy

Roll back, Patch updates no dependency on Development teams

Complements additional skill set

Sharing of efforts and cross pollination DevOps

Without DevOps Silo operations

Dev QA Operations

15 CSS Corp | Confidential | www.csscorp.com

Testing framework for DevOps implementation

Development(agile/iterative)

QA Testing(PC, Laptop & Mobile)

UAT Production

Build Failure

Build Pass

Defect Management Tool & process

QA Pass UAT Pass

Test Cases repository (Automated scripts) leverage Selenium, Test link integration

Continuous integration coupled with continuous automation

16 CSS Corp | Confidential | www.csscorp.com

What will it look like after DevOps implementation?

Sprint 1 Sprint 2 Sprint N

Development

Functional AutomationNon Functional Automation

Automated Unit Testing

Automated Unit Testing

Automated Unit Testing

Functional AutomationNon Functional Automation

Functional AutomationNon Functional Automation

Continuous Automated regression testing

Continuous Automated integration testing

Release

..

..

17 CSS Corp | Confidential | www.csscorp.com

Tools for DevOps implementation

Deployment automation : Fai, KickStart, preseed, Cobbler

Configuration management : Puppet, Chef, cfengine, bcfg2

Build Automation :Jenkins, ant, Cruise control, Hudson

Test Automation : QTP, Autoit, Testlink, Test Complete, CA LISA Test

18 CSS Corp | Confidential | www.csscorp.com

How do I start Dev Ops implementation

Perks and Benefits Starbucks cards etc

Join Developer, QA and Operations team members

Start small with a mini project, assess the success factors

19 CSS Corp | Confidential | www.csscorp.com

Difficulties & Barriers for implementing DevOps

Common Cultural barriers 49 percent of respondents who

had no plans to implement DevOps identified a lack of manager buy-in as a blockerthis was the most frequently identified barrier for this group, followed by "lack of team buy-in" (38%).

48 percent of all respondents indicated that one of the biggest difficulties in implementing DevOps was that the value wasn't understood outside of their group.

Source: Puppetlabs

20 CSS Corp | Confidential | www.csscorp.com

Benefits of DevOps What kind of metrics will help assess DevOps?

Achieve 25 times more frequent code deployments

Reduce build failures/outages by 60 times

Key Metrics Number of Deployments Number of Successful Builds Change Deployment Lead time Validation time per change Quality of deployed change

21 CSS Corp