Integrating effective testing & automation for successful ...· Integrating effective testing & automation

  • View

  • Download

Embed Size (px)

Text of Integrating effective testing & automation for successful ...· Integrating effective testing &...

  • 1 CSS Corp | Confidential | CSS Corp | Confidential |

    Integrating effective testing & automation for successful adoption of DevOps

    20th May, 2014SCQAA-Orange County, CA

  • 2 CSS Corp | Confidential |


    Anil Parambath

    Vice President - Solution Engineering

    PRabhu Meruga

    Director - Solution Engineering

  • 3 CSS Corp | Confidential |


    What is DevOps-History, Evolution and Prominence

    Why DevOps now? Todays Application development needs

    Deployment statistics of todays modern world

    Build Failures impact

    Development life cycle survey results

    Challenges Collaboration, Release and Integration testing

    Drivers for DevOps

    Elements of DevOps

    Testing framework for DevOps

    Integrating continuous testing into DevOps understand the differences

    how to integrate testing into DevOps? What are the tools available

    Difficulties, Barriers & Benefits


  • 4 CSS Corp | Confidential |

    DevOps history, evolution and prominence


    (software engineering)

    Technology Operations (IT teams)

    Quality Assurance


    Started in Belgium as DevOps

    days in 2009 by Patrick

    Cross departmental integration concept

    Idea Culture Practice

  • 5 CSS Corp | Confidential |

    Build, listen. Mash-up scenarios. Mobile first

    Develop Fast. Succeed Fast

    Ease of use. Avoid feature overload. Dont replace Simplicity



    Quality, Performance, UsabilityCoverage

    Why DevOps? Todays dynamic application development needs

  • 6 CSS Corp | Confidential |

    Deployment challenges of modern world

    Social networking sites such as face book on average have minimum 2 builds per day and maximum 7 builds per day

    Open source code sharing firms like have 500 builds/day and 100 deploys/day

  • 7 CSS Corp | Confidential |

    Development methodology survey results from Agile conference

    Gartner: By 2015, DevOps will emerge as niche strategy employed by large cloud providers into a mainstream strategy for 20% of the global 2000 Organizations.

  • 8 CSS Corp | Confidential |

    Collaboration challenges between Dev, QA and Ops

    Development completed

    Deployment completed

    Incorrect version deployed

    Checked it again its the right version

    Oops we picked the wrong code

    Now we got the right one





    io 2 Request for new

    environment setupPlease wait we are busy




    Request for new environment setup

    I have daily buildCannot support daily build

    Consistent build deployment needed

    and many other scenarios.

  • 9 CSS Corp | Confidential |

    Release flavors

    Hourly Daily Weekly Fortnightly Monthly Quarterly Half Yearly Yearly

    Business Critical systems-Social Media, Space, Defense, e-Commerce, financial

    Some of the changes in business critical systems are pushed on an hourly basis

    Release vary based on the criticality up to quarterly releases

    Large scale ERP updates, Operating system patches, Anti virus updates, Embedded systems, traditional internal applications

    Some of the changes are very slow depending on the nature of business and components involved. Eg Hardware/embedded

  • 10 CSS Corp | Confidential |

    Integration testing becomes more complex




    Continuous Integration server

    Submit Code branch

















    Production Farm

    QA Lab

    Performance Lab

    UAT Lab

    Operations Lack of controls in Parallel Dev methodology

    Heterogeneous channels PC, laptop, Mobile

    Systems Integration Mobile apps, ERP, Web, Client

    API testing ( too many bugs escape downstream)

    Lack of performance engineering & automation

    Multiple environments to setup and maintain Huge number of production applications

    monitoring Lack of visibility into customer experience

  • 11 CSS Corp | Confidential |

    Drivers for DevOps implementation

    My enterprise has waterfall modelWe are in agile mode

    Need for on the fly multiple environments creation and maintenance

    Code synchronization challenges across environments

    Business demanding frequent releases to catch up with Users demand

    Increased usage of mobile devices and applications



  • 12 CSS Corp | Confidential |

    Elements of DevOps

    Continuous Integration

    Continuous Deployments

    Test Driven Development

    Continuous Automation testing

  • 13 CSS Corp | Confidential |

    DevOps can resolve the challenges and ensure quality of applications.

    Continuous integration tools- Jenkins, Selenium grid

    Automated build deployment process Automation build



    IT Operations

    QA/Testing team

    User Community

    Automated monitoring of the deploys

    Jump into action in case of deadlocks

    Automated daily builds into production Faster release cycles for new features No dependency on IT operations human

    resources Alert management systems

  • 14 CSS Corp | Confidential |

    With and Without DevOps understand the differences

    Automated monitoring of the deploys leveraging DevOps tools

    Has visibility into application monitoring for better root cause analysis

    Reduces dependency on operations

    Very focused on development and coding

    Line of responsibility does not encourage cross skill set training

    Raises tickets to deploy code, monitor application logs, set up environment

    Very focused on waiting for the code deploy

    Line of responsibility does not encourage creating code snippets for any kind of automation

    Raises tickets to deploy code, set up environment

    Continuous automation testing for regression

    Has visibility into security testing and application monitoring

    On the fly environments creation for QA needs

    Very focused on operations alone

    No perspective on developing deployment scripts

    Low focus on validation and purpose of monitoring

    Takes time to create environments

    Motivated to create or design code for various support activities

    Environment creation and monitoring made easy

    Roll back, Patch updates no dependency on Development teams

    Complements additional skill set

    Sharing of efforts and cross pollination DevOps

    Without DevOps Silo operations

    Dev QA Operations

  • 15 CSS Corp | Confidential |

    Testing framework for DevOps implementation


    QA Testing(PC, Laptop & Mobile)

    UAT Production

    Build Failure

    Build Pass

    Defect Management Tool & process

    QA Pass UAT Pass

    Test Cases repository (Automated scripts) leverage Selenium, Test link integration

    Continuous integration coupled with continuous automation

  • 16 CSS Corp | Confidential |

    What will it look like after DevOps implementation?

    Sprint 1 Sprint 2 Sprint N


    Functional AutomationNon Functional Automation

    Automated Unit Testing

    Automated Unit Testing

    Automated Unit Testing

    Functional AutomationNon Functional Automation

    Functional AutomationNon Functional Automation

    Continuous Automated regression testing

    Continuous Automated integration testing




  • 17 CSS Corp | Confidential |

    Tools for DevOps implementation

    Deployment automation : Fai, KickStart, preseed, Cobbler

    Configuration management : Puppet, Chef, cfengine, bcfg2

    Build Automation :Jenkins, ant, Cruise control, Hudson

    Test Automation : QTP, Autoit, Testlink, Test Complete, CA LISA Test

  • 18 CSS Corp | Confidential |

    How do I start Dev Ops implementation

    Perks and Benefits Starbucks cards etc

    Join Developer, QA and Operations team members

    Start small with a mini project, assess the success factors

  • 19 CSS Corp | Confidential |

    Difficulties & Barriers for implementing DevOps

    Common Cultural barriers 49 percent of respondents who

    had no plans to implement DevOps identified a lack of manager buy-in as a blockerthis was the most frequently identified barrier for this group, followed by "lack of team buy-in" (38%).

    48 percent of all respondents indicated that one of the biggest difficulties in implementing DevOps was that the value wasn't understood outside of their group.

    Source: Puppetlabs

  • 20 CSS Corp | Confidential |

    Benefits of DevOps What kind of metrics will help assess DevOps?

    Achieve 25 times more frequent code deployments

    Reduce build failures/outages by 60 times

    Key Metrics Number of Deployments Number of Successful Builds Change Deployment Lead time Validation time per change Quality of deployed change

  • 21 CSS Corp