Service Virtualization: Delivering Complex Test Environments on Demand

Parasoft Proprietary and Confidential 1

2014-11-12

Service VirtualizationDelivering Complex Test Environments on Demand

Frank Jennings; Director, Performance Test & Engineering, TQM, Comcast

Mark Lambert; VP Products and Support, Parasoft Corporation

Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2

Drivers of the SDLC

SDLC Speed is the

difference between a

first mover and a

follower

Damages associated

with software failure

are increasing and

very real

Brand equity is critical

and Quality drives

brand loyalty


Current Pressures of the SDLC

Adoption of Agile to meet Acceleration

Requires Continuous Infrastructure

Need to assess Business Risk of rapidly changing code

Needs Continuous Testing with automated Quality Gates

Unit Test is not enough

Testing complexity grows further into the SDLC

DevOps requires infrastructure to “shift left” On-demand Environment access


Complexity is a Barrier to Innovation

3rd Party System

Evolving ComponentMainframe

Scheduled Access


An application is not just “one application” …


… it’s made up of „several components” …


… and they are not all ready at the same time.


How do we ensure quality of the components?

Static Analysis, Unit Testing, Code Review


How to test without dependant components?

API testing

Reusable Virtual Assets


Test as soon as the components are ready


Service Virtualization delivers simulated application behavior

in dev / test environmentsallowing an organization to test

on-demand

Service Virtualization


How to apply Service Virtualization

1. Identify the assets;

Which types of asset are good to simulate?

2. Model behaviour;

How much is enough?

3. Deploy and reuse;

Provide easy ad-hock access and sharing

4. Automate Provisioning;

Integrated into a Continuous Test Environment


When to use SV? Access vs Configuration

Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications

Configuration Complex to configure No control Limited variability Consumes test time

Test Access

Simple Difficult

Configura

tion

Complex

Internal

Service

Internal

Application 3rd Party

Service

ERP

Mainframe

External

Application

External

Database

Internal

Database

Cloud

App

Message

Queue

ESB

Web

Server


When to use SV? Access vs Configuration

Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications

Configuration Complex to configure No control Limited variability Consumes test time

Test Access

Simple Difficult

Configura

tion

Complex

Internal

Service

Internal

Application 3rd Party

Service

ERP

Mainframe

External

Application

External

Database

Internal

Database

Cloud

App

Message

Queue

ESB

Web

Server


Virtual Lab / Server

Virtualization

(i.e. VMWare)


The Total Cost of System Access

Service Virtualization dramatically reduces the cost of

delivering a dev / test environment

Hardware Savings

Access EfficiencyStaged w/ Virtual Test Lab

Staged Application Instance

System Behavior

Cost

Only manage the functionality you need

Only manage data you need Low-Risk “disposable instance”

that can be accessed at anytime Dramatically reduce test

environment setup and management



Setting up Service Virtualization

Define &

Capture

Initiated from the system under test,

the user has the ability to capture

detail from a live monitor that

analyzes system traffic, from

analyzing transaction logs or by

modeling virtual behavior within the

Parasoft Virtualized interface.

Model &

Deploy

After the virtualized artifact has been

captured, users can now instruct the

details of the virtualized asset

behavior. This includes: performance,

data sources and conditional response

criteria. The virtualized asset is then

provisioned for simplified uniform

access across teams and business

partners.

Provision &

Consume

The virtualized asset can now be called

for unit, functional and performance tests.

The virtualized asset can be leveraged by

any test suite – including Parasoft Test.


Example: Capturing current behavior

ApplicationUnder Test

2 Capture

4 Deploy

3 Create

Application

Database

Service

QA and Test

Performance TestEngineer

Mainframe

Define Monitors1

Development

Service

Test

LoadRunner


Provision the Environment on Demand

ApplicationUnder Test

Application

Database

Service

Mainframe

EnvironmentManagement

5 Manage

6 Consume

QA and Test

Performance TestEngineer

Development

QC/ALM

Rational

Continuous Integration


Rapid Environment Access


Reduced wait time for test team by 60% for a major media conglomerate

The Challenge

Large agile development effort to adopt Service Oriented Architecture (SOA)

High risk project but the Test team “stuck waiting for the first build”

Development of functionality was not easy to coordinate as different teams had different schedules; not all finished at the same time

Agile/Parallel development limited by system dependencies

Iteration

Iteration

Iteration

Team A

Team B

Team C

Iteration Iteration

Iteration Iteration Iteration

Iteration Iteration

Current

Development/Testing

dependencies



The Solution

Use descriptions of the new services (WSDL, XSD, example JSON payloads) to build Virtual assets prototyping the new functionality.

Test team builds tests with against the prototypes with SOAtest and the independent development tests use the prototypes to perform early stage Integration Testing


Iteration

Iteration

Iteration

Team A

Team B

Team C

Iteration Iteration

Iteration Virtual Asset Iteration

Virtual Asset Iteration

Current

Development/Testing

dependencies



The Business Benefits

Met business goals and timelines, were able to test functionality “as soon as” it was available. Practiced TDD against prototypes to get a head-start on ‘full system testing’


Iteration

Iteration

Iteration

Team A

Team B

Team C

Iteration Iteration

Iteration Iteration Iteration

Iteration Iteration

✔✖✔

✖✔✖


Provisioning of test environments went from weeks to minutes from large government organization

The Challenge Test results did not reflect actual performance in production

Inability to test multiple projects at once

Testing environments were unstable and unreliable

Limited access to mainframes, ERPs, or 3rd party systems



The Solution Combining Service Virtualization and VMWare technology to build a

testing environment that properly replicates the production environment while allowing multiple virtual copies of that environment to be provisioned on-demand.


PV

PV

PVPV

PV

PV

PV

PV



The Business Benefit Developers empowered to provision their own test environments.

Provisioning delays reduced from weeks to minutes.

Project reliability enhanced.


&


Reduced cost of creation and management of Test Environments by 85% for a major international bank

The Challenge

Complex test environments including; ATM, website and mobile front-end systems with WebSphere, Mainframe, MQ, SAP and DataPower back-end systems

Each test system costs >$2m, therefore only two environments available

Active development at every level of the system resulting in full end-to-end testing requiring careful coordination and sub-system teams often offline or unavailable due to upgrades

Database

SAP

Mainframe

Application

Mobile Application development and extension of current systems


The Solution

Create a library of virtual assets for teams to share and use Environment Manager to replicate and manage Virtual test labs using different performance charactistics for different load scenarios.

Use automated test solution to test and validate at each layer independantly




Database

SAP

Mainframe

Application

Virtual AssetVirtual

AssetVirtual Asset


Asset

Virtual Asset


AssetVirtual AssetVirtual

Asset

Virtual Asset Virtual

Asset

Virtual Asset

✔✖


The Business Benefit

Improved efficiency, enabling Agile development, with the teams able to test in isolation without effecting full end-to-end system tests.




✔✖

✔✖

✔✖

✔✖


Comcast Case Study: Applying Service Virtualization to

Performance Testing


Network Provising

Inventory Billing

Enterprise Services Portal

Location DB

Commercial Residential

Fron

tend

OSS / B

SSM

idd

leware

Ne

two

rk

Order Entry

CustomerCare

SelfService

Order EntryCustomer

CareSelf Service eCommerce

Voice Response

System

Commercial Services Platform

Ticketing

Inventory

Authentication

Offer MgmtCustomer &

AccountProvisioning Location CDV Services

Event Mgmt

Customer & Account

Activation

Order Fullfillment

Order Mgmt

Business Process Mgmt

Order Management

Billing

Self Install Process

UI Widget Warehouse

Knowledge Management

3rd PartyNetworkServices

eMTASTB

ModemsWifi

IMS

CMTSNetwork ElementStatus

Networks Services

TN Inventory Managment

E911 Services

Customer DB Offer DB

Performance Team Test CoverageTest TargetVirtualizedTest with Virtualize

Commercial DBs


System Under Test

Test Automation

Services & DBs

Virtualized Services

Real End Points

Development

Deployment Services Regression Performance Manual Test

QA Stage Production

Unit

Intelligent Proxy

DeploymentAutomation

Continuous Deployment Engine

Code

DeployDev

DeploymentValidation

Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression

FunctionalDeploymentValidation

PerformanceDeploymentValidation


System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression




System Under Test

Test Automation

Services & DBs


Real End Points

Development


QA Stage Production

Unit

Intelligent Proxy



Code

DeployDev


Code Quality

Regression

BaselinePerformance

DeployQA

DeployStage

DeployProd


Regression





Emulates dependencies for the Test Environment

Reduces the complexity for early stage testing

Increases predictability

On-Demand “Test Anytime, Anywhere, Anyway”

Automated Provisioning for different use-cases

Automated Test Data Management/Simulation

Remember: Does not eliminate the need for System/Integration Testing

Parasoft Proprietary and Confidential 46

2014-11-12

Thank you

Questions?

Software

Service Virtualization: Delivering Complex Test Environments on Demand