Case Studies in Solving Testing Constraints using Service ...sast.se/q-moten/2014/Q1_2014_Parasoft.pdf · Paraso&’Proprietary’and’Conﬁden1al’ 1 2/21/14 Case Studies in Solving

Paraso& Proprietary and Confiden1al 1

2/21/14

Case Studies in Solving Testing Constraints using Service Virtualization

[email protected]

Paraso& Proprietary and Confiden1al 2 Paraso& Proprietary and Confiden1al 2

•  Paraso& is supplier automated tes1ng solu1ons –  Since 1984, Los Angeles (US) and Krakow (EU) –  Development tes1ng –  API tes1ng –  Service Virtualiza1on

•  Solu1on Architect for Paraso& Nordics –  Since 1992 in Valida1on & Verifica1on (PhD) –  Joined Paraso& 2004, focus Benelux and Scandinavia –  Member of Dutch Testnet since 1999

•  Also, supervisor of Faris Nizamic –  PhD project on tes1ng in distributed systems –  University of Groningen (NL) –  SW project now with University of Linnaeus (S)

Introduction


We are living in a Distributed World

Mobile devices force the industry to re-think the user experience

APIs drive composite apps and interconnecting multiple dependencies

Cloud challenges ingrained concepts about software development


Bus

Bus

Cloud Resource

Partner

Partner

Bus

Bus

ERP

Modern IT systems are a collec1on of many components, o&en exposes as services

Testing in our Distributed World


Bus

Bus

Cloud Resource

Partner

Partner

Bus

Bus

ERP

For tes1ng this: We need this:



•  Service-‐oriented systems characteris1cs: –  Func1onality spread across the network –  Interdependent services with no single owner –  No full control over a (sub)system

•  Test automa1on -‐ sa1sfying maturity level –  Test tools capable suppor1ng automa1on process –  Test automa1on techniques available to test teams

•  YET, Agile so&ware development is being delayed due to of number of constraints imposed by SOA.

Bus

Bus

Cloud Resource

Partner Partner

Bus

Bus

ERP



A survey conducted in 2012, asking so&ware developers, testers, and performance test engineers a series of ques1ons about their access to test environments. The survey results show:

1.  Applica1ons rely on an average of 8 dependent services 2.   30% of 1me is spend on configura1on on test

environment 3.   71% of developers/testers need to schedule test

environment access 4.  Average 1me block for access is 4 hours 5.   50% of the test plan is actually executed 6.   20% of the dependent services' func1onality is required

for dev / test tasks

Survey on Constraints in Testing


Constraints

Availability Schedule Data Other

limited access to a sub-‐system

no access to a sub-‐system

not sufficient dev/test environments

non-‐instant availability of a sub-‐system

unfinished subsystem not available data

not appropriate data

corrupted data

necessary test-‐data not in sub-‐systems

fees of invoking third-‐party system

limited usage permission of a sub-‐

system

Not allowed data corrup1on of a sub-‐

system

regulatory controls conflic1ng

delivery schedules of sub-‐systems

not appropriate delivery

coordina1on

Survey on Constraints in Testing


“Tes1ng facili1es in Aircra& industry simulate real-‐1me effects of wind, load and pressure, and repeat the standard movements over and over” MeridianBUF

Other industries: Simulation


Other industries: Simulation


For Service Oriented Systems the use of Virtual Services is the current technique to obtain simulated environments.

Service virtualiza-on emulates the behavior of so5ware components to remove dependency constraints on development and tes-ng teams (Wikipedia).

Virtual Services can be modeled or generated from traffic coming to and from real services.

How to simulate environment in SOA


Service Virtualization: How does it work?

Monitor & Capture

Initiated from the system under test, the user has the ability to capture detail from a live monitor that analyzes system traffic, from analyzing transaction logs or by modeling virtual behavior within the Parasoft Virtualized interface.

Model & Deploy

After the virtualized artifact has been captured, users can now instruct the details of the virtualized asset behavior. This includes: performance, data sources and conditional response criteria. The virtualized asset is then provisioned for simplified uniform access across teams and business partners.

Provision & Consume

The virtualized asset can now be called for unit, functional and performance tests. The virtualized asset can be leveraged by any test suite – including Parasoft Test.


Database

Service

Mainframe

Application

Application Under Test

2 Capture

4 Deploy

Server

Traffic Logs

3 Create

Virtualize SME

Desktop

Define Monitors 1

How does it work? Monitor and Model


Database

Service

Mainframe

Application

Application Under Test

Server

5 Provision Environment Manager

6 Consume

How does it work? Provision and Consume


Real back-‐end services

(constrained)

Service Under Test

Web Service 1

Web Service 2

Web Service 3

Virtualize Server

(full control)

Virtual Service 1

Virtual Service 2

Virtual Service 3

Virtual Service Manager

Case Studies in Service Virtualization


Case study 1: Telecom platform

Service Under Test

Back-‐end service

Back-‐end service

In-appropriate test-data



•  Constraints: In-‐appropriate test-‐data –  Backend systems are from 3rd party –  Test cases (“orders”) need to be added manually

•  Solu1on: –  Virtual service to simulate the 2 backend systems –  Simulate business processes (asynchronous transac1ons)

•  Results: Development includes regression tes1ng –  Prepara1on of testcase from a week to several minutes –  Increase test-‐coverage: from ca 10 to unlimited –  ( Stable “access” to backend in test-‐environment)


Automated tests

Service Under Test

Back-‐end service

Back-‐end service

Automated service

virtualizaFon management

Virtual Service

Virtual Service





Case study 2: SOA (e-banking)

Service Under Test

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

E S B

Limited Available sub

systems



•  Main constraint: Unavailable backend services –  Service might be down for load-‐test environment –  Services are in wrong version

•  Solu1on: –  Virtual services of 70 back-‐end systems –  Data in Virtual Services synchronized with Loadtest tool –  Full automated deployment & mapping of endpoints

•  Results: Load-‐tes1ng is now part of Agile-‐cycle –  Efficiency: >50% reducFon in performance test-‐1me –  Possibility to experiment with response 1mes (SLA) –  Performance measurements of ESB infrastructure


ESB Automated tests

Service Under Test

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Automated service

virtualizaFon management

Virtual Service

Virtual Service

Virtual Service

Virtual Service

Virtual Service

Virtual Service



Case study 3: Student administration



•  New interfaces by central government body –  No test environment available (DUO)

•  Created a collec1on of Virtual Assets –  Including asynchronous services to simulate address modifica1ons etc –  Contains over 4000 student records, including school history

•  Available for three different test-‐environments –  Via SSL available over the Internet –  Used by all Dutch universi1es and polytechnics.










Case study 4: Pension application

Service Under Test

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

Back-‐end service

E S B

Cost of Staged Environm

ents


Case study 4: Pension application

•  New implementa1on pension applica1on: –  Outsourced development –  6 releases a year –  Around 20 backend systems –  MQ middleware

•  Many staged environments: –  Development, Test, Acceptance, Pre-‐produc1on, Training, Produc1on (6) –  External connec1ons (GBA) without data-‐synchroniza1on –  Applied 80 / 20 rule (not full simula1on of all back-‐ends)

•  Used Service Virtualiza1on to: –  Simulate (remove) external sub-‐systems –  Automated provision data in the several databases –  Load-‐tests without impac1ng other parts of organiza1on


Case study 5: Validation environment

•  Energy market introduced New XML protocol for gas transport –  From EDI to XML based (EdiGAS) –  AS2 protocol (and new SHA2 cer1ficates)

•  Needed a “test tool” (cer1fica1on environment) to cer1fy over 120 market par1es, both na1onal and interna1onal –  Had to act like a chameleon

•  Use Virtualize (in combina1on with ESB and Database) to have fully automated cer1fica1on environment –  Dynamic responses based on role of the business partner –  Automated verifica1on of the test-‐scenario’s –  SSL Cer1fica1on and authen1ca1on


Case study 5: Validation environment

Hosted TestTool

Remote Party


Summary

•  Service Virtualiza1on solves constrains in tes1ng distributed systems –  Availability –  Scheduling –  Data

•  Benefits depends of the situa1on: –  Cost reduc1on in infrastructure for staged environments –  Speed up the overall tes1ng 1me –  Solve the test-‐data management problem –  Enabler of test-‐automa1on

Bus

Bus

Partner Partner

Bus

Bus

ERP


Summary

•  Enabler for Con1nuous Tes1ng and Delivery in distributed environments –  Key for Agile methodologies, DevOps –  Integra1on with Jenkins etc.

•  Allows Environment Based Tes1ng: –  Not only the front door, also the back door –  Project with University of Groningen and Linnaeus


Further reading and information

Email: Rix.Groenboom@Paraso&.nl Mobile: +31-‐62-‐240-‐1968

Documents

Case Studies in Solving Testing Constraints using Service ...sast.se/q-moten/2014/Q1_2014_Parasoft.pdf · Paraso&’Proprietary’and’Conﬁden1al’ 1 2/21/14 Case Studies in Solving