A Ranger4 Guide to Integration Testing and Service Virtualization

A Ranger4 Guide to

Integration Testing &Service Virtualization

www.ranger4.com © Ranger4 2014 1

http://www.google.com/url?q=http%3A%2F%2Fwww.ranger4.com&sa=D&sntz=1&usg=AFQjCNFVDEXbd4GQxVLiqSaw6TEFPEyAtQ

Contents

1.0 What is Integration Testing and Service Virtualization?1.1 Integration Testing, Service Virtualization and DevOps

2.0 Why should you do it?3.0 What you should look for in a vendor

3.1 Creating virtual components3.2 Key Capabilities3.3 A Shared Infrastructure3.4 Routing3.5 Questions to ask during your evaluation

4.0 Considerations at implementation time4.1 Measuring success4.2 Don’t let developers write the tests on their own code4.3 Manage requirements: establish an incremental integration plan

4.3.2 Identify services to virtualize4.4 Techniques to consider

4.4.1 Employ test virtualization4.4.2 Use continuous system-level testing and asset sharing4.4.3 Plan for effective data management4.4.4 Reduce E2E testing and isolate the GUI4.4.5 Test earlier (shift left)4.4.6 Avoid the Big Bang

5.0 Ranger4 Recommendations



1.0 What is Integration Testing and Service Virtualization?

Integration testing is a phase in software testing when individual software modules areaggregated and tested as a group. This phase occurs after unit testing and before validationtesting. The inputs to the integration testing phase are the software modules that havebeen unit tested - they are combined into larger groups tests are applied to the groups asper their definition in an integration test plan. The output is the integrated system readyfor system testing.

Service virtualization is a method to emulate the behavior of specific components inheterogeneous component-based applications and is used to provide software developmentand QA/testing teams access to dependent system components that are needed to exercisean application under test (AUT), but are unavailable or difficult-to-access for developmentand testing purposes. Virtualizing the behaviour of the dependent components means thattesting can proceed without needing to access the actual live components.

1.1 Integration Testing, Service Virtualization and DevOps

The DevOps movement has been born out of the increasing conflict between IT developmentand operations teams as a result of businesses’ need to deliver more innovation to theircustomers via applications and infrastructures of ever-increasing complexity. Wheredevelopment are all about change, operations are driven to control the stability of theenvironments they manage and deliver the highest possible uptime to the business - andchange puts all of that at risk.

With the advent of agile development, and its definition of done, done, done, another newcomplaint has arisen: developers feel like testers are intruding on their "need for speed” andtesters are finding defects in code after the developers have moved on.

Integration testing and service virtualization helps development, testing and operations workbetter together.


http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FUnit_testing&sa=D&sntz=1&usg=AFQjCNE_I-iWC9XV8CCI98vko96lu87acw

http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FUnit_testing&sa=D&sntz=1&usg=AFQjCNE_I-iWC9XV8CCI98vko96lu87acw


2.0 Why should you do it?

Performing integration testing and service virtualization has a number of high level benefits:

- Improved software quality- Increased consistency, efficiency, reliability and predictability in the software developmentlifecycle- Fewer defects- Reduction in time spent testing- Elimination of bottlenecks- Doing more with less- Increased project delivery capability / reduction of project risk- Reduction in transactions lost due to production outages- Avoidance of additional hardware infrastructure expenses- Less time/cost associated with troubleshooting- Improved collaboration between development, testing and operations teams- Business-value based innovation to market faster- Happier customers- Reduction in spend on third party consulting/testing fees

Service virtualization delivers benefits for all types of testing including functional (manualand automated), integration, and performance testing.

Service virtualization emulates the behavior of software components to remove dependencyconstraints on development and testing teams. These constraints occur in complex,interdependent environments when a component connected to the application under test is:

● Not complete● Still evolving● Controlled by a third-party or partner● Available for testing only in limited capacity or at inconvenient times● Difficult to provision or configure in a test environment● Needed for simultaneous access by different teams with varied test data setup and

other requirements● Restricted or costly to use for load and performance testing



3.0 What you should look for in a vendor

Service virtualization solutions have the following characteristics:

- Application emulation: Virtual components can simulate the behaviour of an entireapplication or a specific component

- Multiple test environments: Developers and testers can create test environments by usingvirtual components configured for their needs

- Same testing tools: Developers and testers can use the same testing tools that they havetraditionally used — the tools don’t know the difference between a real system and a virtualservice

The virtual components are created to simulate a real environment through two basic entrypoints:

1) Observing the system in action: Build virtual components by listening to the networktraffic of the service that you want to emulate

2) Reading the descriptions of the system: Build virtual components using other sources ofinformation such as service specifications - one example is a Web Services DescriptionLanguage (WSDL) file, which describes the operations offered by a service along with theparameters it expects and the data it returns

When selecting your service virtualization solution it’s essential to recognize the needs ofdifferent roles within your business. Not everyone needs or wants to define virtualcomponents, but many testers need to access virtual components in their testenvironments.

3.1 Creating virtual components

The best service virtualization solutions make it easy to construct virtual components that:

- Mimic the behavior of the real component providing the service- Respond with realistic data- Process requests within configurable throughput ranges- Can be switched on and off, as the real service becomes available, without having toreconfigure the deployed application



3.2 Key Capabilities

When you’re looking for a service virtualisation solution for your business, here are somekey capabilities you should be looking for:

1. A tool that can create and maintain virtual components2. A hosting environment for virtual components3. Testers can configure their environment with virtual components with ease4. Data extracted from production environments is presented in an easy-to-use way5. Data is captured during recording in a way that allows for easy creation of a

virtualized service6. Data can be privatized or obfuscated as needed7. Multiple types of boundary conditions can be tested8. Data from virtual components can be externalized to allow for easy updating - it will

be easy for the author of a new test scenario to add relevant data to a spreadsheet9. A method for observing and recording messaging conversations10. A virtual service solution that’s flexible enough to allow you to switch back and forth

between the real component and virtual components when testing11. Virtual services can be developed in a personal environment on a desktop12. A shared environment for service virtualization can be used across a development

team

3.3 A Shared Infrastructure

Developers and testers need a shared infrastructure for hosting virtual components andmultiple environments need to be maintained in parallel with different mixes of real andvirtual components. An environment binds a set of variables from the logical view of yoursystem to specific virtual and physical resources (identified by URLs, host addresses, ports,or other connection settings). By creating multiple environments (for example, developerprivate, SIT, and UAT), you can run tests against different configurations during each phaseof the product life cycle.

3.4 Routing

It’s essential to be able to control which traffic is routed to a real service and which isrouted to virtual services across environments. After the environment is established, you



should be able to run all your tests without knowing the difference between real and virtualservices. Your chosen service virtualization tool should allow this change in routing withoutneeding any changes to the application code / configuration. You should be able to adjustvirtual components throughout.

3.5 Questions to ask during your evaluation

1. How easy to use is the solution?2. How much and what level of training is required?3. Can you create virtual components from recordings and / or design specifications?4. What test types does the solution support? e.g. Manual, automated (integration and

functional), and performance test5. Can you create virtual components that allow you to test different scenarios? e.g

happy path, alternative flow, and negative testing6. How do you create virtual components that enable you to test what-if conditions?7. Does the solution offer the ability to quickly deploy and manage virtualized services

through an administrative console?8. Can you toggle between live systems and virtual services without having to

reconfigure your application deployment?9. Can the solution manage the complexity of your application environment (ranging

from data-driven and correlated response sequences to full stateful databaseemulation)?

10. Can you automatically schedule and execute tests supported by virtual componentsupon the availability of a new application build?

11. Can you create, modify, and deploy virtual components without requiring your teamsto learn new programming skills?

12. How easy will it be to share and reuse virtual components across teams?13. Can your teams develop in parallel environments?14. Will your solution scale to accommodate very large teams as you grow?15. Does the solution require a long term professional services engagement to help you

get started and continue to move forward?

Technology constantly changes, so check the software vendor you select is committed todelivering new functionality regularly to ensure the solution remains current - ask to see theproduct roadmap and question them about their release cycles and software maintenanceprogram. To help speed your adoption process and keep your productivity levels up, makesure your vendor provides excellent support with access to trained technicians - ask for areference call and check with another customer that this is the case.



4.0 Considerations at implementation time

The evolution of applications is accelerating. Applications are not discrete islands but buildon complex, interconnected sets of services including disparate technologies, developers,deployment topologies and organizations. Developers are directed to deliver high-qualityapplications while testing expenses are often limited. A combination of automatedintegration testing and test virtualization can help test teams to improve quality and keepup with the rate of change.

4.1 Measuring success

A simple measure of success for a test manager is the ratio of captured defects versusescaped defects. However, success or failure is not simply determined by the number ofdefects that have escaped into production. Categorization of defects to determine wherethe defect should have been found can dramatically reveal the efficiency or inefficiency ofyour testing. For example, if a functional defect is found during end-to-end system testing,the costs of remediation would far exceed the costs of fixing the defect as it wasintroduced in an earlier development phase. The increased costs would be due to factorssuch as: more regression testing, more test resources, usage of more live-like environmentsand greater requirement for coordination.

4.2 Don’t let developers write the tests on their own code

Having developers write their own testing simulations or stubs to validate their own codechanges or new code is a bad idea - analogous to having a student mark his own homeworkor exam. Testers, who know how the functionality should be tested and the data requiredto properly test the various scenarios, can easily create virtual components and tests,making them available to the development team.

4.3 Manage requirements: establish an incremental integration plan

Accurate and unambiguous requirements are particularly vital for integration testing.Business processes must be defined and broken down into their constituent parts so thatcorrespondingtesting requirements can be defined at the most granular level of service interaction.Requirements coverage is thus tracked at:

- A functional level- A service interaction level



- The business process level- At full system level

When granular requirements are fully visible, project leads and test managers can moreeasily collaborate to plan an effective and controlled release schedule. Gradually controllingthe introduction of functions and components into test environments makes it faster andeasier to isolate faults and defects.

4.3.2 Identify services to virtualize

You need to identify the right services to virtualize - start by bringing together the keydevelopment stakeholders involved in the application life cycle and find out where yourorganization experiences the most testing pain. Then ask these questions:

1. Do you have all the environments you need for integration testing?2. Are these test environments available to teams throughout the development cycle?3. Do you experience downtime due to unavailable test environments?4. How often does downtime occur? How long do your teams usually have to wait?5. What’s the impact on time and cost due to testing downtime?6. Does your application interface with third-party services?7. Do you need to pay for and schedule access to these third-party interfaces prior to

scheduling your tests? How much does this cost?8. Who controls the information needed for creating test environments?9. Do individuals or teams conflict with each other when scheduling the sharing of test

environments (or parts of a test environment)?

The answers to these questions should enable you to identify a prioritised area where youcan start your project.

4.4 Techniques to consider

After establishing an incremental integration plan for the project, test professionals shouldconsider the following techniques to help achieve a more proactive test procedure:

4.4.1 Employ test virtualization

In test virtualization, reals component are replaced by virtual components (“stubs”). Thesevirtual components can be used to model and simulate real system behaviour. Virtualizingthe environment helps to eliminate application test dependencies and reduces the setup andinfrastructure costs of traditional testing environments. A proactive test plan should use test



virtualization to make virtual components available for key service components, allowingsituations to be simulated and tested more easily.

4.4.2 Use continuous system-level testing and asset sharing

Running full regression cycles whenever new or virtualized components are introducedprovides an immediate feedback loop to the development team who can run the exact samescripts, replicate and resolve issues with minimal effort. This promotes innovation ratherthan remediation. Testing tools encourage developers and test teams to collaborate bysharing integration tests and virtualized services throughout the software developmentlifecycle (DevOps).

4.4.3 Plan for effective data management

Representative, appropriate data is needed in order to support the necessary test coverageand achieve the appropriate level of confidence in a delivered solution. It’s recommendedthat data considerations begin at the requirements stage and are factored into testcreation and execution. Given limited timescales and budgets it is usually necessary to useequivalence partitioning or boundary analysis to identify the data that is absolutely essentialto the project. Test data management is such an important activity that it is often assignedto dedicated individuals.

4.4.4 Reduce E2E testing and isolate the GUI

Since integration testing is incremental, E2E testing takes on less significance. When aproactive test approach is implemented, far less time is spent performing costly end-to-endtesting - by the time full end-to-end functionality has been created in the test environment,the functional, integration and business-process-level tests will have been executed manytimes. Incremental and continuous testing will have removed many of the risks of theintegrated solution.

It’s recommended that end-to-end testing is focused on driving through the E2E processesvia the various GUI front ends. Implementing automated testing that occurs at the servicelayer and bypasses the GUI makes it faster to create and execute tests and delivers a morerobust process than using GUI-driven automated scripts.

It’s recommended that GUI components are isolated in environments where they interfacewith virtualized services - this ensures a high rate of test execution. The GUI componentscan then be periodically introduced, validated and withdrawn through the project life cycle(they should only be formally introduced after completion of all other integration testing).Combining automation at multiple isolated layers of an application and a more predictable



execution environment for GUI tests helps mitigate the traditional challenges of a GUI-onlyapproach to testing.

4.4.5 Test earlier (shift left)

The above considerations are all undertaken in order to test more effectively at an earlierpoint in the software development lifecycle. It is widely accepted that testing and defectresolution are more expensive if undertaken at the latter stages of integration.

4.4.6 Avoid the Big Bang

A traditional “Big Bang” test method brings all integration points together for end-to-endtesting. With this test method, there is a sudden threshold at which it becomes possible torun many more test cases. The increase in the number of cases causes a sudden drop in thepercentage that are tested and passed.

With Big Bang testing, a huge proportion of the project’s risk has been deferred until late inthe development cycle when all of the components are available. This profile needs to bereversed by addressing integration risk earlier and more continuously (shifting left).



5.0 Ranger4 Recommendations

Ranger4 recommend’s IBM’s solutions for integration testing and service virtualization(formerly known as GreenHat):

● Rational Test Workbench: Delivers a comprehensive test automation solutionincluding regression, load, integration, and performance testing across web, mobile,and traditional applications. By providing an end-to-end solution in a single offering,Rational Test Workbench allows teams to significantly improve team agility,maximize productivity, and deliver software at the pace the business demands.

● Rational Test Virtualization Server: Enables the deployment of virtual services inorder to simulate real system behavior so teams can perform integration testingearlier and more frequently in the development cycle. Virtual services can be reusedand shared easily across teams to significantly reduce the expense of creating andmaintaining test environments.

● Rational Performance Test Server: Provides application and integration level loadgeneration to provide a complete view of performance and scalability across allapplication components. This workload test engine maximizes the use of your testinfrastructure to quickly deploy load scenarios and create large-scale system tests toavoid the costly impact of poor application performance.

Gartner rated IBM as a Leader in their 2013 Magic Quadrant for Integrated Quality Suites,identifying them as having one of the broadest portfolios in the testing tool market withcoverage not just for core test management and test automation, but also static analysis(both code quality and security), unit testing, and a comprehensive solution for test data.

Ranger4 are ideally placed to help you evaluate IBM’s integration test and servicevirtualization suite and well-versed in helping our customers write business cases to justifytheir software investments.

We often find that testing is one solution area of several that will help an organisationbenefit from a full-blown DevOps implementation and have developed our DevOps MaturityAssessment which baselines and scores an organization’s current capabilities on our DevOpsMaturity Index and provides a fit-assessed roadmap to a desired future state.

To find out more, please visit www.ranger4.com.

