Mark Walton CzechTest 2013 «Virtual Test Environment» 1 (43) Slide 1 (300) Whats the biggest obstacle in real-life software testing? Mark Walton CzechTest

Mark Walton ● CzechTest 2013 ● «Virtual Test Environment» ● 1 (43) Slide 1 (300)

What’s the biggest obstacle in real-life software testing?

Mark Walton ● CzechTest 2013 ● «Virtual Test Environment»


Test design?Test management?

Test automation?

Trac

eabi

lity?

Well?


… it’s TEST ENVIRONMENT!


The Solution?

Test environment virtualization

Mark, tester in:• Telecomm• Banking• Insurance• Contact: [email protected]


• Common test environment problems

• How does test environment virtualization work?

• Business benefits• Vendors


Typical Environment Problems:1. Huge cost2. Dispersed responsibility3. Maintenance4. Double or even triple environments5. Access difficulties6. Simulation7. Special: SOA, cloud and Web-services




1. Huge Cost (A)• Because you need special stuff:– Expensive server– Separate network– Telephone exchange– A Maserati in southern Mediterranean– Huge amount of data– A something that does not exist yet


1. Huge Cost (B)• Because you need special tools:– Air-conditioned test laboratory – Hardware tools to be able to see:• In-Circuit Emulator• Logic Analyser• Oscilloscope• Listening instruments

– Burglary-proof test laboratory




2. Dispersed Responsibility• Who takes back-ups?• Who installs and configures software

and data?• Who gives you access rights?• Who are your system administrator’s

prime customers?• Who has access priority?




3. Maintenance• Administration and

order-keeping• Between test passes• During project• After the project• And do not forget corrective

maintenance!




4. Many Environments• Developers’ own environment• Subsystem test environment• Integration environment• System test environment• Performance test environment• Pre-production environment• Final production environment




5. Access Difficulties• Many things are hidden in

real systems• Expensive hardware tools

may be required• Testing versus debugging

purposes• Remote access• Access booking - conflicts




6. Simulation Costs•Building and debugging simulators• Simulator maintenance and support

Sub-system A

Sub-system B simulator

Test A

Sub-system A

Sub-system B

Joint test

Sub-system B

Sub-system A simulator

Test B




What is a «cloud»?This is «cloud»!Application on server…

… used by (relatively) thin clients…

… through Internet


Application on server provides relatively small functions, called «services»…

… used by (relatively) thin clients…

… through Internet, using so-called SOAP messages

«Web services»?

SOA?

These are «Web

services»


Web-services use SOA,• … or, are an example of SOA (Service

Oriented Architecture)

SOAP

WSDLWSDL


7. SOA, Cloud and Web-services• Test environment:– For the development of a cloud application,

simulating many clients with different user profiles– For the development of client applications,

simulating one or many cloud applications (SOA services)






Half-way Summary 1• Traditional test environment:


Half-way Summary 2• Virtual test environment:


Virtual Test Environment• You could build it all by yourself, of

course,• … but it is better to acquire a pret-a-

porter framework, a tool• Not all functionality need to be

virtualized, but only the interesting part of its behaviour• Hence the name ABV (application

behaviour virtualization)


How Is ABV Created?1. Either through recording

(capturing) of the behaviour of the real system (installing the virtualizer as a proxy server)

2. Or from interface specifications (perhaps in WSDL)

3. Or from old transaction logs


ABV Configuration• Various behaviours for different

projects, versions or customers• Incorrect behaviour simulated to test

failure scenarios• ABV can be delivered locally or

globally• Or even through SOA for commercial

purposes






Mitigates the Problems 1(2)• Missing or unstable components

• Unavailable physical test environments;

• Inaccessible or costly third-party systems;

• Very expensive, complex or secret parts of systems, not to be used in testing (mainframe, financial, ERP);

• Many users of test environment resources (access and configuration hard)


Mitigates the Problems 2(2)• Using agile methods, when a large system is

developed by a number of separate agile teams, each in need of own test environment;

• A great number of systems, with which the SUT must be integrated;

• The need to access test environment from various locations in a distributed project


But, Keep This in Mind:There exists an army of IT people used to the current state of affairs, and even proud of their ability to cope with chaos, fear and trembling, sweat, blood and tears! Professor Martin Tornquist, founder and head of T&M Testes de Software (www.tmtestes.com.br), says:

It is easier to convince CFO than CIO to invest and improve testing and requirements

management, because CFO notices the cost of poor quality, while CIO parasitizes on it.


And Read This• professionaltester.

com/files/PT-issue18.pdf






Parasoft

• parasoft.com/jsp/products/virtualize_splash.jsp

Utilities


CA technologies • CA LISA®• www.ca.com/us/virtual.aspx• itko.com/solutions/index.jsp• www.ca.com/us/content/

Integration/itko.aspx


IBM - GreenHat• servicevirtualization.com/

group/greenhat• http://www-01.ibm.com/

software/rational/servicevirtualization/solutions/• testmagazine.co.uk/

2011/09/suiting-customer-needs/


Hewlett-Packard• http://www.hp.com/sbso/

serverstorage/article/virtualization-oct.html• http://h18004.www1.hp.com/

products/servers/vmware/index.html?psn=servers


Grid Tools• http://www.grid-tools.com/

products/intelligent_virtual_services.php


Learn More• en.wikipedia.org/wiki/

Service_virtualization#References• Note:

• Virtualization is not only for Web-services testing (but much of it is)• Test environment virtualization need not be

available in cloud (but often is)• Virtualization Tool ǂ Virtual Test Environment

until configured and programmed!


The biggest obstacle – no

longer!

Mark Walton ● CzechTest 2013 ● «Virtual Test Environment»

Documents

Mark Walton CzechTest 2013 «Virtual Test Environment» 1 (43) Slide 1 (300) Whats the biggest obstacle in real-life software testing? Mark Walton CzechTest