WEBINAR

Small is Beautiful – How to Fully Automate your Testing and Test Case Design

Huw Price, Managing Director – Grid Tools

• Automatically generate the smallest number of test cases from requirements for

maximum coverage

• Push test cases out as automated tests

• Remove bottlenecks caused by the manual creation, execution and maintenance

of test cases

This Agile Designer™ webinar will demonstrate how you

can:

Huw Price

Managing Director and co-founder of Grid-Tools

What’s Supposed to Happen

TesterBusiness AnalystUser Developer

The end user knows what they want

The analyst specifies what that is.

The programmer writes the code

The tester tests the program

What’s going wrong?

• Requirements are stored disparately & In different formats

• They are typically incomplete and ambiguous

Requirements Code Test Cases Test Data Environments

The developer requires clarification & defects

enter the code

• Test cases are manually created and executed

• Provide just 10-20% coverage and defects are detected late

• Cannot respond to change requests• Never get deleted

• Contains sensitive information, and time is spent securing it

• Provides poor coverage, and testers spend half their time finding or making data

• Is large, and 20% of the SDLC is spent finding data

• Is unavailable ‘upstream’

Change Request

• Are shared with other teamsor are not complete yet

• Are legacy or expensive mainframe systems

Hardware & Data Dependencies • Automation is used late

• Scripts are manually developed

• It cannot respond to change• Tests the same functionality

Automation

Grid Tools - Products

• Automation Scripts

• User Stories

• Optimized Test Cases

• Backlog and Complexity

• Test Data

• Requirements and Coverage

• Data Coverage

• Find and Allocate Test Data

• Subset and Mask Data

• Synthetic Data

• Virtualized Messages

• Test Data on Demand

• Selenium• QTP• IBM Rational• Ranorex• SilkTest• Microsoft UI Automation• CA• Parasoft

Javelin Automation

Dataviz

Javelin Data Orchestration

A Simple Form Easy to test?

Test case design

• Currently manual

• No formal processes.

• “Exploratory Testing” the norm: poor requirements lead to poor overall testing, with testers having to fill in the gaps

• No linkage to test data – process is manual, painstaking and very time-consuming.

• No flexibility for change requests: a critical weakness in an agile environment. Changes take longer than the original requirement!

What is Testing Coverage?

The Business Thinks it’s:

• Code Covered

• Number of Test Covered – Tests Run

• Percentage of use cases

• All Paired Combinations

It is actually:

• Designing Sufficient Tests To VERIFY That The Design And Code Correctly Implement The Requirements

• Did you get the right answer for the right reason - Two or more defects may sometimes cancel each other out - Observability

Different Coverage Techniques

Combinatorial – All Pairs – Constrained All Pairs

• Does not support Expected Results - You have to work it out for each combination which is very time consuming

• Combinatorial is something that ‘accidentally’ increases functional coverage (and only to a point)

• Combinatorial does not give you any solid metrics on how good your testing actually is – you can only infer that x out of y combinations have been covered, but it has no bearing in actual functional logic.

• You end up with lots of false errors that have to be checked by hand

Copyright Bender RBT Inc. 2009 11RBT24261

Cause-Effect GraphingObservable Events and Path Sensitizing

• Assume C and F are not observable events.

• Assume A is stuck at FALSE.

• Enter as a test case A(T), B(T), D(T), E(T).

• Results should be C(T), F(T) and G(T).

T

T

TT

T

T

T

Different Coverage Techniques

Different Coverage Techniques

Optimized Flow Chart Modelling

• Advanced Graph Optimization

• Most projects already have a flow chart

• Similar results to Cause and effect

• Supports Looping

• Supports Constraints

• Not applicable for non sequential based logic

Optimizer Demo

• Different Optimization Methods

• Managing Change

The path explorer tool and optimization algorithms

Create Perfect Test Cases

• Generate the smallest number of test cases with maximum coverage

• Test more functionality in fewer tests

• Measure test coverage and know that every requirement has been tested

The cost and complexity tool in Agile Designer™

Model Based Testing – Step by Step

Model Based Testing – Step by Step

CUSTOMER NUMBER: 123456

CUSTOMER NAME: HUW PRICE

CUSTOMER COUNTRY: UK

VALIDATION

PROCESS

VALIDATION

PROCESSDATA UPDATED

Validation Coverage - Add Related Data Logic

ALL POSSIBLE ALL PAIRS ALL EDGES

Model Based Testing – Process CoverageALL POSSIBLE

ALL PAIRS

ALL EDGES & PATH HINTS

Model Based Testing

Most testing is – Frankly:

• Random

• Unstructured

• Repetitive

• Not thorough enough

• Can’t be measured

• Can’t keep up

• Too slow

Model based testing, lets you define what is supposed to happen and then test that.

Model based testing is:

• Accurate

• Structured

• Thorough

• Measurable

• Can keep up with change

• Allows you to measure Risk

• Lets you automate very quickly

Automated tests: manual generation

• Automated testing frameworks are heavily scripted

• Script generation is usually done manually

• Alternative solution: use scriptless automation frameworks – But your back to Manual test case design

• Optimal solution: from automated test case generation, through automated script generation, to automated execution. Automating the automation.

Active Automation

• If you could create better manual tests faster, would you?

• If you could create automation scripts as quickly as manual test cases, would you?

Active Automation

• Scripts Are Created Automatically

• Testing is Model Based

• Use a component library of covered tests

• Test Data is created or found as part of the Automation

Active Automation

Active Automation

Record an Automation Script

Import it into Agile Designer

Design the logic

Auto create your automation robots

If something changes, auto create new robots!

Cl Cl

Cl

ClN

O

H

N

N

N

H

O

Cl

Cl

Cl Cl

Model Based Testing – Use Libraries of Test Cases

Agile Designer – Data driven or Flow Driven

LOGIC

DATA DRIVEN

AUTOMATION

AUTOMATION HARNESS

DATA 1

DATA 2

DATA 3

DATA 4

FLOW DRIVEN

AUTOMATION 1 – DATA 1

AUTOMATION 2 – DATA 2

AUTOMATION 3 – DATA 3

AUTOMATION 4 – DATA 4

AUTOMATION 5 – DATA 5

You can drive using a static flow and feed it with test data and expected results

You can drive by auto generating brand new test scripts for each path

Or a combination

Automation Demo

• Capture Automation Assets

• Define the logic

• Create the automation robots

Shift left the effort of testing

Business Analyst

User

Requirements

Programmer

Tester

Tester

Test Cases

& Test Scripts

Program

• Concentrate effort into the requirements gathering stage

• User and BA map requirements to unambiguous, active flowcharts

• Testers can generate automated test scripts and test cases directly from the requirements

• If a user makes a change to the requirements, the tests can be automatically updated

Active Automation

• If you could create better manual tests faster, would you?

• If you could create automation scripts as quickly as manual test cases, would you?

•Yes please!

Q & A

If you have any further questions please contact us on

agile-sales@agile-designer.com

For more information on what Agile Designer™ can do for

you, please contact:

www.agile-designer.com

agile-sales@agile-designer.com

or by phone:

UK: +44 (0)1865 884 600 or US: +1 888 603 1213

THANK YOU FOR WATCHING