GDP Rich UI-Testing Practice

8/8/2019 GDP Rich UI-Testing Practice

1/21


2/21

GoalsGoals

Time to market

Efficiency - no redundancy on test caseexecution


3/21

Challenge: How do we measure?Challenge: How do we measure?

How do we normally test? We use test cases

We use unit tests

What do we measure code covered by unit

tests What we cant measure what is tested by

test cases Could we measure how much code is actually

tested?

Could we measure how much dead code we have?


4/21

MeasurementMeasurement

What is measurement? Is it number of test cases?

Is it how many days we spent for ad-hoc testing?

Is it number of days we spent to create/review test

scenarios?

Each test case covers some limited numberof source code lines

We could measure how much source code lines(SCL) were used by particular test case

Using numbers of SCL by each test case we couldmeasure how much untested code we have,

We could identify unused code by not-covered by

test cases


5/21

ExampleExample

Project with 210 source code lines (Java/JavaScript/Ajax/etc)

Test plan has 5 test cases

Regular scenario (guessingstyle)

Potentially we havecoverage for 210 lines(no instrument tocheck)

Hopefully all source codelines covered by testcases

Potentially we have

somewhere 10 lines ofnot tested code?

Scenario with Source CodeCoverage info

We have complete set ofsource code linestested by each testcase

We have complete set ofsource code lines thatare not testes


6/21

How to identify test casesHow to identify test casesneeded to check all changes inneeded to check all changes inthe codethe code

Not tested code

TestCa

sesto

Execute


7/21

Test casesTest cases

Standard test case

How many steps do you have inyour test cases? 30-200?

Each step depends on previoussteps

How many times you test same

code using same conditions?

If 25th step fails how youll testit again?

Proposed test case

A few steps

No dependencies data requiredto test functionality is createdin database right before testcase initialization

Test case tests only a piece of thevery specific requirement

Can be used for both manual andautomation

Easy to support/understand

We call it Atomic test case

Atom ic te st case is:Scenario for very specific feature

Scenario that is not related to any other test case

Original test case is number of many atom ic test cases

Each test case has its own coverage by source code lines


8/21

Atomic Functional Test CasesAtomic Functional Test Cases

Atomic test cases that aimed to verifyparticular functionality

Benefits

Coverage: measurable Maintainability: extremely high maintainability.

Case covers very small (atomic) piece ofrequirement

Execution: easy to execute. Every case could be

executed separately even in different days bydifferent test engineers

Dependency: test cases are independent. Everycase prepares preconditions (application, data) forits run

Automation: extremely easy to automate


9/21

Another example:Another example:Conventional Test CaseConventional Test Case


10/21

Document for atomic test caseDocument for atomic test case

T bilit t iT bilit t i


11/21

Traceability matrixTraceability matrix(functionality+testcase+source(functionality+testcase+sourcecode)code)

Traceab ility l inks toget herIssue for new functionality with specification document

Specification document with functional test case

Functional test case wit h automated test script

Coverage for source code lines by test case

Black box t est coverage by manual/automa te d te sts


12/21

BenefitsBenefits

We wont test few times same code We reduce number of cases to test

We wont release untested code We identify what wasnt tested


13/21


14/21

Test case pre-requisitesTest case pre-requisites

To start with a test case we have to create data to test

Regular way

Create all data manuallyusing standard flowsusing application UI

Depends on test casecomplexity, may takeabout 30 minutes

Proposed way:

If we have one atomic testcase for data creation wedont need to test it again

We can pre-populate data indatabase using some tool

XML based databasefixtures

Data that shouldbe already

present indatabase topass test case

XML based GUI fixtures

Data that shouldbe entered into

form to passtest case


15/21

Atomic test case executionAtomic test case execution

As plug-in to Maven

Atomic test cases for per-commit or nightly builds

Pre-requisites creation takes milliseconds

Test cases can be run against any browser withoutchanges

Performance

Individual atomic test case takes about 90 sec.

About 480 test cases per night


16/21

How does it work?How does it work?Coverage toolCoverage tool

Automated test cases (using WebUIinteraction API)

or QA manually interact w ith application withinCoverage tool through browser

Coverage tool tracks the source code lines affectedby those activities

overagetool

Browser

- -Auto test cases

Manual TCexecution App DB


17/21

Covera ge t ool collect ingCoverage t ool collect ing

coveragecoverage

Step Action

1.1:1.2 User/WebDriver activities

2.1:2.2 Communication with server

3.1 Cobertura tracks the code execution

4. 1:4.2 Dev Plugin controls the code execution onCode Server

5.1 Emma tracks the code execution

6.1 The stop-tracking command request

7.1:7.2 Get client side coverage

8.1:8.2 Get Emma runtime coverage

9.1:9.2 Get Cobertura runtime coverage

10.1:10.2 Persist combined coverage for an activity toDB


18/21

BenefitsBenefits

We can measure the code covered by test casesbeing ran either manually or by automated tool

We can determine the code not covered by testcases

We can reduce the testing time by executing theonly test cases against affected source code


19/21

33rdrd party projects testingparty projects testing

Testing of 3rd party project? No documentation?

No understanding how to test?

Quality is good enough?

Start with coverage, start with test cases

Having coverage for any test case, you can extend youtest coverage using tool for measuring

You can add more and more test cases for another areas

Future steps You can MEASURE how many test cases affected (comparing to

coverage for previous version) You can UNDERSTAND how much code lines are not covered by

your test cases

You can EXPLAIN how much dead code is in the project

You can ASKdevelopers to explain how to test their code


20/21

And again all togetherAnd again all together

Atomic test cases

Test case code coverage Minimizing of number of test cases required to

test all changes

Automated or Manual

Measurable!


21/21

Q&AQ&A

Documents

GDP Rich UI-Testing Practice