Gop-stop automated testing

Gop-stop automated testingTsimafei Avilin

Meaning…

Gop-stop (slang) – robbery; an attack aimed at theft of

property of a victim committed with the use of violence,

or with the threat of violence…

Translation: гоп-стоп - грабёж; нападение с целью хищения имущества потерпевшего,

совершённое с применением насилия, либо с угрозой применения насилия

What's stealing?

• Time

• Money

• Mental energy (so called - demotivation)

• People

• Projects

• Your cases …. ?

SPRING COMAQA CONFERENCE 2017

Why does it happen…sometimes?

- Inexperienced managers

- Inexperienced workers

- Few controlled circumstances


How can identify that?

Usual phrases sound like:

- “We have to see 100% parity and coverage!”

- “We have to automate 100% test cases!”

- “You promised to complete this test set last week!”

- “Why our tests are still failing?”

- “You promised to complete that at 8 P.M.!”

- “Why this test case is still failing???”

- “We are doing code cleaning…”


Stop this demagogy,

give me the real example!!!


Framework issues…


Test automation framework side:

- too complicated framework with many spikes -> hard

maintainability,

- automated test case is longer then two listings;

- no stable test cases set -> CI builds time to time fails;

- extensive programming -> do as many code lines as possible

w/out encapsulation;

- test data is given by different "providers": hard-coded, as files in

a project, from api provider (e.g.TFS), etc.

Framework issues…


Real-life example:

- developed framework with complicated architecture;

- about 15 bad-designed test cases (each test case is about 150-

200 lines of code) and all these tests got failed both in CI and

even locally have been developed;

- test data is given by different "providers": hard-coded, as files in

a project, from api provider (e.g.TFS)

CI results for such framework…


0

50

100

150

200

250

300

Nu

mb

er

of te

st ca

se

s

Date

Failed test cases Passed test cases

WTF???

Process issues…


Testing and development process side:

- it takes much time newcomer to start working on a project;

- much time and effort is going to fix some unique situations

instead of working on main functionality/important bugs;

- switching between the projects each few days;

- complicated relationships between the customer and the vendor

-- solve problem - not people!

- the customer's PMs cannot make a final decision and put it on

the team shoulders;

Process issues…


Potential solutions…


Common recommendations:

-- dive deeply into designed architecture;

- focus on specific pieces of an application instead of long E2E

testing;

- try to not automate 100% if the tests set will be unstable;

Audited framework recommendations:

- define strategy of preparing test data (in the framework all

business entities are created in the test case directly, which

takes sometimes about 50 lines of code or more…);

- stop developing new automated test cases and encapsulate

members to improve and simplify test cases design;

- improve quality of functional test cases;


Software

Gop-stop automated testing