EXTENT-2016: Managing QA for Complex Systems in Agile Development Framework

Alexey Zverev CEOExactpro

Testing Complex Systems within Agile Development Framework

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 2

QA management at Exactpro

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49www.exactpro.com

Alexey Zverev, Co-CEO, Exactpro

http://www.exactpro.com/

http://exactpro.com/


Introduction



Agenda

1. Fundamental principles of QA Management

2. The anti-patterns

3. Summary of our principles vs. the anti-patterns

4. Questions & Answers



Fundamental principles of QA Management

QA is a continuous learning and development process

1. Learn how the system should operate in production

2. Learn about the actual behaviour

3. Make sure all discrepancies are either fixed or documented

4. Develop Test Design, Test Tools and an Automated Test Library through a series of

small iterations

This is a permanent process throughout the delivery cycle and

after the actual Go-Live, whilst building the test tools adds more efficiency,

and therefore, we find as many issues as possible.



The anti-patterns

1. “The Requirement Traceability Matrix”

2. “Test Automation”

3. “The Test Library”

4. “The Test Plan”

5. “Non-Functional Testing”

6. “The Test Tool”

7. “The Scrum Team”



“The Requirements Traceability Matrix”

“We need to write Test Cases at the

beginning of the project”

“All Test Cases must contain direct

links to Requirements”

“Stakeholders must see test coverage and correspondence between tests

cases and software requirements”



“Test Automation”

“We must automate 100% of our

Test Library.

Let’s start now”

“Complete regression must

happen overnight as part of the

build process”

“As a result of test automation we should reduce the QA team to a few

people”



“The Test Library”

“We must document all our test scenarios and

put them into Test Management Software”

“Test Library consists of Test Suites which

consist of Test Scenarios which consist of

Test Cases”

“Once we accomplish this we will start test automation”

“Exploratory Testing? What is it? No we don’t do it”

“All test cases contain detailed test steps and

Testers must put Pass/Fail as they execute them”



“The Test Plan”

“Let’s estimate QA effort required to deliver this project”

“Lets plan QA tasks and start implementing them in waterfall model”

“Let’s monitor process closely and put our effort into achieving targets and milestones”



“Non-Functional Testing”

“We need to test that the system

supports 10,000 orders per second”

“We need to make sure that latency is

within expected limits”

“We will allocate production-like

hardware to focus on capacity and

latency measurements”



“The Test Tool”

“We have purchased a licence

for a magical test automation

tool”

“We will use it for all of our

test automation purposes”

“It allows interacting with GUI / via FIX / via SWIFT and has the capacity to

record/replay test scenarios”

“It is expensive”



“The Scrum Team”

“We are using scrum and will test

everything as a part of the sprint”

“Testing is done and automated by

Developers who can temporarily

allocate their time within scrum”

“We will put some QA engineers into Scrum team and let them write unit tests”

“We are using DevOps we don’t need QA, QA is dead”



Summary of our principles vs. the anti-patterns

1. Developing team knowledge vs. “The Requirement Traceability Matrix”

2. Building software to test software vs. “Test Automation”

3. Test Design vs. “The Test Library”

4. Agile iterative approach vs. “The Test Plan”

5. Testing at the confluence of Functional and Non-Functional Testing vs. “Non-Functional

Testing”

6. Pragmatic in-house development of test tools vs. “The The Test Tool”

7. The QA team is a pacemaker vs. “The Scrum Team”



Questions & Answers

Thank you!


Technology

EXTENT-2016: Managing QA for Complex Systems in Agile Development Framework