Agile Testing Introduction

Agenda

Agile Recap

Agile Testing Overview

Agile Testing Game

Agile QA Common Approach

Test Strategy in Agile Environment

Advanced Problems For Discussion

Summary Activity

IMT Confidential 2

Agile Recap

IMT Confidential 3

Agile Recap

IMT Confidential 4

Testing terminology – Unit testing

• Automatic test on code level run every night or after new code is added – to verify no bugs

introduced

– Integration testing

• Testing interactions between different modules of the system

– Feature testing • Performed by a QA when a feature is finished implemented

– System testing

• To verify whether the system as a whole meets the specified requirements

– ExploratoryTesting • Exploratory testing is simultaneous learning, test design, and test execution.

– Regression Testing

• Test to detect side effects from changes to the system, i.e after bug fix

– Retesting

• Verification of bug fix

– Compatibility testing • Verification of compatibility between AMX applications and the different OS, office applications, browsers

and databases etc that we support

5 thinking levels of Tester. -----------------------------------

You don't believe it until you see it. You don't believe everything that you see.


• Agile testing is a software testing practice that follows the principles of agile software development and involves all members of a cross-functional agile team.

• It is a set of activities intended to ensure that products satisfy customer requirements in a systematic, reliable fashion.

• In SCRUM (agile) QA is the responsibility of everyone in R&D, not only the Test team. QA is all the activities we do to ensure correct quality during development of new products.

IMT Confidential 6

https://en.wikipedia.org/wiki/Software_testing



https://en.wikipedia.org/wiki/Agile_software_development






• Agile testing focuses

on testing iteratively,

as often as stable

code base is

available, until quality

is achieved from end

customer’s

perspective.

IMT Confidential 7


11/19/2015

IMT Confidential 8


Agile Testing Traditional SQA

Agile Testers are integrated, full-fledged

members of dev team and participate in

planning, estimating and all team

activities.

Testing is performed by a separate

organization or group reporting to SQA

Manager.

Agile testers work hand-in-hand with

development and product manager.

SQA teams may work with other groups

but with limited interaction and well-

establishment boundaries.

Business requirements are written a

piece at a time to accommodate

changing business needs.

SQA receive detailed business

requirements and schedules up front.

Throughout the release additional test

scenarios are uncovered through

exploratory testing.

When development is presumed

complete, the application is delivered to

SQA for testing

IMT Confidential 9

Agile Testing Overview Agile Testing Traditional SQA

Agile testing is continuous and

feedback is provided during all stages

of development.

SQA executes a complete cycle of test

and reports defects back to

development.

Agile testing follows a fluid, continuous

process with defects fixed as they are

found.

When testing is complete,

development fixes defects and

delivers another release to SQA

Agile tester are expected to radiate

information and provide complete

visibility into all test-related activities for

the benefit of the larger group.

Traditional SQA teams often keep test

design, implementation and progress

specifics within the SQA site.

Quality software is always ready to

be delivered.

The process of test and fix repeats

until time runs out.

IMT Confidential 10


IMT Confidential 11

• Agile Testing Mindset


IMT Confidential 12

• Exploratory Testing in Agile Environment


IMT Confidential 13


Factor Scripted Exploratory

Domain Knowledge Lack of domain knowledge

can be overcome during test

design

Requiring sufficient domain

knowledge

System Complexity Allows for careful design of

test scripts

Relies on skill set of tester

Level of Doc Require good doc to be

supplied

Just assumes good domain

knowledge

Timeframes and Dealines Significant lead-in time

required

Little or no lead-in time

required

Efficiency

? ?

Coverage ? ?

Verification ? ?

Acceptable Risk Level ? ?

Reproducticibility ? ?


IMT Confidential 14



• Challenges in Agile Testing – Short Iterations & Releases

• 1-week, 2-week, 3-4 week release cycles, even shorter feedback cycles

• Validate new “user stories”, ensure old functionality continues to work, communicate bugs, fix

them and validate again.

– Less time for documentation

• Need tools/formats for documentation that cover essence of the test but are lightweight

– Dealing with ‘Slacker’ and ‘Asker’

– Is the number of defects reported a good metric to measure testing

effectiveness?

– Is automation required only when regression testing is needed?

– How to manage Time zone difference?

– How to cope and deal with changes of requirement

IMT Confidential 15

Agile QA Game

IMT Confidential 16

Agile Testing Game

• Building IMT Agile Testing Manifesto

IMT Confidential 17

Candidate Final

Duplicate Not Clear


IMT Confidential 18


Responsible for Test Preparation

Take part in requirement elaboration

Attend Sprint planning meeting

Collaborate well with Product Owner to clarify user

stories/PBI and the concept of tests

Prioritize all tasks

Estimate time for testing features/backlogs

Identify and mitigate risks

19


Responsible for Test Execution

Attend Daily Stand up Meeting

Design Test cases

Automate acceptance test cases wherever possible

Perform regression, exploratory & acceptance testing

Write/update & execute acceptance criteria

Identify the gap between what the customer expects

and what the team delivers

20


Responsible for Test Evaluation Be part of Sprint review & Retrospective meeting

Responsible for evaluating test deliverables

Provide continuous feedback to development team

21

Agile Quality Assurance Strategy

• Test strategy in agile development

IMT Confidential 22

Test Strategy in agile

• We start our QA activities at Release planning stage

• Release planning – Create test plans

• High level test planning

• budgeting enough time

– Participate in story sizing

– Typical Test plan will include • Scope of testing

• New functionalities which are being tested

• Types/ level of testing based on the complexity of the features being tested

• Performance and load testing

• Infrastructure consideration (Test environment/software)

• Risks/ mitigation plans

• Resourcing

• Milestones and deliverables

Test Strategy in agile…

• Each Sprint – Participate in sprint planning

– Estimate tasks(QA input)

– Write test cases using stories(QA)

• High level test cases before coding begins- guide dev

• Detailed test cases written during coding

– Unit/ Integration tests(Dev) • Automatic test on code level run every night or after new code is added. Bugs found are fixed immediately

– Pair tests(Dev + QA) • Performed when a part of a feature is finished implemented. Bugs found are fixed immediately

– Feature tests(QA) • Performed when a feature is finished implemented. Errors found are logged in main project in bug system

and prioritized during next sprint planning. After all prioritized bugs are fixed, the feature is ready for System

Test at the end of the project

– Reviews (stories, req, test cases with customer, programmer)

• Increase collaboration and communications


• Each Sprint contd…

– Test automation

• Automate new functional test cases

• Run automated regression tests

– Perform non functional testing (load, security, usability etc)

– Demo to the stakeholders


• Hardening sprint

– A hardening sprint can be used for bug fixes in previous sprints. Bugs that are prioritized will be considered here

– The end game(System test)

Every member in the team is involved in this.

• Mock deploy on staging

• Smoke test on staging

• Perform non functional testing (load, security, usability etc)

• Complete regression test

• Perform UAT tests

• Other tests

– Compatibility(DB upgrade etc)

– Installation

– Localization etc


• Release – Define Exit criteria

• Testing activities (bug severity, open bugs, coverage levels, metrics

etc)

• Product Artifacts (user guide, installation guides, other necessary

documentation are in place)

– Participate in retrospectives

• Start

• Stop

• Continue

Iteration metrics

To enhance predictability…

– Measuring progress

• Burn down charts

• Estimated vs. Actual time for tasks

– Defect metrics

• Root cause analysis

• Results from defect tracking system

– Traceability metrics

• Test coverage (story vs. test cases)

• Code coverage

Advanced Problems

• Agile Testing Interview Questions

– What is Agile Testing and how is it different to traditional

waterfall or the V model?

– What is your approach when requirements change

continuously?

– What are good characteristics of an Agile Tester / QA?

– What are the two key factors when working as a QA in an

Agile team?

– What are the three main roles in Scrum?

IMT Confidential 29


IMT Confidential 30


• Impact Analysis

IMT Confidential 31


• IMT Projects lesson learnt

– Xfinity Home Security

– Precyse

– Precyse Automation

– TabSquare

– …………………………………

IMT Confidential 32

Summary Activity

• Using mind map to draw the map like below

to summary your knowlegde of Agile Testing

IMT Confidential 33

Summary Activity • An example of mind map about Agile Testing Day

IMT Confidential 34

Conclusion By Image

11/19/2015

IMT Confidential 35

Main References

IMT Confidential 36

http://www.istqb.org/

An Le, Tester Mindset

in Agile Scrum Model

http://jp.imt-soft.com/Blogs/Agile-Scrum/Tester-Mindset-in-

Agile-Scrum-Model

http://jp.imt-soft.com/Blogs/Agile-Scrum/Tester-Mindset-in-Agile-Scrum-Model















Main References

IMT Confidential 37

Nguyen Vu Hung, Tran Son Hai and et al, Tài

liệu Hướng dẫn Scrum dịch từ bản gốc được

cung cấp bởi Ken Schwaber và Jeff Sutherland,

Main References

IMT Confidential 38

Thank You

IMT Confidential 39

Questions

IMT Confidential 40

• Lazada Story Question:

• Recap Question: List the

pros and cons of Agile

Testing in Outsource

Project?

Presentations & Public Speaking

Agile Testing Introduction