Software Testing Life Cycle

Limited Business Circulation

Page:1Version 1.1/Testing Essentials/001

QSIT Copyright

Software Testing Essentials 1 www.qsitglobal.com

Testing Essentials Unit 4

QSIT- Quality Solutions for Information Technology Pvt. Ltd.,

A 706 & 707, Mittal Tower,BANGALORE- 560 001,INDIAURL: www.qsitglobal.comPH: +91 80 41134334Fax: +91 80 25597445


Testing Life Cycle Overview

• Test Strategy

• Test Planning

• Test Case Design

• Test Execution

• Testing Types

• Testing Tools Overview



QSIT Copyright


Normally, testing of any Large Systems will be in TWO parts.

The functional verification and validation towards the Requirement Spec and Performance evaluation against the indicated requirements

Testing activity is involved right from the beginning of the project

V&V Process model to successfully deliver a project on timeThis model is followed for a complete project life cycle model

Testing Life Cycle Overview

Software Testing Essentials 4www.qsitglobal.com

SDLC Vs STLC

UAT Planning

System test planning

Integration test

planning

Unit test planning

Coding

URS

SRS

HLD

LLD

User Acceptance

Testing

System Testing

Integratio

n testing

Unit

testing

Delivery production

deployment

Maintenance and

enhancement

Verification Validation



QSIT Copyright


Software Testing Life Cycle


STLC Phases

� Requirement analysis

� Test Strategy

� Prepare Test Plan

� Test Case Designing

� Test Case Execution

� Bug Reporting, Analysis and Regression Testing

� Inspection and Release

� Client Acceptance and

Support During acceptance

� Test Summary Analysis



QSIT Copyright


� All the business and test objectives are identified and test strategy shall be defined in order to meet those objectives.

� As far as possible all the test objectives to be met are defined in quantitative terms in the test strategy.

� Examples of test objectives can be like 90 % of code coverage, finding at least a minimum of 100 defects etc.

Test Strategy - Objective


Test Strategy

A test strategy will typically cover the following aspect

� Definition of test objective

� Strategy to meet the specified objective

� Overall testing approach

� Test Environment

� Test Automation requirements

� Metric Plan

� Risk Identification, Mitigation and

Contingency plan

� Details of Tools usage

� Specific Document templates

used in testing



QSIT Copyright


Test Strategy

� Test strategy is statement of overall approach of testing to meet the business and test objectives.

� It is a plan level document and has to be prepared in the requirement stage of the project.

� It identifies the methods, techniques and tools to be used for testing .

� It can be a project or an organization specific.

� Developing a test strategy which effectively meets the needs of the organization/project is critical to the success of the software development

� An effective strategy has to meet the project and business objectives

� Defining the strategy upfront before the actual testing helps in planning the test activities


� Test approach will be based on the objectives set for testing

� Test approach will detail the way the testing to be carried out

� Types of testing to be done viz Unit, Integration and system

testing

� The method of testing viz Black–box, White-box etc.,

� Details of any automated testing to be done

Test Strategy - Approach



QSIT Copyright


� All the Hardware and Software requirements for carrying out testing shall be identified in detail.

� Any specific tools required for testing will also be identified

� If the testing is going to be done remotely, then it has to be considered during estimation

Test Strategy – Test Environment


� Metric plan will contain the measures that will indicate the effectiveness of testing .

� All the metrics to be tracked for testing like defect density ,residual defect density ,code coverage etc are identified in test strategy .

� The objectives set for testing are also measured in the metric plan.

Test Strategy – Metric Plan



QSIT Copyright


� Risk analysis should carried out for testing phase

� The risk identification will be accomplished by identifying causes-

and-effects or effects-and-causes

� The identified Risks are classified into to Internal and External

Risks.

� The internal risks are things that the testing team

can control or influence.

� The external risks are things beyond the control or

influence of the testing team

Test Strategy – Risks


Test Strategy – Risks

� Once Risks are identified and classified, the following activities will be carried out

– Identify the probability of occurrence

– Identify the impact areas – if the risk were to occur

– Risk mitigation plan – how avoid this risk?

– Risk contingency plan – if the risk were to occur what do we do?



QSIT Copyright


� Tools required in the project and rationale to use that particular

tool(s) should be documented.

� Automation requirements should be identified and analyzed

�Which phases need to be automated

� How much of testing needs to be automated

� Based on the automation requirements, tools strategy should be

prepared.

Test Strategy – Tools Usage


Prepare Test Plan

This involves following activities:

� Objective

� Scope

� Environment

� Schedule/ Mile Stone

� Entry/Exit Criteria

� Resources & Responsibility

testplan template

IEEE testplan template



QSIT Copyright


Test Case Writing:

A specific set of steps and data along with expected results for a particular test objective. A test case should only test one limited subset of a feature or functionality.

Test Case


A typical test case sheet contains these fields…

� Test Case ID: It is unique number given to test case in order to

be identified.

� Test description: The description if test case you are going to

test.

� Revision history: Each test case has to have its revision history

in order to know when and by whom it is created or modified.

� Function to be tested: The name of function to be tested.

� Priority/Build/time: The priority of the Test Case, Build version

of the test case and the Minimum time-to-execute the test case

is also mentioned.

Test Case Design



QSIT Copyright


� Environment: It tells in which environment you are testing.

� Test Setup: Anything you need to set up outside of your

application for example printers, network etc.,

� Test Execution: It is detailed description of every step of

execution.

� Expected Results: The description of what you expect the

function to do.

� Actual Results: Pass /Failed.

If pass – What actually happens when you run the test

If failed - Description of what you've observed.

Test Case Design (Contd…)


Characteristics of a Good Test:

� Tests are likely to catch bugs

� No redundancy

� Not too simple or too complex.

TIP: Test case is going to be complex if you have

more than one expected results.




QSIT Copyright


Test Data Inputs:

� Before testing, the tester should plan what kind of

data he is giving for test.

� Give data inputs as functional, boundary, stress,

performance, usability values etc.

Test Data Inputs


Test Scripts:

� A test script is the executable form of a test.

� It defines the set of actions to carry out in order to conduct a test

and it defines the expected outcomes and results that are used to

identify any deviance in the actual behavior of the program from

the logical behavior in the script (errors during the course of that

test).

� In essence it is a program written for a human computer (tester)

to execute.

TestCase Template TestCase Status Template




QSIT Copyright


Test Case Template


• Test execution is culmination of testing activities which involves executing the planned test cases and conducting of the tests.

• Test execution phase broadly involves execution and reporting.

Test Execution



QSIT Copyright


Test execution consists of following activities to be performed

� Creation of test setup or Test bed

� Execution of test cases on the setup

� Test Methodology used

� Collection of Metrics

� Defect Tracking and Reporting

� Regression Testing

Test Execution - Activities


Test Execution - Cycle

�Test Data: Inputs which are to exercise the system under test.



QSIT Copyright


� All the defects/bugs found during testing is logged and they are

classified based on the severity and priority.

� They are assigned to developers for fixing.

� Each defect found is logged and tracked to closure.

� Test summary report shall be prepared containing the details like

numbers of test cases passed/failed ,number of cycles of testing.

� Generally test defect logging and tracking tools are used for this

purpose.

Test report template

Test Execution – Defect Tracking & Reporting


Bug

A fault in a program which causes the program to perform in an

unintended or unanticipated manner or deviation from the requirement specification or the design specification is referred as a bug.

Test Execution – Defect Tracking & Reporting



QSIT Copyright


Bug Life Cycle?

Validate

Close

Duplicate/ Invalid

Submit Bug

Review, verify and qualify

Defer Assign

More Info

Fix/Change

Update

Cancel

In-work


Defer Bug

Deferred is where in the developer says that he cannot fix that bug

� Reasons for that would be:

� It may change entire contents

� Technology Problem

� Cost of fixing

Or

� Whenever we accept some changes in the modules

� Whenever there is less time available they may say that they will fix in future release



QSIT Copyright


Reporting/Logging a Bug/Defect

A Bug/Defect is reported with the following details

� Defect ID : Bug ID _1546

� Build ID : B04

� Module Name : Sent Items

� Status : Open /Assigned/Fixed/Closed/Reopen

� Severity : Blocker/Critical/Major/Minor

� Priority : High/Medium/Low

� Test Case Name: ABC/Yahoo_sent_items

� Test Environment : WinXP, Netscape Navigator

� Found By : Name of the tester



Brief Description : Sent mails are not listed in sent items page

Description : Following are the steps to Reproduce the defects

� Open browser and enter “Url”

� Login with a valid user name and password

� Click on compose link

� Enter valid data in To, CC, BCC and in body of the mail and

click on send button

� Click on sent items

� Expected results : sent mail should be listed in sent items page

� Actual Result : Sent items is not displayed

this difference shows you the defect



QSIT Copyright



Process to be followed while you track a defect manually

� Find a bug

� Check in the repository to make sure its not duplicate

� Prepare a defect report

� Send a copy to dev lead or development team distribution list and mark a cc to testing lead

� Enter the details in defect tracking sheet (optional)


Classification of Bugs

Two attributes are used whenever a Bug/Defect is detected

� Severity – Impact of the bug on the application

Blocker: This bug prevents testers from testing the software.

Critical : The software crashes, hangs, or causes you to lose data.

Major : A major feature is broken.

Trivial : A cosmetic problem, such as misspelled or misaligned text.

Enhancement: Request for new feature or enhancement.

� Priority – Impact on the developer/management

High

Medium

Low



QSIT Copyright


Examples on Severity and Priority

Which Does not Effect the Performance of the Application

� Spelling Mistakes

� Grammatical Mistakes

Example 1:

Severity : Minor

Priority : Low

Example 2:

Welcome page of Yahoo shows Well to Yahoo –

Severity : Low

Priority : High

Example on Low Severity and Priority Bug

Enter Pwsd

Well to Yahoo

Example on low severity and a high priority Bug



If you are going to test Dominos Pizza web site, in the home page the door delivery phone number is displaying wrong one in this situation severity wise it won’t affect any thing but it will affect the stake holder business so its “Low severity & High Priority”

� If you take the example of Citi bank web site think the login page is not working then its “High severity & High Priority”

Example on Low Severity and High Priority Bug

Example on High Severity and High Priority Bug



QSIT Copyright



In the big bazaar application the items that is to be brought has to be listed and has to be categorized according to category, it should not take under food items cloths and under cloths it should not take foodMajor Severity , Low Priority

Logo Has ChangedCritical Severity, Low Priority

minimizes when the application is invoked,solution is: the application should be MaximizedMinor Severity, Medium Priority

Accepts blank in the Password, Accepts Critical Severity, Medium Priority

You have entered Wrong password/Wrong UsernameMinor Severity, High Priority


� Testing is done in rounds. Software passes though several rounds

before it gets released.

� Testing is stopped when there are many major bugs and critical

system errors, when further testing is not feasible

� Testing is stopped when the system becomes un stable during

test.

� When those critical errors are fixed and system is stable, the next

round of testing is started .

� The test cases are executed all over again in each round.

Test Execution – Start & Stop Criteria



QSIT Copyright


� Metrics are important to collect for any project as it helps in

monitoring the health of project /product /application under test.

� Metrics are collected during testing depending on the objective of

testing.

� There are different kinds of metrics which can be used in testing.

� For example LOC (Lines of Code) is a metric for Unit testing. i.e.,

Code related metrics comes in the category of Unit testing.

� Test case related metrics can be used for all kinds of testing types,

since for every type of test, we will write test cases.

� Normally source of errors/defects is requirements, design or

Implementation. Testing can not be the source of errors/defects.

Test Execution – Metrics


• Typical testing metrics– No of LOC tested per day – No of test cases executed per day– Number of test cases failed– Number of bugs reported – Number of rejects of bugs reported– Number of persons working on the test plan during the month– Total Effort spent on test execution in person hours– Number of simulations made during the month– Number of out of norms per month– Average time taken to find a problem categorized by priority– Defect density– Review efficiency




QSIT Copyright


• Having a complete Metrics data helps in building the “Trustworthy, Capable, Reliable and Predictable” Product or Application.

– Productivity Increase

– Field error rate Decrease

– Customer Overhead reduces

– Improvement release on release

– On time delivery

– Risks in Program identified and proactively intimated

– Decrease in Cost of Quality

– Decrease in Re-work

– Helps in identifying the area which needs focus in the project/organization for improvement

– Helps in arriving at better estimates for the project



• Software testing 10 rules?

– Test early and test often.

– Integrate the application development and testing life cycles.

– Formalize a testing methodology; you'll test everything the same way and you'll get uniform results.

– Develop a comprehensive test plan; it forms the basis for the testing methodology.

– Use both static and dynamic testing.

Test Execution – Testing Rules ?



QSIT Copyright


Test Execution – Testing Rules ?– Define your expected results.

– Understand the business reason behind the application. You'll write a better application and better testing scripts.

– Use multiple levels and types of testing (regression, systems, integration, stress and load).

– Review and inspect the work, it will lower costs.

– Don't let your programmers check their own work; they'll miss their own errors.


• What makes a good test engineer?

– A good test engineer has a 'test to break' attitude.

– An ability to take the point of view of the customer

– A strong desire for quality, and an attention to detail.

– Tact and diplomacy for maintaining a cooperative relationship with developers.

– An ability to communicate with both technical (developers) and non-technical (customers, management) people.

Testing



QSIT Copyright


Testing– Previous software development experience can be helpful, gives the tester an appreciation from the developers' point of view, and reduce the learning curve in automated test tool programming.

– Judgment skills are needed to assess high-risk areas of an application on which to focus testing efforts when time is limited.

– Notice little things that others miss/ignore (See symptom not bug)

– Good testers use files, Databases and all the other accoutrements of an organized mind

– Testers are fundamentally honest and incorruptible

– Last but not the least is “Patience”


An effective testing team includes a mixture of members who has

– Testing expertise

– Tools expertise

– Database expertise

– Domain/Technology expertise

Testing Team Structure



QSIT Copyright


� The testing team must be properly structured, with defined roles and

responsibilities that allow the testers to perform their functions with

minimal overlap.

� There should not be any uncertainty regarding which team member

should perform which duties.

� The test manager will be facilitating any resources required for the

testing team.

� Clear Communication protocol should be defined with in thetesting team to ensure proper understanding of roles andresponsibilities.

� The roles chart should contain both on-site and off-shore teammembers.

Testing Team – Role and Responsibilities


• Test Manager

– Single point contact between

onsite and offshore team

– Prepare the project plan

– Test Management

– Test Planning

– Interact with onsite lead

Client QA manager

– Team management

– Work allocation to the team

– Test coverage analysis




QSIT Copyright


• Test Lead

– Resolves technical issues for the product group

– Provides direction to the team members

– Performs activities for the respective product group

– Review and Approve of Test Plan / Test cases

– Review Test Script / Code

– Approve completion of Integration testing

– Conduct System / Regression tests

– Ensure tests are conducted as per plan

– Reports status to the Offshore Test Manager



• Test Engineer

– Development of Test cases and Scripts

– Test Execution

– Result capturing and analysing

– Defect Reporting and Status reporting




QSIT Copyright


Collaborative Approach - Roles of Developers & Testers

Knowledge transfer from Development Team

� How the Product/Module Works?

� Intentions of each feature

� Areas or features which are more risk/error prone

� the schedule for delivering the remaining feature sets

� Concerns about hidden/abstract activities

Knowledge transfer from Testing Team

� Test coverage including types & levels of testing

� The feature as understood by the testers

� Required information so as to make feature more granular

� Testing phases and schedule

� What testers expect from the development team

Documents

Software Testing Life Cycle