50
Quality Assessment Handbook QASS i Quality Assessment Handbook

Qa Handbook

Embed Size (px)

DESCRIPTION

Testing Policy

Citation preview

Page 1: Qa Handbook

Quality Assessment Handbook

QASS

i Quality Assessment Handbook

Page 2: Qa Handbook

REVISION HISTORY

Version Number

Effective Date

Brief description of change

Affected Section(s)

Prepared By

Reviewed By Approved By

ii Quality Assessment Handbook

Page 3: Qa Handbook

REVISION HISTORY................................................................................................................................ II

1. POLICY STATEMENT................................................................................................................... 3

2. INTRODUCTION........................................................................................................................... 4

2.1. TESTING APPROACH..........................................................................................................................42.2. “V” MODEL......................................................................................................................................52.3. STLC THREADS................................................................................................................................6

3. THREAD: TEST DEFINITION....................................................................................................... 8

3.1. ACTIVITY: PERFORM TEST ANALYSIS.......................................................................................................93.2. ACTIVITY: DEFINE TESTING TECHNIQUES...............................................................................................103.3. ACTIVITY: VALIDATE REQUIREMENTS....................................................................................................113.4. ACTIVITY: DEVELOP TESTING STRATEGY.................................................................................................12

4. THREAD: TEST DESIGN............................................................................................................. 13

3.1. ACTIVITY: DEFINE TRACEABILITY TRIAGE...............................................................................................143.2. ACTIVITY: INFRASTRUCTURE REQUIREMENTS..........................................................................................153.3. ACTIVITY: TEST DATA REQUIREMENTS..................................................................................................16

5. THREAD: TEST DEVELOPMENT................................................................................................17

5.1. ACTIVITY: DEVELOP TEST SUITES..........................................................................................................175.2. ACTIVITY: GENERATE TEST DATA.........................................................................................................185.3. ACTIVITY: SET UP TEST SYSTEM...........................................................................................................19

6. THREAD: TEST EXECUTION...................................................................................................... 21

6.1. ACTIVITY: EXECUTE TEST SCRIPTS.........................................................................................................216.2. ACTIVITY: ITERATE TEST EXECUTION.....................................................................................................226.3. ACTIVITY: TEST REPORTING AND METRICS.............................................................................................23

7. THREAD: TEST EXIT.................................................................................................................. 24

7.1. ACTIVITY: CREATE EXIT REPORT...........................................................................................................247.2. ACTIVITY: DISTRIBUTE EXIT REPORT......................................................................................................25

8. THREAD: DEFECT MANAGEMENT............................................................................................26

8.1. ACTIVITY: DEFECT PREVENTION...........................................................................................................278.2. ACTIVITY: DEFECT DETECTION.............................................................................................................288.3. ACTIVITY: ROOT CAUSE ANALYSIS........................................................................................................298.4. ACTIVITY: DEFECT RESOLUTION...........................................................................................................308.5. ACTIVITY: PROCESS IMPROVEMENT......................................................................................................31

9. TEST MANAGEMENT................................................................................................................. 32

9.1. ESTABLISH AND DISTRIBUTE TEST PLAN.................................................................................................32

1 Quality Assessment Handbook

Page 4: Qa Handbook

9.2. OBTAIN COMMITMENT TO TEST PLAN..................................................................................................339.3. RECONCILE WORK AND STAFF..............................................................................................................339.4. MONITOR PROGRESS AGAINST PLAN....................................................................................................339.5. MANAGE DEFECTS TO CLOSURE...........................................................................................................33

10. SERVICE OFFERING................................................................................................................ 34

10.1. FUNCTIONAL TESTING....................................................................................................................3410.2. SYSTEM TESTING...........................................................................................................................3410.3. INTEGRATION TESTING...................................................................................................................3410.4. AUTOMATION TESTING..................................................................................................................3410.5. USABILITY TESTING........................................................................................................................3410.6. REGRESSION TESTING....................................................................................................................3510.7. PERFORMANCE TESTING.................................................................................................................35

11. TOOL BOX............................................................................................................................... 36

2 Quality Assessment Handbook

Page 5: Qa Handbook

1. POLICY STATEMENT

CORP IT QAT’s mission is to effectively, and efficiently provide timely, accurate, and useful quality management information and services.

As we move towards automating complex business processes, quality of the implementation becomes all the more critical and complex. By defining an adaptable and flexible testing policy, we agree to adhere and follow this document for all testing services within CORP IT.

Depending on project objectives, the process outlined in this document CORP IT Quality Assessment Team will recommend the degree of quality risk mitigation required. This will then determine the extent of the Services, Threads, and Activities that should be adopted to minimize quality risks and deliver faultless software to our business.

3 Quality Assessment Handbook

Page 6: Qa Handbook

2. INTRODUCTION

Software testing is a core quality control component of any software development lifecycle and essential to effective software quality assurance. Testing is the means of achieving two major objectives – Verification (You built it right) and Validation (You built the right thing). By themselves, Verification and Validation do not guarantee software quality; various test management threads, and other aspects of software engineering are required.

Quality Assessment Hand Book focuses on defining QASS’s testing approach and reflects its years of experience and resulting best practices in this area. Like any other framework, this Hand Book is designed to be flexible and customizable so that it can be tailored for use on a variety of testing initiatives. Specifically, this is designed to support testing on the following types of initiatives:

ORGANIZATION’S IT Managed Development Projects

Both production support changes and enhancement requests on ORGANIZATION’S IT Managed Applications

Stand-alone Testing Service for any software developed by teams outside ORGANIZATION’S IT or other third-party

2.1. TESTING APPROACH

This document identifies the QASS’s testing methodology as implemented across all projects. Core elements of QASS’s approach to testing are -

Adaptable to all project types and methodologies by carefully and thoughtfully integrating with project management

Support of Process Improvement Goals as defined by various functional teams in ORGANIZATION’S IT

Risk Based Testing to detect the software’s technical risks, and to control critical path failures

Defect Management to include collection of defect measures and causal analysis to identify, remove and prevent defects as close to injection as possible

4 Quality Assessment Handbook

Page 7: Qa Handbook

2.2. “V” MODEL

The V-Model is a structured testing framework emphasizing quality from the initial requirements stage through the final testing stage. The framework can be used with any project management or system development methodology. It focuses on testing throughout the development life-cycle, early development of test requirements, and early detection of errors. Each major deliverable in the development process is assessed, verified, validated and tested.

5 Quality Assessment Handbook

Page 8: Qa Handbook

“V” - Model

2.3. STLC THREADS

STLC Threads

Our STLC can be classified into threads of work performed across the phases of SDLC. The goal of this process is to depict the relationship and integration between testing and software development activities.

6 Quality Assessment Handbook

Page 9: Qa Handbook

The SDLC phases shown on the diagram are for illustrative purposes and are not defined within this document. A quick look at our STLC threads -

Test Definition– Defines the overall scope, strategy and performance objectives of the testing to be performed

Test Design – Decide environment and set up, testing techniques to be used, test data, and test sets & scenarios for automation and manual testing

Test Development –Build test scripts, and create test data; Create and configure test systems

Test Execution –Execute scripts, log defects, report metrics

Test Exit –Deliver Test Exit Report to confirm the software is ready to be deployed

Defect Management – Manage defect prevention, tracking, from detection till closure in compliance to the approach defined in this document

7 Quality Assessment Handbook

Page 10: Qa Handbook

3. THREAD: TEST DEFINITION

This thread defines overall test scope, Testing Strategy and performance objectives of the tests to be performed. A Testing Strategy driven by the risk analysis is delivered at the end of the thread that, details on planning and executing the testing phase in perfection.

The Testing Strategy provides the roadmap for the overall testing effort. Its primary goal is to provide a management-level view of all testing to be accomplished and the sequencing and dependency between the various types and iterations of testing.

This thread contains the following activities:

Perform test analysis

Define testing techniques

Validate requirements

Develop Testing Strategy

8 Quality Assessment Handbook

Page 11: Qa Handbook

3.1. ACTIVITY: PERFORM TEST ANALYSIS

The purpose of this activity is to identify and assess the software’s technical risks and overall impact on the business context so the optimal testing scope can be determined and planned.

Ultimately the question of how much testing is enough is a business decision to be made by the business owner that owns the application software. The optimal level of testing cannot be decided without the assistance of the business area owners.

9 Quality Assessment Handbook

Page 12: Qa Handbook

3.2. ACTIVITY: DEFINE TESTING TECHNIQUES

The purpose of this activity is to assess how every item on scope would be tested and what technique could be applied to test a particular feature of the item. Test scope and specifications (from the Test Analysis) grouped by levels, modules etc. are required to focus on testing. Document the test structure with applicable testing techniques, test data, and approach for each area. Ensure the structure allows the specification coverage to be measured.

From your own experience, or other experience you have access to, identify existing techniques that will either meet the requirements of the test approach, or can be adapted to meet them.

10 Quality Assessment Handbook

Page 13: Qa Handbook

3.3. ACTIVITY: VALIDATE REQUIREMENTS

Defects introduced early in the effort to engineer a system due to poorly identified requirements are generally seen as a major factor leading to high system and software costs, especially if the defective requirements are undetected until later development phases in the lifecycle. The ultimate savings due to error detection, diagnosis, and correction before a trial system is produced are generally great.

Traceability matrix is created by associating requirements with the work products that satisfy them and is used to ensure all the requirements are met. The traceability matrix helps in locating affected system components when there is a requirement change or a defect identified. This thread addresses requirements validation through Requirements Traceability Matrix ensuring that the

set of requirements is correct, complete, and consistent

a system can be created that satisfies the requirements

and the solution can be tested to prove that it satisfies the requirements

11 Quality Assessment Handbook

Page 14: Qa Handbook

3.4. ACTIVITY: DEVELOP TESTING STRATEGY

Develop a Testing Strategy to provide a roadmap for testing that defines the testing scope, objective, process and plan. Other areas that will be addressed in the Testing Strategy document include:

Test Infrastructure & Tool Requirements

Test Data Development & Management Process

Defect Management Process - Refer the thread Defect Management for additional information on this topic.

Software Risk Profile

12 Quality Assessment Handbook

Page 15: Qa Handbook

4. THREAD: TEST DESIGN

This thread decides test environment setup, test scenarios, test data design for manual, automation, functional, and non-functional testing. Test design sets the foundation to accomplish the most important test objective – ‘find the majority of the errors with a minimum amount of effort and time’.

Structured testing implies that test design techniques are applied, possibly supported by tools and guidelines. This thread aims at deriving and selecting test sets and subsequently test scenarios from requirements and design specifications. Traceability Matrix will be revisited to translate the current version to tangible test sets, test types, test data and test environment.

This thread contains the following Activities:

Define traceability triage

Infrastructure requirements

Test data requirements

13 Quality Assessment Handbook

Page 16: Qa Handbook

3.1. ACTIVITY: DEFINE TRACEABILITY TRIAGE

Using EAST1 technique, prioritize test scenarios based on identified product risks. Based on items/ features to be tested, environmental needs, inter-dependencies and any special procedural requirements, prioritize the test scenarios in the order of preferred execution.

Also identifies the right technique to execute each scenario and update the traceability matrix as appropriate.

1 EAST – Enterprise Application System Test is a technique devised by QASS to ensure design of efficient test strategies covering heterogeneous domains and distributed technologies. QASS takes utmost care while designing unique test strategies and test data, based on the impact analysis of the product/ change on applications in the enterprise context.

14 Quality Assessment Handbook

Page 17: Qa Handbook

3.2. ACTIVITY: INFRASTRUCTURE REQUIREMENTS

The purpose of this activity is to decide required environmental settings which include hardware and software requirements to perform testing activities of all identified testing techniques. Management procedures are created to setup, maintain and manage the test environment.

15 Quality Assessment Handbook

Page 18: Qa Handbook

3.3. ACTIVITY: TEST DATA REQUIREMENTS

The purpose of this activity is to identify specific test data necessary to support test scenarios and techniques.

A test data set is called ideal if all the application errors get identified with the minimum size of data set. Major ways of obtaining the right test data is to get from production and front-end, than generating manually. Often times, it is good to go with the mix of real-time and generated data to suit specific scenarios.

QASS has automated test data loads for critical applications and scenarios, and will continue to contribute to the same.

16 Quality Assessment Handbook

Page 19: Qa Handbook

17 Quality Assessment Handbook

Page 20: Qa Handbook

5. THREAD: TEST DEVELOPMENT

Test development activities follow documentation threads, definition and design. The objective of this thread is to develop test suites, test data and test systems.

Test development thread contains the following activities

Develop test suites

Generate test data

Set up test system

5.1. ACTIVITY: DEVELOP TEST SUITES

The purpose of this activity is to build manual and automation test scripts for the scenarios defined by the Traceability Matrix. A Test Coverage Matrix is created by cloning the Traceability Matrix, with detailed test scripts and steps. Often times, this will be the final test script document unless otherwise decided by the Test Manager.

18 Quality Assessment Handbook

Page 21: Qa Handbook

5.2. ACTIVITY: GENERATE TEST DATA

The purpose of this activity is to generate test data as defined in the test data requirements. Specific test required to be able to run the test scripts is created using automated scripts or tools or manually. Archive the data to allow a restore of the initial situation in the future.

QASS also focuses on creating generic test data consisting of master data and some initial content of primary data. The data however, is stored in the form of scripts and data files making it portable to any environment. Test environments themselves are considered as test data storages and production environments as test data generators (apart from scripts).

Include RBAC for testers and application user management, as demanded by Test Coverage Matrix.

19 Quality Assessment Handbook

Page 22: Qa Handbook

5.3. ACTIVITY: SET UP TEST SYSTEM

Test Systems are those environments in which tests are conducted. A managed and controlled test system is indispensable for any testing. QASS reviews the usability of the designated test system and works with Systems and Applications Administration team in resolving any deviations. Managing and controlling a test system is the responsibility of the SAAT, while QASS owns the system.

QASS owns Test Systems across multiple technologies and platforms supporting most of the applications. New Systems are built occasionally on existing environments, to support adhoc/ short term application developments and complete access is provided to QASS. Test data is loaded to the identified Test System, and the necessary configurations at web layers, middle-tiers and database layers are set up.

20 Quality Assessment Handbook

Page 23: Qa Handbook

21 Quality Assessment Handbook

Page 24: Qa Handbook

6. THREAD: TEST EXECUTION

Test execution is an important activity that ensures quality of the software/application which is under assessment. The objective of this thread is to execute test scripts with reference to the Test Coverage Matrix. During the test execution thread, defects are detected and defect reports are written. These are logged into the test tracker Mantis, which automatically alerts all relevant personnel. Proper defect classification scheme is established and Defect Management thread describes more about handling the defect lifecycle until closure.

6.1. ACTIVITY: EXECUTE TEST SCRIPTS

Tests are executed in line with previously specified test coverage and schedule. The documented test scripts are executed manually or automatically with the help of testing tools.

Test Coverage Matrix ensures all requirements are tested thoroughly, in the order specified using the generated test data. Quality of the application should be decided based on test scripts status (i.e. Passed, Failed and Blocked).

This activity is supported by Defect Management and the tasks are repeated in case of open functional, non-functional discrepancies with the expected results. The next iteration begins after the fixes are migrated to Test System.

22 Quality Assessment Handbook

Page 25: Qa Handbook

23 Quality Assessment Handbook

Page 26: Qa Handbook

6.2. ACTIVITY: ITERATE TEST EXECUTION

Start a new iteration if the previous test fails, after the Test System is updated with fixes. The purpose of this activity is to ensure quality product is migrated to production.

The general myth with most of the PMs is that testing increases the delivery time and is always the main reason for delayed launches. Actually testing is not the time-consuming activity in SDLC. Analyzing and fixing the detected defects are the time-consuming activities.

After the code patching is done, QASS repeats the testing from the beginning. Recommended best practice is that the resolving defects should be done in parallel to testing and patch up the code towards the end and release for next round/ iteration of testing.

24 Quality Assessment Handbook

Page 27: Qa Handbook

6.3. ACTIVITY: TEST REPORTING AND METRICS

Testing metric is just like any other tool used to measure quality of software product to serve best to the customer. It is important to keep track of the progress of testing and its effectiveness along with product quality. QASS generates and distributes defect reports using the DAST2 tools.

At a basic level, simple defect reports are generated, in an ongoing basis. During (and also after) test execution, one gets to see number of defects by status, severity and category to help Project Manager initiate next steps.

Application and server performance metrics will be distributed for all non-functional testing tasks.

2 DAST – Defect Analytics and Statistical Trends is a quality model devised by QASS for defect management. This technique allows realistic defect categorization, defect analysis, and trend reporting of applications covered under EAST.

25 Quality Assessment Handbook

Page 28: Qa Handbook

7. THREAD: TEST EXIT

The thread Test Exit confirms the closure of testing, on a product by issuing an exit report to all the stakeholders. Contents of Test Exit Report include an executive summary of testing, statement on product quality, open risks and limitations, known defects if any, along with the state of the product-readiness for deployment.

7.1. ACTIVITY: CREATE EXIT REPORT

The purpose of this activity is to draft an executive summary on the progress of testing, and the product quality. Test Manager should study the open risks, known defects, and limitations with the Business Analyst and (or) Business User to quantify the impact before exiting testing phase.

26 Quality Assessment Handbook

Page 29: Qa Handbook

7.2. ACTIVITY: DISTRIBUTE EXIT REPORT

Exit report is a valuable statement which acts as a certificate for the application to move to the next stage. Distribute the created Exit Report to PM and other stakeholders of the project.

27 Quality Assessment Handbook

Page 30: Qa Handbook

8. THREAD: DEFECT MANAGEMENT

Delivering software without defects is not possible, but the number and impact can be minimized with an effective Defective Management process leading to minimizing Cost of Failures.

The primary goal of the Defect Management thread is preventing defects which when impossible, detect early on and reduce the impact. Imperfect processes and poor adherence to one cause majority of the defects and hence processes should be altered to prevent defects. QASS has entwined the Defect Management process within its STLC threads.

Defect Management Process

28 Quality Assessment Handbook

Page 31: Qa Handbook

8.1. ACTIVITY: DEFECT PREVENTION

Defect Prevention focuses on avoiding defects before and during coding rather than detecting and resolving later in the SDLC. Identifying the process related root causes and creating action plans that prevent the re-occurrence is always recommended for Project Management.

Product related Defect Prevention begins with EAST in the thread Test Design that helps identify critical risks associated with the product. This helps in planning for the expected defects by estimating and minimizing the impact.

29 Quality Assessment Handbook

Page 32: Qa Handbook

8.2. ACTIVITY: DEFECT DETECTION

Main purpose of Defect Detection is to find defects early on the process, before Cost of failures becomes expensive. QASS’s well-articulated threads ensure efficient defect detection throughout the STLC, with the help of DAST and EAST.

QASS has a well-defined defect category model that seamlessly fits into DAST, which helps in avoiding conflicts during defect acknowledgement.

30 Quality Assessment Handbook

Page 33: Qa Handbook

8.3. ACTIVITY: ROOT CAUSE ANALYSIS

One of the best problem solving methods is Root Cause Analysis, which aims at eliminating root causes as opposed to resolving obvious issues. QASS follows failure-based reactive approach of detection, analysis, and resolution.

The tasks involved in this activity are the same irrespective of the process or product root cause. QASS recommends 5 Why technique to identify root cause and corrective/ preventive actions. Test Manager should organize Defect Review meeting with relevant participants, to arrive at CAR (Causal Analysis Report) that triggers resolution.

31 Quality Assessment Handbook

Page 34: Qa Handbook

8.4. ACTIVITY: DEFECT RESOLUTION

Defect Resolution follows the distribution of Causal Analysis Report (CAR). The resolution process begins with scheduling and implementing the fix. The patched up code is finally migrated to Test System for re-testing and closure of the defect. Every time a code build is released to QASS, this triggers regression and re-testing of features.

32 Quality Assessment Handbook

Page 35: Qa Handbook

8.5. ACTIVITY: PROCESS IMPROVEMENT

This activity mandates the team to revisit the process that originated the defect to fix what caused the defect. The cause however, was identified in the earlier activity for Root Cause Analysis. The focus is more on the human-factor which when implemented will improve effectiveness of a process.

This activity adds more value by identifying other possible defects that may not have been detected thus, strengthening the proactive measures.

33 Quality Assessment Handbook

Page 36: Qa Handbook

9. TEST MANAGEMENT

QASS’s Test Management methodology is guided by the following principles -

Test Strategically – defining enterprise Testing Strategy

Test to Mitigate Risk – analyze and prioritize business risks to test highest risk areas first, so as to strike optimal balance of risk mitigation through testing against the defined scope

Test Early & Continuously – start as soon as possible and continue from development through deployment

Test Visibly –frequently review success criteria and constantly deliver defect metrics

Automate for Efficiency – improve test efficiency with appropriate automation

Test Independently – focus on independent and impartial testing

Quality Assessment Team treats testing as a managed process, which involves in managing all the threads discussed so far, alongside a few repeatable tasks. Test Management ensures that the strategy is based on product risk assessment and defines what testing is required, when, how and by whom. Commitments are also established with Stakeholders and revised as needed.

Most often the STLC threads are implemented by QASS members. Very rarely are the individual threads implemented by other groups like – Business Analysts, Process Support Analysts, Developers, Administrators, End Users, Business Users, Vendors (in case of SAAS/ COTS), and or the engagement team (when outsourced within or outside the organization). However, the Test Management is the responsibility of QASS by all means.

Test Manager monitors and controls testing to ensure it is going as planned and actions are taken if deviations occur. Test Management consists of the following repeatable tasks.

9.1. ESTABLISH AND DISTRIBUTE TEST PLAN

Test planning involves performing a product risk assessment on the test object and defining a differentiated test strategy based on the risks identified (as described in the thread Test Definition). It also involves developing test estimates for the testing to be performed, and a top-level work breakdown structure and test milestones clearly defining the scope of the estimate.

Estimates in conjunction with strategy are used as the basis for managing testing and communication to all stakeholders.

34 Quality Assessment Handbook

Page 37: Qa Handbook

9.2. OBTAIN COMMITMENT TO TEST PLAN

Select relevant stakeholders from customers, end users, developers, architects, analysts, testers, project managers, administrators, and others who may be affected by, or may affect, the product as well as the test process.

This task requires the test manager to organize review (walk through) of the estimates and strategy with all stakeholders and more specifically with business analyst. Using the WBS, test manager obtains commitment from relevant stakeholders responsible for performing and supporting test plan execution.

9.3. RECONCILE WORK AND STAFF

Review the test estimates and strategy to reflect available and estimated staff. Discuss differences between available staff and estimates on WBS with Project Managers.

Reconcile the differences by lowering or deferring technical performance, allocating more staff, finding ways to increase productivity, adjusting staff-skill mix or revising the schedule.

9.4. MONITOR PROGRESS AGAINST PLAN

Test manager should monitor and control the testing phase against the plan for performing the process and take appropriate actions as demanded by the context. Test environments, test data management, maintaining traceability-coverage, defect management and test execution are the activities that should be monitored and controlled.

Test Manager also reviews status with project managers periodically and with higher management as felt necessary and, ensure adherence to the process, standards, and procedures, and address any non-compliances.

9.5. MANAGE DEFECTS TO CLOSURE

Refer to the thread Defect Management for more detailed discussion. Any discrepancies detected during test execution are reported as defects, when there are differences between actual and expected results. Appropriate actions on the defects are decided by the project team in a defect review meeting organized by test manager. The defect is then tracked to closure using defined defect management process.

35 Quality Assessment Handbook

Page 38: Qa Handbook

10. SERVICE OFFERING

QASS provides testing services based on the software development process, or testing objectives, or by the level of specificity of the test. Following are the various test services offered by the team.

10.1. FUNCTIONAL TESTING

Functional testing is an investigatory testing phase, where the focus is to have almost a destructive attitude and test not only the design, but also the behavior and even the believed expectations of the customer. It is also intended to test up to and beyond the bounds defined in the software/hardware requirements specification(s).

10.2. SYSTEM TESTING

Objective of System Testing is to test a completely integrated system to verify that it meets the requirements. Basically is a black-box testing that is based on overall requirements specifications, covers all combined parts of a system.

10.3. INTEGRATION TESTING

Integration testing seeks to verify the interfaces between components against a software design. Objective of this test is to expose defects in the interfaces and interaction between integrated components (modules). Progressively larger groups of tested software components corresponding to elements of the architectural design are integrated and tested until the software works as a system.

10.4. AUTOMATION TESTING

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions

10.5. USABILITY TESTING

Usability Testing is needed to check if the user interface is easy to use and understand.

36 Quality Assessment Handbook

Page 39: Qa Handbook

10.6. REGRESSION TESTING

Regression Testing focuses on finding defects after a major code change has occurred. Specifically, it seeks to uncover software regressions, or old bugs that have come back. Typically, regressions occur as an unintended consequence of program changes, when the newly developed part of the software collides with the previously existing code.

The depth of testing depends on the phase in the release process and the risk of the added features. They can either be complete, for changes added late in the release or deemed to be risky, to very shallow, consisting of positive tests on each feature, if the changes are early in the release or deemed to be of low risk.

10.7. PERFORMANCE TESTING

The term “Performance Testing” is often used interchangeably with ’stress’ and ‘load’ testing. QASS does recommend the tests as applicable to the context -

LOAD TESTING

It’s a performance testing to check system behavior under heavy loads, to determine at what point the system’s response time degrades or fails.

STRESS TESTING

System is stressed beyond its specifications to check how and when it fails. This test is usually performed under heavy loads like - putting large number beyond storage capacity, complex database queries, continuous input to system or database load.

37 Quality Assessment Handbook

Page 40: Qa Handbook

11. TOOL BOX

Refer to the Quality Governance Site - https://teams.portal.Organization’sinc.com/support/IT/appserv/qass/Tool%20Box/Forms/AllItems.aspx

NAME PURPOSE DFN

DES

DEV

EXE

EXIT

DEFMGT

TSTMGT

QASS-CHK-COMPATIBILITYTEST.XLSX Checklist to follow while executing OS, Browser Compatibility Tests

QASS-CHK-ETLTEST.XLSX Checklist to refer while testing ETLs (not necessarily on BI platform alone)

QASS-CHK-GUITEST.XLSX Web based GUI testing checklist, helps you test all UI controls

QASS-CHK-INTERFACETEST.XLSX Checklist to refer while testing application interfaces

QASS-CHK-MIGRATION.XLSX This helps you ensure proper migrations, to environments

QASS-CHK-PERFTEST.XLSX Checklist to refer for performance testing QASS-CHK-REQRVW.XLSX Requirements review checklist QASS-CHK-SECURITYTEST.XLSX Checklist used during Security Testing QASS-CHK-USABILITYTEST.XLSX Testers refer this checklist for all Usability

Tests

QASS-GDL-EAST.PDF Detailed document on Enterprise Application System Test (conceptualized and implemented by QASS), concept and framework

QASS-GDL-DEFATTRIBUTES.XLSX Defect attributes and life cycle, as applied in QASS Mantis Test Tracker

QASS-GDL-DAST.PDF Detailed document on Defect Analytics and Statistical Trends (Conceptualized and Implemented by QASS), concept and framework

QASS-GDL-ENVMANAGEMENT.PDF Discusses Test Environment management activities like data refresh etc.

QASS-GDL-QAHANDBOOK.PDF Detailed document on Quality Assessment Framework conceptualized and applied by QASS

QASS-TPL-ADHOCTESTSTRATEGY.DOCX

Test Strategy template used for adhoc test requests, and testing small projects, or support tickets

38 Quality Assessment Handbook

Page 41: Qa Handbook

NAME PURPOSE DFN

DES

DEV

EXE

EXIT

DEFMGT

TSTMGT

QASS-TPL-CAR.DOCX Template of Causal Analysis Report used to report root causes of critical defects and preventive/ corrective actions thereof

QASS-TPL-ISSUELOG.XLSX Used to record issues/ questions/ clarifications sought during the testing phase

QASS-TPL-MOM.DOCX Compact template for recording minutes of meetings

QASS-TPL-PERFTESTRESULTS.XLSX Template to share performance test results QASS-TPL-PERFTESTSCRIPTS.XLSX Template to draft and maintain performance

test scenarios/ scripts

QASS-TPL-PERFTESTSTRATEGY.DOCX Test Strategy template used for all performance test requests

QASS-TPL-RACIMATRIX.XLSX To list all activities, tasks, dependencies, deliverables and stakeholders of all phases

QASS-TPL-RELEASENOTE.XLSX Details the test release/ build, for every iteration

QASS-TPL-TESTCOVERAGE.XLSX Document that maintains and manages test coverage with respect to traceability matrix

QASS-TPL-TESTESTIMATES.XLSX Template to share test estimates (no estimation guidelines included)

QASS-TPL-TESTEXIT.PPTX Test exit report, delivered at the end of testing

QASS-TPL-TESTRESULTS.XLSX Template to share test results (non-performance)

QASS-TPL-TESTSCRIPTS.XLSX Template to maintain test scripts (non-performance)

QASS-TPL-TESTSTRATEGY.DOCX Test Strategy template used for large and medium projects

39 Quality Assessment Handbook