8/3/2019 Auditing Software Testing Process
1/4
Auditing Software Testing ProcessAuthor : Exforsys Inc. Published on: 5th Jun 2006
Auditing Software Testing Process
In this tutorial you will learn about Auditing Software Testing Process - Introduction, Types of
Testing Process Audits, Audit to verify compliance, Audit for process improvement/problem
solving,
Audit for Root Cause Analysis, Internal Audits, External Audits, Why Audit Software TestingProcess? How to Audit, What can be audited? and Summary.
Introduction:
To ensure transparency and reliability of the IT systems it may be necessary to audit the
Software Development Processes including the most important aspectSoftware TestingProcess.
Auditing is an important activity in organizations. In the context of testing it helps us ensure that
the Testing processes are followed as defined.
Types of Testing Process Audits
There can be various reasons to conduct Audits. The Audits may serve aim to achieve certaindefinite goals. Based on that we can classify them as follows:
Audit to verify compliance: In this type of auditing the prime motivation is to judge if theprocess complies with a standards. In these scenarios, the actual testing process is compared with
the documented process. For example ISO Standards require us to define our Software testing
process. The audit will try to verify if we actually conducted the testing as documented
Audit for process improvement/problem solving:In this type of audit the motivation is to audit and trace the various steps in the process and try to
weed out process problems. For instance it is observed that too many software defects escapeddetection even though the testing process was apparently followed. So the audit is done as a
preliminary step to collect facts and analyze them.
Audit for Root Cause Analysis
http://www.addthis.com/bookmark.php?v=250&winname=addthis&pub=exforsys&source=tbx32-250&lng=en-US&s=google_plusone&url=http%3A%2F%2Fwww.exforsys.com%2Ftutorials%2Ftesting%2Fauditing-software-testing-process.html&title=Auditing%20Software%20Testing%20Process&ate=AT-exforsys/-/-/4e19d523169f40de/1&frommenu=1&uid=4e19d523c90bb2f2&ufbl=1&ct=1&pre=http%3A%2F%2Fwww.google.co.in%2Furl%3Fsa%3Dt%26source%3Dweb%26cd%3D1%26ved%3D0CDgQFjAA%26url%3Dhttp%253A%252F%252Fwww.exforsys.com%252Ftutorials%252Ftesting%252Fauditing-software-testing-process.html%26rct%3Dj%26q%3Dwhat%2520internal%2520auditing%2520in%2520software%2520%2520testing%26ei%3DItQZTtewHI-qrAfI7ejPAQ%26usg%3DAFQjCNG_AT8AdKqX9Vxzw3RZypN5KZeH6w%26sig2%3DYD5RcI&tt=08/3/2019 Auditing Software Testing Process
2/4
In this type of audit the motivation is to audit the testing process is to find a Root Cause of a
specific problem. For example the customers discovered a huge problem with the software. Sowe retrace our testing steps to find out what went wrong in this specific case.
Internal Audits
Typically the internal audits are initiated from within the organizations
External Audits
External Audits are done by and initiated by external agencies
Why Audit Software Testing Process?
Auditing Test Process helps the management understand if the process is being followed as
specified. Typically Testing audit may be done for one or more of the following factors:
To ensure continued reliability and integrity of the process To verify compliance of standards (ISO, CMM, etc) To solve process related problems
To find the root cause of a specific problem
To detect or prevent Fraud
To improve the Testing process
Auditing of the Testing process may also be done if the Software Product is a mission criticalone such as used for Medical Life Support Systems
This is done to prevent any loop holes or bugs in the system
How to Audit
Typically the Audit of the Testing Process will include the following steps:
reviewing the Testing process as documented in the Quality Manual
This helps the auditor understand the process as defined.
Reviewing the deliverable documents at each step
Document reviewed include
............... Test Strategy
............... Test Plans
............... Test Cases
............... Test Logs
............... Defects Tracked
8/3/2019 Auditing Software Testing Process
3/4
............... Test Coverage Matrix
............... any other relevant records
Each of the above document provides a certain level of traceability that the process was followedand the necessary steps were taken
Interviewing the Project Team at various levels PM, Coordinator, Tester
Interviewing the Project Team members gives an understanding of the thought process prevalent
in those conducting the Testing Process.This can provide valuable insights over an above what was actually documented
ISACAww.isaca.org provides guidelines and standards for Auditing Information Systems &
Software Development Lifecycle
CISA stands for Certified Information Systems Auditor
Similarly independent agencies may verify the Test Processes and SDLC for ensuring
compliance with FDA ( Food and Drug Administration)
What can be audited?
Whether the test process deliverables exist as specified
The only thing that can be really verified in an audit is that the process deliverables exist. Theprocess deliverables are taken as a proof that the necessary steps were taken to do the testing. For
example if Test Logs exist, we assume that testing was done and the Test Logs were created as aresult of actual tests executed.
A separate exercise may be initiated to verify the authenticity of the Test Logs or other test
deliverables
Whether test cases created covered all requirements/use cases
This analysis reveals if the test coverage was sufficient. It indicates that whether the testing teamdid the best to provide adequate amount of testing
Whether all Defects were fixed
The Status of all the Defects logged is checked to verify if all were fixed and verified
8/3/2019 Auditing Software Testing Process
4/4
Whether there are any known bugs in the software released
Sometimes all the defects may not be fixed, the software may be released with known problems.Test Logs would indicate the actual results and evidence of any bugs being present.
Whether the levels of testing was effective enough
If Defects pass thru the various levels of testing undetected, it may reflect poorly on the
effectiveness of the testing process
What were the number of defects (Defect Leaks) that went by undetected in each phase Number of iterations of testing in each level Time taken to test each module/component This data may be used for process improvement Versions of source code actually tested