Software Quality Assurance - PLANIT TESTING · This paper attempts to dispel some common misconceptions regarding the roles of Testing and Software Quality Assurance (SQA) ... related

WHITEPAPER: Software Quality Assurance

Copyright © 2013 by Planit Software Testing, All rights reserved. This document, or any part thereof, may not be reproduced or stored in a retrieval system, or transmitted

in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 1

Software Quality Assurance Is it the same as Testing?

Teji Chopra, Senior Test Consultant

Planit Software Testing

Abstract

This paper attempts to dispel some common misconceptions regarding

the roles of Testing and Software Quality Assurance (SQA) by referring

to existing industry information and extensive professional experience. It

provides insights into the differences between these terms, while also

outlining challenges and providing recommendations for successful

SQA teams.

Planit Software Testing

www.planit.net.au

www.planittesting.co.nz




Starting point

The terms Testing and Software Quality Assurance (SQA)

are often used almost interchangeably in the IT industry

with testing professionals often classed as quality

assurance professionals.

So, are these terms the same?

Although both have quality as their overall objective, a

fundamental difference between the two is that testing is

performed after a product has been built, or in the case of

static testing, after a document has been written. In

contrast, quality assurance consists of activities to ensure

that quality will be built into the product.

To appreciate the differences between Testing and SQA

further, it is important to first understand the closely

related concepts, Quality Control (QC) and Quality

Assurance (QA).

Quality control

ISO9000 defines Quality Control as a part of Quality

Management focussed on fulfilling quality requirements.

Quality Control is a set of activities designed to evaluate

whether a developed product (project document,

developed system etc.) meets customer requirements. It

ensures that delivered products are checked for quality

and determines how well it is built. Its focus is to find

defects and to ensure that they are corrected.

QC is the responsibility of the project team.

Testing forms an integral part of Quality Control, as

displayed in Figure 1. However, not all QC activities are

testing activities. Code inspections, technical reviews and

stage gates are other examples of Quality Control

activities.

Quality assurance

ISO9000 defines Quality Assurance as a part of quality

management focused on providing confidence that

quality requirements will be fulfilled.

The goal of QA is to provide assurance that a product will

meet the customer’s quality expectations. It consists of

activities designed to ensure that quality will be built into

the product. These activities usually precede development

of the product and continue while the development is in

progress.

It is a QA responsibility to develop and implement

processes and standards to improve the development life

cycle and to make sure that these procedures are

followed. The focus of QA is defect prevention, processes

and continual improvement of these processes.

While QA is a proactive activity, QC is reactive.

Examples of QA activities include establishing standards

and processes, quality audits, selection of tools and

training.

The relationship between QA and QC is depicted in Figure

2 below.

How do test and QA team

responsibilities differ?

Test teams perform test basis document reviews, test

planning, test analysis and design, test validation and

verification, and test reporting through the different test

levels.

In contrast, SQA teams perform the following functions:

Implement organisational quality policies,

standards and processes

Quality Control Testing Quality Assurance

FIGURE 2

Quality Control Testing

FIGURE 1




Assist projects with preparing software quality

assurance or project quality plans

Assure that project processes conform to quality

plans

Conduct regular audits of project products and

processes, and present regular assessments to

senior management

Escalate situations where there are deviations

from guidelines or standards

Assure that:

o Independent reviews are conducted

o Change control procedures for projects are in

place

o Configuration management procedures for

projects are in place

o Procedures are in place for identification and

management of Risks

o There are retrospectives or lessons learned

processes planned and conducted

Provide assurance through the system

development life cycle

Conduct continuous improvements to QA process

and guidelines based on lessons learned

Although these attributes are labelled as QA team

responsibilities, it should be noted that this does not

mean that the QA team will develop these artefacts, but

rather assure they are produced in a manner that is ‘fit for

purpose’.

As part of their assurance role, QA teams may also review

requirement traceability matrices, project document

samples, software configuration management activities,

design, code etc.

How do test and SQA

planning and

documentation differ?

Test teams prepare test strategies and plans based on

test basis documents such as business requirements and

solution design documents. These test planning

documents cover test processes across the various

planned test levels including details such as the test

approach, entry and exit criteria between levels, detailed

test schedules, environment requirements, defect

management, test management and reporting.

In contrast, software quality assurance or quality plans

deal with a broader set of activities across the life cycle.

This ties in with project management methodologies such

as Prince2 that encourage the use of project quality plans

and quality logs that are developed early in the life cycle,

when initiating a project.

A typical Project Quality Plan includes customer quality

expectations, acceptance criteria, quality responsibilities,

planned quality control and audit processes, stage gates,

configuration management plans and change

management procedures. Quality plans for projects use

the organisation’s quality policies, standards or guidelines

as a basis, where they exist.

Monitoring of the Project Quality Plan during the project

is done by continually updating results of planned quality

activities in the Quality Log.

There is an overlap between Risk Management and

Quality Management and therefore, the Risk Register can

play an important input into the preparation of Quality

Plans.

Who should perform the

QA function in an

organisation?

The need for a software quality assurance team grows

with the organisation’s size and its level of quality

policies. Where such a team is required, it is essential that

the QA function remains independent from project and

operational teams. Their reporting line, however, must

provide them with strong support, as and when required.

Project management methodologies, for example, Prince2

are consistent with this aspect of retaining independence

of the quality assurance function. Prince2 recognises

three interests that must be represented on the Project

Board at all times – Business, User and Supplier, and goes

on to recommend project assurance roles that align to

each of these interests. Importantly, Prince 2 recognises

that all of these project assurance roles must be

independent of the project manager. Project Board

members may use members of the independent QA team

to monitor quality aspects of their individual interest.

Some organisations have the QA function embedded

within their enterprise Project Management Office (PMO)

departments. This meets the independence criteria,

however, organisations following this model must ensure

that this team has trained and/or specialised quality

assurance analysts.




Observations and

recommendations for

successful SQA teams

While assuming responsibility for software quality

assurance, I experienced a number of challenges. Some of

these challenges are outlined below together with

learnings and recommendations for organisations

considering creating a SQA team.

Independence

As outlined above, to be successful, SQA teams need to

be independent from project and delivery teams. This

provides the team with the ability to conduct objective

assessment of projects.

A question that arises is whether the Testing and SQA

function should reside in the same team. Although this

may work well in smaller organisations, in my experience,

this has the disadvantage of creating a possible conflict of

interest when monitoring testing activities. It also tends to

emphasise the misconception that testing and SQA are

essentially the same.

An option to resolve this is to embed the SQA function

within the enterprise PMO, or depending upon the size

and quality policies of the organisation, have a separate

team reporting to a senior manager who is responsible

for the function.

Project team relationships

If quality assurance analysts are too process-oriented,

insisting on processes or documentation that may not

add much value, it could strain relationships with project

managers. For example, the level of discipline imposed

on smaller and lower risk projects would be different from

that for complex, higher risk ones. There may also be

differences based on the methodology used, for example,

Waterfall or Agile.

SQA teams would find it much easier to work with project

teams if they keep in mind the ‘fit for purpose’ principle.

Providing guidance and assistance to project teams,

forms a basis for maintaining good relationships, which is

an important aspect of successful QA teams.

Senior management support

I have managed a SQA team that was strongly supported

by senior management and also one that had only

lukewarm support. There was a world of difference

between the two experiences. With the latter, for

example, escalations were not dealt with in time and

problems were carried through the life cycle leading to

production problems and higher than anticipated costs.

In such instances, it is essential that SQA managers have

open discussions with the leadership team.

SQA teams can only ensure adherence to processes and

organisational policies if there is strong and consistent

leadership team support.

Employing the right people

Another ingredient for successful SQA teams is

employing the right staff. People with experience in the

system development life cycle or software engineering

make good candidates for QA roles. Some training in ISO

and/or CMMI principles would supplement knowledge of

someone with a prior development background and keen

interest in quality principles.

Checklists

Standard checklists are a useful mechanism to conduct

reviews or audits of projects, particularly if they are

developed in line with the phases of the development life

cycle. For example, in the Design phase a checklist

question could be ‘Is there traceability between design

and requirement elements?’

To avoid frustration from project managers, I learnt that it

is important to ensure early stakeholder engagement to

gain their feedback when a project is initiated, and when

changes are proposed to checklists.

Communication and reporting

Although, regular reporting to senior management is

important, developing the right templates and metrics

that provide the senior managers with what they require

ensures that these reports are given due consideration.

This is best accomplished by conducting meetings with

relevant senior management providing them with options

and obtaining their feedback.

SQA teams need to continually get approval for changes

to quality processes and standards and ensure effective

communication with stakeholders.

Continuous improvement

Lessons learned from projects provide a SQA team with a

basis for evaluating its quality processes and guidelines,




and incorporating continual improvements. We learned

several lessons that required making changes to our

procedures. This included developing checklists, flexibility,

maintaining good stakeholder relationships and making

improvements to our regular management reports.

Since continuous improvements may also require

changes to the System Development Methodology, it is

recommended that a SQA team maintains an IT

department’s development methodology.

Benefits

A successful SQA team can add considerable value for an

organisation. Some of these benefits include,

Higher quality products

Consistency in processes used for delivery

Continued improvement of organisation

processes

Lower overall costs of delivery

Increased application support documentation

Disadvantages Upfront costs in staffing of quality assurance

analysts

Increased processes that could generate

frustration in some staff

Conclusion

Comparing the differences in activities and responsibilities between Quality Control and Quality Assurance provides us with a

good appreciation of these different terms. QC validates that a specific deliverable meets standards and specifications. In

contrast, QA is a broad function covering proactive planning and monitoring throughout the development life cycle. Testing

on the other hand, forms an integral part of QC. For an organisation to effectively implement Quality Management

processes, these streams must work in tandem. It is recommended that organisations carefully consider the various possible

challenges and have appropriate plans in place prior to embarking on quality management processes.

About the Author

Teji Chopra is a Senior Test Consultant with Planit Software Testing. He has extensive IT

experience in commercial and government sectors that include roles in test management,

software quality assurance and project management

Documents

Software Quality Assurance - PLANIT TESTING · This paper attempts to dispel some common misconceptions regarding the roles of Testing and Software Quality Assurance (SQA) ... related