QQAASSSQA / V & V

May 15, 2009

QQAASSQuestions you have…

• So what is SQA?

• What do I need to know about it?

• Why is it important to me?

• What is V & V and how does it fit with SQA?

• Who will be impacted by the intro of SQA/V&V?

• How will SQA / V&V effect the next release of SDLC, EPIP, ExPIP?

• What is being added to our capital/expense processes that I need to know about?

QQAASSSimple definitions• Quality testing – process of planning and executing

tests to identify defects, failures, and incompatibilities • Quality control – measured degree of conformance to

functional requirements, standards (according to Good Software Engineering Practices – GSEP) e.g. inspections, audits, KPIs)

• Quality Assurance (aka. SQA) – activities planned (process) and conformance

established in (requirements, standards, budget) provide adequate confidence in software development for business solutions.

– it is the implementation of policies and procedures intended to prevent defects from occurring in the first place.

QQAASSQuality Software

Portability

Product

transition

Reusability

Inte

rope

rabi

lity

Correctness

Pro

du

ct

op

erat

ion

Rel

iabi

lity

Eff

icie

ncy

Integrity

Usability

Product revisionMaintainability

Flexibility

Testability

Source: McCall’s factor model tree

QQAASSWhat do I need to know about SQA?

• Brief history of Quality’s history

• The exponential cost of defects

• Everyone has the responsibility for quality

• Measuring quality will help us better manage quality

• Sound requirements are key…we build code and maintain code, when we build requirements, we should maintain requirements

QQAASSHeritage of Software Quality Assurance

• Early 50’s software procured by Dept of Defense (DoD) – Behind schedule– Over budget– Had many technical problems– Software never worked as intended– Projects cancelled after before anything of value could

be delivered– Lacked visibility of software development projects

• From this evolved from base principles of Independent Verification & Validation (IV&V)

QQAASSBirth of IV & V• For Atlas Missile Program

– “independent software tester” was established hired to perform unbiased software testing

– Role of “IV&V contractor” was born– During the 70’s, software development expanded and

companies experienced the same thing the DoD experienced

– Even into the 80’s many projects in development became “disasters”

– Usage today of IV&V contractors is quite comprehensive

– Additionally, from this “software crisis”, the emergence of Software Quality Assurance (SQA) was born

QQAASSSQA, the Early Years• Initially, SQA struggled to show value

– In 1990’s software complexity increased significantly– Competitive business pressures existed– Professionals performing SQA functions lacked formal

training– Many learned SQA practices on-the-job– Learning institutions failed to recognize SQA as a

legitimate discipline

• SQA eventually matured and was recognized and now provides exponential value in today’s complex software development process environment

QQAASSSo, What is SQA?• Pillars of SQA (Quality starts at the beginning)

– Product• Solid requirements

• Project Dev plans and Quality plans

• Configuration management plans

• Software quality metrics and KPIs

– Process• Formal design reviews

• Peer reviews, Inspections and Audits

• Comprehensive Software testing

• Quality training and participation

• Project, process, & quality standards and compliance

QQAASSIEEE/ISO 9000 Definition of SQA

• Software Quality Assurance is: a systematic, planned set of actions necessary to provide adequate confidence that the software development process or the maintenance process of a software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operation within the budgetary confines.

QQAASSWhy is SQA important to me?

• Provides a framework for each participant to be responsible for quality

• Enables empowerment as each participant engages in product delivery

• Shared accountability for meeting our customer expectations and delivering quality solutions.

QQAASS

Software defects identified early in the development lifecycle, translates into a significant cost savings over the duration of the project

Exponential Cost of a Defect

0

2000

4000

6000

Definition Elaboration Implementation Transition Production

Phase

Co

st

Cost

QQAASSNine major causesof software errors

• Faulty requirements definition• Business – IT communication failures• Deliberate deviations from software requirements• Logical design errors• Coding errors• Non-compliance with documentation and coding

instructions• Shortcomings of the testing process• User interface procedure errors• General documentation errors

Source: Software Quality Assurance by Daniel Galin

QQAASSWhere will I see SQA and how will I recognize it?

Analysis• Requirements Development and Management• Change Impact Analysis• Requirements Walk-throughs & Peer Reviews

Design• Requirements Traceability and Repository• Design Document review• Design Walk-throughs & Peer Reviews• BRD & FSD Reconciliation• Functional Decomposition

Coding• Code Walk-throughs, Inspections and Audits• Peer Reviews• Software Configuration Mgmt Plan

Testing• Test Plan Walk-throughs • Causal Analysis• KPIs & Metrics• Separation of QA Testing and UAT testing

QQAASS

It must be understood that quality cannot be the assigned function of any one person or organization; rather, it must be the primary responsibility of every person involved in the development of the product.

QQAASSWhat is V & V and how does it fit with SQA?

• Verification ensures that “you built it right.”

• Validation ensures that “you build what the customer asked for.”

• V & V laid the foundation for SQA

QQAASSBRD’s

FSD’s

DDD’sReviews

Walk-throughsAudits

Integration& SmokeTesting

Functional& Regression

Testing

UAT & BetaTesting

Verification &Validation Model

QQAASSV & V Practices

• Design Reviews – participants are project leads and business sponsors and are charged with formally ensuring the design documents are reviewed and approved before the project development begins

• Peer Reviews: Walk-throughs – informal review of designs and constructed code which includes an overview of the objective and developed artifacts

• Peer Reviews: Inspections – formal review of designs and constructed code with an objective focused on corrective action – contribute more significantly to overall SQA

• Software Audits – evaluation of conformance of software products to standards, guidelines, plans and procedures

• Traceability – mapping of requirements, specifications, and tests to ensure complete coverage

QQAASSMetrics & KPIs• In-process QA Metrics

– M1: Test Execution Progress– M2: Test Execution by Functional Areas– M3: Open Defects in Release by Status– M4: Test Design Coverage by Modules– M5: Code Stability by Builds

• Testing Effectiveness Metrics– M6: Defect Leakage– M7: Defect Removal Efficiency (DRE)– M8: DRE Trend by Releases

• Production Stability Metrics– M9: Production Defects by Severity– M10: Trend of Production Defects (weekly, monthly)

QQAASSSample SQA Dashboard

views

92%

1%7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

QA BETA PROD

7%

93%

Prod

QA

Defect Removal Efficiency - M7

Defect Leakage – M6 Code Stability by Release

0102030405060

Release

Def

ects

Defects addressedin Release Notes

Defects foundin Release

M5

QQAASSBenefits of SQA

• SQA activities provide independent and objective assessments of the processes and quality of the product with time to react.

• The further along a project is within the development lifecycle; the more costly it becomes to resolve defects.

• Requirements management greatly enhances processes, which provide a clear goal of complete and testable requirement specifications.

• Design and code inspections, as well as post mortems, assist in improving process performance and better efficiencies for future projects.

QQAASSCritical Success factors of SQA

• Senior Management support of SQA initiative and process enhancements

• Establish teams’ commitments to follow and comply with SQA process

• Monitor and analyze the SQA process adoption (KPIs, PPQA)

QQAASSQuotes

• “To change the quality of your software, you need to change the process used to create it” – Watts Humphrey

• “We can’t manage what you can’t measure” – W. Edwards Deming

QQAASSIn summary • This presentation was to introduce you to

Software Quality Assurance

• We defined the differences between verification and validation (V & V)

• Highlighted benefits and critical success factors for SQA

• Thank you

• Questions ?