Project Management QA in Brief

8/12/2019 Project Management QA in Brief

1/24

Project ManagementProject Management

Week 9Week 9

Quality AssuranceQuality Assurance

Lecture OverviewLecture Overview

What is Quality Assurance?

Do we need a quality management system forsoftware?

Impact of ISO9000

ISO9001 & ISO9000.3

TickIT

The Capability Maturity Model

ISO9000 Vs CMM

WHAT IS QUALITY ASSURANCE?Quality assurance is the term used to describe those activitieswhich ensure that a contractually acceptable product is deliveredby a project - In other words, a quality piece of software is onewhich, above all e lse, meets the customer's requirements.

Quality assurance includes many of the good practices whichwe have already described, such as holding reviews, producing aproject plan and documenting activities. It is important to pointout that quality assurance is as much concerned with theprocesses used to create a software system as the system itself.



2/24

The production of almost any piece of software requires a

certain amount of communication between people, from thesmallest program to the largest system.

For example, imagine how much communication would beinvolved when a company's entire finance department andboard of directors have to communicate their ideas to a softwaredeveloper's project team consisting of maybe 15, 20 or evenmore people - Not only is there communication between thecustomer's staff and the developer's, but there is a great deal ofcommunication needed within the developer's organisationbetween members of the project team.


In a high-quality product, it is imperative that communication iseffective, or the developer could end up producing a product thecustomer does not want, or different members of the project teamcould end up producing incompatible program units!

One of the aims of quality assurance, therefore, is to ensure thatthis communication is effective, i.e. to ensure that the developer, inparticular the developer's project team, has understood correctly thecustomer's requirements.

High-quality software is characterised by many other attributeswhich are not directly related to the customer's requirements, suchas:


Efficiency

Reliability

Testability

Maintainability

Usability



3/24

Efficiency - refers to the behaviour of a system inrelation to the resources of the computer system onwhich it executes. Most often the efficiency of a systemis described in terms of its execution speed and storageuse. If a system executes as fast as was specified butconsumes twice as much storage as was expected,maybe in order to achieve the performance requirement,it is likely to incur the customer's displeasure.


Reliability - relates to the number of errors in a pieceof software, and hence is a measure of the number oftimes a software system fails to perform correctly - Onemight expect a system containing 10 errors to meet acustomer's requirements more frequently than onecontaining 100. (However, this might not be the case;

just one of the 10 may be more serious than all of the100.)


Testability - refers to the ease with which a softwaresystem can be tested - For example, if a system containsprogram units with a large amount of tortuous logic thenit will be difficult to test; if it is difficult to test itsdesign or implementation may be poor and it may beless likely to meet customer requirements.



4/24

Maintainability - refers to the ease with which asystem can be changed once it is in operation - Forexample, it should be straightforward to replace afunction in a system if its system design exhibits loosecoupling and high cohesion.


Usability - is the ease with which the system canbe used; if a system is easy to use, if it has aconsistent and clear model of how it should be used,then it is less likely to be misused.

This characteristic has much to do with thehuman-computer interface which is a topic in itself.

It is not just a matter of aesthetics or taste sincemisuse could result in financial loss, environmentaldamage, or loss of life.


Quality Management SystemsQuality Management SystemsFor SoftwareFor Software

Quality System : the organisational structure,

responsibilities, procedures, processes and resources forimplementing quality management;

Quality management : that aspect of the overallmanagement function that determines and implementsthe quality policy.

Quality : the totality of features and characteristics ofa product or service that bear on its ability to satisfystated and implied needs.


5/24


A quality system is the organisational structure and soon that controls and influences the quality of a suppliersproducts and services.

Quality is what makes your customer happy - virtuallyeverything in a software development organisationinfluences quality, so in practice the quality system isthe means for managing the software development.


Some examples of what may be parts of a quality systemfor software:

Schedule and agenda for executive meetings;Assignments of authorities and responsibilities in the

company;Procedures for project management / Templates for

documents;Procedures for reviews and tests / Procedures for handling

customer complaints / Records of employee training;

Procedures for internal audits;Procedures for handling changes to specifications andprogram / The central product library for software.


The basic requirement of a quality system is that itworks.

If an eternal editor can see that your quality systemworks, he or she would probably only be able to findminor things to criticise.


6/24

Do We Need A Quality ManagementDo We Need A Quality ManagementSystems For SoftwareSystems For Software

So why bother with a quality system? Well, there are a

few possible reasons:You may want to modify the software;Your customers may require that your software works;You may have to convince your customer beforehand

that you will be able to deliver a suitable product;You may have to hire p rogrammers who have families

and private lives, and who are not prepared to work dayand night for three years;

The original whiz kids may quit and start a business oftheir own;

You may even have product liabilities.

If you want to know what you'll get and when youget it, you had better have a quality system of somesort.

An intelligent application of the requirements in ISO9001 will make a software supplier better, theoperative word here being intelligent.

Many of the principles of quality management canbe usefully applied to software development, providedthe particular features of software quality problems areborne in mind.


The problems of software are not unique. Userrequirements are often highlighted as the worst problemarea.

Juran highlighted this area in manufacturing 40 yearsago - complexity requires careful management in allcontexts, but software cannot claim a monopoly here asquality problems of software development represent aparticular blend of problems, rather than somethingcompletely different.



7/24

It is suggested that there are four principal aspects to a QMS

for software development:Development procedures.This includes the use of design and developmentmethodologies and tools, testing and associated stafftraining.

Quality control.This includes many activities for the monitoring of qualityduring development, e.g. planning, progress meetings, usersign-off, configuration management, change control,documentation control, design reviews, code walkthroughs,error reporting, system testing and acceptance testing.


Quality improvement.This includes all activities aimed at establishing ahuman quality culture amongst the staff, such asquality improvement teams, quality circles and soon.

Quality assurance.Where a quality system is in place, QA becomes themonitoring of the system itself to ensure that it is

being carried out correctly.


A quality system is designed to move maintenance toearlier in the lifecycle - this will lead to a reduction in bothtime and effort.

All these benefits must be shown to happen in practice -Once quantified in financial terms, the benefits must beweighed against the cost, which may be considered in twostages.

First there is the cost of introducing a qualitymanagement system - Once established there are specificcosts associated with certification.



8/24

In 1988 the cost of introducing a QMS to a typicalsupplier (a company with 50-100 employees andturnover of 3,000,000) was estimated at 230,000 ayear - In addition, set-up costs were estimated at120,000.


Standards are generally defined in terms of a model of bestpractice, against which all others may be compared.

The role of standards is not to build the proverbial bettermousetrap, but to ensure conformance to the standard.

The standard never improves quality directly nor ensuresperfection - It should, however, ensure that the correct proceduresare in place and being carried out.

The standard provides a model, and the accreditation (audit)procedure the incentive to ensure that things are done directly.


An audit is an evaluation of your quality system anddocumentation - Your organisation may undergo several typesof audits:

First-party auditsSecond-party auditsThird-party audits

The first-party audit is an internal quality system auditperformed by the supplier (your organisation) on its own qualitysystem.

The second-party audit is a qua lity system audit performed byyour customer on the (supplier & your organisation).

Quality AuditsQuality Audits


9/24

The third-party audit is a quality system audit

performed by an auditor on the supplier in order toachieve certification for one of the ISO 9000 Standards -The third-party auditor must be independent of both thecustomer and the supplier - as Third-party audits cannotbe performed by the customer or the supplier.



What does the auditor look for when performing a third-party audit?

Do your documents conformto the requirements of the Standard?

Do your operations conform to the documents?

Do your records showpast conformanceto your documents?

Standard

On-goingoperation

Qsuality

records

ControlledDocuments

The accreditation (audit) process provides the number of

potential benefits to the supplier:it provides external validation to see whether the

investment made in the QMS is being effective;it gives the supplier and their quality system

external credibility;it allows the supplier to sell to those customers

who insist on accreditation as a condition of tender;it qualifies the supplier to be included in buyers

guides compiled by the accreditation bodies andcirculated to potential customers.



10/24

The cost of creating a satisfactory QMS to one of theISO 9000 series standards is small in relation to the costof setting up the QMS in the first place. The figures fora typical supplier in 1988 were estimated at 10,500initially, with a further annual cost of 4,500.

The advantage of third -party over second-partyaccreditation (audits) is that the supplier only has tosatisfy one accreditor/auditor (rather than, say, to have to

justify ones quality practices to six different customers).

The Impact of ISO9000The Impact of ISO9000

The ISO 9000 standard with which we are concernedis ISO 9001, since it applies to quality assurance indesign, development, production, installation andservicing.

This standard was written for the manufacturingindustry, and this poses some problems when applying itto the development and maintenance of software.

ISO 9001 and ISO 9000.3ISO 9001 and ISO 9000.3

In manufacturing (for example, kettles), design is a

relatively minor activity - Instead, the cost for eachmanufactured item is notable, so when a few items have beenproduced, production is by far the major part of the activity.

Therefore, when we talk about quality or productivityproblems and improvements in manufacturing, we tend to focuson production.

Software development, however, is nearly 100% design.Production means to copy executable code to diskettes, tapes, orROMS, and is performed and checked automatically. So, whentalking about quality & productivity, we focus on design.



11/24

The functionality and complexity of software and complexelectronics are many orders of magnitude greater than thoseof ordinary appliances.

Thus the need for control is greater in softwaredevelopment than in those of other appliances; at the sametime, that control is more difficult to define and apply.

ISO 9001 covers design, but it focuses on production - evenfor a production expert the text in the standard is brief andneeds explanation - to apply it to software development thestandard must be interpreted and exp lained still further.


In 1991 ISO published a guide for this purpose ISO9000.3, Quality Management Systems Part 3 Guidelinefor the application of ISO 9001 to the development, supplyand maintenance of software.

The guideline is organised into three main groups:General requirements on the company and its

management;Requirements on projects and the maintenance

phase;Requirements on supporting activities (those

activities independent of phases);Software engineers complain that ISO does not tell us how

to develop quality software. - It was never intended to.


The standard is aimed solely at being a tool for thecustomer.

Basically ISO 9001 makes the supplier implement thebasic management of software development, and thestandard then enforces visibility, so that the customercan see what the developers are doing and judge it.

In practice, ISO9001 and 9000.3 can be used as guidesfor the suppliers management, helping them to controldevelopment and gain insight into what is really goingon.



12/24

By the end of the 80s the ISO standards had becomequite popular in Europe - Manufacturers were certified to theISO standards in increasing numbers.

Some had considerable computer departments developingsoftware for internal use and the certification of thesedepartments varied depending on the auditors competenceand the attitude of the certification bod y.

Companies with software as part of their products startedto join in, and soon pure software houses joined in. Industrywas becoming apprehensive about ISO 9001 certification ofsoftware development and maintenance.


It was feared that different certificates might havevery different values that the standard was non-standard.

The British software industry, together with theBritish Department of Trade and Industry, launched aninitiative to amend the situation and called it TickIT .

The goal was to establish effective and unifiedcertification of software development and maintenance.


TickIT is a system for certifying software development

organisations to ISO 9001.It comprises of 6 elements:An interpretation of ISO 9001 for software;A standard set of requirements on the competence and

behaviour of certification auditors;A standardised training course for certification auditors;A registration scheme for certified auditors;A system for accrediting certification bodies for

conducting TickIT certifications;A logo to be used on certificates to show TickIT

certification.

TickITTickIT


13/24

The British National Accreditation Council forCertification Bodies is the only national authority issuingaccreditation to certification bodies.

Even companies in Sweden and the USA areaccredited by NACCB.

TickITTickIT

What's really special about TickIT certification is theauditors.

TickIT auditors are registered by the InternationalRegister of Certificated Auditors (IRCA) in London.TickIT auditors come in three levels:

Provisional 'TickIT Auditor,Senior TickIT Auditor,Lead TickIT Auditor.

TickIT AuditorsTickIT Auditors

In order to become one, you have to fulfil several

requirements:You must yourself have worked for at least three years insoftware development, including all different types of work.

You must have successfully concluded an approved one-week TickIT auditor's course ending with a formalexamination.

You must have experience as a manager.To become Senior or Lead TickIT auditor, you must have

experience in conducting and leading, respectively, TickITcertifications.

There are further requirements regarding your personalattributes.



14/24

IRCA shows 0 signs of taking the requirements for

TickIT auditors seriously.In 1993, it was reported that 15% of the applicants for

registration as TickiT auditors were not called for aninterview, and of those interviewed, 25% failed.

All this means is that when you apply for TickITcertification you know that your software developmentand maintenance procedures will be judged by well-trained auditors with personal experience in softwaredevelopment.


The certificate you just received and put up on theboardroom wall is valid for a certain period of time,usually three years.

After that time, you will have to apply for certificationand do it all over again - Hopefully, the process will bemuch simpler this time.

So when you have received your certificate theauditors will be back soon - In your contract with thecertification body, it is stated that they will do regularaudits twice a year during the time of the certificate.

Maintaining The CertificateMaintaining The Certificate

Often an organisation is kept on its toes un til thecertification is done, but then the reaction comes, and thedisciplined of working is replaced with the usual sloppiness.

This is the reason why the certification body will be back tocheck that your company is not one of these organisations.

These regular surveillance audits are less comprehensivethan the certification audit, and usually the auditorsconcentrate on different areas of your quality system eachtime.



15/24

During the term of the certificate, you can still lose it- In some situations, the certification body has theobligation or authority to withdraw your certificate.

Examples include:If you fail to correct within the prescribed time

a non-conformance found in a surveillance audit;If you use the certificate improperly in your

marketing (e.g. indicating that this is a certificateof the quality of your products);

If you dont pay the bills from the certificationbody.


Summary of ISO 9000:It is concerned with what you do in software

development, not how you do it;It provides some d irection, however, not necessarily

sufficient direction;It is predominantly a tool for software buyers, not

builders.


One of the most promising current methods ofmanaging the development of high quality softwaresystems can be found in the work of the AmericanSoftware Engineering Institute (SEI) at Carnegie MellonUniversity;

The SEI has worked for a number of years to set astandard for managing American software developmentfrom all over the world to American industry.

The Capability Maturity ModelThe Capability Maturity Model


16/24

The Capability Maturity Model(CMM) provides agraduated set of software development process goals, ascale for assessing the maturity levels of existingsoftware processes, and a programme to drive processimprovement.

The underlying principles of the CMM stand behindall current approaches to software development processassessment and improvement, especially ISO.


The Capability Maturity Model for SoftwareThe Capability Maturity Model for Software describes the

principles and practices underlying software process maturityand is intended to help software organizations improve thematurity of their software processes in terms of an evolutionarypath from ad hoc, chaotic processes to mature, disciplinedsoftware processes.

The CMM is organized into five maturity levels - A maturity

level is a well-defined evolutionary plateau toward achieving amature software process. Each maturity level provides a layer inthe foundation for continuous process improvement.


The Five Maturity LevelsThe following characterizations of the five maturity levelshighlight the primary process changes made at each level:1) Initial The software process is characterized as ad hoc, andoccasionally even chaotic. Few processes are defined, andsuccess depends on individual effort and heroics.2) Repeatable Basic project management processes areestablished to track cost, schedule, and functionality. Thenecessary process discipline is in place to repeat earliersuccesses on projects with similar applications.



17/24


18/24

Software Project Tracking and Oversight (PT)Establish adequate visibility into actual progress so thatmanagement can take effective actions when thesoftware project's performance deviates significantlyfrom the software plans.

Software Subcontract Management (SM)Select qualified software subcontractors and managethem effectively.


Software Quality Assurance (QA)Provide management with appropriate visibility into theprocess being used by the software project and of the productsbeing built.

Software Configuration Management (CM)Establish and maintain the integrity of the products of thesoftware project throughout the project's software life cycle.

The key process areas at level 3 address both project andorganizational issues, as the organization establishes aninfrastructure that institutionalizes effective softwareengineering and management processes across all projects, assummarized below:


The key process areas at level 3 address both projectand organizational issues, as the organization establishesan infrastructure that institutionalizes effective softwareengineering and management processes across allprojects, as summarized below:



19/24

Organization Process Focus (PF)Establish the organizational responsibility for softwareprocess activities that improve the organization's overallsoftware process capability.

Organization Process Definition (PD)Develop and maintain a usable set of software processassets that improve process performance across theprojects and provide a basis for cumulative, long-termbenefits to the organization.


Training Program (TP)Develop the skills and knowledge of individuals so theycan perform their roles effectively and efficiently.

Integrated Software Management (IM)Integrate the software engineering and managementactivities into a coherent, defined software process that istailored from the organization's standard softwareprocess and related process assets.


Software Product Engineering (PE)

Consistently perform a well-defined engineering processthat integrates all the software engineering activities toproduce correct, consistent software products effectivelyand efficiently.

Inter-group Coordination (IC)Establish a means for the software engineering group toparticipate actively with the other engineering groups sothe project is better able to satisfy the customer's needseffectively and efficiently.



20/24

Peer Reviews (PR)Remove defects from the software work products earlyand efficiently- An important corollary effect is todevelop a better understanding of the software workproducts and of the defects that can be prevented.

The key process areas at level 4 focus on establishinga quantitative understanding of both the softwareprocess and the software work products being built, assummarized below:


Quantitative Process Management (QP)Control the process performance of the software projectquantitatively.

Software Quality Management (QM)Develop a quantitative understanding of the quality of the project'ssoftware products and achieve specific quality goals.

The key process areas at level 5 cover the issues that both theorganization and the projects must address to implement

continuous and measurable software process improvement, assummarized below:


Defect Prevention (DP)

Identify the causes of defects and prevent them from recurring.

Technology Change Management (TM)Identify beneficial new technologies (i.e., tools, methods, andprocesses) and transfer them into the organization in an orderlymanner.

Process Change Management (PC)Continually improve the software processes used in theorganization with the intent of improving software quality,increasing productivity, decreasing the cycle time for PD.



21/24

Common FeaturesFor convenience, each of the key process areas is

organized by common features - The common featuresare attributes that indicate whether the implementationand institutionalization of a key process area iseffective, repeatable, and lasting.

The five common features, followed by their two-letter abbreviations, are listed below:


Common FeaturesCommitment to Perform (CO)

Describes the actions the organization must take toensure that the process is established and will endure -Includes practices on policy and leadership.

Ability to Perform (AB)Describes the preconditions that must exist in theproject or organization to implement the software

process competently - Includes practices on resources,organizational structure, training, and tools.


Common Features Activities Performed (AC)

Describes the roles and procedures necessary toimplement a key process area. Includes practices onplans, procedures, work performed, tracking, andcorrective action.

Measurement and Analysis (ME)Describes the need to measure the process and analyzethe measurements. Includes examples of measurements.



22/24

Common FeaturesVerifying Implementation (VE)

Describes the steps to ensure that the activities areperformed in compliance with the process that hasbeen established. Includes practices on managementreviews and audits.


Clearly there is a strong correlation between ISO 9001and the CMM, although some issues in ISO 9001 are notcovered in the CMM, and some issues in the CMM arenot addressed in ISO 9001.

The levels of detail differ significantly: chapter 4 inISO 9001 is about five pages long; sections 5, 6, and 7 inISO 9000-3 comprise about 11 pages; and the CMM isover 500 pages long.

There is some judgment involved in deciding the

exact correspondence, given the different levels ofabstraction.

ISO 9001 Vs CMMISO 9001 Vs CMM

The clauses in ISO 9001 with no strong relationships

to the CMM key process areas, and which are not welladdressed in the CMM, are purchaser-supplied product(4.7) and handling, storage, packaging and delivery(4.15).

The clause in ISO 9001 that is addressed in the CMMin a completely distributed fashion is servicing (4.19) -The clauses in ISO 9001 for which the exactrelationship to the CMM is subject to significant debateare corrective action (4.14) and statistical techniques(4.20).



23/24

The biggest difference, however, between these twodocuments is the emphasis of the CMM on continuousprocess improvement. ISO 9001 addresses the minimumcriteria for an acceptable quality system.

It should also be noted that the CMM focuses strictlyon software, while ISO 9001 has a much broader scope:hardware, software, processed materials, and services[Marquardt91].


The biggest similarity is that for both the CMM andISO 9001, the bottom line is Say what you do; do whatyou say. The fundamental premise of ISO 9001 is thatevery important process should be documented andevery deliverable should have its quality checkedthrough a quality control activity.

ISO 9001 requires documentation that containsinstructions or guidance on what should be done or howit should be done.


The CMM shares this emphasis on processes that are

documented and practiced as documented - Phrases suchas conducted according to a documented procedureand following a written organizational policycharacterize the key process areas in the CMM.

The CMM also emphasizes the need to recordinformation for later use in the process and forimprovement of the process - This is equivalent to thequality records of ISO 9001 that document whether ornot the required quality is achieved and whether or notthe quality system operates effectively.



24/24

This statement is controversial in itself. Some

members of the international standards communitymaintain that if you read ISO 9001 with insight (betweenthe lines so to speak), it does address continuous processimprovement.

There is faith that weaknesses will improve over time,especially given regular surveillance audits - Correctiveaction can be interpreted in this way, although that maynot be consistently done today - This will undoubtedlybe one of the major topics for the next revisioncycle for ISO 9001.


Lecture OverviewLecture Overview

What is Quality Assurance?

Do we need a quality management system forsoftware?

Impact of ISO9000

ISO9001 & ISO9000.3

TickIT

The Capability Maturity Model

ISO9000 Vs CMM