Quality Management Systems: Efficiency Improvement in Software Development Karl Heinrich Möller

CMMMö/3.6.2002

Slide 1Efficiency Improvement of Software Development

ICEL

Quality Management Systems:Efficiency Improvement

in Software Development

Karl Heinrich MöllerGaertnerstr. 29

D-82194 GroebenzellTel: +49(8142)570144Fax: +49(8142)570145

Email: [email protected]

CMMMö/3.6.2002


ICEL

Contents

• Principles• Capability Maturity Model (CMM)

– Overview– Questions– Results

• Improvement Process– People within the improvement process– Organising the improvement process (example)– Charter of the groups engaged– Piloting the new methods– Application of the new methods

• Experience• Rules

CMMMö/3.6.2002


ICEL

PrinciplesVision

quality

productivity elapsed time

• derive objectives of the organisation• objectives achieved?

• who sets the objectives ?• who controls the achievement?• how is the achievement controlled?

CMMMö/3.6.2002


ICEL

org.-level 1org.-level 3


employeeemployeeemployeeemployee

PrinciplesVision

organisation to be improved

employee

org.-level 1org.-level 1org.-level 1Change Agent

Change Agentchange agent

improvement project

project leader

sponsor

externalsupport

org.-level 4

CMMMö/3.6.2002


ICEL

PrinciplesVision

• PlanPlan the method

• DoIntroduce the method

• CheckCheck whether the goal has been achieved

• ActImprove the method

Plan

Act

Check

Do

CMMMö/3.6.2002


ICEL

Initial

Repeatable

Defined

Managed

Optimizing

Process is informal andad hoc

Project managementpractices areinstitutionalized

Technical practices areintegrated withmanagement practicesand institutionalized

Product and process arequantitatively controlled

Process improvementis institutionalized

Level Process Characteristics

Risk

Quality

Productivity

Results

Capability Maturity Model (CMM)

CMMMö/3.6.2002


ICEL

Maturity Level 1

• what has been achieved:every individual does its best to perform the work

• what is missing:cooperation of the individuals only if they believe there is a need

CMMMö/3.6.2002


ICEL

Maturity Level 2

• what has been achieved :the individuals of every project are organising their work to achieve the objectives of the project as far as they are understood

• what is missing:cooperation of the projects only if they believe there is a need (every project e.g. has its own configuration management system); processes are not efficient

CMMMö/3.6.2002


ICEL

Maturity Level 3

• what has been achieved :organisation has a defined development process valid for all projects, which is to be followed by everybody; process is optimised by qualitative criteria

• what is missing:no quantitative criteria (faults are found too late, fault prevention insufficient, many changes in late phases etc.)

CMMMö/3.6.2002


ICEL

Maturity Level 4

• what has been achieved :process is controlled and continuously improved by quantitative criteria

• what is missing:process improvements are endured and not created; introducing new methods is insufficiently planned and controlled

CMMMö/3.6.2002


ICEL

Maturity Level 5

• what has been achieved :introducing new methods is planned and controlled

• what is missing:nothing

CMMMö/3.6.2002


ICELSoftware Process AssessmentStructure of Questions (Clusters / Process Criteria)

Process Criteria

1.Organisation and

Resources

2.Process oriented

Functions

3. Life Cycle independent

Functions

4. Life Cycle Functions

1.1Organisationsal

structure

1.2

Training

1.3

Communication

1.4

Technology

2.1 ProcessDefinition

2.2 Processmeasurement

2.3 ProcessImprovement

3.1Project Planning

3.2

Project Control

3.4 ConfigurationManagement

3.3 Qualityassurance

3.5 SubsupplierManagement

3.6 Risk-Management

3.7 QualityManagement

4.6 Acceptance

4.7 Application andMaintenance

4.5 Integration andSystem Test

4.4 Implementation

4.3Design

4.2 RequirementManagement

Spec. SystemProperties

4.1

CMMMö/3.6.2002


ICELDevelopment Process: Conformance Diagram Site Assess. (I)

1 2 3 4 5

Organization

Training

Communication

Technologies Management

Process Definition

Process Measurement

Process Improvement

Project Planning

Project Tracing

Quality Assurance

Configuration Mgmt.

Contractor Mgmt.

Risk Management

Quality Management

CMMMö/3.6.2002


ICEL

Development Process: Conformance Diagram Site Assess. (II)

1 2 3 4 5

Customer Requirements

SW Requirements

Architecture Design

Detailed Design

Implementation

Integration

System Test

Acceptance Test

Deployment +Maintenance

CMMMö/3.6.2002


ICEL

Development Process: Conformance Diagram Project Assess. 11 2 3 4 5

Organization

Training

Communication

Process Definition

Process Measurement

Process Improvement

Project Planning

Project Tracing

Quality Assurance

Configuration Mgmt.

Contractor Mgmt.

Risk Management

Quality Management

Customer Requirements

SW Requirements

Architecture Design

Detailed Design

Implementation

Integration

System Test

Deployment + Maintenance

CMMMö/3.6.2002


ICEL

Improvement Process 1From Analysis to an Improved Status

Analysis Design Imple-mentation Test Use

identify 3 to 5 processes with the highest potential for improvement2

1

34

5

initial

repeatable

defined

managed

optimizing

method 1implementing




method 1test

method 2test

method 3test

method 1use

CMMMö/3.6.2002


ICEL

Improvement Process 2 From Analysis to an Improved Status

Imple-mentation

Test Use



method 2test

method 3test


method 4test

method 4use


method 1test

method 1use

Check

Act

CMMMö/3.6.2002


ICEL

Roles within the Improvement Process

• SponsorHead of organisation

• Coachmembers of (another) organisation who make the change happen (full-time)

• Change Agentmembers of organisation who make the change happen (part-time)

• Championmembers of organisation who support the change

• Targetmembers of organisation whose attitudes have to be changed

CMMMö/3.6.2002


ICEL

Software Engineering Process Group Example

S ub-T eam 11R ev iew

S ub-T eam 12T es t

S ub-T eam 13D PP

Proc es s T eam 1D efec t E lim inat ion P roc es s

S ub-T eam 21A naly s is

S ub-T eam 22D es ign

S ub-T eam 23P roduc t P lann ing

P roc es s T eam 2Produc t C rea t ion P roc es s

S ub-T eam 31L ib P roduc t ion

S ub-T eam 32 P roduc t ion

S ub-T eam 33Q M -M anagem ent

P roc es s T eam 3 P roc es s C ont ro l

Sub-T eam 41M etric s

Sub-T eam 42C os t C ont ro l

P roc es s T eam 4P roc es s O rgan iz a t ion

SE P G

CMMMö/3.6.2002


ICEL

CharterSoftware Engineering Process Group SEPG

• Led by project leader• Supported by sponsor• Consisting of leaders of process teams• Controls improvement process

– Distribution of resources– Members (especially sponsor) stand for the project– Help in case of trouble– Decides on introduction of new methods

CMMMö/3.6.2002


ICEL

Charter Process Team

• Led by a member of SEPG• Controls sub teams• Consists of the members of the sub teams• Godfather supports introduction of new

methods

CMMMö/3.6.2002


ICEL

CharterSub Team (Technical Experts Group)

• Opinion leaders of the organisation• Designs, implements, tests, and introduces

new methods• Members become coaches and change agents

CMMMö/3.6.2002


ICEL

changeagent

Introduction of New Method 1Overview

technical experts group (subgroup) defines the new method

pilot 1 pilot 2

time

group a group b group c

Piloting

Introductionstep 1

Introductionstep 2

coach

proc

ess

team

con

trols

pilo

ting

and

intro

duct

ion

group d group e group f

CMMMö/3.6.2002


ICEL




Introduction of New Method 2Technical experts team (Sub Team) designs it


employee



improvement project

project leader

sponsor

org.-level 4

CMMMö/3.6.2002


ICEL




Introduction of New Method 3Piloting


employee



improvement project

project leader

sponsor

org.-level 4 actions:part 1Sub team gets pilot to support new methodpart 2pilot has been convinced that new method makes work more easy (and is of advantage to the organisation)

CMMMö/3.6.2002


ICEL

PilotingSteps

• Kick off meeting to join in members of the pilot• Members of pilot get every support• Measure degree of application and effectiveness• Final report together with the members of pilot

– Method makes work easy– Is really applied– Has advantage for the organisation

CMMMö/3.6.2002


ICEL

org.-level 3

org.-level 2

Introduction of New Method 4Convincing Management


employee

org.-level 1Change Agent


improvement project

project leader

sponsor

org.-level 4

action:improvement project convinces every level of management (especially middle management) of the advantages of new method

CMMMö/3.6.2002


ICEL

Rules for Cooperation 1Understanding the Improvement Project

• Without change of mental attitudes there is no acceptance of new methods

• Pressure is a bad means to enforce mental change

• To simply wait for understanding will not lead to a timely change

• To make a change happen real hard work with the people to be convinced is necessary

• Change will only occur with the majority of people in the organisation being in favour of change

CMMMö/3.6.2002


ICEL

Rules for Cooperation 2Rules of Enforcement

• Arguing until organisation knows nothing against• Every action must succeed the first time

(estimation of resistance must be accurate)– Coach or change agent is able to do it– Needs support from project leader or department head (should not

be done often) – Needs support of the sponsor (should not be done directly)

• Enforcement in the beginning must be superseded by trustful cooperation

CMMMö/3.6.2002


ICEL

Defect Detection RatesDesignRequire-

mentsLevel

Managed4

Defined3

Repeatable2

5% 10% 50% 20% 10% <5%

0% 0% 5% 15% 50% 30%

3% 5% 7% 25% 50% 10%

Cost per defect in DM

Average costper defect

3.700 DM

9.600 DM

12.900 DM

Imple-menting

Modul Test

Integr. Syst.-Test

Use

500 500 500 6.000 12.000 20.000

CMMMö/3.6.2002


ICEL

Potential of ImprovementExample for Fault Cost Reduction

Defect cost are a certain fraction of total development cost

There is an exponential increase in cost if time between seeding and detection becomes longer:

- detection of defects as early as possible- prevent defect - reduce number of undetected defects in

each phase of development process

Potentials for cost reduction:level 1 level 2 : 17 % of defect costlevel 2 level 3 : 22 % of defect costlevel 3 level 4 : 19 % of defect costlevel 4 level 5 : 44 % of defect cost

Defectcreation

rate

Defect detection

rate

Reaquire-ments Design Coding Functional

TestSystem

TestUse

Cost of defects(rates)

5

4

3

2

1

10%40% 50%

5% 20%40%

20%10% <5%

3% 5% 7%60%

20%5%

0% 2% 3% 55%32% 8%

0% 0% 3% 30%50%

17%

33%50%15%2%0%0%

1 11 1 6 1220

CMMMö/3.6.2002


ICELRelation of Maturity Level and Process CharacteristicsProduct Size: 500 KLOC

Highest gains for field defects and cost(Reported by Lockheed)

Initial

2

1

3

4

5

Repeatable

Defined

Managed

Optimizing

Elapsed Time[Time]

Field Defects[Defects/kLOC]

Productivity[LOC/hr]

ex-tra-po-la-tion

Cost[Cost/Project]

1%

3%

11%

22%

100%

9%

17%

33%

66%

100%

55%

65%

75%

85%

100%

100%

70%

40%

27%

20%

0 50 1000 50 1000 50 100 0 50 100

CMMMö/3.6.2002


ICEL

General experiences Effort and Elapsed Time

• 2 to 3 years are needed to improve projects by 1 maturity level

• 2 to 3% of development cost is a typical effort for process maturity of an organisation that

– is at SEI-Level 2 to 3 and– is insufficient to improve SEI-Maturity

• Up to 5% is needed to improve the maturity level

CMMMö/3.6.2002


ICEL

Gains for Improving from Maturity Level 2.5 to 3.5 Quality, Productivity and Elapsed Time

• Productivity is increased by about 50%• Elapsed time is reduced by about 15%• Field defect rate is cut by half

• Compared to 3 to 5% of effort for the process improvement

Documents

Quality Management Systems: Efficiency Improvement in Software Development Karl Heinrich Möller