Software Measurement & Metrics

2

A Quote on Measurement

“When you can measure what you are speaking about and express it innumbers, you know something about it; but when you cannot measure,when you cannot express it in numbers, your knowledge is of a meagerand unsatisfactory kind; it may be the beginning of knowledge, but youhave scarcely, in your thoughts, advanced to the stage of science.”

LORD WILLIAM KELVIN (1824 – 1907)

3

Reasons to MeasureTo characterize in order to

Gain an understanding of processes, products, resources, and environmentsEstablish baselines for comparisons with future assessments

To evaluate in order toDetermine status with respect to plans

To predict in order toGain understanding of relationships among processes and productsBuild models of these relationships

To improve in order toIdentify roadblocks, root causes, inefficiencies, and other opportunities for improving product quality and process performance

4

Categories of Software MeasurementTwo categories of software measurement

Direct measures• Software process (cost, effort, etc.)• Software product (lines of code produced, execution

time, defects reported over time, etc.)

Indirect measures• Software product (functionality, quality, complexity,

efficiency, reliability, maintainability, etc.)

Project metrics can be consolidated to create process metrics for an organization

5

Establishing a Metrics Baseline

By establishing a metrics baseline, benefits can be obtained at the software process, product, and project levelsBaseline data must have the following attributes

Data must be reasonably accurate (guesses should be avoided)Data should be collected for as many projects as possibleMeasures must be consistent (e.g., a line of code must be interpreted consistently across all projects)Past applications should be similar to the work that is to be estimated

6

Software Metrics Baseline Process

SoftwareEngineeringProcess

SoftwareProject

SoftwareProduct

Data Collection

MetricsComputation

MetricsEvaluation

Measures

Metrics

Indicators

Product Metrics

Ensuring the Quality of Product• Size• Complexity• Structure & Architecture• Product Quality• Defects

Product MetricsSize

number of elements • lines of code • number of documentation pages, • number of test cases • etc

development metrics • time metrics • cost metrics • consumption of resources metrics

size of components • number of modules/objects • average size of components

Defects MetricsDefects per unit sizeTypes/LevelsStages Introduced/FixedDefect Discovery RateDefect Removal EfficiencyResultsCosts/Efforts for finding& fixing

10

Defect Removal Efficiency

Defect removal efficiency provides benefits at both the project and process levelDRE = E / (E + D)The ideal value of DRE is 1, which means no defects are found after deliveryDRE encourages a software team to institute techniques for finding as many errors as possible before delivery

1

ii

ii EE

EDRE

Failure Intensity Setting FI in terms of reliability

is failure intensity

R is reliability

t is natural unit (time, etc.)

95.0

1ln

Rfor

t

Ror

t

R

Reliability and Failure IntensityReliability for 1 hour

mission timeFailure intensity

0.36800 1 failure / hour

0.90000 105 failure / 1000 hours

0.95900 1 failure / day

0.99000 10 failure / 1000 hours

0.99400 1 failure / week

0.99860 1 failure / month

0.99900 1 failure / 1000 hours

0.99989 1 failure / year

MTTF and Failure Intensity

MTTF MTTFA

MTTF MTTR MTBF

MTTF

1

System Reliability A system usually consists of components.

Components may have

Different reliability

Different dependencies among each other

System reliability is a function of the reliabilities of the (sub-) components and of the relationships between the components.

Serial System Reliability System is composed of n independent serially

connected components.

Failure of any component has a cross system effect, i.e., results in failure of the whole system.

A serial system has always smaller reliability than its components.

R1 R2 ...

The system is composed of 4 independent serially connected components

R1 = 0.95

R2 = 0.87

R3 = 0.82

R4 = 0.73

Rsystem = 0.95 0.87 0.82 0.73 = 0.4947

Serial system reliability is smaller than any individual reliability of the components

Parallel System Reliability

System is composed of n independent components connected in parallel. Failure of all components results in the failure of the whole system (principle of active redundancy).

R1

R2

...

The system is composed of 4 independent components connected in parallel

R1 = 0.95

R2 = 0.87

R3 = 0.82

R4 = 0.73

Rsystem = 1 – ((1 – 0.95) (1 – 0.87)

(1 – 0.82) (1 – 0.73)) = 0.9996

Parallel system reliability is greater than any individual reliability of the components

Metrics for Specification Quality

Requirements in a specification

Specificity (lack of ambiguity)

nnn nffr

r

ui

nnQ

Architectural Design Metrics

For hierarchical architectures (e.g. call and return architectures)Structural complexity of a module i is defined as,

S (i) = f2 out (i)

Architectural Design Metrics

Data complexity provides an indication of the complexity in the internal interface for a module i and is defined as,

where v(i) is the number of input and output variables that are passed to and from module i

1)(

)()(

if

iviD

out

Architectural Design Metrics

System Complexity is defined as the sum of structural and data complexity,

C(i)=S(i)+D(i)

Increase in these values also increases in the overall architectural complexity

Class-oriented Metrics-CK Metrics Suite

Weighted methods per class (WMC)

Depth of inheritance tree (DIT)Number of children (NOC)Coupling between object classes (CBO)Response for a class (RFC)Lack of cohesion in methods (LCOM)

iCWMC

Class Oriented Metrics - The MOOD Metrics Suite

Method inheritance factor (MIF)

))()((

)(

idii

ii

CMCM

CMMIF

Halstead metric for source code

n1=number of distinct operators in programn2= number of distinct operands in programN1=total number of operator occurrencesN2=total number of operand occurrences

Length N can be estimated as,

Program volume may be defined as,

222121 loglog nnnnN

)(log 212 nnNV

Halstead metric for testing

Program Level PL is expressed as,

Halstead effort e can be computed as,

)()2(

1

2

21n

NnPL

PL

Ve

Metrics for Maintenance

IEEE Std 982.1-1988 recommends Software Maturity Index (SMI)It provides indication of stability of software product (based on changes that occur for each release of the product)

MT=number of modules in current release

Fc=number of modules in current release that are changed

Fa=number of modules in current release that are added

Fd=number of modules from preceding release that are deleted in current release

Metrics for Maintenance

The software maturity index is computed as,

As SMI approaches 1.0, the product begins to stabilize.SMI used for planning maintenance and release

T

dcaT

M

FFFMSMI

)(

29

Metrics for Software QualityCorrectness

This is the number of defects per KLOC, where a defect is a verified lack of conformance to requirementsDefects are those problems reported by a program user after the program is released for general use

MaintainabilityThis describes the ease with which a program can be corrected if an error is found, adapted if the environment changes, or enhanced if the customer has changed requirementsMean time to change (MTTC) : the time to analyze, design, implement, test, and distribute a change to all users• Maintainable programs on average have a lower MTTC

30

Metrics for Software Quality

IntegrityThreatSecurity

Usability

]security))1((1[ threatIntegrity

Thank you!