Software Measurement & Metrics
2
A Quote on Measurement
“When you can measure what you are speaking about and express it innumbers, you know something about it; but when you cannot measure,when you cannot express it in numbers, your knowledge is of a meagerand unsatisfactory kind; it may be the beginning of knowledge, but youhave scarcely, in your thoughts, advanced to the stage of science.”
LORD WILLIAM KELVIN (1824 – 1907)
3
Reasons to MeasureTo characterize in order to
Gain an understanding of processes, products, resources, and environmentsEstablish baselines for comparisons with future assessments
To evaluate in order toDetermine status with respect to plans
To predict in order toGain understanding of relationships among processes and productsBuild models of these relationships
To improve in order toIdentify roadblocks, root causes, inefficiencies, and other opportunities for improving product quality and process performance
4
Categories of Software MeasurementTwo categories of software measurement
Direct measures• Software process (cost, effort, etc.)• Software product (lines of code produced, execution
time, defects reported over time, etc.)
Indirect measures• Software product (functionality, quality, complexity,
efficiency, reliability, maintainability, etc.)
Project metrics can be consolidated to create process metrics for an organization
5
Establishing a Metrics Baseline
By establishing a metrics baseline, benefits can be obtained at the software process, product, and project levelsBaseline data must have the following attributes
Data must be reasonably accurate (guesses should be avoided)Data should be collected for as many projects as possibleMeasures must be consistent (e.g., a line of code must be interpreted consistently across all projects)Past applications should be similar to the work that is to be estimated
6
Software Metrics Baseline Process
SoftwareEngineeringProcess
SoftwareProject
SoftwareProduct
Data Collection
MetricsComputation
MetricsEvaluation
Measures
Metrics
Indicators
Product Metrics
Ensuring the Quality of Product• Size• Complexity• Structure & Architecture• Product Quality• Defects
Product MetricsSize
number of elements • lines of code • number of documentation pages, • number of test cases • etc
development metrics • time metrics • cost metrics • consumption of resources metrics
size of components • number of modules/objects • average size of components
Defects MetricsDefects per unit sizeTypes/LevelsStages Introduced/FixedDefect Discovery RateDefect Removal EfficiencyResultsCosts/Efforts for finding& fixing
10
Defect Removal Efficiency
Defect removal efficiency provides benefits at both the project and process levelDRE = E / (E + D)The ideal value of DRE is 1, which means no defects are found after deliveryDRE encourages a software team to institute techniques for finding as many errors as possible before delivery
1
ii
ii EE
EDRE
Failure Intensity Setting FI in terms of reliability
is failure intensity
R is reliability
t is natural unit (time, etc.)
95.0
1ln
Rfor
t
Ror
t
R
Reliability and Failure IntensityReliability for 1 hour
mission timeFailure intensity
0.36800 1 failure / hour
0.90000 105 failure / 1000 hours
0.95900 1 failure / day
0.99000 10 failure / 1000 hours
0.99400 1 failure / week
0.99860 1 failure / month
0.99900 1 failure / 1000 hours
0.99989 1 failure / year
MTTF and Failure Intensity
MTTF MTTFA
MTTF MTTR MTBF
MTTF
1
System Reliability A system usually consists of components.
Components may have
Different reliability
Different dependencies among each other
System reliability is a function of the reliabilities of the (sub-) components and of the relationships between the components.
Serial System Reliability System is composed of n independent serially
connected components.
Failure of any component has a cross system effect, i.e., results in failure of the whole system.
A serial system has always smaller reliability than its components.
R1 R2 ...
The system is composed of 4 independent serially connected components
R1 = 0.95
R2 = 0.87
R3 = 0.82
R4 = 0.73
Rsystem = 0.95 0.87 0.82 0.73 = 0.4947
Serial system reliability is smaller than any individual reliability of the components
Parallel System Reliability
System is composed of n independent components connected in parallel. Failure of all components results in the failure of the whole system (principle of active redundancy).
R1
R2
...
The system is composed of 4 independent components connected in parallel
R1 = 0.95
R2 = 0.87
R3 = 0.82
R4 = 0.73
Rsystem = 1 – ((1 – 0.95) (1 – 0.87)
(1 – 0.82) (1 – 0.73)) = 0.9996
Parallel system reliability is greater than any individual reliability of the components
Metrics for Specification Quality
Requirements in a specification
Specificity (lack of ambiguity)
nnn nffr
r
ui
nnQ
Architectural Design Metrics
For hierarchical architectures (e.g. call and return architectures)Structural complexity of a module i is defined as,
S (i) = f2 out (i)
Architectural Design Metrics
Data complexity provides an indication of the complexity in the internal interface for a module i and is defined as,
where v(i) is the number of input and output variables that are passed to and from module i
1)(
)()(
if
iviD
out
Architectural Design Metrics
System Complexity is defined as the sum of structural and data complexity,
C(i)=S(i)+D(i)
Increase in these values also increases in the overall architectural complexity
Class-oriented Metrics-CK Metrics Suite
Weighted methods per class (WMC)
Depth of inheritance tree (DIT)Number of children (NOC)Coupling between object classes (CBO)Response for a class (RFC)Lack of cohesion in methods (LCOM)
iCWMC
Class Oriented Metrics - The MOOD Metrics Suite
Method inheritance factor (MIF)
))()((
)(
idii
ii
CMCM
CMMIF
Halstead metric for source code
n1=number of distinct operators in programn2= number of distinct operands in programN1=total number of operator occurrencesN2=total number of operand occurrences
Length N can be estimated as,
Program volume may be defined as,
222121 loglog nnnnN
)(log 212 nnNV
Halstead metric for testing
Program Level PL is expressed as,
Halstead effort e can be computed as,
)()2(
1
2
21n
NnPL
PL
Ve
Metrics for Maintenance
IEEE Std 982.1-1988 recommends Software Maturity Index (SMI)It provides indication of stability of software product (based on changes that occur for each release of the product)
MT=number of modules in current release
Fc=number of modules in current release that are changed
Fa=number of modules in current release that are added
Fd=number of modules from preceding release that are deleted in current release
Metrics for Maintenance
The software maturity index is computed as,
As SMI approaches 1.0, the product begins to stabilize.SMI used for planning maintenance and release
T
dcaT
M
FFFMSMI
)(
29
Metrics for Software QualityCorrectness
This is the number of defects per KLOC, where a defect is a verified lack of conformance to requirementsDefects are those problems reported by a program user after the program is released for general use
MaintainabilityThis describes the ease with which a program can be corrected if an error is found, adapted if the environment changes, or enhanced if the customer has changed requirementsMean time to change (MTTC) : the time to analyze, design, implement, test, and distribute a change to all users• Maintainable programs on average have a lower MTTC
30
Metrics for Software Quality
IntegrityThreatSecurity
Usability
]security))1((1[ threatIntegrity
Thank you!