Upload
vijayasekar
View
217
Download
0
Embed Size (px)
Citation preview
8/14/2019 Proficience - SQAM - Module 4
1/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 11
Module 4Module 4Software MetricsSoftware Metrics
8/14/2019 Proficience - SQAM - Module 4
2/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 22
When you can measure what you areWhen you can measure what you are
speaking about and express it inspeaking about and express it innumbers, you know something about it;numbers, you know something about it;
but when you cannot measure, whenbut when you cannot measure, when
you cannot express it in numbers, youryou cannot express it in numbers, your
knowledge is of a meager andknowledge is of a meager andunsatisfactory kind; it may be theunsatisfactory kind; it may be the
beginning of knowledge, but you havebeginning of knowledge, but you have
scarcely, in your thoughts, advanced toscarcely, in your thoughts, advanced to
the stage of science Lord Kelvinthe stage of science Lord Kelvin
You cannot control what you cannotYou cannot control what you cannot
measure - DeMarcomeasure - DeMarco
8/14/2019 Proficience - SQAM - Module 4
3/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 33
MeasurementMeasurement
Economic measurements determine priceEconomic measurements determine price
and pay increasesand pay increases
Measurements in radar systems enable usMeasurements in radar systems enable us
to detect air craftto detect air craft Measurements enable doctors to diagnoseMeasurements enable doctors to diagnose
specific illnessesspecific illnesses
Measurements in atmospheric systemsMeasurements in atmospheric systemsbasis for weather predictionbasis for weather prediction
8/14/2019 Proficience - SQAM - Module 4
4/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 44
MeasurementMeasurement
A process by which numbers orA process by which numbers or
symbols are assigned to attributes ofsymbols are assigned to attributes of
entities in the real world in such aentities in the real world in such a
way as to describe them according toway as to describe them according to
clearly defined rules.clearly defined rules.
8/14/2019 Proficience - SQAM - Module 4
5/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 55
Measurement helps..Measurement helps..
Understand about software products,Understand about software products,
processes and servicesprocesses and services
Evaluate against the establishedEvaluate against the established
standards and goalsstandards and goals
Control resources and processesControl resources and processes
Predicts the attributes of software inPredicts the attributes of software infuturefuture
8/14/2019 Proficience - SQAM - Module 4
6/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 66
Information, Data & MetricsInformation, Data & Metrics
8/14/2019 Proficience - SQAM - Module 4
7/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 77
Measurement at differentMeasurement at different
levelslevels
8/14/2019 Proficience - SQAM - Module 4
8/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 88
Use of Measurement at different levelsUse of Measurement at different levels
8/14/2019 Proficience - SQAM - Module 4
9/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 99
MeasurementMeasurement
CharacteristicsCharacteristics Simple and robustSimple and robust
Be a part of the processBe a part of the process
RepeatableRepeatable Easy to measureEasy to measure
Non threateningNon threatening
Non evaluativeNon evaluative
8/14/2019 Proficience - SQAM - Module 4
10/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1010
Steps to design a metricSteps to design a metric
Identify metrics customersIdentify metrics customers Target goalsTarget goals Ask questionsAsk questions
Select metricsSelect metrics Standardize definitionsStandardize definitions Choose a measurement functionChoose a measurement function Establish a measurement methodEstablish a measurement method
Define a decision criteriaDefine a decision criteria Define reporting mechanismsDefine reporting mechanisms Determine additional qualifiersDetermine additional qualifiers
8/14/2019 Proficience - SQAM - Module 4
11/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1111
Goal Question MetricGoal Question Metric
Goal 1
Q1
Goal 2
Q2 Q3 Q4
M1 M2 M3 M4 M5 M6 M7 M8
8/14/2019 Proficience - SQAM - Module 4
12/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1212
GQM - ExampleGQM - Example
Goal: Reduce maintenance costs by 50% within a yearGoal: Reduce maintenance costs by 50% within a year
Questions:Questions:
- How much do we spend each month?How much do we spend each month?
- What is the fraction of cost on maintaining eachWhat is the fraction of cost on maintaining each
application?application?- How much money do we spend on corrective, adaptive andHow much money do we spend on corrective, adaptive and
preventive maintenance?preventive maintenance?
Metrics:Metrics:
- Total Budget on maintenance- Total Budget on maintenance
- Hours spent on for maintenance for each applicationHours spent on for maintenance for each application
- Hours spent on for every type of maintenance and costHours spent on for every type of maintenance and cost
involvedinvolved
One more example..One more example..
8/14/2019 Proficience - SQAM - Module 4
13/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1313
Class work..Class work..
Improve Schedule EstimationImprove Schedule Estimation
accuracy by 10% within 6 monthsaccuracy by 10% within 6 months
Reduce system testing defects forReduce system testing defects for
the next 3 development projectsthe next 3 development projects
Reduce time to close a defect byReduce time to close a defect by
40% within 3 months40% within 3 months
8/14/2019 Proficience - SQAM - Module 4
14/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1414
Metrics ExamplesMetrics Examples
Product size: Kloc, FP, Object classes, Screens,Product size: Kloc, FP, Object classes, Screens,Interface points, etc.Interface points, etc.
Effort and Schedule: Time sheetsEffort and Schedule: Time sheets
Cost: Cost of Quality, ReworkCost: Cost of Quality, Rework
Reviews: Preparation rate, Review Rate, DefectReviews: Preparation rate, Review Rate, DefectDensityDensity
Planning: Effort variance, schedule variance,Planning: Effort variance, schedule variance,ProductivityProductivity
Requirements:Requirements VolatilityRequirements:Requirements VolatilityDefects: Severity, Defect Density, Defect RemovalDefects: Severity, Defect Density, Defect Removal
Effectiveness, Defect Distribution, etcEffectiveness, Defect Distribution, etc
Maintenance: Turn around time, BacklogMaintenance: Turn around time, Backlog
Management IndexManagement Index
8/14/2019 Proficience - SQAM - Module 4
15/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1515
Software Measurement SinsSoftware Measurement Sins
Lack of management commitment Educate!Lack of management commitment Educate!
Measuring too much, too soon Start simple and balancedMeasuring too much, too soon Start simple and balanced
Measuring too little, too late Measure the aspectsMeasuring too little, too late Measure the aspects
together!together!
Measuring the wrong things Right audience!Measuring the wrong things Right audience! Imprecise metrics definitions Standardize!Imprecise metrics definitions Standardize!
Use the data for evaluating individuals Never!Use the data for evaluating individuals Never!
Use metrics to motivate, than understand Focus on resultsUse metrics to motivate, than understand Focus on results
Collecting data that is not used Inform or remove!Collecting data that is not used Inform or remove!
Lack of communication and training Handbook, website,Lack of communication and training Handbook, website,
short meetshort meet
Misinterpreting metrics data Do not over react!Misinterpreting metrics data Do not over react!
8/14/2019 Proficience - SQAM - Module 4
16/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1616
Establishing a measurement programEstablishing a measurement program
Problems to be tackled areProblems to be tackled are
lack of management commitmentlack of management commitment
imprecise or inappropriate objectivesimprecise or inappropriate objectives
unclear linkage to business and managementunclear linkage to business and managementgoalsgoals
differing management and staff expectationsdiffering management and staff expectations
assessment criteria not definedassessment criteria not defined
lack of feedbacklack of feedback
concerns about misuse of informationconcerns about misuse of information
cultural change neededcultural change needed
willingness to continue in the longer term.willingness to continue in the longer term.
8/14/2019 Proficience - SQAM - Module 4
17/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1717
Establishing a measurement program-Establishing a measurement program-
Step1Step11.1. Appoint an executive sponsor committed,Appoint an executive sponsor committed,dynamic, strong motivator, sets hard anddynamic, strong motivator, sets hard and
realistic targetsrealistic targets
2.2. Establish scope and goals ExamplesEstablish scope and goals Examples
identify where changes can improveidentify where changes can improve
productivityproductivity
assess the effectiveness of reviews andassess the effectiveness of reviews and
testingtesting
determine the effort spent on reworkdetermine the effort spent on rework
identify trends in defectsidentify trends in defects
8/14/2019 Proficience - SQAM - Module 4
18/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1818
Establishing a measurement program-Step 3Establishing a measurement program-Step 3
3. Define Key Performance Indicators, measures, models 3. Define Key Performance Indicators, measures, models
ProductivityProductivity
Function Points per person monthFunction Points per person month
QualityQuality
percentage effort spent on reworkpercentage effort spent on rework
percentage effort spent on reviewspercentage effort spent on reviews
effectiveness of reviewseffectiveness of reviews
effectiveness of systems testingeffectiveness of systems testing
average effort to correct defects found during developmentaverage effort to correct defects found during development
percentage bad fixes during developmentpercentage bad fixes during development
EstimatesEstimatesEstimated vs. actual effort (variance)Estimated vs. actual effort (variance)
Estimated vs. actual timescale (variance)Estimated vs. actual timescale (variance)
8/14/2019 Proficience - SQAM - Module 4
19/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 1919
Establishing a measurement program-Step 4Establishing a measurement program-Step 4
4. Establish a data collection Infrastructure4. Establish a data collection Infrastructure
who collects data?who collects data?
when is it collected?when is it collected?
where is it collected from?where is it collected from? how is it collected and validated?how is it collected and validated?
who analyses and presents the results?who analyses and presents the results?
8/14/2019 Proficience - SQAM - Module 4
20/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 2020
Establishing a measurement program-Step 5Establishing a measurement program-Step 5
5. Explain the Program:5. Explain the Program:
Workshop for managersWorkshop for managers
Workshop for project teams andWorkshop for project teams and
developersdevelopersTraining for staff in measurement andTraining for staff in measurement and
analysisanalysis
Questions to be answered:Questions to be answered:
Do teams understand the purpose of theDo teams understand the purpose of theprogram?program?
Do teams support the program?Do teams support the program?
Do teams know what is expected of them?Do teams know what is expected of them?
8/14/2019 Proficience - SQAM - Module 4
21/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 2121
Establishing a measurement program-Step 6Establishing a measurement program-Step 6
6. Devise success criteria Periodic review6. Devise success criteria Periodic review
Is the right information being generated atIs the right information being generated at
thethe
right time?right time? What is the quality of the information?What is the quality of the information?
Have baselines and targets been set for eachHave baselines and targets been set for each
KPI?KPI?
Have we got quantifiable information to aidHave we got quantifiable information to aiddecision making?decision making?
Has information been acted upon?Has information been acted upon?
Is it understood when to act on theIs it understood when to act on the
information?information?
8/14/2019 Proficience - SQAM - Module 4
22/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 2222
Establishing a measurement program-Step 7Establishing a measurement program-Step 7
7. Set baselines and targets for KPI7. Set baselines and targets for KPI
Identify when projects provide data and when data isIdentify when projects provide data and when data is
assessedassessed
Establish annual (or more frequent) baselines and targets,Establish annual (or more frequent) baselines and targets,
derived from quantifiable informationderived from quantifiable information Assess performance against baselines and targetsAssess performance against baselines and targets
Use data generated in first 3-6 months to set initialUse data generated in first 3-6 months to set initial
baselinesbaselines
Base initial targets on achievements of knownBase initial targets on achievements of known
improvement programsimprovement programs
Review and revise baselines and targets when measuredReview and revise baselines and targets when measured
information becomes availableinformation becomes available
8/14/2019 Proficience - SQAM - Module 4
23/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 2323
Establishing a measurement program-Step 8Establishing a measurement program-Step 8
8. Develop a Feedback mechanism8. Develop a Feedback mechanism
Use effective methods of presenting andUse effective methods of presenting and
reporting resultsreporting results
Dash boards Green, Yellow, RedDash boards Green, Yellow, Red
8/14/2019 Proficience - SQAM - Module 4
24/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 2424
Metrics for SMEMetrics for SME
Keep it simpleKeep it simple
Customize to local needsCustomize to local needs
Be sure that it adds valueBe sure that it adds value
ScheduleSchedule EffortEffort
Defect Removal EffectivenessDefect Removal Effectiveness
8/14/2019 Proficience - SQAM - Module 4
25/26
SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 2525
Cost of Software MetricsCost of Software Metrics
For small groups, initial cost is 3-8%For small groups, initial cost is 3-8%
then drops to less than 1% afterthen drops to less than 1% after
familiarityfamiliarity
In major organizations, initial cost isIn major organizations, initial cost is
relatively high then it drops to lessrelatively high then it drops to less
than 5%.than 5%.
Source: Software Metrics, Fenton & FleegerSource: Software Metrics, Fenton & Fleeger
8/14/2019 Proficience - SQAM - Module 4
26/26