View
407
Download
1
Category
Tags:
Preview:
Citation preview
Software Development Metrics You
Can Count On
Arthur Hicken
Evangelist, Parasoft
21 Apr 2015 International Conference on Software Quality - ICSQ 2015
Takeaways
Basic understanding of available SDLC metrics
Learn which metrics are dangerous and how to recognize them
Learn how to mine the data effectively.
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 2
What Are Metrics
“A software metric is a quantitative measure of a
degree to which a software system or process
possesses some property. Since quantitative
measurements are essential in all sciences, there
is a continuous effort by computer science
practitioners and theoreticians to bring similar
approaches to software development.”
- Software metric - Wikipedia
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 3
Why Should I care about
Metrics?
What is my quality
What is my risk
Where is my risk
Is it good enough
Is it done
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 4
The Problem
“The software industry lacks standard metric and
measurement practices. Almost every software
metric has multiple definitions and ambiguous
counting rules… The result of metrics problems is
a lack of solid empirical data on software costs,
effort, schedules, quality, and other tangible
matters.”
- Strengths and Weaknesses of Software Metrics, 2006
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 5
Common Metrics
• Balanced scorecard
• Bugs per line of code
• Code coverage
• Cohesion
• Comment density
• Connascent software
components
• Coupling
• Cyclomatic complexity
(McCabe's complexity)
• DSQI (design structure
quality index)
• Halstead Complexity
• Instruction path length
• Maintainability index
• Number of classes and
interfaces
• Number of lines of code
• Program execution time
• Program load time
• Program size (binary)
• Function Points and
Automated Function Points
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 6
Metrics with a Capital M
Some metrics have taken on a life of their own
• Complexity
• Cohesion
• Coupling
• Maintainability
• KLOC
There are no silver bullets, no single metric that
defines “good” vs. “bad” software
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 7
Averages Aren’t Normal
• Averages help us
understand a trend
• More nuanced
analysis is more
productive
• Frequently the
average represents no
real data point
• Real risks hide behind
averages – IE slow
components9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 8
How Do You Measure Quality
• No – there is no way to quantify quality
• Yes – measurement is fundamental to quality
Can it be done?
• Complexity isn’t complicated
• Which developer is most productive?
No single metric
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 10
What Can You Measure
• Code churn
• Field bugs
• Static analysis findings
• Test failures
• Coverage
• Performance
• Counts (lines, files, …)
• Bug Arrival Rates
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 11
Varied Tool Chain
• Different teams use
different tools
• Difficult to unify
• Getting the most from
open-source
• How to get uniform
outcome
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 12
Getting Consistent Results
Map the tools
• Tricky but necessary
Get the results in one place
• data driven
Proper reporting
• Combined results
• By tool
Back-fill capabilities
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 13
Some Types of Graphics
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 14
• The instantaneous value of something
Gauges
• An incrementing and decrementing value
Counters
• Display multivariate data
Radar / Spider
• The statistical distribution of values in a stream of data
Histograms
• Show how one variable is correlated with another
Scatter plot
Checking Your Work
• Did you get the right numbers?
– Are they going in the right direction?
• Are you measuring enough?
– Are unexpected things happening?
• Are the measurements automatic?
– Manual estimates are inconsistent
– Multiple layers of manual collection yield compound
rounding errors
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 16
Getting Value
• Consistency is key
• Automation
• Tweak based on feedback loop
– Perform post-mortem on problems
– Verify that measurements are meaningful
• Doing the same thing will yield the same result
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 17
Keys to Success
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 20
Central Quality Hub
Control and Visibility
Real-time feedback & compliance
Aggregation into
actionable information
Bridge technical
finding with business impact
Harnessing “Big” Data
• Aggregate data
• Correlate data
• Mine data
• Create
– Reports
– Dashboards
– Tasks
– Alerts
• Continuous testing/delivery/release
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 21
Why Analyze
What’s Working / Not Working
What’s important
What’s critical
What’s more/less important than I thought
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 22
Names Can Be Misleading
• Complexity isn’t complex – ask a developer
• Cyclomatic is roughly 1 + (number of loops) +
(number of if statements)
– Clean but long if-then-else gets flagged
– Short complicated code is “OK”
• a=b+c; can be unbelievably complex using function pointers
• In other words
– Branches alone doesn’t make the code complicated
as regards maintainability or probability of bugs
– Good indicator of how many tests are needed
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 24
KLOC
• Ups and downs
• How to count
– White space
– Comments
– Multiple statements on one line
– CONSISTENCY
• Meaningful?
• IEEE study – correlation to bugs
– A Critique of Software Defect Prediction Models
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 27
Problem: Who’s the Best
• Lines of code written
– Is it difficult
– Is it good
– Is it maintainable
– Is less/more better
– Comments?
• Field bugs
• Tasks completed
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 28
Analyzing Code Maintainability
Complicated
• Difficult to understand
Complexity
• The interactions between a number of entities
Coupling
• The manner and degree of interdependence between software modules
Cohesion
• The degree to which the elements of a module belong together
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 29
Problem: Where’s the Risk
• Which component / code is vulnerable
• Multiple data points
– Static code analysis (CWE)
– Unit test results
– Source control data
– QA and manual testing
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 30
Problem: Am I on Track?
• Have to know what “DONE” is first
• Need connection to requirements
• Need connection to source control
• Need connection to development management
data (scrums etc)
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 31
Problem: Where Are the Bugs
• No single metric can predict bugs
• Enhanced data is helpful
• Testability itself shows bugs
– Can you use auto test generation?
• Link to field bug-tracking system is valuable
• Correlate to Q&A activities
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 32
Coverage Metrics
• Some Types
– Functional
– Statement
– Branch
– Condition
– Modified Condition / Decision Coverage (MCDC)
• Meaning
• Danger of chasing coverage
• Reasonable answer – How Much is Enough?
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 33
A Word on Debt
• Debt is a real thing
– We incur debt consciously and accidentally
• Real debt isn’t trivial to measure
– Wrong frameworks, models, algorithms
• Anyone claiming they can fix 10,000 hours of
debt in a day is selling snake oil
• Static analysis alone cannot fully calculate debt
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 34
Conclusions
• Avoid simple metrics for decision making
• Be sure the metric means what you think it does
• Be sure that the practice you’re planning will
affect your metric
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 35
Q&A
• Web
– http://www.parasoft.com
• Blog
– http://alm.parasoft.com http://codecurmudgeon.com
• Social
– Facebook: https://www.facebook.com/parasoftcorporation
– Twitter: @Parasoft @CodeCurmudgeon
– LinkedIn: http://www.linkedin.com/company/parasoft
– Google+: +Parasoft +ArthurHickenCodeCurmudgeon
9-11 Mar 2015 International Conference on Software Quality - ICSQ 2015 36
Recommended