Embed Size (px)
Citation preview
TESTING METRICSMAKING YOUR TESTS
VISIBLEAlper Mermer
WHY WOULD YOU CARE ABOUT METRICS ?
Make your team aware of the trends
WHY WOULD YOU CARE ABOUT METRICS ?
Share the pain
WHY WOULD YOU CARE ABOUT METRICS ?
Watch production
WHY WOULD YOU CARE ABOUT METRICS ?
Improve ownership
WHY WOULD YOU CARE ABOUT METRICS ?
Prevent issues
WHY WOULD YOU CARE ABOUT METRICS ?
Not all metrics are needed in every project
WHICH METRICS SHOULD YOU TRACK ?
Measure every service performance seperately
Define sets of scenarios to run in every build
Performance Measurements Through Time
WHICH METRICS SHOULD YOU TRACK ?
Build mechanisms to warn about unexpected variances
in servicesWatch overall system performance as well
Keep an eye on trends
Performance Measurements Through Time
WHICH METRICS SHOULD YOU TRACK ?
Break builds with high security risks
For lesser security risks, watch the trend
Number Of Security Warnings
WHICH METRICS SHOULD YOU TRACK ?
Test coverage (?)Cyclomatic Complexity
Code Quality Metrics
WHICH METRICS SHOULD YOU TRACK ?
Depth of InheritanceDistance from Main
Code Quality Metrics
WHICH METRICS SHOULD YOU TRACK ?
Tests that cover your requirements are listed hereIncreased time in Run Time affects feedback time
Serves as a motivation for testers
API + GUI Test Count And Run Time
WHICH METRICS SHOULD YOU TRACK ?
Should help to increase focus on quality
Decreasing number of higher level bugs boosts
up morale
Defects By Priority/Severity
WHICH METRICS SHOULD YOU TRACK ?
Lesser number of bugs increases the confidence in the product
An increase in the number of higher level bugs should start an
investigation
Defects By Priority/Severity
WHICH METRICS SHOULD YOU TRACK ?
Trend in number of defects from production is importantIt affects product and company image
Should be closely monitored and analyzed
Number of Defects From Production
WHICH METRICS SHOULD YOU TRACK ?
Serves as an indicator of problematic areasShould initiate discussions around prioritization
Defects From Production by Module
WHICH METRICS SHOULD YOU TRACK ?
A team internal metricA nice indicator of your testing process
It is expected to be very high in a mature shift left testing culture
Mean Time Between Build Failures
LAST WORDS
Metrics Are There to Help Your Brain, Not Replace It
