Upload
alliance-global-services
View
3.504
Download
2
Embed Size (px)
DESCRIPTION
In an Agile project, what testing metrics should you care about? How do you best collect these metrics so they are meaningful? Agile projects impose unique challenges compared to traditional methodologies because of the key tenets of the Agile manifesto. From rapid execution cycles to preference for working software over documentation, it may seem that effectively measuring testing of Agile projects is prohibitively difficult. During this webinar, we will present an overview of what metrics need to be captured specifically in an Agile project that will provide the business with a valuable snapshot of the quality of a release.
Citation preview
Metrics that Matter
Measuring Quality in Agile Project
Sreekanth Singaraju, VPSharon Lee, Director of Marketing
2
Agenda
• Agile Overview• Testing in Agile• Importance of Metrics• Key Metrics
– Test and Execution Coverage – Defect identification efficiency – Technical Debt – Test confidence level
• Tools
3
Why Organizations Use Agile
• Develop iteratively with business needs
• Deliver working software rapidly
• Flexibility to change with business needs
• Incorporate end-user feedback
• Highly productive technical teams
Deliver what end-users really want in an iterative fashion
4
Role of Testing in Agile
• Testing may be done by people who aren’t called “QA” or “Tester”; e.g. Business Analysts.
• Some teams may use developers for system or acceptance testing.
• Dedicated testers bring two benefits– Focus on customer usage over technical implementation– Focus on uncovering flaws over confirming completeness
What product or service does testing provide?“Tested, Debugged Software” – Wrong
Testing provides Information about the status of software under development to make informed decisions.
5
Testing in Agile Projects
Agile Testing Quadrants provide a concise way to communicate the different layers of testing needed and how they can be executed
6
Testing in Agile Projects
• Testing is time-boxed like development
• Functionality to be tested is not limited like Dev
• Difficult to achieve “fully-tested” status
• Defects found do not reflect Test Results
• Testing spread across Dev and Test teams
Unique Challenges on Agile Projects
7
Metrics Objectives
8
Metrics Objectives
For Business Stakeholders provide:
• All the information they need to make decisions, and no more.
• Information at the level of detail they can use.
• Information at the scope they care about (team, project, program, line of business, enterprise, industry)
• Information pertaining to the time frame they care about (day, iteration, release, project, strategic milestone)
9
Metrics Objectives
Communicate the following aspects:
• The level of testing that was accomplished in a sprint, release etc.
• The efficiency with which defects have been identified in different testing stages
• The cost of trade-offs being made in each sprint
• The overall confidence level in the testing efforts
10
Coverage Metrics
PrincipleCommunicate to all stakeholders the coverage provided by formal testing
Dimensions• Use Validation points to evaluate application’s compliance with requirements• Weight them by business criticality
DefinitionDesigned Coverage = Designed Validation Points / All Validation Points (Estimated)Executed Coverage = Executed Validation Points / All Validation Points (Estimated)
Best Practices• Keep coverage metrics simple• Introduce granularity to address business needs• Trend to show progress (or declines – great case for additional resources)
11
Defect Identification Efficiency
PrincipleIdentify how defects are identified and created. Evaluate compliance of Agile principles.
Dimensions• Number of defects identified in different stages of software lifecycle• Number of defects created in different stages of software lifecycle• Use key Software stages to draw valuable inferences
DefinitionProportion of defects identified by each stage of a lifecycle – Ex – Testing –56%, UAT – 18%, Production - 4%, other - 22%
Best Practices• Start with defects identified first – then introduce defects created• Use “Other” to catch exceptions and introduce granularity incrementally• Factor in defect severity if that information is readily available
12
Technical Debt
PrincipleProvide a financial impact of technical choices on the overall product resulting from technology, process and implementation choices made during development
Dimensions• Quality of software code• Unresolved defects that are business or quality critical• Impact of technical or architectural choices
DefinitionCost estimates for redressing most critical (Business perspective) defects or technical choices
Best Practices• Explosive- Handle and communicate with care• Be conservative – use a very strong criteria to evaluate “criticality”• Use several available tools to evaluate software code quality
13
Confidence Level
PrincipleDemonstrate the confidence of the testing team in the testing efforts executed on a release with the resources provided for testing.
Informational• Test Execution Coverage• Expertise of resources to test the application• Quantitative results of defects including Defect identification efficiency
DefinitionBased on the evaluation of the 3 dimensions above rank each of them on a scale of 0(Poor) – 1(High). Overall confidence = (3.Coverage.Resources.Results)/(Coverage+Resources+Results)
Best Practices• Be prepared with supporting facts• Define clearly laid out criteria for ranking
14
Key Success Drivers
• Alignment with ultimate stakeholders’ goals
• Won’t be perfect so don’t wait for them to be
• Keep them simple
• Collaborate with other functions
• Trending more valuable than snapshots
Key best practices to succeed at creating
metrics.
15
Tools
Quality Management Systems• HP Quality Center• Silk Central• Rally
Automation Tools• QTP• Selenium• Silk
Code Quality Analysis• CAST AIP• Coverity• Radar
16
QUESTIONS & DISCUSSION
www.allianceglobalservices.com
17
Thank YouFor more information email us at: [email protected]