Embed Size (px)
DESCRIPTION
Presentation for SQuAD on May 12, 2009. Abstract: In an agile environment, testing alone is not sufficient to ensure quality. Many other factors come into play to ensure true quality. This session will explore the meaning of "quality" and give some examples of how it can be enhanced. Richard and Bob will also explain the principles behind an agile process which achieves a quality product as well as a potential workflow to implement the process. This will include how QA integrates with the team to avoid creating a "mini-waterfall" situation.
Citation preview
Testing != QualityRethinking quality and how to achieve it in an agile world
Bob HartmanRichard Lawrence
“How does QA fit into all of this?”
are conflicting goals
Speed/value
quality
and
Or are they?
How can quality enable speed and value?
What is quality?
How do we ensure our software has it?
What is quality?
The goal of a software org:Produce valuable software now and in the future.
A feature can be modeled with a cumulative cash flow chart
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09
Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
-$300,000
-$200,000
-$100,000
$0
$100,000
$200,000
$300,000
$400,000
$500,000
$600,000
Or, we can remove the investment and just look at value flow
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Value flow for a feature that never works right
Value flow for a feature that fails for some users some of the time
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Value flow for a feature with downtimes
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Value flow for a feature that doesn’t scale
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Quality is the foundation of value
A team’s productivity can be measured with a velocity chart
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Velocity of a team with a well-designed, high quality system
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Velocity of a team with a poorly-designed, low quality system
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Quality is the foundation of speed
Photo credit: woodsy on sxc.hu
Quality is what ensures: Value now and in the future
How do we ensure our software has it?
Testing at the end doesn’t workIteration Start Iteration End
The Plan
The Reality
Plan Dev Test
What’s really going on?
The Plan
The Reality
Dev Test
Dev Test
Dev expands to
fill the available
time
Dev to test is really a loop
Build quality in vs. inspecting for it at end
Inspect for quality Dev TestPlan
Build quality in
Manual testing doesn’t scale
Sprint 1 Sprint 2 Sprint 3 Sprint 4
New feature testing
Sprint 5
Testing that doesn’t happen,
but should
(or
happens
in
overtime)
Regression testing
Testing capacity
Automate and get regression tests for free
Sprint 1 Sprint 2 Sprint 3 Sprint 4
New feature testing
Sprint 5
Automated tests that are now
regression tests
Testing capacity
What is really being tested?
1 2 3 4 5 Total1 5
2 4
3 3
4 2
5 1
IterationsFe
atur
es
Which features need to work best?
Never Used45%
Rarely19%
Sometimes16%
Often13%
Always7%
Source: The Standish Group
Usage of features in a typical software product
But without regression it is different
1 2 3 4 5 Total1 1
2 1
3 1
4 1
5 1
IterationsFe
atur
es
Minimal regression is a little better
1 2 3 4 5 Total1 3
2 3
3 3
4 2
5 1
IterationsFe
atur
es
But what will happen where we don’t have regression tests running?
1 2 3 4 5 Total1 3
2 3
3 3
4 2
5 1
= Missed tests, which Murphy’s Law says will lead to defects being missed!
IterationsFe
atur
es
Build and deploy processes can kill speed
Define acceptance criteria and automate
tests
Develop Package Deploy to test
env
Test
DONEThe Plan
The Reality Loops, waits, errors…more time
To ensure quality in an agile world:
• Move testing to the front of the process• Collaborate• Automate tests (using agile-friendly tools)• Automate/accelerate build and deploy• Work on user stories one at a time, all the way
to DONE
