Acceptance
Test Driven
Development
(ATDD)Lean-Agile Acceptance
Test-Driven Development (
Ken Pugh, 2011
ATDD
Acceptance tests are
developed collaboratively
between the :
Customer
Developer
tester
Lean-Agile Acceptance Test-Driven Development (2011) Ken Pugh
Acceptance test driven development – involves developing
requirements tests prior to implementation
Chapter 2
ATDD Lineage
ATDD comes from:
Example driven development (EDD)
Behavior-driven development (BDD)
Story-driven development (SDD)
Domain-driven Design (DDD)
Executable acceptance test-driven
development (EATDD)
Lean-Agile Acceptance Test-Driven Development (2011) Ken Pugh
Testing Strategy –
Chapter 3
Testing Matrix Chapter 3
Where
Tests
Run
Positive & Negative Testing
Positive testing – ensure the program works
as expected
Negative testing – check to see that
program does not create unexpected
results
Acceptance Testing –
Chapter 4
Requirements & Tests are
Linked
Tests clarify and amplify requirements
Can’t have a requirement without a test or
vice versa
Tests written after the code is written is a
new requirement or a new detail on an
existing requirement
Chapter 4
Requirement Examples
Examples of requirements clarify the
requirements and can be used as tests
Requirement Examples -
Example
Requirement Examples -
Example
Testing
Tests of business rules can be executed in at
least four ways:
1. Through the user interface of a
transaction that invokes the business rule
2. Development of a user interface that
directly invokes the business rule
3. Unit test implemented in a language’s
unit testing framework
4. Automated test that communicates with
the business rule module
Example project –
Chapter 5
Charter & Features
Charter objectives - project acceptance tests
Features (high level requirements) - feature acceptance criteria
ATDD Process Chapter 5
CharterVision - Describes the end-state of the result of the project, why
project should be done.
Mission - Path that will be taken to get to the vision.
Scope – boundaries of the project and key issues that the
project addresses (optional)
Deliverable – tangible outputs (optional)
Objectives - Measurable criteria for the success of the project
Principles - Statement of values that a team uses to inform
decision making.
CharterVision - Describes the end-state of the result of the project, why project should be done.
Mission - Path that will be taken to get to the vision.
Scope – boundaries of the project and key issues that the project addresses (optional)
Deliverable – tangible outputs (optional)
Objectives - Measurable criteria for the success of the project -project acceptance tests
Principles - Statement of values that a team uses to inform decision making.
Objectives – Acceptance
Tests
Objectives must be visible by external observation
SMART :
Specific
Measureable
Achievable
Relevant
Time boxed
Video
store
rental
example
Releases
Possibly create mini-charters for each project release
ATDD Process
High Level Requirements
High level requirements or features
Create a list of possible features.
Select those that accomplish the objectives.
Estimate if the selected features can be implemented
to achieve the objectives.
If not, reevaluate the objectives.
Video Rentals Brain Storm
Features to Accomplish
Objectives
Feature Acceptance Criteria
Sequence & Schedule Features
User Stories – Chapter 6
User stories - Story acceptance criteria
ATDD Process
User Stories Chapter 6
Break “features” into smaller “user stories” (also
called “requirement story”).
Only breakdown enough features to fill the
team’s time until the next feature breakdown.
Constraint stories also belong here .
Example – software must be developed using
open source software.
User Story Process
While creating user stories , list one or more
acceptance criteria for each story
These acceptance criteria are expanded to
specific acceptance tests just before the story
is develop
Make stories brief. Details are gathered just
prior to, or during, the story’s implementation
Sprint PlanningEstimate the business value and effort for each
story – ratio gives a rough “Return On
Investment” (ROI) – use ROI to determine what
belongs in next sprintAs ROI flattens
may terminate
project for
other projects
with higher ROI
If iterations involve approximately
the same effort, the slope of the
business value curse is roughly the
ROI
User Story Breakdowns
Break large stories into smaller – but business
value for a story is not achieved until all
needed stories are developed
Developer stories don’t give customer direct
value, but may be needed
Developer Stories
Some customer stories require framework development, translating to many developer stories. Example -development organization develops common modules or components
Developer stories add technical value and make it easier to track units of work
Acceptance criteria for these defined by the developers, not the customers
If considerable framework development is needed, - have an embedded technical project and a “technical product owner”, who can break a customer stories into technical customer stories, each with acceptance criteria, is needed
Roles
Role – “hats” people wear. One person can perform different roles.
Role attributes: Frequency of use
Domain expertiseComputer expertise
General goal
Can create “persona” for each role.
Persona – imaginary person described with lots of detail.
User Stories
As a <role>, I want to <do something> so
that <reason>.
Want stories to satisfy INVEST
I – independent
N – negotiable
V – valuable
E – estimatable
S – small
T – testable
User Stories
Story
Accept
ance
Criteria
Story Estimates
Estimates needed:
Business value – relative worth of a story
to the business
Effort - include all work required to deliver
the store including implementing the
code, testing it, and any other work
involved.
Business Value
Business value:
Increased revenue (sales, royalties, fees)
Decreased expenses
Using fewer resources
More efficient use of resources
Customer satisfaction
Product promoters/satisfiers/detractors
Staying in business
Avoiding risk
Estimating Business Value
Order the stories and assign values :
Fibonacci series (1, 2, 3, 5, 8, 13, …)
power of 2 (1, 2, 4, 8, …)
Collaborating on
Scenarios – Chapter 7
Use cases (scenarios) - acceptance tests
ATDD Process
Use Case
Use case – describes a sequence of actions
and reactions between the user and the
software
Use cases can be highly details, fairly
lightweight, or anywhere in between
Don’t create use cases for stories until they are
needed, helping to avoid “analysis paralysis”
Sample Use Case
As the clerk, I want to check out a CD for a customer
so that I can keep track of who has rented it.
Use Case Template
Name – brief name given to user story
Description – can be the user story
Actor – almost always the role
Sample Use Case
As the clerk, I
want to check
out a CD for a
customer so
that I can keep
track of who
has rented it.
Sample Use Case - Exceptions
As the clerk, I want to check out a CD for a customer
so that I can keep track of who has rented it.
Sample Use Case - Alternatives
As the clerk, I want to check out a CD for a customer
so that I can keep track of who has rented it.
Sample Use Case - Tests
As the clerk, I want to check out a CD for a customer
so that I can keep track of who has rented it.
Outline acceptance tests – at least one for the main
course, each exception, and each alternative
Alternative Ways to Generate
Scenarios
Scenarios can be generated :
• Story map
• Conceptual flow
Story
Map
Conceptual Flow
Test Anatomy – Chapter 8
Finally the tests
ATDD Process
Developing Tests
Customer, developer and tester develop tests together
Tester may envision extra tests
New tests may be discovered during implementation, but if this occurs frequently, the team can investigate the root cause of missing tests
Test Structure
Six Tests Similar Tests
Test Flow – Check Out CD
Sample
Test
…
Test CasesTest case is a test scenario with actual data.
Data can be placed in tables
Tables can be useful for expressing tests:. Three common types of tables:
Calculation
Data
Action
Calculation Table
Format:
? Indicates output
Examples
Data Table
Data tables can be used for input values or
expected values
Examples
Can include “data” in the name. If data
meets certain criteria, specify it after the
name
Action Table
First column: enter, press and check.
Examples
Use Tables or Not
Tests Needed
At least one test is needed for:
1. Every scenario (happy path)
2. Every alternate path
3. Every exception
more later….
Scenario Tests– Chapter 9
Cross-Story/Scenario Tests
Cross-Story – stories that have issues with other stories
Example:
Scenario for monitor that expects response from each disk each second or reports a failure
Scenario that turns a disk off if it isn’t accessed after 60 seconds.
Tests for each scenario may work, but need one or more tests for the interactions (integration testing)
Example – Late Rental & CD
Limit Reached
Multi-Level Tests
Run tests at multiple levels
Example – scenario that involves business
rules. Business rule tests are usually simpler
than UI tests (business rules don’t require
much setup). Test separately. Test business
rules, and later test UI.
UI Tests
Create a working system early and test
against objectives. (Could create a
prototype.)
Alternate Interfaces
Tests Needed
At least one test is needed for:
1. Every scenario (happy path)
2. Every alternate path
3. Every exception
4. Cross story tests
5. Business rules (multi-level)
6. UI tests
User Story Breakup –
Chapter 10
Boundary Conditions
Consider boundary conditions and if more
tests are needed.
System Boundary –
Chapter 11
External Interfaces
Credit Card Processor
Bank Transfer
Component Tests
Component Tests - continued
Component Tests - continued
Explore ways the credit card could be
denied and ask the customer what to do in
each case. Tests can be built for each.
Development Review –
Chapter 12
Workflow Tests
Need to test the entire workflow to make
sure there are no issues between related
stories and that the entire flow is usable.
Tests NeededAt least one test is needed for:
1. Every scenario (happy path)
2. Every alternate path
3. Every exception
4. Cross story tests
5. Business rules (multi-level)
6. UI tests
7. External interfaces
8. Component
9. Usability testing
10. Quality attribute testing
Revisit Chapter 1
Test Conditions (Chapter 1)
Tests should:
1. Fail for a well defined reason (the reason
the test was created)
2. No other test should fail for the same
reason, (otherwise you have a
redundant test)
3. A test should not fail for any other reason
Test Doesn’t Fail for Other
Reasons
A test should not fail for any other reason –
hard to achieve.
Tests have setup, action/trigger and
expected result. A test may fail because the
setup did not work or the action/trigger did
not function properly.