Atdd half day_new_1_up

ATDDAcceptanceTest Driven Development

J a r e d R i c h a r d s o n A g i l e A r t i s a n s

@ J a r e d R i c h a r d s o n

Who are you?

Jared RichardsonCoAuthored GROWS Methodology

Author Ship It! and Career 2.0

Screencast editor PragProg.com

2nd public signatory of the Agile Manifesto

Started AgileRTP in 2007!

AgileArtisans.com

Who are you?

What would you say you do here?

Roles?

Responsibilities?

Why are you here?

What are your tests for this session?

ObjectivesDefine ATDD

Why requirements are not enough

Who to involve when

Turning requirements into tests

What to test

How ATDD can make your entire company more productive

Outline

Terms and definitions

Requirements and tests

Software development

ATDD examples

Anatomy of a test

Tables (Grid Driven Development)

Take Aways

What’s a test?

Who writes them?

When are they written?

What do they verify?

Provide an example

Primary benefit?

Secondary benefits?

Breaks

Halfway (90 minutes)

Background

Lean-Agile Acceptance Test Driven Development

by Ken Pugh

One Rule

There are exceptions to every statement

Except this one

Familiar

Working

Smooth

Teaching

Dreyfus ModelofSkillsAcquisition

en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition

Familiar

Working

Smooth

Teaching

Dreyfus ModelofSkillsAcquisition

Recipes

Intuition

Principles

Key Points

Skills don’t map

dilbert.com/strip/1990-09-13

Key Points

Beginners need steps

What is ATDD?

Acceptance Test Driven Development

External view of the system

Examines visible effects

Inputs

Outputs

State changes

External interfaces16

Definitions

Acceptance criteria -> General idea

Acceptance test -> specific. Pass/fail.

Implementation independent

Triad -> Customer, developer, tester

Fast Car

Who wants one?

Criteria

Closed course. Measure acceleration.

Accelerate 0 to 60 in X seconds

flickr.com/photos/vodcars/4132650949

Fast Car

flickr.com/photos/fordeu/14296246942

0 to 60 in 30 seconds

0 to 60 in 5 seconds.

Top speed is 61.

0 to 60 in 5. Top speed is 120.

60 to 120 in 5 min

0 to 60 in 5. 60 to 120 in 20 sec.

Range of 1/4 mile.

0 to 60 in 30 seconds

0 to 60 in 5 seconds.

Top speed is 61.

0 to 60 in 5. Top speed is 120.

60 to 120 in 5 min

0 to 60 in 5. 60 to 120 in 20 sec.

Range of 1/4 mile.

Range of 1,000 miles. Cost 7 billion

Fast Car

flickr.com/photos/alpi-costerni/8161111001

Why ATDD?Rework down from 60% to 20%

Workflows work 1st time

Little room miscommunication

Saves time

Getting business rules RIGHT

Tighter cross functional team integration

Crisp visible story completion criteria

Automation yields reduced testing time21

Requirements and Tests

Requirement

Needed by a user to solve a problem

Needed by a system to satisfy contract, standard, spec, regulation...

Requirements Include

Functions (what software does)

Constraints (limits)

Requirements Should not

Include

Implementation details

Specific manifestation

A Brownie Spec

Ingredient Parts by weight

Sugar 23.0Flour 21.0Shortening 16.8 Nuts 16.0whole eggs 13.0Cocoa 5.5Dextrose, anhydrous 4.4Salt

Brownie Spec

a. Whip eggs in large bowl on high speed until light and fluffy.b. Combine sugars, cocoa, salt, and leavening; add to beaten eggs, and whip on high speed until thick.c. Add shortening slowly while mixing on low speed.d. Scrape bowl and whip on high speed until thick.e. Mix flour, nuts, and flavors together and fold into batter; mix until uniform.f. Pour batter into pan at a rate that will yield uncoated brownies which, when cut such as to meet the dimension requirements specified in 3.4f, will weigh approximately 35 grams each. g. Bake at 3500F until done (45 minutes).h. Cut into 2 inch squares

Inter-related

Can't have one w/o the other

Failing test is a requirement

Passing test valid system spec

ATDD as a Prism

Requirements

Software Development

Exercise What Issues?

Current software dev process

Write down your org’s issues

Groups of 3 to 8

Pass to front

Developers Testers

Customers

The Team

Customer Unit

Product owner

Business analysts

Developer/Tester Unit

Programmers

Testers

Developers Testers

Customers

The Triad

Traditional Value Stream

Elicit&Requirements&

Analyze&Requirements& Design&&

Code& Test&

Why go back?

Deploy&

Requirements

Write Code

The Traditional Approach

Rewrite

“Discussion”

Typically a confrontation &

argument

The demo forces a discussion of the

requirementsAn isolated event,

primarily worked by the customer proxy

Write Code

Requirements

The Agile Approach

Rewrite

Before significant time and $$ is

invested

Discuss

Include the Golden Triad

Dev QA

Customer38

Mistakes?

Misunderstandings

Missed requirements

Mis other

Mistakes

Feedback corrects misunderstandings

Quick feedback is better than slow feedback

Desired Actual

Slow Feedback

Actual

Fast Feedback

Agile Value Stream

Elicit&Requirements&

Analyze&Requirements&With&Tests&

Design&&

Code&With&Tests& Deploy&

Move Testing Forward

Two Types of Testing

To find defects

To prevent defects

Finding Defects

Identifying mistakes

flickr.com/photos/mag3737/444557606

preventing Defects

Saves development time

Saves QA time

Saves money

Author tests (write)

customer, tester, developer together

Connect to the system (automate)

developer

Run the tests (execute)

developer, testers, customers, CI system

Test% write% bind% run%

Two Ways

Code%write% run%bind%

Code%write%

Test First

Write the test first

Write the code after

One molds the other

First in wins

Testing Strategies

Testing Focus

Verification vs

ValidationVerification Are the requirements right Focus of acceptance tests

Validation Have we got the right requirements? Customer really do it this way? How do you test this?

Where to test?

Acceptance Testing

Applica'on*/**Middle*'er*

Core*Business*Logic**

Persistence*

Acceptance Tests

Tests all the way down

Requirement* Requirement**

Test* Test*

Requirement*

Service*Implementa4on*

Customer Acceptance Tests

Developer Acceptance Tests

Service*Implementa4on*

Requires services to implement

Acceptance Test Examples

Temperature

Input in Celsius

Output in Fahrenheit

How long to code?

Temperature

F = C x 9/5 + 32

Temperature

Tests?

Temperature

Test Binning

Celsius' Fahrenheit' Notes'0" 32"100" 212" Needed?"

Celsius' Fahrenheit' Notes'*273.15" *459.67" Precision"

Formula Tests

Precision Tests

Celsius' Fahrenheit' Notes'*273.15" *459.67" 0"Kelvin""*273.151" Error" Below"0"Kelvin""500" 932" Maximum"–"Needed?"

Limit Tests

Temperature

Input in Celsius

Output in Fahrenheit

How long to code?

Is This A Good Spec?

Business Rules

If a Customer Rating is Good and the Order Total is less than or equal $10.00, Then give no discount Otherwise give a 1% discount

If Customer Rating is Excellent, Then give a discount of 1% for any order

If the Order Total is greater than $50.00, Then give a discount of 5%

Rule Table => Test

Another Business Rule

Email coupon codes

If a customer enters the code, provide a 20% discount

Discount an item or the order?

Implementation

Testing script

Program interface

Xunit framework

ATDD framework

Testing Script

Tools for testers

(Usually) GUI based

Testing Script

Log in as a Customer who is rated GoodStart an OrderAdd items totally $10.01Complete order Verify $0.10 discount

Repeat for 10 other similar cases

Program Interface

Code something (CLI or GUI)

Screen

Discount Method

Discount Percentage Screen Customer Type: Good Order Total: 10.01 Percentage: 1 %

C:>DiscountPercentage Good 10.01 Percentage: 1%

Xunit Test

class TestCase { testDiscountPercentageForCustomer() { SomeClass o = new SomeClass()

assertEquals(0, o.computeDiscount(10.0, Good)); assertEquals(1, o.computeDiscount(10.01, Good)); assertEquals(1, o.computeDiscount(50.01, Good)); assertEquals(1, o.computeDiscount(.01, Excellent)); assertEquals(1, o.computeDiscount(50.0, Excellent));

assertEquals(5, o.computeDiscount(50.01, Excellent));

Fit Test

Tables as Requirements and Tests

Discount Rule Customer Rating Order Total Discount Percentage

Good <= $10.00 0% Otherwise 1%

Excellent Any 1% > $50.00 5%

Discount))Test)Order)total) Customer)ra3ng) Discount)percentage?)$10.00% Good% 0%%$10.01% Good% 1%%$50.01% Good% 1%%$.01% Excellent% 1%%$50.00% Excellent% 1%%$50.01% Excellent% 5%%

Requirement

Context

Context Diagram

Process'

Reports

Commands Views

Another Context Diagram

Process'

Another System Shared Database

Commands

Requests

Responses

Stores

Retrievals

Engine Controller

Engine&Controller&

Engine Rotation Speed

Gasoline Flow Control

Spark Timing

Acceleration / De-acceleration

Air Flow Control

Embedded Contexts

Sub$ System Two

Input Output

System One

Internal I/O

Car Context

Drive&sha*&&

Acceleration De-acceleration Movement

Trans-mission&

Engine&& Wheels&

What to test?RPMs?Miles per hour

Software Context

Core or API

Developer Acceptance

Customer Acceptance

Middle-tier

What to test?Elements in an array?Behavior of the system

What To Test

BehaviorActions

Changesvia the API

via the same mechanism your GUI doesNot elements in a data structure

Not fields in the database

Use Cases and Tests

Use Case Template

Name—Identifier to easily reference it by Description—Brief note Actor—Who initiates the use casePre-conditions—What must be true before the use case is initiated Post-conditions—What’s true if the use case successfully executesMain course—Steps that show the sequence of interactions

Example: Part One

Name—Check Out CD.Description—Check out a CD for a customer.Actor—Clerk. Pre-conditions—The customer has an identification. The CD

has an identity. Post-conditions—The CD is recorded as rented. The rental

contract is printed.Main Course:

1. The clerk enters the customer identification and CD identifier into the system.

2. The system records the information.3. The system prints a contract that the customer signs.

Example: Part Two

Exceptions:1a. Customer identification is not recognized.

Clerk repeats step 1. 1b. The customer violates the CD Rental Limit business rule.

The clerk notifies the customer of the violation.The use case is abandoned.

Business Rule:CD Rental Limit

A customer can rent only three CDs at any one time. Alternatives

3a. The printer jams.The clerk fills out the contract by hand. The use case exits.

Acceptance Testing

The Triad create acceptance tests for stories

Tests from general (expected path) ...

... to specific (exceptions)

Examples

Rent a CD (expected path)

Bad Customer ID (Enter it wrong)

CD Rental Limit (already has 3. Try 4.)

Print jam (simulate by removing paper)

Test Anatomy

Test Flow

Verify

Test Flow

Ini$al'System'State'

Setup'(Given))'

Trigger'(When)'

Assert'(Then)'

Final'System'State'and'Output'

Action or Event

ExpectedSystem'State'and'Output'

Given/When/Then

Given = Use case pre-conditions When = Main course (or exceptional course)Then = Use case post-conditions

Given/when/Then

Given (Setup)Customer has ID (initial system state)CD has ID (initial system state)CD is not currently rented (initial system state)

When (Trigger)Clerk checks out CD (action)

Then (Verify)CD recorded as rented (final system state)Rental contract printed (output)

Given/when/Then

GivenCustomer has IDCD has IDCD is not currently rented

WhenClerk checks out CD

Then (Verify)CD recorded as rentedRental contract printed

Context Diagram

Internal or External Persistence

Output is response to change address

Change address Send mail

External repository for address

Input for send mail

Internal or External Persistence

How to test internal persistence? Through another action Through a “see-through” method APIs should be symmetricalAdd/DeleteModify/View

Change address

Internal persistence of address (state)

Send mail

Addressed mail

Exercise

Write questions (tests) for:“copying a file from one directory to another”

Describe in terms of:Given: some stateWhen: copy (file, source_directory,

destination_directory)Then: expected state

Source'Directory'Contents'

Des/na/on'Directory'Contents'

Copy'File''

Des/na/on'Directory'Contents?'

Anything'else?''

X' X' X'

Given When Then (Expected)

Scenario

Example

Copy'File''

Des/na/on'Directory''Contents?'

Source'Directory'Contents?'

Content'Comparison?'

Notes'

X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#

Standard#copy##

X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#

X#overwri?en#

X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##

X#not#overwri?en#

What about modification date and other attributes? Make more columns

Example

Copy'File''

Notes'

Standard#copy##

X#overwri?en#

X#not#overwri?en#

Example

Copy'File''

Notes'

Standard#copy##

X#overwri?en#

X#not#overwri?en#

Example

Copy'File''

Notes'

Standard#copy##

X#overwri?en#

X#not#overwri?en#

Tables

Table-driven requirements can reduce ambiguityTables can be testsDiscovery

Tables bring to light overlooked assumptionsDraws out unstated requirements

Table Types

Calculation

Action

Calculation Table

Calc Table Example

Discount Calculation

Item Total Customer Rating Discount Percentage?

$10.00 Good 0%

!  Exists&(or&should&exist)&–&for&Given&and&Then&

Title Data

Value Name 1 Value Name 2

Value for 1 Value for 2

Another value for 1 Another value for 2

Data Table Example

Customer Data

Name ID

James 007

Maxwell 86

Customer Data Name Begins with="’J"

Name ID

James 007

Text Requirements

User can log in

What are the rules?

Login Grid

User Name Password Result Notes

Action Table

Enter enters data into an entry fieldPress initiates a process, such as a Submit buttonCheck sees if a result is equal to an expected

Action Table Example

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Press Rent

Check Rented True

Can be just “When” or a stand-alone Given-When-Then

When Then

Copy File ExampleCopy%a%file%(happy%path)%%!  Given&a&source&&!  and&target&directory&&&&&&&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F

Copy File

Enter Source Directory A

Enter Target Directory B

Enter File name X

Press Copy

!  When&copy&request&is&made&

&&&&&&

!  Then&file&is&copied&&&&&&&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F,X

The Action

!  Can$drive$a$GUI$

!  Or$a$method$$CopyFile(Directory source, Directory target, Filename file)

Business Rules Test

Usually many

One test per rule

Example

Rates&

Type& Days& Cost?&Regular( 2( $2(Golden(Oldie( 3( $1.50(Hot(Stuff( 6( $12(Hot(Stuff( 50(( IGBTYOT((

CD(Rental(Rates(Regular(($2(/(2(days(plus($1(/(day((Golden(Oldie($1(/(2(days(plus($(.50(/(day(Hot(Stuff(($4(/(2(days(plus($2(/(day(

UI & Business Rules

Email&Valida)on&(Middle/Tier)&

Value&& Valid?&&manager@sam.com, yes,someone@@somewhere.com, no,

!#$%&*+7/=?^_,{|}~@sam.com, yes,

Ajax call to middle-tier

Submit,

Middle-tier test

Alternatives

Can use "script" in tables or separately

System Boundaries

External Interfaces

!  Connec&ons(to(external(systems(need(to(have(test(doubles((mocks)(

!  Random(events(may(need(to(be(simulated((!  Test(doubles(give(repeatability(and(speed((

External system, device, service

Random events

External Interface Example

CD Rental System

Credit Card Processor

Sam’s Bank

Charge Transfer To

Confirmation or Denial Bank

Statement

Example of a Credit Card Test Double

CD Rental System

Credit Card Processor Test Double

Charge

Confirmation or Denial

Example of a Email Test Double

CD Rental System

Email Server Test Double

Confirmation or Denial

Test verifies that email sent to proper address with proper contents

Complex Business Rule and

Separation

Complex Business Rule

Field&One& Field&Two& Field&Three& Field&Four& Result&?&>&20& <&50&or&blank& >=100& Y& Allow&Otherwise&or&blank&

DNC& DNC& DNC& Disallow&

DNC& Otherwise& DNC& DNC& Disallow&DNC& DNC& Otherwise&or&

blank&DNC& Disallow&

DNC& DNC& DNC& N&or&blank& Disallow&

A business rule determines whether a user is allowed to perform a certain operation Fields may contain values or be blank

Field&One& Result?&>&20& Allow&Blank& Disallow&Otherwise& Disallow&

Field&Two& Result?&<&50& Allow&Blank& Allow&Otherwise& Disallow&

Field&Three& Result&?&>=100& Allow&Blank& Disallow&Otherwise& Disallow&

Field&Four& Result?&&Y& Allow&N& Disallow&Blank& Disallow&Otherwise& ??&

Field&One& Field&Two& Field&Three& Field&Four& Result&?&Allow& Allow& Allow& Allow& Allow&Disallow& DNC& DNC& DNC& Disallow&DNC& Disallow& DNC& DNC& Disallow&DNC& DNC& Disallow& DNC& Disallow&DNC& DNC& DNC& Disallow& Disallow&

Google Maps

What can you test w/o a UI?

Google Maps

No UI?

Search phrase => List of places

Place => Position

Zoom Level => List of maps

Position => List what's there

Test Evaluation

WARNING!!

Acceptances tests do NOT replace interactive communication!

They provide focus for the communication

Test Evaluation

Remove redundant redundancyBusiness domain terms

Shared between customer unit and developer/tester units

Avoid lots of input and output columnsBreak into smaller tables

Guidelines

Develop tests & automation separately First, understand Second, automate (if appropriate)

Automate tests for regression Run in continuous integration

Cover 100% of function (if practical)

Separation of Concerns

Separate business rules from how results of business rules are displayed

Separate calculation of a business rule (such as a customer rating) from the use of that business rule (such as to give a discount)

Separate each use case or step in a workflow Separate out validation of an entity from use of entity

Separation and Abstraction

One story together:When the build fails, send an email to the

administrator Separate and abstracted:

When the build fails, generate an alert When an alert is generated, notify the

responsible individual When the responsible individual is notified, use

that individual’s notification preference (email) Test the notification itself works (e.g. email

works)

What is a Good Test?

Fails reliably (for a known reason)

No false positives

No other test covers the same area

NOTE: Tests at the "top" often forced to assume business rules... aim for 1 failure at each level

Use Alternate Tables

Sudoku Puzzle 1 4 7

2 5 8 3 6 9

4 7 1 5 8 2

6 9 3 7 1 4

8 2 5 9 3 6

Sudoku Solution 1 6 5 4 9 8 7 3 2 9 2 4 3 5 7 6 8 1 8 7 3 2 1 6 5 4 9 4 9 8 7 3 2 1 6 5 3 5 7 6 8 1 9 2 4 2 1 6 5 4 9 8 7 3 7 3 2 1 6 5 4 9 8 6 8 1 9 2 4 3 5 7 5 4 9 8 7 3 2 1 6

Create tables in alternative forms, if standard not appropriate

Required(Performance((

Puzzle(Type( Time(to(Solve?( Notes(

Easy% .1%seconds%% See%easy%determina2on%

Impossible% 1%second% No%solu2on%%

Other uses for Tests

Estimation Aids

Degree of Doneness

Process note

Requirement Testable Requirement

Developable Requirement

At least one example for each test

Not schedulable Preferably all examples for all tests

process Note

Code%write%

Write previous iteration

Write within iteration

SCMDevelopers

Unit Tests

Deploy to Dev

Integration Tests

First Look

Continuous Testing

On Demand Deploy to QA

Manual Testing

Not the End.... The Beginning!

Objective Review

Understand ATDD in context of SDLC

Learn how to turn requirements into tests

What is a good acceptance test?

ATDD as a communication vehicle

Our OutlineTerms and definitions

Software development

ATDD examples

Anatomy of a test

Tables

System boundary145

Primary goalsDiscover ambiguous requirements and gaps in requirements

early onCreate a record of business/development understandingGive feedback on quality

Secondary goalsUse acceptance tests as an executable regression testMeasure your progress towards "done” completenessMeasure the complexity of requirements.Use the tests as a basis for user documentation

Convert requirements into tests

Write tests before code

Write tests in grids (spreadsheets)

Never write code without tests

Include the Golden Triad

Atdd half day_new_1_up

Software

Agile Tour 2009 Coding Dojo Kata ATDD

ATDD - Desarrollo Dirigido por Test de Aceptación

Algorithm Theoretical Definition Document (ATDD) …hsaf.meteoam.it/documents/ATDD/SAF_HSAF_ATBD-13_1_0.pdf · IPF Institut für Photogrammetrie und Fernerkundung (of TU-Wien, in

Concordion & ATDD (Lightning Talk)

Proiect Practica ATdd

Whitepaper: Supporting ATDD/BDD with SpecLog, …speclog.net/uploads/ATDD_BDD_SpecLog_SpecFlow_TFS... · Whitepaper: Supporting ATDD/BDD with SpecLog, SpecFlow and Team Foundation

ATDD by Example 2012

ATDD With Robot Framework

Atdd Lab Exercises

ATDD BY Example With Cucumber

ATDD with Behat and Selenium (LDNSE6)

ATDD and Conversations

ATDD Using Robot Framework

ATDD Make Love, Not War @ Agile Thailand

Convergido: TDD + ATDD + BDD + xUnit Patterns + Dependency Injection

ATDD In The Enterprise Context

Raconteur: bringing ATDD to Visual Studio

ATDD with SpecFlow

Acceptance Test Driven Development (ATDD) · ATDD Lineage ATDD comes from: Example driven development (EDD) Behavior-driven development (BDD) Story-driven development (SDD) Domain-driven

Algorithm Theoretical Definition Document (ATDD) for product …hsaf.meteoam.it/documents/ATDD/ATDD-03.pdf · 2014-11-12 · Algorithms Theoretical Definition Document, 31 August