Atdd half day_new_1_up

ATDDAcceptanceTest Driven Development

J a r e d R i c h a r d s o n A g i l e A r t i s a n s

@ J a r e d R i c h a r d s o n

1

Who are you?

2

Jared RichardsonCoAuthored GROWS Methodology

Author Ship It! and Career 2.0

Screencast editor PragProg.com

2nd public signatory of the Agile Manifesto

Started AgileRTP in 2007!

Coach

AgileArtisans.com

3

Who are you?

4

What would you say you do here?

Roles?

Responsibilities?

Why are you here?

What are your tests for this session?

5

ObjectivesDefine ATDD

Why requirements are not enough

Who to involve when

Turning requirements into tests

What to test

How ATDD can make your entire company more productive

6

Outline

Terms and definitions

Requirements and tests

Software development

ATDD examples

Anatomy of a test

Tables (Grid Driven Development)

7

Take Aways

What’s a test?

Who writes them?

When are they written?

What do they verify?

Provide an example

Primary benefit?

Secondary benefits?

8

Breaks

Halfway (90 minutes)

9

Background

Lean-Agile Acceptance Test Driven Development

by Ken Pugh

10

One Rule

There are exceptions to every statement

Except this one

11

Start

Familiar

Working

Smooth

Teaching

Dreyfus ModelofSkillsAcquisition

en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition

12

Start

Familiar

Working

Smooth

Teaching

Dreyfus ModelofSkillsAcquisition

Steps

Recipes

Intuition

Stuck

Principles

13

Key Points

Skills don’t map

dilbert.com/strip/1990-09-13

14

Key Points

Beginners need steps

15

What is ATDD?

Acceptance Test Driven Development

External view of the system

Examines visible effects

Inputs

Outputs

State changes

External interfaces16

Definitions

Acceptance criteria -> General idea

Acceptance test -> specific. Pass/fail.

Implementation independent

Triad -> Customer, developer, tester

17

Fast Car

Who wants one?

Criteria

Closed course. Measure acceleration.

Test

Accelerate 0 to 60 in X seconds

flickr.com/photos/vodcars/4132650949

18

Fast Car

flickr.com/photos/fordeu/14296246942

0 to 60 in 30 seconds

0 to 60 in 5 seconds.

Top speed is 61.

0 to 60 in 5. Top speed is 120.

60 to 120 in 5 min

0 to 60 in 5. 60 to 120 in 20 sec.

Range of 1/4 mile.

19

0 to 60 in 30 seconds

0 to 60 in 5 seconds.

Top speed is 61.

0 to 60 in 5. Top speed is 120.

60 to 120 in 5 min

0 to 60 in 5. 60 to 120 in 20 sec.

Range of 1/4 mile.

Range of 1,000 miles. Cost 7 billion

Fast Car

flickr.com/photos/alpi-costerni/8161111001

20

Why ATDD?Rework down from 60% to 20%

Workflows work 1st time

Little room miscommunication

Saves time

Getting business rules RIGHT

Tighter cross functional team integration

Crisp visible story completion criteria

Automation yields reduced testing time21

Requirements and Tests

22

Requirement

Needed by a user to solve a problem

Needed by a system to satisfy contract, standard, spec, regulation...

23

Requirements Include

Functions (what software does)

Constraints (limits)

24

Requirements Should not

Include

Implementation details

Specific manifestation

25

A Brownie Spec

Ingredient Parts by weight

Sugar 23.0Flour 21.0Shortening 16.8 Nuts 16.0whole eggs 13.0Cocoa 5.5Dextrose, anhydrous 4.4Salt

26

Brownie Spec

a. Whip eggs in large bowl on high speed until light and fluffy.b. Combine sugars, cocoa, salt, and leavening; add to beaten eggs, and whip on high speed until thick.c. Add shortening slowly while mixing on low speed.d. Scrape bowl and whip on high speed until thick.e. Mix flour, nuts, and flavors together and fold into batter; mix until uniform.f. Pour batter into pan at a rate that will yield uncoated brownies which, when cut such as to meet the dimension requirements specified in 3.4f, will weigh approximately 35 grams each. g. Bake at 3500F until done (45 minutes).h. Cut into 2 inch squares

27


Inter-related

Can't have one w/o the other

Failing test is a requirement

Passing test valid system spec

28

ATDD as a Prism

Requirements

Tests

ATDD

29

Software Development

30

Exercise What Issues?

Current software dev process

Write down your org’s issues

Groups of 3 to 8

Pass to front

31

Developers Testers

Customers

The Team

32

Customer Unit

Product owner

Business analysts

Users

QA

33

Developer/Tester Unit

Programmers

Testers

QA

34

Developers Testers

Customers

The Triad

35

Traditional Value Stream

Elicit&Requirements&

Analyze&Requirements& Design&&

Code& Test&

Why go back?

Deploy&

36

Requirements

Write Code

The Traditional Approach

Demo

Rewrite

“Discussion”

Typically a confrontation &

argument

The demo forces a discussion of the

requirementsAn isolated event,

primarily worked by the customer proxy

37

Demo

Write Code

Requirements

The Agile Approach

Rewrite

Before significant time and $$ is

invested

Discuss

Include the Golden Triad

Dev QA

Customer38

Mistakes?

Misunderstandings

Missed requirements

Mis other

39

Mistakes

Feedback corrects misunderstandings

Quick feedback is better than slow feedback

40

Desired Actual

Slow Feedback

Actual

Fast Feedback

41

Agile Value Stream

Elicit&Requirements&

Analyze&Requirements&With&Tests&

Design&&

Code&With&Tests& Deploy&

42

Move Testing Forward

43

Two Types of Testing

To find defects

To prevent defects

44

Finding Defects

Waste

Identifying mistakes

flickr.com/photos/mag3737/444557606

45

preventing Defects

Saves development time

Saves QA time

Saves money

46

Steps

Author tests (write)

customer, tester, developer together

Connect to the system (automate)

developer

Run the tests (execute)

developer, testers, customers, CI system

Test% write% bind% run%

47

Two Ways

Code%write% run%bind%

Code%write%

run%

bind%

48

Test First

Write the test first

Write the code after

One molds the other

First in wins

49

Testing Strategies

Meszaros, XUNIT TEST PATTERNS: REFACTORING TEST CODE, Fig 6.1 "Purpose of Tests" p. 51, © 2007 Pearson Education, Inc

50

Testing Focus

51

Verification vs

ValidationVerification Are the requirements right Focus of acceptance tests

Validation Have we got the right requirements? Customer really do it this way? How do you test this?

52

Where to test?

53

Acceptance Testing

Applica'on*/**Middle*'er*

Core*Business*Logic**

UI*

Persistence*

Acceptance Tests

54

Tests all the way down

Requirement* Requirement**

Test* Test*

Requirement*

Test*

Test*

Service*Implementa4on*

Test*

Customer Acceptance Tests

Developer Acceptance Tests

Service*Implementa4on*

Requires services to implement

55

Acceptance Test Examples

56

Temperature

Input in Celsius

Output in Fahrenheit

How long to code?

57

Temperature

F = C x 9/5 + 32

58

Temperature

Tests?

59

Temperature

60

Test Binning

Celsius' Fahrenheit' Notes'0" 32"100" 212" Needed?"

Celsius' Fahrenheit' Notes'*273.15" *459.67" Precision"

Formula Tests

Precision Tests

Celsius' Fahrenheit' Notes'*273.15" *459.67" 0"Kelvin""*273.151" Error" Below"0"Kelvin""500" 932" Maximum"–"Needed?"

Limit Tests

61

Temperature

Input in Celsius

Output in Fahrenheit

How long to code?

62

Is This A Good Spec?

63

Business Rules

If a Customer Rating is Good and the Order Total is less than or equal $10.00, Then give no discount Otherwise give a 1% discount

If Customer Rating is Excellent, Then give a discount of 1% for any order

If the Order Total is greater than $50.00, Then give a discount of 5%

64

Rule Table => Test

?

?

65

Another Business Rule

Email coupon codes

If a customer enters the code, provide a 20% discount

Discount an item or the order?

66

Implementation

Testing script

Program interface

Xunit framework

ATDD framework

67

Testing Script

Tools for testers

(Usually) GUI based

68

Testing Script

Log in as a Customer who is rated GoodStart an OrderAdd items totally $10.01Complete order Verify $0.10 discount

Repeat for 10 other similar cases

69

Program Interface

Code something (CLI or GUI)

Screen

Discount Method

Discount Percentage Screen Customer Type: Good Order Total: 10.01 Percentage: 1 %

C:>DiscountPercentage Good 10.01 Percentage: 1%

70

Xunit Test

class TestCase { testDiscountPercentageForCustomer() { SomeClass o = new SomeClass()

assertEquals(0, o.computeDiscount(10.0, Good)); assertEquals(1, o.computeDiscount(10.01, Good)); assertEquals(1, o.computeDiscount(50.01, Good)); assertEquals(1, o.computeDiscount(.01, Excellent)); assertEquals(1, o.computeDiscount(50.0, Excellent));

assertEquals(5, o.computeDiscount(50.01, Excellent));

} }

71

Fit Test

72

Tables as Requirements and Tests

Discount Rule Customer Rating Order Total Discount Percentage

Good <= $10.00 0% Otherwise 1%

Excellent Any 1% > $50.00 5%

Discount))Test)Order)total) Customer)ra3ng) Discount)percentage?)$10.00% Good% 0%%$10.01% Good% 1%%$50.01% Good% 1%%$.01% Excellent% 1%%$50.00% Excellent% 1%%$50.01% Excellent% 5%%

Requirement

Test

73

Context

74

Context Diagram

Process'

User

Reports

Commands Views

75

Another Context Diagram

Process'

User

Another System Shared Database

Commands

Views

Requests

Responses

Stores

Retrievals

76

Engine Controller

Engine&Controller&

Engine Rotation Speed

Gasoline Flow Control

Spark Timing

Acceleration / De-acceleration

Air Flow Control

77

Embedded Contexts

Sub$ System Two

Input Output

System One

Internal I/O

78

Car Context

Drive&sha*&&

Acceleration De-acceleration Movement

Trans-mission&

Engine&& Wheels&

&

What to test?RPMs?Miles per hour

79

Software Context

Core or API

Developer Acceptance

Test

Customer Acceptance

Test

Middle-tier

GUI

What to test?Elements in an array?Behavior of the system

80

What To Test

BehaviorActions

Changesvia the API

via the same mechanism your GUI doesNot elements in a data structure

Not fields in the database

81

Use Cases and Tests

82

Use Case Template

Name—Identifier to easily reference it by Description—Brief note Actor—Who initiates the use casePre-conditions—What must be true before the use case is initiated Post-conditions—What’s true if the use case successfully executesMain course—Steps that show the sequence of interactions

83

Example: Part One

Name—Check Out CD.Description—Check out a CD for a customer.Actor—Clerk. Pre-conditions—The customer has an identification. The CD

has an identity. Post-conditions—The CD is recorded as rented. The rental

contract is printed.Main Course:

1. The clerk enters the customer identification and CD identifier into the system.

2. The system records the information.3. The system prints a contract that the customer signs.

84

Example: Part Two

Exceptions:1a. Customer identification is not recognized.

Clerk repeats step 1. 1b. The customer violates the CD Rental Limit business rule.

The clerk notifies the customer of the violation.The use case is abandoned.

Business Rule:CD Rental Limit

A customer can rent only three CDs at any one time. Alternatives

3a. The printer jams.The clerk fills out the contract by hand. The use case exits.

85

Acceptance Testing

The Triad create acceptance tests for stories

Tests from general (expected path) ...

... to specific (exceptions)

86

Examples

Rent a CD (expected path)

Bad Customer ID (Enter it wrong)

CD Rental Limit (already has 3. Try 4.)

Print jam (simulate by removing paper)

87

Test Anatomy

88

Test Flow

Setup

Use

Verify

Setup

Use

Verify

Test Flow

89

Test Flow

Ini$al'System'State'

Test'

Setup'(Given))'

Trigger'(When)'

Assert'(Then)'

Final'System'State'and'Output'

Action or Event

ExpectedSystem'State'and'Output'

90

Given/When/Then

Given = Use case pre-conditions When = Main course (or exceptional course)Then = Use case post-conditions

91

Given/when/Then

Given (Setup)Customer has ID (initial system state)CD has ID (initial system state)CD is not currently rented (initial system state)

When (Trigger)Clerk checks out CD (action)

Then (Verify)CD recorded as rented (final system state)Rental contract printed (output)

92

Given/when/Then

GivenCustomer has IDCD has IDCD is not currently rented

WhenClerk checks out CD

Then (Verify)CD recorded as rentedRental contract printed

93

Context Diagram

94

Internal or External Persistence

Output is response to change address

Change address Send mail

External repository for address

Input for send mail

95

Internal or External Persistence

How to test internal persistence? Through another action Through a “see-through” method APIs should be symmetricalAdd/DeleteModify/View

Change address

Internal persistence of address (state)

Send mail

Addressed mail

96

Exercise

Write questions (tests) for:“copying a file from one directory to another”

Describe in terms of:Given: some stateWhen: copy (file, source_directory,

destination_directory)Then: expected state

Source'Directory'Contents'

Des/na/on'Directory'Contents'

Copy'File''

Des/na/on'Directory'Contents?'

Anything'else?''

X' X' X'

Given When Then (Expected)

97

Scenario

B

A X

Copy%

B X

A X

98

Example



Copy'File''

Des/na/on'Directory''Contents?'

Source'Directory'Contents?'

Content'Comparison?'

Notes'

X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#

Standard#copy##

X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#

X#overwri?en#

X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##

X#not#overwri?en#

What about modification date and other attributes? Make more columns


99

Example



Copy'File''




Notes'


Standard#copy##


X#overwri?en#


X#not#overwri?en#



100

Example



Copy'File''




Notes'


Standard#copy##


X#overwri?en#


X#not#overwri?en#



101

Example



Copy'File''




Notes'


Standard#copy##


X#overwri?en#


X#not#overwri?en#



102

Tables

103

Tables

Table-driven requirements can reduce ambiguityTables can be testsDiscovery

Tables bring to light overlooked assumptionsDraws out unstated requirements

104

Table Types

Calculation

Action

Data

105

Calculation Table

106

Calc Table Example

Discount Calculation

Item Total Customer Rating Discount Percentage?

$10.00 Good 0%

Given

When

Then

107

Data

!  Exists&(or&should&exist)&–&for&Given&and&Then&

Title Data

Value Name 1 Value Name 2

Value for 1 Value for 2

Another value for 1 Another value for 2

108

Data Table Example

Customer Data

Name ID

James 007

Maxwell 86

Customer Data Name Begins with="’J"

Name ID

James 007

109

Text Requirements

User can log in

What are the rules?

110

Login Grid

User Name Password Result Notes

111

Action Table

Enter enters data into an entry fieldPress initiates a process, such as a Submit buttonCheck sees if a result is equal to an expected

value

112

Action Table Example

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Press Rent

Check Rented True

Can be just “When” or a stand-alone Given-When-Then

Given

When Then

113

Copy File ExampleCopy%a%file%(happy%path)%%!  Given&a&source&&!  and&target&directory&&&&&&&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F

Copy File

Enter Source Directory A

Enter Target Directory B

Enter File name X

Press Copy

!  When&copy&request&is&made&

&&&&&&

!  Then&file&is&copied&&&&&&&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F,X

114

The Action

!  Can$drive$a$GUI$

!  Or$a$method$$CopyFile(Directory source, Directory target, Filename file)

115

Business Rules Test

Usually many

One test per rule

116

Example

Rates&

Type& Days& Cost?&Regular( 2( $2(Golden(Oldie( 3( $1.50(Hot(Stuff( 6( $12(Hot(Stuff( 50(( IGBTYOT((

CD(Rental(Rates(Regular(($2(/(2(days(plus($1(/(day((Golden(Oldie($1(/(2(days(plus($(.50(/(day(Hot(Stuff(($4(/(2(days(plus($2(/(day(

117

UI & Business Rules

Email&Valida)on&(Middle/Tier)&

Value&& Valid?&&[email protected], yes,someone@@somewhere.com, no,

!#$%&*+7/=?^_,{|}[email protected], yes,

Email

Ajax call to middle-tier

Submit,

Middle-tier test

118

Alternatives

Can use "script" in tables or separately

119

System Boundaries

120

External Interfaces

!  Connec&ons(to(external(systems(need(to(have(test(doubles((mocks)(

!  Random(events(may(need(to(be(simulated((!  Test(doubles(give(repeatability(and(speed((

External system, device, service

Random events

Time

121

External Interface Example

CD Rental System

Credit Card Processor

Sam’s Bank

Charge Transfer To

Confirmation or Denial Bank

Statement

122

Example of a Credit Card Test Double

CD Rental System

Credit Card Processor Test Double

Charge

Confirmation or Denial

123

Example of a Email Test Double

CD Rental System

Email Server Test Double

Email

Confirmation or Denial

Test verifies that email sent to proper address with proper contents

124

Complex Business Rule and

Separation

125

Complex Business Rule

Field&One& Field&Two& Field&Three& Field&Four& Result&?&>&20& <&50&or&blank& >=100& Y& Allow&Otherwise&or&blank&

DNC& DNC& DNC& Disallow&

DNC& Otherwise& DNC& DNC& Disallow&DNC& DNC& Otherwise&or&

blank&DNC& Disallow&

DNC& DNC& DNC& N&or&blank& Disallow&

A business rule determines whether a user is allowed to perform a certain operation Fields may contain values or be blank

126


Field&One& Result?&>&20& Allow&Blank& Disallow&Otherwise& Disallow&

Field&Two& Result?&<&50& Allow&Blank& Allow&Otherwise& Disallow&

Field&Three& Result&?&>=100& Allow&Blank& Disallow&Otherwise& Disallow&

Field&Four& Result?&&Y& Allow&N& Disallow&Blank& Disallow&Otherwise& ??&

127


Field&One& Field&Two& Field&Three& Field&Four& Result&?&Allow& Allow& Allow& Allow& Allow&Disallow& DNC& DNC& DNC& Disallow&DNC& Disallow& DNC& DNC& Disallow&DNC& DNC& Disallow& DNC& Disallow&DNC& DNC& DNC& Disallow& Disallow&

128

Google Maps

What can you test w/o a UI?

129

Google Maps

No UI?

Search phrase => List of places

Place => Position

Zoom Level => List of maps

Position => List what's there

130

Test Evaluation

131

WARNING!!

Acceptances tests do NOT replace interactive communication!

They provide focus for the communication

132

Test Evaluation

Remove redundant redundancyBusiness domain terms

Shared between customer unit and developer/tester units

Avoid lots of input and output columnsBreak into smaller tables

133

Guidelines

Develop tests & automation separately First, understand Second, automate (if appropriate)

Automate tests for regression Run in continuous integration

Cover 100% of function (if practical)

134

Separation of Concerns

Separate business rules from how results of business rules are displayed

Separate calculation of a business rule (such as a customer rating) from the use of that business rule (such as to give a discount)

Separate each use case or step in a workflow Separate out validation of an entity from use of entity

135

Separation and Abstraction

One story together:When the build fails, send an email to the

administrator Separate and abstracted:

When the build fails, generate an alert When an alert is generated, notify the

responsible individual When the responsible individual is notified, use

that individual’s notification preference (email) Test the notification itself works (e.g. email

works)

136

What is a Good Test?

Fails reliably (for a known reason)

No false positives

No other test covers the same area

NOTE: Tests at the "top" often forced to assume business rules... aim for 1 failure at each level

137

Use Alternate Tables

Sudoku Puzzle 1 4 7

2 5 8 3 6 9

4 7 1 5 8 2

6 9 3 7 1 4

8 2 5 9 3 6

Sudoku Solution 1 6 5 4 9 8 7 3 2 9 2 4 3 5 7 6 8 1 8 7 3 2 1 6 5 4 9 4 9 8 7 3 2 1 6 5 3 5 7 6 8 1 9 2 4 2 1 6 5 4 9 8 7 3 7 3 2 1 6 5 4 9 8 6 8 1 9 2 4 3 5 7 5 4 9 8 7 3 2 1 6

Create tables in alternative forms, if standard not appropriate

Required(Performance((

Puzzle(Type( Time(to(Solve?( Notes(

Easy% .1%seconds%% See%easy%determina2on%

Impossible% 1%second% No%solu2on%%

138

Other uses for Tests

Estimation Aids

Degree of Doneness

139

Process note

Requirement Testable Requirement

Developable Requirement

At least one example for each test

Not schedulable Preferably all examples for all tests

140

process Note

Code%write%

run%

bind%

Code%write%

run%

bind%

Write previous iteration

Write within iteration

141

SCMDevelopers

Watch

Build

Unit Tests

Deploy to Dev

Integration Tests

First Look

Continuous Testing

On Demand Deploy to QA

Manual Testing

142

Not the End.... The Beginning!

143

Objective Review

Understand ATDD in context of SDLC

Learn how to turn requirements into tests

What is a good acceptance test?

ATDD as a communication vehicle

144

Our OutlineTerms and definitions


Software development

ATDD examples

Anatomy of a test

Tables

System boundary145

Recap

Primary goalsDiscover ambiguous requirements and gaps in requirements

early onCreate a record of business/development understandingGive feedback on quality

Secondary goalsUse acceptance tests as an executable regression testMeasure your progress towards "done” completenessMeasure the complexity of requirements.Use the tests as a basis for user documentation

146

Rules

Convert requirements into tests

Write tests before code

Write tests in grids (spreadsheets)

Never write code without tests

Include the Golden Triad

147

Further Reading

GROWS Method http://GrowsMethod.com

BDD Behavior Driven Development (by Dan North)

Cucumber http://cukes.info

Tellurim (hosted Cucumber) http://te52.com

Junit http://jaredrichardson.net/articles/junit-tutorial.html

FitNesse fitnesse.org

148

Exercise

You listed your development issues...

Would ATDD help, hurt, or be neutral

Why?

Write down your answer and pass it up

149

Take Aways

What’s a test?

Who writes them?

When are they written?

What do they verify?

Provide an example

Primary benefit?

Secondary benefits?

150

151

151

152

GrowsMethod.com

AgileArtisans.com

[email protected]@GrowsMethod.com

152