View
4
Download
0
Category
Preview:
Citation preview
1
1 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
It Ain’t Testable Until it’s Tested
Gerard MeszarosIndependent Consultant
CTO of FeedXL.Com
itake2017@gerardm.com
These slides: http://itake2017.testAutomationPatterns.com
2 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Product & I.T.
I.T.
EmbeddedTelecom
My Background
Gerard Meszarositake2017@gerardm.com
•Software developer
•Development manager
•Project Manager
•Software architect
•OOA/OOD Mentor
•Requirements (Use Case) Mentor
•XP/TDD Mentor
•Agile PM Mentor
•Test Automation Consultant & Trainer
•Lean/Agile Coach/Consultant
2
3 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Product & I.T.
I.T.
EmbeddedTelecom
My Background
•Software developer
•Development manager
•Project Manager
•Software architect
•OOA/OOD Mentor
•Requirements (Use Case) Mentor
•XP/TDD Mentor
•Agile PM Mentor
•Test Automation Consultant & Trainer
•Lean/Agile Coach/Consultant Gerard Meszarositake2017@gerardm.com
4 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Key Messages
• Systems are hard to test in the whole
• Better to ensure parts work properly before integrating
• Full TDD is great but hard to achieve
• Component Tests provide more direct value at lower cost
• Especially when dealing with legacy code
3
5 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Outline
• Why is (Automated) Testing Necessary?
• Why is Traditional Test Automation Hard?– Systems diagram without preamble/legend
• Why is TDD Hard? (Optional)– Hard to convince and a process (hard to enforce)
6 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Audience Survey
https://lmsa.site-ym.com/news/221726/Sex-and-Gender-Medical-Education-National-Student-Survey.htm
4
7 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Questions
• Who likes using software?
• Who likes using BUGGY software?
• Why is it buggy?
• How hard is *your* code to test?
• What makes code hard to test?
• Should code be easy to test?
• How can we make our code easier to test?
• Whose job is “quality”?
• What kinds of tests do we need?
• Difference between EDD and BDD?
8 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Johnny’s Report Card
http://fabulesslyfrugal.com/report-card-freebies-for-good-grades/
5
9 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Our Report Card
http://fabulesslyfrugal.com/report-card-freebies-for-good-grades/
Functionality
Performance
Ease of Change
Reliability
10 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Our Report Card
http://fabulesslyfrugal.com/report-card-freebies-for-good-grades/
Functionality
Performance
Ease of Change
Reliability
Report Card
6
11 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
QA as Quality Assessment
12 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Dev
Test & Fix “Battleship”
DevelopmentQuality
Assessment
Requirements
B6! Missed!
Requirements
Test
7
13 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Insanity:
• Doing the same thing over and over again and expecting different results
14 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
The Toyota Way
8
15 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
5 Why’s
www.123rf.com
16 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
5 Why’s of Test Automation Being Hard
Tests need to interact via
the UITest
Automation is Hard!
Product isn’t designed for testability
Why? Why?
Why?
Tests are very complex
9
17 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
5 Why’s of Test Automation Being Hard
Dev doesn’t know how (or care)
Tests need to interact via
the UITest
Automation is Hard!
It’s QA’s problem
Tests are automated after
Dev Complete
QA isn’t involved in product
definition
Product isn’t designed for testability
Why? Why?
Why?
Why?
Why?
Why?
Tests are very complex
Why?
Why?
18 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
5 Why’s of Test Automation Being Hard
Dev doesn’t know how (or care)
Tests need to interact via
the UITest
Automation is Hard!
It’s QA’s problem
Tests are automated after
Dev Complete
QA isn’t involved in product
definition
QA is too busy testing last release
Product isn’t designed for testability
Why? Why?
Why?
Why?
Why?
Why?
Why?
Why?
Tests are very complex
Why?
Why?
QA is supposed to be
“independent”
Why?
10
19 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Systems Thinking
A B+
C-
Notation:+ Increasing- Reducing
20 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Systems Thinking
A B+
C-
+ +
Notation:+ Increasing- Reducing
ReinforcingCycle
BalancingCycle
11
21 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Systems Thinking
A B+
+
Notation:+ Increasing- Reducing
Vicious orVirtuous
Cycle
(Depending on whether A & B
are good or bad)
22 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
5 Why’s of Test Automation Being Hard
Dev doesn’t know how (or care)
Tests need to interact via
the UITest
Automation is Hard!
It’s QA’s problem
Tests are automated after
Dev Complete
QA isn’t involved in product
definition
QA is too busy testing last release
Product isn’t designed for testability
Why? Why?
Why?
Why?
Why?
Why?
Why?
Why?
Tests are very complex
Why?
Why?
QA is supposed to be
“independent”
Why?
12
23 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Test Automation – Systems Thinking Diagram
Dev doesn’t know how (or care)
Tests need to interact via
the UITest
Automation is Hard!
It’s QA’s problem
Tests are automated after
Dev Complete
QA isn’t involved in product
definition
QA is too busy testing last release
Product isn’t designed for testability
+ +
+
+
+
+
+
+
Tests are very complex
+
+
QA is supposed to be
“independent”
+
A Vicious Cycle
24 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Root, Root Cause: Conway’s LawThe Architecture of System will resemble the
Organization that Built it.
Development Team
TestTeam
BusinessTeam
Req’ts Code
13
25 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Resulting Architecture
Specifi-cations
Product Code
Test Code
Bus Test
Dev
26 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Automated Tests are Just Code
That Tests Other Code
Who Tests This Code?Who Should Write This Code?
14
27 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
TDD Is One Solution
But is hard to implement because:
• Requires everyone to do it
• all the time
• And is a large cultural change
And it doesn’t solve the whole problem:
• Ensures “code quality” but not “product quality”
• Can build great software that doesn’t work as a whole
28 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Example-Driven DevelopmentDevelopment flow
Potentially Executable
Specifications
ExampleInterpreter
Code ProductCode
DevTest
DevDevTestBus Test
Expertise or Roles , not Teams!
15
29 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Expertise or Roles , not Teams!
Example-Driven DevelopmentRuntime View:
ProductCode
ExampleInterpreter
Code
ReportCard
Potentially Executable
Specifications
DevTest
DevDevTestBus Test
30 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Expertise or Roles , not Teams!
Example-Driven DevelopmentFeedback Loops:
Potentially Executable
Specifications ProductCode
ExampleInterpreter
Code
ReportCard
What they say
How they say it
Dev
DevTest
DevTestBus Test
16
31 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Expertise or Roles , not Teams!
Example-Driven DevelopmentFeedback Loops:
Potentially Executable
Specifications ProductCode
ExampleInterpreter
Code
ReportCard
What they say
How they say it
DevTest
DevTestBus TestDev
Isn’t This “Just” Test Automation By Another Name?
32 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Tests vs. Checks vs. Examples?ref: http://www.satisfice.com/blog/archives/856
17
33 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Preparing Good Examples
34 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Example: Notifying of Bank Transactions
Configure Notification
Rules
AccountHolder
Process Transaction
TransactionSettlement
Not
ify
base
d on
cha
rge
Type
18
35 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Example: Notifying of Bank Transactions
Configure Notification
Rules
AccountHolder
Process Transaction
TransactionSettlement
Not
ify
base
d on
cha
rge
Type
Not
ify
base
d on
cha
rge
Cont
inen
tN
otif
y ba
sed
on c
harg
e Co
untr
yN
otif
y ba
sed
on c
harg
e Ci
ty &
Sta
te
36 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Example: Notifying of Bank Transactions
Configure Notification
Rules
AccountHolder
Suspend Notification
Resume Notification
Process Transaction
TransactionSettlement
Not
ify
base
d on
cha
rge
Type
Not
ify
base
d on
cha
rge
Cont
inen
tN
otif
y ba
sed
on c
harg
e Co
untr
yN
otif
y ba
sed
on c
harg
e Ci
ty &
Sta
te
Sus
pend
Not
ific
atio
n
19
37 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Checking Notifications• Open MegaBank app
• Log in as “BobMa” with password ******
• Click on “Manage Notifications” tab
• Click on “Add New Rule” button
• Select account “10035692877”
• Type “Default Rule” into field “rule name”
• Type “1000” into field “threshold amount”
• Click on “all transaction types” radio button
• Click on “all locations” radio button
• Click on “save changes” button
• …
• …
38 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Checking Notifications – 1/2Given:
User and Accounts
Another attempt:
When: Notification
Rule is Configured
Then:Notification Rule Configured and
Active
Then:Notification Rule
is Active
20
39 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Checking Notifications – 2/2
Then: ExpectedNotifications
When: The Transactions to be processed
Another attempt:
40 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Test Automation Pyramid
Pyramid originally proposed by Mike Cohn
Exploratory Tests
Unit Tests
ComponentTests (API)
SystemTests (GUI)
21
41 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Behavior Specification at Right Level
ComponentTests (API)
SystemTests (GUI)
Broad Narrow
Det
ail
High
Low
Scope
Unit Tests TDD/BDD
ATDD/EDD
42 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Behavior Specification at Right Level
ComponentTests (API)
SystemTests (GUI)
Broad Narrow
Det
ail
High
Low
Scope
ATDD/EDD
22
43 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Behavior Specification at Right Level
ComponentTests (API)
SystemTests (GUI)
Unit Tests TDD/BDD
ATDD/EDD
44 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Behavior Specification at Right Level
ComponentTests (API)
SystemTests (GUI)
Broad Narrow
Det
ail
High
Low
Scope
Unit Tests TDD/BDD
ATDD/EDD
23
45 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Behavior Specification at Right Level
Broad Narrow
Det
ail
High
Low
Scope
ComponentTests (API)
SystemTests (GUI)
46 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Test Automation Pyramid - Redrawn
ComponentTests (API)
Broad Narrow
Det
ail
High
Low
Scope
Unit TestsTDD/ BDD
ATDD/EDDSystemTests
24
47 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Test Automation Pyramid - Redrawn
Broad Narrow
Det
ail
High
Low
Scope
ComponentTests (API)
SystemTests ATDD/EDD
48 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Behavior Specification at Right Level
Broad Narrow
Det
ail
High
Low
Scope
ComponentTests (API)
SystemTests
25
49 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
ComponentTests (API)
SystemTests
Behavior Specification at Right Level
Rules &Algorithms
Multi-Use Case
Workflows
IncompleteSpec
Toomuch detailUnmaintainable
Det
ail
High
Low
Broad NarrowScope
Transactions(Use Cases)
50 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Toomuch detailUnmaintainable
Changing Level of Abstraction/Detail
Rules &Algorithms
Multi-Use Case
WorkflowsTransactions(Use Cases)
IncompleteSpec
Broad Narrow
Det
ail
High
L
ow
Scope
26
51 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Refactoring Workflow ExampleGiven:
User and Accounts
When: Notification
Rule is Configured
Then:Notification Rule Configured and
Active
Then:Notification Rule
is Active
52 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Refactoring Workflow Example
“If it isn’t essential to conveying the essence of the behavior, it is essential to
not include it.”
27
53 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Then: ExpectedNotifications
When: The Transactions to be processed
Refactoring Workflow Example
54 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Refactoring Workflow Example
28
55 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Refactoring Workflow Example
56 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Refactoring Workflow ExampleGiven: User &
Thresholds
When: Transactions
Are Processed
Then: We Expect
Notifications
29
57 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
9991113333
Given: User &
Thresholds
When: Transactions
Are Processed
Then: We Expect
Notifications
credit
credit
Refactoring Workflow Example
58 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
9991113333
Refactoring Workflow Example
Broad Scope (Multi-Actor);
Minimum Detail (per Actor/Transaction);
Given: User &
Thresholds
When: Transactions
Are Processed
Then: We Expect
Notifications
credit
credit
30
59 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Filling in the Details
Rules &Algorithms
Multi-Use Case
WorkflowsTransactions(Use Cases)
IncompleteSpec
Too much detailUnmaintainableBroad Narrow
Det
ail
High
Low
Scope
60 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Business Rule Example
Process Transaction
Threshold per Charge Type
Configuration
31
61 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Process Transaction
Threshold per Charge Type
Narrow Scope (Single Rule)
High Detail (Everything that matters)
Example:
Configuration Given these rules
Then: The answer should be
Business Rule ExampleWhen we ask
NotificationRequired? with this transaction:
62 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Life Cycle of an Example
User Goal
Feature
StoryTitle
Story Narrative
Story Scenarios
Story Examples
Executable Examples
Satisfied Examples
FormalizeAutomate
DevelopProduct
Make ConcreteAdd Data
DefineAcceptance
Criteria
WorkBreakdown
Product Design & Validation
Elaborate
Each
32
63 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Who Does Activities on an Example
User Goal
Feature
StoryTitle
Story Narrative
Story Scenarios
Story Examples
Executable Examples
Satisfied Examples
FormalizeAutomate
DevelopProduct
Make ConcreteAdd Data
DefineAcceptance
Criteria
WorkBreakdown
Product Design & Validation
Elaborate
DevTestBus
DevTest Bus
DevTest
Bus
Each
64 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Test-After Architecture
System Under Test
ProcessTransaction
ConfigureNotificationThreshold
TransactionInterface
ConfigurationUser
Interface Should weNotify
andDo
Notification?
NotificationRules
NotificationLog
Bus
Workflow Test
Test Dev
! ! ! !
33
65 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
TransactionInterface
ConfigurationUser
Interface
System Under TestConfigure
NotificationThreshold
Example-Driven ArchitectureWorkflow Examples Drive API
Workflow Example
Dev
Workflow Keyword
Interpreter
Bus
Test
ProcessTransaction
Process Transaction (including:
Do We Notify and How to Notify)
Dev
66 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
TransactionInterface
ConfigurationUser
Interface
System Under TestConfigure
NotificationThreshold
Example-Driven Architecture... but not Component Structure
Workflow Example
Dev
Workflow Keyword
Interpreter
Bus
Test Dev
NotificationMethod Example
NotificationRule Example Process
Transaction
System Under Test
34
67 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Example-Driven ArchitectureRules Examples Drive Component Dev’t
Bus
Test
NotificationMethod Example
NotificationRule Example
System Under Test
68 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
System Under Test
Example-Driven ArchitectureRules Examples Drive Component Dev’t
DoNotification.
NotificationLog
NotificationMethod Example
NotificationRule Example
Should weNotify?
ExampleInterpreter
ExampleInterpreter
DevBus
TestDev
NotificationRules
35
69 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
NotificationRules
Example-Driven ArchitectureRules Examples Ensure Testability
Should weNotify?
NotificationRules Fake
NotificationRule Example
ExampleInterpreter
DevBus
TestDev
70 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
System Under Test
Example-Driven ArchitectureRules Examples Ensure Testability
DoNotification.
NotificationRules
NotificationLog
NotificationMethod Example
Bus
TestDev
NotificationRules
NotificationRules Fake
NotificationLog Fake
NM ExampleInterpreter
Dev
36
71 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
System Under Test
Example-Driven ArchitectureUnit Tests Fill in Gaps in Coverage
DoNotification.
NotificationRules
NotificationLog
NotificationMethod Example
Bus
TestDev
NotificationRules
NotificationRules Fake
NotificationLog Fake
NM ExampleInterpreter
Dev
UnitTestsUnit
TestsUnitTestsUnit
Tests
72 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
System Under Test
Putting it All Together
DoNotification.
ProcessTransaction
NotificationRules
NotificationLog
NotificationMethod Example
NotificationRules Fake
NotificationRule Example
Should weNotify?
ExampleInterpreter
ExampleInterpreter
DevBus
Test Dev
UnitTestsUnit
TestsUnitTestsUnit
Tests
TransactionInterface
ConfigurationUser
Interface
ConfigureNotificationThreshold
Workflow Example
Workflow Keyword
Interpreter
37
73 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
tools
Tools
As adapted by Gerard Meszaros
74 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Tools (My Prefs Only)
38
75 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
In Conclusion
76 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Test Automation Pyramid - Redrawn
BusinessRule Examples
Broad Narrow
Det
ail
High
Low
Scope
Unit Testsor Specs (BDD)
TDD/ BDD
ATDD/EDDWorkflowExamples
39
77 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
After State – Back to Our Systems Diagram
Dev doesn’t know how (or care)
Tests need to interact via
the UITest
Automation is Hard!
It’s QA’s problem
Tests are automated after
Dev Complete
QA isn’t involved in product
definition
QA is too busy testing last release
Product isn’t designed for testability
+ +
+
+
+
+
+
+
Tests are very complex
+
+
QA is supposed to be
“independent”
+
X
XX
X X
XXX
78 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
After State – Back to Our Systems Diagram
Dev knowshow (and cares)
Examples don’t need to interact via
the UI
Automated checking is
Easy!
Automation is Dev’sproblem
Examples are automated before Dev
starts
QA isinvolved in product
definition
QA has time for Examples and Exploratory
testing
Product isdesigned for testability
+ +
+
+
+
+
+
+
Examples are simple
+
+
A Virtuous Cycle
40
79 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Benefits• Defining Examples first helps us understand
– What the system should do
– How it should be structured to ease testability
• Using business language for examples– Reduces opportunities for “lost in translation”
• Developers responsible for automation– Ensures the design is testable
– Provides immediate feedback on function & testability.
– Reduces duplication with unit tests
• Applicable to new code *and* legacy code– Start with Workflow or Component tests, not unit tests
80 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Thank You!Gerard Meszaros
itake2017@gerardm.com http://www.xunitpatterns.com
Slides: http://itake2017.testAutomationpatterns.com
Call me when you:• Want to transition to Agile or Lean• Want to do Agile or Lean better• Want to teach developers how to test• Need help with test automation strategy• Want to improve your test automation
Jolt Productivity Award winner - Technical Books
Available on MSDN:
41
81 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Tools Selection
• Tools must support the strategy– Support business/testers to write specs/tests in
business language» More or less “natural” language
– Let them write specs before the specs are automated or the system is built» This disqualifies Capture & Replay tools
– Can be automated by technical people, later» Separation of business skills from technical skills
» e.g. Table or Keyword-based tools
– Can be run by anyone, especially developers» This disqualifies expensive tester-specific tools
82 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Business Rules Tools - A Sampling
• Fit– Column Fixture
• SLIM (Fitnesse) – Decision Table
• Robot Framework – Parameterized Testcase
42
83 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
Workflow Spec Tools (my fav’s)
• Gherkin (Given-When-Then terminology)– Cucumber (various tech-specific implementations)
– JBehave (Java/JVM specific)
– SpecFlow (.Net)
• Other GWT (Given-When-Then terminology)– Robot Framework GWT
» https://blog.codecentric.de/en/2009/11/givenwhenthen-and-example-tables-using-the-robot-framework/
• Robot Framework– Keyword-based Testcases
Resource: Agile Alliance Functional Testing Tools program
84 Copyright 2017 Gerard Meszaroshttp://itake2017.testAutomationPatterns.com
More Complete Survey of Tools
• Agile Alliance Functional Testing Tools program Yahoo! group– https://groups.yahoo.com/neo/groups/aa-ftt/info
• Agile Alliance Functional Testing Tools spreadsheet– https://docs.google.com/spreadsheet/ccc?key=0Apag_J
97l3CTdHR0WS1sZGFVaFA0dEpXYWRqLXBxV3c#gid=1
Recommended