Embed Size (px)
Citation preview
BDD, TDD, and ATDD for the EnterpriseAdam Satterfield, VP of QA, BetterCloudKevin Dunne, VP of Business Development, QASymphony
“Absorb what is useful, discard what is useless and add what is specifically your own.”
— Bruce Lee
AGENDA• Understanding typical software development and testing challenges
• Introduce test-first methodologies• Benefits of implementing test-first methodologies• Review the state of test-first methodologies in industry• Investigate BetterCloud’s successful implementation of ATDD• Q&A
Current Problems in Software Dev & Test
#qualityjam16
Where We Came From
Traditional development cycles often model the Rational Unified Process:
Source: http://www.psa-software.com/_img/_knowledge_center/rup.jpg
#qualityjam16
The Broken Game of Telephone
Running the processes in parallel introduces the risk as the requirements get handed off multiple times and customer expectations change with time:
Source: http://lh5.ggpht.com/_g0-GZzIBNms/SloJ3LOGy3I/AAAAAAAAAK0/FvyLacg_Q28/s800/conversations.pngg
#qualityjam16
Why QA Breaks Down in Agile
• QA kept out of the loop• QA does not complete their acceptance tests until after the sprint had started
• Dev and QA out of sync• Lack of understanding or inability to qualify when QA is “Done” with testing.
#qualityjam16
Why We Chose It
We would have obviously chosen a more efficient process, but we were constrained by many limitations, including:
Environment Creation Code Merges On-Premise Prevalence
Desktop Focus Lack of Collaboration Off-Shore Development
#qualityjam16
What’s Changed
Many of our prior limitations have been replaced, based on macro trends around technology and industry:
Containers have simplified the process dramaticallyEnvironment Creation
Git has replaced Subversion as the industry standardCode Merges
Cloud adoption is at an all time high, increased securityOn-Premise Prevalence
Prevalence of Web, Mobile, Internet of thingsDesktop Focus
Increase in teamwork, chat and collaboration technologyLack of Collaboration
Shifts towards rural sourcing, onshoring of laborOff-Shore Development
Introducing Test-First Methodologies
#qualityjam16
How the Process Has Adapted
Now that we have freed ourselves of past limitations, the process has been shifted to one that aligns more with our needs:
Traditional Approach
Test-First Approach
Design Requirements Code Test Deploy
Design (Automated) Test Code Refactor Deploy
#qualityjam16
TDD vs. ATDD vs. BDD
Test-First methodologies were coined “Test Driven Development”. Less technically focused versions called Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD) also emerged:
Test Driven Development (TDD)
Behavior Driven Development BDD
Acceptance Test Driven Development (ATDD)
Unit Test Driven Development (“Technical
TDD”)
#qualityjam16
What’s the Difference?
ATDD and BDD are similar in that they both try to make TDD more accessible to business users. The major functional difference comes down to how the tests are structured:
"I think this definition leaves out a key piece, we are focusing on collaboration and learning. Having worked on a project that was using 'ATDD', in 2005 I think, we had the same goals then as BDD without
the Given When Then language.“
— Wes Williams
#qualityjam16
Pros and Cons
ATDD/BDD offer benefits over more Unit Focused/Technical TDD, but also has its drawbacks:
Pros• Increased understanding of tests from business stakeholders
• Increased collaboration early in the cycle• More focus on customer and business needs
• Higher involvement of business in development and quality
Cons• Addition of more tooling in the development/delivery chain
• Greater time spent defining tests and specifications
• Demands stronger contributors in requirements, dev, test
• Often increases the automation needs in an organization
#qualityjam16
User Stories – Sprint Planning
The user story is the “What.” Allows the business to explain a business need while leaving development to choose the best technical approach
Example:As a paying customer,I want to have the ability to update my billing information on my recurring subscription,So that I can keep my subscription current if my information changes.
#qualityjam16
Acceptance Criteria
Acceptance criteria are how we define “done.” They’re measures for how a system should perform. They:
• Define the system behavior.• Ensure features work as expected.• Provide the team a better gauge of the amount of work needed to complete a story.• Guide Development and QA testing.
#qualityjam16
Acceptance Criteria Example
Examples based on user story:
• There should be a navigation element the user can select from to update their billing information
• The user should have the ability to update either their billing address or their payment methods for the recurring payment
• A notification should be shown to the user stating that changes will not go into effect until next billing cycle
#qualityjam16
Acceptance Tests
Verify the work: Acceptance tests evaluate the acceptance criteria using the “Given-When-Then” format.
Example:Given: The given clause sets the stage for the initial scenario being tested or the stage the user is at prior to testingWhen: The when clause describes the action the user performs on the systemThen: The then clause describes the system’s reaction to the action performed in the when clause
#qualityjam16
Acceptance Test Example
Example:Given the customer has navigated to the billing pageWhen the customer clicks on “Update Billing Information” and clicks “Change Address,”Then: the customer is presented with a form to enter a new address.
Example based on Acceptance Criteria:
Benefits of Test-First Approaches
#qualityjam16
How Can TDD Help Us?
Test Driven Development brings several major benefits to organization, most notably:
1. Move Testing Up Front – prevents having to rush testing at the end of the cycle2. Bake in Automation from Day 1 – protects against getting behind with test and
automation coverage3. Build More Testable Software – requires developers to think about testability,
and create more robust software4. Push to Customers When Ready – allows you to push software to customers
just in time, as it is developed
#qualityjam16
Move Testing Up Front
Moving Testing Up Front removes the risk of having to make compromises at the end of the cycle on quality or on-time delivery:
Traditional Development Timeline
Ends on: Day 1 Day 3 Day 14 Day 20 Day 21
Design Requirements Code Test Deploy
There is risk in this process that any process, typically Code, will run over and either squeeze development, or push release dates. TDD removes it!
#qualityjam16
Defect Costs Increase as Code Matures
Design Implementation Testing Maintenance0
20
40
60
80
100
120
Phase/Stage of the S/W Development in Which the Defect is Found
Cost of Resolving Defect
#qualityjam16
Bake in Automation Up Front
In traditional development, automation is often built after the code is developed, which has significant limitations:
Test development is more costly, since we cannot access the code to make it more testable (more details to come)Tests are slower and more brittle, with higher levels of maintenance, if we can only access the UITest coverage is incomplete, as we must chose strategically where to build out automation coverage
Moving to TDD flips the process and forces developers to write code to satisfy tests, increasing automation coverage, speed, and reducing cost
#qualityjam16
Build for Testability
Source: http://zeroturnaround.com/wp-content/uploads/2015/12/PUZZLE-1-min.png
Moving towards BDD will force your developers to build an application that can be tested well at the Unit, Integration, and UI levels:
#qualityjam16
UI Test
s
Integration Tests
Unit Tests
Building a Complete Testing Strategy
Moving towards BDD will also demand a more complete testing strategy focused on more than just UI testing:
UI Tests
Integration Tests
Unit Test
s
#qualityjam16
Push Features When They Are Complete
TDD paired with continuous delivery will allow you to push features as they become ready, if you’d like to:
Old Way
New Way
Code Feature ACode Feature B
Code Feature CTest Deploy
WaitWait
Code Feature ACode Feature B
Code Feature C
WriteTests
DeployDeploy
Deploy
State of Test-First in Industry
#qualityjam16
State of Test First – Respondent Overview
Most respondents are yet to implement TDD, or just getting started
#qualityjam16
Learning the Ropes Online
Most people are learning about TDD online, due to lack of good in-person content!
#qualityjam16
Struggling for Developer Buy-In
The biggest stumbling block continues to be getting developers to write tests
#qualityjam16
Transition to Agile First, Then TDD
Most organizations move to TDD after mastering Scrum/Kanban Agile approach
How BetterCloud Does ATDD
#qualityjam16
A Typical Sprint
Dev Code/Test Code/Test Code/Test Code/Test Code/Test
Day 1 Day 2 Day 3 Day 4 Day 5
FA Test Plan Test Test - Color Test Test TestTest upload to Qtest
DEV
QA
Everyone Sprint Planning Arch Review Sprint Planning Internal Demo
Dev Code/Test Code/Test Code/Test Code/Test Code/Test
Day 6 Day 7 Day 8 Day 9 Day 10
FA Test Plan Test - Color Test - Color Test Test Test Test Test
DEV
FA
Everyone Sprint Planning Arch Review Sprint Planning Internal Demo
Dev Bugs/Refactor Bugs/Refactor Bugs/Refactor Bugs/Refactor Bugs/Refactor
Day 11 Day 12 Day 13 Day 14 Day 15
FA Test Plan Test - Color Test Test - ProductionTest Test Test
DEV
FA
Everyone Sprint Planning Arch Review Sprint Planning Demo & Retrospective
#qualityjam16
How Does the Process Work with our Tools?
User Story Acceptance Criteria Testing Tasks
Test Case Test Runs
JIRA
qTest
When the Test Run passes, complete the
testing task
DONE!
#qualityjam16
Improvements for Automation
BetterCloud writes automated cases in Groovy with the Spock framework:
• Spock - higher level language, like a manual acceptance test
• Groovy - object oriented programming language, drives desired activity in the application
Reduced the amount of time between functionality being delivered and automation coverage being created
#qualityjam16
Groovy and Spock Example
#qualityjam16
QA Improvements
• QA have become the Subject Matter Experts (SME’s) for the product• Acceptance test cases get created during sprint planning which frees
up time in sprint to test• Team can focus on more complex test scenarios including system
testing and leading UAT sessions with the business• QA and Dev can discuss the split of testing to limit overlap between
Unit, Integration and Acceptance Testing
#qualityjam16
Tips
• Include the test automation team early and often in sprint planning• Do not think “inside the tool” with test automation. Learn about other
efficiencies that can be gained, like creating scripts in powershell or a chrome extension tool.
• Focus on communication and collaboration. Those are the two key points to a successful Agile team
• Both Manual and automated testers working together is key
Q&A
