To atdd-and-beyond

John Ferguson Smart

To ATDD And BeyondBetter Automated Acceptance Testing on the JVM

John Ferguson Smart

ConsultantTrainerMentorAuthorSpeakerCoder

The Plan

1) What is ATDD?

2) How can I do it

3) Let’s see it work

The story of your app

Acceptance Test Driven Development

Features/Epics

Make money by selling our stuff online

Display catalog

Order products

Build a wishlist

Browse CatalogIn order to find stuff I would like to buyAs a customerI want to be able to browse through the catalog

User stories

☑ See all the top-‐level categories in the catalog☑ Browse through the sub-‐categories☑ See all the products in a category

Acceptance criteria

User stories

☑ See all the top-‐level categories in the catalog☑ Browse through the sub-‐categories☑ See all the products in a category

Acceptance criteria

Scenario: See all top-level categoriesGiven I want to browse the catalogWhen I am on the home pageThen I should see the following product categories: Clothing, Accessories, Shoes

Automated acceptance test

Implemented development tests Implemented acceptance tests

Automated acceptance test

Automated acceptance criteria

Define your goals

keep you on track

Provide better visibility

Allow faster release cycles

Reduce Risk

Provide more value

Delivery Time

Using ATD

31% faster delivery

Higher Quality

Defect R

Using ATD

4 4mes less defects

Why only do QA at the end of the project?

Real quality cannot be injected at the end

It must be part of the process

Leave the boring stuff to the automated tests...

Let testers focus on more intelligent testing

...and empower your QA team

Keeping an eye on things

(Think “Two-CDs”)

2 Automate your acceptance criteria

1 Discover your acceptance criteria

4 Execute your acceptance tests

3 Implement your acceptance criteria

Feature: Browse CatalogIn order to find items that I would like to buyAs a customerI want to be able to browse through the catalog

Story: Browse by categoryIn order to find items more easilyAs a customerI want to be able to browse through the product categories

Acceptance CriteriaSee all the top-level categoriesBrowse through the category hierarchyShould display the correct products for each categoryEach category should have the correct sub-categories

Define acceptance criteria for each story

Clarify the acceptance criteria with examples

Story: Browse by categoryIn order to find items more easilyAs a customerI want to be able to browse through the product categories

Narrative:In order to find items more easilyAs a customerI want to be able to see what product categories exist

We now have an executable requirement

...but they will be reported as ‘pending’

Narrative:In order to find items more easilyAs a customerI want to be able to see what product categories exist

4 Execute your acceptance tests

Functional Test Coverage

“What was tested”

“What’s been done”

What have we tested?

Broken

In Progress

What have we finished?

Finished

Specified but no tests

Partially done?

What have we finished?

Better organised requirements

Stay on top of your scenarios

Better visibility on what is doneMore targeted reporting

CapabilityIn order to increase the number of items I sellAs a sellerI want buyers to be able to view ads for items they might want to purchase

FeatureIn order to increase sales of advertised articlesAs a sellerI want potential buyers to be able to display only the ads for articles that they might be interested in purchasing.

StoryIn order to find the items I am interested in fasterAs a buyerI want to be able to list all the ads with a particular keyword in the description or title.

Goal: In order to increase revenue from commissions on classified ads salesAs the head of the classified ads departmentI want to increase the number of items sold via our classified ads

Keep your scenarios organized

Store your requirements where it suits you

RequirementsTagProvidergetRequirements()getParentRequirementOf(testOutcome)

MyRequirementProvider

Meta:@issue MYPROJ-123

Scenario: Search by keywordGiven that I want to find products in the "<range>" rangeWhen I search for products by keyword "<keywords>"Then I should see a product with the title <expectedTitle>

CapabilityIn order to increase the number of items I sellAs a sellerI want buyers to be able to view ads for items they might want to

FeatureIn order to increase sales of advertised articlesAs a sellerI want potential buyers to be able to display only the ads for articles

MYPROJ-123StoryIn order to find the items I am interested in fasterAs a buyerI want to be able to list all the ads with a particular keyword in the description or title.

Customize Thucydides to work for you

• Non-functional requirements• Iterations• Current iteration vs regression tests• Acceptance tests vs more detailed tests• Related features• System components• ...

Use tags for orthogonal concerns

@RunWith(ThucydidesRunner)@Issue("NC-1")@WithTag(name="Browse Ads")class BrowseAdsByCategory { @Managed public WebDriver webdriver

@ManagedPages(defaultUrl = "http://www.newsclassifieds-uat.appspot.com") public Pages pages

@Steps BuyerSteps buyer

@Test void "Browse ads by category"() { buyer.opens_home_page() buyer.chooses_classification "Furniture & Homewares" buyer.should_see_search_results_for "Furniture & Homewares"

buyer.chooses_entry(1).from().other_results() buyer.should_see_details_for_the_selected_ad()

buyer.returns_to_search_results() buyer.should_see_search_results_for "Furniture & Homewares" }}

Tagging in JUnit

Use tags for orthogonal concernsMeta:@tag component:search

Examples:| range | keywords | expectedTitle || Simone | Simone | 3by3 Simone || Grey Steel Classic | Steel | 3by3 Grey Steel Classic || KV Bags | KV | KV Classic Satchel |

Tagging in JBehave

Use tags for orthogonal concernsMeta:@tag component:search

Examples:| range | keywords | expectedTitle || Simone | Simone | 3by3 Simone || Grey Steel Classic | Steel | 3by3 Grey Steel Classic || KV Bags | KV | KV Classic Satchel |

More structured tests

Have you got web tests that look like this?

Page Objects To The Rescue!

Better encapsulationEasier to readEasier to maintainStill a bit unclear what we are doing

Test Steps To The Rescue!

Step methods focus on what we are doing

Step implementations detail the how

@Step("The carousel should display {0} at a time") def should_see_a_number_of_visible_ads_in_the_carousel(int adCount) { assert homePage.visibleCarouselAds.size() == adCount }

@Step public void proceedToPayment() { adPreviewPage.proceedToPayment(); }

Steps are ordinary methods

@Step public void searches_for(String searchTerms) { homePage.searchFor(searchTerms); }

They can have parameters

@Pending @Step public void searches_for(String searchTerms) { // Not done yet }

They can be pending

You can customize the title

@Step def browses_product_categories(String... categories) { selects_top_level_category(categories.head()) categories.tail().each { subcategory -> browse_through_subcategory(subcategory) } }

@Step def selects_top_level_category(String category) { homePage.select_top_level_category category } @Step def browse_through_subcategory(subcategory) { homePage.select_subcategory(subcategory) }

Steps can call other steps

Test reports document the what and the how

Focus on what we are doing, not how we do itMore reusabilityBetter reporting

Even better with JBehave!

Narrative:In order to find items more easilyAs a customerI want to be able to browse through the product categories

Scenario: Browse through product categoriesGiven I am on the home pageWhen I browse through the product categories Clothing, Mens, ShirtsThen I should see a product with the title 3by3 Milano Stretch Cotton Shirt

JBehave describes what we are doing

Use normal Thucydides steps in the implementation

Scenario: Browse through product categoriesGiven I am on the home pageWhen I browse through the product categories Clothing, Mens, ShirtsThen I should see a product with the title 3by3 Milano Stretch Cotton Shirtclass BrowseByCategorySteps extends BigCommerceJBehaveSteps{

@Steps CustomerSteps customer;

@Given('I am on the home page') public void givenIAmOnTheHomePage() { customer.opens_home_page() } @When('I browse through the product categories $categories') public void whenIBrowseThroughTheProductCategories(List<String> categories) { customer.browses_product_categories(*categories) }

@Then('I should see a product with the title "$expectedTitle"') public void thenIShouldSeeAProductWithTheTitle(String expectedTitle) { customer.should_see_product withTitle(expectedTitle) }}

Use normal Thucydides steps

Context

Illustrations

Living documentationEasier to readEasier to maintainMore work maintaining the .story files

Better Page Objects

Fluent selectorsFluent matchersHTML tablesFluent waits

Thucydides Page Object support

Thucydides Fluent Selectorspublic class HomePage extends PageObject {

@FindBy(name="adFilter.searchTerm") WebElement searchTerm;

@FindBy(css=".keywords button") WebElement search;

public HomePage2(WebDriver driver) { super(driver); }

public void chooseCategory(String category) { findBy("#category-select").then(".arrow").then().click(); findBy("#category-select").then(By.linkText(category)).then().click(); }

public void enterKeywords(String keywords) { $(searchTerm).type(keywords); }

public void performSearch() { $(search).click(); }}

Fluent selectors

Thucydides helper methods

Thucydides Fluent Selectorsclass ShoppingCartPage extends PageObject {

ShoppingCartPage(WebDriver driver) { super(driver) }

@FindBy(css = ".CartContents tbody tr") List<WebElement> shoppingCartItems

List<CartItem> getCartItems() { shoppingCartItems.collect { cartItemfromElement(it) } }

CartItem cartItemfromElement(WebElement element) { Integer quantity = Integer.parseInt(itemQuantity(element)) String product = productName(element) BigDecimal itemPrice = priceOf(itemPrice(element)) BigDecimal totalPrice = priceOf(totalPrice(element))

return CartItem.containing(quantity).productsCalled(product). withAnItemPriceOf(itemPrice). andATotalOf(totalPrice) } ...

Page Object returns domain classes

Building a CartItem from the WebElement

private String productName(WebElement cartItem) { cartItem.findElement(By.className("ProductName")).text }

private String totalPrice(WebElement cartItem) { cartItem.findElement(By.className("CartItemTotalPrice")).text }

private String itemPrice(WebElement cartItem) { cartItem.findElement(By.className("CartItemIndividualPrice")).text }

private String itemQuantity(WebElement cartItem) { def itemQuantityDropdown = cartItem.findElement( By.xpath(".//td[contains(@class,'CartItemQuantity')]/select")) element(itemQuantityDropdown).selectedValue }

Classic WebDriver

private String productName(WebElement cartItem) { element(cartItem).findBy(".ProductName").text }

private String totalPrice(WebElement cartItem) { element(cartItem).findBy(".CartItemTotalPrice").text }

private String itemPrice(WebElement cartItem) { element(cartItem).findBy(".CartItemIndividualPrice").text }

private String itemQuantity(WebElement cartItem) { element(cartItem).findBy(".CartItemQuantity").then("select").selectedValue }

Using Thucydides Fluent Selectors

private String productName(WebElement cartItem) { $(cartItem).findBy(".ProductName").text }

private String totalPrice(WebElement cartItem) { $(cartItem).findBy(".CartItemTotalPrice").text }

private String itemPrice(WebElement cartItem) { $(cartItem).findBy(".CartItemIndividualPrice").text }

private String itemQuantity(WebElement cartItem) { $(cartItem).findBy(".CartItemQuantity").then("select").selectedValue }

Using short-hand Thucydides Fluent Selectors

private String productName(WebElement cartItem) { return $(cartItem).findBy(".ProductName").getText(); }

private String totalPrice(WebElement cartItem) { return $(cartItem).findBy(".CartItemTotalPrice").getText(); }

private String itemPrice(WebElement cartItem) { return $(cartItem).findBy(".CartItemIndividualPrice").getText(); }

private String itemQuantity(WebElement cartItem) { return $(cartItem).findBy(".CartItemQuantity").then("select").getSelectedValue(); }

(Pure Java version)

Thucydides Fluent Selectors List<String> getYAxes() { def axesElements = findAll(".dygraph-axis-label-y"); axesElements.collect { WebElement axis -> axis.text } }

Finding multiple elements

Thucydides Fluent Matchers

Thucydides Fluent Matchersimport static org.hamcrest.Matchers.is;import static net.thucydides.core.matchers.BeanMatchers.the;... @Steps public DeveloperSteps developer;

@Test public void should_search_for_artifacts_by_name() { developer.opens_the_search_page(); developer.searches_for("Thucydides"); developer.should_see_artifacts_where(the("ArtifactId", is("thucydides")), the("GroupId", is("net.thucydides"))); }

Using Thucydides matchers

And Hamcrest matchers

import static net.thucydides.core.matchers.BeanMatcherAsserts.shouldMatch;

public class DeveloperSteps extends ScenarioSteps {

SearchPage searchPage; SearchResultsPage searchResultsPage;

public DeveloperSteps(Pages pages) { super(pages); searchResultsPage = pages.getPage(SearchResultsPage.class); searchPage = pages.getPage(SearchPage.class); }

@Step public void opens_the_search_page() { searchPage.open(); }

@Step public void searches_for(String search_terms) { searchPage.enter_search_terms(search_terms); searchPage.starts_search(); }

@Step public void should_see_artifacts_where(BeanMatcher... matchers) { shouldMatch(searchResultsPage.getSearchResults(), matchers); }}

Applying the matchers

The Page Object returns a list of POJOs

public List<Artifact> getResults() { List<WebElement> rows = resultTable.findElements(By.xpath(".//tr[td]")); return convert(rows, toArtifacts()); }

private Converter<WebElement, Artifact> toArtifacts() { return new Converter<WebElement, Artifact>() { public Artifact convert(WebElement row) { List<WebElement> cells = row.findElements(By.tagName("td")); String groupId = cells.get(0).getText(); String artifactId = cells.get(1).getText(); String latestVersion = cells.get(2).getText(); return new Artifact(groupId, artifactId, latestVersion); } }; }

The page object should return POJOs

Thucydides Fluent MatchersList<Person> persons = Arrays.asList(new Person("Bill", "Oddie"), new Person("Tim", "Brooke-Taylor"));

shouldMatch(persons, the("firstName", is(not("Tim"))));shouldMatch(persons, the("firstName", startsWith("B")));

Matcher work with simple POJOs

Thucydides and Tablesimport static net.thucydides.core.pages.components.HtmlTable.rowsFrom;

public class SearchResultsPage extends PageObject {

WebElement resultTable;

public SearchResultsPage(WebDriver driver) { super(driver); }

public List<Map<String, String>> getSearchResults() { return rowsFrom(resultTable); }}

Convenience methods: convert a table to a map of Strings

Thucydides and Tables @Test public void clicking_on_artifact_should_display_details_page() { developer.opens_the_search_page(); developer.searches_for("Thucydides"); developer.open_artifact_where(the("ArtifactId", is("thucydides")), the("GroupId", is("net.thucydides")));

developer.should_see_artifact_details_where(the("artifactId", is("thucydides")), the("groupId", is("net.thucydides"))); }

Using matchers to click on a row in a table

Thucydides and Tables @Test public void clicking_on_artifact_should_display_details_page() { developer.opens_the_search_page(); developer.searches_for("Thucydides"); developer.open_artifact_where(the("ArtifactId", is("thucydides")), the("GroupId", is("net.thucydides")));

developer.should_see_artifact_details_where(the("artifactId", is("thucydides")), the("groupId", is("net.thucydides"))); } @Step public void open_artifact_where(BeanMatcher... matchers) { searchResultsPage.clickOnFirstRowMatching(matchers); }

import static net.thucydides.core.pages.components.HtmlTable.filterRows;... @FindBy(css="#resultTable") WebElement resultTable;

public void clickOnFirstRowMatching(BeanMatcher... matchers) { List<WebElement> matchingRows = filterRows(resultTable, matchers); WebElement targetRow = matchingRows.get(0); WebElement detailsLink = $(targetRow).findBy(".//a[contains(@href,'artifactdetails')]"); detailsLink.click(); }

Pass through matchers

Return matching rows

Thucydides Fluent Waits

Convenience methodsExtends the WebDriver Wait API

Thucydides Fluent Waits void addToCart(Integer quantity) { $(quantitySelection).selectByVisibleText(quantity.toString()) $(addToCart).click() waitForTextToAppear("added to your cart") }

A simple wait

void uploadExistingImage(String imageUrl) { // upload an image ... waitForPresenceOf(".photo") }

Wait for an element to be rendered

def waitForPageToLoad() { waitFor(500).milliseconds() waitForAbsenceOf("#loading") }

Wait for an element to disappear

Thucydides Fluent Waits def searchByLocation(postcode) { element(location).typeAndTab(postcode) element(locationGo).click() waitFor("//div[@class='f-item'][contains(.,'${postcode}')]") }

Waiting for an XPath expression

def filterByLocation(String state) { element(locationList).click() waitFor("#location-select li a") findBy("//span[@id='location-select']//a[contains(.,'$state')]").then().click() waitForCondition().until(stateIsChosen(state))

Function<? super WebDriver, Boolean> stateIsChosen(final String state) { return new Function<? super WebDriver, Boolean>() {

@Override Boolean apply(driver) { return state == getCurrentSelectedState() }; } }

Wait for a non-trivial condition

What are we waiting for

Thucydides Assertsdef shouldDisplaySubcategory(subCategory) { findBy(".SubCategoryList").then(By.linkText(subCategory)).shouldBeCurrentlyVisible()}

Check that this element is visible

@FindBy(id=”search”)WebElement searchButton

def searchShouldBeEnabled() { searchButton.shouldBeCurrentlyEnabled()}

Check that this element is enabled

Smarter Webdriver coding

Prefer CSS to XPath

Use Nested Finds

Nested Findspublic void chooseCategory(String category) { findBy("//div[@id='category-select']//a[contains(@class,'arrow')]").click(); findBy("//div[@id='category-select']//a[.,contains($category)]").click();}

public void chooseCategory(String category) { findBy("#category-select").then(".arrow").then().click(); findBy("#category-select").then(By.linkText(category)).then().click();}

Hard to read, hard to maintain

A more readable approach

Confine Webdriver to your Page Objects

Encapsulate Page Objectsclass ShoppingCartPage extends PageObject {

public ShoppingCartPage(WebDriver driver) { super(driver) }

public List<CartItem> getCartItems() { shoppingCartItems.collect { cartItemfromElement(it) } }

@FindBy(css = ".CartContents tbody tr") private List<WebElement> shoppingCartItems

private CartItem cartItemfromElement(WebElement element) { Integer quantity = Integer.parseInt(itemQuantity(element)) String product = productName(element) BigDecimal itemPrice = priceOf(itemPrice(element)) BigDecimal totalPrice = priceOf(totalPrice(element))

Public API

Webdriver stays private

Webdriver Query = JDBC Query~

Learning More

github.com/thucydides-webtestshttp://thucydides.info

http://thucydides-webtests.com

John Ferguson Smart

Thank You

To ATDD And BeyondBetter Automated Acceptance Testing on the JVM

To atdd-and-beyond

Documents

ATDD (Acceptance Test Driven Dev)

Unit Testing, TDD and ATDD

ATDD by Example 2012

Change Anything with Cucumber and ATDD

ATDD It’s Not Just About Writing Tests

Atdd for web apps

Atdd Lab Exercises

ATDD In The Enterprise Context

ASOS Tech Community Live: ATDD

BDD, ATDD, Page Objects: The Road to Sustainable Web Testing

At2009 Coding Dojo ATDD

ATDD Digital Booklet

A QUICK GUIDE TO IMPLEMENT ATDD - STPCon€¦ · TO IMPLEMENT ATDD raj@testim.io 2 Developer Evangelist at ... JSON/HTTP UI Automation APPIUM Server Cucumber with Appium Work Flow

Guide to ATDD · Business Value “Done” means Done Frequent Delivery Alignment. Defining Acceptance Test Driven Development (ATDD) 5 Acceptance Test Driven Development is a practice

Model-based Testing:Taking BDD/ATDD to the Next Level

Raconteur: bringing ATDD to Visual Studio

An ATDD Case Study

ARTICLE ATDD – An applied approach to continuous delivery · These tests can be implemented using frameworks like: Aspire Systems - ATDD – An applied approach to continuous delivery

Sprint3r tpse2014-atdd-with-robot-framework

ATDD and Conversations