To atdd-and-beyond

John Ferguson Smart

To ATDD And BeyondBetter Automated Acceptance Testing on the JVM

John Ferguson Smart

ConsultantTrainerMentorAuthorSpeakerCoder

The Plan

1) What is ATDD?

2) How can I do it

3) Let’s see it work

The story of your app

Acceptance Test Driven Development

Features/Epics

Make money by selling our stuff online

Display catalog

Goals

Order products

Build a wishlist

Browse CatalogIn order to find stuff I would like to buyAs a customerI want to be able to browse through the catalog

User stories


User stories

☑ See all the top-‐level categories in the catalog☑ Browse through the sub-‐categories☑ See all the products in a category

Acceptance criteria


User stories

☑ See all the top-‐level categories in the catalog☑ Browse through the sub-‐categories☑ See all the products in a category

Acceptance criteria

Scenario: See all top-level categoriesGiven I want to browse the catalogWhen I am on the home pageThen I should see the following product categories: Clothing, Accessories, Shoes

Automated acceptance test

Implemented development tests Implemented acceptance tests

Automated acceptance test

Automated acceptance criteria

Define your goals


keep you on track


Provide better visibility


Allow faster release cycles


Reduce Risk


Provide more value

Delivery Time

Trad

i4on

al

Using ATD

D

31% faster delivery


Higher Quality

Defect R

ate

Trad

i4on

al

Using ATD

D

4 4mes less defects

Why only do QA at the end of the project?

Real quality cannot be injected at the end

It must be part of the process

Leave the boring stuff to the automated tests...

Let testers focus on more intelligent testing

...and empower your QA team

20

Keeping an eye on things

21

(Think “Two-CDs”)

22

2 Automate your acceptance criteria

1 Discover your acceptance criteria

4 Execute your acceptance tests

3 Implement your acceptance criteria

23


Feature: Browse CatalogIn order to find items that I would like to buyAs a customerI want to be able to browse through the catalog

Story: Browse by categoryIn order to find items more easilyAs a customerI want to be able to browse through the product categories

Acceptance CriteriaSee all the top-level categoriesBrowse through the category hierarchyShould display the correct products for each categoryEach category should have the correct sub-categories

Define acceptance criteria for each story

24




Clarify the acceptance criteria with examples

25


Story: Browse by categoryIn order to find items more easilyAs a customerI want to be able to browse through the product categories



Narrative:In order to find items more easilyAs a customerI want to be able to see what product categories exist


We now have an executable requirement

26


...but they will be reported as ‘pending’

27


Narrative:In order to find items more easilyAs a customerI want to be able to see what product categories exist


28


29


JUnit

30

4 Execute your acceptance tests

31

32

33

34

35

36

Functional Test Coverage

“What was tested”

vs

“What’s been done”

What have we tested?

Broken

Works

In Progress

What have we finished?

Finished

Specified but no tests

Partially done?




Better organised requirements

Stay on top of your scenarios



Better visibility on what is doneMore targeted reporting

CapabilityIn order to increase the number of items I sellAs a sellerI want buyers to be able to view ads for items they might want to purchase

FeatureIn order to increase sales of advertised articlesAs a sellerI want potential buyers to be able to display only the ads for articles that they might be interested in purchasing.

StoryIn order to find the items I am interested in fasterAs a buyerI want to be able to list all the ads with a particular keyword in the description or title.

Goal: In order to increase revenue from commissions on classified ads salesAs the head of the classified ads departmentI want to increase the number of items sold via our classified ads

Keep your scenarios organized

Store your requirements where it suits you

RequirementsTagProvidergetRequirements()getParentRequirementOf(testOutcome)

MyRequirementProvider

Meta:@issue MYPROJ-123

Scenario: Search by keywordGiven that I want to find products in the "<range>" rangeWhen I search for products by keyword "<keywords>"Then I should see a product with the title <expectedTitle>

CapabilityIn order to increase the number of items I sellAs a sellerI want buyers to be able to view ads for items they might want to

FeatureIn order to increase sales of advertised articlesAs a sellerI want potential buyers to be able to display only the ads for articles

MYPROJ-123StoryIn order to find the items I am interested in fasterAs a buyerI want to be able to list all the ads with a particular keyword in the description or title.

Customize Thucydides to work for you

• Non-functional requirements• Iterations• Current iteration vs regression tests• Acceptance tests vs more detailed tests• Related features• System components• ...

Use tags for orthogonal concerns

Use tags for orthogonal concerns

@RunWith(ThucydidesRunner)@Issue("NC-1")@WithTag(name="Browse Ads")class BrowseAdsByCategory { @Managed public WebDriver webdriver

@ManagedPages(defaultUrl = "http://www.newsclassifieds-uat.appspot.com") public Pages pages

@Steps BuyerSteps buyer

@Test void "Browse ads by category"() { buyer.opens_home_page() buyer.chooses_classification "Furniture & Homewares" buyer.should_see_search_results_for "Furniture & Homewares"

buyer.chooses_entry(1).from().other_results() buyer.should_see_details_for_the_selected_ad()

buyer.returns_to_search_results() buyer.should_see_search_results_for "Furniture & Homewares" }}

Tagging in JUnit

http://www.newsclassifieds-uat.appspot.com

http://www.newsclassifieds-uat.appspot.com

Use tags for orthogonal concernsMeta:@tag component:search


Examples:| range | keywords | expectedTitle || Simone | Simone | 3by3 Simone || Grey Steel Classic | Steel | 3by3 Grey Steel Classic || KV Bags | KV | KV Classic Satchel |

Tagging in JBehave

Use tags for orthogonal concernsMeta:@tag component:search


Examples:| range | keywords | expectedTitle || Simone | Simone | 3by3 Simone || Grey Steel Classic | Steel | 3by3 Grey Steel Classic || KV Bags | KV | KV Classic Satchel |

More structured tests

Have you got web tests that look like this?

Have you got web tests that look like this?

Page Objects To The Rescue!

Page Objects To The Rescue!

Better encapsulationEasier to readEasier to maintainStill a bit unclear what we are doing

Test Steps To The Rescue!

Step methods focus on what we are doing


Step implementations detail the how

@Step("The carousel should display {0} at a time") def should_see_a_number_of_visible_ads_in_the_carousel(int adCount) { assert homePage.visibleCarouselAds.size() == adCount }


@Step public void proceedToPayment() { adPreviewPage.proceedToPayment(); }

Steps are ordinary methods

@Step public void searches_for(String searchTerms) { homePage.searchFor(searchTerms); }

They can have parameters

@Pending @Step public void searches_for(String searchTerms) { // Not done yet }

They can be pending

You can customize the title


@Step def browses_product_categories(String... categories) { selects_top_level_category(categories.head()) categories.tail().each { subcategory -> browse_through_subcategory(subcategory) } }

@Step def selects_top_level_category(String category) { homePage.select_top_level_category category } @Step def browse_through_subcategory(subcategory) { homePage.select_subcategory(subcategory) }

Steps can call other steps


Test reports document the what and the how


Focus on what we are doing, not how we do itMore reusabilityBetter reporting

Even better with JBehave!

Narrative:In order to find items more easilyAs a customerI want to be able to browse through the product categories

Scenario: Browse through product categoriesGiven I am on the home pageWhen I browse through the product categories Clothing, Mens, ShirtsThen I should see a product with the title 3by3 Milano Stretch Cotton Shirt

JBehave describes what we are doing

Use normal Thucydides steps in the implementation



Scenario: Browse through product categoriesGiven I am on the home pageWhen I browse through the product categories Clothing, Mens, ShirtsThen I should see a product with the title 3by3 Milano Stretch Cotton Shirtclass BrowseByCategorySteps extends BigCommerceJBehaveSteps{

@Steps CustomerSteps customer;

@Given('I am on the home page') public void givenIAmOnTheHomePage() { customer.opens_home_page() } @When('I browse through the product categories $categories') public void whenIBrowseThroughTheProductCategories(List<String> categories) { customer.browses_product_categories(*categories) }

@Then('I should see a product with the title "$expectedTitle"') public void thenIShouldSeeAProductWithTheTitle(String expectedTitle) { customer.should_see_product withTitle(expectedTitle) }}

Use normal Thucydides steps




Context

What

How

Illustrations




Living documentationEasier to readEasier to maintainMore work maintaining the .story files

Better Page Objects

Fluent selectorsFluent matchersHTML tablesFluent waits

Thucydides Page Object support

Thucydides Fluent Selectorspublic class HomePage extends PageObject {

@FindBy(name="adFilter.searchTerm") WebElement searchTerm;

@FindBy(css=".keywords button") WebElement search;

public HomePage2(WebDriver driver) { super(driver); }

public void chooseCategory(String category) { findBy("#category-select").then(".arrow").then().click(); findBy("#category-select").then(By.linkText(category)).then().click(); }

public void enterKeywords(String keywords) { $(searchTerm).type(keywords); }

public void performSearch() { $(search).click(); }}

Fluent selectors

Thucydides helper methods

Thucydides Fluent Selectorsclass ShoppingCartPage extends PageObject {

ShoppingCartPage(WebDriver driver) { super(driver) }

@FindBy(css = ".CartContents tbody tr") List<WebElement> shoppingCartItems

List<CartItem> getCartItems() { shoppingCartItems.collect { cartItemfromElement(it) } }

CartItem cartItemfromElement(WebElement element) { Integer quantity = Integer.parseInt(itemQuantity(element)) String product = productName(element) BigDecimal itemPrice = priceOf(itemPrice(element)) BigDecimal totalPrice = priceOf(totalPrice(element))

return CartItem.containing(quantity).productsCalled(product). withAnItemPriceOf(itemPrice). andATotalOf(totalPrice) } ...

Page Object returns domain classes

Building a CartItem from the WebElement







private String productName(WebElement cartItem) { cartItem.findElement(By.className("ProductName")).text }

private String totalPrice(WebElement cartItem) { cartItem.findElement(By.className("CartItemTotalPrice")).text }

private String itemPrice(WebElement cartItem) { cartItem.findElement(By.className("CartItemIndividualPrice")).text }

private String itemQuantity(WebElement cartItem) { def itemQuantityDropdown = cartItem.findElement( By.xpath(".//td[contains(@class,'CartItemQuantity')]/select")) element(itemQuantityDropdown).selectedValue }

Classic WebDriver







private String productName(WebElement cartItem) { element(cartItem).findBy(".ProductName").text }

private String totalPrice(WebElement cartItem) { element(cartItem).findBy(".CartItemTotalPrice").text }

private String itemPrice(WebElement cartItem) { element(cartItem).findBy(".CartItemIndividualPrice").text }

private String itemQuantity(WebElement cartItem) { element(cartItem).findBy(".CartItemQuantity").then("select").selectedValue }

Using Thucydides Fluent Selectors







private String productName(WebElement cartItem) { $(cartItem).findBy(".ProductName").text }

private String totalPrice(WebElement cartItem) { $(cartItem).findBy(".CartItemTotalPrice").text }

private String itemPrice(WebElement cartItem) { $(cartItem).findBy(".CartItemIndividualPrice").text }

private String itemQuantity(WebElement cartItem) { $(cartItem).findBy(".CartItemQuantity").then("select").selectedValue }

Using short-hand Thucydides Fluent Selectors







private String productName(WebElement cartItem) { return $(cartItem).findBy(".ProductName").getText(); }

private String totalPrice(WebElement cartItem) { return $(cartItem).findBy(".CartItemTotalPrice").getText(); }

private String itemPrice(WebElement cartItem) { return $(cartItem).findBy(".CartItemIndividualPrice").getText(); }

private String itemQuantity(WebElement cartItem) { return $(cartItem).findBy(".CartItemQuantity").then("select").getSelectedValue(); }

(Pure Java version)

Thucydides Fluent Selectors List<String> getYAxes() { def axesElements = findAll(".dygraph-axis-label-y"); axesElements.collect { WebElement axis -> axis.text } }

Finding multiple elements

Thucydides Fluent Matchers

Thucydides Fluent Matchersimport static org.hamcrest.Matchers.is;import static net.thucydides.core.matchers.BeanMatchers.the;... @Steps public DeveloperSteps developer;

@Test public void should_search_for_artifacts_by_name() { developer.opens_the_search_page(); developer.searches_for("Thucydides"); developer.should_see_artifacts_where(the("ArtifactId", is("thucydides")), the("GroupId", is("net.thucydides"))); }

Using Thucydides matchers

And Hamcrest matchers



import static net.thucydides.core.matchers.BeanMatcherAsserts.shouldMatch;

public class DeveloperSteps extends ScenarioSteps {

SearchPage searchPage; SearchResultsPage searchResultsPage;

public DeveloperSteps(Pages pages) { super(pages); searchResultsPage = pages.getPage(SearchResultsPage.class); searchPage = pages.getPage(SearchPage.class); }

@Step public void opens_the_search_page() { searchPage.open(); }

@Step public void searches_for(String search_terms) { searchPage.enter_search_terms(search_terms); searchPage.starts_search(); }

@Step public void should_see_artifacts_where(BeanMatcher... matchers) { shouldMatch(searchResultsPage.getSearchResults(), matchers); }}

Applying the matchers

The Page Object returns a list of POJOs



public List<Artifact> getResults() { List<WebElement> rows = resultTable.findElements(By.xpath(".//tr[td]")); return convert(rows, toArtifacts()); }

private Converter<WebElement, Artifact> toArtifacts() { return new Converter<WebElement, Artifact>() { public Artifact convert(WebElement row) { List<WebElement> cells = row.findElements(By.tagName("td")); String groupId = cells.get(0).getText(); String artifactId = cells.get(1).getText(); String latestVersion = cells.get(2).getText(); return new Artifact(groupId, artifactId, latestVersion); } }; }

The page object should return POJOs

Thucydides Fluent MatchersList<Person> persons = Arrays.asList(new Person("Bill", "Oddie"), new Person("Tim", "Brooke-Taylor"));

shouldMatch(persons, the("firstName", is(not("Tim"))));shouldMatch(persons, the("firstName", startsWith("B")));

Matcher work with simple POJOs

Thucydides and Tablesimport static net.thucydides.core.pages.components.HtmlTable.rowsFrom;

public class SearchResultsPage extends PageObject {

WebElement resultTable;

public SearchResultsPage(WebDriver driver) { super(driver); }

public List<Map<String, String>> getSearchResults() { return rowsFrom(resultTable); }}

Convenience methods: convert a table to a map of Strings

Thucydides and Tables @Test public void clicking_on_artifact_should_display_details_page() { developer.opens_the_search_page(); developer.searches_for("Thucydides"); developer.open_artifact_where(the("ArtifactId", is("thucydides")), the("GroupId", is("net.thucydides")));

developer.should_see_artifact_details_where(the("artifactId", is("thucydides")), the("groupId", is("net.thucydides"))); }

Using matchers to click on a row in a table

Thucydides and Tables @Test public void clicking_on_artifact_should_display_details_page() { developer.opens_the_search_page(); developer.searches_for("Thucydides"); developer.open_artifact_where(the("ArtifactId", is("thucydides")), the("GroupId", is("net.thucydides")));

developer.should_see_artifact_details_where(the("artifactId", is("thucydides")), the("groupId", is("net.thucydides"))); } @Step public void open_artifact_where(BeanMatcher... matchers) { searchResultsPage.clickOnFirstRowMatching(matchers); }

import static net.thucydides.core.pages.components.HtmlTable.filterRows;... @FindBy(css="#resultTable") WebElement resultTable;

public void clickOnFirstRowMatching(BeanMatcher... matchers) { List<WebElement> matchingRows = filterRows(resultTable, matchers); WebElement targetRow = matchingRows.get(0); WebElement detailsLink = $(targetRow).findBy(".//a[contains(@href,'artifactdetails')]"); detailsLink.click(); }

Pass through matchers

Return matching rows

Thucydides Fluent Waits

Convenience methodsExtends the WebDriver Wait API

Thucydides Fluent Waits void addToCart(Integer quantity) { $(quantitySelection).selectByVisibleText(quantity.toString()) $(addToCart).click() waitForTextToAppear("added to your cart") }

A simple wait

void uploadExistingImage(String imageUrl) { // upload an image ... waitForPresenceOf(".photo") }

Wait for an element to be rendered

def waitForPageToLoad() { waitFor(500).milliseconds() waitForAbsenceOf("#loading") }

Wait for an element to disappear

Thucydides Fluent Waits def searchByLocation(postcode) { element(location).typeAndTab(postcode) element(locationGo).click() waitFor("//div[@class='f-item'][contains(.,'${postcode}')]") }

Waiting for an XPath expression

def filterByLocation(String state) { element(locationList).click() waitFor("#location-select li a") findBy("//span[@id='location-select']//a[contains(.,'$state')]").then().click() waitForCondition().until(stateIsChosen(state))

}

Function<? super WebDriver, Boolean> stateIsChosen(final String state) { return new Function<? super WebDriver, Boolean>() {

@Override Boolean apply(driver) { return state == getCurrentSelectedState() }; } }

Wait for a non-trivial condition

What are we waiting for

Thucydides Assertsdef shouldDisplaySubcategory(subCategory) { findBy(".SubCategoryList").then(By.linkText(subCategory)).shouldBeCurrentlyVisible()}

Check that this element is visible

@FindBy(id=”search”)WebElement searchButton

def searchShouldBeEnabled() { searchButton.shouldBeCurrentlyEnabled()}

Check that this element is enabled

Smarter Webdriver coding


Prefer CSS to XPath


Use Nested Finds

Nested Findspublic void chooseCategory(String category) { findBy("//div[@id='category-select']//a[contains(@class,'arrow')]").click(); findBy("//div[@id='category-select']//a[.,contains($category)]").click();}

public void chooseCategory(String category) { findBy("#category-select").then(".arrow").then().click(); findBy("#category-select").then(By.linkText(category)).then().click();}

Hard to read, hard to maintain

A more readable approach


Confine Webdriver to your Page Objects

Encapsulate Page Objectsclass ShoppingCartPage extends PageObject {

public ShoppingCartPage(WebDriver driver) { super(driver) }

public List<CartItem> getCartItems() { shoppingCartItems.collect { cartItemfromElement(it) } }

@FindBy(css = ".CartContents tbody tr") private List<WebElement> shoppingCartItems

private CartItem cartItemfromElement(WebElement element) { Integer quantity = Integer.parseInt(itemQuantity(element)) String product = productName(element) BigDecimal itemPrice = priceOf(itemPrice(element)) BigDecimal totalPrice = priceOf(totalPrice(element))


Public API

Webdriver stays private


Webdriver Query = JDBC Query~

Learning More

github.com/thucydides-webtestshttp://thucydides.info

http://thucydides-webtests.com

http://thucydides.info

















John Ferguson Smart

Thank You

To ATDD And BeyondBetter Automated Acceptance Testing on the JVM

Documents

To atdd-and-beyond