99
Java Testing Tools Roundup

Developer Testing Tools Roundup

Embed Size (px)

DESCRIPTION

An overview of testing tools and techniques available to Java developers

Citation preview

Page 1: Developer Testing Tools Roundup

Java Testing Tools Roundup

Page 2: Developer Testing Tools Roundup

What will we cover todayHow to name your testsHamcrest MatchersParameterized testsJUnit RulesMockitoSpockGebWeb testingThucydides and easyb

Agenda

Page 3: Developer Testing Tools Roundup

Most importantly...

Practice Test Driven Development

Page 4: Developer Testing Tools Roundup

Name your tests well

What’s in a name

"What's in a name? That which we call a roseBy any other name would smell as sweet."

Romeo and Juliet (II, ii, 1-2)

Page 5: Developer Testing Tools Roundup

The 10 5 Commandments of Test WritingI. Don’t say “test, say “should” insteadII. Don’t test your classes, test their behaviourIII. Test class names are important tooIV. Structure your tests wellV. Tests are deliverables too

What’s in a name

Page 6: Developer Testing Tools Roundup

Don’t use the word ‘test’ in your test names

What’s in a name

testBankTransfer()

testWithdraw()

testDeposit()

Page 7: Developer Testing Tools Roundup

testBankTransfer()

testWithdraw()

testDeposit()

Do use the word ‘should’ in your test names

What’s in a name

tranferShouldDeductSumFromSourceAccountBalance()

transferShouldAddSumLessFeesToDestinationAccountBalance()

depositShouldAddAmountToAccountBalance()

Page 8: Developer Testing Tools Roundup

Your test class names should represent context

What’s in a name

When is this behaviour applicable?

What behaviour are we testing?

Page 9: Developer Testing Tools Roundup

Write your tests consistently‘Given-When-Then’ or ‘Arrange-Act-Assert’ (AAA)

What’s in a name

@Test public void aDeadCellWithOneLiveNeighbourShouldRemainDeadInTheNextGeneration() { String initialGrid = "...\n" + ".*.\n" + "...";

String expectedNextGrid = "...\n" + "...\n" + "...\n";

Universe theUniverse = new Universe(seededWith(initialGrid)); theUniverse.createNextGeneration(); String nextGrid = theUniverse.getGrid(); assertThat(nextGrid, is(expectedNextGrid)); }

Prepare the test data (“arrange”)

Do what you are testing (“act”)

Check the results (“assert”)

Page 10: Developer Testing Tools Roundup

Tests are deliverables too - respect them as suchRefactor, refactor, refactor!Clean and readable

What’s in a name

Page 11: Developer Testing Tools Roundup

Express Yourself with HamcrestWhy write this...

when you can write this...

import static org.junit.Assert.*;...assertEquals(10000, calculatedTax, 0);

import static org.hamcrest.Matchers.*;...assertThat(calculatedTax, is(10000));

“Assert that are equal 10000 and calculated tax (more or less)” ?!

Don’t I just mean “assert that calculated tax is 10000”?

Page 12: Developer Testing Tools Roundup

Express Yourself with HamcrestWith Hamcrest, you can have your cake and eat it!

String color = "red"; assertThat(color, is("blue"));

assertThat(calculatedTax, is(expectedTax));

Readable asserts

String[] colors = new String[] {"red","green","blue"};String color = "yellow";assertThat(color, not(isIn(colors)));

Informative errors

Flexible notation

Page 13: Developer Testing Tools Roundup

Express Yourself with HamcrestMore Hamcrest expressiveness

String color = "red";assertThat(color, isOneOf("red",”blue”,”green”));

List<String> colors = new ArrayList<String>();colors.add("red");colors.add("green");colors.add("blue");assertThat(colors, hasItem("blue"));

assertThat(colors, hasItems("red”,”green”));

assertThat(colors, hasItem(anyOf(is("red"), is("green"), is("blue"))));

Page 14: Developer Testing Tools Roundup

Home-made Hamcrest MatchersCustomizing and extending HamcrestCombine existing matchersOr make your own!

Page 15: Developer Testing Tools Roundup

Home-made Hamcrest MatchersCustomizing Hamcrest matchersYou can build your own by combining existing Matchers...

List stakeholders = stakeholderManager.findByName("Health");Matcher<Stakeholder> calledHealthCorp = hasProperty("name", is("Health Corp"));assertThat(stakeholders, hasItem(calledHealthCorp));

Create a dedicated Matcher for the Stakeholder class

Use matcher directly with hasItem()

“The stakeholders list has (at least) one item with the name property set to “Health Corp””

Page 16: Developer Testing Tools Roundup

Home-made Hamcrest MatchersWriting your own matchers in three easy steps!

public class WhenIUseMyCustomHamcrestMatchers { @Test public void thehasSizeMatcherShouldMatchACollectionWithExpectedSize() { List<String> items = new ArrayList<String>(); items.add("java"); assertThat(items, hasSize(1)); }}

We want something like this...

I want a matcher that checks the size of a collection

Page 17: Developer Testing Tools Roundup

Home-made Hamcrest MatchersWriting your own matchers in three easy steps!

public class HasSizeMatcher extends TypeSafeMatcher<Collection<? extends Object>> { private Matcher<Integer> matcher;

public HasSizeMatcher(Matcher<Integer> matcher) { this.matcher = matcher; }

public boolean matchesSafely(Collection<? extends Object> collection) { return matcher.matches(collection.size()); }

public void describeTo(Description description) { description.appendText("a collection with a size that is"); matcher.describeTo(description); }}

Extend the TypeSafeMatcher class

Provide expected values in the constructor

Do the actual matching

Describe our expectations

So let’s write this Matcher!

Page 18: Developer Testing Tools Roundup

Home-made Hamcrest MatchersWriting your own matchers in three easy steps!

import java.util.Collection;import org.hamcrest.Factory;import org.hamcrest.Matcher;

public class MyMatchers { @Factory public static Matcher<Collection<? extends Object>> hasSize(Matcher<Integer> matcher){ return new HasSizeMatcher(matcher); }}

Use a factory class to store your matchers

All my custom matchers go in a special Factory class

Page 19: Developer Testing Tools Roundup

Home-made Hamcrest MatchersWriting your own matchers in three easy steps!

import static com.wakaleo.gameoflife.hamcrest.MyMatchers.hasSize;import static org.hamcrest.MatcherAssert.assertThat;

public class WhenIUseMyCustomHamcrestMatchers {

@Test public void thehasSizeMatcherShouldMatchACollectionWithExpectedSize() { List<String> items = new ArrayList<String>(); items.add("java"); assertThat(items, hasSize(1)); }}

Hamcrest-style error messages

Page 20: Developer Testing Tools Roundup

Home-made Hamcrest MatchersBut wait! There’s more!

@Test public void weCanUseCustomMatchersWithOtherMatchers() { List<String> items = new ArrayList<String>(); items.add("java"); assertThat(items, allOf(hasSize(1), hasItem("java"))); }

Combining matchers

@Test public void weCanUseCustomMatchersWithOtherMatchers() { List<String> items = new ArrayList<String>(); items.add("java"); items.add("groovy"); assertThat(items, hasSize(greaterThan(1))); }

Nested matchers

Page 21: Developer Testing Tools Roundup

Using Parameterized Tests

Data-Driven Unit Tests

Page 22: Developer Testing Tools Roundup

Using Parameterized TestsParameterized tests - for data-driven testingTake a large set of test data, including an expected resultDefine a test that uses the test dataVerify calculated result against expected result

{2, 0, 0}{2, 1, 2}{2, 2, 4}{2, 3, 6}{2, 4, 8}{2, 5, 10}{2, 6, 12}{2, 7, 14}

...Data

Test

x = a * b

Verify

Page 23: Developer Testing Tools Roundup

Using Parameterized TestsParameterized testsExample: Calculating income tax

Page 24: Developer Testing Tools Roundup

Using Parameterized Tests

Parameterized tests with JUnit 4.8.1What you need:Some test dataA test class with matching fields And some testsAnd an annotation

Income Expected Tax$0.00 $0.00$10,000.00 $1,250.00$14,000.00 $1,750.00$14,001.00 $1,750.21$45,000.00 $8,260.00$48,000.00 $8,890.00$48,001.00 $8,890.33$65,238.00 $14,578.54$70,000.00 $16,150.00$70,001.00 $16,150.38$80,000.00 $19,950.00$100,000.00 $27,550.00

public class TaxCalculatorDataTest { private double income; private double expectedTax; public TaxCalculatorDataTest(double income, double expectedTax) { this.income = income; this.expectedTax = expectedTax; }}

public class TaxCalculatorDataTest { private double income; private double expectedTax; public TaxCalculatorDataTest(double income, double expectedTax) { super(); this.income = income; this.expectedTax = expectedTax; } @Test public void shouldCalculateCorrectTax() {...}}

@RunWith(Parameterized.class)public class TaxCalculatorDataTest { private double income; private double expectedTax; public TaxCalculatorDataTest(double income, double expectedTax) { super(); this.income = income; this.expectedTax = expectedTax; } @Test public void shouldCalculateCorrectTax() {...}}

Page 25: Developer Testing Tools Roundup

Income Expected Tax$0.00 $0.00$10,000.00 $1,250.00$14,000.00 $1,750.00$14,001.00 $1,750.21$45,000.00 $8,260.00$48,000.00 $8,890.00$48,001.00 $8,890.33$65,238.00 $14,578.54$70,000.00 $16,150.00$70,001.00 $16,150.38$80,000.00 $19,950.00$100,000.00 $27,550.00

Using Parameterized TestsHow it works@RunWith(Parameterized.class)public class TaxCalculatorDataTest { private double income; private double expectedTax;

@Parameters public static Collection<Object[]> data() { return Arrays.asList(new Object[][] { { 0.00, 0.00 }, { 10000.00, 1250.00 }, { 14000.00, 1750.00 }, { 14001.00, 1750.21 }, { 45000.00, 8260.00 }, { 48000.00, 8890.00 }, { 48001.00, 8890.33 }, { 65238.00, 14578.54 }, { 70000.00, 16150.00 }, { 70001.00, 16150.38 }, { 80000.00, 19950.00 }, { 100000.00, 27550.00 }, }); }

public TaxCalculatorDataTest(double income, double expectedTax) { super(); this.income = income; this.expectedTax = expectedTax; }

@Test public void shouldCalculateCorrectTax() { TaxCalculator calculator = new TaxCalculator(); double calculatedTax = calculator.calculateTax(income); assertThat(calculatedTax, is(expectedTax)); }}

This is a parameterized test

The @Parameters annotation indicates the test data

The constructor takes the fields from the test data

The unit tests use data from these fields.

Page 26: Developer Testing Tools Roundup

Parameterized Tests in EclipseRun the test only onceEclipse displays a result for each data set

Income Expected Tax$0.00 $0.00$10,000.00 $1,250.00$14,000.00 $1,750.00$14,001.00 $1,750.21$45,000.00 $8,260.00$48,000.00 $8,890.00$48,001.00 $8,890.33$65,238.00 $14,578.54$70,000.00 $16,150.00$70,001.00 $16,150.38$80,000.00 $19,950.00$100,000.00 $27,550.00

Using Parameterized Tests

Page 27: Developer Testing Tools Roundup

Using Parameterized TestsExample: using an Excel Spreadsheet

@Parameterspublic static Collection spreadsheetData() throws IOException { InputStream spreadsheet = new FileInputStream("src/test/resources/aTimesB.xls"); return new SpreadsheetData(spreadsheet).getData();}

Page 28: Developer Testing Tools Roundup

Using Existing and Custom JUnit RulesCustomize and control how JUnit behaves

JUnit Rules

Page 29: Developer Testing Tools Roundup

The Temporary Folder Rule

JUnit Rules

public class LoadDynamicPropertiesTest {

@Rule public TemporaryFolder folder = new TemporaryFolder();

private File properties;

@Before public void createTestData() throws IOException { properties = folder.newFile("messages.properties"); BufferedWriter out = new BufferedWriter(new FileWriter(properties)); // Set up the temporary file out.close(); }

@Test public void shouldLoadFromPropertiesFile() throws IOException { DynamicMessagesBundle bundle = new DynamicMessagesBundle(); bundle.load(properties); // Do stuff with the temporary file }}

Create a temporary folder

Use this folder in the tests

The folder will be deleted afterwards

Prepare some test data

Page 30: Developer Testing Tools Roundup

JUnit RulesThe ErrorCollector RuleReport on multiple error conditions in a single test

public class ErrorCollectorTest {

@Rule public ErrorCollector collector = new ErrorCollector(); @Test public void testSomething() { collector.addError(new Throwable("first thing went wrong")); collector.addError(new Throwable("second thing went wrong")); String result = doStuff(); collector.checkThat(result, not(containsString("Oh no, not again"))); }

private String doStuff() { return "Oh no, not again"; }}

Two things went wrong here

Check using Hamcrest matchers

Page 31: Developer Testing Tools Roundup

public class ErrorCollectorTest {

@Rule public ErrorCollector collector = new ErrorCollector(); @Test public void testSomething() { collector.addError(new Throwable("first thing went wrong")); collector.addError(new Throwable("second thing went wrong")); String result = doStuff(); collector.checkThat(result, not(containsString("Oh no, not again"))); }

private String doStuff() { return "Oh no, not again"; }}

JUnit RulesThe ErrorCollector RuleReport on multiple error conditions in a single test

All three error messages are reported

Page 32: Developer Testing Tools Roundup

JUnit RulesThe Timeout RuleDefine a timeout for all tests

public class GlobalTimeoutTest {

@Rule public MethodRule globalTimeout = new Timeout(1000); @Test public void testSomething() { for(;;); }

@Test public void testSomethingElse() { }}

No test should take longer than 1 second

Oops

Page 33: Developer Testing Tools Roundup

Parallel testsSetting up parallel tests with JUnit and Maven

<project...> <plugins> ... <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.5</version> <configuration> <parallel>methods</parallel> </configuration> </plugin> </plugins> ... <build> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId>

<version>4.8.1</version> <scope>test</scope> </dependency> </dependencies> </build> ...</project>

Needs JUnit 4.8.1 or better

Needs Surefire 2.5

‘methods’, ‘classes’, or ‘both’

Page 34: Developer Testing Tools Roundup

Continuous TestingContinuous Tests with InfinitestInfinitest is a continuous test tool for Eclipse and IntelliJRuns your tests in the background when you save your code

Page 35: Developer Testing Tools Roundup

Continuous TestingUsing InfinitestWhenever you save your file changes, unit tests will be rerun

Error message about the failed test

Project containing an error

Failing test

Page 36: Developer Testing Tools Roundup

Mocking with styleMockito - lightweight Java mocking

Account

balancenumbergetFees(feeType)

import static org.mockito.Mockito.*;....Account accountStub = mock(Account.class);when(accountStub.getFees(FeeType.ACCOUNT_MAINTENANCE)).thenReturn(4.00);when(accountStub.getFees(FeeType.TRANSACTION_FEE)).thenReturn(0.50);

assertThat(accountStub.getFees(FeeType.TRANSACTION_FEE), is(0.50));

Low-formality mocking

Page 37: Developer Testing Tools Roundup

Mocking with style

Mockito - lightweight Java mocking

AccountDao accountDao = mock(AccountDao.class);Account newAccount = mock(Account.class);when(accountDao.createNewAccount(123456)).thenReturn(newAccount) .thenThrow(new AccountExistsException() );

Manage successive calls

AccountDao

createNewAccount(String id)

Page 38: Developer Testing Tools Roundup

Mocking with style

Mockito - lightweight Java mocking

@Mock private AccountDao accountDao...

Mockito annotations

AccountDao

createNewAccount(String id)

Page 39: Developer Testing Tools Roundup

Mocking with style

Mockito - lightweight Java mocking

when(accountStub.getEarnedInterest(intThat(greaterThan(2000)))).thenReturn(10.00);

Account

balancenumbergetEarnedInterest(year)

Use matchers

Page 40: Developer Testing Tools Roundup

Mocking with style

Mockito - lightweight Java mocking

@Mock private AccountDao accountDao...// Test stuff...verify(accountDao).createNewAccount( (String) isNotNull());

Verify interactions

AccountDao

createNewAccount(String id)

Page 41: Developer Testing Tools Roundup

Spock - Unit BDD in GroovySpecifications in Groovyimport spock.lang.Specification;

class RomanCalculatorSpec extends Specification { def "I plus I should equal II"() { given: def calculator = new RomanCalculator() when: def result = calculator.add("I", "I") then: result == "II" }}

Specifications, not tests

Page 42: Developer Testing Tools Roundup

Spock - Unit BDD in GroovySpecifications in Groovy def "I plus I should equal II"() { when: "I add two roman numbers together" def result = calculator.add("I", "I") then: "the result should be the roman number equivalent of their sum" result == "II" }

BDD-style

Page 43: Developer Testing Tools Roundup

Spock - Unit BDD in GroovySpecifications in Groovy def "I plus I should equal II"() { when: "I add two roman numbers together" def result = calculator.add("I", "I") then: "the result should be the roman number equivalent of their sum" result == "II" }

BDD-style

This is the assert

I plus I should equal II(com.wakaleo.training.spocktutorial.RomanCalculatorSpec) Time elapsed: 0.33 sec <<< FAILURE!Condition not satisfied:

result == "II"| |I false 1 difference (50% similarity) I(-) I(I)

at com.wakaleo.training.spocktutorial .RomanCalculatorSpec.I plus I should equal II(RomanCalculatorSpec.groovy:17)

Page 44: Developer Testing Tools Roundup

Spock - Unit BDD in GroovySpecifications in Groovydef "The lowest number should go at the end"() { when: def result = calculator.add(a, b)

then: result == sum where: a | b | sum "X" | "I" | "XI" "I" | "X" | "XI" "XX" | "I" | "XXI" "XX" | "II" | "XXII" "II" | "XX" | "XXII" }

Data-driven testing

Page 45: Developer Testing Tools Roundup

Spock - Unit BDD in GroovySpecifications in Groovydef "Messages published by the publisher should only be received by active subscribers"() {

given: "a publisher" def publisher = new Publisher()

and: "some active subscribers" Subscriber activeSubscriber1 = Mock() Subscriber activeSubscriber2 = Mock()

activeSubscriber1.isActive() >> true activeSubscriber2.isActive() >> true

publisher.add activeSubscriber1 publisher.add activeSubscriber2

and: "a deactivated subscriber" Subscriber deactivatedSubscriber = Mock() deactivatedSubscriber.isActive() >> false publisher.add deactivatedSubscriber

when: "a message is published" publisher.publishMessage("Hi there")

then: "the active subscribers should get the message" 1 * activeSubscriber1.receive("Hi there") 1 * activeSubscriber2.receive({ it.contains "Hi" })

and: "the deactivated subscriber didn't receive anything" 0 * deactivatedSubscriber.receive(_)}

Setting up mocks

Asserts on mocks

Page 46: Developer Testing Tools Roundup

Geb - Groovy Page Objects

DSL for WebDriver web testingimport geb.*   Browser.drive("http://google.com/ncr") {  assert title == "Google"   // enter wikipedia into the search field  $("input", name: "q").value("wikipedia")   // wait for the change to results page to happen  // (google updates the page without a new request)  waitFor { title.endsWith("Google Search") }   // is the first link to wikipedia?  def firstLink = $("li.g", 0).find("a.l")  assert firstLink.text() == "Wikipedia"   // click the link  firstLink.click()   // wait for Google's javascript to redirect  // us to Wikipedia  waitFor { title == "Wikipedia" }}

Concise expression language

Higher level than WebDriver

Power asserts

Page 47: Developer Testing Tools Roundup

The story of your app

ATDDor Specification by example

Page 48: Developer Testing Tools Roundup

As a job seekerI want to find jobs in relevant categoriesSo that I can find a suitable job

User stories

Features/Epics

Page 49: Developer Testing Tools Roundup

As a job seekerI want to find jobs in relevant categoriesSo that I can find a suitable job

User stories

☑  The  job  seeker  can  see  available  categories  on  the  home  page☑  The  job  seeker  can  look  for  jobs  in  a  given  category☑  The  job  seeker  can  see  what  category  a  job  belongs  to

Acceptance criteria

Page 50: Developer Testing Tools Roundup

As a job seekerI want to find jobs in relevant categoriesSo that I can find a suitable job

User stories

☑  The  job  seeker  can  see  available  categories  on  the  home  page☑  The  job  seeker  can  look  for  jobs  in  a  given  category☑  The  job  seeker  can  see  what  category  a  job  belongs  to

scenario "A job seeker can see the available job categories on the home page",{ when "the job seeker is looking for a job", then "the job seeker can see all the available job categories"}

Automated acceptance test

Acceptance criteria

Page 51: Developer Testing Tools Roundup

scenario "A job seeker can see the available job categories on the home page",{ when "the job seeker is looking for a job", then "the job seeker can see all the available job categories"}

Implemented development tests Implemented acceptance tests

Automated acceptance test

Page 52: Developer Testing Tools Roundup

or how not to have web tests like this

The art of sustainable web tests

Page 53: Developer Testing Tools Roundup

The Three Ways of Automated Web Testing

Record/Replay

Scripting

Page Objects

Page 54: Developer Testing Tools Roundup

Record-replay automated tests

Promise Reality

Page 55: Developer Testing Tools Roundup

Record-replay automated tests

Page 56: Developer Testing Tools Roundup

Script-based automated tests

Selenium RC

HTMLUnit

Canoe Webtest

JWebUnit

Watir

Page 57: Developer Testing Tools Roundup

Script-based automated tests

Selenium RC

HTMLUnit

Canoe Webtest

JWebUnit

Watir

Page 58: Developer Testing Tools Roundup

How about Page Objects?

Hide unnecessary detail

Reusable

Low maintenance

2

Page 59: Developer Testing Tools Roundup

A sample Page ObjectA web page

Page 60: Developer Testing Tools Roundup

A sample Page Object

lookForJobsWithKeywords(values : String)getJobTitles() : List<String>

FindAJobPage

A Page Object

Page 61: Developer Testing Tools Roundup

A sample Page Object

public class FindAJobPage extends PageObject {

WebElement keywords; WebElement searchButton;

public FindAJobPage(WebDriver driver) { super(driver); }

public void lookForJobsWithKeywords(String values) { typeInto(keywords, values); searchButton.click(); }

public List<String> getJobTitles() { List<WebElement> tabs = getDriver() .findElements(By.xpath("//div[@id='jobs']//a")); return extract(tabs, on(WebElement.class).getText()); }}

An implemented Page Object

Page 62: Developer Testing Tools Roundup

A sample Page Object

public class WhenSearchingForAJob {

@Test public void searching_for_a_job_should_display_matching_jobs() { FindAJobPage page = new FindAJobPage(); page.open("http://localhost:9000"); page.lookForJobsWithKeywords("Java"); assertThat(page.getJobTitles(), hasItem("Java Developer")); }}

A test using this Page Object

Page 63: Developer Testing Tools Roundup

Sustainable web tests

Are we there yet?

Page 64: Developer Testing Tools Roundup

The high-level view

Acceptance Tests

So  where  are  we  at?

Page 65: Developer Testing Tools Roundup

Page Objects

Implementation focus

Page  Objects  rock!

Page 66: Developer Testing Tools Roundup

How do we bridge the gap?

Page 67: Developer Testing Tools Roundup

How do we bridge the gap?

Test steps

Page 68: Developer Testing Tools Roundup

scenario "A job seeker can see the available job categories on the home page",{ when "the job seeker is looking for a job", then "the job seeker can see all the available job categories"}

Automated

scenario "The user can see the available job categories on the home page",{ when "the job seeker is looking for a job", { job_seeker.open_jobs_page() } then "the job seeker can see all the available job categories", { job_seeker.should_see_job_categories "Java Developers", "Groovy Developers" }}

Implemented

JobSeekerSteps

open_jobs_page()should_see_job_categories(String...  categories)...

JobSeekerSteps

open_jobs_page()should_see_job_categories(String...  categories)...

JobSeekerSteps

open_jobs_page()should_see_job_categories(String...  categories)...

Step libraries

Page 69: Developer Testing Tools Roundup

scenario "The user can see the available job categories on the home page",{ when "the job seeker is looking for a job", { job_seeker.open_jobs_page() } then "the job seeker can see all the available job categories", { job_seeker.should_see_job_categories "Java Developers", "Groovy Developers" }}

JobSeekerSteps

open_jobs_page()should_see_job_categories(String...  categories)...

JobSeekerSteps

open_jobs_page()should_see_job_categories(String...  categories)...

JobSeekerSteps

open_jobs_page()should_see_job_categories(String...  categories)...

Step libraries

Page Objects

Implemented Tests

Page 70: Developer Testing Tools Roundup

help organize your tests

Test steps

Page 71: Developer Testing Tools Roundup

are a communication tool

Test steps

Page 72: Developer Testing Tools Roundup

are reusable building blocks

Test steps

Page 73: Developer Testing Tools Roundup

Test steps

help estimate progress

Page 74: Developer Testing Tools Roundup

And so we built a tool...

Page 75: Developer Testing Tools Roundup

Webdriver/Selenium 2 extension

Organize tests, stories and features

Measure functional coverage

Record/report test execution

Page 76: Developer Testing Tools Roundup
Page 77: Developer Testing Tools Roundup

Thucydides in action A simple demo app

Page 78: Developer Testing Tools Roundup

Defining your acceptance tests

scenario "The administrator deletes a category from the system",{ given "a category needs to be deleted", when "the administrator deletes a category", then "the system will confirm that the category has been deleted", and "the deleted category should no longer be visible to job seekers",}

focus on business value

scenario "The administrator adds a new category to the system",{ given "a new category needs to be added to the system", when "the administrator adds a new category", then "the system should confirm that the category has been created", and "the new category should be visible to job seekers",}

...defined in business terms

scenario "A job seeker can see the available job categories on the home page",

{ when "the job seeker is looking for a job", then "the job seeker can see all the available job categories"}

High level requirements...

Page 79: Developer Testing Tools Roundup

Organizing your requirements

public class Application {

@Feature public class ManageCompanies { public class AddNewCompany {} public class DeleteCompany {} public class ListCompanies {} }

@Feature public class ManageCategories { public class AddNewCategory {} public class ListCategories {} public class DeleteCategory {} }

@Feature public class BrowseJobs { public class UserLookForJobs {} public class UserBrowsesJobTabs {} }}

Features

Stories

Page 80: Developer Testing Tools Roundup

Implementing your acceptance testsusing "thucydides"thucydides.uses_steps_from AdministratorStepsthucydides.uses_steps_from JobSeekerStepsthucydides.tests_story AddNewCategory

scenario "The administrator adds a new category to the system",{ given "a new category needs to be added to the system", { administrator.logs_in_to_admin_page_if_first_time() administrator.opens_categories_list() } when "the administrator adds a new category", { administrator.selects_add_category() administrator.adds_new_category("Scala Developers","SCALA") } then "the system should confirm that the category has been created", { administrator.should_see_confirmation_message "The Category has been created" } and "the new category should be visible to job seekers", { job_seeker.opens_jobs_page() job_seeker.should_see_job_category "Scala Developers" }}

We are testing this story

Narrative style

Step through an example

An acceptance criteria

Still high-level

Page 81: Developer Testing Tools Roundup

Some folks prefer JUnit...@RunWith(ThucydidesRunner.class)@Story(AddNewCategory.class)public class AddCategoryStory {

@Managed public WebDriver webdriver;

@ManagedPages(defaultUrl = "http://localhost:9000") public Pages pages;

@Steps public AdministratorSteps administrator;

@Steps public JobSeekerSteps job_seeker;

@Test public void administrator_adds_a_new_category_to_the_system() { administrator.logs_in_to_admin_page_if_first_time(); administrator.opens_categories_list(); administrator.selects_add_category(); administrator.adds_new_category("Java Developers","JAVA"); administrator.should_see_confirmation_message("The Category has been created");

job_seeker.opens_job_page(); job_seeker.should_see_job_category("Java Developers"); }

@Pending @Test public void administrator_adds_an_existing_category_to_the_system() {}}

Thucydides handles the web driver instances

Using the same steps

Tests can be pending

Page 82: Developer Testing Tools Roundup

Defining your test stepspublic class AdministratorSteps extends ScenarioSteps {

@Step public void opens_categories_list() { AdminHomePage page = getPages().get(AdminHomePage.class); page.open(); page.selectObjectType("Categories"); }

@Step public void selects_add_category() { CategoriesPage categoriesPage = getPages().get(CategoriesPage.class); categoriesPage.selectAddCategory(); }

@Step public void adds_new_category(String label, String code) { EditCategoryPage newCategoryPage = getPages().get(EditCategoryPage.class); newCategoryPage.saveNewCategory(label, code); }

@Step public void should_see_confirmation_message(String message) { AdminPage page = getPages().get(AdminPage.class); page.shouldContainConfirmationMessage(message); }

@StepGroup public void deletes_category(String name) { opens_categories_list(); displays_category_details_for(name); deletes_category(); }}

A step library

High level steps...

...implemented with Page Objects

...or with other steps

Page 83: Developer Testing Tools Roundup

Defining your page objectspublic class EditCategoryPage extends PageObject {

@FindBy(id="object_label") WebElement label;

@FindBy(id="object_code") WebElement code;

@FindBy(name="_save") WebElement saveButton;

public EditCategoryPage(WebDriver driver) { super(driver); }

public void saveNewCategory(String labelValue, String codeValue) { typeInto(label, labelValue); typeInto(code, codeValue); saveButton.click(); }}

Provides some useful utility methods...

but otherwise a normal WebDriver Page Object

Page 84: Developer Testing Tools Roundup

Data-driven testing

categories.csv

public class DataDrivenCategorySteps extends ScenarioSteps { public DataDrivenCategorySteps(Pages pages) { super(pages); }

private String name; private String code;

@Steps public AdminSteps adminSteps;

public void setCode(String code) {...} public void setName(String name) {...}

@Step public void add_a_category() { adminSteps.add_category(name, code); }}

Test data

Test steps

Page 85: Developer Testing Tools Roundup

Data-driven testing

categories.csv

public class DataDrivenCategorySteps extends ScenarioSteps { public DataDrivenCategorySteps(Pages pages) { super(pages); }

private String name; private String code;

@Steps public AdminSteps adminSteps;

public void setCode(String code) {...} public void setName(String name) {...}

@Step public void add_a_category() { adminSteps.add_category(name, code); }}

@Steps public DataDrivenCategorySteps categorySteps;

@Test public void adding_multiple_categories() throws IOException { steps.login_to_admin_page_if_first_time(); steps.open_categories_list();

withTestDataFrom("categories.csv").run(categorySteps).add_a_category(); }

Test data

Test steps

Call this step for each row

Page 86: Developer Testing Tools Roundup

Now run your tests

Page 87: Developer Testing Tools Roundup

Displaying the results in easyb

Page 88: Developer Testing Tools Roundup

Displaying the results in easyb

Page 89: Developer Testing Tools Roundup

Thucydides reports

Browse the features

Page 90: Developer Testing Tools Roundup

Browse the stories

Browse the stories

Page 91: Developer Testing Tools Roundup

Browse the stories

Browse the test scenarios

Page 92: Developer Testing Tools Roundup

Illustrating the test paths

Test scenarios

A test scenario

Steps

Page 93: Developer Testing Tools Roundup

Illustrating the test paths

Test scenarios

Test scenario

Steps

Screenshots

Page 94: Developer Testing Tools Roundup

Functional coverage

Features

Page 95: Developer Testing Tools Roundup

Functional coverage

User stories

Page 96: Developer Testing Tools Roundup

Functional coverage

Passing tests

Failing tests

Pending tests

Page 97: Developer Testing Tools Roundup

Functional coverage

Test scenarios

Page 98: Developer Testing Tools Roundup

Functional coverage

Page 99: Developer Testing Tools Roundup

Thank you!

John Ferguson SmartEmail: [email protected]

Web: http://www.wakaleo.comTwitter: wakaleo