1

Advanced Testing Concepts& TDD

2

Testing vs. DBC

• DBC: Exhaustive correctness– Post conditions should work on all inputs– Often, highly complicated– Just as error prone as the method itself

• Tests: Sample-based correctness– Associate a sample input with expected output– Much simpler to write, maintain

• In theory, tests examine only a fracture of the input space – How come they work?– (Answer later today)

3

Responsibility of a Test Method

Option 1:Tests one method on several subject objects

• Test methods (in a test class) will tend to use the same set of subjects

• => fields will hold subjects • => @Before will initialize the fields

• (Example: CommonGUIItems_Test)

4

Responsibility of a Test Method (cont.)

Option 2: Test several methods on a single subject object

• Each test method will have its own subject (local variable)• Use a factory method to streamline the creation • Test methods may tend to run the same “testing scenario”

– (but with different inputs/output)

• (Example: Model_Test.java)

5

Testing Interaction Among Objects

• Object x invokes Y.m()

• Y.m()’s effect cannot be easily tested– Encapsulation– Affect behavior but not state– Interacts with external entities (DBs, Sockets, …)

• Solution: Replace Y with a Test Double

6

Test Doubles

A generic term for any case where one replaces a production object for testing

purposes

Source: http://www.martinfowler.com/bliki/TestDouble.html

7

public class Reporter {

private PrintWriter pw; public Reporter(PrintWriter pw) { this.pw = pw; } public void writePerson(String firstName, String lastName) { writePerson(firstName, null, lastName); }

public void writePerson(String firstName, String middleName, String lastName) { middleName == null ? "" : " " + middleName.charAt(0) + "."; pw.println(lastName + ", " + firstName + middleName); } }

8

public class Reporter_Test {

private static class WriterSpy extends PrintWriter { public WriterSpy() { super(System.out); }

@Override public void println(String s) { last = s; } public String last; }

@Test public void test1() { WriterSpy s = new WriterSpy(); Reporter r = new Reporter(s);

r.writePerson("John", "Winston", "Lennon"); Assert.assertEquals("Lennon, John W.", s.last);

}}

9

Kinds of Test Doubles

• Dummy objects: passed around but never actually used. Usually used to fill parameter lists.

• Fake objects: have working implementations, but take some shortcut which makes them not suitable for production

10

Kinds of Test Doubles (cont.)

• Stub object: provide canned answers to calls made during the test, not responding to anything outside the test's needs

• Spy objects: Stubs that also record some information based on how they were called

• Mock object: Pre-programmed with expectations. They can throw an exception if they receive a call they don't expect and are checked during verification to ensure they got all the calls they were expecting.

11

Mocking Example

• Following two slides: A regular, JUnit-only test – Examines interaction of Order, Warehouse

• Following that, the same test with EasyMock

12

public class OrderWarehouse_Test { private static String TALISKER = "Talisker"; private Warehouse warehouse = new WarehouseImpl();

@Before public void init() { warehouse.add(TALISKER, 50); }

@Test public void testOrderIsFilledIfEnoughInWarehouse() { Order order = new Order(TALISKER, 50); order.fill(warehouse); assertTrue(order.isFilled()); assertEquals(0, warehouse.getInventory(TALISKER)); }

...

13

...

@Test public void testOrderDoesNotRemoveIfNotEnough() { Order order = new Order(TALISKER, 51); order.fill(warehouse); assertFalse(order.isFilled()); assertEquals(50, warehouse.getInventory(TALISKER)); }

14

EasyMock

• A library for creating Mock objects– Notable competitor: JMock

• Façade class: EasyMock– Many static methods

• Test class creates mock object – EasyMock.createMock(MyInterface.class)– Returns a mock implementing MyInterface

• Test class records expectations – the messages that will be sent to the mock object– Calls appropriate methods on the mock object

• EasyMock.replay() – stop recording, start responding• EasyMock.verify() – make sure expectations were matched

15

private Warehouse warehouse; private static String TALISKER = "Talisker"; @Before public void init() { warehouse = EasyMock.createMock(Warehouse.class); }

@Test public void testFillingRemovesInventoryIfInStock() { EasyMock.expect(warehouse.hasInventory(TALISKER,50)) .andReturn(true); warehouse.remove(TALISKER, 50); EasyMock.replay(warehouse);

Order order = new Order(TALISKER, 50); order.fill(warehouse); Assert.assertTrue(order.isFilled()); EasyMock.verify(warehouse);}

16

Clients and Providers

• Definition: X is a client of Y if X uses Y– E.g.: A library (provider) is used by a program (client)

• Two types of usage: Static vs. Dynamic– Source code reflects only the static usage– The dominant factor is the dynamic usage

• Providers do not know their clients – Often a provider will be used by multiple clients

• => Adapting client to provider is easier – Than adapting provider to client

• => The basic power scheme: Provider shapes client– Easier to develop a provider– Provider changes something – clients pay the price

17

Testing A Chain of Clients

• A set of classes: X1, X2, ... X9 - all part of a single program

– Xi is a client of Xi+1

– Each class is written by a different person– X9's developer introduced a change that causes a bug in X1

• No testing: – X1’s developer detect the defect.

• Debugs and comes to the conclusion that X2 is responsible

– X2’s developer does the same – blames X3

– Ultimately: 8 developers searched for a bug that does not exist

• Testing: – X9's developer will get a red bar– Will not commit the code. End of story– The powers are now balanced:

provider shapes client, but client tests shape provider

18

The Column of Numbers Metaphor

• Best way to sum up a column of numbers:– Do it twice in two different ways (up, down)– Much better than doing the same way twice – Much better than doing it once

• Double checking in software: Assume that X is a client of Y– Class Y is checked by Y's test– Class Y is checked by X's test

19

High vs. Low• All sort of terms

– Acceptance– System– Functional– Unit testing

• High level testing– Better coverage (for same amount of code)– You have to manually debug

• Low level testing – Less coverage– Quickly pin-pointing bugs

• High level tests compound the effect of “double checking”– That’s why testing is effective – Despite the fact that they examine only a fracture of all inputs

20

TDD: Test Driven Design

• Before any change - writing a test– The test should capture the change you are about to do

• Rationale: – Many times a new class is written to meet an interface– Correct signatures are enforced by the compiler– What about correct behavior?– TDD let's you specify both interface and behavior

21

TDD vs. Documents• Just like use-cases: a description of wishes

– Thus, instead of writing use cases write tests

• Documents lie, tests don't• Comments lie, code doesn't

• Tests will run thousands of times – Ensuring that every version of the program meets its

expectations

• Documents are read at most N times – N = 3? 4? 5?

22

Basic Cycle of TDD– Pre-coding:

• Update from the source repository• Make sure all tests are green

– Coding:• Write tests that capture the new functionality• Make the program compile • Make sure the new tests fail• Write code until the tests are green

– Post-coding• Resolve conflicts with the source repository• Make sure all tests are green• Commit

23

“Pure” TDD

• Write a test for a very simple input

• Make sure it fails

• Write the simplest code that passes the test– Even hardcode

• Repeat with a more complicated input– Refactor after each pass

24

Practices

• Found a bug?– Capture it with a test– while(test_is_failing) keep_fixing

• In a suite, shorter tests should be placed first

• Always verify that you start from Green

• (Think about): Happy & Sad paths

• Testing code should be simpler than subject code