How to Test Asynchronous Code

by Felix Geisendörfer

19.05.2011 (v2)

@felixge

Twitter / GitHub / IRC

Core Contributor

&

Module Author

node-mysql node-formidable

- Joined the mailing list in June 26, 2009- When I joined there where #24 people- Now mailing list has close to 4000 members members- First patch in September 2009

How to Test Asynchronous

Code

- Disclaimer: Don’t listen to me : )

The current approach

test('something', function(done) { doSomethingAsync(function() { assert.equals(...);

done(); });});

The current approach

test('something', function(done) { doSomethingAsync(function() { assert.equals(...);

done(); });});

If you have multiple tests, how will exceptions be linked?

db.query('SELECT A', function() { db.query('SELECT B', function() { db.query('SELECT C', function() { db.query('SELECT D', function() { // WTF }); }); });});

- Who here is running node in production?- Raise hands if you have a test suite for your node.js projects- Raise hands if you are happy with it- But why is it so difficult?

test('something', function(done) { doSomethingAsync(function() { assert.equals(...);

done(); });});

F%$! that shit

Forget that shit

What can we do?

- We looked at asynchronous testing frameworks- We really wanted to do TDD- But everything we tried was hard

Take out the I/O

Stub / Mock Dependencies

Synchronous Tests

Unit TestsIf you have nested callbacks (or any other I/O) in your tests, you are not unit testing

Microtestshttp://anarchycreek.com/2009/05/20/theyre-called-microtests/

• It is short, typically under a dozen lines of code.

• It is always automated.

• It does not test the object inside the running app, but instead in a purpose-built testing application.

• It invokes only a tiny portion of the code, most usually a single branch of a single function.

• It is written gray-box, i.e. it reads as if it were black-box, but sometimes takes advantage of white-box knowledge. (Typically a critical factor in avoiding combinatoric issues.)

• It is coded to the same standard as shipping code, i.e. the team’s best current understanding of coding excellence.

• It is vault-committed source, with a lifetime co-terminous with the functionality it tests.

• In combination with all other microtests of an app, it serves as a ‘gateway-to-commit’. That is, a developer is encouraged to commit anytime all microtests run green, and discouraged (strongly, even nastily) to commit otherwise.

• It takes complete control of the object-under-test and is therefore self-contained, i.e. running with no dependencies on anything other than the testing code and its dependency graph.

• It runs in an extremely short time, milliseconds per test.

• It provides precise feedback on any errors that it encounters.

• It usually (not always) runs entirely inside a single computer.

• It usually (not always) runs entirely inside a single process, i.e. with few extra-process runtime dependencies.

• It is part of a collection all or any subset of which is invokable with a single programmer gesture.

• It is written before the code-change it is meant to test.

• It avoids most or all usage of ‘awkward’ collaborators via a variety of slip-and-fake techniques.

• It rarely involves construction of more than a few classes of object, often one or two, usually under five.

Approach

• Load SUT in a Sandbox using v8 / node.js

• Inject dependencies using the global object of the sandbox

• Don’t use test doubles for internals, only for peers

Internal: An object that has the same life span as its hostPeers: Something that is being passed in / out of the SUT

Test Doubles

• Dummy Object

• Test Stub

• Test Spy

• Mock Object

• Fake Object

• Temporary Test Stub

http://xunitpatterns.com/Mocks,%20Fakes,%20Stubs%20and%20Dummies.html

node-fake

node-fake

var fake = require('fake')();var object = {};

fake.expect(object, 'method');

object.method();

node-fake

var fake = require('fake')();var object = {};

var objectMethodCall = fake.stub(object, 'method');

object.method();

assert.equals(objectMethodCall.calls.length, 1);

node-fake

var fake = require('fake')();

var MyClass = fake.class('MyClass');fake .expect('new', MyClass) .withArgs(1, 2);

var myClass = new MyClass(1, 2);

node-microtest

node-microtest

var fs = require('fs');

module.exports = function(path) { var file = fs.createReadStream(path); file.pipe(process.stdout);};

cat.js

node-microtest

var test = require('microtest').module('cat.js');

test.requires('fs');test.context.process = { stdout: test.object('stdout'),};

var cat = test.compile();

test-cat.js

node-microtesttest.describe('cat', function() { var PATH = test.value('path'); var file = test.object('file');

test .expect(test.required.fs, 'createReadStream') .withArgs(PATH) .andReturn(file);

test .expect(file, 'pipe') .withArgs(test.context.process.stdout);

cat(PATH);});

test-cat.js

Benefits

We take the position that the real benefit of extensive microtest-driven development isn't higher quality at all. Higher quality is a side effect of TDD. Rather, the benefit and real purpose of TDD as we teach it is sheer productivity: more function faster.

Mike Hill

Benefits

• Simpler implementations

• Short change / verification cycles

• Tests for edge cases (error handling, race conditions, etc.)

There really are only two acceptable models of development: "think and analyze" or "years and years of testing on thousands of machines".

Linus Torvalds

Disadvantages

• Requires gray / white box knowledge of implementation

• Lots of test code needs to be written

• Sometimes feels awkward

http://martinfowler.com/articles/mocksArentStubs.html#ClassicalAndMockistTesting

Numbers

node-mysql

node-mysql

0

425

850

1.275

1.700

library tests

Lines of code

library vs test code: 1x : 1.35x

Library: 1240 LoC (+600 LoC MySql constants)Tests:Tests: 1673 LoC

node-mysql

0

325

650

975

1.300

integration micro

integration vs micro tests: 1x : 2.89x

Lines of code

Micro Tests: 1243 LoCIntegration tests: 430 LoC

node-mysql

0

100

200

300

400

integration micro

Assertions

integration vs micro tests: 1x : 8.15x

Micro Tests: 375 assertsIntegration tests: 46 asserts

Transloadit

Transloadit

0

3.250

6.500

9.750

13.000

library tests

Lines of code

library vs test code: 1x : 2.04x

Library: 6184 LoCTests: 12622 LoC

Not included: Fixture data, server configuration, customer website, dependencies we developed

Transloadit

0

2.500

5.000

7.500

10.000

integration micro

integration vs. micro tests: 1x : 4.30x

Lines of code

Integration tests: 2254 LoCMicro Tests: 9695 LoC

Transloadit

0

750

1.500

2.250

3.000

integration micro

Assertions

integration vs. micro tests: 1x : 12.30x

Integration tests: 189 assertsMicro Tests: 2324 asserts

tl;dr

• Don’t use integration tests to show the basic correctness of your software.

• Write more microtests.

Questions?

☝Thank You

Thank You

Node.js Workshop Cologne

nodecologne.eventbrite.com

Use discount code ‘berlinjs’ for 10% off