How BDD style Unit Testing helps you write better

test code (and some caveat)

@ihower at RubyKaigi 2011 Ruby Taiwan & OptimisDev

About Me

• Chang, Wen-Tien 張文鈿

• a.k.a. ihower

• http://ihower.tw

• http://twitter.com/ihower

I’m ihower and here is my blog and twitterI have program ruby from 2006 for ruby on railsAnd I’m Yet another rubyist since rails.

From Taiwan

3.5hrs

I come from Taiwan which is far away about 4hours flightI’m sorry that I can’t speak Japanese, and English is not my native languageIf I speak Chinese, I guess you will escape now. So please forgive my english. I hope it’s simple to understand.

I work at Optimis International, which is an america company in Taipei

Ruby Taiwanhttp://ruby.tw

Beside daily job, I’m the founder and organizer of Ruby Taiwan community, I found the community since 2008.

RubyConf Taiwan 2011/8/26-27http://rubyconf.tw

We are organizing rubyconf taiwan now. The date is augest 26 and 27 and it open registration now.Welcome to Taiwan.

Last year photo. Nice small meeting room. pretty sweet.

Agenda

• Why Unit Testing?

• How to do unit test in BDD style?

• Some BDD caveat

Today’s agenda. There will be three part:First, I will talk about why, then how to do, finally talk about some caveat.

How many people do unit test?

Before get into BDD, how many people here do unit test all the way?Could you raise your hand?

How many people use BDD framework

(RSpec)?

And how many people use rspec?

VerificationAre we writing the code right?Unit Test

Customer Test(Acceptance Test)

ValidationAre we writing the right software?

Two kinds of test:

There are roughly two kinds of test:Unit test and Customer test(or called Acceptance test)Unit test is about verification, the question is are we writing the code right?Customer test is about validation, the question is are we writing the right software?

Test individual class and methodUnit Test

Customer Test(Acceptance Test)

Test functionality of the over-all system

Two kinds of test:

Unit test test individual class and methodAnd Customer Test test functionality of the over-all system

Today is all about Unit Testing

Today is all about unit testing

1. Why Unit Testing?

Let get started part one

1. VerificationDo we write the code right?

First, it does verification

How to verify this?There are three execution path

def correct_year_month(year, month)

if month > 12 if month % 12 == 0 year += (month - 12) / 12 month = 12 else year += month / 12 month = month % 12 end end return [year, month] end

For example, how to verify this simple if else code.There are three execution path which means if you manually testyou need play it 3 times in different way

via UI?

via irb console?

Manually test is inefficient

Unit testing give us instant coding feedback

it "should return correct year and month" do correct_year_month(2011, 3).should == [2011, 3] correct_year_month(2011, 14).should == [2012,2] correct_year_month(2011, 24).should == [2012,12]end

automatic unit testing can give us instant feedbackyou can test many times you want.

2.Check regressionSafety Net: when you add new feature and refactoring

test also check the regressionit’s a safety net when you add new feature or refactoringIt can give you confidence that you won’t break up any existed behavior

3.API designguide you write better API, especially test first.

third, Since test code is the client for your production code,so if the test code is hard to write, it may means your API is not good.Writing unit testing can guide you write better API, especially when you write test first.

4.DocumentationWhat the result should be?

Communication is the key for software development.

finally, uniting test can be your code documentation.It can answer you what’s the class and method result should be?Communication the key for successful software development.

Brief Summary

• Your code is not trivial: Automatic test can save your verify/debug time

• Your code is not throwing way:Regression test

• Test first: Guide you write better API

• Tests as Documentation

2. How to do unit test in BDD style?

xUnit Framework

• Each test has four-phase:

• Setup

• Exercise

• Verify (assertion)

• TeardownFirst, We use xUnit framework to test code. Each test should be isolation and has four phase:1. Setup Data2. Exercise production code3. Verify the result 4. Cleanup the data

Ruby Test::Unitclass OrderTest < Test::Unit::TestCase

def setup @order = Order.new end def test_order_amount assert_equal 0, @order.amount end end

1

23

The is the classical ruby test::unit codeRed 1 is setup Red 2 is exerciseRed 3 is verify. We assert the result equal to zero

Minitest::Specdescribe Order do

before do @order = Order.new end it "should have default amount is 0" do @order.amount.must_equal 0 end

end

1

2 3

Here is the BDD style using minitest::specred 1 is setupred 2 is exercisered 3 is verify. the result must equal to zero

First impression about BDD

• Syntax is different

• Like specification

• More readable

So, what’s the first impression about bdd style?Ok, the syntax is different, it looks like spec and may more readable Otherwise the structure is not much different

What’s BDD?

• An improved xUnit Framework

• Focus on clearly describe the expected behavior

• The emphasis is Tests as Documentation rather than merely using tests for verification.

Let’s back to see what’s BDD means.It’s not totally new test framework, it just an improved xUnit frameworkIt focus on describe the expected behaviorIt emphasize tests as documentation, not just for verification.

Terminology changedNew paradigm: Executable Specification

• “Test” becomes “Spec”

• “Assertion” becomes “Expectation”

• “test method” becomes “example” (RSpec)

• “test case” becomes “example group” (RSpec)

For correspond to the new paradigm: the test is the executable specificationThe “test” becomes “spec”“Assertion” becomes “Expectation”

Ruby BDD tools

• RSpec

• Minitest/Spec

In Ruby, there are some choices:RSpec is the most powerful, has many features and many awesome syntax.Minitest/spec is the Ruby 1.9 build-in xunit library. It’s simple, fast, and just works!

Learn BDD style

• It’s about syntax

• syntax

• syntax

• syntax

Let’s learn how to write BDD style test code.Basically, it’s just learn about syntax.

BDD style element (1)Context

one test case

a class or string

describe Order do # ...end

# or

describe "A Order" do # ...end

Instead of test case class, BDD style use “describe” block for one test caseIt accept parameter which means the stuff you want to describe, usually a class name or just description text.

Can be nested

#method means instance method

.method means class method

describe Order do

describe "#amount" do # ... end

describe ".size" do # ... end

end

“describe” block can be nested.And inner “describe” usually describe method.Pound means instance methoddot means class method

In RSpec,alias_method :context, :describe

describe Order do

context "when initialized" do describe "#amount" do # ... end

describe ".size" do # ... end

end

end

1

2

3

3

Is RSpec, it also alias describe to context.it used to organize test code into different context.

BDD style element (2)Declarative test

one test method: ityou can write test description in plain text

describe Order do

describe "#amount" do describe "when user is vip" do it "should discount five percent if total > 1000" do # ... end it "should discount ten percent if total > 10000" do # ... end end end

Each test method is “it” blockAnd it accept one parameter which is description text

Setup method: beforedescribe Order do

before do @user = User.new( :is_vip => true ) @order = Order.new( :user => @user ) end end

the setup method is “before” block

describe Order do

before do @user = User.new( :is_vip => true ) @order = Order.new( :user => @user ) end describe "#amount" do before do @order2 = Order.new( :user => @user ) end end end

nested beforebtw, the outer data will be shared, make the test faster

1

2

“before” block also support nestedMore important, the outer setup data will be shard for all inside test methods.Make the test faster since we need not create again every time.

https://github.com/citrusbyte/contest

class SomeTest < Test::Unit::TestCase setup do @value = 1 end

test "sample test" do assert_equal 1, @value end

context "a context" do setup do @value += 1 end

test "more tests" do assert_equal 2, @value end

endendThere are library which make test/unit support context and declarative test ,

like contest

https://github.com/jm/context

class UserTest < Test::Unit::TestCase

context "A new User" do before do @user = User.first end

test "should have the right full_name" do assert_equal "Dude Man", @user.full_name end end

Or context

Rails support declarative test(but no nested context)

class PostsControllerTest < ActionController::TestCase

setup @post = posts(:one) end test "should show post" do get :show, :id => @post.id assert_response :success end

end

Even rails, It supports declarative test syntax. It use “test” block But no nested context.

BDD style element (3)Expectation

Expectation(Assertion)Minitest/Spec

@order.amount.must_equal 1900@order.amount.wont_equal 2000lambda{ order.ship! }.must_raise NotPaidError

BDD style assertion is called expectation.In menitest/spec, the methods are must_equal, wont_equal, must_raiseIn unit/test, some people may confuse assert method that first parameter and second parameter, which one is actual and which one is expect. But BDD style, it’s impossible to confuse it.

Expectation(Assertion)RSpec

@order.amount.should == 1900@order.amount.should_not == 2000lambda{ order.ship! }.should_raise NotPaidError# orexpect { order.ship! }.to raise_error(NotPaidError)

Here is rspec version. should equal and should_not equal

Matchyhttps://github.com/jm/matchy

x.should == 42y.length.should_not be(4)lambda { raise "FAIL" }.should raise_error

There is also library can make test/unit support bdd style expectation only.

BDD cares aboutthe output

Output (rspec)

Really nice code documentation

RSpec output :)

RSpec does the output very well, you can see really nice code documentation

HTML format

Minitest Output :(

But I can’t not find how to generate code documentation in minitest/spec

require 'minitest/pride'# Show your testing pride!

I found there is a minitest/pride you can required! It may help format the output, so I try it!

Very funny @_@

And here is the output....... very funny. Good job!

Use turn gemgem 'minitest' # use gem version instead of stdlibrequire 'minitest/spec'require 'turn'

MiniTest::Unit.use_natural_language_case_names = trueMiniTest::Unit.autorun

Anyway, There is a gem called trun which used in Rails 3.1It can generate nice test output for unit/testSo here I require it.Btw, I suggest you use the latest minitest gem version, not standard library. Since the gem version is much new. I totally agree tenderlove suggestion yesterday that we should move some standard library into gem.

Minitest::Spec output :)

Here is the output, very nice.

Brief Summary

• Three BDD style elements:

• context

• declarative test

• expectation

• test output can be code documentation

So...

Why Syntax Matters?

http://www.nytimes.com/2010/08/29/magazine/29language-t.html

Your language shape how you think!

Language will not limit you, but it does influence how you think.

For example:

test_datetime_format

# ruby/test/logger/test_logger.rbclass TestLogger < Test::Unit::TestCase

def test_datetime_format dummy = STDERR logger = Logger.new(dummy) log = log_add(logger, INFO, "foo") assert_match(/^\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d.\s*\d+ $/, log.datetime) logger.datetime_format = "%d%b%Y@%H:%M:%S" log = log_add(logger, INFO, "foo") assert_match(/^\d\d\w\w\w\d\d\d\d@\d\d:\d\d:\d\d$/, log.datetime) logger.datetime_format = "" log = log_add(logger, INFO, "foo") assert_match(/^$/, log.datetime) end

end

This is a test code from ruby core for Loggerit test datetime_format method

the same test in rubyspec# rubyspec/logger/logger/datetime_format_spec.rbdescribe "Logger#datetime_format" do # ... it "returns the date format used for the logs" it "returns nil logger is using the default date format"end

describe "Logger#datetime_format=" do # ...

it "sets the date format for the logs" do format = "%Y" @logger.datetime_format = "%Y" @logger.datetime_format.should == "%Y" @logger.add(Logger::WARN, "Test message") @log_file.rewind

regex = /2[0-9]{3}.*Test message/ @log_file.readlines.first.should =~ regex end

it "follows the Time#strftime format" endThis is the same test in rubyspec

you can see it divide into many describe and it block.

In this case:

• Test::Unit version just do verify

• Spec version not only verify but also describe its behavior:

• It help us understand what’s the method should do

• It increases the test coverage

The BDD syntax guides you focus on the

expected behavior

Emphasis on Tests as Documentation

Rather than merely using tests for

verification

3. BDD style caveat

1. Too magic?

Metaprogramming?

• The BDD framework using metaprograming should not be the problem.

• The problem is some (RSpec) advanced awesome syntax may make the test code hard to understand or not intuitive.

Metaprogramming implementation should be the problem. If it is, then many ruby library is bad including rails.

The problem is some advanced awesome syntax may hard to understand.

describe Order do its(:status) { should == "New" } end

implicit subject(RSpec)

For example. RSpec has a feature callsd implict subject.You can omit the receiver of expectation method.It also has a syntax call its, its...... its.... well, hard to explain... let see another equal version

describe Order do before do @order = Order.new end it "should have default status is New" do @order.status.should == "New" end end

Equal to this version:

oh.... I see...its will covert Order instance and call its method

well, seems magic!

describe PostsController do describe "PUT #update" do before do @post = Factory(:post) end it "allows an author to edit a post" do sign_in @post.author put :update, :id => post.id, :body => "new content"

response.should be_successful end

it "does not allow a non-author to edit a post" do sign_in Factory(:user) put :update, :id => post.id, :body => "new content"

response.should be_forbidden end endend

would like DRY?

1

2

3

2

3

Another example to test rails controller:red 1 is setup partred 2 is exercisered 3 is verify

The red 2 exercise are almost the same, so some people may want to DRY.

But readable?describe PostsController do describe "PUT #update" do let(:post) { Factory(:post) }

before do sign_in user put :update, :id => post.id, :body => "new content" end

context "when logged in as post's author" do let(:user) { post.author }

it "allows the post to be updated" do response.should be_success end end

context "when logged in as another user" do let(:user) { Factory(:user) }

it "does not allow the post to be updated" do response.should be_forbidden end end endend

3

3

2

1

So they move the code into before blockand rspec has a feature called “let”, it’s a lazy and memorized method for setup data.

Finally in the test method, Only left assertion: response.should be_success

Four-phase Test form is key to understand the

test code• Setup

• Exercise

• Verify (assertion)

• Teardown

Why? I think for test code:four-phase test form is the key to understand.

When I see one test method, I must need to figure out what’s the setup, exercise and verify.

So, it should be easy to trace.

2.English-like DSL?http://pragdave.blogs.pragprog.com/pragdave/2008/03/

the-language-in.html

Second caveat is english-like DSL.Dave Thomas has a nice article explain this problem.

“The language in a DSL should be the language of the domain, not the natural

language of the developer. “ by DaveThomas

He said...

so... To be english-like is not DSL’s purpose. The easy to use and readable DSL API is the purpose

# Baconit 'should have an object identity' do @ary.should.not.be.same_as Array.newend

the be is just “self”

Let’s see one obvious example. This is a bacon test code. There is a be method inside method chain, but it does nothing and just return self.

It existed only because we want to make it like english. Is it right?

RSpec matcher (1)

target.should be_true # targer.should == truetarget.should be_false # targer.should == falsetarget.should be_nil # targer.should == nil

target.should be_a_kind_of(Array) # target.class.should == Arraytarget.should be_an_instance_of(Array) # target.class.should == Array

On the other hand, RSpec has core feature called matcher for expectation.It also make the test code readable. In some way, It’s also very easy to use.

like * be_true, * be_false * be_a_kind_of

RSpec matcher (2)target.should respond_to(:foo) # target.repond_to?(:foo).should == true

target.should have_key(:foo) # target[:foo].should_not == nil

target.should include(4) # target.include?(4).should == true

target.should have(3).items # target.items.length == 3

There are more example like

* have_key* include

RSpec matcher (3)

target.should be_empty # target.empty?.should == true

target.should be_blank # target.blank?.should == true

target.should be_admin # target.admin?.should == true

any prefix be_ matcher will covert it to that method with question mark.

RSpec matcher is very powerful

# Custom Matcherdescribe 9 do it "should be a multiple of 3" do 9.should be_a_multiple_of(3) # (9 % 3).should == 0 endend

# RSpec supports custom matchers DSLRSpec::Matchers.define :be_a_multiple_of do |expected| match do |actual| actual % expected == 0 endend

RSpec matcher is very powerful since it even provide DSL to define your custom matcher which make the your test readable.

Some matchers I like, but some I dislike.

It becomes personal taste :|

3. Deep nested context?describe Order do before do @order = Order.new end describe "#amount" do before do @order2 = Order.new( :total => 2000) end describe "when user is vip" do before do @user = User.new(:is_vip => true) end it "should discount five percent if total >= 1000" do # ... end end endend

1

1

1

There are three nested context here. For the most inner test method, it has three before block outside.

Deep nested is hatefulIt’s hard to understand what’s setup going on in sub-sub-sub-sub context

Deep nested context is hateful, It hard to trace the setup part when there is sub-sub-sub-sub context

Four-phase Test form is key to understand the

test code• Setup

• Exercise

• Verify (assertion)

• Teardown

Again. Four-phase Test form:

Deep context make setup part hard to trace.

That’s why Rails does not support nested context.

Rails team like flat test structure.

4. Too mockist?

• rspec-rails gem divides tests into model,controller,view,helper and encourages you use mocks.

• should_receive easily make people test implementation rather than result by default. Make your test fragile.

RSpec ship with very nice mock feature,And RSpec-rails divides tests into four layer which encourage you use mocks.

But mocks has many caveats too, It make the test fragile easily.

5.Performance penalty?

• minitest/spec performance is almost equal to test/unit. It’s really fast.

• RSpec performance is slower comparing to test/unit.

minitest/spec performance is pretty well, you need not to worry about it.

RSpec is slower, But when running test, I think the bottleneck is your production code and the testing framework factor is not big problem.

Conclusion

• BDD testing framework is unlike ActiveRecord for SQL, Rails for web development. It does not “simplify” test. It’s not an abstract tool for testing.

• So, yes, It’s not required and some people think it’s unnecessary.

yes, it’s not required...

But it’s very valuable for your team!

• The BDD syntax guides you focus on the expected behavior, and increasing the test coverage.

• Emphasis on Tests as Documentation rather than merely using tests for verification.

It focus on describe the expected behavior and increase the test coverageIt emphasize tests as documentation, not just for verification.

Thanks for listening!Please tweet me if you have questions:

@ihower

Reference

• RSpec 讓你愛上寫測試 slides, ihower OSDC.TW 2011

• xUnit Test Patterns

• http://avdi.org/devblog/2011/04/07/rspec-is-for-the-literate/

• http://objectreload.com/articles/2010/09/thoughts-on-testing-part-1.html