Testing: 1… 2… 3…An Introduction to Developer Testing

Will Greenwill@hotgazpacho.orghttp://hotgazpacho.org/@hotgazpacho

CaveatI am Not an Expert

http://www.flickr.com/photos/eprater/2375127206/

I am not Kent Beck

• Father of Extreme Programming

• Pioneer of Software Design Patterns

• Originator of CRC Cards• One of original

signatories of the Agile Manifesto

• Author, JUnit (unit testing framework for Java)

• Author of several books, including Test Driven Development: By Example

• Co-Author: Refactoring: Improving the Design of Existing Code

http://en.wikipedia.org/wiki/Kent_Beck

I am not Ward Cunningham

• Creator of the Wiki• CTO of AboutUs.com• Pioneer of Extreme

Programming• Inventor of FIT

(Framework for Integrated Tests)

http://en.wikipedia.org/wiki/Ward_Cunningham

I am not Martin Fowler

• Chief Scientist at Thoughtworks

• Co-Author: Refactoring: Improving the Design of Existing Code

• Co-Author: Patterns of Enterprise Application Architecture

• Author of several other books

• One of original signatories of the Agile Manifesto

http://en.wikipedia.org/wiki/Martin_Fowler

I am not Robert C. Martin

• Author, Clean Code: A Handbook of Agile Software Craftsmanship

• Author, Agile Principles, Patterns, and Practices in C#

• Author, Agile Software Development: Principles, Patterns and Practices

• Creator of FITNesse(combining FIT with a Wiki)

• One of original signatories of the Agile Manifesto

• Founder and President of Object Mentor, Inc.

http://en.wikipedia.org/wiki/Robert_Cecil_Martin

I amWill Green

• I am an Aspiring Software Craftsman

• Co-host of The Iron Languages Podcasthttp://ironlanguages.net/

• I Read A LOT• I Practice A LOT• I Learn A LOT

http://hotgazpacho.org/

Ready to drink from the fire hose?

http://www.flickr.com/photos/joevans/4292116723/

Don’t Think “Test”To-may-to, To-mah-to…

http://www.flickr.com/photos/rhinoneal/2706246863/sizes/m/

http://twitter.com/ampgt/status/21689847806

Testing as Design

Testing as Design

http://twitter.com/jbogard/statuses/19838298235

Testing as Design

http://twitter.com/venkat_s/status/24480762059

Tests aren’t Tests

“Test” doesn’t describe what is going on

Tests are specification Tests are design Test suites are an artifact of the

design process

TDD: Test Driven Design

Write the code you wish you had Hard to test == Hard to use

Better to find this out now than 2 months from now when you have to change it

Done right, helps you adhere to the Principles of Object Oriented Design

SOLIDLike a Rock

http://www.flickr.com/photos/gustty/2908229462/

Dependency Management

Dependency Management is Hard The single greatest influencer of

software quality The foundation for creating software that

exhibits the qualities we desire

Poorly Managed Dependencies

Lead to code that is Hard to change Fragile Not reusable

Well Managed Dependencies

Lead to code that is Flexible Robust Reusable

Qualities of Software

UNDESIRABLE

Hard to change Fragile Not reusable

DESIRABLE

Easy to change Robust Reusable

Dependency Management and SOLID

SOLID is The foundational set of principles for

managing dependencies Therefor the foundation for creating

software that exhibits the qualities we desire

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/SOLID_5F00_6EC97F9C.jpg

What is SOLID?

Single Responsibility

Principle

Open Closed Principle

Liskov Substitution

Principle

Interface Segregation

Principle

Dependency Inversion

Principle

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/SingleResponsibilityPrinciple2_5F00_71060858.jpg

Single Responsibility Principle

A class should have one, and only one, reason to change

http://www.objectmentor.com/resources/articles/srp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/OpenClosedPrinciple2_5F00_2C596E17.jpg

Open Closed Principle

You should be able to extend the behavior of a class, without modifying it

Classes should be open for extension but closed for modification

http://www.objectmentor.com/resources/articles/ocp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/LiskovSubtitutionPrinciple_5F00_52BB5162.jpg

Liskov Substitution Principle

Derived classes must be substitutable for their base classes

http://www.objectmentor.com/resources/articles/lsp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/InterfaceSegregationPrinciple_5F00_60216468.jpg

Interface Segregation Principle

Make fine-grained interfaces that are client-specific

http://www.objectmentor.com/resources/articles/isp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/DependencyInversionPrinciple_5F00_0278F9E2.jpg

Dependency Inversion Principle

Depend on abstractions, not on specific implementations

http://www.objectmentor.com/resources/articles/dip.pdf

Test Driven DesignThe Path to Success: 3 Simple Rules

http://www.flickr.com/photos/stuckincustoms/4070581709/

Rule #1

You are not allowed to write any production code unless it is to make a failing unit test pass.

Rule #2

You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.

Rule #3

You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

The Flow of Test Driven Design

1. Write a Test2. Watch Test Fail3. Write Code4. Watch Test Pass5. Refactor6. See step 1

http://gamesfromwithin.com/when-is-it-ok-not-to-tdd

TDD Demo!

Fibonacci Number Generator Sequence of numbers

First two numbers are 0 and 1 Each subsequent number is the sum of

the two previous numbers 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144,

…

How To Test

http://www.flickr.com/photos/modestchanges/3214701196/

An Example

Test Naming

Method being tested Conditions Expected Outcome

Testing Pattern

Arrange, Act, Assert Separates what is being tested from

the setup and verification steps Makes some Test Smells more

obvious Assertions mixed in with “Act” code. Test methods that try to test too much at

once

Arrange

Arrange all necessary preconditions and inputs

Act

Act on the object or method under test

Assert

Assert that the expected results have occurred

Things to Remember

Triple A Take Small Steps Run tests after every change Tests are Code; maintain them!

Test Doubles

http://www.flickr.com/photos/herbstkind/3791698698/

Mock Repository?

Mock? Stub? What is this?

Test Doubles

Think “stunt double” Replacement for a Production object

for testing purposes There are many types of Test

Doubles, each with its own purpose

Dummy Objects

Passed around but never used Simply to fill parameter lists

Fake Objects

Have working implementations Usually just enough for them to work in

the context of the specific test However, not suitable for Production

E.g. In-Memory Database

Stubs

Provide canned answers to SUT Usually don’t respond to anything

more than what was programmed for the context of the test

Spies

Stubs that record information on how they were called

Allows inspection for Assertion Example

Email Service that records the messages it was sent (without actually sending them)

Mocks

Pre-programmed with expectations Specify how they should be called by

the SUT Throws an exception if they receive a

call they were not expecting Checked during the Assert to ensure

they got all the calls they were expecting

RefactoringYou could do that better…

http://www.flickr.com/photos/29108968@N06/2852038148/

Why Refactoring?

Improve human readability of code Reduce the complexity in order to

improve maintainability Create a more expressive internal

structure

It’s about DESIGN!!

What is Refactoring?

A disciplined technique for restructuring an existing body of code

Altering code’s internal structure without changing its external behavior

A series of small steps that do very little on their own

Risk of change is mitigated by running automated tests after each step

Refactoring is NOT

An undisciplined technique Making changes to code that

change its external behaviorOne giant step Done without a safety net of

automated tests

You Are Not Refactoring

If you are doing any of those things, your are not refactoring.

You’re just changing things willy-nilly.

http://www.flickr.com/photos/brymo/2807243701/

How NOT to TestYeah, don’t do that

http://www.flickr.com/photos/bobcatnorth/1199505016/

Test Smells

Things that make you go “that just ain’t right” Code Smells Test Smells Project Smells

Most of what follows is summarized from xunitpatterns.com/Test%20Smells.html the book XUnit Test Patterns:

Refactoring Test Code

Test Smells – Code Smells

Smells that are found while looking at test code

Obscure Test Conditional Test Logic Hard-to-Test Code Test Code Duplication Test Logic in Production

Obscure Test

There is trouble understanding what behavior a test is verifying too much or too little information in the

test method Possible Solutions

Keep tests small and focused Do not depend on external resources

(files, databases, web services, etc.)

Conditional Test Logic

Hard to know exactly what a test is going to do when it really matters conditional logic to handle when SUT

fails to return valid data Loops to verify contents of collections Conditionals to verify complex objects

Possible Solutions Custom Assertion Methods

Hard to Test Code

Code is difficult to test Highly Coupled Code Asynchronous Code Untestable Test Code

Possible Solutions Reduce coupling by using TDD and

adhering to SOLID principles Use Test Doubles to isolate the SUT

Test Code Duplication

The same code is repeated many times Cut-and-Paste Code Reuse Reinventing the Wheel

Possible Solutions Refactor your test code to create Utility

Methods

Test Logic in Production

Production code contains logic that should only be exercised during tests Conditional logic “For Tests Only” Test Dependency in Production Test-Specific Equality

Possible Solutions Test-specific sub-classes of SUT Manage Test Dependencies Custom Assertions

Test Smells – Behavior Smells

Smells that jump out at you while running tests

Assertion Roulette Erratic Test Fragile Test Frequent Debugging Manual Intervention Slow Tests

Assertion Roulette

Hard to tell which of several assertions in the same test caused the test to fail Test tries to verify too much Assertions with unclear failure messages

Possible Solutions One logical assertion per test Include a failure message for the assertion Use a GUI test runner

Erratic Test

One or more test pass or fail intermittently Tests depend on each other Resource Leakage / Scarcity Shared state Using different/random values for each test run

Possible Solutions Use a fresh fixture for each test Clean up after yourself Use Database sandboxes or a Fake Database

Fragile Test

A test fails to compile or run when the SUT is changed in ways that do not affect the part the test is exercising Data Sensitivity Over-specifying behavior Context Sensitivity

Possible Solutions Do not depend on a database Encapsulate setup behind Creation Methods Abstract away dependencies (including Time)

Frequent Debugging

Manual debugging is required to determine the cause of most test failures

Caused by Lack of Defect Localization (missing

tests) Infrequent Test Runs

Possible Solutions Only create behavior after a failing test

case Runs tests as part of Continuous

Integration builds

Manual Intervention

A test requires a person to do some manual action each time it is run, or verify results manually Lack of attention to automated test fixture setup Test is not self-checking Test conditions that are hard to generate

programmatically Possible Solutions

Invest in automating test setup Use Assertion Methods Use Test stubs to simulate test conditions

Slow Tests

Tests take a long time to run Interaction with external resources (Databases,

File Systems, Web Services) Heavy Test fixture rebuilds for every test Asynchronous Test / Explicit delays in the test Too many tests

Possible Solutions Abstract away database, file system, web services Use an immutable shared fixture Separate event from the behavior it executes Run a fast subset more, and the entire suite less

Test Smells – Project Smells

Smells that a project manager can watch out for

Buggy Tests Developers Not Writing Tests High Test Maintenance Cost Production Bugs

Buggy Tests

Bugs are regularly found in automated tests Fragile Tests Obscure Tests Hard to Test Code

Possible Solutions Learn to write tests properly Refactor legacy code to make testing easier

& more robust Use TDD

Developers Not Writing Tests

The Usual Excuses Not Enough Time Hard to Test Code Wrong Test Automation Strategy

Possible Solutions Learn & Practice ▪ writing tests takes less time as you become

more familiar with doing it

High Test Maintenance Cost

Too much time is spent maintaining tests Fragile Tests Obscure Tests Hard to Test code

Possible Solutions Learning good Test Automation Practicing TDD

Production Bugs

Too many bugs found during formal test or in production Infrequently Run Tests Untested Code Missing Unit Tests Lost Tests

Possible Solutions Speed up test suite Use TDD Write tests to cover untested code Ensure all tests are part of some suite that is run

regularly

.NET Testing Tools

http://www.flickr.com/photos/booleansplit/2376359338/

Testing Frameworks

NUnit, MBUnit, XUnit.net Testing frameworks in the xUnit tradition Assertion Engine with syntactical sugar

on top

Support Libraries

Object Factories NBuilder

Test Doubles / Isolation Frameworks Hand-rolled mock objects Rhino Mocks Moq NMock EasyMock.NET Typemock Isolator Telerik Just Mock

Test Runners

NUnit command line NUnit GUI TestDriven.NET & Visual Nunit

Addins for Visual Studio JetBrains ReSharper or DevExpress

CodeRush Integrates with Visual Studio test status next to the test method in the editor

JetBrains TeamCity Continuous Integration server

Test Guidance

Pair Programming (Dev and QA) Code Reviews Automated Tools

Typemock Test Lint

Resources

http://www.flickr.com/photos/44442915@N00/3467438742/

Agile Principles, Patterns, and Practices in C#

Robert C. MartinMicah Martin

Working Effectively with Legacy Code

Michael Feathers

Refactoring: Improving the Design of Existing Code

Martin FowlerKent BeckJohn BrantWilliam OpdykeDon Roberts

The Art of Unit Testing with Examples in C#

Roy Osherove

Growing Object-Oriented Software, Guided by Tests

Steve FreemanNat Pryce

Testing ASP.NET Web Applications

Jeff McWherterBen Hall

MSDN Magazine Articles

June 2008 – Patterns in Practice: The Open Closed Principlehttp://msdn.microsoft.com/en-us/magazine/cc546578.aspx

October 2008 – Patterns in Practice: Cohesion and Couplinghttp://msdn.microsoft.com/en-us/magazine/cc947917.aspx

December 2008 – Patterns in Practice: Design for Testabilityhttp://msdn.microsoft.com/en-us/magazine/dd263069.aspx

September 2007 – Unit Testing: Exploring the Continuum of Test Doubleshttp://msdn.microsoft.com/en-us/magazine/cc163358.aspx

Videos

TDD with Kent Beckhttp://www.pragprog.com/screencasts/v-kbtdd/test-driven-development

Test First and Test Driven conversations with Corey Haines and JB Rainsbergerhttp://programmingtour.blogspot.com/2009/07/test-first-and-test-driven-conversation.html

Moving Specificity Towards the Tests with Corey Haines and JB Rainsbergerhttp://programmingtour.blogspot.com/2009/08/moving-specificity-towards-tests-with.html

Codemanship’s Videos with Jason Gormanhttp://www.youtube.com/user/parlezuml

TekPub Concepts serieshttp://tekpub.com/production/concepts

Podcasts

Hanselminutes Podcast 145 - SOLID Principles with Uncle Bob - Robert C. Martinhttp://www.hanselminutes.com/default.aspx?showID=163

Hanselminutes Podcast 150 – Uncle Bob Martin: SOLID: This Time With Feelinghttp://www.hanselminutes.com/default.aspx?showID=168

Hanselminutes Podcast 146 – Test Driven Development is Design – The Last Word on TDD with Scott Bellwarehttp://www.hanselminutes.com/default.aspx?showID=164

Hanselminutes Podcast 31 – Test Drivenhttp://www.hanselminutes.com/default.aspx?showID=42

Coding QA Podcast 41 – iPhone TDDhttp://codingqa.com/index.php?post_id=622464

Software Engineering Radio Podcast 155 - Johannes Link & Lasse Koskela on TDDhttp://www.se-radio.net/2010/02/episode-155-johannes-link-lasse-koskela-on-tdd/

Web Sites

The Principles of Object Oriented Designhttp://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod

xUnit Test Patternshttp://xunitpatterns.com/

Object Mentor Bloghttp://blog.objectmentor.com/

Martin Fowler’s Bloghttp://martinfowler.com/bliki/

Roy Osherove’s Bloghttp://weblogs.asp.net/ROsherove/