TDD and the Legacy Code Black Hole

TEST-DRIVEN DEVELOPMENTAND THE

LEGACY CODE BLACK HOLE

Noam Kfir

NOAM KFIR• Consultant and Trainer

• Telerik Developer Expert• Ranorex Professional• ISTQB Reviewer• Agile Practitioners Meetup Co-

organizer

• Specialize in test automation forboth testers and coders• Ranorex, Selenium…• TDD, BDD, Unit & Integration Testing…• JavaScript, C#…

AGENDA

• What Is Legacy Code?• The Legacy Code Black Hole• The Legacy Code Dilemma• Unit Testing• JavaScript Tests• ECMAScript 6

• Test-Driven Development (TDD)• Changing Legacy Code• Clean Code• Dependency Breaking

Techniques

WHAT IS LEGACY CODE?

STABLE

• It works…

• Tested in the wild, probably for a long time

• If it ain’t broke, don’t fix it…?

ANCIENT

• Was written a long long time ago• Uses “a previous language, architecture, methodology, or

framework”• Uses unsupported technologies

• Prohibitively expensive to rewrite or replace

INHERITED

• Somebody else wrote it but now we’re responsible for it• Programmers who left (or got promoted…)• Outsourcing• Acquisition• Purchased third-party libraries/frameworks/components• Adopted from open source

• Expensive to learn, integrate and continuously maintain

STRIKES FEAR IN THE HEARTS OF MORTALS

• Except maybe for that one irreplaceable ninja programmer…• Nobody else understands it• Everybody else is afraid to touch it

• Hard to predict how changes will affect the rest of the system

ALL CODE AS SOON AS IT IS WRITTEN

• We tend to focus on the future

• Our code will eventually become ancient• Somebody else will eventually inherit it• Others will eventually fear it

CODE WITHOUT TESTS

• Michael C. Feathers in Working Effectively with Legacy Code

• Tests provide some control• Safety net• Live documentation• Feedback

• Can delay entropy, but not prevent it

THE LEGACY CODE BLACK HOLE

CODE IS ENTROPIC

• Systems become more complex• Technical debt tends to grow• Legacy code holds evolution back• Bugs on legacy code tend to accumulate

UNTIL IT BECOMES A BLACK HOLE

• Not all legacy code is a black hole

• We know we have a problem when legacy code starts swallowing up everything around it – especially code and time

• It’s usually already too late – very expensive and difficult to fix

THE LEGACY CODE DILEMMA

OPTION 1 – IGNORE LEGACY CODE

• Make the choice to continue to incur more technical debt• Find creative work arounds that avoid touching the legacy code

• The default option – without it, there would be no black holes

• It’s a cost/benefit analysis• Depends a lot on company culture, constraints and goals

OPTION 2 - REFACTOR

• Make the fewest incremental changes necessary to align the legacy code with its new goals

• A refactoring is a small, safe and focused change to an internal structure that does not affect the behavior of the containing system

• A lot of refactoring means doing many incremental refactorings

• Only theoretically safe because we don’t have tests!

OPTION 3 - RESTRUCTURE

• Make larger changes to the external behavior of legacy code using current technologies while maintaining most of its original design

• Often includes a lot of refactorings in addition to the external changes• Often involves a partial redesign of the legacy code and/or the system

it interacts with

• Not safe but often necessary to account for previously unforeseen features or integrations

OPTION 4 - REWRITE

• Reimplement the legacy code completely using current technologies and design principles

• Usually means the legacy code is deleted and its functionality (or a subset) is reimplemented elsewhere from scratch

• Option of last resort• Expensive

OUR FOCUS – REFACTORING AND RESTRUCTURING

• Ignoring and rewriting are legitimate options but not interesting to us

• We deal with legacy code after ignoring it fails• We only rewrite legacy code if we can’t first refactor or

restructure it

• So we will focus on refactoring and restructuring

REFACTOR• Always small• Safe (theoretically)• Internal flow and behavior• Does not affect the system• Incremental

• Usually bigger• Not safe• External flow and behavior• Affects the system• Not incremental

RESTRUCTURE

REFACTORING VS. RESTRUCTURING

UNIT TESTING

WHAT ARE UNIT TESTS?

• Unit tests verify that pieces of code in an applicationbehave as expected in isolation

• There is no consensus on the definition for unit• A unit is typically a method that performs a specific action

• Units should be small• Different approaches accept different levels of granularity

WHAT IS A GOOD TEST?

• Checks correctness – verifies a single behavior• Maintainable – short, concise, readable• Atomic – independent from other tests• Automated – runs quickly and needs no human intervention• Provides immediate feedback

• Above all: Trustworthy

• All normal programming rules still apply!

ARRANGE, ACT, ASSERT

• Arrange – Prepare the dependencies and components• Act – Execute the code being tested• Assert – Verify the code behaves as expected and returns the

correct result

• Sometimes called Given/When/Then

CONVENTIONS

• We rely on conventions to ensure consistency

• Includes code style, structure, naming rules, etc.

• There are more opinions than programmers• The most important thing is to stick to the project’s convention

GUIDELINES – “DO”

• Treat test code the same as production code• Re-use test code• The DRY principle applies to test code as well

• Atomic tests• Tests should be able to run in any order without affecting other tests

• Test isolated units• Try to keep the units as small as possible

GUIDELINES – “DON’T”

• Avoid test logic (e.g., “if” and “switch” statements in test code)• Avoid testing internal (encapsulated) state and behavior• Avoid testing more than one unit• Avoid multiple asserts• Difficult to name the containing test• Difficult to see the results at a glance• Execution stops on first failure• Can’t see the big picture (e.g. when one problem has multiple symptoms)

JAVASCRIPT TESTS

MOCHA

• Mocha is a testing framework for JavaScript• Can run on the client or the server• Based on Jasmine but intentionally without assertions and

spies

• Installed via npm• Mocha specs cannot be run directly• Must be run with the mocha utility, but can be executed with

other tools

MOCHA TEST STRUCTURE

• Mocha files are composed of suites, tests and asserts

• Suites (describe) contain tests, before and after code, and can be nested• Tests (it) execute the code being tested and use asserts to verify the results• Asserts (chai) verify the results comply with expectations and report failures

• Asynchronous test support provided by done parameter of it callback• Thenable promises also supported by simply returning them from it callback

CHAI

• Chai is a popular fluent assertion library with a fluent syntax

• Provides three different styles or approaches (assert, expect and should)

• We will use the expect style

expect(actualValue).to.be.equal(expectedValue);expect(actualValue).to.be.undefined;expect(actualValue).to.be.above(minimumValue);

SINON

• Sinon provides test spies, stubs and mocks

• Spies – functions that record everything that happens to them• Stubs – spies that can modify the function’s behavior• Mocks – similar to spies except that they also assert expectations

const callback = sinon.spy();foo(callback);expect(callback.called).to.be.true;

KARMA

• Karma is a JavaScript test runner• Relies on a configuration file – karma.conf.js

• Knows how to run mocha and report the results in many different ways

• Has good integration with many tools

• Can run tests in PhantomJS (the headless browser) or in real browsers

ECMASCRIPT 6

ES6 OVERVIEW

• JavaScript underwent a massive revolution in 2015• The language semantics have changed and many features have been

added• Many features supported by modern browsers and Node, but not all• Use Babel to transpile to ES5• We use a subset of the new features• Learn more about ES6+ and its features online:• https://egghead.io/courses/learn-es6-ecmascript-2015• http://es6katas.org/

VARIABLE ASSIGNMENT

• let – variable declaration with block scope• const – constant declaration with block scope

• Use block scope instead of the function scope used by var• Less susceptible to bugs and unexpected side effects than var• Have the same syntax as var• Can be used in the same places as var

ES6 ARROW FUNCTIONS

• => – lambda functions

const double = (value) => value * 2;

• Can be declared in the same places as regular functions• Do not affect the this keyword

TEMPLATE STRINGS

• `${expression}` – performs string interpolation

const student = { name: 'Alex' };let value = `name: ${student.name}`;

• Uses back-ticks• Resolves expression when the string is parsed• The expression must be in context

ES6 CLASSES

• class – declares a JavaScript class

class Bar {}

class Foo extends Bar { constructor() {} doSomething() {}}

• Syntactic sugar for prototypes with new semantics

ES6 DESTRUCTURING

• Uses {} on left side of assignment – shorthand for extracting members

const { port } = options; // const port = options.port;

function foo( { port } ) {}foo( { port: 8080 } );

• Works with objects and arrays• Supports head/tail semantics with the rest operator

ES6 PROPERTY SHORTHAND

• Variable names identical to assigned property names can be omitted

function foo() { const port = 8080; return { host: 'localhost', port };}

ES6 SPREAD OPERATOR

• ... – expands an array

const values = [1, 2, 3];const clone = [...values]; // [1, 2, 3]foo(...values); // foo(1, 2, 3);const [head, ...tail] = values; // head == 1, tail == [2, 3]

• Supported in arrays, function calls (instead of apply) and destructuring

ES6 REST PARAMETER

• ...name – effectively params

function foo(operation, ...items);foo('sum', 1, 2, 3);

• name can be any legal name• name is an array

ES6 MODULES

• import – imports members from specified namespaces• export – exports specified members

import { map } from 'lodash';export const value = 3;

• Universal way to declare modules (browser and Node)• Not fully implemented yet

TEST-DRIVEN DEVELOPMENT (TDD)

WHAT IS TDD?

• Test-Driven Development is a methodology whose purpose is to help programmers build software safely• For our purposes, TDD refers also to BDD and ATDD

• It’s not about the tests!• Tests are a tool that helps focus on the design and establish

trust

• TDD encourages emergent design

EMERGENT DESIGN

• We assume that it is impossible to plan the final design in advance

• So we rely on programming principles, collaboration, knowledge of the domain and our skill and experience to build the software

• Instead of planning every detail ahead of time, we rely on tentative plans and iterative feedback cycles and let the code evolve on its own

• A design emerges – partly guided and partly evolutionary

EMERGENT DESIGN AND LEGACY CODE

• Recall our dilemma – whether to ignore, refactor, restructure or rewrite

• The difficulty with legacy code is that it doesn’t conform to the design used by the rest of the system

• To what extent do we want it to conform?• How much are we willing to invest in forcefully reshaping its design?• How can we refactor or restructure it as safely and cheaply as possible?

CLEAN CODE

• No single definition but you know it when you see it• “Clean code always looks like it was written by someone who cares”• Michael C. Feathers

• Good designs emerge only we write clean code

• Some key principles: DRY, design patterns, SOLID principles, meaningful names, expression of intent, purposeful functions, the Law of Demeter, the Boy Scout Rule, avoiding side effects, and more

TDD AND LEGACY CODE

• Legacy code can be very tricky to unravel

• Even if we don’t use TDD on a regular basis, it’s especially helpful in these cases

• The careful iterative step-by-step process protects us

CHANGING LEGACY CODE

TESTING LEGACY CODE

• Legacy code has already been tested in the real world, so it’s probably stable

• Writing tests for legacy code is very difficult• Usually requires changing the code• Usually requires complicated tests

• Only write tests for legacy code that you need to interact with• Never change legacy code without having a clear purpose

BEWARE THE LABYRINTH

• Changing legacy code often feels like trying to find our way out of a labyrinth

• We have to go back a few times and try new paths

• It’s a trial and error process, but we can make educated guesses

VERSION CONTROL

• Use version control wisely to create safe restore points and avoid changing the central branches

• Work on a separate branch• Commit often

• You may need to roll back several times when working with tangled code

• VCSs are extremely useful for working our way out of the labyrinth

THE LEGACY CODECHANGE ALGORITHM

1. Identify change points – what has to change to make the code testable

2. Find test points – figure out what needs to be tested and what to test for

3. Break dependencies – make the legacy code testable4. Write tests – anchor the existing behavior before making

real changes5. Make changes and refactor – gradually improve the design

1 – IDENTIFY CHANGE POINTS

• Looks for seams and their enabling points• “A seam is a place where you can alter behavior in your program

without editing in that place.”• “Every seam has an enabling point, a place where you can make

the decision to use one behavior or another.”• The most useful seams are object seams

• Requires a basic understanding of the architecture and design

2 – FIND TEST POINTS

• Analyze the code• Trace the values through the code or the symbol usage in the editor

• Look for places that might be affected by your changes• You will have to test these places before you write the new features

• Look for dependencies• You may have to write tests for some to ensure other things don’t break

3 – BREAK DEPENDENCIES

• Use techniques to carefully change the internal structure of the legacy code

• Avoid the temptation to change many things at once, go step-by-step

• The purpose is to make the legacy code testable, not to improve its design

• Design improvements are a secondary benefit, not the main goal

4 – WRITE TESTS

• Remember that tests have to fail first• Either create a test that fails due to an intentional mistake, and then

fix it• Or make a tiny change in your legacy code to break a good test, and

then restore it

• Try to cover all the test points

5 – MAKE CHANGES AND REFACTOR

• Write the new features• Use TDD and refactor it• Don’t forget to refactor the tests too

CLEAN CODE

KEEP IT DRY

• Don’t Repeat Yourself

• Be lazy, but not lazy

DESIGN PATTERNS

• “A software design pattern is a general reusable solution to a commonly occurring problem within a given context in software design. It is not a finished design that can be transformed directly into source or machine code.”• https://en.wikipedia.org/wiki/Software_design_pattern

• Design patterns are building blocks• Provide a language for effectively communicating complex interactions in code

• Always use design patterns

THE SOLID PRINCIPLES

• Single Responsibility Principle• do just one thing, have one reason to changeSRP• Open Closed Principle• open for extension, closed for changeOCP• Liskov Substitution Principle• all implementations should behave consistentlyLSP• Interface Segregation Principle• implement only necessary abstractionsISP• Dependency Inversion Principle• externalize dependencies and rely on abstractionsDIP

ADDITIONAL CONSIDERATIONS

• Use meaningful names• Expression of intent• Avoid side effects• The Law of Demeter• Purposeful functions• The Boy Scout Rule

DEPENDENCY BREAKING TECHNIQUES

DECIDING WHETHER TO WRITE TESTS

• Not all legacy code is testable at first

• It may take a different route to get there• Other things may have to be refactored before a certain test

can be written

• An alternative is to write a higher-level test (integration, end-to-end…)

• If you must make a change and cannot write a test now, be more careful

LOW HANGING FRUIT

• Go for the easy things first

• Lowers the fear barrier• Improves the design a bit so the rest becomes easier too• Changing the code helps you understand the code better

SPROUT METHODS

• Instead of adding new behavior to an existing method, create a new method with the new behavior and call it from the old method

• Develop the new method using TDD

SPROUT CLASSES

• Similar to Sprout Methods• Create a new class for the new behavior instead of a new

method

• Useful for classes that are difficult to create in tests• Also useful for very complicated methods and classes

• Eventually more behavior will probably more to the testable sprout class

WRAP METHOD

• Basically, the Extract Method Refactoring

• The idea is to preserve the SRP and not add additional behavior to an existing method, if possible

• So the content of the method is extracted, a new method is created for the new behavior, and the old method calls them both

WRAP CLASS

• Similar to Wrap Method

• Extract a class or interface and create a Decorator with the new behavior

SUBCLASS

• Create a derived class that overrides the implementation that can’t be tested

• Ensure the remaining behavior is reachable and testable

• Test the subclass implementation

EXTRACT ALGORITHMS

• Flatten nested decision trees

• Create an interface base class for a decision

• Derive implementations for each flattened decision

• Change the original flow so uses the decision classes instead of the tree

DEPENDENCY INJECTION

• Instead of creating new instances of classes inside your method, supply the instances from outside

• The test can supply an stub instead of the dependency

SUMMARY

THANK YOU!

Test-Driven Developmentand the

Legacy Code Black Hole

Noam KfirConsultant & Trainernoam@kfir.cc | http://noam.kfir.cc | @NoamKfir