TestBoxCFUG Meetup

July 19th

Who am I

Gavin Pickin – developing Web Apps since late 90s

● New Addition to Ortus Solutions● ContentBox Evangelist

What else do you need to know?● Blog - http://www.gpickin.com● Twitter – http://twitter.com/gpickin● Github - https://github.com/gpickin

Let’s get on with the show.

So who is already using testing?

Trick Question - You are all testing your apps.

● Most people look like this guyClicking around in the browser yourself

● Setup Selenium / Web Driver to click around for you

● Structured Programmatic Tests

Bugs Hurt You

Bugs Hurt

•Bugs hurt – the later in the process, the harder to

fix.

•Test Early and Often

–Find them before they rot your foundation

•Testable Code is Maintainable Code

Types of Testing

● Black/White Box● Unit Testing● Integration Testing● Functional Tests● System Tests● End to End Tests● Sanity Testing

● Regression Test● Acceptance Tests● Load Testing● Stress Test● Performance Tests● Usability Tests● + More

Important Testing Types

•Unit Testing

–Test behavior of individual objects

•Integration Testing

–Test Entire Application from Top Down

•UI verification testing

Verification via HTML/Visual elements

Important Testing Tools

•TestBox (Run BDD and MXUnit style)

•IDE - CF Builder / Eclipse

•Mocking Framework

•ANT

•Jenkins, Bamboo, Teamcity, other Cis

•Selenium

•Jmeter or Webstress Tool, Apache AB

What is TestBox?

TestBox is a next generation testing framework for ColdFusion (CFML) that is based on BDD (Behavior Driven Development) for providing a clean obvious syntax for writing tests. It contains not only a testing framework, runner, assertions and expectations library but also ships with MockBox, A Mocking & Stubbing Framework. It also supports xUnit style of testing and MXUnit compatibilities.

More info: https://www.ortussolutions.com/products/testbox

Integration Tests

● Integration Tests several of the pieces together

● Most of the types of tests are variations of an Integration Test

● Can include mocks but can full end to end tests including DB / APIs

What is Unit Testing

“unit testing is a software verification and validation method

in which a programmer tests if individual units of source code

are fit for use. A unit is the smallest testable part of an

application”

- wikipedia

Unit Testing

•Can improve code quality -> quick error discovery

•Code confidence via immediate verification

•Can expose high coupling

•Will encourage refactoring to produce > testable code

•Remember: Testing is all about behavior and expectations

Style of Testing - TDD vs BDD

● TDD = Test Driven Development● Write Tests● Run them and they Fail● Write Functions to Fulfill the Tests● Tests should pass● Refactor in confidence● Test focus on Functionality

Style of Testing - TDD vs BDD

● BDD = Behavior Driven Development● Actually similar to TDD except:● Focuses on Behavior and Specifications● Specs (tests) are fluent and readable● Readability makes them great for all levels of testing in the

organization

● Hard to find TDD examples in JS that are not using BDD describe and it blocks

TDD Example

Test( ‘Email address must not be blank’, function(){

notEqual(email, “”, "failed");

});

BDD Example

Describe( ‘Email Address’, function(){

It(‘should not be blank’, function(){

expect(email).not.toBe(“”);

});

});

BDD Matchers

expect(true).toBe(true);

expect(true).toBe(true);

expect(true).toBe(true);

expect(true).toBe(true);

BDD Matchers

expect(true).not.toBe(true);

expect(true).not.toBe(true);

expect(true).not.toBe(true);

expect(true).not.toBe(true);

expect(true).not.toBe(true);

Matcher Examples

expect(true).toBe(true);

expect(a).not.toBe(null);

expect(a).toEqual(12);

expect(message).toMatch(/bar/);

expect(message).toMatch("bar");

expect(message).not.toMatch(/quux/);

expect(a.foo).toBeDefined();

expect(a.bar).not.toBeDefined();

BDD Example

describe("Hello world function", function() {

it(”contains the word world", function() {

expect(helloWorld()).toContain("world");

});});

New BDD Examplefeature( "Box Size", function(){

describe( "In order to know what size box I need

As a distribution manager

I want to know the volume of the box", function(){

scenario( "Get box volume", function(){

given( "I have entered a width of 20

And a height of 30

And a depth of 40", function(){

when( "I run the calculation", function(){

then( "the result should be 24000", function(){

// call the method with the arguments and test the outcome

expect( myObject.myFunction(20,30,40) ).toBe( 24000 );

});

});

});

});

});

});

Installing Testbox

Install Testbox – Thanks to Commandbox - this is easy.

# box install testbox

Next, we need to decide how you want to run Testbox

Create a Runner.cfm File to Run your tests<cfsetting showDebugOutput="false">

<!--- Executes all tests in the 'specs' folder with simple reporter by default --->

<cfparam name="url.reporter" default="simple">

<cfparam name="url.directory" default="tests.specs">

<cfparam name="url.recurse" default="true" type="boolean">

<cfparam name="url.bundles" default="">

<cfparam name="url.labels" default="">

<!--- Include the TestBox HTML Runner --->

<cfinclude template="/testbox/system/runners/HTMLRunner.cfm" >

Create a test suite// tests/specs/CFCTest.cfc

component extends="testbox.system.BaseSpec" {

function run() {

it( "will error with incorrect login", function(){

var oTest = new cfcs.userServiceRemote();

expect( oTest.login( 'gavin@gavin.com', 'topsecret').result ).toBe('400');

});

}

}

Create 2nd Test Suite// tests/specs/APITest.cfccomponent extends="testbox.system.BaseSpec" { function run() { describe("userService API Login", function(){ it( "will error with incorrect login", function(){ var email = "gavin@gavin.com"; var password = "topsecret”; var result = ""; http url="http://www.testableapi.local.com:8504/cfcs/userServiceRemote.cfc?method=login&email=#email#&password=#password#" result="result”;

expect( DeserializeJSON(result.filecontent).result ).toBe('400'); }); });

}}

Run tests in Browser with Runner

Run tests with Grunt

● Install Testbox Runner – Thanks Sean Coyne

# npm install testbox-runner

● Install Grunt Shell

# npm install grunt-shell

Then, we need to add Grunt Configuration

Add Grunt Config - 1

module.exports = function (grunt) {

grunt.loadNpmTasks('grunt-shell');

grunt.initConfig({ … })

}

Add Grunt Config - 2

Watch: {

…

cfml: {

files: [ "cfcs/*.cfc"],

tasks: [ "testbox" ]

}}

Add Grunt Config - 3

shell: {

testbox: {

command: "./node_modules/testbox-runner/index.js --colors --runner http://www.testableapi.local.com:8504/tests/runner.cfm --directory /tests/specs --recurse true”

}

}

Add Grunt Config - 4

grunt.registerTask("testbox", [ "shell:testbox" ]);

grunt.loadNpmTasks('grunt-contrib-jasmine');

grunt.loadNpmTasks('grunt-contrib-watch');

Grunt Config Gists

Jasmine + Testbox

https://gist.github.com/gpickin/9fc82df3667eeb63c7e7

TestBox output with Grunt

TestBox JSON output

Testbox has several runners, you have seen the HTML one, this Runner uses the JSON runner and then formats it.

http://www.testableapi.local.com:8504/tests/runner.cfm?reporter=JSON&directory=%2Ftests%2Fspecs&recurse=true

Running TestBox in Sublime 2

Install PackageControl into Sublime Text

Install Grunt from PackageControl

https://packagecontrol.io/packages/Grunt

Update Grunt Sublime Settings for paths{

"exec_args": { "path": "/bin:/usr/bin:/usr/local/bin” }

}

Then Command Shift P – grunt

TestBox Output in Sublime 2

Run TestBox with CommandBox

Run the tests in the box.json file.

# testbox run

Watch the folders and run tests on change

# testbox watch

More info on TestBox usage from CommandBox: https://www.ortussolutions.com/blog/using-testbox-watch-to-automate-your-testing-suite

More Testing - Mocking with MockBox

"A mock object is an object that takes the place of

a ‘real’ object in such a way that makes testing

easier and more meaningful, or in some cases,

possible at all"

by Scott Bain - Emergent Design

More Testing - IntegratedStart using an easy, fluent API for your integration tests!

More Info: https://github.com/elpete/integrated

Api - https://elpete.github.io/integrated/

Some real life exampleshttps://github.com/framework-one/fw1/tree/develop/tests maybe?

https://github.com/Ortus-Solutions/TestBox/tree/development/tests

https://github.com/foundeo/cfdocs/tree/master/tests

https://github.com/foundeo/cfmlparser/tree/master/tests/tests

https://github.com/foundeo/toscript/tree/master/tests

https://github.com/foundeo/bolthttp/tree/master/test

https://github.com/foundeo/cfml-security/tree/master/tests/tests/secureupload

Some real life examples cont.https://github.com/ryanguill/emit/tree/master/tests

https://github.com/ryanguill/cfmlBase62/tree/master/tests

https://gitlab.com/ryanguill/FPcfc/tree/master/tests

https://gitlab.com/ryanguill/template-cfc/tree/master/tests

https://github.com/MotorsportReg/sidecar/tree/master/tests/basic

https://github.com/MotorsportReg/sidecar/tree/master/tests/basic