Testing AutomatonLuis Majano @lmajano

Who am I

• Luis Majano

• Computer Engineer

• Imported from El Salvador

• CEO of Ortus Solutions

• Creator of all things Box

Why people don’t testWhy we need to test

How to testHow to automate

Where to start?

Write Tests

WHY PEOPLE DON’T TEST

COMFORT

WHY PEOPLE DON’T TEST

New MethodologyNew Learned Behavior

It is a leap….

BIGGEST LIE IN SOFTWARE DEV

Don’t worry, we will create the

tests and refactor it later!

Bugs cost money

Cost To Fix Time detected Requirements Design Building Testing Post-Release

Time Introduced

Requirements 1x 3x 5-10x 10x 10-100x

Design --- 1x 10x 15x 25-100x

Building -- -- 1x 10x 10-25x

^ Kaner, Cem; James Bach, Bret Pettichord (2001). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. p. 4. ISBN 0-471-08112-4.

• Just do it!

• You will get dirty

• It can hurt (a little)

• Learned behavior

NO MORE EXCUSES

IT WILL ACCELERATE YOUR DEVELOPMENT

TDD/BDD=ProcessContinuousIntegrationContinuousDelivery

ContinuousImprovement

AUTOMATION

TestBox : What you get

API Docs

Core

Test Browser

Test Harness

Test Runner

Samples

install testbox —saveDev

install testbox@be —saveDev

Test Harness

Automated test results!

xUnit/BDD Test Bundles

Harness bootstrap

HTML Runner

ANT Runner

Test Bundle CFCs• Inherits from testbox.system.BaseSpec

•TesBox will then execute all tests/specs/suites within 1 or more bundles

RUN TESTS

• Execute bundles via the URL• http://localhost/tests/bundle.cfc?method=runRemote

• SOAP Runner• HTTP/REST Runner• ANT Runner• NodeJS Runner• CommandBox Runner• Custom Runner

TESTBOX REPORTERS

•ANTJunit : Variant of JUnit for Jenkins•Codexwiki : Mediawiki syntax•Console : Server console•Doc : Semantic HTML•Dot : Awesome Dots•JSON•JUnit•Raw : CFML Struct•Simple : A basic HTML reporter•Text : Back to the 80’s, Text•MinText : Great for CommandBox Runners•XML •Tap : A test anything protocol reporter•Min : Minimalistic Report

MANUAL TESTING

BundleCFC Runner

CommandBox Runnerpackage set testbox.runner = "http://127.0.0.1:57538/tests/runner.cfm"

testbox run

1

2

{ "runner": "http://127.0.0.1:57538/tests/runner.cfm", "directory": "/tests/specs", "recurse": true}

NodeJS Test Runnernpm install -g testbox-runner

create .testbox-runnerrc

1

2

Run it : testbox-runner2

ANT TESTING

DEVELOPER AUTOMATION

1.Grunt Watchers + LiveReload2.Grunt Watchers + testbox-runner3.Grunt Watchers + commanbox4.ColdBox Elixir

CONTINOUS INTEGRATION

CI SERVERS

Jenkins + CommandBoxwget http://mirrors.jenkins-ci.org/war-stable/latest/jenkins.war

box server start warpath=jenkins.war

1

2

Travis CI - Sign Uphttps://travis-ci.org/

Sign Up

1

2

Accounts

Activate For Repositories

1

2

Create .travis.yml3

Commit to repository4

Travis CI - Getting Started

TRAVIS + ANTlanguage: javabranches: only: - developmentsudo: requireddist: trustybefore_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- box install- box server startbefore_script:# Startup the app- curl http://localhost:49616/script: - ant -f workbench/build.xml

TRAVIS + COMMANDBOXlanguage: javasudo: requireddist: trustybefore_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- cd tests- box install- box server startbefore_script:# Startup the app- curl http://localhost:49615/script: > testResults="echo $(box testbox run)"; echo "$testResults"; if grep -i "\[Failures: [1-9][0-9]\?[0-9]\?\]\|\[Errors: [1-9][0-9]\?[0-9]\?\]\|<t[^>]*>\|<b[^>]*>" <<< $testResults; then exit 1; fi

MULTIPLE VERSIONS

MULTIPLE ENGINES

TRAVIS + MULTIPLE ENGINESlanguage: javasudo: requiredjdk:- oraclejdk8env: matrix: - ENGINE=lucee@4.5 - ENGINE=lucee@5 - ENGINE=adobe@2016 - ENGINE=adobe@11 - ENGINE=adobe@10before_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- box installbefore_script:- box server start cfengine=$ENGINE port=8500script: > testResults="$(box testbox run runner='http://127.0.0.1:8500/tests/runner.cfm' )"; echo "$testResults"; if grep -i "\[(Failures|Errors): [1-9][0-9]\?[0-9]\?\]\|\[Errors: [1-9][0-9]\?[0-9]\?\]\|<t[^>]*>\|<b[^>]*>" <<< $testResults; then exit 1; finotifications: email: false

Resources

• TestBox : ortussolutions.com/products/testbox

• CommandBox: ortussolutions.com/products/commandbox

• Slack: boxteam.herokuapp.com

• CFML Slack: #box-products

• travis-ci.org

• jenkins.io

Thank you!