Automated Testing With Jasmine, PhantomJS and Jenkins

1

Automated Testing With Jasmine, PhantomJS &

JenkinsAisha Kaliel | Ronn Abueg

Drupalcon 2013 Portland

1

2May 2013

Agenda

1.Who we are2.Testing JS code and front-

end• Testing without the

browser• Continuous Integration

2

3May 2013




o Listing the types of tests

o Looking at some test tools

o Writing and running tests

Agenda

3

4May 2013

Agenda




o What is a headless browser

o Scripting with a headless browser

o Testing with a headless browser

4

5May 2013

Agenda




o Deploying codeo Integrating testing with

buildso Review build results

5

6May 2013

Who are we?

Aisha Kalieldrupal: akaliel

Ronn Abueg drupal: ronnbottwitter: @ronnbot

Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration

twitter: @workatplayfacebook: www.facebook.com/workatplaywebsite: www.workatplay.com

6

7May 2013

We work with Drupal...

Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 7

8May 2013

And other CMS/frameworks


9May 2013

And other CMS/frameworks


10May 2013

Repetitive Testing: Why not automate?


10

11May 2013

Why test in general


11

12May 2013

Automated testing...


• Saves time and money

• Improves accuracy

• Increases test coverage

• Does what manual testing cannot

• Helps developers and testers

• Improves team moral

• and continuous integration is the cornerstone of Agile development

12

13May 2013




o Listing the types of tests

o Looking at some test tools

o Writing and running tests

Agenda

13

14May 2013

Types of tests

GUI

Unit

Service / Integration

Test Pyramid•concepted by Mike Cohn•higher level tests at the top•larger number of test below•high level tests can encompass many low level tests, e.g.ofor form test to pass, functional units must work


15May 2013

Tackling the testing pyramid

GUI

Unit



automating browser

JS testing

16May 2013

Testing tools

• Selenium• Watir• Windmill• Mocha• QUnit• Jasmine• and etc.


automating browser

JS testing

17May 2013

Testing tools

• Selenium• Watir• Windmill• Mocha• QUnit• Jasmine


18May 2013

Comparison

Selenium

•Automates browsers

•Simulates user actions

•There's an IDE, a Firefox extension, that allows to record, edit, and debug tests

Jasmine

•Open-source testing framework for JavaScript

•Does not depend on DOM or any other JavaScript frameworks

•Unit tests your JS code


19May 2013

Why Jasmine?

• Runs anywhere Javascript can

• Does not intrude on the application

• Plays well with IDEs

• Encourages good testing principles


20May 2013

What is Jasmine?

• Behavior-driven development framework

• Does not require require a DOM, or depend on any other JavaScript frameworks

• Clean, obvious syntax


21May 2013

What is Jasmine?




• Specialized version of test-driven development

• Focuses on the "desired behavior" of software


22May 2013

What is Jasmine?


• Does not require require the DOM, or depend on any other JavaScript frameworks



23May 2013

What is Jasmine?




Still has access to the DOM!


24May 2013

What is Jasmine?





25May 2013

Tackling the testing pyramid with Jasmine

GUI

Unit



26May 2013


GUI

Unit



27May 2013


GUI

Unit



28May 2013

Jasmine Features

• Test suites/specs• Expectations• Matchers• Setup and teardown• Spies• Asynchronous Support


29May 2013

Feature to test: basic calculator


30May 2013

Basic calculator


31May 2013

Basic calculator


32May 2013

Basic calculator


33May 2013

Test the calculator: Suites


34May 2013

Test the calculator: Specs


35May 2013

Testing the calculator: Expectations


36May 2013

Feature: Matchers

• expect(x).toBe(y);

• expect(x).toEqual(y);

• expect(x).toMatch(/pattern/);

• expect(x).toBeDefined();

• expect(x).toBeUndefined();

• expect(x).toBeNull();

• expect(x).toBeTruthy();

• expect(x).toBeFalsy();

• expect(x).toContain(y);

• expect(x).toBeLessThan(y);

• expect(x).toBeGreaterThan(y);

• expect(x).toBeCloseTo(y, 0);

• expect(function(){ fn(); }).toThrow(e);


37May 2013

Feature: Negative Matchers

• expect(x).not.toBe(y);

• expect(x).not.toEqual(y);

• expect(x).not.toMatch(/pattern/);

• expect(x).not.toBeDefined();

• expect(x).not.toBeUndefined();

• expect(x).not.toBeNull();

• expect(x).not.toBeTruthy();

• expect(x).not.toBeFalsy();

• expect(x).not.toContain(y);

• expect(x).not.toBeLessThan(y);

• expect(x).not.toBeGreaterThan(y);

• expect(x).not.toBeCloseTo(y, 0);


38May 2013

Testing the calculator: Matchers


39May 2013

Testing the calculator: More specs!


40May 2013

Feature: Setup and Teardown


41May 2013

Testing the calculator: Setup


42May 2013

Testing the calculator: Spies


43May 2013

Testing the calculator with DOM tests


44May 2013

Testing the calculator: Nested describe blocks


45May 2013

Testing the calculator: Setup and Teardown


46May 2013

Feature: Asynchronous Support


47May 2013



48May 2013



49May 2013

Testing the calculator: Reporter


50May 2013

Testing the calculator: Reporter


51May 2013

Testing the calculator: Filtering the Reporter


52May 2013

Running the test page without a browser


53May 2013

Agenda




o What is a headless browser

o Scripting with a headless browser

o Testing with a headless browser

53

54Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration May 2013

What is a headless browser

• Web browser without a GUI

• Renders web pages

• Outputs to non-humans i.e. other software

• Useful for:o testingo parsingo screen capture

54

55Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration May 2013

Enter the PhantomJS• Headless WebKit

• Scriptable with JavaScript/CoffeeScript

• Support for various web standards:o DOM handlingo CSS selectoro JSONo Canvaso SVG

55

56May 2013

PhantomJS basic how toCreate a script: hello.js

console.log('Hello, DrupalCon Portland!');

phantom.exit();

Run script in the command line

phantomjs hello.js

Output

Hello, DrupalCon Portland!


57May 2013

PhantomJS modulesWebPageA WebPage object encapsulates a web page.

SystemA set of functions to access system-level functionality.

FileSystemA set of API functions available to access files and directories.

WebServerUsing an embedded web server module called Mongoose, PhantomJS script can start a web server.


58May 2013

PhantomJS basic how toPage Loading: loadpage.js

var page = require('webpage').create();

page.open('http://portland2013.drupal.org', function () {

page.render('drupalcon.png');

phantom.exit();

});


59May 2013

PhantomJS basic how toPage Loading with arguments

var page = require('webpage').create(),

system = require('system');

page.open(system.args[1], function (status) { if (status == 'success') { console.log('Loaded the address: ' + system.args[1]); } phantom.exit(); });


60May 2013






61May 2013






62May 2013






63May 2013

PhantomJS basic how toRun script in the command line

phantomjs loadpage.js http://www.google.com

Output

Loaded the address: http://www.google.com


64May 2013

PhantomJS basic how toCode Evaluation

var page = require('webpage').create();

page.open(url, function (status) {

var title = page.evaluate(function () {

return document.title;

}); console.log('Page title is ' + title); });


65May 2013

Running Jasmine with PhantomJS

Github: phantomjs / examples / run-jasmine.jsLocated at: https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js


66May 2013

Running Jasmine with PhantomJSFrom https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js


67May 2013



68May 2013



69May 2013



70May 2013



71May 2013



72May 2013



73May 2013



74May 2013



75May 2013



76May 2013



77May 2013



78May 2013



79May 2013



80May 2013



81May 2013



...

...

...

81

82May 2013

PhantomJS featuresHeadless web testingLightning-fast testing without the browser with various test frameworks.

Page automationAccess and manipulate web pages with the standard DOM API, or with usual JavaScript libraries.

Screen captureProgrammatically capture web contents including CSS, SVG and Canvas.

Network monitoringAutomate performance analysis, track page loading and export as standard HAR format.


83May 2013

PhantomJS custom modules (v1.7)RequireUsers can reference their own modules from the file system using require function.

General usage

var server = require('webserver').create();

var Awesome = require('MyAwesomeModule');

Awesome.do();


84

CI

May 2013

Lets integrate it with Jenkins


85May 2013

Agenda




o Deploying codeo Integrating testing with

buildso Review build results

85

86May 2013

Meet Jenkins

• Leading open-source Continuous Integration server

• Built with Java

• Plug-in support

• RSS/E-mail/IM support


87May 2013

Jenkins in action - dashboard


88May 2013

Jenkins in action - project


89May 2013

Jenkins in action - build


90May 2013

Jenkins in action - build detail


91May 2013

Deploying code - console output of build


92May 2013

PhantomJs & Jasmine with XML

Github: detro / phantomjs-jasminexml-exampleLocated at: https://github.com/detro/phantomjs-jasminexml-example


93May 2013

Jasmine Reporter Plugin

Github: ../ test / lib / jasmine-reportersLocated at: https://github.com/detro/phantomjs-jasminexml-example/tree/master/test/lib/jasmine-reporters


94May 2013

Jasmine Reporter Plugin

Github: ../ test / lib / jasmine-reportersLocated at: https://github.com/detro/phantomjs-jasminexml-example/tree/master/test/lib/jasmine-reporters


95May 2013

PhantomJS XML Runner

Github: ../ test / phantomjs_jasminexml_runner.jshttps://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js


96May 2013




97May 2013




98May 2013




99May 2013




100May 2013




101May 2013




102May 2013




103May 2013

Running PhantomJS in Jenkins: configuration


104May 2013

Running PhantomJS in Jenkins: Console Output


105May 2013

Publishing test results in Jenkins: configuration


106May 2013

Publishing test results in Jenkins: configuration


107May 2013

Publishing test results in Jenkins: Console Output


108May 2013

Review build and test results


109May 2013

Review build and test results


110May 2013

Review test results


111May 2013

Review test results


112May 2013

Agenda




112

113May 2013

Agenda




o Wrote and ran tests in Jasmine

o We are able to do various types of testsautomatically

o This is more accurate than manual testing

113

114May 2013

Agenda




o Executed Jasmine tests via PhantomJS

o The tests can be ran by non-humans

o This saves us resources

114

115May 2013

Agenda




o Added tests execution during builds on Jenkins

o Reviewed test results in Jenkins after builds

o Allows us to be more agile

115

116

From Mint Digital's blog, original cartoon by xkcd

http://xkcd.com/303/

117May 2013

Thank you!

Evaluate this session at:

portland2013.drupal.org/scheduleEnjoy the rest of

Drupalcon 2013 Portland

117

Technology

Automated Testing With Jasmine, PhantomJS and Jenkins