Full-Stack End-to-End Test Automation with

Node.jsOne year later

ForwardJS 2017Chris Clayman

Mek Stittri

2

First things first

Airware, we are a drone company.We build complete enterprise drone solutions; Cloud Platform, Ground Control Station, Drone Integration and etc

We are Hiring! - http://www.airware.com/careers● iOS Software Engineer● Javascript Software EngineerOr just come talk to us :)

3

A Bit About UsThe Automation team at Airware, we do drone stuffs.Continuous Deployment & Test Automation tools aka Quality Ops

Chris Clayman@chrisclaymangithub.com/kidmillions

Mek Stittri@mekdevgithub.com/mekdev

Overheard at the Monday workshops

"I don't write tests, testing sucks"

Let's talk About Test Automation

Let us give you a warm hug.

Why Node.js for Test Automation

QA, Automation engineers

Frontend engineers

Backend engineers

Java

Javascript

Java

Python

Javascript

Java

Ruby

Javascript

Node.js

Company A Company B Company C

Node.js

Javascript

Node.js

Airware

- Node.js adoption, more and more company moving to node.js- Share code with the whole engineering team- Play to the engineering team’s strength

True Full-Stack Delivery Team

Functional tests that interacts with the webapp as a user would.

Full-Stack End-to-End Automation

Visual

UI Interaction

Http Restful API

Javascript Unit tests

Microservice Integration

Unit tests

DB & Data Schema tests

Martin Fowler - Test Pyramid, Google Testing Blog2014 Google Test Automation Conference

Tools for tests● Visual Diff Engine ● Selenium-Webdriver ● REST API Client

SmallUnit tests

BigE2E tests

MediumIntegration tests

z’ Test Pyramid

Cost of test

Amount of tests

Client implementation that maps to your HTTP RESTful Backend API

Simulating browser AJAX calls and acts like a user.Via GET / POST / PUT / DELETE requests and etc..

Request libraries:- requests - https://github.com/request/request- request-promise - https://github.com/request/request-promise- etc...

REST API Client

Anyone here use selenium for automation ?

Industry standard for web browser automation

Language-neutral coding interface: C#, JavaScript , Java, Python, Ruby

Official Project - https://github.com/SeleniumHQ/selenium

Selenium Webdriver

$ npm i selenium-webdriver

Visual Diff Engine

Bitmap comparison tool for screenshot validation.

Validates a given screenshot against a baseline image

Available tools:- Applitools - https://applitools.com/ integrates with selenium JS- Sikuli, roll your own framework - http://www.sikuli.org/

Baseline Changes

Timeline

Q2 2015

The Early AgeRapid Prototyping Phase

The Middle Age“The Promise Manager”

Evaluating multiple test frameworks

Test Runner- Mocha / Karma- Ava / JestAPI - Http library- request- co-request- request-promise- Supertest / ChakramUI - Selenium bindings- Nightwatch- WebdriverIO- selenium-webdriver (WebdriverJS)

Q3 2015 Q3 2016 - Present

- ES6/ES2015- Promises- ESLint- Went with Mocha- Picked selenium-webdriveraka WebDriverJsBuilt-in Promise Manager(synchronous like behavior)- Went with co-requestGenerator calls, easier(synchronous like behavior)- Applitools visual diff

- ES7/2016+- Async Await- Babel- ESLint + StandardJS- Flowtype (facebook)- selenium-webdriver- Switched to request lib

Async / Await solves a lot of the previous problems, no more callbacks or thenables

The Modern Age“Async/Await” and beyond

Lessons Learnt

We made mistakes so you don’t have to!

Recipe for a Node/Selenium

Framework

- Representation of Pages (Page Objects)- Readable tests- Frontend testability- Robust backend API

- Clients to interact with said API- De-async testing approach- Visual tests- Accurate and descriptive reporting

Automation Framework Ingredients

- Representation of a page or component.

- We use class syntax- Abstract UI interaction

implementation to a BasePage or UI utility outside of the pageobject

- Map page objects 1-to-1 to pages and components

Page Object Model (POM)

const CHANGE_PASSWORD_BTN = 'button.change-passwd'

class ProfilePage extends BasePage {

clickChangePassword () {

click(CHANGE_PASSWORD_BTN)

}

}

Readable Tests

clusterImageGallery.currentSelectedImageAnnotationIndicatorExistsInTrack()

Great work, Chip!

- Anyone should be able to read or write tests- Abstract low level webdriver implementation

outside of page objects, e.g. Navigation utility- Long-as-hell method names are actually OK

as long as they clarify the test

- Testability is a product requirement- Cook data models into your elements- Stop Xpath in its tracks. Use CSS selectors

Frontend Testability

Xpath Selector//div[@class='example']//a

CSS Selectordiv.example a

Frontend Testability- Data attributes for testing framework - image name, image uuid- Clear and concise selectors (ideally separate from styling concerns)

data attributes: image id, name

Concise selectors- class .main-image- Image track .slick-track- Selection status .selected

- UI tests should limit UI interaction to only the functionality being tested

- Build a client to speak to your backend through HTTP (still act as an ‘outsider’ user)

- Setup test data, simply navigate to it, and interact

- If you can’t expose your backend, bootstrap test data beforehand

Use the Backend API

it('Validate Bob\'s profile', function () {

const user = apiClient.getUser('bob')

// if you know Bob's user id,

// you can go straight to his profile

driver.goToProfile(user.id)

const title = profilePage.getTitle()

assert.isEqual(title, 'Bob\'s Killer Profile')

})

The gordian knot:

- Selenium actions and the WebDriver API are asynchronous

- JavaScript is built for asynchronicity- Tests should run in a synchronous

manner and in a wholly deterministic order

De-asyncing Tests

How to cleanly write async actions in a sync manner?

….sandwiches?

It’s like we finish each other’s….

- At first, we used the promise manager extensively- Heavily dependent on selenium-webdriver control flows

The promise manager / control flow● Implicitly synchronizes asynchronous actions● Coordinates the scheduling and execution of all commands● Maintains a queue of scheduled tasks and executes them in order

De-asyncing Via Promise Manager

driver.get('http://www.google.com/ncr');driver.findElement({name: 'q'}).sendKeys('webdriver');driver.findElement({name: 'btnGn'}).click();

driver.get('http://www.google.com/ncr') .then(function() { return driver.findElement({name: 'q'});}) .then(function(q) { return q.sendKeys('webdriver');}) .then(function() { return driver.findElement({name: 'btnG'});}) .then(function(btnG) { return btnG.click();});

test.it('Verify data from both frontend and backend', function() { var apiClient = new ApiClient(); var projectFromBackend; // API Portion of the test var flow = webdriver.promise.controlFlow(); flow.execute(function *(){ yield webClient.login(USER001_EMAIL, USER_PASSWORD); var projects = yield apiClient.getProjects(); projectFromBackend = projectutil.getProjectByName(projects, QA_PROJECT); }); // UI Portion of the test var login = new LoginPage(driver); login.enterUserInfo(USER001_EMAIL, USER_PASSWORD); var topNav = new TopNav(driver); topNav.getProjects().then(function (projects){ Logger.debug('Projects from backend:', projectsFromBackend); Logger.debug('Projects from frontend:', projects); assert.equal(projectsFromBackend.size, projects.size);});

- Not Readable / flat- Not testrunner-agnostic- Non-standard

API part of the testHTTP Requests

Issues with Promise Manager

Context switch to REST API calls

UI Part of the testSelenium & Browser

var test = require('selenium-webdriver/testing');

JobsPage.prototype.getJobList = function () { this.waitForDisplayed(By.css('.job-table')); const jobNames = []; const defer = promise.defer(); const flow = promise.controlFlow(); const jobList = this.driver.findElement(By.css('.job-table')); // get entries flow.execute(() => { jobList.findElements(By.css('.show-pointer')).then((jobs) => { // Get text on all the elements jobs.forEach((job) => { let jobName; flow.execute(function () { job.findElement(By.css('.job-table-row-name')).then((element) => { element.getText().then((text) => { jobName = text; });

// look up more table cells... }); }).then(() => { jobNames.push({ jobName: jobName }); }); }); }); }).then(() => { // fulfill results defer.fulfill(jobNames); }); return defer.promise;};

Bad complexity + promise chaining =

Async / Await

function notAsync () {

foo().then((bar) => {

console.log(bar)

});

}

async function isAsync() {

const bar = await foo();

console.log(bar)

}

Node > 7.6ES5

Given function foo() that returns a promise...

C’mon gang, let’s write a test!https://github.com/airware/webdriver-mocha-async-await-example

- When reporting, consider what information you need in order to be successful- Failures First- Flakiness vs. Failure- Assertion messaging - Screenshots- Video

- Screenshots very useful if done right- Integrate them into test reports- Use them for visual testing (shoutout to Applitools)

Reporting and Visual Testing

Applitools - Automated Visual Testing

"I don't write tests, testing sucks"

"Test automation is easy with

JavaScript "

When You Talk About Test Automation

Airware github repo code example:https://github.com/airware/webdriver-mocha-async-await-example

ForwardJS OrganizersDave, Taylor, Tracy and team

Special Thanks

Saucelabs - browser CI infrastructureFeedback on selenium and node.js prototyping

● Neil Manvar● Kristian Meier● Adam Pilger● Christian Bromann

ApplitoolsAutomated Visual Diff Engine from Applitools

● Moshe Milman● Matan Carmi● Adam Carmi● Ryan Peterson

MochaJSGithub#mochajs

Test runner

Selenium WebdriverGithub

BrowserAutomation

WebdriverIOGithub@webdriverioEasy to useSelenium APIMature Mobile APIs

AppiumGithub@AppiumDevs

Mobile & Native Apps Automation

JavaScript Community and Open Source Projects. The world runs on OSS - Please help support these projects!

Thank you!

Q & A