Test Automation with Cucumber JVM, Selenium, and Mocha

Test Automation with Cucumber JVM, Selenium, and Mocha

Sanjay Gidwani, Salesforce.com, Director @gidzone

Shuji Ui, Sonoma Partners, Technical Solution Architect @shujiui

Safe harborSafe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

Shuji UiShuji Ui

Sonoma Partners

Salesforce.com Technical Solution Architect

@shujiui

Sonoma Partners

Salesforce.com Technical Solution Architect

@shujiui

Sanjay GidwaniSanjay Gidwani

Salesforce.com

Director of Salesforce.com team

@gidzone

Salesforce.com

Director of Salesforce.com team

@gidzone

Agenda

• Why test automation is important in Agile development?

• What is BDD?

• How did we automate tests?

Cucumber JVM + Selenium

Mocha

Jenkins

Agile project and automation

• How many people are in Agile project right now?

• Does your development team write unit tests for non-Apex code?

• Does your QA team automate E2E tests?

• Continuous Integration?

Sprint 13 Agile project experience

• Each sprint is 2 weeks, 13 sprints Agile project

• Custom Visualforce / JavaScript project

• 5 developers + 3 QA + 1.5 QA automation developer

• JavaScript unit tests with Mocha

• Regression testing automation using Cucumber JVM + Selenium

1.5 developers worked on implementing automation scripts and code

Write automation for previous sprint

Behavior-Driven Development• By writing specs/test cases in plain text, developers, business analysts, and QAs

with shared tools and a shared process to collaborate on software development.

• Based on test-driven development – Write test first

http://en.wikipedia.org/wiki/Behavior-driven_development

Frameworks Jasmine for JavaScript unit tests - http://pivotal.github.io/jasmine/

Mocha for JavaScript unit tests - http://visionmedia.github.io/mocha/

Cucumber for E2E tests - http://cukes.info/

Frank for iOS with Cucumber - http://www.testingwithfrank.com/

Cucumber JVMCucumber

•A testing framework based on BDD (Behavior-driven development)

•Writing test cases in plain text and then Cucumber generates Ruby, Java, .Net, Flex codes

•Generate reports

•Written in Ruby

http://cukes.info/

Cucumber JVM

A pure Java implementation of Cucumber

https://github.com/cucumber/cucumber-jvm

Selenium IDE and Remote Control

Selenium IDE

Firefox extension, allows you to record, edit, and debug selenium tests in Firefox. It exports Selenium Remote Control Java code.

http://docs.seleniumhq.org/projects/ide/

Selenium Remote Control Java client and the server

Selenium Remote Control (RC) is a test tool that allows user to write automated web application UI tests.

Selenium Remote Control

•IE8 was the browser requirement

•We created EC2 Windows

instance.

•Runing RC server on the Windows

instance.

•RC server starts up IE8.

Mocha

• JavaScript unit test framework• The syntax is similar to Jasmin

• We used Mocha because we used Brunch assembler for project. The assembler came with Mocha.

• Running on Node.js• Comes with several reporters

http://visionmedia.github.io/mocha/

How did we use Mocha unit tests? (Sanjay)

• Running Mocha in real browser, for example, IE8.

• Used for UI testing, e.g. filtering, displaying error based

on a condition…etc.

• QA Lead communicated to QA engineers and

developers to balance Mocha UI unit tests and e2e test

using Cucumber JVM + Selenium.

• Can generate report.

Continuous Integration with Jenkins (Sanjay)

• Open source Continuous Integration tool written in Java.• The project was forked from Hudson.• Easy to set up and running. • Active plugin community. • Our team installed Jenkins in our EC2 Ubuntu instance.

http://jenkins-ci.org/

Cucumber JVM Jenkins Plugin

https://github.com/masterthought/jenkins-cucumber-jvm-reports-plugin-java

Demo: Testing Flexible Contact Chatter-Follow Sample App with Cucumber JVM and Selenium

• Flexible Contact Chatter-Follow Sample App

• Write specs/tests in plain text and convert them to Java

code snippets.

• Selenium IDE and add Selenium IDE generated JUnit

code to the Java Code snippets.

• commit the code to git and run them with Jenkins.

• Cucumber JVM report.

Flexible Chatter Contact Follow App

1. One of our clients wants to use “Sync contacts users

follow in Chatter” option in Salesforce to Outlook

configurations

2. They want an app to make their assistants to control

chatter-follow for other people.

Flexible Chatter Contact Follow App Solution

1. Created a “Contacts Relationship” custom object as a

junction object between “Contact” and “User” objects.

2. When a “Contacts Relationship” record is created, an

Apex trigger is fired to make “User” follow “Contact”

relationship in Chatter.

Agile project: Sprint 1 and User Story 001

Suppose we are in Sprint 1 of an Agile project and the Flex

Contact Chatter-Follow is User Story 001.

Here is what to do before writing code:

1. BA and TA gather requirements from client.

2. TA and Dev design solution and create mockup/diagram.

3. BA and Dev writes scenarios and confirm with QA.

4. QA writes test cases and confirms with BA and Dev.

BA, QA, and Dev write tests in plain text

Cucumber JVM generates snippets from the plain text case

Selenium IDE to record a test case

Export JUnit code from Selenium IDE

Merge Selenium generated code to Cucumber generated snippets

Dynamic generated IDs in SFDC

• When a client wants to run integration automation code in

2 different sandbox orgs. HTML DOM ids are different

between 2 sandbox orgs.

Workaround

• xpath option in Selenium instead of using IDs to locate

HTML tags.

• QA automation developers need to learn xpath.

• Depending on page xpath is difficult to maintain.

Run the test cases in Jenkins and generates report

Running Mocha and generating the report

Conclusion

End of the 13 sprints, QA developers wrote e2e test code for about 90 User Stories and more than 200 test cases.

Developers wrote unit tests for all User Stories.

Jenkins run nightly and send report to QA/Dev team. The automation saved significant amount of QA time and delivered quality code to the client.

Sanjay GidwaniSanjay Gidwani

DirectorSalesforce.com

@gidzone

Shuji UiShuji Ui

Technical Solution Architect Sonoma Partners

@shujiui