NoVaTAIG

April 7, 2010Automated In-Browser Testing with

Cucumber and Watir

Rudy Regner Mike Perz Erik Scheirer Rasik Pandey

Rudy Regner• Introductory Notes

– We’re all on linkedin.com.– Send us questions regarding any details presented

tonight

• Presentation Goals and Objectives– Mike Perz: Test Automation Framework– Erik Scheirer: End-to-End SDLC Architect– Rus Pandey: BDD Tools Expert

Our Situation

• Writing regression tests for an app in production– Only required to support IE

• Rails v1.2.3– Precludes using Webrat latest version– ?? Could possibly have used earlier version ??

• why regression tests for earmarks?– Bug made it through manual testing – Encapsulate critical functionality in automated

regression test scripts to decrease likelihood of this failure occurring again.

Mike Perz• How accomplished the automation framework• Challenges encountered

Our Learning Curve• Started with Watir

– Easy to get up and running• Added Cucumber (RSpec) • Replaced Watir with Webrat and Selenium

– Sought flexibility gain through re-use of Webrat to drive Selenium, Mechanize or straight Rails

• Productivity dropped with Selenium– Watir can manipulate the DOM as objects. (you can iterate

through the cells of an html table)– With Selenium you would have to find each cell individually.– Script debugging difficult without irb

• You can attach to a browser window* (with Watir)• Converted existing Selenium code to Watir

Cucumber Feature snippet

associated Step Definition

Scenario Outline example (similar to fitnesse tables)

Cucumber Test Summary Output (sample)

Once we switched back to Watir

• At first our tests were dependent on each other– They needed to be run in a particular order. – We used Rake to accomplish this.

• Open 4 browsers with 4 different users (Watir::IE.new_process vs. Watir::IE.new)

– Now we can attach Watir to whichever browser session we need

• Watir::IE.attach

– We use SQLite to keep track of which user is associated with which browser session

Interesting side benefits to writing automated tests

• Writing automated tests helped/made us understand the app better.

• Found application bugs that were missed earlier

Other useful tools

• Watircuke• IE Developer Toolbar• Firebug• FireWatir recorder:

www.itest2.com/downloads

Erik Scheirer• Spreading the Developer Joy• Benefits of a single executable Cucumber

artifact end-to-end

Status - Your Organization

• What Rules? Confusion? Sanity?• Even if its sane, it probably can be more so.• Some groups have managed a high degree

of sanity.• But what most organizations are

experiencing is ...

Chaos

• little Trust between groups + infighting• bad communication• no ‘Process’ (or its limping, bleeding, out

the door)• no time to implement anything new,

constantly fighting fires instead

This approach

• provides an end-to-end process via architecture, convention, and tools

• is easy to implement• is easy to customize to exacting technical

needs and group culture

This approach

• builds better quality• builds more trust between technical and

non-technical stakeholders• utilizes resources more effectively

Ingredients

• CTBE - catch that bug early!• Unit tests, feature and black-box testing,

via one single artifact• in this case an Agile approach (scrum)• artifact re-use creates self-reinforcing

system

One Artifact, multiple uses

• developer can create cucumbers as part of T/BDD, and also to capture ‘drive-by requirements’

• QA can create ‘cukes’ for regression tests• Cukes can be fed from dev. to QA or vice

versa• Cukes can be used by Continuous

Integration, and in Production monitoring

Next, the Process from multiple perspectives:

Stakeholder interaction,design and architecture,

includes non-technical

Iterations

testing & daily work The overall lifecycle

The daily grind

Time

Building Trust - spreading the joyBuilding Trust -

spreading the joy

Conclusion•There is a way to achieve more success, better utilization of existing resources, and get more sleep; using a simple, adaptable architecture coupled with process, convention,

and tools.

•Did we mention ‘metrics’ and how these can be used to CTBE in this manner, too?

• Its also all open-source, so its customizable to a great degree.

• Thank you for listening.

Rus Pandey• How BDD would work• Other possible tools

Ideal BDD Process:

• 1. Describe behaviour in plain text• 2. Write a step definition in cucumber• 3. Run and watch it fail• 4. Write code to make the step pass• 5. Run again and see the step pass• 6. Repeat 2-5 for all your functionality

Performance:

• Since it takes 3 hours to run a test suite we should investigate using spork and --drb and testjour to distribute the test load if possible to speed up test runs.

Test Web apps in any language: Drive a full or headless browser using one of these

• *Webrat – Ruby acceptance testing for web applications • *Capybara – Acceptance testing framework with a webrat-like API and support for multiple

backends, including RackTest, Selenium, Celerity and Culerity • *Steam – Drives a fast headless browser with Javascript support. Support for the normal

webrat step definitions, see Setting up Steam • *Testing PHP app using Webrat • *WebDriver – Drives IE, Firefox, Chrome • *Watir – Drives IE (Windows only), see: Setting up FireWatir • *FireWatir – Drives Firefox on Windows, OSX and GNU/Linux • *SafariWatir – Drives Safari (OS X only) • *ChromeWatir – Drives Google Chrome • *Celerity – Drives a fast headless browser with Javascript support. Examples here • *Culerity – For when you can’t run your app under jRuby • *Selenium – Runs any browser (any OS), see: Setting up Selenium • *Mechanize – Runs a headless browser (any OS)

Examples of Cucumber being used in the real world across

various languages/platforms to test all sorts of things.• http://wiki.github.com/aslakhellesoy/cucumb

er/tutorials-and-related-blog-posts