Continuous Testing and New Tools for Automation

Using Selenium, Sauce Labs, GitHub, and Travis-CI**Presentation given at StarWest 2014

How I Spent My Summer Testing

Continuous Testing on the Beach

What is Continuous Integration?

“Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.”

http://www.thoughtworks.com/continuous-integration

My Continuous Testing Process

1. Commit and push

2. Set up test server and env

3. Check out the code

4. Set up the test dependencies

5. Set up the test databases

6. Run the tests

7. (Optionally) deploy code

Using GitHub for my Repo

My Continuous Testing Process

1. Commit and push

- Borrowed from Armin Ronacher Flaskr tutorial - https://github.com/mitsuhiko/flask/tree/master/examples/flaskr/

My Super Simple Demo App

My Super Simple Demo App

GitHub and the .travis.yml file

Using Travis for my Builds

What is Travis-CI?

“Travis-CI is a hosted, distributed continuous integration service used to build and test projects hosted at GitHub. Travis-CI automatically detects when a commit has been made and will try to build the project and run tests.”

— http://en.wikipedia.org/wiki/Travis_CI

My Continuous Testing Process

1. Commit and push

2. Set up test server and env

3. Check out the code

4. Set up the test dependencies

5. Set up the test databases

language: Python

python: - 2.7

before_script: - lots of shell scripting here for setup for apache, etc.

script: - flask --app=flaskr run &> flask.log &

Simple .travis.yml example

Travis checks out the code…

Dependencies and DB Setup

• Flask configuration and restart with document root and port

• SQLite setup routines create the DB and CREATE TABLE

• “pip install” runs to fetch and install various dependency packages

Using Selenium for my Tests

Need help getting started with Selenium? Download the Selenium Bootcamp series here!

My Continuous Testing Process

1. Commit and push

2. Set up test server and env

3. Check out the code

4. Set up the test dependencies

5. Set up the test databases

6. Run the tests

My Super Simple Test Case

1.Go to index page2.Click Login link3.Select Username text field element4.Enter our username5.Select Password text field element6.Enter our password7.Click login button

My Super Simple Test in Python

1.Go to index page2.Click Login menu3.Select Username text field element4.Enter our username5.Select Password text field element6.Enter our password7.Click login button

A Selenium server can be installed and run in the build env via .travis.yml

Using Sauce for our Test Runs

Try Sauce for free, get an account here!

Running Tests on Sauce

Sauce Connect is what enables the grid to run inside the Travis env

Configuring Travis for Sauce

gem install travis

travis init

travis encrypt SAUCE_USERNAME=your_sauce_username --addtravis encrypt SAUCE_ACCESS_KEY=XXXXXXXXXXXXXXXXX --add

Next, initialize your project for use with Travis CI:

First, install the Travis gem locally:

We need to encrypt our credentials for safe use in GitHub:

language: pythonpython: - 2.7

env: global: - secure: HOTmOq6r+fjDDvr7gzETG7rS9IQtZ7QQ= - secure: NFM+4hE1VdaGs/lhaiVdn9Vi9P5L8Nb2t=

addons: sauce_connect: true

before_script: - lots of shell scripting here for apache, etc.

script: - flask --app=flaskr run &> flask.log & - py.test -n4 --boxed example.py

.travis.yml example for Sauce Connect

My Continuous Testing Process

1. Commit and push

2. Set up test server and env

3. Check out the code

4. Set up the test dependencies

5. Set up the test databases

6. Run the tests

7. (Optionally) deploy code

language: pythonphp: - 2.7env: global: - secure: HOTmOq6r+fjDDvr7gzETG7rS9IQtZ7QQ= - secure: NFM+4hE1VdaGs/lhaiVdn9Vi9P5L8Nb2t=addons: sauce_connect: truebefore_script: - lots of shell scripting here for apache, etc.script: - flask --app=flaskr run &> flask.log &

deploy: - your deployment instructions go here

.travis.yml example for deployment

Demo

Q&A