Continuous Testing at Code.org

@bcjordanBrian Jordan, software engineer at Code.org

Code.org

non-profit

expanding participation in CS

Hour of Code

230 million tried Hour of Code

now what?

full curricula, district partnerships,

professional development, policy change

K-12 CS Curriculum

25k K-12 teachers trained

courses: 300k teachers 10mm students

how did Code.org start automated testing?

development in 2013-2014

why test? what is Code.org's testing context?

the context

the context

the context

the context

the context

so how do we test all that?

how we started Selenium testing

how our tests are organized and run

selenium-webdriver cucumber

takeaways

Build a library of steps

Build a library of test levels

Build a library of annotations

@no_mobile@no_ie9

@skip

@db_access@as_student

@eyes

who writes tests?

who/what runs tests?

what challenges did we end up facing?

challenge:multiple browsers

solution:selenium-webdriver

Sauce Labs

challenge:testing new changes

solution:chromedriver sauce-connect

solution (soon):pull request tests in CI

challenge:

interaction, drag-and-drop

solution:jQuery Simulate

$(selector).drag…

challenge:

visual, responsive changes

hackathon solution?

not easy...

actual thing!

using eyes

converted feature

ignoring regions

what do tests look like when running?

(not) wasting time

speeding up test runs

2014: 60-90 min of tests

parallelizationat runner level

2015: 60-90 min of tests

to 20-30 min of tests

justifying tests

the bug collection

missing button

duplicate button

Firefox getBBox()

JS % error: node build issue

Chrome 50 offsetWidth

my favorite bug ever

who investigates failures?

what about failures pass tests?

dev of the day

what's next?

per-PR tests

managing more eyes tests

cross-browser responsive eyes tests

github.com/tourdedave/responsive-web-testing

takeaways

start small

everybody tests

periodically invest in speed-ups

code.org/help

github.com/code-dot-org

Thanks!

@bcjordanBrian Jordan, software engineer at Code.org

code.org/help

github.com/code-dot-org

Thanks!

@bcjordanBrian Jordan, software engineer at Code.org