1/x
Quality continuously delivered
Agile Testing in an Open Source Project
Oliver Goetz – BREDEX GmbH
2/x
About me..
Oliver Goetz
M.A. Sociology
Functional Test Consultant at BREDEX GmbH since2011
QA in Jubula project
Test Manager in customer project
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
3/x
Agenda
Introduction• Introduction of the Jubula project
How do you do it? A typical release cycle• More than the sum of its parts – Roles and parts in the team• When do we start? – Basic time conditions• We need a plan! – Creation of User Stories and tasks• Let´s do it! – Implementing the tasks• Confidence is good, but… - Quality Assurance• Release the dogs! - Internal Continuous Delivery and external
Release
The Toolbox – Used tools
Conclusion
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
4/x
Introduction – The Jubula project
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
Code released under EPL (seit 2011)„Eclipse for Testers“
Open Source
GUIdancerStandalone(2004 - 2015)
Feature identic since 2013
JubulaStandalone
(2011 - today)=
5/x
Introduction – The Jubula project
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
6/x
Introduction – The Jubula project
Create tests early
Don´t have to write code
Good structure for tests
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
7/x
How do you do it? More than the sum of its parts – Roles and parts in the team
Product Owner
Developers
QA
Students and Apprentices
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
8/x
Stakeholders• Customers
• Community
• Test Consultants from BREDEX
• Team itself
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? More than the sum of its parts – Roles and parts in the team
9/x
How do you do it? When do we start? – Basic time conditions
Sprints• Three weeks long
• Start: Sprint Setup
• End: Sprint Review and Lessons Learned
• Between: User Stories meetings
Releases• Official major/minor standalone release: usually every six
months
• Official beta standalone releases: usually after every Sprint
• Official Eclipse releases: usually every six months
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
10/x
How do you do it? When do we start? – Basic time conditions
How does this contribute to a higher quality?• <= 4 weeks too long, >= 2 weeks too short
• Sprint Setup: Overview, Availability, Meet the commitment
• Sprint Review: Why did(´nt) we realize all stories
• User Stories Meetings: Forecast, Estimation
• Releases: Fast feedback
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
11/x
User Stories• External/Internal Customers
• Team itself
• BREDEX Test Consultants
• Community
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? We need a plan! – Creation of User Stories and tasks
12/x
User Stories• PO stores them in the backlog
• PO writes acceptance criteria
• Results in (not so detailed) Epics
or User Stories
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? We need a plan! – Creation of User Stories and tasks
13/x
User Stories• User Stories Meeting
• Team discusses User Stories and creates tasks for them
• Estimation: implement, document and test (in this order) • A (up to 12h), B (up to 20h) and C (up to 40h)
• Availability
• User Stories/tasks assigned to certain team members basedon their expertise and availability
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? We need a plan! – Creation of User Stories and tasks
14/xATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? We need a plan! – Creation of User Stories and tasks
15/xATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? We need a plan! – Creation of User Stories and tasks
How does this contribute to a higher quality?• User stories come from „real“ users real problems
• This includes the team
• Discussion leads to common understanding
• Abstract estimations to clarify complexity• But still try to keep it simple
• Small team <-> large project specialists• Knowledge transfer where possible, two people for one topic
16/x
Tasks/User Stories implemented in the order of theirBacklog rank
Usually the assigned developer implements• Speaks with the „topic associate“
• Otherwise he delegates
After implementing the task is set to „Done“• Now the task has to be documented
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Let´s do it! – Implementing the tasks
17/x
How does this contribute to a higher quality?• Important stuff is done first
• Assigned developer is „specialist“ but passes his knowledge
• Implementing before documenting encourages communication
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Let´s do it! – Implementing the tasks
18/x
Task set to done and documented
Does the docu meet the acceptance criteria? docugets tested
Now the actual task/implementation gets tested
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Confidence is good, but… – QA – Manual Test
19/x
Automated tests using Jubula
Several different Test Suites (69)• Tests for the ITE
• CaA tests for supported toolkits
Written by tester and developer
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Confidence is good, but… – QA – Automated Tests
20/x
User Story/Task contains Automation of AcceptanceTest
Developer starts Tester starts
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Confidence is good, but… – QA – Acceptance Test Driven Dev
21/x
Only a small number of Unit Tests for new features• Started late
• For new features
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Confidence is good, but… – QA – Unit Tests
22/x
How does this contribute to a higher quality?• Testing documentation Is this what we need/want
• Automated Functional tests Safety net
• Communication supports detecting bugs early
• ATDD based on functional tests
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Confidence is good, but…
23/x
Testing Sprint before an official release
Performed by all team members
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? A thing of the past? – Exploratory testing weeks
24/x
How does this contribute to a higher quality?• Cover parts of the application which are not covered by
automated tests
• Raise confidence of the whole team
• Find blind spots of the QA
• Developers get a feeling for „their“ application
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? A thing of the past? – Exploratory testing weeks
25/x
Nightly Hudson Builds
The next day everyone works with „the current“
Quick feedback about quality
Motivation to continue to deliver a high quality product
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Release the dogs! – Internal Continuous Delivery and external Release
26/x
Other BREDEX projects get new version as soon as„their“ feature has been implemented
After every sprint a „Sprintly“ is released and can bedownloaded from testing.bredex.de by everyone
Every six months an official release is uploaded
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Release the dogs! – Internal Continuous Delivery and external Release
27/x
Other BREDEX projects get new version as soon as„their“ feature has been implemented
After every sprint a „Sprintly“ is released and can bedownloaded from testing.bredex.de by everyone
Every six months an official release is uploaded• Trying to shorten this period
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Release the dogs! – Internal Continuous Delivery and externalRelease
28/x
How does this contribute to a higher quality?• Quick feedback on newly implemented features
• Quick feedback from the „real“ users
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
How do you do it? Release the dogs! – Internal Continuous Delivery and externalRelease
29/x
The Toolbox Used tools
Eclipse as IDE
Hudson for Continuous Integration
Jubula for automated tests
Jira as ALM tool
Bugzilla for bugtracking
And some more…
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
30/x
Conclusion
A very small, steadily changing team requires specialmeasures to gain a high quality product• Say goodbye to rigid team roles!
• Sometimes Scrum has to be „adapted“ to the team!
Continuos Delivery is only possible if everyone in theteam cares about the quality (assurance)
Continuos Delivery raises the trust in your product
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered
31/x
Thank you for listening!
www.bredex.de
testing.bredex.de
www.eclipse.org/jubula/
ATD 2015 - Oliver Goetz - BREDEX GmbH - Quality continuosly delivered