Fosdem 2016 - Community Driven Testing

Community Driven Testing: Lessons learned from the BuddyUp Pilot project

Marcia Knous and Ioana Chiorean

What is BuddyUp?

● Single support entry point for users - integrates support offered on hardware, software or service issues

● Web app that enables on-device notifications for an instant answer to any question you have

A Firefox OS support app

What is BuddyUp?A Firefox OS support app

Why BuddyUp?

● An opportunity to try something new!

● No dedicated QA resources available for the project

● App was planned to ship by default on the Firefox OS 2.5 release

The Mozilla mission in action: human collaboration across an open platform

Pilot Project Objectives

● Teach contributors skills that would span the mobile testing landscape

● Give contributors a sense of what is like to be part of a QA testing team

● Make it easy for any FxOS user interested to help to become and feel part of a larger community of helpers

● Develop community mentors

Who participated?

● Two mentors - Marcia Knous and Ioana Chiorean

● Three community members○ Christos○ Fredy○ Karthikeyan

● First team meeting held in Mozilla Paris space in February, 2015 (Developers, Project Management, UX)

Ioana, Karth and Fredy discussing the project

Meetup attendes

How we structured testing

● App divided into three parts○ Testing as someone asking a question○ Testing as someone answering a question○ Profile and Settings

● Owners created a short test plan and test cases for each area

How we structured testing

● Periodic smoketests run to catch regressions● Community dialed into project meetings and reported status

How did they benefit?

● Gained knowledge in mobile testing, which can help them get a job

● Gained knowledge of how to approach testing an app from a project perspective

● Worked equally with the staff, as an important member of the team - they attended team meetings, sent reports, smoketested

How did they benefit?

Challenges and Lessons

● Time zone challenges

● Time based requests are difficult

● Lead Developer left for some time, and the project lost momentum

● Resource balancing

● No automation infrastructure meant lots of repetitive manual testing

Going further

● Created two tiger teams for Firefox OS features (Pin the Web/Control Center)

● Provided better documentation that can be applied to other projects as well

● Realized that in-person training is way more efficient than online meetings so we will try to duplicate this

How BuddyUp experience can help us more

Questions?