QA Mission Plan - LibreOffice Conference · QA Mission Plan: Where we're headed in 2015 Howdy! Everyone in Europe thinks that the people in the US are a bunch of cowboys, so we figured

QA Mission Plan:Where we're headed in 2015

Robinson Tryon & Joel MaderoNotorious Legends of QA

LibreOffice Annual Conference – BernSeptember 3rd, 2014

https://creativecommons.org/licenses/by-sa/4.0/

Robinson Tryon & Joel MaderoLibreOffice Annual Conference – BernSeptember 3rd, 2014


Howdy!

Everyone in Europe thinks that the people in the US are a bunch of cowboys, so we figured that we'd style our presentation appropriately

We've got a bundle of topics to present to you today

The organizers only agreed to let us talk if we kept it to a half-hour, so don't worry, it'll be over soon.

Have questions?

Joel: Shut up and let us talk!

Robinson: Interrupt us anytime!



Kudos to Joel

Law student

Translation: Super-busy all the time

Roped-him in to giving a talk at the annual conference, with one catch:

I had to co-present

He still manages to find time to do tons of QA work

Will continue to be a valuable addition to the QA Team for years to come

Not going to send him out to the pasture quite yet

Bug: Verrrry Sloowww SVG load: https://bugs.freedesktop.org/83426

https://bugs.freedesktop.org/83426



RobinsonThe Vermont Kid

Volunteer in LibreOffice QA

Just hired in August as QA Engineer for The Document Foundation

Most people in the US know me from my LibreOffice outreach work

If you're ever attending a Free Software-related conference in the US, email me – I'll probably be attending or should be attending



Topics

Mission Plan

Bugzilla Migration

Statistics

Bibisect & GUI

Improvements to Bugzilla

US Summer Hackfest

Suggestions for team recruitment

Cross-project "Bug-bees"



Have questions?

Please feel free to ask questions during my talk

Active participation helps to keep you awakeIf you have longer questions/comments, we may time at the end for Q&A



QA Has a Mission Plan?

Work in progress...

Plan can help keep QA Team focused

Gives people outside QA Team better understanding of our goals

Measure our success



Bugzilla Migration

Bugzilla currently hosted on Freedesktop.org

We don't have direct access to Bugzilla or underlying OS

Unable to modify/update/improve Bugzilla

Can be difficult to contact FDO admins

Currently identifying and recruiting admins for Bugzilla, the BSA, and the VM

Are you interested? Come talk to us!

Updating Bugzilla code to ease transition

Coordinating with Norbert, QA Team members, and the developers



Bugzilla Improvements

Specific improvements to QA workflows

Identify common tasks for automation

Collaborate and brainstorm with QA Engineers in other projects

Wikimedia

Mozilla

GNOME

Provide onboarding information and tutorials within the bugtracker

Wiki: https://wiki.documentfoundation.org/QA/Bugzilla/LibreOffice_Bugzilla_Proposal#Post-Migration

https://wiki.documentfoundation.org/QA/Bugzilla/LibreOffice_Bugzilla_Proposal#Post-Migration



Speaking of Bugzilla...

Our volunteers have been doing an amazing job with UNCONFIRMED bugs

Consistently pushing that line lower

Looking forward to a coordinated, concurrent push in October between Seattle LibreFest and online participants



Statistics:UNCONFIRMED + NEEDINFO



Statistics:UNCONFIRMED + NEEDINFO + RESOLVED



Future of Bibisect

Repositories for Windows, OS X, and GNU/Linux

15+ QA volunteers happily bibisecting away!

Windows buildbot pushing to repository daily

Thanks, cloph!

OS X and GNU/Linux will push to repositories daily

Faster identification of commit range that introduced a regression

Get new QA contributors confident using bibisect

GUI for Bibisecting

Windows: Promising work by Florian Reisinger

GNU/Linux (and cross-platform): On our roadmap



QA Team:We do more than just poke at bug reports!



Multimedia Support:“Houston, we have a problem”

QA Team noticed increasing # of multimedia bugs

Originally on OS X

Expanded to include Windows

GNU/Linux seemed ok



Multimedia Support:The Problem

No technical documentation of supported images/audio/video in LibreOffice

No documentation describing how to enable multimedia support

Growing number of bug reports on multiple platforms

No existing test plan or test results



Multimedia Support:The Solution

Identification of community members on multiple teams

Recruitment of testers on multiple platforms

Creation of new wiki page[1] documenting supported formats

Identification/creation of standard test files and standardized testing methods

[1] https://wiki.documentfoundation.org/Media_Support

https://wiki.documentfoundation.org/Media_Support



Multimedia Support:The Result

Discovered issues on Windows and OS X

Learned about replacement/neutering of media playback framework on OS X

United Devs and QA Team members to build, triage, and test specific new versions of LibreOffice

Created a better resource for users who seek to report multimedia bugs



Lessons Learned?



QA is not static

QA needs to be nimble

Aggregate information to classify families of problems

Still have many “pre-OO” problems

A stitch in time saves nine:

Engineer a solution that prevents/simplifies related issues



QA Meetings

Try changing format of QA Meetings

Shorter meetings

Directly engage all meeting participants

IRC may allow more participation

People who aren't as comfortable speaking English, shy, or who have noisy children[1]

Lower threshold to join the meeting

[1] Extra noise on a poor-quality phone connection is a serious concern



QA Events & Recruitment:New Approaches

Social media

New types of outreach events

Connecting with people running Software Freedom Day and Document Freedom Day events

Incorporate recruitment into standard QA workflow

Joel does a great job here: Be more like him!

Invite candidates based on bug quality and professed interest

Consistency in how QA Team recruits?

[1] Actual claim from QA Team member



QA Events & Recruitment

As members move on due to graduating from school, new jobs, etc.. we are working to replace them

QA Team continues to have strong leadership

Identifying best practices to recruit and retain volunteers

Running QA-specific events to grow the QA Team

Organized hackfest in Boston

Leading LibreOffice Community “Bug-Triaging-Bee” in Seattle this October

Strong interest in community events in Portland, OR and Austin, TX



QA: US Summer Hackfest

Joel & Robinson present from QA Team

Hosted by Xamarin

Other participants

Michael M., Immanuel, Algot

Martin (Inkscape developer)

Two Xamarin developers

Dick & Jill Miller



Hackfest: Lessons

Search diligently for participants (esp. developers)

The early hackfest catches no developers

10:00 – 20:00 ok

11:00 – 21:00 better

Unclear how far this trend continues

Simplify advertisement

Be clear about expectations when people join

Maintain list of EasyHacks/EasyTasks for new people to grab

Physical whiteboard, Etherpad, or similar

Work your local contacts



Hackfest: Venue Lessons

Confirm venue well ahead of time

Well-signed, open venue requires less involvement than hidden, locked-down venue

Centralized venue encourages participation

Free parking at venue is appreciated

Accommodations close to venue maximizes effective time



We ♥ Universities!Tips for a successful Hackfest/LibreFest

Partner with a University

Make sure that the school will be in session

Contact CS department early

Find enthusiastic, FOSS-friendly professors or graduate students

Ask department to promote hackfest to classes

Does the school have a LUG or computer club?

Reach out to the OpenHatch project

Has an Open Source Comes to Campus event been run in the local area?



Cross-project “Bug Bees”

LibreOffice is a huge project

Coordinating bug triage with other projects is valuable in many ways

Fresh new QA work: Energizing and exciting

New software, new experience, perhaps new tools

Make friends that we can poach may spend time with us

Smaller project: Have a very visible effect

Identify projects that are interested in partnering-up

Inkscape? VLC?



The End

Robinson TryonQA EngineerThe Document FoundationLibreOffice Community Outreach Herald

Email: [email protected]: [email protected]

URL: https://conference.libreoffice.org/2014/talks/speeches-qa/#qa-mission-plan--where-we-are-headed-in-2015

Joel MaderoBoard MemberThe Document FoundationLibreOffice QA CowboyEpic Law Student

Email: [email protected]

Any questions?



Sources and Licensing for y'allPDF graphic by FSFE – public domainhttp://www.pdfreaders.org/graphics.en.html

Walrus with TDF eyes by Robinson Tryon,based on Cartoon Walrus by Studio Fibonacci – public domainhttps://openclipart.org/detail/169963/cartoon-walrus-by-studiofibonacci

Cowboy with guns by j4p4n – public domainhttps://openclipart.org/detail/191808/cowboy-with-guns-by-j4p4n-191808

Cowboy hat by nicubunu – public domainhttps://openclipart.org/detail/14836/cowboy-hat-by-nicubunu

Sheriff star by studio_hades – public domainhttps://openclipart.org/detail/160033/sheriff-star-by-studio_hades

Barbed wire by Muga – public domainhttps://openclipart.org/detail/19289/barb-wire-by-muga-19289

Cowboy on a horse with a gun by j4p4n – public domainhttps://openclipart.org/detail/182389/cowboy-on-a-horse-with-a-gun-by-j4p4n-182389

Cowboy at rodeo by liftarn – public domainhttps://openclipart.org/detail/181766/cowboy-at-rodeo-by-liftarn-181766

Firefox Logo by Mozilla Foundationhttps://wiki.mozilla.org/Firefox3.5/LogosFirefox is a registered trademark of the Mozilla Foundation.

Android robot by Google – CC-BY 3.0https://developer.android.com/distribute/tools/promote/brand.htmlReproduced or modified from work created and shared by Google

Toy unicorn by spacefem – public domainhttps://openclipart.org/detail/184660/toy-unicorn-by-spacefem-184660

All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 4.0 License (unless otherwise specified). "LibreOffice" and "The Document Foundation" are registered trademarks. Their respective logos and icons are subject to international copyright laws. The use of these therefore is subject to the trademark policy.



You fell off the end

mailto:[email protected]

mailto:[email protected]

https://conference.libreoffice.org/2014/talks/speeches-qa/#qa-mission-plan--where-we-are-headed-in-2015



Nooooooooooooooo.......

http://www.pdfreaders.org/graphics.en.html

https://openclipart.org/detail/169963/cartoon-walrus-by-studiofibonacci

https://openclipart.org/detail/191808/cowboy-with-guns-by-j4p4n-191808

https://openclipart.org/detail/14836/cowboy-hat-by-nicubunu

https://openclipart.org/detail/160033/sheriff-star-by-studio_hades

https://openclipart.org/detail/19289/barb-wire-by-muga-19289

https://openclipart.org/detail/182389/cowboy-on-a-horse-with-a-gun-by-j4p4n-182389

https://openclipart.org/detail/181766/cowboy-at-rodeo-by-liftarn-181766

https://wiki.mozilla.org/Firefox3.5/Logos

https://creativecommons.org/licenses/by/3.0/

https://developer.android.com/distribute/tools/promote/brand.html

https://openclipart.org/detail/184660/toy-unicorn-by-spacefem-184660

http://wiki.documentfoundation.org/TradeMark_Policy

https://creativecommons.org/licenses/by-sa/4.0/

Documents

QA Mission Plan - LibreOffice Conference · QA Mission Plan: Where we're headed in 2015 Howdy! Everyone in Europe thinks that the people in the US are a bunch of cowboys, so we figured