Testing beyond software, a case study

Embed Size (px)


So far, testing has always been related to hardware and, specially, to software development. We tend to agree that building new pieces of software, programs, applications... requires testing efforts and we, as a community, struggle to establish, strengthen and refine these efforts; because we believe that testing is never in vain, we believe that testing adds value to wherever it is attached to. If we believe that testing adds value to wherever it is attached to, why don’t apply testing efforts beyond software development? In this presentation, I will explain a work in progress case study of testing applied not to software development, but to software usage, for real-time production and launching of on-line market research surveys in a complex and highly demanding environment.

Text of Testing beyond software, a case study

  • 1. Testing beyond software A case study@Mauri_Edo

2. Where is the action?Online market research projects +1.850.000 interviews in +1.000 projects (2012) Proprietary tools@Mauri_Edo 3. Online market research? Yes, online surveys mainly But online surveys are simple! Hmm... Are they? @Mauri_Edo 4. Mental image 3 minutes Text inputs Single response Multiple response Simple routing@Mauri_Edo 5. Real image 15 minutes Plus: Flash / jQuery Loops Client-side scripting @Mauri_Edo 6. The problem: Faulty surveys being launched Lets add some testing and quality to this process! Cool! But where to start? @Mauri_Edo 7. Gather information! Primary source Ask questions! Secondary sources Observation Current issues Current quality level @Mauri_Edo 8. Information gathering results Mission: Compliance with requirements Create a safety net before survey reaches the customer for approval Stakeholders: Customers Respondents @Mauri_Edo 9. Triage Little bets*Iterate ApproachDiversify Persevere! *A Peter Sims ideaMonitor DIY @Mauri_Edo 10. Disclaimer: Work in progress, potential fatal errors included@Mauri_Edo 11. Current status FormSpecCodingC.A.T.LaunchSurvey testingForm reviewSpec review Test designC.A.T. = Customer Acceptance TestingTest execution E.T. Data validation @Mauri_Edo 12. Challenge #1: culture Missing background on quality and testing: Knowledge transference (and recycling) Message consistency Communicate, communicate, communicate Aversion to change: Start small, gain momentum Prove your victories Gain followers to get more followers @Mauri_Edo 13. Challenge #2: miscommunication Miscommunication = confusion, mistakes, stress Lead by example: - Daily stand-up meetings - Weekly sync meetings Empower communication in others: - General, non-project related, meetings - Cross-team trainings - Stand up and talk! @Mauri_Edo 14. Challenge #3: development pace Survey programming cycle shouldnt be dramatically extended in time (anonymous co-worker)Challenge everything! Any deadlocks available? Testability + test utilities Smoke tests Smart checklists Testing depth Time available @Mauri_Edo 15. Error #1: inattention to specs Specs are valuable! Promote research and improvement in order to achieve specs ownership Make related issues visible Test against specs compliance @Mauri_Edo 16. Error #2: client modifications Before... FormSpecCodingC.A.T.Launch FixesAfter... FormSpecCodingC.A.T.LaunchExpress TestingFull regression testing? Maybe...@Mauri_Edo 17. Error #3: issue unawareness Who is aware of the issues? Respondent support team (functional, content) Application support team (internal) Connect the teams Active search for issues: Soft launch + data consistency analysis Repeatable process, anytime Dont wait for bugs to knock at your door @Mauri_Edo 18. Lessons learned@Mauri_Edo 19. About quality systems... The (successful) implementation of a quality system needs the bosses approval AND the embrace of the regular workers Quality might start with testing but it has to permeate through the rest of teams / areas@Mauri_Edo 20. About testing... Be adaptable, a full regression test suite execution is not the solution to all problems Communication is key Make yourself visible!Test strategies require lots of testing as well @Mauri_Edo 21. About challenges... Quality can add value everywhere, beyond software development Future job, anyone?@Mauri_Edo 22. Thanks! Questions? Mauri Edo @Mauri_Edo testingfuncional.wordpress.com