How to be an explorer of software - v11 - EuroStar · A Check Has Three Elements 1. An observationlinked to… 2. A decision rulesuch that… 3. both observation and decision rule

How to be an explorer of software

Huib Schoots

@huibschoots

Who am I?• Agile coach & Scrum master

• Context-driven software tester

• Rapid Software Testing teacher• Trainer, coach, writer, speaker, leader & storyteller

• Curious & lifelong learner• Passionate & energetic people lover

• Books & Apple gadget collector

• Trombone player

• Avid Gamer

• Beer brewer • Star Wars &LEGO freak

• Aspiring magician

Thank you:• Keri Smith for your inspiration!• Ruud Cox en Joris Meerts for the many discussions

on this topic and help with preparing thisworkshop

• Obviously James Bach and Michael Bolton forsharing their knowledge about Rapid Software Testing

Some slides are taken from Rapid Software Testing and are used withpermission. Rapid Software Testing is developed by James Bach andMichael Bolton. SeE Also: http://www.satisfice.com/info_rst.shtml

Many of the pictures are taken from the book “How to be an explorer of the world”. This book is writen by Keri Smith and published by Penguinbooks.

Buy this book and practice!

Acknowledgements

• Introduction• What is testing?• Part I: analysis• Part II: Coverage• Part III: Testing story

Agenda for today

I will ask many difficult and critical questions (a.k.a. Socratic method). I ask them because it is important to fully understand the concepts we discuss today. Just say “pass” or “help me” if you don’t feel comfortable.

Install software

Ghost - publishing platformhttp://ghost.org à create your own (free) accountOR use mine: http://huib.testlab.site/

XMIndhttp://www.xmind.net/download

Introductory discussion…

Let’s Discuss:

• What is testing? • What do testers do?• What makes testing exploratory?

What do you do when you test?

• Name activities and skills

Definitions

• Activity:a thing that a person or group does

• Skill:the ability to do something well. Something Can be learned and practiced.

• Trait/characteristic/aptitude:your natural abilities. They are a part of who you are, what you do and why you do it.Cannot be trained. But Can be encouraged or discouraged.

Source: how to be an explorer of the world - Keri Smith


And Testers do too!



What is testing?What do testers

do?

Testerslighttheway

Thisisourrole.Weseethingsforwhattheyare.

Wemakeinformeddecisionsaboutqualitypossible,becausewethinkcriticallyaboutsoftware.

Source:RapidSoftwareTesting– JamesBach&MichaelBolton

Testerslighttheway:theriskgap

What we know

What we need to know

Ourknowledgeofthestatusoftheproduct.


Callthischecking, nottesting

Observe Evaluate Report

Interactwiththeproductinspecificwaysto collectspecificobservations.

Applyalgorithmicdecisionrulestothoseobservations.

Reportanyfailedchecks.

means

operatingaproducttocheckspecificfacts

aboutit…


ACheckHasThreeElements1. Anobservation linkedto…2. Adecisionrule suchthat…3. bothobservationanddecisionrulecanbeapplied

algorithmically.

Acheck canbeperformed

byahumanwhohasbeeninstructednot tothink

(andwhoisslowandvariable)

byamachinethatcan’t think

(butthatisquickandprecise)Source:RapidSoftwareTesting– JamesBach&MichaelBolton

Acquiringthecompetence,motivation,andcredibilityfor…

Testingis…

creatingtheconditionsnecessaryfor…

…sothatyouhelpyourclientstomakeinformeddecisionsaboutrisk.

evaluatingaproductbylearningaboutitthroughexperimentation,whichincludesto

somedegree:questioning,study,modeling,observationandinference,including…

operatingaproducttocheckspecificfactsaboutit…


Testing’sMissionisFocusedonLearningTestershelptodefendthevalueoftheproduct

bylearning onbehalfofourclients.

execution discovery

investigation

learningreporting

design


Exercise 1.1: survey session

25Source: how to be an explorer of the world - Keri Smith

Survey Testing

Any testing that has as its primary goal learning about the design, purposes, testability, and possibilities of the product. Survey testing tends to be open and playful. It provides a foundation for effective, efficient testing, later on.


Heuristics:GeneratingSolutionsQuicklyandInexpensively

• Heuristic(adjective):servingtodiscoverorlearn

• Heuristic(noun):afalliblemethodforsolvingaproblem

ormakingadecision

“Heuristicreasoningisnotregardedasfinalandstrictbutasprovisionalandplausibleonly,whosepurposeistodiscoverthesolutiontothepresentproblem.”

GeorgePolya,HowtoSolveIt


Concise Documentation

TheCuffChecklistSource:RapidSoftwareTesting– JamesBach&MichaelBolton

Wanttocovertheproduct?

§ Structure § Platform

§ Function § Operations

§ Data § Time

§ Interfaces

Remember:“SanFranciscoDepot”Source:RapidSoftwareTesting– JamesBach&MichaelBolton

Exercise 1.2: survey session

• Work in pairs

• Use Ghost

• Do a survey session to Create a MApOF THE SOFTWARE (PRODUCT coverage outline )

• Use SFDIPOT

• Make notes

30

Definitions• PRODUCT Coverage outline:a systematic presentation of test conditions.

• test condition:anything about the product that you might want to cover in the course of testing.

• To coverexamine SOMETHING for the purposes of testing.

• coverage:how much testing we’ve done with respect to some model.

It’s the extent to which we have traveled over some map of the product.

Visual coverage is like a map

DailyTestSessions

Project

Strategy

Gene

ral

Ongoing

Plan

ning

LowTechDashboard

TestCharters

(weekly)planningsession

Wrap-up/Debrief

TestSession

Issues BugsRiskCatalogHeuristics TestIdeas

RisksTestPlan Coverage

TestLog

Wrap-up/Debrief

TestSession

TestLog

Wrap-up/Debrief

TestSession

TestLog

SessionSheets

CharterPatterns:Evolvingteststrategy

• IntakeSessions(Goal:negotiatemission)“InterviewtheprojectmanagerabouttestingXmind.”

• SurveySessions(Goal:learnproduct)“FamiliarizeyourselfwithXmind.”

• SetupSessions(Goal:createtestinginfrastructure)“Developalibraryofmindmaps fortestingXmind.”

• AnalysisSessions(Goal:getideasfordeepcoverage)“IdentifytheprimaryfunctionsofXmind.”“Constructaproductcoverageoutline.”“Brainstormtestideas.”“Prepareastatemodelforstate-basedtesting.”“Performacomponentrisk-analysistoguidefurthertesting.”“DiscoveralltheerrormessagesinXmind.”


CharterPatterns:Evolvingteststrategy

• DeepCoverageSessions(Goal:findtherightbugs)“Performscenariotestingbasedonthescenarioplaybook.”

“Performatourthatachievesdouble-transitionstatecoverage.”

“Performsteeplechaseboundarytestingonthemajordataitems.”

“TesteacherrormessageinXmind.”

“Performafunctiontourusingthe2300nodemindmap.”

• ClosureSessions(Goal:getreadytorelease)“Verifythelatestfixes.”

“Re-testtutorialwiththelatestbuild.”

“Reviewhelpfilesandreadme.”

“GooverdeferredbugswithCustomerSupportpeople.”

“Performclean-machineinstalltest.”


Exercise 2: Deep coverage session• Work in pairs• use Ghost

• Use your map from Exercise 1• Now TEST one specific part

of Ghost in detail:Test the Format functionality

• Make notes!

Why storytelling?

• We understand stories faster and easier• We remember stories better• Stories convince more• Stories motivate more• Stories connect• Stories provide grip• Stories create manageable• Stories influence

Source:Storypower– Verhalendieorganisatieveranderen– AnnetScheringa

What is a test report?

• A test report is any description, explanation, or justification of thestatus of a test project.

• A comprehensive test report is all of those things together.

• A professional test report is onecompetently, thoughtfully, andethically designed to serve yourclients in that context.

• A test report isn’t “just the facts.” It’s a story about facts.

Bron: Rapid Software Testen – James Bach & Michael Bolton

Totestistoconstructthreestories

Level 1: A story about the status of the PRODUCT……abouthowitfailed,andhowitmight fail...…inwaysthatmattertoyourvariousclients.

Level 2: A story about HOW YOU TESTED it……howyouconfigured,operatedandobservedit……aboutwhatyouhaven’ttested,yet……andwon’ttest,atall…

Level 3: A story about the VALUE of the testing……whattherisksandcostsoftestingare……howtestable(ornot)theproductis……thingsthatmaketestingharderorslower……whatyouneedandwhatyourecommend…

WhyshouldIbepleasedwithyourwork?


Exercise 3: Testing Story

• groupS of 4• Use your notes from earlier exercises

• What did you see? • What have you tested?

• Prepare a (written or oral) report on Ghost

Using the testing story

• Building a story during testing• Status• Wrap-up and debrief• reporting• Overview and insight

Why exploring is powerful

• Do what really needs to be done• It facilitates experimentation,

serendipity, discovery and learning• It finds more bugs• Create engagement: help people use their

brains• Take advantage of tacit knowledge and

skill• Use insights from experiments to inform

the next• Using the full creative power of

exploration


Next steps…

• Practice, Practice, Practice!– Focus and Defocus– Alternate approaches: playful vs. Deliberative– Note taking– Coverage reporting– (Self) management– Creating test ideas fast

• Observe others test

• Observe yourself test

(use the heuristics from the “Exploratory Testing Dynamics” list form the RST appendices: http://www.satisfice.com/rst-appendices.pdf)

SizeValue

Lessons Learned?

References• SomeoftheseslidesaretakenfromRapidSoftwareTestingbyJamesBach&MichaelBolton

http://www.satisfice.com/info_rst.shtml• KeriSmith

http://www.kerismith.comhttp://redefineschool.com/keri-smith-explore-art/http://www.kerismith.com/popular-posts/100-ideas/http://superforest.org/2010/03/the-rebels-manifesto-keri-smith/http://www.kerismith.com/blog/revolution/

• ExploratoryTesting3.0http://www.satisfice.com/blog/archives/1509

• “EvolvingUnderstandingofExploratoryTesting”and“Structures ofExploratoryTesting”http://www.developsense.com/resources.html

• Testcasesarenottesting:towardsacultureoftestperformance byJamesBach&AaronHodderhttp://www.testingcircus.com/testing-trapeze-2014-february-edition/

• TestingandCheckingRefinedhttp://www.satisfice.com/blog/archives/856

• Testingstoryhttp://www.developsense.com/blog/2012/02/braiding-the-stories/http://www.developsense.com/blog/2012/02/delivering-the-news-test-reporting-part-3/

• Thesciencebehindstorytellinghttp://www.melcrum.com/research/strategy-planning-tactics/science-behind-storytelling

• WhyYourBrainLovesGoodStorytellinghttp://hbr.org/2014/10/why-your-brain-loves-good-storytelling/

• WhyStoriesMatterhttp://storyrhetoric.com/blog/2014/12/17/why-stories-matter

• Misconceptionsabouttestingwww.huibschoots.nl/wordpress/?p=1118

• Collectedlinksonmywebsite:http://www.huibschoots.nl/links (takealookatthesocialsciencesectiontoo)

• Alessoninexploratorytestinghttp://trishkhoo.com/2012/10/a-lesson-in-exploratory-testing/

[email protected]@huibschoots

www.huibschoots.nl/blog

Prof. Dr. Dorgelolaan 305613 AM Eindhoven

The Netherlands

Tel: +31 40 2021803

Documents

How to be an explorer of software - v11 - EuroStar · A Check Has Three Elements 1. An observationlinked to… 2. A decision rulesuch that… 3. both observation and decision rule