The Magazine for Agile Developers and Agile Testers

© iStockphoto.com/ThomasVogel

January 2011

issue 5www.agilerecord.com freedigitalversion madeinGermany ISSN2191-1320

77www.agilerecord.com

Thefirstfewthingsthatcometoourmindwhenwetalkaboutautomationarerepeatability,reducedtimeandcosts,reusability,reliability andbetter quality of softwarebeingdeveloped.Withthe increasing use of the Agile developmentmethodology (in-crementaland iterativemodelofdevelopment) inthesoftwareindustry,theroleandbenefitsofautomationinAgilehasalwaysbeenunderscrutiny.“Howcanyouautomatestuffwhenyousu-relyknowthatthefeaturebeingdevelopedissubjecttochangein thenext iteration?” isabigquestionposedbyqualityassu-ranceteams.

The objective of this article is to bring out the risks and chal-lengestheproductteamsfacewhileautomatingtestsusinganAgiledevelopmentmethodology.TheauthorsalsosuggestsomestrategiesthatproductteamscanadoptsothatautomationcanbeeffectivelydevelopedandusedinanAgilecontext.

How Agile works?AccordingtoWikipedia,“Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solu-tions evolve through collaboration between self-organizing, cross-functional teams.” Agile advocates frequent releases inshortdevelopmentcycles.InAgile,thedevelopmentofsoftwareis broken down into smaller tasks based on product features.Thecompletesoftwaredevelopmentlifecycle(requirementana-lysis,design,developmentandtesting)isfollowedforeachtask.Thetaskistargetedtobecompletedinaperiodof2-4weeks(atypicaldurationofdevelopmentcycle).Whenthetaskiscomple-tedthenextdevelopmentiterationbeginswherenewandincre-mentaltasksaretakenupbytheproductteams.Theobjectiveofadevelopmentcycleisnottodevelopsoftwareofproductionqualitybut tohaveearlyand frequent feedbackson the tasks(oronthedevelopedfeatures)fromthecustomers.Thisnotonlyhelpstheproductteamstoreducerisksbutalsohelpsthemtobeflexibletochanges.

NowthatweunderstandhowAgileworks,let’sanalyzethechal-lengesQAteamsfacewhiledevelopingautomationandtherisksassociatedwithautomationinanAgileset-up.

Challenges and risks of automationTestautomationispreferablydoneinsituationswhere:

• thetasksarehighlyredundant.

• thesoftwareistestedmanytimesandisquitestable.

• thepossibilityoffeaturegettingchangedisminimal.

• thetasksaresorepetitivethatavoidinghumanerrorisad-vantageous.

WithAgile,thesoftwaregetsdevelopedincrementallyin‘N’de-velopmentcycles.Planninganddevelopingautomatedtestsforafeaturebecomesdifficultinthisdevelopmentmode.

ListedbelowarethechallengesandrisksofautomationinAgile.

My basis for automation is no longer valid!InAgile,thereisapossibilityoffeaturesgettingremovedduringcustomerinteractionsattheendofeverydevelopmentcycle.Youmighthavespenttimeautomatingtests fora featureofawebapplicationandinthenextcycleyourealizethefeaturenomoreexistsorhasbeendrasticallychanged.Insuchsituations,eitheralotofreworkisrequiredonthescriptsortheteststhatyouhaveautomatedmaynolongerbeusablefortestingthewebapplica-tion.ThisresultsinwastageofeffortonthepartofQAteams.

Maintaining the automation is costlyInAgilemethodology,thesoftwaregetsdevelopedincrementally.Asmallchangeinthewebpageofawebapplicationoramodifi-cationinGraphicalUserInterfaceofanapplicationwouldmeanspending time inmodifying thealreadydeveloped test scripts.Itprovescostlyfortheautomationengineerstoinvesttimeandenergymaintainingthesameautomationscript(s)witheveryde-velopmentcycle.

© iStockphoto.com

/nullplus

Automation in Agileby Chetan Giridhar & Sunil Ubranimath

78 www.agilerecord.com

Test Bed MaintenanceTheessentialrequirementsfortesting(softwareorhardware)fallunderthecategoryof‘testbed’.Forachangeinthesoftwarede-signorthecode,aneedalsoarisesforchangingthetestbed.Forexample,additionofafeaturecanleadtoincreaseintestdatafiles,maintenanceofwhichgetsdifficultattimes.

Automation involves time and planningThisisoneoftherisksassociatedwithautomationinAgile.Auto-mationessentiallymeansdevelopingoftestscriptswhichcouldhelpinreducingexecutiontimesandcosts.Buttheflipsideis,writing test scripts requiresagoodamountofplanning,whichtakes time. Agile development cycles are typically 2-4 weeks,whichdoesn’tallowforsufficientplanningforautomation.

On what scale do I design my automation?Agoodautomationframeworkisonewhichhasarobustarchitec-tureand library. InAgileenvironments it’sdifficult forautoma-tionengineerstoworkonthearchitectureoractuallybuildtheframework,asthearchitectureorthedesigntheydevelopedinprevious development cyclesmay not be suitable for creatingandrunningautomatedtestsfornewfeaturesbeingdeveloped.ThereisalwaysanuncertaintyonwhatscaletheautomationcanbedesignedorbuiltinAgile.

Automation teams different from QA teamsIt iscommonpractice forproductcompanies tohaveseparateautomation and QA teams because QA engineersmay not beskilledinwritingtestscriptsorbuildingframeworks.Theworkofanautomationengineerisoftencloselyrelatedtobothdevelop-mentandQA.Theautomationengineershouldhaveanattitudeof breaking the codeandat the same time shouldbeable towrite codeasdevelopersdo.Withadifferent setof engineersdeveloping the test scripts there is the advantage of having adifferentperspectivefortestingthesamesoftware.However,thedisadvantageofthisapproachisthatautomationengineershavetoconstantlycommunicatewithdevelopmentandQA,whichme-ansmoretimespentbytheseteamswithautomationengineers.ThisisariskwhenitcomestoanAgilecontextasthereisalwaysashortageoftimeinAgile.Moreover,automationengineersaresobusyworkingonthedevelopmentofscriptsandlibrariesthattheygetinsufficienttimetogainproductknowledge,whichdef-eatsthepurposeoftesting.

Change in focus on the part of automation engineers Oftenautomationengineersgetoverinvolvedinwriting“great”code.Engineersgetbusyinwritingtestscripts,buildinglibrariesorframeworksthatmaybeefficient,butmaynothelpintestingthe softwaremore effectively. In Agile environments, the con-sequences of such situations are fatal as the amount of timeavailable forcreatinggoodcode is reduced.This isoneof therisksassociatedwithautomationinAgile.

Product testabilityOneofthenon-functionalcharacteristicsofsoftwareistestabili-ty.AccordingtoWikipedia,“Softwaretestabilityisthedegreetowhichasoftwareartifact(i.e.asoftwaresystem,softwaremodu-

le,requirements-ordesigndocument)supportstestinginagiventestcontext.”Alowerdegreeoftestabilityoftenimpliesmoretes-tingeffort.Oneofthefactorsonwhichtestabilityisdependentisautomatability(thedegreetowhichasoftwareundertestcanbesubjectedtoautomatedtesting). Ifawebapplicationrespondstouserrequestsin,say,0.2seconds,thisisanacceptableper-formancemetric,butifitisnotopeningwithinatimeperiodof1-2minutes,thewebapplicationcanbedeemednon-testable.InAgiletheproductisdevelopedinanincrementalfashionandperformance(eventhoughverycrucial)isNOTconsideredatthedesignphaseof theproduct.Suchproductsmaywork functio-nallycorrectbutiftheperformanceisnotgood,testingofsuchapplicationsbecomesdifficult. Investing time in automationofnontestableproductswouldbearisk.

Choice of Automation Tools is importantWeunderstand thatautomation in theearly stagesofproductdevelopmentinAgileisdifficult.Butasmoredevelopmentcyclesarecompleted,thefeaturesgetdevelopedandthesystembeco-messtable.Itisthusverycrucialtoselectagoodtestingtoolforautomationthatcanbeusedthroughouttheproductlifecycle.

CommunicationCommunicationplaysavitalroleinanAgilecontext.Itisimpor-tantforautomationengineerstocommunicatewellwithQA,de-velopmentteams,aswellaswiththeotherstakeholdersoftheprojecttocorrectlyunderstandtherequirements.Workingincol-laborationandcommunicatingwellusingdifferentchannels(likeScrummeetings)isthekeytosuccess.

Suggested automation strategies for AgileAutomationispossibleandshouldbedoneinAgile,butthefol-lowingpointsmustbekeptinmind.

Quick GainsConsiderautomationfor featuresthatcanprovidequickgains.Anysmalltask,iffoundrepeatableandvalidthroughoutthepro-ductlifecycle,wouldbeconsideredasagoodcandidateforauto-mation.Notonlydoesittakelesstimeforbuildingtheautomati-on,butitisalsoadvantageousasthesametasksareperformedforeverydevelopmentcycle.

Loosely coupled automation frameworkA very generic test automation frameworkwhere tests can becontributed easily for the new features in development cycleswouldbethekeytoautomationinAgile.ItbecomesdifficultforQAandautomationengineerstocontributetestsinahard-codedframework. A very loosely coupled framework that is indepen-dentoftheproductortheproductfeaturesaccommodatestestsfornewfeaturesfairlyeasily.

Building on the source code and unit testsItmaybeworthwhileforautomationengineerstohavealookatthesourcecodeandtheunittestsandreusethesameforbuil-dingtheirautomatedtests.Changesdoneinthesourcecodeforaccommodatinganewfeatureorbuildingontheexistingfeaturecanbeeasilyreflectedinautomationbyfollowingthisapproach.

80 www.agilerecord.com

Chetan Giridharhas more than 5 years of experience working as a software engineer in re-search and product organi-zations. Chetan is an avid blogger and has a blogspot which he updates with ar-ticles, blogs and publica-tions. You can reach him at cjgiridhar@gmail.com.

Sunil Ubranimathhas more than 7 years of experience in the software industry and has good ex-posure on Agile practices. He has worked in the Agile environment for more than 3 years. You can reach Sunil at uvsunil@gmail.com

> About the authorThisalsodevelops rapportbetweendevelopmentandautoma-tionengineersand inawaycodegetsreviewedbyautomationengineers.

Vertical DevelopmentDevelopmentofafeatureisdeemedtobecompleteonlywhenthe functional (alongwith robusterrorhandling)andnon-func-tional aspects of the feature are complete. Development canhappenintwodistinctways.Horizontaldevelopmentinvolvesde-velopmentoffunctionalitiesonly.Thiswouldmeanwhenallthefeaturesarefunctionallycorrect,errorhandlingandperformanceaspectswouldbeaddressedforallthefeatures.Contrarytothisapproach,verticaldevelopmentwouldmeandevelopingonefea-tureinclusiveofallfunctionalrequirements,robusterrorhand-lingandnon-functionalaspectsfortheparticularrequirement.InAgile,itisadvisabletofollowvertical(couldalsobereferredtoasdepth-wise)developmentbecausethiswouldhelpautomationengineersplanbetterforwritingthetestscriptsforfeaturequali-fication.Withverticaldevelopment,nonfunctionalrequirementswillbeconsideredearlywhichimprovesthetestabilityofthecom-ponentundertest.

ConclusionInthisarticleweintroducedtheAgiledevelopmentmethodology.We also considered the challenges ofwriting automated testsinanAgileset-upalongwiththerisksthatautomationcanposewhileworkinginAgile.Thearticlealsosuggestedsomepracticesthatautomationengineerscanfollowsothat theycandevelopanduseautomationwitheaseinAgile.■

References• www.wikipedi.org–thefreeencyclopedia

• Paperon‘AgileTestAutomation’

www.satisfice.com/articles/agileauto-paper.pdf

ByJamesBach

• Automated Testing in anAgileEnvironment–http://www.tampabayqa.com/Images/Agile.ppt

ByBobCrewsandKenArneson,Checkpoint TechnologiesInc.