IMPROVE UNIT TESTING FOR JAVA WITH AUTOMATION

IMPROVE UNIT TESTING FOR JAVA WITH AUTOMATION

BEST PRACTICES FOR JAVA DEVELOPERS

03 Introduction:ImproveUnitTesting 03 WhatIsUnitTesting?

03 ImproveUnitTestingWithAutomatedTestingTools

04 BestPracticesforDevelopers 04 UnitTestingBestPractices:HowtoGettheMost outofYourTestAutomation

10 JUnitTutorial:SettingUp,Writing,andRunningJavaUnitTests

17 MockinginJava:HowtoAutomateaJavaUnitTest, IncludingMockingandAssertions

24 HowtoCreateJUnitParameterizedTests

34 GetMoreOutofUnitTestingandReduceMaintenance EffortsWithRuntimeAnalysis

Table of Contents

Improve Unit Testing for Java With AutomationBest Practices for Java Developers

2

Introduction: Improve Unit TestingWHAT IS UNIT TESTING?Unittestingisthepracticeoftestingindividualunitsorcomponentsofanapplicationinordertovalidatethateachofthoseunitsisworkingproperly.Generally,aunitshouldbeasmallpartoftheapplication—inJavait'softenasingleclass.Notethatthereisnostrictdefinitionof"unit"here,anditisuptothedevelopertodecidethescopeoftestedcodeforeachtest.

Peoplesometimescontrasttheterm"unittesting"with"integrationtesting"or"end-to-endtesting".Thedifferenceisthat,generally,unittestingisdonetovalidatethebehaviorofanindividualtestableunit,whereasintegrationtestsarevalidatingthebehaviorofmultiplecomponentstogether,ortheapplicationasawhole.Asmentionedabove,thedefinitionforwhatconstitutesa"unit"isnotstrictlydefined,andit'suptoyoutodecidethescopeforeachtest.Ifthescopeistoobroad,itmaynotbepossibletodeterminewhyatestfailureoccurred.

Withthesechallenges,unittestingjustisn'teasy.Itrequiresalotofdevelopmentskillandeffort,andittakescommitmentandtimetomaintaintestsuites.ThisebookprovideshelpfultipsandtechniquesaswellasbestpracticestohelpyouimproveunittestingwithJUnitandParasoftJtest.

IMPROVE UNIT TESTING WITH AUTOMATED TESTING TOOLS Withautomatedtestingtools,developersareabletoreducelate-cycledefectswithbetterunittestsandautomatedstaticcodeanalysis.Theycanfocusmoretimeonnewfeaturedevelopmentforthebusiness.Developersalsobenefitfromimmediatefeedback.They'reabletorapidlyidentifywhethertheircodechangesarebreakingfunctionalityintheapplicationandaddressingitquickly.

Parasofthasfocusedonimprovingautomatedtestingforover30years.ParasoftJtestisakeyenablerofdeliveringqualityatspeedforunittesting.ThisintegratedJavasolutionenablesdevelopmentteamstobeagileanddeliverfasterwithoutsacrificingquality,makingthebusinesssuccessful.


3

https://www.parasoft.com/products/parasoft-jtest/

Best Practices for DevelopersUNIT TESTING BEST PRACTICES: HOW TO GET THE MOST OUT OF YOUR TEST AUTOMATION Unittestingisawell-knownpractice,butthere'slotsofroomforimprovement!Thissectionwillcoverthemosteffectiveunittestingbestpractices,includingapproachesformaximizingyourautomationtoolsalongtheway.Itwillalsodiscusscodecoverage,mockingdependencies,andoveralltestingstrategies.

WHYUNITTEST? Unittestingisaproventechniqueforensuringsoftwarequality,withplentyofbenefits.Hereare(morethan)afewgreatreasonstounittest:

» Unittestingvalidatesthateachpieceofyoursoftwarenotonlyworksproperlytoday,butcontinuestoworkinthefuture,providingasolidfoundationforfuturedevelopment.

» Unittestingidentifiesdefectsatearlystagesoftheproductionprocess,whichreducesthecostsoffixingtheminlaterstagesofthedevelopmentcycle.

» Unit-testedcodeisgenerallysafertorefactor,sincetestscanbere-runquicklytovalidatethatbehaviorhasnotchanged.

» Writingunittestsforcesdeveloperstoconsiderhowwelltheproductioncodeisdesignedinordertomakeitsuitableforunittesting,andmakesdeveloperslookattheircodefromadifferentperspective,encouragingthemtoconsidercornercasesanderrorconditionsintheirimplementation.

» Includingunittestsinthecodereviewprocesscanrevealhowthemodifiedornewcodeissupposedtowork.Plus,reviewerscanconfirmwhetherthetestsaregoodonesornot.

It'sunfortunatethatalltoooften,developerseitherdon'twriteunittestsatall,don'twriteenoughtests,ortheydon'tmaintainthem.Unittestscansometimesbetrickytowrite,ortime-consumingtomaintain.Sometimesthere'sadeadlinetomeet,anditfeelslikewritingtestswillmakeusmissthatdeadline.Butnotwritingenoughunittestsornotwritinggoodunittestsisariskytraptofallinto.

Sopleaseconsiderthefollowingbest-practicerecommendationsonhowtowriteclean,maintainable,automatedteststhatgiveyouallthebenefitsofunittesting,withaminimumamountoftimeandeffort.


4

https://www.parasoft.com/what-is-shift-left-testing/

UNITTESTINGBESTPRACTICES Let’slookatsomebestpracticesforbuilding,running,andmaintainingunittests,toachievethebestresults.

UnitTestsShouldBeTrustworthy Thetestmustfailifthecodeisbrokenandonlyifthecodeisbroken.Ifitdoesn't,wecannottrustwhatthetestresultsaretellingus.

UnitTestsShouldBeMaintainableandReadable Whenproductioncodechanges,testsoftenneedtobeupdated,andpossiblydebuggedaswell.So,itmustbeeasytoreadandunderstandthetest,notonlyforwhoeverwroteit,butforotherdevelopersaswell.Alwaysorganizeandnameyourtestsforclarityandreadability.

UnitTestsShouldVerifyaSingleUseCase Goodtestsvalidateonethingandonethingonly,whichmeansthattypically,theyvalidateasingleuse-case.Teststhatfollowthisbestpracticearesimplerandmoreunderstandable,andthatisgoodformaintainabilityanddebugging.Teststhatvalidatemorethanonethingcaneasilybecomecomplexandtime-consumingtomaintain.Don'tletthishappen.

Anotherbestpracticeistouseaminimalnumberofassertions.Somepeoplerecommendjustoneassertionpertest(thismaybealittletoorestrictive);theideaistofocusonvalidatingonlywhatisneededfortheuse-caseyouaretesting.

UnitTestsShouldBeIsolated Testsshouldberunnableonanymachine,inanyorder,withoutaffectingeachother.Ifpossible,testsshouldhavenodependenciesonenvironmentalfactorsorglobal/externalstate.Teststhathavethesedependenciesarehardertorunandusuallyunstable,makingthemhardertodebugandfix,andendupcostingmoretimethantheysave(seetrustworthy,above).

MartinFowler,afewyearsago,wroteabout"solitary"vs"sociable"code,todescribedependencyusageinapplicationcode,andhowtestsneedtobedesignedaccordingly.Inhisarticle,"solitary"codedoesn'tdependonotherunits(it'smoreself-contained),whereas"sociable"codedoesinteractwithothercomponents.Iftheapplicationcodeissolitary,thenthetestissimple,butforsociablecodeundertest,youcaneitherbuilda"solitary"or"sociable"test.A"sociabletest"wouldrelyonrealdependenciesinordertovalidatebehavior,whereasa"Solitarytest"isolatesthecodeundertestfromdependencies.ThisisvisuallyshowninFigure1.Youcanusemockstoisolatethecodeundertestandbuilda"solitary"testfor"sociable"code.We'lllookathowtodothatbelow.


5

Ingeneral,usingmocksfordependenciesmakesourlifeeasierastesters,becausewecangenerate"solitarytests"forsociablecode.Asociabletestforcomplexcodemayrequirealotofsetupandmayviolatetheprinciplesofbeingisolatedandrepeatable.Butsincethemockiscreatedandconfiguredinthetest,itisself-contained,andwehavemorecontroloverthebehaviorofdependencies.Plus,wecantestmorecodepaths.Forinstance,youcanreturncustomvaluesorthrowexceptionsfromthemock,inordertocoverboundaryorerrorconditions.

UnitTestsShouldBeAutomated Makesuretestsarebeingruninanautomatedprocess.Thiscanbedaily,oreveryhour,orinaContinuousIntegrationorDeliveryprocess.Thereportsneedtobeaccessibletoandreviewedbyeveryoneontheteam.Asateam,talkaboutwhichmetricsyoucareabout:codecoverage,modifiedcodecoverage,numberoftestsbeingrun,performance,etc.Alotcanbelearnedbylookingatthesenumbers,andabigshiftinthosenumbersoftenindicatesregressionsthatcanbeaddressedimmediately.

UseaGoodMixtureofUnitandIntegrationTests MichaelCohn'sbook,SucceedingwithAgile:SoftwareDevelopmentUsingScrum,addressesthisusingatestingpyramidmodel(seeillustrationinFigure2).Thisisacommonlyusedmodeltodescribetheidealdistributionoftestingresources.Theideaisthatasyougoupthepyramid,testsareusuallymorecomplextobuild,morefragile,slowertorun,andslowertodebug.Lowerlevelsaremoreisolatedandmoreintegrated,faster,andsimplertobuildanddebug.Therefore,automatedunittestsshouldmakeupthebulkofyourtests.

Figure 1: Comparison of Sociable and Solitary Tests. Source: Martin Fowler, 2014, "UnitTest"

Figure 2: Testing Pyramid Model


6

https://www.goodreads.com/book/show/6707987-succeeding-with-agile

https://www.martinfowler.com/bliki/UnitTest.html

Unittestsshouldvalidateallofthedetails,thecornercasesandboundaryconditions,etc.Component,integration,UI,andfunctionaltestsshouldbeusedmoresparingly,tovalidatethebehavioroftheAPIsorapplicationasawhole.Manualtestsshouldbeaminimalpercentageoftheoverallpyramidstructurebutarestillusefulforreleaseacceptanceandexploratorytesting.Thismodelprovidesorganizationswithahighlevelofautomationandtestcoverage,sothattheycanscaleuptheirtestingeffortsandkeepthecostsassociatedwithbuilding,running,andmaintainingtestsataminimum.

UnitTestsShouldBeExecutedWithinanOrganizedTestPractice Inordertodrivethesuccessofyourtestingatalllevels,andmaketheunit testingprocessscalableandsustainable,youwillneedsomeadditionalpractices inplace.Firstofall,thismeanswritingunittestsasyouwriteyourapplicationcode.Someorganizationswritethetestsbeforetheapplicationcode(test-driven or behavior-drivenprogramming).Theimportantthingisthattestsgohand-in-handwiththeapplicationcode.Thetestsandapplicationcodeshouldevenbereviewedtogetherinthecodereviewprocess.Reviewshelpyouunderstandthecodebeingwritten(becausetheycanseetheexpectedbehavior)andimproveteststoo!

Writingtestsalongwithcodeisn'tjustfornewbehaviororplannedchanges,it’scriticalforbugfixestoo.Everybugyoufixshouldhaveatestthatverifiesthebug isfixed.Thisensuresthatthebugstaysfixedinthefuture.

Adoptazero-tolerancepolicyforfailingtests.Ifyourteamisignoringtestresults, thenwhyhavetestsatall?Testfailuresshouldindicaterealissuessoaddress thoseissuesrightaway—beforetheywasteQA'stime,orworse,theygetintothereleasedproduct.

Thelongerittakestoaddressfailures,themoretimeandmoneythosefailureswillultimatelycostyourorganization.So,youshouldruntestsduringrefactoring,runtestsrightbeforeyoucommitcode,anddon'tletataskbeconsidered"done"untilthetestsarepassingtoo.

Finally,maintainthosetests.Asmentionedpreviously,ifyou'renotkeepingthosetestsuptodatewhentheapplicationchanges,theylosetheirvalue.Especiallyiftheyarefailing,failingtestsarecostingtimeandmoneytoinvestigateeachtimetheyfail.Refactorthetestsasneeded,whenthecodechanges.

Asyoucansee,maximizingyourreturnsonmoneyandtimeinvestedinyourunittestsrequiressomeinvestmentinapplyingbestpractices.Butintheend,therewardsareworththeinitialinvestment.


7

https://www.parasoft.com/4-tips-for-adopting-test-driven-development-tdd-in-your-organization/

https://www.parasoft.com/how-and-why-to-adopt-bdd-in-software-development/

WHATABOUTCODECOVERAGE? Ingeneral,codecoverageisameasurementofhowmuchoftheproductioncodeisexecutedwhileyourautomatedtestsarerunning.Byrunningasuiteoftestsandlookingatcodecoveragedata,youcangetageneralsenseofhowmuchofyourapplicationisbeingtested.

Therearemanykindsofcodecoverage—themostcommononesarelinecoverageandbranchcoverage.Mosttoolsfocusonlinecoverage,whichjusttellsyouifaspecificlinewascovered.Branchismoregranular,asittellsyouifeachpaththroughthecodeiscovered.

Codecoverageisanimportantmetricbutrememberthatincreasingitisameanstoanend.It’sgreatforfindinggapsintesting,butit'snottheonlythingtofocuson.Becarefulnottospendtoomuchefforttryingtoachieve100%coverage–itmaynotevenbepossibleorfeasible,andreallythequalityofyourtestsistheimportantthing.Thatbeingsaid,achievingatleast60%coverageforyourprojectsisagoodstartingpoint,and80%ormoreisagoodgoaltoset.Obviously,it'suptoyoutodecidewhatthatgoalshouldbe.

It'salsovaluableifyouhaveautomatedtoolsthatnotonlymeasurecodecoveragebutalsokeeptrackhowmuchmodifiedcodeisbeingcoveredbytests,becausethiscanprovidevisibilityintowhetherenoughtestsarebeingwrittenalongwithchangesinproductioncode.

Figure3showsanexamplecodecoveragereportfromParasoftDTP,thereporting andanalyticshub,thatyoucannavigatethroughifyouareusingParasoftJtestforyourunittesting:

Figure 3: Example Code Coverage Report


8

https://www.parasoft.com/products/parasoft-dtp/


https://www.parasoft.com/products/parasoft-jtest/java-unit-testing/

Anotherthingtokeepinmindisthat,whenwritingnewtests,becarefuloffocusingonlinecoveragealone,assinglelinesofcodecanresultinmultiplecodepaths,somakesureyourtestsvalidatethesecodepaths.Linecoverageisausefulquickindicator,butitisn'ttheonlythingtolookfor.

Themostobviouswaytoincreasecoverageissimplytoaddmoretestsformore codepaths,andmoreusecasesofthemethodundertest.Apowerfulwaytoincreasecoverageistouseparameterizedtests.ForJUnit4,therewasthebuiltinJUnit4Parameterizedfunctionalityandthird-partylibrarieslikeJunitParams.JUnit5has built-inparameterization.

Finally,ifyouaren'talreadytrackingtestcoverage,wehighlyrecommendyoustart.Thereareplentyoftoolsthatcanhelp,likeParasoftJtest.Startbymeasuringyourcurrentcoveragenumbers,thensetgoalsforwhereitshouldbe,addressimportantgapsfirst,andthenworkfromthere.

SUMMARYOFUNITTESTINGBESTPRACTICES Althoughunittestingisaproventechniqueforensuringsoftwarequality,it’sstillconsideredaburdentodevelopersandmanyteamsarestillstrugglingwithit.Inordertogetthemostoutoftestingandautomatedtestingtools,testsmustbetrustworthy,maintainable,readable,self-contained,andbeusedtoverifyasingleusecase.Automationiskeytomakingunittestingworkableandscalable.

Inaddition,softwareteamsneedtopracticegoodtestingtechniques,suchaswritingandreviewingtestsalongsideapplicationcode,maintainingtests,andensuringthatfailedtestsaretrackedandremediatedimmediately.Adoptingtheseunittestingbestpracticescanquicklyimproveyourunittestingoutcomes.


9

JUNIT TUTORIAL: SETTING UP, WRITING, AND RUNNING JAVA UNIT TESTSThistutorialwillhelpyouunderstandthebasicsandscaleyourunittestingpracticelikeapro.

BeforewegointoJUnits,let'stalkalittlebitaboutunittestingandregressiontestingandwhytheymatteringeneral.We'llgetintosomegoodexamplesaswegoon.

UNITTESTING Unittestingisaformofwhiteboxtesting,inwhichtestcasesarebasedoninternalstructure.Thetesterchoosesinputstoexploreparticularpathsanddeterminestheappropriateoutput.Thepurposeofunittestingistoexaminetheindividualcomponentsorpieceofmethods/classestoverifyfunctionality,ensuringthebehaviorisasexpected.

Theexactscopeofa“unit”isoftenlefttointerpretation,butaniceruleofthumbisforaunittocontaintheleastamountofcodethatperformsastandalonetask(e.g.asinglemethodorclass).Thereisagoodreasonthatwelimitscopewhenunittesting--ifweconstructatestthatincorporatesmultipleaspectsofaproject,wehaveshiftedfocus,fromfunctionalityofasinglemethod,tointeractionbetweendifferentportionsofthecode.Ifthetestfails,wedon'tknowwhyitfailed,andweareleftwonderingwhetherthepointoffailurewaswithinthemethodwewereinterestedin,orinthedependenciesassociatedwiththatmethod.

REGRESSIONTESTING Complementingunittesting,regressiontestingmakescertainthatthelatestfix,enhancement,orpatchdidnotbreakexistingfunctionality,bytestingthechangesyou'vemadetoyourcode.Changestocodeareinevitable,whethertheyaremodificationsofexistingcodeoraddingpackagesfornewfunctionality--yourcodewillcertainlychange.Itisinthischangethatthemostdangerlies,sowiththatinmind,regressiontestingisamust.

WHATISJUNIT? JUnitisaunittestingframeworkfortheJavaprogramminglanguagethatplaysabigroleinregressiontesting.Anopen-sourceframework,itisusedtowriteandrunrepeatableautomatedtests.

Aswithanythingelse,theJUnitframeworkhasevolvedovertime.ThemajorchangetomakenoteofistheintroductionofannotationsthatcamealongwiththereleaseofJUnit4,whichprovidedanincreaseinorganizationandreadabilityofJUnits.TherestofthisblogpostwillbewrittenfromusagesofJUnit4and5.


10

HOWTOSETUPJUNIT ThemorecommonIDEs,suchasEclipseandIntelliJ,willalreadyhaveJUnitfunctionalityinstalledbydefault.IfoneisnotusinganIDEandperhapsrelyingsolelyonabuildsystemsuchasMavenorGradle,theinstallationofJUnit4/5ishandledviathepom.xmlorbuild.gradle,respectively.ItisimportanttonotethatJUnit5wassplitintothreemodules,oneofthosebeingavintagemodulethatsupportsannotation/syntaxofJUnit4.

JUNIT4 ToaddJUnit4toyourMaven,buildthefollowingtothepom.xml. Bemindfulofversion:

<dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency>

ForGradle,addthefollowingtothebuild.gradle:

apply plugin: 'java'

dependencies { testCompile 'junit:junit:4.12' }

JUNIT5 AddingJUnit5isabitdifferent.BecauseofthemodularfashionofJUnit5,aBOMisusedtoimportallaspects.Ifonlyparticularclassesareneeded,individualgroupsorartifactscanbespecified.

ToaddJUnit5toMaven,addthefollowingtopom.xml:

<dependency> <groupId>org.junit</groupId> <artifactId>junit-bom</artifactId> <version>5.2.0</version> <scope>test</scope> </dependency>

ForGradle,addthefollowingtothebuild.gradle:

apply plugin: 'java'

dependencies { implementation 'org.junit:junit-bom:5.2.0' }


11

Althoughnottypicallyneeded,theraw.jarfile,whichallowsonetousetheJUnitframework,isalsoaccessibletomanuallyputontheclasspath.GithousesthecodeforJUnit.JUnit4hasthe.jaravailabletodownloaddirectly.ItismostcommontoincludetheJUnit5.jarfilesusingabuildmanagementsystem,likeMavenorGradle.SeetheJUnit5docsonlineforspecifics.

WRITINGUNITTESTS:THEANATOMYOFAJUNIT Nowthatwetalkedalittleaboutunittestingandsetup,let'smoveontoactualconstructionandexecutionofthesetests.TobestillustratethecreationofJUnits, wewanttostartwithsomethingbasic.Intheexampleimagebelow,wehavea simplemethod(left)thatconvertsFahrenheittoCelsius,andtheJUnit(right)associatedwithourmethod.TheJUnitsarenumberedinsectionsbelowandwe'lldiscusseachindetail.

Sections1and2 TheseareimportsfortheJUnitlibrariesneededtoleveragethetestingframework.TheimportedlibrariescanbespecifieddowntoaparticularfunctionalityofJUnitbutarecommonlyimportedwithasteriskstohaveaccesstoallfunctionality.

Section3 Thishasthestartofourtestclass,andtheimportantthingtotakenoteofhereisthenamingconventionfortheclass,whichfollowsClassNameTest.

Section4 Here,weseeourfirstJUnit-specificsyntax,anannotation.AnnotationsareextremelyimportantwhencreatingJUnits.ThisishowJUnitknowswhattodowiththeprocessingsectionofcode.Inourexamplecase,wehavean@Testannotation,whichtellsJUnitthatthepublicvoidmethodtowhichitisattachedcanberunasatestcase.

Figure 4: Example Code With Example Unit Test


12

https://github.com/junit-team/junit5/

Therearemanyotherannotations,butthemorecommonare@Before(whichrunssomestatement/preconditionbefore@Test,publicvoid),@After(whichrunssomestatementafter@Test,publicvoide.g.resettingvariables,deletingtemporaryfiles,variables,etc.),and@Ignore(whichignoressomestatementduringtestexecution--notethat@BeforeClassand@AfterClassareusedforrunningstatementsbeforeandafteralltestcases,publicstaticvoid,respectively).

Section5 Thetakeawayhereisagainnamingconvention.Notethestructure,testMethodName.

Section6 Hereweconstructanewinstanceofourclassobject.Thisisnecessarysowecancallthemethodwearetestingonsomething.Withoutthisobjectinstance,wecannottestthemethod.

Section7 Variablesassociatedwiththemethodneedtobeestablished,soherewedeclarevariablescorrespondingtoourmethod.Theseshouldbegivenmeaningfulvalues(note:ifaparameterisanobject,onecaninstantiateit,ormockit),sothatourtest hasmeaning.

Section8 Thisvariabledeclarationcouldbearguedasoptional,butit'sworthwhileforthesakeoforganizationandreadability.Weassigntheresultsofourmethodbeingtestedtothisvariable,usingitasneededforassertionsandsuch.

Section9 Theassertmethods(whicharepartoftheorg.junit.Assertclass)areusedindeterminingpass/failstatusoftestcases.Onlyfailedassertionsarerecorded.Likewithannotations,therearemanyassertoptions.InourexampleJUnitabove,weuseassertEquals(expected,actual,delta).Thistakesintheexpectedoutcome,whichtheuserdefines,theactual,whichistheresultofthemethodbeingcalled,andthedelta,whichallowsforimplementinganalloweddeviationbetweenexpectedandactualvalues.Thepurposeofanassertionisvalidation.AlthoughnotrequiredtorunyourJUnit,failingtoaddassertionsarguablydefeatsthepurposeofyourtest.Withoutassertions,youhavenoverificationandatmostasmoketest,whichgivesfeedbackonlywhenatesterrorsout.


13

HOWTORUNAJUNIT Chooseyourownadventure!Here,wewilllookatthreewaystorunJUnits:straightfromthecommandline,fromtheIDE(EclipseandIntelliJ),andusingbuildsystems(MavenandGradle).

How to Run a JUnit From the Command Line TorunaJUnitdirectlyfromthecommandline,youneedafewthings:JDKonyourpath,rawJunitjarfile,andthetestcases.Thecommandisasfollows(thisexampleisforJUnit4):

java -cp /path/to/junit.jar org.junit.runner.JUnitCore <test class name>

NOTE: it is unlikely in a professional setting that one would be running a test manually from the command line, without some build system, but the ability is there.

How to Run a JUnit From the IDE Eclipse TorunfromEclipse,fromyourPackageExplorerlocateyourJUnittest,inwhicheverfolderyouhavedesignateditto.Right-click,andmovedowntoRunAsJUnitTest.ThiswillexecuteyourtestandopenanewJUnitwindowifnotalreadyopen.

Figure 5: Eclipse Menu to Run as JUnit Test


14

IntelliJ RunningatestinIntelliJisverysimilartoEclipse.FromtheProjectwindow,locatetest,right-click,andselectRun‘testName’.LikeEclipse,aJUnitwindowwillopenwiththeresultsofthetest.

Figure 6: IntelliJ Menu to Run a Unit Test


15

How to Run a JUnit Using Build Systems

Maven Mavenmaderunningtestssimple.Ensureyouareintheproperlocationfromyourcommandline,andtheprojectpom.xmlisproperlyconfigured.ThenyoucanrunthefollowingtoexecuteyourJUnits:

Torunentiretestsuite:

mvn test

Torunsingle/specifictest(s):

mvn -Dtest=TestName test

Gradle Gradle,likeMaven,maderunningtestssimple.

Torunentiretestsuite:

gradlew test

Torunsingle/specifictest(s):

gradlew -Dtest.single=testName test

NOTE: Maven and Gradle are their own monster -- what is shown here is minimal to cover the basics.

CONTINUINGWITHUNITTESTING Ourexampleranthroughaverysimplesnippetofcode,andofcourse,thisisjustthestartofunittesting.Morecomplexmethodscalldatabasesorothermethods,buttoreassurefunctionalityweneedisolation,whichweachievethroughmocking.Mockinghelpsusisolateunitsofcodetofocusourvalidation.FrameworkscommonlyusedformockingareMockitoandPowerMock.

Thebenefitsofunittestingareclear:

» Identifydefectswithisolationandfocusedtesting.

» Assurebehaviorofindividualmethodsorpiecesofmethods.

» Helpstoensureadditionormodificationofcodedoesnotbreaktheapplication.

» Boundaryanalysismakesiteasiertocheckforinvalid/badinput.

» Testeveryaspectofthemethodtoincreasecodecoverage.

It'shelpfultodeploypowerfulunittestingtoolslikeParasoftJtestthatcanremedymuchofthepainassociatedwithJUnitsandsavedevelopersvaluabletime.


16


MOCKING IN JAVA: HOW TO AUTOMATE A JAVA UNIT TEST, INCLUDING MOCKING AND ASSERTIONSWhatismockinginJava?Youcanauto-generateaunittestwithasinglebuttonclick,includingallofthemockingandvalidations.

Goodunittestsareagreatwaytomakesurethatyourcodeworkstodayandcontinuestoworkinthefuture.Acomprehensivesuiteoftests,withgoodcode-basedandbehavior-basedcoverage,cansaveanorganizationalotoftimeandheadaches.Andyet,itisnotuncommontoseeprojectswherenotenoughtestsarewritten.Infact,somedevelopershaveevenbeenarguingagainsttheirusecompletely.

WHATMAKESAGOODUNITTEST? Therearemanyreasonswhydevelopersdon’twriteenoughunittests.Oneofthebiggestreasonsistheamountoftimetheytaketobuildandmaintain,especiallyinlarge,complexprojects.Incomplexprojects,oftenaunittestneedstoinstantiateandconfigurealotofobjects.Thistakesalotoftimetosetupandcanmakethetestascomplex(ormorecomplex)thanthecodeitistesting,itself.

Let’slookatanexampleinJava:

public LoanResponse requestLoan(LoanRequest loanRequest, LoanStrategy strategy) { LoanResponse response = new LoanResponse(); response.setApproved(true); if (loanRequest.getDownPayment().compareTo(loanRequest.getAvailableFunds()) > 0) { response.setApproved(false); response.setMessage("error.insufficient.funds.for.down.payment"); return response; } if (strategy.getQualifier(loanRequest) < strategy.getThreshold(adminManager)) { response.setApproved(false); response.setMessage(getErrorMessage()); } return response; }


17

HerewehaveamethodthatprocessesaLoanRequest,generatingaLoanResponse.NotetheLoanStrategyargument,whichisusedtoprocesstheLoanRequest.Thestrategyobjectmaybecomplex–itmayaccessadatabase,anexternalsystem,orthrowaRuntimeException.TowriteatestforrequestLoan(),weneedtobeawareofwhichtypeofLoanStrategywe'retestingwithandweprobablyneedtotestthemethodwithavarietyofLoanStrategyimplementationsandLoanRequestconfigurations.

AunittestforrequestLoan()maylooklikethis:

Asyoucansee,there’sawholesectionofourtestwhichjustcreatesobjectsandconfiguresparameters.Itwasn’tobviouslookingattherequestLoan()methodwhatobjectsandparametersneedtobesetup.Tocreatethisexample,wehadtorunthetest,addsomeconfiguration,thenre-runagainandrepeattheprocessoverandover.WespenttoomuchtimefiguringouthowtoconfiguretheAdminManagerandtheLoanStrategyinsteadoffocusingonourmethodandwhatneededtobetestedthere.AndIstillneedtoexpandourtesttocovermoreLoanRequestcases,morestrategies,andmoreparametersforAdminDao.

Additionally,byusingrealobjectstotestwith,thistestisactuallyvalidatingmorethanjustthebehaviorofrequestLoan()—we'redependingonthebehaviorofAvailableFundsLoanStrategy,AdminManagerImpl,andAdminDaoinorderforourtesttorun.Effectively,we'retestingthoseclassestoo.Insomecases,thisisdesirable,butinothercasesitisnot.Plus,ifoneofthoseotherclasseschanges,thetestmaystartfailingeventhoughthebehaviorofrequestLoan()didn’tchange.Forthistest,wewouldratherisolatetheclassundertestfromitsdependencies.

@Test public void testRequestLoan() throws Throwable { // Set up objects DownPaymentLoanProcessor processor = new DownPaymentLoanProcessor();

LoanRequest loanRequest = LoanRequestFactory.create(1000, 100, 10000); LoanStrategy strategy = new AvailableFundsLoanStrategy();

AdminManager adminManager = new AdminManagerImpl(); underTest.setAdminManager(adminManager); Map<String, String> parameters = new HashMap<>(); parameters.put("loanProcessorThreshold", "20");

AdminDao adminDao = new InMemoryAdminDao(parameters); adminManager.setAdminDao(adminDao);

// Call the method under test LoanResponse response = processor.requestLoan(loanRequest, strategy);

// Assertions and other validations }


18

MOCKINGINJAVA Onesolutionforthecomplexityproblemistomockthosecomplexobjects.Forthisexample,let'sstartbyusingamockfortheLoanStrategyparameter:

@Test public void testRequestLoan() throws Throwable { // Set up objects DownPaymentLoanProcessor processor = new DownPaymentLoanProcessor(); LoanRequest loanRequest = LoanRequestFactory.create(1000, 100, 10000); LoanStrategy strategy = Mockito.mock(LoanStrategy.class);

Mockito.when(strategy.getQualifier(any(LoanRequest.class))).thenReturn(20.0d);

Mockito.when(strategy.getThreshold(any(AdminManager.class))).thenReturn(20.0d);

// Call the method under test LoanResponse response = processor.requestLoan(loanRequest, strategy);

// Assertions and other validations }

Let’slookatwhat’shappeninghere.WecreateamockedinstanceofLoanStrategy usingMockito.mock().SinceweknowthatgetQualifier()andgetThreshold() willbecalledonthestrategy,wedefinethereturnvaluesforthosecallsusingMockito.when(…).thenReturn().Forthisthetest,wedon’tcarewhattheLoanRequestinstance’svaluesare,nordoweneedarealAdminManageranymorebecauseAdminManagerwasonlyusedbytherealLoanStrategy.

Additionally,sincewearen'tusingarealLoanStrategy,wedon’tcarewhattheconcreteimplementationsofLoanStrategymightdo.Wedon’tneedtosetuptestenvironments,dependencies,orcomplexobjects.WearefocusedontestingrequestLoan()–notLoanStrategyorAdminManager.Thecode-flowofthemethodundertestisdirectlycontrolledbythemock.

ThistestisaloteasiertowritewithMockitothanhavingtocreateacomplexLoanStrategyinstance.Buttherearestillsomechallenges.

» Forcomplexapplications,testsmayrequirelotsofmocks.

» IfyouarenewtoMockito,youneedtolearnitssyntaxandpatterns.

» Youmaynotknowwhichmethodsneedtobemocked.

» Whentheapplicationchanges,thetests(andmocks)needtobeupdatedtoo.


19

SOLVINGMOCKINGCHALLENGESWITHAJAVAUNITTESTGENERATOR WedesignedParasoftJtesttohelpaddressthechallengesaboveandmanagetherisksofJavasoftwaredevelopment.

Ontheunittestingsideofthings,ParasoftJtesthelpsyouautomatesomeofthemostdifficultpartsofcreatingandmaintainingunittestswithmocks.Fortheaboveexample,itcanauto-generateatestforrequestLoan()withasinglebutton-click,includingallofthemockingandvalidationsyouseeintheexampletest.

Below,the“Regular”actionintheParasoftJtestUnit Test AssistantToolbargeneratesthefollowingtest:

@Test public void testRequestLoan() throws Throwable { // Given DownPaymentLoanProcessor underTest = new DownPaymentLoanProcessor(); // When double availableFunds = 0.0d; // UTA: default value double downPayment = 0.0d; // UTA: default value double loanAmount = 0.0d; // UTA: default value

LoanRequest loanRequest = LoanRequestFactory.create(availableFunds, downPayment, loanAmount); LoanStrategy strategy = mockLoanStrategy(); LoanResponse result = underTest.requestLoan(loanRequest, strategy); // Then // assertNotNull(result); }

Allthemockingforthistesthappensinahelpermethod:

private static LoanStrategy mockLoanStrategy() throws Throwable { LoanStrategy strategy = mock(LoanStrategy.class); double getQualifierResult = 0.0d; // UTA: default value when(strategy.getQualifier(any(LoanRequest.class))).thenReturn(getQualifierResult);

double getThresholdResult = 0.0d; // UTA: default value

Figure 7: Test Generation With Parasoft Jtest


20


when(strategy.getThreshold(any(AdminManager.class))).thenReturn(getThresholdResult);

return strategy; }

Allthenecessarymockingissetupforus—ParasoftJtestdetectedthemethodcallstogetQualifier()andgetThreshold()andmockedthemethods.OnceweconfigurevaluesintheunittestforavailableFunds, downPayment,etc,thetestisreadytorun(wecouldalsogenerateaparameterizedtestforbettercoverage!).Notealsothattheassistantprovidessomeguidanceastowhichvaluestochangebyitscomments,“UTA:defaultvalue”,makingtestingeasier.

Thissavesalotoftimeingeneratingtests,especiallyifwedon’tknowwhatneedstobemockedorhowtousetheMockitoAPI.

HANDLINGCODECHANGES Whentheapplicationlogicchanges,thetestsoftenneedtochangealso.Ifthetestiswell-written,itshouldfailifyouupdatethecodewithoutupdatingthetest.Often,thebiggestchallengeinupdatingthetestisunderstandingwhatneedstobeupdated,andhowexactlytoperformthatupdate.Iftherearelotsofmocksandvalues,itcanbedifficulttotrackdownwhatthenecessarychangesare.

Toillustratethis,let’smakesomechangestothecodeundertest:

public LoanResponse requestLoan(LoanRequest loanRequest, LoanStrategy strategy) { ... String result = strategy.validate(loanRequest); if (result != null && !result.isEmpty()) { response.setApproved(false); response.setMessage(result); return response; } ... return response; }

WehaveaddedanewmethodtoLoanStrategy – validate(),andarenowcallingitfromrequestLoan().Thetestmayneedtobeupdatedtospecifywhatvalidate()shouldreturn.


21

Withoutchangingthegeneratedtest,let’srunitwithintheParasoftJtestUnit TestAssistant:

ParasoftJtestdetectedthatvalidate()wascalledonthemockedLoanStrategy argumentduringourtestrun.Sincethemethodhasnotbeensetupforthemock,theassistantrecommendsthatwemockthevalidate()method.The“Mockit”quick-fixactionupdatesthetestautomatically.Thisisasimpleexample–butforcomplexcodewhereitisn’teasytofindthemissingmock,therecommendationandquick-fixcansaveusalotofdebuggingtime.

Afterupdatingthetestusingthequickfix,wecanseethenewmockandsetthedesiredvalueforvalidateResult:

private static LoanStrategy mockLoanStrategy() throws Throwable {

LoanStrategy strategy = mock(LoanStrategy.class); String validateResult = ""; // UTA: default value

when(strategy.validate(any(LoanRequest.class))).thenReturn(validateResult); double getQualifierResult = 20.0d;

when(strategy.getQualifier(any(LoanRequest.class))).thenReturn(getQualifierResult); double getThresholdResult = 20.0d;

when(strategy.getThreshold(any(AdminManager.class))).thenReturn(getThresholdResult); return strategy; }

WecanconfigurevalidateResultwithanon-emptyvaluetotesttheusecasewherethemethodentersthenewblockofcode,orwecanuseanemptyvalue(ornull)tovalidatebehaviorwhenthenewblockisnotentered.

Figure 8: Example of Mocking in Parasoft Jtest


22

ANALYZINGTHETESTFLOW Theassistantalsoprovidessomeusefultoolsforanalyzingthetestflow.Forinstance,hereistheflowtreeforourtestrun:

SUMMARYOFMOCKINGINJAVA Youcanautomatemanyaspectsofunittesting.ParasoftJtesthelpsyougenerateunittestswithlesstimeandeffort,reducingthecomplexityassociatedwithmocking.Italsomakesmanyotherrecommendationstoimproveexistingtestsbasedonruntimedata,andhassupportforparameterizedtests,SpringApplicationtests,andPowerMock(formockingstaticmethodsandconstructors).

Figure 9: The Parasoft Jtest Unit Test Assistant’s Flow Tree, showing calls made during test execution.


23


HOW TO CREATE JUNIT PARAMETERIZED TESTSParameterizedtestsareagoodwaytodefineandrunmultipletestcases,wheretheonlydifferencebetweenthemisthedata.Here,welookatthreedifferentframeworkscommonlyusedwithJUnittests.

Whenwritingunittests,itiscommontoinitializemethodinputparametersandexpectedresultsinthetestmethoditself.Insomecases,usingasmallsetofinputsisenough;however,therearecasesinwhichweneedtousealargesetofvaluestoverifyallofthefunctionalityinourcode.Parameterizedtestsareagoodwaytodefineandrunmultipletestcases,wheretheonlydifferencebetweenthemisthedata.Theycanvalidatecodebehaviorforavarietyofvalues,includingbordercases.Parameterizingtestscanincreasecodecoverageandprovideconfidencethatthecodeisworkingasexpected.

ThereareanumberofgoodparameterizationframeworksforJava.Inthisarticle,wewilllookatthreedifferentframeworkscommonlyusedwithJUnittests,withacomparisonbetweenthemandexamplesofhowthetestsarestructuredforeach.Finally,wewillexplorehowtosimplifyandexpeditethecreationofparameterizedtests.

JUNITPARAMETERIZEDTESTFRAMEWORKS Let’scomparethe3mostcommonframeworks:JUnit4,JunitParams,andJUnit5.EachJUnitparameterizationframeworkhasitsownstrengthsandweaknesses.

JUnit4 Pros » ThisistheparameterizationframeworkbuiltintoJUnit4,soitrequiresnoadditional externaldependencies. » ItsupportsolderversionsofJava(JDK7andolder).

Cons » Testclassesusefieldsandconstructorstodefineparameters,whichmaketests moreverbose. » Itrequiresaseparatetestclassforeachmethodbeingtested.

JunitParams Pros » Simplifiesparametersyntaxbyallowingparameterstobepasseddirectly toatestmethod. » Allowsmultipletestmethods(eachwiththeirowndata)pertestclass. » SupportsCSVdatasources,aswellasannotation-basedvalues (nomethodrequired).

Cons » RequirestheprojecttobeconfiguredwiththeJunitParamsdependency. » Whenrunninganddebuggingtests,alltestswithintheclassmustberun—it isnotpossibletorunasingletestmethodwithinatestclass.


24

JUnit5 Pros » ThisparameterizationframeworkisbuiltintoJUnit5andimproveswhatwas includedwithJUnit4. » HasasimplifiedparametersyntaxlikeJunitParams. » Supportsmultipledata-setsourcetypes,includingCSVandannotation (nomethodrequired). » Eventhoughnoextradependenciesarerequired,morethanone.jarisneeded.

Consideration » RequiresnewerversionsofJavaandyourpreferredbuildsystem(e.g.Gradle orMavenSurefire).ChecktheJUnit5specificationsfordetails.

EXAMPLEOFAJUNITPARAMETERIZEDTEST Asanexample,supposethatwehaveamethodthatprocessesloanrequestsforabank.Wemightwriteaunittestthatspecifiestheloanrequestamount,downpaymentamount,andothervalues.Wewouldthencreateassertionsthatvalidatetheresponse—theloanmaybeapprovedorrejected,andtheresponsemayspecifythetermsoftheloan.

public LoanResponse requestLoan(float loanAmount, float downPayment, float availableFunds)

{ LoanResponse response = new LoanResponse(); response.setApproved(true);

if (availableFunds < downPayment) { response.setApproved(false); response.setMessage("error.insufficient.funds.for.down.payment");

return response; }

if (downPayment / loanAmount < 0.1) { response.setApproved(false); response.setMessage("error.insufficient.down.payment"); }

return response; }

Viewraw. Viewparameterizedtestexample.


25

https://gist.githubusercontent.com/ParasoftExamples/8b28da81e3a3717041169fbf880cb26b/raw/7af7a49d6a82b6e958772f95814ff4cd3f1e9250/parameterized_test_example_1.java

https://gist.github.com/ParasoftExamples/8b28da81e3a3717041169fbf880cb26b#file-parameterized_test_example_1-java

First,let’slookataregulartestfortheabovemethod:

@Test public void testRequestLoan() throws Throwable {

// Given| LoanProcessor underTest = new LoanProcessor();

// When LoanResponse result = underTest.requestLoan(1000f, 200f, 250f);

// Then assertNotNull(result); assertTrue(result.isApproved()); assertNull(result.getMessage()); }

Viewraw. Viewparameterizedtestexample2.

Inthisexample,wearetestingourmethodbyrequestinga$1000loan,witha $200downpaymentandindicatingthattherequestorhas$250inavailablefunds. Thetestthenvalidatesthattheloanwasapprovedanddidn’tprovideamessagein theresponse.

InordertomakesurethatourrequestLoan()methodistestedthoroughly,weneedtotestwithavarietyofdownpayments,requestedloanamounts,andavailablefunds.Forinstance,let’stesta$1millionloanrequestwithzerodownpayment,whichshouldberejected.Wecouldsimplyduplicatetheexistingtestwithdifferentvalues,butsincethetestlogicwouldbethesame,itismoreefficienttoparameterizethetestinstead.

Wewillparameterizetherequestedloanamount,downpayment,andavailablefunds,aswellastheexpectedresults:whethertheloanwasapproved,andthemessagereturnedaftervalidation.Eachsetofrequestdata,alongwithitsexpectedresults, willbecomeitsowntestcase.


26

https://gist.githubusercontent.com/ParasoftExamples/5a4210eb830dd8bcda0abcc20ffa3832/raw/5261999e865c88073aa50caa84f0da9a8c7ec387/parameterized_test_example_2.java

https://gist.github.com/ParasoftExamples/5a4210eb830dd8bcda0abcc20ffa3832#file-parameterized_test_example_2-java

ANEXAMPLEOFAPARAMETERIZEDTESTUSINGJUNIT4PARAMETERIZED Let’sstartwithaJUnit4Parameterizedexample.Tocreateaparameterized test,wefirstneedtodefinethevariablesforthetest.Wealsoneedtoinclude aconstructortoinitializethem:

@RunWith(Parameterized.class) public class LoanProcessorParameterizedTest {

float loanAmount; float downPayment; float availableFunds; boolean expectApproved; String expectedMessage;

public LoanProcessorParameterizedTest(float loanAmount, float downPayment,

float availableFunds, boolean expectApproved, String expectedMessage)

{ this.loanAmount = loanAmount; this.downPayment = downPayment; this.availableFunds = availableFunds; this.expectApproved = expectApproved; this.expectedMessage = expectedMessage; }

// ...

}


Here,weseethatthetestusesthe@RunWithannotationtospecifythatthetestwillrunwiththeJUnit4Parameterizedrunner.Thisrunnerknowstolookforamethodwhichwillprovidethevalue-setforthetest(annotatedwith@Parameters),initializethetestproperly,andrunthetestswithmultiplerows.

Notethateachparameterisdefinedasafieldinthetestclass,andtheconstructorinitializesthesevalues(youcanalsoinjectvaluesintofieldsusingthe@Parameter annotationifyoudon’twanttocreateaconstructor).Foreachrowinthevalue-set,theParameterizedrunnerwillinstantiatethetestclassandruneachtestintheclass.


27

https://gist.githubusercontent.com/ParasoftExamples/b6f712fc7328d00a175a1b244a9365c4/raw/f7103b147a4a29c715b1a62b326561d93f803f7e/parameterized_test_example_3.java

https://gist.github.com/ParasoftExamples/b6f712fc7328d00a175a1b244a9365c4#file-parameterized_test_example_3-java

Let’saddamethodwhichprovidestheparameterstotheParameterizedrunner:

@Parameters(name = "Run {index}: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}")

public static Iterable<Object[]> data() throws Throwable

{ return Arrays.asList(new Object[][] {

{ 1000.0f, 200.0f, 250.0f, true, null } }); }


Thevalue-setsarebuiltasaList of Objectarraysbythedata()method,[email protected]@Parameterssetsthenameofthetestusingplaceholders,whichwillbereplacedwhenthetestruns.Thismakesiteasiertoseevaluesintestresults,aswewillseelater.Currently,thereisonlyonerowofdata,testingacasewheretheloanshouldbeapproved.Wecanaddmorerowstoincreasecoverageofthemethodundertest.

@Parameters(name = "Run {index}: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}")

public static Iterable<Object[]> data() throws Throwable

{ return Arrays.asList(new Object[][] {

{ 1000.0f, 200.0f, 250.0f, true, null },

{ 1000.0f, 50.0f, 250.0f, false, "error.insufficient.down.payment" },

{ 1000.0f, 200.0f, 150.0f, false, "error.insufficient.funds.for.down.payment" } }); }



28

https://gist.githubusercontent.com/ParasoftExamples/85f9efae920bae0881b1f1fdfa93ed1e/raw/339b15e40eefe579ba664d42af7bf47dace620bc/parameterized_test_example_4.java

https://gist.github.com/ParasoftExamples/85f9efae920bae0881b1f1fdfa93ed1e#file-parameterized_test_example_4-java

https://gist.githubusercontent.com/ParasoftExamples/7d3ef919eb158feb2ff1b54f3bc1d3b4/raw/1ba4b0edc34771f7ab46c6a039f04d932f21a956/parameterized_test_example_5.java

https://gist.github.com/ParasoftExamples/7d3ef919eb158feb2ff1b54f3bc1d3b4#file-parameterized_test_example_5-java

Here,wehaveonetestcasewheretheloanwouldbeapproved,andtwocasesinwhichitshouldnotbeapprovedfordifferentreasons.Wemaywanttoaddrowsinwhichzeroornegativevaluesareused,aswellastestboundaryconditions.

Wearenowreadytocreatethetestmethod:

@Test

public void testRequestLoan() throws Throwable

{ // Given LoanProcessor underTest = new LoanProcessor();

// When LoanResponse result = underTest.requestLoan(loanAmount, downPayment, availableFunds);

// Then assertNotNull(result); assertEquals(expectApproved, result.isApproved()); assertEquals(expectedMessage, result.getMessage()); }

Viewraw. Parameterizedtestexample6.

Here,wereferencethefieldswheninvokingtherequestLoan()methodandvalidatingtheresults.

JUNITPARAMSEXAMPLE TheJunitParamslibrarysimplifiesparameterizedtestsyntaxbyallowingparameterstobepasseddirectlytothetestmethod.Theparametervaluesareprovidedbyaseparatemethodwhosenameisreferencedinthe@Parametersannotation.

@RunWith(JUnitParamsRunner.class) public class LoanProcessorParameterizedTest2 {

@Test

@Parameters(method = "testRequestLoan _ Parameters") public void testRequestLoan(float loanAmount, float downPayment, float availableFunds, boolean expectApproved, String expectedMessage) throws Throwable { ... }


29

https://gist.githubusercontent.com/ParasoftExamples/304155114c112e784753d467b6a5fd02/raw/3eea16f88ed907054c656a81ac6bcf9601c3441d/parameterized_test_example_6.java

https://gist.github.com/ParasoftExamples/304155114c112e784753d467b6a5fd02#file-parameterized_test_example_6-java

@SuppressWarnings("unused") private static Object[][] testRequestLoan _ Parameters() throws Throwable {

// Parameters: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}

return new Object[][] {

{ 1000.0f, 200.0f, 250.0f, true, null }, { 1000.0f, 50.0f, 250.0f, false, "error.insufficient.down.payment"}, { 1000.0f, 200.0f, 150.0f, false, "error.insufficient.funds.for.down.payment" } }; } }


JunitParamshastheadditionalbenefitthatitsupportsusingCSVfilestoprovidevaluesinadditiontoprovidingthevaluesincode.Thisallowsthetesttobedecoupledfromthedataanddatavaluestobeupdatedwithoutupdatingthecode.

JUNIT5EXAMPLE JUnit5addressessomeofthelimitationsandshortcomingsofJUnit4.LikeJunitParams,JUnit5alsosimplifiesthesyntaxofparameterizedtests.Themostimportantchangesinsyntaxare:

» Thetestmethodisannotatedwith@ParameterizedTestinsteadof@Test.

» Thetestmethodacceptsparametersdirectly,insteadofusingfields andaconstructor.

» The @RunWithannotationisnolongerneeded.

DefiningthesameexampleinJUnit5wouldlooklikethis:

public class LoanProcessorParameterizedTest {

@ParameterizedTest(name="Run {index}: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}")

@MethodSource("testRequestLoan _ Parameters") public void testRequestLoan(float loanAmount, float downPayment, float availableFunds,

boolean expectApproved, String expectedMessage) throws Throwable


30

https://gist.githubusercontent.com/ParasoftExamples/a4f09960f5fb3e4addb0f539cf2152c9/raw/505a53e894c3b7311309404f143fd9f512fedc46/parameterized_test_example_7.java

https://gist.github.com/ParasoftExamples/a4f09960f5fb3e4addb0f539cf2152c9#file-parameterized_test_example_7-java

{ ... }

static Stream<Arguments> testRequestLoan _ Parameters() throws Throwable {

return Stream.of(

Arguments.of(1000.0f, 200.0f, 250.0f, true, null),

Arguments.of(1000.0f, 50.0f, 250.0f, false, "error.insufficient.down.payment"),

Arguments.of(1000.0f, 200.0f, 150.0f, false, "error.insufficient.funds.for.down.payment") ); } }


EFFICIENTLYCREATEPARAMETERIZEDTESTS Asonemightimagine,writingtheaboveparameterizedtestcanbeabitofwork.Foreachparameterizedtestframeworkthereissomeboilerplatecodethatneedstobewrittencorrectly.Itcanbehardtorememberthecorrectstructure,andittakestimetowriteout.Tomakethismucheasier,youcanuseParasoftJtesttogenerateparameterizedtests,automatically,liketheonesdescribedabove.Todothis,simplyselectthemethodyouwanttogenerateatestfor(inEclipseorIntelliJ):

Thetestisgenerated,usingdefaultvaluesandassertions.Youcanthenconfigure thetestwithrealinputvaluesandassertionsandaddmoredatarowstothe data()method.

Figure 10: Select Parameterized Test Generation in Parasoft Jtest


31

https://gist.githubusercontent.com/ParasoftExamples/94a71f99d280bfb4f876769e8e87b686/raw/10cbae5901846c383da6e00a5df89322fbe33010/parameterized_test_example_8.java

https://gist.github.com/ParasoftExamples/94a71f99d280bfb4f876769e8e87b686#file-parameterized_test_example_8-java


RUNNINGTHEPARAMETERIZEDTEST ParasoftJtestcanrunparameterizedtestsdirectlyinbothEclipseandIntelliJ.

Notethatthenameofeachtest,asshown,includesinputvaluesfromthedatasetandexpectedresultvalues.Thiscanmakedebuggingthetestmucheasierwhenitfails,sincetheinputparametersandexpectedoutputsareshownforeachcase.

YoucanalsousetheRunAllactionfromParasoftJtest:

Itanalyzesthetestflowandprovidesdetailedinformationabouttheprevioustestrun.Thisallowsyoutoseewhathappenedinthetestwithoutneedingtorerunthetestwithbreakpointsordebuggingstatements.Forinstance,youcanseeparameterizedvaluesintheVariablesview:

Figure 11: The JUnit View in Eclipse

Figure 12: The Flow Tree View in Parasoft Jtest

Figure 13: The Variables View in Parasoft Jtest


32

SUMMARYOFCREATINGJUNITPARAMETERIZEDTESTS Eachofthethreeframeworksthatwereviewedarefinechoicesandworkwell.IfusingJUnit4,JunitParamsispreferredoverthebuilt-inJUnit4Parameterizedframework,duetothecleanerdesignofthetestclassesandtheabilitytodefinemultipletestmethodsinthesameclass.However,ifusingJUnit5,werecommendthebuilt-inJUnit5frameworksinceitaddressestheshortcomingsinJUnit4andrequiresnoextralibraries.WealsolikeusingParasoftJtest’sunittestingcapabilitiestomakecreation,execution,anddebuggingofparameterizedtestsmoreefficient.


33

GET MORE OUT OF UNIT TESTING AND REDUCE MAINTENANCE EFFORTS WITH RUNTIME ANALYSIS Torealizethebenefitsofunittesting,youcanobserveaunittestduringitsexecutionviaruntimeanalysis.Runtimeanalysisduringunittestexecutioniscriticaltoimprovingtestefficiencyandeffectiveness.

Unittestingisabestpracticetotestindividualunits/componentsofasoftware,butitcanbetediousandcostlyforJavadevelopers.It'spainstakingtotesteachunitforcorrectbehaviorwithmanualassertions,andisolateeachmethodwithmocking,andunittestingitselfisopentobugsandmisunderstoodbehavior.Toimprovethissituation,youcanusearuntimeanalysistooltodetectdataandcontrolflow,externaldependencies,andtocalculatetestcodecoverage.

Withthiscollecteddatafromtheruntimeanalysis,anenterprise-gradesolutionlikeParasoftJtestcanpromptthedeveloperabouthowtoimprovethetests,byautomaticallyrecommendingassertionsforcorrectbehavior,andmethodsformockingtoimprovetestisolation.ThisintegrationbetweenautomaticunittestgenerationandruntimeanalysisreducesthemanualinterventionrequiredforunittestingforJava.

BENEFITSOFUNITTESTING Unittestingisawell-knownpractice,butitsimplementationrequiresimprovementinmanyprojects.Unittesting,donewell,improvestheagilityofagileprocess,increasesqualityandsecurity,andbringslong-termcostsavings.

Unfortunately,regardlessofthesebenefits,developerscanstillstrugglewithunittesting,despitethedesiretoachievebetterresults.Theamountoftimeandeffortneededfortestcreationandmaintenancecanbetoomuchtojustifyincreasingtestingefforts.Often,testsuitesarefragilebecauseofpoorunit/objectisolationfromdependencies.Propermockingofdependenciesbecomesthebaneofsoftwaretesters,asdoescreatingtheassertionsneededtodeterminecorrectprogramlogic.Evenparameterizingtestsforscenarioscanbetediousandtimeconsuming.

Softwaredevelopmentteamsmustaddresstheseproblemswithtestcreation,isolation,andmaintenanceiftheywanttoachievethebenefitsofthoroughunittesting.Theanswerstartswithtestautomationtools,butsimplyautomatingtheexecutionoftestsandcollectingresultsisn’tenough.Runtimeanalysis,theprocess ofobservingarunningexecutableandrecordingkeymetrics,isaninnovativeway tohelpimproveunittestingcreation,mocking,andteststability.


34


RUNTIMEANALYSISCANIMPROVEUNITTESTING Inmostcases,developersdon’tconsiderruntimeanalysisimportantinearlystagesofunittesting.Mosttoolsareusedforcatchingerrorsthatunittestingmissed,orsimplyincalculatingcodecoverage.Butwhilethesebenefitsareimportant,runtimeanalysiscanalsoobservetheexecutionofthefirstiterationofaunittesttomakerecommendationstoimprovethetestanddetectchangestothetestruntimeenvironmentthatinterferewithteststability.

TestframeworkssuchasJUnitcreatesparsecodethatrequiresfurtherdeveloperinput.Thisworkistedious,soitcanbeautomatedtofillinmoreofthedetailsbasedontheobservedprogramlogic.Forexample,thefollowingunittestcanbeautomaticallygeneratedbyParasoftJtest:

Figure 14: Unit Test Generation in Parasoft Jtest


35

Similarly,forunittestswithparameterizedinputs,shownbelow:

Sincethecreatedtestsareexecutablefromthestart,theycanbeobservedbyruntimeanalysisforbothresultsandexecutionflow.Forexample,atestmayfailduetoaraisedexception,shownbelow.

DETECTINGDEPENDENCIESANDMOCKINGWITHRUNTIMEANALYSIS Inaddition,runtimetoolsobservetheexecutionpathintodependenciesandrecommendpotentialmockstoincreasetheisolationofthetest.Althoughvisualinspectionofanobjectundertestwillrevealitsdependencies,automatingthedetectionandmockingofthesedependenciessaveslotsoftediousanderror-pronework.

Figure 15: P arameterized Test Generation in Parasoft Jtest

Figure 16: Exception Error Shown in Parasoft Jtest Unit Test Assistant


36

Intheexamplebelow,ParasoftJtestoffersthedeveloperachoiceofwhattomockbasedontheexecutiontraceoftheunittest:

Inthiscase,addingamockablemethodpatternaddsthemethodtoalistofmockstobehandledbyamockingframeworksuchasPowerMock.

Mockingstaticconstructorsarealsopossible,asshownbelow.

IMPROVINGTESTINGFIDELITYWITHRUNTIMEANALYSIS Withfullknowledgeoftheexecutionflow,plusparametersusedinmethodcalls,runtimeanalysiscanbeusedtoprovideusefulrecommendationstothedeveloper toimprovethetestcode.Althoughassertionsareprovided,statically,whena testiscreated,theymaynotbeenabledorcorrect.Attestexecution,failedandmissingassertionstriggerwarningswhichthenleadtorecommendationstoremedytheproblem.

Figure 17: Execution Trace of Unit Test

Figure 18: Example for Mocking Constructors


37

Forexample,aftercreatinganewtest,andnorecommendedassertionshavebeenuncommented,youwouldseethefollowing:

Whateverhappens,itistheconstantfeedbackaboutcorrectiveactionforassertionsthatclosestheloopontestcreationtocompleteunittesting.Additionally,astheunitundertestischanged,thesechangescanbedealtwithinthesamemanner,continuallyreducingthemanualtestmaintenancerequired.

Orifanassertionfails,forexample,thefollowingisdisplayed:

Figure 19: Example of Assertion Recommendations

Figure 20: Example of Assertion Failure


38

IMPROVINGTESTSTABILITYWITHRUNTIMEANALYSIS Runtimeanalysiscanalsodetectchangesinthetestenvironmentduringexecutionthatimpacttheabilitytorecreateanidenticaltestenvironmentforsubsequenttests.Teststhatpassatonetimeandfaillatercanbeacauseofgreatfrustrationandlosttimeandeffort.Someexamplesofinstabilitiesthatyoucandetectwithruntimeanalysisincludethefollowing:

» Achangedsystempropertyduringatestthathasn'tchangedbacktoitsoriginalstate.Asubsequenttestmayrelyonthisproperty.

» Additionalexecutionthreadsinthebackgroundthatmayinterferewithatestrun.

» Anewfilecreationduringtestexecutionthatmightimpactsubsequentrunsiftheyrelyonthefileanditscontents.

» Modifiedstaticfieldsthatmightimpactfutureteststhatusethesesamefields.

It’scriticalthateachtestexecutionhasanidenticalstartingpoint,toensurereliableresults.Preventingtestinstabilitywithruntimedetectionremovesguessworkfromthetestdebugphase.

CONCLUSION Youcanseethatruntimeanalysisisn'tjustforcomputingcodecoverage.Runtimeanalysisduringtestexecutioniscriticaltoimprovingtestefficiencyandeffectiveness.Monitoringexecutionpathsprovidesinformationaboutdependenciestoimprovethehandlingofdependenciesandmocking.Assertionscanbemonitored,andautomaticrecommendationsimprovetestfidelity.Detectingchangesintheruntimetestenvironmentthataffectteststabilityremovesfrustrationandreducesdebuggingcyclesfortestcode.


39

TAKE THE NEXT STEPLearnhowParasoftJtestcanhelpyouimproveyourJava codequalityandteamproductivity.Contactustoday.

ABOUTPARASOFT Parasofthelpsorganizationscontinuouslydeliverqualitysoftwarewithitsmarket-proven,integratedsuiteofautomatedsoftwaretestingtools.Supportingtheembedded,enterprise,andIoTmarkets,Parasoft’stechnologiesreducethetime,effort,andcostofdeliveringsecure,reliable,andcompliantsoftwarebyintegratingeverythingfromdeepcodeanalysisandunittestingtowebUIandAPItesting,plusservicevirtualizationandcompletecodecoverage,intothedeliverypipeline.Bringingallthistogether,Parasoft’sawardwinningreportingandanalyticsdashboarddeliversacentralizedviewofqualityenablingorganizationstodeliverwithconfidenceandsucceedintoday’smoststrategicecosystemsanddevelopmentinitiatives—cybersecure,safety-critical,agile,DevOps,andcontinuoustesting.


40


https://www.parasoft.com/contact/

Documents

IMPROVE UNIT TESTING FOR JAVA WITH AUTOMATION