MasteringSoftwareTestingwithJUnit5

ComprehensiveguidetodevelophighqualityJavaapplications

BoniGarcía

BIRMINGHAM-MUMBAI

MasteringSoftwareTestingwithJUnit5

Copyright©2017PacktPublishing

Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.

Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.

PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.

Firstpublished:October2017

Productionreference:1231017

PublishedbyPacktPublishingLtd.

LiveryPlace

35LiveryStreet

Birmingham

B32PB,UK.

ISBN978-1-78728-573-6

www.packtpub.com

Credits

Author

BoniGarcía

CopyEditor

CharlotteCarneiro

Reviewers

LuisFernándezMuñoz

AshokKumarS

ProjectCoordinator

RitikaManoj

CommissioningEditor

SmeetThakkar

Proofreader

SafisEditing

AcquisitionEditor

NigelFernandes

Indexer

AishwaryaGangawane

ContentDevelopmentEditor

MohammedYusufImaratwale

Graphics

JasonMonteiro

TechnicalEditor

RalphRosario

ProductionCoordinator

ShraddhaFalebhai

AbouttheAuthorBoniGarcíahasaPhDdegreeoninformationandcommunicationstechnologyfromTechnicalUniversityofMadrid(UPM)inSpainsince2011.Hisdoctoraldissertationwasfocusedontestautomationforwebapplications.Heisanauthorofmorethan30researchpapers,includinginternationalconferences,journals,andmagazines.

Atthetimeofwritingthisbook,hewasworkingasaresearcheratKingJuanCarlosUniversity(URJC),andalsoasassistantprofessoratDigitalArtandTechnologyUniversity(U-tad)inSpain.Hehasparticipatedindifferentresearchinitiatives.Forinstance,heismemberofKurentoproject,wherehehascreatedatestingframeworkforWebRTCapplications.HealsoparticipatesinthecoordinationofElasTest,aprojectaimedtocreateanelasticplatformforend-to-endtestingofdifferenttypesofapplications.

Boniisanactivememberonthefreeopensourcesoftwarecommunitywithabigemphasisonsoftwaretestingandwebengineering.Amongothers,heistheownerandmaindeveloperoftheWebDriverManagerandselenium-jupiterprojects(JUnit5extensionforSelenium).

Firstofall,Iwouldliketothankmygirl,Verónica,forheressentialsupportwhilewritingthisbook.Withoutyou,thismanuscriptwouldhavebeensimplyimpossible.But,mostimportantly,thankyouforsharingyourlifewithme.Iwouldalsoliketothanktomysisters,YolyandInma,foralltheirhelp,notonlywiththisbook,butinlifeingeneral.Thankyouforbeingalwaysthere.Inaddition,Iwanttodedicatethisbooktomynieces,Andrea,Silvia,andLaura,whoarethemostamazingpeopleintheuniverse(atleastthepartIknow).Ialsowouldliketorememberheremyparents,PabloandDolores.ThereisnothingIwouldlikemorethanifyouwerehere.Iknowthatapartofyouislivingwithus,inyourdaughters,grandchildren,andmyself.Imissyousomuch.Finally,Ialsowanttodedicatethisbooktothemostimportantpersonintheworld,mylittleboy,Pablo.Son,wheneveryoureadthis,IwantyoutoknowthatIlearnedwhathappinessreallymeansbylookingintoyoureyes.Iwouldliketobelievethesewordscantravelintimeandspacetogiveyouabighuganytime.Rememberthatyourparentsloveyoumorethananything.

AbouttheReviewersLuisFernándezMuñozinthelast25yearshehasbeenservingasaProfessorandResearcheratTechnicalUniversityofMadrid(UPM)andKingJuanCarlosUniversity(URJC),bothinSpain.hehasalsocollaborateasConsultorandTrainerindifferentpublicandprivatenationalandinternationalinstitutions.Hismainexpertisecomprisesallthesoftwareengineeringdisciplines,fromcodingandtesting,toanalysis,design,architecture,andprojectmanagement,usuallyaroundC++/JavaandWeb/JEEtechnologies.

AsaConsultorinprivatecompanies,hehasparticipatedindifferentprojectsasdifferentroles,frommanagementtodevelopmentofcodeandtests.Inthedifferentareashehasworked,hehighlightcloudcomputing,combatsimulationandphysicalactivityexpertsystems,amongothers.

Additional,myentrepreneurspirittookmetobecomeintheoneofco-foundersofTSCompany,anUPMstartupwith25employeesinonlyfouryears.

HehasdevelopedmyresearchendeavorintheNaturalComputingGroup,inwhichhemadehisPhDdissertationfocusedonParallelAlgorithmsfortheApplicationofEvolutionRulesinEndomembranePSystems.Afterthat,hehasdirectedsomeothersPhDdissertations.Thankstoallofthis,hehasparticipatedinmorethan60publications,inconferences,journals,andseveralresearchprojects.

Thisversatilecareerhasgivenmetheopportunityoflearnbyreadingandlisteningtorelevantpeopleinthefield,whichisoneofhismainhobbies.Deepreflections,discussionsandbuildingthementalstructureofconceptualideasrigorously,buthisway,ishismotivationchallenge.Traveling,chatting,andventuring-withoutlosinghismind-isthewayheenjoy’sthebestinlife.

AshokKumarSisanAndroiddeveloperresidinginBangalore,India.Agadgetenthusiast,hethrivesoninnovatingandexploringthelatestgizmosinthetechspace.HehasbeendevelopingsoftwaresforallGooglerelatedtechnologies,andhealsoaGooglecertifiedAndroiddeveloper.Astrongbelieverinspirituality,heheavilycontributestotheopensourcecommunityasane-spiritualritualtoimprovehise-karma.HeregularlyconductsworkshopsaboutAndroidapplicationdevelopmentinlocalengineeringcollegesandschools.Healsoorganizesmultipletecheventsathis

organizationandheisaregularattendeeofalltheconferencesthathappenonAndroidandJavarelatedtechnologiesinthesiliconvalleyofIndia(Bengaluru).HealsorunsaYouTubechannel,calledAndroidABCD,wherehediscussesallaspectsofAndroiddevelopment,includingquicktutorials.Having4yearsofprofessionalexperience.CurrentlyworkingwithDunstTechnologiesPvtLtdasfull-timeSr.MobileEngineer.HehasextensivelyworkedonAndroidNativeapplicationsrangingfromEnterpriseapplicationstocommerceapplication.Ispendmostofmytimeinexploringbrilliantarchitecturesandlibraries.Ihaveastronginterestincodequality,testing,andautomation,andallthree.heisaspeakeratAndroidconferencesthathappen’sinBengaluru,Apartfromallofit,Iamalsoaphotographer,Astoryteller.

www.PacktPub.comForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.

DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusatservice@packtpub.comformoredetails.

Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.

https://www.packtpub.com/mapt

Getthemostin-demandsoftwareskillswithMapt.MaptgivesyoufullaccesstoallPacktbooksandvideocourses,aswellasindustry-leadingtoolstohelpyouplanyourpersonaldevelopmentandadvanceyourcareer.

Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser

CustomerFeedbackThanksforpurchasingthisPacktbook.AtPackt,qualityisattheheartofoureditorialprocess.Tohelpusimprove,pleaseleaveusanhonestreviewonthisbook’sAmazonpageathttps://www.amazon.com/dp/1787285731.

Ifyou’dliketojoinourteamofregularreviewers,youcane-mailusatcustomerreviews@packtpub.com.WeawardourregularreviewerswithfreeeBooksandvideosinexchangefortheirvaluablefeedback.Helpusberelentlessinimprovingourproducts!

TableofContentsPreface

WhatthisbookcoversWhatyouneedforthisbookWhothisbookisforConventionsReaderfeedbackCustomersupport

DownloadingtheexamplecodeErrataPiracyQuestions

1. RetrospectiveOnSoftwareQualityAndJavaTestingSoftwarequality

QualityengineeringRequirementsandspecificationQualityAssurance

ISO/IEC-25000VerificationandValidation

SoftwaredefectsStaticanalysis

SoftwaretestingTestinglevels

UnittestingIntegrationtestingSystemtesting

TestingmethodsBlack-boxtestingWhite-boxtestingNon-functionaltesting

TestingtypesOthertestingapproaches

TestingframeworksfortheJVMJUnit3

StandardtestsinJUnit3TestexecutioninJUnit3

JUnit4StandardtestsinJUnit4TestexecutioninJUnit4AdvancedfeaturesofJUnit4

JUnitecosystem

Summary2. What’sNewInJUnit5

RoadtoJUnit5JUnit5motivation

ModularityJUnit4runnersJUnit4rules

JUnit5inceptionJUnit5community

JUnit5architectureTestEngineSPITestLauncherAPI

RunningtestsinJUnit5JupitertestswithMavenJupitertestswithGradleLegacytestswithMavenLegacytestswihGradleTheConsoleLauncherJupitertestsinJUnit4

IntelliJEclipse

TheextensionmodelofJUnit5TestlifecycleConditionalextensionpointsDependencyinjectionThird-partyextensions

Summary3. JUnit5StandardTests

TestlifecycleTestinstancelifecycleSkippingtestsDisplaynames

AssertionsJupiterassertions

GroupofassertionsAssertingexceptionsAssertingtimeouts

Third-partyassertionlibrariesTaggingandfilteringtests

FilteringtestswithMavenMavenregularsupport

FilteringtestswithGradleMeta-annotations

ConditionaltestexecutionAssumptions

NestedtestsRepeatedtestsMigrationfromJUnit4toJUnit5

RulesupportinJupiterSummary

4. SimplifyingTestingWithAdvancedJUnitFeaturesDependencyinjection

TestInfoParameterResolverRepetitionInfoParameterResolverTestReporterParameterResolver

DynamictestsTestinterfacesTesttemplatesParameterizedtests

@ValueSource@EnumSource@MethodSource@CsvSourceand@CsvFileSource@ArgumentsSourceArgumentconversion

ImplicitconversionExplicitconversion

CustomnamesJava9

JUnit5andJava9compatibilityBeyondJUnit5.0Summary

5. IntegrationOfJUnit5WithExternalFrameworksMockito

MockitoinanutshellJUnit5extensionforMockito

SpringSpringinanutshell

SpringmodulesIntroductiontoSpringTestTestingSpringBootapplications

JUnit5extensionforSpringSelenium

SeleniuminanutshellJUnit5extensionforSelenium

Cucumber

CucumberinanutshellJUnit5extensionforCucumber

DockerDockerinanutshellJUnit5extensionforDocker

AndroidAndroidinanutshellGradlepluginforJUnit5inAndroidprojects

RESTRESTinanutshellUsingRESTtestlibrarieswithJupiter

Summary6. FromRequirementsToTestCases

TheimportanceofrequirementsTestplanningTestdesign

EquivalencepartitioningBoundaryanalysisTestcoverage

SoftwaretestingprinciplesThepsychologyoftesting

Testanti-patternsCodesmells

Summary7. TestingManagement

SoftwaredevelopmentprocessesContinuousIntegration

JenkinsTravisCI

TestreportingMavenSurefireReportAllure

Defect-trackingsystemsStaticanalysisPuttingallpiecestogether

FeaturesandrequirementsDesignTests

Summary

PrefaceHumansarenotperfectthinkers.Atthetimeofthiswriting,softwareengineersarehumanbeings.Mostofthem.Forthatreason,writinghigh-quality,usefulsoftwareisareallydifficulttask.Aswewilldiscoverinthisbook,softwaretestingisoneofthemostimportantactivitiescarriedoutbysoftwareengineers(thatis,developers,programmers,ortesters)towarrantyalevelofqualityandconfidenceinagivenpieceofsoftware.

JUnitisthemostusedtestingframeworkfortheJavalanguage,andoneofthemostremarkableinsoftwareengineeringingeneral.Nowadays,JUnitismuchmorethanaunittestingframeworkforJava.Aswewilldiscover,itcanbeusedtoimplementdifferenttypesoftests(suchasunit,integration,end-to-end,oracceptancetests)usingdifferentstrategies(suchasblack-boxorwhite-box).

OnSeptember10,2017,theJUnitteamreleasedJUnit5.0.0.ThisbookismainlyfocusedonthisnewmajorreleaseofJUnit.Aswewilldiscover,JUnit5hassupposedacompleteredesignoftheJUnitframework,improvingimportantfeatures,suchasmodularization(JUnit5architectureiscompletelymodular),composability(theextensionmodelofJUnit5allowstointegratethird-partyframeworksintheJUnit5testlifecycleisaneasyway),andcompatibility(JUnit5supportstheexecutionofJUnit3and4legacytestsinthebrand-newJUnitPlatform).Allofit,followingamodernprogrammingmodelbasedonJava8andcompliantwithJava9.

Softwareengineeringinvolvesamultidisciplinarybodyofknowledgewithastrongimpetusforthechange.Thisbookprovidesacomprehensivereviewofmanydifferentaspectsrelatedtosoftwaretestingfrom,mainlyfollowinganopensourcepointofview(JUnitisopensourcefromitsinception).Inthisbook,inadditiontoJUnit,youlearnhowtousethird-partyframeworksandtechnologiesinourdevelopmentprocess,namely,Spring,Mockito,Selenium,Appium,Cucumber,Docker,Android,RESTservices,Hamcrest,Allure,Jenkins,TravisCI,Codecov,orSonarCube,amongothers.

WhatthisbookcoversChapter1,RetrospectiveOnSoftwareQualityAndJavaTesting,providesadetailedreviewofsoftwarequalityandtesting.Theobjectiveofthischapteristoclarifytheterminologyofthisdomaininanintelligibleway.Moreover,thischapterprovidesasummarythehistoryofJUnit(version3and4)andalsosomeJUnitenhancers(forexample,librariesthatcanbeusedtoextendJUnit).

Chapter2,What’sNewInJUnit5,firstintroducesthemotivationtocreateaversion5ofJUnit.Then,thischapterdescribesthemaincomponentsoftheJUnit5architecture,namely,Platform,Jupiter,andVintage.Next,wediscoverhowtorunJUnittests,forexample,usingdifferentbuildtoolssuchasMavenorGradle.Finally,thischapteristheextensionmodelofJUnit5,whichallowsextendingthecorefunctionalityofJUnit5byanythirdparty.

Chapter3,JUnit5StandardTests,givesadetaileddescriptionofthebasicfeaturesofthenewJUnit5programmingmodel.Thisprogrammingmodel,togetherwiththeextensionmodel,iscalledJupiter.Inthischapter,youlearnaboutthebasictestlifecycle,assertions,taggingandfilteringtests,conditionaltestexecution,nestedandrepeatedtests,andfinallyhowtomigratefromJUnit4.

Chapter4,SimplifyingTestingWithAdvancedJUnitFeatures,provideadetaileddescriptionoftheJUnit5features,suchasdependencyinjection,dynamictests,testinterfaces,testtemplates,parameterizedtests,compatibilitywithJava9,andplannedfeaturesfortheforJUnit5.1(notreleasedyetatthetimeofthiswriting).

Chapter5,IntegrationOfJUnit5WithExternalFrameworks,talksabouttheintegrationofJUnit5withexistingthird-partysoftware.Thisintegrationcanbedoneindifferentways.Typically,theJupiterextensionmodelshouldbeusedtointeractwithexternalframeworks.ThisisthecaseofMockito(apopularmockframework),Spring(aJavaframeworkaimedtocreatedenterpriseapplicationsbasedondependencyinjection),Docker(acontainerplatformtechnology),orSelenium(testframeworkforwebapplications).Inaddition,developerscanreusetoJupitertestlifecycletointeractwithothertechnologies,forexample,AndroidorRESTservices.

Chapter6,FromRequirementsToTestCases,providesasetofbestpracticesaimedtohelpasoftwaretestertowritemeaningfultestcases.Consideringthe

requirementsasthebasisofsoftwaretesting,thischapterprovidesacomprehensiveguideforcodingtestsavoidingtypicalmistakes(anti-patternsandcodesmell).

Chapter7,TestingManagement,isthefinalchapterofthebook,anditsobjectiveistoguidethereadertounderstandhowsoftwaretestingactivitiesaremanagedinalivingsoftwareproject.Tothataim,thischapterreviewsconceptssuchasContinuousIntegration(CI),buildservers(Jenkins,Travis),testreporting,ordefecttrackingsystems.Toconcludethebook,acompleteexampleapplicationtogetherwithdifferenttypesoftests(unit,integration,andend-to-end)ispresented.

WhatyouneedforthisbookInordertounderstandtheconceptspresentedinthisbookbetter,itishighlyrecommendedtoforktheGitHubrepository,whichcontainsthecodeexamplespresentedinthisbook(https://github.com/bonigarcia/mastering-junit5).Intheauthor’sopinion,touchingandplayingwiththecodeisessentialtoachieveaquickhands-onunderstandingoftheJUnit5testingframework.Asintroducedbefore,thelastchapterofthisbookprovidesacompleteapplicationexamplecoveringsomeofthemostimportanttopicsofthisbook.Thisapplication(calledRatemycat!)isalsoavailableonGitHub,intherepositoryhttps://github.com/bonigarcia/rate-my-cat.

Inordertoruntheseexample,youwillneedJDK8orhigher.YoucandownloadtheOracleJDKfromitswebsite:http://www.oracle.com/technetwork/java/javase/downloads/index.html.Inaddition,itishighlyrecommendedtouseanIntegratedDevelopmentEnvironment(IDE)toeasethedevelopmentandtestingprocess.Aswewilldiscoverinthisbook,atthetimeofthiswritingtherearetwoIDEsfullycompliantwithJUnit5,namely:

Eclipse4.7+(Oxygen):https://eclipse.org/ide/.IntelliJIDEA2016.2+:https://www.jetbrains.com/idea/.

IfyouprefertorunJUnit5fromthecommandline,twopossiblebuildtoolscanbeused:

Maven:https://maven.apache.org/Gradle:https://gradle.org/

WhothisbookisforThisbookistargetedforJavasoftwareengineers.Forthatreason,thispieceofliteraturetriestospeakthesamelanguagethanthereader(thatis,Java)andthereforeitisdrivenbyworkingcodeexamplesavailableontheaforementionedpublicopensourceGitHubrepositories.

ConventionsInthisbook,youwillfindanumberoftextstylesthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestylesandanexplanationoftheirmeaning.Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:“The@AfterAlland@BeforeAllmethodsareexecutedonlyonce”.

Ablockofcodeissetasfollows:packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.api.Test;

classStandardTest{

@Test

voidverySimpleTest(){

assertTrue(true);

}

}

Anycommand-lineinputoroutputiswrittenasfollows:mvntest

Newtermsandimportantwordsareshowninboldlikethis:“Compatibilityisthedegreetowhichaproduct,systemorcomponentcanexchangeinformationwithotherproducts”.

Warningsorimportantnotesappearinaboxlikethis.

Tipsandtricksappearlikethis.

ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook-whatyoulikedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallygetthemostoutof.

Tosendusgeneralfeedback,simplyemailfeedback@packtpub.com,andmentionthebook’stitleinthesubjectofyourmessage.

Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.

CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.

DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforthisbookfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilesemaileddirectlytoyou.

Youcandownloadthecodefilesbyfollowingthesesteps:

1. Loginorregistertoourwebsiteusingyouremailaddressandpassword.2. HoverthemousepointerontheSUPPORTtabatthetop.3. ClickonCodeDownloads&Errata.4. EnterthenameofthebookintheSearchbox.5. Selectthebookforwhichyou’relookingtodownloadthecodefiles.6. Choosefromthedrop-downmenuwhereyoupurchasedthisbookfrom.7. ClickonCodeDownload.

Oncethefileisdownloaded,pleasemakesurethatyouunziporextractthefolderusingthelatestversionof:

WinRAR/7-ZipforWindowsZipeg/iZip/UnRarXforMac7-Zip/PeaZipforLinux

ThecodebundleforthebookisalsohostedonGitHubathttps://github.com/bonigarcia/mastering-junit5.Wealsohaveothercodebundlesfromourrichcatalogofbooksandvideosavailableathttps://github.com/PacktPublishing/.Checkthemout!

ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks-maybeamistakeinthetextorthecode-wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.

Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.

PiracyPiracyofcopyrightedmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.IfyoucomeacrossanyillegalcopiesofourworksinanyformontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.

Pleasecontactusatcopyright@packtpub.comwithalinktothesuspectedpiratedmaterial.

Weappreciateyourhelpinprotectingourauthorsandourabilitytobringyouvaluablecontent.

QuestionsIfyouhaveaproblemwithanyaspectofthisbook,youcancontactusatquestions@packtpub.com,andwewilldoourbesttoaddresstheproblem.

RetrospectiveOnSoftwareQualityAndJavaTestingInordertomakeanapplepiefromscratch,youmustfirstinventtheuniverse.

-CarlSagan

Thewell-knowntestingframeworkJUnithascomealongwaysinceitsinceptionin1995.OnSeptember10,2017,animportantmilestoneintheprojectlifecycletookplace,i.e.thereleaseofJUnit5.0.0.BeforegoingdeepintothedetailsofJUnit5,itisworthreviewingthestatusquoofsoftwaretesting,inordertounderstandfromwherewehavecome,andwherewearegoing.Tothataim,thischapterprovidesahigh-levelreviewofthebackgroundofsoftwarequality,softwaretesting,andtestingforJava.Concretely,thechapteriscomposedofthreesections:

Softwarequality:Thefirstsectionreviewsthestatusquoinqualityengineering:Qualityassurance,ISO/IEC-2500,Verification&Validation(V&V),andsoftwaredefects(bugs).Softwaretesting:Thisisthemostcommonlyperformedactivitytoguaranteesoftwarequalityandreducethenumberofsoftwaredefects.Thissectionprovidesatheoreticalbackgroundofsoftwaretestinglevels(unit,integration,system,andacceptance),methods(black-box,white-box,andnon-functional),automatedandmanualsoftwaretesting.TestingframeworksfortheJavaVirtualMachine(JVM):ThissectionprovidesasummaryofthemainfeaturesofthelegacyversionsoftheJUnitframework(thatis,versions3and4).Finally,abriefdescriptionofalternativetestingframeworksandenhancerstoJUnitisdepicted.

SoftwarequalitySoftwareisthecollectionofcomputerprograms,relateddata,andassociateddocumentationdevelopedforaparticularcustomerorforageneralmarket.Itisanessentialpartofthemodernworld,andithasbecomepervasiveintelecommunications,utilities,commerce,culture,entertainment,andsoon.ThequestionWhatissoftwarequality?cangeneratedifferentanswers,dependingontheinvolvedpractitioner’sroleinasoftwaresystem.Therearetwomaingroupsofpeopleinvolvedinasoftwareproductorservice:

Consumers:arepeoplewhousesoftware.Inthisgroup,wecandifferentiatebetweencustomers(thatis,peopleresponsiblefortheacquisitionofsoftwareproductsorservices)andusers(thatis,peoplewhousethesoftwareproductsorservicesforvariouspurposes).Nevertheless,thedualrolesofcustomersandusersarequitecommon.Producers:arepeopleinvolvedwiththedevelopment,management,maintenance,marketing,andserviceofsoftwareproducts.

Thequalityexpectationsofconsumersarethatasoftwaresystemperformsusefulfunctionsasspecified.Forsoftwareproducers,thefundamentalqualityquestionisfulfillingtheircontractualobligationsbyproducingsoftwareproductsthatconformtotheServiceLevelAgreement(SLA).Thedefinitionofsoftwarequalitybythewell-knownsoftwareengineerRogerPressmancomprisesbothpointsofview:

Aneffectivesoftwareprocessappliedinamannerthatcreatesausefulproductthatprovidesmeasurablevalueforthosewhoproduceitandthosewhouseit.

QualityengineeringQualityengineering(alsoknownasqualitymanagement)isaprocessthatevaluates,assesses,andimprovesthequalityofsoftware.Therearethreemajorgroupsofactivitiesinthequalityengineeringprocess:

1. Qualityplanning:Thisstageestablishestheoverallqualitygoalbymanagingcustomer’sexpectationsundertheprojectcostandbudgetaryconstraints.Thisqualityplanalsoincludesthestrategy,thatis,theselectionofactivitiestoperformandtheappropriatequalitymeasurementstoprovidefeedbackandassessment.

2. QualityAssurance(QA):Thisguaranteesthatsoftwareproductsandprocessesintheprojectlifecyclemeettheirspecifiedrequirementsbyplanningandperformingasetofactivitiestoprovideadequateconfidencethatqualityisbeingbuiltintothesoftware.ThemainQAactivityisVerification&Validation,butthereareothers,suchassoftwarequalitymetrics,theuseofqualitystandards,configurationmanagement,documentationmanagement,oranexpert’sopinion.

3. Post-QA:Thesestageincludesactivitiesforqualityquantificationandimprovementmeasurement,analysis,feedback,andfollow-upactivities.Theaimoftheseactivitiesistoprovidequantitativeassessmentofproductqualityandidentificationofimprovementopportunities.

Thesephasesarerepresentedinthefollowingchart:

SoftwareQualityEngineeringProcess

RequirementsandspecificationRequirementsareakeytopicinthequalityengineeringdomain.Arequirementisastatementidentifyingacapability,physicalcharacteristic,orqualityfactorthatboundsaproductorprocessneedforwhichasolutionwillbepursued.Therequirementdevelopment(alsoknownasrequirementsengineering)istheprocessofproducingandanalyzingcustomer,product,andproduct-componentrequirements.Thesetofproceduresthatsupportthedevelopmentofrequirements,includingplanning,traceability,impactanalysis,changemanagement,andsoon,isknownasrequirementsmanagement.Therearetwokindsofsoftwarerequirements:

Functionalrequirementsareactionsthattheproductmustdotobeusefultoitsusers.Theyarisefromtheworkthatstakeholdersneedtodo.Almostanyactionsuchas,inspecting,publishing,ormostotheractiveverbscanbeafunctionalrequirement.Non-functionalrequirementsareproperties,orqualities,thattheproductmusthave.Forexample,theycandescribepropertiessuchasperformance,usability,orsecurity.Theyareoftencalledqualityattributes.

Anotherimportanttopicstronglylinkedwiththerequirementsisthespecification,whichisadocumentthatspecifiesinacomplete,precise,verifiablemanner,therequirements,design,behavior,orothercharacteristicsofasystem,andoftentheproceduresfordeterminingwhethertheseprovisionshavebeensatisfied.

QualityAssuranceQualityAssurance(QA)isprimarilyconcernedwithdefiningorselectingstandardsthatshouldbeappliedtothesoftwaredevelopmentprocessorsoftwareproduct.DanielGalin,theauthorofthebookSoftwareQualityAssurance(2004)definedQAas:

Systematic,plannedsetofactionsnecessarytoprovideadequateconfidencethatthesoftwaredevelopmentandmaintenanceprocessofasoftwaresystemproductconformstoestablishedspecificationaswellaswiththemanagerialrequirementsofkeepingthescheduleandoperatingwithinthebudgetaryconfines.

TheQAprocessselectstheV&Vactivities,tools,andmethodstosupporttheselectedqualitystandards.V&Visasetofactivitiescarriedoutwiththemainobjectiveofwithholdingproductsfromshipmentiftheydonotqualify.Incontrast,QAismeanttominimizethecostsofqualitybyintroducingavarietyofactivitiesthroughoutthedevelopmentandmaintenanceprocessinordertopreventthecausesoferrors,detectthem,andcorrectthemintheearlystagesofdevelopment.Asaresult,QAsubstantiallyreducestheratesofnon-qualifyingproducts.Allinall,V&VactivitiesareonlyapartofthetotalrangeofQAactivities.

ISO/IEC-25000Variousqualitystandardshavebeenproposedtoaccommodatethesedifferentqualityviewsandexpectations.ThestandardISO/IEC-9126wasoneofthemostinfluentialinthesoftwareengineeringcommunity.Nevertheless,researchersandpractitionersdetectedseveralproblemsandweaknessesinthisstandard.Forthatreason,theISO/IEC-9126internationalstandardissupersededbytheISO/IEC-25000seriesofinternationalstandardsonSoftwareproductQualityRequirementsandEvaluation(SQuaRE).Thissectionprovidesahigh-leveloverviewofthisstandard.

TheISO/IEC-2500qualityreferencemodeldistinguishesdifferentviewsonsoftwarequality:

Internalquality:Thisconcernsthepropertiesofthesystem,thatcanbemeasuredwithoutexecutingit.Externalquality:Thisconcernsthepropertiesofthesystem,thatcanbeobservedduringitsexecution.Qualityinuse:Thisconcernsthepropertiesexperiencedbyitsconsumerduringoperationandmaintenanceofthesystem.

Ideally,thedevelopment(processquality)influencestheinternalquality;then,theinternalqualitydeterminestheexternalquality.Finally,externalqualitydeterminesqualityinuse.Thischainisdepictedinthefollowingpicture:

ISO/IEC-2500ProductQualityReferenceModel

ThequalitymodelofISO/IEC-25000dividestheproductqualitymodel(thatis,theinternalandexternalattributes)intoeighttop-levelqualityfeatures:functionalsuitability,performanceefficiency,compatibility,usability,reliability,security,maintainability,andportability.Thefollowingdefinitionshavebeenextracteddirectlyfromthestandard:

Functionalsuitability:Thisrepresentsthedegreetowhichaproductorsystemprovidesfunctionsthatmeetstatedandimpliedneedswhenusedunderspecifiedconditions.Performanceefficiency:Thisrepresentstheperformancerelativetotheamountofresourcesusedunderstatedconditions.Compatibility:Thisisthedegreetowhichaproduct,systemorcomponentcanexchangeinformationwithotherproducts,systemsorcomponents,and/orperformitsrequiredfunctions,whilesharingthesamehardwareorsoftwareenvironment.Usability:Thisisthedegreetowhichaproductorsystemcanbeusedbyspecifieduserstoachievespecifiedgoalswitheffectiveness,efficiency,andsatisfactioninaspecifiedcontextofuse.Reliability:Thisisthedegreetowhichasystem,product,orcomponentperformsspecifiedfunctionsunderspecifiedconditionsforaspecifiedperiodoftime.Security:ThisisthedegreetowhichaproductorsystemprotectsinformationanddatasothatpersonsorotherproductsorsystemshavethedegreeofdataaccessappropriatetotheirtypesandlevelsofauthorizationMaintainability:Thisrepresentsthedegreeofeffectivenessandefficiencywithwhichaproductorsystemcanbemodifiedtoimproveit,correctit,oradaptittochangesinenvironmentandinrequirementsPortability:Thisisthedegreeofeffectivenessandefficiencywithwhichasystem,product,orcomponentcanbetransferredfromonehardware,software,orotheroperationalorusageenvironmenttoanother

Ontheotherhand,theattributesofqualityinusecanbecategorizedintothefollowingfivecharacteristics:

Effectiveness:Thisistheaccuracyandcompletenesswithwhichusersachievespecifiedgoals.Efficiency:Thesearetheresourcesexpendedinrelationtotheaccuracyandcompletenesswithwhichusersachievegoals.Satisfaction:Thisisthedegreetowhichuserneedsaresatisfiedwhenaproductorsystemisusedinaspecifiedcontextofuse.Freedomfromrisk:Thisisthedegreetowhichaproductorsystemmitigatesthepotentialrisktoeconomicstatus,humanlife,health,ortheenvironment.Contextcoverage:Thisisthedegreetowhichaproductorsystemcanbeusedwitheffectiveness,efficiency,freedomfromrisk,and

satisfactioninbothspecifiedcontextsofuseandincontextsbeyondthoseinitiallyexplicitlyidentified.

VerificationandValidationVerificationandValidation-alsoknownasSoftwareQualityControl-isconcernedwithevaluatingthatthesoftwarebeingdevelopedmeetsitsspecificationsanddeliversthefunctionalityexpectedbytheconsumers.Thesecheckingprocessesstartassoonasrequirementsbecomeavailable,andcontinuethroughallstagesofthedevelopmentprocess.Verificationisdifferenttovalidation,althoughtheyareoftenconfused.

ThedistinguishedprofessorofcomputerscienceBarryBoehmexpressedthedifferencebetweenthembackin1979:

Verification:arewebuildingtheproductright?Theaimofverificationistocheckthatthesoftwaremeetsitsstatedfunctionalandnon-functionalrequirements(thatis,thespecification).Validation:arewebuildingtherightproduct?Theaimofvalidationistoensurethatthesoftwaremeetsconsumer’sexpectations.Itisamoregeneralprocessthanverification,duetothefactthatspecificationsdonotalwaysreflecttherealwishesorneedsofconsumers.

V&VactivitiesincludeawidearrayofQAactivities.AlthoughsoftwaretestingplaysanextremelyimportantroleinV&V,otheractivitiesarealsonecessary.WithintheV&Vprocess,twobiggroupsoftechniquesofsystemcheckingandanalysismaybeused:

Softwaretesting:ThisisthemostcommonlyperformedactivitywithinQA.Givenapieceofcode,softwaretesting(orsimplytesting)consistsofobservingasampleofexecutions(testcases),andgivingaverdictonthem.Hence,testingisanexecution-basedQAactivity,soaprerequisiteistheexistenceoftheimplementedsoftwareunits,components,orsystemtobetested.Forthisreason,itissometimescalleddynamicanalysis.Staticanalysis:ThisisaformofV&Vthatdoesnotrequireexecutionofthesoftware.Staticanalysisworksonasourcerepresentationofthesoftware:eitheramodelofthespecificationofdesignorthesourceortheprogram.Perhaps,themostcommonlyusedareinspectionsandreviews,whereaspecification,design,orprogramischeckedbyagroupofpeople.Additionalstaticanalysistechniquesmaybeused,suchasautomatedsoftwareanalysis(thesourcecodeofaprogramischecked

forpatternsthatareknowntobepotentiallyerroneous).

Itshouldbenotedthatthereisastrongdivergenceofopinionaboutwhattypesoftestingconstitutevalidationorverification.Someauthorsbelievethatalltestingisverificationandthatvalidationisconductedwhenrequirementsarereviewedandapproved.Otherauthorsviewunitandintegrationtestingasverificationandhigher-ordertesting(forexample,systemorusertesting)asvalidation.Tosolvethisdivergence,V&Vcanbetreatedasasingletopicratherthanastwoseparatetopics.

SoftwaredefectsKeytothecorrectnessaspectofV&Vistheconceptofsoftwaredefects.Thetermdefect(alsoknownasbug)referstoagenericsoftwareproblem.TheIEEEStandard610.12proposethefollowingtaxonomyrelatedtosoftwaredefects:

Error:Ahumanactionthatproducesanincorrectresult.Errorscanbeclassifiedintotwocategories:1. Syntaxerror(programstatementthatviolatesoneormorerulesof

thelanguageinwhichitiswritten).2. Logicerror(incorrectdatafields,out-of-rangeterms,orinvalid

combinations).Fault:Themanifestationofanerrorinthesoftwaresystemisknownasafault.Forexample,anincorrectstep,process,ordatadefinition.Failure:Theinabilityofthesoftwaresystemtoperformitsrequiredfunctionsisknownas(system)failure.

Thetermbugwasfirstcoinedin1946bythesoftwarepioneerGraceHooper,whenamothtrappedinrelyofanelectromechanicalcomputercausedasystemmalfunction.Inthisdecade,thetermdebugwasalsointroduced,astheprocessofdetectingandcorrectingdefectsinasystem.

Inadditiontothislevelofgranularityfordefects,itisalsointerestingtocontemplateincidencesassymptomsassociatedwithafailureperceivedbythesoftwareconsumer.Allinall,error,faults,failures,andincidencesaredifferentaspectsofsoftwaredefects.Acausalrelationexistsbetweenthesefouraspectsofdefects.Errorsmaycausefaultstobeinjectedintothesoftware,andfaultsmaycausefailureswhenthesoftwareisexecuted.Finally,incidenceshappenwhenfailuresareexperiencedbythefinaluserorcostumer.DifferentQAactivitiescanbecarriedouttotrytominimizethenumberofdefectswithinasoftwaresystem.AsdefinedbyJeffTianinhisbookSoftwareQualityEngineering(2005),thealternativescanbegroupedintothefollowingthreegenericcategories:

Defectpreventionthrougherrorremoval:Forexample,theuseofcertainprocessesandproductstandardscanhelptominimizetheinjectioncertainkindsoffaultsintothesoftware.

Defectreductionthroughfaultdetectionandremoval:Thetraditionaltestingandstaticanalysisactivitiesareexamplesofthiscategory.Wediscoverthespecifictypesofthesemechanismsinthebodyofthischapter.Defectcontainmentthroughfailureprevention:Theseactivitiesaretypicallyoutofthescopeofthesoftwaresystem.Theobjectiveofcontainmentistominimizethedamagecausedbysoftwaresystemfailures(forexample,wallstocontainradioactivematerialincaseofreactorfailures).

SoftwaredefectchainandassociatedQAactivities

StaticanalysisStaticanalysisofasoftwarepieceisperformedwithoutexecutingthecode.Thereareseveraladvantagestosoftwareanalysisovertesting:

1. Duringtesting,errorscanhideothererrors.Thissituationdoesnothappenwithstaticanalysis,becauseitisnotconcernedwithinteractionsbetweenerrors.

2. Incompleteversionsofasystemcanbestaticallyanalyzedwithoutadditionalcost.Intesting,ifaprogramisincomplete,testharnesseshavetobedeveloped.

3. Staticanalysiscanconsiderbroaderqualityattributesofasoftwaresystem,suchascompliancewithstandards,portability,andmaintainability.

Therearedifferentmethodsthatcanbeidentifiedasstaticanalysis:

Inspection(firstproposedbyMichaelFaganin1976)areexaminationsofsoftwareartifactsbyhumaninspectorsaimedatdiscoveringandfixingfaultsinthesoftwaresystems.Allkindsofsoftwareassetsaresubjecttobeinspected,forexamplethespecification,designmodels,andsoon.Theprimaryreasonfortheexistenceofinspectionisnotwaitingfortheavailabilityofexecutableprograms(suchasintesting)beforestartingperforminginspection.Reviewistheprocessinwhichagroupofpeopleexaminethesoftwareanditsassociateddocumentation,lookingforpotentialproblemsandnon-conformancewithstandards,andotherpotentialproblemsoromissions.Nowadays,reviewsarefrequentlycarriedoutfornewcodebeforebeingmergedinasharedsourcecoderepository.Typically,thereviewisdonebyadifferentpersontothecodeauthorwithinthesameteam(peerreview).Thisprocessisquiteexpensiveintermsoftimeandeffort,butontheotherside,whencorrectlyperformed,ithelpstoensureahighinternalcodequalityreducingpotentialrisks.

Awalkthroughisaspecialformofreview.AccordingtoIEEEStandardforSoftwareReviews,awalkthroughisaformofsoftwarepeerreviewinwhichadesignerorprogrammerleadsmembersofthedevelopmentteamandotherinterestedparties

throughasoftwareproduct,andtheparticipantsaskquestionsandmakecommentsaboutpossibleerrors,violationofdevelopmentstandards,andotherproblems.

Automatedsoftwareanalysisassessesthesourcecodeusingpatternsthatareknowntobepotentiallydangerous.Thistechniqueisusuallydeliveredascommercialoropensourcetoolsandservices,commonlyknownaslintorlinter.Thesetoolscanlocatemanycommonprogrammingfaults,analyzethesourcecodebeforeitistested,andidentifypotentialproblemsinordertore-codethembeforetheymanifestthemselvesasfailures.Theintentionofthislintingprocessistodrawacodereader’sattentiontofaultsintheprogram,suchas:1. Datafaults:Thismayincludevariablesdeclaredbutneverused,

variablesassignedtwicebutneverusedbetweenassignments,andsoon.

2. Controlfaults:Thismayincludeunreachablecodeorunconditionalbranchesintoloops.

3. Input/outputfaults:Thismayincludevariablesoutputtwicewithnointerveningassignment.

4. Interfacefaults:Thismayincludeparameter-typemismatches,parameterundermismatches,non-usageoftheresultsoffunctions,uncalledfunctionsandprocedures,andsoon.

5. Storagemanagementfaults:Thismayincludeunassignedpointers,pointersarithmetic,andsoon.

Halfwaybetweenstaticanalysisanddynamictestingwefindanespecialwayofsoftwareevaluation,calledformalverification.Thiskindofassessmentprovidesmechanismstocheckthatasystemoperatesaccordingtoitsformalspecification.Tothataim,softwareistreatedasamathematicalentitywhosecorrectnesscanbeprovedusinglogicaloperations,combiningdifferenttypesofstaticanddynamicevaluation.Nowadays,formalmethodsarenotwidelyadoptedmainlyduetoscalabilityproblems.Projectsusingthesetechniquesaremostlyrelativelysmall,suchascriticalkernelsystems.Assystemsgrow,theeffortrequiredtodevelopaformalspecificationandverificationgrowexcessively.

SoftwaretestingSoftwaretestingconsistsofthedynamicevaluationofthebehaviorofaprogramonafinitesetoftestcases,suitablyselectedfromtheusuallyinfiniteexecutionsdomain,againsttheexpectedbehavior.Thekeyconceptsofthisdefinitionaredepictedasfollows:

Dynamic:TheSystemUnderTest(SUT)isexecutedwithspecificinputvaluestofindfailuresinitsbehavior.Thus,theactualSUTshouldensurethatthedesignandcodearecorrect,andalsotheenvironment,suchasthelibraries,theoperatingsystemandnetworksupport,andsoon.Finite:Exhaustivetestingisnotpossibleorpracticalformostrealprograms.Theyusuallyhavealargenumberofallowableinputstoeachoperation,plusevenmoreinvalidorunexpectedinputsandthepossiblesequencesofoperationsareusuallyinfiniteaswell.Testersmustchooseanumberoftestssothatwecanrunthetestsintheavailabletime.Selected:Sincethereisahugeorinfinitesetofpossibletestsandwecancanaffordtorunonlyasmallfractionofthem,thekeychallengeoftestingishowtoselecttheteststhataremostlikelytoexposefailuresinthesystem.Expected:Aftereachtestexecution,itmustbedecidedwhethertheobservedbehaviorofthesystemwasafailureornot.

Softwaretestingisabroadtermencompassingawidespectrumofdifferentconcepts.Thereisnouniversalclassificationforallthedifferenttestingformsavailableintheliterature.Fortheshakeofclarity,inthisbookweclassifythedifferentformoftestsusingthreeaxis,namelytestinglevel(unit,integration,system,andacceptance),testingmethods(black-box,white-box,andnon-functionaltesting),andtestingtypes(manualandautomated).

Nextsectionsprovidemoredetailsaboutalloftheseconcepts,whicharesummarizedinthefollowingdiagram:

Taxonomyofsoftwaretestinginthreecategories:levels,methods,andtypes

Forexample,aswewilldiscover,aJUnittestthatexercisesamethodinaclassaccordingtoitsfunctionalbehaviourcanbeseenasanautomatedunitblack-boxtest.Whenafinalconsumerusesasoftwareproducttovalidateifworksasexpected,accordingthetaxonomybeforewecanseethisasamanualblack-boxacceptancetest.Itshouldbenoticedthannotallpossiblecombinationofthesethreeaxesisalwaysmeaningful.Forinstance,non-functionaltests(example,performance)istypicallycarriedoutautomaticallyandatsystemlevels(itwouldbeveryunlikelytodomanuallyoratunitlevel).

TestinglevelsDependingonthesizeoftheSUTandthescenarioinwhichitisexercised,testingcanbecarriedoutatdifferentlevels.Inthisbook,weclassifythedifferenttestinglevelsinfourphases:

Unittesting:Here,individualprogramunitsaretested.Unittestingshouldfocusonthefunctionalityofobjectsormethods.Integrationtesting:Here,unitsarecombinedtocreatecompositecomponents.Integrationtestingshouldfocusontestingcomponents,interfaces.Systemtesting:Here,allofthecomponentsareintegratedandthesystemistestedasawhole.Acceptancetesting:Here,consumersdecidewhetherornotthesystemisreadytobedeployedintheconsumerenvironment.Itcanbeseenasahigh-levelfunctionaltestingperformedatsystemlevelbyfinalusersorcustomers.

Thereisnouniversalclassificationinthemanydifferentformsoftesting.Regardingtestinglevels,inthisbook,weusetheaforementionedclassificationoffourlevels.Nevertheless,otherlevelsorapproachesarepresentintheliterature(forexample,systemintegrationtestingorregressiontesting).Inthelastpartofthissection,wecanfindareviewofdifferenttestingapproaches.

Thefirstthreelevels(unit,integration,andsystem)aretypicallycarriedoutduringthedevelopmentphasesofthesoftwarelifecycle.Thesetestsaretypicallyperformedbydifferentrolesofsoftwareengineers(thatis,programmers,testers,QAteam,andsoon).Theobjectiveofthesetestsistheverificationofthesystem.Ontheotherside,thefourthlevel(acceptance)isatypeofusertesting,inwhichpotentialorrealusersareusuallyinvolved(validation).Thefollowingpictureprovidesagraphicaldescriptionoftheseconcepts:

TestinglevelsanditsrelationshipwithV&V

UnittestingUnittestingisamethodbywhichindividualpiecesofsourcecodearetestedtoverifythatthedesignandimplementationforthatunithavebeencorrectlyimplemented.Therearefourphasesexecutedinsequenceinaunittestcasearethefollowing:

Setup:Thetestcaseinitializesthetestfixture,thatisthebeforepicturerequiredfortheSUTtoexhibittheexpectedbehavior.Exercise:ThetestcaseinteractswiththeSUT,gettingsomeoutcomefromitasaresult.TheSUTusuallyqueriesanothercomponent,namedtheDepended-OnComponent(DOC).Verify:Thetestcasedetermineswhethertheexpectedoutcomehasbeenobtainedusingassertions(alsoknownaspredicates).Teardown:ThetestcasetearsdownthetestfixturetoputtheSUTbackintotheinitialstate.

ThesephasesanditsrelationshipwiththeSUTandDOCisillustratedasfollows:

Unittestgenericstructure

Unittestingisdonewiththeunitundertestinisolation,thatis,withoutinteractingitsDOCs.Tothataim,testdoublesareemployedtoreplaceanycomponentsonwhichtheSUTdepends.Thereareseveralkindsoftestdoubles:

AdummyobjectsimplysatisfiestherealobjectAPIbutitisneveractuallyused.Thetypicalusecasefordummyobjectsiswhentheyarepassedasparameterstomeetthemethodsignature,butthenthedummyobjectisnotactuallyused.Afakeobjectreplacestherealobjectwithasimplerimplementation,for

example,anin-memorydatabase.Astubobjectreplacestherealobjectprovidinghard-codedvaluesasresponses.Amockobjectalsoreplacestherealobject,butthistimewithprogrammedexpectationsasresponses.Aspyobjectisapartialmockobject,meaningthatsomeofitsmethodsareprogrammedwithexpectations,buttheothersusetherealobject’simplementation.

IntegrationtestingIntegrationtestingshouldexposedefectsintheinterfaces,andtheinteractionbetweenintegratedcomponentsormodules.Therearedifferentstrategiesforperformingintegrationtesting.Thesestrategiesdescribetheorderinwhichunitsaretobeintegrated,presumingthattheunitshavebeenseparatelytested.Examplesofcommonintegrationstrategiesarethefollowing:

Top-downintegration:Thisstrategystartswiththemainunit(module),thatis,therootoftheproceduraltree.Anylower-levelmodulethatiscalledbythemainunitshouldbesubstitutedbyatestdouble.Oncetestersareconvincedthatthemainunitlogiciscorrect,thestubsaregraduallyreplacedwiththeactualcode.Thisprocessisrepeatedfortherestofthelower-unitintheproceduraltree.Themainadvantageofthisapproachisthatdefectsaremoreeasilyfound.Bottom-upintegration:Thisstrategystartsthetestingprocesswiththemostelementaryunits.Largersubsystemsareassembledfromthetestedcomponents.Themainadvantageofthistypeisthattestdoublesarenotneeded.Adhocintegration:Thecomponentsareintegratedinthenaturalorderinwhicharefinished.Itallowsanearlytestingofthesystem.Testdoublesareusuallyrequired.Backboneintegration:Askeletonofcomponentsisbuiltandothersaregraduallyintegrated.Themaindisadvantageofthisapproachisthecreationofthebackbone,whichcanbelabor-intensive.

Anotherstrategycommonlyreferredintheliteratureisbig-bangintegration.Inthisstrategy,testerswaituntilallormostoftheunitsaredevelopedeintegrated.Asaresult,allthefailuresarefoundatthesametime,makingverydifficultandtime-consumingtocorrecttheunderlyingfaults.Ifpossible,thisstrategyshouldbeavoided.

SystemtestingSystemtestingduringdevelopmentinvolvesintegratingcomponentstocreateaversionofthesystemandthetestingtheintegratedsystem.Itverifiesthatthecomponentsarecompatible,interactscorrectly,andtransfertherightdataattherighttime,topicallyacrossitsuserinterfaces.Itobviouslyoverlapswithintegrationtesting,butthedifferencehereisthatsystemtestingshouldinvolveallthesystemcomponentstogetherwiththefinaluser(typicallyimpersonated).

Thereisanspecialtypeofsystemtestingcalledend-to-endtesting.Inthisapproach,thefinaluseristypicallyimpersonated,thatis,simulatedusingautomationtechniques.

TestingmethodsTestingmethods(orstrategies)definethewayfordesigningtestcases.Theycanberesponsibilitybased(black-box),implementationbased(whitebox),ornon-functional.Black-boxtechniquesdesigntestcasesonthebasisofthespecifiedfunctionalityoftheitemtobetested.White-boxonesrelyonsourcecodeanalysistodeveloptestcases.Hybridtechniques(grey-box)testingdesignstestcasesusingbothresponsibility-basedandimplementation-basedapproaches.

Black-boxtestingBlack-boxtesting(alsoknownasfunctionalorbehavioraltesting)isbasedonrequirementswithnoknowledgeoftheinternalprogramstructureordata.Black-boxtestingreliesonthespecificationofthesystemorthecomponentthatisbeingtestedtoderivetestcases.Thesystemisablack-boxwhosebehaviorcanonlybedeterminedbystudyingitsinputsandtherelatedoutputs.Therearealotofspecificblack-boxtestingtechniques;someofthemostwell-knownonesaredescribedasfollows:

Systematictesting:ThisreferstoacompletetestingapproachinwhichSUTisshowntoconformexhaustivelytoaspecification,uptothetestingassumptions.Itgeneratestestcasesonlyinthelimitingsensethateachdomainpointisasingletonsub-domain.Insidethiscategory,someofthemostcommonlyperformedareequivalencepartitioningandboundaryvalueanalysis,andalsologic-basedtechniques,suchascause-effectgraphing,decisiontable,orpairwisetesting.Randomtesting:Thisisliterallytheantithesisofsystematictesting-thesamplingisovertheentireinputdomain-.Fuzztestingisaformofblack-boxrandomtesting,whichrandomlymutateswell-formedinputsandteststheprogramontheresultingdata.Itdeliversrandomlysequencedand/orstructurallybaddatatoasystemtoseeiffailuresoccur.GraphicUserInterface(GUI)testing:Thisistheprocessofensuringthespecificationofsoftwarewithagraphicinterfaceinteractingwiththeuser.GUItestingisevent-driven(forexample,mousemovementsormenuselections)andprovidesafrontendtotheunderlyingapplicationcodethroughmessagesormethodcalls.GUItestingatunitlevelisusedtypicallyatthebuttonlevel.GUItestingatsystemlevelexercisestheevent-drivennatureoftheSUT.Model-basedtesting(MBT):Thisisatestingstrategyinwhichtestcasesarederivedinpartfromamodelthatdescribessome(ifnotall)aspectsoftheSUT.MBTisaformofblack-boxtestingbecausetestsaregeneratedfromamodel,whichisderivedfromtherequirementsdocumentation.Itcanbedoneatdifferentlevels(unit,integration,orsystem).Smoketesting:ThisistheprocessofensuringthecriticalfunctionalityoftheSUT.Asmoketestcaseisthefirsttoberunbytestersbefore

acceptingabuildforfurthertesting.Failureofasmoketestcasewillmeanthatthesoftwarebuildisrefused.Thenameofsmoketestingderiveselectricalsystemtesting,wherebythefirsttestwastoswitchonandseeifitsmoked.

Sanitytesting:ThisistheprocessofensuringthebasicfunctionalityoftheSUT.Similarlytosmoketesting,sanitytestsareperformedatthebeginningofthetestprocess,butitsobjectiveisdifferent.SanitytestsaresupposedtoensurethattheSUTbasicfeaturescontinueworkingasexpected(i.e.therationalityoftheSUT),beforeconductingmoreexhaustivetests.

Smokeandsanitytestingareusuallyconfusingtermsinthesoftwaretestingcommunity.Itiscommonlyacceptedthatbothkindoftestsareperformedtoavoidwastingeffortinrigoroustestingwhenthesetestsfail,beingthemaindifferencetheirtarget(criticalvs.basicfunctionality).

White-boxtestingWhite-boxtesting(alsoknownasstructuraltesting)isbasedonknowledgeoftheinternallogicofanapplication’scode.Itdeterminesiftheprogram-codestructureandlogicisfaulty.White-boxtestcasesareaccurateonlyifthetesterknowswhattheprogramissupposedtodo.

Black-boxtestingusesonlythespecificationtoidentifyusecases,whilewhite-boxtestingusestheprogramsourcecode(implementation)asthebasisoftestcaseidentification.Bothapproaches,usedinconjunction,shouldbenecessaryinordertoselectagoodsetoftestcasesfortheSUT.Someofthemostsignificantwhite-boxtechniquesareasfollows:

Codecoveragedefinesthedegreeofsourcecode,whichhasbeentested,forexample,intermsofpercentageofLOCs.Thereareseveralcriteriaforthecodecoverage:1. Statementcoverage:Thelineofcodecoveragegranularity.2. Decision(branch)coverage:Controlstructure(forexample,if-else)

coveragegranularity.3. Conditioncoverage:Booleanexpression(true-false)coverage

granularity.4. Pathscoverage:Everypossibleroutecoveragegranularity.5. Functioncoverage:Programfunctionscoveragegranularity.6. Entry/exitcoverage:Callandreturnofthecoveragegranularity.

Faultinjectionistheprocessofinjectingfaultsintosoftwaretodeterminehowwell(orbadly)someSUTbehaves.Defectscanbesaidtopropagate,andinthatcase,theireffectsarevisibleinprogramstatesbeyondthestateinwhichtheerrorexisted(afaultbecameafailure).MutationtestingvalidatestestsandtheirdatabyrunningthemagainstmanycopiesoftheSUTcontainingdifferent,single,anddeliberatelyinsertedchanges.Mutationtestinghelpstoidentifyomissionsinthecode.

Non-functionaltestingThenon-functionalaspectsofasystemcanrequireconsiderableefforttotest.Withinthisgroupitcanbefounddifferentmeansoftesting,forexample,performancetestingconductedtoevaluatethecomplianceofaSUTwithspecifiedperformancerequirements.Theserequirementsusuallyincludeconstraintsaboutthetimebehaviorandresourceusage.Performancetestingmaymeasureresponsetimewithasingleuserexercisingthesystemorwithmultipleusersexercisingthesystem.Loadtestingisfocusedonincreasingtheloadonthesystemtosomestatedorimpliedmaximumload,toverifythesystemcanhandlethedefinedsystemboundaries.Volumetestingisoftenconsideredsynonymouswithloadtesting,yetvolumetestingfocusesondata.Stresstestingexercisesbeyondnormaloperationalcapacitytotheextentthatthesystemfails,identifyingactualboundariesatwhichthesystembreaks.Theaimofstresstestingistoobservehowthesystemfailsandwherethebottlenecksare.

Securitytestingtriestoensurethefollowingconcepts:confidentiality(protectionagainstthedisclosureofinformation),integrity(ensuringthecorrectnessoftheinformation),authentication(ensuringtheidentityoftheuser),authorization(determiningthatauserisallowedtoreceiveaserviceorperformanoperation),availability(ensuringthatthesystemperformsitsfunctionalitywhenrequired),andnon-repudiation(ensuringthedenialthatanactionhappened).Authorizedattemptsforevaluatingthesecurityofsysteminfrastructureisoftenknownaspenetrationtesting.

Usabilitytestingfocusesonfindinguserinterfaceproblems,whichmaymakethesoftwaredifficulttouseormaycauseuserstomisinterprettheoutput.Accessibilitytestingisthetechniqueofmakingsurethatourproductisaccessibility(theabilitytoaccessthesystemfunctionality)compliant.

TestingtypesTherearetwomaintypestocarryingoutsoftwaretesting:

Manualtesting:ThisistheprocessofassessingtheSUTisdonebyahuman,typicallyasoftwareengineerorthefinalconsumer.Inthistypeoftesting,wecanfindtheso-calledexploratorytesting,whichisatypeofmanualtestinginwhichhumantestersevaluatethesystembyinvestigatingandfreelyevaluatingthesystemusingitspersonalperception.Automatedtesting:ThisistheprocessofassessingtheSUTinwhichthetestingprocess(testexecution,reporting,andsoon)iscarriedoutwithspecialsoftwareandinfrastructurefortesting.ElfriedeDustin,inherbookImplementingAutomatedSoftwareTesting:HowtoSaveTimeandLowerCostsWhileRaisingQuality(2009),definedAutomatedSoftwareTesting(AST)asthe:

Applicationandimplementationofsoftwaretechnologythroughouttheentiresoftwaretestinglifecyclewiththegoaltoimproveefficienciesandeffectiveness.

ThemainbenefitsofASTare:anticipatedcostsavings,shortenedtestduration,heightenedthoroughnessofthetestsperformed,improvementoftestaccuracy,improvementofresultreportingaswellasstatisticalprocessing,andsubsequentreporting.

AutomatedtestsaretypicallyexecutedinbuildserversinthecontextofContinuousIntegration(CI)processes.Moredetailsaboutthisareprovidedinchapter7,TestingManagement.

ASTismosteffectivewhenimplementedwithinaframework.Testingframeworksmaybedefinedasasetofabstractconcepts,processes,proceduresandenvironmentsinwhichautomatedtestswillbedesigned,created,andimplemented.Thisframeworkdefinitionincludesthephysicalstructuresusedfortestcreationandimplementation,aswellasthelogicalinteractionsamongthosecomponents.

Strictlyspeaking,thatdefinitionofframeworkisnotveryfarfromwhatwecanunderstandbylibrary.Inordertomakethedifferenceclearer,considerthefollowingquotefromthewell-knownsoftwareengineeringguruMartin

Folwer:

Alibraryisessentiallyasetoffunctionsthatyoucancall,thesedaysusuallyorganizedintoclasses.Eachcalldoessomeworkandreturnscontroltotheclient.Aframeworkembodiessomeabstractdesign,withmorebehaviorbuiltin.Inordertouseityouneedtoinsertyourbehaviorintovariousplacesintheframeworkeitherbysubclassingorbyplugginginyourownclasses.Theframework’scodethencallsyourcodeatthesepoints.

Visualexplanationofthedifferencebetweenlibraryandframework

Frameworksarebecomingmoreandmoreimportantinmodernsoftwaredevelopment.Theyprovideacapabilityhighlydesiredinsoftware-intensivesystems:reusability.Thisway,largeapplicationswillendupconsistingoflayersofframeworksthatcooperatewitheachother.

OthertestingapproachesAsintroducedatthebeginningofthissection,thereisnoanuniversaldefinitionforthedifferentformsoftesting.Inthissectionwereviewsomeofthemostcommonlyvarietiesoftestingavailableintheliteraturenotcoveredsofar.Forinstance,whenthetestingprocessisperformedtodeterminewhetherthesystemmeetsitsspecifications,itisknownasconformancetesting.Whenanewfeatureorfunctionalityisintroducedtoasystem(wecancallitabuild),thewayoftestingthisnewfeatureinknownasprogressiontesting.Inadditiontothat,tocheckthatthenewintroducedchangesdonotaffectthecorrectnessoftherestofthesystem,theexistingtestcasesareexercised.Thisapproachiscommonlyknownasregressiontesting.

Whenthesysteminteractswithanyexternalorthird-partysystem,anothertestingcouldbedone,knownassystemintegrationtesting.Thiskindoftestingverifiesthatthesystemisintegratedtoanyexternalsystemsproperly.

Userorcustomertestingisastageinthetestingprocessinwhichusersorcustomersprovideinputandadviceforsystemtesting.Acceptancetestingisatypeofusertesting,buttherecanalsobedifferenttypesofusertesting:

Alphatesting:Thistakesplaceatdevelopers’sites,workingtogetherwiththesoftware’sconsumers,beforeitisreleasedtoexternalusersorcustomers.Betatesting:Thistakesplaceatcustomer’ssitesandinvolvestestingbyagroupofcustomerswhousethesystemattheirownlocationsandprovidefeedback,beforethesystemisreleasedtoothercustomers.Operationaltesting:Thisisperformedbytheenduserinitsnormaloperatingenvironment.

Finally,releasetestingreferstotheprocessoftestingaparticularreleaseofasystemperformedbyaseparateteamoutsidethedevelopmentteam.Theprimarygoalofthereleasetestingprocessistoconvincethesupplierofthesystemthatisgoodenoughforuse.

TestingframeworksfortheJVMJUnitisatestingframeworkwhichallowstocreateautomatedtests.ThedevelopmentofJUnitwasstartedbyKentBeckandErichGammainlate1995.Sincethen,thepopularityoftheframeworkhasbeengrowing.Nowadays,itisbroadlyconsideredasthedefactostandardfortestingJavaapplications.

JUnitwasdesignedtobeaunit-testingframework.Nevertheless,itcanbeusedtoimplementnotjustunittests,butalsootherkindsoftests.Aswewilldiscoverinthebodyofthisbook,dependingonhowthetestlogicexercisesthepieceofsoftwareundertest,atestcaseimplementedwithJUnitcanbeconsideredasanunit,integration,system,andevenacceptancetest.Allinall,wecanthinkofJUnitasamulti-purposetestingframeworkforJava.

JUnit3SincetheearlyversionsofJUnit3,theframeworkcanworkwithJava2andhigher.JUnit3isopensourcesoftware,releasedunderCommonPublicLicense(CPL)Version1.0andhostedonSourceForge(https://sourceforge.net/projects/junit/).ThelatestversionofJUnit3wasJUnit3.8.2,releasedonMay14,2007.ThemainrequirementsintroducedbyJUnitintheworldoftestingframeworkswerethefollowing:

1. Itshouldbeeasytodefinewhichtestswillrun.2. Theframeworkshouldbeabletoruntestsindependentlyofallother

tests.3. Theframeworkshoulddetectandreporterrorstestbytest.

StandardtestsinJUnit3InJUnit3,inordertocreatetestcases,weneedtoextendtheclassjunit.framework.TestCase.ThisbaseclassincludestheframeworkcodethatJUnitneedstoautomaticallyrunthetests.Then,wesimplymakesurethatthemethodnamefollowsthetestXXX()pattern.Thisnamingconventionmakesitcleartotheframeworkthatthemethodisaunittestandthatitcanberunautomatically.

Thetestlifecycleiscontrolledinthesetup()andtearDown()methods.TheTestCasecallssetup()beforerunningeachofitstestsandthencallsteardown()wheneachtestiscomplete.Onereasontoputmorethanonetestmethodintothesametestcaseistosharethesametestfixture.

Finally,inordertoimplementtheverificationstageinthetestcase,JUnit3definesseveralassertmethodsinautilityclassnamedjunit.framework.Assert.Thefollowingtablesummarizesthemainassertionsprovidedbythisclass:

Method Description

assertTrueAssertsthataconditionistrue.Ifitisn’t,themethodthrowsanAssertionFailedErrorwiththegivenmessage(ifany).

assertFalseAssertsthataconditionisfalse.Ifitisn’t,themethodthrowsanAssertionFailedErrorwiththegivenmessage(ifany).

assertEqualsAssertsthattwoobjectsareequal.Iftheyarenot,themethodthrowsanAssertionFailedErrorwiththegivenmessage(ifany).

assertNotNullAssertsthatanobjectisnotnull.Ifitis,themethodthrowsanAssertionFailedErrorwiththemessage(ifany).

assertNullAssertsthatanobjectisnull.Ifitisn’t,themethodthrowsanAssertionFailedErrorwiththegivenmessage(ifany).

assertSame

Assertsthattwoobjectsrefertothesameobject.Iftheydonot,themethodthrowsanAssertionFailedErrorwiththegivenmessage(ifany).

assertNotSame

Assertsthattwoobjectsdonotrefertothesameobject.Iftheydo,themethodthrowsanAssertionFailedErrorwiththegivenmessage(ifany).

failFailsatest(throwingAssertionFailedError)withthegiven

message(ifany).

ThefollowingclassshowsasimpletestimplementedwithJUnit3.8.2.Aswecansee,thistestcasecontainstwotests.Beforeeachtest,themethodsetUp()willbeinvokedbytheframework,andaftertheexecutionofeachtest,themethodtearDown()willbealsoinvoked.Thisexamplehasbeencodedsothatthefirsttest,namedtestSuccess()finishescorrectly,andthesecondtestnamedtestFailure()endswithanerror(theassertionthrowsanexception):

packageio.github.bonigarcia;

importjunit.framework.TestCase;

publicclassTestSimpleextendsTestCase{

//Phase1:Setup(foreachtest)

protectedvoidsetUp()throwsException{

System.out.println("<Setup>");

}

//Test1:Thistestisgoingtosucceed

publicvoidtestSuccess(){

//Phase2:Simulationofexercise

intexpected=60;

intreal=60;

System.out.println("**Test1**");

//Phase3:Verify

assertEquals(expected+"shouldbeequalsto"

+real,expected,real);

}

//Test2:Thistestisgoingtofail

publicvoidtestFailure(){

//Phase2:Simulationofexercise

intexpected=60;

intreal=20;

System.out.println("**Test2**");

//Phase3:Verify

assertEquals(expected+"shouldbeequalsto"

+real,expected,real);

}

//Phase4:Teardown(foreachtest)

protectedvoidtearDown()throwsException{

System.out.println("</Ending>");

}

}

AllthecodeexamplesexplainedinthisbookareavailableontheGitHubrepositoryhttps://github.com/bonigarcia/mastering-junit5.

TestexecutioninJUnit3JUnit3allowstoruntestcasesbymeansofJavaapplicationscalledtestrunners.JUnit3.8.2providesthreedifferenttestrunnersoutofthebox:twographical(SwingandAWTbased)andonetextualthatcanbeusedfromthecommandline.TheJUnitframeworkprovidesseparateclassloadersforeachtest,inordertoavoidsideeffectsamongtests.

Itisacommonpracticethatbuildtools(suchasAntorMaven)andIntegratedDevelopmentEnvironments-IDE-(suchasEclipseandIntelliJ)implementitsownJUnittestrunner.

ThefollowingimageshowswhattheprevioustestlookslikewhenweusetheJUnitSwingrunner,andalsowhenweuseEclipsetorunthesametestcase.

ExecutionofanJUnit3testcaseusingthegraphicalSwingtestrunnerandalsowiththeEclipsetestrunner

WhenatestisnotsucceededinJUnit,itcanbefortworeasons:afailureoranerror.Ontheonehand,afailureiscausedbyanassertion(Assertclass)whichisnotmeet.Ontheotherhand,anerrorisanunexpectedconditionnotexpectedbythetest,suchasaconventionalexceptioninthesoftwareundertest.

AnotherimportantcontributionofJUnit3istheconceptofthetestsuite,whichisaconvenientwaytogroupteststhatarerelated.TestsuitesareimplementedbymeansoftheJUnitclassjunit.framework.TestSuite.Thisclass,inthesamewayasTestCase,implementstheframeworkinterfacejunit.framework.Test.

AdiagramcontainingthemainclassesandmethodsofJUnit3isdepictedasfollows:

CoreJUnit3classes

ThefollowingsnippetshowsanexampleoftheuseoftestsuitesinJUnit3.Inshort,wecancreateagroupoftestssimplyinstantiatingaTestSuiteobject,andthenaddsingletestcasesusingthemethodaddTestSuite():

packageio.github.bonigarcia;

importjunit.framework.Test;

importjunit.framework.TestSuite;

publicclassTestAll{

publicstaticTestsuite(){

TestSuitesuite=newTestSuite("Alltests");

suite.addTestSuite(TestSimple.class);

suite.addTestSuite(TestMinimal.class);

returnsuite;

}

}

Thistestsuitecanbelaterexecutedusingatestrunner.Forexample,wecouldusethecommand-linetestrunner(junit.textui.TestRunner)andthecommandline,asfollows:

Testsuiteexecutedusingthetextualtestrunnerandthecommandline

JUnit4JUnit4isstillanopensourceframework,thoughthelicensechangedwithrespecttoJUnit3,fromCPLtoEclipsePublicLicense(EPL)Version1.0.ThesourcecodeofJUnit4ishostedonGitHub(https://github.com/junit-team/junit4/).

OnFebruary18,2006,JUnit4.0wasreleased.Itfollowsthesamehigh-levelguidelinesthanJUnit3,thatis,easilydefinetest,theframeworkruntestsindependently,andtheframeworkdetectsandreporterrorsbythetest.

OneofthemaindifferencesofJUnit4withrespecttoJUnit3isthewaythatJUnit4allowstodefinetests.InJUnit4,Javaannotationsareusedtomarkmethodsastests.Forthisreason,JUnit4canonlybeusedforJava5orlater.AsthedocumentationofJUnit4.0statedbackin2006:

ThearchitectureofJUnit4.0isasubstantialdeparturefromthatofearlierreleases.Insteadoftaggingtestclassesbysubclassingjunit.framework.TestCaseandtaggingtestmethodsbystartingtheirnamewith‘test’,younowtagtestmethodswiththe@Testannotation.

StandardtestsinJUnit4InJUnit4,the@Testannotation(containedinpackageorg.junit)representsatest.Anypublicmethodcanbeannotatedwith@Testtomakeitatestmethod.

Inordertosetupthetestfixture,JUnit4providesthe@Beforeannotation.Thisannotationcanbeusedinanypublicmethod.Similarly,anypublicmethodannotatedwith@Aftergetsexecutedaftereachtestmethodexecution.JUnit4providestwomoreannotationstoenhancethetestlifecycle:@BeforeClassand@AfterClass.Theyareexecutedonlyoncepertestclass,beforeandafteralltests,respectively.ThefollowingpicturedepictsthelifecycleofaJUnit4testcase:

JUnit4testlifecycle

@Beforeand@Aftercanbeappliedtoanypublicvoidmethods.@AfterClassand@BeforeClasscanbeappliedtoonlypublicstaticvoidmethods.

ThefollowingtablesummarizesthemaindifferencesbetweenJUnit3andJUnit4seensofar:

Feature JUnit3 JUnit4

Testdefinition testXXXpattern @Testannotation

Runbeforethefirsttest Notsupported @BeforeClassannotation

Runafterallthetests Notsupported @AfterClassannotation

Runbeforeeachtest OverridesetUp()method @Beforeannotation

Runaftereachtest OverridetearDown()method @Afterannotation

Ignoretests Notsupported @Ignoreannotation

Theorg.junit.Assertclassprovidesstaticmethodstocarryoutassertions(predicates).Thefollowingarethemostusefulassertionmethods:

assertTrue:Iftheconditionbecomesfalse,theassertionfailsandAssertionErroristhrown.assertFalse:Iftheconditionbecomestrue,theassertionfailsandAssertionErroristhrown.assertNull:Thischeckswhethertheargumentisnull,otherwisethrowsAssertionErroriftheargumentisnotnull.assertNotNull:Thischeckswhethertheargumentisnotnull;otherwise,itthrowsAssertionErrorassertEquals:Thiscomparestwoobjectsorprimitivetypes.Moreover,iftheactualvaluedoesn’tmatchtheexpectedvalue,AssertionErroristhrown.assertSame:Thissupportsonlyobjectsandcheckstheobjectreferenceusingthe==operator.assertNotSame:ThisistheoppositeofassertSame.

ThefollowingsnippetsprovideasimpleexampleofaJUnit4testcase.Aswecansee,itistheequivalenttestcaseasseenintheprevioussection,thistimeusingtheJUnit4programmingmodel,thatis,using@Testannotationtoidentifytestsandotherannotations(@AfterAll,@After,@BeforeAll,@Before)toimplementthetestlifecycle(setupandteardowntestfixture):

packageio.github.bonigarcia;

importstaticorg.junit.Assert.assertEquals;

importorg.junit.After;

importorg.junit.AfterClass;

importorg.junit.Before;

importorg.junit.BeforeClass;

importorg.junit.Test;

publicclassTestSimple{

//Phase1.1:Setup(foralltests)

@BeforeClass

publicstaticvoidsetupAll(){

System.out.println("<SetupClass>");

}

//Phase1.2:Setup(foreachtest)

@Before

publicvoidsetupTest(){

System.out.println("<SetupTest>");

}

//Test1:Thistestisgoingtosucceed

@Test

publicvoidtestSuccess(){

//Phase2:Simulationofexercise

intexpected=60;

intreal=60;

System.out.println("**Test1**");

//Phase3:Verify

assertEquals(expected+"shouldbeequalsto"

+real,expected,real);

}

//Test2:Thistestisgoingtofail

@Test

publicvoidtestFailure(){

//Phase2:Simulationofexercise

intexpected=60;

intreal=20;

System.out.println("**Test2**");

//Phase3:Verify

assertEquals(expected+"shouldbeequalsto"

+real,expected,real);

}

//Phase4.1:Teardown(foreachtest)

@After

publicvoidteardownTest(){

System.out.println("</EndingTest>");

}

//Phase4.2:Teardown(foralltest)

@AfterClass

publicstaticvoidteardownClass(){

System.out.println("</EndingClass>");

}

}

TestexecutioninJUnit4TheconceptofthetestrunnerisalsopresentinJUnit4,butitwasslightlyimprovedwithrespecttoJUnit3.InJUnit4,atestrunnerisaJavaclassusedtomanageatest’slifecycle:instantiation,callingsetupandteardownmethods,runningthetest,handlingexceptions,sendingnotifications,andsoon.ThedefaultJUnit4testrunneriscalledBlockJUnit4ClassRunner,anditimplementstheJUnit4standardtestcaseclassmodel.

ThetestrunnertobeusedinaJUnit4testcasecanbechangedsimplyusingtheannotation@RunWith.JUnit4providesacollectionofbuilt-intestrunnersthatallowstochangethenatureofthetestclass.Inthissection,wearegoingtoreviewthemostimportantones.

Torunagroupoftests(thatis,atestsuite)JUnit4providestheSuiterunner.Inadditiontotherunner,theclassSuite.SuiteClassesallowstodefinetheindividualtestclassesbelongingtothesuite.Forexample:

packageio.github.bonigarcia;

importorg.junit.runner.RunWith;

importorg.junit.runners.Suite;

@RunWith(Suite.class)

@Suite.SuiteClasses({TestMinimal1.class,TestMinimal2.class})

publicclassMySuite{

}

Parameterizedtestsareusedtospecifydifferentinputdatathatisgoingtobeusedinthesametestlogic.Toimplementthiskindoftests,JUnit4providestheParameterizedrunner.Todefinethedataparametersinthistypeoftest,weneedtoannotateastaticmethodoftheclasswiththeannotation@Parameters.ThismethodshouldreturnaCollectionofthetwo-dimensionalarrayprovidinginputparametersforthetest.Now,therewillbetwooptionstoinjecttheinputdataintothetest:1. Usingtheconstructorclass.2. Annotatingclassattributeswiththeannotation@Parameter.

Thefollowingsnippetsshowanexampleofthelatter:packageio.github.bonigarcia;

importstaticorg.junit.Assert.assertTrue;

importjava.util.Arrays;

importjava.util.Collection;

importorg.junit.Test;

importorg.junit.runner.RunWith;

importorg.junit.runners.Parameterized;

importorg.junit.runners.Parameterized.Parameter;

importorg.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)

publicclassTestParameterized{

@Parameter(0)

publicintinput1;

@Parameter(1)

publicintinput2;

@Parameter(2)

publicintsum;

@Parameters(name="{index}:input1={0}input2={1}sum={2}?")

publicstaticCollection<Object[]>data(){

returnArrays.asList(

newObject[][]{{1,1,2},{2,2,4},{3,3,9}});

}

@Test

publicvoidtestSum(){

assertTrue(input1+"+"+input2+"isnot"+sum,

input1+input2==sum);

}

}

TheexecutionofthistestonEclipsewouldbeasfollows:

ExecutionofaParameterizedtestinEclipse

JUnittheoriesareanalternativetoJUnit’sparameterizedtests.AJUnittheoryisexpectedtobetrueforalldatasets.Thus,inJUnittheories,wehaveamethodprovidingdatapoints(thatis,theinputvaluestobeusedforthetest).Then,weneedtospecificamethodannotatedwith@Theorywhichtakesparameters.Thetheoriesinaclassgetexecutedwitheverypossiblecombinationofdatapoints:

packageio.github.bonigarcia;

importstaticorg.junit.Assert.assertTrue;

importorg.junit.experimental.theories.DataPoints;

importorg.junit.experimental.theories.Theories;

importorg.junit.experimental.theories.Theory;

importorg.junit.runner.RunWith;

@RunWith(Theories.class)

publicclassMyTheoryTest{

@DataPoints

publicstaticint[]positiveIntegers(){

returnnewint[]{1,10,100};

}

@Theory

publicvoidtestSum(inta,intb){

System.out.println("Checking"+a+"+"+b);

assertTrue(a+b>a);

assertTrue(a+b>b);

}

}

Takealookattheexecutionofthisexample,againinEclipse:

ExecutionofaJUnit4theoryinEclipse

AdvancedfeaturesofJUnit4OneofthemostsignificantinnovationsintroducedinJUnit4wastheuseofrules.Rulesallowflexibleadditionorredefinitionofthebehaviorofeachtestmethodinatestclass.Aruleshouldbeincludedinatestcasebyannotatingaclassattributewiththeannotation@Rule.ThetypeofthisattributeshouldinherittheJUnitinterfaceorg.junit.rulesTestRule.ThefollowingrulesareprovidedoutoftheboxinJUnit4:

ErrorCollector:ThisruleallowsexecutionofatesttocontinueafterthefirstproblemisfoundExpectedException:ThisruleallowstoverifythatatestthrowsaspecificexceptionExternalResource:ThisruleprovidesabaseclassforRulesthatsetupanexternalresourcebeforeatest(afile,socket,server,databaseconnection,andsoon)andguaranteetotearitdownafterwardTestName:ThisrulemakesthecurrenttestnameavailableinsidetestmethodsTemporaryFolder:ThisruleallowscreationoffilesandfoldersthatshouldbedeletedwhenthetestmethodfinishesTimeout:ThisruleappliesthesametimeouttoalltestmethodsinaclassTestWatcher:Itisabaseclassforrulesthatwillkeepalogofeachpassingandfailingtest

AnotheradvanceJUnit4featuresallowto:

Executetestsisagivenorder,usingtheannotation@FixMethodOrder.CreateassumptionsusingtheclassAssume.Thisclassoffersmanystaticmethods,suchasassumeTrue(condition),assumeFalse(condition),assumeNotNull(condition),andassumeThat(condition).Beforeexecutingatest,JUnitcheckstheassumptionspresentinthetest.Ifoneoftheassumptionsfail,theJUnitrunnerignoresthetestswithfailingassumptions.JUnitprovidesatimeoutvalue(inmilliseconds)inthe@Testannotationtomakesurethatifatestrunslongerthanthespecifiedvalue,thetestfails.CategorizetestsusingthetestrunnerCategoriesandidentifythetypesoftestannotatingthetestsmethodwiththeannotationCategory.

MeaningfulexamplesforeachofoneoftheearliermentionedfeaturescanbefoundintheGitHubrepository(https://github.com/bonigarcia/mastering-junit5).

JUnitecosystemJUnitisoneofthemostpopulartestframeworksfortheJVM,anditisconsideredoneofthemostinfluentialframeworksinsoftwareengineering.WecanfindseverallibrariesandframeworksthatprovideadditionalfunctionalityontopofJUnit.Someexamplesoftheseecosystemenhancersare:

Mockito(http://site.mockito.org/):Thisisthemockframework,whichcanbeusedinconjunctionwithJUnit.AssertJ(http://joel-costigliola.github.io/assertj/):ThisisthefluentassertionslibraryforJava.Hamcrest(http://hamcrest.org/):Thisisthelibrarywithmatchersthatcanbecombinedtocreateflexibleandreadableassertions.Cucumber(https://cucumber.io/):ThisisthetestingframeworkthatallowstorunautomatedacceptancetestswritteninaBehavior-DrivenDevelopment(BDD)style.FitNesse(http://www.fitnesse.org/):Thisisthetestingframeworkdesignedtosupportacceptancetestingbyfacilitatingdetailedreadabledescriptionsofsystemfunctions.

WhileJUnitisthelargesttestingframeworkfortheJVM,itisnottheonlyone.ThereareseveralothertestingframeworksavailablefortheJVM.Someexamplesare:

TestNG(http://testng.org/):ThisisthetestingframeworkinspiredfromJUnitandNUnit.Spock(http://spockframework.org/):ThisisthetestingandspecificationframeworkforJavaandGroovyapplications.Jtest(https://www.parasoft.com/product/jtest/):ThisistheautomatedJavatestingandstaticanalysisframeworkmadeanddistributedbythecompanyParasoft.Scalatest(http://www.scalatest.org/):ThisisthetestingframeworkforScala,Scala.js(JavaScript),andJavaapplications.

ThankstoJUnit,testinghasmovedtoacentralpartofprogramming.Consequently,theunderlyingtestingmodelimplementedinJUnit,hasbeenportedtoasetoftestingframeworksoutsidetheboundaryoftheJVM,inthe

so-calledxUnitfamily.Inthismodel,wefindtheconceptsoftestcase,runner,fixture,suite,testexecution,report,andassertion.Tonameafew,considerthefollowingframeworks.AllofthemfallintothexUnitfamily:

GoogleTest(https://github.com/google/googletest):Google’sC++testingframework.JSUnit(http://www.jsunit.net/):UnittestingframeworkforJavaScript.Mocha(https://mochajs.org/):UnittestingframeworkrunningonNode.js.NUnit(https://www.nunit.org/):UnittestingframeworkforMicrosoft.NET.PHPUnit(https://phpunit.de/):UnittestingframeworkforPHP.SimplyVBUnit(http://simplyvbunit.sourceforge.net/):UnittestingframeworkforVB.NET.Unittest(https://docs.python.org/3/library/unittest.html):UnittestingframeworkforPython.

SummarySoftwarequalityisakeyconceptinsoftwareengineering,sinceitdeterminesthedegreeinwhichasoftwaresystemmeetsitsrequirementsanduserexpectations.VerificationandValidationisthenamegiventosetofactivitiesaimedtoassessasoftwaresystem.ThegoalofV&Vistoensurethequalityofapieceofsoftwarewhilereducingthenumberofdefects.ThetwocoreactivitiesinV&Varesoftwaretesting(evaluationofarunningpieceofsoftware)andstaticanalysis(assessmentofsoftwareartefactswithoutitsexecution).

Automatedsoftwaretestinghasexperiencedbiggestadvancesinthelastfewdecades.Inthisarena,theJUnitframeworkhasaremarkableposition.JUnitwasdesignedtobeaunitframeworkfortheJVM.Nowadays,itisafactthatJUnitisthemostpopulartestframeworksintheJavacommunity,providingacomprehensiveprogrammingmodeltocreateandexecutetestcases.Inthenextsection,wewilldiscoverthefeaturesandcapabilitiesprovidedbythenewversionoftheframework,JUnit5.

What’sNewInJUnit5Thosewhocanimagineanything,cancreatetheimpossible.

-AlanTuring

JUnitisthemostimportanttestingframeworkfortheJVMandoneofthemostinfluentialinsoftwareengineeringingeneral.JUnit5isthenextgenerationofJUnit,anditsfirstGeneralAvailability(GA)version(5.0.0)wasreleasedonSeptember10,2017.Aswewilldiscover,JUnit5supposesasmallrevolutionwithrespecttoJUnit4,providingacompletelynewarchitecture,programming,andextensionmodel.Thischaptercoversthefollowingcontent:

RoadtoJUnit5:Inthefirstsection,wewilldiscoverthemotivationtocreateanewmajorversionofJUnit(thatis,thelimitationsofJUnit4),thedesignprinciplesguidingthedevelopmentofJUnit5,andfinallythedetailsoftheJUnit5opensourcecommunity.JUnit5architecture:JUnit5isamodularframeworkcomposedofthreemajorcomponents,namedPlatform,Jupiter,andVintage.RunningtestsinJUnit5:WewilldiscoverhowtorunJUnit5testsusingpopularbuildtools,suchasMavenorGradle,andalsowithIDEssuchasIntelliJorEclipse.TheextensionmodelofJUnit5:Theextensionmodelallowsforthird-partylibrariesandframeworkstoextendtheJUnit5programmingmodelwiththeirownadditions.

RoadtoJUnit5SoftwaretestinghaschangedalotsincethefirstreleaseofJUnit4in2006.Sincethen,notonlyhaveJavaandtheJVMhasevolved,butalsoourtestingneedsmatured.Wearenotwritingjustunittestsanymore.Instead,inadditiontoverifyingasinglepieceofcode,softwareengineersandtestersdemandotherkindsoftests,suchasintegrationandend-to-endtests.

Inaddition,ourexpectationsabouttestingframeworkshavegrown.Nowadays,wedemandadvancedcapabilitiesfortheseframeworks,suchasextensibilityormodularity,tonameafew.Inthissection,wediscoverthemainlimitationsofJUnit4,thevisionofJUnit5,andthecommunitysupportingitsdevelopment.

JUnit5motivationAccordingtoseveralstudies,JUnit4isthemostusedlibraryforJavaprojects.Forinstance,TheTop100JavalibrariesonGitHubisawell-knownreportpublishedbyOverOps(@overopshq),asoftwareanalyticscompanyfocusedonlarge-scaleJavaandScalacodebases.

Initseditionof2017,thisreportanalyzedtheimportstatementsofuniqueJavalibrariesthatareusedbythetop1,000JavaprojectsonGitHub(bystars).Inthelightoftheresults,JUnit4istheundisputedkingofJavaLibraries:theimportsofthepackagesorg.junitandorg.junit.runnerappearinthefirstandsecondposition,respectively:

TheTop20JavalibrariesonGitHub

Despitethisfact,JUnit4isaframeworkcreatedmorethanadecadeago,andthereareimportantseverallimitationsthatimposeacompleteredesignoftheframework.

ModularityFirstofall,JUnit4isnotmodular.Asdepictedinthefollowingpicture,thearchitectureofJUnit4iscompletelymonolithic.AllthecapabilitiesofJUnit4areprovidedbythejunit.jardependency.Asaresult,differenttestmechanisms,suchastestdiscoveryandexecution,aretightlycoupledinJUnit4.

TheJUnit4Architecture

JohannesLink,oneoftheJUnit5coreteammembers,summarizesthisprobleminaninterviewforJaxmagazineonAugust13,2015(duringtheinceptionofJUnit5):

ThesuccessofJUnitasaplatformpreventsthedevelopmentofJUnitasatesttool.ThebasicproblemwewanttosolveisexecutingtestcasesbyseparatingasufficientlypowerfulandstableAPI.

JUnit4runnersTheJUnit4’srunnerAPIalsohasanimportantdeterrent.Asdescribedinchapter1,RetrospectiveonsoftwarequalityandJavatesting,inJUnit4arunnerisaJavaclassusedtomanageatest’slifecycle.TherunnerAPIinJUnit4isquitepowerful,nevertheless,ithasanimportantdrawback:runnersarenotcomposable,thatis,wecanonlyuseasinglerunneratatime.

Forexample,aparameterizedtestcannotbecombinedwiththeSpringtestsupport,duetothefactthatbothtestswouldusetheirownrunnerimplementation.ThinkinginJava(seethesnippetsgivenfollow),eachtestcaseusesitsownunique@RunWithannotation.ThefirstoneusestheParameterizedrunner:

importorg.junit.Test;

importorg.junit.runner.RunWith;

importorg.junit.runners.Parameterized;

@RunWith(Parameterized.class)

publicclassMyParameterizedTest{

@Test

publicvoidmyFirstTest(){

//mytestcode

}

}

WhilethissecondexampleisusingtheSpringJUnit4ClassRunnerrunner,itwouldnotbecombinedwiththepreviousoneduetoalimitationonJUnit4(runnersarenotcomposable):

importorg.junit.Test;

importorg.junit.runner.RunWith;

importorg.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)

publicclassMySpringTest{

@Test

publicvoidyetAnotherTest(){

//mytestcode

}

}

JUnit4rulesDuetothestrictlimitationofuniquenessofaJUnit4runnerwithinthesametestclass,version4.7ofJUnitintroducedtheconceptofmethod-levelrules,whichareannotatedfieldsinatestclasswith@Rule.Theserulesallowforadditionorredefinitionoftestbehaviorbyexecutingsomecodebeforeandaftertheexecutionofthetest.JUnit4.9alsoincorporatestheconceptofclass-levelrules,whicharerulesthatareexecutedbeforeandafteralltestswithintheclass.Theserulesareidentifiedbyannotatingstaticfieldswith@ClassRule,asshowninthefollowingexample:

importorg.junit.ClassRule;

importorg.junit.Test;

importorg.junit.rules.TemporaryFolder;

publicclassMyRuleTest{

@ClassRule

publicstaticTemporaryFoldertemporaryFolder=newTemporaryFolder();

@Test

publicvoidanotherTest(){

//mytestcode

}

}

Whilerulesaresimplerandmostlycompostable,theyhaveotherdrawbacks.ThemaininconveniencewhenusingJUnit4rulesforcomplextestsisthatwearenotabletouseasingleruleentityformethod-levelandclass-level.Attheendoftheday,thisimposeslimitationstocustomizethelifecyclemanagement(thebefore/afterbehavior).

JUnit5inceptionEventhoughJUnit4wasthedefaulttestingframeworkformillionsofJavadevelopersworldwide,noneoftheactiveJUnitmaintainerswerepaidbytheiremployertodothatwork.Forthatreason,andinordertoovercomethedrawbacksofJUnit4,inJuly2015JohannesLinkandMarcPhilippstartedtheJUnitLambdacrowdfundingcampaign(http://junit.org/junit4/junit-lambda-campaign.html)onIndiegogo(aninternationalcrowdfundingwebsite):

JUnitLambdaCrowdfundingCampaign

JUnitLambdawasthenamegiventotheproject,whichwastheseedofthecurrentJUnit5framework.TheinclusionofthewordlambdaintheprojectnameenforcestheideaofusingJava8fromtheverybeginningoftheproject.QuotingtheJUnitLambdaprojectsite:

Thegoalistocreateanup-to-datefoundationfordeveloper-sidetestingontheJVM.ThisincludesfocusingonJava8andabove,aswellasenablingmanydifferentstylesoftesting.

TheJUnitLambdaCrowdfundingCampaignranfromJulytoOctober2015.

Itwasasuccess,raising53,937eurosfrom474individualsandcompaniesworldwide.Fromthispoint,theJUnit5kick-offteamwascreated,joiningpeoplefromEclipse,Gradle,IntelliJ,orSpring.

TheJUnitLambdaprojectbecameJUnit5,andthedesignprinciplesguidingthedevelopmentprocesswerethefollows:

Modularization:Asintroducedbefore,JUnit4wasnotmodular,andthiscausessomeproblems.Fromitsinception,JUnit5architectureismuchcompletelymodular,allowingdeveloperstousethespecificpartsoftheframeworktheyrequire.Powerfulextensionmodelwithfocusoncomposability:Extensibilityisamustformoderntestingframeworks.Therefore,JUnit5shouldprovideseamlessintegrationwiththird-partyframeworks,suchasSpringorMockito,tonameafew.APIsegregation:Decoupletestdiscoveryandexecutionfromtestdefinition.Compatibilitywitholderreleases:SupportingtheexecutionoflegacyJava3andJava4inthenewJUnit5platform.Modernprogrammingmodelforwritingtests(Java8):Nowadays,moreandmoredeveloperswritecodewithJava8newfeatures,suchaslambdaexpressions.JUnit4wasbuiltonJava5,butJUnit5hasbeencreatedfromscratchusingJava8.

JUnit5communityThesourcecodeofJUnit5ishostedonGitHub(https://github.com/junit-team/junit5).AllmodulesoftheJUnit5frameworkhavebeenreleasedunderthetermsoftheopensourcelicenseEPLv1.0.Thereisoneexceptiontothisrule,sincethemodulecalledjunit-platform-surefire-provider(describedlater)hasbeenreleasedusingApacheLicensev2.0.

TheroadmapoftheJUnitdevelopment(https://github.com/junit-team/junit5/wiki/Roadmap)andthedefinitionandstatusofthedifferentreleasesandmilestones(https://github.com/junit-team/junit5/milestones/)arepubliconGitHub.Thefollowingtablesummarizesthisroadmap:

Phase Date Release

0.Crowdfunding

FromJuly2015toOctober2015 -

1.Kickoff FromOctober20to22,2015 -

2.Firstprototype

FromOctober23,2015totheendofNovember2015

-

3.Alphaversion February1,2016 5.0Alpha

4.Firstmilestone

July9,20165.0M1:Stable,documentedIDE-facingAPIs(LauncherAPIandEngineSPI),dynamictests

5.Additionalmilestones

July23,2016(5.0M2)

November30,2016(5.0M3)

April1,2017(5.0M4)

5.0M2:Bugfixandminorimprovementrelease

5.0M3:JUnit4interoperability,additionaldiscoveryselectors

5.0M4:Testtemplates,repeatedtests,andparameterizedtests

5.0M5:Dynamiccontainersandminor

July5,2017(5.0M5)

July16,2017(5.0M6)

APIchanges

5.0M6:Java9compatibility,scenariotests,additionalextensionAPIsforJUnitJupiter

6.Releasecandidate(RC)

July30,2017

July30,2017

August23,2017

5.0RC1:Finalbugfixesanddocumentationimprovements

5.0RC2:FixGradleconsumptionofjunit-jupiter-engine

5.0RC3:Configurationparametersandbugfixes

7.Generalavailability(GA)

September10,2017 5.0GA:Firststablerelease

TheJUnit5contributorsaremorethanjustdevelopers.Contributorsarealsotesters,maintainers,andcommunicators.Atthetimeofwriting,thetopJUnit5contributorsonGitHubare:

SamBrannen(@sam_brannen):CoreSpringFrameworkandJUnit5committer.EnterpriseJavaConsultantatSwiftmind.Spring&JUnittrainer.Conferencespeaker.MarcPhilipp(@marcphilipp):SeniorSoftwareEngineeronLogMeIn,activecontributortoopensourceprojectssuchasJUnitorUsus.Conferencespeaker.JohannesLink(@johanneslink):Programmerandsoftwaretherapist.JUnit5supporter.MatthiasMerdes:LeadDeveloperatHeidelbergMobilGmbH,Germany.

TopJUnit5contributorsonGitHub

ThefollowinglistprovidesacollectionofonlineJUnit5resources:

Officialwebsite(http://junit.org/junit5/).Sourcecode(https://github.com/junit-team/junit5/).JUnit5developerguide(http://junit.org/junit5/docs/current/user-guide/).Referencedocumentation.TwitteroftheJUnitteam(https://twitter.com/junitteam).Usually,thetweetsaboutJUnit5aretaggedwith#JUnit5(https://twitter.com/hashtag/JUnit5).Issues(https://github.com/junit-team/junit5/issues).ProblemsorsuggestionsforadditionalfunctionalityonGitHub.QuestionsonStackOverflow(https://stackoverflow.com/questions/tagged/junit5).StackOverflowisapopularquestion-and-answerwebsiteforcomputerprogramming.Thetagjunit5shouldbeusedtoaskquestionsaboutJUnit

5.JUnit5JavaDoc(http://junit.org/junit5/docs/current/api/).JUnit5Gitter(https://gitter.im/junit-team/junit5),aninstantmessagingandchatroomsystemusedtodiscussdirectlywiththeJUnit5teammembersandotherpractitioners.OpenTestAlliancefortheJVM(https://github.com/ota4j-team/opentest4j).ItisaninitiativestartedbytheJUnit5team,anditsobjectiveistoprovideaminimalcommonfoundationfortestinglibraries(JUnit,TestNG,Spock,andsoon)andthird-partyassertionlibraries(Hamcrest,AssertJ,andsoon)ontheJVM.TheideaistouseacommonsetofexceptionsthatIDEsandbuildtoolscansupportinaconsistentmanneracrossalltestingscenarios(sofarthereisnostandardfortestingontheJVM,andtheonlycommonbuildingblockistheJavaexceptionjava.lang.AssertionError).

JUnit5architectureTheJUnit5frameworkhasbeendesignedtobeconsumedbydifferentprogrammaticclients.ThefirstgroupofclientsareJavatests.ThesetestscanbebasedonJUnit4(testswhichusethetestlegacyprogrammingmodel),JUnit5(testswhichusethebrandnewprogrammingmodel),andevenotherkindsofJavatests(thirdparty).Thesecondgroupofclientsarebuildtools(suchasMavenorGradle)andIDEs(suchasIntelliJorEclipse).

Inordertoachievetheintegrationofallthesepiecesinalooselycoupledmanner,JUnit5wasdesignedtobemodular.Asdepictedinthefollowingpicture,theJUnit5frameworkiscomposedofthreemajorcomponents,calledPlatform,Jupiter,andVintage:

JUnit5Architecture:high-levelcomponent

Thehigh-levelcomponentsoftheJUnit5architectureareenumeratedasfollows:

Thefirsthigh-levelcomponentiscalledJupiter.Itprovidesthebrand-newprogrammingandextensionmodeloftheJUnit5framework.InthecoreofJUnit5,wefindtheJUnitPlatform.ThiscomponentisaimedtobecomethefoundationforanytestingframeworkexecutedintheJVM.Inotherwords,itprovidesmechanismstorunJupitertests,legacyJUnit4,andalsothird-partytests(forexample,Spock,FitNesse,andsoon).Thelasthigh-levelcomponentoftheJUnit5architectureiscalled

Vintage.ThiscomponentallowsrunninglegacyJUnittestsontheJUnitPlatformoutofthebox.

Let’stakeacloserlookatthedetailsofeachcomponenttofindouttheirinternalmodules:

JUnit5Architecture:modules

Ascanbeseeninthepicturepreceding,therearethreetypesofmodule:

TestAPIs:Thesearethemodulesfacingusers(thatis,softwareengineerandtesters).ThesemodulesprovidetheprogrammingmodelforaparticularTestEngine(forexample,junit-jupiter-apiforJUnit5testsandjunitforJUnit4tests).TestEngines:Thesemodulesallowtoexecuteakindoftest(Jupitertests,legacyJUnit4,orotherJavatests)withintheJUnitPlatform.TheyarecreatedbyextendingthegeneralPlatformEngine(junit-platform-engine).TestLauncher:ThesemodulesprovidetheabilityoftestdiscoveryinsidetheJUnitplatformforexternalbuildtoolsandIDEs.ThisAPIisconsumedbytoolssuchasMaven,Gradle,IntelliJ,andsoon,usingthejunit-platform-launchermodule.

Asaresultofthismodulararchitecture,theJUnitframeworkexposesasetof

interfaces:

AnAPI(ApplicationProgrammingInterface)towritetests,theJupiterAPI.ThedetaileddescriptionofthisAPIiswhatitisknownastheJupiterprogrammingmodelanditisdescribedindetailinchapters3,JUnit5StandardTestsandchapter4,SimplifyingTestingWithAdvancedJUnitFeaturesofthisbook.AnSPI(ServiceProviderInterface)todiscoverandexecutetests,theEngineSPI.ThisSPIistypicallyextendedbytestengines,whichintheendprovidetheprogrammingmodelstowritetests.AnAPIfortestdiscoveryandexecution,theLauncherAPI.ThisAPIistypicallyconsumedbyprogrammaticclients,thatareIDEsandbuildtools.

APIandSPIarebothasetsofassets(typicallyclassesandinterfaces)usedbysoftwareengineersforagivenpurpose.ThedifferenceisthatAPIiscalledwhileSPIisextended.

TestEngineSPITheTestEngineSPIallowsforcreatingtestexecutorsontopoftheJVM.IntheJUnit5framework,therearetwoTestEngineimplementationsoutofthebox:

Thejunit-vintage-engine:ThisallowsrunningJUnit3and4testsintheJUnitplatform.Thejunit-jupiter-engine:ThisallowsrunningJUnit5testsintheJUnitplatform.

Moreover,third-partytestlibraries(forexample,Spock,TestNG,andsoon)canplugintotheJUnitPlatformbyprovidingacustomTestEngine.Todothat,theseframeworksshouldcreateitsownTestEnginebyextendingtheJUnit5interfaceorg.junit.platform.engine.TestEngine.Inordertoextendthisinterface,threemandatorymethodsmustbeoverridden:

getId:Theuniqueidentifierforthetestengine.discover:Thelogictofindandfilterthetest(s).execute:Thelogictorunthepreviouslyfoundtest(s).

ThefollowingexampleprovidestheskeletonforacustomTestEngine:packageio.github.bonigarcia;

importorg.junit.platform.engine.EngineDiscoveryRequest;

importorg.junit.platform.engine.ExecutionRequest;

importorg.junit.platform.engine.TestDescriptor;

importorg.junit.platform.engine.TestEngine;

importorg.junit.platform.engine.UniqueId;

importorg.junit.platform.engine.support.descriptor.EngineDescriptor;

publicclassMyCustomEngineimplementsTestEngine{

publicstaticfinalStringENGINE_ID="my-custom-engine";

@Override

publicStringgetId(){

returnENGINE_ID;

}

@Override

publicTestDescriptordiscover(EngineDiscoveryRequestdiscoveryRequest,

UniqueIduniqueId){

//Discovertest(s)andreturnaTestDescriptorobject

TestDescriptortestDescriptor=newEngineDescriptor(uniqueId,

"Mytest");

returntestDescriptor;

}

@Override

publicvoidexecute(ExecutionRequestrequest){

//UseExecutionRequesttoexecuteTestDescriptor

TestDescriptorrootTestDescriptor=

request.getRootTestDescriptor();

request.getEngineExecutionListener()

.executionStarted(rootTestDescriptor);

}

}

AlistofexistingTestEngines(forexample,Specsy,Spek,andothers)ismaintainedbythecommunityinthewikilocatedintheGitHubsiteoftheJUnit5team:https://github.com/junit-team/junit5/wiki/Third-party-Extensions.

TestLauncherAPIOneofthegoalsofJUnit5istomaketheinterfacebetweenJUnitanditsprogrammaticclients(buildtoolsandIDEs)morepowerfulandstable.Tothataim,theTestLauncherAPIhasbeenimplemented.ThisAPIisusedbyIDEsandbuildtoolsfordiscovering,filtering,andexecutingtests.

LookingcloseratthedetailsofthisAPI,wefindtheclassLauncherDiscoveryRequest,whichexposesafluentAPItoselectthelocationoftests(forexampleclasses,methods,orpackages).Thisgroupoftestscanbefiltered,forexample,usingamatchpattern:

importstatic

org.junit.platform.engine.discovery.ClassNameFilter.includeClassNamePatterns;

importstaticorg.junit.platform.engine.discovery.DiscoverySelectors.selectClass;

importstaticorg.junit.platform.engine.discovery.DiscoverySelectors.selectPackage;

importorg.junit.platform.launcher.Launcher;

importorg.junit.platform.launcher.LauncherDiscoveryRequest;

importorg.junit.platform.launcher.TestPlan;

importorg.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;

importorg.junit.platform.launcher.core.LauncherFactory;

//Discoverandfiltertests

LauncherDiscoveryRequestrequest=LauncherDiscoveryRequestBuilder

.request()

.selectors(selectPackage("io.github.bonigarcia"),

selectClass(MyTest.class))

.filters(includeClassNamePatterns(".*Test")).build();

Launcherlauncher=LauncherFactory.create();

TestPlanplan=launcher.discover(request);

Afterthat,theresultingtestsuitecanbeexecutedusingtheclassTestExecutionListener.Thisclasscanbealsousedtogetfeedbackandreceiveevents:

importorg.junit.platform.launcher.TestExecutionListener;

importorg.junit.platform.launcher.listeners.SummaryGeneratingListener;

//Executingtests

TestExecutionListenerlistener=newSummaryGeneratingListener();

launcher.registerTestExecutionListeners(listener);

launcher.execute(request);

RunningtestsinJUnit5Atthetimeofwriting,Jupitertestscanbeexecutedinseveralways:

Usingabuildtool:Maven(implementedinthemodulejunit-plaform-surefire-provider)orGradle(implementedinthemodulejunit-platform-gradle-plugin).UsingtheConsoleLauncher:Acommand-lineJavaapplicationthatallowstolaunchtheJUnitPlatformfromtheconsole.UsinganIDE:IntelliJ(sinceversion2016.2)andEclipse(sinceversion4.7,Oxygen).

Aswearegoingtodiscover,andduetothemodulararchitectureofJUnit5,weneedtoincludethreedependenciesinourprojects:onefortheTestAPI(toimplementtests),anotherfortheTestEngine(toruntests),andthelastoneoftheTestLauncher(todiscovertests).

JupitertestswithMavenInordertorunJupitertestswithinaMavenproject,weneedtoconfigurethepom.xmlfileproperly.Firstofall,weneedtoincludethejunit-jupiter-apimoduleasadependency.Thisisneededtowriteourtest,andtypicallywithtestscope:

<dependencies>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-api</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

</dependencies>

Ingeneral,itisrecommendedtousethelatestversionofthedependencies.Inordertocheckwhatitthatversion,wecancheckitonMavenCentral(http://search.maven.org/)

Then,themaven-surefire-pluginhastobedeclared.Internally,thispluginneedstwodependencies:theTestLauncher(junit-platform-surefire-provider)andtheTestEngine(junit-jupiter-engine):

<build>

<plugins>

<plugin>

<artifactId>maven-surefire-plugin</artifactId>

<version>${maven-surefire-plugin.version}</version>

<dependencies>

<dependency>

<groupId>org.junit.platform</groupId>

<artifactId>junit-platform-surefire-provider</artifactId>

<version>${junit.platform.version}</version>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-engine</artifactId>

<version>${junit.jupiter.version}</version>

</dependency>

</dependencies>

</plugin>

</plugins>

</build>

AllthesourcecodeofthisbookispubliclyavailableontheGitHubrepositoryathttps://github.com/bonigarcia/mastering-junit5.

Lastbutnotleast,weneedtocreateaJupitertestcase.Sofar,wehavenotlearnedhowtoimplementJupitertests(thispartiscoveredinchapter3,JUnit5StandardTests).Nevertheless,thetestweexecutehereisthesimplesttesttodemonstratetheexecutionoftheJUnit5framework.AJupitertest,inits

minimalexpression,isjustaJavaclassinwhichone(ormore)ofitsmethodsareannotatedwith@Test(packageorg.junit.jupiter.api).Thefollowingsnippetprovidesanexample:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importorg.junit.jupiter.api.Test;

classMyFirstJUnit5Test{

@Test

voidmyFirstTest(){

Stringmessage="1+1shouldbeequalto2";

System.out.println(message);

assertEquals(2,1+1,message);

}

}

JUnitrequiresJava8(orhigher)atruntime.However,wecanstilltestcodethathasbeencompiledwithpreviousversionsofJava.

Asshowninthefollowingpicture,thistestcanbeexecutedusingthecommandmvntest:

RunningJupitertestswithMaven

JupitertestswithGradleNow,wearegoingtostudythesameexample,butthistimeexecutedwithGradle.Therefore,weneedtoconfigurethebuild.gradlefile.Inthisfile,weneedtodefine:

ThedependencyfortheJupiterAPI(junit-jupiter-api).ThedependencyfortheTestEngine(junit-jupiter-engine).ThepluginfortheTestLauncher(junit-platform-gradle-plugin).

Thecompletesourceofbuild.gradleisasfollows:buildscript{

repositories{

mavenCentral()

}

dependencies{

classpath("org.junit.platform:junit-platform-gradle-

plugin:${junitPlatformVersion}")

}

}

repositories{

mavenCentral()

}

applyplugin:'java'

applyplugin:'eclipse'

applyplugin:'idea'

applyplugin:'org.junit.platform.gradle.plugin'

compileTestJava{

sourceCompatibility=1.8

targetCompatibility=1.8

options.compilerArgs+='-parameters'

}

dependencies{

testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")

testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")

}

WeusethecommandgradletesttorunourJupitertestfromthecommandlinewithGradle:

RunningJupitertestswithGradle

LegacytestswithMavenThefollowingistheimagewewanttorunthelegacytest(JUnit4inthiscase)insidetheJUnitPlaform:

packageio.github.bonigarcia;

importstaticorg.junit.Assert.assertEquals;

importorg.junit.Test;

publicclassLegacyJUnit4Test{

@Test

publicvoidmyFirstTest(){

Stringmessage="1+1shouldbeequalto2";

System.out.println(message);

assertEquals(message,2,1+1);

}

}

Tothataim,inMaven,wefirstneedtoincludetheoldJUnit4dependencyinourpom.xml,asfollows:

<dependencies>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>4.12</version>

<scope>test</scope>

</dependency>

</dependencies>

Then,weneedtoincludemaven-surefire-plugin,usingthefollowingdependenciesfortheplugin:theTestEngine(junit-vintage-engine)andtheTestLauncher(junit-platform-surefire-provider):

<build>

<plugins>

<plugin>

<artifactId>maven-surefire-plugin</artifactId>

<version>${maven-surefire-plugin.version}</version>

<dependencies>

<dependency>

<groupId>org.junit.platform</groupId>

<artifactId>junit-platform-surefire-provider</artifactId>

<version>${junit.platform.version}</version>

</dependency>

<dependency>

<groupId>org.junit.vintage</groupId>

<artifactId>junit-vintage-engine</artifactId>

<version>${junit.vintage.version}</version>

</dependency>

</dependencies>

</plugin>

</plugins>

</build>

Theexecutionfromthecommandlinewillalsobeusingthecommandmvntest:

RunningLegacytestswithMaven

LegacytestswihGradleIfwewanttoexecutethesametestpresentedintheexamplebefore(io.github.bonigarcia.LegacyJUnit4Test),butthistimeusingGradle,weneedtoincludethefollowinginourbuild.gradlefile:

ThedependencyforJUnit4.12.ThedependencyfortheTestEngine(junit-vintage-engine).ThepluginfortheTestLauncher(junit-platform-gradle-plugin).

Thus,thecompletesourceofbuild.gradlewouldbeasfollows:buildscript{

repositories{

mavenCentral()

}

dependencies{

classpath("org.junit.platform:junit-platform-gradle-

plugin:${junitPlatformVersion}")

}

}

repositories{

mavenCentral()

}

applyplugin:'java'

applyplugin:'eclipse'

applyplugin:'idea'

applyplugin:'org.junit.platform.gradle.plugin'

compileTestJava{

sourceCompatibility=1.8

targetCompatibility=1.8

options.compilerArgs+='-parameters'

}

dependencies{

testCompile("junit:junit:${junitLegacy}")

testRuntime("org.junit.vintage:junit-vintage-engine:${junitVintageVersion}")

}

Theexecutionfromthecommandlinewouldbeasfollows:

RunningLegacytestswithGradle

TheConsoleLauncherTheConsoleLauncherisacommand-lineJavaapplicationthatallowslaunchingtheJUnitPlatformfromtheconsole.Forexample,itcanbeusedtorunVintageandJupitertestsfromthecommandline.

AnexecutableJARwithalldependenciesincludedispublishedinthecentralMavenrepositoryunderthejunit-platform-console-standaloneartifact.ThestandaloneConsoleLaunchercanbeexecutedasfollows:

java-jarjunit-platform-console-standalone-version.jar<Options>

TheexampleGitHubrepositoryjunit5-console-launchercontainsasimpleexamplefortheuseoftheConsoleLauncher.Asdepictedinthefollowingpicture,arunconfigurationentryhasbeencreatedinEclipse,runningthemainclass,org.junit.platform.console.ConsoleLauncher.Then,thetestclassnameispassedasanargumentusingtheoption--select-classandthequalifiedclassname(inthisexample,io.github.bonigarcia.EmptyTest).Afterthat,wecanruntheapplication,obtainingthetestresultintheintegratedconsoleofEclipse:

ExampleofConsoleLauncherinEclipse

JupitertestsinJUnit4JUnit5hasbeendesignedtobeforwardandbackwardcompatible.Ontheonehand,theVintagecomponentsupportsrunninglegacycodeonJUnit3and4.Ontheotherhand,JUnit5providesaJUnit4runnerthatallowstorunJUnit5inIDEsandbuildsystemsthatsupportJUnit4,butdoesnotyetsupportthenewJUnitPlatform5directly.

Let’sseeoneexample.ImaginewewanttorunaJupitertestinanIDEdoesnotsupportJUnit5,forexample,anoldversionofEclipse.Inthiscase,weneedtoannotateourJupitertestwith@RunWith(JUnitPlatform.class).TheJUnitPlatformrunnerisaJUnit4-basedrunner,whichenablestorunanytestwhoseprogrammingmodelissupportedontheJUnitPlatforminaJUnit4environment.Therefore,ourtestwouldresultasfollows:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importorg.junit.jupiter.api.Test;

importorg.junit.platform.runner.JUnitPlatform;

importorg.junit.runner.RunWith;

@RunWith(JUnitPlatform.class)

publicclassJUnit5CompatibleTest{

@Test

voidmyTest(){

Stringmessage="1+1shouldbeequalto2";

System.out.println(message);

assertEquals(2,1+1,message);

}

}

IfthistestiscontainedinaMavenproject,ourpom.xmlshouldcontainthefollowingdependencies:

<dependencies>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-api</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-engine</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.junit.platform</groupId>

<artifactId>junit-platform-runner</artifactId>

<version>${junit.platform.version}</version>

<scope>test</scope>

</dependency>

</dependencies>

Ontheotherhand,foraGradleproject,ourbuild.gradleisthefollowing:buildscript{

repositories{

mavenCentral()

}

dependencies{

classpath("org.junit.platform:junit-platform-gradle-

plugin:${junitPlatformVersion}")

}

}

repositories{

mavenCentral()

}

applyplugin:'java'

applyplugin:'eclipse'

applyplugin:'idea'

applyplugin:'org.junit.platform.gradle.plugin'

compileTestJava{

sourceCompatibility=1.8

targetCompatibility=1.8

options.compilerArgs+='-parameters'

}

dependencies{

testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")

testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")

testCompile("org.junit.platform:junit-platform-runner:${junitPlatformVersion}")

}

IntelliJIntelliJ2016.2+hasbeenthefirstIDEwhichsupportstheexecutionofJupitertestsnatively.Asshowninthefollowingscreenshot,anyJupitertestcanbeexecutedusingtheintegratedfunctionsoftheIDE:

RunningaJupitertestinIntelliJ2016.2+

EclipseEclipse4.7(Oxygen)hasbetasupportforJUnit5.Thankstothis,EclipseprovidestheabilityofrunningJupitertestsdirectlyinEclipse,asshowninthefollowingscreenshot:

RunningaJupitertestinEclipse4.7+

Moreover,Eclipse4.7(Oxygen)providesawizardtocreateJupitertestsinasimpleway,asshowninthefollowingpictures:

EclipsewizardtocreateJupitertests

TheextensionmodelofJUnit5Asintroducedbefore,JupiteristhenamegiventothenewprogrammingmodelofJUnit5,describedindetailinchapter3,JUnit5standardtestsandchapter4,SimplifyingtestingwithadvancedJUnitfeatures,togetherwiththeextensionmodel.TheextensionmodelallowstoextendtheJupiterprogrammingmodelwithcustomadditions.Thankstothis,third-partyframeworks(suchasSpringorMockito,tonameafew)canachieveinteroperabilitywithJUnit5inaseamlessway.Theextensionsprovidedbytheseframeworkswillbestudiedinchapter5,IntegrationofJUnit5withexternalframeworks.Inthecurrentsection,weanalyzethegeneralperformanceoftheextensionmodelandalsotheextensionsprovidedoutoftheboxinJUnit5.

IncontrasttoformerextensionpointsinJUnit4(thatis,testrunnersandrules),theJUnit5extensionmodelconsistsofasingle,coherentconcept:theExtensionAPI.ThisAPIallowstoextendthecorefunctionalityofJUnit5byanythirdparty(toolvendor,developers,andsoon).ThefirstthingweneedtounderstandaboutextensionsinJupiteristhateachnewextensionimplementsaninterfacecalledExtension.Thisinterfaceisamarkerinterface,thatis,aJavainterfacewithnofieldormethods:

packageorg.junit.jupiter.api.extension;

importstaticorg.apiguardian.api.API.Status.STABLE;

importorg.apiguardian.api.API;

/**

*Markerinterfaceforallextensions.

*

*@since5.0

*/

@API(status=STABLE,since="5.0")

publicinterfaceExtension{

}

InordertomakeeasethecreationofJupiterextensions,JUnit5providesasetofextensionspointswhichallowstoexecutecustomcodeindifferentpartsofthetestlifecycle.ThefollowingtablecontainsasummaryoftheextensionpointsinJupiter,anditsdetailsarepresentedinthenextsections:

Extensionpoint Implementedbyextensionswhichwantto…

TestInstancePostProcessorProvideadditionalbehaviorjustafterthetest

instantiation

BeforeAllCallbackProvideadditionalbehaviorbeforealltestsareinvokedinatestcontainer

BeforeEachCallbackProvideadditionalbehaviortotestsbeforeeachtestisinvoked

BeforeTestExecutionCallbackProvideadditionalbehaviortotestsimmediatelybeforeeachtestisexecuted

TestExecutionExceptionHandler Handleexceptionsthrownduringtestexecution

AfterAllCallbackProvideadditionalbehaviortotestcontainersafteralltestshavebeeninvoked

AfterEachCallbackProvideadditionalbehaviortotestsaftereachtesthasbeeninvoked

AfterTestExecutionCallbackProvideadditionalbehaviortotestsimmediatelyaftereachtesthasbeenexecuted

ExecutionCondition Conditionatethetestexecutionatruntime

ParameterResolver Resolveparametersatruntime

Oncewecreatedanextension,inordertouseit,weneedtousetheannotationExtendWith.Thisannotationcanbeusedtoregisteroneormoreextensions.Itcanbedeclaredoninterfaces,classes,methods,fields,andeveninotherannotations:

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

publicclassMyTest{

@ExtendWith(MyExtension.class)

@Test

publicvoidtest(){

//Mytestlogic

}

}

TestlifecycleThereareasetofextensionpointsaimedatcontrollingthelifecycleoftests.Firstofall,theTestInstancePostProcessorcanbeusedtoexecutesomelogicafterthetestinstantiation.Afterthat,therearedifferentextensionswhichcontrolthepre-teststage:

TheBeforeAllCallbackdefinesthelogicexecutedbeforealltests.TheBeforeEachCallbackdefinesthelogicexecutedbeforeatestmethod.TheBeforeTestExecutionCallbackdefinesthelogicexecutedimmediatelybeforeatestmethod.

Similarly,thereareextensionstocontrolthepost-testphases:

TheAfterAllCallbackdefinesthelogicexecutedafteralltests.TheAfterEachCallbackdefinesthelogicexecutedafteratestmethod.TheAfterTestExecutionCallbackdefinesthelogicexecutedimmediatelyafteratestmethod.

InbetweentheBefore*andAfter*callbacks,thereisanextensionthatprovidesawayforcollectingexceptions:theTestExecutionExceptionHandler.

Allthesecallbacks,andtheirorderinthetestlifecyclaredepictedinthefollowingpicture:

Lifecycleofextensioncallbacks

Let’sseeanexample.WecreatedanextensioncalledIgnoreIOExceptionExtension,whichimplementsTestExecutionExceptionHandler.Inthisexample,theextension

checkswhetherornottheexceptionisIOException.Ifso,theexceptionisdiscarded:

packageio.github.bonigarcia;

importjava.io.IOException;

importorg.junit.jupiter.api.extension.ExtensionContext;

importorg.junit.jupiter.api.extension.TestExecutionExceptionHandler;

publicclassIgnoreIOExceptionExtension

implementsTestExecutionExceptionHandler{

@Override

publicvoidhandleTestExecutionException(ExtensionContextcontext,

Throwablethrowable)throwsThrowable{

if(throwableinstanceofIOException){

return;

}

throwthrowable;

}

}

Considerthefollowingtestclass,whichcontainstwotests(@Test).Thefirstoneisannotatedwith@ExtendWithandourcustomextension(IgnoreIOExceptionExtension):

packageio.github.bonigarcia;

importjava.io.IOException;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

publicclassExceptionTest{

@ExtendWith(IgnoreIOExceptionExtension.class)

@Test

publicvoidfirstTest()throwsIOException{

thrownewIOException("IOException");

}

@Test

publicvoidsecondTest()throwsIOException{

thrownewIOException("MyIOException");

}

}

Whenexecutingthistestclass,thefirsttestissucceededduetothefactthattheIOExceptionhasbeeninternallyhandledbyourextension.Ontheotherhand,thesecondwillfailsincethatexceptionisnothandled.

Theexecutionofthistestclassintheconsolecanbeseeninthenextscreenshot.NotethatweselectthetesttobeexecutedusingtheMavencommandmvntest-Dtest=ExceptionTest:

Outputofignoreexceptionexample

ConditionalextensionpointsInordertocreateextensionsthatactivateordeactivatetestsdependingonagivencondition,JUnit5providesoneconditionalextensionpointcallledExecutionCondition.Thefollowingsnippetshowsthedeclarationofthisextensionpoint:

packageorg.junit.jupiter.api.extension;

importstaticorg.apiguardian.api.API.Status.STABLE;

importorg.apiguardian.api.API;

@FunctionalInterface

@API(status=STABLE,since="5.0")

publicinterfaceExecutionConditionextendsExtension{

ConditionEvaluationResultevaluateExecutionCondition

ExtensionContextcontext);

}

Theextensioncanbeusedtodeactivateeitheralltestsinacontainer(likelyaclass)orindividualtests(likelyatestmethod).ExamplesofthisextensionareprovidedinthesectionConditionalTestExecutionofchapter3,JUnit5StandardTests.

DependencyinjectionTheParameterResolverextensionprovidesdependencyinjectionatmethodlevel.Inthisexample,wecanseehowanargumentisinjectedinthetestmethodwithacustomimplementationofParameterResolvercalledMyParameterResolver.Followingthecode,wecanseethatthisresolverwillsimplyinjecthard-codedStringparameterswiththevaluemyparameter:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.extension.ExtensionContext;

importorg.junit.jupiter.api.extension.ParameterContext;

importorg.junit.jupiter.api.extension.ParameterResolutionException;

importorg.junit.jupiter.api.extension.ParameterResolver;

publicclassMyParameterResolverimplementsParameterResolver{

@Override

publicbooleansupportsParameter(ParameterContextparameterContext,

ExtensionContextextensionContext)

throwsParameterResolutionException{

returntrue;

}

@Override

publicObjectresolveParameter(ParameterContextparameterContext,

ExtensionContextextensionContext)

throwsParameterResolutionException{

return"myparameter";

}

}

Then,thisparameterresolvercanbeusedinatest,declaringitasusualusingtheannotation@ExtendWith:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

publicclassDependencyInjectionTest{

@ExtendWith(MyParameterResolver.class)

@Test

publicvoidtest(Objectparameter){

System.out.println("Myparameter"+parameter);

}

}

Finally,ifweexecutethistest(forexampleusingMavenandthecommandline),wecanseehowtheinjectedparameterisloggedinthestandardoutput:

Outputofdependencyinjectionextensionexample

Third-partyextensionsIntherealworld,extensionstypicallyimplementseveralofthepreviouslyexplainedextensionpoints.Forexample,SpringExtension(explainedindetailinchapter5,IntegrationofJUnit5withexternalframeworks)implementstheextensionspointsBeforeAllCallback,TestInstancePostProcessor,ParameterResolver,amongothers.ThefollowingsnippetprovidesthestructureofSpringExtension:

packageorg.springframework.test.context.junit.jupiter;

importorg.junit.jupiter.api.extension.*;

publicclassSpringExtensionimplementsBeforeAllCallback,

AfterAllCallback,

TestInstancePostProcessor,BeforeEachCallback,AfterEachCallback,

BeforeTestExecutionCallback,AfterTestExecutionCallback,

ParameterResolver{

@Override

publicvoidafterTestExecution(TestExtensionContextcontext)

throwsException{

//implementation

}

//Restofmethods

}

AlistofexistingJUnit5extensions(forexample,Spring,Selenium,Docker,andothers)ismaintainedbythecommunityinthewikilocatedintheGitHubsiteoftheJUnit5team:https://github.com/junit-team/junit5/wiki/Third-party-Extensions.Someofthemarealsodetailedinchapter5,IntegrationofJUnit5withexternalframeworks.

SummaryThischapterprovidesanoverviewoftheJUnit5testingframework.DuetothelimitationsofJUnit4(monolithicarchitecture,impossibilityofcomposetestrunners,andlimitationsoftestrules),anewmajorversionoftheframeworkwasneeded.Inordertocarryouttheimplementations,theJUnitLambdaprojectstartedacrowdfundingcampaignin2015.Asaresult,theJUnit5developmentteamwasborn,andtheGAreleaseoftheframeworkwasreleasedonSeptember10,2017.

JUnit5wasdesignedtobemodern(thatis,usingJava8andJava9compliantfromtheverybeginning)andmodular.ThethreemajorcomponentswithinJUnit5are:Jupiter(newprogramminganextensionmodel),Platform(foundationforanytestingframeworkexecutedintheJVM),andVintage(integrationwithlegacyJUnit3and4tests).Atthetimeofthiswriting,JUnit5testscanbeexecutedusingbuildtools(MavenorGradle)andalsowithIDEs(IntelliJ2016.2+orEclipse4.7+).

TheextensionmodelofJUnit5allowstoextendthecorefunctionalityofJUnit5byanythirdparty.InordertocreateJUnit5extensions,weneedtoimplementoneorseveralJUnitextensionpoints(suchasBeforeAllCallback,ParameterResolver,orExecutionCondition,amongothers),andthenregistertheextensioninourtestsusingtheannotation@ExtendWith.

Inthenextchapter3,JUnit5StandardTests,wearegoingtolearnthebasicsoftheJupiterprogrammingmodel.Inotherwords,wearegoingtolearnhowtocreatestandardJUnit5tests.

JUnit5StandardTestsTalkischeap.Showmethecode.

-LinusTorvalds

JUnit5providesabrand-newprogrammingmodelcalledJupiter.WecanseethisprogrammingmodelasanAPIforsoftwareengineersandtesterswhichallowtocreateJUnit5tests.ThesetestsarelaterexecutedontheJUnitPlatform.Aswewilldiscover,theJupiterprogrammingmodelallowstocreatemanydifferenttypesoftests.ThischaptertacklesthebasicsofJupiter.Tothataim,thischapterisstructuredasfollows:

Testlifecycle:Inthissection,weanalyzethestructureoftheJupitertests,describingtheannotationsinvolvedinthemanagementofthetestlifecycleintheJUnit5programmingmodel.Then,wediscoverhowtoskiptests,andalsohowtoannotatetestswithacustomdisplayname.Assertions:Inthissection,firstwepresentabriefoverviewoftheverificationassets,calledassertions(alsoknownaspredicates).Second,westudyhowtheassertionshavebeenimplementedinJupiter.Finally,wepresentseveralthird-partylibrariesaboutassertions,providingsomeexamplesforHamcrest.Taggingandfilteringtests:Inthissection,firstwewilllearnhowtolabelJupitertests,thatis,howtocreatetagsinJUnit5.Then,wewilllearnhowtofilterourtestsusingMavenandGradle.Finally,wearegoingtoanalyzehowtocreatemeta-annotationsusingJupiter.Conditionaltestexecution:Inthissection,wewilllearnhowtodisabletestsbasedonagivencondition.Afterthat,wemakeareviewoftheso-calledassumptionsinJupiter,whichareamechanismprovidedoutoftheboxbyJupitertoruntestsonlyifcertainconditionsareasexpected.Nestedtests:ThissectionpresentshowJupiterallowstoexpresstherelationshipamongagroupoftests,callednestedtests.Repeatedtests:ThissectionreviewshowJupiterprovidestheabilitytorepeatatestaspecifiednumberoftimes.MigrationfromJUnit4toJUnit5:ThissectionprovidesasetofhintsaboutthemaindifferencesbetweenJUnit5anditsimmediateantecessor,thatis,JUnit4.Then,thissectionpresentsthesupportforseveralJUnit4ruleswithinJupitertests.

TestlifecycleAswesawinChapter1,RetrospectiveonsoftwarequalityandJavatesting,aunittestcaseiscomposedoffourstages:

1. Setup(optional):First,thetestinitializesthetestfixture(beforethepictureoftheSUT).

2. Exercise:Second,thetestinteractswiththeSUT,gettingsomeoutcomefromitasaresult.

3. Verify:Third,theoutcomefromthesystemundertestiscomparedtotheexpectedvalueusingoneorseveralassertions(alsoknownaspredicates).Asaresult,atestverdictiscreated.

4. Teardown(optional):Finally,thetestreleasesthetestfixturetoputtheSUTbackintotheinitialstate.

InJUnit4,thereweredifferentannotationstocontrolthesetestphases.JUnit5followsthesameapproach,thatis,JavaannotationsareusedtoidentifydifferentmethodswithinJavaclasses,implementingthetestlifecycle.InJupiter,alltheseannotationsarecontainedinthepackageorg.junit.jupiter.api.

ThemostbasicJUnitannotationis@Test,whichidentifiesthemethodsthathavetobeexecutedastests.Therefore,aJavamethodannotatedwithorg.junit.jupiter.api.Testwillbetreatedasatest.ThedifferenceofthisannotationwithrespecttoJUnit4’s@Testistwofolded.Ontheonehand,theJupiter@Testannotationdoesnotdeclareanyattributes.InJUnit4,@Testcandeclarethetesttimeout(aslongattributewiththetimeoutinmilliseconds),ontheotherhand,inJUnit5,neithertestclassesnortestmethodsneedtobepublic(thiswasarequirementinJUnit4).

TakealookatthefollowingJavaclass.Possibly,itisthesimplesttestcasewecancreatewithJupiter.Ithassimplyamethodwiththe@Testannotation.Thetestlogic(thatistheexerciseandverifystagesasdescribedbefore)wouldbecontainedinsidethemethodmyTest.

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Test;

classSimpleJUnit5Test{

@Test

voidmySimpleTest(){

//Mytestlogichere

}

}

TheJupiterannotations(alsolocatedinthepackageorg.junit.jupiter.api)aimedtocontrolthesetupandteardownstagesinJUnit5testsaredescribedinthefollowingtable:

JUnit5annotation Description JUnit4’s

equivalence

@BeforeEachMethodexecutedbeforeeach@Testinthecurrentclass

@Before

@AfterEachMethodexecutedaftereach@Testinthecurrentclass

@After

@BeforeAllMethodexecutedbeforeall@Testinthecurrentclass

@BeforeClass

@AfterAllMethodexecutedafterall@Testinthecurrentclass

@AfterClass

Methodsannotatedwiththeseannotations(@BeforeEach,@AfterEach,@AfterAll,and@BeforeAll)arealwaysinherited.

ThefollowingpicturedepictstheorderofexecutionoftheseannotationsinaJavaclass:

Jupiterannotationstocontrolthetestlyfecycle

Let’sgobacktothegenericstructurefortestswesawatthebeginningofthissection.Now,weareabletomaptheJupiterannotationstocontrolthetestlifecyclewiththedifferentpartsofatestcase.Asillustratedinthefollowing

picture,wecarryoutthesetupstagebyannotatingmethodswith@BeforeAlland@BeforeEach.Then,wecarryouttheexerciseandverifystagesinmethodsannotatedwith@Test.Finally,wecarryouttheteardownprocessinthemethodswith@AfterEachand@AfterAll.

RelationshipamongtheunittestcasesstagesandtheJupiterannotations

Let’sseeasimpleexample,whichusesalltheseannotationsinasingleJavaclass.Thisexampledefinestwotests(thatis,twomethodsannotatedwith@Test),andwedefineadditionalmethodsfortherestofthetestlifecyclewiththeannotations@BeforeAll,@BeforeEach,@AfterEach,and@AfterAll:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.AfterAll;

importorg.junit.jupiter.api.AfterEach;

importorg.junit.jupiter.api.BeforeAll;

importorg.junit.jupiter.api.BeforeEach;

importorg.junit.jupiter.api.Test;

classLifecycleJUnit5Test{

@BeforeAll

staticvoidsetupAll(){

System.out.println("SetupALLTESTSintheclass");

}

@BeforeEach

voidsetup(){

System.out.println("SetupEACHTESTintheclass");

}

@Test

voidtestOne(){

System.out.println("TEST1");

}

@Test

voidtestTwo(){

System.out.println("TEST2");

}

@AfterEach

voidteardown(){

System.out.println("TeardownEACHTESTintheclass");

}

@AfterAll

staticvoidteardownAll(){

System.out.println("TeardownALLTESTSintheclass");

}

}

Ifwerunthistestclass,first@BeforeAllwillbeexecuted.Then,thetwotestmethodswillbeexecutedsequentially,thatis,thefirstoneandthentheother.Ineachexecution,thesetupmethodannotatedwith@BeforeEachwillbeexecutedbeforethetest,andthenthe@AfterEachmethod.ThefollowingscreenshotshowsanexecutionofthetestsusingMavenandthecommandline:

ExecutionofaJupitertestwhichcontrolsitslifecycle

TestinstancelifecycleInordertoprovideexecutioninisolation,theJUnit5frameworkcreatesanewtestinstancebeforeexecutingtheactualtest(thatis,themethodannotatedwith@Test).Thisper-methodtestinstancelifecycleisthebehaviorintheJupitertestandalsoinitsantecessors(JUnit3and4).Asanovelty,thisdefaultbehaviorcanbechangedinJUnit5,simplybyannotatingatestclasswith@TestInstance(Lifecycle.PER_CLASS).Usingthismode,thetestinstancewillbecreatedonceperclass,insteadofoncepertestmethod.

Thisper-classbehaviorimpliesthatitispossibletodeclarethe@BeforeAlland@AfterAllmethodsasnon-static.Thisisbeneficialtobeusedinconjunctionwithsomeadvancedcapabilities,suchasnestedtestordefaulttestinterfaces(explainedinthenextchapter).

Allinall,andtakingintoaccounttheextensioncallback(asexplainedintheTheextensionmodelofJUnit5sectionofChapter2,What’snewinJUnit5),therelativeexecutionorderofusercodeandextensionsisdepictedinthefollowingpicture:

Relativeexecutionorderofusercodeandextensions

SkippingtestsTheJupiterannotation@Disabled(locatedinthepackageorg.junit.jupiter.api)canbeusedtoskiptests.Itcanbeusedatclasslevelormethodlevel.Thefollowingexampleusestheannotation@Disabledatmethodlevelandthereforeitforcestoskipthetest:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Disabled;

importorg.junit.jupiter.api.Test;

classDisabledTest{

@Disabled

@Test

voidskippedTest(){

}

}

Asshowninthefollowingscreenshot,whenweexecutethisexample,thetestwillbecountedasskipped:

Disabledtestmethodconsoleoutput

Inthisotherexample,theannotation@Disabledisplacedattheclasslevelandthereforeallthetestscontainedintheclasswillbeskipped.Notethatacustommessage,typicallywiththereasonofthedisabling,canbespecifiedwithintheannotation:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Disabled;

importorg.junit.jupiter.api.Test;

@Disabled("Alltestinthisclasswillbeskipped")

classAllDisabledTest{

@Test

voidskippedTestOne(){

}

@Test

voidskippedTestTwo(){

}

}

Thefollowingscreenshotshowshowthetestcaseisskippedwhenitisexecuted(inthisexampleusingMavenandthecommandline):

Disabledtestclassconsoleoutput

DisplaynamesJUnit4identifiedtestsbasicallywiththenameofthemethodannotatedwith@Test.Thisimposesalimitationonnametests,sincethesenamesareconstrainedbythewayofdeclaringmethodsinJava.

Toovercomethisproblem,Jupiterprovidestheabilityofdeclaringacustomdisplayname(differenttothetestname)fortests.Thisisdonewiththeannotation@DisplayName.Thisannotationdeclaresacustomdisplaynameforatestclassoratestmethod.Thisnamewillbedisplayedbytestrunnersandreportingtools,anditcancontainspaces,specialcharacters,andevenemojis.

Takealookatthefollowingexample.Weareannotatingthetestclass,andalsothethreetestmethodsdeclaredinsidetheclasswithacustomtestnameusing@DisplayName:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Test;

@DisplayName("Aspecialtestcase")

classDisplayNameTest{

@Test

@DisplayName("Customtestnamecontainingspaces")

voidtestWithDisplayNameContainingSpaces(){

}

@Test

@DisplayName("(╯°Д°)╯")voidtestWithDisplayNameContainingSpecialCharacters(){

}

@Test

@DisplayName(" ")

voidtestWithDisplayNameContainingEmoji(){

}

}

Asaresult,weseetheselabelswhenexecutingthistestinaJUnit5compliantIDE.ThefollowingpictureshowstheexecutionoftheexampleonIntelliJ2016.2+:

Executionofatestcaseusing@DisplayNameinIntelliJ

Ontheotherhand,thedisplaynamecanbealsoseeninEclipse4.7(Oxygen)ornewer:

Executionofatestcaseusing@DisplayNameinEclipse

AssertionsAsweknow,thegeneralstructureofatestcaseiscomposedoffourstages:setup,exercise,verify,andteardown.Theactualtesthappensduringthesecondandthirdstage,whenthetestlogicinteractswiththesystemundertest,gettingsomekindofoutcomefromit.Thisoutcomeiscomparedwiththeexpectedresultintheverifystage.Inthisstage,wefindwhatwecallassertions.Inthissection,wetakeacloserlookatthem.

Anassertion(alsoknownasapredicate)isabooleanstatementtypicallyusedtoreasonaboutsoftwarecorrectness.Fromatechnicalpointofview,anassertioniscomposedofthreeparts(seetheimageafterthelist):

1. First,wefindtheexpectedvalue,whichcomesfromwhatwecalltestoracles.Atestoracleisareliablesourceofexpectedoutputs,forexample,thesystemspecification.

2. Second,wefindtherealoutcome,whichcomesfromtheexercisestagemadebythetestagainsttheSUT.

3. Finally,thesetwovaluesarecomparedusingsomelogiccomparator.Thiscomparisoncanbedoneinmanydifferentways,forexample,wecancomparetheobjectidentity(equalsornot),themagnitude(higherorlowervalue),andsoon.Asaresult,weobtainatestverdict,which,intheend,isgoingtodefineifthetesthassucceededorfailed.

Schematicviewofanassertion

JupiterassertionsLet’smoveontotheJUnit5programmingmodel.JupitercomeswithmanyoftheassertionmethodssuchastheonesinJUnit4,andalsoaddsseveralthatcanbeusedwithJava8lambdas.AllJUnitJupiterassertionsarestaticmethodsintheAssertionsclasslocatedinorg.junit.jupiterpackage.

Thefollowingpictureshowsthecompletelistofthesemethods:

CompletelistofJupiterassertions(classorg.junit.jupiter.Assertions)

ThefollowingtablereviewsthedifferenttypesofbasicassertionsinJupiter:

Assertion Description

fail Failsatestwithagivenmessageand/orexception

assertTrue Assertsthatasuppliedconditionistrue

assertFalse Assertsthatasuppliedconditionisfalse

assertNull Assertsthatasuppliedobjectisnull

assertNotNull Assertsthatasuppliedobjectisnotnull

assertEquals Assertsthattwosuppliedobjectsareequal

assertArrayEquals Assertsthattwosuppliedarraysareequal

assertIterableEquals Assertsthattwoiterableobjectsaredeeplyequal

assertLinesMatch AssertsthattwolistsofStringsareequals

assertNotEquals Assertsthattwosuppliedobjectsarenotequal

assertSame Assertsthattwoobjectsarethesame,comparedwith==

assertNotSame Assertsthattwoobjectsaredifferent,comparedwith!=

Foreachoftheassertionscontainedinthetable,anoptionalfailuremessage(String)canbeprovided.Thismessageisalwaysthelastparameterintheassertionmethod.ThisisasmalldifferencewithrespecttoJUnit4,inwhichthismessagewasthefirstparameterinthemethodinvocation.

ThefollowingexampleshowsatestusingtheassertEquals,assertTrue,andassertFalseassertion.Notethatweareimportingthestaticassertionmethodsatthebeginningoftheclassinordertoimprovethereadabilityofthetestlogic.Intheexample,wefindtheassertEqualsmethod,inthiscasecomparing

twoprimitivetypes(itcouldalsobeusedforobjects).Second,themethodassertTrueevaluatesifabooleanexpressionistrue.Third,themethodassertFalseevaluatesifaBooleanexpressionisfalse.Inthiscase,noticethatthemessageiscreatedasaLamdbaexpression.Thisway,assertionmessagesarelazilyevaluatedtoavoidconstructingcomplexmessagesunnecessarily:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.junit.jupiter.api.Assertions.assertFalse;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.api.Test;

classStandardAssertionsTest{

@Test

voidstandardAssertions(){

assertEquals(2,2);

assertTrue(true,

"Theoptionalassertionmessageisnowthelastparameter");

assertFalse(false,()->"Really"+"expensive"+"message"

+".");

}

}

ThefollowingpartsofthissectionreviewtheadvanceassertionsprovidedbyJupiter:assertAll,assertThrows,assertTimeout,andassertTimeoutPreemptively.

GroupofassertionsAnimportantJupiterassertionisassertAll.Thismethodallowstogroupdifferentassertionsatthesametime.Inagroupedassertion,allassertionsarealwaysexecuted,andanyfailureswillbereportedtogether.

ThemethodassertAllacceptsavargargsoflambdaexpressions(Executable…)orastreamofthose(Stream<Executable>).Optionally,thefirstparameterofassertAllcanbeaStringmessageaimedtolabeltheassertiongroup.

Let’sseeanexample.Inthefollowingtest,wearegroupingacoupleofassertEqualsusinglambdaexpressions:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertAll;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importorg.junit.jupiter.api.Test;

classGroupedAssertionsTest{

@Test

voidgroupedAssertions(){

Addressaddress=newAddress("John","Smith");

//Inagroupedassertionallassertionsareexecuted,andany

//failureswillbereportedtogether.

assertAll("address",()->assertEquals("John",

address.getFirstName()),

()->assertEquals("User",address.getLastName()));

}

}

Whenexecutingthistest,allassertionsofthegroupwillbeevaluated.Sincethesecondassertionfails(lastnamedoesnotmatch),onefailureisreportedinthefinalverdict,ascanbeseeninthefollowingscreenshot:

Consoleoutputofgroupedassertionsexample

AssertingexceptionsAnotherimportantJupiterassertionisassertThrows.Thisassertionallowstoverifyifagivenexceptionisraisedinapieceofcode.Tothataim,themethodassertThrowsacceptstwoarguments.First,theexceptionclassexpected,andsecond,anexecutableobject(lambdaexpression),inwhichtheexceptionissupposedtohappen:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.junit.jupiter.api.Assertions.assertThrows;

importorg.junit.jupiter.api.Test;

classExceptionTest{

@Test

voidexceptionTesting(){

Throwableexception=

assertThrows(IllegalArgumentException.class,

()->{

thrownewIllegalArgumentException("amessage");});

assertEquals("amessage",exception.getMessage());

}

}

TheisexpectingIllegalArgumentExceptiontobethrown,andthisisactuallyhappeninginsidethislambdaexpression.Thefollowingscreenshotshowsthatthetestactuallysucceeds:

ConsoleoutputofassertThrowsexample

AssertingtimeoutsToassesstimeoutsinJUnit5tests,Jupiterprovidestwoassertions:assertTimeoutandassertTimeoutPreemptively.Ontheonehand,assertTimeout,allowsustoverifythetimeoutofagivenoperation.Inthisassertion,theexpectedtimeisdefinedusingtheclassDurationofthestandardJavapackagejava.time.

Wearegoingtoseeseveralrunningexamplestoclarifytheuseofthisassertionmethod.Inthefollowingclass,wefindtwotestsusingassertTimeout.Thefirsttestisdesignedtobesucceeded,duetothefactthatweareexpectingthatagivenoperationtakeslessthan2minutes,andwearedoingnothingthere.Ontheotherside,thesecondtestwillfail,sinceweareexpectingthatagivenoperationtakesamaximumof10milliseconds,andweareforcingittolast100milliseconds.

packageio.github.bonigarcia;

importstaticjava.time.Duration.ofMillis;

importstaticjava.time.Duration.ofMinutes;

importstaticorg.junit.jupiter.api.Assertions.assertTimeout;

importorg.junit.jupiter.api.Test;

classTimeoutExceededTest{

@Test

voidtimeoutNotExceeded(){

assertTimeout(ofMinutes(2),()->{

//Performtaskthattakeslessthan2minutes

});

}

@Test

voidtimeoutExceeded(){

assertTimeout(ofMillis(10),()->{

Thread.sleep(100);

});

}

}

Whenweexecutethistest,thesecondtestisdeclaredasfailedbecausethetimeouthasbeenexceededin90milliseconds:

ConsoleoutputofassertTimeoutfirstexample

Let’sseeacouplemoretestsusingassertTimeout.Inthefirsttest,assertTimeoutevaluatesapieceofcodeasalambdaexpressioninagiventimeout,obtainingitsresult.Inthesecondtest,assertTimeoutevaluatesamethodinagiventimeout,obtainingitsresult:

packageio.github.bonigarcia;

importstaticjava.time.Duration.ofMinutes;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.junit.jupiter.api.Assertions.assertTimeout;

importorg.junit.jupiter.api.Test;

classTimeoutWithResultOrMethodTest{

@Test

voidtimeoutNotExceededWithResult(){

StringactualResult=assertTimeout(ofMinutes(1),()->{

return"hithere";

});

assertEquals("hithere",actualResult);

}

@Test

voidtimeoutNotExceededWithMethod(){

StringactualGreeting=assertTimeout(ofMinutes(1),

TimeoutWithResultOrMethodTest::greeting);

assertEquals("helloworld!",actualGreeting);

}

privatestaticStringgreeting(){

return"helloworld!";

}

}

Inbothcases,theteststakelesstimethanexpectedandthereforebothofthemaresucceeded:

ConsoleoutputofassertTimeoutsecondexample

TheotherJupiterassertionfortimeoutsiscalledassertTimeoutPreemptively.ThedifferencewithassertTimeoutPreemptivelywithrespecttoassertTimeoutisthatassertTimeoutPreemptivelydoesnotwaituntiltheendoftheoperation,andtheexecutionisabortedwhentheexpectedtimeoutisexceeded.

Inthisexample,thetestwillfailsincewearesimulatinganoperationwhichlasts100milliseconds,andwehavedefinedatimeoutof10milliseconds:

packageio.github.bonigarcia;

importstaticjava.time.Duration.ofMillis;

importstaticorg.junit.jupiter.api.Assertions.assertTimeoutPreemptively;

importorg.junit.jupiter.api.Test;

classTimeoutWithPreemptiveTerminationTest{

@Test

voidtimeoutExceededWithPreemptiveTermination(){

assertTimeoutPreemptively(ofMillis(10),()->{

Thread.sleep(100);

});

}

}

Inthisexample,whenthetimeoutof10msisreached,instantlythetestisdeclaredasafailure:

ConsoleoutputofassertTimeoutPreemptivelyexample

Third-partyassertionlibrariesAswehaveseen,thebuilt-inassertionsprovidedoutoftheboxforJupiteraresufficientformanytestingscenarios.Nevertheless,therearetimeswhenmoreadditionalfunctionality,suchasmatchers,canbedesiredorrequired.Insuchsituations,theJUnitteamrecommendstheuseofthefollowingthird-partyassertionlibraries:

Hamcrest(http://hamcrest.org/):anassertionframeworktowritematcherobjectsallowingrulestobedefineddeclaratively.AssertJ(http://joel-costigliola.github.io/assertj/):fluentassertionsforJava.Truth(https://google.github.io/truth/):anassertionsJavalibrarydesignedtomaketestassertionsandfailuremessagesmorereadable.

Inthissection,wearegoingtomakeabriefreviewofHamcrest.ThislibraryprovidedtheassertionassertThat,whichallowstocreatereadablehighlyconfigurableassertions.ThemethodassertThatacceptstwoarguments:firsttheactualobject,andsecondaMatcherobject.Thismatcherimplementstheinterfaceorg.hamcrest.Matcher,andenablesapartialoranexactmatchforanexpectation.Hamcrestprovidesdifferentmatcherutilities,suchasis,either,or,not,andhasItem.TheMatchermethodsusethebuilderpattern,allowingtocombineoneormorematcherstobuildamatcherchain.

InordertouseHamcrest,firstweneedtoimportthedependencyinourproject.InaMavenproject,thismeansthatwehavetoincludethefollowingdependencyinourpom.xmlfile:

<dependency>

<groupId>org.hamcrest</groupId>

<artifactId>hamcrest-core</artifactId>

<version>${hamcrest.version}</version>

<scope>test</scope>

</dependency>

IfweareusingGradle,weneedtoaddtheequivalentconfigurationwithinthebuild.gradlefile:

dependencies{

testCompile("org.hamcrest:hamcrest-core:${hamcrest}")

}

Asusual,itisrecommendedusingthelatestversionofHamcrest.WecancheckitontheMavencentralweb(http://search.maven.org/).

ThefollowingexampledemonstrateshowtouseHamcrestinsideaJupitertest.Concretely,thistestusestheassertionassertThattogetherwiththematcherscontainsString,equalTo,andnotNullValue:

packageio.github.bonigarcia;

importstaticorg.hamcrest.CoreMatchers.containsString;

importstaticorg.hamcrest.CoreMatchers.equalTo;

importstaticorg.hamcrest.CoreMatchers.notNullValue;

importstaticorg.hamcrest.MatcherAssert.assertThat;

importorg.junit.jupiter.api.Test;

classHamcrestTest{

@Test

voidassertWithHamcrestMatcher(){

assertThat(2+1,equalTo(3));

assertThat("Foo",notNullValue());

assertThat("Helloworld",containsString("world"));

}

}

Asshowninthefollowingscreenshot,thistestisexecutedwithnofailure:

ConsoleoutputofexampleusingtheHamcrestassertionlibrary

TaggingandfilteringtestsTestclassesandmethodscanbetaggedintheJUnit5programmingmodelbymeansoftheannotation@Tag(packageorg.junit.jupiter.api).Thosetagscanlaterbeusedtofiltertestdiscoveryandexecution.Inthefollowingexample,weseetheuseof@Tagatclasslevelandalsoatmethodlevel:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

@Tag("simple")

classSimpleTaggingTest{

@Test

@Tag("taxes")

voidtestingTaxCalculation(){

}

}

AsofJUnit5M6,thelabelfortaggingtestsshouldmeetthefollowingsyntaxrules:

Atagmustnotbenullorblank.Atrimmedtag(thatis,tagsinwhichleadingandtrailingwhitespacehavebeenremoved)mustnotcontainawhitespace.AtrimmedtagmustnotcontainISOcontrolcharactersnorthefollowingreservedcharacters:,,(,),&,|,and!.

FilteringtestswithMavenAswealreadyknow,weneedtousemaven-surefire-plugininaMavenprojecttoexecuteJupitertest.Moreover,thispluginallowsustofilterthetestexecutioninseveralways:filteringbyJUnit5tagsandalsousingtheregularinclusion/exclusionsupportofmaven-surefire-plugin.

Inordertofilterbytags,thepropertiesincludeTagsandexcludeTagsofthemaven-surefire-pluginconfigurationshouldbeused.Let’sseeanexampletodemonstratehow.ConsiderthefollowingtestscontainedinthesameMavenproject.Ontheonehand,alltestsinthisclassaretaggedwiththefunctionalword.

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

@Tag("functional")

classFunctionalTest{

@Test

voidtestOne(){

System.out.println("FunctionalTest1");

}

@Test

voidtestTwo(){

System.out.println("FunctionalTest2");

}

}

Ontheotherhand,alltestsinthesecondclassaretaggedasnon-functionalandeachindividualtestisalsolabeledwithmoretags(performance,security,usability,andsoon):

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

@Tag("non-functional")

classNonFunctionalTest{

@Test

@Tag("performance")

@Tag("load")

voidtestOne(){

System.out.println("Non-FunctionalTest1(Performance/Load)");

}

@Test

@Tag("performance")

@Tag("stress")

voidtestTwo(){

System.out.println("Non-FunctionalTest2(Performance/Stress)");

}

@Test

@Tag("security")

voidtestThree(){

System.out.println("Non-FunctionalTest3(Security)");

}

@Test

@Tag("usability")

voidtestFour(){

System.out.println("Non-FunctionalTest4(Usability)");

}

}

Asdescribedbefore,weusetheconfigurationkeywordsincludeTagsandexcludeTagsintheMavenpom.xmlfile.Inthisexample,weincludethetestwiththetagfunctionalandexcludenon-functional:

<build>

<plugins>

<plugin>

<artifactId>maven-surefire-plugin</artifactId>

<version>${maven-surefire-plugin.version}</version>

<configuration>

<properties>

<includeTags>functional</includeTags>

<excludeTags>non-functional</excludeTags>

</properties>

</configuration>

<dependencies>

<dependency>

<groupId>org.junit.platform</groupId>

<artifactId>junit-platform-surefire-provider</artifactId>

<version>${junit.platform.version}</version>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-engine</artifactId>

<version>${junit.jupiter.version}</version>

</dependency>

</dependencies>

</plugin>

</plugins>

</build>

Asaresult,whenwetrytoexecuteallthetestswithintheproject,onlytwowillbeexecuted(thosewiththetagfunctional),andtherestarenotrecognizedastests:

Mavenexecutionoftestfilteringbytags

MavenregularsupportTheregularinclusion/exclusionsupportoftheMavenplugincanstillbeusedtoselectwhichtestsaregoingtobeexecutedbymaven-surefire-plugin.Tothataim,weusethekeywordsincludesandexcludestoconfigurethetestnamepatternusedtofiltertheexecutionbytheplugin.Noticethatforbothinclusionsandexclusions,regularexpressionscanbeusedtospecifyapatternofthetestfilenames:

<configuration>

<includes>

<include>**/Test*.java</include>

<include>**/*Test.java</include>

<include>**/*TestCase.java</include>

</includes>

</configuration>

<configuration>

<excludes>

<exclude>**/TestCircle.java</exclude>

<exclude>**/TestSquare.java</exclude>

</excludes>

</configuration>

Thesethreepatterns,thatis,theJavafilescontainingthewordTestorendingwithTestCase,areincludedbydefaultbyamaven-surefireplugin.

FilteringtestswithGradleLet’smovenowtoGradle.Aswealreadyknow,wecanalsouseGradletorunJUnit5tests.Regardingthefilteringprocess,wecanselectthetesttobeexecutedbasedon:

Thetestengine:Usingthekeywordengineswecanincludeorexcludethetestenginetobeused(thatisjunit-jupiterorjunit-vintage).TheJupitertags:Usingthekeywordtags.TheJavapackages:Usingthekeywordpackages.Theclassnamepatterns:UsingthekeywordincludeClassNamePattern.

Bydefault,allenginesandtagsareincludedinthetestplan.OnlytheclassnamecontainingthewordTestsisapplied.Let’sseeaworkingexample.WereusethesametestspresentedintheformerMavenproject,butthistimeinaGradleproject:

junitPlatform{

filters{

engines{

include'junit-jupiter'

exclude'junit-vintage'

}

tags{

include'non-functional'

exclude'functional'

}

packages{

include'io.github.bonigarcia'

exclude'com.others','org.others'

}

includeClassNamePattern'.*Spec'

includeClassNamePatterns'.*Test','.*Tests'

}

}

Noticethatweareincludingthetagsnon-functionalandexcludingfunctional,andthereforeweexecutefourtests:

Gradleexecutionoftestfilteringbytags

Meta-annotationsThefinalpartofthissectionisaboutthedefinitionofmeta-annotations.TheJUnitJupiterannotationscanbeusedinthedefinitionofotherannotations(thatis,canbeusedasmeta-annotations).Thatmeansthatwecandefineourowncomposedannotationthatwillautomaticallyinheritthesemanticsofitsmeta-annotations.ThisfeatureisveryconvenienttocreateourcustomtesttaxonomybyreusingtheJUnit5annotation@Tag.

Let’sseeanexample.Considerthefollowingclassificationfortestcases,inwhichweclassifyalltestsasfunctionalandnon-functional,andthenwemakeanotherlevelunderthenon-functionaltests:

Exampletaxonomyfortests(functionalandnon-functional)

Withthatschemeinmind,wearegoingtocreateourcustommeta-annotationsforleavesofthattreestructure:@Functional,@Security,@Usability,@Accessiblity,@Load,and@Stress.Noticethatineachannotationweareusingoneormore@Tagannotations,dependingonthestructurepreviouslydefined.First,wecanseethedeclarationof@Functional:

packageio.github.bonigarcia;

importjava.lang.annotation.ElementType;

importjava.lang.annotation.Retention;

importjava.lang.annotation.RetentionPolicy;

importjava.lang.annotation.Target;

importorg.junit.jupiter.api.Tag;

@Target({ElementType.TYPE,ElementType.METHOD})

@Retention(RetentionPolicy.RUNTIME)

@Tag("functional")

public@interfaceFunctional{

}

Then,wedefinetheannotation@Securitywithtagsnon-functionalandsecurity:packageio.github.bonigarcia;

importjava.lang.annotation.ElementType;

importjava.lang.annotation.Retention;

importjava.lang.annotation.RetentionPolicy;

importjava.lang.annotation.Target;

importorg.junit.jupiter.api.Tag;

@Target({ElementType.TYPE,ElementType.METHOD})

@Retention(RetentionPolicy.RUNTIME)

@Tag("non-functional")

@Tag("security")

public@interfaceSecurity{

}

Similarly,wedefinetheannotation@Load,butthistimetaggingwithnon-functional,performance,andload:

packageio.github.bonigarcia;

importjava.lang.annotation.ElementType;

importjava.lang.annotation.Retention;

importjava.lang.annotation.RetentionPolicy;

importjava.lang.annotation.Target;

importorg.junit.jupiter.api.Tag;

@Target({ElementType.TYPE,ElementType.METHOD})

@Retention(RetentionPolicy.RUNTIME)

@Tag("non-functional")

@Tag("performance")

@Tag("load")

public@interfaceLoad{

}

Finallywecreatetheannotation@Stress(withtagsnon-functional,performance,andstress):

packageio.github.bonigarcia;

importjava.lang.annotation.ElementType;

importjava.lang.annotation.Retention;

importjava.lang.annotation.RetentionPolicy;

importjava.lang.annotation.Target;

importorg.junit.jupiter.api.Tag;

@Target({ElementType.TYPE,ElementType.METHOD})

@Retention(RetentionPolicy.RUNTIME)

@Tag("non-functional")

@Tag("performance")

@Tag("stress")

public@interfaceStress{

}

Now,wecanuseourannotationstotag(andlaterfilter)tests.Forinstance,inthefollowingexampleweareusingtheannotation@Functionalatclasslevel:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Test;

@Functional

classFunctionalTest{

@Test

voidtestOne(){

System.out.println("Test1");

}

@Test

voidtestTwo(){

System.out.println("Test2");

}

}

Wecanalsooutannotationsatmethodlevel.Inthefollowingtest,weannotatethedifferenttests(methods)withdifferentannotations(@Load,@Stress,@Security,and@Accessibility):

packageio.github.bonigarcia;

importorg.junit.jupiter.api.Test;

classNonFunctionalTest{

@Test

@Load

voidtestOne(){

System.out.println("Test1");

}

@Test

@Stress

voidtestTwo(){

System.out.println("Test2");

}

@Test

@Security

voidtestThree(){

System.out.println("Test3");

}

@Test

@Usability

voidtestFour(){

System.out.println("Test4");

}

}

Allinall,wecanfilterthetestbysimplychangingtheincludedtags.Ontheonehand,wecanfilterbythetagfunctional.Noticethatinthiscase,onlytwotestsareexecuted.ThefollowingsnippetshowstheoutputofthiskindoffilteringusingMaven:

Filteringtestbytags(functional)usingMavenandthecommandline

Ontheotherhand,wecanalsofilterwithdifferenttags,suchasnon-functional.

Thefollowingpictureshowsanexampleofthistypeoffiltering,thistimeusingGradle.Asusual,wecanplaywiththeseexamplesbyforkingtheGitHubrepository(https://github.com/bonigarcia/mastering-junit5):

>

Filteringtestbytags(non-functional)usingGradleandthecommandline

ConditionaltestexecutionInordertoestablishcustomconditionsfortestexecution,weneedtousetheJUnit5extensionmodel(introducedinChapter2,What’snewinJUnit5,inthesectionTheextensionmodelofJUnit5).Concretely,weneedtousetheconditionalextensionpointcalledExecutionCondition.Thisextensioncanbeusedtodeactivateeitheralltestsinaclassorindividualtests.

Wearegoingtoseeaworkingexampleinwhichwecreateacustomannotationtodisabletestsbasedontheoperativesystem.Firstofall,wecreateacustomutilityenumerationtoselectoneoperativesystem(WINDOWS,MAC,LINUX,andOTHER):

packageio.github.bonigarcia;

publicenumOs{

WINDOWS,MAC,LINUX,OTHER;

publicstaticOsdetermine(){

Osout=OTHER;

StringmyOs=System.getProperty("os.name").toLowerCase();

if(myOs.contains("win")){

out=WINDOWS;

}

elseif(myOs.contains("mac")){

out=MAC;

}

elseif(myOs.contains("nux")){

out=LINUX;

}

returnout;

}

}

Then,wecreateanextensionofExecutionCondition.Inthisexample,theevaluationisdonebycheckingwhetherornotthecustomannotation@DisabledOnOsispresent.Whentheannotation@DisabledOnOsispresent,thevalueoftheoperativesystemiscomparedwiththecurrentplatform.Dependingontheresultofthatcondition,thetestisdisabledorenabled.

packageio.github.bonigarcia;

importjava.lang.reflect.AnnotatedElement;

importjava.util.Arrays;

importjava.util.Optional;

importorg.junit.jupiter.api.extension.ConditionEvaluationResult;

importorg.junit.jupiter.api.extension.ExecutionCondition;

importorg.junit.jupiter.api.extension.ExtensionContext;

importorg.junit.platform.commons.util.AnnotationUtils;

publicclassOsConditionimplementsExecutionCondition{

@Override

publicConditionEvaluationResultevaluateExecutionCondition(

ExtensionContextcontext){

Optional<AnnotatedElement>element=context.getElement();

ConditionEvaluationResultout=ConditionEvaluationResult

.enabled("@DisabledOnOsisnotpresent");

Optional<DisabledOnOs>disabledOnOs=AnnotationUtils

.findAnnotation(element,DisabledOnOs.class);

if(disabledOnOs.isPresent()){

OsmyOs=Os.determine();

if(Arrays.asList(disabledOnOs.get().value())

.contains(myOs)){

out=ConditionEvaluationResult

.disabled("Testisdisabledon"+myOs);

}

else{

out=ConditionEvaluationResult

.enabled("Testisnotdisabledon"+myOs);

}

}

System.out.println("-->"+out.getReason().get());

returnout;

}

}

Moreover,weneedtocreateourcustomannotation@DisabledOnOs,whichisalsoannotatedwith@ExtendWithpointingtoourextensionpoint.

packageio.github.bonigarcia;

importjava.lang.annotation.ElementType;

importjava.lang.annotation.Retention;

importjava.lang.annotation.RetentionPolicy;

importjava.lang.annotation.Target;

importorg.junit.jupiter.api.extension.ExtendWith;

@Target({ElementType.TYPE,ElementType.METHOD})

@Retention(RetentionPolicy.RUNTIME)

@ExtendWith(OsCondition.class)

public@interfaceDisabledOnOs{

Os[]value();

}

Finally,weuseourannotation@DisabledOnOsinaJupitertest.importorg.junit.jupiter.api.Test;

importstaticio.github.bonigarcia.Os.MAC;

importstaticio.github.bonigarcia.Os.LINUX;

classDisabledOnOsTest{

@DisabledOnOs({MAC,LINUX})

@Test

voidconditionalTest(){

System.out.println("ThistestwillbedisabledonMACandLINUX");

}

}

IfweexecutethistestinaWindowsmachine,thetestisnotskipped,aswecanseeinthissnapshot:

Executionofconditionaltestexample

AssumptionsInthispartofthissectionisabouttheso-calledassumptions.Assumptionsallowustoonlyruntestsifcertainconditionsareasexpected.AllJUnitJupiterassumptionsarestaticmethodsintheclassAssumptions,locatedinsidetheorg.junit.jupiterpackage.Thefollowingscreenshotshowsallthemethodsofthisclass:

Methodsoftheclassorg.junit.jupiter.Assumptions

Ontheonehand,themethodsassumeTrueandassumeFalsecanbeusedtoskiptestswhosepreconditionsarenotmet.Ontheotherhand,themethodassumingThatisusedtoconditiontheexecutionofapartinatest:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.fail;

importstaticorg.junit.jupiter.api.Assumptions.assumeFalse;

importstaticorg.junit.jupiter.api.Assumptions.assumeTrue;

importstaticorg.junit.jupiter.api.Assumptions.assumingThat;

importorg.junit.jupiter.api.Test;

classAssumptionsTest{

@Test

voidassumeTrueTest(){

assumeTrue(false);

fail("Test1failed");

}

@Test

voidassumeFalseTest(){

assumeFalse(this::getTrue);

fail("Test2failed");

}

privatebooleangetTrue(){

returntrue;

}

@Test

voidassummingThatTest(){

assumingThat(false,()->fail("Test3failed"));

}

}

Noticethatinthisexample,thetwofirsttests(assumeTrueTestandassumeFalseTest)areskippedsincetheassumptionsarenotmet.Nevertheless,intheassummingThatTesttest,onlythispartofthetest(alambdaexpressioninthiscase)isnotexecuted,butthewholetestisnotskipped:

Executionofassumptionstestexample

NestedtestsNestedtestsgivethetestwritermorecapabilitiestoexpresstherelationshipandorderinagroupoftests.JUnit5makesiteffortlesstonesttestclasses.Wesimplyneedtoannotateinnerclasseswith@Nestedandalltestmethodsintherewillbeexecutedaswell,goingfromtheregulartests(definedinthetop-levelclass)tothetestsdefinedineachoftheinnerclasses.

Thefirstthingweneedtotakeintoaccountisthatonlynon-staticnestedclasses(thatisinnerclasses)canserveas@Nestedtests.Nestingcanbearbitrarilydeep,andthesetupandteardownforeachtest(thatis,@BeforeEachand@AfterEachmethods)areinheritedinthenestedtests.Nevertheless,innerclassescannotdefinethe@BeforeAlland@AfterAllmethods,duetothefactthatJavadoesnotallowstaticmembersininnerclasses.However,thisrestrictioncanbeavoidedusingtheannotation@TestInstance(Lifecycle.PER_CLASS)inthetestclass.AsdescribedinthesectionTestinstancelifecycleinthischapter,thisannotationforcetoinstanceatestinstanceperclass,insteadofatestinstancepermethod(defaultbehavior).Thisway,themethods@BeforeAlland@AfterAlldonotneedtobestaticandthereforeitcanbeusedinnestedtests.

Let’sseeasimpleexamplecomposedbyaJavaclasswithtwolevelsofinnerclasses,thatis,theclasscontainstwonestedinnerclassesannotatedwith@Nested.Aswecansee,therearetestsinthethreelevelsoftheclass.Noticethatthetopclassdefinedasetupmethod(@BeforeEach),andalsothefirstnestedclass(calledInnerClass1intheexample).Inthetop-levelclass,wedefineasingletest(calledtopTest),andineachnestedclasswefindanothertest(calledinnerTest1andinnerTest2,respectively):

packageio.github.bonigarcia;

importorg.junit.jupiter.api.BeforeEach;

importorg.junit.jupiter.api.Nested;

importorg.junit.jupiter.api.Test;

classNestTest{

@BeforeEach

voidsetup1(){

System.out.println("Setup1");

}

@Test

voidtopTest(){

System.out.println("Test1");

}

@Nested

classInnerClass1{

@BeforeEach

voidsetup2(){

System.out.println("Setup2");

}

@Test

voidinnerTest1(){

System.out.println("Test2");

}

@Nested

classInnerClass2{

@Test

voidinnerTest2(){

System.out.println("Test3");

}

}

}

}

Ifweexecutethisexample,wecantracetheexecutionofthenestedtestsbysimplylookingtotheconsoletraces.Notethatthetop@BeforeEachmethod(calledsetup1)isalwaysexecutedbeforeeachtest.Therefore,thetraceSetup1isalwayspresentintheconsolebeforetheactualtestexecution.Eachtestalsowritesalinetheconsole.Aswecansee,thefirsttestlogsTest1.Afterthat,thetestsdefinedintheinnerclassesareexecuted.ThefirstinnerclassexecutesthetestinnerTest1,butafterthat,thesetupmethodofthetop-levelclassandthefirstinnerclassareexecuted(loggingSetup1andSetup2,respectively).

Finally,thetestdefinedinthelastinnerclass(innerTest2)isexecuted,butasusual,thecascadeofsetupmethodsisexecutedbeforethetest:

Consoleoutputoftheexecutionofthenestedtestexample

Nestedtestscanbeusedinconjunctionwiththedisplayname(thatis,the

annotation@DisplayName)tohelptoproduceanicelyreadabletestoutput.Thefollowingexampledemonstrateshow.Thisclasscontainsthestructuretotesttheimplementationofastack,thatis,alast-in-first-out(LIFO)collection.Theclassisdesignedtofirsttestthestackwhenitisjustinstantiated(themethodisInstantiatedWithNew).Afterthat,thefirstinnerclass(WhenNew)issupposedtotestthestackasanemptycollection(methodsisEmpty,throwsExceptionWhenPoppedandthrowsExceptionWhenPeeked).Finally,thesecondinnerclassissupposedtotestwhenthestackisnotempty(methodsisNotEmpty,returnElementWhenPopped,andreturnElementWhenPeeked):

packageio.github.bonigarcia;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Nested;

importorg.junit.jupiter.api.Test;

@DisplayName("Astacktest")

classStackTest{

@Test

@DisplayName("isinstantiated")

voidisInstantiated(){

}

@Nested

@DisplayName("whenempty")

classWhenNew{

@Test

@DisplayName("isempty")

voidisEmpty(){

}

@Test

@DisplayName("throwsExceptionwhenpopped")

voidthrowsExceptionWhenPopped(){

}

@Test

@DisplayName("throwsExceptionwhenpeeked")

voidthrowsExceptionWhenPeeked(){

}

@Nested

@DisplayName("afterpushinganelement")

classAfterPushing{

@Test

@DisplayName("itisnolongerempty")

voidisNotEmpty(){

}

@Test

@DisplayName("returnstheelementwhenpopped")

voidreturnElementWhenPopped(){

}

@Test

@DisplayName("returnstheelementwhenpeeked")

voidreturnElementWhenPeeked(){

}

}

}

}

Theobjectiveofthistypeoftestistwofolded.Ontheonehand,theclassstructureprovidesanorderfortheexecutionofthetests.Ontheotherhand,theuseof@DisplayNameimprovesthereadabilityofthetestexecution.WecanseethatwhenthetestisexecutedinanIDE,concretelyinIntelliJIDEA.

Executionofnestedtestusing@DisplayNameonIntellijIDEA

RepeatedtestsJUnitJupiterprovidesfortheabilitytorepeatatestaspecifiednumberoftimessimplybyannotatingamethodwith@RepeatedTest,specifyingthetotalnumberofrepetitionsdesired.Eachrepeatedtestbehavesexactlyasaregular@Testmethod.Moreover,eachrepeatedtestpreservesthesamelifecyclecallbacks(@BeforeEach,@AfterEach,andsoon).

ThefollowingJavaclasscontainsatestthatisgoingtoberepeatedfivetimes:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.RepeatedTest;

classSimpleRepeatedTest{

@RepeatedTest(5)

voidtest(){

System.out.println("Repeatedtest");

}

}

Duetothefactthatthistestonlywritesaline(Repeatedtest)inthestandardoutput,whenexecutingthistestintheconsole,wewillseethattracefivetimes:

Executionofrepeatedtestintheconsole

Inadditiontospecifyingthenumberofrepetitions,acustomdisplaynamecanbeconfiguredforeachrepetitionviathenameattributeofthe@RepeatedTestannotation.Thedisplaynamecanbeapatterncomposedofacombinationofstatictextanddynamicplaceholders.Thefollowingarecurrentlysupported:

{displayName}:Thisisthenameofthe@RepeatedTestmethod.

{currentRepetition}:Thisisthecurrentrepetitioncount.{totalRepetitions}:Thisisthetotalnumberofrepetitions.

Thefollowingexampleshowsaclasswiththreerepeatedtestsinwhichthedisplaynameisconfiguredwiththepropertynameof@RepeatedTest:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.RepeatedTest;

importorg.junit.jupiter.api.TestInfo;

classTunningDisplayInRepeatedTest{

@RepeatedTest(value=2,name="{displayName}

{currentRepetition}/{totalRepetitions}")

@DisplayName("Repeat!")

voidcustomDisplayName(TestInfotestInfo){

System.out.println(testInfo.getDisplayName());

}

@RepeatedTest(value=2,name=RepeatedTest.LONG_DISPLAY_NAME)

@DisplayName("Testusinglongdisplayname")

voidcustomDisplayNameWithLongPattern(TestInfotestInfo){

System.out.println(testInfo.getDisplayName());

}

@RepeatedTest(value=2,name=RepeatedTest.SHORT_DISPLAY_NAME)

@DisplayName("Testusingshortdisplayname")

voidcustomDisplayNameWithShortPattern(TestInfotestInfo){

System.out.println(testInfo.getDisplayName());

}

}

Inthistest,thedisplaynamefortheserepeatedtestswillbeasfollows:

ForthetestcustomDisplayName,thedisplaynamewillfollowthelongdisplayformat:

Repeat1outof2.Repeat2outof2.

ForthetestcustomDisplayNameWithLongPattern,thedisplaynamewillfollowthelongdisplayformat:

Repeat!1/2.Repeat!2/2.

ForthetestcustomDisplayNameWithShortPattern,thedisplaynameinthistestwillfollowtheshortdisplayformat:

Testusinglongdisplayname::repetition1of2.Testusinglongdisplayname::repetition2of2.

Executionofrepeatedtestexampleinconjunctionwith@DisplayName

MigrationfromJUnit4toJUnit5JUnit5doesnotsupportJUnit4features,suchasRulesandRunners,natively.Nevertheless,JUnit5providesagentlemigrationpathviatheJUnitVintagetestengine,whichallowsustoexecutelegacytestcases(includingJUnit4butalsoJUnit3)onthetopoftheJUnitPlatform.

ThefollowingtablecanbeusedtosummarizethemaindifferencesbetweenJUnit4and5:

Feature JUnit4 JUnit5

Annotationspackage

org.junit org.junit.jupiter.api

Declaringatest @Test @Test

Setupforalltests @BeforeClass @BeforeAll

Setuppertest @Before @BeforeEach

Teardownpertest @After @AfterEach

Teardownforalltests

@AfterClass @AfterAll

Taggingandfiltering

@Category @Tag

Disableatestmethodorclass

@Ignore @Disabled

Nestedtests NA @Nested

Repeatedtest Usingcustomrule

@Repeated

Dynamictests NA @TestFactory

Testtemplates NA @TestTemaplate

Runners @RunWithThisfeatureissupersededbytheextensionmodel(@ExtendWith)

Rules @Ruleand@ClassRule

Thisfeatureissupersededbytheextensionmodel(@ExtendWith)

RulesupportinJupiterAsdescribedbefore,JupiterdoesnotsupportJUnit4rulesnatively.Nevertheless,theJUnit5teamrealizedthatJUnit4rulesarewidelyadoptedinmanytestcodebasesnowadays.InordertoprovideaseamlessmigrationfromJUnit4toJUnit5,theJUnit5teamimplementedthejunit-jupiter-migrationsupportmodule.Ifthismoduleisgoingtobeusedinaproject,themoduledependencyshouldbeimported.ExamplesforMavenareshownhere:

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-migrationsupport</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

TheGradledeclarationforthisdependencyislikethis:dependencies{

testCompile("org.junit.jupiter:junit-jupiter-

migrationsupport:${junitJupiterVersion}")

}

TherulesupportinJUnit5islimitedtothoserulessemanticallycompatiblewiththeJupiterextensionmodel,includingthefollowingrules:

junit.rules.ExternalResource(includingorg.junit.rules.TemporaryFolder).junit.rules.Verifier(includingorg.junit.rules.ErrorCollector).junit.rules.ExpectedException.

InordertoenabletheserulesinJupitertests,thetestclassshouldbeannotatedwiththeclass-levelannotation@EnableRuleMigrationSupport(locatedinthepackageorg.junit.jupiter.migrationsupport.rules).Letusseeseveralexamples.First,thefollowingtestcasedefinesandusesaTemporaryFolderJUnit4rulewithinaJupitertest:

packageio.github.bonigarcia;

importjava.io.IOException;

importorg.junit.Rule;

importorg.junit.jupiter.api.AfterEach;

importorg.junit.jupiter.api.BeforeEach;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.migrationsupport.rules.EnableRuleMigrationSupport;

importorg.junit.rules.TemporaryFolder;

@EnableRuleMigrationSupport

classTemporaryFolderRuleTest{

@Rule

TemporaryFoldertemporaryFolder=newTemporaryFolder();

@BeforeEach

voidsetup()throwsIOException{

temporaryFolder.create();

}

@Test

voidtest(){

System.out.println("Temporaryfolder:"+

temporaryFolder.getRoot());

}

@AfterEach

voidteardown(){

temporaryFolder.delete();

}

}

Whenexecutingthistest,thepathofthetemporaryfolderwillbeloggedonthestandardoutput:

ExecutionofJupitertestusingaJUnit4TemporaryFolderrule

ThefollowingtestdemonstratestheuseoftheErrorCollectorruleinaJupitertest.Noticethatthecollectorruleallowstheexecutionofatesttocontinueafteroneormoreproblemsarefound:

packageio.github.bonigarcia;

importstaticorg.hamcrest.CoreMatchers.equalTo;

importorg.junit.Rule;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.migrationsupport.rules.EnableRuleMigrationSupport;

importorg.junit.rules.ErrorCollector;

@EnableRuleMigrationSupport

classErrorCollectorRuleTest{

@Rule

publicErrorCollectorcollector=newErrorCollector();

@Test

voidtest(){

collector.checkThat("a",equalTo("b"));

collector.checkThat(1,equalTo(2));

collector.checkThat("c",equalTo("c"));

}

}

Theseproblemsarereportedtogetherattheendofthetest:

ExecutionofJupitertestusingaJUnit4ErrorCollectorrule

Finally,theExpectedExceptionruleallowsustoconfigureatesttoanticipateagivenexceptiontobethrownwithinthetestlogic:

packageio.github.bonigarcia;

importorg.junit.Rule;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.migrationsupport.rules.EnableRuleMigrationSupport;

importorg.junit.rules.ExpectedException;

@EnableRuleMigrationSupport

classExpectedExceptionRuleTest{

@Rule

ExpectedExceptionthrown=ExpectedException.none();

@Test

voidthrowsNothing(){

}

@Test

voidthrowsNullPointerException(){

thrown.expect(NullPointerException.class);

thrownewNullPointerException();

}

}

Inthisexample,evenwhenthesecondtestraisesaNullPointerException,thetestwillbemarkedashavingsucceededsincethatexceptionwasexpected.

ExecutionofJupitertestusingaJUnit4ExpectedExceptionrule

SummaryInthischapter,weintroducedthebasicsofthebrand-newprogrammingmodeloftheJUnit5framework,knownasJupiter.ThisprogrammingmodelprovidesarichAPIthatcanbeusedbypractitionerstocreatetestcases.ThemostbasicelementofJupiteristheannotation@Test,whichidentifiesthemethodsinJavaclassestreatedastests(thatislogicwhichexercisesandverifiesaSUT).Moreover,therearedifferentannotationsthatcanbeusedtocontrolthetestlifecycle,namely,@BeforeAll,@BeforeEach,@AfterEach,and@AfterAll.OtherusefulJupiterannotationsare@Disabled(toskiptests),@DisplayName(toprovideatestname),@Tag(tolabelandfiltertests).

Jupiterprovidesarichsetofassertions,whicharestaticmethodsintheclassAssertionsusedtoverifyiftheoutcomeobtainedfromtheSUTcorrespondswithsomeexpectedvalue.Wecanimposeconditionsforthetestexecutioninseveralways.Ontheonehand,wecanuseAssumptionstoonlyruntests(orapartofthose)ifcertainconditionsareasexpected.

WehavelearnedhownestedtestscanbecreatedsimpleannotatinginnerJavaclasseswith@Nested.Thiscanbeusedtocreatetestexecutionsfollowinganordergiventhenestedclassesrelationship.WehavealsostudiedhoweasyistocreatedrepeatedtestusingtheJUnit5programmingmodel.Theannotation@RepeatedTestisusedtothataim,providingtheabilitytorepeatatestaspecifiednumberoftimes.Finally,wehaveseenhowJupiterprovidessupportforseverallegacyJUnit4testrules,includingExternalResource,Verifier,andExpectedException.

Inthechapter4,SimplifyingTestingWithAdvancedJUnitFeatures,wecontinuediscoveringtheJUnitprogrammingmodel.Concretely,wereviewtheadvancefeaturesofJUnit5,namely,dependencyinjection,dynamictests,testinterfaces,testtemplates,parameterizedtests,compatibilityofJUnit5andJava9.Finally,wereviewsomeoftheplannedfeaturesinthebacklogforJUnit5.1,notimplementedyetatthetimeofthiswriting.

SimplifyingTestingWithAdvancedJUnitFeatures

Simplicityistheultimatesophistication.-LeonardodaVinci

Sofar,wehavediscoveredthebasicsofJupiter,thebrand-newprogrammingmodelprovidedbytheJUnit5framework.Moreover,Jupiterprovidesarichrangeofpossibilitieswhichallowstocreatedifferenttypesoftestcases.Inthischapter,wereviewtheseadvancedfeatures.Tothataim,thischapterisstructuredasfollows:

Dependencyinjection:Thissectionfirsttakesalookatdependencyinjectionforconstructorsandmethodsintestclasses.Then,itreviewsthethreeparameterresolversprovidedoutoftheboxinJupiter.TheseresolversallowtoinjectobjectsofTestInfo,RepetitionInfo,andTestReporterinsidetests.Dynamictests:ThissectiondiscusseshowdynamictestsareimplementedinJUnit5,usingthemethodsdynamicTestandstream.Testinterfaces:ThesectionreviewstheJupiterannotationsthatcanbedeclaredontestinterfacesanddefaultmethods.Testtemplates:JUnit5introducestheconceptofatemplatefortestscases.Thesetemplateswillbeinvokedmultipletimes,dependingontheinvocationcontexts.Parameterizedtests:InthesamewayasJUnit4,JUnit5providescapabilitiestocreatetestsdrivenbydifferentinputdata,thatis,aparametrizedtest.WewilldiscoverthatthesupportforthiskindoftesthasbeensignificantlyenhancedintheJupiterprogrammingmodel.Java9:OnSeptember21,2017,Java9released.Aswewilldiscover,JUnit5hasbeenimplementedtobecompatiblewithJava9,withspecialemphasisonthemodularityfeatureofJava9.

DependencyinjectionInformerJUnitversions,testconstructorsandmethodswerenotallowedtohaveparameters.OneofthemajorchangesinJUnit5isthatbothtestconstructorsandmethodsarenowallowedtoincludeparameters.Thisfeatureenablesthedependencyinjectionforconstructorsandmethods.

AsintroducedinChapter2,What’sNewInJUnit5ofthisbook,theextensionmodelhasanextensionthatprovidesdependencyinjectionsforJupitertests,calledParameterResolver,whichdefinesanAPIfortestextensionsthatwishtodynamicallyresolveparametersatruntime.

Ifatestconstructororamethodannotatedwith@Test,@TestFactory,@BeforeEach,@AfterEach,@BeforeAll,or@AfterAllacceptsaparameter,thatparameterisresolvedatruntimebyaresolver(objectwithparentclassParameterResolver).Therearethreebuilt-inresolversregisteredautomaticallyinJUnit5:TestInfoParameterResolver,andRepetitionInfoParameterResolver,TestReporterParameterResolver.Werevieweachoneoftheseresolversinthissection.

TestInfoParameterResolverGivenatestclass,ifamethodparameterisoftypeTestInfo,theJUnit5resolverTestInfoParameterResolversuppliesaninstanceofTestInfocorrespondingtothecurrenttestasthevalueforthedeclaredparameter.TheTestInfoobjectisusedtoretrieveinformationaboutthecurrenttest,suchasthetestdisplayname,thetestclass,thetestmethod,orassociatedtags.

TestInfoactsasadrop-inreplacementfortheTestNamerulefromJUnit4.

TheclassTestInfoisplacedinthepackageorg.junit.jupiter.apiandoffersthefollowingAPI:

StringgetDisplayName():Thisreturnsthedisplaynameofthetestorcontainer.Set<String>getTags():Thisgetsthesetofalltagsforthecurrenttestorcontainer.Optional<Class<?>>getTestClass():Thisgetstheclassassociatedwiththecurrenttestorcontainer,ifavailable.Optional<Method>getTestMethod():Thisgetsthemethodassociatedwiththecurrenttest,ifavailable.

TestInfoAPI

Let’sseeanexample.Noticethatinthefollowingclass,boththemethodsannotatedwith@BeforeEachand@TestacceptsaparameterofTestInfo.ThisparameterisinjectedbyTestInfoParameterResolver:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.BeforeEach;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.TestInfo;

classTestInfoTest{

@BeforeEach

voidinit(TestInfotestInfo){

StringdisplayName=testInfo.getDisplayName();

System.out.printf("@BeforeEach%s%n",displayName);

}

@Test

@DisplayName("Mytest")

@Tag("my-tag")

voidtestOne(TestInfotestInfo){

System.out.println(testInfo.getDisplayName());

System.out.println(testInfo.getTags());

System.out.println(testInfo.getTestClass());

System.out.println(testInfo.getTestMethod());

}

@Test

voidtestTwo(){

}

}

Therefore,inthebodyofeachmethod,weareabletousetheTestInfoAPItogetthetestinformationatruntime,asthefollowingscreenshotdemonstrates:

ConsoleoutputofdependencyinjectionofTestInfoobjects

RepetitionInfoParameterResolverThesecondresolverprovidedoutoftheboxinJUnit5iscalledRepetitionInfoParameterResolver.Givenatestclass,ifamethodparameterina@RepeatedTest,@BeforeEach,or@AfterEachmethodisoftypeRepetitionInfo,theRepetitionInfoParameterResolverwillsupplyaninstanceofRepetitionInfo.

RepetitionInfocanbeusedtoretrieveinformationaboutthecurrentrepetitionandthetotalnumberofrepetitionsforthecorresponding@RepeatedTest.TheAPIofRepetitionInfoofferstwomethods,asshowninthescreenshotafterthelist:

intgetCurrentRepetition():Getsthecurrentrepetitionofthecorresponding@RepeatedTestmethodintgetTotalRepetitions():Getsthetotalnumberofrepetitionsofthecorresponding@RepeatedTestmethod

RepetitionInfoAPI

TheclassherecontainsasimpleexamplefortheuseofRepetitionInfo:packageio.github.bonigarcia;

importorg.junit.jupiter.api.RepeatedTest;

importorg.junit.jupiter.api.RepetitionInfo;

classRepetitionInfoTest{

@RepeatedTest(2)

voidtest(RepetitionInforepetitionInfo){

System.out.println("**Test"+

repetitionInfo.getCurrentRepetition()

+"/"+repetitionInfo.getTotalRepetitions());

}

}

Ascanbeseeninthetestoutput,weareabletoreadtheinformationabouttherepeatedtestatruntime:

TheconsoleoutputofdependencyinjectionofRepetitionInfoobjects.

TestReporterParameterResolverThelastbuilt-inresolverinJUnit5isTestReporterParameterResolver.Again,givenatestclass,ifamethodparameterisoftypeTestReporter,theTestReporterParameterResolversuppliesaninstanceofTestReporter.

TestReporterisusedtopublishadditionaldataaboutthetestexecution.ThedatacanbeconsumedthroughthemethodreportingEntryPublished,andthen,itcanberequestedbyIDEsorincludedintestreports.EachTestReporterobjectstoresinformationasamap,thatis,akey-valuecollection:

TestReporterAPI

ThistestprovidesasimpleexampleofTestReporter.Aswecansee,weusetheinjectedtestReporterobjecttoaddcustominformationusingkey-valuepairs:

packageio.github.bonigarcia;

importjava.util.HashMap;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.TestReporter;

classTestReporterTest{

@Test

voidreportSingleValue(TestReportertestReporter){

testReporter.publishEntry("key","value");

}

@Test

voidreportSeveralValues(TestReportertestReporter){

HashMap<String,String>values=newHashMap<>();

values.put("name","john");

values.put("surname","doe");

testReporter.publishEntry(values);

}

}

DynamictestsAsweknow,inJUnit3,weidentifiedtestsbyparsingmethodnamesandcheckingwhethertheystartedwiththewordtest.Then,inJUnit4,weidentifiedtestsbycollectingmethodsannotatedwith@Test.Bothofthesetechniquessharethesameapproach:testsaredefinedatcompiletime.Thisconceptiswhatwecallstatictesting.

Statictestsareconsideredalimitedapproach,especiallyforthecommonscenarioinwhichthesametestissupposedtobeexecutedforavarietyofinputdata.InJUnit4,thislimitationwasaddressedinseveralways.Averysimplesolutiontotheproblemistolooptheinputtestdataandexercisingthesametestlogic(JUnit4examplehere).Followingthisapproach,onetestisexecuteduntilthefirstassertionfails:

packageio.github.bonigarcia;

importorg.junit.Test;

publicclassMyTest{

@Test

publicvoidtest(){

String[]input={"A","B","C"};

for(Strings:input){

exercise(s);

}

}

privatevoidexercise(Strings){

System.out.println(s);

}

}

AmoreelaboratesolutionistousetheJUnit4supportforparameterizedtests,usingtheparameterizedrunner.Thisapproachdoesnotcreatetestsatruntimeeither,itsimplyrepeatsthesametestseveraltimesdependingontheparameters:

packageio.github.bonigarcia;

importjava.util.Arrays;

importjava.util.Collection;

importorg.junit.Test;

importorg.junit.runner.RunWith;

importorg.junit.runners.Parameterized;

importorg.junit.runners.Parameterized.Parameter;

importorg.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)

publicclassParameterizedTest{

@Parameter(0)

publicIntegerinput1;

@Parameter(1)

publicStringinput2;

@Parameters(name="Mytest#{index}--inputdata:{0}and{1}")

publicstaticCollection<Object[]>data(){

returnArrays

.asList(newObject[][]{{1,"hello"},{2,"goodbye"}});

}

@Test

publicvoidtest(){

System.out.println(input1+""+input2);

}

}

WecanseetheexecutionoftheprecedingexampleintheEclipseIDE:

ExecutionofJUnit4’sparameterizedtestinEclipse

Ontheotherhand,JUnit5allowstogeneratetestatruntimebyafactorymethodthatisannotatedwith@TestFactory.Incontrastto@Test,a@TestFactorymethodisnotatestbutafactory.A@TestFactorymethodmustreturnaStream,Collection,Iterable,orIteratorofDynamicTestinstances.TheseDynamicTestinstancesareexecutedlazily,enablingdynamicgenerationoftestcases.

Inordertocreateadynamictest,wecanusethestaticmethoddynamicTestoftheclassDynamicTestlocatedintheorg.junit.jupiter.apipackage.Ifweinspectthesourcecodeofthisclass,wecanseethataDynamicTestiscomposedofadisplaynameinformoftheStringandoneexecutableobject,whichcanbeprovidedaslambdaexpressionsorasmethodreferences.

Let’sseeseveralexamplesofdynamictests.Inthefollowingexample,thefirstdynamictestwillfail,duetothefactwearenotreturningtheexpectedcollectionofDynamicTests.ThenextthreemethodsareverysimpleexamplesthatdemonstratethegenerationofCollection,Iterable,andIteratorofDynamicTestinstances:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importstaticorg.junit.jupiter.api.DynamicTest.dynamicTest;

importjava.util.Arrays;

importjava.util.Collection;

importjava.util.Iterator;

importjava.util.List;

importorg.junit.jupiter.api.DynamicTest;

importorg.junit.jupiter.api.TestFactory;

classCollectionTest{

//Warning:thistestwillraiseanexception

@TestFactory

List<String>dynamicTestsWithInvalidReturnType(){

returnArrays.asList("Hello");

}

@TestFactory

Collection<DynamicTest>dynamicTestsFromCollection(){

returnArrays.asList(

dynamicTest("1stdynamictest",()->

assertTrue(true)),

dynamicTest("2nddynamictest",()->assertEquals(4,2

*2)));

}

@TestFactory

Iterable<DynamicTest>dynamicTestsFromIterable(){

returnArrays.asList(

dynamicTest("3rddynamictest",()->

assertTrue(true)),

dynamicTest("4thdynamictest",()->assertEquals(4,2

*2)));

}

@TestFactory

Iterator<DynamicTest>dynamicTestsFromIterator(){

returnArrays.asList(

dynamicTest("5thdynamictest",()->

assertTrue(true)),

dynamicTest("6thdynamictest",()->assertEquals(4,2

*2))).iterator();

}

}

Theseexamplesdonotreallyexhibitdynamicbehavior,butmerelydemonstratethesupportedreturntypes.NotethatthefirsttestisgoingtofailduetoJUnitException:

Consoleoutputofthefirstexamplefordynamictestexecution

Thefollowingexampledemonstrateshoweasyitistogeneratedynamictestsforagivensetofinputdata:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.DynamicTest.dynamicTest;

importjava.util.stream.Stream;

importorg.junit.jupiter.api.DynamicTest;

importorg.junit.jupiter.api.TestFactory;

classDynamicExampleTest{

@TestFactory

Stream<DynamicTest>dynamicTestsFromStream(){

Stream<String>inputStream=Stream.of("A","B","C");

returninputStream.map(

input->dynamicTest("Displaynameforinput"+input,

()->{

System.out.println("Testing"+input);

}));

}

}

Noticethat,intheend,threetestswereexecuted,andthesethreetestswerecreatedatruntimebyJUnit5:

Consoleoutputofthesecondexamplefordynamictestexecution

ThereisanotherpossibilitytocreatedynamictestsinJUnit5,usingthestaticmethodstreamoftheclassDynamicTest.Thismethodneedsaninputgenerator,afunctionthatgeneratesadisplaynamebasedonaninputvalue,andatestexecutor.

Let’sseeanotherexample.Wecreateatestfactory,providingtheinputdataasanIterator,adisplaynamefunctionusingalambdaexpression,andfinally,atestexecutorimplementedwithanotherlambdaexpression.Inthisexample,thetestexecutorbasicallyassertswhetherornottheinputintegerisevenorodd:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importstaticorg.junit.jupiter.api.DynamicTest.stream;

importjava.util.Arrays;

importjava.util.Iterator;

importjava.util.function.Function;

importjava.util.stream.Stream;

importorg.junit.jupiter.api.DynamicTest;

importorg.junit.jupiter.api.TestFactory;

importorg.junit.jupiter.api.function.ThrowingConsumer;

classStreamExampleTest{

@TestFactory

Stream<DynamicTest>streamTest(){

//Inputdata

Integerarray[]={1,2,3};

Iterator<Integer>inputGenerator=Arrays.asList(array).iterator();

//Displaynames

Function<Integer,String>displayNameGenerator=(

input)->"Datainput:"+input;

//Testexecutor

ThrowingConsumer<Integer>testExecutor=(input)->{

System.out.println(input);

assertTrue(input%2==0);

};

//Returnsastreamofdynamictests

returnstream(inputGenerator,displayNameGenerator,

testExecutor);

}

}

Thetestwillfailforoddinputs.Aswecansee,twooutofthreetestswillfail:

Consoleoutputofdynamictestexecution(examplethree)

TestinterfacesInJUnit5,therearedifferentrulesrelativetotheuseofannotationsinJavainterfaces.Firstofall,weneedtobeawarethat@Test,@TestFactory,@BeforeEach,and@AfterEachcanbedeclaredoninterfacedefaultmethods.

DefaultmethodsisafeatureofJavaintroducedinversion8.Thesemethods(declaredusingthereservekeyworddefault)allowstodefineadefaultimplementationforagivenmethodwithinaJavainterface.Thiscapabilitycanbeusefulforbackwardcompatibilitywithexistinginterfaces.

ThesecondruleregardingJUnit5andinterfacesisthat@BeforeAlland@AfterAllcanbedeclaredonstaticmethodsinatestinterface.Moreover,ifthetestclass,whichimplementsagiveninterface,isannotatedwith@TestInstance(Lifecycle.PER_CLASS),themethods@BeforeAlland@AfterAlldeclaredontheinterfacedonotneedtobestatic,butdefaultmethods.

ThethirdandfinalruleconcerninginterfacesinJUnit5is@ExtendWithand@Tagcanbedeclaredontestinterfacestoconfigureextensionsandtags.

Let’sseesomesimpleexamples.Inthefollowingclass,wearecreatinganinterface,notaclass.Inthisinterface,weusetheannotations@BeforeAll,@AfterAll,@BeforeEach,and@AfterEach.Ontheonehand,wedefine@BeforeAll,@AfterAllasstaticmethods.Ontheotherhand,wearedefining@BeforeEachand@AfterEachasJava8defaultmethods:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.AfterAll;

importorg.junit.jupiter.api.AfterEach;

importorg.junit.jupiter.api.BeforeAll;

importorg.junit.jupiter.api.BeforeEach;

importorg.junit.jupiter.api.TestInfo;

importorg.slf4j.Logger;

importorg.slf4j.LoggerFactory;

publicinterfaceTestLifecycleLogger{

staticfinalLoggerlog=LoggerFactory

.getLogger(TestLifecycleLogger.class.getName());

@BeforeAll

staticvoidbeforeAllTests(){

log.info("beforeAllTests");

}

@AfterAll

staticvoidafterAllTests(){

log.info("afterAllTests");

}

@BeforeEach

defaultvoidbeforeEachTest(TestInfotestInfo){

log.info("Abouttoexecute{}",testInfo.getDisplayName());

}

@AfterEach

defaultvoidafterEachTest(TestInfotestInfo){

log.info("Finishedexecuting{}",testInfo.getDisplayName());

}

}

WeareusingthelibrarySimpleLoggingFacadeforJava(SLF4J)inthisexample.TakealookatthecodeonGitHub(https://github.com/bonigarcia/mastering-junit5)fordetailsonthedeclarationofdependencies.

Inthisexample,weareusingtheannotationTestFactorytodefineadefaultmethodinaJavainterface:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importstaticorg.junit.jupiter.api.DynamicTest.dynamicTest;

importjava.util.Arrays;

importjava.util.Collection;

importorg.junit.jupiter.api.DynamicTest;

importorg.junit.jupiter.api.TestFactory;

interfaceTestInterfaceDynamicTestsDemo{

@TestFactory

defaultCollection<DynamicTest>dynamicTestsFromCollection(){

returnArrays.asList(

dynamicTest("1stdynamictestintestinterface",

()->assertTrue(true)),

dynamicTest("2nddynamictestintestinterface",

()->assertTrue(true)));

}

}

Finally,weusetheannotation@Tagand@ExtendWithinanotherinterface:packageio.github.bonigarcia;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.extension.ExtendWith;

@Tag("timed")

@ExtendWith(TimingExtension.class)

publicinterfaceTimeExecutionLogger{

}

Allinall,wecanusetheseinterfacesinourJupitertests:packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importorg.junit.jupiter.api.Test;

classTestInterfaceTestimplementsTestLifecycleLogger,

TimeExecutionLogger,

TestInterfaceDynamicTestsDemo{

@Test

voidisEqualValue(){

assertEquals(1,1);

}

}

Inthistest,thefactofimplementingallthepreviouslydefinedinterfaceswillprovidetheloggingcapabilitiesimplementedinthedefaultmethods:

Consoleoutputoftestimplementingseveralinterfaces

TesttemplatesA@TestTemplatemethodisnotaregulartestcasebutatemplatefortestcases.Methodannotatedlikethiswillbeinvokedmultipletimes,dependingontheinvocationcontextreturnedbytheregisteredproviders.Thus,testtemplatesareusedtogetherwitharegisteredTestTemplateInvocationContextProviderextension.

Letseeasimpleexampleofatesttemplate.Inthefollowingsnippet,wecanseeamethodannotatedwith@TestTemplate,andalsodeclaringanextensionofthetypeMyTestTemplateInvocationContextProvider:

packageio.github.bonigarcia;

importorg.junit.jupiter.api.TestTemplate;

importorg.junit.jupiter.api.extension.ExtendWith;

classTemplateTest{

@TestTemplate

@ExtendWith(MyTestTemplateInvocationContextProvider.class)

voidtestTemplate(Stringparameter){

System.out.println(parameter);

}

}

TherequiredprovidedimplementstheJupiterinterfaceTestTemplateInvocationContextProvider.Inspectingthecodeofthisclass,wecanseehowtwoStringparametersareprovidedtothetesttemplate(inthiscase,thevaluefortheseparametersareparameter-1andparameter-2):

packageio.github.bonigarcia;

importjava.util.Collections;

importjava.util.List;

importjava.util.stream.Stream;

importorg.junit.jupiter.api.extension.Extension;

importorg.junit.jupiter.api.extension.ExtensionContext;

importorg.junit.jupiter.api.extension.ParameterContext;

importorg.junit.jupiter.api.extension.ParameterResolver;

importorg.junit.jupiter.api.extension.TestTemplateInvocationContext;

importorg.junit.jupiter.api.extension.TestTemplateInvocationContextProvider;

publicclassMyTestTemplateInvocationContextProvider

implementsTestTemplateInvocationContextProvider{

@Override

publicbooleansupportsTestTemplate(ExtensionContextcontext){

returntrue;

}

@Override

publicStream<TestTemplateInvocationContext>

provideTestTemplateInvocationContexts(

ExtensionContextcontext){

returnStream.of(invocationContext("parameter-1"),

invocationContext("parameter-2"));

}

privateTestTemplateInvocationContextinvocationContext(Stringparameter){

returnnewTestTemplateInvocationContext(){

@Override

publicStringgetDisplayName(intinvocationIndex){

returnparameter;

}

@Override

publicList<Extension>getAdditionalExtensions(){

returnCollections.singletonList(newParameterResolver(){

@Override

publicbooleansupportsParameter(

ParameterContextparameterContext,

ExtensionContextextensionContext){

returnparameterContext.getParameter().getType()

.equals(String.class);

}

@Override

publicObjectresolveParameter(

ParameterContextparameterContext,

ExtensionContextextensionContext){

returnparameter;

}

});

}

};

}

}

Whenthetestisexecuted,eachinvocationofthetesttemplatebehaveslikearegular@Test.Inthisexample,thetestisonlywritingtheparameterinthestandardoutput.

Consoleoutputoftesttemplateexample

ParameterizedtestsParameterizedtestsareaspecialkindsoftestsinwhichthedatainputisinjectedinthetestinordertoreusethesametestlogic.ThisconceptwasalreadyaddressedinJUnit4,asexplainedinChapter1,RetrospectiveOnSoftwareQualityAndJavaTesting.Aswewouldexpect,parameterizedtestsarealsoimplementedinJUnit5.

Firstofall,inordertoimplementaparameterizedtestinJupiter,weneedtoaddthejunit-jupiter-paramstoourproject.WhenusingMaven,thatmeansaddingthefollowingdependency:

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-params</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

Asusual,asageneralrule,itisrecommendedtousethelatestversionoftheartifacts.Tofindoutthat,wecancheckouttheMavencentralrepository(http://search.maven.org/).

WhenusingGradle,thejunit-jupiter-paramsdependencycanbedeclaredasfollows:

dependencies{

testCompile("org.junit.jupiter:junit-jupiter-

params:${junitJupiterVersion}")

}

Then,weneedtousetheannotation@ParameterizedTest(locatedinthepackageorg.junit.jupiter.params)todeclareamethodwithinaJavaclassasaparameterizedtest.Thistypeoftestbehavesexactlythesameasaregular@Test,meaningthatallthelifecyclecallbacks(@BeforeEach,@AfterEach,andsoon)andextensionscontinueworkinginthesameway.

Nevertheless,theuseof@ParameterizedTestisnotenoughtoimplementaparameterizedtest.Togetherwith@ParameterizedTest,weneedtospecifyatleastoneargumentprovider.Aswewilldiscoverinthissection,JUnit5implementsdifferentannotationstoprovidedatainput(thatis,parametersfortests)fromdifferentsources.Theseargumentproviders(implementedasannotationsinJUnit5)aresummarizedinthefollowingtable(eachoftheseannotationsarelocatedinthepackageorg.junit.jupiter.params.provider):

Arguments

providerannotation

Description

@ValueSource UsedtospecifyanarrayofliteralvaluesofString,int,long,ordouble

@EnumSource

Argumentsourceforconstantsofaspecifiedenumeration(java.lang.Enum)

@MethodSourceProvidesaccesstovaluesreturnedbystaticmethodsoftheclassinwhichthisannotationisdeclared

@CsvSourceArgumentsourcewhichreadscomma-separatedvalues(CSV)fromitsattribute

@CsvFileSourceArgumentsourcewhichisusedtoloadCSVfilesfromoneormoreclasspathresources

@ArgumentsSource

Usedtospecifyacustomargumentprovider(thatis,aJavaclassthatimplementstheinterface)org.junit.jupiter.params.provider.ArgumentsProvider)

@ValueSourceTheannotation@ValueSourceisusedinconjunctionwith@ParameterizedTesttospecifyaparameterizedtestinwhichtheargumentsourceisanarrayofliteralvaluesofString,int,long,ordouble.Thesevaluesarespecifiedinsidetheannotation,usingtheelementsstrings,ints,longs,ordoubles.Considerthefollowingexample:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.ValueSource;

classValueSourceStringsParameterizedTest{

@ParameterizedTest

@ValueSource(strings={"Hello","World"})

voidtestWithStrings(Stringargument){

System.out.println("Parameterizedtestwith(String)parameter:"

+argument);

assertNotNull(argument);

}

}

Themethodofthisclass(testWithStrings)definesaparameterizedtestinwhichanarrayofStringisspecified.DuetothefactthattwoStringargumentsarespecifiedintheannotation@ValueSource(inthisexample"Hello"and"World"),thetestlogicwillbeexercisedtwice,oncepervalue.Thisdataisinjectedinthetestmethodusingtheargumentofthemethod,inthiscasethroughtheStringvariablenamedargument.Allinall,whenexecutingthistestclass,theoutputwillbeasfollows:

Executionofaparameterizedtestusing@ValueSourceandStringargumentprovider

Wecanalsouseintegerprimitivetypes(int,long,anddouble)withinthe@ValueSourceannotation.Thefollowingexampledemonstrateshow.Themethodsofthisexampleclass(namedtestWithInts,testWithLongs,andtestWithDoubles)usetheannotation@ValueSourcetodefinetheargumentsintheformofintegervalues,usingtheprimitivetypesint,long,anddouble,

respectively.Tothataim,theelementsints,longs,anddoublesof@ValueSourceneedtobespecified:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.ValueSource;

classValueSourcePrimitiveTypesParameterizedTest{

@ParameterizedTest

@ValueSource(ints={0,1})

voidtestWithInts(intargument){

System.out.println("Parameterizedtestwith(int)argument:"+

argument);

assertNotNull(argument);

}

@ParameterizedTest

@ValueSource(longs={2L,3L})

voidtestWithLongs(longargument){

System.out.println(

"Parameterizedtestwith(long)

argument:"+argument);

assertNotNull(argument);

}

@ParameterizedTest

@ValueSource(doubles={4d,5d})

voidtestWithDoubles(doubleargument){

System.out.println("Parameterizedtestwith(double)

argument:"+argument);

assertNotNull(argument);

}

}

Ascanbeseeninthepicturehere,eachtestisexecutedtwice,sinceineach@ValueSourceannotationwespecifytwodifferentinputparameters(typeint,long,anddouble,respectively).

Executionofaparameterizedtestusing@ValueSourceandprimitivetypes

@EnumSourceTheannotation@EnumSourceallowstospecifyaparameterizedtestinwhichtheargumentsourceisaJavaenumerationclass.Bydefault,eachvalueoftheenumerationwillbeusedtofeedtheparameterizedtest,oneatatime.

Forexample,inthefollowingtestclass,themethodtestWithEnumisannotatedwith@ParameterizedTestinconjunctionwith@EnumSource.Aswecansee,thevalueofthisannotationisTimeUnit.class,whichisastandardJavaannotation(packagejava.util.concurrent)usedtorepresenttimeduration.ThepossiblevaluesdefinedinthisenumerationareNANOSECONDS,MICROSECONDS,MILLISECONDS,SECONDS,MINUTES,HOURS,andDAYS:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importjava.util.concurrent.TimeUnit;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.EnumSource;

classEnumSourceParameterizedTest{

@ParameterizedTest

@EnumSource(TimeUnit.class)

voidtestWithEnum(TimeUnitargument){

System.out.println("Parameterizedtestwith(TimeUnit)

argument:"+argument);

assertNotNull(argument);

}

}

Therefore,theexecutionofthistestwillbecarriedoutseventimes,thatis,oneperTimeUnitenumerationvalue.Wecancheckthisinthetraceoftheoutputconsolewhenexecutingthetest:

Executionofparameterizedtestusing@EnumSourceandTimeUnit.class

Moreover,the@EnumSourceannotationallowstofilterthemembersoftheenumerationinseveralways.Toimplementthisselection,thefollowingelementscanbespecifiedwithina@EnumSourceannotation:

mode:Constantvaluewhichdeterminesthetypeoffiltering.Thisisdefinedasanenumerationintheinnerclassorg.junit.jupiter.params.provider.EnumSource.Mode,andthepossiblevaluesare:

INCLUDE:Usedtoselectthosevalueswhosenamesaresuppliedviathenameselement.Thisisthedefaultoption.EXCLUDE:Usedtoselectallvaluesexceptthosesuppliedwiththenameselement.MATCH_ALL:Usedtoselectthosevalueswhosenamesmatchthepatternsinnameselement.MATCH_ANY:Usedtoselectthosevalueswhosenamesmatchanypatterninthenameselement.

names:Thearrayofstringwhichallowstoselectagroupofenumconstants.Thecriteriaforinclusion/exclusionisdirectlylinkedtothevalueofmode.Inaddition,thiselementalsoallowstodefineregularexpressionstoselectthenamesofenumconstantstobematched.

Considerthefollowingexample.Inthisclass,therearethreeparameterizedtests.Firstone,namedtestWithFilteredEnum,usestheclassTimeUnittofeedthe@EnumSourceargumentprovider.Moreover,theenumconstantsetisfilteredusingtheelementnames.Aswecansee,onlytheconstant"DAYS"and"HOURS"willbeusedtofeedthistest(takeintoaccountthatthedefaultmodeisINCLUDE):

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importstaticorg.junit.jupiter.params.provider.EnumSource.Mode.EXCLUDE;

importstaticorg.junit.jupiter.params.provider.EnumSource.Mode.MATCH_ALL;

importjava.util.concurrent.TimeUnit;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.EnumSource;

classEnumSourceFilteringParameterizedTest{

@ParameterizedTest

@EnumSource(value=TimeUnit.class,names={"DAYS","HOURS"})

voidtestWithFilteredEnum(TimeUnitargument){

System.out.println("Parameterizedtestwithsome(TimeUnit)

argument:"+argument);

assertNotNull(argument);

}

@ParameterizedTest

@EnumSource(value=TimeUnit.class,mode=EXCLUDE,names={

"DAYS","HOURS"})

voidtestWithExcludeEnum(TimeUnitargument){

System.out.println("Parameterizedtestwithexcluded(TimeUnit)

argument:"+argument);

assertNotNull(argument);

}

@ParameterizedTest

@EnumSource(value=TimeUnit.class,mode=MATCH_ALL,names=

"^(M|N).+SECONDS$")

voidtestWithRegexEnum(TimeUnitargument){

System.out.println("Parameterizedtestwithregexfiltered

(TimeUnit)argument:"+argument);

assertNotNull(argument);

}

}

Thus,whenexecutingthisclassintheconsole,theoutputweobtainisthefollowing.Regardingthefirsttest,wecanseethatonlytracesfor"DAYS"and"HOURS"arepresent:

Executionofparameterizedtestusing@EnumSourceusingfilteringcapabilities

Considernowthesecondtestmethod,namedtestWithExcludeEnum.Thistestisexactlythesameasbeforewithadifference:themodehereisEXCLUSION(insteadofINCLUSION,chosenbydefaultintheprevioustest).Allinall,intheexecution(seescreenshotbefore)whencanseethatthistestisexecutedfivetimes,peroneoftheenumconstantdifferenttoDAYSandHOURS.Tocheckthat,trackthetraceswiththesentence"Parameterizedtestwithexcluded(TimeUnit)argument".

Thethirdandlastmethodofthisclass(calledtestWithRegexEnum)definesaninclusionmode,MATCH_ALL,usingaregularexpressiontofiltertheenumeration(inthiscase,itisalsoTimeUnit).Theconcreteregularexpressionusedinthisexampleis^(M|N).+SECONDS$,whichmeansthatonlywillbeincludedinthoseenumconstantsstartingwithMorNandendingwithSECONDS.Ascanbecheckedintheexecutionscreenshot,therearethreeTimeUnitconstantsmatchingtheseconditions:NANOSECONDS,MICROSECONDS,andMILISECONDS.

@MethodSourceTheannotation@MethodSourceallowstodefinethenameofthestaticmethodinwhichtheargumentsforthetestareprovidedasaJava8Stream.Forinstance,inthefollowingexample,wecanseeaparameterizedtestinwhichtheargumentproviderisastaticmethodcalledstringProvider.Inthisexample,thismethodreturnsaStreamofString‘sandthereforetheargumentofthetestmethod(callledtestWithStringProvider)acceptsoneStringargument:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importjava.util.stream.Stream;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.MethodSource;

classMethodSourceStringsParameterizedTest{

staticStream<String>stringProvider(){

returnStream.of("hello","world");

}

@ParameterizedTest

@MethodSource("stringProvider")

voidtestWithStringProvider(Stringargument){

System.out.println("Parameterizedtestwith(String)argument:"

+argument);

assertNotNull(argument);

}

}

Whenrunningtheexample,wecanseehowthetestisexecutetwice,onceperStringcontainedintheStream.

Executionofaparameterizedtestusing@MethodSourceandStringargumentprovider

ThetypeoftheobjectscontainedintheStreamisnotrequiredtobeString.Infact,thistypecanbeanything.Let’sconsideranotherexample,inwhich@MethodSourceislinkedtoastaticmethod,whichreturnsasStreamofcustomobjects.Inthisexample,thistypeisnamedPerson,andhereitisimplemented

asaninnerclasswithtwoproperties(nameandsurname).packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importjava.util.stream.Stream;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.MethodSource;

classMethodSourceObjectsParameterizedTest{

staticStream<Person>personProvider(){

Personjohn=newPerson("John","Doe");

Personjane=newPerson("Jane","Roe");

returnStream.of(john,jane);

}

@ParameterizedTest

@MethodSource("personProvider")

voidtestWithPersonProvider(Personargument){

System.out.println("Parameterizedtestwith(Person)argument:"+

argument);

assertNotNull(argument);

}

staticclassPerson{

Stringname;

Stringsurname;

publicPerson(Stringname,Stringsurname){

this.name=name;

this.surname=surname;

}

publicStringgetName(){

returnname;

}

publicvoidsetName(Stringname){

this.name=name;

}

publicStringgetSurname(){

returnsurname;

}

publicvoidsetSurname(Stringsurname){

this.surname=surname;

}

@Override

publicStringtoString(){

return"Person[name="+name+",surname="+surname+"]";

}

}

}

Asthefollowingscreenshotshows,whenexecutingthisexample,theparameterizedtestisexercisetwice,onceperPersonobjectscontainedintheStream("JohnDoe"and"JaneRoe").

Executionofparameterizedtestusing@MethodSourceandcustomobjectargumentprovider

Wecanalsouse@MethodSourcetospecifyargumentproviderswhichcontainintegerprimitivetypes,concretelyofint,double,andlong.Thefollowingclasscontainsanexample.Wecanseethreeparameterizedtests.Thefirstone(namedtestWithIntProvider)usestheannotation@MethodSourcetolinkwiththestaticmethodintProvider.Inthebodyofthismethod,weusethestandardJavaclassIntStreamtoreturnanStreamofintvalues.Thesecondandthirdtest(calledtestWithDoubleProviderandtestWithLongProvider)arequitesimilar,butusingaStreamofdoubleandlongvalues,respectively:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importjava.util.stream.DoubleStream;

importjava.util.stream.IntStream;

importjava.util.stream.LongStream;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.MethodSource;

classMethodSourcePrimitiveTypesParameterizedTest{

staticIntStreamintProvider(){

returnIntStream.of(0,1);

}

@ParameterizedTest

@MethodSource("intProvider")

voidtestWithIntProvider(intargument){

System.out.println("Parameterizedtestwith(int)argument:"+

argument);

assertNotNull(argument);

}

staticDoubleStreamdoubleProvider(){

returnDoubleStream.of(2d,3d);

}

@ParameterizedTest

@MethodSource("doubleProvider")

voidtestWithDoubleProvider(doubleargument){

System.out.println(

"Parameterizedtestwith(double)argument:"+argument);

assertNotNull(argument);

}

staticLongStreamlongProvider(){

returnLongStream.of(4L,5L);

}

@ParameterizedTest

@MethodSource("longProvider")

voidtestWithLongProvider(longargument){

System.out.println(

"Parameterizedtestwith(long)argument:"+argument);

assertNotNull(argument);

}

}

Thus,whenexecutingthisclass,therewillbesixtestsexecuted(threeparameterizedtestswithtwoargumentseach).

Inthefollowingscreenshot,wecancheckthisbyfollowingthetraceswrittenbyeachtesttothestandardoutput:

Executionofparameterizedtestusing@MethodSourceandprimitivetypesargumentprovider

Finally,withregardsto@MethodSourceparameterizedtests,itisworthittoknowthatthemethodprovidersareallowedtoreturnaStreamofdifferenttypes(objectsorprimitivetypes).Thisisveryconvenientforreal-worldtestcases.Forexample,thefollowingclassimplementsaparameterizedtestinwhichtheargumentproviderisamethodreturningargumentsofmixedtypes:Stringandint.Theseparametersareinjectedinthetestasmethodarguments(calledfirstandsecondintheexample).

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotEquals;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importjava.util.stream.Stream;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.Arguments;

importorg.junit.jupiter.params.provider.MethodSource;

classMethodSourceMixedTypesParameterizedTest{

staticStream<Arguments>stringAndIntProvider(){

returnStream.of(Arguments.of("Mastering",10),

Arguments.of("JUnit5",20));

}

@ParameterizedTest

@MethodSource("stringAndIntProvider")

voidtestWithMultiArgMethodSource(Stringfirst,intsecond){

System.out.println("Parameterizedtestwithtwoarguments:

(String)"+first+"and(int)"+second);

assertNotNull(first);

assertNotEquals(0,second);

}

}

Asusual,therewillbetestexecutionsasentriescontainedintheStream.Inthiscase,therearetwo:"Mastertering"and10,andthen"JUnit5"and20.

Executionofparameterizedtestusing@MethodSourcewithdifferenttypesofarguments

@CsvSourceand@CsvFileSourceAnotherwaytospecifythesourceofargumentsforparameterizedtestsisusingcomma-separatedvalues(CSV).Thiscanbedoneusingtheannotation@CsvSource,whichallowstoembedCSVcontentasStringinthevalueoftheannotation.

Considerthefollowingexample.ItcontainsaJupiterparameterizedtest(namedtestWithCsvSource),whichisusingtheannotation@CsvSource.ThisannotationcontainsanarrayofStrings.Ineachelementofthearray,wecanseethereisadifferentvalueseparatedbycommas.

ThecontentoftheCSVisautomaticallyconvertedtoStringandint.TofindoutmoreabouttheimplicittypeconversionmadeinparametersbyJUnit5,takealooktothesectionArgumentconversioninthischapter.

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotEquals;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.CsvSource;

classCsvSourceParameterizedTest{

@ParameterizedTest

@CsvSource({"hello,1","world,2","'happy,testing',3"})

voidtestWithCsvSource(Stringfirst,intsecond){

System.out.println("Parameterizedtestwith(String)"+first

+"and(int)"+second);

assertNotNull(first);

assertNotEquals(0,second);

}

}

Allinall,whenexecutingthistestclass,therewillbethreesingletests,eachperentryinthearray.Eachexecutionwillbeinvoked,passingtwoargumentstothetest.ThefirstoneisnamedfirstanditstypeisString,andsecondoneiscalledsecondanditstypeisint.

Executionofparameterizedtestusing@CsvSource

IftheamountofCSVdataisbig,itmightbemoreconvenientusingtheannotation@CsvFileSourceinstead.ThisannotationallowstofeedtheparameterizedtestwithaCSVfilelocatedintheclasspathoftheproject.Inthefollowingexample,weusethefileinput.csv:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotEquals;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.CsvFileSource;

classCsvFileSourceParameterizedTest{

@ParameterizedTest

@CsvFileSource(resources="/input.csv")

voidtestWithCsvFileSource(Stringfirst,intsecond){

System.out.println("Yetanotherparameterizedtestwith

(String)"+first+"and(int)"+second);

assertNotNull(first);

assertNotEquals(0,second);

}

}

Internally,theannotation@CsvFileSourcelocatesthefileusingthemethodgetResourceAsStream()ofthestandardJavaclassjava.lang.Class.Therefore,thepathofthefileisinterpretedasapathlocaltothepackageclasswearecallingitfrom.Sinceourresourceislocatedintherootoftheclasspath(intheexampleitislocatedinthefoldersrc/test/resources),weneedtolocateitas/input.csv.

Locationandcontentofinput.csvintheexamplewith@CsvFileSource

ThefollowingscreenshotshowstheoutputofthetestwhenitisexecutedwithMaven.SincetheCSVhasthreerowsofdata,therearethreetestexecutions,eachonewithtwoparameters(firstoneasStringandsecondoneasint):

Executionofparameterizedtestusing@CsvFileSource

@ArgumentsSourceThelastannotationaimedtospecifythesourceofargumentsforparameterizedtestsinJUnit5is@ArgumentsSource.Withthisannotation,wecanspecifyacustom(andreusableindifferenttests)class,whichwillcontaintheparametersforthetest.Thisclassmustimplementtheinterfaceorg.junit.jupiter.params.provider.ArgumentsProvider.

Let’sseeanexample.ThefollowingclassimplementsaJupiterparameterizedtest,inwhichtheargumentssourcewillbedefinedintheclassCustomArgumentsProvider1:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.ArgumentsSource;

classArgumentSourceParameterizedTest{

@ParameterizedTest

@ArgumentsSource(CustomArgumentsProvider1.class)

voidtestWithArgumentsSource(Stringfirst,intsecond){

System.out.println("Parameterizedtestwith(String)"+first

+"and(int)"+second);

assertNotNull(first);

assertTrue(second>0);

}

}

Thisclass(namedCustomArgumentsProvider1)hasbeenimplementedonourside,andduetothefactthatitimplementstheinterfaceArgumentsProvider,mustoverridethemethodprovideArguments,inwhichtheactualdefinitionofparametersforthetestisimplemented.Lookingatthecodeoftheexample,wecanseethatthismethodreturnsaStreamofArguments.Inthisexample,wearereturningacoupleofentriesintheStream,eachonewithtwoarguments(Stringandint,respectively):

packageio.github.bonigarcia;

importjava.util.stream.Stream;

importorg.junit.jupiter.api.extension.ExtensionContext;

importorg.junit.jupiter.params.provider.Arguments;

importorg.junit.jupiter.params.provider.ArgumentsProvider;

publicclassCustomArgumentsProvider1implementsArgumentsProvider{

@Override

publicStream<?extendsArguments>provideArguments(

ExtensionContextcontext){

System.out.println("Argumentsprovidertotest"

+context.getTestMethod().get().getName());

returnStream.of(Arguments.of("hello",1),

Arguments.of("world",2));

}

}

NoticealsothatthisargumenthasanargumentoftypeExtensionContext(packageorg.junit.jupiter.api.extension).Thisargumentisveryusefultoknowthecontextinwhichthetestisexecuted.Asillustratedinthescreenshothere,ExtensionContextAPIoffersdifferentmethodstofindoutdifferentattributesofthetestinstance(testmethodname,displayname,tags,andsoon).

Inourexample(CustomArgumentsProvider1),thecontextisusedtowritethetestmethodnameinthestandardoutput:

ExtensionContextAPI

Thus,whenexecutingthisexample,wecanseetwotestsbeingexecuted.Moreover,wecancheckthelogtracewiththetestmethod,thankstotheExtensionContextobjectinside,outArgumentsProviderinstance:

Executionofparameterizedtestusing@ArgumentsSource

Severalargumentsourcescanbeappliedtothesameparameterizedtest.Infact,thiscanbedoneintwodifferentwaysintheJupiterprogramming

model:

Usingseveralannotationof@ArgumentsSourcetogetherwiththesame@ParameterizedTest.Thiscanbedonesince@ArgumentsSourceisajava.lang.annotation.Repeatableannotation.Usingtheannotation@ArgumentsSources(noticethesourceispluralhere).Thisannotationissimplyacontainerforoneormore@ArgumentsSource.Thefollowingclassshowsasimpleexample:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.ArgumentsSource;

importorg.junit.jupiter.params.provider.ArgumentsSources;

classArgumentSourcesParameterizedTest{

@ParameterizedTest

@ArgumentsSources({

@ArgumentsSource(CustomArgumentsProvider1.class),

@ArgumentsSource(CustomArgumentsProvider2.class)})

voidtestWithArgumentsSource(Stringfirst,intsecond){

System.out.println("Parameterizedtestwith(String)"+first

+"and(int)"+second);

assertNotNull(first);

assertTrue(second>0);

}

}

Supposingthatthesecondargumentprovider(CustomArgumentsProvider2.class)specifiestwoormoresetsofargument,whenexecutingthetestclasstherewillbefourtestexecutions:

Executionofparameterizedtestusing@ArgumentsSources

ArgumentconversionTosupportusecasessuchas@CsvSourceand@CsvFileSource,Jupiterprovidesanumberofbuilt-inimplicitconverters.Moreover,theseconverterscanbeimplementedbasedonspecificneedsbymeansofexplicitconverters.Thissectioncoversbothtypesofconversions.

ImplicitconversionInternally,JUnit5handlesasetofrulesfortheconversionofparametersfromStringtotheactualargumenttype.Forexample,if@ParameterizedTestsdeclaresaparameteroftypeTimeUnit,butthedeclaredsourceisaString,internallythisStringwillbeconvertedtoTimeUnit.ThefollowingtablesummarizestherulesofimplicitconversionsinJUnit5forparameterizedtestarguments:

TargetType Example

boolean/Boolean "false"->false

byte/Byte "1"->(byte)1

char/Character "a"->'a'

short/Short "2"->(short)2

int/Integer "3"->3

long/Long "4"->4L

float/Float "5.0"->5.0f

double/Double "6.0"->6.0d

Enumsubclass "SECONDS"->TimeUnit.SECONDS

java.time.Instant "1970-01-01T00:00:00Z"->Instant.ofEpochMilli(0)

java.time.LocalDate "2017-10-24"->LocalDate.of(2017,10,24)

java.time.LocalDateTime "2017-03-14T12:34:56.789"->LocalDateTime.of(2017,3,14,12,34,56,789_000_000)

java.time.LocalTime "12:34:56.789"->LocalTime.of(12,34,56,789_000_000)

java.time.OffsetDateTime"2017-03-14T12:34:56.789Z"->OffsetDateTime.of(2017,3,14,12,34,56,789_000_000,ZoneOffset.UTC)

java.time.OffsetTime "12:34:56.789Z"->OffsetTime.of(12,34,56,789_000_000,ZoneOffset.UTC)

java.time.Year "2017"->Year.of(2017)

java.time.YearMonth "2017-10"->YearMonth.of(2017,10)

java.time.ZonedDateTime "2017-10-24T12:34:56.789Z"->ZonedDateTime.of(2017,10,24,12,34,56,789_000_000,ZoneOffset.UTC)

Thefollowingexampleshowsseveralexamplesofimplicitconversion.Thefirsttest(testWithImplicitConversionToBoolean)declaresaStringsourceas"true",butthen,theexpectedargumenttypeisBoolean.Similarly,thesecondtest("testWithImplicitConversionToInteger")makesanimplicitconversionfromStringtoInteger.Thethirdtest(testWithImplicitConversionToEnum)convertstheinputStringtoTimeUnit(enumeration),andfinallythefourthtest(testWithImplicitConversionToLocalDate)producesaconversiontoLocalDate:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importjava.time.LocalDate;

importjava.util.concurrent.TimeUnit;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.ValueSource;

classImplicitConversionParameterizedTest{

@ParameterizedTest

@ValueSource(strings="true")

voidtestWithImplicitConversionToBoolean(Booleanargument){

System.out.println("Argument"+argument+"isatypeof"

+argument.getClass());

assertTrue(argument);

}

@ParameterizedTest

@ValueSource(strings="11")

voidtestWithImplicitConversionToInteger(Integerargument){

System.out.println("Argument"+argument+"isatypeof"

+argument.getClass());

assertTrue(argument>10);

}

@ParameterizedTest

@ValueSource(strings="SECONDS")

voidtestWithImplicitConversionToEnum(TimeUnitargument){

System.out.println("Argument"+argument+"isatypeof"

+argument.getDeclaringClass());

assertNotNull(argument.name());

}

@ParameterizedTest

@ValueSource(strings="2017-07-25")

voidtestWithImplicitConversionToLocalDate(LocalDateargument){

System.out.println("Argument"+argument+"isatypeof"

+argument.getClass());

assertNotNull(argument);

}

}

Wecanchecktheactualtypeoftheargumentintheconsole.Eachtestwritesalineinthestandardoutputwiththevalueandthetypeofeachargument:

Executionofparameterizedtestsusingimplicitargumentconversion

ExplicitconversionIftheimplicitconversionprovidedbyJUnit5isnotenoughtocoverourneeds,wecanusetheexplicitconversioncapability.Thankstothisfeature,wecanspecifyaclasswhichisgoingtomakethecustomconversionofparametertypes.Thiscustomconverterisidentifiedwiththeannotation@ConvertWith,referringtotheargumenttobeconvertedwith.Considerthefollowingexample.Thisparameterizedtestdeclaresacustomconverterforitstestmethodargument:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importjava.util.concurrent.TimeUnit;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.converter.ConvertWith;

importorg.junit.jupiter.params.provider.EnumSource;

classExplicitConversionParameterizedTest{

@ParameterizedTest

@EnumSource(TimeUnit.class)

voidtestWithExplicitArgumentConversion(

@ConvertWith(CustomArgumentsConverter.class)String

argument){

System.out.println("Argument"+argument+"isatypeof"

+argument.getClass());

assertNotNull(argument);

}

}

OurcustomconvertedisaclassthatextendstheJUnit5’sSimpleArgumentConverter.Thisclassoverridesthemethodconvert,inwhichtheactualconversiontakesplace.Intheexample,wesimplytransformwhateverargumentsourcetoString.

packageio.github.bonigarcia;

importorg.junit.jupiter.params.converter.SimpleArgumentConverter;

publicclassCustomArgumentsConverterextendsSimpleArgumentConverter{

@Override

protectedObjectconvert(Objectsource,Class<?>targetType){

returnString.valueOf(source);

}

}

Allinall,whenthetestisexecuted,thesevenenumerationconstantsdefinedinTimeUnitwillbepassedasargumentstothetest,priorconversiontoStringinCustomArgumentsConverter:

Executionofparameterizedtestsusingexplicitargumentconversion

CustomnamesThelastfeaturerelatedwithparameterizedtestsinJUnit5hastodowiththedisplaynameofeachexecutionoftests.Aswelearned,aparameterizedtestisusuallyexecutedasseveralsingletests.Therefore,fortheshakeoftraceability,itisgoodpracticetolinkeachtestexecutionwiththeargumentsource.

Tothataim,theannotation@ParameterizedTestacceptsanelementcallednameinwhichwecanspecifyacustomname(String)forthetestexecution.Moreover,inthisString,wecanuseseveralbuilt-inplaceholders,asdescribedinthefollowingtable:

Placeholder Description

{index} Currentinvocationindex(firstoneis1,secondis2,…)

{arguments} Comma-separatedargumentscompletelist

{0},{1},… Valueforanindividualargument(firstoneis0,secondis2,…)

Let’sseeasimpleexample.Thefollowingclasscontainsaparameterizedtestwhoseargumentsaredefinedusinga@CsvSourceannotation.Thetestmethodacceptstwoarguments(Stringandint).Inaddition,wearespecifyingtheelementnameoftheannotation@ParameterizedTestwithacustommessage,usingtheplaceholdersforthecurrenttestinvocation({index})andalsoforthevaluesofeachargument:thefirstone({0})andthesecondone({1}):

packageio.github.bonigarcia;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.CsvSource;

classCustomNamesParameterizedTest{

@DisplayName("Displaynameoftestcontainer")

@ParameterizedTest(name="[{index}]firstargument=\"{0}\",second

argument={1}")

@CsvSource({"mastering,1","parameterized,2","tests,3"})

voidtestWithCustomDisplayNames(Stringfirst,intsecond){

System.out.println("Testingwithparameters:"+first+"and"+

second);

}

}

WhenexecutingthistestinanIDE(IntelliJinthefollowingscreenshot),wecanseehowthedisplaynameisdifferentforeachtestexecution:

ExecutionofparameterizedtestsusingcustomnamesinIntelliJIDE

Java9Java9wasreleasedforGeneralAvailability(GA)onSeptember21,2017.TherearemanynewfeaturesshippedwithJava9.Amongthem,modularityisthedefiningfeatureforJava9.

Sofar,therehasbeenaproblemofmodularityinJava,especiallysignificantforlargecodebases.Everypublicclasscanbeaccessedbyanyotherclassintheclasspath,leadingtoinadvertentusageofclasses.Inaddition,theclasspathpresentspotentialproblems,suchastheinabilitytoknowwhetherornotthereareduplicatedJARs.Tosolvetheseproblems,Java9providestheJavaPlatformModuleSystem,whichallowstocreatemodularJARfiles.Thistypeofmodulescontainsanadditionalmoduledescriptorcalledmodule-info.java.Thecontentofsuchfilesisquitesimple:itdeclaresdependenciestoothermodulesusingthekeywordrequires,andexportsitsownpackageswiththekeywordexports.Allnon-exportedpackagesareencapsulatedinthemodulebydefault,forexample:

modulemymodule{

exportsio.github.bonigarcia;

requiresmydependency;

}

Wecanrepresenttherelationshipbetweenthesemodulesasfollows:

ExampleofrelationshipbetweenmodulesinJava9

OthernewcapabilitiesofJava9aresummarizedinthefollowinglist:

TheusemodulesallowtocreateaminimalruntimeJDKoptimizedforthegivenapplication,insteadofusingafullyJDKinstallation.ThiscanbeachieveusingthetoolthejlinkshippedwithJDK9.Java9providesaninteractiveenvironmenttoexecuteJavacode,directlyfromtheshell.ThistypeofutilityiscommonlyknownasRead-Eval-Print-Loop(REPL),whichiscalledJShellinJDK9.Collectionfactorymethods,Java9providesthecapabilityofcreating

collections(forexample,listsorsets)andpopulatestheminasingleline:

Set<Integer>ints=Set.of(1,2,3);

List<String>strings=List.of("first","second");

StreamAPIimprovements:StreamswasintroducedinJava8,andtheyallowtocreatedeclarativepipelinesoftransformationsoncollections.InJava9,themethodsdropWhile,takeWhile,andofNullableareaddedtotheStreamAPI.Privateinterfacemethods:Java8providesdefaultmethodsoninterfaces.ThelimitationsofaristhatdefaultmethodsinJava8mustbepublic.Now,inJava9,thesedefaultmethodscanbealsoprivate,helpingtostructurebettertheirimplementation.HTTP/2:Java9supportsoutofthebox,version2ofHTTPandalsoWebSockets.MultireleaseJARs:Thisfeatureallowstocreatealternativeversionsofclasses,dependingontheversionoftheJREexecutingtheJAR.Tothataim,underthefolderMETA-INF/versions/<java-version>,wecanspecifydifferentversionsofcompiledclasses,whichwillusedonlywhentheJREversionmatchestheversion.ImprovedJavadoc:Lastbutnotleast,Java9allowstocreateHTML5compliantJavadocwithanintegratedsearchcapability.

JUnit5andJava9compatibilitySinceM5,allJUnit5artifactsareshippedwithcompiledmoduledescriptorsforJava9,declaredinitsJARmanifest(fileMANIFEST.MF).Forexample,thecontentofthemanifestfortheartifactjunit-jupiter-apiM6isthefollowing:

Manifest-Version:1.0

Implementation-Title:junit-jupiter-api

Automatic-Module-Name:org.junit.jupiter.api

Build-Date:2017-07-18

Implementation-Version:5.0.0-M6

Built-By:JUnitTeam

Specification-Vendor:junit.org

Specification-Title:junit-jupiter-api

Implementation-Vendor:junit.org

Build-Revision:3e6482ab8b0dc5376a4ca4bb42bef1eb454b6f1b

Build-Time:21:26:15.224+0200

Created-By:1.8.0_131(OracleCorporation25.131-b11)

Specification-Version:5.0.0

WithregardstoJava9,theinterestingthingisthedeclarationAutomatic-Module-Name.ThisallowstotestmodulestorequiretheJUnit5modulesimplybyaddingthefollowinglinestoitsmoduledescriptorfile(module-info.java):

modulefoo.bar{

requiresorg.junit.jupiter.api;

}

BeyondJUnit5.0JUnit5.0GA(GeneralAvailability)wasreleasedonSeptember10,2017.Furthermore,JUnitisalivingproject,andnewfeaturesareplannedforthenextrelease,thatis,5.1(withnoreleaseagendascheduledatthetimeofwriting).ThebacklogforthenextreleaseofJUnit5canbeseenonGitHub:https://github.com/junit-team/junit5/milestone/3.Amongother,thefollowingfeaturesareplannedforJUnit5.1:

Scenariotests:Thisfeaturehastodowiththecapabilityoforderingdifferenttestmethodswithinaclass.Todothat,thefollowingannotationsareplanned:

@ScenarioTest:Aclass-levelannotationusedtodenotethatatestclasscontainsstepsthatmakeupasinglescenariotest.@Step:Amethod-levelannotationusedtodenotethatatestmethodisasinglestepwithinthescenariotest.Supportforparalleltestsexecution:ConcurrencyisoneofthemainaspectstobeimprovedinJUnit5.1,andthereforethesupportofoutoftheboxconcurrenttestexecutionisplanned.

Mechanismforterminatingdynamictestsearly:ThisisanenhancementoftheJUnit5.0supportfordynamictests,introducingatimeouttostoptheexecutionbeforeitterminatesitself(toavoiduncontrollednon-deterministicexecutions).Severalimprovementsintestreporting,suchascapturingstdout/stderrandincludeintestreports,providereliablewaytogettheclass(classname)ofexecutedtestmethods,orspecifytheorderoftestsintestreports,amongothers.

SummaryThischaptercontainsacomprehensivesummaryoftheadvancecapabilitiestowriterichJupitertestsdrivenbyexamples.First,wehavelearnedthatparameterscanbeinjectedinconstructorandmethodsintestclasses.JUnit5providesthreeparameterresolversoutofthebox,namelyresolverforparametersofthetypeTestInfo(toretrieveinformationaboutthecurrenttest),resolverforparametersofthetypeRepetitionInfo(toretrieveinformationaboutthecurrentrepetition),andresolverforparametersofthetypeTestReporter(topublishadditionaldataaboutthecurrenttestrun).

AnothernewfeatureimplementedinJupiteristheconceptofdynamictests.SofarinJUnit3and4,testsaredefinedatcompiletime(thatisstatictests).Jupiterintroducestheannotation@TestFactorythatallowstogeneratetestatruntime.AnothernewconceptprovidedbytheJupiterprogrammingmodelarethetesttemplates.Thesetemplatesredefinedusingtheannotation@TestTemplateandarenotregulartestcasesbutratheratemplatefortestcases.

JUnit5implementsanenhancementsupportforparameterizedtests.Inordertoimplementthistypeoftests,theannotation@ParameterizedTestmustbeused.Togetherwiththisannotation,anargumentprovidershouldbealsospecified.Tothataim,severalannotationsareprovidedinJupiter:@ValueSource,@EnumSource,@MethodSource,@CsvSource,@CsvFileSource,and@ArgumentSource.

Inthechapter5,IntegrationOfJUnit5WithExternalFrameworks,wearegoingtolearnhowJUnit5interactswithexternalframeworks.Concretely,wearegoingtoreviewseveralJUnit5extension,whichprovidescapabilitiestouseMockito,Spring,Selenium,Cucumber,orDocker.Moreover,wepresentaGradleplugin,whichallowstoexecutetestswithinanAndroidproject.Finally,wefindouthowtouseseveralRESTlibraries(forexample,RESTAssuredorWireMock)totestRESTfulservices.

IntegrationOfJUnit5WithExternalFrameworks

IfIhaveseenfurtherthanothers,itisbystandingupontheshouldersofgiants.

-IsaacNewton

AsdescribedinChapter2,What’sNewinJUnit,theextensionmodelofJUnit5allowsustoextendthecorefunctionalityofJUnit5byathirdparty(toolvendor,developers,andsoon).IntheJupiterextensionmodel,anextensionpointisacallbackinterfacethattheextensionimplementsandthenregisters(activates)intheJUnit5framework.Aswewilldiscoverinthischapter,theJUnit5extensionmodelcanbeusedtoprovideseamlessintegrationwithexistingthird-partyframeworks.Concretely,inthischapter,wereviewJUnit5extensionforthefollowingtechnologies:

Mockito:Mock(testdouble)unittestingframework.Spring:AJavaframeworkforbuildingenterpriseapplications.Selenium:Atestingframeworktoautomatethenavigationandassessmentofwebapplications.Cucumber:TestingframeworkwhichallowsustocreateacceptancetestswrittenfollowingaBehavior-DrivenDevelopment(BDD)style.Docker:Asoftwaretechnologywhichallowsustopackandrunanyapplicationasalightweightandportablecontainer.

Moreover,wediscoverthattheJUnit5extensionmodelisnottheonlywaytointegratewiththeexternalworld.Concretely,westudyhowJUnit5canbeusedtogetherwiththefollowing:

Android(mobileoperatingsystembasedonLinux):WecanrunJupitertestsinanAndroidprojectusingaGradlepluginforJUnit5.REST(architecturalstylefordesigningdistributedsystems):WecaninteractandverifyRESTservicessimplyusingthird-partylibraries(suchasRESTAssuredorWireMock),orusingthefullyintegratedapproachofSpring(teststogetherwiththeserviceimplementation).

MockitoMockito(http://site.mockito.org/)isanopensourcemockunittestingframeworkforJava,firstreleasedinApril2008.Ofcourse,MockitoisnottheonlymockframeworkforJava;thereareothers,suchasthefollowing:

EasyMock(http://easymock.org/).JMock(http://www.jmock.org/).PowerMock(http://powermock.github.io/).JMockit(http://jmockit.org/).

Wecansaythat,atthetimeofwriting,MockitoisthepreferredmockframeworkinJavatestsforthemostdevelopersandtesters.Tojustifythatclaim,weusethefollowingscreenshot,whichshowstheevolutionofthetermsMockito,EasyMock,JMock,PowerMock,andJMockitinGoogleTrends(https://trends.google.com/)from2004to2017.Atthebeginningofthisperiod,wecanseetherewasasignificantinterestonEasyMockandJMock;nevertheless,Mockitowasmoreindemandcomparedwiththerestoftheframeworks:

GoogleTrendsevolutionofMockito,EasyMock,JMock,PowerMock,andJMockit

MockitoinanutshellAsintroducedinChapter1,RetrospectiveonSoftwareQualityandJavaTesting,therearedifferentlevelsofsoftwaretesting,suchasunit,integration,system,oracceptance.Regardingunittests,theyshouldbeexecutedinisolationforasinglepieceofsoftware,forexample,anindividualclass.Theobjectiveinthisleveloftestsistoverifythefunctionalityoftheunitandnotofitsdependencies.

Inotherwords,wewanttotestwhatisknownastheSystemUnderTest(SUT)butnotitsDepended-OnComponents(DOCs).Toachievethisisolation,weusetypicallytestdoublestoreplacetheseDOCs.Mockobjectsareakindoftestdouble,whichareprogrammedwithexpectationsabouttherealDOC.

Infewwords,Mockitoisatestingframeworkthatallowsmockobjectcreation,stubbing,andverification.Tothataim,MockitoprovidesanAPItoisolatetheSUTanditsDOCs.Generallyspeaking,usingMockitoinvolvesthreedifferentsteps:

1. Mockingobjects:InordertoisolateourSUT,weusetheMockitoAPItocreatemocksofitsassociatedDOC(s).Thisway,weguaranteethattheSUTisnotdependingonitsrealDOC(s),andourunittestisactuallyfocusedontheSUT.

2. Settingexpectations:Thedifferentialaspectofmocksobjectwithrespecttoothertestdoubles(suchasstub)isthatmockobjectscanbeprogrammedwithcustomexpectationsaccordingtotheneedsoftheunittest.ThisprocessintheMockitojargonisknownasstubbingmethods,inwhichthesemethodsbelongtothemocks.Bydefault,mockobjectsmimicthebehaviorofrealobjects.Inpracticalterms,itmeansthatmockobjectsreturnappropriatedummyvaluessuchasfalseforBooleantypes,nullforobjects,0forintegerorlongreturntypes,andsoon.MockitoallowsustochangethisbehaviorwitharichAPI,whichallowsstubbingtoreturnaspecificvaluewhenamethodiscalled.

Whenamockobjectisnotprogrammedwithanyexpectation(thatis,ithasnostubbingmethod),technicallyspeaking,itisnotamockobjectbutadummyobject(takealookatChapter1,RetrospectiveonSoftwareQualityandJavaTestingforthe

definition).

3. Verification:Attheendoftheday,wearecreatingtests,andthus,weneedtoimplementsomekindofverificationfortheSUT.MockitoprovidesapowerfulAPItocarryoutdifferenttypesofverifications.WiththisAPI,weassesstheinteractionswiththeSUTandDOCs,verifyingtheinvocationorderwithamock,orcapturingandverifyingtheargumentpassedtoastubbedmethod.Furthermore,theverificationcapabilitiesofMockitocanbecomplementedwiththebuilt-inassertioncapabilitiesofJUnitorusingathird-partyassertionlibrary(forexample,Hamcrest,AssertJ,orTruth).SeesectionAssertionswithinChapter3,JUnit5StandardTests.

ThefollowingtablesummarizestheMockitoAPIsgroupedbytheaforementionedphases:

MockitoAPI Description Phase

@Mock

ThisannotationidentifiesamockobjecttobecreatedbyMockito.ThisisusedtypicallyforDOC(s).

1.Mockingobjects

@InjectMocks

Thisannotationidentifiestheobjectinwhichthemocksaregoingtobeinjected.Thisisusedtypicallytotheunitwewanttotest,thatis,ourSUT.

1.Mockingobjects

@Spy

Inadditiontomocks,Mockitoallowsustocreatespyobjects(thatis,apartialmockimplementation,sincetheyusetherealimplementationinnon-stubbedmethods).

1.Mockingobjects

Mockito.when(x).thenReturn(y)

Mockito.doReturn(y).when(x)

Thesemethodsallowustospecifythevalue(y)thatshouldbereturnedbythe

2.Settingexpectations(stubbing

stubbedmethod(x)ofagivenmockobject.

methods)

Mockito.when(x).thenThrow(e)

Mockito.doThrow(e).when(x)

Thesemethodsallowustospecifytheexception(e)thatshouldbethrownwhencallingastubbedmethod(x)ofagivenmockobject.

2.Settingexpectations(stubbingmethods)

Mockito.when(x).thenAnswer(a)

Mockito.doAnswer(a).when(x)

Unlikereturningahardcodedvalue,adynamicuser-definedlogic(Answera)isexecutedwhenagivenmethod(x)ofthemockisinvoked.

2.Settingexpectations(stubbingmethods)

Mockito.when(x).thenCallRealMethod()

Mockito.doCallRealMethod().when(x)

Thismethodallowsustherealimplementationofamethodinsteadthemockedone.

2.Settingexpectations(stubbingmethods)

Mockito.doNothing().when(x)

Whenusingaspy,thedefaultbehavioriscallingtherealmethodsoftheobject.Inordertoavoidtheexecutionofavoidmethodx,thismethodisused.

2.Settingexpectations(stubbingmethods)

BDDMockito.given(x).willReturn(y)

BDDMockito.given(x).willThrow(e)

BDDMockito.given(x).willAnswer(a)

BDDMockito.given(x).willCallRealMethod()

Behaviour-drivendevelopmentisatestmethodologyinwhichtestsarespecifiedintermsofscenariosandimplementedasgiven(initialcontext),when(eventoccurs),andthen(ensuresomeoutcomes).MockitosupportsthistypeofteststhroughtheclassBDDMockito.Thebehaviorofthestubbedmethods(x)isequivalenttoMockito.when(x).

2.Settingexpectations(stubbingmethods)

Mockito.verify()

Thismethodverifiestheinvocationofmockobjects.Thisverificationcanbeoptionallyenhancedusingthefollowingmethods:

times(n):Thestubbedmethodisinvokedexactlyntimes.never():Thestubbedmethodisnevercalled.atLeastOnce():Thestubbedmethodisinvokedatleastonce.atLeast(n):Thestubbedmethodiscalledatleastntimes.atMost(n):Thestubbedmethodiscalledatthemostntimes.only():Amockfailsifanyothermethodiscalledonthemockobject.timeout(m):Thismethodiscalledinmmillisecondsatthemost.

3.Verification

Mockito.verifyZeroInteractions()

Mockito.verifyNoMoreInteractions()

Thesetwomethodsverifythatastubbedmethodhasnointeractions.Internally,theyusethesameimplementation.

3.Verification

@Captor

ThisannotationallowsustodefineanArgumentChaptorobject,aimedtoverifytheargumentspassedtoastubbedmethod.

3.Verification

Itfacilitatesverifying

Mockito.inOrder whetherinteractionswithamockwereperformedinagivenorder.

3.Verification

Theuseofthedifferentannotationsdepictedinprecedingthetable(@Mock,@InjectMocks,@Spy,and@Captor)isoptional,althoughitisrecommendablefortheshakeoftestreadability.Inotherwords,therearealternativestotheuseofannotationusingdifferentMockitoclasses.Forinstance,inordertocreateaMock,wecanusetheannotation@Mockasfollows:

@Mock

MyDocdocMock;

ThealternativetothiswouldbeusingthemethodMockito.mock,asfollows:MyDocdocMock=Mockito.mock(MyDoc.class)

ThefollowingsectionscontainscomprehensiveexamplesusingtheMockitoAPIsdescribedinprecedingtablewithinJupitertests.

JUnit5extensionforMockitoAtthetimeofthiswriting,thereisnoofficialJUnit5extensiontouseMockitoinJupitertests.Nevertheless,theJUnit5teamprovidesasimplereadytouseJavaclassimplementingasimplebuteffectiveextensionforMockito.ThisclasscanbefoundintheJUnit5userguide(http://junit.org/junit5/docs/current/user-guide/),anditscodeisthefollowing:

importstaticorg.mockito.Mockito.mock;

importjava.lang.reflect.Parameter;

importorg.junit.jupiter.api.extension.ExtensionContext;

importorg.junit.jupiter.api.extension.ExtensionContext.Namespace;

importorg.junit.jupiter.api.extension.ExtensionContext.Store;

importorg.junit.jupiter.api.extension.ParameterContext;

importorg.junit.jupiter.api.extension.ParameterResolver;

importorg.junit.jupiter.api.extension.TestInstancePostProcessor;

importorg.mockito.Mock;

importorg.mockito.MockitoAnnotations;

publicclassMockitoExtension

implementsTestInstancePostProcessor,ParameterResolver{

@Override

publicvoidpostProcessTestInstance(ObjecttestInstance,

ExtensionContextcontext){

MockitoAnnotations.initMocks(testInstance);

}

@Override

publicbooleansupportsParameter(ParameterContextparameterContext,

ExtensionContextextensionContext){

return

parameterContext.getParameter().isAnnotationPresent(Mock.class);

}

@Override

publicObjectresolveParameter(ParameterContextparameterContext,

ExtensionContextextensionContext){

returngetMock(parameterContext.getParameter(),extensionContext);

}

privateObjectgetMock(Parameterparameter,

ExtensionContextextensionContext){

Class<?>mockType=parameter.getType();

Storemocks=extensionContext

.getStore(Namespace.create(MockitoExtension.class,

mockType));

StringmockName=getMockName(parameter);

if(mockName!=null){

returnmocks.getOrComputeIfAbsent(mockName,

key->mock(mockType,mockName));

}else{

returnmocks.getOrComputeIfAbsent(mockType.getCanonicalName(),

key->mock(mockType));

}

}

privateStringgetMockName(Parameterparameter){

StringexplicitMockName=

parameter.getAnnotation(Mock.class).name()

.trim();

if(!explicitMockName.isEmpty()){

returnexplicitMockName;

}elseif(parameter.isNamePresent()){

returnparameter.getName();

}

returnnull;

}

}

Thisextension(amongothers)isplannedtobereleasedintheopensourceprojectJUnitPioneer(http://junit-pioneer.org/).ThisprojectismaintainedbyNicolaiParlog,JavadeveloperandauthoroftheblogCodeFX(https://blog.codefx.org/).

Inspectingtheprecedingclass,wecancheckthatitissimplyausecaseoftheJupiterextensionmodel(describedinchapter2,What’sNewInJUnit5,ofthisbook),whichimplementstheextensionscallbackTestInstancePostProcessorandParameterResolver.Thankstothefirst,afterthetestcaseisinstantiated,thepostProcessTestInstancemethodisinvoked,andinthebodyofthismethod,theinitializationofmocksiscarriedout:

MockitoAnnotations.initMocks(testInstance)

ThishasthesameeffectthatusingtheJUnit4runnerforMockito:@RunWith(MockitoJUnitRunner.class).

Inaddition,thisextensionalsoimplementstheinterfaceParameterResolver.Thatmeansthatdependencyinjectionatmethodlevelwillbeallowedintests,whichregistertheextension(@ExtendWith(MockitoExtension.class)).Inparticular,theannotationwillinjectmockobjectsfortestparametersannotatedwith@Mock(locatedinpackageorg.mockito).

Let’sseesomeexamplestoclarifytheuseofthisextensiontogetherwithMockito.Asusual,wecanfindthesourcecodeofthisexamplesontheGitHubrepositoryhttps://github.com/bonigarcia/mastering-junit5.Acopyoftheprecedingextension(MockitoExtension)iscontainedintheprojectjunit5-mockito.Toguidetheseexamples,weimplementatypicalusecaseinsoftwareapplications:theloginofauserinasoftwaresystem.

Inthisusecase,wesupposethatauserinteractswithasystemmadeupbythreeclasses:

LoginController:Theclasswhichreceivestherequestfromtheuser,returningaresponseasaresult.ThisrequestisdispatchedtotheLoginServicecomponent.LoginService:Thisclassimplementsthefunctionalityoftheusecase.To

thataim,itneedstoconfirmwhetherornottheuserisauthenticatedinthesystem.Tothat,itneedstoreadthepersistencelayer,implementedintheLoginRepositoryclass.LoginRepository:Thisclassallowstoaccessthepersistencelayerofthesystem,typicallyimplementedbymeansofadatabase.ThisclasscanalsobecalledDataAccessObject(DAO).

Intermsofcomposition,therelationshipofthesethreeclassesareisfollowing:

Loginusecaseclassdiagram(compositionrelationshipamongtheclasses)

Thesequencediagramofthetwobasicoperationsinvolvedintheusecase(loginandlogout)isdepictedinthefollowingchart:

Loginusecasesequencediagram

WeimplementthisexamplewithseveralsimpleJavaclasses.First,theLoginControllerusestheLoginServicebycomposition:

packageio.github.bonigarcia;

publicclassLoginController{

publicLoginServiceloginService=newLoginService();

publicStringlogin(UserFormuserForm){

System.out.println("LoginController.login"+userForm);

try{

if(userForm==null){

return"ERROR";

}elseif(loginService.login(userForm)){

return"OK";

}else{

return"KO";

}

}catch(Exceptione){

return"ERROR";

}

}

publicvoidlogout(UserFormuserForm){

System.out.println("LoginController.logout"+userForm);

loginService.logout(userForm);

}

}

TheUserFormobjectisasimpleJavaclass,sometimescalledPlain-OldJavaObject(POJO),withtwopropertiesusernameandpassword:

packageio.github.bonigarcia;

publicclassUserForm{

publicStringusername;

publicStringpassword;

publicUserForm(Stringusername,Stringpassword){

this.username=username;

this.password=password;

}

//Gettersandsetters

@Override

publicStringtoString(){

return"UserForm[username="+username+",password="+password

+"]";

}

}

Then,theservicedependsontherepository(LoginRepository)fordataaccess.Inthisexample,theservicealsoimplementsauserregistryusingaJavalistinwhichtheauthenticatedusersarestored:

packageio.github.bonigarcia;

importjava.util.ArrayList;

importjava.util.List;

publicclassLoginService{

privateLoginRepositoryloginRepository=newLoginRepository();

privateList<String>usersLogged=newArrayList<>();

publicbooleanlogin(UserFormuserForm){

System.out.println("LoginService.login"+userForm);

//Preconditions

checkForm(userForm);

//Sameusercannotbeloggedtwice

Stringusername=userForm.getUsername();

if(usersLogged.contains(username)){

thrownewLoginException(username+"alreadylogged");

}

//Calltorepositorytomakelogic

booleanlogin=loginRepository.login(userForm);

if(login){

usersLogged.add(username);

}

returnlogin;

}

publicvoidlogout(UserFormuserForm){

System.out.println("LoginService.logout"+userForm);

//Preconditions

checkForm(userForm);

//Usershouldbeloggedbeforehand

Stringusername=userForm.getUsername();

if(!usersLogged.contains(username)){

thrownewLoginException(username+"notlogged");

}

usersLogged.remove(username);

}

publicintgetUserLoggedCount(){

returnusersLogged.size();

}

privatevoidcheckForm(UserFormuserForm){

assertuserForm!=null;

assertuserForm.getUsername()!=null;

assertuserForm.getPassword()!=null;

}

}

Finally,theLoginRepositoryisasfollows.Forthesakeofsimplicity,insteadofaccessingarealdatabase,thiscomponentimplementsamapinwhichthecredentialsofthehypotheticaluserofthesystemarestored(wherekey=username,andvalue=password):

packageio.github.bonigarcia;

importjava.util.HashMap;

importjava.util.Map;

publicclassLoginRepository{

Map<String,String>users;

publicLoginRepository(){

users=newHashMap<>();

users.put("user1","p1");

users.put("user2","p3");

users.put("user3","p4");

}

publicbooleanlogin(UserFormuserForm){

System.out.println("LoginRepository.login"+userForm);

Stringusername=userForm.getUsername();

Stringpassword=userForm.getPassword();

returnusers.keySet().contains(username)

&&users.get(username).equals(password);

}

}

Now,wearegoingtotestoursystemusingJUnit5andMockito.Firstofall,wetestthecontrollercomponent.Sincewearedoingunittests,weneedtoisolatetheLoginControllerloginfromtherestofthesystem.Todothat,weneedtomockitsdependencies,inthisexample,theLoginServicecomponent.UsingtheSUT/DOCterminologyexplainedatthebeginning,inthistest,ourSUTis

theclassLoginControlleranditsDOCistheclassLoginService.

ToimplementourtestwithJUnit5,firstweneedtoregistertheMockitoExtensionwith@ExtendWith.Then,wedeclaretheSUTwith@InjectMocks(classLoginController)anditsDOCwith@Mock(classLoginService).Weimplementtwotests(@Test).Firstone(testLoginOk)specifieswhenthemethodloginofmockloginServiceiscalled,thismethodshouldreturntrue.Afterthat,theSUTisactuallyexercised,anditsresponseisverified(inthiscase,thereturnedStringmustbeOK).Moreover,theMockitoAPIisusedagaintoassessthatnomoreinteractionswiththemockLoginServiceisdone.Thesecondtest(testLoginKo)isequivalent,butstubbingthemethodlogintoreturnfalseandthereforetheresponseoftheSUT(LoginController)mustbeKOinthiscase:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.mockito.Mockito.verify;

importstaticorg.mockito.Mockito.verifyNoMoreInteractions;

importstaticorg.mockito.Mockito.verifyZeroInteractions;

importstaticorg.mockito.Mockito.when;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classLoginControllerLoginTest{

//Mockingobjects

@InjectMocks

LoginControllerloginController;

@Mock

LoginServiceloginService;

//Testdata

UserFormuserForm=newUserForm("foo","bar");

@Test

voidtestLoginOk(){

//Settingexpectations(stubbingmethods)

when(loginService.login(userForm)).thenReturn(true);

//ExerciseSUT

StringreseponseLogin=loginController.login(userForm);

//Verification

assertEquals("OK",reseponseLogin);

verify(loginService).login(userForm);

verifyNoMoreInteractions(loginService);

}

@Test

voidtestLoginKo(){

//Settingexpectations(stubbingmethods)

when(loginService.login(userForm)).thenReturn(false);

//ExerciseSUT

StringreseponseLogin=loginController.login(userForm);

//Verification

assertEquals("KO",reseponseLogin);

verify(loginService).login(userForm);

verifyZeroInteractions(loginService);

}

}

Ifweexecutethistest,simplyinspectingthetracesonthestandardoutputwecancheckthattheSUThavebeenactuallyexecuted.Inaddition,weassurethattheverificationstagehasbeensucceededinbothtestssincebothofthemhavepassed:

ExecutionofunittestofLoginControllerLoginTestwithJUnit5andMockito

Let’smovenowtootherexampleinwhichthenegativescenarios(thatis,errorsituations)aretestedforthecomponentLoginController.Thefollowingclasscontainstwotests,firstone(testLoginError)isdevotedtoassesstheresponseofthesystem(itshouldbeERROR)whenanullformisused.Inthesecondtest(testLoginException),weprogramthemethodloginofthemockloginServicetoraiseanexceptionwhenanyformisusedfirst.Then,weexercisetheSUT(LoginController)andassessthattheresponseisactuallyanERROR:

Notethatweareusingtheargumentmatcherany(providedoutoftheboxbyMockito)whensettingtheexpectationsforthemockmethod.

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.mockito.ArgumentMatchers.any;

importstaticorg.mockito.Mockito.when;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classLoginControllerErrorTest{

@InjectMocks

LoginControllerloginController;

@Mock

LoginServiceloginService;

UserFormuserForm=newUserForm("foo","bar");

@Test

voidtestLoginError(){

//Exercise

Stringresponse=loginController.login(null);

//Verify

assertEquals("ERROR",response);

}

@Test

voidtestLoginException(){

//Expectation

when(loginService.login(any(UserForm.class)))

.thenThrow(IllegalArgumentException.class);

//Exercise

Stringresponse=loginController.login(userForm);

//Verify

assertEquals("ERROR",response);

}

}

Again,whenrunningthetestsintheshell,wecanconfirmthatbothoftestsarecorrectlyexecutedandtheSUTisexercised:

ExecutionofunittestofLoginControllerErrorTestwithJUnit5andMockito

Let’sseeanexampleusingtheBDDstyle.Tothataim,theclassBDDMockitoisused.Noticethatthestaticmethodgivenofthisclassisimportedintheexample.Then,fourtestsareimplemented.Infact,thesefourtestsareexactlythesameimplementedinthepreviousexamples(LoginControllerLoginTestandLoginControllerErrorTest),butthistimeusingtheBDDstyleandamorecompactstyle(one-linercommands).

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.mockito.ArgumentMatchers.any;

importstaticorg.mockito.BDDMockito.given;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classLoginControllerBDDTest{

@InjectMocks

LoginControllerloginController;

@Mock

LoginServiceloginService;

UserFormuserForm=newUserForm("foo","bar");

@Test

voidtestLoginOk(){

given(loginService.login(userForm)).willReturn(true);

assertEquals("OK",loginController.login(userForm));

}

@Test

voidtestLoginKo(){

given(loginService.login(userForm)).willReturn(false);

assertEquals("KO",loginController.login(userForm));

}

@Test

voidtestLoginError(){

assertEquals("ERROR",loginController.login(null));

}

@Test

voidtestLoginException(){

given(loginService.login(any(UserForm.class)))

.willThrow(IllegalArgumentException.class);

assertEquals("ERROR",loginController.login(userForm));

}

}

Theexecutionofthistestclasssupposesthatfourtestsareexecuted.Asshowninthefollowingscreenshot,allofthempass:

ExecutionofunittestofLoginControllerBDDTestwithJUnit5andMockito

Let’smovenowtothenextcomponentofoursystem:LoginService.Inthefollowingexample,weaimtounittestthatcomponent,andthusfirstweusetheannotation@InjectMockstoinjecttheSUTinourtest.Then,theDOC(LoginRepository)ismockedusingtheannotation@Mock.Theclasscontainsthree

tests.Thefirst(testLoginOk)isdevotedtoverifytheansweroftheSUTwhenacorrectformisreceived.Thesecondtest(testLoginKo)verifiestheoppositescenario.Finally,thethirdtestalsoverifiesanerrorsituationofthesystem.Theimplementationofthisservicekeepsaregistryoftheuserslogged,andwillnotallowedtologinthesameusertwice.Forthisreason,weimplementedatest(testLoginTwice),whichverifiesthattheexceptionLoginExceptionisraisedwhenthesameusertriestologintwice:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertFalse;

importstaticorg.junit.jupiter.api.Assertions.assertThrows;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importstaticorg.mockito.ArgumentMatchers.any;

importstaticorg.mockito.Mockito.atLeast;

importstaticorg.mockito.Mockito.times;

importstaticorg.mockito.Mockito.verify;

importstaticorg.mockito.Mockito.when;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classLoginServiceTest{

@InjectMocks

LoginServiceloginService;

@Mock

LoginRepositoryloginRepository;

UserFormuserForm=newUserForm("foo","bar");

@Test

voidtestLoginOk(){

when(loginRepository.login(any(UserForm.class))).thenReturn(true);

assertTrue(loginService.login(userForm));

verify(loginRepository,atLeast(1)).login(userForm);

}

@Test

voidtestLoginKo(){

when(loginRepository.login(any(UserForm.class))).thenReturn(false);

assertFalse(loginService.login(userForm));

verify(loginRepository,times(1)).login(userForm);

}

@Test

voidtestLoginTwice(){

when(loginRepository.login(userForm)).thenReturn(true);

assertThrows(LoginException.class,()->{

loginService.login(userForm);

loginService.login(userForm);

});

}

}

Asusual,theexecutionofthetestinshellgivesusanideaofhowthingshavegone.Wecancheckthattheloginservicehasbeenexercisedfourtimes(since

inthethirdtest,wedidtwice).ButduetothefactthattheLoginExceptionwasexpected,thattestissucceeded(aswelltheothertwo):

ExecutionofunittestofLoginServiceTestwithJUnit5andMockito

Thefollowingclassprovidesasimpleexampleforcapturingtheargumentofamockobject.WedefineaclasspropertyoftypeArgumentCaptor<UserForm>,whichisannotatedwith@Captor.Then,inthebodyofthetest,theSUT(LoginServiceinthiscase)isexercisedandtheargumentofthemethodloginarecaptured.Finally,thevalueofthisargumentisassessed:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.mockito.Mockito.verify;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.ArgumentCaptor;

importorg.mockito.Captor;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classLoginServiceChaptorTest{

@InjectMocks

LoginServiceloginService;

@Mock

LoginRepositoryloginRepository;

@Captor

ArgumentCaptor<UserForm>argCaptor;

UserFormuserForm=newUserForm("foo","bar");

@Test

voidtestArgumentCaptor(){

loginService.login(userForm);

verify(loginRepository).login(argCaptor.capture());

assertEquals(userForm,argCaptor.getValue());

}

}

Onceagain,intheconsole,wecheckthattheSUTwasexercisedandthetestisdeclaredassuccessful:

ExecutionofunittestofLoginServiceChaptorTestwithJUnit5andMockito

ThelastexampleweseeinthischapterrelatedtoMockitohastodowiththeuseofanspy.Asintroducedbefore,bydefault,anspyusestherealimplementationinnon-stubbedmethods.Therefore,ifwedonotstubmethodsinanspyobject,whatwegetistherealobjectinourtest.Thisiswhathappensinthenextexample.Aswecansee,weareusingtheLoginServiceasourSUT,andthenwespytheobjectLoginRepository.Duetothefactthatinthebodyofthetestswearenotprogrammingexpectationsinthespyobject,weareassessingtherealsysteminthetest.

Allinall,thetestdataispreparedtogetalogincorrect(usingusernameasuserandpasswordasp1,whichispresentinthehardcodedvaluesintherealimplementationofLoginRepository),andthensomedummyvaluesforanunsuccessfullogin:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertFalse;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Spy;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classLoginServiceSpyTest{

@InjectMocks

LoginServiceloginService;

@Spy

LoginRepositoryloginRepository;

UserFormuserOk=newUserForm("user1","p1");

UserFormuserKo=newUserForm("foo","bar");

@Test

voidtestLoginOk(){

assertTrue(loginService.login(userOk));

}

@Test

voidtestLoginKo(){

assertFalse(loginService.login(userKo));

}

}

Intheshell,wecancheckthatbothtestswerecorrectlyexecuted,andinthiscase,therealcomponents(bothLoginServiceandLoginRepository)wereactuallyexercised:

ExecutionofunittestofLoginServiceSpyTestwithJUnit5andMockito

TheseexamplesdemonstrateseveralofthecapabilitiesofMockito,butofcoursenotall.Forfurtherinformation,visittheofficialMockitoreferenceathttp://site.mockito.org/.

SpringSpring(https://spring.io/)isanopensourceJavaframeworkforbuildingenterpriseapplications.ItwasfirstwrittenbyRodJohnsontogetherwithhisbookExpertOne-on-OneJ2EEDesignandDevelopmentinOctober2002.TheoriginalmotivationofSpringwasgettingridofthecomplexityofJ2EE,providingalight-weightinfrastructureaimedtoeasethedevelopmentofenterpriseapplicationusingsimplePOJOsasbuildingblocks.

SpringinanutshellThecoretechnologyoftheSpringFrameworkisknownasInversionofControl(IoC),whichistheprocessofinstantiatingobjectsoutsidetheclassinwhichtheseobjectsareactuallyused.TheseobjectsareknownasbeansorcomponentsintheSpringjargonandarecreatedassingletonobjectsbydefault.TheentityinchargeofthecreationofbeansisknownastheSpringIoCcontainer.ThisisachievedbyDependencyInjection(DI),whichistheprocessofprovidingdependenciesofoneobjectinsteadofconstructingthemitself.

IoCandDIareoftenusedinterchangeably.Nevertheless,asdepictedintheparagraphearlier,theseconceptsarenotexactlythesame(IoCisachievedthroughDI).

Asdepictedinthenextpartofthissection,Springisamodularframework.ThecorefunctionallyofSpring(thatis,IoC)isprovidedinthespring-contextmodule.Thismoduleprovidestheabilityofcreatingapplicationcontext,thatis,theSpring’sDIcontainer.TherearemanydifferentwaystodefineapplicationcontextsinSpring.Twoofthemostsignificanttypesarethefollowing:

AnnotationConfigApplicationContext:Applicationcontext,whichacceptsannotatedclassestoidentifytheSpringbeanstobeexecutedinthecontainer.Inthistypeofcontext,beansareidentifiedbyannotatingplainclasseswiththeannotation@Component.ItisnottheonlyonetodeclareaclassasaSpringbean.Therearefurtherstereotypesannotations:@Controller(stereotypeforpresentationlayer,usedinthewebmodule,MVC),@Repository(stereotypeforthepersistencelayer,usedinthedataaccessmodule,calledSpringData),and@Service(usedintheservicelayer).Thesethreeannotationsareusedtoseparatethelayersofanapplication.Finally,classesannotatedwith@ConfigurationallowstodefineSpringbeansbyannotatingmethodswith@Bean(theobjectreturnedbythesemethodswillbeSpringbeanslivinginthecontainer):

Springstereotypesusedtodefinebeans

ClassPathXmlApplicationContext:Applicationcontext,whichacceptsbeandefinitionsdeclaredinanXMLfilelocatedintheprojectclasspath.

Theannotation-basedcontextconfigurationwasintroducedinSpring2.5.TheSpringIoCcontaineristotallydecoupledfromtheformatinwhichconfigurationmetadata(thatis,beandefinition)isactuallywritten.Nowadaysmanydeveloperschoseannotation-basedconfigurationratherthanXMLbased.Forthisreason,inthisbook,wearegoingtouseonlyannotation-basedcontextconfigurationintheexamples.

Let’sseeasimpleexample.Firstofall,weneedtoincludethespring-contextdependencyinourproject.Forexample,asaMavendependency:

<dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-context</artifactId>

<version>${spring-context.version}</version>

</dependency>

Then,wecreateanexecutableJavaclass(thatis,withamainmethod).Noticethatinthisclassthereisoneannotationatclasslevel:@ComponentScan.ThisisaveryimportantannotationinSpring,sinceitallowstodeclarethepackageinwhichSpringwilllookforbeansdefinitionintheformofannotations.Ifspecificpackagesarenotdefined(justlikeintheexample),scanningwilloccurfromthepackageoftheclassthatdeclaresthisannotation(intheexamplethepackageio.github.bonigarcia).Inthebodyofthemainmethod,wecreatetheSpringapplicationcontextwithAnnotationConfigApplicationContext.Fromthatcontext,wegettheSpringcomponentwhoseclassisMessageComponent,andwewritetheresultofitsgetMessage()methodonthestandardoutput:

packageio.github.bonigarcia;

importorg.springframework.context.annotation.AnnotationConfigApplicationContext;

importorg.springframework.context.annotation.ComponentScan;

@ComponentScan

publicclassMySpringApplication{

publicstaticvoidmain(String[]args){

try(AnnotationConfigApplicationContextcontext=new

AnnotationConfigApplicationContext(

MySpringApplication.class)){

MessageComponentmessageComponent=context

.getBean(MessageComponent.class);

System.out.println(messageComponent.getMessage());

}

}

}

ThebeanMessageComponentisdefinedinthefollowingclass.Noticethatitis

declaredastheSpringcomponentsimplyusingtheannotation@Componentatclasslevel.Then,inthisexample,weareinjectinganotherSpringcomponentcalledMessageServiceusingtheclassconstructor:

packageio.github.bonigarcia;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.stereotype.Component;

@Component

publicclassMessageComponent{

privateMessageServicemessageService;

publicMessageComponent(MessageServicemessageService){

this.messageService=messageService;

}

publicStringgetMessage(){

returnmessageService.getMessage();

}

}

Atthispoint,itisworthreviewingthedifferentmannerstocarryoutdependencyinjectionofSpringcomponents:

1. Fieldinjection:Theinjectedcomponentisaclassfieldannotatedwith@Autowired,justliketheexamplebefore.Asabenefit,thiskindofinjectionremovescluttercodesuchassettermethodsorconstructorparameters.

2. Setterinjection:Theinjectedcomponentisdeclaredasafieldintheclass,andthenasetterforthisfieldiscreatedandannotatedwith@Autowired.

3. Constructorinjection:Thedependencyisinjectedintheclassconstructor,whichisannotatedwith@Autowired(3-ainthediagramhere).Thisisthewayshownintheexampleearlier.AsofSpring4.3,itisnotrequiredanymoretoannotatetheconstructorwith@Autowiredtocarryouttheinjection(3-b).

Thelatestwayofinjection(3-b)wasseveralbenefits,suchasthepromotionoftestabilitywithouttheneedofreflectionmechanism(implemented,forexample,bymockinglibrary).Inaddition,itcanmakedeveloperstothinkoverthedesignoftheclass,sincemanyinjecteddependenciessupposemanyconstructorparameters,andthisshouldbeavoided(Godobjectanti-pattern).

Differentwaysofdependencyinjection(Autowired)inSpring

ThelastcomponentinourexampleisnamedMessageService.NotethatisalsoaSpringcomponent,thistimeannotatedwith@Servicetoremarkitsservicenature(fromafunctionalperspective,itwouldbethesamethanannotatingtheclasswith@Component):

packageio.github.bonigarcia;

importorg.springframework.stereotype.Service;

@Service

publicclassMessageService{

publicStringgetMessage(){

return"Helloworld!";

}

}

Now,ifweexecutethemainclassofthisexample(calledMySpringApplication,seethesourcecodehere),wecreateanannotation-basedapplicationcontextwithatrywithresources(thiswaytheapplicationcontextwillbeclosedattheend).TheSpringIoCcontainerwillcreatetwobeans:MessageServiceandMessageComponet.Usingtheapplicationcontext,weseekthebeanMessageComponetandinvokeitsmethodgetMessage,whichisfinallywritteninthestandardoutput:

packageio.github.bonigarcia;

importorg.springframework.context.annotation.AnnotationConfigApplicationContext;

importorg.springframework.context.annotation.ComponentScan;

@ComponentScan

publicclassMySpringApplication{

publicstaticvoidmain(String[]args){

try(AnnotationConfigApplicationContextcontext=new

AnnotationConfigApplicationContext(

MySpringApplication.class)){

MessageComponentmessageComponent=context

.getBean(MessageComponent.class);

System.out.println(messageComponent.getMessage());

}

}

}

SpringmodulesTheSpringframeworkismodular,allowingdeveloperstouseonlytheneededmodulesprovidedbytheframework.Thecompletelistofthismodulescanbefoundonhttps://spring.io/projects.Thefollowingtablesummarizessomeofthemostimportantones:

Springproject Logo Description

SpringFramework

ProvidescoresupportforDI,transactionmanagement,webapplications(SpringMCV),dataaccess,messaging,andsoon.

SpringIOPlatform

BringstogetherthecoreSpringAPIsintoacohesiveandversionedfoundationalplatformformodernapplications.

SpringBoot

Simplifiesthecreationofstandalone,production-gradeSpring-basedapplicationswiththeminimalconfiguration.Itfollowstheconvention-over-configurationapproach.

SpringData

SimplifiesdataaccessbymeansofcomprehensiveAPIstoworkwiththerelationaldatabases,NoSQL,map-reducealgorithms,andsoon.

SpringCloud

Providesasetoflibrariesandcommonpatternsforbuildinganddeployingdistributedsystemsandmicroservices.

SpringSecurity

ProvidescustomizableauthenticationandauthorizationcapabilitiesforSpring-basedapplications.

SpringIntegration

Providesalightweight,POJO-basedmessagingforSpring-basedapplicationstointegratewithexternalsystems.

SpringBatch

Providesalightweightframeworkdesignedtoenablethedevelopmentofrobustbatchapplicationsforoperationsofenterprisesystems.

IntroductiontoSpringTestSpringamodulecalledspring-test,whichsupportsunitandintegrationtestingofSpringcomponents.Amongotherfeatures,thismoduleprovidestheabilitytocreateSpringapplicationcontextfortestingpurposesorcreatemockobjectsthattotestourcodeinisolation.Therearedifferentannotationssupportingthistestingcapabilities.Alistofthemostsignificantoneisthefollowing:

@ContextConfiguration:ThisannotationisusedtodeterminehowtoloadandconfigureanApplicationContextforintegrationtests.Forexample,itallowstoloadtheapplicationcontextfromannotatedclasses(usingtheelementclasses)orbeandefinitionsdeclaredinXMLfiles(usingtheelementlocations).@ActiveProfiles:Thisannotationisusedtoinstructthecontaineraboutwhichdefinitionprofilesshouldbeactiveduringtheapplicationcontextloading(forexample,developmentandtestprofiles).@TestPropertySource:Thisannotationisusedtoconfigurethelocationsofthepropertiesfilesandtheinlinepropertiestobeadded.@WebAppConfiguration:ThisannotationisusedtoinstructtheSpringcontextthatApplicationContextloadedisWebApplicationContext.

Inaddition,thespring-testmoduleoffersseveralcapabilitiestocarryoutdifferentactionstypicallyrequiredintests,namely:

Theorg.springframework.mock.webpackagecontainsasetofServletAPImockobjects,usefulfortestingwebcontexts.Forinstance,theobjectMockMvcallowstoperformHTTPrequests(POST,GET,PUT,DELETE,andsoon)andverifytheresponse(statuscode,contenttype,orresponsebody).Theorg.springframework.mock.jndipackagecontainsanimplementationoftheJavaNamingandDirectoryInterface(JNDI)SPI,whichcanbeusedtosetupasimpleJNDIenvironmentfortests.Forinstance,usingtheclassSimpleNamingContextBuilderwecanmakeaJNDIdatasourceavailableinourtests.Theorg.springframework.test.jdbcpackagecontainstheclassJdbcTestUtils,whichisacollectionofJDBCutilityfunctionsaimedtosimplifystandarddatabaseaccess.Theorg.springframework.test.utilpackagecontainstheclass

ReflectionTestUtils,whichisacollectionofutilitymethodstosetanon-publicfieldorinvokeaprivate/protectedsettermethodwhentestingtheapplicationcode.

TestingSpringBootapplicationsAsintroducedbefore,SpringBootisaprojectoftheSpringportfolioaimedtosimplifythedevelopmentofSpringapplications.ThemainbenefitsofusingSpringBootaresummarizedasfollows:

ASpringBootapplicationisjustaSpringApplicationContextinwhichtheprincipalconventionoverconfigurationisused.Thanktothis,itisfastertogetstartedwiththeSpringdevelopment.Theannotation@SpringBootApplicationisusedtoidentifythemainclassinaSpringBootproject.Arangeofnon-functionalfeaturesareprovidedoutofthebox:embeddedservletcontainers(Tomcat,Jetty,andUndertow),security,metrics,healthchecks,orexternalizedconfiguration.Acreationofstandalonerunningapplicationsthatjustrunusingthecommandjava-jar(evenforwebapplications).SpringBootcommandlineinterface(CLI)allowstorunGroovyscriptsforquicklyprototypingwithSpring.SpringBootworksinthesamewayasanystandardJavalibrary,thatis,touseit,wesimplyneedtoaddtheappropriatespring-boot-*.jarinourprojectclasspath(typicallyusingbuildtoolssuchasMavenorGradle).SpringBootprovidesanumberofstartersaimedtoeasetheprocessofaddingthedifferentlibrariestotheclasspath.Thefollowingtablecontainsseveralofthosestarters:

Name Description

spring-boot-

starter Corestarter,includingauto-configurationsupportandlogging

spring-boot-

starter-batch StarterforusingSpringBatch

spring-boot-

starter-cloud-

connectors

StarterforusingSpringCloudConnectors,whichsimplifiesconnectingtoservicesinCloudplatformslikeCloudFoundryandHeroku

spring-boot-

starter-data-jpa StarterforusingSpringDataJPAwithHibernate

spring-boot-

starter-

integration

StarterforusingSpringIntegration

spring-boot-

starter-jdbc StarterforusingJDBCwiththeTomcatJDBCconnectionpool

spring-boot-

starter-test

StarterfortestingSpringBootapplicationswithlibraries,includingJUnit,Hamcrest,andMockito

spring-boot-

starter-

thymeleaf

StarterforbuildingMVCwebapplicationsusingThymeleafviews

spring-boot-

starter-web

Starterforbuildingweb,includingREST,applicationsusingSpringMVC.UsesTomcatasthedefaultembeddedcontainer

spring-boot-

starter-

websocket

StarterforbuildingWebSocketapplicationsusingSpringFramework’sWebSocketsupport

ForcompleteinformationaboutSpringBootvisittheofficialreference:https://projects.spring.io/spring-boot/.

SpringBootprovidesdifferentcapabilitiestosimplifythetests.Forinstance,itprovidesthe@SpringBootTestannotation,whichisusedatclasslevelintestclasses.ThisannotationwillcreateApplicationContextforthesetests(similarlyto@ContextConfigurationbutforSpringBootbasedapplications).Aswehaveseeninthesectionbefore,inthespring-testmodule,weusetheannotation@ContextConfiguration(classes=…)tospecify,whichbeandefinition(Spring@Configuration)tobeloaded.WhentestingSpringBootapplicationsthisisoftennotrequired.SpringBoot’stestsannotationswillsearchtheprimaryconfigurationautomaticallyifnotexplicitlydefineone.Thesearchalgorithmworksupfromthepackagethatcontainsthetestuntilitfindsa@SpringBootApplicationannotatedclass.

SpringBootalsofacilitatestheuseofmocksforSpringcomponents.Tothat,theannotation@MockBeanisprovided.ThisannotationallowsdefiningaMockitomockforabeaninsideourApplicationContext.Itcanbenewbeans,butalsotoitcanreplaceasingleexistingbeandefinition.Mockbeansareautomaticallyresetaftereachtestmethod.Thismethodisusuallyknownasin-containertesting,incounterparttoout-of-container,inwhichamocklibrary(example,Mockito)isusedtounittesttheSpringcomponentsinisolationandwithouttheneedofaSpringApplicationContext.ForexampleofbothtypesofunittestsforSpringapplicationsisshowninthenextsection.

JUnit5extensionforSpringInordertointegratethespring-testcapabilitiesintoJUnit5’sJupiterprogrammingmodel,SpringExtensionhasbeendeveloped.Thisextensionispartofthespring-testmodule,asofSpring5.Let’sseeseveralexamplesofJUnit5andSpring5together.

Let’ssupposewewanttomakeanintegrationin-containertestoftheSpringapplicationdescribedintheformersection,madeupofthreeclasses:MySpringApplication,MessageComponent,andMessageService.Aswehavelearned,inordertoimplementaJupitertestagainstthisapplication,weneedtomakethefollowingsteps:

1. Annotateourtestclasswith@ContextConfigurationtospecifywhichApplicationContextneedstobeloaded.

2. Annotateourtestclasswith@ExtendWith(SpringExtension.class)toenablespring-testintoJupiter.

3. InjecttheSpringcomponentwewanttoassessinourtestclass.4. Implementourtest(@Test).

Forexample:packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.test.context.ContextConfiguration;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

@ExtendWith(SpringExtension.class)

@ContextConfiguration(classes={MySpringApplication.class})

classSimpleSpringTest{

@Autowired

publicMessageComponentmessageComponent;

@Test

publicvoidtest(){

assertEquals("Helloworld!",messageComponent.getMessage());

}

}

ThisisaverysimpleexampleinwhichtheSpringcomponentcalledMessageComponentisassessed.Whenthistestisstarted,ourApplicationContextisinitiatedwithandallourSpringcomponentsinside.Afterthat,inthis

example,thebeanMessageComponentisinjectedinthetest,whichisassessedsimplycallingthemethodgetMessage()andverifyingitsresponse.

Itisworthtoreviewwhichdependenciesareneededforthistest.WhenusingMaven,thesedependenciesarethefollowing:

<dependencies>

<dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-context</artifactId>

<version>${spring.version}</version>

</dependency>

<dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-test</artifactId>

<version>${spring.version}</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-api</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

</dependencies>

Ontheotherside,ifweuseGradle,thedependenciesclausewouldbeasfollows:

dependencies{

compile("org.springframework:spring-context:${springVersion}")

testCompile("org.springframework:spring-test:${springVersion}")

testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")

testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")

}

Notethatinbothcasesthespring-contextdependencyisneededtoimplementtheapplication,andthenweneedspring-testandjunit-jupitertotestit.Inordertoimplementtheequivalentapplicationandtest,butthistimeusingSpringBoot,firstwewouldneedtochangeourpom.xml(whenusingMaven):

<projectxmlns="http://maven.apache.org/POM/4.0.0"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0

http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<groupId>io.github.bonigarcia</groupId>

<artifactId>junit5-spring-boot</artifactId>

<version>1.0.0</version>

<parent>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-parent</artifactId>

<version>2.0.0.M3</version>

</parent>

<properties>

<junit.jupiter.version>5.0.0</junit.jupiter.version>

<junit.platform.version>1.0.0</junit.platform.version>

<java.version>1.8</java.version>

<maven.compiler.target>${java.version}</maven.compiler.target>

<maven.compiler.source>${java.version}</maven.compiler.source>

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

</properties>

<dependencies>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter</artifactId>

</dependency>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-test</artifactId>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-api</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

</dependencies>

<build>

<plugins>

<plugin>

<artifactId>maven-surefire-plugin</artifactId>

<dependencies>

<dependency>

<groupId>org.junit.platform</groupId>

<artifactId>junit-platform-surefire-provider</artifactId>

<version>${junit.platform.version}</version>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-engine</artifactId>

<version>${junit.jupiter.version}</version>

</dependency>

</dependencies>

</plugin>

<plugin>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-maven-plugin</artifactId>

<executions>

<execution>

<goals>

<goal>repackage</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

<repositories>

<repository>

<id>spring-milestones</id>

<url>https://repo.spring.io/libs-milestone</url>

</repository>

</repositories>

<pluginRepositories>

<pluginRepository>

<id>spring-milestones</id>

<url>https://repo.spring.io/milestone</url>

</pluginRepository>

</pluginRepositories>

</project>

Orourbuild.gradle(whenusingGradle):buildscript{

ext{

springBootVersion='2.0.0.M3'

junitPlatformVersion='1.0.0'

}

repositories{

mavenCentral()

maven{

url'https://repo.spring.io/milestone'

}

}

dependencies{

classpath("org.springframework.boot:spring-boot-gradle-

plugin:${springBootVersion}")

classpath("org.junit.platform:junit-platform-gradle-

plugin:${junitPlatformVersion}")

}

}

repositories{

mavenCentral()

maven{

url'https://repo.spring.io/libs-milestone'

}

}

applyplugin:'java'

applyplugin:'eclipse'

applyplugin:'idea'

applyplugin:'org.springframework.boot'

applyplugin:'io.spring.dependency-management'

applyplugin:'org.junit.platform.gradle.plugin'

jar{

baseName='junit5-spring-boot'

version='1.0.0'

}

compileTestJava{

sourceCompatibility=1.8

targetCompatibility=1.8

options.compilerArgs+='-parameters'

}

dependencies{

compile('org.springframework.boot:spring-boot-starter')

testCompile("org.springframework.boot:spring-boot-starter-test")

testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")

testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")

}

InordertotransformourrawSpringapplicationintoSpringBoot,ourcomponents(intheexamplecalledMessageComponentandMessageService)wouldbeexactlythesame,butourmainclasswouldchangeabit(seehere).Noticethatweusetheannotation@SpringBootApplicationatclasslevel,implementingthemainmethodwiththetypicallybootstrappingmechanismofSpringBoot.Justforloggingpurposes,weareimplementingamethodannotatedwith@PostConstruct.Thismethodwillbetriggeredjustbeforetheapplicationcontextisstarted:

packageio.github.bonigarcia;

importjavax.annotation.PostConstruct;

importorg.slf4j.Logger;

importorg.slf4j.LoggerFactory;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.boot.SpringApplication;

importorg.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication

publicclassMySpringBootApplication{

finalLoggerlog=LoggerFactory.getLogger(MySpringBootApplication.class);

@Autowired

publicMessageComponentmessageComponent;

@PostConstruct

privatevoidsetup(){

log.info("***{}***",messageComponent.getMessage());

}

publicstaticvoidmain(String[]args)throwsException{

newSpringApplication(MySpringBootApplication.class).run(args);

}

}

Theimplementationofthetestwouldbestraightforward.Theonlychangeweneedtodoistoannotatethetestwith@SpringBootTestinsteadof@ContextConfiguration(SpringBootautomaticallylooksforandstartsourApplicationContext):

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.boot.test.context.SpringBootTest;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

@ExtendWith(SpringExtension.class)

@SpringBootTest

classSimpleSpringBootTest{

@Autowired

publicMessageComponentmessagePrinter;

@Test

publicvoidtest(){

assertEquals("Helloworld!",messagePrinter.getMessage());

}

}

Executingthetestintheconsole,wecanseethatactuallytheapplicationisstartedbeforethetest(noticetheunmistakablespringASCIIbanneratthebeginning).

Afterthat,ourtestusestheApplicationContexttoverifyoneSpringcomponent,andasaresultthetestissucceeded:

ExecutionoftestusingSpringBoot

Tofinishwiththispart,weseeasimplewebapplicationimplementedwithSpringBoot.Withrespecttothedependencies,theonlychangeweneedtodoistoincludethestartedspring-boot-starter-web(insteadofthegenericspring-boot-starter).That’sit,wecanstartimplementingourSpring-basedwebapplication.

Wearegoingtoimplementaverysimple@Controller,thatis,theSpringbean,whichhandlestherequestfromthebrowsers.Inourexample,theonlyURLmappedbythecontrolleristhedefaultresource/:

packageio.github.bonigarcia;

importstaticorg.springframework.web.bind.annotation.RequestMethod.GET;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.stereotype.Controller;

importorg.springframework.web.bind.annotation.RequestMapping;

@Controller

publicclassWebController{

@Autowired

privatePageServicepageService;

@RequestMapping(value="/",method=GET)

publicStringgreeting(){

returnpageService.getPage();

}

}

ThiscomponentinjectsaservicecalledPageService,responsibleofreturningtheactualpagetobeloadedinresponsetotherequestto/.Thecontentofthisserviceisalsoverysimple:

packageio.github.bonigarcia;

importorg.springframework.stereotype.Service;

@Service

publicclassPageService{

publicStringgetPage(){

return"/index.html";

}

}

Byconvention(weareusingSpringBoothere),thestaticresourceforSpring-basedwebapplicationsarelocatedinafoldercalledstaticwithintheprojectclasspath.FollowingthestructureofMaven/Gradleproject,thisfolderislocatedinthesrc/main/resourcespath(seescreenshotbelow).Notethattherearetwopagesthere(weswitchfromonetotheotherinthetests,staytuned):

Contentoftheexampleprojectjunit5-spring-boot-web

Let’smoveonnottheinterestingpart:thetests.WeareimplementingthreeJupitertestsinthisproject.Thefirstoneisdevotedtoverifyadirectcalltothepage/index.html.Asdepictedbefore,thistestneedstousetheSpringextension(@ExtendWith(SpringExtension.class))andbedeclaredasSpringBoottest(@SpringBootTest).Tocarryouttherequesttowebapplication,weuseaninstanceoftheMockMvc,verifyingtheresponseinseveralways(HTTPresponsecode,content-type,andresponsecontentbody).ThisinstanceisautomaticallyconfiguredusingtheSpringBootannotation@AutoConfigureMockMvc.

OutofSpringBoot,insteadofusing@AutoConfigureMockMvc,theobjectMockMvccanbecreatedusingabuilderclasscalledMockMvcBuilders.Inthiscase,theapplicationcontextisusedasparameterforthatbuilder.

packageio.github.bonigarcia;

importstaticorg.hamcrest.core.StringContains.containsString;

importstatic

org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;

importstatic

org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;

importstatic

org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.springframework.beans.factory.annotation.Autowired;

import

org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;

importorg.springframework.boot.test.context.SpringBootTest;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

importorg.springframework.test.web.servlet.MockMvc;

@ExtendWith(SpringExtension.class)

@SpringBootTest

@AutoConfigureMockMvc

classIndexTest{

@Autowired

MockMvcmockMvc;

@Test

voidtestIndex()throwsException{

mockMvc.perform(get("/index.html")).andExpect(status().isOk())

.andExpect(content().contentType("text/html")).andExpect(

content().string(containsString("Thisisindex

page")));

}

}

Again,runningthistestintheshell,wecheckthattheapplicationisactuallyexecuted.Bydefault,theembeddedTomcatlistenstheport8080.Afterthat,testisexecutedsuccessfully:

Consoleoutputofin-containerfirsttest

Secondtestissimilar,butasadifferentialfactoritusesthetestcapability@MockBeantooverrideaspringcomponent(inthisexample,PageService)byamock.Inthebodyofthetest,firstwestubthemethodgetPageofthemocktochangethedefaultresponseofthecomponenttoredirect:/page.html.Asaresult,whenrequestingtheresource/inthetestwiththeobjectMockMvc,wewillobtainanHTTP302response(redirect)totheresource/page.html(whichisactuallyanexistingpage,asshownintheprojectscreenshot):

packageio.github.bonigarcia;

importstaticorg.mockito.Mockito.doReturn;

importstatic

org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;

importstatic

org.springframework.test.web.servlet.result.MockMvcResultMatchers.redirectedUrl;

importstatic

org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.springframework.beans.factory.annotation.Autowired;

import

org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;

importorg.springframework.boot.test.context.SpringBootTest;

importorg.springframework.boot.test.mock.mockito.MockBean;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

importorg.springframework.test.web.servlet.MockMvc;

@ExtendWith(SpringExtension.class)

@SpringBootTest

@AutoConfigureMockMvc

classRedirectTest{

@MockBean

PageServicepageService;

@Autowired

MockMvcmockMvc;

@Test

voidtest()throwsException{

doReturn("redirect:/page.html").when(pageService).getPage();

mockMvc.perform(get("/")).andExpect(status().isFound())

.andExpect(redirectedUrl("/page.html"));

}

}

Similarly,intheshellwecanconfirmthattheteststartstheSpringapplicationandthenitisexecutedcorrectly:

Consoleoutputofin-containersecondtest

Thelasttestinthisprojectisanexampleofanout-of-containertest.Intheprevioustestexamples,theSpringcontextwasusedwithinthetest.Ontheotherside,thefollowingreliescompletelyinMockitotoexercisethecomponentsofthesystem,thistimewithoutstartingtheSpringapplicationcontext.NotethatweareusingtheMockitoExtensionextensionhere,usingthecomponentWebControllerasourSUT(@InjectMocks)andthecomponentPageServiceasDOC(@Mock):

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importstaticorg.mockito.Mockito.times;

importstaticorg.mockito.Mockito.verify;

importstaticorg.mockito.Mockito.when;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

classOutOfContainerTest{

@InjectMocks

privateWebControllerwebController;

@Mock

privatePageServicepageService;

@Test

voidtest(){

when(pageService.getPage()).thenReturn("/my-page.html");

assertEquals("/my-page.html",webController.greeting());

verify(pageService,times(1)).getPage();

}

}

Thistime,intheexecutionofthetest,wedonotseespringtracessincetheapplicationcontainerwasnotstartedbeforeexecutingthetest:

Consoleoutputofout-of-containertest

SeleniumSelenium(http://www.seleniumhq.org/)isanopensourcewebtestingframework,sinceitsinceptionin2008hasestablisheditselfasthedefactowebautomationlibrary.Inthenextsection,wearegoingtoreviewthemainfeaturesofSeleniumandhowtouseitfromJUnit5tests.

SeleniuminanutshellSeleniumiscomposedbydifferentprojects.First,wefoundtheSeleniumIDE.ItisaFirefoxpluginimplementingtheRecordandPlayback(R&P)patternforwebapplications.Thus,itallowstorecordmanualinteractionswithFirefoxandtheplaybackthatrecordinginanautomatedfashion.

ThesecondprojectwasnamedSeleniumRemoteControl(RC).Thiscomponentwascapableofdrivingdifferenttypesofbrowserautomaticallyusingdifferentprogramminglanguages,suchasJava,C#,Python,Ruby,PHP,Perl,orJavaScript.ThiscomponentinjectedaJavaScriptlibrary(calledSeleniumCore)intheSUT.ThislibrarywascontrolledwithanintermediatecomponentcalledSeleniumRCServerwhichreceivesrequestsfromthetestcode(seethefollowingfigure).SeleniumRChadimportantsecurityproblemsduetosame-originpolicy.

Forthatreason,itwasdeprecatedon2016infavorofSeleniumWebDriver:

SeleniumRCschema

WereviewSeleniumRCjusttointroduceSeleniumWebDriver.Nowadays,SeleniumRCisdeprecatedanditsuseishighlydiscouraged.

Fromafunctionalpointofview,SeleniumWebDriverisequivalenttoRC(thatis,allowstocontrolbrowsersusingcode).Asadifferentialaspect,SeleniumWebDrivermakescallstothebrowserusingeachbrowser’snativesupportforautomation.ThelanguagebindingsprovidedbySeleniumWebDriver(labeledasTestinnextfigure)communicateswithandabrowser-specificbinary,whichactsasabridgebetweenrealbrowser.Forinstance,thisbinaryiscalledchromedriver(https://sites.google.com/a/chromium.org/chromedriver/)forChromeandgeckodriver(https://github.com/mozilla/geckodriver)forFirefox.The

communicationbetweentheTestandthedriverisdonewithJSONmessagesoverHTTPusingtheso-calledJSONWireProtocol.

Thismechanism,originallyproposedbytheWebDriverteamisstandardizedintheW3CWebDriverAPI(https://www.w3.org/TR/webdriver/):

SeleniumWebDriverschema

ThelastprojectoftheSeleniumportfolioiscalledSeleniumGrid.ItcanbeseenasextensionofSeleniumWebDriver,sinceitallowsdistributingbrowserexecutiononremotemachines.ThereareanumberofNodes,eachrunningondifferentoperatingsystemsandwithdifferentbrowsers.TheHubserverkeepsatrackofthenodesandproxiesrequeststothem(seefigurebelow):

SeleniumGridschema

ThefollowingtablesummarizesthemainfeaturesoftheWebDriverAPI:

WebDriverfeatureanddescription Example

WebDriverobjectcreation:

ItallowstocreateWebDriverinstances,whichareusedfromthetestcodetocontrolabrowserremotely.

WebDriverdriver=newFirefoxDriver();

WebDriverdriver=newChromeDriver();

WebDriverdriver=newOperaDriver();

Navigation:

ItallowstonavigatetoagivenURL.

driver.get("http://junit.org/junit5/");

Locateelements:

Itallowstoidentifyelementswithawebpage(WebElement)usingdifferentstrategies:byid,name,classname,CSSselector,linktext,tagname,orXPath

WebElementwebElement=

driver.findElement(By.id("id"));

driver.findElement(By.name("name"));

driver.findElement(By.className("class"));

driver.findElement(By.cssSelector("cssInput"));

driver.findElement(By.linkText("text"));

driver.findElement(By.tagName("tagname"));

driver.findElement(By.xpath("/html/body/div[4]"));

Interactwithelements:

FromagivenWebElement,wecancarryoutdifferenttypesofautomatedinteraction,suchasclickelements,typetextorclearinputfields,readattributes,andsoon.

webElement.click();

webElement.sendKeys("text");

webElement.clear();

Stringtext=webElement.getText();

Stringhref=webElement.getAttribute("href");

Stringcss=webElement.getCssValue("css");

Dimensiondim=webElement.getSize();

booleanenabled=webElement.isEnabled();

booleanselected=webElement.isSelected();

booleandisplayed=webElement.isDisplayed();

Handlewaits:

WebDrivercanhandlewaitbothexplicitandimplicitly.

//Explicit

WebDriverWaitwait=newWebDriverWait(driver,

30);

wait.until(ExpectedConditions);

//Implicitwait

driver.manage().timeouts().implicitlyWait(30,

SECONDS);

XPath(XMLPathLanguage)isalanguagetobuildexpressionstoparseandprocessXML-likedocuments(forexample,HTML)

JUnit5extensionforSeleniumInordertosimplifytheuseofSeleniumWebDriverinJUnit5,theopensourceJUnit5extensioncalledselenium-jupitercanbeused.ThisextensionhasbeenbuiltusingthedependencyinjectioncapabilityprovidedbytheextensionmodelofJUnit5.Thankstothisfeature,differenttypesobjectscanbeinjectedinJUnit5in@Testmethodsasparameters.Concretely,selenium-jupiterallowstoinjectsubtypesoftheWebDriverinterface(forexample,ChromeDriver,FirefoxDriver,andsoon).

Usingselenium-jupiterisveryeasy.First,weneedtoimportthedependencyinourproject(typicallyastestdependency).InMaven,itisdoneasfollows:

<dependency>

<groupId>io.github.bonigarcia</groupId>

<artifactId>selenium-jupiter</artifactId>

<version>${selenium-jupiter.version}</version>

<scope>test</scope>

</dependency>

selenium-jupiterdependsonseverallibraries,whichareaddedinourprojectastransitivedependencies,namely:

Selenium-java(org.seleniumhq.selenium:selenium-java):JavalibraryforSeleniumWebDriver.WebDriverManager(io.github.bonigarcia:webdrivermanager):JavalibraryforautomaticSeleniumWebDriverbinariesmanagementinruntimeforJava(https://github.com/bonigarcia/webdrivermanager).Appium(io.appium:java-client):JavaclientforAppium,testingframeworkthatextendsSeleniumtoautomatetestingofnative,hybrid,andmobilewebapps(http://appium.io/).

Onceselenium-jupiterisincludedinourproject,weneedtodeclareselenium-jupiterextensioninourJUnit5test,simplyannotatingitwith@ExtendWith(SeleniumExtension.class).Then,weneedtoincludeoneormoreparametersinour@TestmethodswhosetypesimplementtheWebDriverinterface,andselenium-jupitercontrolthelifecycleoftheWebDriverobjectinternally.HeWebDriversubtypessupportedbyselenium-jupiterarethefollowing:

ChromeDriver:ThisisusedtocontrolGoogleChromebrowser.FirefoxDriver:ThisisusedtocontrolFirefoxbrowser.

EdgeDriver:ThisisusedtocontrolMicrosoftEdgebrowser.OperaDriver:ThisisusedtocontrolOperabrowser.SafariDriver:ThisisusedtocontrolAppleSafaribrowser(onlypossibleinOSXElCapitanorgreater).HtmlUnitDriver:ThisisusedtocontrolHtmlUnit(headlessbrowser,thatis,abrowserwithoutGUI).PhantomJSDriver:ThisisusedtocontrolPhantomJS(anotherheadlessbrowser).InternetExplorerDriver:ThisisusedtocontrolMicrosoftInternetExplorer.Althoughthisbrowserissupported,InternetExplorerisdeprecated(infavorofEdge)anditsuseishighlydiscouraged.RemoteWebDriver:Thisisusedtocontrolremotebrowsers(SeleniumGrid).AppiumDriver:Thisisusedtocontrolmobiledevices(AndroidandiOS).

Considerthefollowingclass,whichusesselenium-jupiter,thatis,declaringtheSeleniumextensionusing@ExtendWith(SeleniumExtension.class).Thisexampledefinesthreetests,whicharegoingbeexecutedusinglocalbrowsers.Firstone(namedtestWithChrome)usesChromeasbrowsers.Tothataim,andthankstothedependencyinjectionfeatureofselenium-jupiter,themethodsimplyneedstodeclareamethodargumentusingthetypeChromeDriver.Then,inthebodyofthetest,theWebDriverAPIisinvokedinthatobject.Notethatthistestsimpleopensawebpageandassertsthatthetitleisasexpected.Next,test(testWithFirefoxAndOpera)issimilar,butthistimeusingtwodifferentbrowsersatthesametime:Firefox(usinganinstanceofFirefoxDriver)andOpera(usinganinstanceofOperaDriver).Thethirdandlasttest(testWithHeadlessBrowsers)declaresandusestwoheadlessbrowsers(HtmlUnitandPhantomJS):

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertNotNull;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.openqa.selenium.chrome.ChromeDriver;

importorg.openqa.selenium.firefox.FirefoxDriver;

importorg.openqa.selenium.htmlunit.HtmlUnitDriver;

importorg.openqa.selenium.opera.OperaDriver;

importorg.openqa.selenium.phantomjs.PhantomJSDriver;

@ExtendWith(SeleniumExtension.class)

publicclassLocalWebDriverTest{

@Test

publicvoidtestWithChrome(ChromeDriverchrome){

chrome.get("https://bonigarcia.github.io/selenium-jupiter/");

assertTrue(chrome.getTitle().startsWith("selenium-jupiter"));

}

@Test

publicvoidtestWithFirefoxAndOpera(FirefoxDriverfirefox,

OperaDriveropera){

firefox.get("http://www.seleniumhq.org/");

opera.get("http://junit.org/junit5/");

assertTrue(firefox.getTitle().startsWith("Selenium"));

assertTrue(opera.getTitle().equals("JUnit5"));

}

@Test

publicvoidtestWithHeadlessBrowsers(HtmlUnitDriverhtmlUnit,

PhantomJSDriverphantomjs){

htmlUnit.get("https://bonigarcia.github.io/selenium-jupiter/");

phantomjs.get("https://bonigarcia.github.io/selenium-jupiter/");

assertTrue(htmlUnit.getTitle().contains("JUnit5extension"));

assertNotNull(phantomjs.getPageSource());

}

}

Inordertoexecuteproperlythistestclass,therequiredbrowsers(Chrome,Firefox,andOpera)shouldbeinstalledbeforehandrunningit.Ontheotherhand,theheadlessbrowsers(HtmlUnitandPhantomJS)areconsumedasJavadependencies,andsothereisnoneedtoinstallthemmanually.

Let’sseeanotherexample,thistimeusingremotebrowsers(thatis,SeleniumGrid).Again,thisclassusestheselenium-jupiterextension.Thetest(testWithRemoteChrome)declaresasingleparametercalledremoteChrome,oftypeRemoteWedbrider.Thisargumentisannotatedwith@DriverUrland@DriverCapabilities,specifyingtheSeleniumServer(orHub)URLandtherequiredcapabilitiesrespectively.Regardingthecapabilities,weareconfiguringtouseaChromebrowserversion59:

Torunthistestproperly,aSeleniumServershouldupandrunninginthelocalhost,andanode(Chrome59)needstoberegisteredintheHub.

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.openqa.selenium.remote.RemoteWebDriver;

@ExtendWith(SeleniumExtension.class)

publicclassRemoteWebDriverTest{

@Test

voidtestWithRemoteChrome(

@DriverUrl("http://localhost:4444/wd/hub")

@DriverCapabilities(capability={

@Capability(name="browserName",value="chrome"),

@Capability(name="version",value="59")})

RemoteWebDriverremoteChrome)

throwsInterruptedException{

remoteChrome.get("https://bonigarcia.github.io/selenium-

jupiter/");

assertTrue(remoteChrome.getTitle().contains("JUnit5

extension"));

}

}

Inthelastexampleofthissection,weuseAppiumDriver.Concretely,wesetupascapabilitiestheuseofaChromebrowserinanAndroidemulateddevice(@DriverCapabilities).Again,thisemulatorneedstobeupandrunninginthemachinerunningthetestbeforehand:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertTrue;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.openqa.selenium.By;

importorg.openqa.selenium.WebElement;

importorg.openqa.selenium.remote.DesiredCapabilities;

importio.appium.java_client.AppiumDriver;

@ExtendWith(SeleniumExtension.class)

publicclassAppiumTest{

@DriverCapabilities

DesiredCapabilitiescapabilities=newDesiredCapabilities();

{

capabilities.setCapability("browserName","chrome");

capabilities.setCapability("deviceName","Android");

}

@Test

voidtestWithAndroid(AppiumDriver<WebElement>android){

Stringcontext=android.getContext();

android.context("NATIVE_APP");

android.findElement(By.id("com.android.chrome:id/terms_accept"))

.click();

android.findElement(By.id("com.android.chrome:id/negative_button"))

.click();

android.context(context);

android.get("https://bonigarcia.github.io/selenium-jupiter/");

assertTrue(android.getTitle().contains("JUnit5extension"));

}

}

Forfurtherexamplesofselenium-jupiter,visithttps://bonigarcia.github.io/selenium-jupiter/.

CucumberCucumber(https://cucumber.io/)istestingframeworkaimedtoautomateacceptancetestswrittenfollowingaBehavior-DrivenDevelopment(BDD)style.CucumberhasbeenwritteninRuby,althoughimplementationsforotherlanguages(includingJava,JavaScript,andPython)areavailable.

CucumberinanutshellCucumberexecutestestsspecifiedwritteninlanguagecalledGherkin.Itisaplaint-textnaturallanguage(forexample,Englishoroneofother60+languagessupportedbyCucumber)withagivenstructure.Gherkinhasbeendesignedtobeusedbynon-programmers,typicallycustomers,businessanalysis,managers,andsoon.

TheextensionforGherkinfilesis.feature.

InaGherkinfile,non-blanklinescanstartwithakeyword,followedbytextinnaturallanguage.Themainkeywordsarethefollowing:

Feature:High-leveldescriptionofthesoftwarefeaturetobetested.Itcanbeseenasausecasedescription.Scenario:Concreteexamplethatillustratesabusinessrule.Scenariosfollowthesamepattern:

Describeinitialcontext.Describeanevent.Describetheexpectedoutcome.

TheseactionsareknownintheGherkinjargonassteps,whicharemainlyGiven,When,orThen:

Therearetwoadditionalsteps:And(usedforlogicalandfordifferentsteps)andBut(usedinfornegativeformofAnd).

Given:Preconditionsandinitialstatebeforethestartofatest.When:Actionstakenbyauserduringatest.Then:OutcomefromactionstakenintheWhenclause.Background:Toavoidrepeatstepsindifferentscenarios,thekeywordbackgroundallowstodeclaredthesesteps,whicharereusedinsubsequentscenarios.ScenarioOutline:Scenariosinwhichstepsaremarkedwithvariables(usingthesymbols<and>).Examples:Ascenariooutlinedeclarationisalwaysfollowedbyoneormoreexamplessections,whichisacontainertablewithvaluesforthe

declaredvariablesintheScenarioOutline.

Whenonelinedoesnotstartwithakeyword,thatlineisnotinterpretedbyCucumber.Itisusedtocustomdescription.

Oncewedefinedourfeaturestobetestedweneedwhatitiscalledstepsdefinition,whichallowstotranslateplaintextGherkinintoactionsthatactuallyexerciseourSUT.InJava,itcanbeeasilydonebyannotationstoannotatemethodsforthestepimplementation:@Given,@Then,@When,@And,and@But.Thestringvalueofeachstepcancontainregularexpressionwhicharemappedasfieldsinthemethod.Seeanexampleinthenextsection.

JUnit5extensionforCucumberThelatestversionsoftheCucumberartifactsforJavaincorporatesaJUnit5extensionforCucumber.ThissectioncontainsacompleteexampleofafeaturedefinedinGherkinandtheJUnit5toexecuteitwithCucumber.Asusual,thesourcecodeofthisexampleishostedonGitHub(https://github.com/bonigarcia/mastering-junit5).

Thestructureoftheprojectcontainingthisexampleisasfollows:

JUnit5withCucumberprojectstructureandcontent

Firstofall,weneedtocreateourGherkinfile,whichisaimedtotestasimplecalculatorsystem.ThiscalculatorwillbetheSUTorourtest.Thecontentofourfeaturefileisasfollows:

Feature:BasicArithmetic

Background:ACalculator

GivenacalculatorIjustturnedon

Scenario:Addition

WhenIadd4and5

Thentheresultis9

Scenario:Substraction

WhenIsubstract7to2

Thentheresultis5

ScenarioOutline:Severaladditions

WhenIadd<a>and<b>

Thentheresultis<c>

Examples:Singledigits

|a|b|c|

|1|2|3|

|3|7|10|

Then,weneedtoimplementourstepsdefinition.Asdescribedearlier,weuseannotationsandregularexpressiontomapthetextcontainedintheGherkinfiletotheactualexerciseofSUTdependingonthestep:

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importcucumber.api.java.en.Given;

importcucumber.api.java.en.Then;

importcucumber.api.java.en.When;

publicclassCalculatorSteps{

privateCalculatorcalc;

@Given("^acalculatorIjustturnedon$")

publicvoidsetup(){

calc=newCalculator();

}

@When("^Iadd(\\d+)and(\\d+)$")

publicvoidadd(intarg1,intarg2){

calc.push(arg1);

calc.push(arg2);

calc.push("+");

}

@When("^Isubstract(\\d+)to(\\d+)$")

publicvoidsubstract(intarg1,intarg2){

calc.push(arg1);

calc.push(arg2);

calc.push("-");

}

@Then("^theresultis(\\d+)$")

publicvoidthe_result_is(doubleexpected){

assertEquals(expected,calc.value());

}

}

Ofcourse,westillneedtoimplementourJUnit5test.ToachievetheintegrationofCucumberandJUnit5,theCucumberextensionneedstoberegisteredinourclassbymeansof@ExtendWith(CucumberExtension.class).Internally,CucumberExtensionimplementstheParameterResolvercallbackoftheJupiterextensionmodel.TheobjectiveistoinjectthecorrespondingtestsoftheCucumberfeatureasJupiterDynamicTestobjectsinthetests.Noticeintheexamplehowa@TestFactoryisused.

Optionally,wecanannotateourtestclasswith@CucumberOptions.ThisannotationallowstoconfiguretheCucumbersettingsforourtest.Theallowedelementsforthisannotationare:

plugin:Built-informatter:pretty,progress,JSON,usage,amongothers.

Default:{}.dryRun:Checksifallstepshavedefinitions.Default:false.features:Pathsofthefeaturesfiles.Default:{}.glue:Pathsforstepdefinitions.Default:{}.tags:Tagsinthefeaturestobeexecuted.Default{}.monochrome:Displaysconsoleoutputinareadableway.Default:false.format:Reportsformattertobeused.Default:{}.strict:Failsifthereareundefinedorpendingsteps.Default:false.

packageio.github.bonigarcia;

importjava.util.List;

importjava.util.stream.Collectors;

importjava.util.stream.Stream;

importorg.junit.jupiter.api.DynamicTest;

importorg.junit.jupiter.api.TestFactory;

importorg.junit.jupiter.api.extension.ExtendWith;

importcucumber.api.CucumberOptions;

importcucumber.api.junit.jupiter.CucumberExtension;

@CucumberOptions(plugin={"pretty"})

@ExtendWith(CucumberExtension.class)

publicclassCucumberTest{

@TestFactory

publicStream<DynamicTest>runCukes(Stream<DynamicTest>scenarios){

List<DynamicTest>tests=scenarios.collect(Collectors.toList());

returntests.stream();

}

}

Atthispoint,weareabletoexecuteourCucumbersuitewithJUnit5.InthefollowingexampleweseetheoutputwhenrunningthetestwithGradle:

ExecutionofJUnit5usingCucumberwithGradle

DockerDocker(https://www.docker.com/)isanopensourcesoftwaretechnology,whichallowstopackandrunanyapplicationasalightweightandportablecontainer.Itprovidesacommand-lineprogram,abackgrounddaemon,andasetofremoteservicesthatsimplifiesthelifecycleofcontainers.

DockerinanutshellHistorically,UNIX-styleoperatingsystemsusedthetermjailtodescribemodifiedisolatedruntimeenvironments.TheLinuxContainers(LXC)projectstartedin2008andbroughttogethercgroups,kernelnamespaces,orchroot(amongothers)toprovidecompleteisolationexecution.TheproblemwithLXCisthedifficulty,andforthatreason,theDockertechnologyemerged.

DockerhidesinunderlyingcomplexityoftheaforementionedresourceisolationfeaturesoftheLinuxkernel(cgroups,kernelnamespaces,andsoon)toallowindependentcontainerstorunwithinasingleLinuxinstance.Dockerprovidesahigh-levelAPI,whichallowstopack,shipandrunanyapplicationasacontainer.

InDocker,acontainercontainsanapplicationanditsdependenciestogether.MultiplecontainerscanrunonthesamemachineandsharethesameOSkernelwithothercontainers.Eachcontainerisrunningasisolatedprocessinuserspace.

Unlikevirtualmachines(VMs),inDockercontainersthereisnoneedofusingahypervisor,whichisthesoftwarethatallowstocreateandrunsVM(example;VirtualBox,VMware,QEMUorVirtualPC).

ThearchitectureofVMandcontaineraredepictedinthefollowingdiagram:

Virtualmachineversuscontainer

TheDockerplatformhastwocomponents:theDockerEngine,whichis

responsibleforcreatingandrunningcontainers;andtheDockerHub(https://hub.docker.com/),acloudservicefordistributingcontainers.TheDockerHubprovidesanenormousnumberofpubliccontainerimagesfordownload.TheDockerEngineisaclient-serverapplicationcomposedbythreemajorcomponents:

Aserverimplementedasadaemonprocess(thedockerdcommand).ARESTAPI,whichspecifiesinterfacesthatprogramscanusetotalktothedaemonandinstructitwhattodo.Acommandlineinterface(CLI)client(thedockercommand).

JUnit5extensionforDockerNowadays,containersarechangingthewaywedevelop,distribute,andrunsoftware.ThisisespeciallyinterestingforContinuousIntegration(CI)testingenvironment,inwhichtheconvergencewithDockerhasadirectimpactontheimprovementofefficiency.

RegardingJUnit5,atthemomentofthiswritingthereisanopensourceJUnit5extensionforDocker,namedJUnit5-Docker(https://faustxvi.github.io/junit5-docker/).ThisextensionactsasclientoftheDockerengineandallowstostartaDockercontainer(downloadedfromtheDockerHub),beforerunningthetestsofaclass.Thatcontainerisstoppedattheendofthetests.InordertouseJUnit5-Docker,firstweneedtoaddthedependencyinourproject.InMaven:

<dependency>

<groupId>com.github.faustxvi</groupId>

<artifactId>junit5-docker</artifactId>

<version>${junit5-docker.version}</version>

<scope>test</scope>

</dependency>

InGradle:dependencies{

testCompile("com.github.faustxvi:junit5-docker:${junitDockerVersion}")

}

TheuseofJUnit5-Dockerisquitestraightforward.Wesimplyneedtoannotateourtestclasswith@Docker.Theelementsavailableinthisannotationarethefollowing:

image:Dockerimagetobestarted.ports:PortmappingfortheDockercontainer.Thisisrequiredsinceatleastoneportmustbevisibleforthecontainertobeuseful.environments:Optionalenvironmentvariablestopasstothedockercontainer.Default:{}.waitFor:Optionallogtowaitforbeforerunningthetests.Default:@WaitFor(NOTHING).newForEachCase:Booleanflag,whichdeterminesifthecontainershouldberecreatedforeachtestcase.Thisvaluewillbefalseifitshouldbecreatedonlyonceforthetestclass.Default:true.

Considerthefollowingexample.Thistestclassusesthe@DockerannotationtostartaMySqlcontainer(containerimageMySQL)andthebeginningofeach

test.Theinternalcontainerportis3306,whichwillbemappedtothehostport8801.Then,severalenvironmentattributesaredefined(MySqlrootpassword,defaultdatabase,andusernameandpassword).Theexecutionofthetestwillnotstartuntilthetracemysqld:readyforconnectionsappearsinthecontainerlog(whichindicatesthattheMySqlinstanceisupandrunning).Inthebodyofthetest,westartaJDBCconnectionagainsttheMySQLinstancerunninginthecontainer.

ThistesthasbeenexecutedinaWindowsmachine.Forthatreason,thehostoftheJDBCURLis192.168.99.100,whichistheIPfortheDockerMachine.ItisatoolwhichallowstoinstallDockerEngineonvirtualhosts,suchasWindowsorMac(https://docs.docker.com/machine/).InaLinuxmachine,thisIPcouldbe127.0.0.1(localhost).

packageio.github.bonigarcia;

importstaticorg.junit.jupiter.api.Assertions.assertFalse;

importjava.sql.Connection;

importjava.sql.DriverManager;

importorg.junit.jupiter.api.Test;

importcom.github.junit5docker.Docker;

importcom.github.junit5docker.Environment;

importcom.github.junit5docker.Port;

importcom.github.junit5docker.WaitFor;

@Docker(image="mysql",ports=@Port(exposed=8801,inner=3306),environments

={

@Environment(key="MYSQL_ROOT_PASSWORD",value="root"),

@Environment(key="MYSQL_DATABASE",value="testdb"),

@Environment(key="MYSQL_USER",value="testuser"),

@Environment(key="MYSQL_PASSWORD",value="secret"),},

waitFor=@WaitFor("mysqld:readyforconnections"))

publicclassDockerTest{

@Test

voidtest()throwsException{

Class.forName("com.mysql.jdbc.Driver");

Connectionconnection=DriverManager.getConnection(

"jdbc:mysql://192.168.99.100:8801/testdb","testuser",

"secret");

assertFalse(connection.isClosed());

connection.close();

}

}

TheexecutionofthistestintheDockerWindowsterminalisasfollows:

ExecutionoftestusingJUnit5-Dockerextension

AndroidAndroid(https://www.android.com/)isanopensourcemobileoperatingsystembasedonamodifiedversionofLinux.ItwasoriginallydevelopedbyastartupnamedAndroid,acquiredandchampionedbyGooglein2005.

AccordingtothereportbyGartnerInc.(AmericanITresearchandadvisorycompany),in2017AndroidandiOSaccountmorethan99%ofglobalsmartphonesales,asshowninthefollowingchart:

Smartphoneoperativesystemmarket.Picturecreatedbywww.statista.com.

AndroidinanutshellAndroidisaLinux-basedsoftwarestackdividedintoseverallayers.Thoselayers,fromdowntotoparethefollowing:

Linuxkernel:ThisisthefoundationoftheAndroidplatform.Thislayercontainsallthelow-leveldevicedriversforthevarioushardwarecomponentsofanAndroiddevice.HardwareAbstractionLayer(HAL):Thislayerprovidesstandardinterfacesthatexposehardwarecapabilitiestothehigher-levelJavaAPIframework.AndroidRuntime(ART):Itprovidesaruntimeenvironmentfor.dexfiles,abytecodeformatdesignedforminimalmemoryfootprint.ARTwasthefirstreleaseonAndroid5.0(seetablebelow).Priortothatversion,DalvikwastheAndroidruntime.NativeC/C++libraries:ThislayercontainsnativelibrarieswritteninCandC++,suchasOpenGLESforhigh-performance2Dand3Dgraphicsprocessing.JavaAPIframework:Theentirefeature-setofAndroidisavailablefordevelopersthroughAPIswritteninJava.TheseAPIsarethebuildingblockforcreatingAndroidapps,forinstance:theViewSystem(forappsUIs),theResourceManager(forI18N,graphics,layouts),theNotificationManager(forcustomalertsinthestatusbar),theActivityManager(tomanagetheappslifecycle),ortheContentProvider(toenableappsaccessdatafromotherapps,suchastheContacts,andsoon).Apps:Androidcomeswithasetofcoreapps,suchasPhone,Contacts,Browser,andsoon.Inaddition,manyothersappscanbedownloadedandinstalledfromGooglePlay(formerlyAndroidMarket):

Androidlayeredarchitecture

Androidhasgonethroughquiteanumberofupdatessinceitsfirstrelease,asdescribedinthefollowingtable:

Androidversion Codename API

levelLinuxkernelversion Releasedate

1.5 Cupcake 3 2.6.27April30,2009

1.6 Donut 4 2.6.29September15,2009

2.0,2.1

Eclair 5,6,7 2.6.29 October26,

2009

2.2 Froyo 8 2.6.32 May20,2010

2.3 Gingerbread 9,10 2.6.35December6,2010

3.0,3.1,3.2 Honeycomb11,12,13 2.6.36

February22,2011

4.0 IceCreamSandwich 14,15 3.0.1 October18,

2011

4.1,4.2,4.3 JellyBean 16,17,18

3.0.31,3.0.21,3.4.0 July9,2012

4.4

KitKat 19,20 3.10 October31,

2013

5.0,5.1 Lollipop 21,22 3.16.1 November12,2014

6.0

Marshmallow 23 3.18.10 October5,2015

7.0,7.1 Nougat 24,25 4.4.1 August22,2016

8.0 AndroidO 26 TBA TBA

Fromadeveloperpointofview,Androidprovidesarichapplicationframework,whichallowstobuildappsformobiledevices.AndroidappsarewrittenintheJavaprogramminglanguage.TheAndroidSoftwareDevelopmentKit(SDK)compileoutJavacodealongwithanydataandresourcefilesintoan.apk(Androidpackage)file,whichcontainscanbeinstalledinAndroid-powereddevices,suchassmartphones,tablets,smartTVs,orsmartwatches.

ForcompleteinformationaboutAndroiddevelopment,visithttps://developer.android.com/.

AndroidStudioistheofficialIDEforAndroiddevelopment.ItisbuiltbasedonIntelliJIDEA.InAndroidStudio,thebuildprocessofAndroidprojectsismanagedbytheGradlebuildsystem.DuringtheAndroidStudioinstallation,twoadditionaltoolscanbealsoinstalled:

AndroidSDK:ThiscontainsallofthepackagesandtoolsrequiredtodevelopAndroidapps.TheSDKManagerallowstodownloadandinstallSDKfordifferentversions(seetheprecedingtable).AndroidVirtualDevice(AVD):Thisisanemulatorthatallowsustomodelanactualdevice.TheAVDManagerallowstodownloadandinstalldifferentemulatedAndroidvirtualdevicesgroupedintofourcategories:phones,tables,TV,andwears.

GradlepluginforJUnit5inAndroidprojectsAtthetimeofthiswriting,thereisnoofficialsupportforJUnit5inAndroidprojects.Tosolvethisproblem,anopensourceGradlepluginnamedandroid-junit5hasbeencreated(https://github.com/aurae/android-junit5).Tousethisplugin,firstweneedtospecifytheproperdependencyinourbuild.gradlefile:

buildscript{

dependencies{

classpath"de.mannodermaus.gradle.plugins:android-junit5:1.0.0"

}

}

Inordertousethisplugininourproject,weneedtoextendourprojectcapabilitiesusingtheclauseapplyplugininourbuild.gradlefile:

applyplugin:"com.android.application"

applyplugin:"de.mannodermaus.android-junit5"

dependencies{

testCompilejunitJupiter()

}

Theandroid-junit5pluginconfiguresthejunitPlatformtask,attachingautomaticallyattachesboththeJupiterandVintageenginesduringthetestexecutionphase.Asanexample,considerthefollowingprojectexample,asusualhostedonGitHub(https://github.com/bonigarcia/mastering-junit5/tree/master/junit5-android).ThefollowingisascreenshotofthisprojectimportedinAndroidStudio:

AndroidprojectcompatiblewithJUnit5onIntelliJ

Now,wearegoingtocreateanAndroidJUnitrunconfigurationofAndroidStudio.Ascanbeseeninthescreenshot,weusetheoptionAllinpackagereferredtothepackagecontainingthetests(io.github.bonigarcia.myapplicationinthisexample):

AndroidJUnitrunconfiguration

Ifwelaunchtheaforementionedrunconfiguration,allthetestsoftheprojectwillbeexecuted.ThesetestscanusetheJUnit4programmingmodel(Vintage)andeventheJUnit5(Jupiter)inaseamlessway:

ExecutiononJupiterandVintagetestswithinanAndroidprojectinIntelliJ

RESTRoyFieldingisanAmericancomputerscientistbornin1965.HeisoneoftheauthorsoftheHTTPprotocolandtheco-authorsoftheApacheWebserver.Intheyear2000,FieldingcoinedthetermREST(shortforREpresentationalStateTransfer)inhisdoctoraldissertationentitledArchitecturalStylesandtheDesignofNetwork-basedSoftwareArchitecture.RESTisanarchitecturalstylefordesigningdistributedsystems.It’snotastandard,butratherasetofconstraints.RESTiscommonlyusedinconjunctionwithHTTP.Ontheonehand,theimplementationswhichfollowsthestrictprinciplesofRESTareoftenreferredasRESTful.Ontheotherhand,thosewhichfollowalooseadherenceofsuchprinciplesarecalledRESTlike.

RESTinanutshellRESTfollowsaclient-serverarchitecture.Theserverisinchargeofhandlingasetofservices,listeningforrequestsmadebyclients.Thecommunicationbetweenclientandservermustbestateless,meaningthatserverdonotstoreanyrecordfromtheclientsandthereforeeachrequestdonefromtheclientmustcontainalltheinformationrequiredfortheservertoprocessitseparately.

ThebuildingblocksofRESTarchitecturesarenamedresources.Resourcesdefinethetypeofinformationthatisgoingtobetransferred.Resourcesshouldbeidentifiedinauniqueway.InHTTP,thewaytoaccesstheresourceittoprovideitsfullURL,alsoknownasAPIendpoint.Eachresourcehasarepresentation,whichisamachine-readableexplanationofthecurrentstateofaresource.Nowadays,representationsareusuallywithJSON,butitcanbedoneinotherformatssuchasXMLorYAML.

Onceweidentifiedtheresourcesandtherepresentationformat,weneedtospecifywhatcanbedonewiththem,thatis,theactions.Actionscouldpotentiallybeanything,althoughthereisasetofcommonactionsthatanyresource-orientedsystemshouldprovide:CRUD(create,retrieve,update,anddelete)actions.RESTactionscanbemappedtotheHTTPmethods(so-calledverbs),asfollows:

GET:Readsaresource.POST:Sendsanewresourcetotheserver.PUT:Updatesagivenresource.DELETE:Deletesaresource.PATCH:updatepartiallyaresource.HEAD:Asksifagivenresourceexistswithoutreturninganyofitsrepresentations.OPTIONS:Retrievesalistofavailableverbsonagivenresource.

InREST,itisimportantthenotionofidempotency.Forexample,GET,DELETE,orPUTaresaidtobeidempotent,sincetheeffectoftheserequestsshouldbethesamewhetherthecommandissentoneorseveraltimes.Ontheotherhand,POSTisnotidempotent,sinceitcreatesadifferentresourceeachtimeitisrequested.

REST,whenbasedonHTTPcanbenefitonstandardHTTPstatuscodes.A

statuscodeisanumberthatsummarizestheresponseassociatedtoit.ThetypicalHTTPstatuscodereusedinRESTare:

200OK:Therequestwentfineandthecontentrequestedwasreturned.NormallyusedonGETrequests.201Created:Theresourcewascreated.UsefulonresponsestoPOSTorPUTrequests.204Nocontent:Theactionwassuccessful,butthereisnocontentreturned.Usefulforactionsthatdonotrequirearesponsebody,suchasaDELETE.301Movedpermanently:Thisresourcewasmovedtoanotherlocationandthelocationisreturned.400Badrequest:Therequestissuedhasproblems(forexample,lackingsomerequiredparameters).401Unauthorized:Usefulforauthenticationwhentherequestedresourceisnotaccessibletotheuserowningtherequest.403Forbidden:Theresourceisnotaccessible,butunlike401,authenticationwillnotaffecttheresponse.404Notfound:TheURLprovideddoesnotidentifyanyresource.405Methodnotallowed.TheHTTPverbusedonaresourceisnotallowed.(forexample,aPUTonaread-onlyresource).500Internalservererror:Agenericerrorcodewhenanunexpectedconditionintheserverside.

Thefollowingpictureshowsanexampleofclient-serverinteractionwithREST.ThebodyoftheHTTPmessagesusesJSONbothforrequestsandresponses:

RESTsequencediagramexample

UsingRESTtestlibrarieswithJupiterRESTAPIsarebecomingmoreandmorepervasivenowadays.Forthatreason,aproperstrategyforassessingRESTservicesisdesirable.Inthissection,wearegoingtolearnhowtouseseveraltestlibrariesinourJUnit5tests.

Firstofall,wecanusetheopensourcelibraryRESTAssured(http://rest-assured.io/).RESTAssuredallowsthevalidationofRESTservicesbymeansofafluentAPIinspiredindynamiclanguagessuchasRubyorGroovy.TouseRESTAssuredinourtestproject,wesimplyneedtoaddtheproperdependencyinMaven:

<dependency>

<groupId>io.rest-assured</groupId>

<artifactId>rest-assured</artifactId>

<version>${rest-assured.version}</version>

<scope>test</scope>

</dependency>

orinGradle:dependencies{

testCompile("io.rest-assured:rest-assured:${restAssuredVersion}")

}

Afterthat,wecanusetheRESTAssuredAPI.Thefollowingclasscontainstwotestexamples.FirstsendsarequesttothefreeonlineRESTservicehttp://echo.jsontest.com/.Thenverifiesiftheresponsecodeandthebodycontentareasexpected.ThesecondtestconsumesanotherfreeonlineRESTservice(http://services.groupkt.com/)andalsoverifiestheresponse:

packageio.github.bonigarcia;

importstaticio.restassured.RestAssured.given;

importstaticorg.hamcrest.Matchers.equalTo;

importorg.junit.jupiter.api.Test;

publicclassPublicRestServicesTest{

@Test

voidtestEchoService(){

Stringkey="foo";

Stringvalue="bar";

given().when().get("http://echo.jsontest.com/"+key+"/"+value)

.then().assertThat().statusCode(200).body(key,

equalTo(value));

}

@Test

voidtestCountryService(){

given().when()

.get("http://services.groupkt.com/country/get/iso2code/ES")

.then().assertThat().statusCode(200)

.body("RestResponse.result.name",equalTo("Spain"));

}

}

RunningthistestinconsolewithMaven,wecancheckthatbothtestssucceed:

ExecutionoftestusingRESTAssured

Inthesecondexample,wearegoingtostudy,inadditiontothetest,wearealsogoingtoimplementtheserverside,thatis,theRESTserviceimplementation.Tothataim,wearegoingtouseSpringMVCandSpringBoot,previouslyintroducedonthischapter(seesectionSpring).

TheimplementationofRESTservicesinSpringisquitestraightforward.First,wesimplyneedtoannotateaJavaclasswith@RestController.Inthebodyofthisclass,weneedtoaddmethodsannotatedwith@RequestMapping.ThesemethodswilllistentothedifferentURLs(endpoints)implementedinourRESTAPI.Theacceptedelementsforthe@RequestMappingare:

value:ThisisthepathmappingURL.method:ThisfindstheHTTPrequestmethodstomapto.params:Thisfindsparametersofthemappedrequest,narrowingtheprimarymapping.headers:hisfindstheheadersofthemappedrequest.consumes:Thisfindsconsumablemediatypesofthemappedrequest.produces:Thisfindsproduciblemediatypesofthemappedrequest.

Ascanbeseeninspectingthecodeofthefollowingclass,ourserviceexampleimplementsthreedifferentoperations:GET/books(toreadallbookinthesystem),GET/book/{index}(toreadabookgivenitsidentifier),andPOST/book(tocreateabook).

packageio.github.bonigarcia;

importjava.util.List;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.http.HttpStatus;

importorg.springframework.http.ResponseEntity;

importorg.springframework.web.bind.annotation.PathVariable;

importorg.springframework.web.bind.annotation.RequestBody;

importorg.springframework.web.bind.annotation.RequestMapping;

importorg.springframework.web.bind.annotation.RequestMethod;

importorg.springframework.web.bind.annotation.RestController;

@RestController

publicclassMyRestController{

@Autowired

privateLibraryServicelibraryService;

@RequestMapping(value="/books",method=RequestMethod.GET)

publicList<Book>getBooks(){

returnlibraryService.getBooks();

}

@RequestMapping(value="/book/{index}",method=RequestMethod.GET)

publicBookgetTeam(@PathVariable("index")intindex){

returnlibraryService.getBook(index);

}

@RequestMapping(value="/book",method=RequestMethod.POST)

publicResponseEntity<Boolean>addBook(@RequestBodyBookbook){

libraryService.addBook(book);

returnnewResponseEntity<Boolean>(true,HttpStatus.CREATED);

}

}

SinceweareimplementingaJupitertestforSpring,weneedtousetheSpringExtensionandalsotheSpringBootTestannotation.Asanovelty,wearegoingtoinjectatestcomponentprovidedbyspring-test,namedTestRestTemplate.

ThiscomponentisawrapperofthestandardSpring’sRestTemplateobject,whichallowstoimplementRESTclientsinaseamlessway.Inourtest,itrequeststoourservice(whichisstartedbeforeexecutingthetests),andresponsesareusedtoverifytheoutcome.

NoticethattheobjectMockMvc(explainedinthesectionSpring)couldbealsousedtotestRESTservices.ThedifferencewithrespecttoTestRestTemplateisthattheformerisusedtotestfromtheclient-side(thatis,responsecode,body,contenttype,andsoon),whilethethelatterisusedtotesttheservicefromtheserverside.Forinstance,intheexamplehere,theresponsestotheservicecalls(getForEntityandpostForEntity)areJavaobjects,whosescopeisonlytheserverside(intheclientside,thisinformationisserializedasJSON).

packageio.github.bonigarcia;

importstaticorg.junit.Assert.assertEquals;

importstatic

org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT;

importstaticorg.springframework.http.HttpStatus.CREATED;

importstaticorg.springframework.http.HttpStatus.OK;

importjava.time.LocalDate;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.boot.test.context.SpringBootTest;

importorg.springframework.boot.test.web.client.TestRestTemplate;

importorg.springframework.http.ResponseEntity;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

@ExtendWith(SpringExtension.class)

@SpringBootTest(webEnvironment=RANDOM_PORT)

classSpringBootRestTest{

@Autowired

TestRestTemplaterestTemplate;

@Test

voidtestGetAllBooks(){

ResponseEntity<Book[]>responseEntity=restTemplate

.getForEntity("/books",Book[].class);

assertEquals(OK,responseEntity.getStatusCode());

assertEquals(3,responseEntity.getBody().length);

}

@Test

voidtestGetBook(){

ResponseEntity<Book>responseEntity=restTemplate

.getForEntity("/book/0",Book.class);

assertEquals(OK,responseEntity.getStatusCode());

assertEquals("TheHobbit",responseEntity.getBody().getName());

}

@Test

voidtestPostBook(){

Bookbook=newBook("I,Robot","IsaacAsimov",

LocalDate.of(1950,12,2));

ResponseEntity<Boolean>responseEntity=restTemplate

.postForEntity("/book",book,Boolean.class);

assertEquals(CREATED,responseEntity.getStatusCode());

assertEquals(true,responseEntity.getBody());

ResponseEntity<Book[]>responseEntity2=restTemplate

.getForEntity("/books",Book[].class);

assertEquals(responseEntity2.getBody().length,4);

}

}

Asshowninthescreenshotbelow,ourSpringapplicationisstartedbeforerunningourtests,whichareexecutedsuccessfully:

OutputofJupitertestusingTestRestTemplatetoverifyaRESTservice.

Toconcludethissection,weseeanexampleinwhichthelibraryWireMock(http://wiremock.org/)isused.ThislibraryallowstomockRESTservices,thatis,aso-calledHTTPmockserver.Thismockservercapturesincomingrequeststotheservice,providingstubbedresponses.ThiscapabilityisveryusefultotestasystemwhichconsumesaRESTservice,buttheserviceisnotavailableduringthetests(orwecantestthecomponentthatcallstheserviceinisolation).

Asusual,weseeanexampletodemonstrateitsusage.Let’ssupposewehaveasystemwhichconsumesaremoteRESTservice.ToimplementaclientforthatserviceweuseRetrofit2(http://square.github.io/retrofit/),whichisahighlyconfigurableHTTPclientforJava.Wedefinetheinterfacetoconsumethisserviceasillustratedintheclassbelow.Noticethattheserviceexposesthreeendpointsaimedtoreadaremotefile(openfile,readstream,andclosestream):

packageio.github.bonigarcia;

importokhttp3.ResponseBody;

importretrofit2.Call;

importretrofit2.http.POST;

importretrofit2.http.Path;

publicinterfaceRemoteFileApi{

@POST("/api/v1/paths/{file}/open-file")

Call<ResponseBody>openFile(@Path("file")Stringfile);

@POST("/api/v1/streams/{streamId}/read")

Call<ResponseBody>readStream(@Path("streamId")StringstreamId);

@POST("/api/v1/streams/{streamId}/close")

Call<ResponseBody>closeStream(@Path("streamId")StringstreamId);

}

ThenweimplementtheclasswhichconsumestheRESTservice.Inthisexample,itisasimpleJavaclasswhichconnectstotheremoteservicegivenitsURLpassedasconstructorparameter:

packageio.github.bonigarcia;

importjava.io.IOException;

importokhttp3.ResponseBody;

importretrofit2.Call;

importretrofit2.Response;

importretrofit2.Retrofit;

importretrofit2.adapter.rxjava.RxJavaCallAdapterFactory;

importretrofit2.converter.gson.GsonConverterFactory;

publicclassRemoteFileService{

privateRemoteFileApiremoteFileApi;

publicRemoteFileService(StringbaseUrl){

Retrofitretrofit=newRetrofit.Builder()

.addCallAdapterFactory(RxJavaCallAdapterFactory.create())

.addConverterFactory(GsonConverterFactory.create())

.baseUrl(baseUrl).build();

remoteFileApi=retrofit.create(RemoteFileApi.class);

}

publicbyte[]getFile(Stringfile)throwsIOException{

Call<ResponseBody>openFile=remoteFileApi.openFile(file);

Response<ResponseBody>execute=openFile.execute();

StringstreamId=execute.body().string();

System.out.println("Stream"+streamId+"open");

Call<ResponseBody>readStream=remoteFileApi.readStream(streamId);

byte[]content=readStream.execute().body().bytes();

System.out.println("Received"+content.length+"bytes");

remoteFileApi.closeStream(streamId).execute();

System.out.println("Stream"+streamId+"closed");

returncontent;

}

}

Finally,weimplementaJUnit5testtoverifyourservice.Notcethatwearecreatingthemockserver(newWireMockServer)andstubbingtheRESTservicecallsusingthestaticmethodsstubFor(...)providedbyWireMockinthesetupofthetest(@BeforeEach).Sinceinthiscase,theSUTisverysimpleandithasnoDOCs,wedirectlyinstantiatetheclassRemoteFileServicealsointhesetupofeachtest,usingthemockserverURLasconstructorargument.Finally,wetestourservice(whichusesthemockserver)simplyexercisingtheobjectcalledwireMockServer,inthisexample,bycallingtothemethodgetFileandassessingitsoutput.

packageio.github.bonigarcia;

importstaticcom.github.tomakehurst.wiremock.client.WireMock.aResponse;

importstaticcom.github.tomakehurst.wiremock.client.WireMock.configureFor;

importstaticcom.github.tomakehurst.wiremock.client.WireMock.post;

importstaticcom.github.tomakehurst.wiremock.client.WireMock.stubFor;

importstaticcom.github.tomakehurst.wiremock.client.WireMock.urlEqualTo;

importstaticcom.github.tomakehurst.wiremock.core.WireMockConfiguration.options;

importstaticorg.junit.jupiter.api.Assertions.assertEquals;

importjava.io.IOException;

importjava.net.ServerSocket;

importorg.junit.jupiter.api.AfterEach;

importorg.junit.jupiter.api.BeforeEach;

importorg.junit.jupiter.api.Test;

importcom.github.tomakehurst.wiremock.WireMockServer;

publicclassRemoteFileTest{

RemoteFileServiceremoteFileService;

WireMockServerwireMockServer;

//Testdata

Stringfilename="foo";

StringstreamId="1";

StringcontentFile="dummy";

@BeforeEach

voidsetup()throwsException{

//LookforfreeportforSUTinstantiation

intport;

try(ServerSocketsocket=newServerSocket(0)){

port=socket.getLocalPort();

}

remoteFileService=newRemoteFileService("http://localhost:"+

port);

//Mockserver

wireMockServer=newWireMockServer(options().port(port));

wireMockServer.start();

configureFor("localhost",wireMockServer.port());

//Stubbingservice

stubFor(post(urlEqualTo("/api/v1/paths/"+filename+"/open-

file"))

.willReturn(aResponse().withStatus(200).withBody(streamId)));

stubFor(post(urlEqualTo("/api/v1/streams/"+streamId+

"/read"))

.willReturn(aResponse().withStatus(200).withBody(contentFile)));

stubFor(post(urlEqualTo("/api/v1/streams/"+streamId+/close"))

.willReturn(aResponse().withStatus(200)));

}

@Test

voidtestGetFile()throwsIOException{

byte[]fileContent=remoteFileService.getFile(filename);

assertEquals(contentFile.length(),fileContent.length);

}

@AfterEach

voidteardown(){

wireMockServer.stop();

}

}

Executingthetestintheconsole,inthetraceswecanseehowtheinternalHTTPservercontrolledbyWireMockisstartedbeforethetestexecution.Then,thethreeRESToperations(openstream,readbytes,closestream)areexecutedbythetest,andfinallythemockserverisdisposed:

ExecutionoftestusingamockRESTserverusingWireMock

SummaryThissectionprovidesadetailedinsightofhowJUnit5canbeusedinconjunctionwiththird-partyframeworks,libraries,andplatforms.ThankstotheJupiterextensionmodel,developerscancreateextensionswhichallowsseamlessintegrationwithexternalframeworkstoJUnit5.First,wehaveseentheMockitoExtension,anextensionprovidedbytheJUnit5teamtouseMockito(anotoriousmockframeworkforJava)inJupitertests.Then,wehaveusedtheSpringExtension,whichistheofficialextensionprovidedintheversion5oftheSpringFramework.ThisextensionintegratesSpringintotheJUnit5programmingmodel.Thisway,weareabletouseSpring’sapplicationcontexts(thatis,theSpring’sDIcontainer)inourtests.

WehavealsoreviewedtheSeleniumExtensionimplementedbyselenium-jupiter,anopensourceprojectprovidingaJUnit5extensionforSeleniumWebDriver(testingframeworkforwebapplications).Thanktothinsextension,wecanusedifferentbrowserstointeractautomaticallywithwebapplicationsandemulatedmobiledevices(usingAppium).Then,wehaveseentheCucumberExtension,allowstospecifyJUnit5acceptancetestsfollowingaBDDstyleusingtheGherkinlanguage.Finally,wehaveseenhowtheopensourceJUnit5-DockerextensioncanbeusedtostartDockercontainers(downloadingtheimagefromDockerHub)beforetheexecutionofourJUnit5tests.

Moreover,wediscoveredthattheextensionmodelisnottheonlywayofinteractingwithexternaltechnologiesbyJUnittests.Forexample,inordertorunJupitertestsinanAndroidproject,wecanusetheandroid-junit5plugin.Ontheotherhand,eventhoughthereisnocustomextensionforassessingRESTservicesusingJUnit5,theintegrationwithsuchlibrariesisstraitforward:wesimplyneedtoincludetheproperdependencyinourprojectanduseitinourtests(forexample,RESTAssured,Spring,orWireMock).

FromRequirementsToTestCasesProgramtestingcanbeusedtoshowthepresenceofbugs,butnevertoshow

theirabsence!-EdsgerDijkstra

Thischapterprovidesabaseofknowledgeaimedtohelpsoftwareengineerstowritemeaningfultestcases.Thestartingpointforthisprocessistheunderstandingoftherequirementsofthesystembeingtested.Withoutthatinformation,itisnotfeasibletodesignnorimplementvaluabletests.Afterthat,severalactionsmightbeexecutedbeforetheactualcodingofthetests,namely,testplanningandtestdesign.Oncewestartthetestcodingprocess,weneedtohaveinmindasetofprinciplestowritecoderight,andalsoasetofanti-patternsandbadsmellstobeavoided.Allthisinformationisprovidedinthischapterinformofthefollowingsections:

Theimportanceofrequirements:Thissectionprovidesageneraloverviewofthesoftwaredevelopmentprocess,startedbythestatementofsomeneedstobecoveredbyasoftwaresystem,andfollowedbyseveralstages,typicallyincludinganalysis,design,implementation,andtests.Testplanning:Adocumentcalledtestplancanbegeneratedatthebeginningofasoftwareproject.ThissectionreviewsthestructureofatestplanaccordingtotheIEEE829StandardforTestDocumentation.Aswewilldiscover,thecompletestatementofatestplanisaveryfine-grainedprocess,especiallyrecommendedforlargeprojectsinwhichthecommunicationamongtheteamisakeyaspectforthesuccessoftheproject.Testdesign:Beforestartingthecodingofthetests,itisalwaysagoodpracticetothinkabouttheblueprintofthesetests.Inthissection,wereviewthemajoraspectstobetakenintoconsiderationtodesignproperlyourtests.Weputtheaccentonthetestdata(expectedoutcome),whichfeedthetestassertions.Inthisregard,wereviewsomeblack-boxdatagenerationtechniques(equivalencepartitioningandboundaryanalysis)andwhite-box(testcoverage).Softwaretestingprinciples:Thissectionprovidesasetofbest-practiceswhichcanhelpuswriteourtests.Testanti-patterns:Finally,theoppositesideisalsoreviewed:whatare

thepatternsandcodesmellstobeavoidedwhenwritingourtestcases.

TheimportanceofrequirementsSoftwaresystemsarebuilttosatisfysomekindofneedtoagroupofconsumers(finalusersorcustomer).Understandingthoseneedsisoneofmostchallengingproblemsinsoftwareengineeringduetothefactthatitisquitecommonthatconsumerneedsarenebulous(especiallyintheearlystagesoftheproject).Moreover,itisalsocommonthattheseneedscanbedeeplychangedthroughouttheprojectlifetime.FredBrooks,awell-knownsoftwareengineer,andcomputerscientist,definesthisprobleminhisseminalbookTheMythicalMan-Month(1975):

Thehardestsinglepartofbuildingasoftwaresystemisdecidingpreciselywhattobuild.Nootherpartoftheconceptualworkisasdifficultasestablishingthedetailedtechnicalrequirements…Nootherpartoftheworksocripplestheresultingsystemifdonewrong.Nootherpartisasdifficulttorectifylater.

Inanycase,consumer’sneedsarethetouchstoneforanysoftwareproject.Fromtheseneeds,alistoffeaturescanemerge.Wedefineafeatureasahigh-leveldescriptionofasoftwaresystemfunctionality.Fromeachfeature,oneormorerequirements(functionalandnon-functional)shouldbederived.Arequirementiseverythingthatbetrueaboutthesoftware,inordertomeettheconsumer’sexpectations.Scenarios(real-lifeexamplesratherthanabstractdescriptions)canbeusefulforaddingdetailstotherequirementsdescription.Thegroupofrequirementsand/orlistoffeaturesofthesoftwaresystemareoftenknownasspecification.

Insoftwareengineering,thestageofdefiningtherequirementsiscalledrequirementselicitation.Inthisstage,softwareengineersneedtoclarifywhatproblemtheyaretryingtosolve.Attheendofthisphase,itisacommonpracticetostartthemodelingofthesystem.Tothataim,amodelinglanguage(typicallyUML)isemployedtocreateagroupofdiagrams.TheUMLdiagrams,whichtypicallyfitsintheelicitationstageistheusecasediagram(modelofthefunctionalityofthesystemanditsrelationshipwiththeinvolvedactors).

Modelingisnotalwayscarriedoutinallsoftwareprojects.Forexample,agilemethodologiesaremorebasedontheprincipleofsketchingratherthaninaformalmodelingstrategy.

Afterelicitation,requirementsshouldberefinedintheanalysisstage.Inthis

phase,thestatedrequirementsareanalysedinordertoresolveincomplete,ambiguousofcontradictoryissues.Asaresult,inthisstageitislikelytocontinuemodeling,usingforexamplehigh-levelclassdiagramsnotlinkedtoanyspecifictechnologyyet.Oncetheanalysisisclear(thatis,thewhatofthesystem),weneedtofindouthowtoimplementit.Thisstageisknownasdesign.Inthedesignphase,theguidelinesoftheprojectshouldbeestablished.Tothataim,anarchitectureofthesoftwaresystemistypicallyderivedfromtherequirements.Again,themodelingtechniquesarebroadlyemployedtocarryoutdifferentaspectsofthedesign.ThereisabunchofUMLdiagramsthatcanbeusedatthispoint,includingstructuraldiagrams(component,deployment,object,package,andprofilediagram)andbehavioraldiagrams(activity,communication,sequence,orstatediagram).Fromthedesign,theactualimplementation(thatis,coding)canstart.

Theamountofmodelingcarriedoutinthedesignstagevariessignificantlydependingondifferentfactors,includingthetypeandsizeofthecompanyproducingthesoftware(multinationals,SMEs,governmental,andsoon),thedevelopmentprocess(waterfall,spiral,prototyping,agile,andsoon),thetypeofproject(enterprise,opensource,andsoon),thetypeofsoftware(custommadesoftware,commercialoff-the-shelf,andsoon),andeventhebackgroundofthepeopleinvolved(experience,career,andsoon).Allinall,thedesignsneedtobeunderstoodasawayofcommunicationbetweenthedifferentrolesofsoftwareengineersparticipatingintheproject.Typically,thebiggertheproject,themorenecessaryafine-graineddesignbasedondifferentmodelingdiagramsis.

Concerningtests,inordertomakeapropertestplan(seenextsectionforfurtherdetails),againweneedtousetherequirementselicitationdata,thatis,thelistofrequirementsand/orfeatures.Inotherwords,inordertoverifyoursystem,weneedtotoknowbeforehandwhatweexpectfromit.UsingtheclassicdefinitionproposedbyBarryBoehm(seechapter1,RetrospectiveOnSoftwareQualityAndJavaTesting),verificationisusedtoanswerthequestionArewebuildingtheproductright?Tothat,weneedtoknowtherequirements,oratleast,thedesiredfeatures.Inadditiontoverification,itwouldbedesirabletocarryoutsomevalidation(accordingtoBoehm:Arewebuildingtherightproduct?).Thisisnecessarysincesometimesthereisagapbetweenwhathasbeenspecified(thefeaturesandrequirements)andtherealneedsoftheconsumer.Therefore,validationisahigh-levelassessmentmethod,andtocarryoutit,thefinalconsumercanbeinvolved(validatingthesoftwaresystemonceitisdeployed).Alltheseideasaredepictedinthefollowingpicture:

Softwareengineeringgenericdevelopmentprocess

Thereisnouniversalworkflowforthetermspresentedsofar(communication,requirementelicitation,analysis,design,implementation/test,anddeployment).Intheprecedingdiagram,itfollowsalinearprocessflows,nevertheless,inpractice,itcanfollowaniterative,evolutionary,orparallelworkflow.

Toillustratethepotentialproblemsinvolvedinthedifferentphasesinsoftwareengineer(analysis,design,implementation,andsoon),itisworthtoreviewtheclassicalcartoonHowprojectreallyworks?Theoriginalsourceofthispictureisunknown(thereareversionsdatingbacktothe1960s).In2007,asitecalledProjectCartoonemerged(http://www.projectcartoon.com/),allowingtocustomizetheoriginalcartoonwithnewscenes.Thefollowingchartistheversion1.5ofthecartoonprovidedonthatsite:

Howprojectsreallywork,version1.5(illustratedcreatedbywww.projectcartoon.com)

Ifwethinkaboutthispicture,wediscoverthattherootoftheproblemscomesfromtherequirements,badlyexplainedbythecustomeratthebeginning,andworstunderstoodbytheprojectleader.Fromthatpoint,thewholesoftwareengineeringprocessturnsintotheChinesewhisperschildrengame.Tosolvealltheseproblemsisoutofthescopeofthisbook,butasagoodstart,weneedtotakespecialcareintherequirements,whichguidethewholeprocess,including,ofcourse,thetests.

TestplanningAfirststepinthetestingpathcanbethegenerationofadocumentcalledtestplan,whichistheblueprinttoconductsoftwaretesting.Thisdocumentdescribestheobjective,scope,approach,focus,anddistributionofthetestingefforts.Theprocessofpreparingsuchdocumentisausefulwaytothinkabouttheneedstoverifyofasoftwaresystem.Again,thisdocumentisespeciallyusefulwhenthesizeoftheSUTandtheinvolvedteamislarge,duetothefactthattheseparationofworkindifferentrolesmakesthecommunicationapotentialdeterrentforthesuccessoftheproject.

AwaytocreateatestplanistofollowtheIEEE829StandardforTestDocumentation.Althoughthisstandardmightbetoomuchformalforthemostofsoftwareprojects,itmightbeworthtoreviewtheguidelinesproposedinthisstandard,andusethepartsneeded(ifany)inoursoftwareprojects.ThestepsproposedinIEEE829arethefollowing:

1. Analyzetheproduct:Thispartreinforcestheideaofextractingtheunderstandingtherequirementsofthesystemfromtheconsumerneeds.Asalreadyexplained,itisnotpossibletotestasoftwareifnoinformationaboutitisavailable.

2. Designtheteststrategy:Thispartoftheplancontainsseveralparts,including:

Definescopeoftesting,thatis,thesystemcomponentstobetested(inscope)andthosepartswhichdonot(outofscope).Asexplainedlater,exhaustivetestingisnotfeasible,andweneedtochoosecarefullywhatisgoingtobetested.Thisisnotasimplechoice,anditcanbedeterminedbydifferentfactors,suchasprecisecustomerrequests,projectbudgetandtiming,andskillsoftheinvolvedsoftwareengineers.Identifytestingtype,thatis,whichlevelsoftestsshouldbeconducted(unit,integration,system,acceptance)andwhichteststrategy(blackbox,whitebox,non-functional).Documentrisks,thatis,potentialproblemswhichmightcausedifferentissuesintheproject.

3. Definethetestobjectives:Inthispartoftheplan,thelistoffeaturestobetestedarelistedtogetherwiththetargetoftestingeachone.

4. Definethetestcriteria:Thesecriteriaaretypicallymadeupbytwo

parts,namely:Suspensioncriteria,forinstancethepercentageoffailedtestsinwhichthedevelopmentofnewfeaturesissuspendeduntiltheteamsolvesallthefailures.Exitcriteria,forexample,thepercentageofcriticalteststhatshouldbepassedtoproceedtonextphaseofdevelopment.

5. Resourceplanning:Thispartoftheplanisdevotedtosummarizetheresourcesrequiredtocarryoutthetestingactivities.Itcouldbepersonnel,equipment,orinfrastructure.

6. Plantestenvironment:Itconsistsofthesoftwareandhardwaresetuponwhichtestaregoingtobeexecuted.

7. Scheduleandestimation:Inthisphase,managersaresupposedtobreakoutthewholeprojectintosmalltasksestimatingtheefforts(person-month).

8. Determinetestdeliverables:Determineallthedocumentsthathastobemaintainedtosupportthetestingactivities.

Ascanbeseen,testplanningisacomplextask,typicallycarriedoutinlargeprojectsbymanagers.Intherestofthischapterwecontinuediscoveringhowtowritetestcases,buthereinafterfromapointofviewclosesttotheactualtestcoding.

TestdesignInordertodesignproperlyatest,weneedtodefinespecificallywhatneedstobeimplemented.Tothataim,itisimportanttorememberwhatisthegenericstructureofatest,alreadyexplainedinchapter1,RetrospectiveOnSoftwareQualityAndJavaTesting.Therefore,foreachtestweneedtodefine:

Whatistestfixture,thatis,therequiredstateintheSUTtocarryoutthetest?Thisisdoneatthebeginningofthetestinthestagecalledsetup.Attheendofthetest,thetestfixturemightbereleasedinthestagecalledteardown.WhatistheSUT,andifwearedoingunittests,whichareitsDOC(s)?Unittestshouldbeinisolationandthereforeweneedtodefinetestdoubles(typicallymocksorspies)fortheDOC(s).Whataretheassertions?Thisakeypartoftests.Withoutassertions,wecannotclaimthatatestisactuallymade.Inordertodesignassertion,itisworthtorecallwhichisitsgenericstructure.Inshort,anassertionconsistsinthecomparisonofsomeexpectedvalue(testdata)andtheactualoutcomeobtainedfromtheSUT.Ifanyoftheassertionsisnegative,thetestwillbedeclaredasfailed(testverdict):

Testcasesandassertionsgeneralschema

Testdataplaysacrucialroleinthetestingprocess.Thesourceoftestdatais

oftencalledtestoracles,andtypicallycanbeextractedfromtherequirements.Nevertheless,therearesomeotherscommonlyusedsourcesfortestsoracles,forexample:

Adifferentprogram,whichproducestheexpectedoutput(inverserelationship).Aheuristicorstatisticaloraclethatprovidesapproximateresults.Valuesbasedontheexperienceofhumanexperts.

Moreover,testdatacanbederived,dependingontheunderlyingtestingtechnique.Whenusingblack-boxtesting,thatis,exercisesomespecificrequirementbasedusingsomeinputandexpectingsomeoutput,differenttechniquescanbeemployed,suchasequivalencepartitioningorboundaryanalysis.Ontheotherside,ifweareusingwhite-boxtesting,thestructureisthebasisforourtestandthereforethetestcoveragewillbekeytoselectthetestinputwhichmaximizesthesecoveragerates.Inthefollowingsections,thesetechniquesarereviewed.

EquivalencepartitioningEquivalencepartitioning(alsoknownasequivalenceclasspartitioning)isablack-boxtechnique(thatis,itreliesintherequirementsofthesystem)aimedtoreducethenumberofteststhatshouldbeexecutedagainstaSUT.ThistechniquewasfirstdefinedbyGlenfordMyersin1978as:

“Atechniquethatpartitionstheinputdomainofaprogramintoafinitenumberofclasses[sets],itthenidentifiesaminimalsetofwell-selectedtestcasestorepresenttheseclasses.”

Inotherwords,equivalencepartitioningprovidesacriteriatoanswerthequestionHowmanytestsdoweneed?Theideaistodivideallpossibleinputtestdata(whichoftenisaenormousnumberofcombinations)inasetofvaluesforwhichweassumetobeprocessedinthesamewaybytheSUT.Wecallequivalenceclassestothesesetsofvalues.TheideaisthattestingonerepresentativevaluewithintheequivalenceclassisconsidersufficientbecauseitisassumedthatallthevaluesareprocessedinthesamewaybytheSUT.

Typically,theequivalenceclassesforagivenSUTcanbegroupedintwotypes:validandinvalidinputs.Theequivalencepartitioningtestingtheoryensuresthatonlyonetestcaseofeachpartitionisneededtoevaluatethebehavioroftheprogramfortherelatedpartition(boththevalidandtheinvalidclasses).ThefollowingprocessdescribeshowtosystematicallycarryouttheequivalencepartitioningforagivenSUT:

1. First,wedeterminethedomainofallpossiblevalidinputsforaSUT.Tofindoutthesevalues,werelyonthespecification(featuresorfunctionalrequirements).OurSUTissupposedtoprocessthesevalues(validequivalenceclass)correctly.

2. Ifourspecificationestablishesthatsomeelementsoftheequivalenceclassareprocesseddifferently,theyshouldassignetoanotherequivalenceclass.

3. Thevaluesoutsidethisdomaincanbeseenasanotherequivalenceclass,thistimeforinvalidinputs.

4. Foreverysingleequivalenceclass,arepresentativevalueischosen.Thisdecisionisanheuristicprocesstypicallybasedonthetesterexperience.

5. Foreverytestinput,thepropertestoutputisalsoselected,andwiththese

valueswewillbeabletocompleteourtestcase(testexerciseandassertions).

BoundaryanalysisAsanyprogrammerknows,faultsoftenappearattheboundaryofaequivalenceclass(forexample,theinitialvalueofanarray,themaximumvalueforagivenrange,andsoon).Boundaryvalueanalysisisamethod,whichcomplementsequivalencepartitioningbylookingattheboundariesofthetestinput.ItwasdefinedbytheNationalInstituteofStandardsandTechnology(NIST)in1981as:

“Aselectiontechniqueinwhichtestdataarechosentoliealong‘boundaries’oftheinputdomain[oroutputrange]classes,datastructures,andprocedureparameters.”

Allinall,toapplyboundaryvalueanalysisinourtests,weneedtoevaluateourSUTexactlyinthebordersofourequivalenceclass.Therefore,typicallytwotestscasesarederivedusingthisapproach:theupperandthelowerboundaryoftheequivalenceclass.

TestcoverageTestcoverageistherateofcodeinSUTthatisexercisedforanyoftheirtests.TestcoverageisveryusefultofindinguntestedpartsofourSUT.Therefore,itcanbetheperfectwhiteboxtechnique(structural)tocomplementtheblackbox(functional).Asageneralrule,atestcoveragerateof80%oraboveisconsideredreasonable.

TherearedifferentJavalibraries,whichallowstomaketestcoverageinasimplemanner,forinstance:

Cobertura(http://cobertura.github.io/cobertura/):Itisanopensourcereportingtool,whichcanbeexecutedusingAnt,Maven,ordirectlyusingthecommandline.EclEmma(http://www.eclemma.org/):ItisanopensourcecodecoveragetoolforEclipse.AsofEclipse4.7(Oxygen),EclEmmaisintegratedoutoftheboxintheIDE.ThefollowingscreenshotshowsanexampleonhowEclEmmahighlightsthecodecoverageonaJavaclassinEclipse:

TestcoveragewithEclEmmainEclipse4.7(Oxygen)

JaCoCo(http://www.jacoco.org/jacoco/):ItisanopensourcecodecoveragelibrarycreatedbytheEclEmmateambasedonotheroldcoveragelibrarycalledEMMA(http://emma.sourceforge.net/).JaCoCoisavailableasaMavendependency.Codecov(https://codecov.io/):Itisacloudsolutionofferingafriendlycodecoveragewebdashboard.Itisfreeforopensourceprojects.

SoftwaretestingprinciplesExhaustivetestingisthenamegiventoatestapproach,whichusesallpossiblecombinationsoftestinputstoverifyasoftwaresystem.Thisapproachisonlyapplicabletotinysoftwaresystemsorcomponentswithaclosefinitenumberofpossibleofoperationsandalloweddata.Inthemajorityofsoftwaresystems,itisnotfeasibletoverifyeverypossiblepermutationandinputcombination,andthereforeexhaustivetestingisjustatheoreticalapproach.

Forthatreason,itissaidthattheabsenceofdefectsinasoftwaresystemcannotbeproved.ThiswasstatedbythecomputersciencepioneerEdsgerW.Dijkstra(seequoteatbeginningofthischapter).Thus,testingis,atbest,sampling,anditmustbecarriedoutinanysoftwareprojecttoreducetheriskofsystemfailures(seechapter1,RetrospectiveOnSoftwareQualityAndJavaTesting,torecallthesoftwaredefecttaxonomy).Sincewecannottesteverything,weneedtotestproperly.Inthissection,wereviewasetofbestpracticestowriteeffectiveandefficienttestcases,namely:

Testsshouldbesimple:Thesoftwareengineerwritingthetest(callhimorhertester,programmer,developer,orwhatever)shouldavoidattemptingtotesthisorherprogram.Inregardstotesting,therightanswertothequestionWhowatchesthewatchmen?Shouldbenobody.Ourtestlogicshouldbesimpleenoughtoavoidanykindofmeta-testing,sincethiswouldleadtoarecursiveproblemoutofanylogic.Indirectly,ifwekeeptestssimple,wealsoobtainanotherdesirablefeature:testswillbeeasytomaintain.Donotimplementsimpletests:Onethingismakesimpletests,andanotherverydifferentstuffistoimplementdummycode,suchasgetterorsetters.Asintroducedbefore,testisatbestsampling,andwecannotwasteprecioustimeinassessingsuchkindofpartofourcodebase.Easytoread:Thefirststepistoprovideameaningfulnameforourtestmethod.Inaddition,thankstotheJUnit5@DisplayNameannotation,wecanprovidearichtextualdescription,whichdefineswithoutJavanamingconstraintsthegoalofthetest.Singleresponsibilityprinciple:Thisisageneralprincipleofcomputerprogrammingthatstatesthateveryclassshouldhaveresponsibilityofasinglefunctionality.Itiscloselyrelatedtothemetricofcohesion.This

principleisveryimportanttobeaccomplishedwhencodingtests:asingletestshouldbeonlyreferredtoagivensystemrequirement.Testdataiskey:Asdescribedinthesectionbefore,theexpectedoutcomefromtheSUTisacentralpartofthetests.Thecorrectmanagementofthesedataiscriticaltocreateeffectivetests.Fortunately,JUnit5providesarichtoolboxtohandletestdata(seesectionParameterizedtestsinchapter4,SimplifyingTestingWithAdvancedJUnitFeatures).Unittestshouldbeexecutedveryfast:Acommonlyacceptedruleofthumbforthedurationofunittestisthataunittestshouldlastasecondatthemost.Toaccomplishthatgoal,itisalsorequiredthatunittestisolatesproperlytheSUT,doublingproperlyitsDOCs.Testmustberepeatable:Defectsshouldbereproducedasmanytimesasrequiredfordeveloperstofindthecauseofthebug.Thisisthetheory,butunfortunatelythisisnotalwaysapplicable.Forexample,inmulti-threadedSUT(areal-timeorserver-sidesoftwaresystems),raceconditionsarelikelytooccur.Inthosesituations,non-deterministicdefects(oftencalledheisenbugs)mightbeexperienced.Weshouldtestpositiveandthenegativescenarios:Thismeanthatweneedtowritetestswithforinputconditionthatassesstheexpectedoutcome,butwealsoneedtoverifywhattheprogramisnotsupposedtodo.Inadditiontomeetitsrequirements,programsmustbetestedtoavoidunwantedsideeffects.Testingcannotbedoneonlyforthesakeofcoverage:Justbecauseallpartsofthecodehavebeentouchedbysometests,wecannotassurethatthosepartshavebeenthoroughlytested.Forthattobetrue,testshavetoanalyzedintermsofreductionofrisks.

ThepsychologyoftestingFromapsychologicalpointofview,theobjectiveoftestingshouldbeexecutingasoftwaresystemwiththeintentoffindingdefects.Understandingthemotivationofthatclaimcanmakethedifferenceinthesuccessofourtests.

Humanbeingstendtobegoaloriented.Ifwecarryoutteststodemonstratethataprogramhasnoerrors,wewilltendtoimplementtestsselectingtestdatawithalowprobabilityofcausingprogramfailures.Ontheotherhand,iftheobjectiveistodemonstratethataprogramhaserrors,wewillincreasetheprobabilityoffindingthem,addingmorevaluetotheprogramthantheformerapproach.Forthatreason,testingisoftenconsideredasadestructiveprocess,sincetestersaresupposedtoprovethattheSUThaserrors.

Moreover,tryingtodemonstratethaterrorsarepresentinthesoftwareisagoalfeasible,whiletryingtodemonstratetheirabsence,asexplainedbefore,itisimpossible.Again,psychologystudiestellusthatpeopleperformpoorlywhentheyknowthatataskisinfeasible.

Testanti-patternsInsoftwaredesign,apatternisareusablesolutiontosolverecurringproblems.Thereareabunchofthem,includingforexamplesingleton,factory,builder,facade,proxy,decorator,oradapter,tonameafew.Anti-patternsarealsopatterns,butundesirableones.Concerningtotesting,itisworthtoknowsomeoftheseanti-patternstoavoidtheminourtests:

Secondclasscitizens:Testcodecontainingalotofduplicatedcode,makingithardtomaintain.Thefreeride(alsoknownasPiggyback):Insteadofwritinganewmethodtoverifyanotherfeature/requirement,anewassertionisaddedtoanexistingtest.Happypath:Itonlyverifiesexpectedresultswithouttestingforboundariesandexceptions.Thelocalhero:Atestdependenttosomespecificlocalenvironment.Thisanti-patterncanbesummarizedinthephraseItworksinmymachine.Thehiddendependency:Atestthatrequiressomeexistingdatapopulatedsomewherebeforethetestruns.Chaingang:Teststhatmustberuninacertainorder,forexample,changingtheSUTtoastateexpectedbythenextone.Themockery:AunittestthatcontainstoomuchtestdoublesthattheSUTisnoteventestedatall,insteadofreturningdatafromtestdoubles.Thesilentcatcher:Atestthatpassesevenifanunintendedexceptionactuallyoccurs.Theinspector:AtestthatviolatesencapsulationthatanyrefactorintheSUTrequiresreflectingthosechangesinthetest.Excessivesetup:Atestthatrequiresahugesetupinordertostarttheexercisestage.Analprobe:Atestwhichhastouseunhealthywaystoperformitstask,suchasreadingprivatefieldsusingreflection.Thetestwithnoname:Testmethodsnamewithnoclearindicatoraboutwhatitisbeingtested(forexample,identifierinabugtrackingtool).Theslowpoke:Aunittestwhichlastsoverfewseconds.Theflickeringtest:Atestwhichcontainsraceconditionswithinthepropertest,makingittofailfromtimetotime.

Waitandsee:Atestthatneedstowaitaspecificamountoftime(forexample,Thread.sleep())beforeitcanverifysomeexpectedbehavior.Inappropriatelysharedfixture:Teststhatuseatestfixturewithoutevenneedthesetup/teardown.Thegiant:Atestclassthatcontainsahugenumberoftestsmethods(GodObject).Wetfloor:Atestthatcreatespersisteddatabutitisnotcleanupatwhenfinished.Thecuckoo:Aunittestwhichestablishessomekindoffixturebeforetheactualtest,butthenthetestdiscardssomehowthefixture.Thesecretcatcher:Atestthatisnotmakinganyassertion,relyingonanexceptiontobethrownandreportingbythetestingframeworkasafailure.Theenvironmentalvandal:Atestwhichrequirestheuseofgivenenvironmentvariables(forinstance,afreeportnumbertoallowssimultaneousexecutions).Doppelganger:Copyingpartsofthecodeundertestintoanewclasstomakevisibleforthetest.Themotherhen:Afixturewhichdoesmorethanthetestneeds.Thetestitall:TeststhatshouldnotbreaktheSingleResponsibilityPrinciple.Linehitter:AtestwithoutanykindofrealverificationoftheSUT.Theconjoinedtwins:TeststhatarecalledunittestsbutarereallyintegrationtestssincethereisnoisolationbetweentheSUTandtheDOC(s).Theliar:Atestthatdoesnottestwhatwassupposedtotest.

CodesmellsCodesmells(alsoknownasbadsmellwhenreferredtosoftware)areundesirablesymptomswithinthesourcecode.Codesmellsarenotproblematicperse,buttheycanevidencesomekindofissuenearby.

Asdescribedinprevioussections,testsshouldbesimpleandeasytoread.Withthatpromises,codesmellsshouldbepresentinourtestsundernocircumstances.Allinall,genericcodesmellsmightbeavoidedinourtests.Someofthemostcommoncodesmellsarethefollowing:

Duplicatedcode:Clonedcodeisalwaysabadideainsoftware,sinceitbreakstheprincipleDon’tRepeatYourself(DRY).Thisproblemisevenworstintests,sincetestlogicmustbecrystalclear.Highcomplexity:Toomanybranchesorloopsmaybepotentiallysimplifiedintosmallerpieces.Longmethod:Amethodthathasgrowntoolargeisalwaysproblematic,anditisaverybadsymptomwhenthismethodisatest.Unappropriatednamingconvention:Variables,class,andmethodnamesshouldbeconcise.Itisconsideredabadsmelltouseverylongidentifiers,butalsouseexcessiveshort(ormeaningless)ones.

SummaryThestartingpointforthetestdesignshouldbethelistofrequirements.Iftheserequirementshavenotbeenformallyelicited,atleastweneedtoknowtheSUTfeatures,whichreflectsthesoftwareneeds.Fromthispoint,severalstrategiescanbecarriedout.Asusual,thereisnouniquepathtoreachourgoal,whichintheendshouldbereducingtherisksoftheproject.

Thischapterreviewedaprocessaimedtocreateeffectiveandefficienttestscases.Thisprocessinvolvestheanalysisofrequirements,definitionofatestplan,designoftestcases,andfinallywritingthetestcases.Weshouldbeawarethat,eventhoughsoftwaretestingistechnicaltask,itinvolvessomeimportantconsiderationsofhumanpsychology.Thesefactorsshouldbeknownbysoftwareengineersandtestersinordertofollowknowbestpracticesandalsoavoidingcommonmistakes.

Inchapter7,Testingmanagement,wearegoingtounderstandhowsoftwaretestingactivitiesaremanagedinalivingsoftwareproject.Tothat,firstwereviewwhenandhowtocarryouttestinginthecommonsoftwaredevelopmentprocesses,suchaswaterfall,spiral,iterative,spiral,agile,ortest-drivendevelopment.Then,theserver-sideinfrastructure(suchasJenkinsorTravis)aimedtoautomatethesoftwaredevelopmentprocessinthecontextofJUnit5isreviewed.Finally,welearnhowtokeeptrackofthedefectsfoundwiththeJupitertestsusingtheso-calledissuetrackingsystemsandtestreportinglibraries.

TestingManagementTheimportantthingisnottostopquestioning.

-AlbertEinstein

Thisisthefinalchapterofthebook,anditsobjectiveistoguidehowtounderstandwhenandhowsoftwaretestingactivitiesaremanagedinalivingsoftwareproject.Tothataim,thischapterisstructuredintothefollowingsections:

Softwaredevelopmentprocesses:Inthissectionwestudywhentestsareexecutedindifferentmethodologies:Behavior-DrivenDevelopment(BDD),Test-DrivenDevelopment(TDD),Test-FirstDevelopment(TFD)andTest-LastDevelopment(TLD).ContinuousIntegration(CI):Inthissection,wewilldiscoverCI,thesoftwaredevelopmentpractice,inwhichtheprocessofbuild,test,andintegrationiscarriedoutcontinuously.Thecommontriggerofthisprocessisusuallythecommitofnewchanges(patches)toasourcecoderepository(forexample,GitHub).Inaddition,inthissection,wewilllearnhowtoextendCI,reviewingtheconceptofContinuousDeliveryandContinuousDeployment.Finally,wepresenttwoofthemostimportantbuildservernowadays:JenkinsandTravisCI.Testreporting:Inthissection,wewillfirstdiscovertheXMLformatinwhichthexUnitframeworkusuallyreportstheexecutionoftests.Theproblemwiththisformatisthatitisnothumanreadable.Forthisreason,therearetoolswhichcovertthisXMLintoafriendlierformat,typicallyHTML.Wereviewtwoalternatives:MavenSurefireReportandAllure.Defecttrackingsystems:Inthissection,wereviewseveralissuetrackes:JIRA,Bugzilla,Redmine,MantisBT,andGitHubissues.Staticanalysis:Inthissection,ontheonehandwereviewseveralautomatedanalysistools(linters)suchasCheckstyle,FindBugs,PMD,andSonarQube.Ontheotherside,wedescribeseveralpeerreviewtools,suchasCollaborator,Crucible,Gerrit,andGitHubpullrequestsreviews.Puttingall,piecestogether:Toconcludethebook,inthefinalsectionwepresentacompleteexampleapplicationinwhichdifferenttypesoftests(unit,integration,andend-to-end)areperformedusingsomeofthemainconceptspresentedalongthisbook.

SoftwaredevelopmentprocessesInsoftwareengineering,thesoftwaredevelopmentprocess(alsoknownasthesoftwaredevelopmentlifecycle)isthenamegiventotheworkflowfortheactivities,actions,andtasksrequiredtocreatesoftwaresystems.AsintroducedinChapter6,FromRequirementstoTestCases,theusualphasesinanysoftwaredevelopmentprocessare:

Definitionofwhat:Requirementselicitation,analysisandusecasemodeling.Definitionofhow:Thesystemarchitectureandmodelingofstructuralandbehavioraldiagrams.Theactualsoftwaredevelopment(coding).Thesetofactivitiesthatmakesthesoftwareavailableforuse(release,installation,activation,andsoon).

Thetiminginwhichtestsaredesignedandimplementedintheoverallsoftwaredevelopmentprocessresultsindifferenttestmethodologies,namely(seediagramafterthelist):

Behavior-DrivenDevelopment(BDD):Atthebeginningoftheanalysisphase,conversationsbetweenthesoftwareconsumer(finaluserorcostumer)andsomeofthedevelopmentteam(typically,projectleader,manager,oranalysts)tookplace.Theseconversationsareusedtoconcretizescenarios(thatis,concreteexamplestobuildupacommonunderstandingofthesystemfeatures).TheseexamplesformthebasistodevelopacceptancetestsusingtoolssuchasCucumber(formoredetailsaboutit,takealooktoChapter5,IntegrationofJUnit5withexternalframeworks.)ThedescriptionofacceptancetestsinBDD(forexample,usingGherkininCucumber)producesbothautomatedtestsanddocumentationthataccuratelydescribetheapplicationfeatures.TheBDDapproachisnaturallyalignedwithiterativeorAgilemethodologies,sinceitisverydifficulttodefinerequirementsupfront,andtheseevolveastheteamlearnsmoreabouttheproject.

ThetermagilewaspopularizedwiththeinceptionoftheAgilemanifestoin2001(http://agilemanifesto.org/).Itwaswrittenby17softwarepractitioners(KentBeck,JamesGrenning,RobertC.

Martin,MikeBeedle,JimHighsmith,SteveMellor,ArievanBennekum,AndrewHunt,KenSchwaber,AlistairCockburn,RonJeffries,JeffSutherland,WardCunningham,JonKern,DaveThomas,MartinFowler,andBrianMarick),andincludesalistof12principlestoguideaniterativeandpeople-centricsoftwaredevelopmentprocess.Basedontheseprinciples,severalsoftwaredevelopmentframeworksemerged,suchasSCRUM,Kanban,orextremeprogramming(XP).

Test-DrivenDevelopment(TDD):TDDisamethodologyinwhichtestsaredesignedandimplementedbeforetheactualsoftwaredesign.Theideaistoconverttherequirementsobtainedintheanalysisstagetospecifictestcases.Then,thesoftwareisdesignedandimplementedtopassthesetests.TDDispartoftheXPmethodology.Test-FirstDevelopment(TFD):Inthismethodology,testsareimplementedafterthedesignstage,butbeforetheactualimplementationoftheSUT.Thisallowstoassurethatthesoftwareunitshavebeenunderstoodcorrectlybeforeitsactualimplementation.ThismethodologyisfollowedintheUnifiedProcess,whichisapopulariterativeandincrementalsoftwaredevelopmentprocess.TheRationalUnifiedProcess(RUP)isawell-knownframeworkimplementationoftheUnifiedProcess.InadditiontoTFD,RUPalsosupportsothermethodologiessuchasTDDandTLD.

Test-LastDevelopment(TLD):Inthismethodology,theimplementationofthetestiscarriedoutaftertheimplementationoftheactualsoftware(SUT).Thistestmethodologyisfollowedbyclassicsoftwaredevelopmentprocesses,suchaswaterfall(sequential),incremental(multi-waterfall)orspiral(risk-orientedmulti-waterfall).

Testmethodologiesduringthesoftwaredevelopmentprocesses

Thereisnouniversalaccepteddefinitionsofthetermspresentedsofar.Theseconceptsaresubjecttocontinuousevolutionanddebate,justlikethesoftwareengineeringitself.Considerthistobeaproposal,whichfitsintoalargenumberofsoftware

projects.

Regardingwhoisresponsibleforcodingthetests,thereisauniversallyacceptedconsensus.ItisbroadlyrecommendedthatunittestsshouldbewrittenbySUTdevelopers.Insomecases,especiallyinsmallteams,thesedevelopersarealsoresponsibleforotherkindsoftests.

Inaddition,theroleofanindependenttestgroup(oftencalledtestersoraQAteam)isalsoacommonpractice,especiallyinlargeteams.Oneoftheobjectiveofthisroleseparationistoremovetheconflictofintereststhatmaybepresentotherwise.Wecannotforgetthattestingisunderstoodasadestructiveactivityfromaphysiologicalpointofview(theobjectiveisfindingdefects).Thisindependenttestgroupisusuallyinchargeontheintegration,system,andnon-functionaltests.Inthiscase,bothgroupsofengineersshouldworkclosely;whiletestsareconducted,developersshouldbeavailabletocorrectfaultsandminimizefutureerrors.

Finally,high-levelacceptancetestsareusuallyconductedinheterogeneousgroupsinvolvingnon-programmers(customers,businessanalysis,managers,andsoon)togetherwithsoftwareengineersortesters(forexample,forimplementthestepdefinitioninCucumber).

ContinuousIntegrationTheconceptofCIwasfirstcoinedon1991byGradyBooch(Americansoftwareengineer,bestknownforthedevelopmentofUMLtogetherwithIvarJacobsonandJamesRumbaugh).TheExtremeProgramming(XP)methodologyadoptedthisterm,makingitverypopular.AccordingtoMartinFowler,CIisdefinedasfollows:

ContinuousIntegrationisasoftwaredevelopmentpracticewheremembersofateamintegratetheirworkfrequently,usuallyeachpersonintegratesatleastdaily-leadingtomultipleintegrationsperday.Eachintegrationisverifiedbyanautomatedbuild(includingtest)todetectintegrationerrorsasquicklyaspossible.

InCIsystems,wecanidentifydifferentparts.First,weneedasourcecoderepository,whichisafilearchivetohostthesourcecodeofoursoftwareproject,typicallyusingaversioncontrolsystem.Nowadays,thepreferredversioncontrolsystemisGit(originallydevelopedbyLinusTorvalds)overoldersolutions,suchasCVSorSVN.Atthemomentofthiswriting,theleadingversioncontrolrepositoryisGitHub(https://github.com/),whichasitsnameindicatesitisbasedonGit.Besides,thereareotheralternatives,suchasGitLab(https://gitlab.com),BitBucket(https://bitbucket.org/),orSourceForge(https://sourceforge.net/).Thelatterwastheleadingforgeinthepast,butisnowadayslessused.

Acopyofthesourcecoderepositoryissynchronizedinthelocalenvironmentofdevelopers.Thecodingworkisdoneagainstthislocalcopy.Developersaresupposedtocommitnewchanges(knownaspatches)totheremoterepositoryinadailybasis.Frequentcommitsallowtoavoidconflicterrorsduetothemutualmodificationofthesamepartsofagivenfile.

ThebasicideaofCIisthateverycommitshouldexecutethebuildandtestthesoftwarewiththenewchanges.Forthatreason,weneedaserver-sideinfrastructurewhichautomatesthisprocess.Thisinfrastructureisknownasbuildserver(ordirectlyCIserver).TwoofthemostimportantbuildserversnowadaysareJenkinsandTravisCI.Detailsofbothofthemareprovidedinnextsubsections.Asaresultofthebuildprocess,thebuildservershouldnotifytheresultoftheprocesstotheorigindeveloper.Iftestsweresuccessful,thepatchismergedinthecodebase:

ContinuousIntegrationprocess

ClosetoCI,thetermDevOpshasgainedmomentum.DevOpscomesfromdevelopmentandoperations,anditisthenamegiventoasoftwaredevelopmentprocessthatemphasizesthecommunicationandcollaborationdifferentteamsinaprojectsoftware:development(softwareengineering),QA(qualityassurance),andoperations(infrastructure).ThetermDevOpsisalsoreferredtoajobposition,typicallyinchargeofthesetup,monitoringanoperationofthebuildservers:

DevOpsareinbetweendevelopment,operationsandQA

Asshowninthenextfigure,theconceptofCIcanbeextendedto:

ContinuousDelivery:WhentheCIpipelinefinishcorrectly,atleastareleaseofsoftwarewillbedeployedtoatestenvironment(forinstance,deployinganSNAPSHOTartifacttoaMavenarchiver).Inthisphase,acceptancetestscanalsobeexecuted.ContinuousDeployment:Asthefinalstepintheautomationtoolchain,thereleaseofthesoftwarecanbereleasedtoaproductionenvironment(forexample,deployingawebapplicationtotheproductionserverforeachcommit,whichachievestopassthecompletepipeline).

ContinuousIntegration,ContinuousDelivery,andContinuousDeploymentchain

JenkinsJenkins(https://jenkins.io/)isanopensourcebuildserverwhichsupportsbuilding,deploying,andautomatinganyproject.JenkinshasbeendevelopedinJava,anditcanbemanagedeasilyusingitswebinterface.TheglobalconfigurationofaJenkinsinstanceincludesinformationaboutJDK,Git,Maven,Gradle,Ant,andDocker.

JenkinswasoriginallydevelopedastheHudsonprojectbySunMicrosystemsin2004.AftertheacquisitionofSunbyOracle,theHudsonprojectwasforkedtoanopensourceproject,renamedtoJenkins.Bothnames(HudsonandJenkins)weremeanttosoundlikestereotypicalEnglishbutlernames.Theideaistheyhelpdeveloperscarryouttedioustasks,justlikeahelpfulbutler.

InJenkins,buildsaretypicallytriggeredbynewcommitsinversioncontrolsystems.Inaddition,buildscanbestartedbyothermechanisms,suchasscheduledcrontaskorevenmanuallyusingtheJenkinsinterface.

Jenkinsishighlyextensiblethankstoitspluginarchitecture.Thankstothose,Jenkinshasbeenextendedtoarichpluginecosystemmadebyvastnumberofthird-partyframeworks,libraries,systems,andsoon.Thisismaintainedbytheopensourcecommunity.TheJenkinspluginportfolioisavailableonhttps://plugins.jenkins.io/.

AttheheartofJenkins,wefindtheconceptofjob.AjobisarunnableentitymonitoredbyJenkins.Asshowninthescreenshothere,aJenkinsjobiscomposedoffourgroups:

Sourcecodemanagement:ThisistheURLofthesourcecoderepository(Git,SVN,andsoon)Buildtrigger:Thisisthemechanismstartingthebuildprocess,suchasnewchangesinthesourcecoderepository,externalscripts,periodically,andsoon.Buildenvironment:Optionalsetup,forexample,deleteworkspacebeforebuildstart,abortthebuildwhenstuck,andsoon.Collectionofstepsofthejobs:ThesestepscanbedonewithMaven,Gradle,Ant,orshellcommands.Afterthose,post-buildactionscanbeconfigured,forexample,toarchiveanartifact,topublishJUnittest

report(wewilldescribethisfeaturelaterinthischapter),emailnotifications,andsoon.

Jenkinsjobconfiguration

AnotherinterestingwayofconfiguringajobisusingaJenkinspipeline,whichisthedescriptionofthebuildworkflowusingthePipelineDSL(adomain-specificlanguagebasedonGroovy).AJenkinspipelinedescriptionistypicallystoredinafilecalledJenkinsfile,whichcanbeunderthecontrolofthesourcecoderepository.Inshort,aJenkinspipelineisdeclarativechainofstagescomposedofsteps.Forexample:

pipeline{

agentany

stages{

stage('Build'){

steps{

sh'make'

}

}

stage('Test'){

steps{

sh'makecheck'

junit'reports/**/*.xml'

}

}

stage('Deploy'){

steps{

sh'makepublish'

}

}

}

}

TravisCITravisCI(https://travis-ci.org/)isadistributedbuildserverusedtobuildandtestsoftwareprojectshostedonGitHub.Travissupportsopensourceprojectswithnocharge.

TheconfigurationofTravisCIisdoneusingafilenamed.travis.yaml.Thecontentofthisfileisstructuredusingdifferentkeywords,including:

language:Projectlanguage,thatis,java,node_js,ruby,python,orphpamongothers(thecompletelistisavailableonhttps://docs.travis-ci.com/user/languages/).sudo:Flagvaluetosetifsuperuserprivilegesareneeded(forexampletoinstallUbuntupackages).dist:BuildscanbeexecutedonLinuxenvironments(UbuntuPrecise12.04orUbuntuTrusty14.04).addons:Declarativeshortcutstobasicoperationsoftheapt-getcommands.install:FirstpartoftheTravisbuildlifecycle,inwhichtheinstallationoftherequireddependenciesisdone.Thispartcanbeoptionallyinitiatedusingbefore_install.script:Actualexecutionofthebuild.Thisphasecanbeoptionallysurroundedbybefore_scriptandafter_script.deploy:Finally,thedeploymentofthebuildcanbeoptionallymadeinthisphase.Thisstagehasitsownlifecyclecontrolledwithbefore_deployandafter_deploy.

YAMLislightweightmarkuplanguageusedbroadlyforconfigurationfilesduetoitsminimalistsyntax.ItwasoriginallydefinedasYetAnotherMarkupLanguage,butthenitwasrepurposedtoYAMLAin’tMarkupLanguagetodistinguishitspurposeasdataoriented.

Thefollowingsnippetshowsanexampleof.travis.yaml:language:java

sudo:false

dist:trusty

addons:

firefox:latest

apt:

packages:

-google-chrome-stable

sonarcloud:

organization:"bonigarcia-github"

token:

secure:"encripted-token"

before_script:

-exportDISPLAY=:99.0

-sh-e/etc/init.d/xvfbstart&

-sleep3

script:

-mvntestsonar:sonar

-bash<(curl-shttps://codecov.io/bash)

TravisCIprovidesawebdashboardinwhichwecancheckthestatusofthecurrentandpastbuildgeneratedintheprojectsusingTravisCIofourGitHubaccount:

TravisCIdashboard

TestreportingFromitsinitialversions,theJUnittestingframeworkintroducedanXMLfileformattoreporttheexecutionoftestsuites.Overtheyears,thisXMLformathasbecomeadefactostandardforreportingtestresults,broadlyadoptedinthexUnitfamily.

TheseXMLcanbeprocessedbydifferentprogramstodisplaytheresultsinahuman-friendlyformat.Thisisforexamplewhatbuildserversdo.Forexample,JenkinsimplementsatoolcalledJUnitResultArchiver,whichparsestoHTMLtheXMLfilesresultingfromthetestexecutionofajob.

DespitethefactthatthisXMLformathasbecomepervasive,thereisnouniversalformaldefinitionforit.JUnittestexecutors(forexample,Maven,Gradle,andsoon)usuallyuseitsownXSD(XMLSchemaDefinition).Forinstance,thestructureofthisXMLreportinMaven(http://maven.apache.org/surefire/maven-surefire-plugin/)isasdepictedinthefollowingdiagram.Notethatatestsuiteiscomposedbyasetofpropertiesandasetoftestcases.Eachtestcasecanbedeclaredasafailure(testwithsomeassertionfailed),skipped(testignored),andanerror(testwithanunexpectedexception).Ifnoneofthesestatesappearinthebodyofthetestsuite,thenthetestisinterpretedassuccessful.Finally,foreachtestcasetheXMLalsostoresthestandardoutput(system-out)andthestandarderroroutput(system-err):

SchemarepresentationforMavenSurefireXMLreports

ThererunFailureisacustomstateimplementedbyMavenSurefireforretryingflaky(intermittent)tests(http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).

WithregardstoJUnit5,theMavenandGradlepluginsusedtorunJupitertests(maven-surefire-pluginandjunit-platform-gradle-pluginrespectively)writetheresultsofthetestexecutionfollowingthisXMLformat.Inthefollowingsections,wearegoingtoseehowtotransformthisXMLoutputtoahumanreadableHTMLreport.

MavenSurefireReportBydefault,maven-surefire-plugingeneratestheXMLresultingfromatestsuiteexecutionas${basedir}/target/surefire-reports/TEST-*.xml.ThisXMLoutputcanbeeasilyparsedtoHTMLusingthepluginmaven-surefire-report-plugin.Tothat,wesimplyneedtodeclarethisplugininthereportingclauseofourpom.xml,asfollows:

<reporting>

<plugins>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-report-plugin</artifactId>

<version>${maven-surefire-report-plugin.version}</version>

</plugin>

</plugins>

</reporting>

Thisway,whenweinvoquetheMavenlifecyclefordocumentation(mvnsite),anHTMLpagewiththetestresultwillbeincludedinthegeneralreport.

Seeanexampleofthereport,madeusingtheprojectjunit5-reportingwithintheGitHubrepositoryexamples(https://github.com/bonigarcia/mastering-junit5):

HTMLreportgeneratedbymaven-surefire-report-plugin

AllureAllure(http://allure.qatools.ru/)isalight-weightopensourceframeworkforgeneratingtestreportsfordifferentprogramminglanguages,includingJava,Python,JavaScript,Ruby,Groovy,PHP,.NET,andScala.Generaliyspeaking,AllureusestheXMLtestoutputandtransformsitinanHTML5-richreport.

AllureprovidessupportforJUnit5projects.ThiscanbedoneusingbothMavenandGradle.RegardingMaven,weneedtodoregisteralistenerinmaven-surefire-plugin.ThislistenerwillbetheclassAllureJunit5(locatedinthelibraryio.qameta.allure:allure-junit5),whichisbasicallyaimplementationoftheJUnit5’sTestExecutionListener.Asdescribedinchapter2,What’sNewInJUnit5,TestExecutionListenerispartoftheLauncherAPI,anditisusedtoreceiveeventsaboutthetestexecution.Allinall,thislistenerallowstoAlluretocompilethetestinformation,whileitisgeneratedintheJUnitplatform.ThisinformationisstoredasJSONfilesbyAllure.Afterthat,wecanusethepluginio.qameta.allure:allure-maventogeneratetheHTML5fromtheseJSONfiles.Thecommandsare:

mvntest

mvnallure:serve

Thecontentofourpom.xmlshouldcontainthefollowing:<dependencies>

<dependency>

<groupId>io.qameta.allure</groupId>

<artifactId>allure-junit5</artifactId>

<version>${allure-junit5.version}</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-api</artifactId>

<version>${junit.jupiter.version}</version>

<scope>test</scope>

</dependency>

</dependencies>

<build>

<plugins>

<plugin>

<artifactId>maven-surefire-plugin</artifactId>

<version>${maven-surefire-plugin.version}</version>

<configuration>

<properties>

<property>

<name>listener</name>

<value>io.qameta.allure.junit5.AllureJunit5</value>

</property>

</properties>

<systemProperties>

<property>

<name>allure.results.directory</name>

<value>${project.build.directory}/allure-results</value>

</property>

</systemProperties>

</configuration>

<dependencies>

<dependency>

<groupId>org.junit.platform</groupId>

<artifactId>junit-platform-surefire-provider</artifactId>

<version>${junit.platform.version}</version>

</dependency>

<dependency>

<groupId>org.junit.jupiter</groupId>

<artifactId>junit-jupiter-engine</artifactId>

<version>${junit.jupiter.version}</version>

</dependency>

</dependencies>

</plugin>

<plugin>

<groupId>io.qameta.allure</groupId>

<artifactId>allure-maven</artifactId>

<version>${allure-maven.version}</version>

</plugin>

</plugins>

</build>

ThesameprocesscanbedoneusingGradle,thistimeusingtheequivalentplugin,io.qameta.allure:allure-gradle.Allinall,thecontentofourbuild.gradlefileshouldcontain:

buildscript{

repositories{

jcenter()

mavenCentral()

}

dependencies{

classpath("org.junit.platform:junit-platform-gradle-

plugin:${junitPlatformVersion}")

classpath("io.qameta.allure:allure-gradle:${allureGradleVersion}")

}

}

applyplugin:'io.qameta.allure'

dependencies{

testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")

testCompile("io.qameta.allure:allure-junit5:${allureJUnit5Version}")

testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")

}

ThefollowingpictureshowsseveralscreenshotsoftheAllurereportgeneratedusingtheabove-mentionedsteps(thefinalresultisthesameusingMavenorGradle).Theprojectofthisexampleiscalledjunit5-allure,asusualhostedinGitHub.

AllurereportsgeneratedinaJUnit5project

Defect-trackingsystemsAdefect-trackingsystem(alsoknownasbugtrackingsystem,bugtracker,orissuetracker)isasoftwaresystemthatkeepstrackofreportedsoftwaredefectsinsoftwareprojects.Themainbenefitsofthiskindofsystemsistoprovideacentralizedoverviewofdevelopmentmanagement,bugreporting,andevenfeaturerequest.Itisalsocommontomaintainalistofpendingitems,oftencalledbacklog.

Thereareabunchofdefect-trackingsystemsavailable,bothproprietaryandopensource.Inthissection,wemakeabriefofseveralofthemostwell-known:

JIRA(https://www.atlassian.com/software/jira):Itisaproprietarydefect-trackingsystemcreatedbyAtlasian.Inadditiontobugandissuetracking,itprovidesmanagementscapabilitiessuchasSCRUMandKanbanboards,alanguagetoqueryissues(JIRAQueryLanguage),integrationwithexternalsystems(forexample,GitHub,Bitbucket),andanadd-onsmechanismtoextendJIRAwithpluginsfromtheAtlasianMarketplace(https://marketplace.atlassian.com/).Bugzilla(https://www.bugzilla.org/):Itisanopensourceweb-based,defect-trackingsystemdevelopedbytheMozillaFoundation.Amongitsfeatures,wecanfindadatabasedesignedtoimproveperformanceandscalability,querymechanismforsearchingdefects,integratede-mailcapabilities,anduserrolesmanagement.Redmine(http://www.redmine.org/):Itisanopensource,web-baseddefect-trackingsystem.Itprovideswikis,forums,timetracking,role-basedaccesscontrol,orGanttchartsforprojectmanagement.MantisBT(https://www.mantisbt.org/):Itisanotheropensource,web-baseddefecttrackingsystemdesignedtobesimplebuteffective.Amongitsfeatures,wecanhighlightitsevent-drivenpluginsystemtoallowsextensionsbothofficialthatthird-party,multi-channelnotificationsystem(e-mail,RSSfeed,Twitterplugin,andsoon),orrole-basedaccesscontrol.GitHubissues(https://guides.github.com/features/issues/):ItisthetrackingsystemintegratedineachGitHubrepository.TheapproachofGitHubissuesistoprovideagenerictrackingsystemfordefects,taskscheduling,discussions,andevenfeaturerequestusingGitHubissues.

Eachissuecanbecategorizedusingacustomizablelabelsystem,participatorsmanagement,andnotifications.

StaticanalysisThisbook,whichisfinishingsoon,hasbeenfocusedonsoftwaretesting.Nosurprises,JUnitisabouttesting.ButasweseeninChapter1,RetrospectiveonsoftwarequalityandJavatesting,althoughsoftwaretestingisthemostcommonlyperformedactivitieswithinVerification&Validation(V&V),itisnottheonlytype.Theotherimportantgroupofactivitiesisstaticanalysis,inwhichthereisnoexecutionofthesoftwaretesting.

Therearedifferentactivitiesthatcanbecategorizedasstaticanalysis.Amongthem,theautomatedsoftwareanalysisisanalternativequiteinexpensiveintermsofrequiredeffort,anditcanhelptoincreasetheinternalcodequalitysignificantly.Inthischapter,wearegoingtoreviewseveralautomatedsoftwareanalysistools,knownaslinters,namely:

Checkstyle(http://checkstyle.sourceforge.net/):ItanalyzesJavacodefollowingdifferentrules,suchasmissingJavadoccomments,theuseofmagicnumbers,namingconventionsofvariablesandmethods,method’sargumentlengthandlinelengths,theuseofimports,thespacesbetweensomecharacters,thegoodpracticesofclassconstruction,orduplicatedcode.ItcanbeusedasEclipseorIntelliJplugin,amongothers.FindBugs(http://findbugs.sourceforge.net/):ItlooksforthreetypesoferrorswithinJavacode:

Correctnessbug:Apparentcodingmistake(forexample,classdefinesequal(Object)insteadofequals(Object).Badpractice:Violationsofrecommendedbestpractices(droppedexceptions,misuseoffinalize,andsoon).Dodgyerrors:Confusingcodeorwritteninawaythatleadstoerror(forexample,classliteralneverused,switchfallthrough,unconfirmedtypecasts,andredundantnullcheck.

PMD(https://pmd.github.io/):Itisacross-languagestaticcodeanalyzer,includingJava,JavaScript,C++,C#,Go,Groovy,Perl,PHP,amongothers.Ithasalotofplugins,includingMaven,Gradle,Eclipse,IntelliJ,andJenkins.SonarQube(https://www.sonarqube.org/):It(formerlyjustSonar)isaweb-based,opensourcecontinuousqualityassessmentdashboard.Itsupportsawidevarietyoflanguages,includingJava,C/C++,Objective-C,C#,andmanyothers.Offersreportsonduplicatedcode,codesmells,code

coverage,complexityandsecurityvulnerabilities.SonarQubehasadistributedflavorcalledSonarCloud(https://sonarcloud.io/).Itcanbeusedforfreeinopensourceprojects,providingaseamlessintegrationwithTravisCIthroughafewlinesofconfigurationin.travis.yml(seethefollowingsnippet),includingtheSonarCloudorganizationidentifierandsecuretoken.TheseparameterscanbeobtainedintheSonarCloudwebadministrationpanel,afterassociatingoutSonarCloudaccountwithGitHub.

addons:

sonarcloud:

organization:"bonigarcia-github"

token:

secure:"encrypted-token"

Afterthat,wesimplyneedtocallSonarCloud,usingMavenorusingGradle:script:

-mvntestsonar:sonar

script:

-gradletestsonarQube

ThefollowingpictureshowstheSonarClouddashboardfortheexampleapplicationRatemycat!,describedinthelastsectionofthischapter:

SonarCloudreportfortheapplicationRatemycat!

Anotheranalysisstatictechniquehighlyadoptedinmanysoftwareprojectsispeerreview.Thismethodisquiteexpensiveintermsoftimeandeffortrequired,butwhencorrectlyapplied,itallowstomaintainverygoodlevelsofinternalcodequality.Nowadaysthereisawiderangeoftoolsaimedtoeasethepeerreviewprocessofsoftwarecodebase.Amongothers,wefindthefollowing:

Collaborator(https://smartbear.com/product/collaborator/):Peercode(anddocumentation)reviewproprietytoolcreatedbythecompanySmartBear.Crucible(https://www.atlassian.com/software/crucible):On-premisescodereviewproprietytoolforenterpriseproducts,createdbyAtlassian.Gerrit(https://www.gerritcodereview.com/):Web-basedcodecollaborationopensourcetool.ItcanbeusedwithGitHubrepositorythroughGerritHub(http://gerrithub.io/).GitHubpullrequestreviews(https://help.github.com/articles/about-pull-request-reviews/):InGitHub,apullrequestisamethodforsubmittingcontributionsinthird-partyrepositories.AspartofthecollaborativetoolsprovidedbyGitHub,pullrequestsallowsreviewsandcommentsinaeasyandintegratedfashion.

PuttingallpiecestogetherInthislastsectionofthebook,wearegoingtoreviewsomeofthemajoraspectscoveredinthisbookwithapracticalexample.Tothataim,acompleteapplicationisdevelopedtogetherwithdifferenttypesoftestsimplementedwithJUnit5.

FeaturesandrequirementsThehistoryofourapplicationbeginswithahypotheticalperson,whichlovescats.Thispersonownsaclowder,andhe/shewouldliketogetfeedbackaboutthemfromtheexternalworld.Forthatreason,thisperson(wecanhim/herourclientfromnowon)contactswithustoimplementawebapplicationwhichsatisfieshis/herneeds.Thenameforthatapplicationwillbe“Ratemycat!”.Inaconversationwiththeclient,weelicitafollowinglistoffeaturesfortheapplicationtobedeveloped:

F1:Eachusershallratealistofcatsbywatchingitsnameandpicture.F2:Therateshallbedoneonceperuserusingastarmechanism(from0.5to5starspercat)andoptionallycommentscouldbeincludedpercat.

Aspartoftheanalysisphaseinourdevelopmentprocess,thosefeaturesarerefinedasalistoffunctionalrequirements(FR)asfollows:

FR1:Theapplicationpresentsalistofcats(composedbynameandpicture)totheenduser.FR2:Eachcatcanberatedindividually.FR3:Therangeforratingcatsisanintervalfrom0.5to5(stars).FR4:Optionallytothenumericratepercat,usersshallincludesomecomments.FR5:Eachenduseronlyshallrateeachcat(commentsand/orstars)once.

DesignSinceourapplicationisquitesimple,wedecidetostoptheanalysisphasehere,withoutmodelingourrequirementsasusecases.Instead,wemoveonmakingahigh-levelarchitecturaldesignofthewebapplicationusingtheclassicalthree-tiermodel:presentation,application(orbusiness)logic,anddatatier.Regardingtheapplicationlogic,asthefollowingpicturedepicts,twocomponentsareneeded.Firstone,calledCatServiceischargeofalltheratingactionsasdescribedintherequirementslist.Secondone,calledCookiesServicesisneededtohandleHTTPCookies,neededtoimplementFR5:

High-levelarchitecturaldesignfortheapplicationRatemycat!

Atthisstage,inthedevelopment,weareabletodecidethemajortechnologiesimpliedintheimplementationourapplication:

Spring5:Thiswillbethefoundationframeworkforourapplication.Concretely,weuseSpringMVCthroughSpringBoottosimplifythecreationofourwebapplication.Moreover,weuseSpringDataJPAusingasimpleH2databasetopersisttheapplicationdata,andThymeleaf(http://www.thymeleaf.org/)astemplateengine(forviewsinMVC).Finally,wealsousetheSpringTestmoduletomakein-containerintegrationtestsinaneasyway.JUnit5:Ofcourse,wecannotuseadifferenttestingframeworkthanJUnit5forourtestscases.Moreover,toimprovethereadabilityofourassertionsweuseHamcrest.Mockito:Inordertoimplementunittestcases,wewillusetheMockitoframework,isolatingtheSUTfromitsDOCsinseveralout-of-containerunittests.SeleniumWebDriver:Wewillalsoimplementdifferentend-to-endtestsusingSeleniumWebDrivertoexerciseourwebapplicationfromJUnit5

tests.GitHub:OursourcecoderepositorywillbehostedinapublicGitHubrepository.TravisCI:OurtestsuitewillbeexecutedeachtimeanewpatchiscommittedtoourGitHubrepository.Codecov:TotrackthecodecoverageofourtestsuitewewilluseCodecov.SonarCloud:Toprovideacompleteassessmentoftheinternalqualityofoursourcecode,wecomplementourtestprocesswithsomeautomaticstaticanalysisusingSonarCloud.

ThescreenshothereshowstheapplicationGUIinaction.Itisnotthemainobjectiveofthissectiontodigdeeperintheimplementationspecificsoftheapplication.VisittheGitHubrepositoryoftheapplicationonhttps://github.com/bonigarcia/rate-my-catfordetailsaboutit.

ScreenshotoftheapplicationRatemycat!

Thepicturesusedtoimplementthisexamplehavebeendownloadedfromthefreeimagesgalleryavailableonhttps://pixabay.com/.

TestsLet’sfocusnowontheJUnit5testsofthisapplication.Weimplementthreetypesoftests:unit,integration,andendtoend.Asintroducedbefore,fortheunittest,weuseMockitotoexercisetheSUTinisolation.Wedecidetounittestthetwomajorcomponentsofourapplication(CatServiceandCookiesServices)usingJavaclassescontainingdifferentJUnit5tests.

Considerthefirsttest(calledRateCatsTest).Ascanbeseenthecode,inthisclasswearedefiningtheclassCatServiceastheSUT(usingtheannotation@InjectMocks)andtheclassCatRepository(whichisusedbyCatServicewithdependencyinjection)astheDOC(usingtheannotation@Mock).Thefirsttestofthisclass(testCorrectRangeOfStars)isanexampleofparameterizedJUnit5tests.TheobjectiveofthistestiftoassesstheratemethodinsideCatService(methodrateCate).Inordertoselectthetestdata(input)forthistest,wefollowablack-boxstrategyandthereforeweusetheinformationoftherequirementsdefinition.Concretely,FR3statestherangeofstarstobeusedintheratingmechanismforcats.Followingaboundaryanalysisapproach,weselecttheedgesoftheinputrange,thatis,0.5and5.Thesecondtestcase(testCorrectRangeOfStars)alsoteststhesamemethod(rateCat),butthistimethetestevaluatestheSUTresponsewhenout-of-rangeinputsexercisetheSUT(negativetestscenario).Then,twomoretestsareimplementedinthisclass,thistimeaimedtoassessFR4(thatis,usingalsocommentstoratecats).NoticethatweareusingtheJUnit5@Tagannotationtoidentifyeachtestwithitscorrespondingrequirement:

packageio.github.bonigarcia.test.unit;

importstaticorg.hamcrest.CoreMatchers.equalTo;

importstaticorg.hamcrest.MatcherAssert.assertThat;

importstaticorg.hamcrest.text.IsEmptyString.isEmptyString;

importstaticorg.junit.jupiter.api.Assertions.assertThrows;

importstaticorg.mockito.ArgumentMatchers.any;

importstaticorg.mockito.Mockito.when;

importjava.util.Optional;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.junit.jupiter.params.ParameterizedTest;

importorg.junit.jupiter.params.provider.ValueSource;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.Cat;

importio.github.bonigarcia.CatException;

importio.github.bonigarcia.CatRepository;

importio.github.bonigarcia.CatService;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

@DisplayName("Unittests(black-box):ratingcats")

@Tag("unit")

classRateCatsTest{

@InjectMocks

CatServicecatService;

@Mock

CatRepositorycatRepository;

//Testdata

Catdummy=newCat("dummy","dummy.png");

intstars=5;

Stringcomment="foo";

@ParameterizedTest(name="Ratingcatwith{0}stars")

@ValueSource(doubles={0.5,5})

@DisplayName("Correctrangeofstarstest")

@Tag("functional-requirement-3")

voidtestCorrectRangeOfStars(doublestars){

when(catRepository.save(dummy)).thenReturn(dummy);

CatdummyCat=catService.rateCat(stars,dummy);

assertThat(dummyCat.getAverageRate(),equalTo(stars));

}

@ParameterizedTest(name="Ratingcatwith{0}stars")

@ValueSource(ints={0,6})

@DisplayName("Incorrectrangeofstarstest")

@Tag("functional-requirement-3")

voidtestIncorrectRangeOfStars(intstars){

assertThrows(CatException.class,()->{

catService.rateCat(stars,dummy);

});

}

@Test

@DisplayName("Ratingcatswithacomment")

@Tag("functional-requirement-4")

voidtestRatingWithComments(){

when(catRepository.findById(any(Long.class)))

.thenReturn(Optional.of(dummy));

CatdummyCat=catService.rateCat(stars,comment,0);

assertThat(catService.getOpinions(dummyCat).iterator().next()

.getComment(),equalTo(comment));

}

@Test

@DisplayName("Ratingcatswithemptycomment")

@Tag("functional-requirement-4")

voidtestRatingWithEmptyComments(){

when(catRepository.findById(any(Long.class)))

.thenReturn(Optional.of(dummy));

CatdummyCat=catService.rateCat(stars,dummy);

assertThat(catService.getOpinions(dummyCat).iterator().next()

.getComment(),isEmptyString());

}

}

Next,unittestevaluatesthecookiesservice(FR5).Tothataim,thefollowingtestusetheclassCookiesServiceasSUT,andthistimewearegoingtomockthestandardJavaobject,whichmanipulatestheHTTPCookies,thatis,javax.servlet.http.HttpServletResponse.Inspectingthesourcecodeofthistest

class,wecanseethatthefirsttestmethod(calledtestUpdateCookies)exercisetheservicemethodupdateCookies,verifyingwhetherornottheformatofthecookiesisasexpected.Nexttwotests(testCheckCatInCookiesandtestCheckCatInEmptyCookies)evaluatesthemethodisCatInCookiesoftheserviceusingapositivestrategy(thatistheinputcatcorrespondswiththeformatofthecookie)andanegativeone(theoppositecase).Finally,thelasttwotests(testUpdateOpinionsWithCookiesandtestUpdateOpinionsWithEmptyCookies)exercisethemethodupdateOpinionsWithCookiesValueoftheSUTfollowingthesameapproach,thatis,checkingtheresponseoftheSUTusingavalidandemptycookie.Allthesetestshavebeenimplementedfollowingawhite-boxstrategy,sinceitstestdataandlogicreliescompletelyinthespecificinternallogicoftheSUT(inthiscasehowthecookiesareformattedandmanaged).

Thistestdoesnotfollowpurewhite-boxapproachinthesenseofitsobjectiveistoexerciseallthepossiblepathswithintheSUT.Itcanbeseenaswhite-boxinthesenseofithasbeendesigneddirectlylinkedtotheimplementationratherthantherequirements.

packageio.github.bonigarcia.test.unit;

importstaticorg.hamcrest.CoreMatchers.containsString;

importstaticorg.hamcrest.CoreMatchers.equalTo;

importstaticorg.hamcrest.CoreMatchers.not;

importstaticorg.hamcrest.MatcherAssert.assertThat;

importstaticorg.hamcrest.collection.IsEmptyCollection.empty;

importstaticorg.mockito.ArgumentMatchers.any;

importstaticorg.mockito.Mockito.doNothing;

importjava.util.List;

importjavax.servlet.http.Cookie;

importjavax.servlet.http.HttpServletResponse;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.mockito.InjectMocks;

importorg.mockito.Mock;

importio.github.bonigarcia.Cat;

importio.github.bonigarcia.CookiesService;

importio.github.bonigarcia.Opinion;

importio.github.bonigarcia.mockito.MockitoExtension;

@ExtendWith(MockitoExtension.class)

@DisplayName("Unittests(white-box):handlingcookies")

@Tag("unit")

@Tag("functional-requirement-5")

classCookiesTest{

@InjectMocks

CookiesServicecookiesService;

@Mock

HttpServletResponseresponse;

//Testdata

Catdummy=newCat("dummy","dummy.png");

StringdummyCookie="0#0.0#_";

@Test

@DisplayName("Updatecookiestest")

voidtestUpdateCookies(){

doNothing().when(response).addCookie(any(Cookie.class));

Stringcookies=cookiesService.updateCookies("",0L,0D,"",

response);

assertThat(cookies,

containsString(CookiesService.VALUE_SEPARATOR));

assertThat(cookies,

containsString(Cookies.CAT_SEPARATOR));

}

@Test

@DisplayName("Checkcatincookies")

voidtestCheckCatInCookies(){

booleancatInCookies=cookiesService.isCatInCookies(dummy,

dummyCookie);

assertThat(catInCookies,equalTo(true));

}

@DisplayName("Checkcatinemptycookies")

@Test

voidtestCheckCatInEmptyCookies(){

booleancatInCookies=cookiesService.isCatInCookies(dummy,"");

assertThat(catInCookies,equalTo(false));

}

@DisplayName("Updateopinionswithcookies")

@Test

voidtestUpdateOpinionsWithCookies(){

List<Opinion>opinions=cookiesService

.updateOpinionsWithCookiesValue(dummy,dummyCookie);

assertThat(opinions,not(empty()));

}

@DisplayName("Updateopinionswithemptycookies")

@Test

voidtestUpdateOpinionsWithEmptyCookies(){

List<Opinion>opinions=cookiesService

.updateOpinionsWithCookiesValue(dummy,"");

assertThat(opinions,empty());

}

}

Let’smoveontothenexttypeoftests:integration.Forthistypeoftest,wearegoingtousethein-containertestcapabilitiesprovidedbySpring.Concretely,weusetheSpringtestobjectMockMvctoevaluatetheHTTPresponsesofourapplicationfromtheclient-side.Ineachtest,differentrequestsareexercisedverifyingiftheresponses(statuscodeandcontenttype)areasexpected:

packageio.github.bonigarcia.test.integration;

importstatic

org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;

importstatic

org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;

importstatic

org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;

importstatic

org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.boot.test.context.SpringBootTest;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

importorg.springframework.test.web.servlet.MockMvc;

@ExtendWith(SpringExtension.class)

@SpringBootTest

@DisplayName("Integrationtests:HTTPreponses")

@Tag("integration")

@Tag("functional-requirement-1")

@Tag("functional-requirement-2")

classWebContextTest{

@Autowired

MockMvcmockMvc;

@Test

@DisplayName("Checkhomepage(GET/)")

voidtestHomePage()throwsException{

mockMvc.perform(get("/")).andExpect(status().isOk())

.andExpect(content().contentType("text/html;charset=UTF-8"));

}

@Test

@DisplayName("Checkratecat(POST/)")

voidtestRatePage()throwsException{

mockMvc.perform(post("/").param("catId","1").param("stars","1")

.param("comment","")).andExpect(status().isOk())

.andExpect(content().contentType("text/html;charset=UTF-8"));

}

@Test

@DisplayName("Checkratecat(POST/)ofannon-existingcat")

voidtestRatePageCatNotAvailable()throwsException{

mockMvc.perform(post("/").param("catId","0").param("stars","1")

.param("comment","")).andExpect(status().isOk())

.andExpect(content().contentType("text/html;charset=UTF-8"));

}

@Test

@DisplayName("Checkratecat(POST/)withbadparameters")

voidtestRatePageNoParameters()throwsException{

mockMvc.perform(post("/")).andExpect(status().isBadRequest());

}

}

Finally,wealsoimplementseveralend-to-endtestsusingSeleniumWebDriver.Inspectingtheimplementationofthistest,wecanseethatthistestisusingtwoJUnit5extensionsatthesametime:SpringExtension(tostart/stoptheSpringcontextwithintheJUnit5tests’lifecycle)andSeleniumExtension(toinjectWebDriverobjectsaimedtocontrolwebbrowsersinthetestmethods).Inparticular,weusethreedifferentbrowsersinoneofthetests:

PhantomJS(headlessbrowser),toassessisthelistofcatsisproperlyrenderedinthewebGUI(FR1).Chrome,toratecatsusingthroughtheapplicationGUI(FR2).Firefox,toratecatsusingtheGUIbutgettinganerrorasaresult(FR2).

packageio.github.bonigarcia.test.e2e;

importstaticorg.hamcrest.CoreMatchers.containsString;

importstaticorg.hamcrest.CoreMatchers.equalTo;

importstaticorg.hamcrest.MatcherAssert.assertThat;

importstatic

org.openqa.selenium.support.ui.ExpectedConditions.elementToBeClickable;

importstatic

org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT;

importjava.util.List;

importorg.junit.jupiter.api.DisplayName;

importorg.junit.jupiter.api.Tag;

importorg.junit.jupiter.api.Test;

importorg.junit.jupiter.api.extension.ExtendWith;

importorg.openqa.selenium.By;

importorg.openqa.selenium.WebElement;

importorg.openqa.selenium.chrome.ChromeDriver;

importorg.openqa.selenium.firefox.FirefoxDriver;

importorg.openqa.selenium.phantomjs.PhantomJSDriver;

importorg.openqa.selenium.support.ui.WebDriverWait;

importorg.springframework.boot.test.context.SpringBootTest;

importorg.springframework.boot.web.server.LocalServerPort;

importorg.springframework.test.context.junit.jupiter.SpringExtension;

importio.github.bonigarcia.SeleniumExtension;

@ExtendWith({SpringExtension.class,SeleniumExtension.class})

@SpringBootTest(webEnvironment=RANDOM_PORT)

@DisplayName("E2Etests:userinterface")

@Tag("e2e")

publicclassUserInferfaceTest{

@LocalServerPort

intserverPort;

@Test

@DisplayName("ListcatsintheGUI")

@Tag("functional-requirement-1")

publicvoidtestListCats(PhantomJSDriverdriver){

driver.get("http://localhost:"+serverPort);

List<WebElement>catLinks=driver

.findElements(By.className("lightbox"));

assertThat(catLinks.size(),equalTo(9));

}

@Test

@DisplayName("RateacatusingtheGUI")

@Tag("functional-requirement-2")

publicvoidtestRateCat(ChromeDriverdriver){

driver.get("http://localhost:"+serverPort);

driver.findElement(By.id("Baby")).click();

StringfourStarsSelector="#form1span:nth-child(4)";

newWebDriverWait(driver,10)

.until(elementToBeClickable

(By.cssSelector(fourStarsSelector)));

driver.findElement(By.cssSelector(fourStarsSelector)).click();

driver.findElement(By.xpath("//*[@id=\"comment\"]"))

.sendKeys("Verynicecat");

driver.findElement(By.cssSelector("#form1>button")).click();

WebElementsucessDiv=driver

.findElement(By.cssSelector("#success>div"));

assertThat(sucessDiv.getText(),containsString("Yourvotefor

Baby"));

}

@Test

@DisplayName("RateacatusingtheGUIwitherror")

@Tag("functional-requirement-2")

publicvoidtestRateCatWithError(FirefoxDriverdriver){

driver.get("http://localhost:"+serverPort);

driver.findElement(By.id("Baby")).click();

StringsendButtonSelector="#form1>button";

newWebDriverWait(driver,10).until(

elementToBeClickable(By.cssSelector(sendButtonSelector)));

driver.findElement(By.cssSelector(sendButtonSelector)).click();

WebElementsucessDiv=driver

.findElement(By.cssSelector("#error>div"));

assertThat(sucessDiv.getText(),containsString(

"Youneedtoselectsomestarsforratingeachcat"));

}

}

Inordertomakeeasierthetraceabilityofthetestexecutions,inalltheimplementedtest,wehaveselectedmeaningfultestnamesusing@DisplayName.Inaddition,forparameterizedtests,weusetheelementnametorefinethetestnameofeachexecutionofthetest,dependingonthetestinput.ThefollowingscreenshotoftheexecutionofthetestsuiteinEclipse4.7(Oxygen):

ExecutionofthetestsuitefortheapplicationRatemycat!inEclipse4.7

Asintroducedbefore,weuseTravisCIasbuildservertoexecuteourtestsduringthedevelopmentprocess.IntheconfigurationofTravisCI(file.travis.yml),wesetuptwoadditionaltoolstoenhancethedevelopmentandtestprocessofourapplication.Ontheonehand,Codecovprovidesacomprehensivetestcoveragereport.Ontheotherhand,SonarCloudprovidesacompletestaticanalysis.BothtoolsaretriggeredbyTravisCIaspartofthecontinuousintegrationbuildprocess.Asaresult,wecanevaluateboththecoveragetestandtheinternalcodequalityofourapplication(suchascode

smells,duplicatedblocks,ortechnicaldebt)alongwithourdevelopmentprocess.

ThefollowingpictureshowsascreenshotoftheonlinereportprovidedbyCodecov(thereportprovidedbySonarCloudwaspresentedintheprevioussectionofthischapter):

\

CodecovreportfortheapplicationRatemycat!

Lastbutnotleast,weareusingseveralbadgesintheREADMEofourGitHubrepository.Concretely,weaddbadgesforTravisCI(statusofthelastbuildprocess),SonarCloud(statusofthelastanalysis),andCodecov(percentageofthelastcodecoverageanalysis):

GitHubbadgesfortheapplicationRatemycat!

SummaryInthischapter,wereviewedseveralconcernsaboutthemanagementsideofthetestingactivities.First,welearnedthattestingcanbemadeindifferentpartsofthesoftwaredevelopmentprocess(softwarelifecycle)dependingonthetestmethodology:BDD(acceptancetestsaredefinedbeforetherequirementanalysis),TDD(testsaredefinedbeforethedesignofthesystem),TFD(testsareimplementedafterthesystemdesign),andTLD(testsareimplementedafterthesystemimplementation).

CIisaprocessmoreandmoreusedinsoftwaredevelopment.Itconsistsontheautomatedbuildandtestofacodebase.Thisprocessistypicallytriggeredwithanewcommitinasourcecoderepository,suchasGitHub,GitLab,orBitbucket.CIisextendedtoContinuousDelivery(whenreleasesaremadetodevelopmentenvironment)andtoContinuousDeployment(whendeploymenttoproductionenvironmentismadecontinuously).Wereviewedtwoofthemostusedbuildserversnowadays:Jenkins(CIasaService)andTravis(in-premises).

Theresomeothertoolsthatcanbeusedtoimprovethemanagementoftests,forexample,reportingtools(suchasMavenSurefireReportorAllure)ordefecttrackingsystems(suchasJIRA,Bugzilla,Redmine,MantisBT,andGitHubissues).Automatedstaticanalysisisagreatcomplementtotesting,forexample,usinglinterssuchasCheckstyle,FindBugs,PMD,orSonarQube,andalsopeerreviewtoolssuchasCollaborator,Crucible,Gerrit,andGitHubpullrequestsreviews.

Toclosethisbook,thefinalsectionofthischapterpresentsacompletewebapplication(namedRatemycat!)anditscorrespondingJUnit5tests(unit,integration,andend-to-end).Itconsistsonawebapplicationsdevelopedandassessedusingdifferenttechnologiespresentedthroughoutthebook,namely,Spring,Mockito,Selenium,Hamcrest,TravisCI,Codecov,andSonarCloud.