Upload
vothuan
View
225
Download
2
Embed Size (px)
Citation preview
Techniquesforbugfinding
Automa'ctestcasegenera'on
LowercoverageLowerfalseposi0vesHigherfalsenega0ves
Fuzzing DynamicsymbolicexecuFon
StaFcanalysis ProgramverificaFon
HighercoverageHigherfalseposi0vesLowerfalsenega0ves
Blackboxfuzzing
• Givenaprogramsimplyfeedrandominputsandseewhetheritexhibitsincorrectbehavior(e.g.,crashes)
• Advantage:easy,lowprogrammercost• Disadvantage:inefficient
– InputsoUenrequirestructures,randominputsarelikelytobemalformed
– InputsthattriggeranincorrectbehaviorisaaverysmallfracFon,probablyofgeVngluckyisverylow
Fuzzing
• AutomaFcallygeneratetestcases• Manyslightlyanomaloustestcasesareinputintoatarget
• ApplicaFonismonitoredforerrors• Inputsaregenerallyeitherfilebased(.pdf,.png,.wav,etc.)ornetworkbased(hWp,SNMP,etc.)
Inputgenerator
Monitor
TestapplicaFon
ProblemdetecFon
• Seeifprogramcrashed– Typeofcrashcantellalot(SEGVvs.assertfail)
• Runprogramunderdynamicmemoryerrordetector(valgrind/purify/AddressSaniFzer)– Catchmorebugs,butmoreexpensiveperrun.
• Seeifprogramlocksup• Rollyourowndynamiccheckere.g.valgrindskins
Regressionvs.FuzzingRegrssion Fuzzing
DefiniFon Runprogramonmanynormalinputs,lookforbadness
Runprogramonmanyabnormalinputs,lookforbadness
Goals Preventnormalusersfromencounteringerrors(e.g.,asserFonfailuresarebad)
PreventaWackersfromencounteringexploitableerrors(e.g.,asserFonfailuresareoUenok)
Enhancement1:MutaFon-Basedfuzzing
• Takeawell-formedinput,randomlyperturb(flippingbit,etc.)
• LiWleornoknowledgeofthestructureoftheinputsisassumed
• AnomaliesareaddedtoexisFngvalidinputs– AnomaliesmaybecompletelyrandomorfollowsomeheurisFcs(e.g.,removeNULL,shiUcharacterforward)
• Examples:ZZUF,Taof,GPF,ProxyFuzz,FileFuzz,Filep,etc.
Seedinput Mutatedinput Runtestprogram
?
Example:fuzzingaPDFviewer
• Googlefor.pdf(about1billionresults)• Crawlpagestobuildacorpus• Usefuzzingtool(orscript)
– CollectseedPDFfiles– Mutatethatfile– Feedittotheprogram– Recordifitcrashed(andinputthatcrashedit)
MutaFon-basedfuzzing
• Supereasytosetupandautomate• LiWleornofileformatknowledgeisrequired• LimitedbyiniFalcorpus• Mayfailforprotocolswithchecksums,thosewhichdependonchallenge
EnhancementII:GeneraFon-BasedFuzzing
• TestcasesaregeneratedfromsomedescripFonoftheinputformat:RFC,documentaFon,etc.–Usingspecifiedprotocols/fileformatinfo–E.g.,SPIKEbyImmunity
• Anomaliesareaddedtoeachpossiblespotintheinputs
• KnowledgeofprotocolshouldgivebeWerresultsthanrandomfuzzing
Inputspec Generatedinputs Runtestprogram
?RFC
MutaFon-basedvs.GeneraFon-based
• MutaFon-basedfuzzer– Pros:Easytosetupandautomate,liWletonoknowledgeofinputformatrequired
– Cons:LimitedbyiniFalcorpus,mayfallforprotocolswithchecksumsandotherhardchecks
• GeneraFon-basedfuzzers– Pros:Completeness,candealwithcomplexdependncies(e.g,checksum)
– Cons:wriFnggeneratorsishard,performancedependsonthequalityofthespec
Howmuchfuzzingisenough?
• MutaFon-based-fuzzersmaygenerateaninfinitenumberoftestcases.Whenhasthefuzzerrunlongenough?
• GeneraFon-basedfuzzersmaygenerateafinitenumberoftestcases.Whathappenswhenthey’reallrunandnobugsarefound?
Codecoverage
• SomeoftheanswerstothesequesFonslieincodecoverage
• Codecoverageisametricthatcanbeusedtodeterminehowmuchcodehasbeenexecuted.
• Datacanbeobtainedusingavarietyofprofilingtools.e.g.gcov,lcov
Linecoverage
• Line/blockcoverage:Measureshowmanylinesofsourcecodehavebeenexecuted.
• Forthecodeontheright,howmanytestcases(valuesofpair(a,b))neededforfull(100%)linecoverage?
if( a > 2 ) a = 2; if( b >2 ) b = 2;
Branchcoverage
• Branchcoverage:Measureshowmanybranchesincodehavebeentaken(condiFonaljmps)
• Forthecodeontheright,howmanytestcasesneededforfullbranchcoverage?
if( a > 2 ) a = 2; if( b >2 ) b = 2;
Pathcoverage
• Pathcoverage:Measureshowmanypathshavebeentaken
• Forthecodeontheright,howmanytestcasesneededforfullpathcoverage?
if( a > 2 ) a = 2; if( b >2 ) b = 2;
BenefitsofCodecoverage
• CananswerthefollowingquesFons–HowgoodisaniniFalfile?–AmIgeVngstucksomewhere? if (packet[0x10] < 7) { //hot path� } else { //cold path } – HowgoodisfuzzerXvs.fuzzerY– AmIgeVngbenefitsbyrunningmulFplefuzzers?
Problemsofcodecoverage
• For:mySafeCopy(char *dst, char* src) { if(dst && src) strcpy(dst, src); }
• Doesfulllinecoverageguaranteefindingthebug?
• Doesfullbranchcoverageguaranteefindingthebug?
EnhancementIII:Coverage-guidedgray-boxfuzzing
• SpecialtypeofmutaFon-basedfuzzing– Runmutatedinputsoninstrumentedprogramandmeasurecodecoverage
– Searchformutantsthatresultincoverageincrease
– OUenusegeneFcalgorithms,i.e.,tryrandommutaFonsontestcorpusandonlyaddmutantstothecorpusifcoverageincreases
– Examples:AFL,libfuzzer
AmericanFuzzyLop(AFL)
Inputqueue
Seedinputs
Nextinput
MutaFon
Executeagainst
instrumentedtarget
branch/edge
coverageincreased?
Addmutanttothequeue
Periodicallycullsthequeuewithout
affecFngtotalcoverage
AFL
• Instrumentthebinaryatcompile-Fme• Regularmode:instrumentassembly• RecentaddiFon:LLVMcompilerinstrumentaFonmode• Provide64KcountersrepresenFngalledgesintheapp• Hashtablekeepstrackof#ofexecuFonofedges
– 8bitsperedge(#ofexecuFons:1,2,3,4-7,8-15,16-31,32-127,128+)
– Imprecise(edgesmaycollide)butveryefficient• AFL-fuzzisthedriverprocess,thetargetapprunsas
separateprocess(es)
Data-flow-guidedfuzzing
• Interceptthedataflow,analyzetheinputsofcomparisons– Incursextraoverhead
• Modifythetestinputs,observetheeffectoncomparisons
• PrototypeimplementaFonsinlibFuzzerandgo-fuzz
Fuzzingchallenges
• Howtoseedafuzzer?– Seedinputsmustcoverdifferentbranches– Removeduplicateseedscoveringthesamebranches
– SmallseedsarebeWer(Why?)
• Somebranchesmightbeveryhardtogetpastasthe#ofinputsstaFsfyingthecondiFonsareverysmall– Manually/automaFcallytransform/removethosebranches
Hardtofuzzcode
void test (int n) { if (n==0x12345678) crash(); }
needs2^32or4billionaWemptsIntheworstcase
Makeiteasiertofuzz
void test (int n) { int dummy = 0; char *p = (char *)&n; if (p[3]==0x12) dummy++; if (p[2]==0x34) dummy++; if (p[1]==0x56) dummy++; if (p[0]==0x56) dummy++; if (dummy==4) crash(); }
needsaround2^10aWempts
Fuzzingrulesofthumb• Input-formatknowledgeisveryhelpful• GeneraFonaltendstobeatrandom,beWerspecsmakebeWerfuzzers
• EachimplementaFonwillvary,differentfuzzersfinddifferentbugs– MorefuzzingwithisbeWer
• Thelongeryourun,themorebugsyoumayfind– ButitreachesaplateauandsaturatesaUerawhile
• Bestresultscomefromguidingtheprocess• NoFcewhereyouaregeVngstuck,useprofiling(gcov,lcov)!