SharingisCaring:UnderstandingandMeasuringThreatIntelligenceSharing

Effectiveness(#ddti)AlexPinto

ChiefDataScientistMLSec Project/Niddel

@alexcpsec@MLSecProject @NiddelCorp

• Previouslyon#ddti• ChallengesatTISharing• MeasuringTISharing• TheFutureofSharing

Agenda

Thisisadata-driventalk!Pleasecheckyouranecdotesatthedoor

Previouslyon#ddti• UsefulMethodsandMeasurementsforHandlingIndicators• AnalysisofThreatIntelligenceFeeds• Indirectly,amethodologyforanalyzingTIProviders

• Combine(https://github.com/mlsecproject/combine)• GathersTIdata(ip/host)fromInternetandlocalfiles

• TIQ-Test(https://github.com/mlsecproject/tiq-test)• RunsstatisticalsummariesandtestsonTIfeeds

TIQ-TEST- TonsofThreat-yTests

• NOVELTY – Howoftendothefeedsupdatethemselves?• AGING – Howlongdoesanindicatorsitonafeed?• POPULATION – Howdoesthispopulationdistributioncomparetomydata?

• OVERLAP– Howdotheindicatorscomparetotheonesyougot?

• UNIQUENESS – Howmanyindicatorsarefoundonlyononefeed?

Puttingthisthreatinteldatatowork

OverlapTest- Outbound

Ihatequotingmyself,but…

KeyTakeaway#1

MORE!=BETTERThreatIntelligenceIndicatorFeeds

ThreatIntelligenceProgram

ConstructiveFeedbackfromtheInternet:

“TISharingisTOTALLYgoingtosolvethis”

Right,folks?Right?

TISharingSolutionPlan:

1. ThebestThreatIntelligenceistheonethatyouanalyzefromyourownincidents(homegrown/organicintelligence)

2. Thereisstrengthinnumbers– verticalherdimmunity!

3. ????????

4. PROFIT!!(oratleastSECURITY!!)

Oratleastaroughstrawman

IfCONSUMINGisforthe1%,whatisthepercentageoforganizationsabletoPRODUCE?

Issue1- BYOTI

Issue2- HerdImmunity

Source:www.vaccines.gov

• Wemaybeabletodetectmore”virusstrains”togetherbutweare*terrible*atinoculation.

• Thethingswedetectthemostmutatetoofast(PyramidofPain)

• Whodidn’tgetimmunized,stillgetssick(FOMO-TI)

Issue?- Whatarewesharing• AUTOMATION-DRIVEN(PLATFORMS)• StraighttothepointIOCsharing

• ANALYST-DRIVEN(COMMUNITIES)• Strategicdata,bestpractices,unstructuredIOCs

• ”Analyst-driven”hasbeenaroundforever(innon-IC,atleastsinceFS-ISACwascreated)

• Thesamepeoplewhobash”justIOCsharing”:• BashSTIX/TAXIIfortryingtoencodecomplexity• TellseveryoneitisIMPOSSIBLEtohireanalysts

TheCognitiveDissonancesofTISharing

Everybody shouldshare! TheCIRCLEOFTRUST

Doyoutrustthegroupenoughtoconsume?

TheTwoSidesoftheTrustCoin

Doyoutrustthegroupenoughtoshare?

Okay,I’llbite

Canwemeasureourcurrentsharingplatformscommunities?

ThreatIntelligenceSharingWewouldliketothankthekindcontributionofdatafromthefinefolksatFacebookThreatExchange andThreatConnect

…andalsothesharingcommunitiesthatchosetoremainanonymous.Youknowwhoyouare,andwe❤ youtoo.

SharingCommunitiesARESocialNetworks

SocialNetworkSelfie SharingCommunitySelfie

Let’slookattheindicatorsfirst

UsingTIQ-TESTOverlapandUniquenesstests

OVERLAPSLIDE

OVERLAPSLIDE

UNIQUENESSSLIDE

Lookslikewewouldgetsimilarqualityona”good”ThreatIntelligenceSharingPlatformaswewouldon

a”paidfeed"

SuggestedMetricsforSharing

• ACTIVITY – Howmanyindicators/postsarebeingshareddaybyday?

• DIVERSITY –Whatisthepercentageofthepopulationthatisactivelysharing?

• FEEDBACK – Areorgscollaboratingonimprovingtheknowledgeinthesharingenvironment?

• TRUST– Howmuchdataisshared”openly”inrelationto”privately”?

Lookingforhealthydynamics

ActivityMetricIsthereanyactualsharinggoing

on?

Lessdata/Delays Moredata/Timely

LargeGroupisroughly40xbiggerthanSmallGroup

Organizationsarelesslikelytoshareiftheyperceivethey”lostcontrol”ofwhocanconsume.

DiversityMetricCheckyoursharingprivilege

Roughly10%oftheorganizationssharedataintothecommunity

Someorganizationsareclearlyinabetterpositionoperationallyandlegallytoshare.Andthatis

expectedduetoourpremises.

FeedbackMetricButisthedataanygood?

🙀 I’msurewecandobetterthanthis🙀

FeedbackMetric• Almostnosupportonautomation-drivenplatforms• Someallowyoutoleave”comments”or”newdescriptors”fortheIOCs– evenbycountingthoseverylow%inrelationtonewshareddata

• Analyst-drivenenvironmentsallowforcollaborationone-mailsandforumpoststodescribeandrefinestrategiesandbestpractices.

Howcanwemakethiscollaborationworkonautomation-drivenplatforms?

TrustMetricArewehelpingallthecommunity

orjustafeworgsatatime?

76%.Again,soundsaboutright

Overall”quality”ofdatagoesuptoo!

TrustMetric• Theroughestimateseemstobethatmorethan80%of”sharing”(IOCs,messages,etc)happensin”privategroups”insidetheinfrastructureofthesharingplatform

• Allcommunitieshavethem:• PartoftheDNAoftheIC/clearedcommunity• Offsetsthetrustequation,butdefeatsthe”herdimmunity”argument• UsuallyMANDATORYoncollaborationwithLEA

Butthenthe”good”dataisnothelping”thecommunity”!Isthereanywaywecanreconcile?

TheFutureofSharing🔮Attheveryleastmyhumble

opinion

#squadgoalsIncreasetheTRUST

amongpeers

ReducetheTECHNICALBARRIERforsharinguseful

information

TRUST:ReputationandAnonymity

AlienVault OTXclearlygotthememo

TRUST:Anonymity+GoodCuration

Somesharingcommunitiesacceptanonymoussubmissionsthattheythencurateanddisseminate

toallorganizations

IOCs

Feedback

TelemetryLESSMATURE

MOREMATURE

With❤ andapologiesto@DavidJBianco

TECHNICALBARRIER:”PyramidofSharing”

Takeaways• IntelligenceSharingisaveryanalyst-centricactivitythatwehavebeentaskedwithscalingoutwithautomation.Nowonderitseemssohard.

• Datacanbeasgoodasapaidfeed,butyouhavetobeintherightcirclesoftrust

• Doesnotsolveanalystshortageandmakingtheindicators/strategiesoperationalintoyourenvironment

Thanks!

• Q&A?• Feedback!

”Themeasureofintelligenceistheabilitytochange."- AlbertEinstein

AlexPinto@alexcpsec

@MLSecProject /@NiddelCorp