DARK BACKGROUND Feedzai How to Choose a Machine Learning ... · Businesses today are engaged in a war against savvy and creative fraudsters. At the same time, they are trying to keep

How to Choose a Machine Learning Platform for Risk 043017

AGUIDEFORMERCHANTS,ACQUIRERS,ANDPAYMENTSERVICEPROVIDERS

HTC043017

Page2

A Guide for Merchants, Acquirers, and Payment Service Providers: How to Choose a Machine Learning Platform for Risk

AGUIDEFORMERCHANTS,ACQUIRERS,ANDPAYMENTSERVICEPROVIDERS

HowtoChooseaMachineLearningPlatformforRisk

TABLEOFCONTENTS

03TheImpactofPoorFraudDetection

04IsitTimetoSwitch?

05WhattoLookForinaMachineLearningPlatform

12ChoosingaMachineLearningPlatform:AChecklist

Page3


01:THEIMPACTOFPOORFRAUDDETECTION

Poorfrauddetectionhasthreebigeconomicimpactsonyourbusiness.

1

First,there’sthefinancialimpactofdenyingalegitimatecustomer.ABusinessInsider

reportfoundthatbusinessesarelosingmoreinfalsedeclinesthanthey’restoppingin

fraud.Foreverydollarabusinesspreventsinfraud,itlosesevenmore–$1.32–to

falsedeclines.1

2

Second,there’sthecostimpactthatchargebackshaveonyourbusiness.Beyondthe

costsyouincur,there'sthemanualtimeyouspenddisputingthem,thefeesyoupay

forthechargebackprocess,thelostgoods,thethreatofthewatchlist,andeventhe

shutdownofyouraccount.

3

Third,there’stheimpactonthespeedandefficiencyofyouroperations.Large

organizationsneedtoreviewandprocessordersasquicklyaspossible.Poorfraud

detectionplatformscreatebottlenecksthatdelayorderfulfillment,frustrate

customers,andthwartyoureffortsduringpromotionsandpeakperiods.

1 http://www.businessinsider.com/the-false-declines-report-the-86-billion-problem-undermining-e-commerce-merchants-fraud-prevention-strategies-2016-7

Page4


02:ISITTIMETOSWITCH?Agrowingbusinessneedsafrauddetectionplatformthatcankeepup.Areyoublockingtoomanylegitimatetransactions?Isyourchargebackratetoohigh?Areyoumissingtoomanyshipments?

Businessestodayareengagedinawaragainst

savvyandcreativefraudsters.Atthesametime,

theyaretryingtokeeppacewithrisingcustomer

expectationsoffastandfrictionlessexperiences.

Businessesarealsocompetingwithother

businessesinanarmsracetoacquirethebest

machinelearningplatformforfrauddetection.

ThedawnofattainableAIhasintroducedusto

newsystemsthatarenowcapableofdetecting

fraudautomatically,bysurfacingpatternsand

insightsthatwereinvisiblebefore.Butall

machinelearningplatformsarenotcreatedequal.

Choosingtherightplatformcanmakeahuge

differencewhenitcomestoprovidinggreat

customerexperiences,drivinggreatoperations,

andwinningthewaronfraud.

Whatdoestherightplatformlooklike?

Thisguidewillhelpyouchoosebyillustratingthe

capabilitiesyouneedwhenconsideringaplatform

that’sfast,thatscales,andthatexplainsits

thinking.We’vealsoprovidedachecklistatthe

endtosummarizethisreport.

Page5


03:WHATTOLOOKFORINAMACHINELEARNINGPLATFORMAGILEMODELS

Cantheplatformsupportmultiplemodels?Howaboutcustommodels?Cantheplatformkeeppacewithfraudasitevolves,andbeproactiveratherthanreactive?

Multiplemodelsareimportantbecause

fraudstersbehavedifferentlydependingon

wheretheyare,andwhatthey’redoing.

Platformsthatcanonlyrunonemodelwillusea

generalizedviewtolookforspecializedbehaviors,

whichresultsinlowerfrauddetectionratesand

higherfalsepositives.

Aplatformwithmultiplemodelsmustalsobe

abletoscoreordersatscale,bytakingallthe

differentinputstreamsfromthesemultiple

modelsandtranslatingthemintoonesingle

storylineorriskprofilethatahumanreviewercan

understand.

Beyondmultiplemodels,cantheplatformsupport

custommodels?

Intherushtobuyamachinelearningplatform,a

businessrunstheriskofpurchasinganoff-the-

shelfsolution.Buteverybusinessisdifferent,and

everymodelshouldbedifferenttoo.Certain

businesseswillwantcustommodelsthatarebuilt

bydatascientistswithdomainexpertiseinspecific

industries.Custommodelsmakeformorepower

andmoreinsight,becausetheytakeintoaccount

theuniquefraudpatternsthateachindustryfaces.

Page6


EXPLAINABLELOGIC

Doestheplatformcontainahuman-readablesemanticlayerthatexplainsitsunderlyinglogic?Machinelearningsystemsthatuseneuralnetworkalgorithms,forinstance,maybeeffectiveinpatterndetection,buttheirlogicishardtodecipher.

Somemachinelearningsystemsmaketheir

decisionsinsideablackbox,whichcreatestwo

seriousproblems.

First,there’sacontrolproblem,becausehumans

can’tmanageandimproveasystemtheydon’t

understand.Second,there’saregulation

problem,becauseabusinesscan’tdocumentor

validatethedecisionsthathappeninsideablack

box.

Comparethattoaplatformthatdoeswhitebox

processingtoprovideclear,human-understandable

reasonsforitsdecisions.Certainmachinelearning

platformspredictpatternsusinganalgorithm

calledRandomForest,whichismadeoftensof

thousandsofdecisiontrees.Awhiteboxsystemcan

takethefewtop-mostfactorsfromthisensemble

ofdecisiontrees,thenweighandcommunicate

themtothehumaninasimpleway.

ALGORITHM PRO CON

RandomForest(akaEnsembleofDecisionTrees)

•Generalizespatternswell•Robusttodifferentinputtypes(texts,numbersofscales,etc.)•Robusttomissingdata•Robusttooutliersanderrors•Fasttotrainandscore•Triviallyparallel•Requireslesstuning•Probabilisticoutput•Canadjustthresholdtotradeoffbetweenprecisionandrecall•Verygoodpredictivepower•FoundtowinmanyMLcompetitions

•Canbecomecomplextointerpretasnumberofdecisionsgrow(inherentnatureofincreasedcapacitytomakedecisions)•Requireslabeleddata

NeuralNetwork

•Abletorepresentcomplexpatterns•Goodpredictivepower

•Cannothandledifferentinputtypes•Needsscalinginputs•Can’thandlemissingvalues•Notscalable•Slow•Requirestuning•Lackofinterpretability•Requireslabeleddata

Page7


Theanatomyofawhitebox

CLEARANSWERSWhatdoweknowaboutthisorder?

REASONSTOALLOW

Description Risk RiskFactor

IPaddressisonlybeingusedbythiscustomer 0.50% 0.9x

IPcountrymatchesshippingcountry 0.50% 0.9x

IPcountrymatchescustomer’shomeandshippingcountries 0.40% 0.7x

IPcountrymatchescustomer’shome,shipping,andbullingcountries 0.30% 0.5x

REASONSTOBLOCK

Description Risk RiskFactor

Customerseenindistantlocationsinashortperiodoftime 60.00% 100.00x

Customerisbehindproxy 6.00% 10.0x

CustomerhasusedtwodifferentIPaddresses 0.90% 1.5x

Customerisusingafreeordisposableemailaddress 0.70% 1.1x

Forexample,awhiteboxexplanationmightsaythattheuserhasbeendeclinedbecausetherewerefour

recenttransactionsindistantlocations.Theideahereistoputhumansincontrolbyarmingthemwiththe

informationtheyneedtoreviewordersmoreintelligently,andtocommunicateclearlyaboutwhycertain

orderswereapprovedordeclined.

Page8


360°VISION

Dataisthefuelforamachinelearningengine.Tobeeffective,aplatformneedstobeabletotakeinalldatafromallsources,soitcaningesteverykindofsignalthatthecustomerisproviding,nomatterthechannelortheusecase.

Manysources,onestory

Manyplatformscanonlytakeindatafromcertain

sourcesorchannels.Forinstance,aplatform

mightlookatpoint-of-saledatabutnotmobile

devices.

Thesesinglechannel,singleusecaseplatforms

canbebreached.Ittakesaplatformwith

omnidataandomnichannelcapabilitiestokeep

pacewithtoday’sfraudsters.

Togetacompleteviewofeverytransaction,a

businessneedsaplatformthatcaninstantlyingest

everykindofdata,whetherit’scomingfroman

internalsystem,likeyourCRMsystem,orexternal

datasources,likeenrichers.

Page9


HYPERGRANULARPROFILES

Aplatformreachesitsdecisionstocreateprofilesofnormalbehavior,andflaginstancesofabnormalbehavior.Butsomemachinelearningplatformscanzoomintoahigherlevelofresolutionthanothers.

COHORTSCOMEBYTHEDOZEN

SEGMENTSOFONECOMEBYTHEHUNDREDSOFMILLION

CustomerSegments(BabyBoomer,Millennial)AccountType(Gold,Platinum)

MerchantCategory(DigitalGoods,Fashion)DeviceClass(iPhone,iPad)

EachCustomer(Susan)EachAccountNumber(134149553)

EachMerchantID(XX4956Y)EachDeviceID(WERB5909O2BYT)

Doestheplatformcreateprofilesofcohorts?Or

doesitusesegment-of-oneprofilestolookat

individualentities?Segment-of-oneprofiles

don’tlookat“womenbetweentheagesof30

and35.”Theylookat“Susan.”Andtheydon’t

lookat“devicesinthiszipcode.”Theylookat

“Susan’sdevice.”

Bycreatingprofilesofmanydifferententities(the

merchant,thedayoftheweek,thecreditcard),

machinelearningplatformscanreachnewlevelsof

sensitivity,anddetectfraudwithgreaterpowerand

accuracy.

Page10


GLOBALCASEMANAGEMENT

Big,complexbusinessesneedascalablecasemanagementtool,onethatmakesdistributedteamseffectiveatmanagingmanualreviewqueuesthatareglobalandround-the-clock.

Organizationscanhavemanyhundredsofpeople

loggingintothesamesystem.Acase

managementtoolneedstohaveself-configurable

workflowssothateveryoneinthisdiverse

userbasecanseetherightactionableinsightsat

therighttime.

Doyouhaveacasemanagementtoolthat

automaticallydistributesworkbasedondifferent

criteria,andprioritizesthemostimportantorders

toreview?

Canteamsacrossfraud,shipping,andcustomer

serviceallgetexactlywhattheyneedoutofthe

samesystem?

Therightcasemanagementtoolalsoneedstomake

surethatmanualreviewqueuesareprioritizedand

sortedinawaythatallowsforbusinesscontinuity.

Otherwisetherewillbegapsinyourqueue

managementthatafraudstercancircumvent.

Page11


SELF-CONFIGURABLEPLATFORM

Inaworldofgas-poweredcars,machinelearningsystemsarethenewelectriccars.Andaplatformthat’sself-configurablegoesonestepfurther.Itmeansyou’renotjustbuyingthecar.You’rebuyingthecarfactory.

Aplatformwithself-configurabledashboardslets

youcustomizethespecificmetricsthatyouwantto

see,soyoucanhaveownershipoveryourcontrol

andcommandcenter,andflexibilitytochoose

businessKPIsthatarerelevanttomanagingfraud.

Maybeyouwanttomeasurechargebackratesby

department,orcountry,orpartner.

Maybeyouwanttoseechargebackratesbasedon

teammembers,orbusinessunit,orproductline.

Typically,ifyouwanttochangeyourdashboard

reporting,youneedtogobacktothevendor.But

withself-configurabledashboardsandreporting,

thecontrolisyours.

AUTOMATICFEEDBACKLOOP

Abigbusinessneedstofindeveryopportunitytoautomate.Amachinelearningplatformshouldbeabletoautomateitsingestionofchargebackdatabyintegratingwiththeacquiringbank’sdata.Otherwiseemployeeswillneedtopulluporders,onebyone,tologchargebacksmanually.

Aplatformshouldalsointegratewithmanual

reviewdecisions.Whenafraudmanagerdoes

researchandmarksasuspicioustransactionas

fraud,therightplatformgetsmovingandfindsthe

patternsassociatedwiththefraud.Theanalysthas

madethemachinesmarter.

Automationletsaplatformthinkindependently,

whichmeansaplatformthatcanfollowthethread

oflogiconitsowntodetectwell-coordinated

criminalbehaviorinrealtime.Forexample,a

platformwithautomationwillfindthepatterns

associatedwithfraudspikes,sothatwhenafraud

ringmovestoadifferentcity,theplatformwill

detecttheirbehaviorimmediately.

Page12


04:CHOOSINGAMACHINELEARNINGPLATFORM:CHECKLIST

☐ AGILEMODELSCantheplatformhavemultiplemodelstosupportyourbusinessatscale?Canitsupportcustommodelsthatarebasedondomainexpertiseofthefraudpatternsspecifictoyourindustry?

☐ EXPLAINABLELOGICDoestheplatformhavewhiteboxAIthatexplainsitsreasoning,putsthehumanincontrol,andleavesapapertrailforregulators?

☐ 360°VISIONWilltheplatformtakeacompleteviewbyingestingeverykindofdatainyourecosystem,protectingthecustomersfromthebeginningoftheirtransactionjourneytotheend?

☐ HYPERGRANULARPROFILESDoestheplatformdetectanomaliesbycreatingnormalbehaviorprofilesforsegmentsofmany?Ordoesitzoomin,andlookatsegment-of-oneprofiles?

☐ GLOBALCASEMANAGEMENTHowmuchscaleandcomplexitycanthecasemanagementtoolhandle?Doyouhaveawaytoautomaticallyprioritizeanddistributemanualreviewwork?

☐ SELF-CONFIGURABLEPLATFORMDoyouhavetheownershipandflexibilitytoself-manageyourdashboardsandreporting,soyoucanseetherightmetricsandKPIsrelevanttofightingfraudatyourorganization?

☐AUTOMATICFEEDBACKLOOPDoesyourplatformintegratewithchargebackdataandmanualreviewdatasothatitcanlearnandimproveonitsown,automatically?

Page13


ABOUTFEEDZAIFeedzaiisAI.We’recodingthefutureofcommercewithtoday’smostadvancedriskmanagementplatformpoweredbybigdataandartificialintelligence.Ourmissionistomakecommercesafe.Theworld’slargestbanks,paymentprovidersandretailersuseFeedzai’smachinelearningtechnologytomanagerisksassociatedwithbankingandshopping,whetherdigitallyorinperson.

www.feedzai.com

[email protected]

[email protected]

Documents

DARK BACKGROUND Feedzai How to Choose a Machine Learning ... · Businesses today are engaged in a war against savvy and creative fraudsters. At the same time, they are trying to keep