FindingIMBHsinStarClusterswithMachineLearning

MarioPasquatoMarieSkłodowskaCurieFellow(Astrofit2)

Is<tutoNazionalediAstrofisicaOsservatoriodiPadova

Incollabora<onwith:MirekGiersz,AbbasAskar,MichelaMapelli,MarioSpera,AlessandroBallone,ElisaBortolas,NicolaGiacobbo,UgoN.diCarlo,MariaC.Artale,AlessandraM.BaLs<,AlessandroA.Trani,FedericoAbbate

mario.pasquato@oapd.inaf.it

Theproblem

IsthisthepictureofanIntermediateMassBlackHole(IMBH)hostornon-host?

mario.pasquato@

oapd.inaf.it

Lookslikethisproblem

Isthisthepictureofadogoracat?

Butit’sreallynotthesame•  Dog/catàhand-labeleddata•  IMBH/noIMBHànosuchdata

≠

mario.pasquato@

oapd.inaf.it

Nottheusualmachinelearningproject

•  lightcurveclassifica<on,galaxymorphology,facerecogni<on,frauddetec<on,sen<mentanalysis...

•  Theyallusehand-labeleddata•  Machinelearningdoessomethingthatwealreadyknowhowtodo,justfaster

Classifica<on

Rawdata

Pre-labeleddata MLmodel

Trainingmanuallabeling

Towardsautoma<cscience

•  Wedonothavepre-labeleddata•  Wemakemockdatafromsimula<ons•  Telldogsfromcatsinpictures(realdata)basedondrawingsofcatsanddogs(simula<ons)...be]ermakegooddrawings

Compe<ngtheories

Classifica<onRawdata

Mockdata MLmodel

Simula?on Training

Indis<nguishable(?)

totallyadog

Trainonthis Testonthissimula<on observa<on

mario.pasquato@

oapd.inaf.it

ApproachI:deepConvolu<onalNeuralNet(CNN)directlyonmockimages

•  Makemock-observa<ons(images,velocitymaps...)•  Trainaconvolu<onalneuralnetwork•  Pixel-by-pixelvaluesareinput,featuresarenothandmadebutlearnedexplain!

IMBH

MOCCAtoMock+CNN•  MOCCASurveyDatabasesimula<onsturnedintoimagesusingCOCOA(Askaretal.2018)

•  MakeandtrainaConvolu<onalNeuralNet(CNN)inKerasontopofTensorflow,inpython

n=700000_zini=0.02_fracb=0.1_w0=9.0_iq=1_isemi=0_ikroupa=0_rbar=60.0_rplum=25.0_kfallb=1.jpeg

n=700000_zini=0.02_fracb=0.95_w0=9.0_iq=1_isemi=4_ikroupa=1_rbar=60.0_rplum=50.0_kfallb=1.jpeg

mario.pasquato@

oapd.inaf.it

Simula<ons:MOCCASurveydatabaseI•  1237simula<onsoutof~2000surviveto12Gyr

•  339formanIMBH>100Msun

•  Cleardis<nc<onbetweenstellarmassandIMBHàclassifica<onproblem✔

0

100

200

300

0 1 2 3 4log10(M MSun)

coun

ts

50Msun

1000Msun

Thedesert

Someexamples

ßPredictedhost(alsorealhost)

notseenintraining

confidence*0.811

flip_flop_n=1200000_zini=0.001_fracb=0.95_w0=9.0_iq=1_isemi=4_ikroupa=1_rbar=60.0_rplum=25.0_kfallb=1.jpeg

Everysnapshotat12GyrturnedintoanimageusingCOCOAbyAskaretal.2018

*outputofthelast(solmax)layer

Flipping/Flopping

Flippeds<llpredictedhost

confidence

0.776

Unexpectedimages

Imagesthatweshouldreallybeunabletoclassifyhave

lowerconfidence

HOST0.565 HOST0.514NONHOST0.587HOST0.565

Alteredimages

Coveringpartsoftheoriginalimage:s<llpre]yconfidentit’sahost

HOST0.829 HOST0.807HOST0.775HOST0.796

HOST0.720 HOST0.816 HOST0.811 HOST0.864

Openingtheblackboxwithablacksquare

Coveringthecenter

NONHOST!confidence

0.644

Importantinforma<onencodedinthecenter

Nicepictures,but?

•  Convolu<onalNeuralNetapproachs<llaworkinprogress

•  Ialsohaveresults...•  ...fromadifferentapproach•  FeaturebasedapproachPasquato,Askar,Giersz&Mapelli2018,MNRASalmostsubmi]ed

classifiedasnon-host

mario.pasquato@

oapd.inaf.it

ApproachII:handmadefeatures+conven<onalmachinelearning

•  Makemock-observa<ons(images)•  Turnimagesintonumbers(e.g.surfacedensityprofile)

•  Learnonthesenumbers

IMBH

Featureengineering

•  Manuallycalculatefeatures(numbersassociatedtoeachimage)

•  e.g.surfacedensityprofiles

•  EachimageàNnumbers

•  TrainclassifiersontheseNnumbers−1.0 −0.8 −0.6 −0.4 −0.2

12

34

log R

log

SApproachII:handmadefeatures+conven<onalmachinelearning

Buildingmodels(Rpackage::func<on)

•  k-nearestneighbor(FNN::knn)•  fullyconnectedneuralnetwork(h2o::h2o.deeplearning)

•  support-vectormachines(e1071::svm)•  randomforest(party::cforest)

mario.pasquato@

oapd.inaf.it

Tes<ngmodels:crossvalida<on

•  Evaluatethetrainedmodelonunseendata,buts<llusealldatawehave(heredata=mockobserva<ons)

•  Splitdata,usesubsetfortrainingandcomplementfortes<ng

•  Loopoverdata(herefive<mes,5-”fold”CV)

mario.pasquato@

oapd.inaf.it

Predic<ngprobabili<esofhos<nganIMBH

−

−−−−

−

−

−

−

−−−

−−−−−−−−

−−

−−−−−−−−−−−−

−

−−−−−−−−−

−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−

−

−

−−−−−−

−−−

−

−−

−

−−−−

−−

−

−

−−−

−−

−−

−−

−−−−−−−−

−−−−

−

−−−−−

−

−

−

−

−

−

−−−−−

−

−−

−−−−−−−

−

−

−−−−−−

−−

−

−

−−−−−−−−−

−

−−−−−−−

−−−

−−

−

−

−−−−−−

−−−

−−

−

−−−

−

−−−−

−−

−

−

−

−−−

−−−−−−−−

−

−−

−

−−−

−

−−

−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−

−

−−−−

−

−−−−−−−−−−−−−−−−−−−−

−

−−

−

−−−−−−

−

−

−−

−−

−

−−−−−−

−−−−−−

−

−

−−−−

−

−

−−

−

−

−−

−

−

−−

−−

−

−−−

−

−−−−−−

−

−

−

−−−

−

−−−−

−−−−

−−−−

−

−−−

−

−

−−−−

−

−

−−

−−−−−

−

−−−

−

−−−−−−

−

−

−−−

−−

−−

−−−−

−−

−

−

−−−−−−−−−

−−

−−

−−−−−−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−

−

−−

−

−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−

−

−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−−−

−−−−−

−

−

−−−−

−

−−

−

−−

−−

−−−−−−−−

−−

−−−−−

−

−−

−−

−

−−−

−

−

−−−

−

−−

−−

−

−

−−

−

−−−−

−

−

−

−−−

−−−−

−−

−−−

−−−

−

−

−

−−−−−

−

−−

−−−

−

−

−

−

−

−

−−−

−

−−−−−

−−−−

−−−−−−

−−

−−−−−−−−−

−−

−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−

−

−−−−

−

−−

−

−−−

−

−−−−−−−

−

−

−

−−

−−

−−−−−−−

−

−

−

−−

−−−−

−−

−

−

−

−−

−

−

−

−

−−

−

−

−

−−

−

−−−−−−−−−−−−

−−−

−

−−−

−

−−−

−−−

−

−

−−

−

−

−

−−−−−−−

−

−

−

−

−−−

−

−

−

−−

−−

−−−−−−

−−

−−

−

−

−

−−−−

−−−

−−−−

−

−

−−−−

−−

−

−−−−

−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−−−−−−−−

−

−−

−

−

−

−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−

−

−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−

−

−

−−−

−

−−

−

−

−

−

−

−−−

−

−

−−

−−−−−−

−−

−

−

−−−

−−

−

−

−

−

−−

−−−−−−−−

−

−

−−−

−−

−−

−

−−−−−

−−−−

−−−−

−−

−

−−

−

−

−

−−

−−−

−−

−

−

−−−−

−−

−−−−−−

−−

−−

−−

−

−−−−

SVM RF k−nn NN

P(IM

BH)

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−

−

−−

−

−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−

−−

−

−−−

−−−−−−−

−−−

−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−

−−−

−

−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−

−−

−

−

−−−

−−−−

−−−−−

−

−−−−

−−−−−

−

−

−−

−−

−

−−−

−

−−−−

−

−

−−

−

−−

−−−−

−

−

−−

−−−−−−−

−

−−

−−−−−

−−

−

−

−

−−−

−−

−−

−

−

−−−−−−

−

−−−

−−−−−−

−

−−−

−

−−

−

−−−−−−

−−

−

−−

−

−−−−

−

−−−−−

−

−

−

−−−

−−−−−−−−−−−−−−−−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−

−−−−−

−

−

−

−

−

−

−−

−

−−−−−−−

−−−−−−−−−−−

−

−−−

−

−

−−

−

−

−−−

−

−

−

−

−

−

−−−−−

−−

−

−−

−−

−−−−−

−−−−

−−−−−−

−−

−

−

−−−−

−

−

−−

−

−

−−

−−

−−−−

−−−−−−

−

−−

−−

−

−−−

−−−−

−−

−−−−

−−−−−−−−−−−

−−

−

−−−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−

−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−

−−

−−−−−

−

−

−

−

−−

−−

−

−

−

−−

−

−

−−

−−−−−

−−

−−

−−−

−

−

−−

−

−−

−−−−

−

−−

−−−

−−−

−

−−

−

−−

−−−

−

−

−−−

−

−−−

−−

−−−

−−

−−

−

−−−−−−−

−−

−−−−

−

−−−−−−−

−

−−−−−

−−−−

−−

−−−−−−

−

−

−

−−−−−−

−−−

−−

−

−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−

−

−−−−

−−−−−−−

−

−−−−−−−

−

−

−

−−

−−

−

−

−−−−−

−

−

−

−−

−−−−−

−

−−−

−−

−

−

−

−

−−

−

−

−−−

−−

−

−

−−−−

−−−−−

−−−

−

−−−−−−−

−−

−−

−

−−

−

−

−

−

−−−−−−

−

−

−

−

−−−

−

−

−

−−

−−−

−−−−−−−−−−−−−−−−

−−−

−−−−

−

−−

−−

−−

−−

−−−

−

−−−−−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−

−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−

−−−

−−−

−

−

−

−

−

−−−

−

−

−−

−−−−−−−

−−

−

−−−

−−

−

−

−−−−

−−

−−−−−−−−

−

−−

−−

−−

−

−−−−−

−−−−

−−

−−

−−

−

−−

−

−

−

−−

−−−−−

−

−−−−−−−−−

−

−−

−

−−

−−

−−

−

−

−−

−

−

−−

−−

−

−

−

−

−−−

−−

−−−

−

−−

−

−−−−

−

−−−−−

−−−

−

−−−

−

−−−−

−

−−

−

−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−

−

−

−

−−−−−−−−−

−

−−−−−−−−−−

−

−−−−−−−−−−−−−−

−

−

−

−

−−

−

−

−

−−−

−−

−

−−−−

−−

−

−−−

−

−−

−−

−−

−−−−−−−

−

−−

−

−

−

−

−−−−

−

−

−

−

−

−

−−−−−

−

−−

−−−−−−−

−

−

−−−−

−−

−−

−

−

−−−−−−

−−−

−

−−−−−−

−

−−

−

−−

−

−

−−−

−−−

−−−

−−

−

−−

−

−−−−

−

−−

−

−

−

−−−

−−−−−−−−

−

−−

−

−

−

−

−

−−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−

−−

−−−−

−

−−−−−−

−

−−−−−−

−

−−−−−−−−−

−

−−−−−−−−

−

−−

−−−−−−−

−

−

−

−

−−

−

−−−

−

−−

−−−−−−

−

−

−−−−

−−

−−

−

−

−−

−

−

−−

−−

−

−−−

−

−−−−−

−−

−

−

−−−

−

−−

−

−

−−−−

−−−−

−

−−−

−

−

−−−−

−

−

−−

−−−−

−−

−−−

−

−−−−−−

−

−−

−−

−

−

−−

−−−−

−−

−

−

−

−−

−−−−−−−−

−−−

−−−−−

−−

−−

−

−−−−−−−−−−−−−−−

−

−−−−−−−−

−

−−−−−−−−

−

−−

−

−

−

−

−

−−−

−

−−−−−−−−−−−−−−−

−

−−−−−−−−−−−

−

−−−

−

−−−−−−−

−

−−

−

−−−−−

−−−−−−−−

−

−

−

−−−−−−−−−−−−−

−

−

−−

−−

−−

−

−−

−

−

−−

−−

−

−

−

−

−

−−

−

−−−−

−

−−−

−−

−−

−−−

−−−

−−

−−

−−

−

−

−

−−

−−−

−−

−−

−−

−

−−

−

−−

−

−

−−−

−−−

−

−−

−−−

−−

−−

−

−

−−−

−−

−

−−

−−−

−

−

−

−

−

−

−

−−

−

−−−−−

−−−−

−−−

−

−

−−−−

−−−−−−−−

−−

−−−

−

−−−−−−

−

−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−−−−−−−−

−

−−−−−

−

−−−−−−−−

−

−−−−−−−−−

−

−−−−

−−

−

−

−−−

−

−

−

−−−−−

−

−

−

−−

−−

−−−−−−−

−

−

−

−−

−−−−

−−

−−−

−−

−

−

−

−

−−

−

−−

−−

−−

−−−

−

−−−

−−−−

−−−−

−−−

−

−−−

−−−

−

−

−−

−

−

−

−−

−−−−−

−

−

−

−

−

−

−

−

−

−

−−

−−

−−−−−−

−−

−−

−

−−−−−−

−−−

−

−−−

−

−

−−−−

−−

−

−−−

−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−−−−

−

−−−

−

−−

−

−−−−−−−−−−−−−−−−−−−

−

−−−

−

−−−−−−−−−−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−

−

−

−−−

−

−−

−

−

−

−

−

−−−

−

−

−

−−−−−−

−

−−

−

−

−−−

−−

−

−

−

−

−−

−−−−−−−−

−

−

−−

−

−−

−−

−

−−−−−

−

−−

−

−

−−−

−

−−

−−

−

−

−

−−

−−−

−

−−

−−

−−−−

−

−−−

−−−

−−

−−

−−

−

−

−

−−0.

00.

20.

40.

60.

81.

0 no IMBH IMBH

realhosts

nonhostsdifferent

modelsà

prob

abilityofb

eingahost

mario.pasquato@

oapd.inaf.it

Probabili<esàclassifica<on

−

−−−−

−

−

−

−

−−−

−−−−−−−−

−−

−−−−−−−−−−−−

−

−−−−−−−−−

−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−

−

−

−−−−−−

−−−

−

−−

−

−−−−

−−

−

−

−−−

−−

−−

−−

−−−−−−−−

−−−−

−

−−−−−

−

−

−

−

−

−

−−−−−

−

−−

−−−−−−−

−

−

−−−−−−

−−

−

−

−−−−−−−−−

−

−−−−−−−

−−−

−−

−

−

−−−−−−

−−−

−−

−

−−−

−

−−−−

−−

−

−

−

−−−

−−−−−−−−

−

−−

−

−−−

−

−−

−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−

−

−−−−

−

−−−−−−−−−−−−−−−−−−−−

−

−−

−

−−−−−−

−

−

−−

−−

−

−−−−−−

−−−−−−

−

−

−−−−

−

−

−−

−

−

−−

−

−

−−

−−

−

−−−

−

−−−−−−

−

−

−

−−−

−

−−−−

−−−−

−−−−

−

−−−

−

−

−−−−

−

−

−−

−−−−−

−

−−−

−

−−−−−−

−

−

−−−

−−

−−

−−−−

−−

−

−

−−−−−−−−−

−−

−−

−−−−−−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−

−

−−

−

−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−

−

−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−−−

−−−−−

−

−

−−−−

−

−−

−

−−

−−

−−−−−−−−

−−

−−−−−

−

−−

−−

−

−−−

−

−

−−−

−

−−

−−

−

−

−−

−

−−−−

−

−

−

−−−

−−−−

−−

−−−

−−−

−

−

−

−−−−−

−

−−

−−−

−

−

−

−

−

−

−−−

−

−−−−−

−−−−

−−−−−−

−−

−−−−−−−−−

−−

−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−

−

−−−−

−

−−

−

−−−

−

−−−−−−−

−

−

−

−−

−−

−−−−−−−

−

−

−

−−

−−−−

−−

−

−

−

−−

−

−

−

−

−−

−

−

−

−−

−

−−−−−−−−−−−−

−−−

−

−−−

−

−−−

−−−

−

−

−−

−

−

−

−−−−−−−

−

−

−

−

−−−

−

−

−

−−

−−

−−−−−−

−−

−−

−

−

−

−−−−

−−−

−−−−

−

−

−−−−

−−

−

−−−−

−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−−−−−−−−

−

−−

−

−

−

−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−

−

−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−

−

−

−−−

−

−−

−

−

−

−

−

−−−

−

−

−−

−−−−−−

−−

−

−

−−−

−−

−

−

−

−

−−

−−−−−−−−

−

−

−−−

−−

−−

−

−−−−−

−−−−

−−−−

−−

−

−−

−

−

−

−−

−−−

−−

−

−

−−−−

−−

−−−−−−

−−

−−

−−

−

−−−−

SVM RF k−nn NN

P(IM

BH)

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−

−

−−

−

−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−

−−

−

−−−

−−−−−−−

−−−

−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−

−−−

−

−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−

−−

−

−

−−−

−−−−

−−−−−

−

−−−−

−−−−−

−

−

−−

−−

−

−−−

−

−−−−

−

−

−−

−

−−

−−−−

−

−

−−

−−−−−−−

−

−−

−−−−−

−−

−

−

−

−−−

−−

−−

−

−

−−−−−−

−

−−−

−−−−−−

−

−−−

−

−−

−

−−−−−−

−−

−

−−

−

−−−−

−

−−−−−

−

−

−

−−−

−−−−−−−−−−−−−−−−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−

−−−−−

−

−

−

−

−

−

−−

−

−−−−−−−

−−−−−−−−−−−

−

−−−

−

−

−−

−

−

−−−

−

−

−

−

−

−

−−−−−

−−

−

−−

−−

−−−−−

−−−−

−−−−−−

−−

−

−

−−−−

−

−

−−

−

−

−−

−−

−−−−

−−−−−−

−

−−

−−

−

−−−

−−−−

−−

−−−−

−−−−−−−−−−−

−−

−

−−−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−

−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−

−−

−−−−−

−

−

−

−

−−

−−

−

−

−

−−

−

−

−−

−−−−−

−−

−−

−−−

−

−

−−

−

−−

−−−−

−

−−

−−−

−−−

−

−−

−

−−

−−−

−

−

−−−

−

−−−

−−

−−−

−−

−−

−

−−−−−−−

−−

−−−−

−

−−−−−−−

−

−−−−−

−−−−

−−

−−−−−−

−

−

−

−−−−−−

−−−

−−

−

−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−

−

−−−−

−−−−−−−

−

−−−−−−−

−

−

−

−−

−−

−

−

−−−−−

−

−

−

−−

−−−−−

−

−−−

−−

−

−

−

−

−−

−

−

−−−

−−

−

−

−−−−

−−−−−

−−−

−

−−−−−−−

−−

−−

−

−−

−

−

−

−

−−−−−−

−

−

−

−

−−−

−

−

−

−−

−−−

−−−−−−−−−−−−−−−−

−−−

−−−−

−

−−

−−

−−

−−

−−−

−

−−−−−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−

−

−−

−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−

−−−

−−−

−

−

−

−

−

−−−

−

−

−−

−−−−−−−

−−

−

−−−

−−

−

−

−−−−

−−

−−−−−−−−

−

−−

−−

−−

−

−−−−−

−−−−

−−

−−

−−

−

−−

−

−

−

−−

−−−−−

−

−−−−−−−−−

−

−−

−

−−

−−

−−

−

−

−−

−

−

−−

−−

−

−

−

−

−−−

−−

−−−

−

−−

−

−−−−

−

−−−−−

−−−

−

−−−

−

−−−−

−

−−

−

−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−

−

−

−

−−−−−−−−−

−

−−−−−−−−−−

−

−−−−−−−−−−−−−−

−

−

−

−

−−

−

−

−

−−−

−−

−

−−−−

−−

−

−−−

−

−−

−−

−−

−−−−−−−

−

−−

−

−

−

−

−−−−

−

−

−

−

−

−

−−−−−

−

−−

−−−−−−−

−

−

−−−−

−−

−−

−

−

−−−−−−

−−−

−

−−−−−−

−

−−

−

−−

−

−

−−−

−−−

−−−

−−

−

−−

−

−−−−

−

−−

−

−

−

−−−

−−−−−−−−

−

−−

−

−

−

−

−

−−

−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−−−−−−−−−−−

−−

−−−−

−

−−−−−−

−

−−−−−−

−

−−−−−−−−−

−

−−−−−−−−

−

−−

−−−−−−−

−

−

−

−

−−

−

−−−

−

−−

−−−−−−

−

−

−−−−

−−

−−

−

−

−−

−

−

−−

−−

−

−−−

−

−−−−−

−−

−

−

−−−

−

−−

−

−

−−−−

−−−−

−

−−−

−

−

−−−−

−

−

−−

−−−−

−−

−−−

−

−−−−−−

−

−−

−−

−

−

−−

−−−−

−−

−

−

−

−−

−−−−−−−−

−−−

−−−−−

−−

−−

−

−−−−−−−−−−−−−−−

−

−−−−−−−−

−

−−−−−−−−

−

−−

−

−

−

−

−

−−−

−

−−−−−−−−−−−−−−−

−

−−−−−−−−−−−

−

−−−

−

−−−−−−−

−

−−

−

−−−−−

−−−−−−−−

−

−

−

−−−−−−−−−−−−−

−

−

−−

−−

−−

−

−−

−

−

−−

−−

−

−

−

−

−

−−

−

−−−−

−

−−−

−−

−−

−−−

−−−

−−

−−

−−

−

−

−

−−

−−−

−−

−−

−−

−

−−

−

−−

−

−

−−−

−−−

−

−−

−−−

−−

−−

−

−

−−−

−−

−

−−

−−−

−

−

−

−

−

−

−

−−

−

−−−−−

−−−−

−−−

−

−

−−−−

−−−−−−−−

−−

−−−

−

−−−−−−

−

−−−−−−−−−−−−−−−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−−−−−−−−

−

−−−−−

−

−−−−−−−−

−

−−−−−−−−−

−

−−−−

−−

−

−

−−−

−

−

−

−−−−−

−

−

−

−−

−−

−−−−−−−

−

−

−

−−

−−−−

−−

−−−

−−

−

−

−

−

−−

−

−−

−−

−−

−−−

−

−−−

−−−−

−−−−

−−−

−

−−−

−−−

−

−

−−

−

−

−

−−

−−−−−

−

−

−

−

−

−

−

−

−

−

−−

−−

−−−−−−

−−

−−

−

−−−−−−

−−−

−

−−−

−

−

−−−−

−−

−

−−−

−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−

−

−

−−−−

−

−−−

−

−−

−

−−−−−−−−−−−−−−−−−−−

−

−−−

−

−−−−−−−−−−−−

−

−−−−−−−−−−−−−

−

−−−−−−−−

−

−

−

−

−−−

−

−−

−

−

−

−

−

−−−

−

−

−

−−−−−−

−

−−

−

−

−−−

−−

−

−

−

−

−−

−−−−−−−−

−

−

−−

−

−−

−−

−

−−−−−

−

−−

−

−

−−−

−

−−

−−

−

−

−

−−

−−−

−

−−

−−

−−−−

−

−−−

−−−

−−

−−

−−

−

−

−

−−0.

00.

20.

40.

60.

81.

0 no IMBH IMBH

realhosts

nonhostsdifferent

modelsà

prob

abilityofb

eingahost

cutoff

Falseposi<ves

Trueposi<ves

movingthecutoffup:lessfalseposi<veslesstrueposi<ves

movingthecutoffdown:morefalseposi<vesmoretrueposi<ves

mario.pasquato@

oapd.inaf.it

Measuringpreformance:ROCcurve

ReceiverOpera<ngCharacteris<c(ROC)curve:frac<onoftrueposi<vesVSfrac<onoffalseposi<ves

mario.pasquato@

oapd.inaf.it

ROCcurvesforourclassifieres•  Howmanytrueposi<vesyougetbyaccep<ngagivenrateoffalseposi<ves

•  Randomclassifier:diagonalline

•  Perfectclassifier:stepto1immediately

•  AreaUndertheCurve(AUC)=overallperformance0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

FPR

TPR

−−−−

SVMRFKNNNN

FPR

TPR

Falseposi<ves/Trueposi<ves•  Falseposi<verate:frac<onofnonhoststhatiswronglyclassifiedashost

•  Trueposi<verate:frac<onofhoststhatiscorrectlyclassifiedashost

Realhosts Realnon-hosts

FalsePosi?veRate

TruePosi?veRate

Claimedhosts

Correctclaims

Correctclaims/total

100 100 10% 80% 90 80 89%

100 100 50% 99% 149 99 66%

100 100 5% 50% 55 50 91%

200 0 10% 80% 160 160 100%

0 200 10% 80% 20 0 0%

Unknown Unknown Choosethis GetthisfromROC

ROCcurvesforourclassifieres

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

FPR

TPR

−−−−

SVMRFKNNNN

Model AUC

Neuralnet 0.94

Randomforest 0.94

Supportvectormachine

0.85

k-nearestneighbor

0.76

Perfomanceofthetwobestclassifiers(NNandRF)isverysimilarNoadhoctuningofclassifierparametersPerformancemeasuredonsnapshotsnotseenintraining

Interpreta<on

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

FPR

TPR

−−−−

SVMRFKNNNN

Ifwe,say,accept10%falseposi<vesWiththerandomforestortheneuralnetweget>80%sensi<vityi.e.80%oftheactualIMBHhostsareactuallyfound

Results•  Fourclassifiers:thebesttwo(neuralnet,randomforest)haveverysimilarROCcurves

•  At5%FPRtheyyield70%TPR•  Scenarios–  IMBHprevalence50%–  IMBHprevalence10%

Realhosts Realnon-

hostsFalsePosi?veRate

TruePosi?veRate

Claimedhosts

Correctclaims

Correctclaims/total

100 100 5% 70% 75 70 93%

20 180 5% 70% 23 14 61%

TosummarizeBasedonlyonsurfacedensityprofiles,withintheMOCCASurveydatabase,ourclassifierswithoutfinetuning,onsnapshotsnotseenintraining,catch70%ofIMBHhostswitha5%falseposi<verate

Needmoremachinelearninginyourproject?dropmealineàmario.pasquato@oapd.inaf.it

Backupslides

ApproachI:deepConvolu<onalNeuralNet(CNN)directlyonmockimages

IMBH

Theselayerstriggersonnoses,beaks,ears,stars...

Theselayersdotheunderstanding

Convolu<onlayers Fullyconnectedlayers

Features

AstronomicalTuringtest

IMBH

Rawdata

Mockdata

Indis<nguishable(?)

Mockdataisindis<nguishablefromrealdataifwecannotbuildanyclassifierthattellsitapart(includingahumanclassifier)Itisweaklyindis<nguishablefromrealdataifwecannotretrainthefullyconnectedlayersoftheIMBH/no-IMBHclassifiertotellitapartfromrealdata

ProsandconsofCNNapproach

•  Computa<onallyintensive•  ButitparallelizesverywellonGPUs

•  Blackbox(rela<velyso...)•  Buttherearewaystolookintotheblackbox

•  Generalizeseasilytoanyimage Velocitymapofa16k-par<cle

NBODY6simula<on,courtesyDr.Mastrobuono-BaLs<Wecanclassifythistoo!