Upload
takeshi-hasegawa
View
70
Download
0
Embed Size (px)
Citation preview
ThisisatranslatedversionofIkaLogtechpresenta6on,originallyinJapaneselanguage.
Transla6onworkiss6llinprogress!
2
What is Splatoon?
A unique third-person shooter – Paint the battlefield with your ink,
and claim turf from the enemy team
– Switch between an Inkling and a Squid for different approaches
– Several gamemodes in addition to turf war, like KOTH and payload
Simple rules, but deep – Various strategies and tactics – Unique weapon classes
Shooter, Charger, Roller, Splatling, and Slosher
Strategy is the key to winning
STAGERECON
• There’smorethanonewayaroundeachmap
• Knowwheretheenemyiscomingfromandhaveaplan
VARIETYOFWEAPONS
• CertainweaponsarebeCersuitedtosomemapsandmodesoverothers
COLLECTPLAY
IkaLog: Data Collector for Splatoon
– Play Splatoon as you would normally
– Your gameplay footage is analyzed
– The data can then be sent to log files, and/or other tools, like stat.ink
GameConsole
Logfiles
IkaLog
stat.ink,Speechapplica6on,etc.
Forwardtodesiredtools
PROCESS
Supported Integrations
VideoStreaming/Recording
AmaRecTV
OnlineDatabase
CSV/JSON,Screenshots
SNSandChat
IkaLog
stat.ink
The online database provided by @fetus_hina – Submit your battle results
and statistics with IkaLog
– Review your past gameplay easily using the website
Review your gameplay on stat.ink
SCOREBOARD
Reviewthescoreboard
later
Filteringallowsforcarefulanalysisofyourpast
gameplay
TIMELINE
Thegraphshowswhathappenedinthegame
Displayskills,deaths,specialweapons,rankedmode
counts/distancesandevents
GLOBALSTATS
Sta6s6csdatafrom
allstat.inkusersisavailable
Seethetrendsamongusers
Timeline Visualization
TheperiodwhenyougotsplaCedand6mespentinac6ve
Alive/Deadstatusofall8players
Team’sspecialweapons,Kills,Deaths
YourTurfScore
Timeline for Ranked Battles
StatusoftheSplatZone
SplatZonecounttheotherteam
earned
SplatZonecountyourteamearned
Thegame-changingmoment
# of IkaLog + stat.ink users
SourcehCps://stat.ink/en6re/user
Avg:200+users,4500+matchesaday
Peak:370usersadayProcessed15,000matches
https://stat.ink/
.96GalDecousersdecreasedaUerthemomentofgameupdate
LongBlasterCustomisgeWngmorepopularsinceSummer
Sourcefootage MaskImage AddedImage
+==
Addi6on(OR)ofsourceimageandcorrectmaskresultsinwhiteimage
Addi6on(OR)ofsourceimageandwrongmaskresultsinnon-whiteimage
• Therearetwofonttypesinthegame.Weshouldcoversinglenumberfontface
– Thefontisknown,soweshouldbeabletoiden6fythem.
• DecidedtohaveMLbasedimageclassifica6on,ratherthanuseofexis6ngOCRlibrary
1)Crop the number from the footage, apply some image filters
2)Generate vertical & horizontal histogram to guess each character’s
position
3)Resize the characters to identical size, and make the image binary
4)Classify the image using KNN
• Basciallysameideawithrecogni6onofnumbers
• 30+%ofaccuracyforsingleclassifica6on.Earnaccuracybyinves6ga6ngmanyframes
– IkaLoginves6gatesapprox.10framesperasec
– ThisexampleshowsIkaLoganalyzed49frames,andfound“96gal_deco”ismostlikely(18frames,36%)->correct.
votes={'supershot':6, 'carbon_deco':1, 'bucketslosher':1, 'octoshooter_replica':1,'splashshield':1, 'sshooter_collabo':5, 'hotblaster':2, 'pablo':1, 'nzap89':6,'sharp_neo':3, 'hotblaster_custom':2, '96gal_deco':18, '52gal':1, 'hokusai':1}
Input LaplacianFilter
Grayscale Classifyusingk-NearestNeighbor
Thanks@itooon
sschooter_collabo
Featuremap
Pros
– IgnoresColor
– Ituses“theshape”formatching
Cons
– Doesn’tworkifinputhasdifferentresolu6on
– Cannotclassifysimilarshapes
Originalclass(longblaster)
Variantversion(longblaster_custom)
Onemorevariant!(Apr2016)
(longblaster_necro)
Conjunc6ononfeaturesbetweenthevariants.Neededtoupdate(orfind)thefeatureextrac6on
longblaster_necro
longblaster_custom
longblaster
IkaLoginternalfeaturesvaluesPloCedin2DusingPCA
Conjunc6ononfeaturesbetweenthevariants.Neededtoupdate(orfind)thefeatureextrac6on
Anothergameresultsceneswith“GearAbilityicons”Theapproachresultsinlowaccuracy…・Sameicon,differentsizeinsingleframe・Usersusesdifferentcaptureresolu6on→Morerobustimageclassifierneeded
• Amountofthedataisalwaysjus6ceformachinelearning
• Usethecrowdtocollectthedata50games/dayperperson
5000gamescanbepossibleby100people
– Field-generateddatahasthenoisesandoutliers• ModelofHDMIcaptureunits,HDMIre-transmiCers,andsoUwareconfigura6on
• It’seasiertocollec6ngtheoutliersfromfield,thentrain,andtestclassifiers
• AccuracyofMachinelearning/DeepLearningisaffectedbypre-processing
• Templatematching-basedalgorithmautoma6callyfixesbroken
inputimages
Wrongaspectra6o offsetedreference
Imagetoclassify
Pros
– BeCerclassifica6onaccuracy
Cons
– Toolargedatasize(20MB->100MB~400MB)
Hardtodistributetheweights
– BreaksWindowsversionofIkaLog
• 90%+ofusersarerunningIkaLogonWindowsPlazorm
• MostofdeeplearningframeworkbreaksPy2EXE(PythonscripttoWindowsexecutableconverter)
– Morecompu6ngpowerneeded
– Input: RGBorHSVcolorvalue(47*45*3=6,345units)
– Output:possibilityofeachclass(91units,applysoUmax)
– Connec6on:alwaysusefullyconnec6on
– Letcomputersthefeatureautoma6cally
• Inthisusecase,deeplearningwillfindproperweightsautoma6cally
• Itwillignorebackgroundcolorsautoma6cally
– TargetPerformance
• Calc.6me:lessthan350msforeachmul6-classclassifica6on(91classes)<3secondsperaframe
• 99.99+%accuracyagainststat.inkposted(Field)data
0
1
2
3
..
..
n
0
1
2
3
…
89
90
InputLayer OutputLayerHiddenLayer
52gal
52gal_deco
96gal
96gal_deco
…
sschooter_wasabi
wakaba
• TestedonAzureML,beforeimplemen6ngthecode
– Trainedclassifiersusingsmalldataset
– Input:Pixelvalues(w/PCA)
– HiddenLayer:1layer、150units
– Output:Probably91(AzureMLmanages)
• Gotgoodresultsanddecidedtomoveforward
– 98+%accuracy(withsmalldataset)
– Enoughrobustnessforanyimages(e.g.los-res)
• UsedChainer(DeepLearningFramework)totrainthemodel
– Supervisedtrainingusingstat.inkdata(0.9Mimages)
– CanbenefitGPUperformance
• TrainedmodelsareconvertedtoFloat16
– 32bitfloat->16bitfloat
– 50%smallerfilesize
CPU GPU #ofjobs
Throughput(images/s)
GPUTime/epoch(s)
RelaWveperf.
(virtual)
Corei7-4790STyp.3.2GHz
NA 1 3,928 234.5s 1X
Corei7-4790STyp.3.2GHz
GeForceGTX760
1 10,371 39.2s 5.9X
Corei7-4790STyp.3.2GHz
GeForceGTX760
4 (Cl)15,988 25.8s(103/4)
9.0XCorei7-4790STyp.3.2GHz
GeForceGTX1080
4 (Cl)30,320 6.25s(25.0s/4) 37.5X
CPU GPU #ofjobs
Throughput(images/s)
GPUTime/epoch(s)
RelaWveperf.
(virtual)
Corei7-4790STyp.3.2GHz
NA 1 3,928 234.5s 1X
Corei7-4790STyp.3.2GHz
GeForceGTX760
1 10,371 39.2s 5.9X
Corei7-4790STyp.3.2GHz
GeForceGTX760
4 (Cl)15,988 25.8s(103/4)
9.0XCorei7-4790STyp.3.2GHz
GeForceGTX1080
4 (Cl)30,320 6.25s(25.0s/4) 37.5X
2XE5-2630Lv3Typ.1.80GHz
TeslaP100 4 (Cl)65,811 2.5s(10.3/4)
93.8X
SpecialThanksto
TypicalLinuxBox
Pre-training(Parametertuning)
– Testedvariouslayerconfigura6on(#ofunits),balancedmodelfilesizeanditsaccuracy
– 4concurrentjobs
– Started03:08am〜6:28pm (15h20m)
Training
– 550epochs/24hrs IncludesdiskIO,crossvalida6ontest6me
– Adoptedthe617thepochforthefirstNN-basedclassifier.
TookApprox.48GPUHoursusing21GBdataset
SpecialThanksto
KNN(Original) ComplexNN ThenewNN
Accuracy Low,oncertainusers 99.99+%
99.99+%
Datasize 20MB 400MB(AlexNet)100MB(GoogleNet)
14MB(Float32)7MB(Float16)
Classifica6on6me@IvyBridge2GHz
(veryfast) ~300ms ~20ms
• Improvedaccuracyofweaponclassifica6on
• Fasterthancomplexneuralnetworkmodels
• Smallerdatasizemakesdistribu6oneasy
SpecialThanksto
• IkaLogincludesownneuralnetworkimplementa6on
– DeepLearningFrameworksbreakWindowsversion
– Re-implementedpropaga6onfunc6on
• Tomakethecodesimple,themodeliskeptsimple
• Compa6blewithLinearFunc6on,andReLUinChainerhCps://github.com/hasegaw/IkaLog/commit/3238b67749334a3c4254aa6f25c005f83e210895
– [email protected]@PYNQ-Z1FPGAboard(Cortex-A9650MHz)
• Applyneturalnetwork-basedapproachtootherclassifiers
– Gearabilityimageclassifier
– Rainmakertouchdowndetec6on
– UseRNNfor“TideofthebaCle”evalua6onfunc6on
• Makethedatasetsmaller
– Thedataset(21GB)isobviouslyredundant
– UsePCA-basedanomalydetec6ontogeneratemoresmallerdata?
• Severalmarchine-learningapproachappliedinIkaLog
– KNearestNeighborforreal-6meclassifica6on
– Neuralnetworkformoreaccurateclassifica6on
• Autofeatureextrac6on,trainingfrom0.9Msamples
• IkaLogandDeepLearning
– MostofDeepLearningframeworkbreaksPy2EXE
– Easytore-implementofpropaga6onifthenetworkissimple
– Canu6lizeexis6ngDNNframeworksandGPUsfortrainingprocess