CacheModelingandOptimizationusingMiniatureSimulations

USENIXAnnualTechnicalConference(ATC’17)July13,2017

CarlWaldspurger CachePhysics,Inc.Trausti Saemundsson CachePhysics,Inc.IrfanAhmad CachePhysics,Inc.NohhyunPark DatosIO,Inc.

Motivation• Cachingimportant,ubiquitous• Optimizevaluablecacheresources– Improveperformance,QoS– Sizing,partitioning,tuning,cliffremoval,…

• Problem:needaccurate,efficientmodels– Complexpolicies,non-linear,workload-dependent– Nogeneral,lightweight,onlineapproach

CachePhysics, Inc. USENIXATC’17 2

CacheModeling

�

���

���

���

���

���

���

���

���

� �� �� �� �� �� �� �� ��

���������

����� ���� ����

• Cacheutilitycurves– Performanceasf(size,…)– Missratiocurve(MRC)– Latencycurve

• Observations– Non-linear,cliffs– Non-monotonicbumps

CachePhysics, Inc. USENIXATC’17 3

CounterStacks[OSDI’14]SHARDS[FAST’15]AET[ATC’16]

MRCConstructionMethods

Exact ApproximateStack AlgorithmsLRU,LFU,…

Any AlgorithmARC,LIRS, 2Q,FIFO,…

CachePhysics, Inc. USENIXATC’17 4

Mattsonalgorithmallsizesatonce

separatesimulationforeachcachesize

CounterStacks[OSDI’14]SHARDS[FAST’15]AET[ATC’16]

miniaturesimulation[ATC’17]

MiniatureSimulation• Simulatelargecacheusingtinyone• Scaledownreferencestream,cachesize– Randomsamplingbasedonhash(key)– Assumesstatisticalself-similarity

• Rununmodifiedalgorithm– LRU,LIRS,ARC,2Q,FIFO,OPT,…– Trackusualstats

CachePhysics, Inc. USENIXATC’17 5

ScalingDown

CachePhysics, Inc. USENIXATC'17 6

refs cache

≈2×

halfkeyspace

halfsizehashkeys(colors)

ScalingDown

CachePhysics, Inc. USENIXATC'17 7

refs cache

≈8×

ScalingDown

CachePhysics, Inc. USENIXATC'17 8

refs cache

≈32×

ScalingDown

CachePhysics, Inc. USENIXATC'17 9

refs cache

≈128×

FlexibleScaling

CachePhysics, Inc. USENIXATC'17 10

Smminisize

samplingrateR

Seemulatedsize

Sm =R × Se

• Time/spacetradeoff– Fixedsamplingrate R– FixedminisizeSm

• Example:Se =1M– R =0.005⇒ Sm=5000– Sm=1000⇒ R =0.001

ExampleMini-SimMRCs

CachePhysics, Inc. USENIXATC’17 11

Mini-SimAccuracy

CachePhysics, Inc. USENIXATC'17 12

• 137real-worldtraces– Storageblocktraces– CloudPhysics,MSR,FIU– 100cachesizespertrace

• MeanAbsoluteError– |exact– approx|– Averageoverallsizes

Mini-SimEfficiency• Variablecosts

– BothspaceandtimescaleddownbyR– R=0.001⇒ simulation1000× smaller,1000× faster

• Fixedcosts– Hashingoverheadforsampling– Footprintforcode,libraries,etc.

• Netimprovement– R=0.001⇒ ~200× smaller,~10× faster– Closerto1000× ifexistingkeyhashormultiplesims

CachePhysics, Inc. USENIXATC'17 13

Mini-SimCacheTuning• Dynamicmulti-modeloptimization– Simulatecandidateconfigurationsonline– Periodicallyapplybesttoactualcache

• Parameteradaptationexperiments– LIRSS stacksize,5mini-simswithf=1.1— 3– 2QA1out size,8mini-simswithKout =50%— 300%– R =0.005,epoch=1Mrefs

CachePhysics, Inc. USENIXATC’17 14

LIRSAdaptationExamples

CachePhysics, Inc. USENIXATC’17 15

2QAdaptationExamples

CachePhysics, Inc. USENIXATC’17 16

TalusCliffRemoval

CachePhysics, Inc. USENIXATC'17 17

• Talus[HPCA’15]– NeedsMRCasinput– Interpolatesconvexhull

• Shadowpartitions𝛼,𝛽– Steerdifferentfractionsofrefstoeach

– Emulatecachesizesonconvexhullviahashing

𝛼

𝛽

TalusforNon-LRUPolicies?• NeedefficientonlineMRCs• Supportdynamicchanges?–WorkloadandMRCevolveovertime– Resizepartitions,lazyvs.eager?–Migratecacheentriesin“wrong”partition?

• Notclearhowtomerge/migratestateCachePhysics, Inc. USENIXATC’17 18

SLIDE:TransparentCliffRemoval• ShardedListwithInternalDifferentialEviction– Singleunifiedcache,nohardpartitions– Deferpartitioningdecisionsuntileviction– Avoidsresizing,migration,complexityissues

• NewSLIDElistabstraction– NochangestoARC,LIRS,2Q,LRUcode– ReplacesinternalLRU/FIFObuildingblocks

CachePhysics, Inc. USENIXATC’17 19

SLIDEList• Augmentconventionallist– Per-itemhashvalue– Hashthresholddeterminescurrent“partition”

• Itemstotallyordered,nohardpartitions• Evictfromtailofover-quotapartition

CachePhysics, Inc. USENIXATC’17 20

SLIDEExperiments• ConstructMRConline– 7mini-sims{⅛,¼,½,1,2,4,8}× cachesize– R=0.005,smoothedmissratios

• UpdateSLIDEsettingsperiodically– Discreteconvexhulleveryepoch(1Mrefs)– Setnew“partition”targetsforSLIDElists

CachePhysics, Inc. USENIXATC’17 21

SLIDE:CliffReduction

CachePhysics, Inc. USENIXATC’17 22

69%ofpotentialgain

48% 38%

SLIDE:LittleImpactwithoutCliffs

CachePhysics, Inc. USENIXATC’17 23

Conclusions• Mini-simextremelyeffective– Robust,generalmethod(ARC,LIRS,2Q,LRU,OPT,…)– Averageerror<0.01with0.1%sampling

• Canoptimizeworkloads/policiesautomatically– Dynamicparametertuning– SLIDEtransparentcliffremoval

CachePhysics, Inc. USENIXATC’17 24