Using Historic Patterns in BothCache Prefetching and Cache

ReplacementJim Gast & Laura Spencer

CS703, Spring 2000

Motivation

• Disk I/O is slow and not getting faster– 10 milliseconds for an unexpected open

• Memory is cheap and getting cheaper– Cache more, read ahead, stay ahead

• LRU is a lousy algorithm• Program modification is hard• History is (comparatively) easy

New metadata per file

• Uses (actual opens from actual programs)• Cache-behind successes• Prefetch successes, Prefetch tries• Prefetch failures

Store history in triads

• Historically, file E is typically followed by– file F after interval IE->F with Bias BE->F

• And E, F is often followed by– file G after interval IEF->G with Bias BEF->G

• Bias is the conditional probability– P(G|E) = P(F|E)*P(G|E,F) = BE->F * BEF->G

Simplifying Assumptions

• Every file is just one page• Unique file name implies unique disk

address• Ignore writes• Any file open takes 10 milliseconds

– No disk contention

Prediction Algorithms

• NONE• Kroger & Long

– 3rd order Trie– We added expected request time

• Ours– 5th order Trie simulated using recursion on 3rd– Including expected request time

Typical Trie - Frequencies

FileList.o

Welem.o Path.o

Welem.oVolID.o

31

229

922

When FileList.o is opened, the bias to prefetch VolID.o is 22 / 31

Typical Trie - Intervals

FileList.o

Welem.o Path.o

Welem.oVolID.o

0.012 0.011

0.0070.009

Predict that VolID.o will be needed in 0.021 seconds

Using 3rd order Trie as 5th order

a.o

b.o

c.o

b.o

c.o

d.o

c.o

d.o

e.o

15

30

15

60

30

20

100

20

20

If a.o is opened, the bias to predict c.o is 50%, e.o is 33%

Bias to predict a future open

n

i ii

iiin GGCount

GGGCountGGCountGGGCountGGGP

4 12

12

21

32121 ),(

),,(),(

),,(),|(

a.o

b.o

c.o

b.o

c.o

d.o

c.o

d.o

e.o

15

30

15

60

30

20

100

20

20

Replacement Algorithms

• LRU– One long queue– Re-queue on fresh open

• ENU– Speculated Queue for files not yet actually

opened– Popular Queue for files worth keeping– Idle Queue for files not yet evicted

Compare 3 items to find victim

Incoming Speculated Popular Idle

Least Old

NewMost

OverdueChoose Most Willing

of the three tails

Each time a file is hinted or opened, it is re-queued

Unexpected Open

OnDisk

Incoming

Idle

Popularor

Arrived

Open(Miss) Evicted

Open(Hit)

Open (NearMiss)

Hinted Open

OnDisk

Incoming

Speculated

Idle

Popularor

Arrived

Hint(PrefetchTry)

Evicted(Prefetch Failure)

Open(Prefetch Hit)

Evicted

Open(PrefetchNearMiss)

Trace-driven Simulation

• Seer traces from UCLA• Nine LINUX servers

– Software development environment• Every file open for several weeks

– We only used one week from one server

Simulation

• 21Cache sizes– 10 pages = much smaller than workingset– 1000 pages = much larger than workingset

• Various levels of aggression– 90% bias (conservative)– 10% bias (aggressive)

• Compared LRU, K&L, Pop, ENU

Measurements

• Total Time Waiting– And Time Fully Busy

• Hits & prefetch Hits• Near Misses & prefetch Near Misses• Misses• Prefetch Failures• Memory Pressure

Total Time Waiting

0.0

100.0

200.0

300.0

400.0

500.0

600.0

10 100 1000

Cache Size

Seco

nds LRU

K&L

ENU

Constant: Bias Threshold 90

All Opens - LRU

0%10%20%30%40%50%60%70%80%90%

100%

10 30 50 100 200 500 1000

Cache Size

MissNear MissHit

All Opens using K&L Algorithm

0%

20%

40%

60%

80%

100%

10 30 50 100 200 500 1000

Cache Size

MissNearMissPF NearMissPF HitHit

All Opens by Algorithm

0%10%20%30%40%50%60%70%80%90%100%

LRU

POP

K&L90

K&L10

ENU90

ENU50

ENU10

At cache size 50

cMiss

cNM

pfNM

pfHits

cHits

All Opens ENU

0%10%20%30%40%50%60%70%80%90%

100%

10 30 50 100 200 500 1000

Cache Size

MissNearMissPF NearMissPF HitHit

Prefetch Mistakes by Bias Threshold

0500

10001500200025003000350040004500

30 100 1000

Cache Size

Mis

take

s

Bias10Bias30Bias50Bias70Bias90

Memory Pressure

1

10

100

1000

10000

100000

100030005000700090001100013000150001700019000210002300025000

Opens

Pres

sure

(avg

evi

ctio

n ag

e)

LRU ENU

)(1 earrivalTimii VictimeCurrentTimeMemPressureMemPressur

IdlePopular

Speculated

Time

CurrentTime

Will

ingn

ess

Idle ENUPopular ENU

ii

ii PopularityeMemPressur

PopularityENUeCurrentTimsWillingnes

Light Memory Pressure

IdlePopularSpeculated

Time

CurrentTime

Will

ingn

ess

ii

ii PopularityeMemPressur

PopularityENUeCurrentTimsWillingnes

Prefetch Failure

IdlePopular

Speculated

Time

CurrentTime

Will

ingn

ess

sss

ss PopularityeMemPressur

FailRateBiasENUeCurrentTimsWillingnes

Willingness for Speculated Files:

Idle Queue close to empty, w/hint

Idle, ENU soon Popular

Speculated

Time

CurrentTime

Will

ingn

ess

Future Work

• Block Level– Intelligent disks

• Model dirty pages• Better Simulation & Traces

– Interleaved requests, limited disks• Real Kernel Mods

– Study long strings without trace times anomoly• Refinements to measurements

– Willingness, Popularity