Embed Size (px)
Citation preview
CS 401
Greedy Algorithms / Optimal Caching / Shortest Path
Xiaorui Sun
1
2
Administrativia Stuffs
Homework 2 is out!
• Due Wednesday October 9
Midterm exam: October 18 in class
• 1pm-1:50pm, SES 138
• You may use a sheet with notes on both sides, but not
textbook and any other paper materials
• You may use a calculator, but not any device with
transmitting functions, especially ones that can access
the wireless or the Internet.
Optimal Caching
Optimal Offline CachingCaching
• Cache with capacity to store ! items.• Sequence of " item requests #$, #&,⋯ , #".• Cache hit: item already in cache when requested.• Cache miss: item not already in cache when requested:
must bring requested item into cache, and evict some existing item, if full.
Goal• Eviction schedule that minimizes number
of evictions.
Farthest-in-future• Evict item in the cache that is not requested until farthest in the
future.
Theorem• [Bellady, 1960s] FIF is an optimal eviction schedule.
Reduced Eviction SchedulesDefinition• A reduced schedule is a schedule that only inserts an item into the cache
in a step in which that item is requested.ClaimGiven any unreduced schedule !, can transform it into a reduced schedule !′with no more cache misses.
a x
an unreduced schedule
ca d ca d ba c ba x ba c ba b ca b c
acdabcaa
a b
a reduced schedule
ca b ca d ca d ca d ba c ba c ba c b
acdabcaa
a b ca a b ca
6
Farthest-In-Future: Analysis
6
Theorem• FIF is optimal eviction algorithm.
Proof. (by induction on number or requests !)
Let " be reduced schedule that satisfies invariant through # requests. We produce "′ that satisfies invariant after # + & requests.
• Consider (# + &)st request ) = )#+&.• Since " and ",-, have agreed up until now, they have the same cache
contents before request # + &.Case 1: () is already in the cache).
"′ = " satisfies invariant. (used " is reduced here)Case 2: () is not in the cache and " and ",-, evict the same element).
"′ = " satisfies invariant.
Invariant: There exists an optimal reduced schedule " that makes the same eviction schedule as ",-, through the first # + & requests.
Farthest-In-Future: Analysis
7
Proof. (continued)Case 3: (! is not in the cache; "#$# evicts %; " evicts & ≠ %).
• begin construction of "′ from " by evicting % instead of &
• now "′ agrees with "#$# on first ) + + requests; we show that having element & in cache is no worse than having element %
– Continue building "’ to be the same as " until forced to be different
j same f same fee
S S'
j same d same fde
S S'j+1
evicted by "#$#evicted by "
Farthest-In-Future: Analysis
8
Proof. (continued)Let !′ be the first time after ! + $ that % and %′ must take a different action,
and let & be item requested at time !′.
Case 3a: & = (.
Can't happen: ( was evicted by Farthest-In-Future so there must be a request for )before (.
Case 3b: & = ).
Element ) can't be in cache of %, so let (′ be the element that % evicts.
• if (′ = (, %′ accesses ) from cache; now % and %′ have same cache• if (* ≠ (, %′ evicts (′ and brings ( into the cache; now % and %′ have the same
cache
Note: %′ is no longer reduced, but can be transformed intoa reduced schedule that agrees with %,-, through step ! + $
same e same f
S S'
j'Action of S must involve e or f (or both)
Farthest-In-Future: Analysis
9
Proof. (continued)Let !′ be the first time after ! + $ that % and %′ must take a different action, and let & be item requested at time !′.
Case 3c: & ≠ ( and & ≠ ). % must evict (.Make %′ evict ); now % and %′ have the same cache. ▪
same e same f
S S'
j'Action of S must involve e or f (or both)
In each case can now extend %’ using rest of % at no extra cost.%’ is optimal, reduced, and agrees with %+,+ for ! + $ stepsOptimality of %+,+ follows by induction.
same g same g
S S'
j'
otherwise %′ would take the same action
Online Caching
10
• Online vs. offline algorithms.Offline: full sequence of requests is known a priori.Online (reality): requests are not known in advance.Caching is among most fundamental online problems in CS.
• LIFO. Evict item brought in most recently.• LRU. Evict item whose most recent access was earliest.
• Theorem. FIF is optimal offline eviction algorithm.Provides basis for understanding and analyzing online algorithms.LRU is k-competitive. [Section 13.8]LIFO is arbitrarily bad.
FIF with direction of time reversed!
Single Source Shortest PathGiven an (un)directed connected graph ! = ($, &) with non-negative edge weights () ≥ 0 and a start vertex ,.
Find length of shortest paths from , to each vertex in !
Cost of path s-2-3-4-t= 9 + 23 + 6 + 6= 44.
s
3
t
2
6
7
45
23
182
9
14
15 5
30
20
44
16
11
6
19
6
length of path = sum of edge weights in path
Dijkstra(!, #, $) {Initialize set of explored nodes % ← {$}
// Maintain distance from $ to each vertices in %) $ ← *
while (% ≠ ,){
Pick an edge (., /) such that . ∈ % and / ∉ % and )[.] + #(.,/) is as small as possible.
Add / to % and define )[/] = )[.] + #(.,/).789:;< / ← ..
}
Dijkstra’s Algorithm
Dijkstra’s Algorithm: Example
0
¥¥
¥
¥
¥
¥
¥
¥
¥
¥¥
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
¥
¥
¥
¥
¥
¥
¥¥
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
¥
¥
¥
¥
¥
¥
¥¥
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
¥
9
¥
¥
¥
¥
¥¥
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
¥
9
¥
¥
¥
¥
¥¥
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
8
9
¥
¥
¥
¥
75
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
8
9
¥
¥
¥
¥
75
¥
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
8
9
¥
¥
¥
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
8
9
¥
¥
¥
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
¥
8
9
¥
¥
¥
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
¥
¥
¥
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
¥
¥
¥
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
¥
10
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
¥
10
¥
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
14
10
20
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
14
10
20
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
14
10
19
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
14
10
19
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
14
10
18
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm: Example
0
24
16
8
9
15
14
10
18
75
13
425
10
1
8
43
4
3 1
5
8
2
5
6
7
9
s
Dijkstra’s Algorithm outputs a tree.
Disjkstra’s Algorithm: CorrectnessTheorem: For any ! ∈ #, the path $% on the tree in the shortest path from & to ! on '. (For all ! ∈ #, ) ! = dist(&, !).)Proof: Induction on # = 1.Base Case: This is always true when # = & .Inductive Step: Say 3 is the 1 + 1 67 vertex that we add to S.Let (!, 3) be last edge on $8.If $8 is not the shortest path, there is a shorter path $ to #.Consider the first time that $ leaves # with edge (9, :). So, ; $ ≥ ) 9 + ;=,> ≥ ) ! + ;%,8 = ) 3 = ; $8 .
A contradiction. Sv
y
u
sx
$8
$Due to the choice of 3$ is the shorter path.
Remarks on Dijkstra’s Algorithm• Algorithm produces a tree of shortest paths to ! following
Parent links (for undirected graph)• Algorithm works on directed graph (with nonnegative weights)• The algorithm fails with negative edge weights. • Why does it fail?
Implementing Dijkstra’s AlgorithmPriority Queue: Elements each with an associated key Operations
• Insert• Find-min
– Return the element with the smallest key
• Delete-min– Return the element with the smallest key and delete it from the data structure
• Decrease-key– Decrease the key value of some element
ImplementationsArrays: • !(#) time find/delete-min, • !(1) time insert/decrease key Binary Heaps:• !(log #) time insert/decrease-key/delete-min, • !(1) time find-minFibonacci heap:• !(1) time insert/decrease-key• !(log #) delete-min• O(1) time find-min
Read wiki!
Dijkstra(!, #, $) {Initialize set of explored nodes % ← {$}
// Maintain distance from $ to each vertices in %) $ ← *Insert all neighbors + of s into a priority queue with value #($,+).
while (% ≠ /){
Pick an edge (0, +) such that 0 ∈ % and + ∉ % and )[0] + #(0,+) is as small as possible.v ¬ delete min element from 6
Add + to % and define )[+] = )[0] + #(0,+).89:;<= + ← 0.
foreach (edge ; = (+,>) incident to +)if (> ∉ %)
if (> is not in the 6)Insert > into 6 with value ) + + #(+,>)
else (the key of > > ) + + #(+,>))Decrease key of + to )[+] + #(+,>).
}
?(@) of delete min,each in ?(log @)
?(@) of insert,each in ?(1)
?(E) of decrease/insert key, each runs in ?(1)
