Chapter 25: All-Pairs Shortest-Paths

1

Chapter 25: All-Pairs Shortest-Paths

2

Some AlgorithmsWhen no negative edges

– Using Dijkstra’s algorithm: O(V3)– Using Binary heap implementation: O(VE lg

V)– Using Fibonacci heap: O(VE + V2log V)

• When no negative cycles– Floyd-Warshall [1962]: O(V3) time

• When negative cycles– Using Bellman-Ford algorithm: O(V2 E) =

O(V4 )– Johnson [1977]: O(VE + V2log V) time based

on a clever combination of Bellman-Ford and Dijkstra

3

A dynamic programming approach

1. characterize the structure of an optimal solution,2. recursively define the value of an optimal solution,3. compute the value of an optimal solution in a bottom-up fashion.

4

The structure of an optimal solution

•Consider a shortest path p from vertex i to vertex j, and suppose that p contains at most m edges. Assume that there are no negative-weight cycles. Hence m ≤ n-1 is finite.

5

The structure of an optimal solution (cont.)

• If i = j, then p has weight 0 and no

edge

• If i≠j, we decompose p into i ~ k -> j

where p’ contains at most m-1

edges.

• Moreover, p’ is a shortest path from i to k and δ(i,j) = δ(i,k) + wkj , where

δ(i,j) dente the shortest weight path

from i to j

'p

6

Recursive solution to the all-pairs shortest-path problem

• Define: lij(m) = minimum weight of any path from i to j that contains at most m edges.

0 if i = j• lij(0) =

∞ if i ≠ j• Then lij(m) = min{ lij(m-1), min1≤k≤n

{lik(m-1) + wkj}} = min1≤k≤n {lik(m-1) + wkj} (why?)

7

Recursive solution to the all-pairs shortest-path

problem• Since shortest path from i to j

contains at most n-1 edges, δ(i,j) = lij(n-1) = lij(n) = lij(n+1) = …• Computing the shortest-path weight

bottom up:– Compute L(1) ,L(2) ,…., L(n-1) where L(m)=(lij (m)) for all i and j– Note that L(1) = W.

8

06

052

04

710

4830

)1(L

0618

20512

11504

71403

42830

)2(L

1

45

2

3

-4

17

2

6

-5

83

4

Example:

W =

L(2) = L(1) x W

9

06158

20512

115047

11403

42330

)3(L

06158

20512

35047

11403

42310

)4(L

L(3) = L(2) x W

L(4) = L(3) x W

10

EXTENDED-SHORTEST-PATHS(L, W)

• Given matrices L(m-1) and W return L(m)

1 n <- L.row2 Let L’ = (l’ij) be a new n x n matrix

3 for i = 1 to n4 for j = 1 to n5 l’ij = ∞

6 for k = 1 to n7 l’ij = min(l’ij, lik + wkj)

8 return L’

11

Matrix Multiplication

Let l(m-1) -> a w -> b l(m) -> c min -> + + -> ‧

time complexity : O(n3)

Cij = k=1 to n aik‧bkj

lij(m) = min1≤k≤n {lik(m-1) + wkj}

12

SLOW-ALL-PAIRS-SHORTEST-PATHS(W)

L(1) = L(0) ‧ W = WL(2) = L(1) ‧ W = W2

L(3) = L(2) ‧ W = W3

‧ ‧ ‧

L(n-1) = L(n-2) ‧W = Wn-1

13

SLOW-ALL-PAIRS-SHORTEST-PATHS(W)

1 n = W.rows2 L(1) = W3 for m = 2 to n-14 let L(m) be a new n x n matrix5 L(m) = EXTENDED-SHORTEST-

PATHS( L(m-1), W )6 return L(n-1)

Time complexity : O(n4)

14

Improving the running time

L(1) = WL(2) = W2 =W．WL(4) =W4 = W2．W2

. . .

i.e., using repeating squaring!

Time complexity: O(n3lg n)

12122)2( )1log()1log()1log()1log(

nnnn

WWWL

15

FASTER-ALL-PAIRS-SHORTEST-PATHS(W)1. n =W.row2. L(1) =W3. m =14. while m < n-1 5. let L(2m) be a new n x n matrix

6. L(2m) = Extend-Shorest-Path(L(m), L(m))7. m = 2m8. return L(m)

FASTER-ALL-PAIRS-SHORTEST-PATHS

16

The Floyd-Warshall algorithm

• A different dynamic programming formulation

‧The structure of a shortest path: Let V(G)={1,2,…,n}. For any pair of

vertices i, j єV, consider all paths from i to j whose intermediate vertices are drawn from {1, 2,…,k}, and let p be a minimum weight path among them.

17

The structure of a shortest path

• If k is not in p, then all intermediate vertices are in {1, 2,…,k-1}.

• If k is an intermediate vertex of p,

then p can be decomposed into i ~ k

~ j where p1 is a shortest path from i

to k with all the intermediate vertices in {1,2,…,k-1} and p2 is a shortest path from k to j with all the intermediate vertices in {1,2,…,k-1}.

1 p 2 p

18

A recursive solution to the all-pairs shortest path

• Let dij(k) =the weight of a shortest

path from vertex i to vertex j with all intermediate vertices in the set {1,2,…,k}.

dij(k) = wij if k = 0

= min(dij(k-1), dik

(k-1) + dkj(k-1)) if k ≥ 1

D(n) = (dij(n)) if the final solution!

19

FLOYD-WARSHALL(W)

1. n = W.rows 2. D(0)= W 3. for k = 1 to n 4. Let D(k) = (dij

(k)) be a new n x n matrix 5. for i = 1 to n 6. for j = 1 to n 7. dij

(k) = min (dij(k-1), dik

(k-1) + dkj(k-

1)) 8. return D(n)

Complexity: O(n3)

20

Constructing a shortest path

• π(0), π(1),…, π(n)

• πij(k) : is the predecessor of the vertex j

on a shortest path from vertex i with all intermediate vertices in the set {1,2,…,k}.

πij(0) = NIL if i=j or wij = ∞

= i if i ≠j and wij < ∞

πij(k) = πij

(k-1) if dij(k-1) ≤ dik

(k-1) + dkj(k-1)

= πkj(k-1) if dij

(k-1) > dik(k-1) + dkj

(k-1)

21

Example

06

052

04

710

4830

)0(D

06

20552

04

710

4830

(1)D

NNNN

NNN

NNNN

NNN

NN

5

44

3

22

111

(0)

NNNN

N

NNNN

NNN

NN

5

1414

3

22

111

(1)

22

06

20552

11504

710

44830

(2)D

NNNN

N

NN

NNN

N

5

1414

223

22

1211

(2)

06

20512

11504

710

44830

)3(D

NNNN

N

NN

NNN

N

5

1434

223

22

1211

(3)

23

06158

20512

35047

11403

44130

(4)D

06158

20512

35047

11403

42310

(5)D

N

N

N

N

N

5434

1434

1234

1244

1241

(4)

N

N

N

N

N

5434

1434

1234

1244

1541

(5)

24

Transitive closure of a directed graph

• Given a directed graph G = (V, E) with V = {1,2,…, n}

• The transitive closure of G is G*= (V, E*) where E*={(i, j)| there is a path from i to j in G}.

Modify FLOYD-WARSHALL algorithm:tij

(0) = 0 if i≠j and (i,j) ∉ E

1 if i=j or (i,j) є Efor k ≥ 1 tij

(k) = tij(k-1) (tik

(k-1) tkj(k-1))

25

TRANSITIVE-CLOSURE(G)1 n = |G.V|2 Let T(0) = (tij

(0)) be a new n x n matrix

3 for i = 1 to n 4 for j =1 to n5 if i == j or (i, j) ∈ G.E6 tij

(0) = 1

7 else tij(0) = 0

8 for k =1 to n9 Let T(k) = (tij

(k)) be a new n x n matrix

10 for i = 1 to n11 for j =1 to n12 tij

(k) = tij(k-1) (tik

(k-1) tkj(k-1))

123 return T(n)

Time complexity: O(n3)

26

Example

1101

0110

1110

0001

(0)T

1101

0110

1110

0001

(1)T

1101

1110

1110

0001

(2)T

1111

1110

1110

0001

(3)T

1111

1111

1111

0001

(4)T

① ②

③④

27

Some AlgorithmsWhen no negative edges

– Using Dijkstra’s algorithm: O(V3)– Using Binary heap implementation: O(VE lg

V)– Using Fibonacci heap: O(VE + V2log V)

When no negative cycles– Floyd-Warshall [1962]: O(V3) time

• When negative cycles– Using Bellman-Ford algorithm: O(V2 E) =

O(V4 )– Johnson [1977]: O(VE + V2log V) time based

on a clever combination of Bellman-Ford and Dijkstra

28

Johnson’s algorithm for sparse graphs

• If all edge weights in G are nonnegative,

we can find all shortest paths in O(V2lg V

+ VE) by using Dijkstra’s algorithm with

Fibonacci heap

• Bellman-Ford algorithm takes O(VE)

• Using reweighting technique

29

Reweighting technique

• If G has negative-weighted edge, we compute a new set of nonnegative weight that allows us to use the same method. The new set of edge weight ŵ satisfies:

• 1. For all pairs of vertices u, v єV, a shortest path from u to v using weight function w is also a shortest path from u to v using the weight function ŵ

• 2. ∀(u,v) є E(G), ŵ(u,v) is nonnegative

30

Lemma: (Reweighting doesn’t change shortest paths)

• Given a weighted directed graph G = (V, E) with weight function w:E→R , let h:V →R be any function mapping vertices to real numbers. For each edge (u,v) є E, ŵ(u,v) = w(u,v) + h(u) – h(v)

• Let P=<v0,v1,…,vk> be a path from vertex v0 to vk Then w(p) = δ(v0,vk) if and only if ŵ(p) = (v0,vk) Also, G has a negative-weight cycle using weight function w iff G has a negative weight cycle using weight function ŵ. 

31

Proof• ŵ(p) = w(p) + h(v0) – h(vk)

ŵ(p) = ŵ(vi-1 ,vi)

= (w(vi-1 ,vi) + h(vi-1)-h(vi))

= w(vi-1 ,vi) + h(v0)-h(vk)

=w(p) + h(v0) – h(vk)

k

i 1

k

i 1

k

i 1

32

Proof

• Because h(v0) and h(vk) do not depend on the path, if one path from v0 to vk is shorter than another using weight function w, then it is also shorter using ŵ. Thus,

w(p) = δ(v0,vk) if and only if ŵ(p) = (v0,vk)

33

Proof

• G has a negative-weight cycle using w iff G has a negative-weight cycle using ŵ. 

• Consider any cycle C=<v0,v1,…,vk> with v0=vk . Then ŵ(C) = w(C) + h(v0) – h(vk) = w(C) .

Question: how to setting the value of h(vi) for all i?

34

Producing nonnegative weight by reweighting

• Given a weighted directed graph G = (V, E)

• We make a new graph G’= (V’,E’), V’ = V ∪ {s}, E’ = E ∪{(s,v): v є V} and w(s,v) = 0, for all v in V

• Let h(v) = δ(s, v) for all v V’• We have h(v) ≤ h(u) + w(u, v) (why?)• ŵ(u, v) = w(u, v) + h(u) – h(v) ≥ 0

35

-41

7

2

6

-5

8

3 42

3

45

1

Example:

36

-4

1

7

2

6

-5

8

3 4 2

3

5 4

1

0S

0

0

0

0

0

37

-4

1

7

2

6

-5

8

3 4-1

-5

-4 0

0

2

3

45

1

0S

0

0

0

0

0

38

0

0

10

2

2

0

13

4 0-1

-5

-4 0

0

2

3

45

1

0S

4

0

0

1

5

h(v) = δ(s, v)ŵ(u, v) = w(u, v) + h(u) – h(v)

39

JOHNSON algorithm

1 Computing G’, where G’.V = G.V ∪ {s}and G’.E= G.E ∪{(s, v): v є G.V} and w(s, v) = 0.

2 if BELLMAN-FORD(G’, w, s)= FALSE3 print “the input graph contains negative

weight cycle”4 else for each vertex v є G’.V5 set h(v) to be the value of δ(s, v) computed

by the BF algorithm6 for each edge (u, v) є G’.E, ŵ(u, v) = w(u, v)

+ h(u) – h(v)

40

JOHNSON algorithm

7 Let D = (duv) be a new n x n matrix

8 for each vertex u є G.V run DIJKSTRA (G, ŵ, u) to compute (u, v)

for all v є V[G] .10 for each vertex v є G.V 11 duv = (u, v) + h(v) – h(u)

12 return DComplexity: O(V2lgV + VE)

41

00

10

2

2

0

13

4 0

2

3

45

1

2/1

2/-3

2/20/-4

0/0

00

10

2

2

0

13

4 0

2

3

45

1 0/0

2/3 0/-4

0/12/-1

00

10

2

2

0

13

4 03

45

1 0/0

0/4

2/7

2/3 0/5

2

42

0 010

2

2

0

13

4 0

2

3

45

1

2/2

0/-1

0/-5

0/02/-2

00

10

2

2

0

13

4 0

2

3

45

12/5

4/8

0/0

2/1

2/6

43

Homework• Practice at home: Exercises: 25.1-1,

25.1-3, 25.1-6• Practice at home : 25.2-4, 25.2-6• Exercises: 25.2-8 (Due: Jan. 4)• Practice at home : 25.3-2, 25.3-6• Exercises: 25.3-5 (Due: Jan. 4)• Bonus: Write a program to find all

pairs shortest path of a graph G. There may exist negative weight cycle on graph G.