Shortest Path

AlgorithmsChapters 24, 25

Example: Positive Weights

s

t

y

x

z

3

6

2 7

6

3

412

5

Example: Negative Weights

s

t

y

x

z

6

5

7

9

2

-38

7

-2

-4

Shortest path problem

Given a weighted, directed graph 𝐺 = 𝑉, 𝐸 ,

with weight function 𝑤:𝐸 → ℝ. The weight

𝑤 𝑝 of a path 𝑝 = 𝑣0, 𝑣1, … , 𝑣𝑘 is the sum of

the weights of its constituent edges

𝑤 𝑝 = σ𝑖=1𝑘 𝑤 𝑣𝑖−1, 𝑣𝑖

Shortest-path weight 𝛿 𝑢, 𝑣 from 𝑢 to 𝑣 is

𝛿 𝑢, 𝑣 = ൝min 𝑤 𝑝 : 𝑢→𝑝𝑣 if there is a path from 𝑢 to 𝑣

∞ 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

The shortest path is any path with shortest

path weight

Problem Variants

Single-source single-destination shortest path

Single-source all-destinations shortest paths

All-sources single-destination shortest paths

All-pairs shortest paths

Optimal Substructure

𝑣0

𝑣1 𝑣𝑘Shortest

path from

𝑣0 to 𝑣𝑘

Optimal Substructure

𝑣0

𝑣1 𝑣𝑘Shortest

path from

𝑣0 to 𝑣𝑘𝑣𝑖

𝑣𝑗

Shortest path from 𝑣𝑖 to 𝑣𝑗

Optimal Substructure

𝑣0

𝑣1 𝑣𝑘Shortest

path from

𝑣0 to 𝑣𝑘𝑣𝑖

𝑣𝑗

Assume that this is not the

shortest path from 𝑣𝑖 to 𝑣𝑗

And this is the shortest path

from 𝑣𝑖 to 𝑣𝑗

𝑣𝑖

𝑣𝑗

Optimal Substructure

𝑣0

𝑣1 𝑣𝑘Shortest

path from

𝑣0 to 𝑣𝑘𝑣𝑖

𝑣𝑗

Assume that this is not the

shortest path from 𝑣𝑖 to 𝑣𝑗

And this is the shortest path

from 𝑣𝑖 to 𝑣𝑗

𝑣𝑖

𝑣𝑗 𝑣𝑘

𝑣0

𝑣1

Then this new path is

better than the optimal

answer from 𝑣0 to 𝑣𝑘which is a contradiction

Cycles in Shortest Path

Negative weight cycles?

Make the shortest path undefined

Positive weight cycles?

Can always be removed to produce a shorter path

0-weight cycles?

Can always be removed while still having a

shortest path

Conclusion: We can disregard cycles in our

solutions

Shortest path representation

For single-source all-destinations shortest

path

𝑣0

𝑣1

𝑣2

𝑣3

𝑣5

𝑣4

𝑣6

Shortest path representation

For single-source all-destinations shortest

path

For each vertex 𝑣𝑖, we store its predecessor

𝑣𝑖 . 𝜋 in the shortest-path tree and the cost of

the shortest path 𝑣𝑖 . 𝑑 = 𝛿 𝑠, 𝑣𝑖

𝑣0

𝑣1

𝑣2

𝑣3

𝑣5

𝑣4

𝑣6

Triangle Inequality

For any edge 𝑢, 𝑣 ∈ 𝐸, we have

𝛿 𝑠, 𝑣 ≤ 𝛿 𝑠, 𝑢 + 𝑤 𝑢, 𝑣

Question: What is the upper limit of the size

of the shortest path in terms of number of

edges?

Bellman-Ford

Algorithm

Initialization

Relax

Relaxation

𝑢

𝑣2

𝑐

𝑠

𝑎

5

3

1

4

2

𝑏3

𝜋 = 𝑐𝑑 = 9

𝜋 = 𝑎𝑑 = 4

Relaxation

𝑢

𝑣2

𝑐

𝑠

𝑎

5

3

1

4

2

𝑏3

𝜋 = 𝑢𝑑 = 6

𝜋 = 𝑎𝑑 = 4

Bellman-Ford Algorithm

𝑉 − 1iterations

𝐸iterations

Running time is 𝑂 𝑉 ⋅ 𝐸

Detecting Negative Cycles

Assume the input graph contains a negative-

weight cycle 𝑐 = 𝑣0, 𝑣1, … , 𝑣𝑘 where 𝑣0 = 𝑣𝑘

σ𝑖=1𝑘 𝑤 𝑣𝑖−1, 𝑣𝑖 < 0 // weight of the cycle

For the sake of contradiction, assume that

∀𝑖: 𝑣𝑖 . 𝑑 ≤ 𝑣𝑖−1. 𝑑 + 𝑤 𝑣𝑖−1, 𝑣𝑖 // inequality

σ𝑖=1𝑘 𝑣𝑖 . 𝑑 ≤ σ𝑖=1

𝑘 𝑣𝑖−1. 𝑑 + 𝑤 𝑣𝑖−1, 𝑣𝑖

σ𝑖=1𝑘 𝑣𝑖 . 𝑑 ≤ σ𝑖=1

𝑘 𝑣𝑖−1. 𝑑 + σ𝑖=1𝑘 𝑤 𝑣𝑖−1, 𝑣𝑖

σ𝑖=1𝑘 𝑤 𝑣𝑖−1, 𝑣𝑖 ≥ 0 // Contradiction

Dijkstra’s Algorithm

Dijkstra’s Algorithm

Bellman-Ford algorithm assumes no negative

cycles but can handle negative-weight edges

In many real applications, even negative-

weight edges are not allowed

E.g., road networks

In this case, Dijkstra’s algorithm can find the

shortest paths more efficiently

Dijkstra’s AlgorithmInit-Single-Source(G,s)

𝑆 = ∅ // Shortest path tree vertices

𝑄 = ∅ // Min-heap

For each vertex 𝑢 ∈ 𝐺. 𝑉

Insert(𝑄, 𝑢)

While 𝑄 ≠ ∅

𝑢 =Extract-Min(𝑄)

𝑆 = 𝑆 ∪ 𝑢

For each vertex 𝑣 ∈ 𝐺. 𝑎𝑑𝑗 𝑢

Relax(𝑢, 𝑣, 𝑤)

If 𝑣. 𝑑 changes

Decrease-Key(𝑄, 𝑣, 𝑣. 𝑑)

Dijkstra’s AlgorithmInit-Single-Source(G, s)

𝑆 = ∅ // Shortest path tree vertices

𝑄 = ∅ // Min-heap

For each vertex 𝑢 ∈ 𝐺. 𝑉

Insert(𝑄, 𝑢)

While 𝑄 ≠ ∅

𝑢 =Extract-Min(𝑄) // 𝑂 𝑙𝑜𝑔 𝑉

𝑆 = 𝑆 ∪ 𝑢

For each vertex 𝑣 ∈ 𝐺. 𝑎𝑑𝑗 𝑢 // 𝐸 times in total

Relax(𝑢, 𝑣, 𝑤) // 𝑂 1 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛

If 𝑣. 𝑑 changes

Decrease-Key(𝑄, 𝑣, 𝑣. 𝑑) // 𝑂 𝑙𝑜𝑔 𝑉

Correctness of Dijkstra’s

𝑆

𝑠

𝑢

𝑧

Shortest path from

s to z

What Dijkstra’s find

For contradiction, let’s assume

that it is incorrect, i.e., not

shortest path

Correctness of Dijkstra’s

𝑆

𝑠

𝑢

𝑥

Real shortest path

𝑦

𝑦. 𝑑 = 𝛿 𝑠, 𝑦

≤ 𝛿 𝑠, 𝑢

≤ 𝑢. 𝑑

𝑢. 𝑑 ≤ 𝑦. 𝑑At the time when 𝑢was chosen to expand

𝑦. 𝑑 = 𝛿 𝑠, 𝑦 = 𝛿 𝑠, 𝑢 = 𝑢. 𝑑

𝑢. 𝑑 = 𝛿 𝑠, 𝑢

All-pairs Shortest

PathsChapter 25

𝑣1

𝑣3

𝑣2

𝑣4

𝑣5

𝑣6

10

75

10 4 3 1

4

4

2

23

Floyd-Warshall

AlgorithmSection 25.2

Floyd-Warshall Algorithm

Let 𝑝 𝑖, 𝑗 be the shortest path from any pair

of vertices 𝑖, 𝑗

An intermediate vertex 𝑙 is any vertex on the

path 𝑝 𝑖, 𝑗 , where 𝑖 ≠ 𝑙 and 𝑗 ≠ 𝑙

In this case, 𝑝 𝑖, 𝑗 = 𝑝 𝑖, 𝑙 ||𝑝 𝑙, 𝑗

Now, let us define 𝑝𝑘 𝑖, 𝑗 as the shortest

path between 𝑖, 𝑗 such that all the

intermediate vertices are in the set

𝑣1, 𝑣2, … , 𝑣𝑘By definition, 𝑝 𝑖, 𝑗 = 𝑝𝑛 𝑖, 𝑗

Shortest paths representation

𝒅 𝒊, 𝒋 𝒗𝟏 𝒗𝟐 𝒗𝟑 𝒗𝟒

𝒗𝟏 0

𝒗𝟐 0

𝒗𝟑 0

𝒗𝟒 0

𝝅 𝒊, 𝒋 𝒗𝟏 𝒗𝟐 𝒗𝟑 𝒗𝟒

𝒗𝟏 NIL

𝒗𝟐 NIL

𝒗𝟑 NIL

𝒗𝟒 NIL

Single-source all-destinations

shorts paths (𝑠 = 𝑣3)

All-sources single-destination

shortest paths (𝑑 = 𝑣2)

Floyd-Warshall Algorithm

𝑖 𝑗

𝑣𝑘

𝑝1 𝑝2

𝑝1 = 𝑝𝑘 𝑖, 𝑣𝑘 = 𝑝𝑘−1(𝑖, 𝑣𝑘) 𝑝2 = 𝑝𝑘 𝑣𝑘, 𝑗 = 𝑝𝑘−1(𝑗, 𝑣𝑘)

𝑝𝑘 𝑖, 𝑗 = 𝑝𝑘 𝑖, 𝑣𝑘 ||𝑝𝑘(𝑣𝑘 , 𝑗)

Recursive Formula

𝑝𝑘 𝑖, 𝑗 = ൞

𝑖, 𝑗

𝑝𝑘−1 𝑖, 𝑗

𝑘 = 0𝑘 ∉ 𝑝𝑘 𝑖, 𝑗

𝑝𝑘−1 𝑖, 𝑣𝑘 ||𝑝𝑘−1(𝑣𝑘 , 𝑗) 𝑘 ∈ 𝑝𝑘 𝑖, 𝑗

𝑑𝑘 𝑖, 𝑗 = ቐ𝑤 𝑖, 𝑗 𝑘 = 0

min 𝑑𝑘−1 𝑖, 𝑗 , 𝑑𝑘−1 𝑖, 𝑘 + 𝑑𝑘−1 𝑘, 𝑗 𝑘 ≥ 1

Pseudo-Code

1. FLOYD-WARSHALL(𝑊)

2. 𝑛 = 𝑊.rows

3. 𝑑0 𝑖, 𝑗 = ∞;∀𝑖, 𝑗

4. for 𝑘 = 1 to 𝑛

5. let 𝐷𝑘 = 𝑑𝑘 𝑖, 𝑗 be a new 𝑛 × 𝑛 matrix

6. for 𝑖=1 to 𝑛

7. for 𝑗=1 to 𝑛

8. 𝑑𝑘 𝑖, 𝑗 = min 𝑑𝑘 𝑖, 𝑗 , 𝑑𝑘−1 𝑖, 𝑘 + 𝑑𝑘−1 𝑘, 𝑗

9. return 𝐷𝑛

𝑣1

𝑣3

𝑣2

𝑣4

𝑣5

𝑣6

10

75

10 4 3 1

4

4

2

23

All-pairs Shortest

PathBased on Matrix Multiplication

Recursive Formulation

Let 𝑙𝑖𝑗(𝑚)

be the minimum weight path from 𝑖 to

𝑗 of at most 𝑚 edges.

𝑙𝑖𝑗(0)

= ቊ0 𝑖𝑓 𝑖 = 𝑗∞ 𝑖𝑓 𝑖 ≠ 𝑗

𝑙𝑖𝑗𝑚 = min 𝑙𝑖𝑗

𝑚−1 , min1≤𝑘≤𝑛

𝑙𝑖𝑘(𝑚−1)

+𝑤𝑘𝑗

= min1≤𝑘≤𝑛

𝑙𝑖𝑘𝑚−1 +𝑤𝑘𝑗

Pseudo-code

1. EXTEND-SHORTEST-PATHS(𝐿, 𝑊)

2. 𝑛 = 𝐿. rows

3. let 𝐿′ = (𝑙′𝑖𝑗) be a new 𝑛 × 𝑛 matrix

4. for 𝑖 = 1 to 𝑛

5. for 𝑗 = 1 to 𝑛

6. 𝑙𝑖𝑗’ = ∞

7. for 𝑘 = 1 to 𝑛

8. 𝑙𝑖𝑗’ = min(𝑙𝑖𝑗’, 𝑙𝑖𝑘 + 𝑙𝑘𝑗)

9. return 𝐿’

Analogy with Matrix Multiplication

𝐶 = 𝐴 × 𝐵

𝑐𝑖𝑗 = σ𝑘=1𝑛 𝑎𝑖𝑘 ∙ 𝑏𝑘𝑗

If 𝐶 = 𝐴 × 𝐴

𝑐𝑖𝑗 = σ𝑘=1𝑛 𝑎𝑖𝑘 ∙ 𝑎𝑘𝑗

For the shortest path algorithm

𝑙𝑖𝑗𝑘 = min1≤𝑘≤𝑛 𝑙𝑖𝑘

𝑘−1 + 𝑙𝑘𝑗𝑘−1

Equivalent to 𝐿𝑘 = 𝐿𝑘−1 ⊛𝐿𝑘−1

Where ⊛ is a redefined matrix multiplication

operation based on addition and minimum