Algorithm Design and Analysis (ADA)

242-535 ADA: 11. Shortest Paths 1

• Objectiveo we describe three algorithms for finding the

shortest path tree through a graph:• Dijkstra, Bellman-Ford, and topological sort

Algorithm Design and Analysis

(ADA)242-535, Semester 1 2014-2015

11. Shortest Paths


1. Problem Overview2. Dijkstra’s Algorithm3. Bellman-Ford Algorithm4. DAGs and Topological Sort5. Summary of Algorithms

Overview


• Edge weights may represent distances, costs, etc.

• Example:o the distances between airports

1. Problem Overview

ORD PVD

MIADFW

SFO

LAX

LGA

HNL

849

802

13871743

1843

10991120

1233337

2555

142

12

05


• The length of a path = the sum of the weights of the edges in the path.

• The shortest path between two verticies = the path having the minimum length.

• Example:o shortest path between Providence and Honolulu

Shortest Paths

Graphs 4

ORD PVD

MIADFW

SFO

LAX

LGA

HNL

849

802

13871743

1843

10991120

1233337

2555

142

12

05


Typical Applications

the fast buying and selling of financial items (e.g. foreign exchange) in different markets to profit from price differences


Arbitrage GraphA chance tomake money.


Property 1A subpath of a shortest path is itself a shortest pathThis means that shortest path has an optimal structure.

Property 2There is a shortest paths tree (SPT) from a start vertex to all the other vertices (e.g. see below with Providence as the start).

Shortest Path Properties

Graphs 7

ORD PVD

MIADFW

SFO

LAX

LGA

HNL

849

802

13871743

1843

10991120

1233337

2555

142

12

05


• A shortest path tree is a tree that spans all the nodes in the graph, using edges that give the shortest paths from the starting node to all the other vertices.

Another SPT

SPT

start nodeis 0


• Build a shortest path tree (SPT)• No negative edge weights allowed.• Due to Edsger W. Dijkstra (1959, when 29).

• Similar to Prim's algorithmo Grow a tree gradually, advancing from vertices taken

from a queueo Dijkstra is a greedy algorithm like Prim'so Instead of a MST, we are building a SPT

2. Dijkstra’s Algorithm


• Each vertex has a entry in a distTo[] array.

• Initialize dist[start] to 0 and all other distTo[] entries to positive infinity ()

• During the execution, the distTo[] values will get smaller. This is called (edge) relaxing.

• At each iteration, the SPT is grown by adding a vertex with the smallest distTo[] value.

Algorithm in Words


• The growing SPT is drawn with black edges

• The crossing edge with the shortest path is chosen by looking at the distTo[] valueso e.g. distTo[w] will

be the smallest which is not already in the SPT

Algorithm Graphically


What is Relaxing?

a

b

c

d

distTo[a] = 3

distTo[b] = 13

distTo[c] = 13

distTo[d] = 10

5

4

8relax'edges'linkedto a

a is inthe SPT

a

b

c

d

distTo[a] = 3

distTo[b] = 8

distTo[c] = 7

distTo[d] = 10

5

4

8

c will be the nextvertex to join SPTif (distTo[b] > distTo[a] + b.weight())

distTo[b] = distTo[a] + b.weight();


• distTo[v] is the length of the shortest known path from the start node to vertex v.o distTo[] values will get smaller (be relaxed)

• edgeTo[v] contains the edge that connects v to its parent in the SPTo this is used to reconstruct the shortest path at the end

of the algorithm

• a priority queue keeps track of vertices that are candidates for being added to the SPT

SPT Data Structures


Data Structures in UseSPT

from 0


• Note that we dont need to maintain a tree data structure for the SPT.

• Instead the edgeTo[] array is used to reconstruct the paths through the SPT.


Building the Path 0 --> 6


DirectedEdge[] edgeTo;double[] distTo;

MinPriQueue<Double, Double> pq;/* this is a priority queue where each element's position is based on a separate value, which is not its vertex number */

Code

3 3 0 17 4 23 2 61remove() add()

change() value


void dijkstra(Digraph graph, int start){ for (int v : graph.allVerts()) distTo[v] = Double.POSITIVE_INFINITY; distTo[start] = 0;

pq.add(start, distTo[start]); // add start node to queue using distTo[] as value

while (!pq.isEmpty()) { int small = pq.remove(); // small now 'in' SPT relax(G, small); }}


void relax(Digraph graph, int u){ for(DirectedEdge e : graph.adj(u)) { int v = e.to(); // e is u --> v edge if (distTo[v] > distTo[u] + e.weight()) { distTo[v] = distTo[u] + e.weight(); edgeTo[v] = e; // e is used in SPT to goto v if (pq.contains(v)) // v already in queue? pq.change(v, distTo[v]); // change v's position in the queue // based on the changed distTo[] value else pq.add(v, distTo[v]); } } }

relax all edgesadjacent to u


Example in DetailStart node

is 0 removed(so 'in' SPT)



Finished queue is empty


• On each iteration of the while-loop:o one vertex is removed from the queue, which means

that it becomes part of the SPT

o the selected vertex changes its adjacent verticies’ distTo[] if they can be made smaller• thr adjacent edges are relaxed• edgeTo[] changes as well

• The algorithm eventually considers every vertex.

Notes


• Inductive statement S(i)o on the ith iteration of the while-loop, the v node is

removed from the queue (and so added to the SPT)

o distTo[v], is the shortest distance from the start node to v

Proof of the Algorithm

continued

Is this statement true?


• Basis (i = 1).o initially distTo[start] == 0, and all other

distTo[]s are Infinityo so the while-loop selection will be the start

nodeo distTo(start) is the shortest path from the start

node to the start node; it is 0

o S(1) is true.

continued


• Induction Case.o Prove that S(k) S(k+1)o Assume that S(k)o On the k+1 iteration, the while-loop removes

node v from the queue o Is distTo[v] the shortest path from start node to

v?


Diagram of Induction Case

removed fromqueue (so nowin SPT)

still in the queue

startnode

w

u

v

Is distTo[v] the shortest path from start node to v ?

x choose vbecause ofdist[v]


• Assume that distTo[v] is not the shortest path from the start node to v.

• That means there must be another path to v:start -> ... -> w -> u -> ... -> v

which is shorter.

Proof by Contradiction

continued

something notalready selectedfrom the queue


• This means that:path_length(a -> ... -> w -> u -> ... -> v)

<path_length(a -> ... -> v)

• which means that: path_length(a -> ... -> w -> u) <path_length(a -> ... -> v)

• which means that:distTo[u] < distTo[v]

continued


• But, if distTo[u] < distTo[v] then u should have been selected instead of v on this iteration.

• Since v was chosen, it means that:distTo[u] ≥ distTo[v], for all u

• So distTo[v] is the shortest path.

continued


• The induction case was:o On the k+1 iteration, select node v.

Is distTo[v] the shortest path from start node to v?

o The answer is yes.o So, S(k+1) is true.

• Since the basis and induction cases are true, S(i) is true for all i.


• Dijkstra's original algorithm did not use a minimum priority queue and so ran in O(|V|2)o it used an array to store the vertices

• The implementation based on a minimum priority queue coded as a minimum heap runs in O(|E| * log |V|)

Running Time


Why O(E log V)?

dijkstra

forwhil

e

for

queue ops

relax()

O(V) look at every vertex

the queue containsat most V vertices

look at every edge from a vertex

look at all edges = E iters

O(log V)due to heap implof the queue

O(E*log V)

O(E*log V)in a dense graphE ≈ V2, so is biggerthan V


Why O(log V)?

† Individual ops are amortized analysis

PQ Operation

remove()

change()

Binary heap

log V

log V

Fib heap †

log V

1

Array

V

1Total E log V V log V + EV2

d-ary Heap

log V

log VE log E/V V

minimumbinaryheap

Fibonacci Heap(multiple trees)


Why Doesn’t Dijkstra Work for Negative Weight Edges?

C

A

D

F

0

45

9

4

1

5-8

distTo[c] will change from 5 to 4+5-8 = 1

• Adding a negative edge can change the distTo[] values for vertices already in the SPT.

• This breaks the algorithm since distTo[] is assumed to be the smallest when its node is selected for SPT

SPTvertices


• Unlike Dijkstra's algorithm, the Bellman-Ford algorithm can be used on graphs with negative edge weights, as long as the graph contains no negative cycles reachable from the source vertex s.

• The presence of such cycles means there is no shortest path, since the total weight becomes lower each time the cycle is traversed.

• Drawback: slower than Dijkstra's algorithm for the same problemo O(|V| * |E|)

3. Bellman-Ford Algorithm


• In graphs with negative weight cycles, some shortest paths will not exist:

Negative Cycles Problem

s t u x

vw

2

1

-4

2

1 1

What is distTo[x]starting from s?


Example

3535

37

28

32

38

26

39

29

34

-120-140

52

-125

28

This examplecan be processed by Bellman-Ford since there are no negative cycles.


SPT Using Bellman-Ford

35

32

52

-125

3934

26

distTo[4] = 26 + 34 + 39 + 52 – 125 = 26


• Bellman-Ford is similar to Dijkstra's Algorithm, but instead of greedily selecting the minimum weight vertex not yet in the SPT, it relaxes all the edges|V | − 1 times.

• The repetitions reduce the distTo[] values to their smallest throughout the graph. Why?

• The SPT visits every node in the graph at most once, so the biggest path in the SPT is at most |V|-1 edges long. So |V|-1 loops is certain to process it all.o assuming there are no negative cycles

Algorithm in Words


• Prove: after |V|-1 passes, all distTo[] values correcto Consider shortest path from s to v:

s v1 v2 v3 v4 v• Initially, distTo[s] = 0 is correct, and doesn’t change.

• After 1 pass through edges, distTo[v1] is correct and doesn’t change

• After 2 passes, distTo[v2] is correct and doesn’t change

• …

• Terminates in |V| - 1 passes (if no negative cycles).



void bellmanFord(Digraph graph, int start){ for (int v : graph.allVerts()) distTo[v] = Double.POSITIVE_INFINITY; distTo[start] = 0;

for (int i=1; i < graph.numV(); i++) relax(graph); for (DirectedEdge e : graph.allEdges()) { int u = e.from(); // edge e is u --> v int v = e.to(); if (distTo[v] > distTo[u] + e.weight()) println("There's no solution"); }}

Code

Initialize distTo[], whichwill reduce to shortest-path value

Relaxation: Make |V|-1 passes, relaxing every edge

Test for solution When do we not get a solution?


void relax(Digraph graph) // a simpler version than Dijkstra{ for (DirectedEdge e : graph.allEdges()) { int u = e.from(); // edge e is u --> v int v = e.to(); if (distTo[v] > distTo[u] + e.weight()) { distTo[v] = distTo[u] + e.weight(); edgeTo[v] = e; } } }


An Example in Detail

s

t

y

x

z

6

5

-2-3

7-48

927

stxyz

distTo[]

0

edgeTo[]

-----

s

t

y

x

z

6

5

-2-3

7-48

927

stxyz

distTo[]

067

edgeTo[]

-s->t-s->y-

The start node is s

after each relax for-loop iteration


s

t

y

x

z

6

5

-2-3

7-48

927

stxyz

distTo[]

06472

edgeTo[]

-s->ty->xs->yt->z

s

t

y

x

z

6

5

-2-3

7-48

927

stxyz

02472

-x->ty->xs->yt->z

s

t

y

x

z

6 -2-3

7-48

927

stxyz

0247-2

-x->ty->xs->yt->z

Finished


• The second for-loop executes |V|-1 times, and processes every edge (in the worst case) inside the body.

• Worse case running time is O(|V|*|E|)

• Note that order in which edges are processed affects how quickly the algorithm finishes

Running Time


• In many applications, edge-weighted digraphs have no directed cycles: they are DAGso Directed Acyclic Graphs

• We can modify a topological sort to find the SPT in a DAG o simpler and faster than Dijskstra

• linear running time: O(E + V)o handles negative edge weightso can be easily extended to solve related problems, such

as finding all the longest paths

4. DAGs and Topological Sort


• A topological sort puts the vertices in an order where all the vertices which "affect / go to" a vertex are before it.

• This is the ordering uses by edge relaxing to calculate the smallest distTo[] values.

• This means that relaxing need only pass through the sorted graph once to reduce all the distTo[] values to their smallest valueso the running time is linear

DAG Shortest Paths


Example of a DAG

32

3526

93

58

37

28

38

39

29

34

40

52

Negative weights are allowed,but there aren't any in this example.


Topological Sort of the Example

5 1 3 6 4 7 0 2

• A topological sort of a DAG is a linear ordering of its vertices such that for every directed edge u v,u comes before v in the ordering.

• Any DAG has at least one topological sort (and often many).


Multiple Sorts


• Topological sort is the same as a reverse postorder visit order added to a standard recursive Depth-First Search (DFS) algorithm.

TopSort = Reverse Postorder


Some Tree Visit Orderings

Reverse postorder: F, G, I, H, B, D, E, C, A (root, right, left)

the usual visitorder used by DFS


Graph Visit Orderings Example

Preorder:

Postorder:

Reverse postorder:

same as a topological sort

4

dfs(graph, 0)


• The different visit orders can be easily implemented by storing the nodes visited by the DFS in various kinds of data structures:o Preorder: Put the vertex on a queue before the dfs()

recursive call

o Postorder: Put the vertex on a queue after the dfs() recursive call

o Reverse postorder: Put the vertex on a stack after the dfs() recursive call

Implementing the Orders


boolean[] marked; // to prevent cycles

Queue<Integer> pre = new Queue<Integer>(); // to store vertices in preorder

Queue<Integer> post = new Queue<Integer>(); // to store vertices in postorder

Stack<Integer> reversePost = new Stack<Integer>(); // to store vertices in reverse postorder

void dfsVisit(Digraph graph){ for (int v : graph.allVerts()) // for multiple 'roots' if (!marked[v]) dfs(graph, v);}

dfsVisit() Code


void dfs(Digraph graph, int v){ pre.add(v); marked[v] = true; for (int w : graph.adj(v)) // v --> w if (!marked[w]) dfs(graph, w); post.add(v); reversePost.push(v);}


• DFS has running time: O(V + E)o dfsVisit() visits every vertex: O(V)o all the calls to dfs() will result in visiting the adjacent

edges to all the vertices, which are all the edges: O(E)

• Since topological sort uses DFS, its running time is also O(V + E)

Running Time



void dagSP(Digraph graph, int start){ for (int v : graph.allVerts()) distTo[v] = Double.POSITIVE_INFINITY; distTo[start] = 0;

Graph gSort = sortTopological(graph); // based on dfsVisit() for (int v : gSort.allVerts()) relax(gSort, v);}

DAG Shortest Path Code


void relax(Digraph graph, int u) // a simpler version than Dijkstra{ for (DirectedEdge e : graph.adj(u)) { int v = e.to(); // edge e is u --> v if (distTo[v] > distTo[u] + e.weight()) { distTo[v] = distTo[u] + e.weight(); edgeTo[v] = e; } } }


Example in Detail



• The topological sort running time: O(V+E)

• The for-loop visits every vertex: O(V)• All the calls to relax() will result in visiting the

adjacent edges to all the vertices, which are all the edges: O(E)

• So the total is O(V+E) + O(V) + O(E) ≈ O(V+E)

Running Time of dagSP()


5. Summary of Algorithms

key benefit

Documents

Algorithm Design and Analysis (ADA)