242-535 ADA: 10. MSTs 1

• Objectiveo look at two algorithms for finding mimimum

spanning trees (MSTs) over graphs• Prim's algorithm, Kruskal's algorithm

Algorithm Design and Analysis

(ADA)242-535, Semester 1 2014-2015

10. Minimum Spanning

Trees (MSTs)

242-535 ADA: 10. MSTs 2

1. Minimum Spanning Tree2. Prim's Algorithm3. Kruskal's Algorithm4. Difference between Prim and

Kruskal

Overview

242-535 ADA: 10. MSTs 3

• A minimum spanning tree T is a subgraph of a weighted graph G which contains all the verticies of G and whose edges have the minimum summed weight.

• Example weighted graph G:

1. Minimum Spanning Tree

A B

CD

EF

3

4

5

163

2

6

2

242-535 ADA: 10. MSTs 4

• A minimum spanning tree (weight = 12):

• A non-minimum spanning tree (weight = 20):

A B

CD

EF

3

4

5

163

2

2

6

A B

CD

EF

3

4

5

163

2

6

2

242-535 ADA: 10. MSTs 5

Typical MST Applications

242-535 ADA: 10. MSTs 6

• MSTs have the optimal substructure property: an optimal tree is composed of optimal subtrees.

• This can be seen by considering how to break a MST into parts:o Let T be an MST of G with an edge (u,v)o Removing (u,v) splits T into two trees T1 and T2

o Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of G2 = (V2,E2) (

• V1 and V2 do not share any vertices

o Proof: w(T) = w(u,v) + w(T1) + w(T2)

• There can’t be a better tree than T1 or T2, or T would be suboptimal

MST Optimality

242-535 ADA: 10. MSTs 7

• Both dynamic programming (DP) and greedy algorithms can be used on problems that exhibit optimality.

• What's the difference between the two approaches?

• DP: use when the problem is optimal and there are repeating sub-problems

• Greedy: use when the problem is optimal, and each sub-problem can be solved without combining/examining smaller sub-sub-problemso this is called the greedy-choice property

Optimality and Coding

242-535 ADA: 10. MSTs 8

• DP solves sub-problems bottom-up since the current sub-problem may depend on sub-sub-problems.

• Greedy algorithms usually execute top-down, since a sub-problem can be solved without using sub-sub-problem solutions.

• In a greedy algorithm, we make whatever choice seems best at the moment and then solve any other sub-problems arising after the choice is made.

242-535 ADA: 10. MSTs 9

• Since a MST is an optimal data structure, it is possible to use dynamic programming (DP) techniqueso e.g. memoization, bottom-up execution

• In fact, both the MST algorithms in this part are greedyo at each iteration the choice of how to grow the MST only

depends on the current state of the MST, not earlier sub-states or simpler versions

DP, Greed and MST

242-535 ADA: 10. MSTs 10

• Prim's algorithm finds a minimum spanning tree T by iteratively adding edges to T.

• At each iteration, a minimum-weight edge is added that does not create a cycle in the current T.

• The new edge must be connected to a vertex which is already in T.

• The algorithm can stop after |V|-1 edges have been added to T.

2. Prim's Algorithm

242-535 ADA: 10. MSTs 11

• Tree prim(Graph G, int numVerts){ Tree T = anyVert(G); for i = 1 to numVerts-1 { Edge e = select an edge of minimum weight that is connected to a vertex already in T and does not form a

cycle in T; T = T + e ; } return T}

Simple Pseudocode

242-535 ADA: 10. MSTs 12

• For the graph G.• 1) Add any vertex to T

o e.g A, T = {A}

• 2) Examine all the edges leaving {A} and add the vertex with the smallest weight.o edge weight

(A,B) 4(A,C) 2(A,E) 3

o add edge (A,C), T becomes {A,C}

Informal Algorithm

continued

A B

CD

EF

3

4

5

163

2

6

2

242-535 ADA: 10. MSTs 13

• 3) Examine all the edges leaving {A,C} and add the vertex with the smallest weight.o edge weight edge weight

(A,B) 4 (C,D) 1(A,E) 3 (C,E) 6(C,F) 3

o add edge (C,D), T becomes {A,C,D}

continued

242-535 ADA: 10. MSTs 14

• 4) Examine all the edges leaving {A,C,D} and add the vertex with the smallest weight.o edge weight edge weight

(A,B) 4 (D,B) 5(A,E) 3 (C,E) 6(C,F) 3 (D,F) 6

o add edge (A,E) or (C,F), it does not mattero add edge (A,E), T becomes {A,C,D,E}

continued

242-535 ADA: 10. MSTs 15

• 5) Examine all the edges leaving {A,C,D,E} and add the vertex with the smallest weight.o edge weight edge weight

(A,B) 4 (D,B) 5(C,F) 3 (D,F) 6(E,F) 2

o add edge (E,F), T becomes {A,C,D,E,F}

continued

242-535 ADA: 10. MSTs 16

• 6) Examine all the edges leaving {A,C,D,E,F} and add the vertex with the smallest weight.o edge weight edge weight

(A,B) 4 (D,B) 5

o add edge (A,B), T becomes {A,B,C,D,E,F}

• All the verticies of G are now in T, so we stop.

continued

242-535 ADA: 10. MSTs 17

• Resulting minimum spanning tree (weight = 12):

A B

CD

EF

3

4

5

163

2

2

6

242-535 ADA: 10. MSTs 18

• Build up the MST (black tree) by repeatedly adding the minimum crossing edge (thick red line) to ito a crossing edge connects

a node in the MST to the rest of the graph.

• Ignore edges that form cycles (grey lines).

Prim's Algorithm Graphically

242-535 ADA: 10. MSTs 19

• The following weighted graph:

Example 2

16 26

38

28

58

2932

34

36

37

93

40

52

35

1719

242-535 ADA: 10. MSTs 20

Building the MST1

2

3

242-535 ADA: 10. MSTs 21

4

5

6

242-535 ADA: 10. MSTs 22

7

8

Finished

242-535 ADA: 10. MSTs 23

boolean marked[]; // for storing the vertices in the MST

Queue<Edge> mst; // for storing the edges in the MST

MinPriQueue<Edge> pq; // for storing the crossing (and ineligible) edges

Prim's in More Detail

242-535 ADA: 10. MSTs 24

void prims(Graph graph, int start){ visit(graph, start); while (!pq.isEmpty()) { Edge e = pq.remove(); // get lowest weight edge int v = e.either(); // (v,w) are the ends of the edge e int w = e.other(v); if (!marked[v] || !marked[w]) { mst.add(e); // add edge to MST queue if (!marked[v]) // visit vertex v or w visit(graph, v); if (!marked[w]) visit(graph, w); } }} // end of prims()

242-535 ADA: 10. MSTs 25

void visit(Graph graph, int v)/* Mark v and add to priority queue all the edges from v to unmarked vertices */{ marked[v] = true; for (Edge e : graph.adj(v)) if (!marked[e.other(v)]) pq.add(e);}

242-535 ADA: 10. MSTs 26

• The running time depends on the implementation of the minimum priority queue, which uses a minimum heap.o An add() takes time O(log n), and remove is O(1)o At most E edges are added to the priority queue and at

most E are removed.

• The algorithm has a worst case running time of O(E log E).

Running Time

242-535 ADA: 10. MSTs 27

• Krukal's algorithm finds a minimum spanning tree T by iteratively adding edges to T.

• At each iteration, a minimum-weight edge is added that does not create a cycle in the current T.

• The new edge can come from anywhere in G.

• The algorithm can stop after |V|-1 edges have been added to T.

3. Kruskal's Algorithm

242-535 ADA: 10. MSTs 28

• Tree kruskal(Graph G, int numVerts){ Tree T = allVerts(G); for i = 1 to numVerts-1 { Edge e = select an edge of minimum weight

from G which does not form a cycle in T;

T = T + e ; } return T}

Simple Pseudocode

242-535 ADA: 10. MSTs 29

• Graph G:

Example 1

ab c d

ef g h

i j k l

2 3 13

4

3

4 2

1

3

2 3

4 3

3 1

3

iter edge 1 (c,d) 2 (k,l) 3 (b,f) 4 (c,g) 5 (a,b) 6 (f, j) 7 (b,c) 8 (j,k) 9 (g,h) 10 (i, j) 11 (a,e)

continued

242-535 ADA: 10. MSTs 30

• Minimum spanning tree (weight = 24):

ab c d

ef g h

i j k l

2 3 13

4

3

4 2

1

3

2 3

4 3

3 1

3

242-535 ADA: 10. MSTs 31

Example 2

next MSTedge

edgessorted byweight

black edgesare usedin MST

242-535 ADA: 10. MSTs 32

Queue<Edge> mst; // for storing the edges in the MST

MinPriQueue<Edge> pq; // for storing the all the edges from G

UnionFind uf; /* used to maintain disjoint sets of vertices: one for vertices in the MST, and other sets for nodes outside */

Kruskal's in More Detail

242-535 ADA: 10. MSTs 33

void kruskal(Graph graph){ pq.addAll(graph.edges()); // add all edges to pri queue uf.makeSets(graph.vertices()); // create disjoint sets of V's

while ((!pq.isEmpty()) && (mst.size() < graph.vertices().size()-1)) { Edge e = pq.remove(); // get lowest weight edge int v = e.either(); // (v,w) are the ends of the edge e int w = e.other(v); if (!uf.isConnected(v, w)) { // are v and w not in same set? // only one of v or w must be in the MST set uf.union(v, w); // combine v and w's sets mst.add(e); // add edge to mst } }} // end of kruskal()

242-535 ADA: 10. MSTs 34

• As with Prim's algorithm, the running time depends on the implementation of the minimum priority queue, which uses a minimum heap.

• The algorithm also uses a Union-find data structureo E find() and isConnected() calls, which are both O(log n)

• The algorithm has a worst case running time of O(E log E) – same as Prim'so in practice, Prim's algorithm is usually faster than

Kruskal's

Running Time

242-535 ADA: 10. MSTs 35

• A disjoint-set data structure keeps track of a set of elements split into disjoint (non-overlapping) subsets.

• Union-find consists of two main operations:o find(): report which subset a particular element is ino union(): join two subsets into a single subseto others: makeSets(), isConnected(), etc.

Union-find

242-535 ADA: 10. MSTs 36

242-535 ADA: 10. MSTs 37

• Prim's algorithm chooses an edge that must be connected to a vertex in the minimum spanning tree T.

• Kruskal's algorithm chooses an edge from G that may or may not be connected to a vertex in T.

4. Difference between Prim and

Kruskal