Minimum Spanning Trees

• Definition

• Two properties of MST’s

• Prim and Kruskal’s Algorithm– Proofs of correctness

• Boruvka’s algorithm

• Verifying an MST

• Randomized algorithm in linear time

Problem Definition

• Input– Weighted, connected undirected graph G=(V,E)

• Weight (length) function w on each edge e in E

• Task– Compute a spanning tree of G of minimum total weight

• Spanning tree– If there are n nodes in G, a spanning tree consists of n-1

edges such that no cycles are formed

Example

A B C D

E F G

1

3

8

2

4

7

5

69Input:

A B C D

E F G

1

3

8

2

4

7

5

69Output:

Two Properties of MST’s

• Cycle Property: For any cycle C in a graph, the heaviest edge in C does not appear in the minimum spanning tree– Used to rule edges out

• Cut Property: For any proper non-empty subset X of the vertices, the lightest edge with exactly one endpoint in X belongs to the minimum spanning forest– Used to rule edges in

Cycle Property Illustration/Proof

• Proof by contradiction: – Suppose T is an MST with such an edge e.

– Derive a contradiction showing that T is not an MST

A B C D

E F G

1

3

8

2

4

7

5

69

A B C D

E F G

1

3

8

2

4

7

5

69

Cut Property Illustration/Proof

• Proof by contradiction:– Suppose T is an MST without such an edge e.

– Derive a contradiction showing that T is not an MST

A B C D

E F G

1

3

8

2

4

7

5

69

A B C D

E F G

1

3

8

2

4

7

5

69

Three Classic Greedy Algorithms

• Kruskal’s approach– Select the minimum weight edge that does not form a

cycle

• Prim’s approach– Choose an arbitrary start node v– At any point in time, we have connected component N

containing v and other nodes V-N– Choose the minimum weight edge from N to V-N

• Boruvka’s approach– Prim “in parallel”

• Illustrate the execution of Kruskal and Prim’s algorithms on the following input graph. Let D be the arbitrary start node for Prim’s algorithm.

Example

A B C D

E F G

1

3

8

2

4

7

5

69

Prim Implementation

• Use a priority queue to organize nodes in V-N to facilitate finding closest node.– Extract-Min operation– Decrease-key operation

• Describe how we could implement Prim using a priority queue.

A B C D

E F G

1

3

8

2

4

7

5

69

Running Time of Prim

• How many extract-min operations will we perform?

• How many decrease-key operations will we perform?

• How much time to build initial priority queue?• Given binary heap implementation, what is the

running time?• Fibonacci heap:

– Decrease-key drops to O(1) amortized time

Kruskal Implementation

• Kruskal’s Algorithm– Adding edge (u,v) to the current set of edges T

forms a cycle if and only if u and v are in the same connected component.

A B C D

E F G

1

3

8

2

4

7

5

69

Disjoint Set Data Stucture (Ch 21)

• Given a universe U of objects– Maintain a collection of sets Si such that

• Unioni Si = U• Si intersect Sj is empty

– Find-set(x): Returns set Si that contains x– Merge(Si, Sj): Returns new set Sk = Si union Sj

• Describe how we can implement cycle detection with this data structure.

A B C D

E F G

1

3

8

2

4

7

5

69

Running Time of Kruskal

• How many merges will we perform?• How many Find-set operations will we

perform?– Each can be implemented in amortized (V) time

where is a very slow growing function.

• What other operations do we need to implement?

• Overall running time?

Proofs of Correctness

• Why do we know each edge that is added in Kruskal’s algorithm is part of an MST?

• Why do we know that each edge added in Prim’s algorithm is part of an MST?

Boruvka’s Algorithm

• Prim “in parallel”• Boruvka Step:

– We have a graph of vertices– For each v in V, select the minimum weight edge

connected to v– Update

• Contract all selected edges, replacing each connected component by a single vertex

• Delete loops, and keep only the lowest weight edge in a multi-edge

• Run Boruvka steps until we have a single node

Boruvka Step IllustrationA B C D

E F G

1

3

8

2

9

7

5

64

6A B C D

E F G

1

3

8

2

9

7

54

A’ B’7

Boruvka’s Algorithm Analysis

• Correctness:– How can we verify that each edge we add is

part of an MST?

• Running time:– What is the running time of a Boruvka Step?– How many Boruvka steps must we implement

in the worst case?

Verification Problem

• Input– Weighted, connected undirected graph G=(V,E)

• Weight (length) function w on each edge e in E

– A spanning tree T of G

• Task– Answer yes/no if T is a minimum spanning tree for G

• Two key concepts– Decision problem: problem with yes/no answer– Verification: Is it easier to verify an answer as correct

as compared to generating an answer?

Key idea: T(u,v)

• Suppose we have a tree T• For each edge (u,v) not in T, let T(u,v) be the heaviest

edge on the (u,v) path in T• If w(u,v) > w(T(u,v)), then (u,v) should not be in T.

A B C D

E F G

1

3

8

2

4

7

5

69

• Consider edge (B,F) with weight 7.• T(B,F) = (C,G) which has weight 6.• (B,F) is appropriately not in T as w(B,F) > w(C,G).

Verification Algorithm

• For each edge (u,v) not in T, find T(u,v).

• Compare w (u,v) to w(T(u,v))

• If all are ok, then return yes, else return no.

• Running time?– O(E) time to perform comparisons– Sophisticated techniques to find all the T(u,v)

in O(E) time.

Randomized Algorithm

1. Run Baruvka’s Step 2 timesa) If there were originally n nodes, how many in reduced problem

2. Random Samplinga) Make a smaller subgraph by choosing each edge with

probability ½b) Recursively compute minimum spanning tree (or perhaps forest)

F on this reduced graphc) Use verification algorithm to eliminate any edges (u,v) not in F

whose weight is more than weight of F(u,v).

3. Apply algorithm recursively to the remaining graph to compute a spanning tree T’

a) Knit together tree from step 1 with F’ to form spanning tree

Graph Go

Step 1: Run 2 Baruvka Steps:Contracted Graph G1 List of edges E1

Step 2: Construct G2 by sampling ½ edges

Step 2: Recursively Construct T2 for graph G2

Step 2: Use T2 to identify edges E2 that cannot be in MST for G1

Graph G3 = G1 – E2

Step 3: Recursively Construct T3 for graph G3

Step 3: Return T3 melded with edges E1 as final tree T4

Tree T4 = T3 union E1

Visualization

Graph Go

Step 1: Run 2 Baruvka Steps:Contracted Graph G1 List of edges E1

Step 2: Construct G2 by sampling ½ edges

Step 2: Recursively Construct T2 for graph G2

Step 2: Use T2 to identify edges E2 that cannot be in MST for G1

Graph G3 = G1 – E2

Step 3: Recursively Construct T3 for graph G3

Step 3: Return T3 melded with edges E1 as final tree T4

Tree T4 = T3 union E1

Analysis Intuition

G2 has at most ½ the edges of G0

G3 edges bounded by ½ nodes of G0