Lecture 18: Minimum Spanning TreesMinimum Spanning Trees

JFK

BOS

ORDPVD

8672704

187849

1447401846 621 J

LAXDFW

SFO BWI

1258

1391

184

946

1090

1846 621

802

1464

1235

337

MIA1121

2342

Courtesy of Goodrich, Tamassia and Olga Veksler Instructor: Yuzhen Xie

O tlineOutlineMinimum Spanning TreesMinimum Spanning TreesPrim’s algorithmK k l’ l ithKruskal’s algorithm

2

Minimum Spanning Trees ORD 10

Spanning subgraphSubgraph of a graph Gcontaining all the vertices of G

PITDEN

1

96 7containing all the vertices of G

Spanning tree (for connected graph)Spanning subgraph that is itself a (free) tree that it’s connected

STLDCA

93

4a (free) tree, that it s connected and has no cycles

Unless graph is a tree itself, many spanning trees are ATLDFW

8 25

many spanning trees are possible

Any graph which is a tree has exactly n – 1 edges,

ATLDFW

Minimum spanning tree (MST)Spanning tree of a weightedy g ,

where n is the number of graph vertices.

Any connected graph with n – 1 edges and n vertices is

Spanning tree of a weighted graph with minimum total edge weight

ApplicationsC i i kn – 1 edges and n vertices is

a tree graph Communications networksTransportation networks

Prim’s Algorithm: Basic IdeaGreedily grow MST from some start vertex sPartition vertices into green and blue clouds

Blue cloud starts with one vertex ss

5 8Blue cloud starts with one vertex sGreen cloud has all the other graph vertices

Iterate until the blue cloud has all the vertices:

56 8

vertices:find the smallest weight edge across the green and blue partitionMark this edge as part of the MSTMark this edge as part of the MSTmove the green endpoint of this edge into the blue cloud s

56 8At the end of the algorithm: 6 8At the end of the algorithm:

blue vertices are connected by marked edgesthere are exactly n - 1 marked edges, so marked edges (and all vertices) form a spanning treeIs this a minimum spanning tree, however?

YES, we will show this later

Prim’s Algorithm: Efficient Implementation

At each iteration, we could search over all edges for the smallest edge which crosses the green/blue partition

This is not as efficient as possibleThis is not as efficient as possible

more efficient approach is similar to Dijkstra’s algorithm We store with each vertex v a label d[v] = the smallest weight f d ti t t i bl l dof an edge connecting v to a vertex in blue cloud At each iteration:

We add to blue cloud vertex u 5

d[u]= 2outside the blue cloud with smallest distance labelWe update the label of any vertex adjacent to

s5

8uw

vd[v]= ∞

d[w]= ∞

2

vertex w adjacent to uif d[w] > weight(u,w) set d[w] = weight(u,w)this is the new distance of

d[v]= ∞

5d[ ]this is the new distance of vertex w to the blue cloud

s5

8

uwvd[v]= 3

d[w]= 5

Prim’s Algorithm: Pseudo-CodeQ ← new priority queue

A priority queue stores the vertices outside the blue cloud

VISITED vertices are in

Q ← new priority queues ← a vertex of Gfor all v ∈ G.vertices()

if v = ssetDistance(v 0)VISITED vertices are in

blue cloudUNVISITED vertices are in green cloudKey: distance

setDistance(v, 0)else

setDistance(v, ∞)setParent(v, ∅)

k UNVISITEDKey: distanceElement: vertex

Locator-based methodsinsert(k e) returns a locator

mark v UNVISITEDl ← Q.insert(getDistance(v), v)setLocator(v,l)

while ¬Q.isEmpty()insert(k,e) returns a locator replaceKey(l,k) changes the key of an item at a locator

We store three labels with

u ← Q.removeMin() mark u VISITEDfor all e ∈ G.incidentEdges(u)

z ← G.opposite(u,e)each vertex:

DistanceParent edge in MSTLocator in priority queue

← G.opposite(u,e)if z is UNVISITED

r ← weight(e)if r < getDistance(z)

setDistance(z r)Locator in priority queue setDistance(z,r)setParent(z,u)Q.replaceKey(getLocator(z),r)

E ampleExampleD72∞

D727

B

CF

4

28

59

2

8 ∞B

CF

4

28

59

2

5 ∞

AE

8

7

38

0 7 AE

8

7

38

0 7

BD7

49

2

5 ∞

7

BD7

49

2

5 4

7

C

A

F

E

28

5

3

9

87

5 ∞C

F

E

28

5

3

9

8

5 4

7

A 70 7 AE

70 7

E ample (contd )Example (contd.)D72

7

BD

CF

74

28

59

2

5 4C

AE

8

7

38

0 7

D727

BD7

42

7

BD

CF

74

28

59

2

5 4

B

CF

4

28

5

3

9

8

5 4

AE

8

7

38

0 3A

E7

8

0 3

Prim’s Algorithm: Correctness ProofTheorem: After every iteration, the marked edges are a part of some MST

Proof (by contradiction):Suppose false. Let (u,v) be the first marked edge after which theorem is false

before adding (u,v), marked edges were a part of some MST T, after adding (u,v), marked edges are not part of any MST

vuwy w

tConsider blue cloud right before u was added to itIn T, there must be path between u and v, it does not include edge (u,v)g ( , )

Let w be first blue vertex after u on this path Let t be the green vertex before w on this pathGet T* from T by remove edge (w,t) adding (u,v)

T* is spanning tree; it has all marked edges and (u,v) weight of T* ≤ weight of T because (u,v) is the smallest weight edge out of the blue cloud

vuw

T* is a MST which has marked edges after marking (u,v)CONTRADICTION! t

Prim’s Algorithm: Complexity Analysis

Edge weights can be negative for Prim’s algorithm

Running time analysis is exactly like that for theRunning time analysis is exactly like that for the Dijkstra’s algorithm

AssumeAssumesetting/getting a label takes O(1) time, graph is represented by the adjacency list structure

Prim algorithm runs in O((n + m) log n) time provided the graph is represented by the adjacency list structure

10

Kruskal’s Algorithm g

A tree with n vertices must Algorithm KruskalMST(G)

have exactly n - 1 edges

A priority queue stores the edges outside the cloud

for each vertex V in G dodefine a Cloud(v) of {v}

let Q be a priority queueInsert all edges into Q using theirg

Key: weightElement: edge

At the end of the algorithm

Insert all edges into Q using their weights as the keyT ∅while T has fewer than n-1 edges doAt the end of the algorithm

We are left with one cloud that encompasses the MSTA tree T which is our MST

gedge e = Q.removeMin()Let u, v be the endpoints of eif Cloud(v) ≠ Cloud(u) then

Add edge e to TgMerge Cloud(v) and Cloud(u)

return T

11

Data Structure for Kruskal Algortihm The algorithm maintains a forest of treesAn edge is accepted if it connects distinct treesWe need a data structure that maintains a partition, i.e., a collection of disjoint sets, with the operations:-find(u): return the set storing ufind(u): return the set storing u-union(u,v): replace the sets storing u and v with their union

12

Representation of a PartitionPartition

Each set is stored in a sequenceqEach element has a reference back to the set

operation find(u) takes O(1) time, and returns the set of which u is a memberwhich u is a member.in operation union(u,v), we move the elements of the smaller set to the sequence of the larger set and update their referencestheir referencesthe time for operation union(u,v) is min(nu,nv), where nuand nv are the sizes of the sets storing u and v

Whenever an element is processed, it goes into a set of size at least double, hence each element is processed at most log n times

13

p g

Kruskal’s Algorithm for each vertex V in G do

define a Cloud(v) of {v}nn

let Q be a priority queue.Insert all edges into Q using their weights as the key

m log n

T ∅while T has fewer than n-1 edges edge e = Q.removeMin()

Let u v be the endpoints of e

mm log nm

1

Let u, v be the endpoints of eif Cloud(v) ≠ Cloud(u) then

Add edge e to TMerge Cloud(v) and Cloud(u)

return T

mmn-1

1account separately

return T 1

Each element is moved at most logn timesIt takes constant amount of time to move 1 elementTh l t ti t i i l

14

There are n elements, time spent merging is n lognTotal running time is n logn + 3n + m logn + 3m +1which is O( (m+n ) logn )

Kruskal Example

BOS8672704

ORDPVD

187849

JFK1258

144740

184

1846 621

802SFO BWI1391

1090

802

1464337

LAXDFW 946

1121

1235

15

MIA2342

ExampleBOS867

2704

Example

ORDPVD

187849

144740JFK

SFO BWI

1258

144

184

1846 621

802

DFW

SFO BWI1391

9461090

1464337

MIA

LAXDFW 946

1121

1235

16

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

740JFK

1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

17

MIA2342

ExampleExampleBOS867

2704

JFK

ORDPVD

187849

144740JFK

SFO BWI

1258184

1846 621

802

DFW

BWI1391

9461090

1464337

MIA

LAXDFW 946

1121

1235

18

MIA2342

ExampleExampleBOS867

2704

849

JFK

ORDPVD

187849

1447401846 621

SFO BWI

1258

1391

184

1846 621

802

LAXDFW

1391

9461090

1464

1235

337

MIA

LAX

1121

1235

19

2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO BWI

1258

144

184

1846 621

802

DFW

SFO BWI1391

9461090

1464337

MIA

LAXDFW 946

1121

1235

20

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO BWI

1258

144

184

1846 621

802

DFW

SFO BWI1391

9461090

1464337

MIA

LAXDFW 946

1121

1235

21

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

22

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

23

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

24

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

740JFK

1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

25

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO BWI

1258

1447 0

184

1846 621

802

DFW

SFO BWI1391

9461090

1464337

MIA

LAXDFW 946

1121

1235

26

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

27

MIA2342

ExampleExampleBOS867

2704

ORDPVD

187849

144740JFK

SFO1258

144740

184

1846 621

802SFO BWI1391

10901464

337

LAXDFW 946

1121

1235

28

MIA2342