CS5234 Combinatorial and Graph Algorithms

CS5234Combinatorial and Graph Algorithms

Welcome!

AdministrativeProblem Sets

Programming Assignment

Exams

Problem set 1 duePS1: Hand it in now!

Problem set 2 released tonightPS2: Submit next week

IVLE (optional, electronic submission)Soon: Once registration completes…

CS5234 Overview

Problem set gradingDistributed grading scheme:

Once registration completes, there will be an

IVLE survey where you choose a week.

Grading supervised (and verified) by the TA.

CS5234 Overview

How to draw pictures?By hand: Either submit hardcopy, or scan, or take a picture with your phone!

Or use a tablet / iPad…Digitally: 1. xfig (ugh) 2. OmniGraffle (mac) 3. Powerpoint (hmmm)4. ???

CS5234 Overview

Programming assignment(s):Berth assignment problem: Last 3-4 weeks of the semester.

Very hard algorithmic problem (related to graph partitioning).

Design the best solution you can!Must know or learn: C++

More details later…

CS5234 Overview

Mid-term examOctober 8 In class

Final exam

November 19 Reading WeekExams will be graded and returned.

CS5234 Overview

Quick ReviewVertex Cover

Definition:Given: graph G = (V,E)Find: smallest cover C ⊆ V such that every edge e ∈ E is covered by some node v ∈ V.

Challenge:NP-complete: No polynomial time algorithm

unless P = NP.

Vertex Cover

Example: (Suboptimal) cover (size 9)

Vertex Cover

Example: Optimal cover? (size 6)

Vertex Cover

Greedy approximation algorithm:repeat: until every edge is covered

1. Let e = (u,v) be an uncovered edge.2. Add u and v to the cover.

Details:

Graph representation: Sorted adjacency list…

Vertex Cover

Analysis:1. For every matching M: OPT ≥ |M|.2. The cover C contains a matching of size |C|/2.3. Therefore: OPT ≥ |C|/2.

Conclusion: |C| ≤ 2OPT 2-approximation algorithm

Vertex Cover

Today’s PlanVertex Cover (Review)

Set Cover (Review + Analysis)

Steiner Trees

Definition:Given: set X

subsets S1, S2, …, Sm where Sj ⊆ XFind: smallest collection I ⊆ {1, …, m}

such that:

Challenge:NP-complete: No polynomial time algorithm

unless P = NP.

Set Cover

Example:Alice: C, C++Bob: C++, JavaCollin: C++, Ruby, PythonDave: C, Java

Choose a good team:Collin and Dave: optimal solution.

Set CoverX = {C, C++, Java, Ruby, Python}

Set Cover

set that contains the mostuncovered elements

Example:

Set CoverSet # uncoveredS1 4S2 6S3 3S4 5S5 4

1. Choose S2.

Example:


1. Choose S2.2. Choose S3.

Example:


1. Choose S2.2. Choose S3.3. Choose S4.

Example:


1. Choose S2.2. Choose S3.3. Choose S4.4. Choose S1

Example:


Output:S1, S2, S3, S4

OPT:S1, S4, S5

To get a good approximation, show a lower bound on OPT.

– “OPT has to be AT LEAST this large.”– E.g., “OPT ≥ |M|”

Set Cover

Example:


Note:12 elements≤ 6 elements per set

OPT ≥ 2

Example:


Note:6 uncovered elements≤ 3 elements per set

OPT ≥ 2

Example:


General rule:k uncovered elements≤ t elements per set

OPT ≥ (k / t)

Assume GREEDY covers elements in order:x1, x2, x3, x4, x5, x6, x7, x8, …, xn

xj = the jth item covered

Notation


Notation

S3 covers 3 items

S6 covers 2 new items

S1 covers 2 new items

S4 covers 1 new item


Notation

3 2 2 1


c1, c2, c3, c4, c5, c6, c7, c8, …, cn

3, 3, 3, 2, 2, 2, 2, 1, …,

cj = number of items covered in same step as xj

Notation

3 2 2 1


– cost(x1) = 1/3– cost(x2) = 1/3– cost(x3) = 1/3– cost(x4) = 1/2– cost(x5) = 1/2 – …

cost(xj) = 1/cj = amount payed to cover xj

Notation

3 2 2 1


cost(xj) = 1/cj = amount payed to cover xj

Notation

3 2 2 1

Example:


Note:6 uncovered elements≤ 3 elements per set

OPT ≥ 2

Assume (x1, x2, …, xj-1) are covered:– How does OPT cover (xj, xj+1, …, xn)?

Analysis


Fact: No set covers more than c(xj) elements in

the set (xj, xj+1, …, xn).

Why? If it did, GREEDY would have chosen it!

(Tip: use the fact that algorithm is GREEDY.)

Analysis


Analysis

As before:n – j + 1 uncovered elements≤ c(xj) elements per set

OPT ≥ (n – j + 1) / c(xj) = (n – j + 1)cost(xj)


Analysis

As before:n – j + 1 uncovered elements≤ c(xj) elements per set

OPT ≥ (n – j + 1) / c(xj) = (n – j + 1)cost(xj)

cost(xj) ≤ OPT / (n – j + 1)


Analysis

3 2 2 1

Algebra

Conclusion:

Theorem:Greedy-Set-Cover is an O(log n) approximation algorithm for set cover.

Set Cover

Today’s PlanVertex Cover (Review)

Set Cover (Review + Analysis)

Steiner Trees

Problem Statement:Given: 2d (Euclidean) map &

set of points of interestFind: shortest set of roads connecting all points

A Warm-Up Problem



A Warm-Up Problem

Algorithm:Compute: for every pair (u,v): distance(u,v)Build: complete graph G = (V, E)

where w(u,v) = distance(u,v)Find: minimum spanning tree of G.

A Warm-Up Problem

Definition: a spanning tree is an acyclic subset of the edges that connects all nodes

Spanning Tree

9

13

4

1 9

5

2

Weight: 32

8

Definition: a spanning tree with minimum weight

Minimum Spanning Tree

9

13

4

1 9

5

28

Definition: a spanning tree with minimum weight

Minimum Spanning Tree

9

13

4

1 9

5

28

Weight: 20

Property 1: No cycles

Properties of MST

Property 2: If you cut an MST, the two pieces are both MSTs.

Properties of MST

Property 3: Cycle propertyFor every cycle, the maximum weight edge is not in the MST.

Properties of MST

max-weight edge on cycle

Property 4: Cut propertyFor every partition of the nodes, the minimum weight edge across the cut is in the MST.

Properties of MST

min-weight edge on cut

Prim’s Algorithm. (Jarnik 1930, Dijkstra 1957, Prim 1959)Basic idea:

– S : set of nodes connected by blue edges.– Initially: S = {A}– Repeat:

• Identify cut: {S, V–S}• Find minimum weight edge on cut.• Add new node to S.

Prim’s Algorithm

H

D

A C

F

G B

E

79

4

5

8

3

15

1210

2

11

13

20

1

16

6

Kruskal’s Algorithm. (Kruskal 1956)

Basic idea:– Sort edges by weight.– Consider edges in ascending order:

• If both endpoints are in the same blue tree, then color the edge red.

• Otherwise, color the edge blue.

Data structure:– Union-Find– Connect two nodes if they

are in the same blue tree.

Kruskal’s Algorithm

H

D

A C

F

G B

E

79

4

5

8

3

15

12

2

11

13

20

16

10

16

Algorithm: O(n2 log n)Compute: for every pair (u,v): distance(u,v)Build: complete graph G = (V, E)

where w(u,v) = distance(u,v)Find: minimum spanning tree of G.

A Warm-Up Problem



Try again…



Thinking outside the box…

Euclidean Steiner Tree:Given: 2d (Euclidean) map &

set of points of interest (POI)Find: set of extra points

shortest set of links connecting all POI

Thinking outside the box…

Metric Steiner Tree:Given: set of required points R

set of optional points Sdistance metric d(., .)

Steiner Tree Problems

u

v

w

Metric Steiner Tree:Given: set of required points R

set of optional points Sdistance metric d(., .)

Find: tree T = (V,E) : R ⊆ V ⊆ (S∪R)• Tree includes all required points.• Tree may include some optional points.

cost of tree is minimized:


General Steiner Tree:Given: set of required points R

set of optional points Sset of edges Eedge weights w(.)


cost of tree is minimized


Three variants:Euclidean Steiner Tree: points in a 2d-planeMetric Steiner Tree: distances form (any) metricGeneral Steiner Tree: arbitrary distances

All three variants are NP-hard.– No polynomial time solutions (unless P =

NP).– Find a good approximation algorithm.


Proposed Algorithm:Step 1: Construct a graph G from points R.

Ignore points S.

Step 2: Find an MST of G.

Question:Is an MST a good approximation of a Steiner Tree?


Challenge:Find the worst example you can for the proposed algorithm.– Euclidean– Metric– General

Worst maximize(cost(MST) / Steiner_OPT).


Euclidean Steiner Tree

Given: A set R of points in the planeFind: Minimum-cost tree spanning R

1

1Cost = 2 Steiner PointCost = 31

1

1

Euclidean metric

Conjecture: the worst-case ratio of (MST-to-Steiner) is .

Metric Steiner Tree Given: Set R and S of points, and a distance metric

Find: Minimum-cost tree spanning R

10 10

10

5

55

MST: 20Steiner Tree: 15Ratio: 1.33

10

10

10

5

55

10

5


10

10

10

5

55

10

5

10

5


Generalize: n-gonMST: 10(n-1)Steiner Tree: 5nRatio: 2(1 – 1/n)

General Steiner Tree Given: Set R and S of points, and a set of weighted edges

Find: Minimum-cost tree spanning R

3000 3000

3000

1

11

MST: 6000Steiner Tree: 3Ratio: 2000

Conclusion:MST is NOT a good approximation for a

general Steiner Tree.

Today’s PlanEuclidean Steiner Tree (skip)

(left as an exercise)

Metric Steiner TreeShow that MST is a 2-approximation.

General Steiner TreesReduce to Metric Steiner Tree.

To get a good approximation, show a lower bound on OPT.

– “OPT has to be AT LEAST this large.”– E.g., “OPT ≥ |M|”

Metric Steiner Tree

Given:– set of required points R– set of optional points S– distance metric d(., .)

Define:– T = (V, E) be the optimal (minimum)

Steiner Tree.

Notation

Consider a DFS traversal of T:(x0, x1, x2, x3, …, xm) where x0 = xm

Example:

Build a cycle…

1

11

1

1

1

11

All other pairs are distance 2.


Example:

Build a cycle…

g h e

b

a

d f

211

111

1

c


Example:

Build a cycle…

g h e

b

a

d f

211

111

1

c

DFS: a g d g f g a h c h b h a e a

Consider a DFS traversal of T:Each edge is included in the DFS traversal twice.cost(DFS) = 2cost(T) = 2OPT

Build a cycle…

g h e

b

a

d f

211

111 1

c


Example:cost(T) = 8cost(DFS) = 16

Example:Replace (a g d) with (a d)

Shortcut Steiner nodes…

g h e

b

a

d f

211

111 1

c


NEW: a d g f g a h c h b h a e a

Triangle Inequality:d(a,d) ≤ d(a,g) + d(g,d)

Hint: use the fact thatdistance is a metric.

Example:Replace (a g d) with (a d)


g h e

b

a

d f

211

111 1

c


NEW: a d g f g a h c h b h a e a

Conclusion:cost(NEW) ≤ cost(DFS)


Example:Replace (d g f) with (d f)


g h e

b

a

d f

211

111 1

c


NEW: a d f g a h c h b h a e a



Example:Continue until done…


g h e

b

a

d f

211

111 1

c


NEW: a d f a c b a e a


Example:Replace (f a c) with (f c)

Remove repeats…

g h e

b

a

d f

211

111 1

c


NEW2: a d f c b a e a

Conclusion:cost(NEW2) ≤ cost(DFS)

Example:Replace (b a e) with (b e)

Remove repeats…

g h e

b

a

d f

211

111 1

c


NEW2: a d f c b e a

Conclusion:cost(NEW2) ≤ cost(DFS)

Example:Final: a d f c b e a11 = cost(Final) ≤ cost(DFS) = 16 = 2OPT

Remove repeats…

g h e

b

a

d f

211

111 1

c

2

2 2 1

2

2

Example:Path: a d f c b e9= cost(Path) ≤ cost(DFS) = 16 = 2OPT

Break the cycle…

g h e

b

a

d f

211

111 1

c

2

2 2 1

2

Example:Spanning tree: a d f c b e9= cost(Spanning tree) ≤ cost(DFS) = 16 = 2OPT

Path is a spanning tree…

g h e

b

a

d f

211

111 1

c

2

2 2 1

2

Example:Spanning tree: a d f c b ecost(MST) ≤ cost(Spanning tree) ≤ 2OPT

Path is a spanning tree…

g h e

b

a

d f

211

111 1

c

2

2 2 1

2

Analysis:1. Let T be an optimal Steiner tree.2. Let DFS be a DFS-traversal of T.3. Let NoSteiner be DFS where we short-cut

past Steiner nodes. 4. Let Rcycle be NoSteiner where we short-

cut past repeated nodes.5. Let Path be Rcycle where we remove the

last edge.

Approximation Proof

Analysis:

1. cost(MST) ≤ cost (Path)

2. cost(Path) ≤ cost(Rcycle)

3. cost(Rcycle) ≤ cost(NoSteiner)

4. cost(NoSteiner) ≤ cost(DFS)

5. cost(DFS) ≤ 2cost(T) = 2OPT

Approximation Proof

By definition of MST.

Trivial.

By triangle inequality.

By construction.

Theorem:A minimum spanning tree is a 2-approximation of the optimal metric Steiner Tree.

Question: Is this analysis tight? 2(1 – 1/n) approximation?

Metric Steiner Tree








cost of tree is minimized


Problem:Minimum spanning tree of R is not a good approximation.

General Steiner Tree

Idea: Reduction1. Construct an instance of Metric Steiner Tree

from the input.2. Solve the Metric Steiner Tree problem (by

finding an MST).3. Translate the solution back.


Beware: Reductions are tricky for approximation algorithmsTypical example:

– Assume two problems ABC and XYZ– Function f : ABC XYZ– Function g : “solutions to XYZ” “solutions to

ABC”– Show:

If S is an optimal solution for f(A), then g(S) is an optimal solution for A.


Beware: Reductions are tricky for approximation algorithmsTypical example:


ABC XYZ

A f(A)

Sg(S)

f

ALGg

Beware: Reductions are tricky for approximation algorithmsProblem: ALG does not find optimal solution

– Function g may not preserve approximation ratio.


ABC XYZ

A f(A)

Sg(S)

f

ALGg


from the input.2. Solve the Metric Steiner Tree problem (by






Construction:1. Required and optional points stay the same.2. For every pair of points (u,v) define:

d(u,v) = distance of shortest path from u to v.

Construct a Metric

Example:– d(A,B) = 10– d(H,E) = 11– d(B,H) = 12– …

Construct a Metric

H

D

A C

F

G B

E

79

4

2

83

15

12

5

11

13

20

16

10

16

How do we find all the shortest paths?

Example:– d(A,B) = 10– d(H,E) = 11– d(B,H) = 12– …

Construct a Metric

H

D

A C

F

G B

E

79

4

2

83

15

12

5

11

13

20

16

10

16

How do we find all the shortest paths?• Dijkstra’s Algorithm : O(VE log V)• Floyd-Warshall : O(V3)

Claim:The function d(.,.) is a metric.

Construct a Metric


Usual properties: (don’t matter)• d(u, u) = 0• d(u, v) = d(v, u)• d(u, v) ≥ 0

Construct a Metric


Triangle Inequality: • Fix some (u, v, w).• d(u,w) ≤ d(u,v) + d(u,w)

– If not, find a shorter path from u to w by going u v w.• Shortest paths always satisfy

triangle inequality, by definition!

Construct a Metric

H

D

A C

F

G B

E

79

4

2

83

15

12

5

11

13

20

16

10

16


from the input (via shortest paths).2. Solve the Metric Steiner Tree problem (by



Given a Steiner Tree T’ for the metric problem:

1. For every edge (u,v) in T’, add the shortest path from (u v) to the graph G.

(Note, G may not be a tree.)

2. Find an MST of G.

Translate back…

Overview:1. Input: IN = (R, S, G, w)2. Construct: MET = (R, S, d) where d = shortest

paths.Show: OPT(MET) ≤ OPT(IN)

3. Solve: Let T’ be the approximately-optimal Steiner tree for MET. Fact: cost (T’) ≤ 2OPT(MET)

4. Translate: Convert T’ into a Steiner tree T for IN.Show: cost(T) ≤ cost(T’)

Conclude: cost(T) ≤ cost(T’) ≤ 2OPT(MET) ≤ 2OPT(IN)

Analysis

Construct: MET = (R, S, d) where d = shortest paths.Show: OPT(MET) ≤ OPT(IN)

1. Let T be an optimal Steiner tree for IN.2. Let T’ be the same tree in MET.3. cost(T’) ≤ cost(T)

• For every edge (u,v): d(u,v) ≤ w(u,v)• Hence the tree only costs less under the distance metric.

4. OPT(MET) ≤ cost(T’) ≤ cost(T) = OPT(IN)

Analysis






Analysis

Translate: Convert T’ into a Steiner tree T for IN.Show: cost(T) ≤ cost(T’)

1. Let G be the graph constructed from T’.2. cost(G) ≤ cost(T’)

• Every edge in T’ corresponds to a “shortest path.”• G is constructed by adding these paths.• Not always equal due to overlapping paths.

3. cost(T) ≤ cost(G)• Remove edges from G to find an MST.

Analysis






Analysis

Theorem:If A is a c-approximation algorithm for Metric Steiner Tree, then we can construct a c-approximation algorithm for General Steiner Tree.



Theorem:There exists a 2-approximation algorithm for General Steiner Tree.


Steiner Tree Example

H

D

A C

F

G B

E

79

4

2

8

3

15

12

5

11

13

20

16

10

16

Shortest Paths: (A,H) = 2(A,D) = 7(A,E) = 9(H,D) = 9(H,E) = 11(D,E) = 10


H

D

A C

F

G B

E

79

4

2

83

15

12

5

11

13

20

16

10

16

H

D

A

E

2

79

10

911

(Look ahead: ignore Steiner nodes.)

Run MST Algorithm


H

D

A C

F

G B

E

79

4

2

83

15

12

5

11

13

20

16

10

16

H

D

A

E

2

79

10

911

Convert solution back to original graph.

Steiner Tree Problem

H

D

A C

F

G B

E

79

4

2

83

15

12

5

11

13

20

16

10

16

H

D

A

E

2

79

10

911


Theorem:There exists a 2-approximation algorithm for General Steiner Tree.

Best known approximation: 1.55






Documents

CS5234 Combinatorial and Graph Algorithms