Optimization and Operations Research II · Optimization and Operations Research II Ricardo Fukasawa [email protected] Department ofCombinatorics andOptimization FacultyofMathematics

Optimization and Operations Research II

Ricardo [email protected]

Department of Combinatorics and OptimizationFaculty of MathematicsUniversity of Waterloo

Nov 26, 2014

R. Fukasawa (C&O) Opt. and OR II 1 / 26

Shortest path

Given G = (V ,E ) with costs ce > 0 for all e ∈ E , and two vertices s andt, find shortest s − t path.

Recall:

Vertices (circles): {A,B,C,D,E,F,G}

Edges (lines): {A,B}, {B,F}, {A,F},{B,C}, . . .

Path is a sequence of adjacentnon-repeated verticesExample: (A,B,F,G,E) is an A− E path

Cost of path is sum of cost of edgesExample: Path (A,B,F,G,E) has cost 18


Shortest path



Shortest path



Shortest path


One typical optimization problem

Used in Operations Research in many applications (often as asubroutine)


Application


Application

We want to have a machine every year from now (year 1) to the end ofyear 6. We want to decide when to keep the old one machine and payincreasing maintenance costs or when to replace it with a new one.Purchase costs at the beginning of each year:

Year1 2 3 4 5 6

Purchase cost 20 19 16 21 18 22Annual maintenance cost for a machine according to age:

Age0 1 2 3 4 5

Repair and maintenance cost 1 3 4 6 7 8How can one determine the best repair policy using the shortest pathproblem?


Shortest path

Given G = (V ,E ) with costs ce > 0 for all e ∈ E , and initial vertex s, findshortest s − v path for all v ∈ V .Dijkstra’s algorithm: Maintain a set S for which we know the shortests − v path of cost d(v) for all v ∈ S .

1 Let S = {s}, d(s) = 02 For all v /∈ S that are adjacent to some vertex in S do

1 Compute d ′(v) = min{d(u) + cuv : u ∈ S , {u, v} ∈ E}

3 Let w be the vertex with smallest d ′(v).

4 Let S ← S ∪ {w} and d(w) = d ′(w).Also, the shortest s − w path will be the shortest s − u path with wappended (where u was the vertex that determined d ′(w) )

5 If S 6= V , go to step 2.


Exercise

Apply Dijkstra’s algorithm starting with s


Exercise

Apply Dijkstra’s algorithm starting with A


Guaranteeing optimality

Question

Suppose you give me a path that you claim is the shortest possible.If I don’t trust you (or how you implemented Dijkstra’s algorithm), howcan you convince me that you are right (i.e. you found the optimalsolution)?



Question


This is in general a hard question to answer without looking throughEVERY single possible solution (or in this instance running Dijkstra’salgorithm myself).



Question



Who is the tallest high-school student in this room?



Question



Who is the tallest high-school student in this room?What is the tallest tree on earth?


Tallest tree on earth

369 feet (112m) high

Discovered in 2007 in Humboldt Redwoods State Park,California, USA. Called “Stratosphere Giant”


Tallest tree on earth

369 feet (112m) high

Discovered in 2007 in Humboldt Redwoods State Park,California, USA. Called “Stratosphere Giant”


After its short four-year reign as World’s Tallest,two hikers... came across a new stand of trees,taller than anyone had ever seen before. Thetallest of the tall is 379 feet 4 inches (116m), 10feet taller than the Giant. It’s now called “Hyper-ion” (2011)

Extracted from NPR.org


Moral of the story

It is much easier to prove that a solution is NOT optimal than to provethat it is.


Introducing barriers

Suppose I want to claim that the path (s, a, c , b, t) is the shortest s − tpathA barrier is defined by a set S ⊂ V such that s ∈ S and t /∈ S .Let δ(S) be the set of edges that go from a vertex in S to a vertex not inS .Let w(S) be a given weight associated to barrier S .



Suppose we define w(S) = 3 for S = {s}.Any path from s to t must cross the above barrier and have length at leastw(S).Shortest path must have length ≥ 3



Suppose we define w(S) = 2 for S = {s, a, c}.Any path from s to t must cross BOTH barriers and have length at leastthe sum of the weights of the barriers.Shortest path must have length ≥ 5



Suppose we define w(S) = 1 for S = {s, a, b, c , d}.Any path from s to t must cross ALL barriers above and have length atleast the sum of the weights of the barrier.Shortest path must have length ≥ 6



Suppose we define w(S) = 1 for S = {s, a}.Any path from s to t must cross ALL barriers above and have length atleast the sum of the weights of the barrier.Shortest path must have length ≥ 7


Shortest path

If the length of each edge is at least the sum of the weights of thebarriers that it crosses, then we have a set of “feasible barriers”


Shortest path


Let l(P) be the length of a path P and w(B) be the weight of a setof feasible barriers B .


Shortest path



We argued that:

l(P) ≥ w(B), for all paths P and feasible barriers B


Shortest path



We argued that:


Hence we have a min-max relationship:

min{l(P) : P is a path} ≥ max{w(B) : B is a set of feasible barriers}


Shortest path



We argued that:


Hence we have a min-max relationship:

min{l(P) : P is a path} ≥ max{w(B) : B is a set of feasible barriers}

Theorem

min{l(P) : P is a path} = max{w(B) : B is a set of feasible barriers}


Exercise

Give a certificate of optimality for shortest A− E path (A,F ,D,E )


Exercise

Give a certificate of optimality for shortest 1− 15 path (1, 2, 6, 10, 14, 15)


Another example: Linear Programming

Producing trains and soldiers subject to carpentry and finishing hours.

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9x1 ≥ 0

x2 ≥ 0

(1)


Bounding the objective

Question

How can we bound the value of 3x1 + 3x2 for any values of (x1, x2)satisfying the constraints below?

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0



Question


max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

3 × 2nd constraint + 3 × 4th constraint



Question


max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0


3 × 1st constraint + 3 × 3rd constraint



Question


max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0


3 × 1st constraint + 3 × 3rd constraint

1st constraint + 2nd constraint + 4th constraint


max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

In general, for any y1, y2, y3, y4 ≥ 0:

3x1+3x2 =

y1(2x1 + x2)+y2(x1 + 3x2)+y3(−x1)+y4(−x2)

= (2y1+y2−y3)x1+(y1+3y2−y4)x2 ≤ 8y1+9y2

as long as(2y1 + y2 − y3) = 3(y1 + 3y2 − y4) = 3

So any solution must have objective value at most 8y1 + 9y2


Duality

This leads to the following related optimization problems:

PRIMAL

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

(P)

DUAL

min 8y1 +9y2s.t. 2y1 +y2 −y3 = 3

y1 +3y2 −y4 = 3y1, y2, y3, y4 ≥ 0

(D)


Duality


PRIMAL

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

(P)

DUAL

min 8y1 +9y2s.t. 2y1 +y2 −y3 = 3

y1 +3y2 −y4 = 3y1, y2, y3, y4 ≥ 0

(D)

Let zP be the value of the optimal solution to (P) and zD be the value ofthe optimal solution to (D).


Duality


PRIMAL

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

(P)

DUAL

min 8y1 +9y2s.t. 2y1 +y2 −y3 = 3

y1 +3y2 −y4 = 3y1, y2, y3, y4 ≥ 0

(D)


Theorem (Weak duality)

zP ≤ zD


Duality


PRIMAL

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

(P)

DUAL

min 8y1 +9y2s.t. 2y1 +y2 −y3 = 3

y1 +3y2 −y4 = 3y1, y2, y3, y4 ≥ 0

(D)


Theorem (Strong duality)zP = zD

and the optimal solutions can be found reasonably fast (and the sametheorem applies to a more general class of optimization problems)


Duality


PRIMAL

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

(P)

DUAL

min 8y1 +9y2s.t. 2y1 +y2 −y3 = 3

y1 +3y2 −y4 = 3y1, y2, y3, y4 ≥ 0

(D)

In our case, we know a solution x1 = 3, x2 = 2 with value 15.We can for instance find y1 = 1.2, y2 = 0.6, y3 = y4 = 0 which gives us anupper bound of 15


How can we find the values of y?

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

As long as(2y1 + y2 − y3) = 3(y1 + 3y2 − y4) = 3

3x1+3x2 = (2y1+y2−y3)x1+(y1+3y2−y4)x2 =

y1(2x1 + x2)+y2(x1 + 3x2)+y3(−x1)+y4(−x2)

≤ 8y1+9y2


How can we find the values of y?

max 3x1 +3x2s.t. 2x1 +x2 ≤ 8

x1 +3x2 ≤ 9−x1 ≤ 0

−x2 ≤ 0

As long as(2y1 + y2 − y3) = 3(y1 + 3y2 − y4) = 3

3x1+3x2 = (2y1+y2−y3)x1+(y1+3y2−y4)x2 =

y1(2x1 + x2)+y2(x1 + 3x2)+y3(−x1)+y4(−x2)

≤ 8y1+9y2

If I want to prove optimality of (x1, x2) = (3, 2), I can only use nonzero yvalues for the constraints that are satisfied at equality for (3, 2)


Exercise

Write the dual optimization problem and the optimal primal and dualsolutions.

max 30x1 +100x2s.t. x1 +x2 ≤ 7

4x1 +10x2 ≤ 40−x1 ≤ −3

−x2 ≤ 0


Exercise

Write the dual optimization problem and the optimal primal and dualsolutions.

min 15A +20Hs.t. A +H = 120

4A −3H ≥ 0−A ≥ −90

−H ≥ −90A ≥ 0

H ≥ 0


Documents

Optimization and Operations Research II · Optimization and Operations Research II Ricardo Fukasawa [email protected] Department ofCombinatorics andOptimization FacultyofMathematics