Math Models of OR: Branch-and-Cut

Math Models of OR:Branch-and-Cut

John E. Mitchell

Department of Mathematical SciencesRPI, Troy, NY 12180 USA

November 2018

Mitchell Branch-and-Cut 1 / 21

Cutting planes

Outline

1 Cutting planes

2 The branch-and-cut algorithm

3 Example

Mitchell Branch-and-Cut 2 / 21

Cutting planes

A cutting plane example

Initial relaxation:

Solve linear optimization relaxation

min{cT x : Ax ≤ b, x ≥ 0}

Solution to linear optimization relaxation

Mitchell Branch-and-Cut 3 / 21

Cutting planes

A cutting plane example

Initial relaxation:

Solve linear optimization relaxation

min{cT x : Ax ≤ b, x ≥ 0}

Solution to linear optimization relaxation

Cutting plane aT1 x = b1

Mitchell Branch-and-Cut 3 / 21

Cutting planes

Add a cutting plane and reoptimize

Solve linear optimization relaxation

min{cT x : Ax ≤ b, x ≥ 0,aT1 x ≤ b1}

Solution to linear optimization relaxation

Cutting plane aT1 x = b1

Mitchell Branch-and-Cut 4 / 21

Cutting planes

Add a cutting plane and reoptimize

Solve linear optimization relaxation

min{cT x : Ax ≤ b, x ≥ 0,aT1 x ≤ b1}

Solution to linear optimization relaxation

Cutting plane aT1 x = b1

Cutting plane aT2 x = b2

Mitchell Branch-and-Cut 4 / 21

Cutting planes

Add a cutting plane and reoptimize

Solve linear optimization relaxation

min{cT x : Ax ≤ b, x ≥ 0,aT1 x ≤ b1,aT

2 x ≤ b2}

Solution to linear optimization relaxation

Cutting plane aT1 x = b1

Cutting plane aT2 x = b2

Solution to relaxation is integral, so it solves the integer optimizationproblem.

Mitchell Branch-and-Cut 5 / 21

The branch-and-cut algorithm

Outline

1 Cutting planes

2 The branch-and-cut algorithm

3 Example

Mitchell Branch-and-Cut 6 / 21

The branch-and-cut algorithm

Branch-and-cut

Branch-and-bound can be improved through the addition of cuttingplanes. We work with integer optimization problems of the form

min cT xsubject to Ax ≤ b (IOP)

x ≥ 0, x ∈ Rn

xi integer, i = 1, . . . ,p.

Mitchell Branch-and-Cut 7 / 21

The branch-and-cut algorithm

The steps of the algorithm

1 Initialize: The initial set L of active nodes consists of just oneproblem, L = {(IOP)}. If a feasible solution x̄ is known, the initialupper bound on the optimal value of (IOP) is set to zu = cT x̄ ;else, we initialize zu =∞.

2 Termination: If L = ∅ then the feasible integral point thatprovided the incumbent upper bound zu is optimal for (IOP).

3 Relaxation: Remove a node IOP l from the set of active nodes.Solve the linear optimization relaxation of IOP l . Let z l be theoptimal value of the relaxation (we allow z l =∞ if the relaxation isinfeasible, and z l = −∞ if the relaxation is unbounded). Let x l bean optimal solution to this relaxation, if the relaxation has anoptimal solution. If the relaxation has an unbounded optimal valuethen let x l either be a fractional extreme ray or a feasible point forthe relaxation with value smaller than zu.

Mitchell Branch-and-Cut 8 / 21

The branch-and-cut algorithm

The steps of the algorithm

1 Initialize: The initial set L of active nodes consists of just oneproblem, L = {(IOP)}. If a feasible solution x̄ is known, the initialupper bound on the optimal value of (IOP) is set to zu = cT x̄ ;else, we initialize zu =∞.

2 Termination: If L = ∅ then the feasible integral point thatprovided the incumbent upper bound zu is optimal for (IOP).

3 Relaxation: Remove a node IOP l from the set of active nodes.Solve the linear optimization relaxation of IOP l . Let z l be theoptimal value of the relaxation (we allow z l =∞ if the relaxation isinfeasible, and z l = −∞ if the relaxation is unbounded). Let x l bean optimal solution to this relaxation, if the relaxation has anoptimal solution. If the relaxation has an unbounded optimal valuethen let x l either be a fractional extreme ray or a feasible point forthe relaxation with value smaller than zu.

Mitchell Branch-and-Cut 8 / 21

The branch-and-cut algorithm

The steps of the algorithm

1 Initialize: The initial set L of active nodes consists of just oneproblem, L = {(IOP)}. If a feasible solution x̄ is known, the initialupper bound on the optimal value of (IOP) is set to zu = cT x̄ ;else, we initialize zu =∞.

2 Termination: If L = ∅ then the feasible integral point thatprovided the incumbent upper bound zu is optimal for (IOP).

3 Relaxation: Remove a node IOP l from the set of active nodes.Solve the linear optimization relaxation of IOP l . Let z l be theoptimal value of the relaxation (we allow z l =∞ if the relaxation isinfeasible, and z l = −∞ if the relaxation is unbounded). Let x l bean optimal solution to this relaxation, if the relaxation has anoptimal solution. If the relaxation has an unbounded optimal valuethen let x l either be a fractional extreme ray or a feasible point forthe relaxation with value smaller than zu.

Mitchell Branch-and-Cut 8 / 21

The branch-and-cut algorithm

The steps (continued)4 Add cutting planes: If desired, search for cutting planes that are

violated by x l ; if any are found, add them to the relaxation andreturn to Step 2.

5 Fathom by infeasibility: If the relaxation is infeasible then IOP l isfathomed. Return to Step 2.

6 Fathom by integrality: If x l is integral then IOP l is fathomed.Update zu ← min{cT x l , zu}. Return to Step 2.

7 Fathom by bounds: If cT x l ≥ zu then IOP l is fathomed. Returnto Step 2.

8 Subdivide: Choose a component i with x li fractional. Create two

new nodes and add them to the set of active nodes:(a) x feasible in (IOP l) with xi ≤ bx l

i c, and(b) x feasible in (IOP l) with xi ≥ dx l

i e.Return to Step 2.

Mitchell Branch-and-Cut 9 / 21

The branch-and-cut algorithm

The steps (continued)4 Add cutting planes: If desired, search for cutting planes that are

violated by x l ; if any are found, add them to the relaxation andreturn to Step 2.

5 Fathom by infeasibility: If the relaxation is infeasible then IOP l isfathomed. Return to Step 2.

6 Fathom by integrality: If x l is integral then IOP l is fathomed.Update zu ← min{cT x l , zu}. Return to Step 2.

7 Fathom by bounds: If cT x l ≥ zu then IOP l is fathomed. Returnto Step 2.

8 Subdivide: Choose a component i with x li fractional. Create two

new nodes and add them to the set of active nodes:(a) x feasible in (IOP l) with xi ≤ bx l

i c, and(b) x feasible in (IOP l) with xi ≥ dx l

i e.Return to Step 2.

Mitchell Branch-and-Cut 9 / 21

The branch-and-cut algorithm

The steps (continued)4 Add cutting planes: If desired, search for cutting planes that are

violated by x l ; if any are found, add them to the relaxation andreturn to Step 2.

5 Fathom by infeasibility: If the relaxation is infeasible then IOP l isfathomed. Return to Step 2.

6 Fathom by integrality: If x l is integral then IOP l is fathomed.Update zu ← min{cT x l , zu}. Return to Step 2.

7 Fathom by bounds: If cT x l ≥ zu then IOP l is fathomed. Returnto Step 2.

8 Subdivide: Choose a component i with x li fractional. Create two

new nodes and add them to the set of active nodes:(a) x feasible in (IOP l) with xi ≤ bx l

i c, and(b) x feasible in (IOP l) with xi ≥ dx l

i e.Return to Step 2.

Mitchell Branch-and-Cut 9 / 21

The branch-and-cut algorithm

The steps (continued)4 Add cutting planes: If desired, search for cutting planes that are

violated by x l ; if any are found, add them to the relaxation andreturn to Step 2.

5 Fathom by infeasibility: If the relaxation is infeasible then IOP l isfathomed. Return to Step 2.

6 Fathom by integrality: If x l is integral then IOP l is fathomed.Update zu ← min{cT x l , zu}. Return to Step 2.

7 Fathom by bounds: If cT x l ≥ zu then IOP l is fathomed. Returnto Step 2.

8 Subdivide: Choose a component i with x li fractional. Create two

new nodes and add them to the set of active nodes:(a) x feasible in (IOP l) with xi ≤ bx l

i c, and(b) x feasible in (IOP l) with xi ≥ dx l

i e.Return to Step 2.

Mitchell Branch-and-Cut 9 / 21

The branch-and-cut algorithm

The steps (continued)4 Add cutting planes: If desired, search for cutting planes that are

violated by x l ; if any are found, add them to the relaxation andreturn to Step 2.

5 Fathom by infeasibility: If the relaxation is infeasible then IOP l isfathomed. Return to Step 2.

6 Fathom by integrality: If x l is integral then IOP l is fathomed.Update zu ← min{cT x l , zu}. Return to Step 2.

7 Fathom by bounds: If cT x l ≥ zu then IOP l is fathomed. Returnto Step 2.

8 Subdivide: Choose a component i with x li fractional. Create two

new nodes and add them to the set of active nodes:(a) x feasible in (IOP l) with xi ≤ bx l

i c, and(b) x feasible in (IOP l) with xi ≥ dx l

i e.Return to Step 2.

Mitchell Branch-and-Cut 9 / 21

The branch-and-cut algorithm

Notes on branch-and-cut

L is the set of active nodes in the branch-and-cut tree. The value of thebest known feasible point for (IOP) is zu, which provides an upperbound on the optimal value of (IOP).

In some situations, a very large number of violated cutting planes arefound in Step 4, in which case it is common to sort the cutting planessomehow (perhaps by violation), and add just a subset.

Mitchell Branch-and-Cut 10 / 21

The branch-and-cut algorithm

Families of cutting planesMany structured integer optimization problems have known families ofvalid inequalities. For example, the subtour elimination constraints arevalid constraints for the traveling salesman problem.

r

s

t

u

v

wUse subtour elimination constraintto tighten relaxation:

xrs + xst + xrt ≤ 2

Mitchell Branch-and-Cut 11 / 21

The branch-and-cut algorithm

Solving the relaxations

The relaxations can be solved using any method for linearprogramming problems.

Typically, the initial relaxation is solved using the simplex method.

Subsequent relaxations are solved using the dual simplex method,since the dual solution for the relaxation of the parent subproblem isstill feasible in the relaxation of the child subproblem.

Further, when cutting planes are added in Step 4, the current iterate isstill dual feasible, so again the modified relaxation can be solved usingthe dual simplex method.

It is also possible to use an interior point method, and this can be agood choice if the linear programming relaxations are large.

Mitchell Branch-and-Cut 12 / 21

The branch-and-cut algorithm

Further notes

Of course, there are several issues to be resolved with this algorithm.

These include the major questions of deciding whether to branch or tocut and deciding how to branch and how to generate cutting planes.

If the objective function and/or the constraints in (IOP) are nonlinear,the problem can still be attacked with a branch-and-cut approach.

Mitchell Branch-and-Cut 13 / 21

Example

Outline

1 Cutting planes

2 The branch-and-cut algorithm

3 Example

Mitchell Branch-and-Cut 14 / 21

Example

A branch-and-cut example

The integer programming problem

min z := −6x1 − 5x2subject to 3x1 + x2 ≤ 11 (Eg0)

−x1 + 2x2 ≤ 5x1, x2 ≥ 0, integer.

is illustrated in the figure on the next slide. The feasible integer pointsare marked. The linear optimization relaxation (or LOP relaxation) isobtained by ignoring the integrality restrictions and is indicated by thepolyhedron contained in the solid lines.

Mitchell Branch-and-Cut 15 / 21

Example

Solve the initial relaxation (Eg0)

x1

x2

3

3

0

x = (237 ,3

57)

Optimal solution to initial relaxationis x = (23

7 ,357), with value −331

7 .Fractional, so branch on x1 ≤ 2versus x1 ≥ 3.

Mitchell Branch-and-Cut 16 / 21

Example

Solve the initial relaxation (Eg0)

x1

x2

3

3

0

x = (237 ,3

57)

Optimal solution to initial relaxationis x = (23

7 ,357), with value −331

7 .Fractional, so branch on x1 ≤ 2versus x1 ≥ 3.

Mitchell Branch-and-Cut 16 / 21

Example

Solve the initial relaxation (Eg0)

x1

x2

3

3

0

x = (237 ,3

57)

Optimal solution to initial relaxationis x = (23

7 ,357), with value −331

7 .Fractional, so branch on x1 ≤ 2versus x1 ≥ 3.

Mitchell Branch-and-Cut 16 / 21

Example

Branch with x1 ≥ 3

x1

x2

3

3

0

x = (3,2)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg1)x1 ≥ 3

x1, x2 ≥ 0, integer.

Solution: x = (3,2).Integral, so get new incumbentsolution, new upper bound onoptimal value of -28.

Mitchell Branch-and-Cut 17 / 21

Example

Branch with x1 ≥ 3

x1

x2

3

3

0

x = (3,2)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg1)x1 ≥ 3

x1, x2 ≥ 0, integer.

Solution: x = (3,2).Integral, so get new incumbentsolution, new upper bound onoptimal value of -28.

Mitchell Branch-and-Cut 17 / 21

Example

Branch with x1 ≥ 3

x1

x2

3

3

0

x = (3,2)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg1)x1 ≥ 3

x1, x2 ≥ 0, integer.

Solution: x = (3,2).Integral, so get new incumbentsolution, new upper bound onoptimal value of -28.

Mitchell Branch-and-Cut 17 / 21

Example

Branch with x1 ≥ 3

x1

x2

3

3

0

x = (3,2)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg1)x1 ≥ 3

x1, x2 ≥ 0, integer.

Solution: x = (3,2).Integral, so get new incumbentsolution, new upper bound onoptimal value of -28.

Mitchell Branch-and-Cut 17 / 21

Example

Branch with x1 ≥ 3

x1

x2

3

3

0

x = (3,2)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg1)x1 ≥ 3

x1, x2 ≥ 0, integer.

Solution: x = (3,2).Integral, so get new incumbentsolution, new upper bound onoptimal value of -28.

Mitchell Branch-and-Cut 17 / 21

Example

Branch with x1 ≤ 2

x1

x2

3

3

0

x = (2,3.5) min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg2)x1 ≤ 2

x1, x2 ≥ 0, integer.

Solution: x = (2,3.5), value -29.5.Fractional, so add cutting plane2x1 + x2 ≤ 7.

Mitchell Branch-and-Cut 18 / 21

Example

Branch with x1 ≤ 2

x1

x2

3

3

0

x = (2,3.5) min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg2)x1 ≤ 2

x1, x2 ≥ 0, integer.

Solution: x = (2,3.5), value -29.5.Fractional, so add cutting plane2x1 + x2 ≤ 7.

Mitchell Branch-and-Cut 18 / 21

Example

Branch with x1 ≤ 2

x1

x2

3

3

0

x = (2,3.5) min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg2)x1 ≤ 2

x1, x2 ≥ 0, integer.

Solution: x = (2,3.5), value -29.5.Fractional, so add cutting plane2x1 + x2 ≤ 7.

Mitchell Branch-and-Cut 18 / 21

Example

Branch with x1 ≤ 2

x1

x2

3

3

0

x = (2,3.5) min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg2)x1 ≤ 2

x1, x2 ≥ 0, integer.

Solution: x = (2,3.5), value -29.5.Fractional, so add cutting plane2x1 + x2 ≤ 7.

Mitchell Branch-and-Cut 18 / 21

Example

Branch with x1 ≤ 2

x1

x2

3

3

0

x = (2,3.5) min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg2)x1 ≤ 2

x1, x2 ≥ 0, integer.

Solution: x = (2,3.5), value -29.5.Fractional, so add cutting plane2x1 + x2 ≤ 7.

Mitchell Branch-and-Cut 18 / 21

Example

Add cutting plane 2x1 + x2 ≥ 7

x1

x2

3

3

0

x = (1.8,3.4)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg3)x1 ≤ 2

2x1 + x2 ≤ 7x1, x2 ≥ 0, integer.

Solution: x = (1.8,3.4), with value−27.8. Value worse thanincumbent integer solution, sofathomed by bounds.

Mitchell Branch-and-Cut 19 / 21

Example

Add cutting plane 2x1 + x2 ≥ 7

x1

x2

3

3

0

x = (1.8,3.4)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg3)x1 ≤ 2

2x1 + x2 ≤ 7x1, x2 ≥ 0, integer.

Solution: x = (1.8,3.4), with value−27.8. Value worse thanincumbent integer solution, sofathomed by bounds.

Mitchell Branch-and-Cut 19 / 21

Example

Add cutting plane 2x1 + x2 ≥ 7

x1

x2

3

3

0

x = (1.8,3.4)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg3)x1 ≤ 2

2x1 + x2 ≤ 7x1, x2 ≥ 0, integer.

Solution: x = (1.8,3.4), with value−27.8. Value worse thanincumbent integer solution, sofathomed by bounds.

Mitchell Branch-and-Cut 19 / 21

Example

Add cutting plane 2x1 + x2 ≥ 7

x1

x2

3

3

0

x = (1.8,3.4)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg3)x1 ≤ 2

2x1 + x2 ≤ 7x1, x2 ≥ 0, integer.

Solution: x = (1.8,3.4), with value−27.8. Value worse thanincumbent integer solution, sofathomed by bounds.

Mitchell Branch-and-Cut 19 / 21

Example

Add cutting plane 2x1 + x2 ≥ 7

x1

x2

3

3

0

x = (1.8,3.4)

min z := −6x1 − 5x2s.t. 3x1 + x2 ≤ 11

−x1 + 2x2 ≤ 5 (Eg3)x1 ≤ 2

2x1 + x2 ≤ 7x1, x2 ≥ 0, integer.

Solution: x = (1.8,3.4), with value−27.8. Value worse thanincumbent integer solution, sofathomed by bounds.

Mitchell Branch-and-Cut 19 / 21

Example

Lifting

Notice that the cutting plane introduced in the second subproblem isnot valid for the first subproblem.

This inequality can be modified to make it valid for the first subproblemby using a lifting technique.

(Not discussed further in this course.)

Mitchell Branch-and-Cut 20 / 21

Example

The branch-and-cut treeThe progress of the algorithm is illustrated below.

(Eg0). Value −33 17

Solution x = (2 37 , 3

57 )

(Eg1). Value −28Solution x = (3, 2)

(Eg2). Value −29.5Solution x = (2, 3.5)

(Eg3). Value −27.8Solution x = (1.8, 3.4)

x1 ≤ 2

x1 ≥ 3

2x1 + x2 ≤ 7

Mitchell Branch-and-Cut 21 / 21