Column Generation: Cutting Stock - Technical University · PDF fileColumn Generation: Cutting Stock – A very applied method Thomas Stidsen [email protected] DTU-Management Technical

1Thomas Stidsen

DTU-Management / Operations Research

Column Generation: Cutting Stock– A very applied method

Thomas Stidsen

[email protected]

DTU-Management

Technical University of Denmark

2Thomas Stidsen


OutlineHistory

The Simplex algorithm (re-visited)

Column Generation as an extension of theSimplex algorithm

A simple example !

3Thomas Stidsen


Introduction to Column GenerationColumn Generation (CG) is an old method,originally invented by Ford & Fulkerson in 1962 !

Benders decomposition algorithm dealt withadding constraints to a master problem

CG deals with adding variables to a masterproblem

CG is one of the most used methods in real lifewith lots of applications.

(Relax it is significantly easier than Benders algo-

rithm)

4Thomas Stidsen


Given an LP problemMin

cTx

s.t.

Ax ≥ b

x ≥ 0

This is a simple “sensible” (in the SOB notation) LPproblem

5Thomas Stidsen


The simplex slack variablesWe introduce a number slack variables, one foreach constraint, (simply included as extra xvariables). They are also positive.

Min

z = cTx

s.t.

Ax = b

x ≥ 0

6Thomas Stidsen


The simplex (intermediate) problemIn each iteration of the simplex algorithm, a numberof basic variables xB may be non-zero (B matrix)and remaining variables are called non − basic xN

(A matrix). In the end of each iteration of thesimplex algorithm, the values of the variables xB

and xN contain a feasible solution.Min

z = cTBxB + cT

NxN

s.t.

BxB + AxN = b

xB, xN ≥ 0

7Thomas Stidsen


A reformulated versionMin

z = cTBxB + cT

NxN

s.t.

xB = B−1b − B−1ANxN

xB, xN ≥ 0

8Thomas Stidsen


A reformulated versionMin

z = cBB−1b + (cN − cBB−1AN)xN

s.t.

xB = B−1b − B−1ANxN

xB, xN ≥ 0

9Thomas Stidsen


Comments to the reformulated versionAt the end of each iteration (and the beginning ofthe next) it holds that:

The current value of the non-basic variablesare: xN = 0.

Hence the current optimization value is:z = cBB−1b.

And hence the values of the basic variables are:xB = cBB−1.

10Thomas Stidsen


Comments to the reformulated versionThe coefficients of the basic variables (xB) inthe objective function are zero ! This simplymeans that the xB variables are not“participating” in the objective function.

The coefficients of the non-basic variables arethe so-called reduced costs: cN − cBB−1AN

Because z = cTBxB + cT

NxN , and we want tominimize, non-basic variables with negativereduced costs can improve the current solution,i.e. cN − cBB−1AN < 0

11Thomas Stidsen


The Simplex AlgorithmxS = bxS = 0while MINj(cj − cBB−1Aj) < 0

Select new basic variable j: cj − cBB−1Aj < 0Select new non-basic variable j′ by

increasing xj as much as possibleSwap columns between matrix B and matrix A

end while

12Thomas Stidsen


Comments to the algorithmThere are three parts of the algorithm:

Select the next basic variable.

Select the next non-basic variable.

Update data structures.

We will now comment on each of the three steps.

13Thomas Stidsen


Selecting the next basic variableTo find the most negative reduced cost, all thereduced costs must be checked:cred = MINj(cj − cBB−1Aj) < 0.

In the revised simplex algorithm, first the systemyB = cB is solved. (In the standard simplexalgorithm, the reduced costs are given directly).

Conclusion: This part (mainly) is dependent on thenumber of variables.

14Thomas Stidsen


Selecting the next non-basic variableThere are the same number of basic variables(to compare) as there are constraints

We are only considering one non-basic variable

Basically we are looking at the system:xB = x∗

B − B−1Ajxj.

The revised simplex algorithm solves thesystem Bd = a, where a is the column of theentering variable.

Conclusion: This part is dependent on the numberof constraints

15Thomas Stidsen


BookkeepingIn the end what needs to be done (in the revisedsimplex algorithm) ?

Update of the matrix’s B and A: Simply swapthe columns.

Conclusion: This part is dependent on the numberof constraints

16Thomas Stidsen


Comments to the algorithmSelecting a non-negative variable principallyrequires checking all the many variables (timeconsuming).

Selecting the new non-basic variable:There are the same number of basicvariables (to compare) as there areconstraintsWe are only considering one non-basicvariableBasically we are looking at the system:xB = x∗

B − B−1Ajxj

Update of the matrix’s B and A: Simply swapthe columns.

17Thomas Stidsen


LP programs with many variables !The crucial insight: The number of non-zerovariables (the basis variables) is (at most) equal tothe number of constraints, hence eventhough thenumber of possible variables (columns) may belarge, we only need a small subset of these in theoptimal solution.

Basis Non−basis

18Thomas Stidsen


The simple column generation ideaThe simple idea in column generation is not torepresent all variables explicitly, represent themimplicitly: When looking for the next basic variable,solve an optimization problem, finding the variable(column) with the most negative reduced cost.

19Thomas Stidsen


The column generation algorithmx∗

S = bx∗

S= 0

repeatrepeat

Select new basic variable j: cj − cBB−1Aj < 0Select new non-basic variable j′ by

increasing xj as much as possibleSwap columns between matrix B and matrix A

until MINj(cj − cBB−1Aj) >= 0x = MIN(cj − cBB−1Aj, LEGAL)

until no more improving variables

20Thomas Stidsen


The reduced costs: cj − cBB−1Aj < 0Lets look at the individual components of thereduced costs cj − cBB−1Aj < 0 :

cj: The original costs for the variable

Aj: The column in the A matrix, which definesthe variable.

cBB−1: For each constraint in the LP problem,this factor is multiplied to the column. cB is thecost vector for the current basic variables andB−1 is the inverse basic matrix.

21Thomas Stidsen


Alternative cBB−1

For an optimization problem:{min cx|Ax ≥ b, x ≥ 0}. Outside the inner loop, thereduced costs are: cN − cBB−1AN ≥ 0 for theexisting variables. Notice that this also holds for thebasic variables for which the costs are all zero, i.e.c − cB−1A ≥ 0. If we rename the term cBB−1 to thename π we can rewrite the expression: πA ≤ c. Butthis is exactly dual feasibility of our original problem! Hence we can use the dual variables from ourinner loop when looking for new variables.

22Thomas Stidsen


The column generation algorithmx∗

S = bx∗

S= 0

repeatπ = MIN(min cx|Ax ≥ b, x ≥ 0)xj = MIN(cj − cBB−1Aj, LEGAL)

until no more improving variables

23Thomas Stidsen


Cutting Stock ExampleThe example which is always referred to regardingcolumn generation is the cutting stock example.Assume you own a workshop where steel rods arecut into different pieces:

Customers arrive and demand steel rods ofcertain lengths: 22 cm., 45 cm., etc.

You serve the customers demands by cuttingthe steel rods into the rigth sizes.

You recieve the rods in lengths of e.g. 200 cm.

24Thomas Stidsen


Cutting Stock ExampleA A A B B C C C

A

A

A

B

C

C

C

B

Log

LOSS

25Thomas Stidsen


Cutting Stock ObjectiveHow can you minimize your material waste (what’sleft of each rod after you have cut the customerpieces) ?

26Thomas Stidsen


Cutting Stock Formulation 1Min:

z =∑

k

yk

s.t.:∑

k

xik = bi ∀i

∑

i

wi · xik ≤ W · yk ∀k

xik ∈ N0 yk ∈ {0, 1}

27Thomas Stidsen


Cutting Stock Formulation 1What is wrong with the above formulation ?

It contains many symmetric solutions, k!. Thismakes the problem extremely hard for branchand bound algorithms.

There are many integer or binary variables

For these reasons, it is never used in practice.

28Thomas Stidsen


New FormulationHow can we change the formulation ?

Instead of focussing on which steel rod aparticular part is to be cut from, look at possiblepatterns used to cut from a steel rod.

The question is then changed to focussing onhow many times a particular pattern is used.

29Thomas Stidsen


Cutting Stock Formulation 2Min:

z =∑

j

xj

s.t.:∑

j

aij · xj ≥ bi ∀i

xj ∈ N0

30Thomas Stidsen


Cutting Stock Formulation 2What are the xj and ai

j ?

aij is a cutting pattern, describing how to cut one

steel rod into pieces.

Notice that any legal solution to formulation 1will always be consisting of a number of cuttingpatterns

xj is then simply the number of times thatcutting pattern is used.

31Thomas Stidsen


Cutting Stock Formulation 2There are two different problems with theformulation:

We will only solve the relaxed problem, i.e. theLP version of the MIP problem, how do we getinteger solutions ?

We need to “consider” all the cutting patterns???

Today we will only deal with the last problem ...

32Thomas Stidsen


Cutting PatternsWe have a number of questions to these cuttingpatterns:

But where do we get the cutting patterns from ?

How many are there ? ( |i|a) = |i|!

a!(|i|−a)! , i.e. a lot !

Where: |i| is the number of different types(lengths) requiredWhere: a is the average number of cutsizesin each of the patterns.

33Thomas Stidsen


Cutting PatternsThis is a very real problem: Even if we had a way of

generating all the legal cutting patterns, our standard

simplex algorithm will need to calculate the “efficient”

variables, but we will not have memory to contain the

variables in the algoritm, now what ?

34Thomas Stidsen


The improving variableHow can we find the improving variable. We knowthat any variable where cj − cBB−1Aj = cj − πAj.But we want not just to choose any variable, butmin cj − πAj, the so called Dantzig rule, which isthe standard non-basic variable selection rule. TheDantzig rule hence corresponds to the originalcosts (which might be zero), subtracted by the dualvariable vector multiplied by the column of the newvariable xj

35Thomas Stidsen


Pattern generation: The subproblemFor each iteration in the Simplex algorithm, we needto find the most negative column. We can do that bydefining a new optimisation problem:

Min z = 1 −∑

i πiaij

36Thomas Stidsen


Pattern generation: Ignore the constantsMax:

z =∑

i

πiaij

s.t.:∑

i

li · aij ≤ L

aij ∈ Z+

What about the j index ?

37Thomas Stidsen


What is the subproblem ?The subproblem is the classical knapsack problem.

The problem is theoretically NP-hard ....

But it is an “easy” NP-hard problem, meaningthat we can solve the problem for relative largesizes of N , i.e. number of items (in this casedifferent lengths)

38Thomas Stidsen


Knapsack solution methodsThe problem may e.g. be solved using dynamicprogramming. We will ignore the problem ofefficiency and just use standard MIP solvers. Noticethat we are going to solve exactly the sameproblem, but with updated profit coefficients.

39Thomas Stidsen


The Column Generation AlgorithmCreate initial columnsrepeat

Solve master problem, find πSolve the subproblem min zsub = 1 −

∑i πi · ai

Add new column problem to master problemuntil zsub ≥ 0

40Thomas Stidsen


Lets look at an example !We have some customers who wants:

Pieces in three lengths: 44 pieces of length 81cm., 3 pieces of length 70 cm. and 48 pieces oflength 68 cm.

We have steel rods of length 218 cm. of unitprice.

How do we get the initial columns ???

41Thomas Stidsen


The initial columnsWe need some initial columns and we basicallyhave two choices:

Start with fake columns, which are so expensivethat we know they will not be in the finalproblem.

If possible, create some more “competetive”columns.

We simply use columns where each column con-

tains 1 in different rows

42Thomas Stidsen


Master problem\ Initial master problem

minimizex_1 + x_2 + x_3

subject tol1: x_1 >= 44l2: x_2 >= 3l3: x_3 >= 48end

43Thomas Stidsen


Sub problem: Given π = [1, 1, 1]\ initial sub problem

maximizea_1 + a_2 + a_3

subject tol: 81a_1 + 70a_2 + 68a_3 <= 218bound

a_1 <= 2a_2 <= 3a_3 <= 3

integera_1 a_2 a_3

end

44Thomas Stidsen


First sub-problem solutionThe best solution (a1, a2, a3) = (0, 0, 3)

45Thomas Stidsen


New Master Problem\ second master problem

minimizex_1 + x_2 + x_3 + x_4

subject tol1: x_1 >= 44l2: x_2 >= 3l3: x_3 + 3x_4 >= 48end

46Thomas Stidsen


First master solutionThe best solution duals (π1, π2, π3) = (1.0, 1.0, 0.33)

47Thomas Stidsen


Second Sub problem: Given π = [1.0, 1.0, 0.33]\ initial sub problem

maximizea_1 + a_2 + 0.33a_3

subject tol: 81a_1 + 70a_2 + 68a_3 <= 218bound

a_1 <= 2a_2 <= 3a_3 <= 3

integera_1 a_2 a_3

end

48Thomas Stidsen


First sub-problem solutionThe best solution (a1, a2, a3) = (0, 3, 0)

49Thomas Stidsen


New Master Problem\ second master problem

minimizex_1 + x_2 + x_3 + x_4 + x_5

subject tol1: x_1 >= 44l2: x_2 + 3x_5 >= 3l3: x_3 + 3x_4 >= 48end

50Thomas Stidsen


Why is this so interesting ???I have told you a number of times that this is themost applied method in OR, why ?

We can solve problems with a "small" numberof constraints and an exponential numbervariables ... this is a special type of LP problembut ...

Through the so called Dantzig-Wolfedecomposition we can change any LP problemto a problem with a reduced number ofconstraints, but an increased number ofvariables ...

By performing clever decompositions we canimprove the LP bounding ...

51Thomas Stidsen


Subproblem solutionUsually (though not always) we need an efficientalgorithm to solve the subproblem. Subproblemsare often one of the following types:

Knapsack (like in the cutting stock problem).

Shortest path problems in graphs.

Hence: To use column generation efficiently youoften need to know something about complexitytheory ...

52Thomas Stidsen


Getting Integer SolutionsThere are several methods:

Rounding up, not always possible ....

Getting integer solutions using (meta)heuristics.This is the reason for the great interest in theset partioning/set covering problem.

Branch and price, hard and quite timeconsuming ...

Documents

Column Generation: Cutting Stock - Technical University · PDF fileColumn Generation: Cutting Stock – A very applied method Thomas Stidsen [email protected] DTU-Management Technical