Upload
pooja-jain
View
10
Download
1
Tags:
Embed Size (px)
DESCRIPTION
lpp
Citation preview
Dual simplex method
Submitted by:
Pooja jain
M.com second semester
Group A roll no. 37
Linear Programming
During the Second World War, the soviet army was probably more efficient thanks to Leonid
Kantorovich. In 1939, this mathematician and economist came up with a new mathematical
technic to solve linear programs, hence improving plans of production at lower costs. In 1975, he
became the only Nobel Prize winner in Economy from the USSR. After the war, methods of
linear programming kept secret got published, including the simplex method by the American
mathematician George Dantzig and the duality theory by the Hungarian-American
mathematician John von Neumann.
Applications in company daily planning rose, increasing efficiency thanks to linear programs.
Applications also include transport, supply chains, shift scheduling, telecommunications, yield
management, assignment, set partitioning, set covering, advertisement. I’ve also heard of DNA
analysis, and I personally use it for cake-cutting.
Let’s be smart robbers
Suppose we have just attacked a village and we want to steal as much as we can. There’s gold
and dollar bills. But we only have one knapsack with a limited volume and we know we can’t
take it all. So we’ll have to choose the volumes of each thing to steal. Also assume that we are
not strong enough to carry too much weight (I do play sports but I don’t go to the gym, so I have
a weak upper body…).
You face what is called a mathematical program or an optimization problem. You need to
maximize the value of your robbery by choosing volumes of gold and 1 dollar bills
with constraints on total volume and total weight. One volume of gold is worth more than one
volume of dollar bills but it’s also heavier.
The value of the robbery is called the objective function. It’s what we want to maximize or
minimize. Volumes of gold and bills are called the variables. Mathematically, we usually write
the mathematical program as follows.
Of course, we usually have complicated notations instead of words… But we’ll get to that!
So far, what we have here is a mathematical program. There are plenty of methods to solve a
mathematical program but none of them can guarantee for sure that we find the solution in a
reasonable time, especially if the number of variable is higher than 10. But in our case, we can
write our mathematical program as a linear program.
A linear program is a mathematical program with a linear objective function and linear
constraints.
In our case, the objective function is linear since, whenever you add a volume of any product, the
total value increases by the value of that volume of that product. That means that we can write
the objective function in the following way:
A linear constraint is a constraint that says that a linear function of variables must be higher,
lesser or equal to a constant. Now, our first constraint says that the sum of all volumes must be
less than some constant limit. This is obviously a linear constraint. Similarly, because the weight
of volumes is the sum of weights of each volume, the total weight is also a linear function.
Hence, the second constraint is also linear: our program is linear.
Cool! Are we done writing our linear program?
Almost. We forgot two constraints. So far volumes are assumed to be real numbers, but they
actually need to be non-negatives. I mean stealing -1 litre of gold… that has no meaning. So we
just need to add constraints that say that volumes are positive. Yet, these are linear constraint
too! So our program is totally linear:
Our problem has two variables. Now any two variables will not necessarily satisfy all the
constraints, in which case we will say that the solution is infeasible. For instance, if the volume
of gold is -1 and the volume of bills is 0, then the solution is infeasible because the
constraint is not satisfied. If our program had no constraints, any solution
would be feasible, but constraints modify the feasible set.
What’s interesting is how each constraint modify the feasible set. As a matter of fact, each
inequality constraint divides the space into two areas, one for solutions satisfying the constraint,
the other for solutions that don’t satisfy the constraint.
The feasible set is the white area. We can see
that they we have a finite number of extreme
points and that we have the other points inside
are points in-between the extreme points. A
theorem says that there always exists an
extreme point that is one of the points that
maximize the objective function.
It’s possible that other points also maximize
the objective function. Still, it means that if
we know all the extreme points of our feasible
set, we can easily solve our problem by testing
them all! And that’s not all! Any mathematical
program can be associated with a dual
program, which is another mathematical program that has interesting properties related to the
initial mathematical program. In the case of linear programming, the dual program has even
more interesting properties and many reasoning in this dual program enable major
improvements.
Duality of LP problems
Each LP problem (called as Primal in this context) is associated with its counterpart known as
Dual LP problem. Instead of primal, solving the dual LP problem is sometimes easier when a)
the dual has fewer constraints than primal (time required for solving LP problems is directly
affected by the number of constraints,) and b) the dual involves maximization of an objective
function (it may be possible to avoid artificial variables that otherwise would be used in a primal
minimization problem).
Duality in Linear Programming
In order to explain duality to, continuing with the example of the smart robber. Basically, the
smart robber wants to steal as much gold and dollar bills as he can. He is limited by the volume
of his backpack and the maximal weight he can carry. Now, let’s notice that we can write the
problem as follows.
The problem we have written here is what we call the primal linear program. The dual program
will totally change our understanding of the problem, and that’s why it’s so cool.
In the primal program, constraints had constant numbers on their right. These constant numbers
are our resources. They say what we are able to do concerning each constraint. The dual problem
consists in evaluating how much our combined resources are worth. If the offer meets the
demand, our resources will be just as much as their potentials, which is the worth of the robbery.
Let’s go more into details. In the dual problem we will attribute values to the resources (as in
“how much they’re worth”). These values are called the dual variables. In our case, we have two
constraints, so we will have 2 dual variables.
The first dual variable, let’s call it ValueVolume refers to the value of one unit of volume. The
second dual variable refers to the value of one unit of weight. Value Weight seems like a right
name for it.
Now, we can write the value of the robbery with these two new variables… Let’s see if we get
the same result.
But how are the values per volume and per weight determined?
If I wanted to sell my resources, potential buyers are going to minimize the value of my
resources. So their valuations are the minimum of the total value. But as a seller, I will argue that
each of my resource is worth a lot, because it enables the robbery of more gold and more bills.
Obviously, the values of resources depend on the actual values of gold and bills per volume.
Let’s have a thought about the value of gold (and then we’ll be able to apply the same reasoning
to bills). If the constraints enabled us to steal one more volume of gold, then incremental value of
the robbery would be at least the value of this one volume of gold, right? It could be more, if we
use the new constraints to steal something else than gold that’s worth more. What I’m saying is
that, if the total volume enabled us to steal one more unit of volume of gold, and if we could
carry one more unit of weight of one volume of gold, then the value of this incremental steal
would be at least the value of one more volume of gold. Let’s write it.
From this we deduce a constraint on dual variables. As we can see, any variable in the primal
problem is associated with a constraint in the dual problem and vice-versa.
We are almost done. Let’s notice the fact that if we increase the total volume, then we have more
possibilities for the primal variables, which are the volumes of stolen gold and bills. Therefore,
the value of a unit of volume cannot be negative. This adds two more constraints on the sign of
dual variables. Now, we’re done and we can write the dual problem.
The dual program of a linear program is a linear program! Let’s have a look at the feasible set.
It’s important to notice that the information
of the primal objective functions appear in
the dual feasible set. The most important
result is the strong duality property: optimal
values of the primal and dual problems
are the same. We can solve the primal
problem simply by solving the dual
problem! And sometimes, the dual problem
can be much more simple than the primal
one.
Primal-Dual relationships
Following points are important to be noted regarding primal-dual relationship:
Primal Dual
Maximization Minimization
Minimization Maximization
Ith
variable ith
constraint
jth
constraint jth
variable
xi >=0 Inequality sign of Ith
constraint:
<= if dual is maximization
>= if dual is minimization
Ith
variable unrestricted Ith
constraint with = sign jth
constraint with = sign jth
variable unrestricted
RHS of jth
constraint Cost coefficient associated with jth
variable in the objective function
Cost coefficient associated with ith
variable in the objective function
RHS of ith
constraint
Primal and Dual Simplex Methods
We’ll consider the smart robber problem, that is we can steal bills or gold, but the total stolen
volume is limited, as well as the total weight stolen.
The simplex method
Recall that, in a linear program, there is necessarily an extreme point that is the optimum.
So we could just list all the extreme points and choose the best one…
There are two problems with listing all the extreme points. First, there can be a lot of them,
especially when considering problems with millions of variables and hundreds of thousands
constraints. Second, finding an extreme point can be quite difficult as it involves solving a
system with all the constraints that may not lead to a feasible solution. Shortly said, listing all the
extreme points is a bad idea.
What the simplex method does is moving from extreme points to strictly better extreme
points until finding an optimal extreme points. This simple idea is considered as one of the
main breakthroughs of the 20th century in
computing!
At any extreme point, the simplex method seeks
for a next feasible constraint base to visit, which is
strictly better than the current constraint base. In
order to do that, it looks at the possible direction
where it can go to reach a next extreme point. The
figure shows what the simplex method would do in
our case:
The simplex method has become famous and has
been used a lot as it enabled the resolution of
problems with millions of variables and hundreds
of thousands of constraints in reasonable time. However, it faces problems in cases of
degeneracy.
Solving an example with simplex method:
Pancakes
3 cups Bisquick
1 cup Milk
2 Eggs
Serves: 6
Waffles
2 cups Bisquick
2 cup Milk
2 Eggs
Serves: 5
You have 24 cups of Bisquick, 18 cups of milk, and 20 eggs. If you want to feed as many people
as possible, how many batches of each should you make?
Step 1
All information about example
Resource Pancakes (p) Waffles (w) Constraints
Bisquick (cups) 3 2 24
Milk (cups) 1 2 18
Eggs 2 2 20
Serves 6 5
Objective Function Z= 6p+5w
Bisquick Constraint 3p+2w ≤ 24
Milk Constraint p+2w ≤ 18
Egg Constraint 2p+2w ≤ 20
Non-negativity conditions p,w ≤ 0
The first step of the simplex method requires that each inequality be converted into an equation.
”less than or equal to” inequalities are converted to equations by including slack variables.
Suppose S1 bisquick cups, S2 milk cups and S3 eggs remain unused in a week. The constraints
become;
3p+2w+S1 = 24
p+2w+S2 = 18
2p+2w+S3 = 20
As unused hours result in no profit, the slack variables can be included in the objective function
with zero coefficients:
Z= 6p+5w+0S1+0S2+0S3
The problem can now be considered as solving a system of 3 linear equations involving the 5
variables in such a way that Z has the maximum value;
Now, the system of linear equations can be written in matrix form. The initial tableau is;
Step 2
Table 1
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3
0 S1 24 3 2 1 0 0
0 S2 18 1 2 0 1 0
0 S3 20 2 2 0 0 1
Zj 0 0 0 0 0 0
Cj-Zj - 6 5 0 0 0
Step 3
Select the pivot column (determine which variable to enter into the solution mix). Choose the
column with the “most positive” element in the objective function row.
Table 2
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3
0 S1 24 3 2 1 0 0
0 S2 18 1 2 0 1 0
0 S3 20 2 2 0 0 1
Zj 0 0 0 0 0 0
Cj-Zj - 6 5 0 0 0
p should enter into the solution mix because each batch of p (pancake) serves 6 compared with
only 5 for each batch of w (waffle).
Step 4
Select the pivot row (determine which variable to replace in the solution mix). Divide the
solution element in each row by the corresponding element in the pivot column. The pivot row is
the row with the smallest non-negative result.
Table 3
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3 Ratio
0 S1 24 3 2 1 0 0 8
0 S2 18 1 2 0 1 0 18
0 S3 20 2 2 0 0 1 10
Zj 0 0 0 0 0 0
Cj-Zj - 6 5 0 0 0
Pivot column
Pivot row
and 3 is pivot
number
S1 Should be replaced by p in the solution mix. 10 pancakes can be made with 20 unused eggs
and 18 pancakes can be made with 18 unused milk cups but only 8 pancakes can be made with
24 bisquick cups. Therefore we decide to make 8 tables.
Now calculate new values for the pivot row. Divide every number in the row by the pivot
number.
Table 4
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3
6 p 8 1 2/3 1/3 0 0
0 S2 18 1 2 0 1 0
0 S3 20 2 2 0 0 1
Zj 0 0 0 0 0 0
Cj-Zj - 6 5 0 0 0
Use row operations to make all numbers in the pivot column equal to 0 except for the pivot
number which remains as 1.
Table 5
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3
6 P 8 1 2/3 1/3 0 0
0 S2 10 0 4/3 -1/3 1 0
0 S3 4 0 2/3 -2/3 0 1
Zj 48 6 4 2 0 0
Cj-Zj - 0 1 -2 0 0
If 8 pancakes are made, then the unused milk cups are reduced by 8 cups (1 cup/pancake
multiplied by 8 pancakes); the value changes from 18 cups to 10 cups. similarly, unused eggs are
reduced by 16 (2 egg/pancake multiplied by 8 pancakes); the value changes from 20 eggs to 4
eggs . Making 8 pancakes results in the service being increased by 48; the value changes from 0
to 48.
Now repeat the steps until there are no positive numbers in the last row.
Select the new pivot column. w should enter into the solution mix.
Select the new pivot row. S3 should be replaced by w in the solution mix.
Table 6
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3 Ratio
6 P 8 1 2/3 1/3 0 0 12
0 S2 10 0 4/3 -1/3 1 0 15/2
0 S3 4 0 2/3 -2/3 0 1 6
Zj 48 6 4 2 0 0
Cj-Zj - 0 1 -2 0 0
Calculate new values for the pivot row. Divide every number in the row by the pivot number.
R1/3
R2 - R1
R3 -2 R1
Use row operations to make all numbers in the pivot column equal to 0 except for the pivot
number.
Table 7
Cj
6 5 0 0 0
Basic
variable Solution P w S1 S2 S3
6 P 4 1 0 1 0 -1
0 S2 2 0 0 1 1 -2
5 W 6 0 1 -1 0 3/2
Zj 54 6 5 1 0 3/2
Cj-Zj - 0 0 -1 0 -3/2
If 6 batches of waffles are made, then the number of pancakes is reduced by 4 of batches
pancakes (2/3 pancake/waffle multiplied by 6 batches of waffles); the value changes from 8
batches of pancakes to 4 batches of pancakes. The replacement of 4 batches of pancakes by 6 of
batches waffles results in the serving being increased by 8; the value changes from 48 to 54.
As the last row contains no positive numbers, this solution gives the maximum value of Z.
The dual simplex
I’m now going to explain what’s happening in the dual as we apply the simplex method to the
primal. This will help us define another way of solving linear programs, known as the dual
simplex. In order to have interesting things to talk about, we’ll assume that the optimal solution
is to steal bills only. This gives us the following primal-dual spaces.
We must choose the worst better dual value because we need to keep the dual reduced costs
positive (if the dual program is a minimization program), so that when we reach a feasible dual
base, we’ll be at the optimum for sure.
Each pivot in the dual space should be done as the one we just did. From the dual orange dot,
only the dual yellow constraint isn’t valid, so we have to get it into the dual constraint base,
which leads us to choose the worst dual value between the yellow and pink dots. The pink dot
has a worse dual value, so we pivot towards it.
Then, from the dual pink dot, only the blue constraint isn’t satisfied, so we’ll get it into the dual
constraint base. We then have to choose between the cyan and the white dots, but, as the white
dot has a worse dual value, we cannot go there. So we’ll go to the cyan dot. The cyan dot is
feasible, thus it is the dual optimum.
Dual Simplex Method
Computationally, dual simplex method is same as simplex method. However, their approaches
are different from each other. Simplex method starts with a non-optimal but feasible solution
whereas dual simplex method starts with an optimal but infeasible solution.
Simplex method maintains the feasibility during successive iterations whereas dual simplex
method maintains the optimality. Steps involved in the dual simplex method are:
The procedure for forming the dual problem is summarized below:
Formation of the dual problem
Given the maximization problem with the problem constraints,
Step1. Use the coefficients and constants in the problem constraints and the objective function to
form a matrix A with the coefficients of the objective function in the last row.
Step2. Interchange the rows and columns of matrix AT, the transpose of A.
Step3. Use the rows of AT to form a minimization problem with ≥ problem constraints.
Forming the dual problem
Primal problem in
algebraic form
Maximise Z= 6p+5w
Subject to
3p+2w ≤ 24
p+2w ≤ 18
2p+2w ≤ 20
And p,w ≥ 0
Primal problem
2 2 24
1 2 18
2 2 20
6 5 1
Dual problem
2 1 2 6
2 2 2 5
24 18 20 1
Dual problem in
algebraic form
Minimise
Z= 24x+18y+20z
Subject to
3x+y+2z ≥ 6
2x+2y+2z ≥ 5
And x,y,z ≥ 0
Step4. Modified problem, as in step 3, is expressed in the form of a simplex tableau.
Table 8
Cj
24 18 20 0 0 M M
Basic
variable Solution X Y Z S1 S2 A1 A2 Ratio
M A1 6 3 1 2 -1 0 1 0 2
M A2 5 2 2 2 0 -1 0 1 5/2
Zj 11M 5M 3M 4M -M -M M M
Cj-Zj - 24-5M 18-3M 20-4M M M 0 0
Step5. Selection of exiting variable: The basic variable with the highest negative value is the
exiting variable. If there are two candidates for exiting variable, any one is selected. The row of
the selected exiting variable is marked as pivotal row.
Step6. Selection of entering variable: The column corresponding to minimum ratio is identified
as the pivotal column and associated decision variable is the entering variable.
Step7. Pivotal operation: Pivotal operation is exactly same as in the case of simplex method,
considering the pivotal element as the element at the intersection of pivotal row and pivotal
column.
Step8. Check for optimality: If all the basic variables have nonnegative values then the
optimum solution is reached. Otherwise, Steps 3 to 5 are repeated until the optimum is reached.
Table 9
Cj
24 18 20 0 0 M
Basic
variable Solution X Y Z S1 S2 A2 Ratio
24 X 2 1 1/2 2/3 -1/3 0 0 4
M A2 1 0 1 2/3 2/3 -1 1 5/2
Zj 48+M 24 12+M 16+2/3M -8+2/3M -M M
Cj-Zj - 0 6-M 4-2/3M 8+2/3 M M 0
Table 10
Cj
24 18 20 0 0
Basic
variable Solution X Y Z S1 S2
24 X 3/2 1 0 1/3 -2/3 1/2
18 Y 1 0 1 2/3 2/3 -1
Zj 54 24 18 20 -4 -6
Cj-Zj - 0 0 0 4 6
Economic interpretation of the dual
The optimal solution to primal problem has already been found and presented in table 7. If we
compare table 7 and 10, we observe that objective functions of the two tables assume identical
values, i.e., 54. We also note that values in the opportunity cost row under column S1 and S2 of
the optimal table of the dual are the same as values under the “solution” column in the optimal
table of the primal problem. We further observe that magnitude of variables x and y are exactly
the same as the entries (with sign changed) in the opportunity cost row under columns S1 and S3
of the optimal table of the primal problem.
Advantages
Knowledge of dual is useful because solution of a linear programming problem may be easier to
obtain through the dual than through the primal problem. For example, consider a primal
problem involving three products all of which have to be processed on eight machines. The
initial simplex table for this problem will have eight rows. Where the same problem can be
solved via its dual, for which the initial table will have only three rows.
There’s something else quite interesting about duality: It gives directly a sensitivity analysis.
Consider our primal problems. It’d be interesting to know how much more I could get, had my
knapsack been larger or my body stronger. This looks like a difficult question in the primal
program, but it’s very obvious in the dual program. If I can have one unit of volume more in the
knapsack, then the incremental worth of the robbery will be Value Volume.
Modern advances in linear programming theory establish the use of the dual simplex algorithm
as a powerful optimization tool. While the performance of the dual simplex was originally
considered to lag that of its most popular primal variant, the dual simplex is widely used today in
practice. One of the more popular applications of the algorithm includes large-scale mixed
integer programming, where row additions break primal feasibility, but typically, dual feasibility
remains intact.
All these interesting properties of the method, coupled with the fact that linear optimization is a
fundamental topic in operations research, ensure that advances in this area will continue to be
positioned at the forefront of the discipline.