1

Linear Programming

Jose RolimUniversity of Geneva

L.P. Jose Rolim 2

What is Linear Programming?

Linear programming (LP) is a mathematical method for selecting the best solution from the available solutions of a problem.

Method:• State the problem and define variables whose values will

be determined.• Develop a linear programming model:

Write the problem as an optimization formula (a linear expression to be minimized or maximized)

Write a set of linear constraints• An available LP solver (computer program) gives the values

of variables.

L.P. Jose Rolim 3

Types of LP

LP – all variables are real.

ILP – all variables are integers.

MILP – some variables are integers, others are real.

L.P. Jose Rolim 4

A single variable problem

Consider variable x Problem: find the maximum value of

x subject to constraint, 0 ≤ x ≤ 15. Solution: x = 15.

L.P. Jose Rolim 5

Single Variable Problem (Cont.)

Consider more complex constraints: Maximize x, subject to following constraints

• x ≥ 0 (1)• 5x ≤ 75 (2)• 6x ≤ 30 (3)• x ≤ 10 (4)

0 5 10 15x (1)

(2)(3)

(4)

All constraints satisfied Solution, x = 5

L.P. Jose Rolim 6

A Two-Variable Problem

Manufacture of x1 chairs and x2 tables: Maximize profit, P = 45x1 + 80x2 dollars Subject to resource constraints:

• 400 boards of wood, 5x1 + 20x2 ≤ 400 (1)

• 450 man-hours of labor, 10x1 + 15x2 ≤ 450 (2)

• x1 ≥ 0 (3)

• x2 ≥ 0 (4)

L.P. Jose Rolim 7

Solution: Two-Variable Problem

Chairs, x1

Tab

les,

x2

(1)

(2)

0 10 20 30 40 50 60 70 80 90

40

30

20

10

0

(24, 14)

Profi

t increasing

decresing

P = 2200

P = 0

Best solution: 24 chairs, 14 tablesProfit = 45×24 + 80×14 = 2200 dollars

L.P. Jose Rolim 8

Change Chair Profit, $64/Unit

Manufacture of x1 chairs and x2 tables: Maximize profit, P = 64x1 + 80x2 dollars Subject to resource constraints:

• 400 boards of wood, 5x1 + 20x2 ≤ 400 (1)

• 450 man-hours of labor, 10x1 + 15x2 ≤ 450 (2)

• x1 ≥ 0 (3)

• x2 ≥ 0 (4)

L.P. Jose Rolim 9

Solution: $64 Profit/Chair

Chairs, x1

Tab

les,

x2

(1)

(2)

Profi

t increasing

decresing

P = 2880

P = 0

Best solution: 45 chairs, 0 tablesProfit = 64×45 + 80×0 = 2880 dollars

0 10 20 30 40 50 60 70 80 90

(24, 14)

40

30

20

10

0

L.P. Jose Rolim 10

Motivation: A Political Problem

Goal: Win election by winning majority of votes in each region.Goal: Win election by winning majority of votes in each region.

Subgoal: Win majority of votes in each region while Subgoal: Win majority of votes in each region while minimizing advertising cost.minimizing advertising cost.

100,000 voters

200,000 voters

50,000 voters

Thousands of voters who could be won with $1,000 of ads

L.P. Jose Rolim 11

Motivation: A Political Problem (continued)

Thousands of voters representing majority.

urban

suburban

rural

L.P. Jose Rolim 12

General Linear Programs

real numbers

variables

Linear function

Linear inequalities

Linear constraints

L.P. Jose Rolim 13

Overview of Linear Programming

Convex feasible region

Objective function

Objective value

L.P. Jose Rolim 14

Standard Form

objective function

constraints

..

L.P. Jose Rolim 15

Standard Form (compact)

mxn matrixm-dimensional vector

n-dimensional vectors

Can specify linear program in standard form by (A,b,c).Can specify linear program in standard form by (A,b,c).

L.P. Jose Rolim 16

Converting to Standard Form

L.P. Jose Rolim 17

Converting to Standard Form (continued)

Negate coefficients

Transforming minimization to maximizationTransforming minimization to maximization

L.P. Jose Rolim 18

Converting to Standard Form (continued)

If xj has no non-negativity constraint,

replace each occurrence of xj with xj’ – xj”.

Giving each variable a non-negativity constraintGiving each variable a non-negativity constraint

New non-negativity constraints

L.P. Jose Rolim 19

Converting to Standard Form (continued)

Transforming equality constraints to inequality constraintsTransforming equality constraints to inequality constraints

L.P. Jose Rolim 20

Converting to Standard Form (continued)

..

Changing sense of an inequality constraintChanging sense of an inequality constraint

Rationale:

L.P. Jose Rolim 21

Converting Linear Programs into Slack Form

for algorithmic ease, transform all constraints except for algorithmic ease, transform all constraints except non-negativity ones into equalitiesnon-negativity ones into equalities

for inequality constraint:

define slack

slack variable

instead of s

basic

variables

non-basic variables

L.P. Jose Rolim 22

Converting Linear Programs into Slack Form (continued)

objective function

L.P. Jose Rolim 23

Converting Linear Programs into Slack Form (continued)

Compact Form: (N, B, A, b, c, v)Compact Form: (N, B, A, b, c, v)

set of indices of non-basic variables

set of indices of basic variables

Slack Form Example Compact Form

negative of slack form coefficients

L.P. Jose Rolim 24

Shortest Paths

..

Single-pair shortest path: minimize “distance” from source s to sink t.Single-pair shortest path: minimize “distance” from source s to sink t.

Can we replace maximize with minimize here? Why or why not?

L.P. Jose Rolim 25

Maximum Flow

..

L.P. Jose Rolim 26

Minimum Cost Flow

..

L.P. Jose Rolim 27

Multicommodity Flow

..

should be si

L.P. Jose Rolim 28

Solving a Linear Program

Simplex algorithm Geometric interpretation

• Visit vertices on the boundary of the simplex representing the convex feasible region

Transforms set of inequalities using process similar to Gaussian elimination

Run-time • not polynomial in worst-case• often very fast in practice

Ellipsoid method Run-time

• polynomial• slow in practice

Interior-Point methods Run-time

• polynomial• for large inputs, performance can be

competitive with simplex method Moves through interior of feasible region

L.P. Jose Rolim 29

Simplex Algorithm: ExampleBasic Solution

Standard Form

Slack Form

Basic Solution: set each nonbasic variable to 0.Basic Solution: set each nonbasic variable to 0.

Basic Solution: )36,24,30,0,0,0(),( 621 xxx

L.P. Jose Rolim 30

Simplex Algorithm: Example Reformulating the LP Model

Main IdeaMain Idea: In each iteration, reformulate the LP : In each iteration, reformulate the LP model so basic solution has larger objective valuemodel so basic solution has larger objective value

Select a nonbasic variable whose objective coefficient is positive: x1

Increase its value as much as possible.

Identify tightest constraint on increase.

For basic variable x6 of that constraint, swap role with x1.

Rewrite other equations with x6 on RHS.

PIVOT

leaving variable

entering variable

new objective value

L.P. Jose Rolim 31

Simplex Algorithm: Example Reformulating the LP Model

Next Iteration: select xNext Iteration: select x33 as entering variable. as entering variable.

PIVOT

leaving variable

entering variable

)0,0,4/69,2/3,0,4/33(),( 621 xxx New Basic Solution:

new objective value

L.P. Jose Rolim 32

Simplex Algorithm: Example Reformulating the LP Model

)0,0,18,0,4,8(),( 621 xxx

..

Next Iteration: select xNext Iteration: select x22 as entering variable. as entering variable.

PIVOT

leaving variable

entering variable

New Basic Solution:

new objective value

L.P. Jose Rolim 33

Simplex Algorithm: Pivoting

leaving variable entering variable

Rewrite the equation that has xl on LHS to have xe on LHS

Update remaining equations by substituting RHS of new equation for each occurrence of xe.

Do the same for objective function.

Update sets of nonbasic, basic variables.

L.P. Jose Rolim 34

Simplex Algorithm: Pseudocode

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

to be defined later (detects infeasibility)

initial basic solution

optimal solution

detects unboundedness

L.P. Jose Rolim 35

Finding an Initial Solution

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

An LP model whose initial basic solution is not feasibleAn LP model whose initial basic solution is not feasible

L.P. Jose Rolim 36

Finding an Initial Solution(continued)

Auxiliary LP model LAuxiliary LP model Lauxaux::

L.P. Jose Rolim 37

Finding an Initial Solution(continued)

..

L.P. Jose Rolim 38

Finding an Initial Solution(continued)

Original LP model

Laux

Laux in slack form

L.P. Jose Rolim 39

Finding an Initial Solution(continued)

PIVOT

PIVOT

L.P. Jose Rolim 40

Finding an Initial Solution(continued)

L.P. Jose Rolim 41

Linear Programming Duality

max becomes min

RHS coefficients swap places with objective function coefficients

sense changes

x variables go away

y variables appear

L.P. Jose Rolim 42

Duality Example

L.P. Jose Rolim 43

Weak Linear Programming Duality

Any feasible solution to primal LP has value no greater Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.than that of any feasible solution to the dual LP.

L.P. Jose Rolim 44

Weak Linear Programming Duality (continued)

L.P. Jose Rolim 45

Finding a Dual Solution

Finding a dual solution whose value is equal to that of Finding a dual solution whose value is equal to that of an optimal primal solution…an optimal primal solution…

L.P. Jose Rolim 46

Optimality

..