OR-1 2014 1

Chapter 4. How fast is the simplex method

Efficiency of an algorithm : measured by running time with respect to the problem size.

Two models of computation time:arithmetic operation model: assumes each operation (addition,

multiplication, …) takes unit time.bit operation model: counts the number of “bits operations”.

depends on the sizes of numbers written in binary notation. Size of problem : amount of information (size of storage, length of

encoding) to represent the problem in computer (using 2 alphabets 0 and 1)

Ex) positive integer : rational number : m n matrix A :

( means smallest integer not less than , e.g. , )

Size of an LP is a function of ( is largest number (absolute value) in LP, assuming data are given in integers)

OR-1 2014 2

Running time : worst case view point

An algorithm is considered efficient if the worst case running time is bounded from above by a polynomial function of prob-lem size (for large instances only, small instances can be solved fast anyway). (called polynomial time algorithm)

Size of problem encoding

Running time

𝑡

OR-1 2014 3

function is called (is read big oh of ) if there exist a positive constant and a positive integer such that when .e.g.)

Why polynomial function?Suppose it takes 1 sec in running time when , then

Suppose one iteration of simplex method takes polynomial time of the input size (which is true). Then, if the number of iterations is a polynomial function of input length for any LP, it implies that simplex method is a polynomial time algorithm.

10 20 50 60

100 sec 400 sec 2500 sec 3600 sec

1000 sec 1 sec 35.7 years 366 cen-turies

OR-1 2014 4

Empirically, the number of iterations is and

Bad counter example : Klee and Minty (1972)maximize subject to ,

,

If we use largest coefficient rule, number of iterations is , hence running time is not polynomial

OR-1 2014 5

ex) max s.t.

Note that , and feasible region is given approximately as

Hence it is an elongated skewed hypercube with extreme points.(Note that we need 3 equations to define an extreme point in this example. Only one of the lower and upper bound constraints for each variable can be chosen to define an extreme point, hence total of extreme points exist.)

Simplex method with largest coefficient rule searches all ex-treme points until it finds the optimal solution. But, largest in-crease rule finds the optimal solution in one iteration.

OR-1 2014 6

Geometric view

1x

2x

3x

largest increaserule

OR-1 2014 7

Alternative rules :Largest increase rule : counter example by Jeroslow (1973)Bland’s rule : counter example by Avis and Chvatal (1978)

Hence, until now, simplex method is not a polynomial time algo-rithm theoretically. But it performs very well practically.

First polynomial time algorithm for LP : ellipsoid method (1979) by L. G. Khachian. But, practically much inferior to simplex. However, it has some important theoretical implications in de-termining computational complexity of some optimization prob-lems.

Another polynomial time algorithm : Interior point method by L. Karmarkar (1984) : Better than simplex in many cases practically. Many versions.Based on ideas from nonlinear programming.

OR-1 2014 8

Interior point method will be briefly mentioned when we study complementary slackness theorem.

Comparing simplex and interior point method:Interior point method is generally fast, especially for large prob-lems. But simplex is competitive for some problems and recent developments in dual simplex algorithm makes solving LP of large size manageable.In addition, simplex is effective when we solve the LP again after making some changes in data (reoptimization). Such capability is quite important when we solve integer programming problems. But little progress has been made for interior point algorithm in this respect. Recently, interior point method is used for some nonlinear pro-gramming problems (convex programs) with successful results.