A Faster Algorithm for Linear Programming and the Maximum Flow

A Faster Algorithm for Linear Programming and the Maximum Flow Problem I

Yin Tat Lee(MIT, Simons)

Joint work with Aaron Sidford

THE PROBLEM

Linear Programming

Consider the linear program (LP)

where is a matrix.

• is the number of constraints.• is the number of variables.

𝑛=2 ,𝑚=6𝑛=2 ,𝑚=∞

Previous Results

• All of them are iterative methods.

• Start with some initial point .• While is not optimal– Improve

• Time =

• This talk focus on #iter.

= # of constraints = # of variables

We call it efficient if• Polynomial time• Doesn’t use LP solver

Previous Results (Selected)

Year Author # of iter Cost per iter Efficient steps

1947 Dantzig Pivot Yes

1979 Khachiyan Update the Ellipsoid Yes

1984 Karmarkar Solve linear systems Yes

1986 Renegar Solve linear systems Yes

1989 Vaidya Matrix inverse Yes

1994Nesterov,

Nemirovskii Compute volume No

2013 Lee, SidfordSolve Linear Systems Yes

Solve Linear Systems Yes

= # of constraints = # of variables( is omitted)

Remark: In 2013, Mądry shows how to obtain iters for certain LPs!

Outline

Year Author # of iter Cost per iter Efficient steps

1947 Dantzig Pivot Yes

1979 Khachiyan Update the Ellipsoid Yes

1984 Karmarkar Solve linear systems Yes

1986 Renegar Solve linear systems Yes

1989 Vaidya Matrix inverse Yes

1994Nesterov,

Nemirovskii Compute volume No

2013 Lee, SidfordSolve Linear Systems Yes

Solve Linear Systems Yes

= # of constraints = # of variables( is omitted)

Remark: In 2013, Mądry shows how to obtain iters for certain LPs!

LP AND CENTER

A general framework

We can solve linear program by maintaining center.

Somehow, get a “center” first Put the cost constraint there and move it.

Say we can move portion closer each time

After steps, we are done.

A general framework

We can solve linear program by maintaining center.

Somehow, get a “center” first Put the cost constraint there and move it.

Say we can move portion closer each time

After steps, we are done.Why center?

What if we don’t try to maintain a center?

• It is just like simplex method.

It is good now. Still good.

Oh, it touches. What to do?…..

What if we don’t try to maintain a center?

• It is just like simplex method.

It is good now. Still good.

Oh, it touches. What to do?…..

Avoid bad decision by using

global information!

A general framework

Formally, we have (say ):• . Find the center of • While is large– for some fixed – Update the center of

This is called interior point method.

The initial point is easy:

A general way to define a center

Let be a smooth convex function on such that• as .

For example,Standard log barrier:

BarrierFunction

Center

QUALITY OF A CENTER

Rounding

• Assume center is induced by some barrier function .• Look at the ellipsoid induced by at the center .• Call is rounding if for some .

Self concordant barrier

• is a -self concordant barrier function for if– is smooth.– gives rounding.

is not smooth enough Bad rounding.

Rounding Algorithm

For general barrier function :• Repeat– Tighten the cost constraint– Maintain the rounding ellipsoid induced by .

Why iterations?

Why iterations?

Think .• Newton Method (Using smoothness)Given , we can find the center in steps.

Why iterations?

Let be the old center. Using the smoothness, we have

Why iterations?

So, we need

It takes iters.

Why iterations?

• We can reduce the gap by .

Roughly Speaking:Smoothness + rounding gives

iterations LP solvers.

Quality of analytic center is arbitrary bad in !

• Recall the standard log barrier function

• The center is called analytic center.

Is it tight?

• In practice, it takes steps.• Mizuno, Todd, Ye showed it is “usually” correct on first step.• In 2014, Mut and Terlaky showed an example really takes

iterations where is exponential in .

UNIVERSAL BARRIER FUNCTION

Universal Barrier Function

Theorem [NN94]: For any convex set ,

is a -self concordant barrier function.

“Smaller” set has larger polar. Hence, as

Note that .

Kannan-Lovasz-Simonovits Lemma: For any convex set , the second moment matrix

gives a rounding of .

The cost of Universal Barrier

• To get second moment matrix, you need sampling.• To get 1 sampling, you need to do iters of Markov chain.• To do 1 iter of Markov chain, you need to implement

separation oracle for .• If , one need to solve an LP.

Hence, one iteration requires solving many LPs.

The problem:Get an efficient self concordant barrier

function.

VOLUMETRIC BARRIER FUNCTION

Volumetric Barrier Function

In 1989, Vaidya showed

where Why it is volumetric?

For example:

It is a barrier.

Log BarrierVolumetric Barrier

Why Volumetric is good?

Around , we have

where

Example: . Then, . , , .In general, , , if the row is repeated, is decreased by .

For [0,1] interval with 0 repeated times:

𝑆−1 𝐴=(11⋮ ) 𝑆−1 𝐴=(𝑘1/3

1⋮ )

OUR BARRIER FUNCTION

Repeated Volumetric Barrier Function

How about ?

Suppose , around , we have

So, we have

We call where satisfiesWhat is that?

What is that weight?

• Let .

If for all , the ellipsoid is inside.

Our Condition (John Ellipsoid): if .

𝑤𝑖(∞ )(𝑥)=𝜎 𝑖 (√𝑊 (∞ )𝑆𝑥

−1 𝐴) .

The represents John ellipsoid of

Repeated Volumetric Barrier Function

• Recall

We get

Symmetrize Find John Ellipsoid

The barrier function is not perfect!

• The path is piecewise smooth because it may not touch every constraint.

Our Barrier Function

• Standard Log Barrier:

• Volumetric Barrier:

• John Ellipsoid Barrier:

• Regularized John Ellipsoid Barrier (1):

• Regularized John Ellipsoid Barrier (2):

Lewis Weight

We call is Lewis weight for if

Thanks to Cohen and Peng, we know• Let be rows sample of accordingly to ,

• is the maximizer of

i.e, the maximum ellipsoid such that it “insides” the polytopes.• For , is the John ellipsoid for .

max𝑤 ≥ 0

lndet (𝐴𝑇𝑆−1𝑊1−log−1 (𝑚𝑛 )

𝑆− 1𝐴) .

Computing Lewis Weight

• Cohen and Peng showed how to compute it when .• The repeated volumetric barrier: ,

After renormalization, gives “ “Lewis” weight:

• Cohen, L., Peng, Sidford shows that in fact a similar algorithm find constant “approximate” Lewis weight for in .

CONCLUSION

Our Barrier

Given any polytope , let

Theorem: The barrier function gives iterations algorithm for LP of the form

Algorithm:• While– Move the cost constraint– Maintain the regularized John ellipsoid

= # of constraints = # of variables

However…

• My goal is to design general LP algo fast enough to

beat the best maxflow algorithm!

• We obtained

= # of constraints = # of variables

Compute   ( 𝐴𝑇𝐷𝑘 𝐴 )−1min𝐴𝑥 ≥𝑏

𝑐𝑇 𝑥