Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Problems

Convex Relaxations of Non-Convex Mixed Integer Quadratically

Constrained Problems

Dr. Anureet SaxenaAssociate, Research

Axioma Inc.

(Joint Work with Pierre Bonami and Jon Lee)

Dedicated to Prof. Egon Balas

2Anureet Saxena, Axioma Inc.

min aT0xst

xTA ix + aTi x + bi · 0; i = 1 : : :mxj 2 Z; j 2 N I

l · x · u

Integer Constrained VariablesSymmetric Matrices

NOT necessarily positive semidefinite

min aT0xst

A i:Y + aTi x + bi · 0; i = 1 : : :mxj 2 Z; j 2 N I

l · x · uY = xxT

yi j = xi xj

Research Question?

Determine lower bounds on the optimal value of MIQCP by constructing strong convex relaxations of MIQCP.

Disjuncti

ve Programming

Disjunctive Programming

P = f x j Ax ¸ bg

DisjunctionPolyhedral Relaxation

Separation Problem

Given x2P show that x2PD or find an inequality which is satisfied by all points in PD and is violated by x.

Anureet Saxena, Axioma Inc.

T heorem: x 2 PD if and only if the optimalvalue of the following cut generating linear pro-gram (CGLP ) is non-negative.

min ®x ¡ ¯s:t

®= utA + vtD t 8t = 1 : : :q

¯ · utb+ vtdt 8t = 1 : : :q

ut;vt ¸ 0 8t = 1 : : :q

P qt=1(u

t»+ vt»t) = 1

P = f x j Ax ¸ bg

Outer Approximation of MIQCP defined by the incumbent solution

P = f x j Ax ¸ bg

What are the sources of non-convexity in

MIQCP?

P = f x j Ax ¸ bg

• xj2 Z j2 NI

• Elementary 0-1 disjunction

(xj · 0) OR (xj ¸ 1)

• Split Disjunctions

• GUB Disjunctions

Integrality Constraints

All eigenvalues of Y-xxT are equal to zero.

Eigenvectors of Y-xxT associated with non-zero eigenvalues can be used as sources of cuts

Ohh!!I don’t like fractional components. I can use them to get good cuts

Ohh!!I don’t like non-zero eigenvalues. I can use them to get good cuts

Negative Eigenvalues of Y-xxT

If (Y ¡ xxT )c = ¸c where ¸ < 0 then

² (cTx)2 · Y:ccT is a convex quadratic cutwhich cuts o®(x; Y )

² equivalent to imposing the SDP conditionY ¡ xxT ¸ SDP 0 by SOCP cuts.

Positive Eigenvalues of Y-xxT

If (Y ¡ xxT )c = ¸c where ¸ > 0 then

Y:ccT · (cTx)2

is a non-convex quadratic cut which cuts o®(x; Y ).

Univariate non-convex expression

Y:ccT · t2

t = cTx

min(x;Y )2OA cTx max(x;Y )2OA cTx

Y:ccT · (cTx)2

(cTx)2

p(cTx) + q

Y.ccT· p(cTx) + q

SecantApproximation

p1(cTx) + q1 p2(cTx) + q2

µµL µU

"µL (c) · cTx · µ

Y:ccT · p1(cTx) + q1

"µ · cTx · µU(c)

Y:ccT · p2(cTx) + q2

Cutting Plane Algorithm

(x; Y )

(cTx)2 · Y:ccT Y:ccT · (cTx)2

Derive Disjunction

Derive DisjunctiveCut

Convex Quadratic Cut

¸ > 0¸ < 0

Extract E igenvaluesand E igenvectors ofY ¡ xxT .

Sequential ConvexificationT heorem Let c1; : : : ; cn denote a set of mutually-orthogonal unit vectors in Rn, and let

>:(x;Y )

¯¯¯¯¯¯¯

A i:Y + aTi x + bi · 0 i = 1 : : :ml · x · u

Y ¡ xxT ¸ SDP 0

Sj = clconv³Sj ¡ 1 \

n(x;Y ) j Y:cj cTj · (cTj x)

f or j = 1 : : :n

Sn+j = clconv³Sn+j ¡ 1 \

n(x;Y ) j xj 2 f 0;1g

o´f or j = 1 : : :p

T he following statements hold true:

Sn = clconv

>:(x;Y )

¯¯¯¯¯¯¯

A i:Y + aTi x + bi · 0 i = 1 : : :ml · x · u

Y ¡ xxT = 0

Sn+p = clconv

8>>>><

(x;Y )

¯¯¯¯¯¯¯¯¯¯

A i:Y + aTi x + bi · 0 i = 1 : : :ml · x · u

Y ¡ xxT = 0xj 2 f 0;1g j = 1 : : :p

9>>>>=

Y.ccT · (cT x)2

Can we improve the disjunctive cuts by choosing c more

carefully?

We are searching for vectors c which satisfy,

1. Y :ccT > (cT x)2

2. max(x;Y )2OAcTx ¡ min(x;Y )2OAc

Tx is assmall as possible

Improving Disjunctions?

This condition is always satisfied if c belongs to vector space spanned by eigenvectors of Y-xxT associated with positive eigenvalues.

This can be calculated by solving a linear program whose right hand side is a linear function of c

We are searching for vectors c which satisfy,

1. Y :ccT > (cT x)2

2. max(x;Y )2OAcTx ¡ min(x;Y )2OAc

Tx is assmall as possible

Improving Disjunctions?

This condition is always satisfied if c belongs to vector space spanned by eigenvectors of Y-xxT associated with positive eigenvalues.

This can be calculated by solving a linear program whose right hand side is a linear function of c

This problem can be formulated as a mixed integer linear program!!

Univariate Expression Generating Mixed Integer Program (UGMIP)

Cutting Plane Algorithm

Extract E igenvaluesand E igenvectors ofY ¡ xxT .

(x; Y )

(cTx)2 · Y:ccT Y:ccT · (cTx)2

Derive Disjunction

Derive DisjunctiveCut

Convex Quadratic Cut

¸ > 0¸ < 0

MIQCP Reformulations

MIQCP (x)

MIQCP (x,Y)MIQCP (x,Y)RLT + SDP

Disjunctive Cuts

MIQCP (x)Projected Ineq

Lifting

Strengthening

Strengthening ?

Heavy Relaxation

Light Relaxation

Projection

MIQCP Reformulations

MIQCP (x)

MIQCP (x,Y)MIQCP (x,Y)RLT + SDP

Disjunctive Cuts

MIQCP (x)Projected Ineq

Lifting

Strengthening

Strengthening ?

Heavy Relaxation

Light Relaxation

Projection

Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Problems: Projected Formulations

A. Saxena, P. Bonami and J. Lee

Anureet Saxena, Axioma Inc. 26

Projecting the RLT Formulation

Ak:Y + aTkx + bk · 0 k = 1 : : :m

Yi j ¡³lixj + uj xi ¡ liuj

´· 0 8i; j

Yi j ¡³uixj + lj xi ¡ ui lj

´· 0 8i; j

Yi j ¡³uixj + uj xi ¡ uiuj

´¸ 0 8i; j

Yi j ¡³lixj + lj xi ¡ li lj

´¸ 0 8i; j

RLT Inequalities

y¡ij (x) = max f uixj + uj xi ¡ uiuj ; lixj + lj xi ¡ li lj g 8i; j

y+ij (x) = min f lixj + uj xi ¡ liuj ;uixj + lj xi ¡ ui lj g 8i; j

Ak:Y + aTk x + bk · 0 k = 1 : : :m

y¡ij (x) · Yi j · y+ij (x) 8i; j

P(x;Y )

Qx =nx j 9Y s.t. (x;Y ) 2 P(x;Y )

Separation Problem

Given x show that x2Qx or find an inequality which is satisfied by all points in Qx and is violated by x.

min ´

aTk x + bk · ¡ Ak:Y + ´ k = 1 : : :m

ProjLP

T heorem x 2 Qx if and only if the optimalvalue of P rojLP is non-positive.

Anureet Saxena, Axioma Inc 28

min ´

aTk x + bk · ¡ Ak:Y + ´ k = 1 : : :m

Dual Solution(u, B, C)

¡B i j y¡i j (x) ¡ Ci j y+i j (x)

uk¡aTk x+bk

¢· 0

Projected Inequality

min ´

aTk x + bk · ¡ Ak:Y + ´ k = 1 : : :m

• A linear programming separation algorithm

• Handles large number O(n2) of RLT inequalities as bound

constraints

• No of constraints = No of quadratic constraints in the original problemAnureet Saxena, Axioma Inc 30

Surrogate Constraints

xTA1x + aT1x + b1 · 0

xTAmx + aTmx + bm · 0

xTAx + aTx + b· 0 Surrogate Constraint

¡B i j y¡i j (x) ¡ Ci j y+i j (x)

uk¡aTk x+bk

¢· 0

A = B – CB, C ¸ 0

Surrogate Constraint

y¡ij (x) · xixj · y+ij (x)

Can we extract the convex part of the surrogate constraint

Surrogate Constraints

xTA1x + aT1x + b1 · 0

A = B + C – DB ¸SDP 0C, D ¸ 0

xT Bx+X

¡Ci j y¡i j (x) ¡ Di j y+i j (x)

uk¡aTk x+bk

¢· 0

What happens if we add all such convex

quadratic cuts?

Projecting the SDP Formulationmin ´s.t.¡ Ak:Y +´ ¸ aTk x +bk; 8k 2 My¡i j (x) · Yi j · y+i j (x); 8i; j 2 NY +´I ¡ xxT ¸ SD P 0

xT Bx+X

¡Ci j y¡i j (x) ¡ Di j y+i j (x)

uk¡aTk x+bk

¢· 0

Dual Solution

(u, B, C, D)

T heorem x 2 Q+x if and only if the optimal

value of P rojSDP is non-positive.

ProjSDP

Separation Problem is a SDP

Projecting the SDP Formulation

T heorem x 2 Q+x if and only if the optimal

value of the following piecewise linear convexoptimzation problem is non-positive.

maxf F (u;B ) j u 2 § M ; B ¸ SDP 0g;

where § M = fu jPk2M uk = 1; u ¸ 0g and

F (u;B ) =Pi;j

³ Pk2M ukA

kij ¡ B i j

´+ ³y¡ij (x) ¡ xi xj

³ Pk2M ukA

kij ¡ B i j

´¡ ³y+ij (x) ¡ xi xj

+Pk2M uk(x

TAkx) +Pk2M uk

³aTk x + bk

Unconstrained Convex Optimization Problem over the Cartessian product of a simplex and cone of PSD

matrices

Projecting the SDP Formulation

xTAx + aTx + b· 0A = B + C ¡ DB ¸ SDP 0; C;D ¸ 0

Projected Sub Gradient Heuristic

1. Initialize B = Projection of A to the cone of PSD matrices

2. Compute a sub gradient of F(u,B) at B

3. Perform line search along the sub gradient direction

4. Update B and goto 2

Limitations of Projection Theorems

xTA1x + aT1x + b1 · 0

Once the surrogate constraint has been produced very little global information is used in the convexification process

min x3s.t.x1x2 ¡ x1 ¡ x2 ¡ x3 · 0¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

• st_e23 instances from GlobalLib

• OPT = -1.08

• RLT = -3

• SDP + RLT = -1.5

• x = ( 0.811, 0.689, -1.500)

P1 = clconv

¯¯¯¯¯x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

0 · x1;x2 · 1:5

(1:5;0; ¡ 1:5) 2 P1 (0;1:5; ¡ 1:5) 2 P10:5407 (1:5;0; ¡ 1:5) + 0:4593 (0;1:5; ¡ 1:5) = x0:5407 + 0:4593 = 1

The non-convex quadratic constraint and the bound constraints cannot cut off x

min x3s.t.x1x2 ¡ x1 ¡ x2 ¡ x3 · 0¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

• st_e23 instances from GlobalLib

• OPT = -1.08

• RLT = -3

• SDP + RLT = -1.5

• x = ( 0.811, 0.689, -1.500)

P1 = clconv

¯¯¯¯¯x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

0 · x1;x2 · 1:5

(1:5;0; ¡ 1:5) 2 P1 (0;1:5; ¡ 1:5) 2 P10:5407 (1:5;0; ¡ 1:5) + 0:4593 (0;1:5; ¡ 1:5) = x0:5407 + 0:4593 = 1

Global Information

We need a technique for engaging additional constraints in the

problem during the convexification process

x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

12 (x1 + x2)

2 ¡ x1 ¡ x2 ¡ x3 ·12 (x1 ¡ x2)

Spectral Decomposition of"0 0:50:5 0

Univariate non-convex expression

min (x1 ¡ x2)s:t¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

: : : · (x1 ¡ x2)2

(x1 ¡ x2)

(x1 ¡ x2)2

max (x1 ¡ x2)s:t¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

SecantApproximation

(x1 ¡ x2)

(x1 ¡ x2)2 · 0:625(x1 ¡ x2) + 0:375

x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

12 (x1 + x2)

2 ¡ x1 ¡ x2 ¡ x3 ·12 (x1 ¡ x2)

12 (x1+ x2)

2¡ x1¡ x2¡ x3 ·12 (0:625(x1 ¡ x2) + 0:375)

Spectral Decomposition

Secant ApproximationCuts off the incumbent

solution

Eigen Reformulation

xTAx + aTx + b· 0 A =Pj ¸ j vj v

P¸k>0 ¸k

³vTk x

´2+ aTx + b+

P¸k<0 ¸ksk · 0

yk = vTk x 8 k : ¸k < 0

sk = y2k 8 k : ¸k < 0

Eigen Reformulation

xTAx + aTx + b· 0 A =Pj ¸ j vj v

P¸k>0 ¸k

³vTk x

´2+ aTx + b+

P¸k<0 ¸ksk · 0

yk = vTk x 8 k : ¸k < 0

sk = y2k 8 k : ¸k < 0

Directions of maximal non-convexity

Eigen Reformulation

min aT0xs.t.xTAkx + aTk x + bk · 0 ; 8k 2 Mxj 2 Z ; 8j 2 N1

P¸kj>0 ¸kj

³vTkj x

´2+ aTk x + bk +

P¸kj<0 ¸kj skj · 0 ; 8k 2 M

ykj = vTkj x ; 8 j : ¸kj < 0; k 2 Mskj = y2kj ; 8 j : ¸kj < 0; k 2 ML kj · ykj · Ukj ; 8 j : ¸kj < 0; k 2 M :

Geometric correlations along

directions of maximal non-convexity

Eigen Reformulation

• st_glmp_kky instances from GlobalLib

• OPT = -2.5

• RLT = RLT+SDP = -3.0

x4x5+ x6x7 ¡ x3 ¡ z · 0y1 =

1p2x4 ¡

y2 =1p2x6 ¡

Projection along y1 and y2 Can we exploit these correlations in deriving strong cutting planes?

Polarity Cuts

1. Projection

2. Determine Extreme Points

3. Lifting

4. Convexification

s1 · y21

s1 ¡ (L + U)y1 ¡ LU · 0

Polarity Cuts

1. Projection

3. Lifting

4. Convexification

Projection

min y1 max y1

Polarity Cuts

1. Projection

3. Lifting

4. Convexification

Polarity Cuts

1. Projection

3. Lifting

4. Convexification

(L ;L 2)

(U;U2)

Polarity Cuts

1. Projection

3. Lifting

4. Convexification

(L ;L 2)

min ®kyk + ¯ksk ¡ °s.t.®kL k + ¯k(L k)

2 ¡ ° ¸ 0®kUk + ¯k(Uk)

2 ¡ ° ¸ 0®k ¡ ®+k + ®¡k = 0®+k + ®¡k ¡ ¯k = 1®+k ¸ 0; ®¡k ¸ 0; ¯k · 0

(U;U2)

Polar Program

Convex Relaxation

Extreme pointsof the projected set

k2S (®k yk +¯k sk) ¡ °s.t.P

³®kytk +¯k (ytk)

2´¡ ° ¸ 0;8t

¯k · 0; 8k 2 S®k ¡ ®+k +®¡k =0; 8k 2 SP

¡®+k +®¡k ¡ ¯k

®+k ¸ 0; ®¡k ¸ 0

Polarity

• Additional problem constraints induce geometric correlations along directions of maximal non-convexity

• Projection mechanism identifies such correlations

• Polarity uses these correlations to derive strong cutting planes for MIQCP

Eigen Reformulation

• OPT = -2.5

x4x5+ x6x7 ¡ x3 ¡ z · 0y1 =

1p2x4 ¡

y2 =1p2x6 ¡

Eigen Reformulation

• OPT = -2.5

x4x5+ x6x7 ¡ x3 ¡ z · 0y1 =

1p2x4 ¡

y2 =1p2x6 ¡

Polarity cuts close 99.62% of the duality gap!!

Relaxations of MIQCP

RLT + SDPDisjunctive

CutsLow Width

Disjunctions

Projection RLT

Projection RLT + SDP

EigenReformulation

Polarity

Computational Results

Solvers

• Convex Relaxations – IpOpt • Eigenvalue Computations – LAPACK • Linear Programs & Mixed Integer Programs– CPLEX 11• COIN-OR / Bonmin based implementation

Test Bed

• 160 GlobalLIB Instances• 4 Chemical Process Design instances from Lee & Grossman• 90 Box QP Instances

Computational Results

Experiment Setup

• 1 Hour Time limit on each instance

Duality Gap=opt(F inal Relaxation) ¡ RLT

Opt(M I QCP ) ¡ RLT£ 100

GlobalLIB Instances

160 = 129 + 24 + 7

• All MIQCP Instances with upto 50 variables

• x1 x2 x3 x4 x5

• (x1+x2)/x3 ¸ 2x1

• x0.75

Numerical Problems

Zero Duality Gap

Non-Zero Duality Gap

Computational Results (Extended Formulations)

V1 SDP

V2 Disjunctive CutsV1

Y:ccT · (cTx)2

V3 UGMIPV2

GlobalLIB Instances

Summary of % Duality Gap Closed (129 Instances with non-zero Duality Gap)

V1 V2 V3>99.99 % 16 23 2398-99.99 % 1 44 5275-98 % 10 23 2125-75 % 11 22 200-25 % 91 17 13

Average Gap Closed 24.80% 76.49% 80.86%

GlobalLIB Instances

Summary of % Duality Gap Closed (129 Instances with non-zero Duality Gap)

V1 V2 V3>99.99 % 16 23 2398-99.99 % 1 44 5275-98 % 10 23 2125-75 % 11 22 200-25 % 91 17 13

Average Gap Closed 24.80% 76.49% 80.86%

Version (2,3) vs Version 1

% Duality Gap ClosedInstance V1 V2 V3

st_qpc-m3a 0.00% 98.10% 99.16%st_ph13 0.00% 99.38% 98.80%st_ph11 0.00% 99.46% 98.19%ex3_1_4 0.00% 86.31% 99.57%st_jcbpaf2 0.00% 99.47% 99.61%st_ph12 0.00% 99.49% 99.62%ex2_1_9 0.00% 98.79% 99.73%prob05 0.00% 99.78% 99.49%st_glmp_kky 0.00% 99.80% 99.71%st_e24 0.00% 99.81% 99.81%st_ph15 0.00% 99.83% 99.81%st_bsj4 0.00% 99.86% 99.80%st_ph14 0.00% 99.85% 99.86%st_e08 0.00% 99.81% 99.89%st_ht 0.00% 99.81% 99.89%st_pan2 0.00% 68.54% 99.91%ex2_1_1 0.00% 72.62% 99.92%st_fp1 0.00% 72.62% 99.92%st_pan1 0.00% 99.72% 99.92%ex5_2_4 0.00% 79.31% 99.92%st_e02 0.00% 99.88% 99.95%st_kr 0.00% 99.93% 99.95%

st_e33 0.00% 99.94% 99.95%st_z 0.00% 99.96% 99.95%st_qpc-m0 0.00% 99.96% 99.96%st_phex 0.00% 99.96% 99.96%st_e26 0.00% 99.96% 99.96%st_m1 0.00% 99.96% 99.96%ex2_1_6 0.00% 99.95% 99.97%st_fp6 0.00% 99.92% 99.97%st_e07 0.00% 99.97% 99.97%st_glmp_kk92 0.00% 99.98% 99.98%st_ph3 0.00% 99.98% 99.98%st_ph20 0.00% 99.98% 99.98%st_qpk1 0.00% 99.98% 99.98%st_bsj2 0.00% 99.98% 99.96%st_ph2 0.00% 99.98% 99.98%st_ph1 0.00% 99.98% 99.98%ex2_1_5 0.00% 99.98% 99.99%st_fp5 0.00% 99.98% 99.99%ex3_1_3 0.00% 99.99% 99.99%st_bpv2 0.00% 99.99% 99.99%st_qpc-m1 0.00% 99.99% 99.98%st_qpc-m3b 0.00% 100.00% 100.00%

Version (2,3) vs Version 1

st_qpc-m3a 0.00% 98.10% 99.16%st_ph13 0.00% 99.38% 98.80%st_ph11 0.00% 99.46% 98.19%ex3_1_4 0.00% 86.31% 99.57%st_jcbpaf2 0.00% 99.47% 99.61%st_ph12 0.00% 99.49% 99.62%ex2_1_9 0.00% 98.79% 99.73%prob05 0.00% 99.78% 99.49%st_glmp_kky 0.00% 99.80% 99.71%st_e24 0.00% 99.81% 99.81%st_ph15 0.00% 99.83% 99.81%st_bsj4 0.00% 99.86% 99.80%st_ph14 0.00% 99.85% 99.86%st_e08 0.00% 99.81% 99.89%st_ht 0.00% 99.81% 99.89%st_pan2 0.00% 68.54% 99.91%ex2_1_1 0.00% 72.62% 99.92%st_fp1 0.00% 72.62% 99.92%st_pan1 0.00% 99.72% 99.92%ex5_2_4 0.00% 79.31% 99.92%st_e02 0.00% 99.88% 99.95%st_kr 0.00% 99.93% 99.95%

st_e33 0.00% 99.94% 99.95%st_z 0.00% 99.96% 99.95%st_qpc-m0 0.00% 99.96% 99.96%st_phex 0.00% 99.96% 99.96%st_e26 0.00% 99.96% 99.96%st_m1 0.00% 99.96% 99.96%ex2_1_6 0.00% 99.95% 99.97%st_fp6 0.00% 99.92% 99.97%st_e07 0.00% 99.97% 99.97%st_glmp_kk92 0.00% 99.98% 99.98%st_ph3 0.00% 99.98% 99.98%st_ph20 0.00% 99.98% 99.98%st_qpk1 0.00% 99.98% 99.98%st_bsj2 0.00% 99.98% 99.96%st_ph2 0.00% 99.98% 99.98%st_ph1 0.00% 99.98% 99.98%ex2_1_5 0.00% 99.98% 99.99%st_fp5 0.00% 99.98% 99.99%ex3_1_3 0.00% 99.99% 99.99%st_bpv2 0.00% 99.99% 99.99%st_qpc-m1 0.00% 99.99% 99.98%st_qpc-m3b 0.00% 100.00% 100.00%

Observation

Either version 2 or version 3 closes >99% of the duality gap on 44 instances on which version 1 is unable to close any gap.

The relaxation obtained by adding disjunctive cuts can be substantially stronger than the SDP relaxation!!

Linear Complementarity Disjunctions

• Some problems have linear complementarity constraints

xi xj = 0

• These constraints can be used to derive the linear

complementarity disjunctions

(xi=0) OR (xj=0)

which can be used with the medley of other disjunctions to derive

disjunctive cuts

Linear Complementarity Disjunctions

Without Using LCD Using LCDInstance V2 V3 V2 V3

ex9_1_4 0.00% 1.55% 100.00% 99.97%ex9_2_1 60.04% 92.02% 99.95% 99.95%ex9_2_2 88.29% 98.06% 100.00% 100.00%ex9_2_3 0.00% 47.17% 99.99% 99.99%ex9_2_4 99.87% 99.89% 99.99% 100.00%ex9_2_6 87.93% 62.00% 80.22% 92.09%ex9_2_7 51.47% 86.25% 99.97% 99.95%

ObservationLinear Complementarity conditions can be exploited effectively within a disjunctive programming framework to derive strong cuts

All eigenvalues of Y-xxT are equal to zero.

Eigenvectors of Y-xxT associated with non-zero eigenvalues can be used as sources of cuts

What is the effect of these disjunctive cuts on the spectrum of

Y-xxT ?

Spectrum of Y-xxT

% Duality Gap closed by

V1 V2 Instance Chosen < 10 % > 90 % st_jcbpaf2 > 40% < 60% ex9_2_7 < 10% < 10% ex7_3_1

Version 1, 0% Gap Closed

0 5 10 15 20 25 30 35-0.60

Sum of Positive Eigen Values of Y-xxT

Sum of NegativeEigen Values of Y-xxT

Version 2, 99.47% Gap Closed

0 50 100 150 200 250 300-0.60

Version 3, 99.61% Gap Closed

0 20 40 60 80 100 120-0.60

Computational Results (Projected Formulations)

W1 ProjLP

Disjunctive Cuts

W1 PolarityCuts

All experiments were done using the eigen reformulation

Computational Results: GlobalLib (Projected)

Disjunctive Cuts Extended

ProjLP ProjLP + PolarLP ProjLP ProjLP +

PolarLP SDP + RLT SDP + RLT + Dsj

>99.99 % gap closed 19 23 19 23 16 2398-99.99 % gap closed 22 31 5 21 1 4475-98 % gap closed 35 33 17 18 10 2325-75 % gap closed 34 23 26 32 11 220-25 % gap closed 14 14 57 30 91 170-(-0.22) % gap closed 4 4 4 4 0 0Average Gap Closed 70.65% 76.06% 40.92% 60.48% 24.80% 76.49%Average Time taken (sec) 4.616 19.462 0.893 0.814 198.043 978.140

ObservationWe can generate relaxations in the space of x

variables which are almost as strong as those in the extended space even though our computing

times are 100 times smaller on average.

ObservationPolarity cuts can capture a portion of

strengthening derived from disjunctive cuts

Global Information at work!!

Computational Results: Box QP (Projected)

W3 ProjLPProjected GradientHeuristic

W3-SDP ProjLP

All experiments were done using the eigen reformulation

% Duality Gap Closed Time Taken (sec)Instances W3 W3-SDP W3 W3-SDP W3 (Adjusted)spar20* 94.60 - 99.97 91.54 - 99.91 2.49 - 408.36 0.84 - 2.46 0.51 - 1.60spar30* 89.87 - 99.99 51.41 - 98.79 12.33 - 565.88 1.74 - 14.38 3.32 - 14.49spar40* 87.85 - 99.60 21.78 - 89.63 35.77 - 134.8 4.16 - 65.28 13.75 - 49.76spar50* 87.88 - 97.53 11.38 - 50.15 50.22 - 180.96 8.76 - 99.13 28.95 - 76.19spar60* 85.78 - 90.99 0.00 - 0.00 121.83 - 226.11 111.07 - 127.47 86.28 - 141.77spar70* 89.78 - 99.36 0.00 - 53.67 191.12 - 693.28 22.02 - 202.98 92.63 - 143.35spar80* 88.13 - 97.49 2.94 - 56.23 257.62 - 892.96 34.77 - 67.66 121.62 - 230.53spar90* 89.44 - 96.60 5.73 - 50.13 408.73 - 991.04 46.98 - 95.66 184.63 - 294.92spar100* 92.15 - 96.46 8.17 - 51.79 538.03 - 1509.96 75.49 - 112.69 279.41 - 385.64Average 95.19% 50.01% 280.50 37.89 101.57

ObservationConvex quadratic cuts generated using the projected gradient heuristic can capture a

substantial portion of strengthening derived from the SDP+RLT relaxations

ObservationConvex quadratic cuts generated using the projected gradient heuristic can capture a

substantial portion of strengthening derived from the SDP+RLT relaxationsJust using the eigen

reformulation with ProjLP closes 50% of the duality gap !!

spar100-075-1 Instance

• 95.84% gap closed in 1509 sec• 94.84% gap closed in 366 sec

Assessing the Tailing off Behaviour

% Time spent onCut Generation

Instances W3 W3-SDPspar20* 26.28 - 95.77 0.12 - 0.24spar30* 17.78 - 91.48 0.00 - 0.21spar40* 27.5 - 78.37 0.01 - 0.13spar50* 51.02 - 79.73 0.01 - 0.11spar60* 46.29 - 56.61 0.10 - 0.12spar70* 71.13 - 87.7 0.01 - 0.11spar80* 76.37 - 84.44 0.01 - 0.02spar90* 73.44 - 88.25 0.01 - 0.02spar100* 77.49 - 92.3 0.01 - 0.23Average 66.05% 0.04%

Time Spent on Cut Generation

• Increases with version W3 reaching 75% for larger instances

• Remains less than 0.25% for all instances with W3-SDP

• ProjLP can be solved very efficiently

Computational Results: Box QP (Projected)Time to solve

% Duality Gap Closed Time Taken (sec) last relaxationInstances SDPLR SDPA Proj SDPLR SDPA Proj Projspar20* 99.67 - 100 99.67 - 99.99 94.6 - 99.97 0.97 - 56.37 1.98 - 3.39 2.48 - 408.35 0.05 - 0.32spar30* 97.81 - 100 97.81 - 99.99 89.87 - 99.99 3.57 - 243.3 16.66 - 29.33 12.33 - 565.88 0.06 - 0.89spar40* 96.6 - 100 96.6 - 99.99 87.85 - 99.6 10.3 - 515.73 105.68 - 157.83 35.77 - 134.8 0.16 - 1.18spar50* 95.55 - 100 95.55 - 99.99 87.88 - 97.53 41.72 - 926.15 438.77 - 589.17 50.21 - 180.95 0.13 - 0.86spar60* 98.69 - 100 98.69 - 99.99 85.78 - 90.99 88.05 - 532.45 1150.06 - 1408.32 121.83 - 226.1 0.53 - 1.55spar70* 98.46 - 100 98.46 - 99.99 89.78 - 99.36 133.07 - 3600.75 2769.98 - 3721.34 191.11 - 693.27 0.48 - 1.1spar80* 97.85 - 100 97.84 - 99.99 88.13 - 97.49 965.18 - 5413.02 6618.79 - 8285.12 257.61 - 892.95 0.56 - 2.02spar90* 97.83 - 99.99 97.83 - 99.99 89.44 - 96.6 2403.62 - 7049.49 12838.46 - 17048.98 408.73 - 991.04 0.77 - 1.51spar100* 98.17 - 99.38 98.17 - 99.38 92.15 - 96.46 5355.2 - 10295.88 23509.13 - 28604.12 538.02 - 1509.96 0.82 - 2Average 99.40% 99.40% 95.19% 1741.20 5247.04 280.50 0.67

No. ConstraintsNo. Variables Linear Convex (Non-Linear) Computing Time (sec) % Duality Gap Closed

Instances SDP Proj SDP Proj SDP Proj SDP Proj SDP Projspar100-025-1 5151 203 20201 156 1 119 5719.42 1.14 98.93% 92.36%spar100-025-2 5151 201 20201 151 1 95 10185.65 1.52 99.09% 92.16%spar100-025-3 5151 201 20201 150 1 114 5407.09 1.24 99.33% 93.26%spar100-050-1 5151 201 20201 150 1 98 10139.57 1.07 98.17% 93.62%spar100-050-2 5151 201 20201 150 1 113 5355.20 1.26 98.57% 94.13%spar100-050-3 5151 201 20201 150 1 97 7281.26 0.82 99.39% 95.81%spar100-075-1 5151 201 20201 150 1 131 9660.79 2.00 99.19% 95.84%spar100-075-2 5151 201 20201 150 1 109 6576.10 1.23 99.18% 96.47%spar100-075-3 5151 199 20201 147 1 90 10295.88 0.87 99.19% 96.06%

No. ConstraintsNo. Variables Linear Convex (Non-Linear) Computing Time (sec) % Duality Gap Closed

Instances SDP Proj SDP Proj SDP Proj SDP Proj SDP Projspar100-025-1 5151 203 20201 156 1 119 5719.42 1.14 98.93% 92.36%spar100-025-2 5151 201 20201 151 1 95 10185.65 1.52 99.09% 92.16%spar100-025-3 5151 201 20201 150 1 114 5407.09 1.24 99.33% 93.26%spar100-050-1 5151 201 20201 150 1 98 10139.57 1.07 98.17% 93.62%spar100-050-2 5151 201 20201 150 1 113 5355.20 1.26 98.57% 94.13%spar100-050-3 5151 201 20201 150 1 97 7281.26 0.82 99.39% 95.81%spar100-075-1 5151 201 20201 150 1 131 9660.79 2.00 99.19% 95.84%spar100-075-2 5151 201 20201 150 1 109 6576.10 1.23 99.18% 96.47%spar100-075-3 5151 199 20201 147 1 90 10295.88 0.87 99.19% 96.06%

Very little computational overheads at the nodes of the enumeration tree

ObservationStrengthened relaxations produced by our code are almost as strong as the SDP+ RLT relaxations and can be solved in less than 2 sec; state of art SDP

solvers can take upto a couple of hours to solve these relaxations in the extended space

Research Question?

1978-1988 •Data Structures•Theoretical Computer Science

1988-1998 •Linear Programming

1998-2008 •Mixed Integer Linear Programming

2008-2018 • ?Anureet Saxena, Axioma Inc 100

Research Question?

1978-1988 •Data Structures•Theoretical Computer Science

1988-1998 •Linear Programming

1998-2008 •Mixed Integer Linear Programming

2008-2018• Mixed Integer Non-Linear

Programming

Go Global for Global Optimization

Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Problems

Documents

TIGHT CONVEX RELAXATIONSpublications.lightfield-analysis.net/GSC13_siims.pdf · tight convex relaxations for vector-valued labeling bastian goldluecke1, evgeny strekalovskiy 2and

Convex Relaxations of Chance Constrained AC Optimal Power Flow

Inference in Graphical Models via Semideﬁnite Programming …yash/PAPERS/psos.pdf · 2020-05-13 · graphical models using tighter convex relaxations. Within combinatorial optimization,

DQM: Decentralized Quadratically Approximated …aribeiro/preprints/2015_mokhtari_etal_c.pdfDQM: Decentralized Quadratically Approximated Alternating Direction Method of Multipliers

Enablingcertiﬁcationofveriﬁcation-agnosticnetworks via ......Convex relaxation based methods. Much work has relied on linear programming (LP) or similar relaxations for neural-network

MOMENT RELAXATIONS OF OPTIMAL POWER FLOW PROBLEMS · 2020-04-12 · MOMENT RELAXATIONS OF OPTIMAL POWER FLOW PROBLEMS: BEYOND THE CONVEX HULL Daniel K. Molzahn Argonne National Laboratory

A Smoothing Newton-Type Algorithm of Stronger …the Quadratically Constrained Convex Quadratic Programming Zheng-Hai Huang∗, Defeng Sun †, and Gongyun Zhao ‡ June 10, 2004;

Convex Relaxations for Optimization of Power Grids under ... · 9 DTU Electrical Engineering Convex Relaxations for Optimization of Power Grids under Uncertainty Feb 8, 2017. Convexifying

New Convex Relaxations for MRF Inference with Unknown ...oval/publications/WLSKZ...fzhhwang, zjhg@zjut.edu.cn, javen.shi@adelaide.edu.au, pawan@robots.ox.ac.uk Abstract Treating graph

Graphical models and variational methods: …wainwrig/icml08/...Graphical models and variational methods: Message-passing, convex relaxations, and all that Martin Wainwright Department

New Convex Relaxations for MRF Inference with Unknown Graphsoval/publications/WLSKZ-ICCV2019.pdf · 2019-11-07 · New Convex Relaxations for MRF Inference with Unknown Graphs Zhenhua

Convex relaxations for structured sparsity › ... › 661 › bachslides.pdf · • Structured sparsity through submodular functions – Relaxation of the penalization of supports

Convex relaxations for minimum energy orbit transfer problemmopta/mopta04pres/preda_mopta04.pdf · Convex relaxations for minimum energy orbit transfer problem MOPTA04 Hamilton, Canada

Semideﬁnite programming relaxations for semialgebraic problemscba.mit.edu/events/03.11.ASE/docs/Parrilo.pdf · Semideﬁnite programming – convex optimization – sums of squares

ICCV2009: MAP Inference in Discrete Models: Part 6: Recent Advances in Convex Relaxations

CONVEX RELAXATIONS FOR PERMUTATION PROBLEMS

DS: Tighter Lifting-Free Convex Relaxations for Quadratic ... · DS: Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems Florian Bernard 1,2 Christian Theobalt

Convex optimization: applications, formulations, relaxations · Convex optimization: applications, formulations, relaxations J´erome MALICK CNRS, LJK/INRIA, BiPoP team NECS team

Convex Relaxations for Consensus and Non-Minimal …openaccess.thecvf.com/content_ICCV_2019/papers/Probst...Convex Relaxations for Consensus and Non-Minimal Problems in 3D Vision Thomas

Information Relaxations, Duality, and Convex Stochastic ...dbbrown/bio/papers/brown_smith... · Brown and Smith: Information Relaxations, Duality, and Convex Stochastic Dynamic Programs