Recent improvement to MISOCP in CPLEXoptimizationdirect.com/data/informs2015_bonami.pdf · Optimization Direct workshop - INFORMS 2015 - October 31 2015 Recent improvement to MISOCP

IBM Analytics

Pierre BonamiIBM ILOG CPLEXOptimization Direct workshop - INFORMS 2015 - October 312015

Recent improvement to MISOCP in CPLEX

1 ©2015 IBM Corporation

IBM Analytics

CPLEX Optimization Studio news (12.6.2/12.6.3)

CPLEX Optimization Studio 12.6.2 (June2015)

DoCloud services: OPL on the cloud (→ IBMWorkshop and TA20)Performance improvements

CPLEX Optimization Studio 12.6.3 (upcoming)

Distributed MIP enhancementsMILP performance improvements

DOcplex (→ IBM Workshop):A Python modeling layer for CPLEX and CPOPrepared to connect to local or DOcloudFreeNotebook-ready

CPLEX Optimization Studio CommunityEdition


IBM Analytics

CPLEX keeps getting better

→ A. Tramontani TA19, → R. Wunderling WD38


IBM Analytics

Convex MIQCP and MISOCP

min cT x

xTQkx + aTk x ≤ a0

k k = 1, . . . ,m,Ax = b,xj ∈ Z j = 1, . . . , p.

(MIQCP)

Where all quadratic constraints can be represented as second order cones:

Ld := {(x , x0) ∈ Rd+1 :

d∑

i=1

x2

i ≤ x2

0, x0 ≥ 0}.

(Ld defines the (d + 1)-dimensional second order cone.)


IBM Analytics

A Lorentz cone

It is convex


IBM Analytics

Second order cone representability

CPLEX recognizes or automatically reformulates to SOC via basictransformations

Second order cones:

d∑

i=1

x2

i ≤ x2

0, with x0 ≥ 0

Rotated second order cones:

d∑

i=2

x2

i ≤ x0x1, with x0, x1 ≥ 0

Convex quadratic constraints:

xTQx + aT x ≤ a0, with Q � 0


IBM Analytics

More second order cone representability

More convex nonlinear set can be represented

Euclidian norms:||x ||2 ≤ y

||Ax + b||2 ≤ cT x + d

Rational functions

y ≥1

x

y ≥1

x2

Those will not be reformulated automatically by CPLEX.


IBM Analytics

MISOCP

min cT x

(xJi , xhi ) ∈ Ldi i = 1, . . . ,mAx = b,xj ∈ Z j = 1, . . . , p.

(MISOCP)

Algorithms are based on SOCP relaxation

SOCP relaxation solved efficiently by interior point methods

Supported by CPLEX since version 9.0


IBM Analytics

MISOCP Applications

Application SOC Integer

Portfolio optimization Risk, utility, robustness number of assets, min in-vestment

[Bienstock, 1996, Bonami and Lejeune, 2009, Vielma et al., 2008]Truss topology optimiza-tion

Physical forces Cross section of bars

[Achtziger and Stolpe, 2006]Networks with delays Delay as function of traf-

ficPath, flows

[Boorstyn and Frank, 1977, Ameur and Ouorou, 2006]Location with stochasticservices

Demands location model

[Elhedhli, 2006]TSP with neighborhoods(Robotics)

Definition of ngbh. TSP

[Gentilini et al., 2013]Many more... see for eg. http://cblib.zib.de.


IBM Analytics

Agenda

Two basic algorithms in CPLEXSOCP-B&B (since CPLEX 9.0)OA-B&B (since CPLEX 11.0)

Novelties of CPLEXCone disaggregationLift-and-project cuts for MISOCP

Comparison with CPLEX 12.6.1


IBM Analytics

SOCP based branch-and-bound

Straightforward generalization of main MILPalgorithm:

Solve SOCP relaxation at each node of the tree


IBM Analytics




Branch on variables with fractional value

integerfeasible

fathomedbybound

infeasible


IBM Analytics




Branch on variables with fractional value

Prune by infeasibility, bounds and integerfeasibility

integerfeasible

fathomedbybound

infeasible


IBM Analytics

Outer Approximation [Duran and Grossmann, 1986]

min cT x

s.t.

gi (x) ≤ 0 i = 1, . . . ,m,

Ax = b

xj ∈ Z, j = 1, . . . , p.

Idea: Take first-order approximations of constraints at different points xk andbuild an approximating MILP


IBM Analytics

Outer Approximation [Duran and Grossmann, 1986]

min cT x

s.t.

gi (x) ≤ 0 i = 1, . . . ,m,

Ax = b

xj ∈ Z, j = 1, . . . , p.

Idea: Take first-order approximations of constraints at different points xk andbuild an approximating MILP

min cT x

s.t.

gi (xk) +∇gi (x

k)T (x − xk) ≤ 0 i = 1, . . . ,m, k = 1, . . . ,K

xj ∈ Z, j = 1, . . . , p.12 ©2015 IBM Corporation

IBM Analytics

OA Branch-and-cut [Quesada and Grossmann, 1992]

Initialize by solving SOCP relaxation and takingOA’s at the optimum


IBM Analytics



Carry out B&B on OA


IBM Analytics



Carry out B&B on OA

At each integer feasible node:1 Solve SOCP with integers fixed, and enrich the

set of linearizations2 Resolve the LP relaxation of the node with the

new cuts3 Repeat as long as node is integer feasible

integerfeasible


IBM Analytics



Carry out B&B on OA

At each integer feasible node:1 Solve SOCP with integers fixed, and enrich the

set of linearizations2 Resolve the LP relaxation of the node with the

new cuts3 Repeat as long as node is integer feasible

Never prune by integer feasibility


IBM Analytics

The MISOCP solver in CPLEX ≥ 11.0

Both main algorithms available

Selection controled via parameterCPXPARAM_MIP_Strategy_MIQCPStrat

miqcpstrat 1 - SOCP based Branch-and-boundmiqcpstrat 2 - OA based branch-and-cutmiqcpstrat 0 - heuristic decision which of the two to use


IBM Analytics

How bad can outer approximation be?

The following convex MIQCP

min cT x

s.t.∑n

i=1

(

xi −1

2

)2≤ n−1

4

x ∈ Zn

(1)

is infeasible:

The ball is too small to containinteger points

It is large enough to touch everyedge of the hypercube

x y

z


IBM Analytics

Solving (1) with OA cuts

No OA constraint can cut 2vertices of the hypercube

If an inequality cuts twovertices, it cuts the segmentjoining themAny such inequality would cutinto the interior of (1)

x y

z


IBM Analytics




An OA needs at least 2n OA cutsto converge

x y

z


IBM Analytics




An OA needs at least 2n OA cutsto converge

Remark

A basic SOCP branch-and-boundalso enumerates at least 2n

integer solutions

x y

z


IBM Analytics

Experimental illustration

CPLEX 12.4 1 SCIP 2.0.1 Bonmin B-Hybn 2n nodes nodes nodes10 1,024 2,047 720 11,15615 32,768 65,535 31,993 947,01420 1,048,576 2,097,151 1,216,354 . . .

1CPLEX ran in single threaded mode17 ©2015 IBM Corporation

IBM Analytics

Experimental illustration

CPLEX 12.4 1 SCIP 2.0.1 Bonmin B-Hybn 2n nodes nodes nodes10 1,024 2,047 720 11,15615 32,768 65,535 31,993 947,01420 1,048,576 2,097,151 1,216,354 . . .

Remark

Problem is simple for CPLEX/SCIP if variables are 0 − 1: replace x2

i byxi , the contradiction n

4≤ n−1

4follows.

SCIP ≥ 2.1 and CPLEX ≥ 12.6.1 solve it in a blink

1CPLEX ran in single threaded mode17 ©2015 IBM Corporation

IBM Analytics

New in CPLEX 12.6.2/12.6.3

Cone disaggregation for MISOCP

Lift-and-project cuts for MISOCP

Redesigned heuristic choice of most promising algorithm


IBM Analytics

Separable mixed integer convex programs

min cT x

s.t. gi (x) ≤ 0 i = 1, . . . ,mAx = b

xj ∈ Z j = 1, . . . , pl ≤ x ≤ u

(sMINLP)

For i = 1, . . . ,m, gi are convex separable:

gi (x) =

n∑

j=1

gij(xj)

with gij : [lj , uj ] → R convex.


IBM Analytics

Disaggregated formulation

Introduce one variable yij for each elementary function:

min cT x

s.t.n∑

j=1

yij ≤ 0 i = 1, . . . ,m,

gij(xj) ≤ yiji = 1, . . . ,m,j = 1, . . . , n,

Ax = b,xi ∈ Z i = 1, . . . , p,l ≤ x ≤ u.

(sMINLP∗)


IBM Analytics

Cone disaggregation for MISOCP

In standard form the nonlinear constraint describing the second order cone isnot convex separable:

n∑

i=1

x2

i ≤ x2

0

[Vielma et al., 2015] divide the constraint by x0 ≥ 0 to get a convex separableconstraint:

n∑

i=1

x2

i

x0

≤ x0

Introduce y1, . . . , yn and rewrite as:

n∑

i=1

yi ≤ x0

x2

i ≤ x0yi


IBM Analytics

Application to (1) [Hijazi et al., 14]

Extended formulation of (1)

min cT x

s.t.n

∑

i=1

yi ≤ (n − 1)/4

(xi − 0.5)2 ≤ yi i = 1, . . . , n

x ∈ Zn.

Its outer approximation on K points x̄k

min cT x

s.t.n

∑

i=1

yi ≤ (n − 1)/4

2(

xki − 0.5)

(xi − xki ) +(

xki − 0.5)2

≤ yii = 1, . . . , nk = 1, . . . ,K

x ∈ Zn


IBM Analytics

Application to (1) [Hijazi et al., 14]

Extended formulation of (1)

min cT x

s.t.n

∑

i=1

yi ≤ (n − 1)/4

(xi − 0.5)2 ≤ yi i = 1, . . . , n

x ∈ Zn.

Its outer approximation on K points x̄k

min cT x

s.t.n

∑

i=1

yi ≤ (n − 1)/4

2(

xki − 0.5)

(xi − xki ) +(

xki − 0.5)2

≤ yii = 1, . . . , nk = 1, . . . ,K

x ∈ Zn


IBM Analytics

How good can this be?

[Cornuéjols and Li, 2001] showed thatthe empty ball in dimension n has splitrank n (also holds for any outerapproximation in this space)⇒ Practically unsolvable using anyform of split cuts (even conic ones).

Instead the OA of the disaggregatedformulation has (simple) split rank 1.


IBM Analytics

How good can this be?

[Cornuéjols and Li, 2001] showed thatthe empty ball in dimension n has splitrank n (also holds for any outerapproximation in this space)⇒ Practically unsolvable using anyform of split cuts (even conic ones).

Instead the OA of the disaggregatedformulation has (simple) split rank 1.

xi ≤ 0 xi ≥ 1

yi ≥14

2 points suffice to make the mixed-integer set infeasiblex1 = 0 and x2 = 1:

− xi + 0.25 ≤ yi i = 1, . . . , n

xi − 0.75 ≤ yi i = 1, . . . , n

xi ∈ {0, 1}

⇒ yi ≥ 0.25


IBM Analytics

Lift-and-project cuts for MISOCP

Cuts are an essential component of MILP solvers

Can always apply MILP cuts to a linear OA of MISOCP (and we do it)

Can we generate better cuts by looking directly at nonlinear functions?partial answer: as long as the cut generated is linear it could also havebeen obtained from a linear outer approximation

In the past three years, tremendous activity towards conic cuts for conicprogramming [Andersen and Jensen, 2013, Belotti et al., 2013,Kılınç-Karzan and Yıldız, 2015, Modaresi et al., 2015] (among others)

Our goals

linear cutting planes

find suitable OA from which to derive a cut

fast


IBM Analytics

A more complicated empty ellipse

n∑

i=1

(100x2

i − 98xi )−

n2

∑

i=1

4x2ix2i−1 ≤ −1, x ∈ Zn

rotated ellipse that not longer fully disaggregates

results on 12 threads with 12.6.1, 12.6.2, 12.6.2-- (no lift-and-projectcuts) and 12.6.2++ (aggressive lift-and-project cuts), 3 hours time limit

12.6.1 12.6.2-- 12.6.2 12.6.2++n nodes nodes nodes nodes5 2,261 2,045 2,045 1,82510 2,097,151 1,914,797 29 115 >23,125,426 >146,604,478 7,769 1

(Largest model solved in 2.2 sec by 12.6.2, in 5.5 sec by 12.6.2++.)


IBM Analytics

Remarks

Cone disaggregation

automatically applied by default during presolve

only interesting (and done):1 if using the OA-B&B2 for cones that are long enough

Lift-and-project

only done in OA based algorithm

expensive algorithm (may not speed up every “easy” model)


IBM Analytics

Heuristic algorithm selection

Redesigned heuristic to choose algorithm to apply in view of these changes

CPLEX 12.6.2: 245 models solved by at least one method and failed by none

0

20

40

60

80

100

1 10 100 1000 10000 100000

% m

odel

sol

ved

time factor

1262_miqcpstrat_01262_miqcpstrat_11262_miqcpstrat_2

Default strategy picked

OA-B&B 186 times

SOCP-B&B 4 times

55 models identical withboth

"perfect" Heuristic:

2 more models withOA-B&B

9 more models withSOCP-B&B


IBM Analytics

Heuristic algorithm selection

Redesigned heuristic to choose algorithm to apply in view of these changes

CPLEX 12.6.1: 225 models solved by at least one method and failed by none

0

20

40

60

80

100

1 10 100 1000 10000 100000

% m

odel

sol

ved

time factor

1261_miqcpstrat_01261_miqcpstrat_11261_miqcpstrat_2

Default strategy picked

OA-B&B 113 times

SOCP-B&B 46 times

56 models identical withboth

"perfect" Heuristic:

14 more models withOA-B&B

36 more models withSOCP-B&B


IBM Analytics

Troubleshooting

Choosing the right algorithm

Our testing shows OA is better in most cases, but can be dramatically worst:

when SOCP relaxation solved very fast,

when SOCP relaxation very strong,

when linear relaxations are weak due to structure (around the head ofcones).

Simplifying models

Ideas presented by Ed carry over:

Can linearize products by binaries (not done automatically).

Assiociated MILP can provide good solutions.


IBM Analytics

Computational experiments MISOCP 12.6.3

Test bed

CPLEX internal test set: 233 models

CBLIB test set (http://cblib.zib.de): 80 models

Compare

CPLEX 12.6.3 against CPLEX 12.6.1

geometric mean of solve times 2

2All tests are carried on Linux machines: Intel X5650 @ 2.67 GHz, 24 GB RAM, 12

threads, deterministic.29 ©2015 IBM Corporation

IBM Analytics

CPLEX 12.6.1 vs 12.6.3

00.20.40.60.81.0

> 0 sec233 Models

> 1 sec154 Models

×2.78 ×4.76

CPLEX test bed

CPLEX 12.6.1: 29 time limits


00.20.40.60.81.0

> 0 sec66 Models

> 1 sec46 Models

×3.57 ×6.25

CBLIB




IBM Analytics

Legal Disclaimer© IBM Corporation 2015. All Rights Reserved.

The information contained in this publication is provided for informational purposes only. While efforts weremade to verify the completeness and accuracy of the information contained in this publication, it is provided ASIS without warranty of any kind, express or implied. In addition, this information is based on IBM’s currentproduct plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible forany damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothingcontained in this publication is intended to, nor shall have the effect of, creating any warranties orrepresentations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicablelicense agreement governing the use of IBM software.

References in this presentation to IBM products, programs, or services do not imply that they will be available inall countries in which IBM operates. Product release dates and/or capabilities referenced in this presentationmay change at any time at IBM’s sole discretion based on market opportunities or other factors, and are notintended to be a commitment to future product or feature availability in any way. Nothing contained in thesematerials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by youwill result in any specific sales, revenue growth or other results.

Performance is based on measurements and projections using standard IBM benchmarks in a controlledenvironment. The actual throughput or performance that any user will experience will vary depending uponmany factors, including considerations such as the amount of multiprogramming in the user’s job stream, theI/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be giventhat an individual user will achieve results similar to those stated here.

IBM, the IBM logo, CPLEX and SPSS are trademarks of International Business Machines Corporation in theUnited States, other countries, or both.

Intel, Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the UnitedStates and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company,product, or service names may be trademarks or service marks of others.


IBM Analytics

References I

W. Achtziger and M. Stolpe. Truss topology optimization with discrete design variables – Guaranteed global optimalityand benchmark examples. Structural and Multidisciplinary Optimization, 34(1):1–20, 2006.

W. B. Ameur and A. Ouorou. Mathematical models of the delay constrained routing problem. Algorithmic OperationsResearch, 1(2):94–103, 2006.

K. Andersen and A. Jensen. Intersection cuts for mixed integer conic quadratic sets. In M. Goemans and J. Correa,editors, Integer Programming and Combinatorial Optimization, volume 7801 of Lecture Notes in Computer Science,pages 37–48. Springer Berlin Heidelberg, 2013. ISBN 978-3-642-36693-2. doi: 10.1007/978-3-642-36694-9_4.URL http://dx.doi.org/10.1007/978-3-642-36694-9_4.

P. Belotti, J. C. Góez, I. Pólik, T. K. Ralphs, and T. Terlaky. On families of quadratic surfaces having fixed intersectionswith two hyperplanes. Discrete Applied Mathematics, 161(16–17):2778 – 2793, 2013. ISSN 0166-218X. doi:http://dx.doi.org/10.1016/j.dam.2013.05.017. URLhttp://www.sciencedirect.com/science/article/pii/S0166218X13002461.

D. Bienstock. Computational study of a family of mixed-integer quadratic programming problems. MathematicalProgramming, 74:121–140, 1996.

P. Bonami and M. Lejeune. An Exact Solution Approach for Integer Constrained Portfolio Optimization Problems UnderStochastic Constraints. Operations Research, 57:650–670, 2009.

R. Boorstyn and H. Frank. Large-scale network topological optimization. IEEE Transactions on Communications, 25:29–47, 1977.

G. Cornuéjols and Y. Li. On the rank of mixed 0,1 polyhedra. In K. Aardal and B. Gerards, editors, Integer Programmingand Combinatorial Optimization, volume 2081 of Lecture Notes in Computer Science, pages 71–77. Springer BerlinHeidelberg, 2001. ISBN 978-3-540-42225-9. doi: 10.1007/3-540-45535-3_6. URLhttp://dx.doi.org/10.1007/3-540-45535-3_6.

M. A. Duran and I. Grossmann. An outer-approximation algorithm for a class of mixed-integer nonlinear programs.Mathematical Programming, 36:307–339, 1986.

S. Elhedhli. Service System Design with Immobile Servers, Stochastic Demand, and Congestion. Manufacturing & ServiceOperations Management, 8(1):92–97, 2006. doi: 10.1287/msom.1050.0094. URLhttp://msom.journal.informs.org/cgi/content/abstract/8/1/92.


http://dx.doi.org/10.1007/978-3-642-36694-9_4

http://www.sciencedirect.com/science/article/pii/S0166218X13002461

http://dx.doi.org/10.1007/3-540-45535-3_6

http://msom.journal.informs.org/cgi/content/abstract/8/1/92

IBM Analytics

References II

I. Gentilini, F. Margot, and K. Shimada. The travelling salesman problem with neighborhoods: Minlp solution.Optimization Methods and Software, 28(2):364–378, 2013.

H. Hijazi, P. Bonami, and A. Ouorou. An outer-inner approximation for separable mixed-integer nonlinear programs.INFORMS Journal on Computing, 26(1):null, 14. doi: 10.1287/ijoc.1120.0545.

F. Kılınç-Karzan and S. Yıldız. Two term disjunctions on the second-order cone. Mathematical Programming, April 2015.http://link.springer.com/article/10.1007/s10107-015-0903-4.

S. Modaresi, M. Kılınç, and J. Vielma. Intersection cuts for nonlinear integer programming: convexification techniques forstructured sets. Mathematical Programming, pages 1–37, 2015. ISSN 0025-5610. doi: 10.1007/s10107-015-0866-5.URL http://dx.doi.org/10.1007/s10107-015-0866-5.

I. Quesada and I. E. Grossmann. An LP/NLP based branch–and–bound algorithm for convex MINLP optimizationproblems. Computers and Chemical Engineering, 16:937–947, 1992.

J. P. Vielma, S. Ahmed, and G. Nemhauser. A lifted linear programming branch-and-bound algorithm for mixed integerconic quadratic programs. INFORMS Journal on Computing, 20:438–450, 2008.

J. P. Vielma, I. Dunning, J. Huchette, and M. Lubin. Extended Formulations in Mixed Integer Conic QuadraticProgramming. Research Report, 2015.


http://link.springer.com/article/10.1007/s10107-015-0903-4

http://dx.doi.org/10.1007/s10107-015-0866-5

Documents

Recent improvement to MISOCP in CPLEXoptimizationdirect.com/data/informs2015_bonami.pdf · Optimization Direct workshop - INFORMS 2015 - October 31 2015 Recent improvement to MISOCP