CPLEX keeps getting better · LP-based Outer Approximation (OA) B&C, where an LP relaxation of SOCP is updated by separating OA constraints on the fly [Quesada and Grossmann, 1992]

© 2015 IBM Corporation

CPLEX keeps getting better @INFORMS 2015, Philadelphia

Andrea Tramontani CPLEX Optimization – IBM Italy

© 2015 IBM Corporation 2

CPLEX Optimization Studio 12.6.2 (June 2015)

CPLEX Optimization Studio 12.6.3 (upcoming)

DOcplex

A Python modeling layer for CPLEX and CPO

Prepared to connect to local or DOcloud

Free and open source

Integrated with the whole Python software ecosystem

Notebook-ready

Presented at: IBM workshop, SD75, MD30 (ask for the slides)

(Some) news from IBM Optimization


CPLEX MILP performance evolution CPLEX Optimization Studio 12.6.2 (June 2015)

DOcloud services: OPL on the cloud

New parameter CPXPARAM_CPUmask

• CPU mask for binding threads to processors

Performance improvements to CPO

CPLEX optimizer performance

• Exploit symmetry for LP

• Improvements for SOCP

• Improvements to convex MIQCP

• BQP cuts for non-convex (MI)QP

CPLEX Optimization Studio 12.6.3 (upcoming)

Distributed MIP enhancements

• Merge information after racing ramp-up

MILP performance improvements

News from CPLEX Optimization Studio


Deterministic parallel MILP (12 threads)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

>1s >10s >100s

1.0

0

1.0

0

1.0

0

0.9

4

0.9

4

0.9

1

CPLEX 12.6.1

CPLEX 12.6.3

1986

models

1197

models

627

models

Time limits:

41 / 26

Date: 25 October 2015

Testset: MILP: 4245 models

Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threads, deterministic

Timelimit: 10,000 sec

1.06x 1.06x 1.10x

CPLEX 12.6.1 vs. 12.6.3: MILP performance improvement


Node presolve improvements (estimated performance improvement: 2%)

Improved handling of bound strengthening for continuous and general integer

variables

Improved propagation of indicator constraints

• Estimated performance impact on models with indicator constraints: 13%

• But test set is too small

Dynamic search improvement:

Estimated performance impact: 3%

Node cuts improvements:

More aggressive separation

More aggressive filtering

Estimated performance improvement: 5%

MILP performance improvement: summary


0

0.2

0.4

0.6

0.8

1

>1s >10s >100s

1.0

0

1.0

0

1.0

0

0.9

9

0.9

8

0.9

2 CPLEX 12.6.1

CPLEX 12.6.3


Testset: LP: 2271 models



CPLEX 12.6.1 vs.12.6.3: LP performance improvement

678 models 166 models

Time limits:

2 / 3

1.01x 1.09x 1.02x

370 models

Part of the improvement comes from LP symmetry detection:

Roland Wunderling, “Symmetry: What LP Can Learn from MIP”, Wednesday Nov

04, 14:45 - 16:15 (WD 38)

Concurrent LP

(12 threads)


0

0.2

0.4

0.6

0.8

1

>0s >1s

1.0

0

1.0

0

0.6

6

0.4

4

CPLEX 12.6.1

CPLEX 12.6.3


Testset: SOCP: 142 models



CPLEX 12.6.1 vs.12.6.3: QCP/SOCP performance improvement

SOCP barrier

(12 threads)


Time limits:

8 / 0

1.52x 2.26x

Improved dense column handling

Improved handling of short cones

Excluding the 8 time outs for CPLEX 12.6.1:

Speed-up of 1.05x in “>0 secs”

Speed-up of 1.10x in “>1 secs”


Solving convex MIQCP/MISOCP in CPLEX

Every convex MIQCP is converted and solved as an equivalent MISOCP

Two alternative approaches for MISOCP (since CPLEX 11.0)

Non-linear B&B, based on the SOCP relaxation of MISOCP

LP-based Outer Approximation (OA) B&C, where an LP relaxation of SOCP is updated

by separating OA constraints on the fly [Quesada and Grossmann, 1992]

Choice controlled by parameter CPXPARAM_MIP_Strategy_MIQCPStrat

SOCP based B&B (miqcpstrat 1)

LP based B&C (miqcpstrat 2)

By default (miqcpstrat 0) CPLEX selects the algorithm to run in a or “clever” way

min 𝑐𝑇 𝑥

𝑠. 𝑡. 𝐴 𝑥 = 𝑏

(MIQCP) 𝑥𝑇𝑄𝑘𝑥 + 𝑎𝑘

𝑇𝑥 ≤ 𝑏𝑘

𝑥𝑗 ∈ ℤ 𝑗 = 1, … , 𝑝

𝑘 = 1, … , 𝑚


MISOCP performance improvements (CPLEX 12.6.2, June 2015)

Major improvements for Outer Approximation B&C

Cone disaggregation

• Original idea in [Vielma et al., 2015]

Cone strengthening by perspective reformulation

• Original idea in [Gϋnlϋk and Linderoth, 2011]

Lift-and-Project (L&P) cutting planes (available for MILP since CPLEX 12.5.1)

• Linear cuts that exploit non-linear constraints

• LP-based OA scheme from [Kilinç et al., 2011]

• Compact separation LP from [Bonami, 2011]

• Plus our own normalization to truncate the separation LP

Redesigned heuristic to choose algorithm to apply in view of these changes

Much more details were given on Sunday (too late if you missed it …):

Pierre Bonami, “Recent Advances in CPLEX for Mixed Integer Nonlinear

Optimization”, Sunday Nov 01, 16:30 - 18:00 (SD 12)


Deterministic parallel MIQCP (12 threads)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

>0s >1s

1.0

0

1.0

0

0.3

6

0.2

1

CPLEX 12.6.1

CPLEX 12.6.3

233

models

154

models

Time limits:

29 / 4


Testset: MIQCP: 296 models



2.78x 4.76x

CPLEX 12.6.1 vs.12.6.3: Convex MIQCP performance improvement


BQP cuts Solving non-convex MIQP

to global optimality


(MI)QP

(with 𝑄 symmetric)

History of MIQP in CPLEX

min 1

2𝑥𝑇 𝑄 𝑥 + 𝑐𝑇 𝑥

𝑠. 𝑡.

𝑙 ≤ 𝑥 ≤ 𝑢

(MIQP) 𝐴 𝑥 = 𝑏

𝑥𝑗 ∈ ℤ, 𝑗 = 1, … 𝑝

Class 𝒑 𝑸 algorithm V. (Year)

Convex QP

–

0

–

≽ 0

–

Barrier

QP simplex

4.0 (1995)

8.0 (2002)

Convex MIQP > 0 ≽ 0 B&B 8.0 (2002)

Nonconvex QP

–

0

–

𝐴𝑛𝑦

–

barrier (local)

Spatial B&B (global)

12.3 (2011)

12.6.0 (2013)

Nonconvex MIQP > 0 𝐴𝑛𝑦 Spatial B&B (global) 12.6.0 (2013)


Parameter CPX_PARAM_OPTIMALITYTARGET

By default CPLEX assumes you want to solve a convex problem

and returns error CPXERR_Q_NOT_POS_DEF if Q is indefinite

Local solver for non-convex QP:

activated by setting CPX_PARAM_OPTIMALITYTARGET to 2 (or to

CPX_OPTIMALITYTARGET_FIRSTORDER)

Global solver for non-convex (MI)QP:

activated by setting CPX_PARAM_OPTIMALITYTARGET to 3 (or to

CPX_OPTIMALITYTARGET_OPTIMALGLOBAL)


Solving non-convex (MI)QP to global optimality

Spatial B&B (since CPLEX 12.6.0)

Reformulate MIQP by “moving” the non-convex part of 𝑄 to constraints

E.g.: Just add non-convex constraints and rewrite

as a linear objective function

Q-space reformulation

Factorized eigenvector space reformulation

Estabilsh a convex (easy to solve) relaxation of the reformulation

Establish branching rules on solutions of this relaxations

New in CPLEX 12.6.2 (June 2015)

Boolean Quadratic Polytope (BQP) cuts

𝑌 = 𝑥𝑥𝑇 𝑥𝑇𝑄 𝑥 = 𝑄, 𝑌


Box QP

Box-QP is interesting in itself:

Bounds 0 and 1 are w.l.o.g. (every box QP can be scaled to those bounds)

Still NP-hard

Has some academic interest [Vandenbussche and Nemhauser, 2005, Burer and

Vandenbussche, 2009, Chen and Burer, 2012]

Also some applications [Moré and Toraldo, 1989] (usually huge size)

Assuming finite bounds on 𝑥, (box-QP) is a valid relaxation for every non-convex (MI)QP

min 1

2𝑥𝑇 𝑄 𝑥 + 𝑐𝑇 𝑥

𝑠. 𝑡.

0 ≤ 𝑥 ≤ 1

(box-QP)


Proposition [Burer and Letchford, 2009]

Let 𝑌𝑄 be the set where variables 𝑦 represent the products in 𝑄 (excluding

diagonal terms 𝑄𝑖𝑖):

We then have:

Corollary

BQP is the Boolean Quadratic Polytope [Padberg, 1989].

BQP gives a valid relaxation of box-QP.

Every valid cut for BQP is valid for the box-QP.

BQP ∶= conv 𝑥, 𝑌 ∈ 𝑌𝑄 ∶ 𝑥 ∈ *0, 1+𝑛 = conv 𝑥, 𝑌 ∈ 𝑌𝑄 ∶ 𝑥 ∈ 0, 1 𝑛

Box QP and Boolean Quadratic Optimization

𝐸 = 𝑖, 𝑗 : 𝑖 ≠ 𝑗 and 𝑞𝑖𝑗 ≠ 0 ,

𝑌𝑄 = 𝑥, 𝑌 : 𝑦𝑖𝑗 = 𝑥𝑖𝑥𝑗 ∀ (𝑖, 𝑗) ∈ 𝐸 .


BQP cuts for non-convex (MI)QPs

[joint project with IBM Research (Gϋnlϋk and Linderoth)]

BQP can be rewritten as

CPLEX has a lot of technology available to optimize over BQP

In particular, the 0 – 1/2 Chvátal-Gomory cut separator finds strong cutting planes for

BQP

When used in the context of a non-convex MIQP, we call these BQP-cuts

BQP ∶= conv

𝑦𝑖𝑗 ≤ 𝑥𝑖 , 𝑦𝑖𝑗 ≤ 𝑥𝑗 , 𝑦𝑖𝑗 ≥ 𝑥𝑖 + 𝑥𝑗 − 1, ∀(𝑖, 𝑗) ∈ 𝐸

( 𝑥, 𝑌 ∈ *0, 1+𝑛+|𝐸| :

)


BQP cuts for non-convex (MI)QPs

[joint project with IBM Research (Gϋnlϋk and Linderoth)]

BQP cuts in CPLEX 12.6.2 (June 2015)

BQP is a relaxation of Box-QP

Box-QP is a relaxation of any (MI)QP

0 – 1/2 Chvátal-Gomory cuts are strong cutting planes for BQP

We separate BQP cuts for (MI)QP as 0 – 1/2 cuts for the BQP relaxation of (MI)QP

Remarks

By scaling and shifting, separation can always be reduced to the 0 – 1 case

After branching, rescaling using tighter local bounds leads to tighter local cuts.

BQP arise from Box-QP but can be used to strengthen any non-convex (MI)QP

The efficacy of BQP cuts depends on how much side constraints already put inside BQP.

Related approaches

Similar ideas already present in the solvers GloMIQO (v2.0) and BARON (v24.3)

E.g.: Using facets of BQP is equivalent to “edge-concave” facets [Misener et al., 2015,

Meyer & Floudas, 2015] or “multiterm” relaxations [Bao et al., 2009]


0

0.2

0.4

0.6

0.8

1

>0s >1s

1.0

0

1.0

0

0.6

6

0.4

1

CPLEX 12.6.1

CPLEX 12.6.3


Testset: CPLEX test bed: 675 models; Box-QP test bed: 99 models



CPLEX 12.6.1 vs.12.6.3: Global (MI)QP performance improvement

CPLEX test bed


Time limits:

267 / 262

1.52x 2.42x

0

0.2

0.4

0.6

0.8

1

>0s >1s

1.0

0

1.0

0 CPLEX 12.6.1

CPLEX 12.6.3

Box-QP test bed [Vandenbussche & Nemhauser, „05,

Burer & Vandenbussche, „09]

80 models 66 models

Time limits:

56 / 19

70.3x 155x


0

0.2

0.4

0.6

0.8

1

>0s >1s

1.0

0

1.0

0

0.6

6

0.4

1

CPLEX 12.6.1

CPLEX 12.6.3


Testset: CPLEX test bed: 675 models; Box-QP test bed: 99 models



CPLEX 12.6.1 vs.12.6.3: Global (MI)QP performance improvement

CPLEX test bed


Time limits:

267 / 262

1.52x 2.42x

0

0.2

0.4

0.6

0.8

1

>0s >1s

1.0

0

1.0

0

0.7

9

0.6

1

CPLEX 12.6.1

CPLEX 12.6.3

CPLEX test bed

(without Box-QPs)


Time limits:

258 / 254

1.26x 1.65x


0

5

10

15

20

25

30

35

40

45

50

0

100

200

300

400

500

600

700

800

9.0(2003)

10.0(2006)

11.0(2007)

12.1(2009)

12.2(2010)

12.4(2011)

12.5.0(2012)

12.6.0(2013)

12.6.1(2014)

12.6.3(upcoming)

tota

l s

pe

ed

up

nu

mb

er

of

tim

eo

uts

CPLEX MILP performance evolution

10 sec

100 sec

1000 sec


Testset: MILP: 3685 models




Legal Disclaimer

• © IBM Corporation 2015. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and

accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this

information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible

for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is

intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and

conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market

opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in

these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales,

revenue growth or other results.

• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or

performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming

in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an

individual user will achieve results similar to those stated here.

• Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

• Other company, product, or service names may be trademarks or service marks of others.

Documents

CPLEX keeps getting better · LP-based Outer Approximation (OA) B&C, where an LP relaxation of SOCP is updated by separating OA constraints on the fly [Quesada and Grossmann, 1992]