Innovations in CPLEX performance and solver capabilities

Decision Optimization

INFORMS 2013 Workshop - Innovations

INFORMS Fall Conference 2013

Building a Smarter Planet withDecision Optimization

© 2013 IBM Corporation


2 INFORMS 2013 Workshop - Innovations

Workshop presentations

3:00 - 3:30: Innovations in CPLEX performance and solver capabilities –Mary Fenelon

3:30 – 4:15: Expert tips and tricks for the OR practitioner: Everything you need to know about CPLEX-powered modeling and solving – Ed Klotz

4:30 - 5:10 From "Good Modeling" to developing and deploying great enterprise applications – Yianni Gamvros, Yana Ageeva, John Chaves

5:10 – 5:30 CPLEX-based tools and advanced analytics for Academia –Dennis Bly


INFORMS 2013 Workshop - Innovations

INFORMS Fall Conference 2013

Innovations in CPLEX performance and solver capabilities

Mary Fenelon

Development Manager

IBM ILOG CPLEX Optimization Studio

[email protected]




Disclaimer

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM®benchmarks in a controlled environment. The actual throughput orperformance that any user will experience will vary depending upon manyfactors, including considerations such as the amount of multiprogramming inthe user's job stream, the I/O configuration, the storage configuration, and theworkload processed. Therefore, no assurance can be given that an individualuser will achieve results similar to those stated here.




Agenda

New features in ILOG CPLEX Optimization Studio

Distributed computing options for CPLEX Optimizer

Performance




Agenda



Performance




Model development Tools

Optimization Engines

Math Programming

CPLEX Optimizers

(Simplex, Barrier, Mixed

Integer)

Constraint Programming

Constraint-based scheduling

CPLEX CP Optimizers

CPLEX Studio (IDE) - OPL Modeling Language

ILOG Concert Technology (C++, .NET, Java) Java client only

CPLEX Optimization Studio

CPLEX

Enterprise

Server

Connectors

•Microsoft Excel

•MATLAB

•Pytthon

•AMPL

Tools & APIs

•CPLEX Interactive

•C Callable Library




New features

CPLEX 12.5.1 – June 2013– Remote Object, a distributed algorithm toolkit for Java and C++– Additional OPL scripting methods and APIs

CPLEX 12.6 – December 2013– Additional OPL scripting methods– Improved model editor– LP file viewer– New constraints in CP Optimizer– CP Optimizer model dump facility– Global solution to nonconvex QP and MIQP– Distributed MIP algorithm




Additional OPL scripting methods and API

12.5.1 APIs– Relaxation iterator IloOplRelaxationIterator– Conflict iterator IloOplConflictIterator

12.5.1 scripting– Multiple MIP starts

12.6 scripting– Lazy constraints and user cuts– Solution pools– Priorities– Problem queries




Improved model editor

Existing features– Scope selection– Syntax coloring– Search/replace– Syntax errors marked at run time– Etc.

New syntax-aware editing enables– Highlighting occurrences– Renaming per entity type– Refactoring and formatting– Search/find per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing





New syntax-aware editing enables– Highlighting occurrences– Refactoring and formatting– Search/replace per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing



















LP file viewer




New constraints in CP Optimizer

Strong constraint

SameSequence constraint





Benefits– Reinforces the model propagation between variables where the user knows

that the domain reduction is weak– Avoids the manual enumeration of tuple sets for creating

allowedAssignments– Creates stronger allowedAssignments than those obtained by hand

Processing– Enumerate partial solutions by performing a search on the variables of each

strong constraint, but using domain reduction of the whole model– Add an allowedAssignments constraint over the variables specified in the

strong constraint, with a tuple set corresponding to the partial solutions found in the previous search

– Remove constraints which are no longer needed given that the strongconstraint is now present in the model

Strong(x1, x2 … xn)





IloIntTupleSet gha(env, 3);

IloIntArray tuple(env, 3);

for (IloInt i = 0; i < n; i++) {

tuple[0] = i;

for (IloInt j = 0; j < n; j++) {

if (i != j) {

tuple[1] = j;

tuple[2] = Game(i, j, n);

gha.add(tuple);

}

}

}

for (IloInt i = 0; i < nbWeeks; i++) {

for (IloInt j = 0; j < nbGamesPerWeek; j++) {

IloIntVarArray vars(env);

vars.add(home[i][j]);

vars.add(away[i][j]);

vars.add(games[i][j]);

model.add(IloAllowedAssignments(env, vars, gha));

}

}

! Best Branches Non-fixed

* 56 2688 0.53s

* 52 7402 0.99s

* 48 19418 2.38s

* 46 102k 11.87s

* 44 109k 12.81s

* 42 195k 20.22s

* 40 218k 22.43s

* 38 239k 24.47s

for (IloInt i = 0; i < nbWeeks; i++) {

for (IloInt j = 0; j < nbGamesPerWeek; j++) {

model.add(home[i][j] != away[i][j]);

model.add(games[i][j] == Game(home[i][j], away[i][j], n));

IloIntVarArray vars(env);

vars.add(home[i][j]);

vars.add(away[i][j]);

vars.add(games[i][j]);

model.add(IloStrong(env, vars));}

}

! Best Branches Non-fixed W Branch decision

* 50 520 0.44s 6 -

* 46 2468 0.65s 1 -

* 38 4304 0.88s 1 -

* 36 25689 2.98s 1 -

* 34 86973 8.15s 1 -

The original tuple set has 90 tuples, the generated ones have between 5 and 66 tuples.

Using the strong constraint in sports.cpp





CP Optimizer has the concept of an interval sequence. In OPL:dvar sequence seq1[r in Res] in all(t in tasksOnRes[r]) a[t];

The sameSequence constraint constrains two different sequences to have the same order of corresponding intervalssameSequence(seq1, seq2)

The correspondence can be specified explicitly or be implicit from the order of intervals given in the definition of the sequences

The sameSequence constraint can naturally model constraints like no-bypass whereby products must be processed by different machines in the same order (although that order is free at the outset

SameSequence (seq1, seq2)

M

1

M

2Conveyor belt

seq1 seq2

sameSequence(seq1, seq2)





dvar interval itvs[j in Jobs][m in Mchs] size

OpDurations[j][m];

dvar sequence mchs[m in Mchs] in all(j in Jobs) itvs[j][m]

types all(j in Jobs) j;

minimize max(j in Jobs) endOf(itvs[j][nbMchs-1]);

subject to {

forall (m in Mchs)

noOverlap(mchs[m]);

forall (j in Jobs, o in 0..nbMchs-2)

endBeforeStart(itvs[j][o], itvs[j][o+1]);

forall (m in Mchs, j in Jobs: 0<m)

typeOfPrev(mchs[m],itvs[j][m],nbJobs)

== typeOfPrev(mchs[0],itvs[j][0],nbJobs);

}

Using sameSequence in sched_flowshop.mod

dvar interval itvs[j in Jobs][m in Mchs] size OpDurations[j][m];

dvar sequence mchs[m in Mchs] in all(j in Jobs) itvs[j][m];

minimize max(j in Jobs) endOf(itvs[j][nbMchs-1]);

subject to {

forall (j in Jobs, o in 0..nbMchs-2)

endBeforeStart(itvs[j][o], itvs[j][o+1]);

forall (m in Mchs)

noOverlap(mchs[m]);

forall (m in Mchs: 0<m)

sameSequence(mchs[0], mchs[m]);

}




CP Optimizer model dump facility

Model instance can be written to a file– Constraints– Variables– Search phases– Parameter settings

Easier way to communicate instance to IBM Support

Aids in debugging formulation

Parameter to anonymize names




CPLEX Optimizer parameter hierarchy

Provide consistent naming across APIs

Traditional names will be maintained

Easier to find relevant parameters

Based on hierarchy used in the CPLEX Interactive Optimizer




CPLEX Optimizer parameter hierarchy

API Traditional Hierarchical

Callable Library

CPX_PARAM_FRACCUTS CPXPARAM_MIP_Cuts_Gomory

Java IloCplex.IntParam.FracCuts IloCplex.Param.MIP.Cuts.Gomory

.NET Cplex.IntParam.FracCuts Cplex.Param.MIP.Cuts.Gomory

C++ IloCplex::FracCuts IloCplex::Param::MIP::Cuts::Gomory

Python cplex.parameters.mip.cuts.gomory cplex.parameters.mip.cuts.gomory

MATLAB cplex.Param.mip.cuts.gomory cplex.Param.mip.cuts.gomory

Example




Global solution to nonconvex QP and MIQP

When Q is positive-semidefinite, the objective function is convex

When Q is indefinite, the objective function is nonconvex

min ½ xTQx + cTx

s.t. Ax = b

u ≤ x ≤ l

optionally some x integral

Quadratic program (QP) and

mixed integer quadratic program (MIQP)





Solution to convex QP and MIQP in CPLEX since v. 4.0 and v. 8.0, resp.

Solution to first order optimality for nonconvex QP since v. 12.3– Primal-dual interior point algorithm– Requires linear algebra routines for indefinite factorization – CPLEX wrote own parallel factorization and solution routines

Solution to global optimality for nonconvex QP and MIQP in v. 12.6





Global solution requires branching on continuous variables– QP solved as MIQP, even if no integer variables– NP-complete, can be much more time-consuming than solving to local optimum

Solution Algorithm– Spatial branch-and-cut over convex QP relaxation

• Branch on continuous variables• Tighten linear relaxation in local nodes by locally valid cuts

– Automatic selection of linearization of non-convex part in Q• McCormick relaxation• Eigenvector reformulation based on indefinite factorization

– Solve the indefinite problem when an incumbent is found




Agenda



Performance




Distributed computing*

Core speed vs. more cores

Memory bandwidth on a single system limited by single cache and memory bus

High performance servers

Clusters of commodity hardware

Cloud

*Koch, Shinano and Ralphs. What Could a Million Cores Do To Solve Integer Programs? 2012.

http://coral.ie.lehigh.edu/~ted/files/talks/MillionINFORMS12.pdf




Distributed computing

Client-server– CPLEX Enterprise Server or ILOG ODM Enterprise Optimization Server– CPLEX Remote Object

Distributed algorithms– Build your own with the CPLEX Remote Object– For MIP, use the CPLEX distributed MIP algorithm, new in v. 12.6




Distributed computing: Client-server

Optimization Server in ODM Enterprise or CPLEX Enterprise Server– Built to enterprise-class standards– WebSphere Application Server

• Queuing, failover, etc.• Administration

– Oracle, DB2 or Microsoft SQL database to store results– J2EE compliant– Asynchronous, e.g., client can be disconnected

CPLEX Enterprise Server– From IDE

• IDE panel to submit/monitor/control jobs• Results are returned to IDE

– From Java APIs– OPL .mod and .dat files are sent to server

• Data read on server, not client• Data can be in .dat file or from a database, flat files, etc.

ODM Enterprise– With the Java API for CPLEX, CP Optimizer or OPL via Optimization Server custom task




Distributed computing: CPLEX Remote Object

Transparent use of remote hardware

Change single line of code to describe connection

All methods are mapped to messages to remote system

Transport abstraction hides the details of using these protocols– ssh– TCP/IP– MPI– Local

optimize(Data data)

{

IloEnv env;

IloCplex cplex(env, transport

argc, argv);

buildModel(cplex, data);

cplex.solve();

useSolution(cplex);

}

optimize(Data data)

{

IloEnv env;

IloCplex cplex(env);


cplex.solve();

useSolution(cplex);

}




Distributed computing: CPLEX Remote Object

Synchronous or asynchronous

Asynchronous calls support a master/worker distributed parallel programming paradigm for building distributed parallel optimization algorithms

Asynchronous calls return AsyncHandleobjects to join back

Examples provided:– iloparbenders.cpp

– iloparmipopt.cpp

– ParBenders.java

– parbenders.c

– parmipopt.c

optimize(Data data)

{

IloEnv env;

AsyncHandle handle;


argc, argv);


handle = cplex.solve();

doOtherThings();

handle.join();

useSolution(cplex);

}

optimize(Data data)

{

IloEnv env;


argc, argv);


cplex.solve();

useSolution(cplex);

}




Distributed computing: CPLEX distributed MIP

Built on the CPLEX Remote Object technology

A “master” distributes work to multiple “workers”

Two phases– Racing ramp-up*– Each machine uses different settings, and a winner is selected

• Only incumbent objective values and best bounds are communicated• Infinite ramp-up allowed (concurrent MIP)

– Distributed tree• Nodes of the tree created by the winner distributed to workers• Nodes exchanged at synchronization points

– Deterministic

Available in the CPLEX Interactive, and the C, C++, Java, Matlab and Python APIs

*Shinano, Achterberg, Berthold, Heinz, Koch. ParaSCIP – a parallel extension of SCIP. 2010.

http://nbn-resolving.de/urn:nbn:de:0297-zib-11921








Does hardware need to be identical?– No, but workers should be similar in performance to get best overall performance– Master does not have to be similar to workers

When should I use infinite ramp-up?– To exploit performance variability– In order to achieve a 1.5x to 2x speed-up for problems which aren’t too trivial

When should I use distributed search?– Large number of nodes to enumerate– Very hard problems

Do I have to have exclusive access to the machine?– No, but for best overall performance, the worker should have little load

Does it work on HPC systems?– Yes. MPI is the de facto standard for HPC and is one of the supported transport

protocols– Experiments have been done on an IBM Blue Gene Q with 256 workers

FAQs





Why would I want determinism?– The same solution will be reached each time so the downstream

processing will see consistent results– For tuning and debugging of the application

Which MPI distributions are supported?– OpenMPI and MPICH, depending on the platform

What shared memory parallel MIP features are supported?– MIPstarts – Informational callback on the master– Almost all parameters, e.g.

• Gap• Time limits

– Terminate function

FAQs





What shared memory parallel MIP features are not supported?– Control callbacks – Restarts– Global solution to nonconvex MIQP and QP

What do I need to do to set up a distributed run?– CPLEX Interactive on each machine with appropriate shared libraries– Describe the machines in a VMC file or with function arguments– Other steps depend on the transport

When does ramp-up stop?– Controlled via parameters– “Automatic” value balances number of open nodes and solution quality

FAQs




Agenda



Performance




Performance – CP Optimizer 12.6

Contains a new scheduling algorithm

Increases search robustness on medium-sized problems

Provides more optimality proofs on smaller instances

The new search is activated by default

Also some performance increase on integer problems

Version-to-version average solution time ratios12.3/12.2 12.4/12.3 12.5/12.4 12.5.1/12.5 12.6/12.5.1

Scheduling1 worker

1.06 1.01 1.29 1.03 1.69

Scheduling4 workers

1.22 0.97 1.35 1.03 1.63

Integer prob.1 worker

1.10 0.99 0.97 1.05 1.10

Integer prob.4 workers

1.06 1.01 1.01 1.03 1.08




Performance – CPLEX Optimizer 12.6

Average speedup of MIP optimizer on MILP v. 12.6 compared to v. 12.5.1– 15% on hard MILP models (more than 100 sec.)– 5% on non-trivial MILP problems (more than 1 sec.)




0

200

400

600

800

1000

1200

1400

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

nu

mb

er o

f ti

meo

uts

0

50

100

150

200

250

tota

l sp

eed

up

10 sec

100 sec

1000 sec

Date: 28 September 2013

Testset: 3147 models (1792 in 10sec, 1554 in 100sec, 1384 in 1000sec)

Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threads (deterministic since CPLEX 11.0)

Timelimit: 10,000 sec

v6.0

v6.5.3

v7.0v8.0

v9.0

v10.0

v11.0 v12.1v12.2

v12.4v12.5

v12.6

CPLEX MILP performance




Technical Sessions

Analyzing 12 Years of Progress in CPLEX Sunday, 16:30 - 18:00 Roland Wunderling

Recent Developments in CPLEX Monday, 08:00 - 09:30 Tobias Achterberg

Non-convex Quadratic Programming in CPLEX Tuesday, 11:00 - 12:30 Christian Bliek and Pierre Bonami

Tutorial: Performance Variability in Mixed-integer Programming Tuesday, 8:00 – 9:30 Andrea Lodi and Andrea Tramontani

Software Demonstration: Expert Tips and Tricks for Using CPLEX Optimization Studio…for Distributed Computing Tuesday Oct 08, 16:30 - 17:15

Lift-and-Project Cuts in CPLEX 12.5.1 Wednesday, 13:30 - 15:00 Andrea Tramontani

Concurrent root cut loops to exploit random performance variability Wednesday, 15:30 - 17:00 A. Tramontani, M. Fischetti, A. Lodi, D. Salvignin, M. Monaci

Interesting Use Cases for the CPLEX Remote Object Wednesday, 15:30 - 17:00 Laszlo Ladanyi and Daniel Junglas