An Outer Approximation Algorithm for Nonlinear Mixed ...liberti/ewminlp/exler.pdf · An Outer Approximation Algorithm for Nonlinear Mixed-Integer Programming based on Sequential Quadratic

Introduction Algorithm Numerical Results Conclusions

An Outer Approximation Algorithm for NonlinearMixed-Integer Programming based on Sequential

Quadratic Programming with Trust Region Stabilization

O. Exler1 T. Lehmann2 K. Schittkowski1

1University of Bayreuth - Department of Applied Computer Science

2ZIB - Konrad-Zuse-Zentrum Berlin - Department Optimization

... supported by Shell GameChanger (SIEP-EPR Rijswijk), under projectnumber 4600003917

Marseille, April 13, 2010

O. Exler (University of Bayreuth) Outer Approximation based on MISQP EWMINLP April 12-16, 2010 1 / 22


Outline

1 IntroductionProblem FormulationExample: Well Relinking

2 AlgorithmMixed-Integer SQP-Trust-Region MethodLinear Outer ApproximationOuter Approximation based on MISQP

3 Numerical Results100 Academic Test Cases - Schittkowski 200955 Test Cases from Petroleum Engineering

4 Conclusions



Problem Formulation

Mixed Integer Nonlinear Programming (MINLP):

min f (x , y) objective, e.g. costs

s.t. gj(x , y) ≥ 0 , j = 1, . . . ,m constraints, e.g. operational limits

x ∈ X , y ∈ Y

whereX := {x ∈ IRnc : xlb ≤ x ≤ xub} ,

Y := {y ∈ INni : ylb ≤ y ≤ yub} .

Assumptions:

Problem is non-convex and non-relaxable.

The model functions are continuously differentiable subject to thecontinuous variables x ∈ IRnc .

Time-consuming function evaluations.

No equality constraints for simplicity.



Problem Formulation

Mixed Integer Nonlinear Programming (MINLP):

min f (x , y) objective, e.g. costs

s.t. gj(x , y) ≥ 0 , j = 1, . . . ,m constraints, e.g. operational limits

x ∈ X , y ∈ Y

whereX := {x ∈ IRnc : xlb ≤ x ≤ xub} ,

Y := {y ∈ INni : ylb ≤ y ≤ yub} .

Assumptions:

Problem is non-convex and non-relaxable.

The model functions are continuously differentiable subject to thecontinuous variables x ∈ IRnc .

Time-consuming function evaluations.

No equality constraints for simplicity.



Example: Well Relinking

Problem: Maximize total production, three sources with differentproduction characteristics, three available sinks (nc = 3, ni = 9), simplestcase

Variables: Continuous flows x1, x2, x3 and binary decisions for splittingy1, . . . , y9



Example: Well Relinking

Maximize: Total flowx1 + x2 + x3

Splitfactor Constraints: Define switching conditions

y1 + y2 + y3 = 1

y4 + y5 + y6 = 1

y7 + y8 + y9 = 1

Upper Bounds: For pressure at source and mass rate at sink

50y1 + 10y2 + 5y3 ≤ 90− 89x1/900 x1y1 + x2y4 + x3y7 ≤ Qmax

50y4 + 10y5 + 5y6 ≤ 80− 79x2/900 x1y2 + x2y5 + x3y8 ≤ Qmax

50y7 + 10y8 + 5y9 ≤ 70− 69x3/900 x1y3 + x2y6 + x3y9 ≤ Qmax



Mixed-Integer SQP-Trust-Region Method

Approach

Mixed Integer Sequential Quadratic Programming (MISQP).Extension of the well-known SQP trust-region methods from continuousnonlinear optimization (Fletcher 1982, Yuan 1995).Advantages:

Continuous and integer variables are optimized directly andsimultaneously.

Approximation of L∞-penalty function.

Integer variables and computational complexity passed to QPsubproblem.

Efficient computation of a feasible solution, often close to the optimalone.




L∞-penalty function with penalty parameter σ > 0:

Pσ(x , y) = f (x , y) + σ‖g(x , y)−‖∞ ,

Approximation:

Φk(d) := ∇f (xk , yk)Td +1

2dTBkd + σk‖(g(xk , yk) +∇g(xk , yk)Td)−‖∞

where d :=

(dc

d i

)with dc ∈ IRnc and d i ∈ INni .

Subproblem:

min Φk(d)

‖dc‖∞ ≤ ∆ck , ‖d i‖∞ ≤ ∆i

k ,

xk + dc ∈ X , yk + d i ∈ Y ,

dc ∈ IRnc , d i ∈ INni .




Subproblem:

min Φk(d)

‖dc‖∞ ≤ ∆ck , ‖d i‖∞ ≤ ∆i

k ,

xk + dc ∈ X , yk + d i ∈ Y ,


Subproblem reformulation (MIQP):

min ∇f (xk , yk)Td + 12dTBkd + σkδ

δ + gj(xk , yk) +∇gj(xk , yk)Td ≥ 0 , j = 1, . . . ,m ,

‖dc‖∞ ≤ ∆ck , ‖d i‖∞ ≤ ∆i

k ,

xk + dc ∈ X , yk + d i ∈ Y , δ ≥ 0 ,

dc ∈ IRnc , d i ∈ INni , δ ∈ IR .




Subproblem:

min Φk(d)

‖dc‖∞ ≤ ∆ck , ‖d i‖∞ ≤ ∆i

k ,

xk + dc ∈ X , yk + d i ∈ Y ,


Subproblem reformulation (MIQP):

min ∇f (xk , yk)Td + 12dTBkd + σkδ

δ + gj(xk , yk) +∇gj(xk , yk)Td ≥ 0 , j = 1, . . . ,m ,

‖dc‖∞ ≤ ∆ck , ‖d i‖∞ ≤ ∆i

k ,

xk + dc ∈ X , yk + d i ∈ Y , δ ≥ 0 ,

dc ∈ IRnc , d i ∈ INni , δ ∈ IR .




Quotient of actual and predicted improvement of Pσk(xk):

rk =Pσk

(xk , yk)− Pσk(xk + dc

k , yk + d ik)

Φk(0)− Φk(dk).

⇒ Update trust region radii dependent on rk .

Update Trust Region Parameter:

∆ck+1 =

{max(2∆c

k , 4‖dck ‖∞) , rk > 0.9

min(∆ck/4, ‖dc

k ‖∞/2) , rk < 0.1

∆ik+1 =

{max(2∆i

k , 4‖d ik‖∞) , rk > 0.9

min(∆ik/4, ‖d i

k‖∞/2) , rk < 0.1




General Idea

1 Compute function values f (xk , yk), g1(xk , yk),..., gm(xk , yk)

2 Approximate ∇f (xk , yk), ∇g1(xk , yk),..., ∇gm(xk , yk) by differenceformula, at grid points for integer variables

3 Update quasi-Newton Bk matrix using modified BFGS formula

4 Generate MIQP and apply trust region parameter separately forcontinuous and integer variables (∆c

k , ∆ik)

5 Solve the MIQP by any available technique, e.g., branch-and-cut

6 Add second-order correction of Yuan 1995, if necessary

Properties:

Quick computation of feasible integer solutions

Applicable to non-convex and non-relaxable programs

No global convergence, even for convex problems




General Idea

1 Compute function values f (xk , yk), g1(xk , yk),..., gm(xk , yk)

2 Approximate ∇f (xk , yk), ∇g1(xk , yk),..., ∇gm(xk , yk) by differenceformula, at grid points for integer variables

3 Update quasi-Newton Bk matrix using modified BFGS formula

4 Generate MIQP and apply trust region parameter separately forcontinuous and integer variables (∆c

k , ∆ik)

5 Solve the MIQP by any available technique, e.g., branch-and-cut

6 Add second-order correction of Yuan 1995, if necessary

Properties:

Quick computation of feasible integer solutions

Applicable to non-convex and non-relaxable programs

No global convergence, even for convex problems



Linear Outer Approximation

Global stabilization by Linear Outer Approximation

Motivation:

Global lower bounds guarantee optimality forconvex MINLP problems.

Exploration of different areas of the feasibleregion.

Linear Outer Approximation:

Overestimation of feasible region.

Underestimation of objective function.

Linear relaxation of MINLP problem.

Pictures: Grossmann 2002O. Exler (University of Bayreuth) Outer Approximation based on MISQP EWMINLP April 12-16, 2010 11 / 22


Linear Outer Approximation

Master Program: Duran and Grossmann 1986, Fletcher and Leyffer 1994

min η

uk > η

η ≥ f (xp, yp) +∇f (xp, yp)T

(x − xp

y − yp

), p ∈ Tk

0 ≤ gj(xp, yp) +∇gj(xp, yp)T

(x − xp

y − yp

), j = 1, ...,m,

0 ≤ gj(xl , yl) +∇gj(xl , yl)T

(x − xl

y − yl

), j = 1, ...,m, l ∈ Sk

x ∈ X , y ∈ Y , η ∈ IR

Tk : set of previous iterates, for which MINLP for fixed yk solvable, xk isthe corresponding solution.Sk : set of previous iterates, for which MINLP not solvable, xk is thecorresponding solution of a feasibility problem.uk : current upper bound for objective function.



Outer Approximation based on MISQP

Trust Region Part

1 Solve MIQP with ∆ik = 0 fixed, giving d̂k = (d̂c

k , 0) .

2 Solve MIQP with d ik ∈ INni , giving dk = (dc

k , dik) .

3 Evaluate Pσk((xk , yk) + dk) and Pσk

((xk , yk) + d̂k) .

4 IfPσk

((xk , yk) + dk) < Pσk((xk , yk) + d̂k) ,

then use dk . (Improving mixed integer search direction)Else set dk := d̂k .

5 Calculate rk =Pσk

(xk ,yk )−Pσk((xk ,yk )+dk )

Φk (0)−Φk (dk ) .

6 Depending on rk , reduce or increase the trust region radii.

7 If Φk(0)− Φk(dk) small, convergence against stationary point, gotoStep 8 (Outer Approximation Part).Else goto Step 1.



Outer Approximation based on MISQP

Outer Approximation Part

8 Update linear outer approximation master problem by addinglinearizations and solve Master Program, giving (xOA

k , yOAk ) .

If yOAk 6= yOA

l , for all previous OA steps, let k = k + 1 and goto Step 1and repeat Trust Region Part with Step 2.

Else goto Step 1 and repeat Trust Region Part without Step 2.

Theorem

The Algorithm terminates at an optimal solution of convexMINLP-problems.



Codes

MINLP solvers:

MISQPN: Outer Approximation solver supported by MIQP search steps(preliminary implementation)

MISQPN0: MISQPN without MIQP search steps

MISQP: Mixed integer SQP trust region method

MISQPOA: External Outer Approximation solver based on MISQP

MINLPB4: Branch and bound solver based on MISQP



Test Case Library

Collection of 100 academic MINLP test cases.Details:

nc ≤ 21,ni ≤ 100m ≤ 54, including nonlinear equality constraints.

(available:

http://www.math.uni-bayreuth.de/̃ kschittkowski/downloads.htm )

Collection of 55 problems from Petroleum Engineering.Details:

3 ≤ nc ≤ 10,9 ≤ ni ≤ 27m ≤ 30, including linear equality constraints.



100 Academic Test Cases - Schittkowski 2009

Numerical Resultscode nsucc ∆err nerr nfunc ngrad

MISQPN 74 1.19 3 714 35MISQPN0 57 5.68 19 316 21MISQP 91 0.15 1 793 29MISQPOA 96 0.05 1 6,006 102MINLPB4 91 1.16 4 111,611 4,430

Table: Test Results for a Set of 100 Academic Test Problems

nsucc - number of successful test runs, (f (xk , yk)− f ?)/|f ?| ≤ 0.01∆err - average relative deviation of computed solution from known

one, (f (xk , yk)− f ?)/|f ?| , taken over all feasible solutionsnerr - number of test runs terminated by an error messagenfunc - average number of equivalent function calls including function

calls used for gradient approximations, for successful runsngrad - average number of gradient evaluations, for successful runs



100 Academic Test Cases - Schittkowski 2009

code nsucc nloc nerr nfunc ngrad time

MISQP/rx 90 8 2 68 18 0.02MISQP 91 8 1 793 29 15.20

Table: Performance Results of Mixed-Integer versus Continuously Relaxed TestProblems

Observation: At least 10 % of our test problems are non-convex.



55 Test Cases from Petroleum Engineering

Numerical Results

code nsucc ∆err nerr nfunc ngrad

MISQPN 47 0.05 8 14,064 648MISQPN0 33 0.26 21 17,340 807MISQP 34 0.08 1 1,797 82MISQPOA 50 0.03 0 30,359 1,925MINLPB4 55 0.0 0 176,460 8,107

Table: Performance Results for a Set of 55 Well Relinking and Gas Lift TestProblems



Conclusions I

MISQPN - efficient if binary variables are present (preliminaryversion), MIQP search steps increase robustness (see MISQPN0)

MISQP - extremely efficient computation of feasible solutions, lessrobust especially for binary problems

MISQPOA - MISQP called inside, less efficient, but more robust byouter approximations, especially for binary variables.

MINLPB4 - very robust, but highly inefficient, not applicable withtime-consuming function evaluations



Conclusions II

Development of efficient mixed-integer nonlinear optimizationalgorithms

Applicable for solving non-convex and non-relaxable programs

Efficient computation of feasible solutions

Large internal calculation times (MIQP solutions), acceptable due toexpensive simulations in real world applications

Applicable without analytical derivatives

Codes in practical use (Shell, Pace Aerospace Engineering, GeneralElectric, ...)



The End

Thank you for your attention!


Documents

An Outer Approximation Algorithm for Nonlinear Mixed ...liberti/ewminlp/exler.pdf · An Outer Approximation Algorithm for Nonlinear Mixed-Integer Programming based on Sequential Quadratic