Upload
lehanh
View
231
Download
0
Embed Size (px)
Citation preview
General Nonlinear Programming (NLP) Software
CAS 737 / CES 735
Kristin DaviesHamid Ghaffari
Alberto Olvera-SalazarVoicu Chis
January 12, 2006
OutlineIntro to NLPExamination of:
IPOPT PENNON CONOPT LOQO KNITRO
Comparison of Computational ResultsConclusions
Intro to NLPThe general problem:
{ }{ }
.
,,,
,...,1,0)(,...,1,0)(..
)(min
Condefinedfunctionsare
ghfandsetcertainaisCxwhereCx
mJjxgpIixhts
xf
jinn
j
i
ℜ⊆ℜ∈
∈
=∈≤=∈=
Either the objective function or some of the constraints may be nonlinear
(NLP)
Intro to NLP (cont’d…)Recall:
The feasible region of any LP is a convex setif the LP has an optimal solution, there is an extreme point of the feasible set that is optimal
However:even if the feasible region of an NLP is a convex set, the optimal solution might not be an extreme point of the feasible region
Intro to NLP (cont’d…)Some Major approaches for NLP
Interior Point MethodsUse a log-barrier function
Penalty and Augmented Lagrange MethodsUse the idea of penalty to transform a constrained problem into a sequence of unconstrained problems.
Generalized reduced gradient (GRG)Use a basic Descent algorithm.
Successive quadratic programming (SQP)Solves a quadratic approximation at every iteration.
Summary of NLP Solvers
PENNON
AugmentedLagrangian Methods
KNITRO (TR)IPOPT, LOQO(line search)
Interior PointMethods
CONOPT
Reduced GradientMethods
NLP
IPOPT SOLVER (Interior Point OPTimizer)
CreatorsAndreas Wachter and L.T. Biegler at CMU (~2002)
AimsSolver for Large-Scale Nonlinear Optimization problems
ApplicationsGeneral Nonlinear optimizationProcess Engineering, DAE/PDE Systems, Process Design and Operations, Nonlinear Model Predictive control, Design Under Uncertainty
IPOPT SOLVER (Interior Point OPTimizer)
Input FormatCan be linked to Fortran and C code MATLAB and AMPL.
Language / OSFortran 77, C++ (Recent Version IPOPT 3.x)Linux/UNIX platforms and Windows
Commercial/FreeReleased as open source code under the Common Public License (CPL). It is available from the COIN-OR repository
IPOPT SOLVER (Interior Point OPTimizer)
Key ClaimsGlobal Convergence by using a Line Search.
Find a KKT pointPoint that Minimizes Infeasibility (locally)
Exploits Exact Second DerivativesAMPL (automatic differentiation)If not Available use QN approx (BFGS)Sparsity of the KKT matrix.
IPOPT has a version to solve problems with MPEC Constraints. (IPOPT-C)
IPOPT SOLVER (Interior Point OPTimizer)
AlgorithmInterior Point method with a novel line search filter.Optimization Problem
00)(..
)(min
≥=
ℜ∈
xxcts
xfnx
0)(..
)log()()(min )(
=
−= ∑ℜ∈
xcts
xxfxi
il
xln
μϕμ
The bounds are replaced by a logarithmic Barrier term. The method solves a sequence of barrier problems for decreasing values of μl
IPOPT SOLVER (Interior Point OPTimizer)
Algorithm(For a fixed value of )
Solve the Barrier ProblemSearch Direction (Primal-Dual IP)
Use a Newton method to solve the primal dual equations.Hessian Approximation (BFGS update)
Line Search (Filter Method)Feasibility Restoration Phase
lμ
IPOPT SOLVER (Interior Point OPTimizer)
Optimization Problem
00)(..
)(min
≥=
ℜ∈
xxcts
xfnx
0)(..
)log()()(min )(
=
−= ∑ℜ∈
xcts
xxfxi
il
xln
μϕμ
lμ
Outer LoopOuter Loop
The bounds are replaced by a logarithmic Barrier term.
The method solves a sequence of barrier problems for decreasing values of
IPOPT SOLVER (Interior Point OPTimizer)
Algorithm (For a fixed value of )Solve the Barrier Problem
Search Direction (Primal-Dual IP)Use a Newton method to solve the primal dual equationsHessian Approximation (BFGS update)
lμ
IPOPT SOLVER (Interior Point OPTimizer)
InnerLoopInnerLoop
0)(0
0)()(
==−
=−∇+∇
xceXVe
vxcxfμ
λ
Optimality conditionsOptimality conditions
eXvVariablesDual
1−= μ
0)(..
)log()()(min )(
=
−= ∑ℜ∈
xcts
xxfxi
il
xln
μϕμ Barrier Barrier NLPNLP
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−
−∇+∇−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡∇
−∇
eeVXxc
vxcxf
ddd
VXxc
IxcH
kk
k
kkT
kk
vk
k
xk
kk
Tk
kk
μ
λλ )(
)()(
000)(
)(
),(2kkxxk xLH λ∇=
At a Newton's iteration At a Newton's iteration ((xxkk,,λλkk,v,vkk))
⎥⎦
⎤⎢⎣
⎡ −∇+∇−=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−∇∇+Σ+
)()()(
)()(
k
kkT
kk
k
xk
cT
k
kHkk
xcvxcx
dd
IxcxcIH λϕδ
δ μλ
At a Newton's iteration At a Newton's iteration ((xxkk,,λλkk,v,vkk))
Algorithm Core: Solution of this Linear systemAlgorithm Core: Solution of this Linear system
IPOPT SOLVER (Interior Point OPTimizer)
Algorithm (For a fixed value of )Line Search (Filter Method)
A trial point is accepted if improves feasibility or if improves the barrier function
Assumes Newton directions are “Good”especially when using Exact 2nd Derivatives
lμ
vkkkk
xkkkk
dvv
dxx
α
α
+=
+=
+
+
1
1
[ ] [ ]
[ ] [ ]kkk
kkk
xxor
xcxc
ϕαϕ
α
≤
≤
)(
)(If
IPOPT SOLVER (Interior Point OPTimizer)
Line Search - Feasibility Restoration Phase
When a new trial point does not provides sufficient improvement.
0..
)(min 2
2
≥ℜ∈
xts
xcnx
Restore FeasibilityMinimize constraint
violation
00)(..
min 2
2
≥=
−ℜ∈
xxcts
xx kx n
Force Unique SolutionFind closest feasible point.
Add Penalty function
The complexity of the problem increases when complementarity conditions are introduced from:
miywywx
ywxcst
ywxf
ii
ywx mmn
K1,00,,
0),,(.
),,(min
)()(
,,
==
≥=
ℜ∈ℜ∈ℜ∈
( ) ( ) ( )( )
miywywxc
st
ywx
ywxf
ii
m
iim
iin
ii
ywx mmn
K1,00),,(
.
lnlnln
),,(min
)()(
1)(
1)(
1)(
,,
==
=
++− ∑∑∑ ===
ℜ∈ℜ∈ℜ∈
μ
δμ≤)()( ii yw
•The interior Point method for NLPs has been extended to handle complementarity problems. (Raghunathan et al. 2003).
0)()( =ii yw is relaxed as 0)(
)()()(
≥
=+i
iii
ssyw δμ
IPOPT SOLVER (Interior Point OPTimizer)
IPOPT SOLVER (Interior Point OPTimizer)
AdditionalIPOPT 3x. Is now programmed in C++.Is the primary NLP Solver in an undergoing project for MINLP with IBM.
ReferencesIpopt homepage:
http://www.coin-or.org/Ipopt/ipopt-fortran.htmlA. Wächter and L. T. Biegler, On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming, Research Report, IBM T. J. Watson Research Center, Yorktown, USA, (March 2004 - accepted for publication in Mathematical Programming)
PENNON (PENalty method for NONlinear& semidefinite programming)
CreatorsMichal Kocvara & Michael Stingl (~2001)
AimsNLP, Semidefinite Programming (SDP), Linear & Bilinear Matrix Inequalities (LMI & BMI), Second Order Conic Programming (SOCP)
ApplicationsGeneral purpose nonlinear optimization, systems of equations, control theory, economics & finance, structural optimization, engineering
SDP (SemiDefinite Programming)
Minimization of a linear function subject to the constraint that an affine combination of symmetric matrices is positive semidefinite
∑=
+=
≥m
iii
T
FxFxFwhere
xFtsxc
10)(
0)(..min
),...,(matricessymmetric1 0 mFFm +
Linear Matrix Inequality (LMI)defines a convex constraint on x
SDP (SemiDefinite Programming)
-always an optimal point on the boundary
-boundary consists of piecewise algebraic surfaces
SOCP (Second-Order Conic Programming)
Minimization of a linear function subject to a second-order cone constraint
iTiii
T
dxcbxAtsxc
+≤+..min
⎭⎬⎫
⎩⎨⎧
≤∈∈⎥⎦
⎤⎢⎣
⎡= − tuRtRu
tu
C kk ,,1
Called a second-order cone constraint since the unit second-order cone of dimension k is defined as:
Which is called thequadratic, ice-cream,or Lorentz cone
PENNON (PENalty method for NONlinear& semidefinite programming)
Input FormatMATLAB function, routine called from C or Fortran, stand-alone program with AMPL
LanguageFortran 77
Commercial/FreeVariety of licenses ranging from
Academic – single user ($460 CDN) to Commercial – company ($40,500 CDN)
PENNON (PENalty method for NONlinear& semidefinite programming)
Key Claims1st available code for combo NLP, LMI, & BMI constraintsAimed at (very) large-scale problemsEfficient treatment of different sparsitypatterns in problem dataRobust with respect to feasibility of initial guessParticularly efficient for large convex problems
PENNON (PENalty method for NONlinear& semidefinite programming)
AlgorithmGeneralized version of the Augmented Langrangian method (originally by Ben-Tal & Zibulevsky)
( ) 0/)(..)(min
≤iigi pxgptsxfϕ
Augmented Lagrangian
( )∑=
+=gm
iiigii pxgpuxfpuxF
1/)()(),,( ϕ
multiplierLagrange
functionpenaltyparameterpenalty0
sconstraintinequalityof#
,...,1
=
==>
=Augmented Problem =
i
g
i
g
g
u
p
m
mi
ϕ
PENNON (PENalty method for NONlinear& semidefinite programming)
The AlgorithmConsider only inequality constraints from (NLP)
Based on choice of a penalty function, φg, that penalizes the inequality constraints
Penalty function must satisfy multiple properties such that the original (NLP) has the same solution as the following “augmented” problem:
( )0
,...1,0/)(..,)(min
>
=≤ℜ∈
i
giigi
pwith
mipxgptsxxf
ϕ (NLPφ)
[3] Kocvara & Stingl
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm (Cont’d…)The Lagrangian of (NLPφ) can be viewed as a (generalized) augmented Lagrangian of (NLP):
( )∑=
+=gm
iiigii pxgpuxfpuxF
1/)()(),,( ϕ
Lagrange multiplierPenalty parameter
Penalty function
Inequality constraint
[3] Kocvara & Stingl
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS.,...,1,0.)0( 111
gii mipLetgivenbeuandxLet =>
.,...2,1
satisfiediscriteriumstoppingauntilrepeatkFor =
[3] Kocvara & Stingl
( )( )( )
gki
ki
gki
kig
ki
ki
kkkx
k
mippiii
mipxguuii
KpuxFthatsuchxFindi
,...,1,)(
,...,1,/)(
,,)(
1
1'1
11
=<
==
≤∇
+
++
++
ϕ
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
.,...,1,0.)0( 111gii mipLetgivenbeuandxLet =>
InitializationCan start with an arbitrary primal variable , therefore, chooseCalculate initial multiplier valuesInitial p= , typically between 10 - 10000
1iu
01 =xx
[3] Kocvara & Stingl
0π
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
( ) KpuxFthatsuchxFindi kkkx
k ≤∇ ++ ,,)( 11
(Approximate) Unconstrained MinimizationPerformed either by Newton with Line Search, or by Trust RegionStopping criteria:
[3] Kocvara & Stingl
( )( ) ( )( )( ) ( ) 11 ,,,,
/,,
1.0,,
1
2
1'
2
1
2
1
−−∇⋅≤∇
−⋅≤∇
=≤∇
+
++
+
H
kkkxH
kkkx
ki
kig
ki
ki
kkkx
kkkx
puxFpuxF
orpxguupuxF
orpuxF
α
ψα
αα
1)'(' −= ϕψ),,(minarg1 kk
x
k puxFx =+
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
5.0,1 typically≤μ
( )( ) gki
kig
ki
ki mipxguuii ,...,1,/)( 1'1 == ++ ϕ
Update of MultipliersRestricted in order to satisfy:
with a positive
If left-side violated, let If right side violate, let
μμ 11
<<+
ki
ki
uu
μ=newiu
μ/1=newiu
[3] Kocvara & Stingl
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
gki
ki mippiii ,...,1,)( 1 =<+
Update of Penalty ParameterNo update during first 3 iterationsAfterwards, updated by a constant factor dependent on initial penalty parameterPenalty update is stopped if peps (10-6) is reached
[3] Kocvara & Stingl
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm
Choice of Penalty FunctionMost efficient penalty function for convex NLP is the quadratic-logarithmic function:
[4] Ben-Tal & Zibulevsky
holdpropertiesthatsocrtcctc
andrwherertctctct
i
g
6,...,1)log(
)1,1()(
654
322
21
1
=<+−
−∈≥++=ϕ
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm
Overall Stopping Criteria
[3] Kocvara & Stingl
7
1
10
)(1
)()(
)(1
),,()(
−
−
=
<+
−<
+
−
ε
εε
where
xf
xfxfor
xf
puxFxfk
kk
k
kkk
PENNON (PENalty method for NONlinear& semidefinite programming)
Assumptions / WarningsMore tuning for nonconvex problems is still requiredSlower at solving linear SDP problems since algorithm is generalized
PENNON (PENalty method for NONlinear& semidefinite programming)
ReferencesKocvara, Michal & Michael Stingl. PENNON: A Code for Convex and Semidefinite Programming. Optimization Methods and Software, 8(3):317-333, 2003.Kocvara, Michal & Michael Stingl. PENNON-AMPL User’s Guide. www.penopt.com . August 2003.Ben-Tal, Aharon & Michael Zibulevsky. Penalty/Barrier Multiplier Methods for Convex Programming Problems. Siam J. Optim., 7(2):347-366, 1997.Pennon Homepage. www.penopt.com/pennon.html Available online January 2007.