Upload
leanh
View
247
Download
0
Embed Size (px)
Citation preview
Chapter 4 Convex Optimization Problems
Shupeng Gui
Computer Science, UR
October 16, 2015
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 1 / 58
Outline
1 Optimization problems
2 Convex optimization problem
3 Quasiconvex optimization
4 Linear program(LP)
5 Quadratic program(QP)
6 Generalized inequality constraints
7 Vector optimization
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 2 / 58
Basic terminology
Optimization problem in standard form
min f0(x)
s.t. fi(x) ≤ 0, i = 1, 2, ...,m
hi(x) = 0, i = 1, 2, ..., p
x ∈ Rn is the optimization variable
f0 : Rn → R is the objective or cost function
fi : Rn → R, i = 1, 2, ...,m, are the inequality constraint functions
hi : Rn → R are the equality constraint functions
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 3 / 58
Optimal value
optimal value
p∗ = inf{f0(x)|fi(x) ≤ 0, i = 1, 2, ...,m, hi(x) = 0, i = 1, ..., p}
p∗ =∞ if problem is infeasible (no x satisfies the constraints)
p∗ = −∞ if problem is unbounded below
inf means the infimum.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 4 / 58
Optimal value
Example What is the optimal value here?
min f0(x) = (x− x0)2
s.t. x ≥ −6
x ≤ 10
Figure 1: f0 which mentioned above,x0 = 4
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 5 / 58
Optimal value
Example What is the optimal value here?
min f0(x) =1
(x− x0)2
s.t. x = x0
Figure 2: f0 which mentioned above,x0 = 4
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 6 / 58
Optimal Value
Example What is the optimal value here?
min f0(x) = − log(x)
s.t. x > 0
Figure 3: f0 which mentioned above
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 7 / 58
Optimal and locally optimal points
x is feasible if x ∈ dom f0 and it satisfies the constraintsa feasible x is optimal if f0(x) = p∗; Xopt is the set of optimal points
Xopt = {x|fi(x) ≤ 0, i = 1, ...,m, hi(x) = 0, i = 1, ...p, f0(x) = p∗}
x is locally optimal if there is an R > 0 such that x is optimal for
min f0(z)
s.t. fi(z) ≤ 0, i = 1, ...,m
hi(z) = 0, i = 1, ..., p
‖z − x‖2 ≤ R
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 8 / 58
Optimal and locally optimal points
Figure 4: Local optimal points
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 9 / 58
Optimal and locally optimal points
example (with n=1, m=p=0)What is the optimal value and how about the optimal points?
f0(x) =1
x, domf0 = R++
Figure 5: f0(x) = 1/xShupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 10 / 58
Optimal and locally optimal points
example (with n=1, m=p=0)What is the optimal value and how about the optimal points?
f0(x) = − log(x), domf0 = R++
Figure 6: f0(x) = − log(x)
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 11 / 58
Optimal and locally optimal points
example (with n=1, m=p=0)What is the optimal value and how about the optimal points?
f0(x) = x log(x), domf0 = R++
Figure 7: f0(x) = x log(x)
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 12 / 58
Optimal and locally optimal points
example (with n=1, m=p=0)What is the optimal value and how about the optimal points?
f0(x) = x log(x), domf0 = R++
Figure 8: f0(x) = x log(x)
p∗ = −1/e, x = 1/e is optimal.Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 13 / 58
Optimal and locally optimal points
example (with n=1, m=p=0)What is the optimal value and how about the optimal points?
f0(x) = x3 − 3x
Figure 9: f0(x) = x3 − 3x
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 14 / 58
Implicit constraints
the standard form optimization problem has an implicit constraint
x ∈ D =
m⋂i=0
domfi ∩p⋂
i=1
domhi
Here include f0.
we call D the domain of the problem.
the constraints fi(x) ≤ 0, hi(x) = 0 are explicit constraints.
a problem is unconstrained if it has no explicit constraints(m=p=0);
Example
min f0(x) = −k∑
i=1
log(bi − aTi x)
is an unconstrained problem with implicit constraints aTi x < bi,
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 15 / 58
Feasibility problem
find x
s.t. fi(x) ≤ 0, i = 1, ...,m
hi(x) = 0, i = 1, ..., p
can be considered a special case of the general problem with f0(x) = 0
min 0
s.t. fi(x) ≤ 0, i = 1, ...,m
hi(x) = 0, i = 1, ..., p
p∗ = 0 if constraints are feasible; any feasible x is optimal
p∗ =∞ if constraints are infeasible
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 16 / 58
Convex optimization problem
standard form convex optimization problem
min f0(x)
s.t. fi(x) ≤ 0, i = 1, 2, ...,m
aTi x = bi, i = 1, 2, ..., p
f0, f1, ..., fm are convex; equality constraints are affine
problem is quasiconvex if f0 is quasiconvex(and f0, f1, ..., fmconvex)
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 17 / 58
standard form convex optimization problem
Sometimes written as
min f0(x)
s.t. fi(x) ≤ 0, i = 1, 2, ...,m
Ax = b
important property: feasible set of a convex optimization problem isconvex.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 18 / 58
standard form convex optimization problem
Feasible set of a convex optimization problem is convex.ProofRecall the intersection of (any number of) convex sets is convex–Chapter 2It is the intersection of the domain of the problem
D =
m⋂i=0
dom fi
which is a convex set, with m (convex) sublevel sets {x|fi(x) ≤ 0} andp hyperplanes {x|aTi x = bi}
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 19 / 58
standard form convex optimization problem
Example
min f0(x) = x21 + x22
s.t. f1(x) = x1/(1 + x22) ≤ 0
hi(x) = (x1 + x2)2 = 0
f0 is convex; feasible set {(x1, x2)|x1 = −x2 ≤ 0} is convex
not a convex problem (according to our definition): f1 is notconvex, hi is not affine.
equivalent (but not identical) to the convex problem
min x21 + x22
s.t. x1 ≤ 0
x1 + x2 = 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 20 / 58
standard form convex optimization problem
Figure 10: f0(x) = x21 + x22
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 21 / 58
Local and global optima
Any locally optimal point of a convex problem is (globally) optimalProof : suppose x is locally optimal, but there exists a feasible y withf0(y) < f0(x)x locally optimal means there is an R > 0 such that
z feasible, ‖z − x‖2 ≤ R⇒ f0(z) ≥ f0(x)
Consider z = θy + (1− θ)x with θ = R/(2‖y − x‖2)‖y − x‖2 > R, so 0 < θ < 1/2
z is a convex combination of two feasible points, hence also feasible
‖z − x‖2 = R/2 and
f0(z) ≤ θf0(y) + (1− θ)f0(x) < f0(x)
which contradicts our assumption that x is locally optimal
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 22 / 58
Local and global optima
Figure 11: f0(x) = x2
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 23 / 58
Optimality criterion for differentiable
Suppose that f0(x) is differentiable in a convex optimization problem,∀x, y ∈ dom f0
f0(y) ≥ f0(x) +∇f0(x)T (y − x) (first order condition)
Feasible set Xopt = {x|fi(x) ≤ 0, i = 1, ...,m, aTi x = bi, i = 1, ..., p}Criterion x is optimal if and only if it is feasible and
∇f0(x)T (y − x) ≥ 0 for all feasible y
If nonzero, ∇f0(x) defines a supporting hyperplane to feasible set X atx.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 24 / 58
Optimality criterion for differentiable
Figure 12: Supporting Hyperplane
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 25 / 58
Optimality criterion for differentiable
Proof
1 Suppose x ∈ X and satisfies ∇f0(x)T (y−x) ≥ 0 for all feasible y.Then if y ∈ X, we have f0(y) ≥ f0(x), by the first order condition,f0(y) ≥ f0(x) +∇f0(x)T (y − x). This shows x is an optimal pointfor the problem.
2 Suppose x is optimal, but the condition∇f0(x)T (y − x) ≥ 0 for all feasible y does not hold, i.e., for somey ∈ X we have
∇f0(x)T (y − x) < 0
Consider z(t) = tx+ (1− t)y, t ∈ [0, 1]. Since z(t) is on the linesegment between x and y, and the feasible set is convex. z(t) isfeasible.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 26 / 58
Optimality criterion for differentiable
Claim that for small positive 1− t, we have f0(z(t)) < f0(x), sincewhich x is not optimal.
d
dxf0(z(t))|t=1 = ∇f0(x)T (y − x) < 0
so for small positive 1− t, we have f0(z(t)) < f0(x)
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 27 / 58
Optimality criterion for differentiable
unconstrained problem: x is optimal if and only if
x ∈ dom f0, ∇f0(x) = 0
ProofSuppose x is optimal, which means that x ∈ dom f0, and for all feasibley we have ∇f0(x)T (y − x) ≥ 0. Since f0 is differentiable, its domain isopen, so all y sufficiently close to x are feasible.Take y = x− t∇f0(x), where x ∈ R is a parameter. For t small andpositive, y is feasible, and so
∇f0(x)T (y − x) = ∇f0(x)T (−t∇f0(x)) = −t‖∇f0(x)‖22 ≥ 0
Therefore, x is optimal if and only if x ∈ dom f0, ∇f0(x) = 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 28 / 58
Optimality criterion for differentiable
equality constrained problem:
min f0(x) s.t.Ax = b
x is optimal if and only if there exists a v such that
x ∈ dom f0, Ax = b, ∇f0(x) +AT v = 0
Proof∀y, Ay = b. Since x is feasible, every feasible y has the form y = x+ vfor some v ∈ N (A). Then ∇f0(x)T v ≥ 0, ∀v ∈ N (A).If a linear function is nonnegative on a subspace, then it must be zeroon the subspace, so it follows that ∇f0(x)T v = 0,∀v ∈ N (A). In otherwords, ∇f0(x)⊥N (A). Using the fact N (A)⊥ = R(AT ),∇f0(x) ∈ R(AT ), there exists a v ∈ Rp such that
∇f0(x) +AT v = 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 29 / 58
Optimality criterion for differentiable
minimization over nonnegative orthant
min f0(x) s.t. x � 0
x is optimal if and only if
x ∈ dom f0, x � 0, ∇f0(x) � 0, xi(∇f0(x))i = 0, i = 1, ..., n
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 30 / 58
Equivalent convex problems
two problems are (informally) equivalent if the solution of one isreadily obtained from the solution of the other, and vice-versaSome common transformations that preserve convexity.eliminating equality constraints
min f0(x)
s.t. fi(x) ≤ 0, i = 1, 2, ...,m
Ax = b
is equivalent to
minz
f0(Fz + x0)
s.t. fi(Fz + x0) ≤ 0, i = 1, 2, ...,m
where F and x0 are such that
Ax = b⇔ x = Fz + x0 for some z
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 31 / 58
Equivalent convex problems
introducing equality constraints
min f0(A0x+ b0)
s.t. fi(Aix+ bi) ≤ 0, i = 1, 2, ...,m
is equivalent to
minx,yi
f0(y0)
s.t. fi(yi) ≤ 0, i = 1, 2, ...,m
yi = Aix+ bi, i = 0, 1, ...,m
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 32 / 58
Equivalent convex problems
introducing slack variables for linear inequalities
min f0(x)
s.t. aTi x ≤ bi, i = 1, 2, ...,m
is equivalent to
minx,s
f0(x)
s.t. aTi x+ si = bi, i = 1, 2, ...,m
si ≥ 0, i = 1, ...,m
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 33 / 58
Equivalent convex problems
epigraph form
minx,t
t
s.t. f0(x)− t ≤ 0
fi(x) ≤ 0, i = 1, 2, ...,m
Ax = b
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 34 / 58
Equivalent convex problems
minimizing over some variables
minx,t
f0(x1, x2)
s.t. fi(x1) ≤ 0, i = 1, 2, ...,m
is equivalent to
minx,t
f ′0(x1)
s.t. fi(x1) ≤ 0, i = 1, 2, ...,m
where f ′0(x1) = infx2 f0(x1, x2)
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 35 / 58
Quasiconvex optimization
minx,t
f0(x)
s.t. fi(x) ≤ 0, i = 1, 2, ...,m
Ax = b
with f0 : Rn → R quasiconvex, f1, ..., fm convexThis problem can have locally optimal points that are not (globally)optimal
Figure 13: Quasiconvex optimizationShupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 36 / 58
Quasiconvex optimization
convex representation of sublevel sets of f0 if f0 is quasiconvex,there exists a family of functions Φt such that:
Φt(x) is convex in x for fixed t
t-sublevel set of f0 is 0-sublevel set of Φt, i.e.,
f0(x) ≤ t⇐ Φt(x) ≤ 0
example
f0(x) =p(x)
q(x)
with p convex, q concave, and p(x) ≥ 0, q(x) > 0 on dom f0Take Φt(x) = p(x)− tq(x):
for t ≥ 0, Φt convex in x
p(x)/q(x) ≤ t if and only if Φt(x) ≤ 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 37 / 58
Linear program(LP)
min cTx+ d
s.t. Gx � hAx = b
convex problem with affine objective and constraint functions
feasible set is a polyhedron
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 38 / 58
Linear program(LP)
Formulate the LPs to standard form.
min cTx+ d
s.t. Gx+ s = h
Ax = b
s � 0
Hints: x+, positive part of x; x−, negative part of x. x = x+ − x−.
min cTx+ − cTx− + d
s.t. Gx+ −Gx− + s = h
Ax+ −Ax− = b
s � 0, x+ � 0, x− � 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 39 / 58
Linear-fractional program
min f0(x)
s.t. Gx � hAx = b
linear-fractional program
f0(x) =cTx+ d
eT + f,dom f0(x) = {x|eTx+ f > 0}
a quasiconvex optimization problem; can be solved by visectionalso equivalent to the LP (variables y,z)
min cT y + dz
s.t. Gy � hzAy = bz
eT y + fz = 1, z ≥ 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 40 / 58
Generalized linear-fractional program
f0(x) = maxi=1,...,r
cTi x+ di
eTi x+ fi,domf0(x) = {x|eTi x+ fi, i = 1, ..., r}
a quasiconvex optimization problem; can be solved by bisection.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 41 / 58
Quadratic program(QP)
min (1/2)xTPx+ qTx+ r
s.t. Gx � hAx = b
P ∈ Sn+, so objective is convex quadratic
minimize a convex quadratic function over a polyhedron
Example: least-squares
min ‖Ax− b‖22
min xTATAx− 2bTAx+ bT b
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 42 / 58
Quadratically constained quadratic program(QCQP)
min (1/2)xTP0x+ qT0 x+ r0
s.t. (1/2)xTPix+ qTi x+ ri ≤ 0, i = 1, ...,m
Ax = b
Pi ∈ Sn+; objective and constraints are convex quadratic
if P1, ..., Pm ∈ Sn++, feasible region is intersection of m ellipsoids
and an affine set.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 43 / 58
Second-order cone programming
min fTx
s.t. ‖Aix+ bi‖2 ≤ cTi x+ di, i = 1, ...,m
Fx = g
Ai ∈ Rni×n, F ∈ Rp×n
inequalities are called second-order cone (SOC) constraints:
(Aix+ bi, cTi x+ di) ∈ second− order cone inRni+1
for Ai = 0, reduces to an LP; if ci = 0, reduces to a QCQP
more general than QCQP and LP
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 44 / 58
Robust linear programming
the parameters in optimization problems are often uncertain, e.g., in anLP
min cTx
s.t. aTi x ≤ bi, i = 1, ...,m
there can be uncertainty in c, ai, bitwo common approaches to handling uncertainty(in ai, for simplicity)
deterministic model: constraints must hold for all ai ∈ ε〉min cTx
s.t. aTi x ≤ bi, ∀ai ∈ εi, i = 1, ...,m
stochastic model: ai is random variable; constraints must holdwith probability η
min cTx
s.t. prob(aTi x ≤ bi) ≥ η, i = 1, ...,m
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 45 / 58
deterministic approach via SOCP
choose an ellipsoid as εi:
εi = {ai + Piu|‖u‖2 ≤ 1} (ai ∈ Rn, Pi ∈ Rn×n)
center is ai, semi-axes determined by singular values/vectors of Pi
robust LP
min cTx
s.t. aTi x ≤ bi, ∀ai ∈ εi, i = 1, ...,m
is equivalent to the SOCP
min cTx
s.t. aTi x+ ‖P Ti x‖2 ≤ bi, i = 1, ...,m
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 46 / 58
Stochastic approach via SOCP
assume ai is Gaussian with mean ai, covariance Σi(ai ∼ N (ai,Σi))
aTi x is Gaussian r.v. with mean aTi x, variance xTΣix; hence
prob(aTi x ≤ bi) = Φ( bi − aTi x‖Σ1/2
i x‖2
)where Φ(x) = (1/
√2π)
∫ x−∞ e
−t2/2dt is CDF of N (0, 1)
robust LP
min cTx
s.t. prob(aTi x ≤ bi), i = 1, ...,m
with η ≥ 1/2, is equivalent to the SOCP
min cTx
s.t. aTi x+ Φ−1(η)‖Σ1/2i x‖2 ≤ bi, i = 1, ...,m
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 47 / 58
Geometric programming
monomial function
f(x) = cxa11 xa22 ...x
ann , dom f = Rn
++
with c > 0; exponent ai can be any real number posynomialfunction: sum of monomials
f(x) =K∑k=1
ckxa1k1 xa2k2 ...xank
n , dom f = Rn++, dom f = Rn
++
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 48 / 58
Geometric programming
geometric program (GP)
min f0(x)
s.t. fi(x) ≤ 1, i = 1, ...,m
hi(x) = 1, i = 1, ..., p
with fi posynomial, hi monomial
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 49 / 58
Geometric program in convex form
change variables t oyi = log xi, and take logarithm of cost, constraints
monomial f(x) = cxa11 xa22 ...x
ann transforms to
log f(ey1 , ..., eyn) = aT y + b (b = log c)
posynomial f(x) =∑K
k=1 ckxa1k1 xa2k2 ...xank
n transforms to
log f(ey1 , ..., eyn) = log( K∑
k=1
eaTk y+bk
)(bk = log ck)
geometric program transforms to convex problem
min log(K∑k=1
exp(aT0ky + b0k))
s.t. log(
K∑k=1
exp(aTiky + bik)) ≤ 0, i = 1, ...,m
Gy + d = 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 50 / 58
Generalized inequality constraints
convex problem with generalized inequality constraints
min f0(x)
s.t. fi(x) �Ki 0, i = 1, ...,m
Ax = b
f0 : Rn → R convex;fi : Rn → Rki Ki-convex w.r.t proper cone Ki
same properties as standard convex problem (convex feasible set,local optimum is global, etc.)
conic form problem: special case with affine objective andconstraints
min cTx
s.t. Fx+ g �K 0
Ax = b
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 51 / 58
Semidefinite program (SDP)
min cTx
s.t. x1F1 + x2F2 + ...+ xnFn +G � 0
Ax = b
with Fi, G ∈ Sk
inequality constraint is called linear matrix inequality (LMI)
includes problems with multiple LMI constraints: for example,
x1F1 + ...+ xnFn + G � 0, x1F1 + ...+ xnFn + G � 0
is equivalent to single LMI
x1
[F1 0
0 F1
]+x2
[F2 0
0 F2
]+...+xn
[Fn 0
0 Fn
]+
[G 0
0 G
]� 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 52 / 58
LP and SOCP as SDP
LP and equivalent SDPLP:
min cTx
s.t. Ax � b
SDP:
min cTx
s.t. diag(Ax− b) � 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 53 / 58
LP and SOCP as SDP
SOCP and equivalent SDPSOCP:
min fTx
s.t. ‖Aix+ bi‖2 ≤ cTi x+ di, i = 1, ...,m
SDP:
min fTx
s.t.
[(cTi x+ di)I Aix+ bi(Aix+ bi)
T cTi x+ di
]� 0, i = 1, ...,m
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 54 / 58
Vector optimization
general vector optimization problem
min(w.r.t.K) f0(x)
s.t. fi(x) ≤ 0, i = 1, ...,m
hi(x) = 0, i = 1, ..., p
vector objective f0 : Rn → Rq, minimized w.r.t. proper cone K ∈ Rq
convex vector optimization problem
min(w.r.t.K) f0(x)
s.t. fi(x) ≤ 0, i = 1, ...,m
hi(x) = 0, i = 1, ..., p
with f0 K-convex, f1, ..., fm convex.
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 55 / 58
Optimal and Pareto optimal points
set of achievable objective values
O = {f0(x)|x feasible}
feasible x is optimal if f0(x) is the minimum value of Ofeasible x is Pareto optimal if f0(x) is a minimal value of O
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 56 / 58
Multicriterion optimization
vector optimization problem with K = Rq+
f0(x) = (F1(x), ..., Fq(x))
q different objectives Fi: roughly speaking we want all Fi’s to besmall
feasible x∗ is optimal if
y feasible⇒ f0(x∗) � f0(y)
if there exists an optimal point, the objectives are noncompeting
feasible xpo is Pareto optimal if
y feasible, f0(y) �⇒ f0(xpo) = f0(y)
if there are multiple Pareto optimal values, there is a trade-offbetween the objectives
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 57 / 58
Scalarization
to find Pareto optimal points: choose λ �K∗ 0 and solve scalar problem
min λT f0(x)
s.t. fi(x) ≤ 0, i = 1, ...,m
hi(x) = 0, i = 1, ..., p
if x is optimal for scalar problem, then it is Pareto-optimal for vectoroptimization problem.For convex vector optimization problems, can find (almost) all Paretooptimal points by varying λ �K∗ 0
Shupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 58 / 58