Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Semidefinite ProgrammingBasics and SOS
Fernando Mário de Oliveira Filho
Campos do Jordão, 20 November 2013
Available at: www.ime.usp.br/~fmario under talks
Conic programming
Complexity: depends on the cone!
V is a real vector space
Given: c, a1, . . . , am 2 V and b1, . . . , bm 2 R
↵x+ �y 2 C 8 x, y 2 C and ↵,� � 0
max hc, xihai, xi = bi for i = 1, . . . , m
x 2 C
h·, ·i is an inner product
Convex cone: set C ✓ V such that
Examples• V = Rn
and C = Rn+: linear programming
• V = Sn and C = cone of PSD matrices: SDP
A, B 2 Sn
hA,Bi = tr(ATB) =
nX
i,j=1
AijBij = trace inner product
() all eigenvalues are nonnegative
A is PSD (also A ⌫ 0): x
TAx � 0 8 x 2 Rn
() A = QTQ for some Q 2 Rn⇥n
() A =Pn
i=1 �iuiuTi , with �i � 0
Semidefinite programmingGiven: C, A1, . . . , Am 2 Sn and b1, . . . , bm 2 R
max hC,XihAi, Xi = bi for i = 1, . . . , mX ⌫ 0
• Generalizes LP: make all matrices diagonal! • Duality theory: comes from conic programming • Solvable in poly-time: under some conditions (later) • Numerical methods: efficient interior-point methods
✓x z
z y
◆⌫ 0 x, y � 0 and xy � z
2 � 0()
Application: polynomial optimizationIs the polynomial p 2 R[x1, . . . , xn] nonnegative?
Sure, if it is a sum-of-squares: p = q21 + · · ·+ q2k
Exercise: A univariate poly is nonnegative iff it is SOS
Pn,d = nonnegative polys, n variables, degree d
⌃n,d = SOS polys, n variables, degree d
Theorem. ⌃n,d ✓ Pn,d with equality i↵
• bivariate polys (n = 2)
• quadratics (d = 2)
• n = 3, d = 4 David Hilbert 1888
SOS is SDPp 2 R[x1, . . . , xn] degree d
v(x) = vector with all monomials degree d/2
Theorem. p is SOS i↵ p(x) = hQ, v(x)v(x)T i for some Q ⌫ 0.
n = 2, d = 6
v(x) =�1, x, x2
, x
3, y, xy, x
2y, y
2, xy
2, y
3�
0
BBBBBBBBBBBBBB@
1 x x
2x
3y xy x
2y y
2xy
2y
3
x x
2x
3x
4xy x
2y x
3y xy
2x
2y
2xy
3
x
2x
3x
4x
5x
2y x
3y x
4y x
2y
2x
3y
2x
2y
3
x
3x
4x
5x
6x
3y x
4y x
5y x
3y
2x
4y
2x
3y
3
y xy x
2y x
3y y
2xy
2x
2y
2y
3xy
3y
4
xy x
2y x
3y x
4y xy
2x
2y
2x
3y
2xy
3x
2y
3xy
4
x
2y x
3y x
4y x
5y x
2y
2x
3y
2x
4y
2x
2y
3x
3y
3x
2y
4
y
2xy
2x
2y
2x
3y
2y
3xy
3x
2y
3y
4xy
4y
5
xy
2x
2y
2x
3y
2x
4y
2xy
3x
2y
3x
3y
3xy
4x
2y
4xy
5
y
3xy
3x
2y
3x
3y
3y
4xy
4x
2y
4y
5xy
5y
6
1
CCCCCCCCCCCCCCA
A small example
v(x) = (1, x, x2, y, y
2, xy)
p(x, y) = x
4 + 2x3y + x
2y
2 � 6x2y � 6xy2 + 10x2 + 10xy + 9y2 � 30y + 25
Qv(x)v(x)T
0
BBBBBBBB@
� 1 x x
2y y
2xy
1 q11 q12 q13 q14 q15 q16
x q21 q22 q23 q24 q25 q26
x
2q31 q32 q33 q34 q35 q36
y q41 q42 q43 q44 q45 q46
y
2q51 q52 q53 q54 q55 q56
xy q61 q62 q63 q64 q65 q66
1
CCCCCCCCA
0
BBBBBBBB@
� 1 x x
2y y
2xy
1 1 x x
2y y
2xy
x x x
2x
3xy xy
2x
2y
x
2x
2x
3x
4x
2y x
2y
2x
3y
y y xy x
2y y
2y
3xy
2
y
2y
2xy
2x
2y
2y
3y
4xy
3
xy xy x
2y x
3y xy
2xy
3x
2y
2
1
CCCCCCCCA
q11 = 25
q23 + q32 = 0
q25 + q52 + q46 + q64 = �6
Q =X
uiuTi =) p =
X(uT
i v(x))2
Sparse SDPA format: Setup
maximize hF0, Y ihFi, Y i = ci for i = 1, . . . , mY ⌫ 0.
1
2
3 Block 4, a diagonal block
Matrices have a block structure
Sparse SDPA format
maximize hF0, Y ihFi, Y i = ci for i = 1, . . . , mY ⌫ 0.
Text file ‘problem.sdpa’:
Matrix entries: hmatrix number i h block number i h i i h j i h entry i
Right-hand side: c1 c2 · · · cmBlock sizes: s1 s2 · · · sN
Number of constraint matrices: m
Number of blocks: N
Remarks: • Indices always start at 1 • Write the size of a block as a negative number to indicate a diagonal block • Only upper-diagonal entries need to be given!
A feasibility problem
p(x) = x
4 + 6x3 + 15x2 � 10x+ 17
F2 =
0
@0 1 01 0 00 0 0
1
AF1 =
0
@1 0 00 0 00 0 0
1
A
F3 =
0
@0 0 10 1 01 0 0
1
A F4 =
0
@0 0 00 0 10 1 0
1
A F5 =
0
@0 0 00 0 00 0 1
1
A
F0 =
0
@0 0 00 0 00 0 0
1
A
maximize hF0, Y ihFi, Y i = ci for i = 1, . . . , 5
Y ⌫ 0.
Is the polynomial
a sum of squares?
c1 = 17, c2 = �10, c3 = 15, c4 = 6, c5 = 1
v(x) = (1, x, x2)
The SDPA fileF2 =
0
@0 1 01 0 00 0 0
1
AF1 =
0
@1 0 00 0 00 0 0
1
A
F3 =
0
@0 0 10 1 01 0 0
1
A F4 =
0
@0 0 00 0 10 1 0
1
A F5 =
0
@0 0 00 0 00 0 1
1
A
F0 =
0
@0 0 00 0 00 0 0
1
A
c1 = 17, c2 = �10, c3 = 15, c4 = 6, c5 = 1
File ‘sos.sdpa’: 5 ▷ Number of constraint matrices!1 ▷ We have only one block!3 ▷ Which is 3x3!17 -10 15 6 1 ▷ The right-hand side 1 1 1 1 1.0 ▷ Matrix entries!2 1 1 2 1.0 !3 1 1 3 1.0 !3 1 2 2 1.0!4 1 2 3 1.0 !5 1 3 3 1.0
CSDP output
p1(x) = �0.719209404617218x2 � 1.21267812518166x+ 4.12310562561766
p2(x) = 0.482351656352516x2 + 4.41136763929902x
p3(x) = 0.500074706343094x2
p
21 + p
22 + p
23 = x
4 + 6x3 + 15x2 � 10x+ 17
Y =
0
@17.0000000000000 �5.00000000000000 �2.96537634217438
�5.00000000000000 20.9307526843488 3.00000000000000�2.96537634217438 3.00000000000000 1.00000000000000
1
A
0.0 0.0 0.0 0.0 0.0 ▷ Dual solution, only numbers!1 1 1 1 2.886751345948129001e-11 ▷ Dual solution, matrix!1 1 2 2 2.886751345948129001e-11 !1 1 3 3 2.886751345948129001e-11 !2 1 1 1 1.700000000000000000e+01 ▷ Primal solution, matrix!2 1 1 2 -5.000000000000000000e+00 !2 1 1 3 -2.965376342174380397e+00 !2 1 2 2 2.093075268434876079e+01 !2 1 2 3 3.000000000000000000e+00 !2 1 3 3 1.000000000000000000e+00
Other uses of SOS
minimum of p is � () p� � is nonnegative
() p is SOS
Formulate as SDP: max{� : p� � is SOS }
p 2 R[x] is nonnegative in [a, b] () 9 q1, q2 SOS s.t.
p = (x� a)(b� x)q1 + q2
Finding the minimum of a polynomial
Nonnegativity in a domain
Duality in conic programmingC ✓ V a cone
Dual cone: C
⇤= {x 2 V : hx, yi � 0 for all y 2 C }
max hc, xihai, xi = bi for i = 1, . . . , m
x 2 C
Primal standard form:
min y1b1 + · · ·+ ymbmy1a1 + · · ·+ ymam � c 2 C⇤
Dual standard form:
strictly feasible: x in interior of C
strictly feasible: y1a1 + · · ·+ ymam � c in interior of C
Duality theorems
x is primal feasible, y is dual feasible =) hc, xi y
Tb
dual is bounded and strictly feasibleIf:primal attains optimal and values coincideThen:
primal is bounded and strictly feasibleIf:dual attains optimal and values coincideThen:
Strong duality:
Weak duality:
Applied to SDP…
Exercise: PSD cone is self-dual
max hC,XihAi, Xi = bi for i = 1, . . . , mX ⌫ 0
Primal standard form:
min y1b1 + · · ·+ ymbmy1A1 + · · ·+ ymAm � C ⌫ 0
Dual standard form:
strictly feasible: X is positive definite
strictly feasible: y1A1 + · · ·+ ymAm � C is positive definite
SDP and duality gone wrong
Optimal not attained:
min �✓↵ 11 �
◆⌫ 0
max �X12 �X21
X11 = 0, X22 = 1
max �X11 �X22
X11 = 0
2X13 +X22 = 1
X ⌫ 0
min y20
@y1 + 1 0 y2
0 y2 + 1 0y2 0 0
1
A ⌫ 0
Positive duality gap:
Optimal: -1 Optimal: 0
Summary and coming soon…
• Generalizes LP • Fits framework of conic programming • Solvable in poly time (under some “mild” assumptions) • Has a duality theory (inherits from conic programming) • One useful application: SOS polynomials
• Approximation algorithms • Maximum-cut problem • Grothendieck inequalities and applications
Semidefinite programming…
Tomorrow: