Upload
remington-ronnie
View
42
Download
2
Embed Size (px)
DESCRIPTION
A matrix generation approach for eigenvalue optimization. Mohammad R. Oskoorouchi California State University San Marcos San Marcos, CA, USA. International Conference on Continuous Optimization Rensselaer Polytechnic Institute August 2 – 4, 2004. Outline:. Eigenvalue optimization - PowerPoint PPT Presentation
Citation preview
A matrix generation approach for eigenvalue optimization
Mohammad R. Oskoorouchi
California State University San Marcos
San Marcos, CA, USA
International Conference on Continuous Optimization
Rensselaer Polytechnic Institute
August 2 – 4, 2004
Outline:Outline:
• Eigenvalue optimization
• From optimization to feasibility
• Weighted analytic center
• Recovering feasibility
• Adding a p-dimensional matrix
• Adding a column
• Computational experience.
• Future extensions
Consider the following optimization problem:
max. .
min ( )s t
T
F y
G y h
l y u
0 1
where
( ) ,
are linearly independend
m
i ii
ni
F y F y F
F
S
Eigenvalue optimization:
max 0( ) ( )i if y F y F
Maximum eigenvalue function:
• Continuous
• Convex
• nondifferentiable
• nonpolyhedral cone
• Cannot be written as the point-wise maximum of finite number of convex smooth function
(if are linearly independent)iF
The objective function can be cast as a semidefinite program:
Now let be a feasible point and the maximum eigenvalue of has multiplicity , and be a matrix whose columns form a basis for the eigenspace for the maximum eigenvalue.
By restricting to a subcone generated at the query point, we obtain a lower bound for
( ) max{ ( ) : t ( ) 1, 0}f y F y U r U U
( ) f y
( )F yp n pQ
U
y
max
( ) max{ ( ) : t ( ) 1, 0}
( )
T
T
f y F y QUQ r U U
Q F y Q
L
b
et
e the feasible re
: , an
gi n.
d
o
m Ty G y h l y u F
where, and be upper and lower bounds o
,
n the optimal
objective
: , ( )
Define
value.
D y z z f y z
F
*
Note that
min ( )f f f y
( )f y
( ) max{ : 1,..., }Ti if y a y b i m
( ) max{ : 1,2}Ti if y a y b i
( )f y
min ( ), where is convex nondifferentiabley R
f y f
( )f y
( )f y
, ( ):mD y z zf y ¡ ¡
, : , ( )D y z z f y z F
max( ) ( )Tf y z Q F y Q z
( )TQ F y Q zI
Let us study the inequality more closely:
( )f y z
or
01
mT T
i ii
y Q FQ zI Q F Q
0T
i iQ F y F Q zI
11: , ,m T T
D my y C A y c y A
1
1
1 0
, , 1,..., ,
,
mT T
i i i ii
Tm
y y A A Q FQ i m
A I C Q F Q
A
01
mT T
i ii
y Q FQ zI Q F Q
is represented by where
T y CA
D
• is a compact convex set with nonempty interior, composed of linear and semidefinite inequalities.
• contains the optimal solution set
• is referred to as the set of localization
11: , ,m T T
D my y C A y c y A
Weighted Analytic Center
The weighted analytic is defined as the maximizer of the weighted dual potential function:
. .
1
max ( , , ) log det log log
0, 0, 0
Ds t
T
T
m
S s S s
y S C
A y s c
y
S s
f
A
The primal formulation
The weighted analytic center can be alternatively derived by the weighted primal potential function:
10, 0, 0 : 0P mX x X Ax e A
, , log det
log log
P
T
X x C X X
c x x
. .max , ,
, ,
Ps t
P
X x
X x
KKT optimality conditions
From the KKT optimality conditions 0, 0, 0
are optimal if and only if there exist 0, 0, and 0
such that
S s
X x
f
f
1
1 0
1
T
T
m
m
y S C
A y s c
y
X Ax e
XS I
xs e
A
A
Approximate analytic centers
We call , , , , , an approximate analytic center ifX x S s
1
1
2 2 2
0
1 1
T
T
m
m
y S C
A y s c
y
X Ax e
XS I xs e
A
A
The primal directions:Let a strictly primal feasible point is available.
We employ Newton method to compute the primal
directions to obtain an approximate analytic center.
2
2
x lp
dX X XSX
d x X s
d
2 21 1
2 21
where
T T TP P lp m m
P P lp m
G y g
G AX A e e
g C AX c e
A A
A
Projection to the primal null space:
Feasible directions , , should satisfyxdX d and d
2
2
T
Tx x lp
dX dX X q X
d d X A q
d d q
A
1 0x mdX Ad d e A
This condition may not satisfy due to the computational round-off error. We therefore project the primal directions back to the primal null space.
11 1x mq G dX Ad d e
A
Lemma
.5 .5 1
max
Let ( ) , ( ) , ,
where , , are primal directions computed
by the primal algorithm. Let be the eigenvalues of
and .
Then ( ) 0, ( ) 0, 0, for any 0,
i
x
x
i
i i x
X X dX x x d d
dX d and d
s
X dX X x d
X x
where
max 1,
1
min , ,i j i j d
Recovering feasibility:
So far, we reformulated the eigenvalue optimization problem into a convex feasibility problem, and derived a query point as an approximate analytic center of the set of localization:
The eigenvalue function is evaluated at this point and there are two cases:
10, 0, 0 : 0P mX x X Ax e A
11: , ,m T T
D my y C A y c y A
max 0( ) i if y F y F
Adding a p-dimensional matrix:
11: , ,
min , ( )
m TD D my y D y
f y
B
If f is not differentiable at the query point y, an oracle returns a subgradient of f at this point.
The subgradient is in the form of semidefinite inequality and will be used to update the set of localization. Dual set of localization will be updated via
01
( ) ( )m
k T k k T ki i
i
y Q FQ zI Q F Q
( )f yIllustration:
( )f yz
{ , }(: )mD y z y zzf ¡ ¡
Primal set of localization:
10, 0, 0 : 0P mX x X Ax e A
Due to computational difficulties with the deep cuts, in practice we use the primal set to recover centrality.
We now need a strictly feasible point for the updated primal set of localization.
10, 0, 0 : 0P m
Xx X T Ax e
T
A B
Primal updating direction:
1
2 221 1 1
max log det
.
0
1
0
x m
lp x
T
s t
dX T Ad d e
X dX X d d
T
A B
Implementing Newton method, one has
0arg min log det
2T
pT trT T T
V
1Twhere G V B B
Primal updating direction:
The objective function of this problem is composed of a quadratic term and a self-concordant function. Therefore, Newton method is suitable to solve this problem:
0arg min ( ) log det
2T
pT F T trT T T
V
1 1 1
( ) ( )
21
2
F T dT F T
pptr dT T tr dT dT
trT dT tr T dT T dT
V V
Primal updating direction:
The Newton step dT is obtained by setting the gradient of F(T+dT) with respect to dT to zero:
0pT T T pT dT T T dT V V
0pT T T T dT V
Adding a column:
10, 0, 0 : 0P mX x X Ax bx e A
1
2 221 1 1
max log
.
0
1
0
x m
lp x
x
s t
dX Ad bx d e
X dX X d d
x
A
Size
m
Dimension
n
Density
%
Columns Matrices CPU time
mm:ss
50
50
5
50
100
97
108
8
23
22
0
00:24
00:31
00:00
300
5
50
100
76
87
50
34
04:30
05:56
500
5
50
100
99
66
75
42
54
53
12:52
16:31
11:33
200
50
5
50
100
409
401
22
72
76
1
12:45
10:22
00:06
200
5
50
100
326
323
184
106
108
55
36:14
43:22
10:04
300
5
50
100
283
294
317
119
128
121
62:49
83:05
49:12
Size
m
Dimension
n
Density
%
Columns Matrices CPU time
mm:ss
500
10 5
100
88
23
0
0
02:28
00:39
20 5
100
203
30
0
0
06:30
00:51
50 5
100
1418
59
90
2
72:05
01:59
800
10 5
100
68
16
8
1
07:50
01:36
20 5
100
297
52
1
0
31:23
04:33
50 5
100
1311
73
99
0
438:23
05:08
1000
10 5
100
57
27
12
2
12:16
03:07
20 5
100
231
56
13
0
45:19
09:24
50 5
100
1187
105
73
0
382:10
17:41