Cost-Cumulants and Risk Sensitive Control

8/14/2019 Cost-Cumulants and Risk Sensitive Control

http://slidepdf.com/reader/full/cost-cumulants-and-risk-sensitive-control 1/8

5C o s t - C u m u l a n t s a n d

Risk-Sens i t ive Contro l

rtment of Electrical Engineering,

University of N orth Dakota,Grand Forks,

North Dakota, USA

5 .1 I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1061

5 .2 L i n e a r - Q u a d r a t i c - G a u s s i a n C o n t r o l . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1061

5 .3 C o s t - C u m u l a n t C o n t r o l . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1062

5 .3 .1 M i n i m a l C o s t V a r i a n c e C o n t r o l

5 .4 R i s k - S e n s it i v e C o n t r o l . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1063

5 .5 R e l a t i o n s h i p B e t w e e n R i s k - S e n si t iv e a n d C o s t - C u m u l a n t C o n t r o l . . . . . . . . . . .. . . . . 1064

5 .6 A p p l i c a t i o n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1065

5 . 6 .1 R i s k - S e n s i ti v e C o n t r o l A p p l i e d t o S a te l l it e A t t i t u d e M a n e u v e r

5 . 6. 2 M C V C o n t r o l A p p l i e d t o S e i s m ic P r o t e c t i o n o f S t r u c t u re s

5 .7 C o n c l u s i o n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067

R e f e r e n c e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1068

5 . 1 I n t r o d u c t i o n

o s t - c u m u l a n t c o n t r o l , a l s o k n o w n a s s t a ti s ti c a l c o n t r o l , is a n

c o s t c u m u l a n t s . R i s k - s e n s it i v e c o n t r o l i s an opt i -

e t h o d t h a t m i n i m i z e s t h e e x p o n e n t ia l o f t h e

e r a b l e s u m o f al l t h e c o s t c u m u l a n t s.

O p t i m a l c o n t r o l t h e o r y d e a l s w i t h t h e o p t i m i z a t i o n , e i t h e r

i z a t i o n o r m a x i m i z a t i o n , o f a g i v e n c o st c r it e ri o n .

a r e d i s c u s s e d i n t e r m s o f

t s . F i g u r e 5 .1 p r e s e n t s a n o v e r v i e w o f t h e o p t i m a l

t h e r e l a ti o n s h ip s a m o n g d i f fe r e n t o p t i m a l c o n t r o l

2 L i n e a r - Q u a d r a t i c -G a u s s ia n C o n t r o l

h e l i n e a r q u a d r a t i c G a u s s i a n ( LQ G ) c o n t r o l m e t h o d o p t i -

e a n , w h i c h is t h e f i r st c u m u l a n t , o f a q u a d r a t i c c o s t

T y p i ca l s y st e m d y n a m i c s f o r L Q G c o n t r o l a r e g i v e n b y t h e

© 2005 by AcademicPress.

reserved.

d x ( t ) = A x ( t ) d t + B k ( t , x ) d t + E ( t ) d w ( t ) .

y ( t ) d t = C x ( t ) d t + d r ( t ) .(5.1)

H e r e w ( t ) a n d v( t ) a r e v e c t o r B r o w n i a n m o t i o n s . T h e m e a n i n g

o f a B r o w n i a n m o t i o n , s u c h a s w ( t ) , c a n b e g i v e n d i r e c t ly o r i n

t e r m s o f i t s d i f fe r e n t ia l d w ( t ) . In the l a t t e r case , the d w ( t ) is a

G a u s s i a n r a n d o m p r o c e s s w i t h z e r o m e a n , c o v a r i a n c e m a t r i x

W d r , a n d i n d e p e n d e n t i n c r e m e n t s . A s i m i l a r d e s c r i p t i o n a p -

pl ies to d r ( t ) , w i t h c o v a r i a n c e V d t . I t i s a s s u m e d t h a t d w ( t )

a n d d r ( t ) a r e i n d e p e n d e n t . T h e m a t r i c e s A , B , C , a n d E a r e o f

c o m p a t i b l e siz e. I t s h o u l d b e r e m a r k e d t h a t t h e f o r m a l i s m o f

e q u a t i o n 5 .1 i s t h a t o f a s t o c h a s t ic d i f f e r e n t ia l e q u a t i o n . I n t u i -

t iv e ly , o n e t h i n k s o f d i v i d i n g b o t h s i d es o f t h i s e q u a t i o n b y d t

t o o b t a i n t h e m o r e c o l l o q u i a l f o r m . B u t t h e f o r m a l d e r i v a t i v e

o f a B r o w n i a n m o t i o n w h i c h is k n o w n a s w h i t e n o i s e - - i s n o ta w e l l - d e f i n e d r a n d o m p r o c e s s , a n d t h i s m o t i v a t e s a n a l t e r n a t e

w a y o f t h i n k in g .

T h e q u a d r a t i c c o s t c r i te r i o n i s g i v e n b y :

l ( k ) = [ ( x ' Q x + k ' R k ) d t . (5.2)

T h e w e i g h t i n g m a t r i x Q i s a s y m m e t r i c a n d p o s i t i v e s e m i d e f i -

n i t e m a t r i x , a n d R i s a s y m m e t r i c a n d p o s i t i v e d e fi n i te m a t r i x .

1061



1062 C h a n g - H e e W o n

fStochastic control

Cost-cumulant control

1 ( ~ " 7 V ' ~ I 2

LQG

Minimal c o s t ~ U Risk-sensit ivevariance control

H-infinity control ~ G am e heoryN I

Deterministic control

UR E 5.1 Rela tionship Between Various Optim al and Robus t

et a l . , 2000; Jacobson, 1973; W on a nd Sain,

1995; Glover an d Doy le, 1988 ; Whittle, 1990; Rhee and Speyer, 1992;

1973 ; Runolfsson, 1994; Uchida, 1989; Basar and Bernhard,

1991.)

t h e n b e c o m e s a m i n i m i z a t i o n o f t h e

1" = m in E { l ( k ) } . (5 . 3 )k

f u l l- s ta t e f e e d b a c k c o n t r o l p r o b l e m is to c h o o s e t h e c o n t r o l

k a s a f u n c t i o n o f t h e s t at e x s o t h a t t h e c o s t c r i t e r io n o f e q u a t i o n

. 3 i s m i n i m i z e d . T h e g e n e r a l p a r t i a l o b s e r v a t i o n o r o u t p u t

b l e m i s t o c h o o s e t h e c o n t r o l k as a f u n c t i o n

h e o b s e r v a t i o n y s o t h a t t h e c o s t o f e q u a t i o n 5 .3 is m i n i m i z e d .

A s s u m e n o w t h a t t h e p r o b l e m h a s a s o l u ti o n o f th e q u a -

l x r I I x . T h e m a t r i x H c a n b e f o u n d f r o m t h e

0 = l~I (t ) + Q + A ' I I ( t ) + I I ( t ) A - I I ( t ) B R - 1 B ' I I ( t ) , (5 . 4 )

T h e n t h e f u l l- s t a te f e e d b a c k o p t i m a l c o n t r o l l e r i s g i v en b y

T h e s o l u t i o n o f th e o u t p u t f e e d ba c k L Q G p r o b l e m is f o u n d

u s i n g t h e certa inty equiva lence principle . T h e o p t i m a l c o n -

t r o l is f o u n d u s i n g a K a l m a n f i l te r , w h e r e a n o p t i m a l e s t i m a t e )2

i s o b t a i n e d s u c h t h a t E { ( x - 2 ) ' ( x - 2 )1 is m i n i m u m . T h e n

t h is e s t i m a t e i s u s e d a s i f i t w e r e a n e x a c t m e a s u r e m e n t o f t h e

s t at e t o s o l v e th e d e t e r m i n i s t ic L Q G c o n t r o l .

F o r t h e o u t p u t f e e d b a c k c a s e , t h e e s t i m a t e d s t a te s a r e g i v e n b y :

d 2d t = A x + B k + P C V l ( y _ C;~), (5 .6)

where P s a t i s f i e s the forwa rd Riccat i equat ion:

P ( t ) = W + A P ( t ) + P ( t ) A ' - P ( t ) C ' V - l C P ( t ) . (5 . 7 )

I n e q u a t i o n 5 . 7, t h e i n i ti a l c o n d i t i o n i s P ( O ) = c o v ( x 0 ) . F i n a l ly

th e o p t i ma l o u t p u t f eed b a ck co n t ro l l er i s g iven a s :

k ( t , x ) = - R 1 B ' I I ( t ) 2 ( t ) . ( 5 . 8 )

5 .3 C o s t - C u m u l a n t C o n t r o l

5 .3 .1 M inima l Cost Variance Control

M i n i m u m cost variance ( M C V ) c o n t r o l i s a s p e c i a l c a s e o f

c o s t - c u m u l a n t o r s t a t i s t i c a l c o n t r o l w h e r e t h e s e c o n d c u m u -

l a n t, v a r i a n ce , is m i n i m i z e d , w h e r e a s t h e f i r st c u m u l a n t , m e a n ,

i s kep t a t a p re s pec i f i ed l eve l .

H e r e, o p e n - l o o p M C V a n d f u ll -s ta te f e e d b a c k M C V c o n t r o l

l aws a re d i s cus s ed . An o p e n - l o o p c o n t r o l l a w i s a f u n c t i o n

u : [ 0 , tF ] - ~ u w h e r e u is s o m e s p e c i f i e d a l l o w a b l e s e t o f c o n -t ro l va lues . A c l o s ed - l o o p o r f eed b a ck co n t ro l l a w i s a f u n c -

t i o n t h a t d e p e n d s o n t i m e a n d t h e p a s t e v o l u t i o n o f t h e p r o c es s

[i.e., u ( t , x(s ) ; 0 < s < t )1 .

O p e n - L o o p M C V C o n t r o l

C o n s i d e r a l i n e a r s y s t e m ( S a i n a n d L i b e r t y , 1 9 7 1 ) :

d x ( t) = A ( t ) x ( t ) d t + B ( t ) u ( t ) d t + E ( t ) d w ( t ) , (5 . 9 )

a n d t h e p e r f o r m a n c e m e a s u r e:

l = J [ x ' ( t ) Q x ( t ) + u ' ( t ) R u ( t ) ] d t + x ' (t F ) Q F X ( t F ),

o

(5 . 10)

w h e r e w ( t ) i s z e r o m e a n w i t h w h i t e c h a r a c t e r i s t i c s r e l a t i v e t o

the s ys tem , tF i s the f ixed f ina l t ime , x ( t ) C R n i s t h e s t a te o f t h e

s y s t e m , a n d u ( t ) E ~m i s t h e c o n t r o l a c t i o n . N o t e t h a t :

E { d w ( t ) d w ' ( t ) } = W d t. (5 . 11)

T h e f u n d a m e n t a l i d e a b e h i n d m i n i m a l c o s t - v a ri a n c e c o n tr o l

i s t o m i n i m i z e t h e v a r i a n c e o f t h e c o s t c r i te r i o n J :

k ( t , x ) = - R I B ' I I ( t ) x ( t ) . (5 . 5 ) J M v = V A R k { J I , (5 . 12)



C o s t - C u m u l a n t a n d R i s k - S e n s i t iv e C o n t r o l 1063

E k {l } = M , (5.13)

t h e c o s t c r i t e r io n a n d w h e r e t h e s u b s c r i p t k o n E

x p e c t a t i o n b a s e d o n a c o n t r o l l a w k g e n e r a t i n g t h e

o n u ( t) f r o m t h e s t a te x ( t) o r f r o m a m e a s u r e m e n t

t h a t s t a te . B y m e a n s o f a L a g r a n g e m u l t i -X c o r r e s p o n d i n g t o t h e c o n s t r a i n t o f e q u a t i o n 5 .1 3 , o n e

t h e f u n c t io n :

JM v = IX (E k{ l} - - M ) + V A R k{J } , (5.14)

] M Y = tX E k{] } + V A R k{J} . (5.15)

J M v m i n i m i z a t i o n i s d e v e l o p e d f o r t h e

p e n - l o o p c a s e :

u ( t ) = k ( t , x(0) ) . (5 .16)

h e s o l u t i o n i s b a s e d o n t h e d i f f e r e n t i a l e q u a t i o n s :

~ ( t ) = A ( t ) z ( t ) - ~ B ( t ) R 1 B ' ( t ) ~ ( t) . (5.17)

~ ( t) = - A ' ( t ) ( f f t ) - 2 Q z ( t ) - 8 t x Q v ( t ) . ( 5 .1 8 )

i , ( t ) = A ( t ) v ( t ) + E ( t ) W E ' ( t ) y ( t ) . (5.19)

~ ( t) = - A ' ( t ) y ( t ) - Q z ( t ) . ( 5 . 2 0 )

T h e s e e q u a ti o n s h a v e th e b o u n d a r y c o n d i ti o n s :

z(O) = x(0 ). (5.21)

( f f t f ) = 2QF z ( t f ) + 8DQFV(tF). (5.22)

v(0) = 0 . (5 .23)

y ( t F ) = Q F Z ( t F ) . (5.24)

T h e e q u a t i o n s a l s o h a v e t h e c o n t r o l a c t i o n r e l a t i o n s h i p :

v ( t ) = - I - R 1 B ( t ) p (t ) . (5.25)2

T h e v a r i a b l e z ( t ) i s t h e m a t h e m a t i c a l e x p e c t a t i o n o f x ( t ) . T h e

v a r i a b l e p ( t ) c o r r e s p o n d s t o t h e c o s t a t e v a r i a b l e o f o p t i m a l

c o n t r o l t h e o r y b e c a u s e i t i s t h e v a r i a b l e t h a t e n f o r c e s t h e

d i f f e r e n t i a l e q u a t i o n c o n s t r a i n t b e t w e e n z ( t ) a n d v ( t ) . T h e

v a r i a b l e v ( t ) a n d y ( t ) a r e i n t r o d u c e d t o r e d u c e t h e i n t e g r o -

d i f fe rent i a l equa t ion .

Full-State Feedback Minimal Cost-Variance Control

C o n s i d e r t h e I t o s e n s e s t o c h a s t i c d i f f e r e n t i a l e q u a t i o n ( S D E )

w i t h c o n t r o l ( S a i n et al. , 2000) :

d x ( t ) = [ A ( t ) x ( t ) + B ( t ) k ( t , x)] d t + E ( t ) d w ( t ) .

T h e e q u a t i o n a l so h a s t h e c o s t c r i t er i o n :

tF

] ( t , x ( t ) , k ) = J [ x ( t ) ' Q x ( t ) + k ' ( t , x ) R ( t ) k ( t , x ) ] d s

t

+ x ' ( t E ) Q F x ( t F ) . (5.26)

I n M C V c o n t r o l , w e d e f i n e a c l a s s o f a d m i s s i b l e c o n t r o l le r s ,

a n d t h e n t h e c o s t v a r i a n c e i s m i n i m i z e d w i t h i n t h a t c l a s s o f

cont ro l l e rs . Def ine V l ( t , x ; k ) = E { J ( t , x ( t ) , k ) [ x ( t ) = x} and

V 2 ( t , x ; k ) = E { j 2 ( t , x ( t ) , k ) [ x ( t ) = x} . A func t ion M i s an

a d m i s s i b l e m e a n c o s t c r i te r i o n i f t h e r e e x i s ts a n a d m i s s ib l e

c o n t r o l l a w k s u c h t h a t :

V l ( t , x ; k ) = M ( t , x ) , (5 .27)

for al l t E [0, tF] and x E R n .

A m i n i m a l m e a n c o s t - c o n t r o l l aw k ~ s a ti sf ie s V1 ( t , x ; k ~ ) =

V ~ ( t , x ) <_ V l ( t , x ; k )f or t E r , x E R ~ a n d f o r

k ,a n a d -m i s s i b l e c o n t r o l l a w . A n M C V c o n t r o l l a w kvl M satisfies

V 2( t , x ; kv lM ) = V f ( t , x ) <_ V 2( t , x ; k ) f o r t E T , x E N ~

w h e n e v e r k i s a d m i s s i b l e . T h e c o r r e s p o n d i n g m i n i m a l c o s t

v a ri an c e is g iv e n b y V * ( t , x ) = V 2 * ( t , x ) - M 2 ( t , x ) f o r

t E T , x E N ~ . H e r e t h e f u l l -s t a te f e e d b a c k s o l u t i o n o f t h e

M C V c o n t r o l p r o b l e m i s p r e s e n t e d f o r a l i n e a r s y s t e m a n d a

q u a d r a t i c c o s t c r i te r i o n .

T h e n t h e l i n e a r o p t i m a l M C V c o n t r o l l e r i s g i v e n b y ( S a i n e t

al., 2000) :

K v i M ( t, X ) = - R - l ( t ) B ' ( t ) [ . M ( t ) + y ( t ) V ( t ) ] x ,

w h e r e . M a n d V a re t h e s o lu t i o n s o f t h e c o u p l e d R i c c a t i - t y p e

equations ( s u p p r e ss i n g t h e t i m e a r g u m e n t ) :

0 = A) t + A ' M + M A + Q - M B R - 1 B ' A d + ~2dkdBR-1Bl id .

(5 .28)

0 = ~ / + 4 ; k 4 E W f f A d + A li d + Id A - d k d B R - l f f I d

- V B R 1 B 'A d - 2 " I V B R - l f f V , (5 .29)

w i t h b o u n d a r y c o n d i t i o n s A d ( t F ) = Q F and Id(tF) = 0. Once

aga in , i f y a ppr oac hes ze ro , c l ass i c LQ G resul t s a re obta in ed .

T h i s M C V i d e a c a n b e g e n e r a l i z e d t o m i n i m i z e a n y c o s t

c u m u l a n t s . V i e w i n g t h e c o s t f u n c t i o n a s a r a n d o m v a r i a b l e

a n d o p t i m i z i n g a n y c o s t c u m u l a n t i s c a l l e d c o s t - c u m u l a n t o r

s t a t is t i ca l con t ro l .

5 . 4 R i s k - S e n s i t i v e C o n t r o l

A l a r g e c l a s s o f c o n t r o l s y s t e m s c a n b e d e s c r i b e d i n s t a t e -

v a r i a b l e f o r m b y t h e s t o c h a s t ic e q u a t i o n s ( A n d e r s o n a n d

M oore , 1989; W hi t t l e , 1996) :



1 0 6 4 C h a n g - H e e W o n

d x ( t ) = A x ( t ) d t + B k ( t, x ) d t + d w ( t ) .

y ( t ) d t = C x ( t ) d t + d r ( t ) .(5.30)

H e r e , x ( t ) i s a 2 n - d i m e n s i o n a l s t a t e v e c t o r , k ( t , x ) i s a n

m - d i m e n s i o n a l i n p u t v e c t o r , w ( t ) i s a q - d i m e n s i o n a l d i s t u r -

b a n c e v e c t o r o f B r o w n i a n m o t i o n s , y ( t ) i s a p - d i m e n s i o n a l

v e c t o r o f o u t p u t m e a s u r e m e n t s , a n d v ( t ) i s a n r - d i m e n s i o n a l

o u t p u t n o i s e v e c t o r o f B r o w n i a n m o t i o n s t h a t a f fe c t t h e m e a s -

u r e m e n t s b e i n g t a k e n .

T h e r i s k - s e n s i t i v e c o s t c r i t e r i o n i s g i v e n b y :

IRs(O) = - 0 - 1 l o g E k { e - ° l } , ( 5 . 3 1 )

Y c (t ) = ( I + O P ( t ) H ( t ) ) - 1 ( ~ ( t ) + O P ( t ) ~ r ( t ) ). ( 5 . 3 9 )

A s 0 a p p r o a c h e s z e r o , t h e c o s t c r i t e r i o n o f e q u a t i o n 5 .3 1

b e c o m e s E k { l } , a n d t h e m a t r i c e s I I a n d P a r e o b t a i n e d f r o m

t h e R i c c a t i e q u a t i o n s :

O = l ~ I (t ) + Q + A ' H ( t ) + I I ( t ) A - I I ( t ) B R - I B ' I I ( t ) . ( 5 . 4 0 )

P ( t ) = W + A P ( t ) + P ( t ) A ' - P ( t ) C ' V - 1 C P ( t ) . ( 5 . 4 1 )

T h u s , t h e c l as s i c L Q G r e s u l t i s o b t a i n e d a s 0 a p p r o a c h e s z e r o :

w h e r e ] i s t h e c l a ss i c al q u a d r a t i c c o s t c r i t e r i o n :

l = I ( x ' Q x + k ' R k ) d t . ( 5 . 3 2 )

T h e R S co n t r o l p r o b l e m t h e n b e c o m e s a m i n i m i z a t i o n o f

IRs(O) o v e r f e e d b a c k c o n t r o l l e r k :

] ~ s ( O ) = m i n ] R s ( O ) . ( 5 . 3 3 )

A s s u m e a s o l u ti o n o f th e q u a d r a t i c f o r m l x t H x - ~ r'x +

( t e rm s i n d e p e n d e n t o f x) . T h e m a t r i x I I c a n b e f o u n d f r o m

h e R i c c a t i - t y p e e q u a t i o n :

= f I ( t ) + Q + A ' I I ( t ) + H ( t ) A - U ( t ) ( B R 1 B' + 0 W ) H ( t ) ,

( 5 . 3 4 )

T h e n , t h e f u l l - s t a te f e e d b a c k o p t i m a l c o n t r o l l e r i s g i v e n b y

( W h i t t l e , 1 9 9 6 ) :

k ( t, x ) = , R - 1 B ' I I ( t ) x ( t ) + R 1 B ' c r ( t ) , ( 5 . 3 5 )

5 .5 R e l a t i o n s h ip B e t w e e n R i s k - S e n s it iv e

a n d C o s t - C u m u l a n t C o n t r o l

T o se e t h e r e l a ti o n s h i p b e t w e e n R S a n d c o s t - c u m u l a n t c o n t r o l,

c o n s i d e r a c o s t c r i t e r i o n :

tF

J = j [ x ( t ) ' Q x ( t ) + l d ( t , x ) R ( t ) k ( t , x ) ] d s + x ' (t E ) Q F X ( tF ) . ( 5 . 4 2 )

0

C l a ss i ca l L Q G c o n t r o l m i n i m i z e s t h e f ir s t c u m u l a n t o r t h e

m e a n o f t h e c o s t c r i t er i o n o f e q u a t i o n 5 .4 2 . I n M C V c o n t r o l ,

t h e s e c o n d c u m u l a n t o f e q u a t i o n 5 .4 2 is m i n i m i z e d w h i l e t h e

m e a n i s k e p t a t a p r e s p e c i f i e d l ev e l. F u r t h e r m o r e , R S c o n t r o l

m i n i m i z e s a n i n f in i t e li n e a r c o m b i n a t i o n o f t h e c o s t c u m u -

l a n t s . T o s e e t h i s , c o n s i d e r a n R S c o s t c r i t e r i o n :

lR S = - -0 1 lo g (E { e x p ( - 0 1 )} ) , (5 . 4 3 )

w h e r e 0 is a r e a l p a r a m e t e r a n d E d e n o t e s e x p e c t a t i o n . T h e n ,

t h e m o m e n t - g e n e r a t i n g f u n c t i o n o r t h e f i rs t c h a r a ct e r is t i c

f u n c t i o n i s g i v e n b y :

w h e r e ( y (t ) + ( A - H ( B ' R - 1 B ' + 0 W ) ) ' c r ( t) = 0 i s a b a c k -

P ( t ) = W + A P ( t ) + P ( t ) A ' - P ( t ) ( C ' V - a C + 0 Q ) P ( t ) , ( 5 . 3 6 )

( 0 ) = ( X o). T h e u p d a t i n g e q u a t i o n f o r t h e r is k -

e n s i t i v e K a l m a n f i l t e r i s g i v e n b y :

d ~d t = A x + B k + P C V - 1 (y - C z ~) - 0 P Q ~ , (5 . 3 7 )

e r e z t (0 ) = 0 . :~ d e n o t e s t h e m e a n o f x c o n d i t i o n a l o n t h e

h i s t or y . F i n al ly , t h e o p t i m a l o u t p u t f e e d b a c k c o n tr o l l e r

g i v e n a s ( W h i t t l e , 1 9 9 6 ) :

k ( t , x ) = - R - 1 B ' I I ( t ) f c ( t ) + R 1 B ' cr (t ), ( 5 . 3 8 )

w h e r e 2 i s t h e m i n i m a l - s t r e s s e s t i m a t e o f x, g i v e n b y :

q b( s) = E e x p ( - s l ) . ( 5 . 4 4 )

T h e c u m u l a n t g e n e r a t i n g f u n c t i o n t b ( s) i s d e f i n e d b y :

(_1)~t ~ (s ) = l o g + ( s ) = Z T ~ i s i '

i= 1( 5 . 4 5 )

i n w h i c h t h e { [3 i} a r e k n o w n a s t h e c u m u l a n t s o r s o m e t i m e s a s

t h e s e m i - i n v a r i a n t s o f ]. N o w b y c o m p a r i n g e q u a t i o n s 5 .4 3,

5 . 4 4, a n d 5 . 4 5 , i t i s n o t e d t h a t :

: R s : ( - o - ~ i ( ] ) ( O ) i , (5 . 4 6 )

w h e r e f3 i ( ] ) d e n o t e s t h e i th c u m u l a n t o f J w i t h r e s p e c t t o t h e

c o n t r o l l a w k . T h u s , i t is i m p o r t a n t t o n o t e t h a t t h e R S c o s tc r i t e ri o n i s a n i n fi n i te l i n e a r c o m b i n a t i o n o f th e c o s t c u m u -

l a n t s . M o r e o v e r , a p p r o x i m a t i n g t o t h e s e c o n d o r d e r :



C o s t - C u m u l a n t a n d R i s k - S e n s i t iv e C o n t r o l 1065

0JR S = [31(1) -- ~[32(1) + 0( 02)

= E { J } - e _ V A R { J } + 0 ( 0 2 ) .2

(5.47)

s c a n b e v i e w e d as fi r st - a n d s e c o n d - o r d e r a p p r o x i m a -

o b l e m r e sp e c ti v e ly . M i n i m i z i n g t h eu n d e r t h e r e s t r i c t i o n t h a t th e f i rs t c u m u l a n t E { J } exis ts

minimal cost v a r i a n c e ( M C V ) p r o b l e m . M o r e -

, m i n i m i z i n g a n y l in e a r c o m b i n a t i o n o f c o st c u m u l a n t s

i s ti c a l c o n t r o l . T h u s , c la s si c al L Q G c o n t r o l ( o p t i m i z a t i o n o f

r st c u m u l a n t ) , M C V c o n t r o l ( o p t i m i z a t i o n o f t h e s e c o n d

u l a n t ) , a n d R S c o n t r o l ( o p t i m i z a t i o n o f th e i n f in i te

m b e r o f c u m u l a n t s ) a r e a ll s p ec i al c as es o f th e c o s t - c u m u ,

a p p l i c a t i o n o f r i s k -s e n s i ti v e c o n t r o l t o s a te l li te a t t i t u d e

n e u v e r i s g i v e n in t h i s s ec t i o n . A n a p p l i c a t i o n o f m i n i m a l

v a r i a n c e c o n t r o l t o a n e a r t h q u a k e s t r u c t u r e c o n t r o l i s a l so

e . F o r l i n e a r q u a d r a t i c G a u s s i a n a p p l i c a t io n s , s e e

a r k e r n a a k a n d S i v a n (1 9 7 2 ) . F o r m o r e r i s k - s e n s i ti v e c o n t r o l

x a m p l e s , r e fe r t o B e n s o u s s a n ( 1 9 9 2 ) a n d W h i t t l e ( 1 9 9 6 ) .

5 .6 .1 R i s k - S e n s i t i v e C o n t r o l A p p l i e d t o S a t e ll it e

A t t i t u d e M a n e u v e r

h i s s u b s e c t i o n s h o w s t h e s i m u l a t i o n r e s u l t s a s s o c i a t e d w i t h

o d e l o f a g e o s t a t i o n a r y s a t e ll it e e q u i p p e d w i t h a b i a s

o m e n t u m w h e e l o n t h e t h i r d a x is o f b o d y f r am e . T h i s

d e l , s m a l l a t t it u d e a n g l e a n d r o l l / y a w d y n a m i c s a r e a s s u m e d

A r o l l / y a w a t t i tu d e m o d e l o f t h e g e o s t a t i o n a r y s a t e ll it e is

impl i f i ed as the fo l lowing

hw > > m a x {Ii , toe}:

0

0clx( t ) = hwo,

I t t

0

r °+

co s (c~)

L s in ( o O

l i n e a r d i f f e r e n t i a l e q u a t i o n o f

0 1 00 0 1hw x ( t ) d t

0 0 - - I I~

hwoac hw 0

I22 I22

m ( t ) d t + d w ( t )

L/~2J

T h e d w / d t i s G a u s s i a n w h i t e n o i s e r e p r e s e n t i n g t h e d i s t u r b a n c e

t o r q u e , d v / d t is G a u s s i a n w h i t e n o i s e r e p r e s e n t i n g t h e m e a s u r e -

m e n t n o i s e , h w i s t h e w h e e l m o m e n t u m , o ~ i s t h e a n g l e

t h a t t h e p o s i t i v e ro l l ax i s m a k e s w i t h t h e m a g n e t i c t o r q u e r , to c

i s the orb i t a l ra t e , Ii i is th e m o m e n t o f in e r t ia o f th e

i th axis , x = [y, r , '~ , k] is the s tate w ith y aw ( ', /) , rol l (r) ,

m is a d i p o le m o m e n t o f t he m a g n e t i c t o r q u e r ( c o n t ro l ) ,

Be = 1 .07 x 10 7 t e l s a i s the n om ina l mag ne t i c f i e ld s t rength ,

a n d I 4× 4 is a n i d e n t i t y m a t r i x w i t h d i m e n s i o n f o u r . T h e e x p e c t e d

v a l u e o f d w l d t i s ze ro wi th E{ d w / d t x d w / d t } = 0.7 B e, a n d t h e

e x p e c t e d v a l u e o f d v / d t i s ze ro wi th E { d v / d t x d r ~ d r '} =

1 x 1 0 7 . H e r e O = 5 x 1 0 2 w a s c h o s e n f o r t h e d e m o n s t r a t i o n

p u r p o s e , b u t t h i s r is k - s e n s i t iv i t y p a r a m e t e r , 0 s h o u l d b e v i e w e d

a s a n o t h e r d e s i g n p a r a m e t e r j u s t l ik e t h e w e i g h t i n g m a t r i c e s Q

a n d R . B y v a r y i n g t h i s O , d i f f e r en t p e r f o r m a n c e a n d s t a b i li t y

resu l t s can be ob ta ined . Theo re t i ca l ly , a l l O h a t g ive a so lu t ion to

t h e R i c c a t i e q u a t i o n 5 .3 6 a r e p o s s ib l e . T h e n e x t e x a m p l e s h o w s

h o w t o c h o o s e t h is r i s k - se n s i t i v it y p a r a m e t e r t o o b t a i n l a r g e r

s t a b i l i t y m a r g i n . T h e c o n s t a n t s f o r t h e o p e r a t i o n a l m o d e a r eg i v e n a s / 1 1 = 198 8kg "m2, 122 = 18 76 kg 'm 2 , 112 =/ 21 = 0 ,

hw = 55 kg- m2 /s , toc = 0 .00418 deg / s , and 0 = 60 deg . These

v a l u e s a r e a c t u a l p a r a m e t e r s o f t h e g e o s t a t i o n a r y s a te l li te . T h e

ini t ia l condi t ion is [0.5 deg, 0, 0, 0.007 deg/sec] . Final ly,

t h e w e i g h t i n g m a t r i c e s a r e c h o s e n t o b e Q = I 4 x 4 a n d

R = 1 x 10 10

I n t h i s m o d e l , t h e s t a te s a r e m e a s u r e d w i t h t h e s e n s o r n o i s e ,

d v / d t . A K a l m a n f i lt e r is t h e n u s e d t o e s t i m a t e t h e s t a te s . T h e

f o l l o w i n g s i m u l a t i o n s a r e p e r f o r m e d u s i n g M A T L A B , a s o f t -

w a r e p a c k a g e . T h e R S c o n t r o l l e r is f o u n d u s i n g e q u a t i o n 5 .3 5 .

N o t e t h a t b o t h y a w a n d r o l l a n g l e s r e d u c e t o a v a l u e c l o s e t o

t h e o r i g i n . F i g u r e 5 . 2 s h o w s t h e r o l l a n d y a w a n g l e s w i t h

r e s p e c t t o t i m e v a r i a t i o n . A f t e r a b o u t 3 h o u r s , b o t h r o l l a n d

yaw angles s t ay be low 0 .1 degree . In i ti a l ly , l a rge cont ro l ac t io n

i s n e e d e d , b u t a f t e r 3 h o u r s o r s o , le ss t h a n 3 0 0 A t m 2 m a g n e t i c

° 5.4 . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . . ! . . . . . . . . . . . . . . . . ! . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . i . . . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . . .% - % : : . .

' ~ A Z ' ! ; :

~ . ' . : ; . ' ~ : : : :

0 . 1 , - . . . . . ~ . . . . . . . . . . . . . . . . ~ . . . . . . . . . . . . . . . . . : . . . . . . . . . . . . . . . . . : . . . . . . . . . . . . . . . . .

" ~ . . v 4 , , : i i , , .." ~ v ¢ : . ~ . .3 , { ~ , ~ 2 ~ , . a ; . . . . . . t , e . , . 4 . . . : . . .

, ~ % ~ - * " > ' i e ' : " • ' " " " " " . ' ~ ' . ~ , .

0 " : . ~ ' ~ ' *

(5.48) -0 .2 -0 "1 . . . . . . ; . . . . . . . . . . . . . - . - i . . . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . . . !i . . . . . . . . . . . . . . . .

- 0 ' 3 0 5 1 0 1 5 2 0 2 5

T i m e [hours]

d y ( t ) = I 4 x 4 X ( t ) d t + d r ( t ) (5.49 ) FIGURE 5.2 R ol l (Dark) and Yaw (Light) Versus Time, RS Con trol



1066 C h a n g - H e e W o n

0 .5

. . . . . . . . i . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7 > . o . , L G , : . . : i ; . . . . . . . . . . . . . . . ! . . . . . . . . . . . . . . . . ! . . . . . . . . . . . . . . . . i . . . . . . . . . . . . . . . .. . . . . . ":.'.,~ +~,~ , i i i

- o . 1 . . . . . . . . . . . . . ~ . . . . . . . . . . . . . . . . ~ . . . . . . . . . . . . . . . . :: . . . . . . . . . . . . . . . . :: . . . . . . . . . . . . . . . .

i .............. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-0"3 0 5 10 15 20 25

Time [hours ]

FIGUR E 5.3 R ol l D ark) and Yaw (Light) Versus Time, LQG Control

o r que i s r e qu i r e d . I t i s im por t a n t t o no te t ha t de sp i t e t he

e x te r na l d i s tu r ba nc e s , R S c on t r o l l a w p r oduc e s good pe r f o r -

m a nc e .

To c om pa r e t he r e su lt s w i th t he w e l l - know n LQ G c on t r o l l e r ,

t he sy s t e m w a s s im u la t e d w i th a n LQ G c on t r o l l e r . The 0

a pp r oa c he d in f in i t y i n e qua t ion 5 .36 . N o te t ha t e q ua t ion 5 . 36

beco me s a c lassica l R icca t i equa t ion as 0 goes to inf in i ty . This

is shown in F igure 5 .3 . Note tha t in LQG case , i t takes longer

for yaw and ro l l angles to fa l l be low 0 .1 degree , and the

var ia t ion in the angles a re la rger than the RS case . Thus , in

th i s s ense , R S c on t r o l l e r ou tpe r f o r m s the L Q G c on t r o ll e r .

k 3 , c 3

k2,c2

k l , C l

m 3

m2

x 3 ( t )

x 2 ( t )

i i i i i i i i0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

GammaFIGUR E 5.4 Schematic Diagram for Three Degree-of-Freedom

Structure

5 .6 . 2 M C V C o n t r o l A p p l i e d t o S e i s m i c P r o t e c t i o n

o f S t r u c t u r e s

A 3D O R s ing le- ba y s t r uc tu r e w i th a n a c tive t e nd on c on t r o l l e r

as shown in F igure 5 .4 i s cons idered here . The s t ruc ture i s

sub je ct t o a one - d im e ns iona l e a r thqua ke e xc i ta t i on . I f a s im p le

she a r f r a m e m ode l f o r t he s t r uc tu r e i s a s sum e d , t he gove r n ing

e qua t ions o f m o t ion in s t a te spa c e f o r m c a n be w r i t t e n a s:

0 I ] x ( t ) d td x ( t ) = _ M T 1 K s _ M T 1 C ,

E ] E°Iu ( t ) d t + d w ( t ) ,+ M T 1 B , - F s

w he r e t he f o l low ing a pp ly :

M s =

C s =

K , =

ml 0 0 ]

0 m2 0 ] , B~ =

0 0 m3

c l + q - q 0

-c 2 c2 + c3 - c3

0 -c3

k~ + k 2 - k 2

- k 2 k 2 + k 3

0 - k 3

- - 4 k c co s oL"

0

0

, r $ ~

C3o ]- k 3 .

k3

T h e m i, c i , ki are the mass, damping, and stif fness, respectively,

a s soc ia t e d w i th t he i t h f l oo r o f t he bu i ld ing . The kc is thes t if fne s s o f the t e ndo n . T he B r ow n ia n m o t io n w ( t ) w i th

E { d w ( t ) } = 0 a n d E { d w ( t ) d w ' ( t ) } = W d t; in th is example ,

W = 1 .00 x 2~r in2/sec 3 . T he param ete rs were chosen to

1 6 0

15 0

14 0

13 0

12 0q -

110>

100

90

80

70

60 0

FIGUR E 5.5 Optimal V ariance: Full-State Feedback, MCV , 3DO F



Cost-Cumulant and Risk-Sensitive Control 1067

0 . 0 2

0 . 0 1 9t~

Eo ~ 0 . 0 1 8

0 . 0 1 7

i i i i i i i ~ i

0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 1 . 4 1 . 6 1 . 8 2

0 '0 3 c I ~ '. ! ! ! ! ! ! !

I

E

t ........................................ : o +o o 1 5 i ; i ; i . . . .

0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 1 . 4 1 . 6 1 . 8 2

o . o ~ | ! ! ! ! ! ! ! ! ! /

. . . . . . . . . . . . . . . . . : . . . . . . . . . . . . . . . . . . . . . . . . 1° . ° 3 ro o 2 1 , , , , , , :, , , I

' 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 1 . 4 1 . 6 1 . 8 2

x

E

co t o i '.~ o . . . . . . . . .

i i i i i i

0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 1 . 4 1 . 6 1 . 8 2

L f ' )

X

E

% ~ o . ~ ~ o ! , o : o o ; ~ ~ : ~ ~ ! 4 ~ ! ~ , ! ~

¢ .O

X

E

090 . 7 . . . . . . . . . . . . . : : . . . . . . . . :

0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 1 . 4

G a m m a

i i

1 . 6 1 . 8 2

FIGUR E 5.6 Displacements and V elocities. Full-State Feedback, MCV , 3DO F

m a tc h m oda l f r e que nc ie s a nd da m pings o f a n e xpe r im e n ta l

s t ruc ture . The cos t c r i te r ion i s g iven by:

l = J ( z ' ( t ) K ~ ( t ) + k c u 2 ( t ) ) d t ,

tog e the r wi th R = kc, whe re z i s a vec to r of f loor d isp lacem ents

and x = [z ~ ] '.

F igure 5 .5 shows tha t the va r ian ce of the co s t c r i te r ion

decreases as y inc reases . No te tha t the ~ = 0 poi n t cor re spon ds

to the classical LQG case.

F igure 5 .6 shows the RMS d isplacem ent responses o f f i r st

(~xl ) , seco nd (~x2), an d th i rd ((Yx3) f loor and the RM S ve loc i ty

respo nses of f irst (Crx4), sec on d (Crx~), an d thi rd (Crx6) f loor ,

r e spe c tive ly , ve r sus t he M C V pa r a m e te r % I t i s im p or t a n t t o

no te t ha t bo th t h i r d f l oo r R M S d i sp l a c e m e n t a nd ve loc i ty

r e sponse s c a n be de c r e a se d by c hoos ing l a r ge %

5 . 7 C o n c l u s i o n s

Thi s c ha p te r de sc r ibes l i ne a r - qua d r a t ic - G a uss i a n ( LQ G ) , m in -

imal cos t va r iance (MCV) , and r i sk-sens i t ive (RS) contro ls in



1068 C h a n g - H e e W o n

f t h e c o s t c u m u l a n t s . C o s t c u m u l a n t c o n t r o l , w h i c h i s

a r a n d o m v a r ia b le a n d m i n i m i z e s a n y c u m u l a n t o f t h e

i z a t i o n c r i te r i o n . T h e n L Q G , M C V , a n d R S a re a ll sp e c ia l

e s o f co s t c u m u l a n t c o n t r o l w h e r e i n L Q G t h e m e a n , i n

C V t h e v a r ia n c e , a n d i n R S al l c u m u l a n t s o f t h e c o s t f u n c -

o p t i m i z e d . T h i s c h a p t e r p r o v i d e s t h e o p t i m a l c o n t r o l -

u d e c o n t r o l a p p l i c a t i o n u s i n g R S c o n t r o l l e r a n d b u i l d i n g

erson, B .D.O. , and Mo ore , J .B. (1989). Optimal control, linear

quadratic methods. Englewood Cliffs, NJ: Prentice Hall.

T., an d Bernh ard, P. (1991). H~-opt imal control and re lated

min ima x des ign problems . Boston: Birkhauser.

Stochastic contro l of partially observable sys-

tems. Cam bridge: Cam bridge Univers ity Press .

Linear estimation and stochastic control.London: Hals ted Press .

nd Rishel, R.W . (1975). Deterministic and stochastic

optimal control. Ne w Y ork: Springer-Verlag.

, and Doyle, J .C. (1988). State-space form ulae f or all stabil-

iz ing control lers that sa t isfy Ho~-norm bo un d an d re la tions to r isk

sensitivity. Systems and Control Letters 11, 167-172.

nential per form anc e criteria an d their relationship to deterministic

differential games, IEEE Transactions on Autom atic Control, AC-18,

124-131.

Kwakernaak, H., and Sivan, R. (1972). Linear optimal control systems.

New York: John W iley & Sons .

Rhee, I. , and Speyer, J . (1992). Application of a game theoretic

con t ro ll e r to a benchmark p rob lem. Journal of Guidance, Control,

and D ynamics 15(5), 1076-1081.

Runolfsson, T. (1994). The equivalence between infinite-horizon op-

t imal co ntrol o f stochas tic sys tems w ith expone nt ia l-of- integra l

performance index and stochastic differential games. IEE E Transac-

tions on Automatic Control 39(8), 1551-1563.

Sain, M.K., and Liberty, S.R . (1971). Per form ance me asure densities

for a c lass of LQG contro l sys tems. IEEE Transactions on Autom atic

Control AC-1 6 (5), 431-439.

Sain, M.K., Won, C.H., and Spencer, Jr. , B.E (1992). Cumulant mini-

mizat ion and robus t control . In Duncan, T .E. , and Pas ik-Duncan,

B. (Eds.) Stochastic Theory and Adaptive Control Lecture Notes in

Control and Information Services 184 Berlin: Springer-Veflag, pp.

411-425.

Sain, M.K., W on, C.H., Spencer, Jr. , B.F. , and Liberty, S.R. (2000).

Cumulants and r isk-sens i t ive control : A cos t mean and variance

theor y with appl ica t ion to se ismic protect ion of s tructures. In l .A.

Filar, V. Gaitsgory, and K. Mizukami (Eds.), Advances in DynamicGames an d Applications, Annals o f the International Society of

Dynamic Games, Vo l. 5. Boston : Birkhauser.

Uchida, K., and Fujita, M. (1989). On the central controller: Charac-

terizations via differential games an d LEQG co ntro l problems.

Systems and Control Letters 15(1), 9-13.

Whittle, P. (1996). Op timal control, basics and beyond. New York: John

Wiley & Sons .

Won, C.H. (1995). Cost cumulants in risk-sensitive and minimal cost

variance control, Ph.D. Disserta t ion. Univers i ty of No tre Dame.

Documents

Cost-Cumulants and Risk Sensitive Control