Optimal control for discrete -time fractional order ...obzor.bio21.bas.bg/conference/Conference_files/sa19/Ungureanu.pdf · Optimal Control for Discrete- Time Fractional-Order Systems

$Page 1: Optimal control for discrete -time fractional order ...obzor.bio21.bas.bg/conference/Conference_files/sa19/Ungureanu.pdf · Optimal Control for Discrete- Time Fractional-Order Systems$
Optimal control for discrete -time fractional ordersystems with Markovian jumps

Geometry, Integrability and Quantization, Varna 2019

Viorica Mariela Ungureanu

Constantin Brancusi University, Romania

VM Ungureanu ( Constantin Brancusi Univ. ) Optimal control for discrete -time FOS 1 / 22

Optimal Control for Discrete- Time Fractional-OrderSystems

Fractional calculus(FC) began to engage mathematiciansinterest inthe 17th century as evidenced by a letter of Leibniz to LHôspital,dated 30th September 1695, which talks about the possibility ofnon-integer order di¤erentiation. Later on, famous mathematicians asFourier, Euler and Laplace contributed to the foundation of this newbranch of mathematics with various concepts and results.

Nowadays, the most popular denitions of the non-integer orderintegral or derivative are the Riemann-Liouville, Caputo andGrunwald-Letnikov denitions. For a historical survey and the currentstate of the art, the reader is referred to [3], [4], [5],[6], [7] and thereferences therein.




FC nds use in di¤erent elds of science and engineering including theelectrochemistry, electromagnetism, biophysics, quantum mechanics,radiation physics, statistics or control theory (see [6], [8], [4], [9]).Such an example comes from the eld of autonomous guided vehicles,which lateral control seems to be improved by using fractionaladaptation schemes [10]. Also, partial di¤erential equations offractional order were applied to model the wave propagation inviscoelastic media or the dissipation in seismology or in metallurgy[11].The optimal control theory was intensively developed during the lastcentury for deterministic systems dened by integer-order derivatives,in both continuous- and discrete- time cases [12]. Since manyreal-world phenomena are a¤ected by random factors that exercised adecisive inuence on the processes behavior, stochastic optimalcontrol theory had a similar evolution in the recent decades (see [13],[14], [15], [16] and the references therein). However, only a fewpapers address optimal control problems for fractional systems (seee.g. [17], [18], [19], [20], [21], [22]) and fewer consider stochasticfractional systems [23], [24].


Finite-horizon LQ optimal control problem

In what follows we shall discuss a nite-horizon LQ optimal controlproblem for stochastic discrete-time LFSs dened by theGrunwald-Letnikov fractional derivative. As far as we know, thissubject seems to be new for discrete-time LFSs with inniteMarkovian jumps. The case of discrete-time LFSs a¤ected bymultiplicative, independent random perturbations was consideredrecently in [*]Trujillo, J. J., & Ungureanu, V. M. (2018). Optimalcontrol of discrete-time linear fractional-order systems withmultiplicative noise. International Journal of Control, 91(1), 57-69].A determinstic case was considered previously in A. Dzielinski,P. M.Czyronis, Dynamic Programming for Fractional Discrete-timeSystems, 19 th World Congress of IFAC,19(1)(2014).

Following [*], we have used an equivalent linear expanded-statemodel of the discrete-time LFS with jumps and we have rewritten thequadratic cost functional accordingly. Then the original optimalcontrol problem reduces to a LQ optimal control problem for linearstochastic systems with Markovian jumps. The solution is obtained byusing a dynamic programming technique.


Finite-horizon LQ optimal control problem

In what follows we shall discuss a nite-horizon LQ optimal controlproblem for stochastic discrete-time LFSs dened by theGrunwald-Letnikov fractional derivative. As far as we know, thissubject seems to be new for discrete-time LFSs with inniteMarkovian jumps. The case of discrete-time LFSs a¤ected bymultiplicative, independent random perturbations was consideredrecently in [*]Trujillo, J. J., & Ungureanu, V. M. (2018). Optimalcontrol of discrete-time linear fractional-order systems withmultiplicative noise. International Journal of Control, 91(1), 57-69].A determinstic case was considered previously in A. Dzielinski,P. M.Czyronis, Dynamic Programming for Fractional Discrete-timeSystems, 19 th World Congress of IFAC,19(1)(2014).Following [*], we have used an equivalent linear expanded-statemodel of the discrete-time LFS with jumps and we have rewritten thequadratic cost functional accordingly. Then the original optimalcontrol problem reduces to a LQ optimal control problem for linearstochastic systems with Markovian jumps. The solution is obtained byusing a dynamic programming technique.


Why stochastic systems with Markovian jumps?

Stochastic discrete-time systems with Markovian switching can modelmany physical systems which may experience abrupt changes in theirdynamics. Among them we mention the manufacturing systems, the powersystems, the telecommunication systems e.t.c. All these systems su¤erfrequent unpredictable structural changes caused by failures or repairs,connections or disconnections of the subsystems [1].


ExampleA model where the state vector xk is computed by switching (according toa Markov law) between the following LFSs

State i = 1

∆[α]h xk+1 =

1/2 10 1/3

xk +

1/2 00 1/2

uk , k 2 N

State i = 2

∆[α]h xk+1 =

1/2 10 1/1

xk +

1/3 00 1/3

uk , k 2 N

x0 = x 2 Rd ,

Such a model could describe a special solar thermal receiver system wherethe state space Z = f1, 2g could represent an atmospheric conditionwhere i = 1 means "sunny", i = 2 means "cloudy".


Stochastic systems with Markovian jumps

frkgk2N is a homogeneous Markov chain on a probability space(Ω,F ,P) , with the state space Z Z (nite or innite) and Gk isthe σ algebra generated by fri , 0 i k 1g, k 2 N. Considerthe LFS with control

∆[α]xk+1 = Ak (rk )xk +Bk (rk )uk , k 2 N (FOS)

x0 = x 2 Rd , (Init cond FOS)

yk+1 = Ck (rk ) xk (observable output)

where the control sequence u = fukgk2N (the input) belongs to aclass of admissible controls U a (U a = fu = fukgk2N

juk 2 L2 (Ω,Rm) is Gk -measurable for all k 2 Ng) and the operator

∆[α]xk+1 =1hα

k+1

∑j=0(1)j

αj

xk+1j , h > 0

is the one used for the denition of the Grünwald-Letnikov fractional

order derivative. Here

αk

= Γ(α+1)

Γ(k+1)Γ(αk+1) .VM Ungureanu ( Constantin Brancusi Univ. ) Optimal control for discrete -time FOS 9 / 22

Optimal control problem

Let x 2 Rd , i 2 Z and N 2 N be xed. Our optimal control problemO is to minimize the cost functional

Ix ,N ,i (u) = (cost funct)N1∑n=0

EhkCn (rn) xnk2 + < Kn (rn) un, un >

jr0=i

i+

E [< SxN , xN > jr0=i ]

subject to (FOS)-(Init cond FOS), over the class U a of admissiblecontrols.

Here S δIRm ,Kn (i) δnIRm ,for all i 2 Z and E [ξjη = x ]denotes the conditional expectation on the event η = x of anintegrable, real-valued random variable ξ dened on a probabilityspace (Ω,F ,P) .


A linear expanded-state model- The coe¢ cients

For all j 2 N, cj := (1)j

αj + 1

, A0k (i) = h

αAk (i) + αIRd ,

Bk (i) = hαBk (i),i 2 Z and Ak (i) 2 L

RdN

,

Bk (i)2L

Rm ,RdN

, k 2 N are linear and bounded operators denedby

Ak (i) =

0BBBB@A0k (i) c1IRd . cN1IRd

IRd 0 . 0. IRd . .. . . 00 . IRd 0

1CCCCA ,Bk (i) =0BB@Bk (i)0.0

1CCA .(1)

Also,

Ck (i) :

RdN! Rp , Ck (i) (v0, v1, ...) = Ck (i) (v0) , k = 0, ..,N 1

S : (Rd )N ! (Rd )N ,S (v0, ..., vN1) = (Sv0, 0, ..., 0) .VM Ungureanu ( Constantin Brancusi Univ. ) Optimal control for discrete -time FOS 11 / 22

A linear expanded-state model-The equations

XTk =xk , xk1, ..., x0, 0, .., 0

N

and

Xk+1 = Ak (rk )Xk + Bk (rk ) uk , (2)

X0 = (x0, 0, 0...) 2 l2Rd , x0 = x 2 Rd (3)

Ix ,N ,i (u) =N1∑n=0

E [(hCk (rk ) Ck (rk )Xk ,Xk i+ hKk (rk ) uk , uk i) jr0=i ]

+E [(h(AN1 (rN1) SAN1 (rN1))XN1,XN1i+2 h(AN1 (rN1) SBN1 (rN1)) uN1,XN1i+ (4)

h(BN1 (rN1) SBN1 (rN1)) uN1, uN1i)jr0=i ].


Backward discrete-time Riccati equation of control onordered Banach spaces

H is a Hilber space, L (H) denotes the Banach space of all linear andbounded operators A : H ! H and S (H) is the Banach subspace ofL(H) formed by all self-adjoint operators. lZ

S(Rd )= fR = fR (i) 2

S (H) , i 2 Zg, kRkZ = supi2Z kR (i)k < ∞g -Banach orderd spaceE :lZ

S(Rd )! lZ

S(Rd ),E (R) (i) = ∑

j2ZqijR (j) , qij =

P (rn+1 = j jrn=i )(transition matrix of the Markov chain)for all i 2 Z , k 2 f0, 1, ..,N 1g

Pk (i) = Ak (i)E (Pk+1) (i)Ak (i) + Ck (i) Ck (i) (Riccati Syst)

[Ak (i)E (Pk+1) (i)Bk (i)] [Kk (i) + Bk (i)E (Pk+1) (i)Bk (i)]1[Bk (i)E (Pk+1) (i)Ak (i)],

PN (i) = S (Final cond)


Cost functional

TheoremThe cost functional (4) can be equivalently rewritten as

Ix ,N ,i (u) = E [hP0 (i)X0,X0i jr0=i ] + (5)

+N1∑n=0

E [Kn (Pn+1) (rn)]1/2 [Wn (rn)Xn un ]

2 jr0=i ,where, for all R 2 lZ

S(Rd ), i 2 Z , n 2 f0, 1, ..,N 1g,

Kn (R) (i) = Kn (i) + Bn (i)E (R) (i)Bn (i) ,Wn (i) = [Kn (Pn+1) (i)]1 [Bn (i)E (Pn+1) (i)An (i)].


Main result

TheoremLet fPngn=0,..,N1 be the unique solution of the generalized Riccatiequation of control and let Wn, n = 0, ..,N 1 be dened as above. Thecontrol sequence

eu = feu0 = W0X0, ..., eun = WnXn, ..., euN1 = WN1XN1g (6)

minimizes the cost functional Ix ,N ,i (u) and the optimal cost is

minu2U a0,N1

Ix ,N ,i (u) = E [hP0X0,X0i jr0=i ] . (7)


ExampleConsider a time-invariant, nite-dimensional version of the LFS with

control where α = 12 , h = 2, d = 2,m = 2, x0 =

12

,Z = f1, 2g and

Ak (1) =1/2 10 1/3

,Ak (2) =

1/2 10 1/1

,

Q =1/2 1/23/4 1/4

(Transition matrix of the markov

process),Kk (1) =2 11 2

,

Kk (2) =3 22 3

, S =

2 00 2

and

Bk (i) =1/ (i + 1) 0

0 1/ (i + 1)

,

Ck (i) = (i + 1)2

0@ 1 00 10 0

1A , for i = 1, 2.VM Ungureanu ( Constantin Brancusi Univ. ) Optimal control for discrete -time FOS 16 / 22

Example

In this example, the state space of the Markon chain is Z = f1, 2g andthe state vector xk will be computed by switching (according to theMarkov law) between the following two deterministic LFSs

∆[α]xk+1 =

1/2 10 1/3

xk +

1/2 00 1/2

uk , k 2 N

x0 = x 2 Rd ,

∆[α]xk+1 =

1/2 10 1/1

xk +

1/3 00 1/3

uk , k 2 N

x0 = x 2 Rd ,

Our model could describe a special solar thermal receiver system where thestate space Z = f1, 2g of the Markov chain could represent anatmospheric condition where i = 1 means "sunny", i = 2 means "cloudy".


Example

P0 (i) is a dened by a (2N) (2N) symmetric matrix. Therelevant values of P0 (i) correspond to the rst 2 nonzero elements of

X0 =x0, 0, .., 0

N1

,x0 2 R2 because

minu2U a0,N1 Ix ,N ,i (u) = E [hP0X0,X0i jr0=i ] .

P0 (i)not= P (0, i) =

0BBBBBBBB@

P11 (0, i) P12 (0, i) . . P1(2N ) (0, i)P12 (0, i) P22 (0, i) . . P2(2N ) (0, i)

. . . . .

. .

. .

. .P1(2N ) (0, i) P2(2N ) (0, i)

1CCCCCCCCABy implementing in Matlab the above results concerning the Riccatiequation of control and the optimal cost, we get the following numericalresults.





Conclusions and further research

This paper provides a new method ( based on the dynamic programmingapproach) for solving the LQ optimal control problem O. It consists in areformulation of the problem for an associated linear non-fractional system(2)-(3), dened on spaces of higher dimensions. The computer programimplementing this method is simple and fast.The linear approach opensthe way of solving an innte horizon LQ optimal control problem for LFSs.

Further research

innite horizon LQ optimal control problem for LFSsasympototic behavior of the generalized Riccati equation with controlstabilizability, detectability and observability properties of the LFS(other than the exponential stability, because the system cannot beexponentially stabilized).

Thank you!!


E.K. Boukas, Control of Singular Systems with Random AbruptChanges, Communications and Control Engineering Series.Springer-Verlag, Berlin, 2008.

A.A. Kilbas, H.M. Srivastava, J.J. Trujillo, Theory and Applications ofFractional Di¤erential Equations. Elsevier Science, Amsterdam, 2006.

I. Podlubny, Fractional Di¤erential Equations, Academic Press, Newyork, 1999.

K. S. Miller, B. Ross, An introduction to the fractional calculus andfractional di¤erential equations, Wiley-Interscience Publication, NewYork, 1993.

M. Wojciech, J. Kacprzyk,J. Baranowski. Advances in the Theory andApplications of Non-integer Order Systems, 5th Conference onNon-integer Order Calculus and Its Applications, Cracow, Poland.Lecture Notes in Electrical Engineering, 257, 2013.


C.A. Monje et al. Fractional-order Systems and Controls:Fundamentals and Applications, Springer Science & Business Media,London, 2010.

B. Bandyopadhyay, S. Kamal, Stabilization and Control of FractionalOrder Systems: A Sliding Mode Approach, Springer InternationalPublishing, Switzerland, 2015.

U.N.Katugampola, A New Approach To Generalized FractionalDerivatives, Bull. Math. Anal. App. 4(6)(2014), 115.

J. Sabatier,M. Moze, C. Farges, LMI stability conditions for fractionalorder systems, Computers & Mathematics with Applications,59(5)(2010), 1594-1609.

J.I. Suárez, B. M. Vinagre, Y. Q. Chen, A fractional adaptationscheme for lateral control of an AGV, Journal of Vibration and Control14.9-10 (2008): 1499-1511.


F. Mainardi, Fractional Calculus and Waves in Linear Viscoelasticity:An Introduction to Mathematical Models, World Scientic, Singapore,2010.

B. D. A. Anderson, J. B. Moore, Optimal control: linear quadraticmethods, Prentice Hall, Englewood Cli¤s, 1990.

M. Athans, P. L.Falb, Optimal Control: An Introduction to the Theoryand Its Applications, Dover Publications, New York, 2006.

V. Dragan, T. Morozan, A. Stoica, Mathematical Methods in RobustControl of Discrete Time Linear Stochastic Systems, Springer, NewYork, 2013.

O. L.V. Costa, M. D. Fragoso, M. G. Todorov, Continuous-TimeMarkov Jump Linear Systems, Springer Berlin Heidelberg, 2013.

V. M. Ungureanu, Stability, stabilizability and detectability for Markovjump discrete-time linear systems with multiplicative noise in Hilbertspaces, Optimization 63(11)(2014),1689-1712.


C. Tricaud, Y. Q. Chen. An approximate method for numericallysolving fractional order optimal control problems of general form,Computers & Mathematics with Applications 59(5)(2010), 1644-1655.

O.P. Agrawal, A general formulation and solution scheme for fractionaloptimal control problems, Nonlinear Dynamics, 38, 1(2004), 323337.

O.P. Agrawal, A quadratic numerical scheme for fractional optimalcontrol problems, Journal of Dynamic Systems, Measurement, andControl 130(1)(2008),1-6.

G. Idiri, S. Djennounet, M. Bettayeb, Solving fractional optimalcontrol problems using control vector parameterization, Control,Decision and Information Technologies (CoDIT), 2013 InternationalConference on. IEEE, 2013, 555-560.

R. Kamocki, M. Majewski, Fractional linear control systems withCaputo derivative and their optimization, Optimal ControlApplications and Methods 36(6)(2015), 953-967.


A. Dzielinski,P. M. Czyronis, Dynamic Programming for FractionalDiscrete-time Systems, 19th World Congress of IFAC,19(1)(2014),2003-2009.

H. Sadeghian, et al. On the general Kalman lter for discrete timestochastic fractional systems, Mechatronics 23(7)(2013), 764-771.

A. Amirdjanova, S. Chivoret, New method for optimal nonlinearltering of noisy observations by multiple stochastic fractional integralexpansions, Computers & Mathematics with Applications 52(1)(2006),161-178.

V. M. Ungureanu, V. Dragan, T. Morozan, Global solutions of a classof discrete-time backward nonlinear equations on ordered Banachspaces with applications to Riccati equations of stochastic control,Optimal Control Applications and Methods, 34(2)(2013), 164-190.


Documents

Optimal control for discrete -time fractional order ...obzor.bio21.bas.bg/conference/Conference_files/sa19/Ungureanu.pdf · Optimal Control for Discrete- Time Fractional-Order Systems