Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2019-11-06 · and initial instant and state fixed x f x,u,t C R U R t f x f x t Rn u t U R p f n u u w w

Optimal Control

Prof. Daniela Iacoviello

Lecture 4

06/11/2019 Pagina 2

Systems Identification and Optimal Control

Optimal control:

Period: September - December

Credits: 6

Professor: Daniela Iacoviello

System Identification:

Period: February- May

Credits: 6

Professor: Stefano Battilotti

Pagina 206/11/2019Prof.Daniela Iacoviello- Optimal Control

06/11/2019 Pagina 3

Systems Identification and Optimal Control

Optimal control:

Period: September - December

Credits: 6

Professor: Daniela Iacoviello

System Identification:

Period: February- May

Credits: 6

Professor: Stefano Battilotti


06/11/2019 Pagina 4

Prof. Daniela Iacoviello

Department of computer, control and management

engineering Antonio Ruberti

Office: A219 Via Ariosto 25

http://www.dis.uniroma1.it/~iacoviel

Prof.Daniela Iacoviello- Optimal Control Pagina 406/11/2019

06/11/2019 Pagina 5Prof.Daniela Iacoviello- Optimal Control Pagina 506/11/2019

Mailing list:

Send an e-mail to: [email protected]

Subject: Optimal Control 2019

mailto:[email protected]

06/11/2019 Pagina 6

Grading

Project+ oral exam

Example of project (1-3 Students):

-Read a paper on an optimal control problem

-Study: background, motivations, model, optimal control, solution, results

-Simulations

-Conclusions

-References

You must give me, before the date of the exam (about a week):

-A .doc document

-A power point presentation

-Matlab simulation files

Oral exam: Discussion of the project AND on the topics of the lectures


06/11/2019 Pagina 7

Some projects proposed

Application Of Optimal Control To Malaria: Strategies And Simulations

Performance Compare Between Lqr And Pid Control Of Dc Motor

Optimal Low-thrust Leo (Low-earth Orbit) To Geo (Geosynchronous-earth Orbit)

Circular Orbit Transfer

Controllo Ottimo Di Una Turbina Eolica A Velocità Variabile Attraverso Il Metodo

dell'inseguimento Ottimo A Regime Permanente

Optimalcontrol In Dielectrophoresis

On The Design Of P.I.D. Controllers Using Optimal Linear Regulator Theory

Rocket Railroad Car

Optimal Control Of Quadrotor Altitude Using Linear Quadratic Regulator

Optimal Control Of An Inverted Pendulum

Glucose Optimal Control System In Diabetes Treatment ………


06/11/2019 Pagina 8

Optimal Control Of Shell And Tube Heat Exchanger

Optimal Control Analysis Of A Mathematical Model For Unemployment

Time optimal control of an automatic Cableway

Glucose Optimal Control System In Diabetes Treatment

Optimal Control Of Shell And Tube Heat Exchanger

Optimal Control Analysis Of A Mathematical Model For Unemployment

Time Optimal Control Of An Automatic Cableway

Optimal Control Project On Arduino Managed Module For Automatic Ventilation Of

Vehicle Interiors

Optimal Control For A Suspension Of A Quarter Car Model

………

Pagina 806/11/2019

Some projects proposed

Prof.Daniela Iacoviello- Optimal Control

06/11/2019 Pagina 9Pagina 906/11/2019

Example


06/11/2019 Pagina 10Pagina 1006/11/2019

Example


06/11/2019 Pagina 11

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below


06/11/2019 Pagina 12

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below


06/11/2019 Pagina 13

References

B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989

E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,

1993

C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993

A.Calogero, Notes on optimal control theory, 2017

L. Evans, An introduction to mathematical optimal control theory, 1983

H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972

D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004

D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise

Introduction", Princeton University Press, 2011

How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:

Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.

……………………………………..


06/11/2019 Pagina 14

Course outline

• Introduction to optimal control

• Calculus of variations

• Calculus of variations and optimal control

• The maximum principle

• The Hamilton Jacobi Bellman equation

• The linear quadratic regulator

• The minimum time problem

• The linear quadratic gaussian problem


06/11/2019 Pagina 15Pagina 1506/11/2019Prof.Daniela Iacoviello- Optimal Control

Calculus of

variations

Calculus of

variations

and

Optimal

control

The Pontryagin

principle

The LQ Problem

The Minimum time problem

The LQG problem


Calculus of

variations

Calculus of

variations

and

Optimal

control

The Pontryagin

principle

The LQ Problem


The LQG problem


Calculus of

variations

Calculus of

variations

and

Optimal

control

The Pontryagin

principle

The LQ Problem


The LQG problem

R.F.Hartl, S.P.Sethi. R.G.Vickson, A Survey of the

Maximum Principles for Optimal Control Problems with

State Constraints, SIAM Review, Vol.37, No.2,

pp.181-218, 1995


Pontryagin

Lev Semenovich Pontryagin (3 September 1908 – 3

May 1988) was a Soviet Russian mathematician. He was

born in Moscow and

lost his eyesight in a stove explosion when he was 14.

Despite his blindness he was able to

become a mathematician due

to the help of his mother who read

mathematical books and papers to him.

He made major discoveries in a number of fields of

mathematics, including the geometric parts of topology.


Principle of optimality

Prof. D.Iacoviello - Optimal Control 2006/11/2019










Minimizing over [t, tf] is equivalent to minimizing over [t, t1]

and [t1, tf]

t t1 tf

x(t)

xx1(t1)

x2(t1)

x3(t1)

x1(tf)

x2(tf)

x3(tf)


Principle of optimality:

OR:

( )

( )

( )

++

= )((),(),(min),(),(min),(

1

1

1

1 ,,

*f

t

tttu

t

tttu

txGduxLduxLttxJ

f

f

( )

( ) ( )

+= 11

*

,

* ),(),(),(min),(

1

1

ttxJduxLttxJ

t

tttu


•

Let us define the

Function.

If is an optimal solution for the control problem


It means that if a solution is optimal then it is optimal in any subinterval

Proof:

By contradiction: assume that the relation

is not true.


There exists a control and the state defined in

such that

I can define a new solution of the optimal control problem:


Contradiction!!!


The optimality principle:

Hamilton- Jacobi equation

The Pontryagin principle

Problem 1 : Consider the dynamical system:

with:

and initial instant and state fixed

( )tuxfx ,,=

( )RURCt

f

x

ffRUtuRtx npn

0,,)(,)(

ii xtx =)(


For the final values assume: where is a function

of dimension of C1 class.

Assume the constraint with

Define the performance index :

with

( ) 0),( = ff ttx

1+ nf

( ) kduxh

f

i

t

t

= ),(),(

( ) niRURCt

h

tx

hh n ,...,2,1,,

)(, 0 =

( ) ( )=f

i

t

t

f duxLtuxJ ),(),(,,( )RURC

t

L

x

LL n

0,,

Determine

• the value

• the control

• and the state

that satisfy

✓ the dynamical system,

✓ the constraint on the control,

✓ the initial and final conditions

✓ and minimize the cost index .

( ) ,if tt

)(0 RCuo

)(1 RCxo


Hamiltonian function

( ) ( ) ( ) )),(),((,)(,,,, 00 ttutxhuxftuxLuxH TT ++=


Theorem 1 (necessary condition):

Consider an admissible solution such that

IF it is a local minimum

there exist a constant , ,

not simultaneously null such that :

( )*** ,, ftux

00 *1* , fi ttC

( ) f

ff ttxrank =

*

),(


,

**

T

x

H

−=

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Moreover there exists a vectorsuch that:fR

T

ft

T

f tH

txT

f

*

*

*

**

)()(

−=

=

R*

The discontinuities of may occur only in the instants

in which u has a discontinuity and in these instants the

Hamiltonian is continuous.

*


Proof: (some ideas) the necessary condition to be proven is

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*


Definition of C

They do not depend on the control u


Definition of C


Let us compare this espression with the Hamilton- Jacobi equation

Let us choose


Let us define

Remark

If the set U coincides with Rp the minimum

condition reduces to :

0=

u

H


The Pontryagin principle - convex case

Problem 2: Consider the dynamical linear system:

with A and B function of C1 class; assume fixed the initial

and final instants and the initial state, and

Assume

where U is a convex set.

utBxtAx )()( +=

nfff Rtxorfixedxtx = )()(

fip tttRUtu ,)(



with

L convex function with respect to x(t), u(t) in

per ogni

G is a scalar function of C2 class and convex with

respect to x(tf).

( ) ( ) ( ))(),(),(, f

t

t

txGduxLuxJ

f

i

+=

( ) nittURCt

L

x

LL fi

n

i

,...,2,1,,,, 0 =

URn

fi ttt ,


Determine:

the control

and the state

that satisfy

the dynamical system,

the constraint on the control,

the initial and final conditions

and minimize the cost index .

fio ttCu ,0

fio ttCx ,1


Theorem 2 (necessary and sufficient condition):


It is a minimum normal (i.e.λ0 =1)

if and only if there exists an n-dimensional vector

such that :

( )oo ux ,

fio ttC ,1

( ) f

o

ff ttxrank =

),(


oTo

x

tuxH

−=

),,,(

( ) ( ) UttutxHttxH ooooo ,)(),(),()(,),(

nf Rtx )(

oT

f

fo

tdx

dGt

)()( =Moreover, if

Some special cases

06/11/2019 Pagina 45


Problem 3: Consider the dynamical system:

with:

Assume fixed the initial control instant and the initial and final

values :


with

( )uxfx ,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =


ff

ii xtxxtx == )()(

( ) ( ) ))(()(),(,, f

t

t

f txGduxLtuxJ

f

i

+=

( ) 20 ,,...,2,1,, CGniURCx

LL n

i

=



with:

Assume fixed the initial control instant and the initial and final

values :


with

( )uxfx ,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =


ff

ii xtxxtx == )()(

( ) ( ) ))(()(),(,, f

t

t

f txGduxLtuxJ

f

i

+=

( ) 20 ,,...,2,1,, CGniURCx

LL n

i

=

Determine:

▪ the value ,

• the control

• the state

that satisfy:




✓ and minimize the cost index

( ) ,if tt

)(0 RCuo

)(1 RCxo


Hamiltonian function

( ) ( ) ( )uxftuxLuxH T ,)(,,,, 00 +=




Assume the admissible solution is a minimum

there exist a constant

and a n-dimensional vector


( )*** ,, ftux

00

*1* , fi ttC

T

x

H*

*

−=

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **0

****0

*


0*=H


Remark- If is fixed, the condition

is substituted by


0*=H

ft

fi tttkH ,,*

=



with:

( )uxfx ,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =


Assume fixed the initial control instant and the initial state

while for final values assume:

where is a function of dimension of C1 class.


with

ii xtx =)(

( ) 0)( = ftx

nf

( ) 20 ,,...,2,1,, CGniURCx

LL n

i

=

( ) ( ) ))(()(),(,, f

t

t

f txGduxLtuxJ

f

i

+=

Determine:

▪ the value

▪ the control

▪ and the state

that satisfy




✓ and minimize the cost index .

( ) ,if tt

)(0 RCuo

)(1 RCxo




If it is a minimum

there exist a constant and an n-dimensional vector


Moreover there exists a vector such that:

( )*** ,, ftux

00

*1* , fi ttC

fftdx

drank =

*

)(


T

x

H*

*

−=

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **0

****0

*

T

ff

tdx

dt

*

)()(

=

fR

0*=H


Remark- If is fixed condition

Is substituted by


0*=H

ft

fi tttkH ,,*

=



with:

( )tuxfx ,,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =


Assume fixed the initial control instant and the initial state

while for final values assume: where is a

function of dimension of C1 class.


with

ii xtx =)(

( ) 0),( = ff ttx 1+ nf

( ) ( )=f

i

t

t

f duxLtuxJ ),(),(,,

( ) niRURCt

L

x

LL n

i

,...,2,1,,, 0 =

Determine:

▪ the value

▪ the control

▪ and the state

that satisfy

the dynamical system,

the constraint on the control,

the initial and final conditions

and minimize the cost index .

( ) ,if tt

)(0 RCuo

)(1 RCxo


Theorem 5: Consider an admissible solution such that

IF it is a minimum there exist a constant and an

n-dimensional vector not simultaneously null

such that :

( )*** ,, ftux

00

*1* , fi ttC

( ) fff ttx

rank =

*

),(


,

**

T

x

H

−= Rkkd

HH

ft

t

=

+ ,

*

**

( ) ( ) UttutxHttxH ,)(,),(),()(,,),( **0

****0

*

Moreover there exists a vector such that: fR

T

ft

T

f

ft

Htx

tf

*

*

*

**

)()(

−=

=

Remark: If in the cost index a Bolza term is present:


T

ft

T

f

ft

Htx

tf

*

*

*

**

)()(

−=

=

( ) ( )=f

i

t

t

f duxLtuxJ ),(),(,,

In the necessary conditions the following modifications are required:

Example 3 (from L.C.Evans)

Control of production and consumption

Consider a factory whose output can be controlled.

Let’s set:

x(t) the amount of output produced at time t , 0 ≤ t.

Assume we consume some fraction of the output at each time

and likewise reinvest the remaining fraction u(t) .

It is our control, subject to the constraint

0 ≤ u(t) ≤ 1


The corresponding dynamics are:

The positive constant k represents the growth rate of our

reinvestment. We will chose K=1.

Assume as cost index the function:

The aim is to maximize the total consumption of the

output

0)0(

0),()()(

xx

ktxtkutx

=

=

( ) −=

ft

dttxtuuJ

0

)()(1))((


choose k=1

We apply the Pontryagin Principle; the Hamiltonian in the

normal case is:

The necessary conditions are:

( ) ( ) xuxuuxH +−= 1,,

( )

( ) ( ) 1)()()()(max)(),(),(

)()()(

0)(1)()(1)(

10−+=

=

=−−−=

ttxtutxttutxH

txtutx

tttut

u

f


1,0),()()()()()()()()()()()( −+−+ txtttxttxtxtuttxtutx

1,0,1)()(1)()( −− ttttu since x(t)>0


=

1)(0

1)(1)(

tif

tiftu

1,0,1)()(1)()( −− ttttu

1 0( )

0

s

s

if t tu t

if t t T

=

where ts is the switching time

From the equation of the costate, since ,

by continuity we deduce for t<tf, t close to tf, that

thus for such values of t.

Therefore and consequently:

More precisely so long as

and this holds for:

0)( =ft

0)( =tu

1)( t

1)( −=t ttt f −=)(

1)( t

ff ttt −1

ttt f −=)(


For times with t near tf -1 we have

Therefore the costate equation yields:

1− ftt 1)( =tu

( ) )(1)(1)( ttt −=−−−=

Since we have for all

and over this time interval there are no switchings

1)1( =−ft 1− ftt

1)(1

=−− tt fet


Therefore:

For the switching time

Homework: find the switching time

=

fs

s

tttif

ttiftu

0

01)(*

1−= fs tt

st ft

Optimal solution: we should reinvest all

the output

(and therefore consume nothing)

up to time ts and afterwards we should

consume everything

(and therefore reinvest nothing)

Bang-bang control


Documents

Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2019-11-06 · and initial instant and state fixed x f x,u,t C R U R t f x f x t Rn u t U R p f n u u w w