Classical Numerical Methods to Solve Optimal Control … 26.pdf · Lecture – 26 Classical Numerical Methods to Solve Optimal Control Problems Dr. Radhakant Padhi Asst. Professor

Lecture – 26

Classical Numerical Methods to Solve Optimal Control Problems

Dr. Radhakant PadhiAsst. Professor

Dept. of Aerospace EngineeringIndian Institute of Science - Bangalore

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

2

Necessary Conditions of Optimality in Optimal Control

State Equation

Costate Equation

Optimal Control Equation

Boundary Condition

( ), ,HX f t X Uλ

∂= =∂

( ), ,H g t X UX

λ ∂⎛ ⎞= − =⎜ ⎟∂⎝ ⎠

ffX

ϕλ ∂=∂ ( )0 0 :FixedX t X=

( )0 ,H U XU

ψ λ∂⎛ ⎞= ⇒ =⎜ ⎟∂⎝ ⎠


3

Necessary Conditions of Optimality: Salient Features

State and Costate equations are dynamic equations

State equation develops forward whereas Costate equation develops backwards

Optimal control equation is a stationary equation

The formulation leads to Two-Point-Boundary-Value Problems (TPBVPs), which demand computationally-intensive iterative numerical procedures to obtain the optimal control solution


4

Classical Methods to Solve TPBVPs

Gradient Method

Shooting Method

Quasi-Linearization Method

Gradient Method




6

Gradient Method

Assumptions:• State equation satisfied

• Costate equation satisfied

• Boundary conditions satisfied

Strategy:• Satisfy the optimal control equation


7

Gradient Method

( )

( )

( )

( )

0

0

0

f

f

f

T

f ff

tT

t

tT

t

tT

t

J XX

HX dtX

HU dtU

H X dt

φδ δ λ

δ λ

δ

δλλ

⎡ ⎤∂= −⎢ ⎥

∂⎢ ⎥⎣ ⎦

∂⎡ ⎤+ +⎢ ⎥∂⎣ ⎦

∂⎡ ⎤+ ⎢ ⎥∂⎣ ⎦

∂⎡ ⎤+ −⎢ ⎥∂⎣ ⎦

∫

∫

∫


8

Gradient Method

After satisfying the state & costate equations and boundary conditions, we have

( )0

ftT

t

HJ U dtU

δ δ ∂⎡ ⎤= ⎢ ⎥∂⎣ ⎦∫

Select ( ) , 0HU tU

δ τ τ∂⎡ ⎤= − >⎢ ⎥∂⎣ ⎦

0

ft T

t

H HJ dtU U

δ τ ∂ ∂⎡ ⎤ ⎡ ⎤= − ⎢ ⎥ ⎢ ⎥∂ ∂⎣ ⎦ ⎣ ⎦∫This leads to


9

Gradient Method

We select

This lead to

Note:

Eventually,

( ) ( ) ( )1i

i i i HU t U t U tU

δ τ+ ∂⎡ ⎤⎡ ⎤= − = −⎣ ⎦ ⎢ ⎥∂⎣ ⎦

( ) ( )1i

i i HU t U tU

τ+ ∂⎡ ⎤= − ⎢ ⎥∂⎣ ⎦

0

0ft T

t

H HJ dtU U

δ τ ∂ ∂⎡ ⎤ ⎡ ⎤= − ≤⎢ ⎥ ⎢ ⎥∂ ∂⎣ ⎦ ⎣ ⎦∫

0 0HJU

δ ∂= ⇒ =

∂


10

Gradient Method: Procedure

Assume a control history (not a trivial task)

Integrate the state equation forward

Integrate the costate equation backward

Update the control solution• This can either be done at each step while integrating

the costate equation backward or after the integration of the costate equation is complete

Repeat the procedure until convergence

0

(a pre-selected constant)ft T

t

H H dtU U

γ∂ ∂⎡ ⎤ ⎡ ⎤ ≤⎢ ⎥ ⎢ ⎥∂ ∂⎣ ⎦ ⎣ ⎦∫


11

Gradient Method: Selection of

Select so that it leads to a certain percentage reduction of Let the percentage be Then

This leads to

ττ

Jα

0100

ft T

t

H H dt JU U

ατ ∂ ∂⎡ ⎤ ⎡ ⎤ =⎢ ⎥ ⎢ ⎥∂ ∂⎣ ⎦ ⎣ ⎦∫

0

100ft T

t

J

H H dtU U

α

τ =∂ ∂⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥∂ ∂⎣ ⎦ ⎣ ⎦∫


12

Objective:Air-to-air missiles are usually launched from an aircraft in the forward direction. However, the missile should turn around and intercept a target “behind the aircraft”.

To execute this task, the missile should turn around by -180o and lock onto its target (after that it can be guided by its own homing guidance logic). Note: Every other case can be considered as a subset of this extreme scenario!

A Real-Life Challenging Problem

Aircraft

Missile

Target


13

MATHEMATICAL PERSPECTIVE:• Minimum time optimization problem• Fixed initial conditions and free final time problem

SYSTEM DYNAMICS:Equations of motion for a missile in vertical plane. The non-dimensional equations of motion (point mass) in a vertical plane are:

2

2

' sin( ) cos( )1 ' [ sin( ) cos( )]

where prime denotes differentiation with respect to the non-dimensional time

w D w

w L w

M S M C T

S M C TM

γ α

γ α γ

τ

= − − +

= + −



14

2

The non-dimensional parameters are defined as follows:

; ; ;2

where flight Mach number flight path angle thrust mass of the

w wg T a S VT S Mat mg mg a

MT

m

ρτ

γ

= = = =

== == missile reference aerodynamic area

speed of the missile lift coefficient drag coefficient the acceleration due to gr

L

D

SV CC g

== == = avity

the local speed of sound the atmospheric density flight time after launchNOTE: , are usually functions of & (tabulated data)L D

atC C M

ρ

α

= ==



15

0

COST FUNCTION:Mathematically the problem is possed as follows to find thecontrol minimizing cost function:

Constraints (0) 0 , (0) initial Mach numbe

ft

o

J dt

Mγ

=

= =

∫r

( ) 180 , ( ) 0.8of ft M tγ = − =



16

( )2

2

2

Choosing as the independent variable the equations are reformulated as follows:

sin( ) cos( )

cos( ) sin( )

cos

w D w

w L w

w L

S M C T MdMd S M C Tdt a Md g S M C

γ

γ α

γ γ α

γ

− − +=

− +

=−( )

( )20 0 0

( ) sin( )

and the transformed cost function is

cos( ) sin( )

A difficult minimum-time problem has been converted to a relatively

easier fix

f

w

t

w L w

T

dt a MJ dt d dd g S M C T

π π

γ α

γ γγ γ α

− −

+

= = =− +∫ ∫ ∫

ed final-time problem (with hard constraint: ( ) 0.8)!fM γ⎛ ⎞⎜ ⎟=⎝ ⎠



17

TaskSolve the problem using gradient method. Assume (0) 0.5 and engagement height as 5 km. Next, generate the trajectories and tabulate the values of for various values.

Use the following system parf

M

M q

=

ameters (typical for an air-to-air missile):

240 0.0707 2 24,000

0.5 3.12

Use standard atmosphere chart for the atmospheric data.

D

L

m kgS mT NCC

=====

Shooting Method




19

Necessary Conditions of Optimality (TPBVP): A Summary

State Equation

Costate Equation

Optimal Control Equation

Boundary Condition

( ), ,HX f t X Uλ

∂= =∂

( ), , ,H g t X UX

λ λ∂⎛ ⎞= − =⎜ ⎟∂⎝ ⎠

0HU∂

=∂

ffX

ϕλ ∂=∂ ( )0 0 :FixedX t X=


20

Shooting Method


21

Shooting Method


22

Shooting Method


23

Shooting Method


24

Shooting Method

Quasi-Linearization Method




26

Quasi-Linearization MethodProblem:

( )( ) ( )

{ }

Differential Equation: , ,

Boundary condition: ,

, 1, ,

TT T

Ti i i i i

i

Z F Z t Z X

C t Z t C Z b

t t i n

λ⎡ ⎤= ⎣ ⎦= =

∈ ∈ …

Assumption:

0This vector differential equation has a unique solution over , ft t t⎡ ⎤∈ ⎣ ⎦

Trick:The nonlinear multi-point boundary value problem is transformed intoa sequence of linear non-stationary boundary value problems, the solutionof which is made to approximate the solution of the true problem.


27

Quasi-Linearization Method( ) ( )

( )

(1) Guess an approximate solution 1 (it need not satisfy the B.C.)

For updating this solution, proceed with the following steps:

(2) Linearize the system dynamics about

N

N

N

Z t N

Z t

FZZ

=

∂⎡Δ = ⎢∂⎣( )

( ) ( ) ( )

( )( )

( ) ( ) ( ) ( ) ( )( ) ( )

1

To be found

1

1

, where,

(3) Enforce the boundary with respect to the updated solution

, ,

,

N

N N N N

Z

A t

N N

N

N N Ni i i i i i

Ni i

Z Z t Z t Z t

Z A t Z

Z t

C t Z t C t Z t Z t b

C t Z t C t

+

+

+

⎤ Δ Δ −⎥⎦

Δ = Δ

= + Δ =

Δ = − ( ) ( ), Ni i iZ t b+ Philosophy: Solve this linear system

and update the solution!


28

Quasi-Linearization Method:Solution by STM Approach

( ) ( )( ) ( ) ( )

( )

1 1

1

Homogeneous Forcing function

1

(1) From the linearized system dynamics, we can write

,

(2) The solution to the above equa

N N N N

N N N

N

Z Z A t Z Z

A t Z F Z t A t Z

Z t

+ +

+

+

= + −

⎡ ⎤= + −⎣ ⎦

( ) ( ) ( ) ( )

( )

1 1 1 10 0

State transition matrix (STM) Particular solution

10

tion is given by

,

(3) The solution for STM , can be obtained from the fact that it satisfies

N N N N

N

Z t t t Z t p t

t t

+ + + +

+

= Φ +

Φ

( ) ( ) ( )

( )

1 10 0

10 0

the following differential equation and boundary conditions

, ,

,

N N

N

t t A t t tt

t t I

+ +

+

∂ ⎡ ⎤Φ = Φ⎣ ⎦∂Φ =


29


( )

( )

1

1

(4) The particular solution can be obtained by observing that it satisfies the the following differential equation and boundary condition

Substituting the complete solution i

N

N

p t

Z t

+

+

( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( )

( ) ( ) ( )

1 1 1 1 1 10 0 0 0

1 1

n the original equation

, ,

,

N N N N N N

N N

N N

t t Z t p t A t t t Z t p tt

F Z t A t Z

p t A t p t F

+ + + + + +

+ +

∂ ⎡ ⎤ ⎡ ⎤Φ + = Φ +⎣ ⎦ ⎣ ⎦∂⎡ ⎤+ −⎣ ⎦

= + ( ) ( )( )

( ) ( ) ( ) ( )

( )

10

1 1 1 10 0 0 0 0

I1

0

,

(5) The boundary condition can be obtained by observing that

,

0

N N

N

N N N N

N

Z t A t Z

p t

Z t t t Z t p t

p t

+

+ + + +

+

⎡ ⎤−⎣ ⎦

= Φ +

=


30


( )( ) ( )

( ) ( ) ( ) ( )( ) ( ) ( ) ( )

10

1

1 1 10 0

1 10 0

(6) The boundary condition can be obtained as follows

,

, ,

, , ,

N

Ni i i

N N Ni i i i

N Ni i i

Z t

C t Z t b

C t t t Z t p t b

C t t t Z t C t p

+

+

+ + +

+ +

=

Φ + =

Φ = − ( )

( )

( ) ( )( ) ( ) ( ) ( )

1

10

1 10

1 1 1 10 0

STM Particular solution

Solve the above system to obtain

Once is determined, the solution is available from the

STM solution: ,

Ni i

N

N N

N N N N

t b

Z t

Z t Z t

Z t t t Z t p t

+

+

+ +

+ + + +

+

= Φ +


31

Quasi-Linearization Method:Convergence Property

( ){ }0

1

Under the assumption that the problem admits a unique solution for ,

it can be shown that the sequence of vectors converge to the true solution.

Morover, the process can be shown to have

f

N

t t t

Z t+

⎡ ⎤∈ ⎣ ⎦

( ) ( ) ( ) ( ) ( )1 1

"quadratic convergence" in general

i.e., it can be shown that , where .

Further more, for a large class of systems, it can be shown to have "monotone convergence" as well, i.e.

N N N NZ t Z t k Z t Z t k f N+ −− ≤ − ≠

there won't be any over-shooting in the convergence process.

R. Kabala, "On Nonlinear Differential Equations, The Maximum Operation and Monotone Convergence", J. of MathemReference :

atics and Mechanics, Vol. 8, 1959, pp. 519-574.


32

A Demonstrative ExampleProblem: ( ) ( )

12 2 2

0

1Minimize for the system , 0 10.2

J x u dt x x u x= + = − + =∫Solution:

( ) ( )

( )

2 2 2

2

1Hamiltonian: 2

1) State Equation: 2) Optimal Control Equation: 03) Costate Equation: / 2

4) Bounary Conditions:

H x u x u

x x uu u

H x x x

λ

λ λ

λ λ

= + + − +

= − ++ = ⇒ = −

= − ∂ ∂ = − +

( ) ( ) ( )

( )( )

2

0 10, 1 / 0Substituting the expression for in the state equation, we can write , 0 10

2 , 1 0 Solv

x xu

x x x

x x

λ

λ

λ λ λ

= = ∂Φ ∂ =

= − − =

= − + =

Task : e this problem using shooting and quasi-linearization methods.


33

References on Numerical Methods in Optimal Control Design

D. E. Kirk, Optimal Control Theory: An Introduction, Prentice Hall, 1970.

S. M. Roberts and J.S. Shipman, Two Point Boundary Value Problems: Shooting Methods, American Elsevier Publishing Company Inc., 1972.

A. E. Bryson and Y-C Ho, Applied Optimal Control, Taylor and Francis, 1975.

A. P. Sage and C. C. White III, Optimum Systems Control (2nd Ed.), Prentice Hall, 1977.


34

Survey of Classical Methods

M. Athans (1966), “The Status of Optimal Control Theory and Applications for Deterministic Systems”, IEEE Trans. on Automatic Control, Vol. AC-11, July 1966, pp.580-596.

H. J. Pesch (1994), “A Practical Guide to the Solution of Real-Life Optimal Control Problems”, Control and Cybernetics, Vol.23, No.1/2, 1994, pp.7-60.


35

Documents

Classical Numerical Methods to Solve Optimal Control … 26.pdf · Lecture – 26 Classical Numerical Methods to Solve Optimal Control Problems Dr. Radhakant Padhi Asst. Professor