Optimal control

T. F. EdgarSpring 2012

2

Optimal Control• Static optimization (finite dimensions)

• Calculus of variations (infinite dimensions)

• Maximum principle (Pontryagin) / minimum principle

Based on state space models

Min

S.t.

is given

General nonlinear control problem

3

Special Case of

• Minimum fuel:

• Minimum time:

• Max range :

• Quadratic loss:

Analytical solution if state equation is linear, i.e.,

4

“Linear Quadratic” problem - LQP• Note is not solvable in a realistic sense ( is

unbounded), thus need control weighting in • E.g., • is a tuning parameter (affects overshoot)

5

Ex. Maximize conversion in exit of tubular reactor

: Concentration

: Residence time parameter

In other cases, when and are deviation variables,

Objective function does not directly relate to profit

(See T. F. Edgar paper in Comp. Chem. Eng., Vol 29, 41 (2004))

6

• Initial conditions

(a) , or

Set point change, is the desired

(b) , impulse disturbance,

(c) , model includes disturbance term

7

Other considerations: “open loop” vs. “closed loop”

• “open loop”: optimal control is an explicit function of time,

depends on -- “programmed control”

• “closed loop”: feedback control, depends on , but not on .

e.g.,

Feedback control is advantageous in presence of noise,

model errors.

Optimal feedback control arises from a specific optimal

control problems, the LQP.

8

Derivation of Minimum Principle

,

have continuous 1st partial w.r.t.

Form Lagrangian

Multipliers: adjoint variables, costates

9

• Define (Hamiltonian)

()

• Since is Lagrangian, we treat as unconstrained problem with variables: , ,

• Use variations: , , (for => original constraint, the state equation.)

10

• Since , are arbitrary (), then

(n equations. “adjoint equation”)

, “optimality equation” for weak minimum

, (n boundary conditions)

If is specified, then

Two point boundary value problem (“TPBVP”)

11

• Example:

(1st order transfer function)

LQP

,

(but don’t know yet)

12

• Free canonical equations (eliminate )

(1) ( is known)

(2) ,

Combine (1) and (2),

for , initially correct to reduce

13

• Another example:

(double integrator)

14

• Free canonical equations

(, coupled)

Char. Equation:

(4 roots, apply boundary condition)

15

• Can motivate feedback control via discrete time, one step ahead

Set , ( fixed)

Feedback control

16

Continuous Time LQP

,

,

()

17

• Free canonical equations

( given)

( given)

Let (Riccati transformation)

, let (feedback control)

Then we have ODE in

(1)

(2)

18

Substitute Eq. (1) into Eq. (2):

(Riccati ODE)

( backward time integration)

At steady state, for , solve steady state equation.

is symmetric,

19

• Example

,

, ,

Plug into Riccati Equation (Steady state)

Feedback Matrix:

20

• Generally 3 ways to solve steady state Riccati Equation:

(1) integration of ode’s steady state;(2) Newton-Raphson (non linear equation solver);(3) transition matrix (analytical solution).

21

• Transition matrix approach

Reverse time integration (Boundary Condition: at ):

Let

When ,

Partition exponential

22

(1)

(2)

Combine (1) and (2), factor out

Fix integration , is fixed

Boundary condition:

Backward time integration of , then forward time integration

23

Integral Action (eliminate offset)

• Add terms or to objective function

Example:

Augment state equation

(new state variable)

(new control variable)

Calculate feedback control

Integrate:

24

• Second method:

;

Optimal control:

With more state variables, PID controller