Linear Quadratic Regulator (LQR) Linear Quadratic ... · Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) ––––IIIIIIII Prof. Radhakant Padhi Dept. of Aerospace

Lecture Lecture Lecture Lecture –––– 11111111

Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) –––– IIIIIIII

Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi

Dept. of Aerospace Engineering

Indian Institute of Science - Bangalore

Optimal Control, Guidance and Estimation

OPTIMAL CONTROL, GUIDANCE AND ESTIMATION

Prof. Radhakant Padhi, AE Dept., IISc-Bangalore2

Outline

� Summary of LQR design

� Stability of closed-loop system in LQR

� Optimum value of the cost function

� Extension of LQR design• For cross-product term in cost function

• Rate of state minimization

• Rate of control minimization

• LQR design with prescribed degree of stability

� LQR for command tracking

� LQR for inhomogeneous systems

Summary of LQR DesignSummary of LQR DesignSummary of LQR DesignSummary of LQR Design






LQR Design:

Problem Statement

� Performance Index (to minimize):

� Path Constraint:

� Boundary Conditions:

( )( )

( )( )

0

,

1 1

2 2

f

f

t

T T T

f f f

t

L X UX

J X S X X Q X U RU dt

ϕ

= + +∫��

X A X BU= +ɺ

( )

( )00 :Specified

: Fixed, : Freef f

X X

t X t

=



LQR Design:Necessary Conditions of Optimality

� Terminal penalty:

� Hamiltonian:

� State Equation:

� Costate Equation:

� Optimal Control Eq.:

� Boundary Condition:

( ) ( )1

2

T T TH X Q X U RU AX BUλ= + + +

( ) ( )1

2

T

f f f fX X S Xϕ =

X AX BU= +ɺ

( ) ( )/T

H X QX Aλ λ= − ∂ ∂ = − +ɺ

( ) 1/ 0 TH U U R B λ−∂ ∂ = ⇒ = −

( )/f f f fX S Xλ ϕ= ∂ ∂ =



LQR Design:

Derivation of Riccati Equation

Guess ( ) ( ) ( )t P t X tλ =

( )

( )

( )

1

1

T

T

PX PX

PX P AX BU

PX P AX BR B

PX P AX BR B PX

λ

λ−

−

= +

= + +

= + −

= + −

ɺ ɺ ɺ

ɺ

ɺ

ɺ

( ) ( )

( )

1

1 0

T T

T T

QX A PX P PA PBR B P X

P PA A P PBR B P Q X

−

−

− + = + −

+ + − + =

ɺ

ɺ



LQR Design:

Derivation of Riccati Equation

� Riccati equation

� Boundary condition

1 0T TP PA A P PBR B P Q

−+ + − + =ɺ

( ) ( ) is freef f f f f

P t X S X X=

( )f fP t S=



LQR Design:

Infinite Time Regulator Problem

Theorem (By Kalman)

Algebraic Riccati Equation (ARE)

Final Control Solution:

1 0T TPA A P PBR B P Q

−+ − + =

As , for constant and matrices, 0ft Q R P t→ ∞ → ∀ɺ

( )1 TU R B P X K X

−= − = −

Stability of closedStability of closedStability of closedStability of closed----loop system in LQRloop system in LQRloop system in LQRloop system in LQR






LQR Design:

Stability of Closed Loop System

� Closed loop system

� Lyapunov function

( )X AX BU A BK X= + = −ɺ

( ) TV X X PX=

( ) ( )

( ) ( )

( )

1 1

1 1

1

T T

T T

TT T T

T T T T

T T

V X PX X PX

A BK X PX X P A BK X

X A BR B P P P A BR B P X

X PA A P PBR B P Q Q PBR B P X

X Q PBR B P X

− −

− −

−

= +

= − + −

= − + −

= + − + − −

= − −

ɺ ɺ ɺ

00



LQR Design:

Stability of Closed Loop System

( )

( )

1

-1

-1

For 0, 0. Also 0

So 0

Also 0.

Hence, 0

0

Hence, the closed loop system is

always asymptotically stable!

T

T

R R P

PBR B P

Q

PBR B P Q

V X

−> > >

>

≥

+ >

∴ <ɺ



LQR Design:

Minimum value of cost function

( )

( ) ( )

( )

( ) [ ]

( )

0

0

0

0 0

0

1 1

1

0 0 0 0

1

2

1

2

1

2

1 1 1

2 2 2

1 1

2 2

T T

t

TT T T

t

T T

t

T

t tt

T T T

J X Q X U RU dt

X Q X R B PX R R B PX dt

X Q PBR B P X dt

V dt V X PX

X PX X PX X PX

∞

∞

− −

∞

−

∞∞∞

∞ ∞

= +

= + − −

= +

= − = − = −

= − =

∫

∫

∫

∫ ɺ

Extensions of LQR DesignExtensions of LQR DesignExtensions of LQR DesignExtensions of LQR Design






LQR Extensions:

1. Cross Product Term in P.I.

( )0

12

2

T T T

tJ X QX X WU U RU dt

∞

= + +∫

Let us consider the expression:

( ) ( ) ( )

( )

1 1 1

2

TT T T T

T T T T T

T T T

X Q WR W X U R W X R U R W X

X QX U RU U W X X WU

X QX X WU U RU

− − −− + + +

= + + +

= + +



( ) ( ) ( )

( )

0

1 1

0

1 1 1

1 1 1

1

2

1

2

TT T T T

t

Q U

T T

t

J X Q WR W X U R W X R U R W X dt

X Q X U RU dt

∞− − −

∞

= − + + +

= +

∫

∫

��

( )

( )

1

1

1

1

1 1

T

T

X AX BU

AX B U R W X

A BR W X BU

A X BU

−

−

= +

= + −

= − +

= +

ɺ

1

Control Solution

U K X= −

( )

1

1

1

T

T

U U R W X

K R W X

−

−

= −

= − +

LQR Extensions:

1. Cross Product Term in P.I.



LQR Extensions:

2. Weightage on Rate of State

( )

( ) ( )

( ) ( ) ( )

0

0

0

1 1

0

1 1

1

2

1

2

1

2

12

2

12

2

T T T

t

TT T

t

T T T T T T

T T T Tt

Q R W

T T T T T T

t

T T

J X QX U RU X SX dt

X QX U RU AX BU S AX BU dt

X QX U RU X A S A X X A SBUdt

U B S A X U B SBU

X Q A SA X U R B SB U X A SB U dt

X Q X U RU

∞

∞

∞

∞

= + +

= + + + +

+ + +=

+ +

= + + + +

= + +

∫

∫

∫

∫

ɺ ɺ

��

( )0

T

tX WU dt

∞

∫Leads to a cross product case



LQR Extensions:

3. Weightage on Rate of Control

( )

( )

0

0

0

1 ˆ2

Let ,

01 ˆ02

1 ˆ ˆ2

T T T

T T

T T

J X QX U RU U RU dt

XV U

U

QJ V RV dt

R

J Q V RV dt

∞

∞

∞

= + +

= =

= +

= +

∫

∫

∫

X

X X

X X

ɺ ɺ

ɺ



LQR Extensions:


( )

{ }

0

ˆˆ

, 0

0ˆ ˆ

0 0

(1) The dimension of the problem has increased

from to ( )

(2) If , is controllable, it can be shown that

the new syste

BA

X AX BU X X

U V

A BV A BV

I

n n m

A B

= + =

=

= + = +

+

X X X

Note :

ɺ

ɺ

ɺ

��

m is also controllable.



[ ]

1

1

11 121 1

12 22

12 22

1 1

12 22

ˆ ˆ ˆ

ˆwhere is the solution of

ˆ ˆˆ ˆ ˆ ˆ ˆ ˆ ˆ 0

Hence

ˆ ˆˆ ˆ ˆ ˆ0

ˆ ˆ

ˆ ˆ ˆ ˆ

T

T T

T

T

T

V U R B P

P

A P PA PBR B P Q

P P XU R I R P P

UP P

R P X R P U

−

−

− −

− −

= = −

+ − + =

= − = −

= − −

Solution :

X

X

ɺ

ɺ

LQR Extensions:




1 1

12 22ˆ ˆ ˆ ˆHowever, is a dynamic

equation in and hence is not easy for implementation.

For this reason, we want an expression in the RHS

only as a function of and operations on it.

State

TU R P X R P U

U

X

− −= − −ɺ

( )( )

equation:

This suggests:

This is only an approximate solution, unless

X AX BU

U B X AX

m n

+

= +

= −

≥Note :

ɺ

ɺ

LQR Extensions:




( )

( )

12

1 1 1

12 22 22

1 1

12 22 22

1 2

1 2 0

0Proportional Initial conditio

Itegral

ˆ ˆ ˆ ˆ ˆ ˆ

ˆ ˆ ˆ ˆ ˆ

Integrating this expression both sides,

T

T

KK

t

U R P X R P B X R P B AX

R P P B A X R P B X

K X K X

U K X K X z dz U

− − + − +

− + − +

= − − −

= − + −

= − −

= − − +∫

ɺ ɺ ɺ

ɺ��

ɺ

��

n

0 can be obtained using a performance index

without the term

U

U

Note :

ɺ

LQR Extensions:




LQR Extensions:

4. Prescribed Degree of Stability

Condition: All the Eiganvalues of the closed loop system

should lie to the left of line AB

( )( )

0

0

0

21 where, 0

2

1

2

1

2

Let

t T T

t

T Tt t t t

t

T T

t

t

t

J e X QX U RU dt

e X Q e X e U R e U dt

X QX U RU dt

X e X

U e U

α

α α α α

α

α

α∞

∞

∞

= + ≥

= +

= +

=

=

∫

∫

∫ ɶ ɶ ɶ ɶ

ɶ

ɶ

Co-ordinate

transformationα

A

B

jω

σ



( )

( ) ( ) ( )

( )

t

t t

t

t t t

X e X e X

e AX BU e X

A e X B e U e X

X A I X BU

α

α α

α

α α α

α

α

α

α

= +

= + +

= + +

= + +

ɺɶ ɺ

ɺɶ ɶ ɶ

LQR Extensions:


t t

U K X

e U K e X

U K X

α α

= −

= −

= −

ɶ ɶControl Solution:



LQR Extensions:


( )

Modified System:

U K X

X A BK I Xα

= −

= − +

ɶ ɶ

ɺɶ ɶ ( )

Actual System:

U K X

X A BK X

= −

= −ɺ

( )

is designed in such a way that eigenvalues of

will lie in the left-half plane.

K

A BK Iα − +

( )Hence, eigenvalues of will lie to the left

of a line parallel to the imaginary axis, which is located

away by distance from the imaginary axis.

A BK

α

−

LQR Design for Command TrackingLQR Design for Command TrackingLQR Design for Command TrackingLQR Design for Command Tracking






LQR Design for

Command Tracking

Problem:

To design such that a part of the state vector of the

linear system tracks a commanded reference signal.

i.e. , where TT c

N

U

X AX BU

XX r X

X

= +

→ =

ɺ

Solution:

1) Formulate a standard LQR problem.

However, select the matrix properly.

2) Implement the controller as T c

N

Q

X rU K

X

−= −

0Typically

0 0

TTQ

Q

=



LQR Design for

Command Tracking

B

A

- K

+ Xɺ

+∫

LQ Regulation

XU

-

B

A

- K

+ Xɺ∫

+

T

N

X

X

( )

0

cr t

+

LQ Tracking

XU



LQR Design for Command

Tracking with Integral Feedback

Solution (with integral controller):

1) Augment the system dynamics with integral states

0

0

0 0 0

2) Select the matrix properly

(should penalize

T TT TN T T

N NT NN N N

I I

X A A X B

X A A X B U

X I X

Q

= +

ɺ

ɺ

ɺ

( ) ( )0

only and states)

3) Control solution

T I

TT

tT T

T c N T c

X X

U K X r X X r dt = − − −

∫

LQR Design for Inhomogeneous SystemsLQR Design for Inhomogeneous SystemsLQR Design for Inhomogeneous SystemsLQR Design for Inhomogeneous Systems






LQR Design for

Inhomogeneous Systems

Reference



LQR Design for


� To derive the state of a linear (rather linearized) system to the origin by minimizing the following quadratic

performance index (cost function)

( ) ( )0

1 1

2 2

where

, 0 (psdf), 0 (pdf)

ft

T T T

f f f

t

f

J X S X X Q X U RU dt

S Q R

= + +

≥ >

∫

X A X BU C= + +ɺX



LQR Design for


� Performance Index (to minimize):

� Path Constraint:

� Boundary Conditions:

( ) ( )0

1 1

2 2

ft

T T T

f f f

t

J X S X X Q X U RU dt= + +∫

X A X BU C= + +ɺ

( )

( )00 :Specified

: Fixed, : Freef f

X X

t X t

=



LQR Design for


� Terminal penalty:

� Hamiltonian:

� State Equation:

� Costate Equation:

� Optimal Control Eq.:

� Boundary Condition:

( ) ( )1

2

T T TH X Q X U RU AX BU Cλ= + + + +

( ) ( )1

2

T

f f f fX X S Xϕ =

X AX BU C= + +ɺ

( ) ( )/ TH X QX Aλ λ= − ∂ ∂ = − +ɺ

( ) 1/ 0 TH U U R B λ−∂ ∂ = ⇒ = −

( )/f f f fX S Xλ ϕ= ∂ ∂ =



LQR Design for


Guess ( ) ( ) ( ) ( )t P t X t K tλ = +

( )

( )( )( ) ( )( )

1

1

T

T T

PX PX K

PX P AX BU C K

PX P AX BR B PC K

QX A PX K PX P AX BR B PX K PC K

λ

λ−

−

= + +

= + + + +

= + − + +

− + + = + − + + +

ɺ ɺ ɺ ɺ

ɺ ɺ

ɺ ɺ

ɺ ɺ

( )

( )

1

1 0

T T

T T

P PA A P PBR B P Q X

K A K PBR B P PC

−

−

+ + − +

+ + − + =

ɺ

ɺ



LQR Design for


� Riccati equation

� Auxiliary equation

� Boundary conditions

1 0T TP PA A P PBR B P Q

−+ + − + =ɺ

( ) ( ) ( ) is freef f f f f f

P t X K t S X X+ =

( )f fP t S=

( )1 0T TK A PBR B K PC

−+ − + =ɺ

( ) 0f

K t =



LQR Design for


Control Solution:

Note: There is a residual controller even after

This part of the controller offsets the continuous disturbance.

0.X →

( )

1

1

1 1

T

T

T T

U R B

R B PX K

R B PX R B K

λ−

−

− −

= −

= − +

= − −



Thanks for the Attention….!!

Documents

Linear Quadratic Regulator (LQR) Linear Quadratic ... · Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) ––––IIIIIIII Prof. Radhakant Padhi Dept. of Aerospace