Lecture 27_LQR.pdf

Lecture – 27

Linear Quadratic Regulator (LQR) Design – I

Dr. Radhakant PadhiAsst. Professor

Dept. of Aerospace EngineeringIndian Institute of Science - Bangalore

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

2

LQR Design: Problem Objective

To drive the state of a linear (rather linearized) system to the origin by minimizing the following quadratic performance index (cost function)

( ) ( )0

1 12 2

where, 0 (psdf), 0 (pdf)

ftT T Tf f f

t

f

J X S X X Q X U RU dt

S Q R

= + +

≥ >

∫

X A X BU= +X


3

LQR Design: Guideline for Selection of Weighting Matrices

( )( )

2

2

0 (psdf), 0 (psdf), 0 (pdf)

These are usually chosen as diagonal matrices, with

maximum expected/acceptable value of 1/

maximum expected/acceptable value of 1/

maximum expected/a

i f

f

f i

i i

i

S Q R

s x

q x

r

≥ ≥ >

=

=

= ( )2cceptable value of 1/ iu


4

LQR Design: Some Facts to Remember

The pair needs to be controllable and the pairneeds to be detectable

(these are usually chosen as diagonal matrices)

By default, it is assumed that

Constrained problems (state and control inequality constraints) are not considered here. Those will be considered later.

{ },A B

0 (psdf), 0 (psdf), 0 (pdf) fS Q R≥ ≥ >

ft →∞

{ },A Q


5

LQR Design: Problem Statement

Performance Index (to minimize):

Path Constraint:

Boundary Conditions:

( )( )

( )( )

0,

1 12 2

f

f

tT T Tf f f

tL X UX

J X S X X Q X U RU dt

ϕ

= + +∫

X A X BU= +

( )( )

00 :Specified

: Fixed, : Freef f

X X

t X t

=


6

LQR Design:Necessary Conditions of Optimality

Terminal penalty:

Hamiltonian:

State Equation:

Costate Equation:

Optimal Control Eq.:

Boundary Condition:

( ) ( )12

T T TH X Q X U RU AX BUλ= + + +

( ) ( )12

Tf f f fX X S Xϕ =

X AX BU= +

( ) ( )/ TH X QX Aλ λ= − ∂ ∂ = − +

( ) 1/ 0 TH U U R B λ−∂ ∂ = ⇒ = −

( )/f f f fX S Xλ ϕ= ∂ ∂ =


7

LQR Design: Derivation of Riccati Equation

Guess:

Justification:

( ) ( ) ( )t P t X tλ =

( )( )

( )

From functional analysis theory of normed linear space,

lies in the "dual space" of , which is the space consisting

of all continuous linear functionals of .

Reference: Optimization by Vecto

t

X t

X t

λ

r Space Methods D. G. Luenberger, John Wiley & Sons, 1969.


8


Guess ( ) ( ) ( )t P t X tλ =

( )( )( )

1

1

T

T

PX PXPX P AX BU

PX P AX BR B

PX P AX BR B PX

λ

λ−

−

= +

= + +

= + −

= + −

( ) ( )( )

1

1 0

T T

T T

QX A PX P PA PBR B P X

P PA A P PBR B P Q X

−

−

− + = + −

+ + − + =


9


Riccati equation

Boundary condition

1 0T TP PA A P PBR B P Q−+ + − + =

( ) ( )is freef f f f fP t X S X X=

( )f fP t S=


10

LQR Design: Solution Procedure

Use the boundary condition and integrate the Riccati Equation backwards from to Store the solution history for the Riccati matrixCompute the optimal control online

( )f fP t S=

ft 0t

( )1 TU R B P X K X−= − = −


11

LQR Design: Infinite Time Regulator ProblemTheorem (By Kalman)

Algebraic Riccati Equation (ARE)

Note:ARE is still a nonlinear equation for the Riccati matrix. It is not straightforward to solve. However, efficient numerical methods are now available.

A positive definite solution for the Riccati matrix is needed toobtain a stabilizing controller.

1 0T TPA A P PBR B P Q−+ − + =

As , for constant and matrices, 0ft Q R P t→∞ → ∀

Example – 1:

Stabilization of Inverted Pendulum




13

A Motivating Example:Stabilization of Inverted Pendulum

( )

2 2

System dynamics:

, /Linearized about vertical equilibrium point

n nu g Lθ ω θ ω= − =

1 2

1 12

2 2

System dynamics (state space form):

Define: ,0 1 0

0 1n

BXX A

x xx x

ux x

θ θ

ω

= =

⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤= +⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥−⎣ ⎦⎣ ⎦ ⎣ ⎦⎣ ⎦

2 22

0

2

Performance Index (to minimize):

1 1 2

1 0 1 ,0 0

J u dtc

Q Rc

θ∞ ⎛ ⎞= +⎜ ⎟⎝ ⎠

⎡ ⎤= =⎢ ⎥⎣ ⎦

∫

mgθ Lu


14


mgθ Lu1

1 2

2 3

2 2 2 22 22 1 2 2 32 3

2 2 2 23 2 2 3 31 2

2 2 2 22 2 2 2

ARE: 0

Let (a symmetric matrix)

1 0 0 00 0 0 0

Equations:12 1 0

T T

n n n

n

n n

PA A P PBR B P Qp p

Pp p

p p c p c p pp pp p c p p c pp p

p c p pc

ω ω ωω

ω ω ω

−+ − + =

⎡ ⎤= ⎢ ⎥⎣ ⎦

⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤+ − + =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥

⎣ ⎦ ⎣ ⎦⎣ ⎦⎣ ⎦ ⎣ ⎦

− + = ⇒ = ± 4 2

2 21 3 2 3

2 22 3 3 2

0 (repeated)12 0 2

n

n

c

p p c p p

p c p p pc

ω

⎡ ⎤+⎣ ⎦

+ − =

− = ⇒ = ±


15


mgθ Lu

3

2

2 4 22 3 22

2 21 3 2 3

2 21 2 3 3

However, is a diagonal term, which needs to be real and positive.Hence, needs to be positive. Therefore

1 1, 2

Moreover,0

(not needed i

n n

n

n

pp

p c p pc c

p p c p p

p c p p p

ω ω

ω

ω

⎡ ⎤= + + =⎣ ⎦

+ − =

= −

( )

1 2 22 3

22 3

n this problem)

Gain Matrix:

Control:

TK R B P c p c p

u K X c p pθ θ

− ⎡ ⎤= = − −⎣ ⎦

= − = +


16


mgθ Lu

Analysis

( )

2 22

Open-Loop System:1

0

right half pole: unstable system

nn

n

I Aλ

λ λ ωω λ

λ ω

−− = = − =

−

= ±

( )

( )

2 4 2

2 22 2

1/ 22 23 2 2

Define: 1

1 2 2

n

n

n

c

pc

p pc c

ω ω

ω ω

ω ω

= +

= +

= = +

2 2 22 3

Closed-Loop System:0 1

Closed-Loop Poles: 0

CLn

CL

A A BKc p c p

I A

ω

λ

⎡ ⎤= − = ⎢ ⎥− −⎣ ⎦

− =


17


mgθ Lu

Analysis

( )( ) ( )

( )

1/ 22 2 2 2

1/ 2 1/ 22 2 2 21,2

2 4 2 2

Closed-Loop Poles:

2 0

1 1 2 2

Note:

n

n n

n n

j

c

λ ω ω λ ω

λ ω ω ω ω

ω ω ω

+ + + =

= − + ± −

= + >

Both of the closed-loop poles are strictly in the left-half plane.

Hence, the closed-loop is guaranteed to be “asymptotically stable”.

Example – 2:

Finite-time Temperature Control




19

Example: Finite Time Temperature Control Problem

( )

0

where, : Constants : Temperature

: Ambient temperature (Constant = 20 C) : Heat input

n

n

a bu

a b

u

θ θ θ

θ

θ

= − − +

System dynamics :


20

Problem formulations

Case – 1: Case – 2:

( )

2

0

0

Cost Function:

12

30

(Hard constraint)

ft

f f

J u dt

t Cθ θ

=

= =

∫ ( )

( )

2 2

0

0

Cost Function:

1 302

0 : Weightage

i.e. 30

(Soft Constraint)

ft

f f

f

f

J s u dt

s

t C

θ

θ

⎡ ⎤= − +⎢ ⎥

⎢ ⎥⎣ ⎦>

≈

∫


21

Solution:

( ) ( )( ) ( )

( )2

Solution:, 0

, 0 012

0

a a

a a

x

x ax bu x

u ax bu

ax

u bu

θ θ θ θ

θ θ

λ

λ λ

λ

− =

= − + = − =

Η = + − +

∂Η⎛ ⎞= − =⎜ ⎟∂⎝ ⎠∂Η

= ⇒ = −∂

Necessary conditions

x ax bua

u bλ λ

λ

= − +

== −


22

Solution: Case – 1 (Hard constraint)( ) ( )

( )

( )

( ) ( )

2

Taking laplace transform:

0

f f

f

f

a t t a t tf f

a t tf

a t tf

e e

u be

x ax b e

sX s x

λ λ λ

λ

λ

− − −

− −

− −

= =

= −

= − −

− ( ) 2

0

1fatfaX s b e

s aλ −

⎡ ⎤ ⎛ ⎞⎢ ⎥ = − − ⎜ ⎟−⎢ ⎥ ⎝ ⎠⎣ ⎦


23

Solution: Case – 1 (Hard constraint)

( )

( )

( )

22 2

2

2

Unknown

0

1

1 1 12

1Hence ( )2

However, ( ) 10

f

f

f

atf

atf

at at atf

f f a

X s b es a

b ea s a s a

x t b e e ea

x t C

λ

λ

λ

θ θ

−

−

− −

⎛ ⎞= − ⎜ ⎟−⎝ ⎠⎛ ⎞= − −⎜ ⎟− +⎝ ⎠

= − −

= − =


24


( )

( )

( )

2

22

22

2

1( ) 102

10 12

201

( )

f f f

f

f

at at atf f

atf

f at

x t b e e ea

be

a

ab e

x t b

λ

λ

λ

− −

−

−

= = − −

⎛ ⎞= − −⎜ ⎟⎜ ⎟

⎝ ⎠−

=−

= −20 a−

2b ( ) ( ) ( )( )2

10121

f

f f f

at atat at at

at at at

e ee e e

ae e e

−− −

− −

⎛ ⎞ −⎜ ⎟ − =⎜ ⎟− −⎝ ⎠


25


( )10( )

f fat at

f

e ex t

−−=

Note :

( )f fat ate e−−10

(i.e. The boundary condition is "exactly met".)

( )u t b

=

= −

Controller :

( )2

20fa t t aeb

− − −

( ) ( )2

201 f f f

at

at at at

aee b e e− −

⎡ ⎤ ⎡ ⎤−⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥− −⎣ ⎦ ⎣ ⎦


26

Solution: Case – 2 (Soft constraint)

( )

( )

( ) ( )

0 0

2 2

0

2

30 10 .

Hence the cost function is

1 102

10 10

However, we have

2

f

f

f f

t

f f

ff f f f

f

at at atf

C x C

J s x u dt

s x xs

bx t e e ea

θ

λλ

λ − −

→ ⇒ →

⎡ ⎤= − +⎢ ⎥

⎢ ⎥⎣ ⎦⎛ ⎞

= − ⇒ = +⎜ ⎟⎜ ⎟⎝ ⎠

= − −

∫


27


( ) ( )

( )

( )( ) ( )

( )

22

22

22

22

At , 1 102

1. . 1 102

20. .

2 1

20Hence

2 1

f

f

f

f f

f

at ff f f

f

atf

f

ff at

f

a t t a t t ff at

f

bt t x t ea s

bi e es a

s ai e

a s b e

s ae e

a s b e

λλ

λ

λ

λ λ

−

−

−

− − − −

−

= = − − = +

⎡ ⎤+ − = −⎢ ⎥

⎢ ⎥⎣ ⎦⎡ ⎤−⎢ ⎥=⎢ ⎥+ −⎣ ⎦

⎡ ⎤−⎢ ⎥= =⎢ ⎥+ −⎣ ⎦


28


( )

( )

( )

( )

( )

22

2

20

2 1

10

2

f

f

f

f f f

a t t fat

f

ata t t f

at at atf

u t b

s abe

a s b e

s abebe

s bae e e

λ

− −

−

− −

−

= −

⎡ ⎤−⎢ ⎥= −⎢ ⎥+ −⎣ ⎦⎡ ⎤⎢ ⎥⎢ ⎥= −⎢ ⎥+ −⎢ ⎥⎣ ⎦


29

Correlation between hard and soft constraint results

( )( )

( ) ( )

. . 2

. .

As ,

10lim lim1

2

20

. . The "soft constraint" problem behaves like the "hard constraint" problem when .

f ff f f

f f

f

at

S Cs sat at at

f

at

at at H C

f

s

abeu tbae e e

s

ae u tb e e

i es

→∞ →∞−

−

→ ∞

=⎛ ⎞

+ −⎜ ⎟⎜ ⎟⎝ ⎠

= =−

→∞


30

Documents

Lecture 27_LQR.pdf