Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Lecture Lecture Lecture Lecture –––– 19191919
SDRE and DesignsSDRE and DesignsSDRE and DesignsSDRE and Designs
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Optimal Control, Guidance and Estimation
θ - D
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore2
Topics
State-Dependent Riccati Equation
(SDRE) Design
Design
Benchmark Examples
Dθ −
StateStateStateState----Dependent Dependent Dependent Dependent RiccatiRiccatiRiccatiRiccati Equation Equation Equation Equation (SDRE) Design(SDRE) Design(SDRE) Design(SDRE) Design
Dr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiAssociate Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore4
References J. R. Cloutier, “State-Dependent Riccati Equation Techniques: An
Overview”, Proceedings of the American Control Conference, Albuquerque, New Mexico, USA, 1997.
J. R. Cloutier and D. T. Stansbery, “The Capabilities and Art of State-Dependent Riccati Equation-Based Design”, Proceedings of the American Control Conference, Anchorage, AK, USA, 2002.
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett, State-Dependent Riccati Equation Techniques: Theory and Applications.Workshop Notes: American Control Conference, June, 1998.
T. Cimen, “State-Dependent Riccati Equation (SDRE) Control: A Survey”, Proceedings of 17th IFAC World Congress, Seoul, Korea, 2008.
T. Cimen, “Systematic and Effective Design of Nonlinear Feedback Controllers via the State-Dependent Riccati (SDRE) Method”, Annual Reviews in Control, Vo. 34, 2010, pp.32-51.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore5
SDRE Design: Usage
Nonlinear suboptimal control design
• Regulator design
• Servo (tracking) design
• Robust control design
Nonlinear suboptimal observer design
Nonlinear suboptimal filters design
(Essentially, wherever Riccati equation appears,
SDRE concept can be brought in)
( )2/H H∞
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore6
SDRE Design: Problem Statement
Performance Index: (to minimize)
System Dynamics: (control affine)
Conditions:
•••••
( ) ( )( )0
1
2
T T
t
J X Q X X U R X U dt
∞
= +∫
( ) ( )X f X B X U= +ɺ
( ) ( ) ( ) ( ) ( ), , , 1kf X B X Q X R X C k∈ ≥
( )0 0f =
( ) 0 (domain of interest)B X X≠ ∀ ∈Ω
( ) ( )( ) is globally convex True when , 0J Q X R X >
( ) ( ) ( ) ( ) , , is point-wise stabilizablef X A X X A X B X=
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore7
SDRE Design: Procedure
Cost Function
Write the system dynamics
in state-dependent coefficient (SDC) form
Solve the state-dependent
Riccati equation
Construct the controller
( ) ( )X A X X B X U= +ɺ
( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )1 0
T
T
P X A X A X P X Q X
P X B X R X B X P X−
+ +
− =
( ) ( ) ( )
( )
1 TU R X B X P X X
K X X
− = −
= −
( ) ( )( )0
1
2
T T
t
J X Q X X U R X U dt
∞
= +∫
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore8
Implementation Issue
Solve the Riccati equation symbolically by long hand algebra.
Use symbolic software package to solve the Riccati equation symbolically
(e.g. Maple, Mathematica, Matcad etc.)
Solve the Riccati equation online with a high speed computer.
Obtain off line point solution and use gain scheduling.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore9
Example - 1
Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore10
Example - 1
Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.
x
u(0,0)
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore11
Definitions:
Controllability / Observability
Controllability
Observability
( )
( ) ( )
is an observable (detectable) parameterization of the
nonlinear system in a region if the pair ,
is point-wise observable (detectable) in the linear sense .
A X
C X A X
X
Ω
∀ ∈Ω
( )
( ) ( )
is an controllable (stabilizable) parameterization of the
nonlinear system in a region if the pair ,
is point-wise controllable (stabilizable) in the linear sense .
A X
A X B X
X
Ω
∀ ∈Ω
( )Output: Z C X X=
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore12
SDRE Design: Useful Results
In addition to the conditions mentioned earlier, if and it is both a detectable and stabilizable parameterization, then the SDRE approach produces a closed loop system that is “locally asymptotically stable”.
For scalar problems, the resulting SDRE nonlinear controller satisfies all the necessary conditions of optimality; i.e. for scalar problems it always leads to the optimal solution (this is not true for vector case however).
( ) ( )1kA X C k∈ ≥
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore13
SDRE Design: Useful Results
Out of the three necessary conditions,
the optimal control equation is
always satisfied
However, the costate equation
is satisfied only asymptotically (under
certain additional mathematical
conditions). This is the reason for sub-
optimality of the controller in general.
/ 0H U∂ ∂ =
( )/H Xλ = − ∂ ∂ɺ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore14
Convergence of Costate Equation
( )
( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
Let 0, be an arbitrarily large open ball centred
at the origin with radious . Assume that the functions
, , , , along with their gradients
, , , , , 1, are
bounded in
i i i i ix x x x x
r
r
A X B X P X Q X R X
A X B X P X Q X R X i n
< ∞
= …
B
B ( )
( )
0, . Then, in SDRE nonlinear regulation,
under asymptotic stability (i.e. as 0), the necessary condition
/ is asymptotically satisfied at a quadratic rate.
r
X
H Xλ
→
= − ∂ ∂ɺ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore15
SDRE Design: Capabilities
Can directly specify and affect performance through the selection of appropriate state dependent state and control weighting matrices
Can incorporate hard bounds on state and control
Can directly handle unstable and/or non-minimum phase systems
Can preserve beneficial nonlinearities
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore16
Can be used to design servo (tracking)
control
Can be applied to a broad class of
nonlinear systems
Can incorporate extra degree of freedom
(a design parameter) to enhance
performance of the suboptimal controller
SDRE Design: Capabilities
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore17
SDRE Design: Limitations
Can be applied only to a class of
nonlinear problems
Suboptimality of the controller
Non-uniqueness of the parameterization
of the system dynamics
Applicable for infinite-time problems only
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore18
Demands solution of Riccati equation
online, which may not be feasible for
high-dimensional problems (since Riccati
equation is nonlinear)
No analytical guarantee of global stability
for the resulting controller in general.
SDRE Design: Limitations
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore19
SDRE Design: Some Useful Tricks
Presence of state-independent terms
Presence of state-dependent terms that excludes the origin
Uncontrollable and Unstable but Bounded State dynamics
( )
Constant bias:
0 1b bα α= − <ɺ ≪
( )11 1
1
cos 1cos 1 bias
xx x
x
−⇒ +
( )( )1 1
Add a stabilizing term
x xα= −ɺ i
( )0 1α< ≪
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore20
Extra Degree of Freedom
( ) ( )
( ) ( ) ( ) ( )
1 2
3 1
Claim:
Assume that and are two SDC parameterizations.
Then another SDC parameterization can be constructed as a
convex combination of these two parameterizations as follows:
1
A X A X
A X X A X X Aα α = + − ( ) ( )2 , 0 1X Xα≤ ≤
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
1 2
1 2
Proof:
1
1
1
X A X X A X X
X A X X X A X X
X f X X f X f X
α α
α α
α α
+ −
= + −
= + − =
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore21
Example - 2
Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore22
Example - 2
Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.
u
t
x2
x1
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore23
Example - 2
Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.
u
t
x2
x1
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore24
Example - 2
Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.
u
t
x2
x1
α(t)
t
Suboptimal Control DesignSuboptimal Control DesignSuboptimal Control DesignSuboptimal Control Design
Dr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiAssociate Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
θ - D
Courtesy: Ming Xin, Department of Aerospace Engineering
Mississippi State University, USA
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION Prof.
Radhakant Padhi, AE Dept., IISc-Bangalore
26
1. Xin M. and Balakrishnan S. N., “A New Method for Suboptimal
Control of a Class of Nonlinear Systems,” Optimal Control
Applications and Methods, 26(2): 55-83, 2005.
2. Xin M., Balakrishnan S. N., Stansbery D. T., and Ohlmeyer E.J.,
“Nonlinear Missile Autopilot Design with θ - D Technique,” AIAA
Journal of Guidance, Control and Dynamics, 27, 406-417, 2004.
3. Drake D., Xin, M. and Balakrishnan S. N., “A New Nonlinear Control
Technique for Ascent Phase of Reusable Launch Vehicles,” AIAA
Journal of Guidance, Control and Dynamics, 27(6): 938-948, 2004.
4. Radhakant Padhi, Ming Xin and S. N. Balakrishnan, “Suboptimal
Control of a One-dimensional Nonlinear Heat Equation Using POD
and θ - D Techniques”, Optimal Control Applications and Methods,
29(3): pp.191-224, 2008.
References
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
27
Optimal Control Problem
Objective:
( ) ( )0
1
2
ft
T T
t
J x Q x x u R x u dt
→∞
= + ∫
Find a controller u to minimize a cost function
( )( ) ( ) ( )x t f x t Bu t= +ɺ
System Dynamics:
This is an infinite-horizon optimal control problem
control affine form
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
28
Solution to the Optimal Control Problem
min*
uV J=
• Solve the Hamilton-Jacobi-Bellman (HJB) equation
*1 T V− ∂
= −∂
u R Bx
( )T *T
1 T T1 10
2 2
* *V V V−∂ ∂ ∂
− + =∂ ∂ ∂
f x BR B x Qxx x x
• A closed-form solution is very difficult to obtain
where
Challenge
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
29
• Make Approximations
Summary of Technique
0
1
2
T TJ dt
∞ = + +
∑∫
i
i
i=1
x Q x u uD θ R∞∞∞∞
*
0
( , ) i
i
i
Vθ θ
∞
=
∂=
∂∑T x x
x
( ) ( )
= + = + = 0
( )ɺ
A xx f x Bu F x x Bu A + θ x + Bu
θ
Assume:
• Solve perturbed Hamilton-Jacobi-Bellman equation
* * *1
1
1 1( ) 0
2 2
T TT T i
i
i
V V Vθ
∞−
=
∂ ∂ ∂ − + + = ∂ ∂ ∂
∑f x BR B x Q D xx x x
θ - D
Recall:*
1 T V− ∂= −
∂u R B
x
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
30
0 0 0 0
1
0 00
T TA A BRT T T TB Q
−−−−+ − + =+ − + =+ − + =+ − + =
T
c cF x DA AT TT
1 10 10 0 1( , , )θθθθ+ = −+ = −+ = −+ = −
⋮
• Closed-form Optimal Control
0T
1T
nT
Algebraic Riccati Equation
Linear Equation with
constant coefficients
Linear Equation with
constant coefficients⋮
n n n
T
cn ncF xT T TA T DA
0 1 10( , , , , )θθθθ−−−−
+ = −+ = −+ = −+ = −⋯⋯⋯⋯
1
0
( , )n
T i
i
i
T θ θ−
=
= − ∑u R B x x
1
0 0 0
T
cA A BR B T−−−−= −= −= −= −
Note: θ will be cancelled in the final control calculation
Substitute in HJB equation and equate coefficients:
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
31
Construction of
iD (((( ))))0 1
( ), , , ,il t
i i i iD k e F A x T T θθθθ
−−−−
−−−−==== ⋯⋯⋯⋯is constructed as
such that
i i i i i iF A x T T D t F A x T T
0 1 0 1( ( ), , , , ) ( ) ( ( ), , , , )θ ε θθ ε θθ ε θθ ε θ− −− −− −− −− = ⋅− = ⋅− = ⋅− = ⋅⋯ ⋯⋯ ⋯⋯ ⋯⋯ ⋯
( ) 1 il t
i it k eεεεε
−−−−= −= −= −= −with a small number
iD
Note:
TT A x A x TF x T 0 0
1 0
( ) ( )( , , )θθθθ
θ θθ θθ θθ θ= − −= − −= − −= − −
T nTn n
n n j n j
j
T A x A x TF x T T T BR B T
111 1
1 1
1
( ) ( )( , , , , )θθθθ
θ θθ θθ θθ θ
−−−−−−−−− −− −− −− −
− −− −− −− −
====
= − − += − − += − − += − − +∑∑∑∑⋯⋯⋯⋯
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
32
Motivation of Using i
D
Prove the convergence of the series.
Guarantee semi-global asymptotic stability
Reduce the initial control level
Adjust the system transient performance
ik and are primary design parameters to tune the system
performance
il
( ) 1 il t
i it k eεεεε
−−−−= −= −= −= − is used to:
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
33
• Systematic Method of Selecting and Parameters
Ideally, on the optimal path, the Hamiltonian
Procedure is as follows
– An initial value of (ki, li) is given.
– Controller is run, computing H value at each time.
– Iteratively change (ki, li) to minimize H in the least-
square sense.
– This procedure is run offline.
( )*
1 12 2
0T
T T VH x Q x u R u f x B u
∂= + + + = ∂x
ili
k
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
34
Benchmark Example (Kokotovic,1994)
Scalar Problem:
uxxx +−= 3ɺ
dtuxJ )(2
1 2
0
2 += ∫∞
Cost function:
The optimal solution
22)( 243 +−−−−= xxxxxu
Feedback Linearization solution3 2flu x x= − Feedback linearization cancels the beneficial nonlinearity
and results in large control effort when the state is large!
System dynamics:
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
35
SolutionDθ −
2
0 11, ( )A A x x= = −
• Factorize nonlinear term f(x) as
with Q = 1, R = 1
0 1 2T = +
solutionDθ −
2
1 1
1 (1 2)2
2 2
xT D
θ
⋅ += − −
22 2
2
2 1 1 22
1 1 (1 2) 1 (1 2)2 2 2
82 2 2 2
x xT x D D D
θθ
− ⋅ + ⋅ + = − ⋅ − + − −
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
36
SolutionDθ −
2 0 01
( ) ( )0 .98
Tt T A x A x T
D eθ θ
− = − −
0.9 1 12
( ) ( )0 .98
Tt T A x A x T
D eθ θ
− = − −
Di terms in the method play key role.Dθ −
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
37
Figure 3: Scalar problem: x0= [10,10]
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
38
Comparison Between
SDRE and θ - D Methods
Requires adjustment of both weighting matrix and
other design parameters
Requires adjustment of weighting matrices
Same requirement
Needs state dynamics in
terms of state dependant coefficient form
Lesser computational timeHigher Computational Time
Solving a set of Lyapunov equations online
Solving the Riccati equation online
SuboptimalSuboptimal
θθθθ - D MethodSDRE Method
OPTIMAL CONTROL, GUIDANCE
AND ESTIMATION Prof. Radhakant
Padhi, AE Dept., IISc-Bangalore
39
Thanks for the Attention….!!