Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Optimal Control
4SC000 Q2 2017-2018
Duarte Antunes
Recap
1
• Continuous-time optimal control problems
x(t) = f(x(t), u(t)), x(0) = x0, t 2 [0, T ]Z T
0g(x(t), u(t))dt+ gT (x(T ))
Dynamic model
Cost
• We have seen two simple versions of the PMP considering
• problems with no terminal constraints and final cost
• problems with terminal constraints and no final cost
gT (x(T ))
x(T ) = x
Recap
2
• In both problems, we have
(u⇤(t), x⇤(t))
x
⇤(t) = f(x⇤(t), u⇤(t))
�(t) = �( @
@x
f(x⇤(t), u⇤(t)))|�(t)� ( @
@x
g(x⇤(t), u⇤(t)))|
@@uf(x
⇤(t), u⇤(t))|�(t) + @@ug(x
⇤(t), u⇤(t))| = 0
�(T ) = @
@x
g
T
(x⇤(T ))
State eq.
Adjoint eq.
Control eq.
• The boundary conditions are different for the two problems
(no terminal state constraints)
(terminal constraints)
9�(t), t 2 [0, T ] is an optimal path candidate if s.t.
x
⇤(0) = x0
x
⇤(0) = x0 x
⇤(T ) = x
• It is also possible that only the terminal value of some variables are constrained, xi(T ) = xi, i 2 C
Outline
• Pontryagin’s maximum principle
• Examples
• Hamiltonian’s principle in mechanics
• Brachistochrone
• Minimum energy problems
• Minimal time problems for linear systems (bang-bang control)
3
Discussion
• Last lecture we used an informal discretization approach to derive a simplified version of the PMP.
• The Pontryagin’s maximum principle is more general and it allows to consider constraints and free terminal time problems. This requires a different mathematical framework in continuous-time.
CT PMP
CT DP
DT PMP
DT DP
Discretization, step ⌧ ⌧ ! 0
Taking the limit
Optimal path and
policy
Stage decision problem
CT control problem
Optimal path and
policy
The goals for today are
• State the general Pontryagin’s maximum principle.
• Solve minimal time problems with the PMP for linear system with input constraints.
4
Problem formulation
Control system
• Final time and final state can be free or fixed.
Cost functional to be minimized
• Initial time and initial state are fixed.
Assumptionsx(t) 2 Rn u 2 U ⇢ Rm
for all
• , control input may be constrained
• Lipschitz property: for every bounded set , s.t.9L
x0 = x(t0)
x(t) = f(x(t), u(t))
D 2 Rn ⇥ U
|f(x1, u)� f(x2, u)| L|x1 � x2|(x1, u), (x2, u) 2 D
• are continuous.
x(0) = x0
J(u) =R T0 g(x(t), u(t))dt
f, g, @f
@x
, @g
@x
t0 = 0
T xf := x(T )
5
Target set
• Depending on the control objective, the final time and final state can be free or fixed, or can belong to some set.
• All the possibilities are captured by introducing a closed target set and letting be the smallest time such that
• fixed-time, fixed end state for some fixed (previous lecture)
• fixed-time, free-end state (previous lecture)
• or in general for some surface in
• In particular only some of the state variables may be constrained.
• free-time, fixed end state
• or in general
• free-time, free end state (not so common)
S1 Rn
T (T, xf ) 2 S
S = {T}⇥ {xf} T, xf
S = {T}⇥ Rn
{T}⇥ S1
S ⇢ [0,1)⇥ Rn
S = [0,1)⇥ Rn
S = [0,1)⇥ {x1}
S = [0,1)⇥ S1
6
Target set
• We assume that , differentiable
• If the target set is never reached the cost is set to infinity.
S1 = {x 2 Rn : h1(x) = · · · = hn�k(x) = 0} hi : Rn ! R
• We can always consider since it captures not only free time-fixed end state problems ( ) but also fixed-time by considering time as a state
Then, for this new problem
• In particular,
• fixed-time, free end state
• fixed-time, fixed end state
xn+1 = t
x(t)
xn+1(t)
�=
f(x(t), u(t))
1
�
S = [0,1)⇥ S1
S = [0,1)⇥ {x1}
x(0)
xn+1(0)
�=
x0
0
�
S = [0,1)⇥ Rn ⇥ {t1}
S = [0,1)⇥ S1 ⇥ {t1}
S = [0,1)⇥ {x1}⇥ {t1}
7
Terminal cost• To consider the problem with terminal cost
note that
where
• Then
and so there is no loss of generality in our problem formulation.
x(0) = x0x(t) = f(x(t), u(t))
J(u) =R T0 g(x(t), u(t))dt+ gT (xf )
gT (xf ) = gT (0) +R T0
ddtgT (x(t))dt
d
dt
g
T
(x(t)) = @
@x
g
T
(x(t))x(t) = @
@x
g
T
(x(t))f(x(t), u(t))
J(u) =
R T0 g(x(t), u(t))dt+ constant
g(x, u) = g(x, u) + @
@x
g
T
(x)f(x, u)
8
Pontryagin’s maximum principle
Let be an optimal control (in the global sense) and let be the corresponding optimal state trajectory. Then there exist a function and a constant satisfying for all and having the following properties:
1) and satisfyx
⇤ p⇤
Consider an optimal control problem with target set .
H : Rn ⇥ U ⇥ Rn ⇥ R ! Rwith boundary conditions and , where is the Hamiltonian, defined as .
2) For each fixed , the function has a global maximum, i.e.,
hold for all and all .
3) for all
4) The vector is orthogonal to the tangent space to at
t
H(x⇤(t), u⇤(t), p⇤(t), p⇤0) � H(x⇤(t), u, p⇤(t), p⇤0)
(p⇤0(t), p⇤(t)) 6= (0, 0)
u ! H(x⇤(t), u, p⇤(t), p⇤0)
u 2 U
H(x⇤(t), u⇤(t), p⇤(t), p⇤0) = 0
S1
x⇤ =@
@pH(x⇤, u⇤, p⇤, p⇤0)
p⇤ = � @
@xH(x⇤, u⇤, p⇤, p⇤0)
p⇤0 0
S = [0,1)⇥ S1
x
⇤ : [0, T ] ! Rn
p⇤ : [0, T ] ! Rnu⇤ : [0, T ] ! U
t 2 [0, T ]
x
⇤(0) x
⇤(T ) 2 S1
H(x, u, p, p0) = p|f(x, u) + p0g(x, u)
t 2 [0, T ]
t 2 [0, T ]
p⇤(T ) x
⇤(T )
< p⇤(T ), d >= 0 8d, i :< d,
@
@x
hi(x⇤(T ))| >= 0
9
Discussion
• These are still only necessary conditions for optimality and not sufficient. However a local maximum or a local minimum cannot satisfy these conditions and thus these are necessary conditions for global optimality.
• The function should be interpreted as our previous costate, but multiplied by , . This is just a convention.
• The scalar is called the abnormal multiplier. We can replace except in degenerate cases which we will not address (in which case ).
• Accordingly, the Hamiltonian is defined in a slightly different manner
which is obtained by multiplying by our previous definition
p(t) = ��(t)�1
p0
�1
p⇤0 = �1p⇤0 = 0
H(x, u, p,�1) = p|f(x, u)� g(x, u)
H(x, u,�) = �|f(x, u) + g(x, u) = �p|f(x, u) + g(x, u) = �H(x, u, p,�1)
p(t) : [0, T ] ! Rn
10
Conditions 1 and 2
• The conditions stated in point 1) of the Theorem are nothing more that the equations for the state and the co-state that we have obtained before. In our previous notation
• The condition 2) states that has a (global) maximum. In our previous notation this would correspond to a minimum. If there are no input constraints and are differentiable with respect to (our previous setting) then we must have
Thus condition 2 is more general and shall allow us to handle input constraints
x⇤ =@
@pH(x⇤, u⇤, p⇤, p⇤0)
p⇤ = � @
@xH(x⇤, u⇤, p⇤, p⇤0)
u ! H(x⇤(t), u, p⇤(t), p⇤0)
x
⇤ = f(x⇤, u
⇤, p
⇤)
�
⇤ = � @
@x
f(x⇤, u
⇤,�
⇤)|�⇤ � @
@x
L(x⇤, u
⇤,�
⇤)|
u
@
@uH(x⇤, u, p⇤, p⇤0)|u=u⇤ = 0
@
@u
f(x⇤, u
⇤, p
⇤)|�⇤ +@
@u
g(x⇤, u
⇤, p
⇤)| = 0
(as in previous lectures)
(as in previous lectures)
f, g
11
Condition 3
• Condition 3 states that the Hamiltonian must be zero.
• For problems with fixed terminal time, recall the trick
where either or
• This trick leads to an extra variable for the costate and the Hamiltonian for the new problem, denoted by , is given by
• Thus is constant and is constant (as in previous lectures)
Thus condition 3 is more general and shall allow us to handle free time problems for which the Hamiltonian must be zero.
xn+1 = t
x(t)
xn+1(t)
�=
f(x(t), u(t))
1
�
pn+1(t)
pn+1(t) = � @
@xn+1H = 0
pn+1(t) H = 0 H
H
x(0)
xn+1(0)
�=
x0
0
�
S = [0,1)⇥ Rn ⇥ {t1} S = [0,1)⇥ {x1}⇥ {t1}
H =⇥p| pn+1
⇤ f1
�� g = p|f � g| {z }
H
+pn+1
12
Condition 4
• Condition 4 is called the transversality condition
• For fixed terminal time, fixed terminal state problems
free (as in previous lectures)
• For fixed terminal time, free terminal state problems there are no constraints so is free and
S1 = {x 2 Rn : h1(x) = · · · = hn�k(x) = 0} hi : Rn ! R
d
h(x(T )) = x(T )� xf = 0 =) d = 0 =) p
⇤(T )
p⇤(T ) = 0
< p⇤(T ), d >= 0 8d, i :< d,
@
@x
hi(x⇤(T ))| >= 0
13
Condition 4
• Recall that if
• Thus the Hamilton for the problem where we consider is
where can be shown to satisfy condition 1 of the theorem.
• Then (as in previous lectures)
Thus condition 4 is more general.
J(u) =
R T0 g(x(t), u(t))dt+ gT (xf ) =
R T0 g(x(t), u(t))dt+ constant
g(x, u) = g(x, u) + @
@x
g
T
(x)f(x, u)
p = p� @
@x
g
T
(x)|
H(x, p, u) = p|f(x, u)� (g(x, u) + @
@x
gT
(x)f(x, u))
p(T ) = 0 =) p(T ) = � @
@x
g
T
(x(T ))|
g
= p
|f(x, u)� g(x, u)
Historical note
The Pontryagin’s maximum principle was developed by the Pontryagin schools in the Soviet Union in the late 1950s. It was presented to the wider research community at the first IFAC World Congress in Moscow in 1960.
14
Lev Pontryagin (1908-1988) was a Soviet mathematician. He was born in Moscow and lost his eyesight due to a stove explosion when he was 14. Despite his blindness he was able to become one of the greatest mathematicians of the 20th century, partially with the help of his mother Tatyana Andreevna who read mathematical books and papers (source: wikipedia).
Outline
• Pontryagin’s maximum principle
• Examples
• Hamiltonian’s principle in mechanics
• Brachistochrone
• Minimum energy problems
• Minimal time problems for linear systems (bang-bang control)
15
Hamiltonian’s principle in Mechanics
The motion of a conservative system, from time to time is such that the integral
has a stationary value (typically a minimum), where
L = T (u, q)� V (q)
T
V
q
u = q
Lagrangian of the system
Principle of least action
Kinetic energy of the system
Potential energy of the system
Generalized velocity vector
Generalized coordinate vector (state of the system)
0 T
I =R T0 L(u, q)dt
16
Hamiltonian’s principle in Mechanics
The Hamiltonian is then
Consequently, the Euler-Lagrange equations are
� = �@H
@q= �@L
@qcostate eq.
control eq.
state eq. u = q
0 =@H
@u=
@L
@u+ � = 0
Combining the first two equations we obtain
d
dt(@L
@q)� @L
@q= 0
which are the Lagrange’s equations of motion for a conservative system.
H = L+ �u
17
Hamiltonian’s principle in Mechanics
If is not an explicit function of time, the Hamiltonian is constant
Interesting: nature governs the motion of bodies with optimal control!
L
H = L� @L
@uu = T � V � @T
@uu = const
One can show that the kinetic energy satisfies: (think of )T =1
2mv2
@T
@uu = 2T
Hence, we have
�H = T + V = const
that is, the kinetic plus potential energy is constant during the motion.
Outline
• Pontryagin’s maximum principle
• Examples
• Hamiltonian’s principle in mechanics
• Brachistochrone
• Minimum energy problems
• Minimal time problems for linear systems (bang-bang control)
18
Brachistochrone
Find a path between two points in a vertical plane such that a particle sliding without friction along this path takes theshortest possible time to travel from one point to the other.
x
y
a b
yb
Historical note
19
Johann Bernoulli posed the problem of the brachistochrone to the readers of Acta Eruditorum in June, 1696. Five mathematicians responded with solutions: Isaac Newton, Jakob Bernoulli (Johann's brother), Gottfried Leibniz, Ehrenfried Walther von Tschirnhaus and Guillaume de l'Hôpital. Four of the solutions (excluding l'Hôpital's) were published in the same edition of the journal as Johann Bernoulli's. In his paper Jakob Bernoulli showed that its solution is a cycloid. According to Newtonian scholar Tom Whiteside, Newton found the problem in his mail, in a letter from Johann Bernoulli, when he arrived home from the mint at 4 p.m., and stayed up all night to solve it and mailed the solution by the next post. This story gives some idea of Newton's power, since Johann Bernoulli took two weeks to solve it. In an attempt to outdo his brother, Jakob Bernoulli created a harder version of the brachistochrone problem. In solving it, he developed new methods that were refined by Leonhard Euler into what the latter called (in 1766) the calculus of variations. Joseph-Louis Lagrange did further work that resulted in modern infinitesimal calculus. (adapted from Wikipedia)
Johann Bernoulli (1667 – 1748) was a Swiss mathematician, known for his contributions to infinitesimal calculus and educating Leonhard Euler. He was a professor of mathematics at the university of Groningen.
20
Problem formulationLet us relabel x ! t
y ! x
The initial kinetic and potential energy is zero, and we must have the following equation determining the velocity as a function of
Then the total time is the integral of the arc-length over the velocity
mv
2
2= mgx
x
Z b
a
p1 + (x(t))2p2gx(t)
dt
g = 9.8
21
Optimal controlZ b
a
p1 + u(t)2p2gx(t)
dt
x(t) = u(t)Dynamic model
Cost function
Hamiltonian H(x, u,�) = L(x, u) + �u
L(x, u)
Terminal constraints x(a) = 0 x(b) = xb
Optimality conditions
@H
@u= 0
H = constant
p1 + u
⇤(t)
2
p2gx
⇤(t)
� u
⇤(t)
2
p1 + u
⇤(t)
2p2gx
⇤(t)
= constant
22
Optimal solutionThe solutions to the differential equation
x
⇤(t) =
sC � x
⇤(t)
x
⇤(t)
are cycloids determined implicitly by
t(✓) = a+ c(✓ � sin(✓))
x(✓) = c(1� sin(✓))
These equations describe the curve traced by a point on a circle of radius as this circle rolls without slipping on the horizontal axis.
c
Outline
• Pontryagin’s maximum principle
• Examples
• Hamiltonian’s principle in mechanics
• Brachistochrone
• Minimum energy problems
• Minimal time problems for linear systems (bang-bang control)
23
Minimum energy control
How to move a motion system described by a linear equation from point A to point B with minimum energy
x(t) = Ax(t) +Bu(t)
x(0) = x0
x(T ) = xdesired
A
B
minuR T0 g(u(t))dt
Fx
Fy
yx
g(u) is convexAssumption: u 2 R
24
PMP
H(x, u,�) = g(u) + �|(Ax+Bu)
@
@uH(x, u,�) =
@
@ug(u) + �|B = 0
� = �[@
@xH(x, u,�)]| = �A|�
x = Ax+Bu
u = g�1u (��|B)
g�1u :=
@
@ug(u) is invertible since is convexg
�(t) = e�A|t�(0)
x(T ) = xdesiredimposing we obtain a linear system from which we can obtain �(0)
= g�1u (�B|�)
x(t) = e
Atx(0) +
Z t
0e
A(t�s)Bg
�1u (�B
|e
�A|s)ds�(0)
25
Example
A =
0 10 0
�B =
01
�(double integrator)
g(u) =1
4u4
x(0) =
00
�x(T ) =
10
�T = 1
solution:
g�1u (y) = sign(y)y1/3 sign(y) =
8><
>:
1 if y > 0
� 1 if y < 0
0 if y = 0
or equivalently
Substituting in the expression
10
�= x(1) = e
2
40 10 0
3
5
x(0)+
Z 1
0e
2
40 10 0
3
5(1�s) 01
�g
�1u (�
⇥1 0
⇤e
�
2
40 10 0
3
5s
�(0))ds
10
�=
Z 1
0
1� s1
�g�1u (s�1(0)� �2(0))ds
�1(0) = 2�2(0) = 203.54
26
Results
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
u
-5
-4
-3
-2
-1
0
1
2
3
4
5
abs((895185530450641 t)/4398046511104 - 895185530450641/8796093022208)1/3 (heaviside(t - 1) - heaviside(t - 1/2)) - (895185530450641/8796093022208 - (895185530450641 t)/4398046511104)1/3 (heaviside(t - 1/2) - heaviside(t))
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x 2 = d
p/dt
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(3 35807421218025641/3 heaviside(t - 1/2) (1 - 2 t)4/3)/262144 -...+ (3 35807421218025641/3 heaviside(t - 1) ((2 t - 1)4/3 - 1))/262144
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x 1=p
0
0.2
0.4
0.6
0.8
1
(3 35807421218025641/3 heaviside(t) ((3 (1 - 2 t)7/3)/14 - 3/14))/262144 -...+ (3 35807421218025641/3 heaviside(t - 1) ((3 (2 t - 1)7/3)/14 - 3/14))/262144
time t time t
time t
27
Connection to controllability gramianSuppose that , the initial condition is zero and is large
x(t) = Ax(t) +Bu(t) x(T ) = xdesired
g(u) =1
2u2
x(0) = 0
min
Z T
0
1
2u(t)2dt
T (T ! 1)
Then is the identify, the optimal control input isg�1u
�(0)
u(t) = �B|e�A|t�(0)
and can be obtained from
x(T ) =
Z T
0e
A(T�s)BB
|e
�A|sds�(0) =
Z T
0eArBB|eA
|rdre�A|T�(0)
r = T � s
�(0) = e
A|TW (T )�1
x(T ) W (T ) =
Z T
0eArBB|eA
|rdr
28
Connection to controllability gramian
Z T
0
1
2u(t)2dt =
1
2x(T )|W (T )�1
Z T
0e
A(T�r)BB
|e
A|(T�r)dr
| {z }W (T )
W (T )�1x(T )
=1
2x(T )|W (T )�1
x(T )
W (T ) =
Z T
0eArBB|eA
|rdr
If is Hurwitz and is controllable, we can take the limitA (A,B) (T ! 1)
obtaining
which is the controllability gramian - solution to:
W1 =
Z 1
0eArBB|eA
|rdr
0 = AW1 +W1A| +BB|
Optimal cost
Outline
• Pontryagin’s maximum principle
• Examples
• Hamiltonian’s principle in mechanics
• Brachistochrone
• Minimum energy problems
• Minimal time problems for linear systems (bang-bang control)
Minimal time problemsConsider the problem
minT
x(t) = Ax(t) +Bu(t) t � 0x(0) = x0
x(T ) = x
(A,B)
subject to
• We assume that is controllable
• Without loss of generality we can assume x = 0
• We can write the cost as T =R T0 1dt
•For simplicity we assume u(t) 2 R
�c ui(t) c i 2 {1, . . . ,m} u(t) 2 Rm
•Without loss of generality we can assume c = 129
PMPMaximize
or equivalently, minimize
H(x(t), u(t),�(t)) = �(t)|(Ax(t) +Bu(t)) + 1
H(x(t), u(t), p(t),�1) = p(t)|(Ax(t) +Bu(t))� 1
�(t) = �p(t)
� = �@H@x
x = @H@�
terminal constraints ,
x(t) = Ax(t) +Bu(t)
x(0) = x0x(T ) = x
�(0),�(T ) free
the function
30
u ! H(x(t), u(t),�(t))is minimized when
In particularH(x(t), u(t),�(t)) = 0
t = T|�(T )|B| = 1
u(t) = �sign�(t)|B =
(� 1 if �(t)|B > 0
1 if �(t)|B 0u(t) = [�1, 1]
�(t) = �A|�(t)
31
Discussion
u(t) = �sign�(t)|B =
(� 1 if �(t)|B > 0
1 if �(t)|B 0
• The fact that
shows that the input always takes either the maximum or the minimum value.
• For a given initial condition, the control input will switch between the minimum and the maximum value until the state reaches the origin.
• For this reason, this is called bang-bang control.
• As explained in [Bryson,Hu, sec 3.9] there is a numerical procedure to find .
x(t) = Ax(t) +Bu(t)
u(t) = �sign�(t)|B =
(� 1 if �(t)|B > 0
1 if �(t)|B 0
• However, for two-dimensional systems one can typically find solutions by inspection.
• To obtain the optimal path we must find such that when we solve �(0)
we obtainx(T ) = 0
�(0)
�(t) = �A|�(t)
• We also show to solve by inspection a problem for a fourth dimensional system.
2D problem
32
Suppose that we wish to bring an object with a given initial position and velocity to rest at position zero in minimum time
subject to a constraint on the magnitude of the force
minT =
Z T
01dt
�1 u(t) 1 tfor all
(x1(0), x2(0)) x1(T ) = 0 x2(T ) = 0
x1 = 0x1
ux1(t) = x2(t)
x2(t) = u(t)
x(t) = Ax(t) +Bu(t) A =
0 10 0
�B =
01
�
Optimal solution
33
If is an optimal control trajectory, must minimize the Hamiltonian for each , i.e.,
{u⇤(t)|t 2 [0, T ]} u⇤(t)t
Therefore
u
⇤(t) = argmin�1u1[1 + �1(t)x⇤2(t) + �2(t)u]
u⇤(t) =
(1 if �2(t) < 0
�1 if �2(t) � 0
The equations describing the costate are
for some constants .
�2(t) = c2 � c1t�2(t) = ��1(t)
�1(t) = 0 �1(t) = c1
c1, c2
Thus there are only four possibilities.
and we must have . |�2(T )| = 1
Possibilities
34
u⇤(t) =
(1 if �2(t) < 0
�1 if �2(t) � 0�2(t) = c2 � c1t
0 T t
�2(t)
0 T t
�2(t)
0 T t
�2(t)
0 T t
�2(t)
0 T t
�1
1
u⇤(t)
0 T t
�1
1u⇤(t)
0 T t
�1
1
u⇤(t)
0 T t
�1
1u⇤(t)
|�2(T )| = 1
A B C D
Possibilities
35
Possibilities A and C correspond to cases where the origin is achieved by setting the actuation constant ( or ) for a given initial conditionu(t) = 1u(t) = �1
0 = e
ATx0 +
R T0 e
A(T�s)B(�1)ds 0 = e
ATx0 +
R T0 e
A(T�s)Bds
u(t) = �1
0 x1
x2
u(t) = 1
or
x0 =R 0T e
A(�s)B(�1)ds
x0 =R 0T e
A(�s)Bds
x0 =
12T
2
�T
�x0 =
� 1
2T2
T
�
Control trajectories
36
u(t) = ⇠ ⇠ 2 {+1,�1} x1(t) = x1(0) + x2(0)t+ ⇠
t
2
2x2(t) = x2(0) + ⇠t
x1(t)�1
2⇠
(x2(t))2= x1(0)�
1
2⇠
(x2(0))2= constant
0 x1
x2 u(t) = 1 u(t) = �1
0 x1
x2
Switching curve
u(t) = �10 x1
x2
u(t) = 1
(x1(0), x2(0)) trajectories switch when they hit the switching curve
For an initial condition not lying in this curve, we know (from PMP co-state) that the control input can only switch once.
37
Minimum time control for a quadcopter
How to move a (linearized model of a) quadcopter from one hovering position to another one in minimum time?
x(t) = Ax(t) +Bu(t)
x(0) = x0
x(T ) = xdesired
minT
x0 xT
|u(t)| L
38
Model
Linearized model (y-axis)
Thrust
Gravity
y =1
mT sin(✓)
z =
1
mT cos(✓)� gT
y
z ✓
mg
mass
✓ =1
I✓⌧✓
x =⇥y y ✓ ✓
⇤|
x(t) = Ax(t) +Bu(t)
B =
2
664
0001
3
775u =
⌧✓I✓
x(0) =⇥0 0 0 0
⇤|x(T ) =
⇥y 0 0 0
⇤|
|u(t)| L
⇡ g✓
T ⇡ mg
A =
2
664
0 1 0 00 0 g 00 0 0 10 0 0 0
3
775
39
Optimal control input
u(t) = �L sign( a3|{z}��1(0)
6I✓
t3 + a2|{z}�2(0)2I✓
t2 + a1|{z}��3(0)
t+ a0|{z}�4(0)
)
|u(t)| = �L sign(�(t)|B) = �L sign(�4(t))
Polynomial of order 3 has at most 3 roots so there are at most three sign changes in the optimal control input
�(t) = e�A|t�(0) =
2
664
1 0 0 0�t 1 0 0gt2
2 �gt 1 0
� gt3
6gt2
2 �t 1
3
775
2
664
�1(0)�2(0)�3(0)�4(0)
3
775
40
Optimal control input
0 TT1 T2 T3
0 TT1 T2 T3
time
time
u(t) = �L sign(a3t3 + a2t
2 + a1t+ a0)
Conclusion: search over is equivalent to search over switching times (and sign of the control input)
�(0)Ti
0 TT1 T2 T3
0 TT1 T2 T3
time
time
or
41
ResultsFor u =
⌧✓I✓
|u(t)| L I✓ = 0.002 L = 50 y = 1
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
u
-60
-40
-20
0
20
40
60100 heaviside(t - 941/2000) -...- 100 heaviside(t - 4964993749729043/36028797018963968)
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
d/dt3
-10
-8
-6
-4
-2
0
2
4
6
8(45215787882993825 heaviside(t - 1808631515319753/2251799813685248))/562949953421312 -...- 100 t heaviside(t - 1808631515319753/2251799813685248)
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
3
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
(885481 heaviside(t - 941/2000))/80000 -...+ (45215787882993825 t heaviside(t - 1808631515319753/2251799813685248))/562949953421312
T2 = 0.4705
T3 = 0.8032
T ⇤ = 0.941
T1 = 0.1378
time t time t
time t
42
Results
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
v
0
0.5
1
1.5
2
2.5
(483164605226563795468286475929144999878116613455 heaviside(t - 1808631515319753/2251799813685248))/5708990770823839524233143877797980545530986496 -...- (801431249760917353029788835347205 t heaviside(t - 1808631515319753/2251799813685248))/2535301200456458802993406410752
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
y0
0.2
0.4
0.6
0.8
1
(38419753466689 heaviside(t - 941/2000))/19200000000000 -...+ (483164605226563795468286475929144999878116613455 t heaviside(t - 1808631515319753/2251799813685248))/5708990770823839524233143877797980545530986496
time t time t
43
ResultsFor u =
⌧✓I✓
|u(t)| L I✓ = 0.002 y = 1L = 100
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
u
-100
-50
0
50
100
200 heaviside(t - 791/2000) -...- 200 heaviside(t - 6081307241734005/9007199254740992)
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
d/dt3
-15
-10
-5
0
5
10
(208677473753223875 heaviside(t - 8347098950128955/72057594037927936))/9007199254740992 -...- 200 t heaviside(t - 6081307241734005/9007199254740992)
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
3
-1.5
-1
-0.5
0
0.5
1
1.5
(625681 heaviside(t - 791/2000))/40000 -...- 100 t2 heaviside(t - 8347098950128955/72057594037927936)
T2 = 0.39558
T3 = 0.6752
T ⇤ = 0.7910
T1 = 0.1158
time t time t
time t
44
Results
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
v
0
0.5
1
1.5
2
2.5
3
(142486188710187787663341665948145434555802153049375 heaviside(t - 8347098950128955/72057594037927936))/280608314367533360295107487881526339773939048251392 -...- (980 t3 heaviside(t - 8347098950128955/72057594037927936))/3
t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
y0
0.2
0.4
0.6
0.8
1
(19182358974289 heaviside(t - 791/2000))/9600000000000 -...- (245 t4 heaviside(t - 8347098950128955/72057594037927936))/3
time t time t
Concluding remarks
45
• Pontryagin’s maximum principal is a very elaborate theorem and the proof is far from trivial
• We can tackle minimum time problems with the Pontryagin’s maximum principle.
• For systems with linear dynamics this leads to bang-bang control.
• For 2D systems we can solve these by inspection by determining switching curves.
Summary
After this lecture you should be able to
• Solve simple minimum time-problems for linear systems with a two-dimensional state.