Optimal control models of the goal-oriented human locomotion › nlc › slides › 12Jean.pdf · Goal-oriented human locomotion (Berthoz, Laumond et al.) Initial point (x 0;y 0;

Optimal control modelsof the goal-oriented human locomotion

(with Y. Chitour, F. Chittaro, and P. Mason)

Frederic Jean(ENSTA ParisTech, Paris)

Nonlinear Control and Singularities – October 24-28, 2010

F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 1 / 30

Outline

1 Inverse optimal control

2 Modelling goal oriented human locomotion

3 Analysis of Pk(L)General resultsAsymptotic analysisStability results


Inverse optimal control

Outline







Analysis/modelling of human motor control→ looking for optimality principles

Subjects under study:

Arm pointing motions (with J-P. Gauthier, B. Berret)Saccadic motion of the eyesGoal oriented human locomotion

Mathematical formulation: inverse optimal control

Given X = φ(X,u) and a set Γ of trajectories, find a cost C(Xu) suchthat every γ ∈ Γ is solution of

inf{C(Xu) : Xu traj. s.t. Xu(0) = γ(0), Xu(T ) = γ(T )}.



Difficulties:

Dynamical model not always known (← hierarchical optimal control)Limited precision of both dynamical models and costs

⇒ necessity of stability (genericity) of the criterion

Non-unicity of the cost that is solution of the problem.No general method

Validation method: a program in three steps

1 Modelling step: propose a class of optimal control problems

2 Analysis step: enhance qualitative properties of the optimal synthesis(e.g. inactivations)

→ reduce the class of problems

3 Comparison step: numerical methods→ choice of the best fitting L



Difficulties:

Dynamical model not always known (← hierarchical optimal control)Limited precision of both dynamical models and costs

⇒ necessity of stability (genericity) of the criterion

Non-unicity of the cost that is solution of the problem.No general method

Validation method: a program in three steps

1 Modelling step: propose a class of optimal control problems

2 Analysis step: enhance qualitative properties of the optimal synthesis(e.g. inactivations)

→ reduce the class of problems

3 Comparison step: numerical methods→ choice of the best fitting L


Modelling goal oriented human locomotion

Outline






Trajectories of the Human Locomotion

0 θ0(x , y , )

1 1 θ1(x , y , )

0

Goal-oriented human locomotion (Berthoz, Laumond et al.)

Initial point (x0, y0, θ0) → Final point (x1, y1, θ1)(x, y position, θ orientation of the body)

QUESTIONS :

Which trajectory is experimentally the most likely?

What criterion is used to choose this trajectory?



Trajectories of the Human Locomotion

HYPOTHESIS:the chosen trajectory is solution of a minimization problem

min

∫L(x, y, θ, x, y, θ, . . . )dt

among all “possible” trajectories

joining the initial point to the final one.

→ TWO QUESTIONS:

What are the possible trajectories ? dynamical constraints?

How to choose the criterion ? (inverse optimal control problem)



Dynamical Constraints

[Arechavaleta-Laumond-Hicheur-Berthoz, 2006] (=[ALHB])↓

1st experimental observation: if target far enough,

the velocity is perpendicular to the body

v

→ x sin θ − y cos θ = 0 nonholonomic constraint!

(Dubins)−→{x = v cos θy = v sin θ

v = tangential velocity.



Dynamical Model

2nd experimental observation in [ALHB]↓

the velocity has a positive lower bound, v ≥ a > 0,

and is a function (almost constant) of the curvature

⇒ The trajectories may be parameterized by arc-length

(we are only interested by the geometric curves)

→ v ≡ 1.



Dynamical Model

Previous observations: L = L(x, y, θ, θ, . . . )

trajectories obtained through a minimization procedure

⇒trajectories in a complete functional space, θ ∈W k,p,

and

L = L(x, y, θ, θ, . . . , θ(k)).

→ the trajectories are solutions of the optimal control problem

minCL(u) =

∫ T

0L(x, y, θ, θ, . . . , θ(k))dt

among all trajectories of

x = cos θy = sin θ

θ(k) = u

u ∈ Lp,

s.t. (x, y, θ)(0) = (x0, y0, θ0) and (x, y, θ)(T ) = (x1, y1, θ1).



Admissible costs

ELEMENTARY REMARKS :

k ≥ 3 not reasonable ⇒ k = 1 or k = 2.

The whole problem is invariant by rototranslations→ L = L(θ, . . . , θ(k)) independent of (x, y, θ)

0 is the unique minimum of LNormalization: L(0) = 1 ⇒ cost of a straight line = its length

L is convex w.r.t. u

TECHNICAL ASSUMPTIONS :

L is smooth (at least C2)

L is strictly convex w.r.t. u and∂2L

∂u2> 0

L(θ, . . . , θ(k−1), u) ≥ C|u|p for |u| > R.



To summarize

Inverse Optimal Control Problem

Given recorded experimental data (e.g. [ALHB])), infer a cost function Lsuch that the recorded trajectories are optimal solutions of

Pk(L)

minCL(u) =

∫ T

0L(θ, . . . , θ(k))dt (k = 1 or 2)

subject to

x = cos θy = sin θ

θ(k) = uwith (x, y, θ)(0) = (x0, y0, θ0) and (x, y, θ)(T ) = (x1, y1, θ1).

REMARKS :

The time T is not fixed

The initial point can be chosen as X0 = (0, 0, π/2)

The target X1 = (x1, y1, θ1) is far from X0: |(x1, y1)| “large”


Analysis of Pk(L)

Outline





Analysis of Pk(L) General results

Outline






Analysis of Pk(L) – General Results

Proposition

For every target X1, there exists an optimal trajectory of Pk(L).

Every optimal trajectory satisfies the Pontryagin Maximum Principle.

REMARKS :

The control system is controllable

The proof of existence uses standard arguments (cf. Lee & Markus)

The optimal control does not belong a priori to L∞([0, T ]). Howeverit is possible to prove that, for (x1, y1) far away from 0, the optimalcontrol is uniformly bounded.→ not necessary to put an a priori bound on the control



CASE k = 2:

H = p1 cos θ + p2 sin θ + p3θ + p4θ − νL(θ, θ) ≡ 0

No strictly abnormal extremals → optimal traj. are C∞ (ν = 1)

Adjoint Equation: (p1, p2) are constant and

using ∂H∂u = 0, the adjoint equation writes as an ODE:

θ(4) = FL(θ, θ, θ, θ(3); (p1, p2)

),

with initial data: (θ, θ(3))(0) = (π2 , 0) [transversality condition]

CASE k = 1:

H = p1 cos θ + p2 sin θ + p3θ − νL(θ) ≡ 0

No strictly abnormal extremals → optimal traj. are C∞ (ν = 1)

Adjoint Equation: (p1, p2) are constant and

θ = GL(θ, θ; (p1, p2)

), θ(0) =

π

2


Analysis of Pk(L) Asymptotic analysis

Outline






Analysis of P2(L)

We present only the analysis for k = 2; same thing for k = 1.

Simplifying assumption: L(θ, θ) = 1 + ψ(θ) + φ(θ)

REMIND:

Adjoint equation ⇒ fourth-order ODE on θ parameterized by (p1, p2):

θ(4) = FL(θ, θ, θ, θ(3); (p1, p2)

),

with initial data: (θ, θ(3))(0) = (π2 , 0)

H = p1 cos θ + p2 sin θ + p3θ + p4θ − L(θ, θ) ≡ 0



Notations: (x1, y1) = ρ(cosα, sinα),

Θ = (θ, θ, θ, θ(3)).

( )1 y1,y

x

x

α

Proposition

Given ν > 0, |Θ(t)− (α, 0, 0, 0)| < ν for t ∈ [τν , T − τν ].

Proposition

As |(x1, y1)| → ∞, (p1, p2) ∼(x1, y1)

|(x1, y1)|= (cosα, sinα).

(α, 0, 0, 0) equilibrium of the “limit” equation θ(4) = FL(Θ; (x1,y1)|(x1,y1)|

)Consequence: an optimal trajectory has a stable behaviour near the

equilibrium Yeq = (α, 0, 0, 0).



Asymptotic Trajectories

The equilibrium Yeq = (α, 0, 0, 0) is not stable:its stable manifold Ws is 2-dimensional.

Z(0)

eq

Yeq

u

s

u

s

Y → the limit optimal trajectorymust be “contained” in Ws.

→ Close to Yeq, the trajectoryis tangent to the stable spaceof the linearized ODE



Asymptotic Trajectories

Ws contains a 1-parameter family of trajectories: we compute themnumerically and select the one starting with θ(0) = π/2 and θ(3)(0) = 0.

-8 -6 -4 -2 2 4 6 8

-8

-6

-4

-2

2

4

6

8

We obtain also the limit value of p(0)→ initialization of a shooting algorithm.

We can recover dL from the phase portraitof Θ

(Figure: L = 1 + θ2 + θ2)



5 10 15 20

5

10

15

20

25

30


Analysis of Pk(L) Stability results

Outline






Stability results

Let L0 be a cost admissible for k = 1 or k = 2.

Let Lε be a family of costs admissible for k = 2 s.t. Lε → L0 in thefollowing sense:

|Lε(θ, θ)− L0(θ, θ)| or |Lε(θ, θ)− L0(θ)| ≤ C(ε)|θ|p

Proposition

The optimal trajectories (xε, yε, θε, θε) of P2(Lε) converge uniformlyto the optimal trajectories of Pk(L0), i.e.,

dunif((xε, yε, θε, θε), T0

)→ 0,

T0 = {(x, y, θ, θ) s.t. (x, y, θ, θ) or (x, y, θ) is optimal for Pk(L0)}.

With additional technical hypothesis, the adjoint vectors (and so θand θ(3)) also converge uniformly.



Stability results

CONSEQUENCES

The optimal synthesis of a problem Pk(L0) is stable underperturbations of the cost.

Our modelling is ”reasonable” (physiological costs)

A solution of the inverse problem = a cost and his perturbations→ we will look for the simplest cost in this class

Question: k = 1 for the simplest one?

Adjoint equation of a perturbation = perturbation of the adjointequation



Analysis of the case k = 1

H = p1 cos θ + p2 sin θ + θL′(θ)− L(θ) ≡ 0

Remark: If θ(t0) = 0, then p1 cos θ(t0) + p2 sin θ(t0) = 1,

→ depends on one parameter

Proposition

Let T be the set of trajectories s.t. θ = 0 at some time.To any trajectory in T , with θ(t0) = 0, we apply the transformation:

t = t− t0θ(t) = θ(t)− θ(t0)(x(t), y(t)) = Rot(−θ(t0))

[(x(t), y(t))− (x(t0), y(t0))

]Then, for every fixed t, the set of (x(t), y(t), θ(t)), for all trajectories in T ,is a curve in R3.



Numerical test

Numerical test: apply the transformation to the recorded curves.Does it give a curve?

Transformation applied at three different times t1, t2, t3 to ∼700recorded trajectories



Validity of the test (work in progress)

Numerically: same test applied to the solutions of some P2(L)

Transversality argument: for k = 2, a generic cost L should not givesuch a curve through the transformation

Analysis of the optimal synthesis using the asymptotic trajectories



Conclusion

Models with k = 1 should be sufficient.

A deep explanation:

recorded trajectories theoretical solutions



Conclusion

Models with k = 1 should be sufficient.

A deep explanation:

recorded trajectories theoretical solutions


Documents

Optimal control models of the goal-oriented human locomotion › nlc › slides › 12Jean.pdf · Goal-oriented human locomotion (Berthoz, Laumond et al.) Initial point (x 0;y 0;