NONLINEAR TRACKING OVER COMPACT SETS WITH LINEAR H

NONLINEAR TRACKING OVER COMPACT SETS WITH LINEARDYNAMICALLY VARYING H∞ CONTROL∗

STEPHAN BOHACEK† AND EDMUND JONCKHEERE†

SIAM J. CONTROL OPTIM. c© 2001 Society for Industrial and Applied MathematicsVol. 40, No. 4, pp. 1042–1071

Abstract. Linear dynamically varying H∞ controllers are developed for tracking natural trajec-tories of a broad class of nonlinear systems defined over compact sets. It is shown that the existenceof a suboptimal H∞ controller is related to the existence of a bounded solution to a functionalalgebraic Riccati equation. Even though nonlinear systems running over compact sets could exhibitsensitive dependence on initial conditions, the Riccati solution is continuous in the suboptimal case,but it may be discontinuous in the optimal case.

Key words. nonlinear systems, trajectory tracking, linear parametrically and dynamicallyvarying systems, state space H∞ robust control, functional algebraic Riccati equation

AMS subject classifications. 49K40, 49L20, 37A99, 37B25, 37C05, 37C75, 37F15

PII. S0363012999350584

1. Introduction. Nonlinear tracking has been thoroughly investigated. A pop-ular approach is to linearize the system around an operating point, generate a linearcontroller for each operating point, and “schedule” the controllers in such a way thatthe closed-loop system remains stable as the operating point changes. In this ap-proach, the nonlinear tracking error system is modeled, approximately, as a linearsystem with parameters that vary as the operating point varies. Such systems havebeen extensively studied [3], [4], [5], [6], [25], [28], [33] and are known as linear para-metrically varying (LPV) systems.

For the purpose of comparing the various LPV concepts, it is convenient to in-troduce linear set-valued dynamically varying (LSVDV) systems [11]:

[x (k + 1)

z (k)

]=

[Aθ(k) B1θ(k)

B2θ(k)

Cθ(k) D1θ(k)D2θ(k)

] x (k)w (k)u (k)

,(1)

θ (k + 1) ∈ F (θ (k)) ⊆ Θ,with θ (0) = θo and x (0) = xo.

Here the parameter vector θ varies according to a set-valued dynamical system, con-tinuous for the Hausdorff metric; w is the disturbance input, u the control, and z thecontrolled output.

In the most traditional LPV approach [4], [6], [5], [23], [25], all that is knownabout the parameter dynamics is that F(θ) = Θ. The advantage of this model is thatif Θ is a convex polytope, then there are many computationally efficient controllersynthesis methods [14]. Most of these approaches generate a suboptimal solution viaa linear matrix inequality (LMI). However, these approaches can be conservative.

A slight refinement of the above LPV method consists of putting bounds on therate at which the system parameters vary, i.e., F(θ(k)) = Bθ(k)(∆), the ball with

∗Received by the editors January 13, 1999; accepted for publication (in revised form) January31, 2001; published electronically November 28, 2001. This research was supported in part by NSFgrant ECS-98-12594 and in part by AFOSR grant F49620-93-I-0505.

http://www.siam.org/journals/sicon/40-4/35058.html†Department of Electrical Engineering—Systems, University of Southern California, 3740 Mc-

Clintock Ave., Room 306, Los Angeles, CA 90089-2563 ([email protected], [email protected]).

1042

TRACKING WITH LINEAR DYNAMICALLY VARYING CONTROL 1043

radius ∆ and with its center at θ(k). There are efficient methods based on func-tional LMIs for designing controllers for these modified LPV systems [16], [33], [34],[35]. However, these design methods could fail when the parameters vary drastically,for example, when the controller needs to account for failures which lead to suddenchanges in the system parameters [21]. Also, typically, these methods are conservative.Nonconservative LPV approaches are pursued in [11] and [29].

Another popular type of LPV system is the jump linear (JL) system [15], [19].Here Θ = {Θ1,Θ2, . . . } is discrete, and F(θ(k)) is equipped with a probability measuredepending on θ(k) only, so that the transition among the Θi’s is a Markov chain. Thejump linear method for designing a controller for a such system is optimal (hence,nonconservative). The controller is provided by the solution to a system of coupledRiccati equations. Furthermore, there are efficient methods to compute the optimalcontroller [1], [2], [12].

A linear dynamically varying (LDV) system is a LSVDV system in which the pa-rameter dynamics are completely known, that is, F(θ(k)) is reduced to a point f(θ(k)).In [8] it was shown that a linear-quadratic (LQ) controller for such a system (withw = 0) can be found by solving a functional algebraic Riccati equation (FARE). Itshould be noted that this functional algebraic Riccati equation is the LDV substitutefor the functional linear matrix inequality of most other LPV approaches. Further-more, the FARE of LDV design provides the optimal solution, while the functionalLMI only provides a suboptimal solution. The mathematical difficulty with the LDVapproach is proving that the solution to the FARE is continuous, in which case thefeedback gain matrix is a continuous function of the parameters. The LPV approachesdescribed above avoid this continuity question by assuming a priori that the solutionto the relevant functional LMI is continuous [33], polynomial [35], affine [16], [34], oreven constant [4], [6], [5], [23], [25]. Since an arbitrary accuracy approximation of adiscontinuous function has to duplicate the exact behavior at the discontinuity points,which are potentially uncountable in number, a discontinuous solution is numericallyintractable, so that the continuity assumption is justifiable. However, it is importantto know how constraining this continuity assumption is.

Tracking trajectories of the important class of hyperbolic nonlinear systems oncompact sets can be accomplished by modeling the dynamics as a Markov chain[22] and resorting to JL methods. However, the resulting closed-loop system is onlystochastically stable, and it is not possible to show directly that the system is robustlystable. For this reason, the typical JL approach is not appropriate for the nonlineartracking problem. The connection between JL and LDV control systems designs isexamined in [10].

While in [8] LDV systems were stabilized using LQ methods, here the same sys-tems are stabilized by means of H∞ methods. This paper shows that, if the parameterdynamics are completely known, then the existence of a suboptimal H∞ controlleris equivalent to the existence of a continuous solution to the FARE. Of particularinterest are LDV systems that arise as linearized versions of nonlinear tracking errordynamics. In this case, it can be shown that the linearization error is a boundedfeedback around the linearized system, so that the H∞ formulation is well-suitedto minimize the effect of the error due to linearization and amplify the domain ofattraction.

The paper proceeds as follows. The next section formalizes the tracking controlproblem of interest and shows how the tracking error dynamics can be approximatedas an LDV system. Section 3 formally develops LDV systems. Section 4 develops the

1044 STEPHAN BOHACEK AND EDMUND JONCKHEERE

suboptimal H∞ controller for this class of systems. Section 5 provides the proofs ofthe main technical results. Section 6 shows that these linear controllers are suitablefor stabilization of nonlinear dynamical systems.

Notation: |x (k)| := (x′ (k)x (k))1/2, ‖x‖[k,j] := (∑ji=k x

′ (i)x (i))1/2, and ‖x‖l2 :=‖x‖[0,∞). If x ∈ R

n, then ‖x‖∞ := maxi≤n |xi| and, if x ∈ Rn×Z, then ‖x‖l∞ :=

supk∈Z|x (k)|. If A is a matrix, then ‖A‖ := sup|x|=1 |Ax|, whereas, if T : l2 → l2,

then ‖T‖ := sup‖x‖l2=1 ‖Tx‖l2 ; the context in which these norms are used will re-

solve potential confusion. If f : Θ × Rm → Θ with Θ ⊂ R

n, then ∂f∂θ (θ, u) denotesthe Jacobian matrix of f where the derivatives are taken with respect to θ and areevaluated at (θ, u) ∈ Θ × R

m. Define ∂f∂u (θ, u) similarly. With reference to system(1), let zθo (u,w, xo) denote the output signal z due to initial conditions θ (0) = θoand x (0) = xo, and input signals u and w. Let zθo (u,w, xo; k) denote this outputat time k. Let zθo (F,w, xo) and zθo (F,w, xo; k) be defined similarly, except that thecontrol u is replaced by the control law defined by F . For succinctness, we often writef (θ) := f (θ, 0) .

2. Problem statement. A dynamical system θ(k + 1) = f(θ(k)), where f ∈C1 (Rn,Rn), gives rise to a string of nested invariant subsets P (f) ⊆ P (f) ⊆ R(f) ⊆NW (f), where P (f) is the periodic set, P (f) its closure, R(f) the closure of therecurrent set, and NW (f) the nonwandering set [22]. We specifically consider systemswhere NW (f) is bounded, in which case P (f), R(f), and NW (f) are compact, andwe choose the domain Θ to be any of those compact invariant sets. More generally,Θ could be taken to be any compact invariant subset. In particular, if f is an AxiomA diffeomorphism satisfying the strong transversality condition, then NW (f) is adisjoint union of attractors [27], which by definition are compact and invariant andhence could be taken to be Θ. If the uniform hyperbolic conditions fails, f could stillhave an attractor, which could be taken to be Θ.

We take the control u to be a small perturbation of the parameters of the nominaldynamics f . More specifically, the nominal and perturbed dynamics are, respectively,

θ (k + 1) = f (θ (k) , 0) + v1 (k) , with θ (0) = θo,(2)

ϕ (k + 1) = f (ϕ (k) , u (k)) + v2 (k) , with ϕ (0) = ϕo,(3)

where1.

f ∈ C1 (Rn × Rm,Rn) ;(4)

2. f (Θ, 0) ⊂ Θ, i.e., Θ is f -invariant, and f (·, 0) : Θ → Θ;3. Θ is a compact subset of R

n.Here {θ (k) : k ≥ 0} is the desired trajectory, and ϕ (k) is the state of the sys-

tem under control. The exogenous inputs v1 (k) and v2 (k) are typically small withθ (k + 1) ∈ Θ. The purpose of v1 is to allow the desired trajectory to occasionallyjump from a point on one orbit to a nearby point on another orbit [9]. On the otherhand, v2 is to allow for some modeling inaccuracies. At time k, it is assumed that bothθ (k) and ϕ (k) are known. The basic objective is to find a control u such that, whenv1 = v2 = 0 and for |θ(0)− ϕ(0)| small enough, we have limk→∞ |ϕ (k)− θ (k)| = 0.

A distinguishing feature of the present approach is that the tracking controllertakes the form of a spatially varying gain F : Θ → R

m×n, guaranteed to be con-tinuous under suitable conditions. As the first and most generic application, given


an arbitrary desired trajectory {θ (k) : k = 0, . . . }, evaluating the controller F alongthe trajectory {θ (k) : k = 0, . . . } yields the time-varying controller Fθ(k) that makesthe nonlinear system ϕ (k + 1) = f(ϕ (k) , Fθ(k) (ϕ (k)− θ (k))) asymptotically trackθ (k + 1) = f (θ (k)). More importantly, the globally defined controller F becomesfully motivated in those specialized applications where there is a need to quicklyadapt the tracking controller to a new reference trajectory without recomputing anew time-varying controller along the new trajectory [9], [13], [20], [21].

If v1 = v2 = 0, then stability of the closed-loop system, which implies asymptotictracking, is guaranteed if |ϕ (0)− θ (0)| < RCapture, where RCapture > 0. If v1 �= 0and/or v2 �= 0, then asymptotic tracking can still be guaranteed if ‖v1 − v2‖l∞ and|ϕ (0)− θ (0)| are small enough and v1(k) − v2(k) is intermittent enough. If v1 − v2

is persistent, then one cannot expect asymptotic tracking; however, under suitable

conditions, the gain‖θ−ϕ‖�∞‖v1−v2‖�∞

can easily be shown to be bounded [7]. Besides, the

effect of the model uncertainty v2 can be minimized using standard H∞ methods.Therefore we shall not pursue investigation of the effects of v1, v2 any further.

The tracking controller design relies on linearizing the tracking error dynamics asfollows. Define the tracking error

x (k) := ϕ (k)− θ (k) .

Then

x (k + 1) = f (ϕ (k) , u (k))− f (θ (k) , 0) .

The first-degree Taylor approximation of f (ϕ (k) , u(k)) around ϕ (k) = θ (k) andu (k) = 0 yields

f (ϕ (k) , u (k)) = f (θ (k) , 0) +Aθ(k) (ϕ (k)− θ (k))

+B2θ(k)u (k) + η (x (k) , u (k) , θ (k)) ,

where

Aθ :=∂f

∂θ(θ, 0) , B2θ

:=∂f

∂u(θ, 0) ,(5)

and η (x (k) , u (k) , θ (k)) accounts for nonlinear terms. Thus

x (k + 1) = Aθ(k)x (k) +B2θ(k)u (k) + η (x (k) , u (k) , θ (k)) .(6)

Since f ∈ C1, η can be decomposed as

η (x (k) , u (k) , θ (k)) = ηx (x (k) , u (k) , θ (k))x (k) + ηu (x (k) , u (k) , θ (k))u (k) ,(7)

where

ηx (x, u, θ)i,j =

∫ 1

0

(∂fi∂xj

(tx+ θ, tu)− ∂fi∂xj

(θ, 0)

)dt(8)

and

ηu (x, u, θ)i,j =

∫ 1

0

(∂fi∂uj

(tx+ θ, tu)− ∂fi∂uj

(θ, 0)

)dt.(9)


Since f ∈ C1 and Θ is compact, if x and u are bounded, then ∂fi∂xj

(tx+ θ, tu) −∂fi∂xj

(θ, 0) is uniformly continuous. In particular, for any ε > 0 there is a δ > 0 such

that, if |x| , |u| < δ, then | ∂fi∂xj (tx+ θ, tu)− ∂fi∂xj

(θ, 0) | < ε. Therefore,

limx→0,u→0

sup {‖ηx (x, u, θ)‖ : |x| < x, |u| < u, θ ∈ Θ} = 0(10)

and

limx→0,u→0

sup {‖ηu (x, u, θ)‖ : |x| < x, |u| < u, θ ∈ Θ} = 0.(11)

If u and x are small, we can approximate the error dynamics as

x (k + 1) = Aθ(k)x (k) +B2θ(k)u (k) ,(12)

θ (k + 1) = f (θ (k) , 0) .

This system is linear in the tracking error x, but the coefficient matrices A and Bvary (generally in a nonlinear way) as θ varies. Since θ (k) varies according to (2), thesystem described in (12) is an LDV system. Before controllers can be developed forsuch systems, linear systems with dynamically varying parameters must be formalized.

3. Linear dynamically varying systems and LQ control. Motivated by thepreceding considerations, a general LDV system is defined as

[x (k + 1)

z (k)

]=

[Aθ(k) B1θ(k)

B2θ(k)

Cθ(k) D1θ(k)D2θ(k)

] x (k)w (k)u (k)

,(13)

θ (k + 1) = f (θ (k)) ,

with θ (0) = θo and x (0) = xo,(14)

subject to the following general conditions:1. Θ ⊂ R

n is compact and f : Θ → Θ is a continuous function.2. A : Θ → R

n×n, B1 : Θ → Rn×l, B2 : Θ → R

n×m, C : Θ → Rp×n, D1 : Θ →

Rp×l, and D2 : Θ → R

p×m are functions that need not be continuous.In the above, θ(k) ∈ Θ is the state of the dynamic system, x (k) ∈ R

n is the stateof the linear system, u (k) ∈ R

m is the control input, w (k) ∈ Rl is the disturbance

input, and z (k) ∈ Rp is the output to be controlled.

It is often assumed that the system coefficient matrices A, B1, B2, C, D1, and D2

are continuous. We will refer to such systems as continuous LDV systems. In section2 it was assumed that f ∈ C1, and since A and B are matrices of partial derivativesof f , A and B are indeed continuous. Thus the tracking error system associated with(2) and (3) can be approximated by a continuous LDV system. However, if a feedbackF : Θ → R

m×n is used to stabilize a continuous LDV system, then the resulting closed-loop system is a continuous LDV system if and only if F is continuous. Although thispaper will focus on stabilizing continuous LDV systems, we cannot assume a priorithat the feedback is continuous. Therefore the definition of an LDV system mustallow for possibly discontinuous coefficient matrices.

Since an LDV system is an uncountable collection of linear time-varying systemsindexed by θ (0), the concept of stability is slightly more complex in the dynamicallyvarying case than it is in the time-varying case.


Definition 3.1. The LDV system (13) is uniformly exponentially stable if, foru (k) = 0 and w (k) = 0, there exist an α ∈ [0, 1) and a β < ∞ such that, for allθ (0) ∈ Θ,

|x (k)| ≤ βαk |x (0)| .System (13) is exponentially stable if, for u (k) = 0, w (k) = 0, and for each θ (0) ∈ Θ,there exist an αθ(0) ∈ [0, 1) and a βθ(0) < ∞ such that, for all x (j) and j ≤ k,

|x (k)| ≤ βθ(0)αk−jθ(0) |x (j)| .

System (13) is asymptotically stable if, for u (k) = 0, w (k) = 0, any |x (0)| < ∞, andany θ (0) ∈ Θ,

|x (k)| → 0 as k → ∞.

Note that an exponentially stable system is stable uniformly in time k, but notnecessarily uniformly in the initial condition θ (0). That is, along any given positivetrajectory

{fk (θ (0)) : k ≥ 0

}, an exponentially stable system is (uniformly in time)

exponentially stable; however, if {θ(0)i : i ≥ 0} is a convergent sequence, with θ(0) =limi→∞ θ(0)i, it is possible that αθ(0)i → 1 while αθ(0) < 1, in which case the systemis exponentially stable, but not θ(0)-uniformly exponentially stable. To emphasizethe difference between exponential and uniformly exponential stability, exponentialstability will occasionally be referred to as uniform in time exponential stability.

In the case of continuous LDV systems, asymptotic, exponential, and uniformexponential stability are equivalent (Proposition 2 in [8]). Since uniformly exponen-tially stable systems are inherently more robust than exponentially stable systems, itis preferable to remain within the confines of continuous LDV systems. Thus, whensynthesizing a feedback for controlling a continuous LDV system, it is important toensure that the feedback is not only asymptotically stabilizing but also continuous.However, to maintain generality, an LDV system is considered stabilizable if thereexists an exponentially stabilizing feedback, that is, the following holds.

Definition 3.2. System (13) is stabilizable if there exists a (not necessarilycontinuous) function F : Z × Θ → R

m×n with bound F θ(0) < ∞ such that, for all

θ (0) ∈ Θ and for all k ≥ 0, we have∥∥Fθ(0) (k)∥∥ ≤ F θ(0), and the system

x (k + 1) =(Aθ(k) +B2θ(k)

Fθ(0) (k))x (k) ,

θ (k) = fk (θ (0))

is exponentially stable. That is, there exist αθ(0) ∈ [0, 1) and βθ(0) < ∞ such that,for any θ (0) ∈ Θ, there exists a time-varying, bounded feedback Fθ(0) (k), which maydepend on θ (0), such that∥∥∥∥∥∥

k−1∏i=j

(Afi(θ(0)) +B2fi(θ(0))

Fθ(0) (i))∥∥∥∥∥∥ ≤ βθ(0)α

k−jθ(0),

where the factors of the matrix product are taken in the proper order.Therefore, along every trajectory

{fk (θ (0)) : k ≥ 0

}, the time-varying system is

(uniformly in time) exponentially stabilizable by means of a function F which, asdefined in Definition 3.2, depends on the initial condition θ (0). In this sense, the


control is not quite “closed-loop,” and more importantly there are no assumptionsabout the global properties of the feedback F . In particular, the feedback may notbe a continuous nor even a uniformly bounded function of θ(0). However, in thecase of continuous LDV systems, it was shown in [8] that a stabilizable system has acontinuous, uniformly exponentially stabilizing feedback F : Θ → R

m×n. In this case,the feedback gain takes the form Fθ(k) and does not depend on the initial conditionθ (0), but only the current state θ (k). Hence the controller is “closed-loop.”

The dual concept of detectability has two versions. The first one is uniformdetectability.

Definition 3.3. System (13) is uniformly detectable if there exists a (not neces-sarily continuous) function H : Θ → R

n×p with uniform bound H < ∞ such that, forall θ ∈ Θ, we have ‖Hθ‖ ≤ H, and the system

x (k + 1) =(Aθ(k) +Hθ(k)Cθ(k)

)x (k) ,

θ (k) = fk (θ (0))

is uniformly exponentially stable. That is, there exist an αd ∈ [0, 1) and a βd < ∞such that, for all θ (0) ∈ Θ,

‖x (k)‖ ≤ βdαkd ‖x (0)‖ .

Definition 3.4. System (13) is detectable if there exists a (not necessarily con-tinuous), function H : Z × Θ → R

n×p with bound Hθ(0) < ∞ such that, for all

θ (0) ∈ Θ and all k, we have∥∥Hθ(0) (k)∥∥ ≤ Hθ(0) < ∞, and the system

x (k + 1) =(Aθ(k) +Hθ(0) (k)Cθ(k)

)x (k) ,

θ (k) = fk (θ (0))

is exponentially stable.If f is invertible, the LDV system has an adjoint system running backwards in

time. If a continuous LDV is detectable and f is invertible, then the adjoint systemis stabilizable. It is easily shown that this implies that the adjoint LDV is in factuniformly stabilizable and therefore that the LDV system is uniformly detectable.Thus, if f is invertible and the LDV system is continuous, then uniform detectabilityand detectability are equivalent. Although stabilizability and uniform detectabilityare slightly asymmetric, to avoid putting extra assumptions on f , stabilizable anduniformly detectable continuous LDV systems will be considered.

Since stabilizability only depends on A, B2, and f , we will say that the triple(A,B2, f) is stabilizable to mean that system (13) is stabilizable. Similarly, we saythat the triple (A,C, f) is uniformly detectable to mean that system (13) is uniformlydetectable.

Since an LDV system is a collection of time-varying systems, the following time-varying Lyapunov stability theorem is useful.

Theorem 3.5. Assume that system (13) is uniformly detectable and w ≡ 0.Then there exist an αθo ∈ [0, 1) and a βθo < ∞ such that, for θ (0) = θo and anyx (j) ∈ R

n,

|x (k)| ≤ βθoαk−jθo

|x (j)|if and only if there exists a sequence

{Xfk(θo) : k ≥ 0

}with bound X : Θ → R such

that∥∥Xfk(θo)

∥∥ ≤ Xθo < ∞, Xfk(θo) = X ′fk(θo) ≥ 0, and

A′fk(θo)Xfk+1(θo)Afk(θo) −Xfk(θo) ≤ −C ′

fk(θo)Cfk(θo).(15)


Furthermore, if (15) is satisfied, then αθo and βθo can be taken to depend only on thebound Xθo and on αd and βd in the definition of detectability.

Proof. For θ (0) fixed, the system is a time-varying system. Thus the theoremis simply a statement about the stability of linear time-varying systems and can befound on page 41 in [18].

Corollary 3.6. Assume that system (13) is uniformly detectable and w ≡ 0.Then there exist an α ∈ [0, 1) and a β < ∞ such that

|x (k)| ≤ βαk |x (0)|if and only if there exists a uniformly bounded function X : Θ → R

n×n with Xθ =X ′θ ≥ 0 such that, for all θo ∈ Θ,

A′fk(θo)Xfk+1(θo)Afk(θo) −Xfk(θo) ≤ −C ′

fk(θo)Cfk(θo).(16)

Proof. Since Xθ is uniformly bounded and the system is uniformly detectable,Theorem 3.5 can be applied at each θo ∈ Θ.

The main result of [8] is the following.Theorem 3.7. Suppose that these conditions hold.1. f : Θ → Θ is continuous and Θ is compact.2. The functions A, B2, C, D2 are continuous.3. D′

2θD2θ

> 0.4. C ′

θD2θ= 0 for all θ ∈ Θ, and (A,C, f) is uniformly detectable.

Then the triple (A,B2, f) is stabilizable if and only if there exists a unique, uniformlybounded solution X2 : Θ → R

n×n such that1. X2 satisfies the FARE

X2θ= A′

θX2f(θ)Aθ

− A′θX2f(θ)

B2θ

(D′

2θD2θ

+B′2θX2f(θ)

B2θ

)−1B′

2θX2f(θ)

Aθ + C ′θCθ;(17)

2. X2θ≥ 0.

In this case, the closed-loop control

uLQ (k) := −(D′

2θ(k)D2θ(k)

+B′2θ(k)

X2f(θ(k))B2θ(k)

)−1

B′2θ(k)

X2f(θ(k))Aθ(k)x (k)(18)

uniformly exponentially stabilizes system (13). Moreover, for |x (0)| < ∞ and w ≡ 0,

x′ (0)X2θ(0)x (0) = inf

{ ∞∑k=0

|z (k)|2 : u ∈ l2

},(19)

where the infimum is attained for u = uLQ. Furthermore, X2 is a uniformly contin-uous function. Finally, if X2θ

(k,N + 1) solves the finite horizon Riccati equation,i.e.,

X2θ(k,N + 1) = A′

fk(θ)X2θ(k + 1, N + 1)Afk(θ) + C ′

fk(θ)Cfk(θ)(20)

−A′fk(θ)X2θ

(k + 1, N + 1)B2fk(θ)

×(D′

2fk(θ)

D2fk(θ)

+B′2fk(θ)

X2θ(k + 1, N + 1)B2

fk(θ)

)−1

×B′2fk(θ)

X2θ(k + 1, N + 1)Afk(θ)


with

X2θ(N + 1, N + 1) = C ′

fN+1(θ)CfN+1(θ),

then X2θ(0, N + 1) → X2θ

uniformly in θ.

4. Linear dynamically varying H∞ control. In the following, the H∞ con-trol problem for LDV systems of the general form (13) will be formulated and thesolution will be provided. There are two related problems.

The first is the finite horizon problem. For all θ ∈ Θ, let the terminal weightingXθ (N + 1, N + 1) ≥ 0 be given. The objective in this problem is to find a controllerFu such that, if

u (k) = Fuθo (k,N + 1)

[x (k)w (k)

]for k ≤ N ,

then we have the following.Objective A. For x (0) = 0 there exists an ε > 0 such that, for w ∈ l2 [0, N ] and

θo ∈ Θ,

‖z‖2[0,N ] − γ2 ‖w‖2

[0,N ] + x′ (N + 1)Xθo (N + 1, N + 1)x (N + 1) ≤ −ε ‖w‖2[0,N ] .

The second problem is the infinite horizon problem, where the objective is to finda uniformly exponentially stabilizing controller Fu such that, if

u (k) = Fuθo (k)

[x (k)w (k)

],

then our objective becomes the following.Objective B. For x (0) = 0 there exists an ε > 0 such that, for w ∈ l2 and θo ∈ Θ,

‖z‖2l2− γ2 ‖w‖2

l2≤ −ε ‖w‖2

l2,

and if w = 0 and x (0) �= 0, then x (k) → 0.If Objective B is achieved, then

‖z‖l2‖w‖l2

< γ.

It will be shown that the solution to Objective B is the limit as N → ∞ ofsolutions to Objective A.

4.1. Finite horizon full information controller. For notational simplicity,we define

[Aθ BθCθ Dθ

]:=

Aθ B1θ

B2θ

Cθ D1θD2θ

0 Il 0

and J =:

[Ip 00 −γ2Il

].

Let Xθo (N + 1, N + 1) ≥ 0 be given. In a recursive manner, define

Xθo (k,N + 1) = A′fk(θo)Xθo (k + 1, N + 1)Afk(θo) + C ′

fk(θo)Cfk(θo)(21)

− Lθo (k,N + 1)′R−1θo

(k,N + 1)Lθo (k,N + 1) ,


where

Rθo (k,N + 1) := D′fk(θo)JDfk(θo) + B′

fk(θo)Xθo (k + 1, N + 1) Bfk(θo),(22)

Lθo (k,N + 1) := D′fk(θo)JCfk(θo) + B′

fk(θo)Xθo (k + 1, N + 1)Afk(θo).(23)

We partition R = [ R1 R′2

R2 R3] and L = [ L1

L2] such that R3 ∈ R

m×m and L2 ∈ Rm×n.

With the assumption that Rθo (k,N + 1) is nonsingular, the Schur decompositionyields

Rθo (k,N + 1)

=

[I R′

2θo(k,N + 1, θo)R

−13θo

(k,N + 1)

0 I

] [ ∇θo (k,N + 1) 00 R3θo

(k,N + 1)

]

×[

I 0R−1

3θo(k,N + 1)R2θo

(k,N + 1) I

],

where

∇θo (k,N + 1) := R1θo(k,N + 1)−R′

2θo(k,N + 1)R−1

3θo(k,N + 1)R2θo

(k,N + 1) .(24)

Note that since R3θo(k,N + 1) = D′

2fk(θo)

D2fk(θo)

+ B′2fk(θo)

Xθo (k,N + 1)B2fk(θo)

and D′2fk(θo)

D2fk(θo)

> 0, we have

R3θo(k,N + 1) > 0(25)

whenever Xθo (k,N + 1) ≥ 0. Hence, if

Xθo (k,N + 1) ≥ 0,(26)

∇θo (k,N + 1) ≤ −+I,(27)

then Rθo (k,N + 1) is nonsingular.For X, R, and ∇ defined as above, it is possible to show by completion of squares

(see [17, p. 485]) that for all x (k) and all u,w ∈ l2 [0, N ] we have

‖z‖2[k,N ] − γ2 ‖w‖2

[k,N ] + x′ (N + 1)Xθo (N + 1, N + 1)x (N + 1)(28)

= x′ (k)Xθo (k,N + 1)x (k)

+N∑j=k

(u (j)− uN (j))′R3θo

(j,N + 1) (u (j)− uN (j))

+N∑j=k

(w (j)− wN (j))′ ∇θo (j,N + 1) (w (j)− wN (j)) ,

with

wN (k) := −∇−1θo

(k,N + 1)L∇θo(k,N + 1)x (k) ,

uN (k) := −R−13θo

(k,N + 1)[

L2θo(k,N + 1) R2θo

(k,N + 1)] [ x (k)

w (k)

],

(29)


and where

L∇θo(k,N + 1) := L1θo

(k,N + 1)−R′2θo

(k,N + 1)R−13θo

(k,N + 1)L2θo(k,N + 1) .

(30)

From (25), (27), and (28), it is clear that, for θ (0) = θo,

x′oXθo (0, N + 1)xo

(31)

= supw∈l2[0,N ]

infu∈l2[0,N ]

{‖z‖2

[0,N ] − γ2 ‖w‖2[0,N ] + x′ (N + 1)Xθo (N + 1, N + 1)x (N + 1)

}.

The above is summarized by the following theorem, which is a straightforwardextension of [17, p. 484].

Theorem 4.1. Let us suppose that D′2fk(θo)

D2fk(θo)

> 0 for all k ≤ N and

Xθo (N + 1, N + 1) ≥ 0. In this case, there exists a causal full information control

u (k) = Fuθo (k,N + 1) [ x (k)w (k) ] that satisfies Objective A if and only if, for 0 ≤ k ≤

N + 1, the following conditions hold:1. Xθo(k,N + 1) satisfies the time-varying Riccati recursion (21).2. For some + > 0, (26) and (27) hold.

In this case, the control given by (29) achieves Objective A.

4.2. Infinite horizon full information controller. The second problem isthe infinite horizon problem where the objective is to find a (uniformly in time)exponentially stabilizing controller F such that, if

u (k) = Fθo (k)

[x (k)w (k)

],

then Objective B can be achieved. The following assumptions on system (13) areneeded:

1. f : Θ → Θ is continuous and Θ is compact.2. The system parameters A, B1, B2, C,D1, and D2 are matrix-valued contin-

uous functions of θ.3. D′

2θD2θ

> 0 for all θ ∈ Θ.

4. For all θ ∈ Θ, we have D′2θ

[Cθ D1θ

]= 0, and the triple (A,C, f) is

uniformly detectable.5. The triple (A,B2, f) is stabilizable.

Assumption 4 is equivalent to the following.4′ The triple (A− B2 (D

′2D2)

−1D′

2C, (I −D2 (D′2D2)

−1D′

2)C, f ) is uniformlydetectable.

Indeed, if Assumption 4′ holds, then the feedback

u (k) = −(D′

2fk(θ)

D2fk(θ)

)−1

D′2fk(θ)

Cfk(θ)x (k)

−(D′

2fk(θ)

D2fk(θ)

)−1

D′2fk(θ)

D1fk(θ)

w (k) + r (k)

converts it to Assumption 4. Perhaps these assumptions could be weakened (forexample, see [30]), but they are common.

The main result of the paper is the following.


Theorem 4.2. Suppose Assumptions 1–5 hold. There exists a (uniformly in

time) exponentially stabilizing controller u (k) = Fuθo (k) [x (k)w (k) ] such that Objective

B can be achieved if and only if there exists a uniformly bounded map X∞ : Θ → Rn×n

such that the following hold:1. X∞ satisfies the FARE

X∞θ= C ′

θCθ +A′θX∞f(θ)

Aθ − L′θR

−1θ Lθ,(32)

where

Rθ := D′θJDθ + B′

θX∞f(θ)Bθ,(33)

Lθ := D′θJCθ + B′

θX∞f(θ)Aθ.

2. For some + > 0 and all θ ∈ Θ,

X∞θ≥ 0,(34)

∇θ := R1θ−R′

2θR−1

3θR2θ

≤ −+I.

3. The closed-loop system

x (k + 1) =(Aθ(k) − Bθ(k)R

−1θ(k)Lθ(k)

)x (k)(35)

is uniformly exponentially stable.In this case, the control

u∞ (k) := −R−13θ(k)

[L2θ(k)

R2θ(k)

] [ x (k)w (k)

](36)

achieves Objective B, X∞ is continuous, and the closed-loop system with control (36),that is,

x (k + 1) =(Afk(θo) −B2

fk(θo)R−1

3fk(θo)

L2fk(θo)

)x (k)

+(B1

fk(θo)−B2

fk(θo)R−1

3fk(θo)

L2fk(θo)

)w (k) ,

is a uniformly (in θ) exponentially stable system.The proof of this theorem is withheld until section 5. The proof entails the major

difficulty of proving continuity relative to θ (0), an issue that does not exist in thetraditional time-varying case of [18] and [26]. Even though our approach is inspiredby [17], [30], and [31], the continuity issue of the LDV case makes it of interest in itsown right.

The control u (k) produced by Theorem 4.2 depends on w (k). Since w (k) ismeant to model the linearization error (see section 6), it will likely depend on u (k).Thus u (k) and w (k) are linked by some algebraic relationship, which may not beeasily solved. The following shows how to find a control u (k) that depends on theinformation x (k) only. This type of control is referred to as strictly causal.

Corollary 4.3. Suppose the assumptions of Theorem 4.2 hold and there existsa controller as described. Suppose also that R1θ

≤ −+I. Then the above control canbe taken to be strictly proper. In particular, the control

u∗ (k) := −(R3θ(k)

−R2θ(k)R−1

1θ(k)R′

2θ(k)

)−1

×(L2θ(k)

−R2θ(k)R−1

1θ(k)L1θ(k)

)x (k)

= −∆−1θ(k)L∆θ(k)

x (k) ,


where

∆θ := R3θ−R2θ

R−11θ

R′2θ

and

L∆θ:= L2θ

−R2θR−1

1θL1θ

,

achieves Objective B.Proof. This corollary follows as a minor variation of the proof of Theorem

4.2.Remark 1. The above results show the importance of the FARE (32). Solving

a functional equation may be computationally difficult. However, in [8], [10] severalmethods for solving the FARE associated with a LQ objective were developed. Thesemethods can easily be extended to solving the FARE (32). Furthermore, the stabilityof (35) can be checked via their respective FAREs.

Remark 2. The continuity of the solution to the FARE is crucial when numeri-cally computing it. For example, suppose that Θ = [0, 1] , that there exists a jumpdiscontinuity at some point 0 ≤ ρ ≤ 1, and that

Xθ :=

{0 if θ ≤ ρ,δ otherwise.

Consider the construction of X, an approximation of X, with error ε < δ, i.e., ‖Xθ −Xθ‖ < ε for all θ ∈ Θ. In general, the point ρ would be estimated via some searchmethod. However, unless ρ is known exactly (which entails an infinite search), ‖Xθ∗ −Xθ∗‖ > ε for some θ∗ ∈ Θ. If θ∗ is a fixed point of f , then ‖Xfk(θ∗) − Xfk(θ∗)‖ > εfor all k, and a similar problem occurs if θ∗ is a recurrent point of f . In general, ifX : Θ → R

n×n is continuous and Θ is compact, then X can be estimated by its valueat a finite number of points. If X is not continuous, such an estimate is not possiblein general. It is this continuity issue, and hence the ability to numerically evaluatethe Riccati solution, that is the main distinction between an LDV controller and afamily of infinite horizon, time-varying controllers.

Remark 3. Another difference between an LDV controller and a family of in-finite horizon, time-varying controllers is that the LDV controller guarantees thatthe closed-loop system is uniformly exponentially stable, whereas the family of time-varying controllers only guarantees stability along every trajectory {θ (k) : k ≥ 0}.One situation in which this distinction is important is noise rejection. For example,suppose that the signal w in system (13) is bounded as ‖w‖l∞ ≤ w. Such a situ-ation arises when the f in (2) is different from the f in (3). Then it follows fromsection 6 that the maximum allowable w depends on the parameters αθo and βθo inthe definition of stability. Hence we write wθo . Now suppose that the system is notuniformly exponentially stable, i.e., there exists a sequence {θ(0)i : i ≥ 0} such thateither limi→∞ αθ(0)i = 1 or limi→∞ βθ(0)i = ∞. In this case, even though wθo > 0 foreach θo, we have limi→∞ wθ(0)i = 0; that is, there is no positive bound on the noisethat results in a stable system for all initial conditions θo.

5. Proof of main theorem.

5.1. Necessity. In the subsequent discussion, we assume the following.Assumption A. Assumptions 1–5 of Theorem 4.2 hold, and there exists a stabi-

lizing controller that achieves Objective B.


Since (A,C, f) is uniformly detectable, D′2θD2θ

> 0, and (A,B2, f) is stabilizable,the optimal stabilizing LDV linear-quadratic controller exists (Theorem 3.7). Thatis, there exists a unique, continuous, bounded function X2 : Θ → R

n×n such thatX ′

2θ= X2θ

≥ 0 solves (17). Furthermore, for w ≡ 0,

infu∈l2

‖z‖2l2= x′

oX2θxo,(37)

and this infimum is attained for u given by (18).Define Xθo (k,N + 1) as in (21) with terminal cost X2fN+1(θo)

. It will be shown

(in Lemma 5.10) that

X∞θ= limN→∞

Xθ (0, N + 1)(38)

provides a solution to (32) (see Lemma 5.6) such that system (35) is uniformly expo-nentially stable (see Lemma 5.9) and inequalities (34) are satisfied (see inequalities(59) and (60)). Furthermore, the convergence in (38) is uniform in θ, and hence X∞θ

is a continuous function in θ (see Lemma 5.10) and the control given by (36) satisfiesObjective B (see Lemma 5.4).

The proof of the following lemma follows from an easy adaptation of the argumentsof Section B.2.3 of [17] and from [31].

Lemma 5.1. If Assumption A holds, Xθo (k,N + 1) is given by (21), and∇θo (k,N + 1) is given by (24), then

1. for θo = θ (0) ∈ Θ, all k ≤ N + 1, and N ≥ 0, we have ∇θo (k,N + 1) ≤ −+Iand Xθo (k,N + 1) ≥ 0;

2. for all θo ∈ Θ, there exists a X∞θo< ∞ such that ‖Xθo (k,N + 1)‖ ≤ X∞θo

for all k ≤ N + 1 and all N ≥ 0;3. Xθo (k,N + 1) is monotone increasing in N .

The bound X∞θodepends on θo, so we cannot say that there exists a single bound

on Xθo (k,N + 1) for all θo ∈ Θ. Since Θ is compact, if X∞ is continuous, then X∞is bounded. However, we have not yet shown that X∞ is continuous.

For fixed θo, Xθo (k,N + 1) exists, is bounded, and is nondecreasing in N . Thus

Xθo (k) := limN→∞

Xθo (k,N + 1)

exists for k < ∞. Furthermore, Xθo (k) solves

Xθo (k) = A′fk(θo)Xθo (k + 1)Afk(θo) + C ′

fk(θo)Cfk(θo) − L′θo (k)R

−1θo

(k)Lθo (k) ,(39)

where

Rθo (k) := D′fk(θo)JDfk(θo) + B′

fk(θo)Xθo (k + 1) Bfk(θo),

Lθo (k) := D′fk(θo)JCfk(θo) + B′

fk(θo)Xθo (k + 1)Afk(θo).

This is simply the Riccati equation associated with the infinite horizon, time-varyingH∞ control problem. Note that, since Xθo (k,N + 1) ≥ 0,

Xθo (k) ≥ 0.(40)

Next, since Xθo (k,N + 1) converges, ∇θo (k) := limN→∞ ∇θo (k,N + 1) exists. Fur-thermore, ∇θo (k,N + 1) ≤ −+I (from the finite horizon problem) implies that∇θo (k)


≤ −+I. Furthermore, since Xθo (k) is bounded and D′2θD2θ

> 0 for all θ ∈ Θ, it isclear from (24) that ∇θo (k) is bounded from below. Hence

−∞ < ∇θo (k) ≤ −+I.(41)

Similarly, define L∇θo(k) as the limit of (30).

Next define

u∞ (k) := Fuθo (k)

[x (k)w (k)

]:= −R−1

3θo(k)[

L2θo(k) R2θo

(k)] [ x (k)

w (k)

](42)

and

w∞ (k) := Fwθo(k)x (k) := −∇−1

θo(k)L∇θo

(k)x (k) .(43)

It will be shown that, with θ (0) = θo, (42) is the best control and (43) is the worstdisturbance (in the sense of Objective B).

Lemma 5.2. For w = 0, the control u (k) = u∞ (k) given by (42) makes theclosed-loop system x(k + 1) = Auθo (k)x(k), where

Auθo (k) := Aθ(k) −B2θ(k)R−1

3θo(k)L2θo

(k) ,(44)

exponentially stable.Proof. Since u∞ (k) = −R−1

3θo(k)L2θo

(k)x (k)−R−13θo

(k)R2θo(k)w (k), the closed-

loop system with w = 0 and u = u∞ is

x (k + 1) =(Aθ(k) −B2θ(k)

R−13θo

(k)L2θo(k))x (k)

= Auθo (k)x (k) .

Set Γθo (k) = Xθo (k) − X2fk(θo)

. By Lemma 5.1, Xθo (k,N + 1) ≥ Xθo (k, k) =

X2fk(θo)

. Thus Xθo (k) = limN→∞ Xθo (k,N + 1) ≥ X2fk(θo)

and Γθo (k) ≥ 0. It is

possible to show (see [17, equation (B.2.39)]) that

Γθo (k) ≥ A′uθo

(k) Γθo (k + 1)Auθo (k)(45)

+A′uθo

(k) Γθo (k + 1)B2θ(k)

×(R3θo

(k)−B′2θ(k)

Γθo (k + 1)B2θ(k)

)−1

B′2θ(k)

Γθo (k + 1)Auθo (k) .

However,

Aθ −B2θ

(D′

2θD2θ

+B′2θX2f(θ)

B2θ

)−1B′

2θX2f(θ)

Aθ

is the closed-loop system if the LQ feedback is used, that is, if w = 0 and u = uLQ,where uLQ is given by (18). After some manipulation, we find

Afk(θo) −B2fk(θo)

(D′

2fk(θo)

D2fk(θo)

+B′2fk(θo)

X2fk+1(θo)

B2fk(θo)

)−1

×B′2fk(θo)

X2fk+1(θo)

Afk(θo)

= Auθo (k)

+B2fk(θo)

(R3θo

(k)−B′2fk(θo)

Γθo (k + 1)B2fk(θo)

)−1

B′2fk(θo)

Γθo (k + 1)Auθo (k)


and

B2fk(θ)

(R3θo

(k)−B′2fk(θ)

Γθo (k + 1)B2fk(θ)

)−1

= B2fk(θ)

(D′

2fk(θ)

D2fk(θ)

+B′2fk(θ)

X2fk+1(θo)

B2fk(θ)

)−1

,

which is bounded since D′2θD2θ

> 0 and X2θ≥ 0 for all θ ∈ Θ. Therefore

ξ (k + 1) = Auθo (k) ξ (k) ,

υ (k) = B′2θ(k)

Γθo (k + 1)Auθo (k) ξ (k)

is a uniformly detectable system. Moreover,

(R3θo

(k)−B′2fk(θo)

Γθo (k + 1)B2fk(θo)

)−1

> 0

and ‖Γθo (k) ‖ = ‖Xθo (k)−X2fk(θo)

‖ ≤ ‖Xθo (k)‖ ≤ X∞θo< ∞. Therefore (45) is a

Lyapunov equation, and Theorem 3.5 implies that

x (k + 1) = Auθo (k)x (k)

is an exponentially stable system.We cannot as yet claim that Au as defined by (44) is uniformly exponentially

stable in the sense of Definition 3.1. To conclude uniformly exponential stability,Γθo (k) , the solution to the Lyapunov equation (45), must be uniformly bounded forall θo ∈ Θ and all k. Γθo (k) is uniformly bounded only if Xθo (k), the solution to(39), is uniformly bounded. Lemma 5.7 will show that Xθo (k) is uniformly bounded.

Lemma 5.3. Let u∞ (k) = Fuθo (k) [x (k)w (k) ], where Fu is given by (42). Let

w∞ (k) = Fwθo(k)x (k) be defined as in (43). Then, for w ∈ l2,

‖zθo (Fu, w, xo)‖2l2− γ2 ‖w‖2

l2

= x′oXθo (k)xo +

∞∑k=0

(w (k)− w∞ (k))′ ∇θo (k) (w (k)− w∞ (k)) .(46)

(See the end of section 1 for the definition of the notation zθo (Fu, w, xo).)

Proof. By the previous lemma, u∞ (k) = Fuθo (k) [x (k)w (k) ] is exponentially stabi-

lizing. Hence if w ∈ l2, then x ∈ l2 and x (k) → 0. Furthermore, Xθo (k) is boundedand, by (41), −∞ < ∇θo (k) ≤ −+I. Thus Fwθo

(k) is bounded, where Fw is definedby (43). Hence w∞ ∈ l2. Thus letting N → ∞ in (28) yields (46).

Now it is shown that the control u∞ achieves Objective B.

Lemma 5.4. If x (0) = 0 and u (k) = u∞ (k) = Fuθo (k) [x (k)w (k) ], then, for all

w ∈ l2, ‖z‖2l2− γ2 ‖w‖2

l2≤ −ε ‖w‖2

l2.

Proof. By Assumption A, there exists an exponentially stabilizing control u∗that satisfies Objective B, that is, if u = u∗, x (0) = 0, and w ∈ l2, then x ∈ l2 and

‖zθo (u∗, w, 0)‖2l2−γ2 ‖w‖2

l2≤ −ε ‖w‖2

l2. Since Xθo (k) is bounded and ∇θo (k) ≤ −+I,

Fuθo (k) and Fwθo(k) are bounded. Therefore u∞ (k) = Fuθo (k) [

xθo (u∗, w, 0; k)w (k) ] ∈ l2,

and w∞ (k) = Fwθo(k)xθo (u∗, w, 0; k) ∈ l2. Thus we can take the limit of (28) as


N → ∞, which yields

−ε ‖w‖2l2≥ ‖zθo (u∗, w, 0)‖2

l2− γ2 ‖w‖2

l2(47)

=

∞∑k=0

(u∗ (k)− u∞ (k))′R3θo

(k) (u∗ (k)− u∞ (k))

+

∞∑k=0

(w (k)− w∞ (k))′ ∇θo (k) (w (k)− w∞ (k))

≥∞∑k=0

(w (k)− w∞ (k))′ ∇θo (k) (w (k)− w∞ (k))

= ‖zθo (Fu, w, 0)‖2l2− γ2 ‖w‖2

l2,

where the last equality follows from Lemma 5.3.From (47), it is clear that u∞ is the best control and w∞ is the worst disturbance

in the sense of Objective B.Lemma 5.5. supw∈l2 infu∈l2 ‖zθo (u,w, xo)‖2

l2− γ2 ‖w‖2

l2= ‖zθo (Fu, w∞, xo)‖2

l2−

γ2 ‖w∞‖2l2= x′

oXθo (0)xo.

Proof. Since ∇θo (k) ≤ −+I, if u (k) = u∞ (k) = Fuθo (k) [x (k)w (k) ], then (46)

implies that

supw∈l2

‖zθo (Fu, w, xo)‖2l2− γ2 ‖w‖2

l2

= x′oXθo (0)xo + sup

w∈l2

∞∑k=0

(w (k)− w∞ (k))′ ∇θo (k) (w (k)− w∞ (k))

= x′oXθo (k)xo,

where w∞ (k) = Fwθo(k)x (k). Therefore

supw∈l2

infu∈l2

‖zθo (u,w, xo)‖2l2− γ2 ‖w‖2

l2≤ supw∈l2

‖zθo (Fu, w, xo)‖2l2− γ2 ‖w‖2

l2(48)

= x′oXθo (k)xo.

Similarly, if w = w∞, then infu∈l2 ‖zθo (u,w∞, xo)‖2l2−γ2 ‖w∞‖2

l2= ‖zθo (Fu, w∞, xo)‖2

l2−

γ2 ‖w∞‖2l2= x′

oXθo (k)xo. Therefore

supw∈l2

infu∈l2

‖zθo (u,w, xo)‖2l2− γ2 ‖w‖2

l2≥ infu∈l2

‖zθo (u,w∞, xo)‖2l2− γ2 ‖w∞‖2

l2

= x′oXθo (k)xo.(49)

Combining inequalities (48) and (49) yields the desired result.Up to this point, θ (0) = θo ∈ Θ has been fixed. Xθo (k) is nothing more than

the stabilizing solution of the time-varying Riccati equation associated with the time-varying system

x (k + 1) = Afk(θo)x (k) +B1fk(θo)

w (k) +B2fk(θo)

u (k) ,

z (k) = Cfk(θo)x (k) +D1fk(θo)

w (k) +D2fk(θo)

u (k) .


Since θo is arbitrary, for all θ ∈ Θ, define X∞ : Θ → Rn×n by

X∞θ:= limN→∞

Xθ (0, N + 1) .(50)

Lemma 5.6. The function

X∞ : Θ → Rn×n,

θ �→ X∞θ

satisfies the FARE (32)–(33), viz.,

X∞θ= A′

θX∞f(θ)Aθ + C ′

θCθ

(51)

− (D′θJCθ + B′

θX∞f(θ)Aθ)′ (

D′θJDθ + B′

θX∞f(θ)Bθ)−1 (

D′θJCθ + B′

θX∞f(θ)Aθ).

Proof. Let f (θ1) = θ2. Clearly,

Xθ1 (N + 1, N + 1) = X2fN+1(θ1)= X2fN (θ2)

= Xθ2 (N,N) .(52)

Next, by (21),

Xθ1 (N,N + 1)(53)

= A′fN (θ1)

Xθ1 (N + 1, N + 1)AfN (θ1) + C ′fN (θ1)

CfN (θ1)

−(D′fN (θ1)

JCfN (θ1) + B′fN (θ1)

Xθ1 (N + 1, N + 1)Af(θ1)

)′×(D′fN (θ1)

JDfN (θ1) + B′fN (θ1)

Xθ1 (N + 1, N + 1) BfN (θ1)

)−1

×(D′fN (θ1)

JCfN (θ1) + B′fN (θ1)

Xθ1 (N + 1, N + 1)AfN (θ1)

)= A′

fN−1(θ2)Xθ2 (N,N)AfN−1(θ2) + C ′

fN−1(θ2)CfN−1(θ2)

−(D′fN−1(θ2)

JCfN−1(θ2) + B′fN−1(θ2)

Xθ2 (N,N)AfN−1(θ2)

)′×(D′fN−1(θ2)

JDfN−1(θ2) + B′fN−1(θ2)

Xθ2 (N,N) BfN−1(θ2)

)−1

×(D′fN−1(θ2)

JCfN−1(θ2) + B′fN−1(θ2)

Xθ2 (N,N)AfN−1(θ2)

)= Xθ2 (N − 1, N) .

Repeating the above, we reach the result:

Xθ1 (k,N + 1) = Xθ2 (k − 1, N) .(54)

Setting k = 0 and θ = θ1 in (21) and substituting Xθ2 (0, N) for Xθ1 (1, N + 1) intothe right-hand side yields

Xθ1 (0, N + 1) = A′θ1Xθ2 (0, N)Aθ1 + C ′

θ1Cθ1

− (D′θ1JCθ1 + B′

θ1Xθ2 (0, N)Aθ1)′ (

D′θ1JDθ1 + B′

θ1Xθ2 (0, N) Bθ1)−1

× (D′θ1JCθ1 + B′

θ1Xθ2 (0, N)Aθ1).(55)

Next we take the limit as N → ∞. In order to take this limit, we must ensure that theright-hand side is continuous in Xθ2 (0, N). Since R3θ2

(0, N) ≥ D′2θ2

D2θ2> 0 and


∇θ2 (0, N) ≤ −+I for allN , and since R3 and∇ are continuous functions ofXθ2 (0, N),we have limN→∞ R3θ2

(0, N) ≥ D′2θ2

D2θ2> 0 and limN→∞ ∇θ2 (0, N) ≤ −+I. Thus(

D′θ1JDθ1 +B′

θ1Y Bθ1

)−1exists for Y in a neighborhood of X∞θ2

. Therefore

limN→∞

(D′θ1JDθ1 + B′

θ1Xθ2 (0, N) Bθ1)−1

=(D′θ1JDθ1 + B′

θ1X∞θ2Bθ1)−1

.(56)

Likewise,

limN→∞

(D′θ1JCθ1 + B′

θ1Xθ2 (0, N)Aθ1)=(D′θ1JCθ1 + B′

θ1X∞θ2Aθ1).(57)

Thus

X∞θ1= limN→∞

Xθ1 (0, N + 1)

= limN→∞

(A′θ1Xθ2 (0, N)Aθ1 + C ′

θ1Cθ1

− (D′θ1JCθ1 + B′

θ1Xθ2 (0, N)Aθ1)′ (

D′θ1JDθ1 + B′

θ1Xθ2 (0, N) Bθ1)−1

× (D′θ1JCθ1 + B′

θ1Xθ2 (0, N)Aθ1))

= A′θ1X∞θ2

Aθ1 + C ′θ1Cθ1

− (D′θ1JCθ1 + B′

θ1X∞θ2Aθ1)′ (

D′θ1JDθ1 + B′

θ1X∞θ2Bθ1)

× (D′θ1JCθ1 + B′

θ1X∞θ2Aθ1).(58)

Since f (θ1) = θ2, equation (51) follows.Now we can drop the dependence on the initial condition θo in R, L, and ∇, that

is, Rθ(k) := Rθo (k), Lθ(k) := Lθo (k), ∇θ(k) := ∇θo (k). As a simple consequence of(40), we find

X∞θ≥ 0,(59)

and by (41), we get

∇θ ≤ −+I.(60)

Thus the best control u∞ and worst disturbance w∞ feedback matrices depend onlyon the current state θ (k). That is, (42) and (43) can be rewritten as

u∞ (k) = Fuθ(k)

[x (k)w (k)

]:= −R−1

3θ(k)

[L2θ(k)

R2θ(k)

] [ x (k)w (k)

](61)

and

w∞ (k) = Fwθ(k)x (k) := −∇−1

θ(k)L∇θ(k)x (k) .(62)

Lemma 5.7. The function X∞ given by (50) is uniformly bounded. That is, thereexists a X∞ < ∞ such that ‖X∞θ

‖ < X∞.Proof. Let X2θ

= X ′2θ

≥ 0 be the solution to (17), and define

ALQθ:= Aθ −B2θ

(D′

2θD2θ

+B′2θX2f(θ)

B2θ

)−1B′

2θX2f(θ)

Aθ

to be the closed-loop state transition matrix with w = 0 and u = uLQ given by (18).Define

v(k) :=∞∑i=k

i∏j=k

ALQfj(θo)

′ (X2fi+1(θo)

B1fi(θo)w (i) + C ′

fi+1(θo)D1fi+1(θo)w (i+ 1)

)


and

Gθo (w, xo; k) :=(D′

2fk(θo)

D2fk(θo)

+B′2fk(θo)

X2fk+1(θo)

B2fk(θo)

)−1

B′2fk(θo)

(63)

×(X2

fk+1(θo)Afk(θo)x (k)− v (k)

).

It is possible to show (see [26, Claim 3, p. 257] or [30, Lemma 9.6]) that if w ∈ l2,then

Gθo (w, xo) = arg inf{‖zθo (u,w, xo)‖l2 : u ∈ l2

},(64)

where the notation zθo (u,w, xo) was introduced at the end of section 1. Note thatGθo (w, xo) and therefore zθo (Gθo (w, xo) , w, xo) are linear in (w, xo).

By assumption, there exists a control satisfying Objective B. Thus Gθo (w, 0) mustalso satisfy Objective B, that is,

‖zθo (Gθo (w, 0) , w, 0)‖2l2− γ2 ‖w‖2

l2≤ −ε ‖w‖2

l2(65)

and

‖zθo (Gθo (w, 0) , w, 0)‖l2 < γ ‖w‖l2 .(66)

If w ≡ 0, then v = 0, and by comparing (18) and (63) we see that Gθo (0, xo; k) =uLQ (k), that is, Gθo (0, xo; k) is the optimal LQ control given by (18). Thus, if w = 0,then by (19) we obtain

‖zθo (Gθo (0, xo) , 0, xo)‖2l2= infu∈l2

‖zθo (u, 0, xo)‖2l2= x′

oX2θoxo,(67)

where X2θis the solution to the Riccati equation (17). It was shown in [8] that X2 is

uniformly bounded. Denote this bound by X2, that is, for all θ ∈ Θ, ‖X2θ‖ ≤ X2 < ∞.

Hence

‖zθo (Gθo (0, xo) , 0, xo)‖2l2= x′

oX2θoxo ≤ X2 |xo| < ∞.(68)

Combining (65), (66), and (68) yields

‖zθo (Gθo (w, xo) , w, xo)‖2l2− γ2 ‖w‖2

l2(69)

= ‖zθo (Gθo (0, xo) , 0, xo) + zθo (Gθo (w, 0) , w, 0)‖2l2− γ2 ‖w‖2

l2

= ‖zθo (Gθo (0, xo) , 0, xo)‖2l2+ 2 〈zθo (Gθo (w, 0) , w, 0) , zθo (Gθo (0, xo) , 0, xo)〉

+ ‖zθo (Gθo (w, 0) , w, 0)‖2l2− γ2 ‖w‖2

l2

≤ ‖zθo (Gθo (0, xo) , 0, xo)‖2l2

+ 2 〈zθo (Gθo (w, 0) , w, 0) , zθo (Gθo (0, xo) , 0, xo)〉 − ε ‖w‖2l2

≤ x′oX2θo

xo + 2γ ‖w‖l2√

x′oxoX2 − ε ‖w‖2

l2

= x′oX2θo

xo + ‖w‖l2(2γ|√

x′oxoX2 − ε ‖w‖l2

)

≤ x′oX2θo

xo + max‖w‖∈R

{‖w‖l2

(2γ |xo|

√X2 − ε ‖w‖l2

)}

≤ x′oX2θo

xo +γ2X2 |xo|2

ε≤ |xo|2

(X2 +

γ2X2

ε

)< ∞.


Thus

supw∈l2

infu∈l2

‖zθo (u,w, xo)‖2l2− γ2 ‖w‖2

l2

= supw∈l2

‖zθo (Gθo (w, xo) , w, xo)‖2l2− γ2 ‖w‖2

l2≤ |xo|2

(X2 +

γ2X2

ε

)< ∞.

Lemma 5.5 implies that

x′oX∞θo

xo = supw∈l2

infu∈l2

‖zθo (u,w, xo)‖2l2− γ2 ‖w‖2

l2≤ |xo|2

(X2 +

γ2X2

ε

)< ∞,

(70)

which concludes the proof.Note that the worst disturbance has the property,

‖w∞‖2l2≤ P |xo|2 ,(71)

where

P :=4γ2X2

ε2|xo|2 .(72)

To see this, observe that if ‖w∞‖2l2

> P |xo|2, equation (69) implies that

‖zθo (Gθo (w∞, xo) , w∞, xo)‖2l2− γ2 ‖w∞‖l2

≤ x′oX2θo

xo + ‖w‖l2(2γ |xo|

√X2 − ε ‖w‖l2

)< x′

oX2θoxo = ‖zθo (Gθo (0, xo) , 0, xo)‖2

l2.

That is, the cost resulting from w ≡ 0 is larger than the cost resulting from w = w∞,which contradicts the maximizing property of w∞.

Lemma 5.8. For w = 0, u (k) = u∞ (k) uniformly exponentially stabilizes thesystem.

Proof. Since X∞ is uniformly bounded, X∞ (θ) defined in Lemma 5.1 is uniformlybounded. The proof of Lemma 5.2 can be applied with no changes to conclude thatAu is uniformly exponentially stable.

Lemma 5.9. The closed-loop system with u (k) = u∞ (k) and w (k) = w∞ (k)is uniformly exponentially stable. In other words, the system x (k + 1) = (Aθ(k)−Bθ(k)R

−1θ(k)Lθ(k))x (k) is uniformly exponentially stable.

Proof. Let |xo| ≤ 1. Define w∞ as

w∞ (k) = Fwθ(k)xθo (Fu, w∞, xo; k) .

Then w∞ is a linear function of xo. By Lemmas 5.5 and 5.7,

supw∈l2

infu∈l2

‖zθo (u,w, xo)‖2l2− γ2 ‖w‖2

l2= ‖zθo (Fu, w∞, xo)‖2

l2− γ2 ‖w∞‖2

l2

= x′oX∞θo

xo ≤ X∞ |xo|2 ,

and, by (71), ‖w∞‖2l2≤ P |xo|2. Thus

‖zθo (Fu, w∞, xo)‖2l2= γ2 ‖w∞‖2

l2+ x′

oX∞θoxo ≤

(γ2P + X∞

) |xo|2 .(73)


Note that, if w (k) = w∞ (k) and u (k) = Fuθ(k)[ x (k)w∞ (k) ], then x (k + 1) = (Aθ(k)−

Bθ(k)R−1θ(k)Lθ(k))x (k).

Define the system

x (k + 1) =(Aθ(k) − Bθ(k)R

−1θ(k)Lθ(k)

)x (k) + r (k) ,(74)

v (k) =

[ −∇−1θ(k)L∇θ(k)

Cθ(k)

]x (k) ,

where

C = C −D1∇−1L∇ +D2

(R−1

3 R2∇−1L∇ −R−13 L2

).

Then v = [ w∞zθo (Fu, w∞, xo) ]. Fix j ≥ 0 and set r (k) = roδ (k − j), that is, r (k) = 0 for

k �= j and r (j) = ro. Then (73) implies that∥∥zfj(θo) (Fu, w∞, ro)

∥∥2l2≤ (γ2P + X∞

) |ro|2.Likewise, ‖w∞‖2

l2≤ P |ro|2. Therefore

‖v‖2l2=∥∥zfj(θo) (Fu, w∞, ro)

∥∥2l2+ ‖w‖2

l2≤ ((γ2P + X∞

)+ P) |ro|2 .(75)

Note that there exists a matrix[

H1 H2

]such that

(A− BR−1L

)+[

H1 H2

] [ −∇−1L∇(C −D1∇−1L∇ +D2

(R−1

3 R2∇−1L∇ −R−13 L2

)) ](76)

is uniformly exponentially stable. For example, set H1 = −B1 − H2D1 and H2 =HdC (C ′C)

+C ′ − B2 (D

′2D2)

−1D′

2, where Hd is the feedback such that A − HdCis uniformly exponentially stable, the existence of which is guaranteed by the de-tectability assumption, and (C ′C)

+is the pseudoinverse of C ′C. Since Hd is bounded,

D′2D2 > 0, D2, B1, C, D1 are uniformly continuous, Θ is compact, and C (C ′C)

+C ′

is bounded,1 there is an H < ∞ such that[

H1 H2

]′ [H1 H2

] ≤ H.Now, let

y (k + 1) =

((Aθ(k) − Bθ(k)R

−1θ(k)Lθ(k)

)+[

H1θ(k)H2θ(k)

] [ Cθ(k)∇−1θ(k)L∇θ(k)

])y (k)

(77)

− [ H1θ(k)H2θ(k)

]v (k) + roδ (k − j) ,

that is, y is an estimate of x. Since system (77) is uniformly exponentially stable,there exists an R < ∞ such that

‖y‖l2 ≤ R∥∥r − [ H1 H2

]v∥∥l2≤ R ‖r‖l2 + HR ‖v‖l2

≤ R |ro|+ HR√((

γ2P + X∞)+ P) |ro|

≤(R+RH

√((γ2P + X∞

)+ P)) |ro| .

1Although C (C′C)+ C is not continuous, ‖C (C′C)+ C‖ ≤ 1.


On the other hand, if the system is initially at rest, that is, y (0) = x (0) = 0, theny (k) = x (k). Thus ‖y‖l2 = ‖x‖l2 , and therefore

‖x‖l2 ≤(R+RH

√((γ2P + X∞

)+ P)) |ro| .

Let Φθo (k, j) be the state transition matrix of system (74) with initial conditionsθ (0) = θo, x (0) = 0, and let r (i) = roδ (i− j). Then we have

‖x‖2[j,∞) =

∞∑i=j

x′ (i)x (i) =∞∑i=j

‖Φθo (i, j) r (j)‖2

≤(R+RH

√((γ2P + X∞

)+ P))2

|ro|2 .

Furthermore, for i ≥ j, ‖Φθo (i, j)‖2 ≤ (R + RH√((

γ2P + X∞)+ P))2. Applying

standard techniques, we find that

K ‖Φθo (j +K, j)‖2=

K∑i=j

‖Φθo (j +K, j)‖2

≤K∑i=j

‖Φθo (j +K, i)‖2 ‖Φθo (i, j)‖2

≤(R+RH

√((γ2P + X∞

)+ P))2 K∑

i=j

‖Φθo (i, j)‖2

≤(R+RH

√((γ2P + X∞

)+ P))4

.

Choosing K ∈ Z such that K ≥ √2(R + RH

√((γ2P + X∞) + P ))4 yields

‖Φ (j +K, j, θo)‖2 ≤ 1/√2. Since this is true for all j and all θo, setting M ∈ Z

with M ≥ 0 and k − (j +MK) < K, we conclude that

‖Φθo (k, j)‖ ≤ ‖Φθo (k, j +MK)‖M∏m=1

‖Φθo (j +mK, j + (m− 1)K)‖

≤(R+RH

√((γ2P + X∞

)+ P))(1

2

)M

≤(R+RH

√((γ2P + X∞

)+ P))

2

(1

2

) k−jK

.

That is, system (74) is uniformly exponentially stable.Lemma 5.10. As N → ∞, Xθ (0, N + 1) → X∞θ

uniformly in θ and X∞ is acontinuous function.

Proof. Let x (k + 1) = (Aθ(k) − Bθ(k)R−1θ(k)Lθ(k))x (k). Then, by Lemma 5.9,

x (k) → 0 uniformly exponentially fast. Set w∞ (k) = Fwθ(k)x (k) as in (62) and

define

wN (k) :=

{w∞ (k) for k ≤ N,0 otherwise.


Since X∞ is uniformly bounded and ∇θ ≤ −+I, it follows that Fw is uniformlybounded. Since x (k) → 0 uniformly exponentially fast and Fw is bounded,w∞ → 0 uniformly exponentially fast. Therefore limN→∞ ‖w∞ − wN‖l2 =limN→∞ ‖w∞‖[N+1,∞) = 0, where the convergence is uniformly exponentially fast.

Recall the following: (31) states that

x′oXθo (0, N + 1)xo(78)

= supw∈l2[0,N ]

infu∈l2[0,N ]

{‖z‖2

[0,N ] − γ2 ‖w‖2[0,N ]

+ x′ (N + 1)Xθo (N + 1, N + 1)x (N + 1)}.

From (19) of Theorem 3.7, it follows that

x (N + 1)′X2fN+1(θo)

x (N + 1) = infu∈l2

‖z‖2[N+1,∞) .(79)

From (64), we have

infu∈l2

‖zθo (u,w, xo)‖2[0,∞) − γ2 ‖w‖2

[0,∞) = ‖zθo (Gθo (w, xo) , w, xo)‖2[0,∞) − γ2 ‖w‖2

[0,∞) .

(80)

Combining (64) and Lemma 5.5 yields

‖zθo (Gθo (w∞, xo) , w∞, xo)‖2[0,∞) − γ2 ‖w∞‖l2 = x′

oX∞θoxo.(81)

From (66), we have

‖zθo (Gθo (w, 0) , w, 0)‖2[0,∞) < γ2 ‖w‖2

[0,∞) ,(82)

and, from inequality (71), we have

‖w∞‖2l2≤ P |xo|2 .(83)

Combining the preceding relations yields the following string:

x′oXθo (0, N + 1)xo

= supw∈l2[0,N ]

infu∈l2[0,N ]

{‖zθo (u,w, xo)‖2

[0,N ] − γ2 ‖w‖2[0,N ]

+ x′ (N + 1)X2fN+1(θo)x (N + 1)

}= sup

{w∈l2:w(k)=0, k≥N}infu∈l2

{‖zθo (u,w, xo)‖2

[0,∞) − γ2 ‖w‖2[0,∞)

}

= sup{w∈l2:w(k)=0, k≥N}

{‖zθo (Gθo (w, xo) , w, xo)‖2

[0,∞) − γ2 ‖w‖2[0,∞)

}≥ ‖zθo (Gθo (wN , xo) , wN , xo)‖2

[0,∞) − γ2 ‖wN‖2[0,∞)

= ‖zθo (Gθo (w∞, xo) , w∞, xo) + zθo (Gθo (wN − w∞, 0) , wN − w∞, 0)‖2[0,∞)

− γ2 ‖wN‖2[0,∞)


= ‖zθo (Gθo (w∞, xo) , w∞, xo)‖2[0,∞) + ‖zθo (Gθo (wN − w∞, 0) , wN − w∞, 0)‖2

[0,∞)

− γ ‖wN‖2[0,∞)

+ 2 〈zθo (Gθo (w∞, xo) , w∞, xo) , zθo (Gθo (wN − w∞, 0) , wN − w∞, 0)〉≥ ‖zθo (Gθo (w∞, xo) , w∞, xo)‖2

[0,∞) − γ2 ‖wN‖2[0,∞)

+ 2 〈zθo (Gθo (w∞, xo) , w∞, xo) , zθo (Gθo (wN − w∞, 0) , wN − w∞, 0)〉≥ ‖zθo (Gθo (w∞, xo) , w∞, xo)‖2

[0,∞) − γ2 ‖w∞‖2[0,∞)

− 2 ‖zθo (Gθo (w∞, xo) , w∞, xo)‖[0,∞) ‖zθo (Gθo (wN − w∞, 0) , wN − w∞, 0)‖[0,∞)

≥ x′oX∞θo

xo − 2

(√γ2 ‖w∞‖2

+ x′oX∞θo

xo

)γ ‖w∞ − wN‖[0,∞)

≥ x′oX∞θo

xo − 2 |xo|(√

γ2P + X∞

)γ ‖w∞ − wN‖[0,∞) .

Lemma 5.1 implies that X∞θo−Xθo (0, N + 1) ≥ 0. Thus

0 ≤ x′o

(X∞θo

−Xθo (0, N + 1))xo ≤ 2γ |xo|

(√γ2P + X∞

)‖w∞ − wN‖[0,∞) .

Since ‖w∞ − wN‖[0,∞) → 0 uniformly in θ and exponentially in N , and since

2γ(√

γ2P + X∞) does not depend on θo, we have Xθo (0, N + 1) → X∞θouniformly

in θ and exponentially in N .Since X2 is continuous, Xθ (0, N + 1) is continuous in θ for N < ∞. Since Θ

is compact, and X (0, N + 1) → X∞ in the uniform metric, Theorem 7.1.4 in [24]implies that X∞ is continuous.

The time-invariant version of the first claim of this lemma can be found in [31].

5.2. Sufficiency. Suppose that the assumptions of the theorem hold and that(32), (33), and (34) hold. It will be shown that the control given by (36) is internallystabilizing and, if u = u∞ as defined by (42), then Objective B is satisfied. This proofis similar to the proof given in [17].

Under the above conditions, (32) can be written as

X∞θ= C ′

θCθ +A′θX∞f(θ)

Aθ − L′2θR−1

3θL2θ

− L′∇θ

∇−1θ L∇θ

.

It follows that[A′θ C ′

θ

B′2θ

D′2θ

] [X∞f(θ)

00 I

] [Aθ B2θ

Cθ D2θ

]

=

[X∞θ

+ L′∇θ

∇−1θ L∇θ

00 0

]+

[L′

2θ

R3θ

]R−1

3θ

[L2θ

R3θ

].

Multiplying both sides of this equality, by [I 0

−R−13θL2θ

I ] on the right and by the

transpose on the left, and taking the (1,1) block yields

X∞θ=(Aθ −B2θ

R−13θ

L2θ

)′X∞f(θ)

(Aθ −B2θ

R−13θ

L2θ

)(84)

− L′∇θ

∇−1θ ∇θ∇−1

θ L∇θ+(Cθ −D2θ

R−13θ

L2θ

)′ (Cθ −D2θ

R−13θ

L2θ

).


Since Aθ − BθR−1θ Lθ = Aθ − B2θ

R−13θ

L2θ− (B1θ

−B2θR−1

3θL2θ

)∇−1θ L∇θ

is assumedto be uniformly exponentially stable, we conclude that the triple((

Aθ −B2θR−1

3θL2θ

),(∇−1θ L∇θ

), f)

is uniformly detectable. Since ∇θ ≤ −+I, ∇−1θ is uniformly bounded. Since X∞ is

uniformly bounded,∇−1θ L∇θ

is uniformly bounded. Thus (84) is a Lyapunov equation,and Corollary 3.6 implies that

ξ (k + 1) =(Aθ(k) −B2θ(k)

R−13θ(k)

L2θ(k)

)ξ (k)(85)

is a uniformly exponentially stable system. Therefore the control u = u∞ is uniformlyexponentially stabilizing.

Since system (85) is uniformly exponentially stable, if u = u∞ and w ∈ l2, thenx ∈ l2 and limk→∞ x (k) = 0. Thus, if u = u∞, then (28) implies that, for all N ,

‖z‖2[0,N ] − γ2 ‖w‖2

[0,N ] + x′ (N + 1)X∞fN+1(θo)x (N + 1)(86)

= x′ (0)X∞θox (0) +

N∑k=0

(w (k)− w∞ (k))′ ∇fk(θo) (w (k)− w∞ (k)) ,

where w∞ (k) := −∇−1fk(θo)

L∇fk(θo)

x (k). Since x,w ∈ l2, it follows that u, z ∈ l2.

Furthermore, ∇ is bounded. Thus we can let N → ∞ in (86), and for xo = 0,

‖z‖2l2− γ ‖w‖2

l2=

∞∑k=0

(w (k)− w∞ (k))′ ∇fk(θo) (w (k)− w∞ (k)) .(87)

Since system (85) is stable and causal, the closed-loop system with u = u∞, viz.,

[x (k + 1)w (k)− w∞ (k)

](88)

=

(Aθ(k) −B2θ(k)

R−13θ(k)

L2θ(k)

) (B1θ(k)

−B2θ(k)R−1

3θ(k)R2θ(k)

)(∇−1

θ(k)L∇

θ(k)

)I

[ x (k)

w (k)

],

is l2-stable and causal. The inverse of this system (see [36]) is[ξ (k + 1)w (k)

]=

[Aθ(k) −Bθ(k)Cθ(k) Dθ(k)

] [ξ (k)w (k)− w∞ (k)

]

with

Aθ = Aθ −B2θR−1

3θL2θ

− (B1θ−B2θ

R−13θ

R2θ

)∇−1θ L∇θ

= Aθ − BθR−1θ Lθ,

Bθ = − (B1θ−B2θ

R−13θ

R2θ

),

Cθ = −∇−1θ L∇θ

,

Dθ = I.


Since ξ (k + 1) = (Aθ(k) − Bθ(k)R−1θ(k)Lθ(k))ξ (k) is uniformly exponentially stable, the

inverse of system (88) is uniformly exponentially stable and hence l2 stable. Thusthere exists a δ > 0 such that, for all θo ∈ Θ,

‖w‖2l2≤ 1

δ‖w − w∞‖2

l2.

Since ∇ ≤ −+I, equation (87) implies that

‖z‖2l2− γ2 ‖w‖2

l2≤ −+ ‖w − w∞‖2

l2≤ −δ+ ‖w‖2

l2= −ε ‖w‖2

l2.

6. Controlling nonlinear systems with linear dynamically varying H∞

controllers. In the preceding section, a technique for stabilizing an LDV systemsubject to an H∞ disturbance rejection requirement was developed. Here, it will firstbe shown that the LDV controller for the linearized tracking error system can also beused to stabilize the nonlinear tracking error dynamics, in a scheme that works alongevery trajectory, provided that the initial tracking error is small enough (section 6.1).Next, some issues quite specific to the H∞ implementation of the tracking scheme (tobe published elsewhere) will be briefly surveyed.

6.1. Stability of a closed-loop nonlinear system. To make H∞ design rele-vant to nonlinear tracking performance improvement, the guiding idea is to write thenonlinearity η in the tracking error dynamics (6) as a feedback from an output z(k)to a disturbance w(k). To this end, we introduce the factorization

η(x, u, θ) =[

ηx (x, u, θ) ηu (x, u, θ)] [ x

u

]= B1θ

η (x, u, θ)[

Cθ D2θ

] [ xu

].

(89)

Here we simply take

B1θ:= In×n, Cθ : =

[In×n0m×n

], D2θ

:=

[0n×mIm×m

],(90)

η (x, u, θ) :=[

ηx (x, u, θ) ηu (x, u, θ)],

where ηx and ηu are defined by (8) and (9). The error due to linearization can bemodeled as a feedback from z to w:

η(x(k), u(k), θ(k)) = B1θ(k)w (k) ,

w(k) = η (x (k) , u (k) , θ (k)) z (k) ,

z (k) = Cθ(k)x (k) +D2θ(k)u (k) .

Here D1 = 0. Clearly B1 : Θ → Rn×n, C : Θ → R

(n+m)×n, and D2 : Θ → R(n+m)×m

are continuous functions, and the triple (A,C, f) is detectable. Hence, assuming that(A,B2, f) is stabilizable, Theorem 4.2 or Corollary 4.3 can be applied to generate acontroller.

Theorem 6.1. Let (4) hold. Define A, B1, B2, C, D1, and D2 as above andassume that the triple (A,B2, f) is stabilizable. Let F be the H∞ controller such thatsupw∈l2 ‖z‖l2/‖w‖l2 < γ for some γ < ∞. Then there exists an RCapture > 0 suchthat, if u (k) = Fθ(k) (ϕ (k)− θ (k)) and |ϕ (0)− θ (0)| < RCapture, then |ϕ (k)− θ (k)| →0 as k → ∞.


Proof. Define η (x, u) := sup {‖η (x, u, θ)‖ : |x| ≤ x, |u| ≤ u, θ ∈ Θ}. By (10) and(11), we obtain

η (x, u) → 0 as x, u → 0.(91)

It follows that there exist x∗, u∗ > 0 such that η (x∗, u∗) ≤ 1γ . Now define

h (x, u, θ) :=

{η (x, u, θ) for |x| < x∗ and |u| < u∗,0n×(n+m) otherwise.

(92)

Thus, for all x and u, supθ∈Θ ‖h (x, u, θ)‖ ≤ 1γ . Consider the following closed-loop

LDV system:

ξ (k + 1) = Aθ(k)ξ (k) +B1θ(k)ω (k) +B2θ(k)

υ (k) ,(93)

ζ (k) = Cθ(k)ξ (k) +D2θ(k)υ (k) ,

ω (k) = h (ξ (k) , υ (k) , θ (k)) ζ (k) ,

υ (k) = Fθ(k)ξ (k) ,

θ (k + 1) = f (θ (k)) .

Since supw∈l2 ‖z‖l2/‖w‖l2 < γ, the small gain theorem of [32] implies that system (93)is externally l2 stable. Since Aθ + B2θ

Fθ is uniformly exponentially stable, system(93) is also internally l2 stable. Therefore there exist a Gx ≥ 1 and a Gu > 0 suchthat ‖ξ‖l∞ ≤ ‖ξ‖l2 < Gx |ξ (0)| and ‖υ‖l∞ ≤ ‖υ‖l2 < Gu |ξ (0)|. Now set

RCapture := min

(x∗

Gx,u∗

Gu

).

If |ξ (0)| ≤ RCapture, then

‖ξ‖l∞ < Gx |ξ (0)| ≤ x∗

and

‖υ‖l∞ < Gu |ξ (0)| ≤ u∗.

By the above inequalities and (92), we conclude that, for all k, h (ξ (k) , υ (k) , θ (k)) =η (ξ (k) , υ (k) , θ (k)). Thus, if

|x (0)| < min

(x∗

Gx,u∗

Gu

),

then, by the uniqueness of solutions to difference equations, the closed-loop LDVsystem

x (k + 1) = Aθ(k)x (k) +B1θ(k)w (k) +B2θ(k)

u (k) ,(94)

z (k) = Cθ(k)x (k) +D2θ(k)u (k) ,

w (k) = η (x (k) , u (k) , θ) z (k) ,

u (k) = Fθ(k)x (k) ,

θ (k + 1) = f (θ (k))

is l2 stable, and furthermore, ‖x‖l∞ < x∗ and ‖u‖l∞ < u∗. Since (94) is the trackingerror of the closed-loop nonlinear system, we conclude that |x (k)| = |ϕ (k)− θ (k)| →0 as k → ∞.


6.2. Further considerations. Writing the nonlinearity η (x, u, θ) as a boundedfeedback η (x, u, θ) from an output z to the input w (see (10), (11), (89)) yields anH∞ design that attenuates the effect of the nonlinearity and hence amplifies theinitial allowable tracking error. It is further possible to optimize this procedure byfactoring η in such a way that ‖η‖ ≤ 1 (see [7] for details). The suboptimal H∞

controller is continuous, and therefore an approximation of the LDV H∞ controllercan be constructed in the same way that an approximation of the LDV quadraticcontroller was constructed in [8]. The fact that the H∞ controller is guaranteed tobe continuous under the condition that it be suboptimal does not prove that thesuboptimality condition for continuity is necessary. In fact, an example based onthe Henon map shows that the suboptimal controller becomes discontinuous as γapproaches γo, the optimal H∞ tolerance.

7. Conclusion. Suboptimal H∞ controllers for LDV systems have been de-veloped. Like the LDV quadratic controllers, these H∞ controllers are continuousfunctions. The H∞ method has distinct advantages over the LQ method, in that H∞

can be tuned to minimize the effect of linearization and it is possible to find a lowerbound on the maximum allowable initial tracking error.

REFERENCES

[1] H. Abou-Kandil, G. Freiling, and G. Jank, Solution and asymptotic behavior of coupled Ric-cati equations in jump linear systems, IEEE Trans. Automat. Control, 39 (1994), pp. 1631–1636.

[2] H. Abou-Kandil, G. Freiling, and G. Jank, On the solution of discrete-time Markovianjump linear quadratic control problems, Automatica J. IFAC, 31 (1995), pp. 765–768.

[3] P. Apkarian, Advanced gain-scheduling techniques for uncertain systems, IEEE Trans. ControlSyst. Technol., 6 (1998), pp. 21–32.

[4] P. Apkarian, P. Gahinet, and G. Becker Self-scheduled H∞ control of linear parameter-varying systems: Design example, Automatica J. IFAC, 31 (1995), pp. 1251–1261.

[5] G. Becker and A. Packard, Robust performance of linear parameterically varying systemsusing parametrically-dependent linear feedback, Systems Control Lett., 23 (1995), pp. 205–215.

[6] G. Becker, A. Packard, and G. Balas, Control of parametrically-dependent linear systems:A single quadratic Lyapunov approach, in American Control Conference, 1993, pp. 2795–2799.

[7] S. Bohacek, Controlling Systems with Complicated Asymptotic Dynamics with Linear Dy-namically Varying Control, Ph.D. thesis, University of Southern California, Los Angeles,CA, 1999.

[8] S. Bohacek and E. Jonckheere, Linear dynamically varying LQ control of nonlinear systemsover compact sets, IEEE Trans. Automat. Control, 46 (2001), pp. 840–852.

[9] S. Bohacek and E. Jonckheere, Linear dynamically varying H∞ control chaos, in NonlinearControl Systems Design Symposium (NOLCOS), Enschede, The Netherlands, IFAC, 1998,pp. 744–749.

[10] S. Bohacek and E. Jonckheere, Linear dynamically varying (LDV) systems versus jumplinear systems, in Proceedings of the AACC American Control Conference, San Diego,CA, 1999, pp. 4024–4028.

[11] S. Bohacek and E. Jonckheere, Analysis and synthesis of linear set valued dynamicallyvarying (LSVDV) systems, in Proceedings of the AACC American Control Conference,Chicago, IL, 2000, pp. 2770–2774.

[12] I. Borno and Z. Gajic, Parallel algorithm for solving coupled algebraic Lyapunov equationsof discrete-time jump linear systems, Comput. Math. Appl., 30 (1995), pp. 1–4.

[13] W. Bown, Mathematicians learn how to tame chaos, New Scientist, 134 (1992), p. 16.[14] S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in

System and Control Theory, SIAM, Philadelphia, PA, 1994.[15] M. Fragoso, J. Ribeiro do Val, and D. Pinto, Jump linear H∞ control: The discrete-time

case, Control Theory Adv. Tech., 10 (1995), pp. 1459–1474.


[16] P. Gahinet, P. Apkarian, and M. Chilali, Affine parameter-dependent Lyapunov functionfor real parameter uncertainty, in Proceedings of the 33rd Conference on Decision andControl, Lake Buena Vista, FL, IEEE, 1994, pp. 2026–2031.

[17] M. Green and D. Limebeer, Linear Robust Control, Prentice–Hall, Englewood Cliffs, NJ,1995.

[18] A. Halanay and V. Ionescu, Time-Varying Discrete Linear Systems, Birkhauser-Verlag,Basel, Switzerland, 1994.

[19] Y. Ji and H. Chizeck, Jump linear quadratic Gaussian control: Steady-state solution andtestable conditions, Control Theory Adv. Tech., 10 (1990), pp. 1459–1474.

[20] J. Burken,Private communication, NASA Dryden Flight Research Center, Edwards, CA, 1999.[21] E. Jonckheere, P. Lohsoonthorn, and S. Bohacek, From Sioux-City to the X-33, in Annu.

Rev. Control, 23 (1999), pp. 91–108.[22] A. Katok and B. Hasselblatt, Introduction to the Modern Theory of Dynamical Systems,

Cambridge University Press, London, UK, 1995.[23] I. E. Kose, F. Jabbari, and W. E. Schmitendorf, A direct characterization of L2-gain con-

trollers for LPV systems, in Proceedings of the 35th Conference on Decision and Control,Kobe, Japan, IEEE, 1996, pp. 3990–3995.

[24] J. R. Munkres, Topology, Prentice–Hall, Englewood Cliffs, NJ, 1975.[25] A. Packard, Gain scheduling via linear fractional transformations Systems Control Lett., 22

(1994), pp. 79–92.[26] M. A. Peters and P. A. Iglesias, Minimum Entropy Control for Time-Varying Systems.

Birkhauser, Boston, Cambridge, MA, 1997.[27] S. Y. Pilyugin, The Space of Dynamical Systems with the C0-Topology. Springer-Verlag, New

York, 1991.[28] J. S. Shamma and M. Athans, Guaranteed properties of gain scheduled control for linear

parameter-varying plants, Automatica J. IFAC, 27 (1991), pp. 559–564.[29] J. S. Shamma and D. Xiong, Set-valued methods for linear parameter varying systems, Auto-

matica J. IFA, 35 (1999), pp. 1081–1089.[30] A. Stoorvogel, The H(infinity) Control Problem: A State Space Approach, Prentice–Hall,

Englewood Cliffs, NJ, 1992.[31] A. A. Stoorvogel and A. J. T. Weeren, The discrete-time Riccati equation related to the

H∞ control problem, IEEE Trans. Automat. Control, 39 (1994), pp. 686–691.[32] M. Vidyasagar, Nonlinear Systems Analysis, Prentice–Hall, Englewood Cliffs, NJ., 1993.[33] F. Wu, X. H. Yang, A. Packard, and G. Becker, Induced l-2 norm control for LPV systems

with bounded parameter variation rates, Internat. J. Robust Nonlinear Control, 6 (1996),pp. 983–998.

[34] J. Yu and A. Sideris, H∞ control with parameteric Lyapunov functions, Systems ControlLett., 30 (1997), pp. 57–69.

[35] J. Yu and A. Sideris, H∞ control with parametric lyapunov functions, in Proceedings of the34th Conference on Decision and Control, New Orleans, LA, IEEE, 1995, pp. 2547–2552.

[36] K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Prentice–Hall, UpperSaddle River, NJ, 1996.

Documents

NONLINEAR TRACKING OVER COMPACT SETS WITH LINEAR H