International Journal of Control, 2013Vol. 86, No. 8, 1324–1337, http://dx.doi.org/10.1080/00207179.2013.801082
Generalised polynomial chaos expansion approaches to approximate stochasticmodel predictive control†
Kwang-Ki K. Kima and Richard D. Braatzb,∗
aUniversity of Illinois at Urbana-Champaign, Urbana, IL, 61801, USA; bMassachusetts Institute of Technology,Cambridge, MA, 02139, USA
(Received 27 December 2012; final version received 27 April 2013)
This paper considers the model predictive control of dynamic systems subject to stochastic uncertainties due to parametricuncertainties and exogenous disturbance. The effects of uncertainties are quantified using generalised polynomial chaosexpansions with an additive Gaussian random process as the exogenous disturbance. With Gaussian approximation of theresulting solution trajectory of a stochastic differential equation using generalised polynomial chaos expansion, convex finite-horizon model predictive control problems are solved that are amenable to online computation of a stochastically robustcontrol policy over the time horizon. Using generalised polynomial chaos expansions combined with convex relaxationmethods, the probabilistic constraints are replaced by convex deterministic constraints that approximate the probabilisticviolations. This approach to chance-constrained model predictive control provides an explicit way to handle a stochasticsystem model in the presence of both model uncertainty and exogenous disturbances.
Keywords: stochastic model predictive control; generalised polynomial chaos expansions; chance constraints; Boole’sinequality; convex relaxation; semidefinite programming; stochastic differential equations
1. Introduction
In recent years, stochastic programming formulations formodel predictive control (MPC, aka receding horizon con-trol) have been intensively studied in the context of manydifferent areas of application including robot and vehi-cle path planning (Blackmore & Ono, 2009; Blackmore,Ono, Bektassov, & Williams, 2010; Blackmore, Ono, &Williams, 2011), network traffic control (Yan & Bitmead,2005), chemical processes (Li, Wendt, & Wozny, 2000;Schwarm & Nikolaou, 1999; van Hessem & Bosgra, 2004)and economics (Couchman, Cannon, & Kouvaritakis, 2006;Herzog, Dondi, & Geering, 2007; Zhu, Li, & Wang, 2004).In such control problems, stochastic models are representedin terms of stochastic differential equations (SDEs) withthe stochasticity resulting from exogenous disturbances,plant/model mismatches and sensor noise.
Robust MPC formulations can be categorised as beingeither deterministic or stochastic, based on the representa-tion of the uncertainties. Deterministic robust MPC (e.g.,see Bemporad & Morari, 1999; Campo & Morari, 1987;Wang, 2002 and references therein) analyses the stabilityand performance of systems against worst-case perturba-tions with the resulting optimisations being min-max prob-lems that are computationally demanding to solve directlyand so are typically replaced by approximate solutions that
∗Corresponding author. Email: [email protected]†A preliminary version of this manuscript was presented in Kim and Braatz (2012a).
are more amenable to implementation. The worst-case per-turbations may have a vanishingly small probability of oc-curring in practice, but any such information on probabili-ties is not taken into account in a deterministic formulation.Analysis or design based on worst-case uncertainties canbe too conservative to be applied in practice, may result inan over-design of process equipment, or can result in in-feasibility during real-time optimisation. From a practicalpoint of view, it is rare that an engineer knows exactly whatvalue for hard bounds to specify on the uncertainty (e.g.knows that the hard bound on uncertainty in a parametershould be exactly 10.6% instead of 11.3%), and a small per-turbation in these bounds can mean the difference becausea closed-loop system being robust to the uncertainties orbeing unstable.
Most parameter estimation algorithms generate mod-els with probabilistic descriptions of the uncertainties. Forsuch models, robustness characterisations are intrinsicallystochastic and can be written in terms of a probability distri-bution or a level of confidence in estimates with probabilis-tic risk of incorrectness. Contrary to deterministic robustMPC, stochastic robust MPC incorporates such probabilis-tic uncertainties and probabilistic violations of constraints,and allows for specified levels of risk during opera-tion. Commonly used probabilistic analysis approaches are
C© 2013 Taylor & Francis
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1325
Monte Carlo (MC) methods, in which many simulations arerun with sampled random variables or random processes.The effects of uncertainty on the closed-loop system arequantified by simulating a large number of individual de-terministic model realisations. While such MC approachesare applicable to most systems, the computational cost canbe prohibitively expensive, especially in real-time optimalcontrol algorithms such as MPC. Apart from simulation-based methods, convex relaxations and approximations fora receding horizon method of the constrained discrete-timestochastic control are considered in Cinquemani, Agar-wal, Chatterjee, and Lygeros (2011), in which convexityof the resultant optimisation is carried out in the basis ofrobust optimisation (Ben-Tal & Nemirovski, 2002; Bert-simas, Brown, & Caramanis, 2011) that includes robustlinear programmes and more generally robust convex pro-grammes (see Ben-Tal, Ghaoui, & Nemirovski, 2009 fordetails of robust convex optimisation). However, such ro-bust optimisation formulations of chance constraints arenot applicable to the cases when the stochastic dynamicalsystem has nonlinear parametric uncertainties, whereas thispaper can manage the system model that is linear parameter-varying Gaussian, for which the system matrices have non-linear dependence of random variables and there are addi-tive Gaussian random processes corresponding to externaldisturbance and measurement noise.
The high computational cost of simulation-based meth-ods has motivated the development of computationally ef-ficient methods for uncertainty analysis that replace or ac-celerate MC methods (Ghanem & Spanos, 1991; Maitre &Kino, 2010; Xiu, 2010). The MPC formulation in this pa-per uses generalised polynomial chaos (gPC) expansions,which is a spectral method to approximate the solution of anSDE that has stochastic parametric uncertainties and exoge-nous disturbances. Polynomial chaos expansions were firstintroduced for turbulence modelling with the uncertaintiesbeing Gaussian random variables (Wiener, 1938), with laterextensions considering other types of common probabilitydistributions (Xiu & Karniadakis, 2002). Recently, many re-searchers have demonstrated the use of (generalised) poly-nomial chaos expansions as a computationally efficient al-ternative to MC approaches for the analysis and controlof uncertain systems (Fisher & Bhattacharya, 2009; Hover,2006, 2008; Kim, 2013; Kim & Braatz, 2012b; Nagy &Braatz, 2007, 2010). In Fisher (2008) and Fisher and Bhat-tacharya (2011), the gPC expansion is applied to formulateoptimal trajectory generation problems in the presence ofrandom uncertain parameters.
This article also presents several probabilistic collisionconditions that are functions of the mean and covarianceof the trajectory. We show that a gPC expansion that canprovide an approximation of the solution of an SDE, inwhich both system parameters and exogenous disturbancesare stochastic, converges in the mean-square sense as thenumber of terms in the expansion increases. The proposed
approximation results in a convex optimisation for the con-trol policy that does not use any sampling and is amenableto online computation.
This paper is organised as follows. Section 2 presentssome mathematical background on gPC expansions. Sec-tion 3 states a stochastic MPC formulation with a chanceconstraint and Section 4 presents several probabilistic con-ditions in terms of chance constraints in which the condi-tions are functions of the mean and covariance of a randomvariable. The main results are presented in Section 5 wherea gPC expansion is used in place of the exact solution ofan SDE in the MPC formulation, which enables the controlpolicy to be computed as the solution to a convex optimisa-tion under the assumption that the computed approximatesolution in the presence of stochastic parametric uncer-tainties and exogenous disturbances is Gaussian. Section6 provides a numerical example to illustrate the proposedapproach for stochastic MPC of dynamic systems subjectto stochastic parametric uncertainty and exogenous dis-turbance. Section 7 presents heuristic convex semi-chanceconstrained MPC problems using gPC expansions and dis-cusses several issues related to gPC expansion theory andassociated computations. Section 8 concludes the paper.
NotationThe following notation is used throughout this article:
〈 ·, ·〉 is the inner product in a Hilbert spaceH; Pr is the prob-ability; E or ¯(·) is the expectation or mean; Var or �(·) is thevariance or covariance; N (a, b) is the Gaussian distributionwith the mean a and the variance b; U(S) is the uniform dis-tribution with the support set S ⊂ R
n; the symbol ∼ means“distributed as;” and erf(·) is the Gaussian error function.For a given sequence of vectors xi ∈ R
ni , col(xi) refers to[xT
1 , . . . , xTk ]T ∈ R
n where n = ∑ki=1 ni . The matrix with
diagonal blocks formed with matrices A1, . . . , Am and theother entries are all zeros is denoted by diag(A1, . . . , Am).S
n ⊂ Rn×n refers to the set of real symmetric matrices and
its subsets Sn+ and S
n++ are used to denote the set of positive
semidefinite and definite matrices, respectively. We also useA 0 for A ∈ S
n+ and A 0 for A ∈ S
n++.
2. Theoretical background
2.1 Characteristics of gPC expansions
Polynomial approximations are commonly used when im-plementing functions on a computing system with the ba-sic assumption being that a finite sum of polynomials canaccurately approximate a function of interest. For poly-nomial approximations, orthogonal polynomials are oftenused, with their properties reviewed below.
2.1.1 Orthogonality
Consider a measure space (X ,M, μ) where X is anonempty set equipped with a σ -algebra M and a measure
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
1326 K.-K.K. Kim and R.D. Braatz
μ. A set of orthogonal polynomials {φn(x)} for x ∈ M isdefined by the orthonormality relation
〈φn, φm〉 �∫X
φn(x)φm(x)dμ(x) ={
1 if n = m,
0 otherwise.(1)
2.1.2 Recurrence relation
Any set of orthogonal polynomials {φn(x)} on the real linesatisfies a three-term recurrence formula (Ball, 1999)
xφn(x) = an+1φn+1(x) + bnφn(x) + anφn−1(x) (2)
for n = 0, 1, . . . Along with φ−1(x) = 0, this formula holdsconsistently and φ0 is always a constant.
2.1.3 Parameterisation of random inputs
For any analysis of a stochastic system, the random inputsmust be specified and characterised appropriately.
2.1.3.1 Random variables. Consider the concatenatedparameter vector θ : � → � ⊆ R
nθ that is a random vectordefined on the events �, where the set � is assumed to beknown. The true parameter θ∗ that is a realisation of a ran-dom variable θ is assumed to be in the set, and the statisticsof the random variable θ is known. For a given probabil-ity distribution of a random parameter of a system, thefirst step of analysis using a gPC expansion is to transformthe parameters to a set of independent random normalisedvariables known as standard random variables (Isukapalli,1999). Performing this step involves finding a diffeomor-phism T: � → � such that θ = T(ζ ) for ζ ∈ � and the stateof a stochastic model x has equivalent representations x(z,t; θ (ω)) = x(z, t; ζ (ω)).
2.1.3.2 Random process and the KL expansion. Theinputs of a system can also be random processes. TheKarhunen-Loeve (KL) expansion has been used to representthe stochastic input quantities in stochastic systems and iscompatible with spectral methods for system identificationand analysis using gPC expansions; i.e. the KL expansionprovides a natural way to parameterise the random processinputs so that such a parameterisation can be exploited inthe spectral analysis to construct basis functions. The de-tails of KL expansions are not presented here due to limitedspace; readers are referred to Ghanem and Spanos (1991),Maitre and Kino (2010), Xiu (2010).
2.2 Universal approximation and convergence ofpolynomial expansions
The Hermite polynomial chaos expansion has a univer-sal approximation property for expanding second-orderrandom processes in terms of orthogonal polynomials
(Cameron & Martin, 1947), and second-order random pro-cesses are processes with finite variance, which applies tomost physical processes (Xiu & Karniadakis, 2002).
Theorem 2.1 Cameron-Martin Theorem (Cameron &Martin, 1947): For any functional f in a Hilbert measurespace (X ,M, μ), there exist a set of polynomials {φi} andconstants {ai} such that
limN→∞
∫X
(f (x) − fN (x))2dμ(x) = 0, (3)
where fN (x) �∑N
i=0 aiφi(x) and aj is obtained from theGalerkin projection ai := 〈f,φi 〉
‖φi‖2 .
The proof of this result is not trivial (interested readersare referred to Cameron & Martin, 1947). The rate of con-vergence depends on the smoothness of the function f andthe type of orthogonal polynomial basis functions {φi} usedfor approximation, and this subject has been heavily stud-ied (e.g., see Newman & Raymon, 1969; Xiu, 2010). Thelimiting behavior of the approximation error ‖f − fN‖ isO(N−p) where p denotes the differentiability of the functionf : X → Y and O(ε) (recall that O(ε) → 0 as ε → 0). Foran analytic function f, the convergence rate is exponential,i.e. ‖f − fN‖ is O(e−αN) for some constant α > 0.
3. Problem statement
Consider a stochastic discrete-time linear parameter-varying system:
xt+1 = A(δ)xt + Bu(δ)ut + Bw(δ)wt, (4)
where δ ∈ � denotes the concatenation of the paramet-ric uncertainties and A : � → R
n×n, Bu : � → Rn×m, and
Bw : � → Rn×nw are uncertain system matrices. Assume
that w ∈ R is a Gaussian white-noise process with knowndistribution, and the initial state x0 and uncertainty δ arerandom variables with known probability density functions(pdfs). Under this stochasticity of parameters and distur-bance, the solution trajectory of system (4) is a randomprocess for which the main goal of analysis is to computeor approximate the statistical properties and the main goalof synthesis is to drive the random process xt to have adesirable statistics.
In finite-horizon stochastic MPC, the goal is to deter-mine a control policy μT� (u0, . . . , uT) that solves theoptimisation
minμT
J (x0, �x0 , μT )
s.t. xt+1 = A(δ)xt + Bu(δ)ut + Bw(δ)wt,
yt = Cxt , Pr[yt /∈ Fy] ≥ β,
ut ∈ U , for t = 0, . . . , T ,
wt ∼ fw, δ ∼ fδ, x0 ∼ fx0 , (5)
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1327
where Fy denotes the forbidden region for the output yt andβ is a lower bound of probabilistic collision avoidance.1
4. Feasibility of chance constraints: probabilisticcollision checking
This section presents four different ways of formulation ofchance constraints corresponding to probabilistic collisionavoidance. In particular, for a motion-planning problem fora mobile system, it is necessary to impose constraints on the(controlled) state or output variables. Such constraints havethe form of a vector inequality η(x) ≤ 0, where x ∈ X ⊂ R
n
refers to the state variables and the function η : Rn → R
m.Due to stochastic nature of the state variables, it is natu-ral to introduce so-called chance constraints of the formPr[η(x) ≤ 0] ≥ α where α ∈ (0, 1) denotes a level of confi-dence. For a probabilistic collision avoidance problem, theformulation of chance constraints depends on the represen-tation of obstacles and mobile agents that have stochasticuncertainty.
4.1 Obstacles as point masses in alarge work space
The probability of collision to obstacles at time t and in thework space W ⊂ R
ns , ns ≤ 3, can be defined as (Lambert,Gruyer, & Pierre, 2008; Toit & Burdick, 2011)
P ct �
∫xv
∫xa
Ic
(xv
t , xat
)dFva
(xv
t , xat
), (6)
where Fva( ·, ·) is the joint cumulative distribution function(cdf), the indicator function for collision is defined by
Ic(xv, xa) �{
1, for Xv(xv)⋂Xa(xa) �= ∅,
0, otherwise,
Xv(xv) and Xa(xa) are the regions occupied by the vehicleand the obstacle whose global reference coordinates are xv
and xa, respectively. Equipped with this definition of proba-bilistic collision, the chance constraint Pr[yt /∈ Fy] ≥ β in(5) can be rewritten as P c
t ≤ 1 − β. Consider the obstaclesas point masses, which occurs when the volume of Xv(xv)is much smaller than the work space W for all xv ∈ W andthe volume of Xa(xa) is 0 for all xa ∈ W.
Lemma 4.1 (Lemma 1 in Toit and Burdick, 2011): Forobstacles as point masses, suppose that xv ∼ N (xv, �xv )and xa ∼ N (xa, �xa ) are independent Gaussian randomvariables. Then Pc ≤ 1 − β can be rewritten as the con-straint on (xv, �xv , xa, �xa ):
(xv − xa)T�−1x (xv − xa) ≥ −2 ln
(1 − β
Vv
√det(2π�x)
),
(7)where �x = �xv + �xa and Vv is the volume of the vehicle.
The constraint (7) is not convex in (xv, xa) even fora fixed β, but is concave in (xv, xa) due to positive defi-niteness of the inverse covariance matrix �−1
x . However, amethod of semidefinite programming (SDP) relaxation canbe used to check its feasibility and solve related optimisa-tions approximately.
Convex relaxation: Suppose that xv is affine in thecontrol input u, i.e. xv = Mu + b with an appropriate ma-trix M and a vector b of compatible dimensions. Then theinequality (7) can be rewritten as
[1u
]T
Q[
1u
]≥ γ ⇐⇒ Tr
(Q
[1u
] [1u
]T)
≥ γ
⇐⇒ Tr(QU ) ≥ γ, U 0,
U11 = 1, Rank(U ) = 1,
(8)
where Q 0 and γ can be appropriately computed from(7). Suppose that a convex quadratic constraint of the formuTQ1u + qT
1 u + q10 ≤ 0 with Q1 0 is imposed on thecontrol input. Minimising the probability of collision underthat quadratic constraint can be represented as the optimi-sation
maxU
Tr(QU )
s.t. Tr(Q1U ) ≤ 0, U 0, U11 = 1, Rank(U ) = 1,
(9)
where the symmetric matrix Q1 satisfies the relation[1u
]TQ1
[1u
]= uTQ1u + qT
1 u + q10. It is well known that
removing the rank constraint Rank(U) = 1 in this particularoptimisation does not change the optimum value, i.e, thecorresponding SDP relaxation is exact (Nesterov, Wolkow-icz, & Ye, 2000). Next, consider a similar problem withthe box-type constraints |ui| ≤ 1 for i = 1, . . .nu in theplace of the quadratic constraint on u. Then minimisationof the probability of collision can be represented as theoptimisation
maxU
Tr(QU )
s.t. Uii ≤ 1, i = 2, . . . , nu + 1,
U 0, U11 = 1, Rank(U ) = 1.
(10)
The associated primal SDP relaxation is the same as theoptimisation (10) without the rank constraint, and its sub-optimality is bounded by
γ � ≤ γ �sdp ≤ π
2γ � (11)
where γ � refers to the optimal value of (10) and γ �sdp refers
to the optimal value of the primal SDP relaxation (Nesterov,1998; Nesterov et al., 2000).
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
1328 K.-K.K. Kim and R.D. Braatz
4.2 Probabilistic safety regions
Instead of quantifying the probability of safety by 1 − Pc,consider the dual definition of probability of safety:
P st �
∫xv
∫xs
Is
(xv
t , xst
)dFvs
(xv
t , xst
), (12)
where xs is a global reference coordinate that characterisesa virtual safety region Xs(xs) and the joint cdf Fvs andthe indicator function Is follow similar definitions as in theprevious section. With this definition of probabilistic safetyregions, the chance constraint Pr[yt /∈ Fy] ≥ β in (5) canbe rewritten as P s
t ≥ β. Consider the obstacles as pointmasses, which is a case when the volume of Xv(xv) is muchsmaller than the work space W for all xv ∈ W and the safetyregion Xs(xs) defines a point or a sequence of points in W.
Lemma 4.2: For point-mass obstacles, suppose that xv ∼N (xv, �xv ) and xs ∼ N (xs , �xs ) are independent Gaus-sian random variables. Then Ps ≥ β can be rewritten as theconstraint on (xv, �xv , xs , �xs ):
(xv − xs)T�−1x (xv − xs) ≤ −2 ln
(β
Vv
√det(2π�x)
),
(13)where �x = �xv + �xs and Vv is the volume of the vehicle.
The constraint (13) is convex in (xv, xs) for a fixed β ∈[0, 1].
4.3 Obstacles as linear constraintsin a work space
Consider the concatenated system output yt. The forbiddenregion for the system output can be defined as a union of Nlinear inequality constraints that is a nonconvex polyhedralset
Fy �N⋃
i=1
{y : hT
i y ≥ bi
}. (14)
Assume that y ∼ N (y, �y) and define ηi � hTi y, which is
a univariate Gausian random variable with mean ηi = hTi y
and variance �ηi= hT
i �yhi . The idea of risk allocationproposed by Ono and Williams (2008) and Blackmore andOno (2009) can be used to derive a conservative convexcondition for the constraint (14).
Lemma 4.3 (Lemmas 1, 2, and 3 in Blackmore and Ono,2009): Consider a chance constraint Pr[y /∈ Fy] ≥ β orthe equivalent condition Pr[y ∈ Fy] ≤ 1 − β where Fy isdefined in (14). Then the feasibility of the constraint
Pr[ηi ≥ bi] ≤ εi, εi ∈ (0, 1), and∑
i
εi = 1 − β (15)
implies the feasibility of the constraint Pr[y /∈ Fy] ≥ β.Furthermore, Pr[ηi ≥ bi] ≤ εi ⇐⇒ 1
2
(1 − erf
(bi−ηi√
2�ηi
)) ≤εi , and the constraint (15) is convex in (ηi , εi) for β ≥ 0.5.
Alternatively, consider the forbidden region for the sys-tem output defined as an intersection of N linear inequalityconstraints that is a convex polyhedral set
F ′y �
N⋂i=1
{y : hT
i y ≤ bi
}. (16)
In this case, the nonconvex chance constraint Pr[y /∈ F ′y] ≥
β can be replaced by the relaxation
Pr[ηi > bi] ≥ εi, εi ∈ (0, 1), for i = 1, . . . , N, (17)
where εi are appropriately defined functions of β.
Lemma 4.4: With the condition εi ≥ 0.5 incorporated intothe constraint (17), the combined constraint is convex in(ηi , εi).
Proof: Pr[ηi > bi] is a concave function in ηi ≥ bi , andfor εi ≥ 0.5, the feasibility of the constraint (17) necessarilyrequires ηi ≥ bi . Thus, if (η1
i , ε1i ) and (η2
i , ε2i ) are feasible
solutions of the constraint (17) and εji ≥ 0.5, j = 1, 2, then
(ηλi , ε
λi ) is also a feasible solution for all λ ∈ [0, 1] where
the superscript λ refers to the λ-convex combination of thefeasible solutions with the superscripts 1 and 2. �
Imposing additional linear constraints on εi does notchange the convexity of the combined constraint. For ex-ample, additional constraints εi ≥ �(β) could be introducedin which �:= β �→[0.5, 1) is a nondecreasing function. How-ever, the most practically useful functional form for the �
is not obvious. One functional form that may be usefulis �(β) = N
√β, in which case β ≥ 0.5N would satisfy the
constraint εi ≥ 0.5.
5. Efficient approximation of feasibilityof probabilistic constraints
The previous section presented methods for formulatingchance constraints for probabilistic collision avoidance un-der stochastic uncertain circumstance and model uncer-tainty. Under the assumption that the state (or output)variables are jointly Gaussian random variables, the resul-tant chance constraints involve imposing constraints on themean and covariance of the state. However, the state of thesystem model (4) is not Gaussian and even computationsof its mean and covariance can necessitate sampling-basedevaluation such as Monte Carlo simulation. This sectionpresents and analyses methods for approximate uncertaintypropagation in a stochastic dynamical system (4) that arebased on generalised polynomial chaos expansions. The
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1329
methods provide numerically tractable computations of themean and covariance of the controlled state variables forwhich closed-forms of the approximate mean and covari-ance can be obtained and the associated chance constraintscan be efficiently evaluated.
5.1 Approximation of uncertainty propagation
Consider the concatenated parametric uncertainty θ :=[xT
0 , δT]T. Suppose that there exists a diffeomorphismT : � → � = � × X such that θ = T(ζ ) and ζ ∈ � isa standard random variable. For application of the spectralmethod based on gPC expansions, assume that the solutionof the SDE (4) has the form
xt ≈ xt �p−1∑i=0
φi(ζ )Xit (18)
which is an approximation of the true solution x with p basisfunctions from the set {φi}. Obtaining the approximate so-lution x involves determining the time-varying determinis-tic coefficients Xi
t . To do this, substitute the approximationx into x of the SDE (4) and solve for the Xi
t by intrusive ornon-intrusive projections onto the probability space of therandom variable ζ whose cdf is given by Fζ . In particular,applying Galerkin projection (Maitre & Kino, 2010) resultsin another SDE
Xt+1 = GXXt + Guut + Gwwt, (19)
where Xt := col(Xit ) ∈ R
np, the initial condition Xi0 =
〈φi(ζ ), x0(ζ )〉, and the matrices G(·) are computed from theinner product (1) defined on a measure space (�,M, Fζ )for the Galerkin projection. The concatenated variables ofinterest over the time-horizon satisfy the equation
X0:T = HXX0 + Huu0:T + Hww0:T , (20)
where X0:T := col(X0, . . . , XT) and the matrices H(·) haveclosed forms in terms of the matrices G(·) in (19). From theassumption of Gaussian white noise wt, the concatenatedcoefficients Xt is a Gaussian random process resulting in aGaussian random variable X0:=T with mean and covariance
X0:T = HXX0 + Huu0:T + Hww0:T ,
�X0:T = HX�X0HTX + Hw�w0:T H T
w.(21)
The following proposition shows that the mean and co-variance of the approximate solution xt have closed-formswith respect to the mean and covariance of the coefficientsof a generalised polynomial chaos expansion given in (21).
Proposition 5.1: The concatenated approximation of thesolution x0:T satisfies
E[x0:T ] = KXX0 + Kuu0:T + Kww0:T , (22)
where the matrices K(·) are functions of G(·) and H(·), andthere exists an affine surjective map � : S
np(T +1) → Sn(T +1)
such that
�x0:T = �(�X0:T ). (23)
Proof: The proof is straightforward. Consider an approx-imate solution using a polynomial expansion (18). Due toindependence of the random parameter θ and the randomprocess wt, its expectation is E[xt ] = E[φ(ζ )T ⊗ In]E[Xt ]where the first expectation is computed w.r.t. the ran-dom vector ζ and the second expectation is computedw.r.t. the random process wt. Since the coefficient Xt
is linear and has affine dependence on the control in-put ut and the external disturbance wt, the concate-nated approximate state E[x0:T ] is of the form givenin (22). Similarly, the variance of the approximate stateE[xt x
Tt ] can be rewritten as E[(φ(ζ )T ⊗ In)XtX
Tt (φ(ζ )T ⊗
In)T] or equivalently, E[Xtφ(ζ )φ(ζ )TXTt ] = E[Xt�XT
t ]
where � � E[φ(ζ )φ(ζ )T] and Xt � [X0t , . . . , X
p−1t ] ∈
Rn×p. From orthonormality of the basis functions, let �
= Ip without loss of generality. The (k, �) element of the
matrix E[xt xTt ] = E[XtX
Tt ] is
∑p−1j=0 X
jt,kX
jt,� and X
jt,kX
jt,�
is an element of the matrix E[XtXTt ]. Therefore, E[xt x
Tt ] is
an affine function of E[XtXTt ], which is equivalent to the
concatenated covariance matrix �x0:T being an affine func-tion of �X0:T . The corresponding mapping is a projectionthat is surjective. �
The random process Xt is Gaussian such that the meanand covariance given by (21) exactly characterise the proba-bility distribution of Xt for all t, whereas the approximationxt to the solution xt is not necessarily Gaussian, due to theadditional randomness of the parameters (x0, δ). However,the mean and covariance of xt can be approximated by themean and covariance of xt given by (22) and (23). Moreprecisely, the next proposition shows the convergence ofthe approximation error in the mean-square sense.
Proposition 5.2: Consider the solution trajectory xt of thesystem (4) and its approximation xt using a gPC expansiongiven by (18) whose coefficients Xt solve (19). Assume thatthe random variables (x0, δ) are independent of the randomprocess wt and Xt is a second-moment process.2 Then ‖xt −xt‖ →m.s. 0 pointwisely in t as p → ∞, where ‖ · ‖ can beany vector p-norm.
Proof: An approximation xt can be explicitly rewritten as∑p−1i=0 φi(ζ (x0, δ))Xi
t (w0:t−1). From Theorem 2.1, for anyrealisation of the random variable w0:t−1 ∈ W t , where Wis the support of wt and ε is greater than zero, there exists
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
1330 K.-K.K. Kim and R.D. Braatz
p ∈ N such that∫�
‖xt − ∑p−1i=0 φi(ζ )Xi
t (w0:t−1)‖2
dμζ (ζ ) ≤ ε for all p ≥ p, where μζ is the probabilitymeasure of ζ . The ε is a function of w0:t−1. Due to thelinear dependence of xt and Xi
t on w0:t−1, which followsfrom (4) and (19), ε = εwT
0:t−1w0:t−1 where ε > 0 is anarbitrary constant that is independent of w0:t − 1. Thisimplies that the mean-square approximation error isbounded above:
∫W t
∫�
∥∥∥∥∥xt −p−1∑i=0
φi(ζ )Xit (w0:t−1)
∥∥∥∥∥2
dμζ (ζ )dμw(w0:t−1)
≤ ε
∫W t
wT0:t−1w0:t−1dμw(w0:t−1) ≤ εM,
where μw is the corresponding probability measure of therandom variable w0:t−1 and M < ∞ whose boundednessfollows from the second-moment assumption of the randomprocess wt. Since ε > 0 is arbitrary, the convergence isensured. �
Furthermore, if the system matrices are analytic func-tions of the random variables (x0, δ) then the convergencerate of the approximation error ‖xt − xt‖ to 0 in mean-square is exponential, which follows from the solution tra-jectory xt being an analytic function of (x0, δ) under thoseassumptions.
5.2 Gaussian approximation and convexificationsof chance-constrained MPC: informationtheoretic justification
For a Gaussian random process yt (or xt), Section 4 showsthat the chance constraint corresponding to probabilisticcollision avoidance Pr[yt /∈ Fy] ≥ β can be rewritten asconditions in terms of its mean yt and covariance �yt
. Inparticular, conditions (13), (15) and (17) are jointly convexin yt (or xt) and the other decision variables (εi), under somemild assumptions.
However, the solution xt of the system dynamics (4) andits spectral approximation xt given in (18) are not gener-ally Gaussian random processes that make the optimisation(5) difficult to solve in the sense that the chance constraintdoes not have a closed-form expression and its feasibil-ity is hard to check. To avoid the use of any sampling orsimulation-based methods to evaluate the feasibility of thechance constraint Pr[yt /∈ Fy] ≥ β, the approximate solu-tion xt is substituted in the place of xt and Gaussian fitting ofthe random variables under consideration is applied. Morespecifically, assume that xt ∼ N ( ¯xt , �xt
), for which thereare closed-form expressions given by (22) and (23).3 A the-oretical justification of this assumption xt ∼ N ( ¯xt , �xt
) canbe made from the principle of maximum entropy (Cover &Thomas, 2006, Chap. 12). Maximum entropy can be usedto determine or approximate a probability distribution thatincorporates only known information. If only the first and
second moments of xt are used to approximate its probabil-ity distribution then the maximum entropy distribution hasthe form N ( ¯xt , �xt
), i.e. a Gaussian distribution. Further-more, since xt converges to xt in the mean-square sense asthe number of basis functions increases, the approximateprobability distribution N ( ¯xt , �xt
) can be made arbitrarilyclose to the probability distribution of xt that maximisesentropy subject to the constraints corresponding to the firstand second moments.
Proposition 5.3: Consider the solution trajectory xt of thesystem (4) and its approximation xt using a gPC expansiongiven by (18) whose coefficients Xt solve (19). Assume thatthe random variables (x0, δ) are independent of the randomprocess wt and Xt is a second-moment process (for notationconvenience, the subscript t is dropped from here on). Sup-pose that a probability density f � solves the optimisation
maxf
−∫
S
f (x) log f (x)dx
s.t. f (x) ≥ 0,
∫S
f (x)dx = 1,∫S
f (x)xidx = Mi, i = 1, 2,
(24)
where S denotes the support for the random variable x,and M1 and M2 correspond to the given first and secondmoments, respectively. Then an approximate Gaussian dis-tribution f2 � N ( ¯x,�x) obtained from the solution of (19)converges to f � as p → ∞ in the L1-norm sense, i.e.,
limp→∞
∫S
|f �(x) − f2(x)|dx = 0.
Proof: Due to limited space, consider the scalar case (theextension to the multivariable case is straightforward).From the principle of maximum entropy, a unique f � hasthe form of eλ0+λ1x+λ2x
2that corresponds to a Gaussian
distribution. Similarly, f2 is a unique maximum entropydistribution that solves the optimisation (24) with givenapproximate moments Mi , i = 1, 2, in the place of Mi
and can be rewritten as eλ0+λ1x+λ2x2
for some constantsλj , j = 0, 1, 2. Since convergence in mean-square im-plies convergence in distribution and Mi can be arbitrar-ily close to Mi as p → ∞ from Proposition 5.2, this im-plies that limp→∞ maxj |λj − λj | = 0. Therefore, for anyarbitrary constant ε > 0, there exists p ∈ N such thatmin{eε, e−ε} ≤ f2(x)/f �(x) ≤ max{eε, e−ε} uniformly inx ∈ S for all p ≥ p. This implies that f2 converges to f � inthe L1-norm sense as p → ∞. �
Remark 1: The above Gaussian approximation is a sub-optimal way to estimate the probability distribution of xt,which produces convex chance constraints that are morecomputationally tractable by ignoring the extra information
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1331
in the higher-order moments of xt . This method of approxi-mation has the same characteristics as the extended Kalmanfilter (EKF) and unscented Kalman filter (UKF) that arewidely used in practical applications although there are notheoretical guarantees that those estimation methods willalways work well or even converge.
Using the Gaussian approximation, the design problemreduces to finding a control policy μT (or u0:T) that solvesthe optimisation
minμT
J (x0, �x0 , μT )
s.t. ¯x0:T = KXX0 + Kuu0:T + Kww0:T ,
�x0:T = �(�X0:T ),
x0:T ∼ N ( ¯x0:T ,�x0:T ), y0:T = (⊕Ti=0C)x0:T ,
(y0:T ,�y0:T ) ∈ F(β) or (y0:T ,�y0:T , ε) ∈ F(β),
u0:T ∈ UT +1,
(25)
where the matrices K(·) and �X0:T , and the injection map �
are precomputed, ⊕Ti=0C � diag(C, . . . , C), and the con-
straints F(β) can be one of the sets:
{(y,�y) : Equation(13), xv = y, �xv = �y
}; (26)
{(y,�y, ε) : Equation(15), η = y, �η = �y
}; (27)
{(y,�y, ε) : Equation(17), η = y, �η = �y, ε ≥ 0.5
},
(28)
where ε = col(εi), β ≥ 0.5 is required for the second feasiblesolution set to be convex in (y, ε), and the first and thethird sets are convex in y and (y, ε), respectively. Withthe standard performance specification that the objectivefunction J is convex quadratic in μT and the setU is a convexpolytope, the optimisation (25) is a convex quadraticallyconstrained quadratic program (QCQP) whenF(β) is givenby (26) and a convex nonlinear program whenF(β) is givenby (27) or (28).
6. A demonstration example
This section compares the accuracy of the proposed gPC-based MPC formulations for a numerical example. Con-sider the parametric uncertain linear time-invariant system
xt+1=[
0.9 + ρ1δ1 0.10.1 0.85
]xt+
[0.25 − ρ2δ2
0.75 + ρ2δ2
]ut+
[1
0.5
]wt
with initial condition x0 = [20, 10]T, ρ1 = 0.001 andρ2 = 0.05 are weights on normalised standard random vari-ables δ1 ∼ N (0, 1) and δ2 ∼ N (0, 1), respectively, and theexogenous process noise wt ∼ N (0, 0.001) is assumed tohave autocorrelation E[wtws] = 0 for all t �= s. This example
2
Figure 1. A controlled trajectory produced by the proposedstochastic MPC formulation. For description of the forbidden re-gions at each step of receding horizon control design to avoidobstacle collisions, the linear inequality constraints (14) are used.To do this, only partial linear constraints are imposed at eachcomputation of receding horizon control input, whereas the fulldescription of the obstacle in this example is indeed an intersectionof linear constraints.
computes the control inputs by solving the optimisation(31), for which the covariance constraints are imposedbased on the 99% level of confidence for collision avoid-ance, and comparing the probability of collisions obtainedfrom different methods presented in the paper. Consider Qt
= diag{100, 100} and Rt = 1 for all t, a prediction horizonof T = 4, and input constraint ut ∈ [ − 0.5, 0.5]. The con-straints are constructed from the obstacle shown in Figure 1.The resultant controlled system trajectory generated by asystem model with fixed parameters δ = [0.01, 0.05] andrandomly chosen exogenous disturbances wt is shown inFigure 1, which avoids the obstacle as desired. Figure 2shows Monte Carlo simulations with 5000 samples of (δ,w), which indicates that the stochastic MPC algorithm waseffective in avoiding the obstacle while allowing the closed-loop trajectory to become rather close to the obstacle so asto optimise the closed-loop performance objective. Figure 3compares the computed probabilities of collision using themethods presented in this paper with the probabilities quan-tified by the Monte Carlo simulations. At each time theprobability of collision obtained by the gPC expansion isvery close to the value computed using either Monte Carloapplied to the original system or Monte Carlo applied to theconvex relaxation. The approximate probabilities of colli-sion follow nearly identical trends to the true probabilitieswhile enabling the optimal control problem at each timeinstance of MPC to be computed from a convex programthat can be solved in polynomial-time.4
To further assess the accuracy of the gPC expansion,let �mc
x (t) and �pcex (t) be the computed covariance of the
controlled system trajectory obtained from Monte Carlo
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
1332 K.-K.K. Kim and R.D. Braatz
Figure 2. Monte Carlo simulations. The red dots correspond tosimulated states at each 4th sampling instance for 5000 samples.
Figure 3. A comparison of the computed probability of colli-sion during the first 20 [sec] of simulation for the true system andthe approximation using a gPC expansion, estimated using MonteCarlo (MC) simulations where the error bars were obtained from1000 Monte Carlo simulations with different sets of 5000 sam-ples. The blue circle refers to the MC simulation result and thered box refers to the collision probability that is obtained from theMC simulations with the convex relaxation (15). For both compu-tations, the corresponding error bars were generated at the 95%confidence level. The widths of the computed confidence inter-vals were smaller than 10−7, which is negligible compared to thecollision probability. The black star refers to the collision proba-bility obtained from the presented gPC method that incorporatesthe convex relaxation (15).
simulations with 5000 samples of (δ, w) and the polyno-mial chaos expansion with a specified order of Hermitepolynomials, respectively. Table 1 compares the worst-casedeviation of max0≤t≤60 ‖�mc
x (t) − �pcex (t)‖F , where ‖ · ‖F
denotes the Frobenius norm, for different degrees of Her-mite polynomials. The approximation error of the covari-ance matrix is small and, as expected from the theoreti-cal analysis, the error in the state covariance matrix de-creases as the number of terms in the polynomial expansionincreases.
Table 1 Covariance approximation errors for different degreesof Hermite polynomial expansions.
Degree of Hermitepolynomials max
0≤t≤60‖�mc
x (t) − �pcex (t)‖F
1st ≈1.0001 × 10−3
2nd ≈6.0878 × 10−4
3rd ≈5.3627 × 10−4
4th ≈2.8365 × 10−4
7. Discussions and further remarks
7.1 Approximate solution using spectral methodswith the KL expansions
This paper considers two sources of uncertainties: (1) para-metric uncertainty and (2) exogenous disturbance. Uncer-tainty propagations induced by parametric uncertainty areapproximated by using a gPC expansion and additive ex-ogenous disturbances affect the coefficients of the resultantgPC expansion. Another possible approach to the sameproblem is to use a KL expansion to approximate the ran-dom process wt, i.e. replace the random process wt by itsprincipal component approximation with random variablesand solve a larger dimension deterministic ordinary dif-ferential equation (ODE) to approximate the true solutionxt. This approach requires higher online computational ex-pense as the dimension of a deterministic ODE increases,even though the system data for such an ODE can beprecomputed.
7.2 Heuristic convexification methods for chanceconstraints with stochastic parametricuncertainty
Here the convexification methods are illustrated for theprototypical stochastic MPC problem
minu0:T −1
E
[T∑
t=1
xTt Qtxt + uT
t−1Rt−1ut−1
]
s.t. xt+1 = A(δ)xt + B(δ)ut ,
Pr�[Htxt ≥ bt ] ≤ εt ,
umin,t ≤ ut ≤ umax,t ,
(29)
with the stochastic uncertainties δ (the incorporation of theexternal noise perturbation is straightforward as describedin the theoretical sections of this paper but not included inthis example to shorten the presentation). The constraintsare defined over the time interval [1, T] for xt and [0, T − 1]for ut, and this time interval consideration is omitted herefor notational convenience and will always be clear fromthe context. The optimisation (29) is further simplifiedby replacing the chance constraint Pr�[Htxt ≥ bt ] ≤ εt byHtE[xt] ≤ bt − β t where β t > 0 is an additional decision
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1333
variable. By doing this, optimisation (29), in which the dy-namic constraint is an SDE, reduces to the deterministicoptimisation
minu0:T −1, β1:T
T∑t=1
(XT
t QtXt + uTt−1Rt−1ut−1
) − γ
T∑t=1
�(βt )
s.t. Xt+1 = FXt + Gut,
c0HtX0t + βt ≤ bt , βt > 0,
umin,t ≤ ut ≤ umax,t ,
(30)
where xt in the constraint of optimisation (29) is approxi-mated by xt in (18), Qt � E
[(φ(ζ ) ⊗ In)TQt (φ(ζ ) ⊗ In)
],
c0�〈1, φ0(ζ )〉, φ(ζ )�col(φi), γ > 0 is a user-defined weightin the optimisation that corresponds to the maximisation ofthe feasibility of the chance constraint Pr�[Htxt ≥ bt ] ≤ εt ,and �(β t) is an incentive for decision variables to maximisethe feasibility of the chance constraint Pr�[Htxt ≥ bt ] ≤εt ; a typical choice can be
∑mi=1 βt,i or max iβ t, i that is
linear in β t, where β t, i is the ith entry of β t.5The con-strained optimisation (30) is a convex QP that can be solvedefficiently.6
For a different formulation of constraints, consider con-straints on the deviation of the solution trajectory from theexpectation:
minu0:T −1
T∑t=1
XTt QtXt + uT
t−1Rt−1ut−1
s.t. Xt+1 = FXt + Gut,
c0HtX0t ≤ bt ,
(Xit )
TWXit − (c0X
0t,i)
2 ≤ σ 2t,i ,
umin,t ≤ ut ≤ umax,t ,
(31)
where Qt and c0 are the same as defined before,W�diag(‖φi‖2), and Xi
t denote the concatenation of co-efficients of the polynomial expansion for the ith state.The constant vector c0 and matrix W can be assumedto be normalised to be 1n and In without loss of gen-erality. The constrained optimisation (31) is a convexQCQP that can be solved efficiently. More precisely, itis not hard to see that optimisation (31) can be rewrittenas
minu
Q0(u) s.t. Qi(u) ≤ 0, i = 1, . . . , mq, (32)
where u � u0:T−1 and Qi are convex quadratic formsfor all i = 0, 1, . . . , mq. From Megretski and Treil(1993),7if the constraints Qi ≤ 0 are regular, i.e. satisfy aconstraint qualification such as Slater’s condition (Boyd &Vandenberghe, 2004, Section 5.2.3), then the static optimi-
Figure 4. A schematic cartoon of constrained state trajectorieswith semi-chance constraints.
sation (32) has the same optimum as the optimisation
maxλ≥0
minu
Q0(u) +mq∑i
λiQi(u) (33)
for which fixing arbitrarily large λi > 0 results in the sameoptimal solution u∗ as obtained from solving the constrainedoptimisation (32).
The conceptual picture of a constrained trajectory inFigure 4 shows how the constraints in (31) can be used toimpose desired bounds on the controlled trajectory.
7.3 The use of concentration-of-measureinequalities for probabilistic validationcertificates of joint chance constraints
This section shows that the Boole inequality can be incorpo-rated into some well-known concentration-of-measure in-equalities to provide probabilistic validation certificates forjoint chance constraints. Consider the constraint H TX ≤ b,or equivalently hT
i X ≤ bi for i = 1, . . . , m where X is a ran-dom vector and hi denotes the ith column of the matrixH. The associated probabilistic constraint can be writtenas Pr[H TX > b] = Pr
[∪mi=1{hT
i X > bi}]. The Boole in-
equality gives an upper bound on this probabilistic violationof constraints:
Pr
[m⋃
i=1
{hT
i X > bi
}] ≤m∑
i=1
Pr[hT
i X > bi
]. (34)
Suppose that bi > 0 for all i = 1, . . . , m without lossof generality. Some concentration-of-measure inequalitiescan be used for upper bounds on the right-hand side of (34)(Boucheron, Bousquet, & Lugosi, 2004):
• The Chernoff’s bound: Pr[hT
i X > bi
] ≤ E[esi hTi
X]esi bi
where si > 0 for all i = 1, . . . , m.
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
1334 K.-K.K. Kim and R.D. Braatz
• The generalised Markov inequality :
Pr[hT
i X > bi
] ≤ E[φi (hTi X)]
φi (bi )where φi : R → R+ for
all i = 1, . . . , m.• The Chebyshev inequality :
Pr[|hT
i X − E[hTi X]| > ti
] ≤ Var(hTi X)
t2i
= hTi Var(X)hi
t2i
.
Here we use the Chebyshev inequality.
Proposition 7.1: If the random vector X satisfies the con-straints on its expectation and variance
hTi X ≤ bi, hT
i Var(X)hi ≤ t2i εi , i = 1, . . . , m (35)
then the inequality HTX ≤ b + t is satisfied with atleast probability 1 − ∑m
i=1 εi , i.e., Pr[H TX ≤ b + t
] ≥1 − ∑m
i=1 εi .
Figure 5 illustrates the outer polytopic certificate (col-ored in red) for the associated chance constraint Pr[H TX >
b] ≤ ε with∑m
i=1 εi ≤ ε. The polytope colored in blue cor-responds to the constraints on the expectation of the trajec-tories. Such certificates can be computed from the resultsin Prop. 7.1.
From (21) and the results in Prop. 5.1, gPC expansionscan provide closed forms for the expectation and variance ofthe controlled predicted state and output trajectories. Thisimplies that any chance constraints of polyhedral inequali-ties can be certificated by deterministic polyhedral inequal-ities that are obtained from incorporating gPC expansions
Figure 5. A cartoon of the outer bound that can be obtained fromthe Boole inequality and concentration-of-measure inequalities.
into the Boole inequality and the Chebyshev inequality ofthe form presented in Prop. 7.1.
7.4 Computation of first-time excursionprobabilities
An important problem in reliability analysis of stochasticdynamical systems is to compute the cumulative probabilitythat the system output variables exceed a given thresholdfor a given time interval [0, T] (Au & Beck, 2001; Greytak& Hover, 2011):
Pf (T ) � Pr
[m⋃
i=1
{∃t ∈ [0, T ] : yi(t) > bi(t)}]
, (36)
where bi(t) denote some specified time-varying thresholdlevels. The probability (36) is called the first-time excursionprobability. It is straightforward to show that computationof Pf(T) is a problem of checking the feasibility of chanceconstraints on the (controlled) system output and the afore-mentioned polynomial chaos expansion methods of controlinput design for stochastic MPC problems can be directlyapplied.
In particular, similar to the probability of violation in(34), the Boole inequality can be used to provide an upperbound on the probability of first-time excursion (36):
Pf (T ) ≤m∑
i=1
Pr [{∃t ∈ [0, T ] : yi(t) > bi(t)}] (37)
for which the probability Pr [{∃t ∈ [0, T ] : yi(t) > bi(t)}]can be approximated by using the methods presented inSections 5, 7.2, or 7.3.
7.5 Affine feedback control policy
The aforementioned methods for computation of a subop-timal control policy use the new measurements to computea control action as well as to initialise the state-transitionconstraint in the optimisations at each step of prediction.In the presence of model/plant mismatch and external dis-turbances, the predicted control trajectory at time k cansignificantly deviate from the true controlled trajectory andthe variance of the trajectory can increase such that the op-timisation is feasible only for a short-time horizon, whichis undesirable in terms of closed-loop stability. This situ-ation can be avoided by incorporating a feedback controlin each step of solving the optimisation, as has been donein many deterministic robust MPC formulations (e.g. seeKothare, Balakrishnan, & Morari, 1996). For example, theaffine control law ut = Ktzt + ν t can be inserted into theoptimisation, where zt is an estimated state or measuredoutput. For a precomputed Kt (or a stationary control gainK), the resultant problem is exactly same as the open-loop
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1335
feedback control in which ν t is the only decision variablein each step of optimisation. If Kt is considered as an ad-ditional decision variable in each step of optimisation, thenthe resulting optimisation will retain the same degree ofconvexification as for ut considered before.
7.6 Time-varying uncertain parameters
Consider the system dynamics given in (4) where the uncer-tain parameter vector δ ∈ � (or θ = (δ, x0) ∈ �, if the initialcondition is considered to be uncertain) is assumed to bean unknown constant vector. In the aforementioned MPCformulations, the uncertain parameters were assumed to befixed only in the prediction step. The approaches can alsobe applied to slowly time-varying uncertain parameters, i.e.when the prediction horizon multiplied by the sampling in-terval is less than the time interval of significant parametervariation. A more accurate study of time-varying uncer-tain parameters can be performed by considering a largedimensional space of uncertain parameters. In particular,for the time-varying uncertain parameter vector δt ∈ �,consider the stacked vector δ0:T −1 � [δ′
0, . . . , δ′T −1]′ ∈ �T
where the superscript ′ denotes the transpose and T denotesthe prediction horizon. Then the approximation based ona polynomial chaos expansion is represented in terms ofthe stacked uncertain parameter vector δ0: T−1. This ap-proach requires more basis functions for the correspondingspectral representation, but the time-dependent coefficientscorresponding to the uncertain parameters in future can beset to zeros, which reduces the computation of projectionsto determine the coefficients of the gPC expansion.
8. Conclusions and future work
This paper considers a new approach for stochastic MPCproblems in the presence of both parametric model uncer-tainty and exogenous stochastic disturbances. To approxi-mate the solution of a stochastic differential equation andsolve the corresponding stochastic MPC problem, a spectralmethod known as generalised polynomial chaos expansionis applied and constraints corresponding to the probabil-ity of safety/collision are imposed on the approximatelypredicted controlled trajectories, based on the model of astochastic differential equation. The first and second mo-ments of the approximate solution were exploited to esti-mate the probability distribution of the true solution. Underthese technical assumptions, the chance constraints werereplaced by convex constraints for the mean and covari-ance of the trajectory that are analytically computed fromthe gPC expansion. It was also shown that concentration-of-measure inequalities combined with the Boole inequal-ity can provide conservative probabilistic certificates forchance constraints of polyhedral inequalities, for which ap-plications of the gPC expansions are straightforward. Fur-ther studies to follow are to apply the presented methods tomore complicated case studies and compare the heuristic
convexification methods discussed in Section 7 to convexnonlinear programmes presented in Section 4, to study thetrade-off between complexity and accuracy.
Notes
1. Fy and β can be time-varying, where the forbidden regionmight correspond to moving objectives and time-varying βcan be used to assign different risk of collision in differenttime sequences in the predicted motions.
2. Consider the time interval [0, T] in which Xt is a second-moment process.
3. The computation of deterministic constant matrices K(·) and�X0:T (or �x0:T ) can be performed off-line.
4. In particular, the computational complexity using a standardinterior-point method (Boyd & Vandenberghe, 2004) is atmost O(�M4log M) where M = npT (n is the dimension ofthe state variables, p is the number of basis functions for agPC expansion, T is the length of prediction horizon), and� denotes the number of probabilistic polyhedral constraints.The average computation time at each sampling instance was≈0.36 CPU seconds for n = 2, p = 3, and T = 5. This com-putation time includes the computation of the optimisationdata, i.e., the time for computing matrices associated withthe objective function and constraints, as well as the time forsolving the resulting constrained optimisation. Optimisationis performed by the CVX toolbox (Grant, Boyd, & Ye, 2009)on a MacBook Pro laptop (2.53 GHz Intel Core 2 Duo, 4GBDDR3).
5. To be a convex program, ‖β t‖2 cannot be used, since it resultsin a concave function term in the objective function in aminimisation problem.
6. By efficiency, it is meant that there is a numerical algorithmwhose convergence is guaranteed and that provides an infea-sibility certificate. Convex programs are such cases.
7. They applied a method of relaxation called the S-procedure(Yakubovich, 1971). Our case is a special case in whichall the quadratic forms are convex, whereas Megretski andTreil (1993) considered more general cases where some ofquadratic forms might be nonconvex. They proposed a suf-ficient condition for the set of quadratic form constraints tobe lossless, i.e., the resultant relaxation obtained from theS-procedure gives an exact optimum.
ReferencesAu, S., & Beck, J. (2001). First excursion probabilities for linear
systems by very efficient importance sampling. ProbabilisticEngineering Mechanics, 16, 193–207.
Ball, J.S. (1999). Orthogonal polynomials, Gaussian quadra-tures, and PDEs. Computing in Science & Engineering, 1,92–95.
Bemporad, A., & Morari, M. (1999). Robust model predictivecontrol: A survey. In F.W. Vaandrager and J.H. van Schuppen(Eds.), Hybrid systems:Computation and control (pp. 31–45).Lecture Notes in Computer Science. New York: Springer.
Ben-Tal, A., Ghaoui, L.E., & Nemirovski, A. (2009). Robust opti-mization. Princeton, New Jersey: Princeton University Press.
Ben-Tal, A., & Nemirovski, A. (2002). Robust optimization–methodology and applications. Mathematical Programming,92, 453–480.
Bertsimas, D., Brown, D.B., & Caramanis, C. (2011). Theory andapplications of robust optimization. SIAM Review, 53, 464–501.
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
1336 K.-K.K. Kim and R.D. Braatz
Blackmore, L., & Ono, M. (2009). Convex chance constrainedpredictive control without sampling. Proceedings of the AIAAGuidance, Navigation, and Control Conference (pp. AIAA–2009–5876). Chicago.
Blackmore, L., Ono, M., Bektassov, A., & Williams, B.C. (2010).A probabilistic particle-control approximation of chance-constrained stochastic predictive control. IEEE Transactionson Robotics, 26, 502–517.
Blackmore, L., Ono, M., & Williams, B.C. (2011). Chance-constrained optimal path planning with obstacles. IEEETransactions on Robotics, 27, 1080–1094.
Boucheron, S., Bousquet, O., & Lugosi, G. (2004). Concentra-tion inequalities. In O. Bousquet, U.V. Luxburg, and G. Rtsch(Eds.), Advanced lectures in machine learning (pp. 208–240).Berlin, Heidelberg: Springer
Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cam-bridge, UK: Cambridge Univ. Press.
Cameron, R., & Martin, W. (1947). The orthogonal developmentof nonlinear functionals in series of Fourier-Hermite func-tionals. The Annals of Mathematics, 48, 385–392.
Campo, P.J., & Morari, M. (1987). Robust model predictive con-trol. Proceedings of American Control Conference (pp. 1021–1026). Minneapolis, MN, USA.
Cinquemani, E., Agarwal, M., Chatterjee, D., & Lygeros, J.(2011). Convexity and convex approximations of discrete-time stochastic control problems with constraints. Automat-ica, 47, 2082–2087.
Couchman, P.D., Cannon, M., & Kouvaritakis, B. (2006). MPCas a tool for sustainable development integrated policy as-sessment. IEEE Transactions on Automatic Control, 51,145–149.
Cover, T.M., & Thomas, J.A. (2006). Elements of informationtheory. Hoboken, NJ: John Wiely & Sons, Inc.
Fisher, J.R. (2008). Stability analysis and control of stochasticdynamic systems using polynomial chaos (PhD dissertation).Texas A&M University, College Station, TX.
Fisher, J., & Bhattacharya, R. (2009). Linear quadratic regulationof systems with stochastic parameter uncertainties. Automat-ica, 45, 2831–2841.
Fisher, J., & Bhattacharya, R. (2011). Optimal trajectory gen-eration with probabilistic system uncertainty using polyno-mial chaos. Journal of Dynamic Systems, Measurement, andContol, 133.
Ghanem, R., & Spanos, P. (1991). Stochastic finite elements: Aspectral approach. New York: Springer-Verlag.
Grant, M., Boyd, S., & Ye, Y. (2009). cvx usersO guide (Tech-nical report, Technical Report Build 711). Available at:http://citeseerx.ist.psu.edu/viewdoc/download.
Greytak, M.B., & Hover, F.S. (2011). A single differential equationfor first-excursion time in a class of linear systems. Transac-tions of the ASME-G-Journal of Dynamic Systems, Measure-ment, and Control, 133, 061020.
Herzog, F., Dondi, G., & Geering, H. (2007). Stochastic modelpredictive control and portfolio optimization. InternationalJournal of Theoretical and Applied Finance, 10, 203–233.
Hover, F.S. (2006). Application of polynomial chaos in stabilityand control. Automatica, 42, 789–795.
Hover, F.S. (2008). Gradient dynamic optimization with Legendrechaos. Automatica, 44, 135–140.
Isukapalli, S.S. (1999). Uncertainty analysis of transport-transformation models. New Brunswick, NJ: Rutgers Univ.
Kim, K.K. (2013). Model-based robust and stochastic control, andstatistical inference for uncertain dynamical systems (PhDthesis). University of Illinois at Urbana-Champaign, Urbana,Illinois.
Kim, K.K.K., & Braatz, R.D. (2012a). Generalized polynomialchaos expansion approaches to approximate stochastic reced-ing horizon control with applications to probabilistic collisionchecking and avoidance. IEEE Multi-Conference on Systemsand Control (pp. 350–355). Dubrovnik, Croatia.
Kim, K.K.K., & Braatz, R.D. (2012b). Probabilistic analysis andcontrol of uncertain dynamic systems: Generalized polyno-mial chaos expansion approaches. Proceedings of AmericanControl Conference (pp. 44–49). Montreal, Canada.
Kothare, M.V., Balakrishnan, V., & Morari, M. (1996). Robustconstrained model predictive control using linear matrix in-equalities. Automatica, 32, 1361–1379.
Lambert, A., Gruyer, D., & Pierre, G.S. (2008). A fast MonteCarlo algorithm for collision probability estimation. Proceed-ings of the International Conference on Control, Automation,Robotics and Vision (pp. 406–411).
Li, P., Wendt, M., & Wozny, G. (2000). Robust model predic-tive control under chance constraints. Computers & ChemicalEngineering, 24, 829–834.
Maitre, O.P.L., & Kino, O.M. (2010). Spectral methods foruncertainty quantification. Dordrecht, NY: Springer.
Megretski, A., & Treil, S. (1993). Power distribution inequalitiesin optimization and robustness of uncertain systems. Journalof Mathematical Systems, Estimation and Control, 3, 301–319.
Nagy, Z.K., & Braatz, R.D. (2007). Distribution uncertaintyanalysis using power series and polynomial chaos expansions.Journal of Process Control, 17, 229–240.
Nagy, Z.K., & Braatz, R.D. (2010). Distributed uncertaintyanalysis using polynomial chaos expansions. IEEE Multi-Conference on Systems and Control (pp. 1103–1108).Yokohama, Japan.
Nesterov, Y. (1998). Semidefinite relaxation and nonconvexquadratic optimization. Optimization Methods and Software,9, 141–160.
Nesterov, Y., Wolkowicz, H., & Ye, Y. (2000). Nonconvexquadratic optimization. In H. Wolkowicz, R. Saigal, and L.Vandenberghe (Eds.), Handbook of semidefinite program-ming: Theory, algorithms, and applications (pp. 361–395).Boston, MA: Kluwer Academic Publishers.
Newman, D.J., & Raymon, L. (1969). Quantitative polynomialapproximation on certain planar sets. Transactions of theAmerican Mathematical Society, 136, 247–259.
Ono, M., & Williams, B.C. (2008). Iterative risk allocation: Anew approach to robust model predictive control with a jointchance constraint. IEEE Conference on Decision and Control(pp. 3427–3432). Cancun, Mexico.
Schwarm, A.T., & Nikolaou, M. (1999). Chance-constrained model predictive control. AIChE Journal, 45,1743–1752.
Toit, N.E.D., & Burdick, J.W. (2011). Probabilistic collisionchecking with chance constraints. IEEE Transactions onRobotics, 27, 809–815.
van Hessem, D.H., & Bosgra, O.H. (2004). Closed-loop stochasticmodel predictive control in a receding horizon implemen-tation on a continuous polymerization reactor example.Proceedings of American Control Conference (pp. 914–919).Boston.
Wang, Y. (2002). Robust model predictive control (PhDdissertation). University of Wisconsin-Madison.
Wiener, N. (1938). The homogeneous chaos. American Journalof Mathematics, 60, 897–936.
Xiu, D. (2010). Numerical methods for stochastic computations:A spectral method approach. Princeton, NJ: PrincetonUniversity Press.
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
International Journal of Control 1337
Xiu, D., & Karniadakis, G.E. (2002). The Wiener-Askeypolynomial chaos for stochastic differential equa-tions. SIAM Journal on Scientific Computing, 24, 619–644.
Yakubovich, V.A. (1971). S-procedure in nonlinear control theory.Vestnik Leningrad University, 2, 62–77, (English translationin Vestnik Leningrad Univ. 4:73–93, 1977).
Yan, Y., & Bitmead, R.R. (2005). Incorporating state estimationinto model predictive control and its application to networktraffic control. Automatica, 41, 595–604.
Zhu, S.S., Li, D., & Wang, S.Y. (2004). Risk control overbankruptcy in dynamic portfolio selection: A generalizedmean-variance formulation. IEEE Transactions on AutomaticControl, 49, 447–457.
Dow
nloa
ded
by [
18.7
.29.
240]
at 0
9:09
23
Dec
embe
r 20
13
Recommended
