Upload
gustavo-rodrigues
View
236
Download
0
Embed Size (px)
Citation preview
8/11/2019 Artigo erro varincia identificao.pdf
1/14
Automatica 45 (2009) 25122525
Contents lists available atScienceDirect
Automatica
journal homepage:www.elsevier.com/locate/automatica
Variance-error quantification for identified poles and zeros
Jonas Mrtensson ,Hkan HjalmarssonACCESS Linnaeus Center, School of Electrical Engineering, KTH Royal Institute of Technology, S-100 44 Stockholm, Sweden
a r t i c l e i n f o
Article history:
Received 11 July 2007
Received in revised form
7 December 2008
Accepted 9 July 2009Available online 10 September 2009
Keywords:
Accuracy of identification
Asymptotic variance expressions
a b s t r a c t
This paper deals with quantification of noise induced errors in identified discrete-time models of causal
linear time-invariant systems, where the model error is described by the asymptotic (in data length)variance of the estimated poles and zeros. The main conclusion is that there is a fundamental difference
in the accuracy of the estimates depending on whether the zeros and poles lie inside or outside the unitcircle. As the model order goes to infinity, the asymptotic variance approaches a finite limit for estimates
of zeros and poles having magnitude larger than one, but for zeros and poles strictly inside the unit circletheasymptotic variance grows exponentially with themodelorder.We analyze howthe variance of polesand zeros is affected by model order, model structure and input excitation. We treat general black-box
model structures including ARMAX and BoxJenkins models.
2009 Elsevier Ltd. All rights reserved.
1. Introduction
Model accuracy is an important issue in system identificationapplications. The experimental conditions, such as excitation
signals, feedback mechanisms, disturbances and measurement
noise,naturally affect theaccuracyof the model,but also the choiceof model structure and model order can have a strong influence.The model error can generally be divided into two parts: bias-errorand variance-error, where the bias-error is dueto un-modeled
dynamics and the variance-error is caused by disturbances whichare modeled as stochastic processes (Ljung,1999;Sderstrm &Stoica, 1989). The focus in this paper is on the variance-error,which will be thedominating part if the model structure is flexibleenough to describe the true underlying dynamics. We will thus
assume that the true system belongs to the model set.The problem of quantifying the expected model error has
received substantial research interest over the past decades. Inparticular, the variance of frequency function estimates has beenstudied extensively. In the mid-eighties Ljung(1985) presented
a variance expression which showed that, for high model orders,the asymptotic variance1 of the frequency function estimate does
The material in this paper was partially presented at the 44th IEEE Conference
on Decision and Control and European Control Conference ECC 2005, 12th15th
December2005 in Seville, Spain, andat the 16thIFAC WorldCongress on Automatic
Control,3rd8thJuly 2005 in Prague,Czech Republic. Thispaperwas recommended
forpublicationin revisedform by AssociateEditorBrett Ninnessunder thedirection
of Editor Torsten Sderstrm. Corresponding author. Tel.: +46 8 7907434; fax: +46 8 7907329.E-mail addresses: [email protected](J. Mrtensson),
[email protected](H. Hjalmarsson).1 Henceforth, asymptotic variancedenotesthe varianceof an estimatedquantity,
normalized by the sample size, as the sample size tends to infinity.
not depend on the model structure but only on the model order.Furthermore, it was shown that the asymptotic variance at aparticular frequency only depends on the ratio of the input andnoise spectra at that particular frequency.
A refined asymptotic variance expression (still asymptotic inmodel order) for frequency function estimates with improvedaccuracy (for many model structures) was proposed in Ninness,Hjalmarsson, and Gustafsson (1999a,b)and expressions that areexact for finite model orders were derived in Ninness andHjalmarsson (2004)and Xieand Ljung (2001, 2004). More recently,a variance expression for finite sample sizes and model ordershas been presented in Hjalmarsson and Ninness (2006). Theoriginal result in Ljung (1985) can also be used for closed loopidentification and this was extended to some alternative closedloop identification methods in Gevers, Ljung, and Van den Hof(2001). The asymptotic variance for parameter estimates in aBoxJenkins model was analyzed inForssell and Ljung(1999)fora whole range of different identification methods. Closed loopasymptotic variance expressions that are exact for finite modelorders are also presented inNinness and Hjalmarsson (2005a,b).
There are numerous other properties of the system, besidesthe frequency function, that could be of interest to estimate.Recently, a geometric interpretation of the asymptotic variance hasbeen developed (Hjalmarsson & Mrtensson, 2007)and, when theunderlying system is causal, linear and time-invariant (LTI), thesetechniques have been used to derive more transparent expressionsfor the asymptotic variance for estimates of system quantitiesthat can be represented by smooth functions of the estimatedparameters (Mrtensson & Hjalmarsson, 2007). In this paper thefocus is on poles and zeros of causal LTI systems. Particularlyinteresting from a control perspective are unstable poles and non-minimum phase zeros, i.e. poles and zeros outside the unit circle,since they pose fundamental performance limitation on the closedloop system(Skogestad & Postlethwaite, 1996).
0005-1098/$ see front matter 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.automatica.2009.08.001
http://www.elsevier.com/locate/automaticahttp://www.elsevier.com/locate/automaticamailto:[email protected]:[email protected]://dx.doi.org/10.1016/j.automatica.2009.08.001http://dx.doi.org/10.1016/j.automatica.2009.08.001mailto:[email protected]:[email protected]://www.elsevier.com/locate/automaticahttp://www.elsevier.com/locate/automatica8/11/2019 Artigo erro varincia identificao.pdf
2/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2513
A series of results regarding the asymptotic variance ofpoles and zeros has been presented over the past years. Non-
minimum phase zeros were treated for different model structuresin Hjalmarsson and Lindqvist (2002), Lindqvist (2001) andMrtensson and Hjalmarsson(2003) where it was found that theasymptotic variance approaches a finitelimit as the model ordergoes to infinity. For ARX models these results were extended
to closed loop identification in Mrtensson and Hjalmarsson(2005a), where also similar results were presented for poleswith magnitude larger than one. Asymptotic variance expressionsthat are exact for finite model orders were derived for zeroswith arbitrary location in Mrtensson and Hjalmarsson(2005b).
It should be noted that the asymptotic variance can be a poormeasure of the variance for finite sample size, see Vuerinckx,Pintelon, Schoukens, and Rolain (2001) where also a more accuratemethod to construct finite sample size confidence bounds forestimated zeros and poles is presented.
In this paper,the results on the asymptotic variance of poles and
zeros referred to above are combined and generalized to a unifiedtheory on the asymptotic variance of poles and zeros of estimatedcausal LTI systems. We put emphasis on presenting, analyzingand interpreting the results, but the derivations (proofs) of the
variance expressions are also included to make the document self-contained. The results hold for prediction error identification.
Outline
This paper is organized in the following way. In Section2 theproblem set-up, the standing assumptions, that will hold through-out the paper, and some notation are introduced. The main tech-
nical results are presented in Section 3. The asymptotic variance ofestimated zeros and poles located in the open unit disc|z| < 1 isdiscussed in Section 4 which is followed by Section 5 covering poleswith magnitude larger than one. Section6covers zeros with mag-nitude at least one and in Section7 some conclusions are drawnbased on the analysis of the variance expressions and some simu-
lations arepresented to investigate theaccuracyof theexpressions.Section 8 contains a brief summary of the findings and Appendix Acontains results that are needed for the proofs of the paper.
Notation
The multiplicity of a root zo of a polynomial A(z)is defined asthe integerk for which the limit limzzo A(z)/(zzo)k = c= 0exists. Two polynomials are said to be coprime if they have nocommon roots. A rational transfer function Q = Qn/Qd, whereQn and Qdare coprime polynomials in z
1, is said to be minimumphase ifQn(z)= 0 or Qd(z)= 0 implies|z| < 1. A zerozo ofQ issaid to be non-minimum phase (NMP) if|zo| > 1.Qis said to bestable ifQd(z)= 0 implies|z| < 1. A rootzp to Qd(z)= 0 with|zp|> 1 is said to be an unstable pole ofQ.
For a transfer function Z(q)with a zero at zo, we will use the
notationZ(q)=Z(q)/(1 zoq1). Itwill be clear from the contextwhich zerozo that is used.
For a rational transfer functionG,Gdenotes a transfer functionwhich satisfies|G(ej)| = |G(ej)| for all for which G(ej) isdefined and which has all its zeros and poles in the closed unitdisc|z| 1. That such a function exists follows from the spectralfactorization theorem.
We shall consider vector-valued complex functions as row
vectors and the inner product of two such functions f(z),g(z):C C1m is defined asf,g = 1
2
f(e
j)g(ej) d whereg denotes the complex conjugate transpose ofg. When fandgare matrix-valued functions, we will still use the notation
f,g to denote the integral 1
2 f(ej)g(ej) d whenever thedimensions off andgare compatible. The space L2consists of all
Fig. 1. Block diagram of SISO LTI system with output feedback.
functionsf: C C1m such thatf,f
8/11/2019 Artigo erro varincia identificao.pdf
3/14
2514 J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525
noise with unit variance, and where we assume that the feedbackcontroller K stabilizes the system. When this system is modeled as
yt= G(q, )ut+H(q, )et,with G and Hgiven by (1), and the parameter vector is estimatedusing prediction error identification with a least-squares costfunction, the asymptotic covariance matrix for the parameter
estimate
N (Nis the sample size, i.e. the number of input/output
measurements) is given by
P=o1,where
= 12
(ej)(ej) d , (4)
where in turn
(q)=
Go
Ho
So
AoRna (q)
So
Ao
o na (q)
GoHo
So
BoRnb (q)
KGoSo
Bo
o nb (q)
0 1Co
o nc(q)
0 1
Do
o nd (q) Go
Ho
So
FoRnf (q)
KGoSo
Fo
o nf (q)
, (5)
where n(q)= [q1, . . . , qn]T, and whereSo
1
1+GoKis the sensitivity function.
It can be shown also, using Gauss approximation formula, thatthe asymptotic variance of an estimate of a root zo of one of thepolynomialsAoFoof the true system is given by
AsVarzo P, (6)
where
dz( )
d
=o
.
For exact conditions for the above asymptotic results to hold werefer to Chapters 89 inLjung(1999).
Since(6)implies the approximation
E|z(N) zo|2 AsVarzo/N (7)for the finite sample size variance of an estimate z(N) of a root ofone of the polynomialsAoFo, AsVarz
o provides information aboutthe variance of root estimates for large sample sizes. The objectiveof this paper is to analyze the right-hand side of(6) and we willmake the following assumptions which will hold throughout the
paper.
Standing Assumptions 2.1. The polynomials Co, Do and Fo areminimum phase. The rational transfer function R is minimumphase. WithK= Kn/Kd , whereKnandKdare coprime polynomialsinz1, it holds that
Acl=AFKd+BKnhas all its roots strictly inside the unit circle. The matrix , definedin(4),is non-singular and the noise variance is positive,o > 0.Finally, all results are valid only for roots of multiplicity 1.
The conditions above are standard in a system identificationcontext, cf.Ljung(1999). The minimum phase conditions onCo,DoandFo guarantee that the one-step ahead predictor of the output
is stable. Thus poles outside the open unit disc|z| < 1 are locatedin Ao. In prediction error identification, only the spectrum of the
Table 1
Details forTheorem 3.1.
Polynomial L(z) i
AozHo(z)Go (z)So (z)R(z) 1
BozHo (z)Go (z)R(z) 1
CozHo (z)
Ho (z)
2
Do zHo (z)Ho (z) 2Fo
zHo(z)Go (z)So (z)R(z) 1 See Remark 3.2.
reference signal matters andR should be interpreted as a spectralfactor of this spectrum and hence the stability and minimumphase conditions on this quantity are not particularly restrictive,save that periodic signals are not covered. Furthermore, the datashould be generated under stationary conditions which require thetransfer functions fromr ande to u and y to be stable. Since, asnoted above, Ao contains all poles on or outside the unit circle,this is equivalent to the closed loop pole polynomial Acl beingminimum phase. The non-singularity ofis a joint condition onidentifiability at
o and persistence of excitation. A sufficient andnecessary condition for this assumption to hold is that the rowsof are linearly independent. This in turn requires both pairs{Co, Do}and{Bo, Fo}to be coprime. We refer toGevers, Bazanella,Bombois and Miskovic(2009)for interesting recent sufficient andnecessary conditions for the asymptotic covariance matrix to benon-singular.
3. General results
In this section we present the main technical results of thispaper. The first result, Theorem 3.1, will give us a generalexpression for the asymptotic variance of a pole or zero identifiedin open or closed loop, and that expression will be a starting pointfor deriving more explicit expressions.
3.1. Characterization in terms of basis functions
Theorem 3.1 (A General Result). The asymptotic variance of a root zo
of any of the polynomials Ao, Bo, Co , Door Fois given by
AsVarzo = o|L(zo)|2K i(zo,zo), (8)whereK i is the ith diagonal element of
K(,z)
nk=1
Bk ()Bk(z), (9)
where{Bk}nk=1 is any orthonormal basis for the rowspace of (defined by(5)).Table1provides L and i for the different cases A oFo.For roots of Ao, Boand Fo, the result(8) holds only if R=0.
Furthermore, when some parameters are known, the above result
still holds, save that the rows of that correspond to the knownparameters should be removed when computing th e subspace used togenerate K.
Proof. The proof is given inAppendix B.
Remark 3.2. In the cases marked by inTables 1and 2we havethatL(zo)= 0 (forAthis only holds in open loop operation). Thisdoes of course not mean that the asymptotic variance is zero. Thereason is that K i(z,z) has a pole at the same location that cancelsthe zero inL. To see this we use that K i(z,z) can be written
K i(z,z)=i (z) 12
(ej)(ej) d 1 i(z), (10)
8/11/2019 Artigo erro varincia identificao.pdf
4/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2515
where iis the ith column of, see Appendix B. Take now, e.g., thecaseCo where L(z
o)= 0 since Co(zo)= 0. In that case there is afactor 1/Co(z
o) in 2(zoc) that cancels the factor Co(z
o) in L(zo).
We illustrate the use ofTheorem 3.1with an example.
Example 3.3. Consider the FIR system
yt
=(1
ej q1)(1
ej q1)ut
1
+et, (11)
where > 0, where the input is white noise with unit variance,i.e.R= 1 and whereetis white noise with varianceo= 1. Wewill useTheorem 3.1to analyze the asymptotic variance of zeroestimates associated with a FIR model of order n.
In order to compute the asymptotic variance for the zero zo =ej we first need to determine L in (8).From the second row inTable 1we obtain L= zHo(z)/Go(z)R(z)for zeros ofBo. For a FIRmodel Ho 1 and furthermore Go(z)= Go(z)/(1zoz1)=
z1(1 ejz1). Thus
L(ej )= 2ej2
1ej2 .From Table 1 it also follows that i = 1 in (8). We thus haveto determine the (1, 1)-element of K and for this we need
an orthonormal basis for the predictor gradient . This can bedone numerically by, e.g. GramSchmidt orthogonalization, andin Section6.4.2we will see that such a basis can sometimes beobtained without computations. In this example we notice that= n 0 so an orthonormal basis is given by zk 0,k=1, . . . , n. Thus we obtain
AsVarzo = o|L(zo)|2K1(zo,zo)
= 4
2(1cos(2))n
k=1|ej2 |2k
= 2
2(1cos(2))1 2n1 2 . (12)
Eq.(12)shows that the asymptotic variance grows exponentially
with the model order n when 1, i.e. forNMP-zeros.Fig. 2shows the zero estimates for 100 different noiserealization when 1000 inputoutput samples are used to estimate(11).ComparingFig. 2(a)(b) shows that for = 1/2 the samplevariance increases indeed quite significantly when the number ofparameters is increased fromn= 3 ton= 5. On the other hand,Fig. 2(c)(d) show that the increase is hardly visible when =3/2for the same change in model order.
Eq.(12)shows also that the asymptotic variance will be largewhen the zeros are close together, i.e. when 0. That this is thecase for estimated zeros is confirmed by comparing Fig. 2(c) and(e) which only differ in the angular position of the zeros.
The objective of this paper is to analyze the asymptotic variance(6)and even though Theorem 3.1provides a characterization of
this quantity for anyof the polynomialsAoFounder any stationaryexperimental condition, the result(8)requires further elaborationin order to provide useful insights. What remains to be doneis to characterize the function K and this in turn requires acharacterization of the rowspace of. We shall be able to providean explicit formula forK onlyfor some special cases but we willbeable to provide usefulboundsonK under quite general conditions.In fact, (8) contains structural information that we can extractwithout having to computeK explicitly. We pursue this idea next.
3.2. Lower bounds in terms of basis functions
Lemma 3.4 (Model Structure Comparison). Consider two differentmodel structures M1 and M2 defined as in (1)(2) corresponding to
predictor gradients 1
and 2
(see (5)), respectively, and supposethat the rowspace of 1 is containedin therowspace of 2. Suppose
that Go= Bo/(AoFo)and Ho= Co/(AoDo)can be described by bothmodel structures. Then the asymptotic variance for any root of AoFo,when M1 is used, is no greater than the corresponding asymptoticvariance whenM2is used.
Proof. The result follows directly from Theorem 3.1 and Lemma A.1.
However, we can also combineTheorem 3.1 andLemma 3.4
to obtain lower bounds on the asymptotic variance where K isreplaced by a quantity that is easier to characterize.
Lemma 3.5 (General Lower Bound). The asymptotic variance of aroot of any of the polynomials Ao, Bo, Co, Do or Fo is bounded frombelow according to
AsVarzo oLlb(zo)2 Klb(zo,zo), (13)
where
Klb(,z)
k
Bk ()Bk(z). (14)
Here{Bk} is any orthonormal basis for the rowspace of the vectorspecified inTable2 for the different polynomials.Table2also defines
the function Llb. In the table, Ry and Ru are transfer functions with allzeros strictly inside th e unit circle and no poles outside the unit circle,satisfying
|Ry(ej)|2 = |R(ej)|2 + o|Ho(ej)|2
|Go(ej)|2 ,
|Ru(ej)|2 = |R(ej)|2 + o|Ho(ej) K(ej)|2.(15)
Proof. The proof is given inAppendix C.
The usefulness of Lemma 3.5 is that the subspace which thebasis functions {Bk} should span is much simpler than thecorresponding space inTheorem 3.1,cf. (5)with the functionsgiven in Table 2. Infact, we will be able to use results in Appendix Ato provide explicit bounds and expressions for Klb.
Corollary 3.6. Equality holds in(13)for FIR and OE models. This alsoholds for BoxJenkins models when the condition
KGoSo
Bonb+nf is orthogonal to
1
CoDonc+nd (16)
is satisfied.
Proof. The proof is given inAppendix D.
Remark 3.7. Notice that(16)holds when the system is operatingin open loop, i.e.K 0.Remark 3.8. The ratio between the two terms
o|Ho|2|Go|2 and |R|
2 that
defineRy is the ratio of the parts of the output spectrum that are
due to the noise and the reference signal. Similarly, the ratio ofthe two terms that define Ru is the ratio of the parts of the inputspectrum that are due to the noise and the reference signal.
The following lemma provides bounds for Ry and Ru definedin(15).
Lemma 3.9 (Bounds on Ry and Ru).When Godoes not have zeros onthe unit circle, it holds that Ry, defined by(15)inLemma3.5,satisfies
1+inf
o|Ho(ej)|2|R(ej) Go(ej)|2
Ry(zo)R(zo)
2 1+sup
o|Ho(ej)|2
|R(ej) Go(ej
)
|2
, (17)
when R is not identically zero, and Ry=o Ho/Gowhen R0.
8/11/2019 Artigo erro varincia identificao.pdf
5/14
2516 J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525
(a)n=3, =1/2, = /4. (b)n=5, =1/2, = /4. (c)n=3, =3/2, = /4.
(d)n=5, =3/2, = /4. (e)n=3, =3/2, = /10.
Fig. 2. Simulation results forExample 3.3.
Table 2Details forLemma 3.5.
Polynomial Llb(z) {Bk}spans rows of
AozHo (z)Go (z)So (z)Ry(z) Go SoRyHoAo na
BozHo (z)Go (z)Ru(z) GoSoRuHo BoFo nb+nf
CozHo (z)Ho (z) 1CoDo nc+nd
Do zHo (z)Ho (z) 1CoDo nc+ndFo
zHo (z)Go (z)So (z)Ru(z) GoSoRuHo BoFo nb+nf See Remark 3.2.
Furthermore when Hoand K are stable, Ru, also defined by(15)inLemma3.5,satisfies
1+inf
o|Ho(ej)K(ej)|2|R(ej)|2
Ru(zo)R(zo)2
1+sup
o|Ho(ej)K(ej)|2|R(ej)|2 , (18)
when R is not identically zero, and
Ru=oHoK wh en R0. (19)Proof. The proof is given inAppendix E.
3.3. Explicit lower and upper bounds
We now provide bounds which indicate how the asymptoticvariance of estimated roots depends on the model order.
Theorem 3.10. The asymptotic variance of a root of any of the
polynomials Ao, Bo, Co, Door Fois bounded according to
o|zo|2c11 |zo|2m2
1 |zo|2 AsVarzo
c21 |zo|2m2
1 |zo|2 , (20)
where m and c1 are given in Table 3 and where c2 = c2(Go, Ho,K, R, o)also depends on which polynomial that z
o belongs to (but
not on any of the orders of the estimated polynomials), and where
m=mabf na+nb+nf+mofor some finite mo0,for zo belongingto Ao, Boand Foand wherem=mcd na+ nb+ nc+ nd+ nf+ mo
for some finite mo0, for zo
belonging to Coand Do.
Proof. The proof is given inAppendix F.
Notice thatTheorem 3.10also applies to the case|zo| = 1, bysetting
1|zo|2m21|zo|2 = m in (20). This is relevant forAoand Bowhich
are allowed to have such roots, but not for the other polynomials
(byStanding Assumptions 2.1).
3.4. Explicit characterization for BoxJenkins models
Building on Corollary 3.6, we can provide explicit upper bounds
for the asymptotic variance of the roots for FIR, output error and
BoxJenkins models under certain restrictions. In some cases it
follows fromCorollary 3.6that these bounds are tight.
8/11/2019 Artigo erro varincia identificao.pdf
6/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2517
Table 3
Details forTheorem 3.10.
m c1
Ao na1
|Ao(zo )|2 min Ho (ej )Ao (ej )
So (ej )Go (ej )Ry (ej )
2Bo nb+nf 1|Bo (zo)Fo(zo )|2 min
Ho (ej )Bo (ej )Fo(ej )So (ej
)Go(ej )Ru(ej
)
2Co nc+nd 1|
Co(z
o)Do (zo )|2 min
Co(ej )Do(e
j )
2
Do nc+nd1
|Co(zo)Do (zo )|2 min Co(ej )Do(ej )2Fo nb+nf 1|Bo (zo)Fo(zo )|2 min
Ho (ej )Bo (ej )Fo(ej )So (ej
)Go(ej )Ru(ej
)
2
Theorem 3.11 (FIR, OE and BJ Models). Assume that the model
structure is FIR, OE or BoxJenkins, where for the latter we assume
that(16)holds.
The asymptotic variances of theroots of the Coand Dopolynomials
are given by
AsVarzo =|Co(1/zo)Do(1/z
o)|2|Z(zo)|2
|zo|2(nc+nd)(|zo|2 1) , (21)
where Z=CoDofor a root of Coand Z=DoCofor a root of Do.Furthermore, denote the denominator of GoSoRuBoFoHo (after cancella-
tions) by A and suppose in the following that the order n of this
polynomial is less than nb+nf.Denote the order of the numerator of GoSoRu
BoFoHo(after cancellations)
by m. Then the asymptotic variance of a root zo = 0of the Bo or Fopolynomials, with|zo| = 1, is bounded from above according to
AsVarzo 1(1 |zo|2)
o|Ho(zo)|2|Go(zo)So(zo)Ru(zo)|2
1|A(1/zo)|2
|A(zo)|2 |zo|2(nb+nf+m)
, (22)
where Ruis defined inLemma3.5.
For a root of Bo with magnitude 1, i.e.
|zo
| = 1, the asymptotic
variance is bounded from above according to
AsVarzo o|zoHo(z
o)|2|Go(zo)So(zo)Ru(zo)|2
nb+nf+mn+n
k=1
1 |k|2|zo k|2
, (23)
where the{k}nk=1are the roots of A.When m = 0, i.e. when the numerator of GoSoRu
BoFoHois constant,
equality holds in(22)and(23).
Proof. The proof is given inAppendix G.
In the following sections we will interpret and elaborate on theresults in this section.
4. Zeros and poles in the open unit disc|z| < 1
Due to the factor |zo|2m in the numerator of(20), Theorem 3.10shows that the asymptotic variance for zeros and poles with
magnitude less than one increases exponentially with the order of
the corresponding polynomial, cf.Example 3.3.Thus, the accuracycan be poor even when only a moderate number of parameters are
used for a certain polynomial. We alert the reader that the result
holds regardless of whether the system is operating in open orclosed loop. Also notice that the asymptotic variance for a root of
Co(Do) can belarge evenifnc(nd) is small. This happens ifnd(nc) is
large since m=nc+ndfor the Coand Dopolynomials (see Table 3).A similar conclusion holds for Boand Fo.
For open loop operation, or whenever (16) holds, an exactformula for the asymptotic variance of the roots of the noise modelpolynomials Co and Do is given in Theorem 3.11 by (21). Thistheorem also gives an exact expression for roots ofFounder certainconditions. Notice that these formulae exhibit the exponentialgrowth with the order discussed in the preceding paragraph.
We conclude that accurate identification of zeros and polesstrictly inside the unit circle is inherently difficult for model
structures of high orders.
5. Unstable poles
In this section we will study the asymptotic variance of esti-mates of poles located strictly outside the unit circle. By assump-tion, all such poles are contained in Ao , see the discussion belowStanding Assumptions 2.1.Since the theory covers stationary ex-perimental conditions, closed loop operation is necessary for theresults in this section to hold.
Theorem 3.10provides generally applicable lower and upperbounds for the asymptotic variance of an unstable pole. However,more transparent expressions canbe obtained by considering largemodel orders. We have the following result.
Theorem 5.1 (Unstable Poles).Consider a root zo of Ao with|zo| >1. The asymptotic variance AsVarzo increases monotonically to a
finite limit as n= na+ nb+ nc+ nd+ nf tends to infinity. Thelimit is bounded from above according to
limn
AsVarzo o Co(zo)Ao(zo)Do(zo)
2 K(zo)R(zo)2 1(1 |zo|2) (24)
and, provided na , bounded from below by
o
Co(zo)Ao(zo)Do(zo)2 K(zo)Ry(zo)
2 1(1 |zo|2) limn AsVarzo, (25)where Ry is defined by (15)inLemma3.5.
Proof. The proof is given inAppendix H.
The upper bound in Theorem 5.1 shows that, contrary to polesstrictly inside the unit circle (see Section 4), the asymptoticvariance for unstable poles converges to a finite limit as the modelorder increases. FromTheorem 5.1we see that the location of thepole is critical for the asymptotic variance when the number ofparameters in A is large. A pole close to the unit circle will havehigh asymptotic variance.Furthermore, we see that also the gain ofthe controller at the pole is important for the asymptotic variance.
Notice that Lemma 3.9 gives that|Ry(zo)| > |R(zo)| whichshows that the lower bound inTheorem 5.1is indeed lower thanthe upper bound in the same theorem. In the following subsectionswe will consider some special cases where Ry can be expressedexplicitly and which will shed further light on the asymptoticvariance of unstable poles.
5.1. High SNR at the output
Corollary 5.2. For a root zo of Aowith|zo|> 1, it holds
limna
AsVarzo
o
Co(zo)Ao(zo)Do(zo) 2 K(zo)R(zo) 2 1(1|zo|2) =1, (26)when
sup
o|Ho(ej)|2|R(ej) Go(ej)|2
0. (27)
Proof. In the limit(27)it follows from(17)that|R(zo)/Ry(zo)| 1 and hence the lower bound(25)equals the upper bound(24)inthe limit(27).
8/11/2019 Artigo erro varincia identificao.pdf
7/14
2518 J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525
Corollary 5.2shows that at high SNR at the output, cf. Remark 3.8,the asymptotic variance of a pole of the system,located outside theunit disc, is approximately equal to the upper bound in(24).
5.2. Proportional signal and noise spectra at the output
When the two terms defining Ry, see Lemma 3.5, areproportional to each other, the lower bound in Theorem 5.1can
be expressed explicitly.
Corollary 5.3. Assume that R= Ho/Go for some > 0. Thenfor a root zo of Aowith|zo|> 1 it holds
1
1+ o
K(zo) Bo(zo)Ao(zo)Fo(zo)
2
1
(1 |zo|2) lim
naAsVarzo. (28)
Proof. With R = Ho/Go = Fo/(BoDo), it follows thatRy=
o+ Fo/(
BoDo) and(25)inTheorem 5.1gives the result.
5.3. Costless identification
The case when there is no external excitation, i.e. R 0, hasbeen coined costless in Bombois, Scorletti, Gevers, Van den Hof,and Hildebrand(2006) since identification can be performed inclosed loop without perturbing the system from normal operatingconditions.
Corollary 5.4. Assume R0. Then for a root zo of Aowith |zo| > 1,it holdsK(zo)
Bo(zo)
Ao(zo)Fo(zo)
21
(1 |zo|2) limna AsVarzo. (29)
Proof. Follows directly fromCorollary 5.3by setting
=0.
InCorollary 5.4we see again the critical role that the pole locationand the controller gainK(zo) play for the asymptotic variance of apole outside the unit disc.
6. Non-minimum phase zeros
Now we turn to the asymptotic variance of non-minimumphase zeros of the system. Here the results below cover both openand closed loop identification.
Theorem 6.1 (NMP-zeros).Consider a root zo with|zo| > 1of Bo.The asymptotic variance increases monotonically to a finite limit asn=na+nb+nc+nd+nftends to infinity.
The limit is bounded from above according to
limn
AsVarzo o|Ho(zo)|2|Go(zo)R(zo)|2 1(1 |zo|2) (30)and, provided nb+nf , bounded from below by
o|Ho(zo)|2|Go(zo)Ru(zo)|2 1(1 |zo|2) limn AsVarzo. (31)Equality holds in(31) for FIR and OE models as well as BoxJenkinsmodels subject to(16).
Proof. The proof is given inAppendix I.
As for poles outside the unit disc, we see that the asymptoticvariance of an NMP-zero converges to a finite limit as the modelorder increases. We see also that the location is important for the
asymptotic variance of a zero. An NMP-zero close to the unit circlewill have large asymptotic variance.
6.1. Open loop operation or high SNR at the input
Corollary 6.2. For a root zo of Bowith|zo|> 1, it holds
limnb+nf
AsVarzo
o
Ho(z
o)
Go(zo)R(zo)
2
1
(1|zo|2)
=1, (32)
when
sup
o|Ho(ej)K(ej)|2|R(ej)|2 0. (33)
In particular,(32)holds in open loop operation, i.e. when K 0.Proof. In the limit(33)it follows from(18)that|R(zo)/Ru(zo)| 1 and hence the lower bound(31)equals the upper bound(30)inthe limit(33).
From Corollary 6.2 we are able to conclude that the upper bound inTheorem 6.1 is attained when thesystem is operating in open loop.Furthermore, for closed loop operation at high SNR at the input, cf.Remark 3.8,and with a large number of parameters in Band/orF,the asymptotic variance becomes equal to the asymptotic variancein open loop operation.
6.2. Proportional signal and noise spectra at the input
When the two terms defining Ru, see Lemma 3.5, areproportional to each other, the lower bound in Theorem 5.1canbe expressed explicitly.
Corollary 6.3. Assume that R=HoK for some > 0 . Then fora root zo of Bowith|zo|> 1 it holds
1
1+ o
1K(zo) Bo(zo)Ao(zo)Fo(zo)
2
1
(1 |zo|2) limnb+nf AsVarzo. (34)
Equality holds in(34) for FIR and OE models as well as BoxJenkins
models subject to(16).Proof. WithR=HoK, it follows thatRu= o+ HoKand(31)inTheorem 5.1gives the result.
Corollary 6.3provides some insight regarding the role played bythe controller for the asymptotic variance of NMP-zeros. SinceK(zo) appears in the denominator in the lower bound(34),a smallcontroller gain will result in a large asymptotic varianceexactlythe opposite to what is the case for poles outside the unit disc, cf.Section5.
6.3. Costless identification
Corollary 6.4. Assume R0. Then for a root zo of Bowith|zo| > 1it holds
1K(zo) Bo(zo)Ao(zo)Fo(zo) 21
(1 |zo|2) limnb+nf AsVarzo. (35)
Equality holds in(35) for FIR and OE models as well as BoxJenkinsmodels subject to(16)
Proof. Follows directly fromCorollary 6.3by setting =0. Asin Corollary 6.3 we seethat Corollary 6.4 embodies the messagethat a small controller gain at the NMP-zero of interest implies alarge asymptotic variance.
6.4. Finite model order results
For FIR, OE and BoxJenkins models we can useTheorem 3.11to obtain results for finite model orders for NMP-zeros.
8/11/2019 Artigo erro varincia identificao.pdf
8/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2519
6.4.1. Upper bounds
Notice thatSo(zo)= 1 for a zero ofBo so the upper bound in
(22)is the same as the asymptotic limit in Corollary 6.2,save for
the factor1|A(1/z
o)|2|A(zo)|2
|zo|2(nb+nf+m)
(36)
which converges to 1 asnb+nf . The upper bound(22)canbe evaluated also for the remaining special cases ofRuused in the
preceding subsections, i.e. proportional noise and signal spectra at
the input and costless identification. The results are finite model
order upper boundsthat areequalto the asymptotic in model order
lower bounds(34)and (35),respectively, but with(36)as added
factor.
6.4.2. Exact results
Theorem 3.11can also be used to obtain exact expressions for
the asymptotic variance. For open loop operation we have the
following result.
Theorem 6.5 (Open Loop/FIR, OE and BJ Models). Assume that the
model structure is FIR, OE or BoxJenkins and that K 0. Assumefurther that Do= 1and that R has no zeros. Denote the polynomialF2o Co/R by Aand suppose that nb+ nfis larger than the order of this
polynomial. Then the asymptotic variance of a root zo =0 of Bo, with|zo| =1, is given by
AsVarzo = 1(1 |zo|2)
o|Ho(zo)|2|Go(zo)R(zo)|2
1|A(1/zo)|2
|A(zo)|2 |zo|2(nb+nf )
. (37)
When|zo| =1, the asymptotic variance of root zo of Bois given by
AsVarzo =
o|zo H
o(zo)
|2
|Go(zo)R(zo)|2
nb+nfn+n
k=1
1 |k|2|zo k|2
, (38)
where the{k}nk=1are the roots of A.Proof. Follows directly fromTheorem 3.11.
Notice that the above theorem is valid not only for NMP-zeros but
for any zeros. For closed loop operation we have the following
result.
Theorem 6.6 (Closed Loop/FIR and OE Models). Assume that the
model structure is FIR or OE and that the controller K= Kn/Kd= 0is such that Knand Kdare coprime stable polynomials such that
Fo=F1Knfor some polynomial F1 . Suppose further that R= K for some > 0and that nb+nfis at least the order of Fo(FoKd+Bo). Thenthe asymptotic variance of a root zo =0 of Bo, with |zo| = 1, is givenby
AsVarzo = 1(1 |zo|2)
1
1+ o
1
|G(zo)K(zo)|2
1|A(1/z
o)|2
|A(zo)
|2 |zo|2(nb+nf )
, (39)
where A=Fo(F1Kd+Bo).
When|zo| = 1, the asymptotic variance is given by
AsVarzo = |zo|2
1+ o
1
|G(zo)K(zo)|2
nb+nfn+n
k=1
1 |k|2|zo k|2
, (40)
where the{k}nk=1are the roots of A.Proof. Follows directly fromTheorem 3.11.
Theorem 6.6 is the NMP-zero counterpart to Theorem 4.1 inNinness and Hjalmarsson(2005b)where asymptotic variance forfrequency function estimates is considered. The condition on R canbe interpreted as that the feedback controller is u = K(r y)(rather than u=Kyras we have assumed throughout the paper)where the referenceris white noise.
7. Interpretations of the results
In the previous sections we have presented a number ofexpressions and simplified bounds for the asymptotic varianceof roots of the polynomials AoFo. We will now summarize and
interpret our major findings.
Roots inside versus outside the unit circle
An important result of this paper is that the asymptoticvariances of zeros and poles strictly inside the unit circle behavefundamentally different from the asymptotic variances of NMP-zeros and unstable poles when the model order is increased.Theorem 3.10 shows that the former grow exponentially whileTheorems 5.1 and 6.1 show that the latter converge to finite limits.Theorems 6.5 and 6.6 show that forFIR, OE andBoxJenkinsmodelsthe convergence for NMP-zeros is exponentially fast, with rate1/|zo|, under certain conditions. Thus roots closer to the unit circleimply slower convergence. Although this has not been proven, webelieve this to be true in general for both poles outsidethe unit disc
and NMP-zeros.
Roots on the unit circle
Combining the upper and lower bounds in Theorem 3.10givesthat for the borderline case with a root on the unit circle, theasymptotic variance grows linearly with the model order. This issimilar to what happens for frequency function estimates (Ljung,1985;Ninness & Hjalmarsson, 2004).
Roots close to each other
FromTheorem 3.1it follows that the asymptotic variance of arootof one ofthe polynomials blowsup whenthere isa nearbyrootof the same polynomial. The reason is that a root is very sensitive
to errors in the parameters of the polynomial in this case. Forexample, for the second order polynomial z2 +1z+2, it holdsthat
dz( )
d =1
2 1
2
2142
12142
T,
which tends to infinity as 21 42, i.e. when the polynomial hasa double root at1/2. In fact the standard asymptotic analysis,which is based on the first order Gauss approximation formula,breaks down for roots of higher multiplicity than 1. This does notimply that the variance of root estimates becomes infinite for,e.g., double roots, instead the convergence rate becomes slowerthan the 1/Nrate given in(7).We will not pursue this here butsimply conclude that the result above indicates that there are
significant problems associated with estimating roots of highermultiplicity than one.
8/11/2019 Artigo erro varincia identificao.pdf
9/14
2520 J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525
NMP-zeros
The universal upper bound
limn
AsVarzo o|Ho(zo)|2
|
Go(zo)R(zo)|2
1
(1 |zo|2) (41)
fromTheorem 6.1is useful when the reference signal is non-zero.
Notice that the bound is model structure independent. It is alsotight in open loop operation and for high signal to noise ratios atthe input (see Corollary 6.2) in closed loop operation, and above wehave argued that the convergence rate with respect to the modelorder is likely to be exponentially fast (at least we have shown thisfor some special cases). Thus the right-hand side of(41)providesin many cases a quite reasonable approximation of the asymptoticvariance for a NMP-zero.
The reason why (41)becomes tight when the signal to noiseratio at the input increases can be understood from the fact thatthe sensitivity function of the closed loop becomes unity whenevaluated at a zero of the system. Thus the feedback does notcontribute to the excitation at a system zero and when the signaltonoise ratio atthe input ishighso thatthe noise fed backto theinput
does little to excite the system, the open loop case is recovered.The above observation has several implications. The firstconcerns how open loop identification compares to closed loopidentification. It is well known that for BoxJenkins models closedloop identification with a reference signal having spectral factorR,so that the part of the input due to the reference signal is SoR, isnever worse than open loop identification with a reference signalhaving spectral factor SoR (Agero & Goodwin, 2007; Bombois,Gevers, & Scorletti, 2005). For NMP-zeros, it follows from(41)thatthis holds for high model orders for any modelstructure also whenRis the spectral factor of the input in open loop identification.
Secondly, the universality of (41) also implies that, forhigh model orders, indirect and direct identification result insimilar asymptotic variance of NMP-zeros. Recall that in indirectidentificationris used as input rather than u(Ljung, 1999).
Thirdly, notice that very different input spectra will result insimilar asymptotic accuracy. We illustrate this with an example.
Example 7.1. Consider the first order ARX system
(10.9q1)yt= (q1 +1.1q2)ut+et, (42)where the input is given byut= rt 10.5q
11+0.3q1ytand wherertand
etare independent white noise, both with unit variance. This shallbe compared with the open loop case when the input is given byut= rt. The input spectra are plotted inFig. 3and in this example,they are very different in open and closed loop (for the closed loopcase both the total input spectrum u and
ru , which is the part
of the spectrum that originates from the reference signal rt, are
plotted). Despite this big difference, the variance of the estimatedzero does not change, at least not for high model orders.
The system (42) is simulated in both open and closed loops.From the closed loop simulations we use both direct andindirect identification. For the open loop case and for the directidentification in the closed loop case, an ARX model is estimatedfrom 1000 samples ofutandyt. For the indirect identification weestimate an ARMAX model from 1000 samples ofrt and y t. Thevariance of the estimated zero, from 5000MonteCarlosimulations,is presented inTable 4.The model orderno represents the order ofthe true system, i.e., na = 1 and nb = 2 in the ARX model andna = 3, nb = 3 and nc= 1 for the ARMAX model. When themodel order is increased, no + k means that na and nb has beenincreased by k , and nc is unchanged. The values in the table are
the variances from the simulations, multiplied byN= 1000. Theasymptotic value 5.8 is the upper bound given by(41).
Table 4
Variance of estimated non-minimum phase zeros from Monte Carlo simulations in
open and closed loops.
n Open loop Closed loop direct id. Closed loop indirect id.
no 2.3 0.21 4.7
no +2 4.0 1.8 6.6no +5 6.9 4.5 7.3no +10 6.9 7.2 6.2n
o
+15 6.0 6.5 5.5Asympt. 5.8 5.8 5.8
frequency
Inputspectrum
r
103
102
101
100
101
1020 0.5 1 1.5 2 2.5 3
u(), closed loop
u(), open loop
u(), closed loop
Fig. 3. Inputspectrum for Example7.1. Theinput spectra are very differentin open
and closed loops. This does not affect the accuracy of the zero estimates.
What we see is that the direct identification in closed loopperforms best for low model orders, but already fromno +5 andup, all three cases give about the same variance, just a little higherthan the asymptotic value.
Next we make some further comments on the accuracy ofestimated non-minimum phase zeros, based on(41).
Usually there is a tradeoff between bias and variance in the
sense that a low order model results in a biased estimate withlow(er) variance and a high order model gives an un-biasedestimate with high(er) variance. When it comes to NMP-zeros,this increase in the variance is typically small and, hence, highorder models can be used without the risk of loosing accuracy,unless the zero is close to the unit circle.
In many identification scenarios the input filter R(q) can bechosen by the user. The variance is proportional to 1/|R(zob )|2 sothat factor should be small. IfR(q) has a pole atzob this factor iszero, but that corresponds to an unstable, and hence infeasible,input filter. An interesting choiceis an autoregressive filter with
just one pole at 1/zob . Such an input filter is optimal in thesense that it has the lowest input power among all LTI filtersthat achieve the same variance, seeMrtensson, Jansson, andHjalmarsson(2005).
Consider open loop identification and suppose for a momentthat Go(q) is known and that only the part 1 zoz1 has tobe estimated. In this scenario,Go(q)= 1 and the input filterRshould be the original input filter in series with the originalGo(q), since the latter part of thesystem shapes theinput to thesystem 1 zoz1, which is to be identified.Notice now that theinput filterR(q)and the factorGo(q)enter the bound(41)as aproduct, i.e. the bound is the same if the product between theinput filterand thesystemthat is identified, excludingthe NMP-zero part, is the same. Thus in the above scenario, the boundremains thesameas when alsoGo(q) is identified. Furthermore,we have fromCorollary 6.2,that as nb+nf , the bound(41) is attained. Combining these observations we have that forhigh model ordersand open loop identification it is insignificantwhetherGo(q)is known or estimated, i.e. knowing Go(q)andonlyestimating 1zoz1 helpslittle for improving the accuracy.
8/11/2019 Artigo erro varincia identificao.pdf
10/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2521
Unstable poles
The universal upper bound
limn
AsVarzo o
Co(zo)
Ao(zo)Do(zo)
2
K(zo)
R(zo)
21
(1 |zo|2) (43)
fromTheorem 5.1is useful when the reference signal is non-zero.
Notice that the bound is model structure independent. It is also
tight for high signal to noise ratios at theoutput (see Corollary 5.2)
and above we have argued that the convergence rate with respect
to the model order is likely to be exponentially fast (at least we
have shown this for some special cases). Thus the right-hand side
of(43)provides in many cases a quite reasonable approximation
of the asymptotic variance for a pole located outside the unit disc.
The upper bound(43)and the corresponding lower bound(25)
indicate that the controller should be designed so that the gain at
the pole is small in order to ensure a small asymptotic variance.
In this context, we point out that if the feedback configuration is
changed to ut= K(q)(rt y t) instead ofu t= rt K(q)yt asabove, the controller dependence in(43)and in other expressions
for the asymptotic variance of unstable poles will disappear. This
is easy to see since this change in controller structure correspondsto replacingRbyKR.
Costless identification
Forcostlessidentification,i.e. when theexternal referenceis not
used to excite the system, the lower bounds 5.4and6.4show that
the controller gain should be small at unstable poles but large at
NMP-zeros in order for the asymptotic variance to be small. Thus
there is a conflict when designing suitable experiments when the
objective is to identify a NMP-zero and an unstable pole that are
close to each other.
8. Summary
In this paper we derive explicit expressions for the asymptotic
variance of estimated poles and zeros of dynamical systems. One
of the main conclusions is that the asymptotic variance of non-
minimum phase zeros and unstable poles is only slightly affected
by the model order, while the variance of minimum phase zeros
and stable poles is very sensitive and grows exponentially with the
model order. Another important observation is that nearby roots
have a very detrimental effect on the asymptotic variance.
The asymptotic variance expressions give structural insight to
how the variance is affected by e.g., model order, model structure
and input excitation.
Acknowledgments
The authors would like to express their gratitude to the
anonymous reviewers for their many insightful remarks and for
spurring us to a major revision which, we believe, significantly
improved the quality of the paper.
Appendix A. Technical results for reproducing kernels
Let{Bk}nk=1 be an orthonormal basis for an n-dimensionalsubspace Sn L2 and let K(,z)denote the reproducing kernelfor Sn, here defined as
K(,z)
nk=1
Bk ()Bk(z). (A.1)
The function K(,z)is called a reproducing kernelsince it holdsthat
f(),K(, ) =f(), f Snf(),K(, ) =ProjSn{f}(), f L2 (A.2)where ProjSn{f}denotes the orthogonal projection off L2ontoSn, see e.g.,Heuberger, Wahlberg, and den Hof(2005).
The reproducing kernel is unique and can also be expressed as
K(,z)=(), 1(z) (A.3)for any whose rowspace spans Sn(Heuberger et al., 2005).
In the following lemmas we present some properties of thereproducing kernels of different subspaces which are used in theproofs of the paper.
Lemma A.1. Consider two subspaces Sn andSm with reproducingkernelsK(,z) andK(,z) and letSnSm. Then it holds thatK(z,z)K(z,z). (A.4)Proof. Let{Bk}nk=1 and{Bk}mk=1 be orthonormal bases for Sn andSm, respectively, with Bk=
Bk,k=1, . . . , n. Then it is clear that
K(z,z) K(z,z)= mk=n+1Bk (z)Bk(z)0,
which concludes the proof.
Lemma A.2. Let the scalar-valued subspaces Sn andSm havereproducing kernels K(,z)andK(,z). Suppose that there existsa constant: 0 < 1 such that for every function f Sn thereexists a function gSmthat fulfills|f(z) g(z)| < |f(z)|, z ZwhereZ is some region including the unit circle. Then it holds that
K(z,z)
1+ 1
2K(z,z), z Z.
Proof. Since K(,z) itself, as a function ofz, belongs to Sn, thereis by assumption a family of functionsgSmsuch that|K(,z) g(z)|< |K(,z)|, z Z.Thus, we can write
K(,z)=g(z) +(z),where
|(z)|< |K(,z)|, z Zwhich also implies that()< K(, )since Z includesthe unit circle. Hence,
|K(,z)| |g(z)| + |K(,z)|,and by using the CauchySchwarz inequality and the fact thatK(, ),K(z, ) =K(,z) we get
|K(,z)| 11 |g(z)| =
1
1 |g(),K(z, )|
11 g()
K(z, ) = 11 g()
K(z,z)= 1
1 K(, ) ()K(z,z)
11 K(, ) + ()K(z,z)
11 (K(, ) + K(, ))
K(z,z)
= 1+ 1 K(, )K(z,z).
8/11/2019 Artigo erro varincia identificao.pdf
11/14
2522 J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525
For =zit holds that K(z,z)= |K(z,z)|and then squaring theinequality above gives the result.
Lemma A.3. Consider the subspace
Sn Span
z1
M(z), . . . ,
zn
M(z)
, (A.5)
where
M(z)
mi=1
(1 iz1), |i|< 1 (A.6)
and nm. The reproducing kernel of the space Snis given by
K(z,z)=
|z|2
1 |z|2
1|M(1/z)|2
|M(z)|2 |z|2n
,
z=0 & |z| =1,nm+
mk=1
1 |k|2|z k|2
, |z| = 1.(A.7)
Furthermore, the reproducing kernel for the space spanned by zk,
k=1, 2, . . . can be expressed asK(z,z)= |z|
2
1 |z|2 , for|z|> 1. (A.8)
Proof. For z : z = 0,|z| = 1, the reproducing kernel for Sn isgiven by
K(,z)= 1 n()n(z)z 1 , (A.9)
where k(z) k
j=11jzzj , see Ninness, Hjalmarsson, and
Gustafsson(1998), and it can also be expressed in terms of thefunctionM(z) by noting that
n(z)= M(z1)
M(z)zn. (A.10)
Combining (A.9) and (A.10) gives the result (A.7) when |z| =1. Theresult when|z| =1 follows directly fromNinness and Gustafsson(1997).
For(A.8)we notice that Bk(z)= zk, k = 1, 2, . . . form anorthonormal basis and hence
K(z,z)=
k=1|z|2k = |z|
2
1 |z|2 , |z|> 1.
Lemma A.4. Consider the subspace
Sn Span Q(z)
M(z)z1, . . . , Q(z)
M(z)zn , (A.11)
where M(z) is given by(A.6)(with mn) andQ(z) q0+q1z1 + +qnqznq (A.12)has all zeros inside the unit circle. LetKnbe the reproducing kernel of
Sn. ThenKn(z,z) is bounded by
Kn(z,z) |z|21 |z|2
1|M(1/z)|
2
|M(z)|2 |z|2(n+nq)
, (A.13)
when z=0 is such that|z| =1 and when|z| =1 by
Kn(z,z)n+nqm+m
k=1
1
|k
|2
|z k|2 . (A.14)
Proof. Define SpasSp Span z1
M(z), . . . ,
zp
M(z)
. (A.15)
If we let p = n+ nq then Sn Sp and Lemma A.1 givesthat Kn(z,z)
Kp(z,z)where
Kp is the reproducing kernel of
Sp. Lemma A.3gives the right-hand sides of(A.13)and (A.14)asexpressions forKp(z,z). Lemma A.5. Consider the subspace
Sn Span
F(z)z1, . . . , F(z)zn
, (A.16)
where F is a rational stable transfer function. Let Kn be thereproducing kernel of Sn. ThenKn(z,z) is bounded according to
1 |z|2n21 |z|2 |F(z)|
2 1
sup
F(ej)2 Kn(z,z) 1 |z|
2n2
1
|z
|2 |F(z)|2
1
inf
F(ej)2
. (A.17)
Proof. Observe that the eigenvalues of
1
2
F(ej)2 n(ej)n(ej)d (A.18)are bounded from below by inf
F(ej)2 and from aboveby sup
F(ej)2, see Grenander and Szeg (1958). Hence theeigenvalues of the inverse of (A.18) are lower bounded by
1
sup|F(ej )|2 and hence
K(z,z)=|F(z)|2
n(z) 1
2
F(ej)2 n(ej)n(ej)d1
n(z)
|F(z)|2 1sup
F(ej)2 n(z)n(z),which gives the lower bound. The upper bound follows using thelower bound for the eigenvalues.
Lemma A.6. LetSnandKnbeasin Lemma A.4 andlet|z|> 1.Whenthe dimension n goes to infinity, the reproducing kernel is given by
limn
Kn(z,z)= |z|21 |z|2 . (A.19)
Proof. Consider the two spaces Sn andSp given by (A.11)and (A.15) and let Kn(,z) andKp(,z) be the associatedreproducing kernels. For any value p of the dimension of thesubspaceSp there is a number r(p) such that there existsparameters ksuch that1L(z) r(p)
k=0kz
k< 1p , z: |z| 1,
i.e. on and outside the unit circle the function 1/L(z) can beapproximated arbitrarily well with an FIR-filter by choosinga sufficiently large number r(p). This is a consequence ofMergelyans theorem, see e.g.Rudin(1987). Let this FIR-filter be
denotedS(z) r(p)
k=0 kzk. A functionf
Spcan be written as
f(z)= 1
M(z)
pk=1
kzk
8/11/2019 Artigo erro varincia identificao.pdf
12/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2523
for some arbitrary parameters k. Now notice that the functionL(z)S(z)f(z) belongs to the subspace Sn if we take n = n(p)=
p+r(p). Thus, for any functionfSp , there isag= LSf Snsuchthat
|f(z) g(z)| = |f(z) L(z)S(z)f(z)|= |1L(z)S(z)||f(z)|
1
p |f(z)|, z: |z| 1.Lemma A.2can now be applied and gives
1 1p
1+ 1p
2 Kp(z,z)Kn(p)(z,z).The sequence Kn(z,z), n= 1, 2, . . . is monotonically increasingand for|z| > 1 Lemma A.4 gives the finite upper boundKn(z,z) |z|
21|z|2 and hence it follows that Kn(z,z)has a limit.
The subsequence Kn(p)(z,z),p= 1, 2, . . . then converges to thesame limit and we can conclude that
limp
Kp(z,z) lim
n
Kn(z,z).
Lemma A.3gives that the lower bound approaches
limpKp(z,z)= |z|2
1 |z|2 ,
which is the same as the upper bound and that proves the result.
Appendix B. Proof ofTheorem 3.1
The starting point is the variance expression(6)which also canbe written as
AsVarzo =o(o), 1(o), (B.1)where is given by (5)and where ( ) is the gradient of the
pole/zero with respect to the parameters. This gradient is given bythe next lemma.
Lemma B.1. Let za(a) be a zero of A(q, a) of multiplicity1. Then
dza(a)
d a
a=oa
= zoaAo(zoa ) na (zoa ),
wheren(q)= [q1, . . . , qn]T. For zeros of the other polynomialsB, C , D and F we get analogous expressions.
Proof. The proof can be found in, e.g., Oppenheim and Schafer(1989).
Now we return to the expression(B.1)and consider a zero ofB(q),which is the most straightforward case. The result for the othercases will be commented at the end of the proof. Consider thefunction J( )= zb(b), which does not depend on a, c, d andf. Thus, fromLemma B.1,
(o)=
0na1
zobBo(zob ) nb (zob )0nc10nd10nf1
.It is now readily verified that
1(zob )
Ho(zob )Bo(z
ob )
Go(
zob )
So(
zob )
R(zo
b ) =
0nb (zob )
0
00
(B.2)
and hence 1(zob )L(z
ob )= (o)with L(z)given inTable 1since
So(zob )=1 regardless of the feedback. Thus, we can write (B.1)as
AsVarzo =o|L(zob )|21 (zob ), 11(zob ). (B.3)Now, using (A.3), we recognize 1 (), 11(z) as thereproducing kernel to the rowspace to and can be written as(A.1),i.e.(9).This now gives the result(8)inTheorem 3.1.
The other cases A, C, D and Ffollow in the same way, exceptfor one detail which is explained here. For brevity only the case
Ais considered here but the result holds also for the other cases.It should first be noted that is singular atzoa . This singularity iscanceled by the factor Ao in the function L , seeTable 1, and westill have that 1(z
oa )L(z
oa ) = (o). This pole/zero cancellation
between L and K i must be made for the expression (8) to makesense.
Appendix C. Proof ofLemma 3.5
First we note that existence of Ry and Ru satisfying (15)is ensured by the spectral factorization theorem. Poles on theunit circle can be extracted before the spectral factorization and
reinserted afterwards and are thus left unchanged. Furthermore,notice that byStanding Assumptions 2.1, R has all zeros strictlyinside the unit circle. Thus the right-hand sides of the equationsin(15)are non-zero on the unit circle and thus Ry and Ru can betaken to have all their zeros strictly inside the unit circle.
Consider now a zero ofBo. Lemma 3.4 gives that the asymptoticvariance when only b and f are used is no greater than theasymptotic variance when allparameters areused. Considering thecase when only band fare used will hence give a lower bound.
Forthis case, it is straightforward to verifythat (ej)(ej)=(ej)(ej) where
G oSoRuHo
1
BoTnb
1
FoTnf
Tand that
(zob ) Ho(z
ob )Bo(z
ob )
Go(zob )So(z
ob )Ru(z
ob )
=nb (zob )
0
.
Thus, if we consider a zero ofBwe can, analogously to the proof ofTheorem 3.1,write(B.1)as
AsVarzo =o|L(zob )|2(zob ), 1(zob ), (C.1)with L(z)= zHo(z)/(Go(z)So(z)Ru(z)). Since So(zob )= 1 we canremove So from L. Now
(), 1(z) is the reproducingkernel for the rowspace of . However, since Boand Foare coprimethis is thesame subspace as the one generated by GoSoRu
BoFoHonb+nf, see
Ninness and Hjalmarsson(2004). This proves(13)for zeros ofBo.The proofs for zeros of the other polynomials are analogous.
Appendix D. Proof ofCorollary 3.6
That we have equality in (13)when only b and f are used,i.e. for FIR and OE models, follows directly from the proof ofLemma 3.5.
We now turn to the BoxJenkins case when(16)holds. Then
=
GoHo
So
BoRnb (q)
KGoSo
Bo
o nb (q)
0 1Co
o nc(q)
01
Doo nd (q)
GoHo
SoFo
Rnf (q) KGoSo
Foo nf (q)
.
8/11/2019 Artigo erro varincia identificao.pdf
13/14
8/11/2019 Artigo erro varincia identificao.pdf
14/14
J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2525
For thelower bound (25) we invoke Lemma 3.5 and then notice
that Lemma A.6 is applicable toKlb(zo,zo) under the conditions in
the theorem. Evaluation ofLlbfor AousingTable 2now gives(25).
Appendix I. Proof ofTheorem 6.1
The proof parallels the proof ofTheorem 5.1,see Appendix H
but in the proof of(30) L is evaluated for Bo instead. Similarly, inthe proof of(31),Llb is evaluated forBousingTable 2.
That equality holds in(31)for OE and FIR models as well as forBoxJenkins models subject to (16) follows from Corollary 3.6. Thisconcludes the proof.
References
Agero, J., & Goodwin, G. (2007). Choosing between open- and closed-loopexperiments in linear system identification. IEEE Transactions on AutomaticControl,52(8), 14751480.
Bombois, X., Gevers, M., & Scorletti, G. (2005). Open-loop versus closed-loopidentification of BoxJenkins models: A new variance analysis. In 44th IEEEconference on decision and control and the European control conference 2005 (pp.31173122).
Bombois, X., Scorletti, G., Gevers, M., Van den Hof, P., & Hildebrand, R. (2006). Leastcostly identification experiment for control.Automatica,42(10), 16511662.
Forssell, U., & Ljung, L. (1999). Closed-loop identification revisited. Automatica,35(7), 12151241.
Gevers, M., Bazanella, A. S., Bombois, X., & Miskovic, L. (2009). Identification andthe information matrix: How to get just sufficiently rich? IEEE Transactions onAutomatic Control(in press). To appear December 2009.
Gevers, M., Ljung, L., & Van den Hof, P. (2001). Asymptotic variance expressions forclosed-loop identification.Automatica,37(5), 781786.
Grenander, U., & Szeg, G. (1958).Toeplitz forms and their applications. Berkley, CA:University of California Press.
Heuberger, P., Wahlberg, B., & den Hof, P. V. (Eds.). (2005). Modeling andidentification with rational orthogonal basis functions. Springer Verlag.
Hjalmarsson, H., & Lindqvist, K. (2002). Identification of performance limitations incontrol using ARX-models. InProceedings of The 15th IFAC world congress.
Hjalmarsson,H., & Mrtensson,J. (2007). A geometricapproach to varianceanalysisin system identification: Theory and nonlinear systems. In Proceedings of the46th IEEE conference on decision and control (pp. 50925097).
Hjalmarsson, H., & Ninness, B. (2006). Least-squares estimation of a class offrequency functions: A finite sample variance expression. Automatica, 42(4),
589600.Lindqvist, K. (2001). On experiment design in identification of smooth linear
systems.Licentiate thesis. Royal Institute of Technology, Stockholm, Sweden.Ljung, L. (1985). Asymptotic variance expressions for identified BlackBox transfer
function models.IEEE Transactions Automatic Control,30(9), 834844.Ljung, L. (1999).System identification: Theory for the user(second ed.). Prentice Hall.Mrtensson, J., & Hjalmarsson, H. (2003). Identification of performance limitations
in control using general SISO-models. In Proceedings of the 13th IFAC symposiumon system identification(pp. 519524).
Mrtensson, J., & Hjalmarsson, H. (2005a). Closed loop identification of unstablepoles and non-minimum phase zeros. In Proceedings of the 16th IFAC worldcongress on automatic control.
Mrtensson, J., & Hjalmarsson, H. (2005b). Exact quantification of the variance ofestimatedzeros.In Proceedingsof the 44thIEEE conferenceon decision and controland European control conference (pp. 42754280).
Mrtensson,J., & Hjalmarsson,H. (2007). A geometricapproach to varianceanalysisin system identification: Linear time-invariant systems. In Proceedings of the46th IEEE conference on decision and control (pp. 42694274).
Mrtensson, J., Jansson,H., & Hjalmarsson,H. (2005).Input design for identificationof zeros. InProceedings of the 16th IFAC world congress on automatic control.
Ninness,B., & Gustafsson,F. (1997).A unifyingconstructionof orthonormalbasesforsystem identification.IEEE Transactions on Automatic Control,42(4), 515521.
Ninness, B., & Hjalmarsson, H. (2004). Variance error quantifications that are exactfor finite model order. IEEE Transactions on AutomaticControl, 49(8), 12751291.
Ninness, B., & Hjalmarsson, H. (2005a). Analysis of the variability of jointinputoutput estimation methods.Automatica,41(7), 11231132.
Ninness,B., & Hjalmarsson,H. (2005b).On thefrequency domain accuracyof closed-loop estimates.Automatica,41(7), 11091122.
Ninness, B., Hjalmarsson, H., & Gustafsson, F. (1998). Generalized Fourier andToeplitz results for rational orthonormal bases. SIAM Journal on Control andOptimization,37(2), 429460.
Ninness, B., Hjalmarsson, H., & Gustafsson, F. (1999a). Asymptotic varianceexpressions for output error model structures. In14th IFAC world congress, Vol.H(pp. 367372).
Ninness, B., Hjalmarsson, H., & Gustafsson, F. (1999b). The fundamental roleof general orthonormal bases in system identification. IEEE Transactions onAutomatic Control,44(7), 13841406.
Oppenheim, A., & Schafer, R. (1989). Discrete-time signal processing. EnglewoodCliffs, NJ: Prentice-Hall.
Rudin, W. (1987).Real and complex analysis(third ed.). McGraw-Hill, Inc..Skogestad, S., & Postlethwaite, I. (1996). Multivariable feedback controlAnalysis and
design. Chichester: John Wiley.Sderstrm, T., & Stoica, P. (1989). Prentice Hall International series in systems and
control engineering,System identification.Vuerinckx, R., Pintelon, R., Schoukens, J., & Rolain, Y. (2001). Obtaining accurate
confidence regions for the estimated zeros and poles in system identification
problems.IEEE Transactions on Automatic Control,46(4), 656659.Xie, L., & Ljung, L. (2001). Asymptotic variance expressions for estimated frequencyfunctions.IEEE Transactions on Automatic Control,46(12), 18871899.
Xie, L., & Ljung, L. (2004). Variance expressions for spectra estimated using auto-regressions.Journal of Econometrics,118(12), 247256.
Jonas Mrtenssonreceived his M.Sc. degree in VehicleEngineering and his Ph.D. degree in Automatic Controlfrom KTH Royal Institute of Technology, Stockholm,Sweden, in 2002 and 2007 respectively. Since 2007 heis a researcher at the School of Electrical Engineering atKTH. His research interests include system identification,presently focusing on modeling and control of internalcombustion engines.
Hkan Hjalmarsson was born in 1962. He received theM.S. degree in Electrical Engineering in 1988, and theLicentiate degree and the Ph.D. degree in AutomaticControl in 1990 and 1993, respectively, all from LinkpingUniversity,Sweden.He hasheld visitingresearch positionsat California Institute of Technology, Louvain Universityand at the University of Newcastle, Australia. He hasserved as an AssociateEditor for Automatica (19962001),and IEEE Transactions on Automatic Control (20052007)and has been Guest Editor for European Journal of Controland Control Engineering Practice. He is Professor at the
School of Electrical Engineering, KTH, Stockholm, Sweden. He is Chair of the IFACTechnical Committee on Modeling, Identification and Signal Processing. In 2001 hereceived the KTH award for outstanding contribution to undergraduate education.His research interests include system identification, signal processing, control andestimation in communication networks and automated tuning of controllers.