Operations Research Letters 28 (2001) 27–34www.elsevier.com/locate/dsw

Constructing a correlated sequence ofmatrix exponentials withinvariant �rst-order properties

Kenneth MitchellComputer Science Telecommunications Program, University of Missouri – Kansas City, 5100 Rockhill Road, 57OC STB,

Kansas City, MO 64110, USA

Abstract

In this paper, we demonstrate a method for developing analytic Markovian tra�c sources in which the correlation structurecan be arbitrarily constructed leaving the marginals invariant. We construct a simple model based on empirical data sets andshow the e�ects of changing the autocorrelation on the behavior of network tra�c. c© 2001 Elsevier Science B.V. All rightsreserved.

Keywords: Autocorrelations; Matrix exponential Distributions; Arrival processes; Communications networks; Markovmodeling; Queueing theory

1. Introduction

Models of networks using the assumption of re-newal arrival tra�c frequently result in bu�er designswhich may signi�cantly under-predict losses whencompared with actual tra�c conditions [8,20]. Short-and long-range dependencies have been observed inLAN tra�c [8] as well as TCP applications overwide-area networks [6]. VBR video tra�c in particularhas been shown to have strong correlations [1,19,21].These correlations cause much higher bu�er over- ow than that predicted in most analytic models. Thedegree of such impact and how the autocorrelationstructure can be incorporated into performance mod-els of these systems is not clear. Attempts to quantifythe impact of autocorrelated network tra�c have been

E-mail address: mitchell@cstp.umkc.edu (K. Mitchell).

made di�cult due to the fact that most analytic tra�csource models have not been able to isolate the e�ectsof the autocorrelations from the marginals.To shed some light on this important problem, we

examined one of the Bellcore Ethernet traces producedby Leland and Wilson et al. In 1989 and 1990, Lelandand Wilson [11] collected traces of several hundredmillion Ethernet packets from Bellcore’s MorristownResearch and Training Center. These packet tracesformed the basis of the conclusion in several papers[9,10,25–27], that renewal models of network tra�cmay poorly re ect the actual tra�c in communicationsnetworks.Eramilli [2] performed simulation experiments in

which the inter-arrival times of the Bellcore datawere shu�ed, allowing him to isolate the e�ects ofthe near-range and long-range autocorrelations onbu�er occupancy. Shu�ing the data in di�erent ways

0167-6377/01/$ - see front matter c© 2001 Elsevier Science B.V. All rights reserved.PII: S 0167 -6377(00)00062 -6

28 K. Mitchell / Operations Research Letters 28 (2001) 27–34

destroys either the near-range or the long-range auto-correlations. The actual inter-arrival times, and there-fore the marginal distribution of the empirical trace,is not a�ected. These experiments, along with ourown [14], demonstrate that it is the ordering of theinter-arrival times which a�ects the autocorrelationstructure. With this in mind, we set out to developanalytical source models in which we can controlthe autocorrelations analytically while leaving themarginals invariant. The resulting process is similarto the discrete autoregressive (DAR) model discussedin Heyman et al. [3] and the transform expandsample (TES) process discussed in Jagerman [4]. Thismodel, however, is a continuous time analytic modeland lends itself readily to Markov chain analytictechniques [15].In Section 2, we brie y describe our techniques for

creating correlated sequences of matrix exponentialprocesses which leave the marginals invariant. In Sec-tion 3, we apply these techniques to construct a simpletwo state Markovian model of Ethernet tra�c. Section4 shows some results of our studies and we concludethe paper in Section 5.

2. Constructing correlated processes

2.1. Matrix exponential distribution

A matrix exponential (ME) distribution is de�ned[12] as a probability distribution with representation(p;B; e), i.e.,

F(t) = 1− p exp(−Bt)e′; t¿0; (1)

where p is the starting operator for the arrival process,B is the process rate operator, and e′ is a summing op-erator. The minus sign in exp(−Bt) represents a nat-ural generalization from a scalar exponential processto a vector process. The order of the representation isindicated by the dimension of the matrix B, and thedegree of the distribution F(t) is the minimal order ofall its representations. The nth moment of the matrixexponential distribution is given by

E[X n] = n!pV ne′; (2)

where V is the inverse of B.The class of matrix exponential distributions is iden-

tical to the class of distributions that possess a rationalLaplace–Stieltjes transform. Matrix exponential distri-

butions are dense in the set of all distributions, so anydistribution can be approximated arbitrarily close witha matrix exponential distribution. The class of ma-trix exponential distributions is closed under mixtures,convolutions, and order statistics of such distributions.A matrix exponential distribution may have an un-derlying probabilistic interpretation (i.e. a phase-typedistribution), but is not necessarily limited to such aninterpretation. Phase-type distributions form a propersubset of matrix exponential distributions.A representation is not unique. If F(t) has repre-

sentation (p;B; e) then (pX ;X−1BX ;X−1e) is alsoa representation for any non-singular matrix X . Theonly limitations on (p;B; e) stem from the requirementthat F(t) must form a distribution function. Conse-quently, (pe′) is a real number between zero and oneand the eigenvalues of B must either be positive realsor must come in complex conjugate pairs with a pos-itive real part. As there are no prescribed structuralor domain restrictions on the components (p;B; e),one can choose a physically based representation (i.e.phase-type), or an algorithmic representation.

2.2. Matrix exponential sequence

A sequence of ME random variables T1; T2; : : : suchthat the joint probability density over any �nite se-quence of consecutive inter-event times is given by

fT1 ;T2 ;:::;Tn(t1; t2; : : : ; tn)

=�(0)exp(−Bt1)L exp(−Bt2)L · · · exp(−Btn)Le′;(3)

where �(t) is a vector representing the internal state ofthe process at time t. The matrix L is the (non-zero)event transition rate operator which generates an eventand starts the next interval in the appropriate startingstate. All matrices and vectors are assumed to be of�nite dimension m. This process is interpreted as astream of events (in this case, successive departuresfrom a matrix exponential process) occurring at timest1; t1+t2; t1+t2+t3; : : :, and inter-event times t1; t2; : : : :Note that if the rank of L is one, then the sequence(T1; T2; : : : ; Tn) is a renewal process. An in�nitesimalcharacterization of the vector process �(t) analogousto the Poisson process is given by Lee et al. [7].

• If at time t the system is in state vector �(t) andno events occur between t and t + h, then at time

K. Mitchell / Operations Research Letters 28 (2001) 27–34 29

t + h the system will be in state vector �(t + h) =�(t)(I − Bh+ o(h))

• If at time t the system is in state vector �(t) anda single event occurs between t and t + h, thenat time t + h the system will be in state vector�(t + h) = �(t)(Lh+ o(h))

This allows the closed-form representation of thestate of the system to be �(t) = �(0) exp(Q∗t), fort ¿ 0, where Q∗ = L− B.There are several ways in which such a sequence

can be constructed. For illustrative purposes we usea “phase-type” viewpoint and review the constructionof a Markov arrival process (MAP). We start with acontinuous time Markov chain with rate matrix Q∗

and invariant steady-state vector, �, with

�Q∗ = 0; �e′ = 1: (4)

Depending on the application, certain transitions rep-resent events of interest. By putting these in a matrixL, we can write Q∗ as

Q∗ =Q∗ − L+ L; (5)

and de�ne

B = L−Q∗: (6)

Internal transition rates are represented by the progressrate operator B and represent transitions that are notof special interest. The vector describing the stateof the process immediately after a special event de-pends on the state immediately before the event andthe transition which created the special event, thus�(t+) = �(t)L. The process just described is a MAPwith D0 = B and D1 = L, see Neuts [17,18].A correlated sequence of matrix exponentials is

more general than a MAP in that we allow imaginaryphases to exist (i.e. Coxian distributions). The ele-ments of the vectors �(t), and matrices B and L maynot have a probabilistic interpretation, but vector val-ued functions on the state �(t) do. The matrices B andL which we construct are of the form which can beused directly in vector balance equations for Marko-vian queueing models. For details see [12,16,24].The joint density function of the �rst n-successiveinter-arrival intervals is a sequence of matrix expo-nentials with representation given in Eq. (3).We assume that the process is in equilibrium, the

steady state being represented by the vector �. Inequilibrium, the starting vector �(0) is the steady

state at embedded arrival points and is denoted by thevector p,

p=�L�Le′

: (7)

The vector � is also the residual vector for the processtill the next event and is related to p by the expression

�= �pV ; (8)

where V is the inverse of B, and � is the mean arrivalrate for the process.The covariance of the sequence of matrix exponen-

tials (3) is given as

cov[X0; Xk ] = pV(VL)kVe′ − (pVe′)2: (9)

See Lipsky [13] for a complete derivation or [16,18]for a short review. If the process is assumed to becovariance stationary, the autocorrelation is obtainedby dividing cov[X0; Xk ] by the variance

var[X0] = 2pV2e′ − (pVe′)2: (10)

De�ne the matrix Y as

Y = VL (11)

and note that the autocorrelations decay matrix geo-metrically as pVY kVe′. Note in particular that L onlyappears in the autocorrelation equation and not in themoment equation (2). A renewal process can be ex-pressed as an uncorrelated sequence of matrix expo-nentials (3) by making the following assignments:

Lr:=Be′p; (12)

�(0):=p: (13)

The rank of the Lr operator is 1 as a result of theproduct of the column vector e′ and the row vector p.

2.3. Constructing a correlated process withinvariant marginals

Generally, we want to construct a correlated se-quence with varying autocorrelations from knownpoint processes. Let a given matrix exponential distri-bution induce a renewal sequence (p;B; e) which canbe represented as a sequence of uncorrelated matrixexponentials using (12) and (13). An autocorrelatedprocess can then be constructed by introducing an L (or Y ) such that p and B remain invariant as follows.

30 K. Mitchell / Operations Research Letters 28 (2001) 27–34

First, �nd the residual vector (�) for renewal ver-sion of the process using Eq. (8). Using Eqs. (6) and(12) constructQ∗=Be′p−B. Now consider the equa-tion �Q∗ = 0. For a given Q∗ its solution is unique,but for a given �, the solution is not, and each Q∗

which satis�es the equation results in a process witha di�erent autocorrelation structure as it leaves �, p,and B invariant.We are interested in developing parsimonious mod-

els, so we start with a single parameter , leading to ageometrically decaying covariance. De�ne the opera-tor L by

L = (1− )Q∗ + B; −16 ¡ 1: (14)

By use of Eqs. (6) and (12), L can be expressed inthe following form:

L = (1− )(Be′p− B) + B: (15)

From Eq. (11), this L induces a transition matrix Y ,

Y = (1− )e′p+ I ; −16 ¡ 1: (16)

We now show that the marginal density of the kth ran-dom variable fTk (tk) at equilibrium remains invariantfor any ; −16 ¡ 1. The expression for themarginaldensity of the kth random variable in a sequence ofmatrix exponentials from Eq. (3) is

fTk (tk) =∫ ∞

0

∫ ∞

0· · ·

∫ ∞

0fT1 ;T2 ;:::;Tn(t1; t2; : : : ; tn)

×(dt1; : : : ; dtk−1; dtk+1; : : : ; dtn)= p(VL )k−1 exp(−Btk)L (VL )n−ke′= p(Y )k−1 exp(−Btk)L (Y )n−ke′: (17)

Using the de�nition of Y (16), the following expres-sions are obtained:

pY = p((1− )e′p+ I)= (1− )p+ p= p; −16 ¡ 1; (18)

Y e′ = ((1− )e′p+ I)e′= (1− )e′ + e′ = e′; −16 ¡ 1: (19)

Substituting expressions (18) and (19) into (17) yields

fTk (tk) = p exp(−Btk)L e′ = p exp(−Btk)BY e′= p exp(−Btk)Be′: (20)

Therefore the marginal distribution is invariant withrespect to .

For any value of −16 ¡ 1, the vectors �, p, andthe operator B remain invariant. We call the measureof persistence of the process, and as → 1, the slopeof the decay of the autocorrelations decreases. Also, as → 1, the time spent in each state increases althoughthe proportion of time spent in each state remains thesame. When = 0, the process is a renewal processand Y =Yr= e′p. For ¡ 0, the lag autocorrelationsalternate in sign.The renewal operator Yr = e′p has a single eigen-

value of pe′ = 1 and an eigenvalue of 0 with mul-tiplicity m − 1, where m is the order of representa-tion. The correlated operator Y has a single eigen-value of 1 and eigenvalue of multiplicity m−1. ThusY k = (1− k)e′p+ kI .The expression for the lag-k autocorrelation be-

comes

corr[X0; Xk ] = c k ; (21)

where the constant c is

c = ( pV 2e′ − ( pVe′)2)=(2pV 2e′ − ( pVe′)2): (22)

The allowable values for are bound such that−16 ¡ 1 and |c |61. In this paper we will assume06 ¡ 1.

3. Constructing a two-state correlated process fromempirical data

We want to determine the relative impact of au-tocorrelation on a simple G=M=1=k performancemodel. To do this, we analyzed such a system asconstructed above and performed our own shu�ingexperiments on the available Bellcore data. By ran-domizing blocks of size n, we can study the relativeimpact of short-range versus long-range autocor-relation, e.g., if n = 100, we eliminate short-rangeautocorrelation up to lag-100 but long-range autocor-relation is not a�ected. As n increases, we eliminatelonger-range autocorrelation until n = ∞ and theentire trace is in random order. For this latter case,the trace should have no autocorrelation structureand should behave as a pure renewal interarrivalstream — which is exactly what happens. Our resultscorroborate those of Erramilli [2] who performedsimilar shu�ing experiments on Bellcore Ethernettraces.

K. Mitchell / Operations Research Letters 28 (2001) 27–34 31

Fig. 1. Inter-arrival time autocorrelation comparison.

Fig. 1 shows the packet inter-arrival time autocor-relation in one of the Bellcore Ethernet traces. Packetinter-arrival time autocorrelation is plotted to lag-1000together with that of a renewal stream constructedfrom the marginals. Clearly, an analytical model witha renewal arrival stream will not be an e�ective surro-gate for a system with autocorrelated arrivals such asthose found in the Bellcore Ethernet packet trace, evenif the renewal stream matches the �rst-order statisticsof the Bellcore trace.The slow rate of decay in autocorrelations is sig-

ni�cant. Erramilli [2] found the rate of decay inlong-range-dependent tra�c to be a power of the lagtime. This characteristic cannot be duplicated in anyMarkovian source model, but only approximated tosome extent.Recent papers however have shown that long-range

dependency in tra�c does not always dominate per-formance. Jelenkovic [5] has observed what are calledweakly stable systems in which arrival processes havestates which generate tra�c with mean inter-arrivaltimes that are shorter than the mean system servicetime. In this scenario the short-range dependenciestend to dominate. Models which use heavy tailed dis-tributions to modulate sources exhibiting long-rangedependency have also found these tendencies in whathave been termed as blow up points [22,23]. Based onthese observations, it may be possible to limit the den-sity of the tail (and thus the order of representation) inthe modulating process. In this example we show thatreasonable approximations can be constructed whichperform well over a wide range of conditions.

Table 1Autocorrelation lags and corresponding values

Model lag

1 0.5800310 0.88248100 0.969491000 0.99675

Using standard techniques, we computed aphase-type hyper-exponential (H2) distribution takenfrom samples of the Bellcore trace. For our example,we use a matrix exponential representation (p;B; e).Thus,

p= [0:2854710695; 0:7145289305]; (23)

B =[0:3747871858; 0

0; 2:998297486

]; (24)

e′ =[11

]: (25)

The renewal process represented by Eqs. (23)–(25)above has �= 1:0 and C2x = 3:223610. Now we wantto make a matrix exponential point process with aspeci�c autocorrelation structure. Using Eq. (16), theexpression for Y is

Y = (1− )e′p+ I =[a+ b ; b− b a− b ; b+ a

]; (26)

where a= 0:2854710695 and b= 0:7145289305.By adjusting the values for , we obtain matrix ex-

ponential point processes with varying autocorrelationstructures. Since the Bellcore trace has been shown toexhibit long-range dependencies, the lag autocorrela-tion of our model will always drop below that of theempirical data at some point. Table 1 shows the val-ues for these varying autocorrelation lags at which ourmodel intersects and falls below that of the empiricaltrace.

4. Numerical results

Fig. 2 compares packet losses from our analyticmodels for varying values with losses resultingfrom a simulation model using the Bellcore Ethernet

32 K. Mitchell / Operations Research Letters 28 (2001) 27–34

Fig. 2. Log. plot of packet loss probabilities for a two-state approximation vs. Bellcore data into a bu�er of size 100 with an exponential server.

Fig. 3. Log. plot of packet loss probabilities for a two-state approximation vs. Bellcore data into a bu�er with an exponential server, �=0:7.

K. Mitchell / Operations Research Letters 28 (2001) 27–34 33

trace. This �gure shows that as the autocorrelation lagincreases, losses from the G=M=1=k model increase.Losses from the renewal inter-arrival stream remainvery low. It is important to note that when the analyti-cal model matches the lag-1 autocorrelation ( =0:58),losses are still very low. For this particular tra�c itappears that matching the slope of the decay of theautocorrelations farther out in the tail are more im-portant than matching the lag-1 autocorrelation. Also,the range of losses experienced in the analytic modelis the result of altering the autocorrelation structureonly. The marginal distribution remains exactly thesame.Fig. 3 shows packet loss probabilities from a sim-

ulation model of the G=M=1=k system and utilization�=0:7 with our analytic model for bu�er sizes in therange of 50–150. This �gure shows that a value of can be chosen such that the performance behavior canbe closely modeled in the range of bu�er sizes understudy. In fact, the decay of the autocorrelations in theanalytical model can be made as arbitrarily close to 0as possible, making the modeling of the behavior oflarge bu�ers possible. For a value of = 0:99, packetloss probabilities are overestimated, but the slope ofthe decay in packet loss probabilities most closelymatches that of the Bellcore Ethernet trace. Of course,as with any Markovian model which tries to approx-imate long-range-dependent behavior, the slope willeventually diverge from that of actual empirical data.

5. Conclusion

To be e�ective, models must incorporate thesecond-order statistics (autocorrelations) of the sys-tems they purport to emulate. We have developedtechniques to allow correlated source models to beconstructed which allow the autocorrelation structureto be modi�ed while leaving the marginals invari-ant. The tra�c we have studied has been shown tohave long-range dependencies and it is evident thatmatching the lag-1 autocorrelation is not enough. Itappears that matching near-term autocorrelation isnot as important as matching the slope of the decayof the autocorrelation structure at some point fartherout in the tail.

References

[1] J. Beran, R. Sherman, M.S. Taqqu, W. Willinger, Long-rangedependence in variable bit rate video tra�c, IEEE Trans.Comm. 43 (2=3=4) (1995) 1566–1579.

[2] A. Erramilli, O. Narayan, W. Willinger, Experimentalqueueing analysis with long-range dependent packet tra�c,IEEE=ACM Trans. Networking 4 (2) (1996) 209–223.

[3] D.P. Heyman, A. Tabatabai, T.V. Lakshman, H. Heeke,Modeling teleconference tra�c from VBR video coders,Conference Record of the International Conference onCommunications (ICC), 1994, pp. 1744–1748.

[4] D.L. Jagerman, B. Melamed, The transform and correlationstructure of TES processes, Part 1: General theory, StochasticModels 8 (2) (1992) 193–219.

[5] P. Jelenkovic, A. Lazar, N. Semret, The e�ect of multipletime scales and subexponentiality in mpeg video streams onqueueing behavior, IEEE J. Selected Areas Comm. 15 (6)(1997) 1052–1071.

[6] S. Klivansky, A. Mukherjee, C. Song, On long-rangedependence in NSFNET tra�c, Technical ReportGIT-CC-94-61, Georgia Institute of Technology, Atlanta, GA,1994.

[7] Y. Lee, A. van de Liefvoort, V. Wallace, Modeling correlatedtra�c with a generalized IPP, Performance Evaluation 40(2000) 99–114.

[8] W.E. Leland, M.S. Taqqu, W. Willinger, D.W. Wilson, On theself-similar nature of Ethernet tra�c, SIGCOMM Symposiumon Communications Architectures and Protocols, 1993, pp.183–193.

[9] W.E. Leland, M.S. Taqqu, W. Willinger, D.V. Wilson, Onthe self-similar nature of Ethernet tra�c (extended version),IEEE=ACM Trans. Networking 2 (1) (1994) 1–15.

[10] W.E. Leland, M.S. Taqqu, W. Willinger, D.V. Wilson, Onthe self-similar nature of Ethernet tra�c, Proceedings of 14thInternational Teletra�c Congress, 1994, pp. 319–328.

[11] W.E. Leland, D.V. Wilson, High time-resolution measurementand analysis of LAN tra�c: implications for LANinterconnection, Proceedings of IEEE Infocom, 1991, pp.1360–1366.

[12] L. Lipsky, Queueing Theory: A Linear Algebraic Approach,MacMillan, New York, 1992.

[13] L. Lipsky, P. Fiorini, W. Hsin, A. van de Liefvoort,Auto-correlation of lag-k for customers departing fromsemi-Markov processes, Technical Report TUM-19506,Technical University Munich, 1995.

[14] K. Mitchell, Analytic models with correlated arrivals, ThirdIFIP Workshop on Tra�c Management and Synthesis ofATM Networks, London, UK, 1999.

[15] K. Mitchell, A. van de Liefvoort, Transient analysis of cellloss probabilities in bu�ers with correlated arrivals, IFIP ATM2000, 2000, pp. 32=1–32=12.

[16] K. Mitchell, A. van de Liefvoort, J. Place, Correlationproperties of the token leaky bucket departure process,Comput. Comm. 21 (1998) 1010–1019.

[17] M. Neuts, Models based on the Markovian arrival process,IEICE Trans. E75-B (1992) 1255–1265.

34 K. Mitchell / Operations Research Letters 28 (2001) 27–34

[18] M. Neuts, Algorithmic Probability: A Collection of Problems(Stochastic Modeling), Chapman & Hall, London, 1995.

[19] P. Pancha, M.E. Zarki, Variable bit rate video transmission,IEEE Comm. 32 (1994) 54–66.

[20] B.E. Partuwo, R. Disney, The e�ects of correlated arrivalson queues, IIE Trans. 25 (1993) 105–110.

[21] D. Reininger, B. Melamed, D. Raychaudhuri, Congestioncontrol in an ATM network, 14th International Teletra�cCongress, 1994, pp. 295–306.

[22] H.-P. Schwefel, L. Lipsky, Bu�er size issues in thepresence of self-similar tra�c, Third IFIP Workshop on theManagement and Design of ATM Networks, 1999, pp. 11=1–11=12.

[23] H.-P. Schwefel, L. Lipsky, Performance results for analyticmodels of tra�c in telecommunications systems, based onmultiple ON-OFF sources with self-similar behavior, 16thInternational Teletra�c Congress, 1999.

[24] A. van de Liefvoort, The moment problem for continuousdistributions, Technical Report CM-1990-02, University ofMissouri-Kansas City, 1990.

[25] W. Willinger, Tra�c modeling for high-speed networks:Theory versus practice, in: F. Kelly, R. Williams (Eds.),Stochastic Networks, The IMA Volumes in Mathematics andits Applications, Springer, Berlin, 1995.

[26] W. Willinger, M.S. Taqqu, W.E. Leland, D.V. Wilson,Self-similarity in high-speed packet tra�c: analysis andmodeling of Ethernet tra�c measurements, Statist. Sci. 10(1) (1995) 67–85.

[27] W. Willinger, D.V. Wilson, W.E. Leland, M.S. Taqqu,On tra�c measurements that defy tra�c models (and viceversa): self-similar tra�c modeling for high-speed networks,Connextion 8 (11) (1994) 14–24.