CHAOTIC SIGNALS AND PHYSICAL SYSTEMS

Henry D. I. Abarbanel

Department of Physics and

Scripps Institution of Oceanography University of California, San Diego

La Jolla, CA 93093-0402

ABSTRACT

The methods for establishing a phase space or state space for a physical system from measurements of a scalar time series are reviewed, and the implications for the analysis of signals of the geometry of strange attractors is discussed. Classifying the physical systems by invariants on the at- tractor is seen to be the analogue of classifying linear sys- tems by the resonant response frequencies. The statisti- cal properties of deterministic systems are used to perform this classifying. Methods for separating a chaotic signal of interest from contamination such as high dimensional 'noise' are discussed. An example of a signal from a non- linear circuit whose equations of evolution are unknown is discussed in detail and all the items mentioned are illus- trated.

1. INTRODUCTION

We identify two goals of signal processing into the tasks of (1) signal analysis - the identification of signal sources, and (2) signal synthesis - the use of sources for communication, structural inquiry, target location, etc.

This talk is about signal analysis for signals with contin- uous broadband spectra arising from a chaotic system; the system is deterministic. My point of view is that of a physi- cist: observe a chaotic signal and make a physically sensible model of the source in terms of differential equations (ordi- nary or partial). I want the parameters in my model to be established by the data, and I want to know what aspects of future system behavior I can use to check the predictive capabilities of my model.

There is nothing new in all this, in a grand sense. This is what physicists have done since Newton! The new de- velopment comes when the system is nonlinear and chaotic. This means that every orbit of the system is intrinsically unstable and because of inevitable experimental noise or measurement error or because of imprecision in the state of the system or round off in computation, one may not expect to compare detailed orbits of one's model with observations - past or present. (Contrast this with the remarkable pre- cision with which we predict the Newtonian orbits of the elements of our solar system!) One must establish appro- priate statistical qualities (in these deterministic systems) to measure, model, and predict.

2. GENERAL IDEAS Chaos requires multidimensional state space. We assume we have N-dimensional dynamics of a variable

measured at a sampling rate l/rs. The signal may be gen- erated by a differential equation so t is continuous, but in reality r, > 0, so we prefer the stroboscopic view char- acterized by discrete time n = 1 , 2 , . . . N O . The discrete time dynamics is a map in N-space which takes vectors x = [XI, 22, . . . X N ] intovectors x -+ f(x) by a nonlinear function f(x). The orbit arises from V(l) by

V(n + 1) = f(V(n)).

In making observations we typically

0 do not know f(x)

0 do not observe V(n).

We usually observe a single & variable, call it p ( t o + nr,) = p(n), which may be one of the components of V(n) or some nonlinear function p(n) = g(V(n)) of the dynamical variables V(n). Our task is to establish multidimensional dynamics from scalar observations.

One more point about the dynamics: if we have an or- bit V(n), small perturbations to this orbit generically move exponentially rapidly away from V(n)-indeed this is char- acteristic of chaos. If the perturbation Iw(n)l << IV(n)l, the dynamics of ~ ( n ) is nearly linear

V(n + 1) + M Df(V(n)). w(.).

w ( n + 1) = f(V(n)) + W(.))

w ( n + 1) = f(V(n) + w(n)) - V(n + 1)

So w(L + 1) arises from w(1) by iterating the linear map Df(x) and evaluating it along the orbit

w(L+ 1) = Df(V(L))*..Df(V(l)).w(l) = DfL'(V). ~ ( 1 ) .

for large L DfL(V) has eigenvalues exoL a = 1,2, ... N ; A1 > A2 > . + + > AN. If any of the A, > 0, the system is unstable to any perturbation. Chaotic systems

IV-113

0-7803-0532-9192 $3.00 0 1992 IEEE

are characterized by one or more A,, called Lyapunov ex- ponents, being positive.

A, > 0 does not mean the system moves infinitely far from V(n). The systems we deal with are driven and damped, and the orbits V(n) reside in a compact region of N-space. All orbits are folded back into this compact region by the dynamics. The set of points in N-space V(1), V(2), . . . V(No) visited by the orbit is called an at- tractor. If any A, > 0, it is a strange attractor.

If we observe only p(n), then we are seeing the geometri- cal figure of the attractor projected from some larger space down to one dimension. Unfolding the geometric struc- ture which is the attractor by choosing d ceordinates built out of the information in the observed p(n) is known as reconstructing phase space or state space. The most con- venient technique for this was suggested by Ruelle [l]. Use as co-ordinates the timedelayed versions of p : p ( n ) , p ( n + T),p(n+2T), . . . where Tis an integer, so in time the delay is TT,. In this way d-dimensional vectors

are formed. This d-dimensional Euclidian space with co- ordinate values p(n).--p(n + T(d - 1)) is our state space. We do not know, nor do we need to know for most pur- poses, the relation of the vectors y(l),y(2), . . . in the re- constructed state space to the vectors V(1), V(2). . . in the physical state space.

3. TIME DELAY; EMBEDDING DIMENSION Choosing T and d knowing only observed p ( n ) will allow us to have a state space. A dimension d large enough to unfold the observations which are projected onto the mea- surement axis p(n) provides and embedding of the attractor in Euclidian space Rd.

The idea of having co-ordinates p(n), . . .p(n + T(d - 1)) is that they are somewhat independent of each other and allow the multidimensional dynamics to be exposed. T = 0 is a clear bad idea; all the co-ordinates are the same. In a chaotic system, large T is also a bad idea since p ( n ) and p ( n + T) are unrelated in any practical sense- the system is chaotic; XI > 0; errors have grown by a factor eXIT.

Chaotic systems are information sources in the strict sense of Shannon. They have a positive entropy. It makes sense, then, to suggest that T be chosen to the average mu- tual information between measurements p(n) at time n and measurements p ( n + T) at time n + T be small, but not zero. The idea [2] of choosing T as the first minimum of

both make sense and is the natural, nonlinear, information theoretic analog of choosing the first zero of the autocorre- lation function as a correlation time and useful time delay.

With this T, what d shall we use? It seems clear that if d is large enough, we will have enough co-ordinate axes to unfold anything. There is a theorem which says that if the dimension of the attractor (geometric objects defined by the points y(n)) is d A , then the integer d just greater than 2dA is sufficient, but not n e c e s s a r m ] . To choose a minimum

d , technically called the embedding dimension, we turn to the idea of the unfolding. When we view the dA dimensional orbit in too low a dimension, the orbit will cross itself in a false way because of the projection to a low dimension. As we increase d we eliminate these false crossings until at least we remove the point crossings, and the geometry is unfolded. Orbit crossings mean that points in state space may be false neighbors due to the projection, which is ob- servational, instead of being neighbors by folding, which is dynamical.

In going from dimension d to dimension d + 1 we add the component p ( n + T d ) to the vector y(n). By comparing this addition for two vectors in dimension d which appear to be neighbors, we can see if they remain neighbors in dimension d + 1. If so, fine; they are true neighbors. If not, alas; they are false neighbors, and the attractor is not unfolded. Continuing until there are no false neighbors (guaranteed when d > 2dA) or the occurrence is infrequent enough, d is established [5].

Choosing d as small as possible is mathematically of no interest-any d > 2dA and any T works in principle. In practice, d larger than neededrequires exponentially much unnecessary computation and allows noise (high dimen- sional chaos not yet unfolded) to muck up any further cal- culations we perform.

4. INVARIANTS

Now we have a space: y(n)cRd. We need quantities in this space independent of y(1) - and thus the specific, observed orbit. For this we need a way to average over state space or over orbits, and ergodicity will ensure the equality of these averages. For this we use the natural or invariant density on the attractor [I].

The integral of p(x) over any volume of state space counts the fraction of points in that volume. It is called invariant since for any g(x) the average

is also

when M ..-* 00. This follows immediately from y(j + 1) =

An important example of g(x) is the fraction of points on the orbit within a distance r of x(O(z) = 0, z < 0; O(z) =

F b (i) 1.

1 7 2 2 0)

M 1

n(r ,x ) = O(r - Jx - y(k)l). k=l

We return to n(r , x) momentarily. A distinct kind of invariant is found in the linearized be-

havior of small perturbations, namely the DFL(y) for the

IV-114

(unknown) dynamics y(1) + y(2) -+ y(m) in reconstructed phase space. In 1968 Oseledec [6] proved that for x defining an orbit which lies in the attractor the eigenvalues of the d x d matrix

1 lim - log ([DFL(x)IT[DFL(x)]) L-oJ 2L

are finite, independent of x and are the Lyapunov expo- nents A,. This multiplicative ergodic theorem is a remark- ably useful dynamical tool. It tells us that the Lyapunov exponents are dynamical invariants which are characteristic of the nonlinear system producing them. They can be used to classify the system. They are unchanged under smooth changes of co-ordinates V -.* p ( n ) = g(V(n)). Any model of the dynamics must reproduce the A,. Happily they can be evaluated from the observed p(n) alone [7].

Another piece of important information is contained in the eigendirections of matrices DFL (x). These directions, called the stable manifolds (for Aa < 0) and the unstable manifolds (for A, > 0) are dependent on the local place on the attractor, but not dependent on the specific orbit one is looking at. This is part of the result of Oseledec's theorem [6], and this invariant property of these manifolds in quite important in methods for separating signals from the dynamical system from contamination such as 'noise'. This will be discussed in some detail by Sidorowich [8],

The A, also serve, by their definition, to quantitatively determine on the average how predictable the physical sys- tem actually is. An error 6y(l) grows to expXILra 6y(l) in time Lr, . When eA1L'a16y(l)l is of the same order as the size of the attractor, prediction is off.

Now turn back to the number density n(r,x). Attractors are inhomogeneous in state space, so it becomes interest- ing to investigate moments of this density, and these are conventionally given as

J

and when r is small, this counts the way the number of points y(j) is distributed in r. Arguments about the inho- mogeneity of the points suggests

C,(r) z T - 0

and the D, are dimensionless quantities known as the generalized dimensions of the attractor. By the previous ar- guments, they are invariants under the dynamics x + F(x), and thus independent of y(1) and characteristic of the sys- tem.

The dimensions D, are closely related to the dimension of the attractor - to be precise, Do is what we called d A earlier, but the variation with q is slow. The most familiar of the dimensions is D2 which goes by the name of the correlation dimension.

Perhaps the most interesting feature of the D, is that they are generally not integers. These fractional dimen- sions (or fractal dimensions) are characteristic of attrac- tors on which the orbits are non-periodic. They are called strange attractors.

Chaos, strange attractors, nonperiodic motion, and frac- tal dimension all go together with at least one positive Aa

and exponential sensitivity to initial conditions or pertur- bations or errors.

The dimensions Dq and the Lyapunov exponents A, are two of the numerous characteristics which are available to classify strange attractors. They have a clear physical in- terpretation and are quantities that any credible model of the dynamics must accurately reproduce.

This talk is constrained to focus on this topic, but the talks by Sidorowich [8] and Meyers to follow will elaborate on how to use the state space picture of dynamics that we have outlined.

Model building can now take, more or less, two routes: (1) a map or differential equation can be suggested by

the researcher. Say some function F(x, p ) is guessed with parameters p . These parameters can be tuned to the data. The dimensions and A, can be determined, and A 1 will de- termine the limits on being able to quantitatively predict any specific orbit of the system.

maps tuned to the way collections of points in state space evolve into other collections of phase space may, easily actually, be constructed. These local maps are of the form

(2)

6 Fa(x) = Cam4m(X)

m=l

and the &,(x) are taken from any complete set of basis functions (polynomials are a good choice). The utility of any given set of (bm(x) depends on what one wishes to do with the map. In a general sense this construction is a problem in approximation theory which involves a smooth way of interpolating in Rd given observations y(n) rRd. It is unlikely there is any general answer as to the basis to use; it must depend on one's aim.

Whatever the aim and regardless of the use of a global or local model, one must demand that the model respect the invariants coming from the data analysis just outlined.

5. CLEANING CHAOS: SEPARATING

Observed signals are always contaminated, and one must remove that contamination before evaluating any of the rel- evant invariants or making models. The job of separating the signal of interest from contamination is, in general out- line, much the same as in linear problems: find something which differs about the signal and the contamination and use this to distinguish between the signals. In the case of a chaotic signal contaminated by high dimensional processes, which might be termed 'noise', other chaotic processes, or other nonchaotic signals, the distinguishing characteristic is that the chaos has a well defined and identifiable geometric structure in phase space. We can use properties of this ge- ometric structure, such as the invariant density mentioned above, to separate two signals. If we know the dynamics

SIGNALS

IV-115

x + F(x), then we can use detailed properties of the func- tion F(x) to separate a signal governed by this dynamics from anything else.

While the methods for signal separation will be covered in other talks, especially that of Myers, there are three general classes of technique for separating signals:

0 We know the dynamics. Then we can use the p rop erties of F(x) to recursively clean the signal along the stable manifolds of F(x) ifi36fWatiditeration and silong its unstable manifolds in backward iteration [9, 10, 111.

0 We know an orbit of the system Then we can use statistical properties of the attractor to determine an effective Markov model of the dynamics and then separate the part of an additively contaminated signal from its contamination [12, 131

0 We receive a signal and know little or nothing. This ‘blind’ case is quite-uncertain. Amazingly, if the signal of interest is much smaller in amplitude than the chaotic contamination, one can, in some circumstances learn enough about the statistics of the chaos to have the signal ‘clean itself’.

Examples of each of these will be given with an emphasis on the latter two. It should be clear that one can ‘mix and match’ these methods. One could use a measured signal to make a model of the dynamics and extract local Jacobians for use in the first method above. The list here is just a mention of the main ideas used in this area.

6. ANEXAMPLE As a “worked example” of all this I present the analysis and then a ‘use’ of data from a nonlinear circuit built at the Naval Research Laboratory by Carroll and Pecora [14]. I do not show the circuit diagram (it is published) because I only received 64,000 data points by e-mail and have never seen the source r, = lo-‘ sec.

We will show the following results from this data set:

0 the irregular time waveform

0 the Fourier power spectrum

0 the average mutual information for T 0 the False Nearest Neighbor test for d

0 the fractal dimension Dz

0 the Lyapunov exponents, and

0 the Lyapunov dimension

These pieces of data analysis demonstrate how we can characterize in an invariant fashion the properties of a non- linear physical system from observations alone and make that characterization in a manner independent of the spe- cific observed orbit.

ACKNOWLEDGEMENTS This work was supported in part by a subcontract from Lockheed Sanders, Inc., under Office of Naval Research Contract NOOO14-91-C-0125 and in part by a subcontract from Lockheed Sanders, Inc., under Army Research Office Contract DAAAL03-91-C-0052.

REFERENCES [l] Eckmann, J.-P. and D. Ruelle, “Ergodic Theory of

Chaos and Strange Attractors”, Rev. Mod. Phys. 57, 617 (1985).

[2] Fraser, A. M. and Swinney, H. L., Phys. Rev., 33A, 1134 (1986); Fraser, A. M., IEEE Trans. on Info. The- ory, 35, 245 (1989); Fraser, A. M., Physica, 34D, 391 (1989).

[3] Maiik, R., in Dynamical Systems and Turbulence, War- wick 1980, eds. D. Rand and 1. S. Young, Lecture Notes in Mathematics 898, (Springer, Berlin), 230 (1981).

[4] Takens, F., in Dynamical Systems and Turbulence, Warwick 1980, eds. D. Rand and L. S. Young, Lec- ture Notes in Mathematics 898, (Springer, Berlin), 366 (1981).

[5]’Kennel, Matthew B., R.’Brown, and H. D. I. Abar- banel, “Determining Minimum Embedding Dimension using the Method of False Nearest Neighbors”, submit- ted to Phys. Rev. A (1991)

[6] Oseledec, V. I., “A Multiplicative Ergodic Theorem. Lyapunov Characteristic Numbers for Dynamical Sys- tems” Trudy Mosk. Mat. Obsc 19, 197 (1968); Moscow Math. Soc. 19, 197 (1968).

[7] Brown, R., Bryant, P. and H.D.I. Abarbanel, “Com- puting the Lyapunov Spectrum of a Dynamical System from Observed Time Series” Phys. Rev. A 43, 2787 (1991). An earlier paper with many of the main results is in Phys. Rev. Lett 65 , 1523 (1990).

[8] Sidorowich, J. J. (“Sid”), “Modeling of Chaotic Time Series for Prediction, Interpolation, and Smoothing”, ICASSP-92 Paper; This Session

[9] Hammel, S. M., “A Noise Reduction Method for Chaotic Systems”,Phys. Lett. 148A, 421 (1990).

[lo] Farmer, J. D. and J. J. (“Sid”) Sidorowich, “Optimal Shadowing and Noise Reduction”, Physica D 47, 373 (1991).

[11] Abarbanel, H. D. I., S. M. Hammel, P.-F. Marteau, and J. J. (“Sid”) Sidorowich, “Scaled Linear Cleaning of Contaminated Chaotic Orbits”, UCSD/INLS Preprint, Spring, 1992.

[12] Marteau, P.-F. and H. D. I. Abarbanel, “Noise Reduc- tion in Chaotic Time Series Using Scaled probabilistic Methods”, accepted for publication in Journal of Non- linear Science.

[13] Myers, C., B. Shin, and A. Singer, “Modeling Chaotic Systems with Hidden Markov Models”, Paper pre- sented to the Meeting CSM/Warwick, 26 August 1991.

[14] Carroll, T. L. and L. M. Pecora, “Synchronizing Chaotic Circuits”, IEEE Trans. Circuits Syst. 38, 453 (1991).

IV-116