A W RICHARDS

Modern ergodic theoryThere is much more to the mathematical study ofGibbs ensembles than the question of whether or nottime averages and ensemble averages are equal

Joel L Lebowitz and Oliver Penrose

The founding fathers of statistical me-chanics, Boltzmann, Maxwell, Gibbsand Einstein, invented the concept ofensembles to describe equilibrium andnonequilibrium macroscopic systems.In trying to justify the use of ensem-bles, and to determine whether theensembles evolved as expected fromnonequilibrium to equilibrium, theyintroduced further concepts such as"ergodicity" and "coarse graining."The use of these concepts raisedmathematical problems that theycould not solve, but like the goodphysicists they were they assumed thateverything was or could be made allright mathematically and went on withthe physics.

Their mathematical worries, how-ever, became the seeds from whichgrew a whole beautiful subject called"ergodic theory." Here we describesome recent (and some not so recent)developments that partially solve someof the problems that worried theFounding Fathers; these results appearnot to be widely known in the physicscommunity, and we hope to remedythe situation. (For a more thoroughreview of the subject, see reference 1.)

Joel L. Lebowitz is chairman of the physicsdepartment at the Belfer Graduate Schoolof Science, Yeshiva University, in NewYork. Oliver Penrose, a member of themathematics faculty at the Open Universi-ty, Walton, UK, worked on this article whileon leave at the Applied Physics Depart-ment, Stanford.

At the top of the page we see Boltzmann,Maxwell, Gibbs and Einstein, the "FoundingFathers" of statistical mechanics.

Although results are not yet wellenough developed to answer all thequestions in this area that are of inter-est to physicists today, such as the der-ivation of kinetic equations or the gen-eral problem of irreversibility, they domake a start.

Ergodic theory is concerned with thetime evolution of Gibbs ensembles. Ithas revealed that there is more to thesubject than the simple question ofwhether a dynamical system is ergodic(which means, roughly, whether thesystem, if left to itself for long enough,will pass close to nearly all the dynam-ical states compatible with conserva-tion of energy). Instead there is a hi-erarchy of properties that a dynamicalsystem may have, each property imply-ing the preceding one, and of whichergodicity is only the first (see figure1). The next one, called "mixing,"provides a clear-cut formulation of thetype of irreversible behavior that peo-ple try to obtain by introducingcoarse-grained ensembles. At the topof our hierarchy is a condition (theBernoulli condition) ensuring that in acertain sense the system, thoughdeterministic, may appear to behave asrandomly as the numbers produced bya roulette wheel.

Some of the mathematical results weshall be discussing have establishedthe positions of some model physicalsystems in this hierarchy. Of particu-lar interest to physicists is the work ofYa. Sinai2 on the hard-sphere system,which shows that this system is bothergodic and mixing. We shall also dis-cuss some work by A. N. Kolmogorov,

V. I. Arnold and J. Moser on systemsof coupled anharmonic oscillators,which shows that, contrary to a com-mon assumption, these systems maynot even reach the "ergodic" rung onthe ladder. (G. H. Walker and J. Fordhave described this work for physi-cists.3)

All the physical systems we shalldiscuss obey classical mechanics, havea finite number of degrees of freedomand are confined to a finite region ofphysical space. The reason for our re-striction to classical mechanics is thata finite quantum system can never ex-hibit any of the properties higher thansimple ergodicity in our hierarchy (al-though, of course, a large quantum sys-tem may approximate closely the be-havior characterized by these con-cepts). This is because the spectrumof a finite quantum system is necessar-ily discrete, whereas for a finite classi-cal system the spectrum (of the Liou-ville operator) can be continuous. Thereason for our restriction to finite sys-tems is that very little is known aboutthe ergodic properties of infinite sys-tems. This lack of knowledge is re-grettable because only for an infinitesystem (by which term we mean thelimit of a finite system as its size be-comes infinite) can one expect to findstrictly irreversible behavior in quan-tum mechanics. Moreover, the dis-tinction between microscopic and mac-roscopic observables, which appears es-sential to any complete theory of irrev-ersibility and kinetic equations, canonly be formulated precisely for infi-nite systems. Much research remains

PHYSICS TODAY/FEBRUARY 1973 2 3

History of ergodic theory

The "ergodic hypothesis" was intro-duced by Boltzmann in 1871. To quoteMaxwell ". . .(it) is that the system, if leftto itself in its actual state of motion,will, sooner or later, pass through everyphase which is consistent with theequation of energy." In our notation"phase" means dynamical state and theoriginal ergodic hypothesis means thatif y and x are any two points on theenergy surface Sf:, then y = 0,(x) forsome t. The ergodic hypothesis thusstated was proven to be false, wheneverS£ has dimensionality greater than one.by A Rosenthal and M. Plancherel in1913. S. G. Brush gives a nice ac-count of the early work on this problem(see reference 5).

The definition of an ergodic systemgiven in equation 1 (page 25) can beshown to be equivalent to what is some-times called the "quasi-ergodic" hy-pothesis, which replaces "every phase"in Maxwell's statement by "every regionFt on SE of finite area," with the further

qualification that this is true for "almostall" dynamical states. Indeed as wepoint out in this article, the fraction oftime that the system will spend in Ft isequal, for an ergodic system, to thefraction of the area of Sg that is occu-pied by Ft.

It was shown by G. D. Birkhoff in1927 that ergodicity is equivalent to theenergy surface being "metrically transi-tive." Stated precisely this means thata system is ergodic on S if and only ifall the regions Ft on S left invariant bythe time evolution, (fit(Ft) = Ft, eitherhave zero area or have an area equal tothe area of S.

The difficult part of Birkhoff's Theo-rem is to show that f*(x), which in-volves taking the time average over infi-nite times, actually exists for almost allx when f(x) is an integrable function. Itis then relatively easy to show that l*(x)is time invariant; that is, f*\(j>t(x)] =f*(x), and that ergodicity is equivalent•to S being metrically intransitive.

to be done on the ergodic theory of in-finite systems, both classical andquantum, but we can be sure that theconcepts to be discussed here will playan important part in it.

Surfaces and ensemble densitiesBefore we go on to discuss the new

results, we review some mathematicaldefinitions.4 If our dynamical systemhas n degrees of freedom, we can thinkof its possible dynamical states geome-trically, as points in a 2n-dimensionalspace (phase space), with n positioncoordinates and n momentum coordi-nates. If the energy of the system is E,then its dynamical state x [= (c/i . . .qn, pi . . . pn)\ must lie on the energysurface H(x) = E, where H is theHamiltonian function. We donote theenergy surface, which is (2n — ^-di-mensional, by SE or just i' and assumethat S is smooth and of finite extent;for example in the case of a system ofharmonic oscillators, for which theHamiltonian is a quadratic form, theenergy surfaces are (2n — l)-dimen-sional ellipsoids.

The time evolution of the systemcauses x to move in phase space, butsince we are assuming our system to beconservative the point x always stayson the energy surface. If the system isin some state x at some time fo then itsstate at any other time t0 + t isuniquely determined by x and t (only).Let us call the new state </>,(*). Thisdefines a transformation tt>, from 6'onto itself. There is one such functionfor each value of t.

We shall want to integrate dynami-cal functions (that is, functions of the

dynamical state) over £>'. When doingthis it is convenient not to measure.(2n— 1 (-dimensional "areas" on the sur-face .S in the usual way but to weightthe areas in such a way that the natu-ral motion of the system on S carriesany region R (after any time t) into aregion <t>t(R) of the same area. Thiscan be accomplished by defining theweighted area of a small surface ele-ment near x, dx, to be such that dxdEis the correct Euclidean 2n-dimension-al volume element of a "pill box" withbase dx and height dE.

By a Gibbs ensemble we mean an in-finitely large hypothetical collection ofsystems, all having the same Hamilto-nian but not necessarily the same dy-namical state. We shall only considerensembles whose systems all have thesame energy, so that their dynamicalstates are distributed in some way oversome energy surface .S. It may happenthat this distribution can be describedby an ensemble density; by this wemean a real-valued function p on Ssuch that the fraction of members ofthe ensemble whose dynamical stateslie in any region R on the surface S is

p( x )dx

with dx the weighted area definedabove. The simplest ensemble densityon 6' is given by

fix) = C (all x in S)

where C is a constant, which can bedetermined from the normalizationcondition )'s(x)dx - 1. This is calledthe microcanonical ensemble on 5'.

The systems constituting the ensem-

ble evolve with time, so that the en-semble density will depend on time.The rule connecting the ensembledensities p, describing the same en-semble at different times t is Liou-ville's theorem, which can be written

p,(x) = p<L<t>-,(x)] ' (a lUallx inS)

where po(x) is the ensemble density attime zero. For example, Liouville'stheorem shows that the density of themicrocanonical ensemble does notchange with time: If

po(*) = C

for all x in S, then Liouville's theoremgives, for any t,

p,(x) = C

for all x in S.

The ergodic problem

The principal success of ensembletheory has been in its application toequilibrium. To calculate the equilib-rium value of any dynamical functionwe average it over a suitable ensemble.The same ensemble also enables us toestimate the magnitude of the fluctua-tions of our dynamical function. Toensure that the calculated averages areindependent of time, we use an invar-iant ensemble; that is, one for whichthe fraction of members of the ensem-ble in every region R on the energy sur-face 6' is independent of time. We al-ready know one invariant ensemble:the microcanonical, whose ensembledensity is uniform on S. Before wecan use it confidently to calculateequilibrium values, however, we wouldlike to be sure that this is the only in-variant ensemble: If other invariantensembles exist then, in principle, theycould do just as well for the calculationof equilibrium properties, and wewould have to choose which to use in aparticular situation.

There are two questions to settle:the first is whether there are any in-variant ensembles on S that do nothave an ensemble density. In generalthere are; for example in the case of ahard-sphere system in a box one couldhave an invariant ensemble whereevery particle moves on the samestraight line being reflected at eachend from a perfectly smooth parallelwall (see figure 2).

The obviously exceptional characterof this motion is reflected mathemati-cally in the fact that this ensemble,though invariant, is confined to a re-gion of zero "area" on S and thereforehas no ensemble density. To set upsuch a motion would presumably bephysically impossible because theslightest inaccuracy would rapidly de-stroy the perfect alignment. It istherefore natural to rule out such ex-ceptional ensembles by adopting theprinciple that any ensemble corre-

24 PHYSICS TODAY/FEBRUARY 1973

sponding to a physically realizable sit-uation must have an ensemble density.

There remains the second part of thequestion: Are there any invariantensembles on S that do have a densitybut differ from the microcanonical en-semble? This is equivalent to the er-godic problem5 in which one comparesthe time averages of a dynamical func-tion /,

i rT

f*(x) = lim ~ f(4>,(x))dt

with its microcanonical ensemble aver-age

(/) = J f(x)dxlj dx

A system is said to be eri>odic on itsenergy surface S if time averages are ingeneral equal to ensemble averages;that is, if for every integrable function/we have

f*(x) (1)

for almost all points x on S. "Almostall" means that if M is the set ofpoints .v for which equation 1 is false,we have J A/C/.V = 0. The answer to oursecond question is given by a theorem,6

which we shall not prove: the micro-canonical ensemble density is the onlyinvariant ensemble—that is, the onlyone satisfying p[4>,(x)\ = p(x) for all xin 5'—if and only if the system is ergo-dic on S.

The physical importance of ergodi-city is that it can be used to justify theuse of the microcanonical ensemble forcalculating equilibrium values andfluctuations. Suppose / is some mac-roscopic observable and the system isstarted at time zero from a dynamicalstate x, for which f(x) has a value thatmay be very far from its equilibriumvalue. As time proceeds, we expectthe current value of/, which is f[4>,(x)],to approach and mostly stay very closeto an equilibrium value with only veryrare large fluctuations away from thisvalue. This equilibrium value shouldtherefore be equal to the time average

I /* because the initial period duringwhich equilibrium is established con-tributes only negligibly to the formuladefining f*(x). The theorem tells usthat this equilibrium value is almostalways equal to (/), the average of / inthe microcanonical ensemble, provided

. the system is ergodic.To justify the use of the microcanon-

ical ensemble in calculating equilibri-um fluctuations we proceed in a simi-lar way. For some observable event A(such as the event that the percentage

: of gas molecules in one half of a con-.tainer exceeds 51%) let R be the regionin phase space consisting of all phasepoints compatible with the event A;that is the event A is observed if andonly if the phase point is in R at timef. If the system is observed over a long

period of time, the fraction of timeduring which event A will be observedis given by the time average g*(xa),where XQ is the initial dynamical stateand g is defined by

(1 if .v is in R,gix) = ioifnot

The ergodic theorem tells us that foralmost all initial dynamical states thisfraction of time is equal to the ensem-ble average of g, which is

(g) -Lii dx

This is just the "probability" of theevent A as calculated in the micro-canonical ensemble on S.

Another way of defining ergodicity isto say that any integrable invariantfunction is constant almost every-where. That is to say, if / is an inte-grable function satisfying the conditionthat

/[</;, (.v)] = f(x)

for all x in S, then there is a constant Csuch that f(x) equals c for almost all x:In other words, the set M of points x

for which fix) does not equal c satisfies.1\'.\jdx = 0. This has the physical in-terpretation that for a Hamiltoniansystem ergodic on S every integrableconstant of the motion is constant onS. Furthermore if ergodicity holds oneach St: then there are no integrableconstants of the motion other thanfunctions of the energy E. Indeed, ifthere were other constants of the mo-tion (for example angular momentumif the Hamiltonian had an axis of sym-metry) we could construct invariantdensities that were not microcanonicalby taking p{x) to be a function of oneof these other constants of the motion,and so the system would clearly benonergodic. When such additionalconstants of the motion exist theymust be taken into account in the sta-tistical mechanics and thermodynam-ics of the system; the standard meth-ods, based on the microcanonical en-semble, must then be generalized forthese systems.

Some special systemsWe now illustrate the idea of ergodi-

city by considering some specific me-chanical systems. The simplest ofthese is the harmonic oscillator, whose

Baker's transformation

Geodesic flow on spaceof constant negative

curvature

Hard-sphere system

Simple harmonicoscillator

Bernoulli system

r

Ksystem

i

Mixing system

<

Ergodic system

Equivalent to roulettewheel

Approach to equilibrium

Use of microcanonicalensemble

Hierarchy of systems. Arrows denote implication: Every mixing system is ergodic. everyBernoulli system equivalent to a roulette wheel, and so on The middle column lists thethree systems discussed in this article—with the "strongest" at the top—as well as theK-system (after A. N. Kolmogorov), which is not discussed. At the left are examples ofthe systems and at the right physical interpretations or implications Figure 1

PHYSICS TODAY FEBRUARY 1973 25

Hamiltonian (in some suitable units) is

H(q, p) = n w (p2 + <?")

where u is the angular frequency. Thetransformation </>, for this system is arotation through angle u;, in the (q,p)plane. The trajectories, which herecoincide with the energy surfaces SE,are circles of radius (2E)1 2. (The sur-face element dx is here proportional tothe ordinary length of an arc segment.)To be invariant under the transforma-tion <j>, an ensemble density on S mustbe unaffected by rotations and is there-fore a constant. It follows, then, thatthe only invariant density is the micro-canonical density and so the simpleharmonic oscillator is ergodic.

Almost as simple is the multipleharmonic oscillator (physically, say, anideal crystal), that is, a system withtwo or more degrees of freedom whosepotential energy is a quadratic form inthe position coordinates. Unlike thesimple harmonic oscillator it cannot beergodic, because it has constants of themotion (the energies of the individualnormal modes) that are not constanton the energy surfaces (the surfaces ofconstant total energy).

It used to be thought that this lackof ergodicity was an accident and thatany small anharmonicity (such aswould inevitably be present in a realsystem) must make the system ergodicby permitting transfer of energy fromone mode to another. In 1954, how-ever, Kolmogorov announced resultsthat contradicted this belief.7 In 1955,Enrico Fermi, J. Pasta and S. W. Ulam8

carried out a computer simulation ofsuch a system. Initially, they excitedone mode only, and instead of theequipartition of the energy between allmodes predicted by the microcanonicalensemble they found that most of itappeared to remain concentrated in afew modes; this indicated that anhar-monic oscillator systems may not beergodic.

The lack of ergodicity was provedrigorously by Kolmogorov, Arnold andMoser.3 They found that if thefrequencies of the unperturbed oscilla-tors are not "rationally connected"(that is, if no rational linear combina-tion of them is zero) then, in general,adding to the Hamiltonian an anhar-monic perturbation sufficiently smallcompared to the total energy does notmake the system ergodic. The unper-turbed trajectories (possible paths ofthe phase point) all lie on n-dimen-sional surfaces in S (which has 2n - 1dimensions) called "invariant tori,"and "KAM" prove that under a weakperturbation most trajectories continueto lie within smooth n-dimensional tori,so that the perturbed system is alsononergodic. The trajectories that donot lie on the new invariant tori are, on

Ensemble with no ensemble density. Hardspheres move up and down the dotted line,which meets the perfectly smooth hardwalls at right angles. Collisions betweenparticles and collisions with the walls do notdeflect the particles from the line if they areperfectly aligned at the start. An ensembleof such systems has no ensemble densitybecause it is concentrated on a region onthe energy surface with zero area (just asthe area of a line or of a line segment in aplane is zero). Figure 2

the other hand, very erratic indeed andmay fill some (2n - 1 (-dimensionalregion densely.9 One consequence ofthis very complicated behavior is thateven though the system is not ergodicthe motion can no longer be decom-posed into independent normal modes.

Similar results probably hold alsofor rationally connected frequencies(which cannot be treated rigorously,although they are of more physical in-terest); thus Michael H§non and CarlHeiles10 carried out computer calcula-tions for the Hamiltonian

H =

whose unperturbed frequencies u;i = 1,u>2 = 1, are rationally connected sincel««i - l»o>2 - 0. They found thatthe energy surfaces with E equal to1/12, 1/8 and probably also 1/6 are notergodic (see figure 3). As seen in thediagrams the fraction of the area corre-sponding to smooth curves (which areresponsible for the nonergodic behav-ior) decreases as the energy increases.

For a general system of anharmonicoscillators, such as a real crystal, weexpect similar behavior, with the frac-tion of SE corresponding to nonergodicbehavior decreasing as E increases, andprobably disappearing altogether atsome critical energy, above which thesystem would be ergodic and perhapsalso show the stronger properties thatwe shall discuss.9 At present very lit-tle is known about the magnitude ofthis critical energy in a system withmany degrees of freedom.

In the case of gases, the situation is

Smooth hard walls

Hard spheres

somewhat different. If there were nointeraction at all between the mole-cules then the energy of each moleculewould be an invariant of the motion, sothat the system (an ideal gas) would benonergodic. The KAM theorem wouldtherefore lead us to expect nonergodi-city to persist in the event of a suffi-ciently weak interaction between theparticles. The actual interactions,however, are not weak because twomolecules very close together repeleach other strongly; consequently thetheorem does not apply. A simplemodel of this type is the hard-spheregas enclosed in a cube with perfectlyreflecting walls or periodic boundaryconditions. Sinai has outlined a proofthat this system is ergodic; he has pub-lished a detailed proof, based on thesame ideas, for a particle moving in aperiodic box containing any number ofrigid convex elastic scatterers. Weshall refer again to this important re-sult.

Mixing

We have seen how to formulate acondition to ensure that the equilibri-um properties of a dynamical systemare determined by its energy alone andcan be calculated from a microcanoni-cal ensemble. This ergodicity condi-tion does not, however, ensure that ifwe start from a nonequilibrium ensem-ble the expectation values of dynami-cal functions will approach their equi-librium values as time proceeds. Anillustrative example is given by theharmonic oscillator. For the har-monic-oscillator system, Liouville'stheorem shows that the ensemble den-sity repeats itself regularly at time in-tervals of 2w/w, therefore all ensemble

26 PHYSICS TODAY FEBRUARY 1973

-0 .4

0

0.4

i i i > i 1 1 1 1

-0.4 -

-0.4 0.4

averages also have this periodicity, andso cannot irreversibly approach theirequilibrium values.

To ensure that our ensembles ap-proach equilibrium in the way wewould expect of ensembles composed ofreal systems, we need a stronger condi-tion than ergodicity. To see what is

- required, let us start at t - 0 withsome ensemble density poU) on S,

. which is supposed to represent the ini-tial nonequilibrium state. At a latertime t the ensemble density is, byLiouville's theorem, po[<j>-t(x)]. Theexpectation value of any dynamicalvariable / at time t is therefore

X f ( x ) p t l [ ( p , ( x ) ] d x (2)

As t becomes large, we would like thisintegral to approach the equilibriumvalue of/, which is (for an ergodic sys-tem) fsfWdx/fsdx. A sufficientcondition for this is that the systemshould satisfy the condition called mix-ing,11 which is that for every pair offunctions / and g whose squares are in-tegrable on .S we require

Hm f(x)g(<t>.,(x))dx =

f f(x)dx f g(x)

I dx

The special case where g is po showsthat integral 2 will approach the equi-librium value of / for large f when thesystem is mixing. Another way oflooking at this condition is that it re-quires every equilibrium time-depen-dent correlation function such as

' (f(x)g[<t>,(x)\) to approach a limit (/) (g)

- 0 . 4

0

0.4

.1' • • • .• -; • . . A

v ' : ' - . ; - . . / - . - : r v C ? " . ' • ' / . . , y

• X ̂ [ 7i i i i \ \ y i i i i i

0.4 -0.4 0 0.4

Nonergodicity of an anharmonic oscillator system with rationally connected frequencies.The Hamiltonian for this system is given on page 26 Michael Henon and Carl Heiles (seereference 10) did computer calculations for this system and found that energy surfacesSE with E equal to 1/12, 1/8 and probably 1/6 are not ergodic. The planes shown hereare intersections of the surface q, equal to zero with SE for E equal to 1/12 (a). 1/8 (b) and1/6 (c), and the points are the intersections of a trajectory with this plane. When the tra-jectory lies on a smooth two-dimensiona4 invariant torus, the intersection points form asmooth curve, but the intersections of an "erratic" trajectory (one that does not lie on asmooth curve) are more or less random. Note that the fraction of area corresponding tosmooth curves (which are responsible for the nonergodic behavior) decreases with in-creasing energy. Figure 3

as t approaches ± ». The conditioncan be shown to be equivalent to thefollowing requirement: if Q and R arearbitrary regions in S, and an ensembleis initially distributed uniformly overQ, then the fraction of members of theensemble with phase points in R attime t will approach a limit as t ap-proaches °°; this limit equals the frac-tion of the area of S occupied by R.

Mixing is a stronger condition thanergodicity: it can easily be shown toimply ergodicity but is not implied byit, as we have seen in the case of a sim-ple harmonic oscillator.

The mathematical definition of mix-ing was introduced by John von Neu-mann12 in 1932 and developed by E.Hopf,13 but goes back to J. WillardGibbs,14 who discusses it by means of ananalogy: ". . . the effect of stirring anincompressible liquid . . . Let us sup-pose the liquid to contain a certainamount of coloring matter which doesnot affect its hydrodynamic properties. . . [and] that the coloring matter is dis-tributed with variable density. If we givethe liquid any motion whatever . . . thedensity of the coloring matter at anysame point of the liquid will be un-changed . . . Yet . . . stirring tends tobring a liquid to a state of uniformmixture."

Gibbs saw clearly that the ensembledensity p, of a mixing system does notapproach its limit in the usual "fine-grained" or "pointwise" sense of p,U)approaching a limit as t"-• °° for eachfixed x. Rather, it is a "coarse-grained" or "weak" limit, in which theaverage of p,(x) over a region R in Sapproaches a limit as f • « for eachfixed R. (A similar distinction appliesin defining the entropy. The fine-

grained entropy -k Jp,(.v) log pt(x)dx,where k is Boltzmann's constant, re-tains its initial value forever, but thecoarse-grained entropy -k fpt*(x) [logpt*(x)]dx, where pt*(x) is a coarse-grained ensemble density obtained byaveraging p,(x) over cells in phasespace, does change for a nonequili-brium ensemble, and approaches as itslimit the equilibrium entropy valuek log f sdx.)

It is sometimes argued that one can-not have a proper approach to equilibri-um for any finite mechanical systembecause of a theorem, due t<> Point-are.15

that every such system eventuallyreturns arbitrarily close to its initialstate. (The time involved, however,will be enormously large for a macro-scopic system. Boltzmann. for exam-ple, estimated a typical Poincare peri-od for 100cm3 of gas to be enormouslylong compared to 10 raised to the power10 raised to the power 10 years.) Here,however, we are considering ensembles,not individual systems, and the mixingcondition guarantees that the ensembledensity eventually becomes indistin-guishable from the microcanonicaldensity and remains so forever after.It is true that individual systems inthe ensemble will return to their initialdynamical states, as required by Poin-care's theorem, but this will happen atdifferent times for different systems, sothat at any particular time only a verysmall fraction of the systems in the en-semble are close to their initial dynam-ical states.

The reason for the irrelevance ofPoincare recurrences in mixing systemsis that the motion of the phase point isvery unstable. Dynamical states thatstart very close to each other in phase

PHYSICS TODAY FEBRUARY 1973 27

A familiar example of "mixing." According to V. I. Arnold and A Avez, the two liquids arerum (twenty percent) and cola (eighty percent), with the result of the mixing process knownas a "Cuba libre." (See reference 1 for details of the process.) Figure 4

space become widely separated as timeprogresses, so that the recurrence timedepends extremely sensitively on theinitial conditions of the motion. (Theimportance of this instability in statis-tical mechanics was first recognized byN. S. Krylov, a Russian physicist whodied in his twenties in 1947; a recentpaper16 gives a theorem that shows thisstatement about instability to be a rig-orous consequence of mixing.) Thistype of instability appears to be char-acteristic of real physical systems, andleads to one sort of irreversibility:even if we could reverse the velocitiesof every particle in a real system thathas been evolving towards equilibrium,the system would not necessarily re-turn or even come close to its initialdynamical state with the velocities re-versed because the unavoidable smallexternal perturbations would be mag-nified. This instability is noticeable inmolecular-dynamics calculations withhard-sphere systems: if we numerical-ly integrate the equations of motionfrom time 0 to t and then try to recover

the initial state by integrating back-wards from time t to time 0, we obtaininstead a completely new state. Thisis because the numerical integration isunstable to small rounding-off errorsmade during the computation, whichplay the same role as external pertur-bations in a real system.

Only a few physical systems havebeen proven so far to be mixing. Themost important is the hard-sphere gas,mentioned above. Sinai's proof thatthis system is ergodic2 also gives thestronger result that it is mixing.Roughly, Sinai's method of provingmixing is to show that the hard-spheresystem is unstable in the sense dis-cussed above. Physically this instabil-ity comes from the fact that a slightchange in direction of motion of anyparticle is magnified at each collisionwith the convex surface of another par-ticle. The full proof for the simplestcase of a "single ' particle movingamong fixed convex scatterers occupiessome 70 pages of difficult mathemat-ics. (G. Gallavotti has provided an el-

ementary introduction to some of theideas.17)

Bernoulli systems

For lack of space we shall not discussthe next concept in our sequence, thatof the "K-system" (after Kolmogorov).For this and related topics such as theKolmogorov-Sinai "entropy" the read-er is referred to reference 1. Insteadwe jump to the last concept in our hi-erarchy, that of a Bernoulli system.By a Bernoulli dynamical system wemean one that is, in a sense we shalldefine, equivalent to an infinite se-quence of spins of a suitably designedroulette wheel.

To make this equivalence more pre-cise, we define a finite partition of theenergy surface S of our dynamical systemas any finite collection of n nonover-lapping regions /?o Rn - I which to-gether cover the whole of .S1. Supposethat some device can be made that willdetermine which of these regions thephase point is in at any time, but givesno information whatever about whichpart of the region it is in. That is,every time we use this measuring de-vice we obtain an outcome that is apositive integer—the label of the regionthe phase point of the system is in atthat time.

Suppose we use the device repeated-ly at intervals of, say, one second. Itsoutcome will be a sequence of positiveintegers from the set |0,. . .,n - lj,which can be extended indefinitely. Ingeneral, we would expect these integersto be correlated; that is, the micro-canonical probability for each new ob-servation depends on what has beenobserved before (as in a Markov pro-cess, for example). This correlationcomes about because the dynamicalstates of the system at different timesare deterministically related, throughthe equations of motion.

The astonishing fact is, however,that for a certain class of dynamical

The baker's transformation recalls the kneading of a piece of dough. We first squash thesquare to half its original height and twice its original width, and then cut the resultingrectangle in half and move the right half of the rectangle above the left. Figure 5

28 PHYSICS TODAY/FEBRUARY 1973

systems it is possible to choose the re-gions i?o>- • Mn-i in such a way thatthe observations made at differenttimes are completely uncorrelated, justlike the numbers shown at differenttimes by a roulette wheel. At thesame time, the regions so chosen giveenough information to discriminate be-tween dynamical states: if two sys-tems have different dynamical statesat some time, then the observationsmade on them cannot yield identicalresults for the observations at everytime. When such regions can be cho-sen, we call the system a Bernoulli sys-tem.

Recently D. Ornstein and B. Weiss18

showed that the class of Bernoulli sys-tems includes a type of dynamical sys-tem (the geodesic flows on a space ofconstant negative curvature) whose er-godic properties are very similar tothose of the hard-sphere system stud-ied by Sinai, and it appears likely thatthe hard-sphere system too is a Ber-noulli system.

As an illustration of a simple Ber-noulli system, consider a system whosephase space is the square 0 < p < 1,0 < q < 1 shown in figure 5, and whose(non Hamiltonian) law of motion isgiven by a mapping known as the bak-er's transformation1^ because it recallsthe kneading of a piece of dough. Ifthe phase point is (p,q) at time i, thenat time r + 1 it is at the point obtainedby squashing the square to a (1/2 X 2)rectangle, then cutting and reassem-bling to form a new square as shown inthe diagram. The formula for thistransformation is

0( > _ f(2p,g/2)ifO<p<l/2l(2p-l,(g /2 + l/2)if

digit after the binary point from p andattaches it to q, so that

4>(O.pip2...,O.q]q2...) =

1/2 < p < 2p.

If p and q are written in binary nota-tion (1/8 in binary notation is0.00100.. ., 1/4 is 0.01000, and so on),the transformation removes the first

Definition of the regions RQ and R-, used toshow that the baker's transformation is aBernoulli system. Figure 6

where the p, and qt take on the values0 and 1. This transformation is invert-ible, and from it we can define $_i asthe inverse of 0 and 0 ± , as the t th ite-ration of 0±i (Only integer values ofthe time are used here, rather than allreal values as in our discussion of dy-namics earlier in this article, but we donot regard this distinction as impor-tant.) Moreover, the transformationpreserves geometrical area, and so theanalog of the microcanonical distribu-tion is just a uniform density.

To see how this completelydeterministic system can at the sametime behave like a roulette wheel, wetake the regions RQ and i?i to be thetwo rectangles 0 < p < 1/2, 1/2 < p <1 shown in figure 6. Suppose thephase point at time 0 is

(p,q) = (O.pip2.. .,0.0x92—)If pi is zero, the system at time 0 is inRo', if Pi is one, the system at time 0 isin/?i. At time 1 phase point is

(0.p2pi...,0.plqlq2...)

and so we observe the phase point inregion RP2- At time 2 it is in Rp3 andso on. Likewise, at time - 1 it is in

References1. V. I. Arnold, A. Avez, Ergodic Problems

of Statistical Mechanics, Benjamin,New York (1968); J. Ford, "The Transi-tion from Analytic Dynamics to Statis-tical Mechanics," to be published inAdvances in Chemical Physics; J. L. Le-bowitz, "Hamiltonian Flows and Rigor-ous Results in Non-equilibrium Statisti-cal Mechanics," in Proceedings of theIUPAP Conference on Statistical Me-chanics, University of Chicago Press(1972); O. Lanford, "Ergodic Theoryand Approach to Equilibrium for Finiteand Infinite Systems," to be publishedin Proceedings of the 100th Anniversaryof the Boltzmann Equation, Vienna,1972; Ya. G. Sinai, "Ergodic Theory,"to be published in Proceedings of the100th Anniversary of the BoltzmannEquation; A. S. Wightman, in Statisti-cal Mechanics at the Turn of the Dec-ade (E. G. D. Cohen, ed.), Marcel Dek-ker, New York (1971).

2. Ya. G. Sinai, Sov. Math.—Dokl. 4, 1818(1963); "Ergodicity of Boltzmann'sEquations" in Statistical Mechanics,Foundations and Applications (T. A.Bak, ed.), Benjamin, New York (1967);Russian Mathematical Reviews 25, 137(1970).

3. G. H. Walker, J. Ford, Phys. Rev. 188,416(1969).

4. O. Penrose, Foundations of StatisticalMechanics, Pergamon, Oxford (1970)chapter III.

5. D. ter Haar, Elements of Statistical Me-chanics, Holt, Rinehart and Winston,New York (1964), Appendix 1, para-

Rqi, at time - 2 in RQ2, and so on.Each observation is determined by adifferent digit in the binary representa-tion of the number pair (p,q).

Since the analog of the microcanoni-cal ensemble for this system has a uni-form density in the square it is not dif-ficult to see that the microcanonicalprobability of each of these digits inthe binary expression for (p,q) is 1/2,and is uncorrelated with all the otherdigits. The observations made at dif-ferent times t (= integer) are thereforeuncorrelated, and so the baker's trans-formation model is a Bernoulli system.

Possibly one should not read toomuch physical meaning into this typeof result, for with a more complicateddynamical system the regions /?o, #i,/?2,- • ., Rn would probably be exceed-ingly complicated sets in phase space,but from a "philosophical" point ofview it is very interesting to see howthe same dynamical system can showperfect determinism on the microscop-ic level and at the same time perfectrandomness on a "macroscopic" level.

It is the interplay of these two appar-ently incompatible levels of descriptionthat give the foundations of statisticalmechanics their fascination.

* * *The work described here was supported bvUS Air Force OS R. Grant AFOSR-7-2430and the US Army Research Office. Durham.

graph 5; O. Penrose, Foundations of Sta-tistical Mechanics, Pergamon, Oxford(1970), page 40; S. G. Brush, TransportTheory and Statistical Physics 1(4), 287(1971).

6. V. I. Arnold, A. Avez, ref. 1, page 16.7. A. N. Kolmogorov, "Address to the 1954

International Congress of Mathemati-cians," [translated in R. Abrahams,Foundations of Mechanics, Benjamin,New York (1967), Appendix D).

8. Enrico Fermi: Collected Papers, Vol-ume II, University of Chicago Press,Chicago (1965) page 978.

9. G. H. Lunsford, J. Ford, J. Math. Phys.13,700(1972).

10. M. Henon, C. Heiles, Astron. J. 69, 73(1964).

11. V.I. Arnold, A. Avez, ref. 1, page 25.12. J. von Neumann, Annals of Math. 33,

587(1932).13. E. Hopf. J. Math, and Phys. 13, 51

(1934); Ergoden Theone, Springer, Ber-lin (1937).

14. J. W. Gibbs, Elementary Principles inStatistical Mechanics, Dover, New York(1960) page 144.

15. H. Poincare, Acta Math. 13, 1 (1890);Oeuvres 7, 262; J. W. Gibbs, reference14, page 139.

16. T. Erber, B. Schweizer, A. Sklar, to bepublished in Comm. Math. Phys.

17. D. Ornstein, B. Weiss, to be publishedin Israel J. Mathematics.

18. P. R. Halmos, Lectures on ErgodicTheory, Mathematical Soc. of Japan(1965) page 9; V. I. Arnold, A. Avez, ref-erence 1, page 8. •

PHYSICS TODAY/FEBRUARY 1973 2 9