NEURONS, DYNAMICSAND COMPUTATION

Brains have long been regarded as biological computers.But how do these collections of neurons perform computations?

John J. Hopfield

The question "How does it work?" is the motivation ofmany physicists. Condensed matter physics, chemicalphysics and nuclear physics can all be thought of asdescriptions of the relation between structure and prop-erties. The components of a biological system have func-tional properties that are particularly relevant to theoperation of the system. Thus it is especially importantin biology to understand the relation between structureand function. Such understanding can be sought at thelevel of the molecule, the cell, the organ, the organismor the social group.

The function of a nervous system is to do computa-tions. Recognizing a friend, walking and understandinga spoken sentence all involve computations. The analysisof the nervous system presented here relates the biophys-ics of nerve cells, statistical physics and dynamical sys-tems to the way a biological "machine" computes.

I use the word "compute" here only in the very fuzzysense of performing a useful task of a kind that a digitalcomputer can also perform. For example, one can pro-gram a digital machine to compare a present image witha set of images generated from a three-dimensional rep-resentation of the head of a friend, and thus in principlethe problem of recognizing a friend can be solved by acomputation. Similarly, the question of how to drive theactuators on a robot given the present posture of therobot and the desired state of dynamic balance is funda-mentally a problem in classical mechanics, which can besolved on a digital computer. While we may not knowhow to write efficient algorithms for these tasks, suchexamples do illustrate that one may usefully describe

John Hopfield is the Roscoe G. Dickinson Professor ofChemistry and Biology al the California Institute ofTechnology, in Pasadena, California.

what the nervous system does as computation. However,that one can use a digital computer to model the outcomesof experiments done on a nervous system does not ipsofacto mean that the brain is a computer, since digitalcomputers can be used to model most physical systems.

For the purposes of this article, we view a computeras an input-output device, with the input and outputsignals in the same general medium or format. Thus ina very simple digital computer, the input is a string ofbits (in time), and the output is another string of bits.The computer produces a transformation on the inputsto generate the outputs. Within this view, the brain isa computer. For example, a million axons carry electro-chemical pulses from the eyes to the brain. Similarsignaling pulses drive the muscles of the vocal tract.When we enter a room, look around and say, "Hello,Jessica," our brain is producing a very complicated trans-formation from one parallel input pulse sequence comingfrom the eyes to another parallel output pulse sequencethat results in sound waves being generated.

The idea of composition is very important in thisview of a computer. The output of one computer can beused as the input for another computer of the samegeneral type, since both signals are in the same medium.Within this view, a digital chip is a computer, and largecomputers are built as composites of smaller ones. Simi-larly, each neuron (see figure 1) is a simple computer,and the brain is a large composite computer made ofneurons.

Computers as dynamical systemsA real, physical digital computer is a dynamical systemand computes by following a path in its space of physicalstates.1 (See figure 2.) Its operation is most simplyillustrated for batch-mode computation, in which all theinputs are supplied at the start of the computation (unlike

40 PHYSICS TODAY FEBRUARY 1994 1994 American Insriture of Physics

interactive computation, in which inputs may continueto arrive during the computation). The computer has Nstorage registers, each storing a single binary digit. Abinary vector of Af bits, such as 10010110000. . . , specifiesthe logical state of the machine at a particular time. Thisbinary state changes into a new state each clock cycle.The transition map describing which state follows whichis implicitly built into the machine by its design. Thusone can describe the machine as a dynamical system thatchanges its discrete state in discrete time.

The user of the machine has no control over thedynamics, which is determined by the state transitionmap. The user's program and data and a standardinitialization procedure prescribe the starting state of themachine. The motion of the dynamical system carriesout the computation. In a batch-mode computation, theanswer is found when a stable point of the discretedynamical system—a state from which there are no tran-sitions—is reached. A particular subset of the state bits

Neurons are simple computers. Each neuron receivesinputs at synapses and computes an output that istransmitted along its axon to as many as 1000 otherneurons. The brain may be regarded as a compositecomputer made up of a network of neurons. (Adaptedfrom ref. 2.) Figure 1

(for example, the contents of a particular machine regis-ter) will then describe the desired answer.

Batch-mode analog computation can be similarly de-scribed with continuous time and state-space variables.The idea of computation as a process carried out by adynamical system in moving from an initial state to afinal state is the same as in the discrete case. In theanalog case, one can think of the motion in state spaceas describing a flow field, and computation is done bymoving with this flow from start to finish. (See figure3.) The final state is typically a point attractor—a loca-tion in the state space to which all nearby states willevolve. (Of course real "digital" machines contain onlyanalog components. The digital description is only acompact representation in fewer variables that containsthe essence of the continuous dynamics.)

One of the most important resources for intelligentbehavior is powerful associative memory, in which partialand perhaps somewhat erroneous knowledge of a memorycan nevertheless give access to the complete memory. Asystem whose dynamics in a high-dimensional state spaceis dominated by a substantial number of point attractorscan be regarded as an associative memory: The locationof a particular point attractor can be obtained from partialinformation (an inexact description of the attractor loca-tion) by merely initializing the system in accord with thepartial information and allowing the dynamics to evolvethe state to the nearest attractor. For such a system tobe useful and biological it must be possible to insertmemories (new attractors) into the system by a biologi-cally plausible algorithm. We will consider this in greaterdetail below. Error-correction codes, used with datatransmission, can also be construed as attractors.

Computations more complicated than the recovery ofmemories can also be directly formulated in terms of thefixed points of dynamical systems. For example, when

PHYSICS TODAY FEBRUARY 1994 4 1

Program and dataQ

r

1i i

1 J r

i

Answer

State of a digital computer follows a paththrough its space of discrete states. Inbatch-mode computation the path goes froman initial state representing the program andthe data to a final stable state representing theanswer. Figure 2

a problem can be posed as an optimization, its solutioncan also often be posed on a dynamical system for whichthe stable (fixed) points are minima of the desired vari-able. The location of the solution is found by followinga trajectory of motion to its end.

A simple dynamical model of neurobiologyFigure 1 depicts the anatomy of a "typical" neuron in amammalian brain.2 In gross terms, it has three regions:dendrites, a cell body and an axon. Each neuron isconnected to approximately 1000 other neurons by struc-tures called synapses. A nerve cell functions as aninput-output device. Inputs to a cell are made at syn-apses on its dendrites or on the cell body. The cellproduces outputs that drive other cells lying at synapsesat the terminals of its axon. When considering a par-ticular synapse, we call the cell producing the output thepresynaptic cell while the one receiving the input ispostsynaptic.

The interior of each cell is surrounded by a mem-

An analog computational system has acontinuous space of states. The state-spaceflow field must be focused onto paths tonegate the effects of errors but is otherwisesimilar to that of a digital system (see figure2). Figure 3

brane of high resistivity and is filled with a conductingionic solution. Ion-specific pumps transport ions such asK+ and Na+ across the membrane, maintaining an elec-trical potential difference between the inside and theoutside of the cell. A cell carries out computations bydynamic changes in the conductivity of particular speciesof ions at synapses and elsewhere in the cell membrane.

A simple model3 captures in mathematical termsmuch of the essence of what a compact nerve cell does.Figure 4 shows the voltage difference u between theinside and the outside of a simple neuron functioning ina brain. The electrical potential is generally slowlychanging, but occasionally it changes very rapidly, pro-ducing a stereotypical voltage spike of about 2 millisec-onds duration. Such a spike is produced every time thecell's interior potential rises above a threshold wthresh ofabout -50 millivolts. After the spike the voltage resetsto a lower value uresetof about -70 millivolts. This "actionpotential" spike is caused by a paroxysm of voltage-de-pendent ion flows across the neuron membrane.

Except for the action potentials, the membrane con-ductivity away from the synapses is approximately con-stant. (We will return too the synapses shortly.) Themembrane is only about 75 A thick, so there is appreciablecapacitance C between the inside and the outside of thecell. If an electrical current i(t) is injected into the cell,the interior potential (except for the action potentials)obeys

Cdt

- (u - u0)

R(1)

where R is the resistance of the cell membrane and u0

is the resting potential to which the cell would drift inthe absence of an external current. For a typical neuron,"o < "threshi s o u wiU decay to uQ when the injected currentvanishes. If i(t) is a large constant current ic, the cellpotential will change in an almost linear fashion betweenu0 and «thresh- An action potential will be generated eachtime uthresh is reached, resetting u to «reset. Neurons thatbehave in this fashion are known as "integrate and fire"neurons. The spiking shown in figure 4 is an experimentalexample of the behavior of an integrate-and-fire neuron.

The time P between the equally spaced action poten-tials is then roughly

P = C "thresh "reset(2)

For small currents the leakage current through the re-sistance R is important, and for small enough constantcurrents the leakage current will prevent the cell fromfiring at all. The black curve in figure 5 shows the firingrate IIP as a function of current ic for a realistic cell.

We will take action potentials to be delta functions,lasting a negligible time. They propagate at constantvelocity along an axon. The transmission is nonlinear,but the shapes of the pulses are actively maintained.When an action potential arrives at a synaptic terminalof an axon, the terminal releases a neurotransmitter(such as acetylcholine or glutamate), which in turn acti-vates specific ionic conductivity channels in the postsyn-aptic dendrite. For reasons including diffusion andchemical inactivation, this conductivity pulse a(t) is nota delta function but has nonzero duration. It can bemodeled as

(Tit) =se

-tt-U/rt<t0

t>t0(3)

where s is the maximum conductivity of the postsynapticmembrane in response to the action potential, and T is

42 PHYSICS TODAY FEBRUARY 1994

the time constant of the pulse. Each synapse from cellj to cell k has its own particular maximum conductivityskj. The conductivity is ion specific, and the current thatflows depends on the chemical potential difference Vllm

between the inside and the outside for that ion. Thusfor a synapse from cell j to cell k, an action potentialarriving on cell / s axon at time t0 causes a current

0 t<t0t>t0

(4)

to flow into cell k. The parameter Skj = Vionsfc/- can haveeither sign, depending on the sign of the free energydifference driving the selected ion type. If S^ is positive,the synapse is "excitatory" because it tends to excite theneuron k to fire. Similarly, a negative Skj correspondsto an "inhibitory" synapse.

An equation of motion can be obtained as follows:For any neuron k, which fires action potentials at timesi* (n = 1, 2, 3, . . .), define the instantaneous firing rate tobe

fk(t) = (5)

In classical electrical circuit theory, the current flowinginto a capacitor as a function of time is a similar sum ofdelta functions because of the discreteness of electrons.The integral of fk(t) over a time interval yields the numberof action potentials occurring within the time interval,and in this sense fk(t) is the instantaneous rate.

Differentiating equation 4 with respect to time yields

(6)

Similarly, for the total current ik into cell k one has

—7 = -~r + ^Skjfj(t) + sensory term (7)

where the "sensory term" is an additional term presentonly for sensory cells. This equation, though exact, isawkward to deal with because the times at which theaction potentials occur are given only implicitly throughequation 1.

Synapse evolution algorithmsThe synaptic strengths Skj can also change with time,both during the development of an immature nervoussystem and as part of the learning and adaptation thatgo on in a mature one. While several such changes areseen in neurobiology, the most interesting variety is onein which the synapse strength S/y changes as a result ofthe roughly simultaneous activity of cells k and j . Thiskind of change is needed if a nervous system is to "learn"the association between two events. A synapse whosechange algorithm involves only the simultaneous activityof the pre- and postsynaptic neurons and no other detailedinformation (other than perhaps when to learn) is calleda Hebbian synapse.4

A simple version of such dynamics might be written

At = aikf.(t) - decay terms (8)

where a is a positive parameter. The "decay terms,"perhaps involving ik and fr are essential if the system isto forget old information. A nonlinearity or control proc-ess is important to keep synapse strength from increasingwithout bound. Also, the parameter a might be variedby neuromodulator molecules that control the overalllearning process. The details of the neurobiology are not

CELLPOTENTIAL

-53 mV •

-70 mV

200 milliseconds TIME-

Action potentials, spikes in the electricalpotential of the inside of a neuron, aregenerated when the cell potential reaches athreshold (-53 mV in this example),discharging the cell. After discharging, thecell resets to about -70 mV. When a constantcurrent is injected into the cell, actionpotentials are generated at a regular rate.(Adapted from data provided courtesy ofJames Schwaber, Du Pont ExperimentalStation, Wilmington, Delaware.) Figure 4

yet thoroughly understood, and equation 8 is only aplaceholder for a more adequate expression. Slightlymore complex synapse change rules of a Hebbian typereproduce results of a variety of experiments on thedevelopment of eye dominance and orientation selectivityof cells in the visual cortex of the cat.5

The synapse evolution algorithm, whatever its form,is one of the dynamical equations of the neural system.The tacit view is that learning and development involvesynapse changes, whereas the dynamics of neural activityis what performs a computation. This need not be thecase, however, and synapse modification should not beignored as a means of doing some kinds of computation.

A synapse change algorithm underlies the mostwidespread application of artificial neural networks. Afeedforward network (one without closed-loop pathways)is a trivial dynamical system, but such a network, withan appropriate set of connections, can solve nontrivialpattern classification problems.6 The computationalpower in this case is embodied in finding the correct setof connections, a process most often done by a highlyartificial synapse change dynamics that is quite unrelatedto biology. Even so, it is the dynamical method ofdetermining the connection strengths that results in auseful feedforward network.

Classical neurodynamicsNeural network dynamics can be described in two ex-treme limits. In one description, the "classical approxi-mation," individual action potentials have little effect andtheir precise timing is unimportant for coding informa-tion. We can then adopt the point of view that dealsonly with large numbers of action potentials in a statis-tical fashion. This paradigm has been used in much ofneurobiology and neuromathematics. In an alternativeparadigm the precise relative timing of action potentialsarriving along different axons is very important, and thedetailed time intervals between action potentials on asingle axon are used to code significant information. Iwill describe this paradigm later.

The classical approximation makes use of the factthat there will be many contributions to the sum on theright-hand side of equation 7 during a reasonable timeinterval as a result of the high connectivity. (The sum

PHYSICS TODAY FEBRUARY 1994 4 3

INPUT CURRENT

Average firing rate of a neuron depends on the inputcurrent. The approximation of "integrate and fire" withoutleakage or saturation would give the blue straight line.Leakage due to the cell's resistance results in the red curve,while noise fluctuations smooth out the sharp break in thecurve at zero firing rate, producing the black curve.Saturation effects in the mechanism that generates actionpotentials set an upper limit on the rate. Figure 5

over j typically includes thousands of cells that may makenonzero contributions.) In that case, it should be per-missible to ignore the spiky nature of fft) and replace itwith a convolution of/}(£) and a smoothing function. Inaddition, fj is presumed to be a function of ic, denotedV(ic), when ic is slowly varying in time. What results islike equation 7, but with hit) now a smooth function of

78time, fft) =

di

Thus7

-jT = -— + 2^SkjV(ij(t)) + sensory term (9)

The main effect of the approximation is to neglect fluc-tuation noise and to assume there are no strong correla-tions between spike events. (A similar description couldbe given in electrical circuits of the passage from consid-ering discrete electrons to regarding charge as a continu-ous variable.)

In many regions of the brain, the dominant connec-tivity is quite short range, and signal propagation delaysare negligible. While propagation delays can easily beintroduced into equation 9, the mathematics of equationswith delays is rather more complicated.

Computation with fixed connectionsWith one set of parameters, equation 9 describes a Vax.With another, it can mimic the electronics in a televisionreceiver. The set of all equations represented by equation9 is far too general to have simple universal properties.If computation is to be done by a convergence to pointattractors in the space of analog variables, one way ofachieving that end is to consider a restricted set ofnetworks that can be shown to converge to fixed points.We will therefore examine networks whose motion canbe understood as the state's moving generally downhillon an "energy function" (or Lyapunov function) that mayhave a complicated landscape with many minima.

The simplest case that is sufficiently flexible to beof interest is a symmetric network,8 defined by Sy = Sjr

The function

(10)xk,j

h 0

where V"1 is the inverse of the function V, V, = V(ij) andthe current Ik in the third term on the right comes fromexternal or sensory inputs, can be shown to always decreaseunder the equations of motion unless all variables havestopped changing. Since E is bounded below, this impliesthat the state will converge to the location of one of theminima of this function. The dynamics is understoodthrough the minimum-seeking nature of equation 9.

While symmetric connections are not the usual casein neurobiology (for example, an excitatory neuron canreceive inputs of either sign but can make only positive

connections to other neurons), there are various circum-stances in which more complex and biological networksare equivalent to symmetric networks. Even feedforwardnetworks can, by an appropriate transformation, be madeequivalent to symmetrically coupled networks.

The behavior of a symmetric system is easiest tounderstand in the limit of high gain, where the sigmoidresponse of the neurons (figure 5) approximates a step.Then, under most circumstances, every stable state musthave each neuron either at maximal activity or zeroactivity, and each such state lies at one of the corners ofa 2w-dimensional hypercube, where N is the number ofneurons. The stable-state problem is then isomorphicwith an Ising spin problem, but with each spin havingthe possible values 0 and 1 (instead of the more usual-1 and 1) and with the S/,j serving as the exchangeinteractions in the Ising Hamiltonian. This connectionwith spin systems, and in a limiting case to a spin glass,has permitted extensive analysis of the stable states.9

(See the article by Haim Sompolinsky in PHYSICS TODAY,December 1988, page 70.)

An associative memory can be constructed as follows.If a "1" is defined to be a neuron firing at maximal rate,and a "0" as a neuron not firing, in the high-gain limita memory is simply a state vector such as Vmem =1,0,0,1,0,0,0,0,1, The synapse change

AS*,- « (2V*Bm-l)(2V4em-l) (11)

will make Vmem a new fixed point of the dynamics, thatis, a new memory. This synapse change is of the Hebbiantype. Networks of a more biological flavor, having fixedpatterns of inhibitory connections and low mean activityrates and carrying memory information only in excitatoryconnections, also function as associative memories.

One can design symmetric networks to find solutionsto many complex tasks that can be posed as minimiza-tions.10 For example, such networks have found solutionsto the classical traveling salesman problem (in which asalesman wishes to visit each member of a set of citiesonce with the minimum amount of traveling) and to somepractical problems of circuit-board layout. Researchersstudy such synthetic problems in part to learn about thecomputational power that can be obtained from a singleconvergence to a fixed point of a symmetric network. Weexpect the highly fed-back neural circuitry of all complexnervous systems to exhibit this computational power. Wemight think of the single convergence to a fixed point asthe fundamental computing step of a brain, just as thesingle clock cycle is the fundamental computing step ofa conventional digital machine. However, the high con-nectivity of the neural system and the analog nature ofits convergence allow the performance of quite complextasks in a single step.

If we abandon the restriction of symmetry, the nextsimplest system is the excitatory-inhibitory network.

44 PHYSICS TODAY FEBRUARY 1994

Midpoint between ears

Signal from left ear

^ Signal from right ear

Binaural localization of sound by an owldepends on the timing of individual actionpotentials. Sequences of spikes from the twoears, representing the sound detected, travelalong antiparallel axons. The two streams ofspikes will tend to coincide and exciteneurons at a location (red) whose positioncorresponds to the time delay between thesounds detected at each ear. Figure 6

The neurons in such a network fall into two classes: Onehas only excitatory outputs; the other has only inhibitoryoutputs. Such networks can oscillate or even exhibitchaos11 (in computer modeling), and it has been difficultto gain enough mathematical control of them to see howto use them for powerful computation. Oscillatory sys-tems are now being intensely investigated, however, be-cause oscillatory behaviors are commonly found in thebrain, for example, in the olfactory bulb and the neo-cortex.

Action potential synchronizationThe foregoing analysis was based on the idea that indi-vidual action potentials are insignificant and that theprecise information about when particular action poten-tials occur is generally not relevant. Information isimplicitly encoded in the short-time average of the num-ber of action potentials generated by each cell. Thisparadigm lies behind most of the studies of the responseof individual neurons in mammals. For example, a neu-ron is called strongly responsive to a stimulus generatedby a moving bar of light if the neuron generates actionpotentials at a high rate only during the presentation ofthat stimulus. Detailed information about the relativetiming of different action potentials is ignored.

In some situations, however, the timing of actionpotentials is very important to biological function. Forexample, the arrival of a synchronized set of actionpotentials on the many axons of the vagus nerve triggersthe contraction of the heart. The timing of this pulse isessential to heart rhythm and blood pressure control.Action potential timing also matters in the binaurallocalization of a sound source by an owl. Each ear hearsa sound of the same form, but one sound is delayed withrespect to the other, and that delay determines theazimuthal location of a sound source. The brain meas-ures the time delay with an array of neurons that detectcoincidences between action potentials propagating with

finite velocity along antiparallel axons from the two ears.(See figure 6.)

In addition, visual object perception involves a set ofspecific computational problems that must be somehowsolved in the brain. One of these is related to how wepiece together the different parts of particular objects inthe visual field, so that an object is seen separated fromthe background of other objects. The idea that whatmoves together is a single object is one of the importantcriteria that the brain seems to use. (If you have everwondered why the door edge behind the subject's headis so plain in the snapshot yet was so invisible in theviewfinder when you were taking the picture, the parallaxcaused by your minor motions is a key element of theanswer.) Coherent oscillations or synchronized actionpotentials may occur in neurons responsive to separatedbut comoving edge elements of a single object. Ideas12

and experiments13 in this direction give impetus to theo-retical work on the synchronization of action potentialsin integrate-and-fire neurons.

Most of the theoretical work has been done to try tounderstand the range of phenomena occurring in simplesystems and has not yet focused on how such systemscan do useful computation. In the simplest case a groupof excitatory integrate-and-fire neurons are all connectedto one another ("all-to-all coupling"), and each couplingis excitatory and of the same strength. When the exter-nal or intrinsic current into each cell is the same (whichwould result in equal firing rates of all the cells if therewere no connections between them) and the synapticcurrent due to an action potential at t0 has the form

lit) =0 t<t0

t>t0

(12)

then the coupling synchronizes the action potentials ofall the cells.14 If the system begins in an arbitrary stateof activity, it will evolve to a state of synchronized firing,a cyclic attractor for the system. If the neurons have arange of firing rates in the absence of connections, or ifthe connections themselves are not uniform, a broaderclass of behaviors occurs, including phase transitions tothe synchronous state and the breakup of the cells intotwo classes, one group synchronized and one not.

In neurobiology, chemical events and molecular con-figuration changes take place between the occurrence ofa presynaptic action potential and the ultimate currentinjection into the postsynaptic cell. Therefore the meas-ured synaptic currents do not rise as a step but increasesmoothly from zero. For this more realistic synapticmodel, in the presence of noise, but with equivalent firingrates and equal all-to-all connections, there is a synchro-nization-desynchronization phase transition as a functionof noise amplitude.15 The less realistic model of equation12 retains a synchronized phase for all noise amplitudes.

PHYSICS TODAY FEBRUARY 1994 4 5

Fixed plate

Spring-block model of earthquakes issomewhat analogous to models of sheets ofneurons that account for individual neuronspikes. Springs connecting the blockscorrespond to synaptic connections betweenneurons, while the slipping of a blockcorresponds to the firing of a neuron.Consequently the types of complex behaviorseen in the earthquake models can beexpected to recur in neurobiology. (Adaptedfrom ref. 1 7.) Figure 7

In an elementary short-time view of neural compu-tation, the computation is performed by a convergence toa point attractor of a dynamical system. The synchroni-zation of spiking neurons is a special case of a moregeneral dynamical system whose motion converges to acyclic attractor. Richer dynamics with more complexattractors should allow attractor networks to solve morecomplicated problems than can be solved with simplepoint attractors.

The phenomena displayed by coupled integrate-and-fire neurons will be richer when the synaptic connectionpatterns are more complex. Even the replacement of theequal all-to-all coupling by a fixed near-neighbor synapticcoupling in two dimensions (to represent aspects of asheet of cells such as occurs in the neocortex) greatlychanges the kinds of behavior that are found. Thisproblem, which does not seem to have been studied inneurobiology, has in a limiting case a very close parallelwith the Burridge-Knopoff model16 of earthquake gen-eration at a junction between tectonic plates. (This pointwas jointly understood in discussions last spring betweenAndreas Herz, John Rundle and me.) In the Burridge-Knopoff model, the junction is represented by a set ofslider "blocks" that are connected to a moving upper plateby springs and are dragged along the lower plate. (Seefigure 7.) The motion of each slider is described bystick-slip friction. Each slider is also connected by othersprings to its nearest neighbors. An earthquake is initi-ated when one slider slips and triggers the motion ofother blocks. A "slip" event corresponds to an actionpotential, the spring from plate to slider corresponds toan external current from elsewhere into each cell, andthe springs between sliders correspond to synaptic con-nections. The slipping is "self-organized"17'18 and pro-duces a power-law distribution of earthquake magni-tudes. While there is no exact correspondence with realneurobiology in this limiting case, it does extend our ideasof the kind of phenomena that can emerge from retainingaction potential timing in neurodynamical equations.

Simplicity and complexityDigital machines and brains both carry out computationby being dynamical systems. A very simple repre-sentation of highly complicated neurobiology leads to adescription in terms of coupled nonlinear differential

equations. The equations presented here are a drasticsimplification of real biological neurons. Many featurescould be added, including propagation delays, a position-dependent intracellular potential and the use of intracel-lular Ca2+ concentrations as dynamical variables. Howimportant are such omitted features?

Physics has often made use of huge simplificationsto get to the heart of issues. For example, conventionalmodels of magnetism usually omit many details of mul-tispin interactions, magnetoelastic coupling and longer-range interactions, and yet they capture much of theessence of magnetic phenomena. Physicists are thereforeaccustomed to ignoring inconvenient details.

Biology, being an evolutionary science, is different.If some quirky detail of neurobiology is useful in animportant but special computation, that detail can beselected for and improved by evolution. As a result, inspecific parts of the brain, particular details that aregenerally negligible elsewhere can be of utmost impor-tance. The highly simplified model dynamics describedin this article is thus far too impoverished to describehow a brain operates.

Nevertheless, network computation with high connec-tivity between analog elements is the means by whichlarge brains gain an intelligence lacking in small nervoussystems. The attractor behavior of equation 9 has provedto be robust to noise and to changes such as the additionof delays and action potentials. Even though the ele-ments we have used are oversimplified abstractions, thisrobustness gives us reason to believe that we are makingprogress in understanding how networks of neurons andsynapses can carry out complex computations.

References1. J. J. Hopfield, in Evolutionary Trends in the Physical Sciences,

M. Suzuki, R. Kubo, eds., Springer-Verlag, New York (1991),p. 295.

2. E. R. Kandel, J. H. Schwartz, Principles of Neural Science, 3rded., Appleton & Lange, Norwalk, Conn. (1991), p. 19.

3. D. Junge, Nerve and Muscle Excitation, Sinauer, Sunderland,Mass. (1981), p. 115, discusses some of the complications ofthe biophysics of this process.

4. J. Hertz, A. Krogh, R. G. Palmer, Introduction to the Theoryof Neural Computation, Addison Wesley, Redwood City, Calif.(1991), p. 16.

5. M. F. Bear, L. N. Cooper, F. F. Ebner, Science 237, 42 (1987).6. See, for example, G. Dreyfus, in Applications of Neural Net-

works, H. G. Schuster, ed., VCH, New York (1992), p. 35.7. While these equations are in common use, they have evolved

somewhat, and do not have a sharp original reference. See,however, H. R. Wilson, J. D. Cowan, Biophys. J. 12, 1 (1972);and W. Gerstner, J. L. van Hemmen, Network 3, 139 (1992).

8. J. J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554 (1982)- 81,3088(1984).

9. E. Domany, J. L. van Hemmen, K. Schulten, eds., Models ofNeural Networks, Springer-Verlag, New York (1991).

10. J. J. Hopfield, D. G. Tank, Biol. Cybern. 52, 141 (1985). Y.Takefuji, Neural Network Parallel Computing Kluwer Boston(1992).

11. W. J. Freeman, Sci. Am., February 1991, p. 78.12. R. Eckhorn, R. Bauer, W. Jordan, M. Brosch, W. Kruse, M.

Munk, H. J. Reitboeck, Biol. Cybern. 60, 121 (1988).13. C. M. Gray, W. Singer, Proc. Natl. Acad. Sci. USA 86, 1698

(1989).

14. Y. Kuramoto, Physica D 50, 15 (1991). R. E. Mirollo, S. H.Strogatz, SIAM J. Appl. Math 50, 1645 (1990). L. F. Abbott,C. Vanvreeswijk, Phys. Rev. E 48, 1483 (1993).

15. M. Tsodyks, I. Mitkov, H. Sompolinsky, Phys. Rev. Lett. 71,1280(1993).

16. R. Burridge, L. Knopoff, Bull. Seismol. Soc. Am. 57,341 (1967).17. Z. Olami, H. J. S. Feder, K. Christensen, Phys. Rev. Lett. 68,

1244(1992).

18. P. Bak, C. Tang, J. Geophys. Res. 94, 15 635(1989). •

46 PHYSICS TODAY FEBRUARY 1994