165

Network models of the basal ganglia David G Beiser*, Sherwin E Huat and James C HoukS

Over the past year, a number of conceptual and mathematical models of the basal ganglia and their interactions with other areas of the brain have appeared in the literature. Even though the models each differ in significant ways, several computational principles, such as convergence, recurrence and competition, appear to have emerged as common themes of information processing in the basal ganglia. Simulation studies of these models have provoked new types of questions at the many levels of inquiry linking biophysics to

behavior.

Figure 1

Thalamus Striatum I Addresses Department of Physiology, M211, Northwestern University Medical School, 303 East Chicago Avenue, Chicago, Illinois 6061 l-3006, USA *e-mail: beiser@nwu.edu te-mail: sherwinhQnwu.edu Se-mail: j-houk@nwu.edu

t 1 Indirect

Direct pathway

pathway

1 - Pallidum <y Subthalamus

Current Opinion in Neurobiology 1997, 7:165-l 90 > 1997 Current Ohkn an Neurolmloov

Electronic identifier: 0959-4388-007-00165

0 Current Biology Ltd ISSN 0959-4368

Abbreviation BABA Taminobutyric acid

Introduction Knowledge of the anatomy and physiology of the basal ganglia has expanded dramatically in recent years (see [l] for a review). This has fostered the development of information-processing models that explore the unique neuronal architecture of this brain region. One architec- tural feature that is likely to be important is the existence of relatively ‘private’ loops (or channels) of connectivity between the cerebral cortex and basal ganglia [2,3], as summarized in Figure 1. A second important feature of the basal ganglia is the specialization of spiny neurons, the principal cells of the striatum, for pattern recognition computations [4]. A third important feature is the division of the striatum into matrix and patch (striosome) compart- ments with specialized neurochemistry and connectivity [l]. The recent models of the basal ganglia reviewed here have, to varying degrees, built upon these striking architectural features.

Organization of the loops (channels) connecting the cerebral cortex and basal ganglia. One or more areas of the cerebral cortex make convergent excitatory projections (white arrows) to a given region of the striatum. Through the direct pathway, striatal spiny neurons make inhibitory projections (dark gray arrows) to the internal segment of the globus pallidus and substantia nigra pars reticulata (pallidurn). The indirect pathway projects to the pallidum via the external pallidum and subthalamic nuclei. The pallidurn, in turn, projects to thalamic relay neurons that make reciprocal excitatory projections to columns in the cerebral cortex.

these signals into new representations of perceptions or action commands. The striatal architecture is well suited to this task.

Models of the striatum Most models of the striatum, the input stage of the basal ganglia, have emphasized pattern recognition or mutual competition, or a combination of the two, to form pattern classification networks. The many discrete areas of the cerebral cortex contain coarsely coded representations of sensory features, motor intentions or representational memories, and some kind of pattern classification operation would facilitate the translation of

Wickens, Kotter and Alexander [!P] have developed a biophysical model that explores the effects of local connectivity, which is dominated by recurrent GABAer- gic (presumed inhibitory) connections, on the response properties of striatal networks. In a separate study of the same model, the same group [6’] has explored the effects of glutamate and dopamine on the generation of complex patterns of network responses. In their connectivity study, Wickens et a/. [S”] modeled three different networks with different topologies of collateral connectivity, such as symmetric versus asymmetric. They found that each network displays a signature pattern of spatiotemporal activity, with areas of sustained firing, long-period traveling waves or short random bursts. This model has many strengths, not the least of which is its judicious mixture of biological realism with abstractions commonly used in large neural networks. The latter study also includes a brief analytical treatment of the model that offers a few general principles concerning the dynamics of inhibitory networks. The capacity of lateral

188 Cognitive neuroscience

inhibitory networks for selective activation has received a more thorough analytical treatment from Fukai and Tanaka [7**], who have shown how ‘winners-share-all’ and ‘winner-take-all’ computations depend on the strength of collateral inhibition in relation to self-inhibition.

Spiny neurons, which are nominally quiescent, display characteristic patterns of subthreshold fluctuations in their membrane potential [8]. A network model by Connolly and Burns [9*] hypothesizes a functional role for these membrane fluctuations and provides an explanation for their origin. The authors posit that subthreshold fluctuations could be involved in coordinating obstacle- avoiding limb movements. The striatum is treated as an electrotonically coupled grid of spiny neurons, each representing a state in a limb’s joint angle space. This use of electrotonic coupling, or passive computation, is a substantial departure from most models of the basal ganglia, which generally rely on the active propagation of neural signals. For its inputs, the model uses a mixture of excitatory and inhibitory afferents from the cortex. Although the model presents an interesting account of the striatum’s potential for passive computation, it extends beyond the anatomy and physiology of the system.

Numerous hypotheses have been proposed regarding the role of recurrent excitatory circuits in sustaining working memory activations through their attractor properties, as will be discussed below. A recent model by Woodward and colleagues [lo’] suggests that recurrent inhibitory circuits could also be used for sustaining such activations. The model abstracts the neostriatum as a three-dimensional array of spiny neurons in which each neuron makes an inhibitory contact with its neighbors. The network’s storage capability rests in its tendency to settle into different stable states of activation in response to patterns of phasic cortical inputs. An overall reading of the state of striatal activation, the authors contend, could be obtained through summation by units in the pallidum that receive convergent inputs from the striatum. Pallidal activations would in effect provide a type of scalar memory. This is an interesting model that emphasizes the processing capabilities of both the local striatal circuitry and the convergent striatopallidal projections. In addition, the model, though inspired primarily by bottom-up evidence such as anatomy, produces simulated patterns of activation that bear general resemblance to actual striatal recordings. One criticism is that approximately 50% of the model’s spiny neurons appear to be active at any given time; this amount of activation would seem to be unrealistically high given what is known about the striatum.

Loop models In recent years, anatomists have reached consensus that the loops interconnecting the basal ganglia with the cerebral cortex are organized in a highly topographic manner. Two contrasting anatomical models of the fun- damental architecture of this system have emerged in

the past few years. Graybiel and Kimura [ll*] have proposed that a given region within the striatum receives prominent convergent input from different cortical areas (e.g. areas a and b in Figure 1). In contrast, Strick and colleagues [12*] have suggested that the fundamental architecture is a closed loop from a given cortical area through a specific channel in the basal ganglia, and then back to the same area of cerebral cortex (e.g. a loop to and from area a in Figure 1, excluding area b). A number of computational models of the basal ganglia have incorporated one or the other of these anatomical models of cortical-basal-ganglionic architecture.

Several current theories of Parkinson’s disease suggest that the depletion of nigrostriatal dopamine neurons produces an imbalance between signals of the direct (i.e. from striatum to pallidurn; see Figure 1) and indirect (i.e. via external pallidum and subthalamus) pathways through the basal ganglia. Further, it has been suggested that an imbalance in these pathways leads to over-activation of neurons in the pallidum and subsequent over-inhibition of target neurons in the thalamus.

A recent model of the basal ganglia provides an account of how dopamine-dependent changes in dopamine Dl and D2 receptor expression and neuropeptide expression could lead to over-inhibition of thalamic neurons within motor outflow pathways [13’,14*]. In addition to exploring the local circuit effects of dopamine depletion, the authors integrate their model of biophysical rearrangement into the broader picture of motor behavior; this, in fact, is the major emphasis of the paper. To explore the effects on motor control, they linked the output of their local-circuit model to a previously published model of trajectory formation [15]. Inputs, on the other hand, consisted of a motor program signal from the premotor cortex. The composite model was then applied to a simulated cursive writing task [14*]. Their model displayed simulated trajectories of reduced amplitude and speed that resemble characteristic Parkinsonian movement impairments, such as akinesia, bradykinesia and micrographia. In this model, the functional role of the basal ganglia appears to be that of modulating the onset and speed of a motor program. In the diseased state, over-inhibition of the thalamus leads to a decrease in the amplitude and a delay in the onset of motor programs. This straightforward interpretation of basal ganglia function provides a simple explanation for the therapeutic mechanism and limitations of pallidotomy and thalamotomy surgeries. However, it does little to explore the computational capabilities of the basal ganglia.

Berns and Sejnowski [16’] hypothesize that the basal ganglia select appropriate cortical motor programs during the generation of sequential actions. They emphasize the interplay between the direct and indirect pathways linking the striatum and pallidum. However, whereas the model of Contreras-Vidal and colleagues [13’,14’] focused on the delicate balance of signal strengths between direct

Network models of the basal ganglia Beiser, Hua and Houk 187

and indirect pathways, that of Berm and Sejnowski [16*] emphasizes the timing differences between these two pathways. Such timing differences, they suggest, could be exploited for blocking competing motor programs via the indirect pathway. They also explore the dynamic consequences of a partially closed circuit, such as the one formed by the basal ganglia’s convergent inputs and segregated output channels. In particular, they suggest that the possibility of chaotic activity within the circuit is squelched by negative feedback from the indirect pathway via cortical projections to the subthalamic nucleus. Even though the reported simulations stop short of demon- strating actual sequence selection, they nevertheless offer several insights into the nature of the interactions between the cerebral cortex and basal ganglia.

Attentional control refers to the ability to effectively engage desired targets, maintain attention in the face of distractions, and disengage attention when either exogenous or endogenous cues are presented. A loop model of the basal ganglia by Jackson and Houghton [17*] promotes a role for the basal ganglia in the voluntary control of visuospatial attention. The model incorporates several computationally salient features of cortical-basal-ganglionic anatomy, including convergent corticostriatal projections and recurrent loops through the frontal cortex.

Considerable evidence now suggests that sustained activ- ity in the frontal cortex could represent working memory. Clearly, these memory signals would be useful for planning and controlling movements. Three variations of a recent model have been used to explore the ability of the basal ganglia to utilize such signals for directing saccades during delayed sequence tasks [18*,19”,20**]. Structurally, the model is quite elaborate, including many types of topographic and non-topographic projections between different cortical and subcortical regions. Interestingly, the model variation by Dominey [19**] does not include the effect of competitive inhibition within the striatum and instead places competition within the superior colliculus. Functionally, the model employs pattern recognition, recurrence and time delays to produce a sequence of oculomotor commands in response to a sequence of inputs. Overall, the flexibility of this model along with its ability to learn and perform different delayed sequencing tasks makes it quite impressive.

Beiser and Houk [Zl”] have modeled the cognitive response of an array of circuits linking the prefrontal cortex, thalamus and basal ganglia during a serial behavior task. The model incorporates some of the computa- tionally salient features previously mentioned, such as convergent corticostriatal projections for spiny neuron context recognition and collateral inhibition for spiny neuron competition. In addition, each circuit includes a bistable corticothalamic loop that sustains detected events within working memory. Sustained working memories are

then used to provide additional context to the model through recursive corticostriatal projections. The model has been tested with a simulated version of a delayed sequencing task and then compared with the results of single-unit studies. When instantiated with randomly distributed corticostriatal weights, different patterns of prefrontal activation resulted from different target se- quences. These patterns represent an unambiguous and spatially distributed encoding of the sequence. Parameter studies of these random networks have been used to compare the computational consequences of collateral and feedforward inhibition within the striatum. An analysis of the model’s receptive fields uncovered an interesting set of cue-, rank- and sequence-related responses that qualitatively resemble responses reported in single-unit studies [Z&24].

Models of reinforcement learning A number of models have assumed that nigrostriatal dopamine inputs to the striatum provide training signals for reinforcement learning. For example, the Arbib and Dominey model [18*] described in the previous section employs this mechanism to adjust the synaptic weights of its striatal spiny neurons. Straight reinforcement learning is relatively inefficient, but is greatly improved when it is guided by an adaptive critic that learns how to predict rewarding situations [ZS”]. Houk, Adams and Barto [26”] have developed a model of an adaptive critic that incorporates the unique patch/matrix architecture of the striatum together with a cellular model of a synaptic trace mechanism based on the rich second-messenger system found in spiny neurons. Gray [27*] has taken a psychiatric perspective to develop some of the issues that an adaptive critic needs to confront, such as relating goal-setting and goal-monitoring operations to the limbic portion of the basal ganglionic network and goal achievement to the sensorimotor portion. Kimura and Graybiel [28*] have developed a model of how nigrostriatal dopamine afferents, in addition to providing reinforcement, can serve as enabling, or gating, signals during the expression of learned responses.

Actions of basal ganglia loops on attractor networks in cortex As mentioned in the introduction, basal ganglia output is directed to many cortical areas (2,3]. Several of these cortical areas have been shown to exhibit attractor dynamics that contribute to associative memory, motor processing and execution. For example, recordings from inferior temporal cortex have revealed sustained activity patterns specific for remembered visual stimuli [29]. Amit’s [30’] theoretical work has shown that these activity patterns can be explained by a recurrent network with attractor dynamics mediating associative memory. Each behaviorally relevant visual pattern is thought to invoke a stable pattern of activation across the population of units in the network. This stable activity pattern, or point attractor, persists even after input to the network is removed. In

188 Cognitive neuroscience

addition to point attractors, recurrent networks can also exhibit dynamic attractor properties that are capable of encoding temporally varying activity such as a movement trajectory. A recurrent network model of motor cortex has shown that movement trajectories can be generated by dynamic attractors [31’]. In this model, trajectories, such as movement in a line or circle, are generated by the dynamic evolution of activity in a population of directionally tuned units.

Because the inferior temporal cortex, motor cortex, and prefrontal cortex all receive input from the pallidum, the basal ganglia are well positioned to regulate the dynamic activity in these cortical areas. Houk and Wise (see [32”,33”]) have outlined a model of how the basal ganglia (and cerebellum) could modify the motor and cognitive functions of the cerebral cortex. In essence, context- relevant information processing allows the striatum to instruct cortical areas as to which sensory inputs or patterns of motor output are behaviorally significant. By shaping specific thalamo-cortical circuits, the basal ganglia should be able to shape and modify the structure of attractor dynamics in the cortex. Then Hebbian learning in these cortical areas [34**] could proceed under the guidance of basal ganglia (or cerebellar) information processing. Hua and Houk [35*] have demonstrated how these conditions could lead to modification of the attractor landscape, allowing a subcortical channel to export its functioning to attractor networks in the cortex for faster and more efficient execution of the learned tasks. Hikosaka et a/. [36*] have also explored the interaction between basal ganglia outflow and Hebbian learning within the cortex in a recent conceptual model of procedural learning.

Distributed modular architectures in the control of behavior The difficulty of modeling a given area of the nervous system without confronting the mosaic of interconnectivity with other areas of the brain is becoming increasingly apparent. Structures that are widely distributed are, in fact, interconnected in a highly modular fashion [37]. In the previous section, we discussed how cortical-cortical attrac- tor networks might be regulated by output from the basal ganglia. Prominent regulatory influences on many areas of the cerebral cortex derive from the cerebellum as well [38]. The schema in Figure 2 illustrates the general plan of distributed modularity that emerges when cortical-basal- ganglionic, cortical-cerebellar and cortical-cortical archi- tectures are considered in combination.

A general scheme for the different styles of signal processing that might result from the different types of modules has been developed by Houk and Wise [32**] on the basis of our current knowledge of signal processing in the motor and premotor cortex. Under the assumption that similarity in modular architecture

Fiaure 2

0 1997 Current Opinion m Neurobiology

General plan of the distributed modular architecture subserving cognition and voluntary action. Many areas of the cerebral cortex have segregated channels for information processing through the basal ganglia and the cerebellum. (Note, however, that some areas of the cerebral cortex that have channels through the basal ganglia lack corresponding channels through the cerebellum.) In this manner, the powerful information-processing mechanisms of the striatum and cerebellar cortex are able to regulate focal areas within the cerebral cortex. The different cortical areas then influence each other through cortical-cortical connections, and actions are controlled through descending projections.

reflects similarities in information processing, Houk [33”] then explored possible mechanisms of cognitive signal processing in the prefrontally linked modules by draw- ing analogies with signal processing in primary motor cortex. This exercise in analogical thinking led to the theory that prefrontal cortical-basal-ganglionic modules are specialized for parallel search, leading to the initiation of a thought (in analogy with the initiation of an action). Prefrontal-cortical+erebellar modules are seen as a mechanism for tuning this search process so as to create a population discharge that accurately represents the thought. The cortical-cortical network is seen as the site in which initiation and tuning operations are coordinated and transmitted to other cortical areas.

Conclusions Since 1995, a number of conceptual, simulation and mathematical models of the basal ganglia have been pub- lished. Particular progress has been made in modeling the striatum as a competitive network for pattern classification. A number of models have dealt with the relationship between the cerebral cortex and basal ganglia as one of input and output. However, these well defined roles break down in other models that emphasize the recurrent nature of cortical-basal-ganglionic interactions whereby cortical representations may be recursively processed. Future modeling studies will need to consider the role that the basal ganglia play in shaping the activations of nodal points within cortical-cortical and cortical-cerebellar networks.

Network models of the basal ganglia Beiser, Hua and Houk 169

Acknowledgements This work was supported by National Institute of hlental Health (NIMH) center grant #PSO-hfH48185.

References and recommended reading Papers of particular interest, published within the annual period of review, have been highlighted as:

. of special interest s of outstanding interest

1.

2.

3.

4.

5. . .

Graybiel AM: Building action repertoires: memory and learning functions of the basal ganglia. Curr Opin Neurobiol I 995. 5:733-741.

Alexander GE, Crutcher MD, Delong MR: Basal ganglia- thalamocortical circuits: parallel substrates for motor, oculomotor, ‘prefrontal’ and ‘limbic’ functions. Prog Brain Res 1990, 65:119-l 46.

Middleton FA, Strick PL: New concepts about the organization of basal ganglia output. In Advances in Neurology, The Basal Ganglia and New Surgical Treatment for Parkinson’s Disease. Edited by Marsden CD, Obeso JA, Ohye C, DeLong MR. Philadelphia: Lippincott-Raven; 1997: in press.

Houk JC: Information processing in modular circuits linking basal ganglia and cerebral cortex. In Models of information Processing in the Basal Ganglia. Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press; 1995:3-g.

Wickens JR, Kotter R, Alexander ME: Effects of local connectivity on sbiatal function: stimulation and analysis of a model. Synapse 1995, 20:261-296.

This model of the striatum explores the effects of homogeneity and symmetry within relatively large networks of collaterally inhibited spiny neurons. Careful attention is given to the biophysical properties and responses of individual spiny neurons. The authors’ results suggest a possible mechanism for the choreic movements of Huntington’s patients.

6. Kotter R, Wickens J: Interactions of glutamate and dopamine . in a computational model of the striatum. J Comput Neurosci

1995, 2:195-214. A network similar to that of Wickens et a/. [5”1 is used to model the spatiotemporal interactions of normal, dopamine-depleted, and dopamine- agonist-treated spiny neurons. The authors’ results demonstrate that inter- esting patterns of activation can be produced by an inhibitory network with homogeneous connections.

7. Fukai T, Tanaka S: A simple neural network exhibiting selective . . activation of neuronal ensembles: from winner-take-all to

winners-share-all. Neural Computation 1997, 9:197-236. This paper provides a thorough analytical discussion of competitive in- hibition, which would appear to have relevance to the study of striatal dy- namics.

6. Wilson CJ: Basal ganglia. In The Synaptic Organization of the Brain. Edited by Shepherd GM. New York: Oxford University Press; 1990:279-316.

9. Connolly Cl, Burns JB: A state-space striatal model. In Mode/s . of Information Processina in the Basal Ganalia. Edited bv

Houk JC, Davis JL, Bei& DG. Cambridge, MA: MIT Press; 1995:163-l 83.

Unlike the majority of striatal models, this abstract treatment focuses on the computational role of subthreshold fluctuations in membrane potential. The striatum is modeled as an electrotonic grid of units that individually corre- spond to different states within a limb’s configuration space.

10. Woodward DJ, Kirillov AB, Myre CD, Sawyer SF: Neostriatal . circuitry as a scalar memory: modeling and ensemble neuron

recording. In Models of Information Processing in the Basal Ganglia. Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MlT Press; 1995:315-336.

Networks of inhibitory units in the neostriatum can produce sustained acti- vations and thus support a type of working memory. The authors discuss the problem of decoding such a distributed working memory.

11. Graybiel AM. Kimura M: Adaptive neural networks in the basal . ganglia. In Models of Information Processing in the Basal Ganglia.

Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MlT Press; 1995:103-l 16.

The authors describe patterns of convergent projections that would appear to provide a single area of the striatum with several homologous body part representations from different areas of the cerebral cortex.

12. Strick PL, Dum RP, Picard N: Macro-organization of the circuits . connecting the basal ganglia with the cortical motor areas. In

Models of Information Processing in the Basal Ganglia Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press; 1995:117-l 30.

This paper presents anatomical evidence supporting the hypothesis that the dominant architecture of the basal ganglia is one of closed trans-striatal loops interconnecting topographically specific regions of the basal ganglia with cortical motor areas.

13. Contreras-Vidal JL, Stelmach GE: A neural model of basal . ganglia-thalamocortical relations in normal and parkinsonian

movement. Biol Cybern 1995, 73~467-476. This model emphasizes the oppositional balance of signals in the direct and indirect pathways from the striatum to the internal globus pallidus. It is suggested that this balance is disturbed by dopamine-dependent changes in Dl and D2 receptor expression in these pathways. This straightforward model casts the circuits of the basal ganglia as single-input/single-output filters.

14. Contreras-Vidal JL, Teulings HL, Stelmach GE: Micrographia in Parkinson’s disease. Neuroreport 1995. 6:2069-2092.

;he model of Contreras-Vidal and Stelmach [13*] is attached to a model of trajectory formation [151. The composite model is altered to simulate the effects of Parkinson’s disease and applied to the task of cursive writing.

15. Bullock 0, Grossberg S, Mannes C: A neural network model for cursive script production. Biol Cybem 1993, 70:15-28.

16. Berns GS, Sejnowski TJ: How the basal ganglia make decisions. . In The Neurobiology of Decision Making. Edited by Damasio AR,

Damasio H, Christen Y. Berlin: Springer-Verlag; 1996:101-l 13. This model explores how timing differences between the direct and indirect pathways linking the striatum and pallidum could play a role in the selection of competing motor programs.

1 7. Jackson S, Houghton G: Sensorimotor selection and the basal . ganglia: a neural network model. In Mode/s of information

Processing in the Basal Ganglia. Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press; 1995:337-367.

This model, while lacking in detail, suggests an interesting role for the basal ganglia in mediating attention.

16. Arbib MA, Dominey PF: Modeling the roles of basal ganglia in . timing and sequencing saccadic eye movements. In Models of

Information Processing in the Basal Ganglia. Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press; 1995:149-l 62.

This model integrates the basal ganglia with the saccade-generating centers of the brainstem. It includes several interesting elements of basal ganglia processing, including corticothalamic working memory loops and dopamine- mediated reinforcement learning in the striatum.

19. . .

Dominey PF: Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning. Biol Cybem 1995, 73265-274.

A model similar to that of Dominey et a/. [20”1 is applied to a complex sequence reproduction task.

20. . .

Dominey PF, Arbib MA, Joseph JP: A model of corticostriatal plasticity for learning oculomotor associations and sequences. J Cogn Neurosci 1995, 7:31 l-336.

An interesting, though somewhat complex, model of oculomotor control ap plied to conditional visual discrimination and sequence reproduction tasks. The model utilizes recurrence to provide contextual inputs regarding previous states. Reinforcement learning is used to associate sensorimotor contexts with correct saccades.

21. Beiser DG, Houk JC: Model of cortical-basal ganglionic . . processing: encoding the serial order of sensory events.

J Neurophysiol t 997, in press. This paper presents a modular network that encodes serial inputs within spatial patterns of activity in the prefrontal cortex. The responses of the model’s units resemble those observed in single-unit studies of sequential tasks [22-241. In addition, the network does not require training.

22. Barone P, Joseph JP: Prefrontal cortex and spatial sequencing in the macaque monkey. Exp Brain Res 1969. 76447-464.

23. Kermadi I, Jurquet Y, Arxi M, Joseph JP: Neural activity in the caudate nucleus of monkeys during spatial sequencing. Exp Brain Res 1993, S&352-356.

24. Kermadi I, Joseph JP: Activity in the caudate nucleus of monkey during spatial sequencing. J Neurophysiol 1995, 74:91 l-933.

25. . .

Barto AG: Adaptive critics and the basal ganglia. In Models of information Processing in the Basal Ganglia. Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press; 1995:215-232.

The mathematical underpinnings of the adaptive critic variant of reinforce- ment learning is discussed in a simple, and intuitive, manner.

190 Cognitive neuroscience

26. Houk JC, Adams JL, Barto AG: A model of how the basal . . ganglia generate and use neural signals that predict

reinforcement In Models of fnformation Processing in the Basal Ganglia. Edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press; 1995:249-270.

A conceptual model that comoares the local circuit of the striosomal soinv neurons ‘to that of an adaptive critic. A detailed mechanism involving both intracellular and intercellular signals is outlined as a possible account for the reward-predicting behavior observed in some spiny neurons.

27. Gray JA: A model of the limbic system and basal ganglia: . aoolications to anxietv and schiroohrenia. In The Coanitive

f&urosciences. Edited-by Gazzaniga MS. Cambridge, MA: MIT Press; 1995:1165-l 176.

This conceptual model is given a psychiatric twist that emphasizes how information-processing operations in the limbic system and ventral striatum help the organism identify series of linked subgoals that are seen to be essential in guiding behavior.

28. Kimura M, Graybiel AM: Role of the basal ganglia in sensory . motor association learning. In Functions of the Cortico-Basal

Ganglia Loop. Edited by Kimura M, Graybiel AM. Tokyo: Springer- Verlag; 1995:2-l 7

This conceptual model provides a link between the limbic system and motor output. It suggests that the nigrostriatal dopamine projections may serve as a phasic reinforcement signal, as well as a gating signal, during the expression of learned responses.

29. Miyashita Y: Inferior temporal cortex: where visual perception meets memory. Annu Rev Neurosci 1993,16:245-263.

30. Amit DJ: The Hebbian paradigm reintegrated: local . reverberations as internal representations. Behav Brain Sci

1995, 16:617-657. The author proposes that local reverberatory networks with attractor dynam- ics are a general feature of cortical function.

31. Lukashin AV, Amirikian BR, Mozhaev VL, Wilcox GL. . Georgopoulos AP: Modeling motor cortical operations by an

attractor network of stochastic neurons. Biol Cybern 1996, 74:255-261.

Whereas point attractors may underlie associative memory, the authors pro- pose that dynamic attractors underlie the generation of the population direc- tion vector. In their model, dynamic attractors allow the temporal evolution of the population vector, leading to the generation of movement trajectories.

32. Houk JC, Wise SP: Distributed modular architectures linking . . basal ganglia, cerebellum and cerebral cortex: their role in

planning and controlling action. Cereeb Cortex 1995. 5:95-l 10. This paper presents an integrated model of motor control. The basal ganglia select behaviorally relevant motor programs whereas the cerebellum shapes

and executes the command. Modular architectures in these areas cooperate to sculpt activity in corresponding cortical modules, which are able to leam and automate movements.

33. . .

Houk JC: On the role of the cerebellum and basal ganglia in cognitive information processing. In Progress in Brain Research. Edited by De Zeeuw Cl, Strata P, Voogd J. Amsterdam: Elsevier Science; 1997:545-554.

The author outlines a theory of how thought processes, such as the instan- tiation of working memory in the prefrontal cortex, utilize modular signal pro- cessing operations in a manner analogous to signal processing in movement control. Mechanisms for distributed learning are also discussed.

34. . .

Amit DJ, Brunei N: Learning internal representations in an attractor neural network with analogue neurons. Network 1995, 6:359-388.

Hebbian learning in an attractor network is explored. The model, which has dual activation and learning dynamics, operates in a palimpsest fashion in which attractors learn the statistics of stimuli presented. Attractors repre- senting stimuli that are no longer presented are forgotten.

35. Hua SE, Houk JC: Cerebellar guidance of premotor network . development and sensorimotor learning. Learning Mem 1997, in

press. A recurrent network comprised of three layers of units endowed with Heb- bian synapses and weight nonalization is used to model the development of a subcortical channel that interconnects the motor cortex and cerebellum. The developed network is then used to model adult plasticity in the motor cortex. This model illustrates a simple mechanism for exporting subcortical knowledge to the cerebral cortex where it can be utilized in a more efficient manner. These concepts are also applicable to subcortical channels through the basal ganglia.

36. Hikosaka 0, Rand MK, Miyachi S, Miyashita K: Procedural . learninp in the monkev. In Functions of the Cortico-Basal

GangliaLoop. Edited by Kimura M, Graybiel AM. Tokyo: Springer- Verlaa: 1995:l E-30.

This paper reports experimental results of a sequential paradigm designed to differentiate procedural from declarative learning. In addition, an interesting conceptual model involving both corticostriatal and corticocortical plasticity is discussed.

37. Wise SP, Houk JC: Modular neuronal architacture for planning and controlling behavior. Biol Commun Dan R Acad Sci Left 1994, 43:21-33.

38. Middleton FA, Strick PS: New concepts regarding the organization of basal ganglia and cerebellar output In Uehara Memorial Foundation Symposium on Integrative and Molecular Approach to Brain Function. Edited by Ito M, Miyashita Y. Tokyo: Elsevier Science; 1997:253-271.