Using temporal binding for hierarchical recruitment of conjunctive concepts over delayed lines

ARTICLE IN PRESS

Neurocomputing 69 (2006) 317–367

0925-2312/$ -

doi:10.1016/j

�CorrespoE-mail ad

URL: htt

www.elsevier.com/locate/neucom

Using temporal binding for hierarchicalrecruitment of conjunctive concepts

over delayed lines

Cengiz Gunaya,c,�, Anthony S. Maidaa,b

aCenter for Advanced Computer Studies, University of Louisiana at Lafayette, Lafayette, LA 70504, USAbInstitute of Cognitive Science, University of Louisiana at Lafayette, Lafayette, LA 70504, USA

cDepartment of Biology, Emory University, Atlanta, GA 30322, USA

Received 9 January 2004; received in revised form 15 December 2004; accepted 1 March 2005

Available online 24 August 2005

Communicated by D. Wang

Abstract

The temporal correlation hypothesis proposes using distributed synchrony for the binding of

different stimulus features. However, synchronized spikes must travel over cortical circuits

that have varying-length pathways, leading to mismatched arrival times. This raises the

question of how initial stimulus-dependent synchrony might be preserved at a destination

binding site. Earlier, we proposed constraints on tolerance and segregation parameters for a

phase-coding approach, within cortical circuits, to address this question [C. Gunay,

A.S. Maida, Temporal binding as an inducer for connectionist recruitment learning over

delayed lines, Neural Networks 16 (5–6) (2003) 593–600]. The purpose of the present paper is

twofold. First, we conduct simulation studies that explore the effectiveness of the proposed

constraints. Second, we place the studies in a broader context of synchrony-driven recruitment

learning [L. Shastri, V. Ajjanagadde, From simple associations to systematic reasoning: a

connectionist representation of rules, variables, and dynamic bindings using temporal

synchrony, Behav. Brain Sci. 16 (3) (1993) 417–451; L.G. Valiant, Circuits of the Mind,

Oxford University Press, Oxford, 1994] which brings together von der Malsburg’s temporal

binding [C. von der Malsburg, The correlation theory of brain function, in: E. Domany,

J.L. van Hemmen, K. Schulten (Ed.), Models of Neural Networks, vol. 2, Physics of Neural

see front matter r 2005 Elsevier B.V. All rights reserved.

.neucom.2005.03.008

nding author.

dresses: [email protected] (C. Gunay), [email protected] (A.S. Maida).

p://www.cacs.louisiana.edu/�maida/Neuroidal.

www.elsevier.com/locate/neucom

ARTICLE IN PRESS

C. Gunay, A.S. Maida / Neurocomputing 69 (2006) 317–367318

Networks, Chapter 2, Springer, New York, 1994, pp. 95–120, (Originally appeared as a

Technical Report at the Max-Planck Institute for Biophysical Chemistry, Gottingen, 1981)]

and Feldman’s recruitment learning [J.A. Feldman, Dynamic connections in neural networks,

Biol. Cybern. 46 (1982) 27–39].

A network based on Valiant’s neuroidal architecture is used to implement synchrony-driven

recruitment learning. Complementing similar approaches, we use a continuous-time learning

procedure allowing computation with spiking neurons. The viability of the proposed binding

scheme is investigated by conducting simulation studies which examine binding errors. In the

simulation, binding errors cause the formation of illusory conjunctions among features

belonging to separate objects. Our results indicate that when tolerance and segregation

parameters obey our proposed constraints, the sets of correct bindings are dominant over sets

of spurious bindings in reasonable operating conditions. We also improve the stability of the

recruitment method in deep hierarchies for use in limited size structures suitable for computer

simulations.

r 2005 Elsevier B.V. All rights reserved.

Keywords: Temporal synchrony; Temporal correlation; Binding; Recruitment learning; Tolerance window;

Phase segregation; Spike response model

1. Introduction

In the brain, functionally and physically separate areas of cortex (e.g., visual,auditory, somatosensory) analyze different features of environmental objects. Thisraises the question of how physically distributed feature representations arecombined to form coherent unitary percepts. This question was first identified asthe binding problem in neural representations by Rosenblatt [46]. The temporal

binding approach to solving this problem was first proposed by von der Malsburg[73]. It has become known as the temporal correlation hypothesis (TCH), which positsthat binding of disparate feature representations is achieved by synchronized neuralfiring across cortical areas [73,74,21,61,62,32,45,11,67]. The viability of TCHdepends on maintaining the synchrony of neural spikes coding features from thesame object and desynchrony of spikes coding features from different objects.

In the temporal binding model, von der Malsburg [73] suggested that in a highlyconnected brain-like structure, synchronously active units may employ a fastsynaptic modification mechanism to form dynamic ensembles. These ensemblesrepresent combinations of features for a unitary percept. This approach eliminatesthe problem of combinatorial explosion associated with static binding mechanismswhere a new unit is needed to represent each possible binding. Temporal binding, byusing time to code space, requires only elementary feature units to be present andallows combinations to be formed dynamically via transient potentials atinterconnecting synapses. The number of units needed, to represent possible entitiesthat a cognitive system is exposed to, is thus lowered from increasing exponentiallyto increasing quadratically with respect to the number of features. The temporalbinding proposal opened the way to many theoretical and simulation studies[75,27,59,47,35,66,65].

ARTICLE IN PRESS

C. Gunay, A.S. Maida / Neurocomputing 69 (2006) 317–367 319

Our previous work [24] noted that some cortical connection topologies posechallenges for maintaining synchrony. These topologies consist of variable-lengthpathways that converge onto some destination area. An example of such a topologyis the converging direct/indirect pathways in Fig. 1. Here, varying delays areobtained with varying the number of synapses crossed by each pathway. If thesources are synchronized, one would expect that the destination readouts would havea way to detect the synchrony at the source. Analytical and simulation studies ofDiesmann et al. [9] show that, if appropriate connectivity is employed, spike volleyspropagating across neuronal pools can become more synchronized. The neuronsfavor synchrony allowing synchronous volleys to propagate protected, filtering outuncorrelated noise. Diesmann et al. did not address the problem of direct/indirectconverging pathways. Our previous work proposed constraints on the timing andintegration properties of these circuits in order to address the problem ofsynchronization.

The purposes of the present work are: (1) to conduct simulation studies thatexplore the effectiveness of the proposed constraints; and, (2) to place the studies in abroader context of synchrony-driven recruitment learning . The present work isintended to help establish a biologically plausible substrate for structured

connectionist models (pioneered by Feldman, Shastri, and others) that are capableof reasoning and other cognitive functions [12,49,59,70,15,71,54].

1.1. Spike timing: tolerance and segregation

The viability of synchrony-driven recruitment learning depends on the accuracy ofneuron spike timing. Fig. 1 shows a neural connection topology that naturally

(synchronized) Intermediate (mismatch)DestinationSource

CarSmall

CurvedRoof

Volks-wagen

Yellow

YellowVolks-wagen

Fig. 1. Connection topology showing possible direct and indirect pathways from an initially synchronized

source of activity converging at a destination. Dashed boxes indicate the hypothesized stages of

processing, in which each individual solid box indicates the localized processing for a single feature.

ARTICLE IN PRESS

10 20

Indirect

0

Direct

t (msec)

Γ

t

lower

square

triangle

upper

Inputs: upper, square,lower, triangle

(a) (b) (c)

Fig. 2. (a) The tolerance window G required to integrate inputs such as shown in Fig. 1. Initially on the

left, spikes corresponding to primitive object properties are synchronized. After traveling over separate

pathways with different delays, their synchrony is degraded, as seen on the right side. By defining this

tolerance window the spikes can still be treated as synchronous. Timings are chosen arbitrarily for

illustration. (b) Activity in separate phase windows for each object in a scene. (c) The scene contains a

triangle in the lower part of the visual field and a square in the upper part.


emerges in a hierarchical recruitment scenario. Spike synchrony is difficult topreserve in this topology because, by virtue of different signal transmission delays,initially synchronous activity may dissipate after passing through separate direct/indirect converging pathways.

The figure illustrates a hypothetical circuit formed by recruited neurons forrecognizing a ‘‘yellow Volkswagen’’ object. In the leftmost part of the figure, theneural firings representing the primitive object properties occur synchronously as thesubject focuses attention. This is consistent with stimulus-dependent synchronybehavior [62]. It is reasonable to require an additional level of processing for thepotentially more complex shape properties than the color property of the object.This allows the circuit to use features such as ‘‘small car’’ and ‘‘curved roof’’ aselements to form the intermediate concept of a Volkswagen independent of color. Iftemporal binding is employed, the signals at the destination need to be synchronousto represent the same object. However, when signal transmission times areconsidered, the degree of synchrony at the final destination is degraded due tosignals crossing varying-length pathways with varying delays.1

This work addresses two related aspects of spike timing in direct/indirectconnection topologies. These are tolerating delays and preventing crosstalk. Fig. 2(a)shows how a tolerance window can be used to integrate signals with varying delaysinto the same response. This window specifies the maximum time allowed betweentwo incoming spikes that can both contribute to generate a particular actionpotential [31,50,53,56]. Its purpose is to implement feature binding. In contrast to

1Similar examples can be formed with multi-modal sensory stimuli. Regarding formation of object

representations, since reaction time for auditory stimuli is faster than for visual stimuli, the signals that

account for the sound of an object are processed faster than the signals representing the image of the

object.

ARTICLE IN PRESS


signals that correspond to the same object, signals that correspond to differentobjects must remain separate, or desynchronized, in order to avoid mismatchingfeatures from different objects as seen in Fig. 2(b). Desynchrony can be obtained byassigning each object to a different tolerance window. This causes objectrepresentations to be segregated into separate phase windows [47,35,59,52,66,28,4].Its purpose is to implement feature segmentation.

This phase-coding approach is one possible method for desynchronizingresponses. This method puts neural activity pertaining to each object in a separatephase window (of a large oscillatory period). This prevents interference as theactivity propagates to deeper structures, preserving its integrity with respect totemporal binding. Many theoretical and simulation studies show that segregationbetween object representations, or desynchronization, can be obtained by inhibitoryprojections. Some propose using global inhibitory projections [35,66], while otherspropose using lateral recurrent inhibitory connections [47,28].

Other support for phase windows arises as a solution to the problem of multiple

instantiation in connectionist models [38,63,64]. These studies offer a biologicallyplausible way to represent multiple instances of objects by placing them in phasewindows. This approach, in contrast to symbolic systems, models more closely thedefects in performance observed in psychological studies. In symbolic systems, it ispossible to instantiate an arbitrary number of representations of the same object. Onthe other hand, in neural models an object representation is usually associated withthe same units. Therefore, there is a problem if the object needs to appear more thanonce in a relation. Thus, phase coding is an appropriate solution to this problem.

The phase-coding approach also has its skeptics. Knoblauch and Palm [29,30]suggested that results of some electrophysiological recording experiments areinconsistent with phase coding. In particular, recordings of activity pertaining tomultiple separate stimuli (e.g., bars moving in opposite directions) cause a flatcorrelogram, whereas phase coding predicts a non-zero time lag of the central peak.Knoblauch and Palm suggested that the TCH can be interpreted differently. In theirmodel, circuits represent bound entities with fast, synchronized oscillations andunbound entities with slow unorganized firing. Circuits can quickly switch betweenthese two states to represent dynamic bindings. Another alternative to phase codingis synchronization among synfire chains for representing dynamic bindings [1].

The present work extends the study of phase segregation by investigating how topreserve synchrony and desynchrony in the presence of delays involving direct/indirect converging pathways.

Here, the relevance of recurrent inhibitory circuits is secondary, since we areinterested in the effect of delays on the synchronous activity coming from upstreamareas, converging to a single destination site. We study the constraints on toleranceand segregation parameters required to maintain the phase coding at the destinationreadout site and its ability to control recruitment learning.

We have chosen the connection topology in Fig. 1 for studying the effects ofdelays. It arises naturally in a recruitment learning framework and more complexconnection topologies can be transformed to this case by hierarchical reduction. Forthis topology, we found constraints on the tolerance and segregation measures that

ARTICLE IN PRESS


satisfies the conditions for temporal binding. For each of the measures, we calculateda lower bound, that is a minimal value.

The need to define measures of tolerance and segregation has been addressedbefore. In forming hierarchical learning structures, Valiant [70] suggested that higherlevels of processing need to operate in slower time scales than lower levels. In sloweroperation, the higher-level units may integrate information from the faster lowerlevels of processing. For instance, high-level units representing a scene may need tointegrate all information about the contents of the scene presented sequentiallybefore giving a response. This implies that the scene unit has a tolerance durationmany times longer than the lower level. Consistent with this view, Newell [42] hasrecognized a time scale hierarchy, ranging from milliseconds to years, in modelingcognitive behavior.

VanRullen and Koch [72] identified relevant experiments in reviewing the theoryof discrete perception. It is known that, if two successive events follow one anothertoo closely, they are perceived as a single event. The critical time interval for thisphenomenon to occur is 20–50ms. This interval is consistent with the time requiredon each item to prevent perceiving illusory conjunctions in visual search experiments[60,67]. VanRullen and Koch [72] reported that, in composing a higher levelrepresentation of a multi-object scene, the perceived number of objects is limited to�10–12 items/s. Overall, a perceptual time frame of �100ms is proposed whilestimuli are grouped to form a single event. Note that, the purpose of theseexperiments is to establish the time frame of a single event in perceiving timing ofmoving scenes. Even though these experiments support our idea of multiple timinglevels, the actual measurements may not apply directly to our current purpose, whichis to establish the timing while building hierarchical representations of static scenes.

1.2. Approaches to recruitment learning

During the 1990s, two approaches to recruitment learning were developedindependently. One approach was pursued by Valiant [69–71] working in theoreticalcomputer science and the other was pursued by Shastri [51,53,55,57] working inconnectionist artificial intelligence (AI). Shastri studies how relational instanceinformation, like that conveyed in the sentence ‘‘John gave Mary a book in thelibrary,’’ might be represented in a brain-like neural network. In humans, suchepisodic information can be learned in ‘one shot,’ in contrast to the gradual learningcommonly exhibited by artificial neural networks. Further, in humans, thehippocampal system has been implicated as the initial storage substrate for suchinformation. Shastri has developed a model of memory acquisition (SMRITI) in thehuman hippocampal system that uses temporal synchrony to drive a variant ofrecruitment learning based on long-term potentiation (LTP).

Shastri links connectionist models to a biological substrate. He focuses on thebinding problem for episodic information which he represents using a role–fillernotation. The matching of the roles and fillers are bindings which need to betransformed from a temporal to a structural representation. Shastri and Ajjanagadde[59] hypothesized that role–filler bindings in a relational instance (like that given

ARTICLE IN PRESS


above) are represented in the human brain via temporal synchrony as patterns oftransient rhythmic activity. In particular, bindings are represented as correlatedfiring, but a relational instance will have multiple bindings that must be mapped toseparate phase windows. Shastri and Ajjanagadde were also aware of the temporarynature of such representations and were perhaps the first to observe that therepresentation could be made more permanent by using recruitment learning to buildstructural representations. They did not report a specific mechanism until [51].Relevant to the present topic, they also observed that there are noisy signalpropagation delays between cortical areas and that there is an issue of maintainingsynchrony in such circumstances. However, they did not address the problem ofdirect/indirect pathways illustrated in Fig. 1.

Valiant addresses problems fundamental to theoretical AI. Among these arebrittleness, the variable-binding problem of predicate calculus, non-monotonicreasoning and learning mechanisms approaching human functioning. The knowl-edge representations that are used in the model exhibit two significant features. Oneis that knowledge is represented redundantly by a number of units followingFeldman [13,14]. The other is use of temporal binding implied by the definition ofrecruitment learning. Valiant argues that to prevent brittleness, intelligent systemsshould use learning mechanisms to acquire skills, as opposed to using prepro-grammed knowledge. Valiant proposes tractable learning procedures that operate onthe network. For this purpose, learning in the probably approximately correct (PAC)sense is employed [68]. Since the present research builds directly on Valiant’s model,it is discussed in detail in Section 2 of this paper.

Valiant’s recruitment learning proposes that, after weight modification, a synapsecannot be modified further. This is for protecting the contents of the information itmemorized. This may seem biologically unrealistic, but there may be new supportingbiological evidence. Matsuzaki et al. [40] performed experiments by syntheticinduction of LTP in the CA1 area of rat hippocampus and then observed sizechanges of dendritic spines in pyramidal neurons using two-photon imaging. Theyidentified two populations of spines: small and large. Small spines may undergo LTPand become larger for �100 min during these experiments. Other in vivoexperiments suggest that these spines are stable for months at a time [22]. Mostinterestingly, Matsuzaki et al. show that large spines are resistant to undergoingfurther LTP. This is consistent with the theory that small spines represent silentsynapses that may potentially undergo LTP. Thus, they become enlarged and tend tostay that way to protect their newly acquired memory possibly with the help ofreadout mechanisms.

The issue of desynchronization, or segregation of signals pertaining to differentobjects into phase-windows, has also been explored [47,35,66,28].

1.3. Direct/indirect connection topologies

Assuming a phase-coding approach, timing is crucial in preserving the integrity ofsignals with respect to temporal binding, especially when synchronized spike volleysmust meet after taking alternate cortical paths. This is similar to the hypothetical

ARTICLE IN PRESS


example discussed in Fig. 1. There are likely many cortical connection topologiesthat exhibit direct/indirect converging pathways. For instance, Fig. 3 shows the M-pathway of the interareal connections between visual areas V1, V2 and V3 [37,2].The propagation pathways in the figure can be confirmed by consulting the visualresponse latencies given in Table 1 [44,34].

Consider a synchronous response that originates in area V1 and which thenpropagates first to area V2 and next to V3. Area V1 has direct connections to V3.Thus, V3 will receive two synchronized spike volleys caused by the same stimulus:one directly from V1, and one through V2.

Even though we are not aware of direct evidence supporting that meetingpathways actually converge at the cellular level, it is reasonable to believe that thearriving signals interact since local cortical circuits are highly interconnected.

Shastri [53] uses a direct/indirect connection topology shown in Fig. 4. Shastri[53,56] describes the recruitment of binding detector cells (BIND) and binding-errordetection (BED) cells. Although Shastri is aware of the issue of signal propagationdelays as evidenced in Shastri and Ajjanagadde [59] and that his model neurons(cf. [57]) include signal transmission delays, he does not report results explaining howhe handles the timing issues in this direct/indirect problem. Fig. 4 shows an areanamed ROLE directly projecting to an area named BED. There is also an indirect

projection from ROLE to BED via BIND. The purpose of BED is to recruit bindingerror detector cells based on associative LTP. This requires synchronized input fromROLE and BIND. Shastri [55] says that, once recruited, a BIND cell takes up to25ms to respond (and 100ms to be recruited if not already recruited). However, histolerance window for coincidence detection is 5ms. He does not discuss how BINDoutputs are synchronized with ROLE outputs. Shastri reports quantitative resultspertaining to number of nodes recruited, but his analyses only use the connectivity ofthe underlying graph structure and do not make reference to signal transmissiontiming properties within the structure. But they should also depend on preserving theintegrity of synchrony. Despite the fact that his model reportedly includestransmission delays, and that theoretically he subscribes to a temporal correlationframework with phase-segregated integration windows, he does not report exactlyhow he gets SMRITI to function. Similar criticisms apply to [57].

Defining a lower bound on the segregation between phases in hierarchicalstructures with direct/indirect converging pathways allows predicting their maximalfrequencies of activity. Conversely, from the observed range of brain activity

V1

4Ca and 4B

V2Thick stripes

V3Layers

LGNM pathway

2∆ + �

∆ + � ∆ + �

Fig. 3. Simple example of direct/indirect connections in visual cortex possibly leading to mismatched

arrival times of spikes. D stands for axonal propagation delay and d stands for synaptic transmission delay

and integration time.

ARTICLE IN PRESS

Table 1

Response latencies in ms in visual areas taken from [34]

Area Earliest Mean

V1 35 72

V2 54 84

V3 50 77

MT 39 76

The early response in MT is due to the connection received from superior colliculus (SC) which we

disregard in this study.

ROLE

BIND

ENT

BED

Fig. 4. A direct/indirect converging pathway circuit used by [53].


frequencies, the maximum depth of this type of recruitment hierarchies can bepredicted. For the parameter values estimated from the visual cortex, and under theassumption of homogeneous conduction speeds, we find that, only two-level deephierarchies can be supported for gamma-band (20–70Hz) frequency range. Forslower activity in the alpha-band (8–12Hz) range, hierarchies up to six levels can beformed. Based on this, we hypothesize that small hierarchies are formed at localareas producing gamma-band activity, and larger hierarchies form over larger areasof cortex producing alpha-band activity. This is consistent with the observation oflocal gamma-band activity and global alpha-band activity (cf. VanRullen and Koch[72]). VanRullen and Koch had a different interpretation, that slow alpha-bandactivity constitutes the ‘‘context’’ and the faster gamma-band activity provides the‘‘content’’ of neural representations.

1.4. Stability issues in hierarchical recruitment

There is a stability issue in hierarchical recruitment because the expected numberof neuroids recruited to represent a concept depends on statistical properties of thenetwork connectivity. When a concept chain is recruited in cascade, such as in deephierarchies like Fig. 5, the variance in the number of recruited neuroids needs to be

ARTICLE IN PRESS


very small. Otherwise, progressing deeper into the hierarchy, the recruited set ofneuroids may either become unreasonably large, or disappear entirely.

Valiant [70] calculated the expected number of recruited units and their variance.He found that the value of the variance was close to the value of its expectation. Ourinitial simulations using a low number of total neuroids per area (102 as compared toValiant’s 108) showed that very few units can be recruited after crossing two levels ina hierarchy such as in Fig. 5. Regarding larger networks, Valiant [70] showed that if108 neuroids were employed with a replication factor of 50, stable recruitment up tofour levels can be achieved. Gerbessiotis [16,17] verified this for asymptotic networks(number of units assumed to be infinite).

Gerbessiotis also provided a rigorous formalism of the expected recruited set size.Shastri [55,58] identified six-layer deep structures in the cortico-hippocampal circuitsto which recruitment learning may be applicable.

The stability issue is more serious in smaller networks. In the present work, wepropose a stabilizing mechanism based on feedback inhibition, to use recruitmentwith small populations. The mechanism applies to larger networks also, foroptimizing their performance. Such a mechanism may be appropriate for beingemployed in small biological networks of localized patches of cortical circuits.However, its biological plausibility needs further investigation.

The use of inhibition as a control mechanism goes back to Marr [39]. Marrproposed a feedforward neural network that can act like an hetero-associativememory in his theory of the archicortex. This model used recurrent global inhibitionproportional to the total excitatory input to retrieve a single memory item from the

Yellow

Velks-wagen

Projectsto

Fig. 5. A hierarchical recruitment scenario. The circles indicate the set of neuroids that represent each

concept, whereas the large ellipses indicate projection sets of these neuroids. Number of neuroids in

intersections vary in an unstable manner when recruitment is used repetitively.

ARTICLE IN PRESS


circuit. Otherwise, either no output, or too many outputs, would be produced.McNaughton and Morris [41] brought this idea to the context of more recentanatomical and physiological knowledge about memory formation in thehippocampal system. Shastri and collaborators [59,56] propose using this type ofrecurrent inhibition, for the same purpose, to limit retrieval of recruited items frommemory in their model of memory acquisition in the hippocampal system. Shastri[56, p. 362] claimed that this can be done by feedforward local circuits and inhibitoryinterneurons, but Shastri’s simulation studies [55,57] do not employ this proposedinhibitory circuitry. The present work follows Shastri’s proposal for using recurrentinhibition to limit excess recruitment. In addition, we propose increasing theconnection density calculated according to Valiant’s proposal in our model tocomplement the limiting function of the inhibition. This is required to counteract theinhibition and amplify small inputs. We name this mechanism boost-and-limit withinour framework [25].

1.5. Summary

Our main objective can be summarized as follows. We investigate the toleranceand segregation parameter constraints for performing temporal binding acrossvarying-delay converging pathways. We present formal and simulation results thatconfirms our earlier timing hypotheses [24]. Our result verifies that the tolerancewindow for keeping coherent representations and the amount of phase segregationto prevent crosstalk for such a given topology can be calculated. This enables bothinterpretation of biological circuits and design of artificial networks. We alsoimprove the recruitment stability in deep hierarchies. This allows using recruitmentin limited size structures suitable for simulation studies.

Our work is consistent with previous work on recruitment learning [49,56,12,8,15].We augment Valiant’s neuroidal model to continuous-time by using the spike

response model (SRM) of [18]. There are also disadvantages of using spiking neuronmodels. They are more complex compared with simplified neural units. Thus theyrequire more sophisticated simulation environments, and they are computationallyexpensive. The SRM is more efficient than other spiking models, such as theintegrate-and-fire (I/F) model, because it allows larger step sizes during simulation.This is because SRM uses time responses of activity functions that do not requireintegration. Other methods that require integrating differential equations often needvery fine step sizes to maintain accuracy.

Complementing other studies on recruitment which mainly provided analyticalcalculations and statistical simulations [70,16,17,56,15], our work extends byimplementing an actual simulator that employs recruitment learning and investigatesits practical applicability. We have uncovered many issues during this simulationstudy such as the instability of the recruitment method.

Our results verify that the tolerance window for keeping coherent representationsand the amount of phase segregationto prevent crosstalk for such a given topologycan be predicted. This enables both interpretation of biological circuits and design ofartificial networks. We also improve the stability of the recruitment method in deep

ARTICLE IN PRESS


hierarchies which allows using recruitment in limited size structures suitablefor computer simulations. The stability mechanism proposed also illustrates aplausible biological mechanisms for maintaining stable signal propagation in thecortex.

The organization of the rest of this paper is as follows. First, in Section 2,recruitment learning is described. The problem of instability in recruitment isdiscussed and solutions are proposed in Section 2.9. In Section 3, timing issues inusing recruitment in certain problematic conditions are explored. Then, measures oftolerance and segregation for maintaining coherence of temporal bindingin direct/indirect connection topologies are defined in Section 4. The methods for testing theproposed hypotheses, followed by the simulation results are given in Section 5.Finally conclusions and future work are given.

2. Recruitment learning

This section describes the recruitment learning procedure based on Valiant’sproposals. The following subsections progressively build the context of therecruitment learning simulation for later sections. Section 2.1 starts by giving abrief summary of the key points of recruitment.

2.1. What is recruitment learning?

Briefly, recruitment learningis a scheme for allocating on-demand representationsfor new concepts [13]. The key feature of recruitment learning is that it operateswithin a static random graph. Vertices in the graph correspond to neural units thatparticipate in representing concepts. The recruitment learning method addresses thequestion of how localist concepts might be allocated in a graph structure like thebrain. The method allows for novel concepts representing conjunctions of conceptsto be allocated by synchronous stimulation of existing concepts. These existingconcepts, upon stimulation, coincidentally activate units where signals converge dueto random interconnections. The two points to emphasize in recruitment learning arerandom connections and synchronous activity, both of which have some biologicalsupport [78,13,70].

Recruitment is an unsupervised learning method. However, once concepts areacquired through recruitment, supervised learning methods can be used to associaterelated concepts [70]. Recruitment can be accomplished with a single example,therefore allowing one-shot learning.

Novel concepts are added to the system only if necessary, similar to the adaptiveresonance theory (ART) model. [6] The latter is an important feature thatdistinguishes recruitment from the monolithic nature of standard artificial neuralnetworks.

We illustrate recruitment learning in the following subsections. Section 2.2introduces Valiant’s neuroidal architecture employing recruitment learning.Section 2.3 describes how we organize our network differently from Valiant’s simple

ARTICLE IN PRESS


random network to test tolerance and segregation. Section 2.4 describes the type ofneural representation that is employed. Section 2.5 makes the relation to temporalbinding explicit. Section 2.6 gives a simple example to illustrate recruitment. Thisexample is revisited in Section 2.7 to analyze the inner workings of the networkduring recruitment.

2.2. A neuroidal architecture

Valiant [70] describes recruitment learning in the framework of his neuroidalarchitecture. In its simplest form, the neuroidal network is formed by a simplerandom interconnection network as seen in Fig. 6(a).

Each node represents a neuroid which is the elementary building block of thenetwork. Apart from being a linear threshold unit (LTU), the neuroid is also a finite

state machine (FSM) that controls the parameters of its LTU (see Fig. 6(b), (c)).In the neuroidal network the FSM detects coincidences among the neuroid’s inputs.It starts at the initial available (A) state and later changes into the memorized (M)state, according to its membrane potential (p).

NTR

0

0

0

0

1

1

1

1

Inputs Weights OutputNo firings

T = ∞

T = ∞

State: Available

p MemorizedState

InitialState

T: threshold

p: potential (net input)

Mp > 2

T = p

p < 2

A∑

(a)

(b) (c)

Fig. 6. Valiant’s neuroidal network: (a) initially blank random interconnection network of neuroids. Also

known as the neuroidal tabula rasa (NTR); (b) the linear threshold unit of the neuroid; (c) state machine of

the neuroid.

ARTICLE IN PRESS


2.3. Structural organization

In order to construct models of varying-length pathways, we organized thenetwork into subparts rather than grouping all neuroids in a single randominterconnection network. Consistent with Valiant [70], we group neuroids in abstractsets influenced by cortical areas to form a multipartite graph structure. Amultipartite graph contains connections only between its separate partitions. Therandom interconnections that enable recruitment are located between areas ratherthan intraareally as seen in Fig. 7(a). The connection probability used in this work isgiven with simulation testbed details in Section 5.1.

2.4. Redundant localist representations

The neural representations that we employ can be described as redundant localistrepresentations. Concepts are represented by a small redundant number (r�10) ofrandomly distributed units within the network. This number r is known as thereplication factor. Since each set represents a single concept, the representation is inessence localist. Redundancy ensures robustness in case some units are lost. This typeof representation is also called modularly distributed or distributed localist [3]. Each ofthese neuroids projects to a large number of neuroids in connected areas, againrandomly, seen in Fig. 7(b).

2.5. Recruitment depends on temporal binding

When projections of two separate concept sets target the same area, common unitsare included in both projection sets due to the statistical properties of the network.can be recruited upon coincident firing of the two sets. The units that receive inputfrom both sources are recruited upon coincident firing of the two concept sets, asseen in Fig. 7(c).

The theory of temporal binding suggests that in the brain features of objects arebound together by synchronous activity. That is, synchronized activity representsfeatures of a single object, whereas desynchronized activity indicates separate objects.Therefore, it can be said that recruitment depends on the principle of temporalbinding. Coincident, or synchronous, activity triggers the memorization mechanismfor recruitment, caused by activity above a preset threshold in units at the intersectionset of the two projections. These units change their internal states and weights topermanently memorize the feature combination. This set of recruited units representsthe high-level concept of the conjunction of the two primitive concepts. The premiseof recruitment learning is that a suitable connection density can be chosen such thatthe recruited set will be expected to contain as many units as the initial concepts.

2.6. How recruitment works

The following simple example illustrates recruitment. Details of the state machineare postponed until the next section. Fig. 8(a) shows a target neuroid d with three

ARTICLE IN PRESS

Area Area

RandomConnections

Yellow

Yellow

YellowVolkswagen

Projectionof Yellow

Projectsto

Volks-wagen

(a) (b)

(c)

Fig. 7. Network structure: (a) interareal random connections; (b) projection set of neuroids representing a

concept; (c) temporal binding activates the intersection set.


binary inputs. The neuroid is initially in the ‘‘Available’’ (A) state with all connectionweights set to 1 (cf. Fig. 6(c)). Consider the following time profile of spikes for thegiven inputs as seen in Fig. 8(b).

�
At time t ¼ 1, input c is active, however, this does not affect the state of the targetneuroid. This is because the net input is insufficient to cause the state transition(see Fig. 6(c)). � At t ¼ 2, a coincidence has occurred. Inputs a and b are active simultaneously.
The net input is above the transition threshold, so the target neuroid changes tothe memorized (M) state. As seen from the weight changes indicated in Fig. 8(b)while weights from a and b retain their original value of 1, the weight from c

decreases to 0. Furthermore, the threshold T is set to the current net input p ¼ 2.As a result, the target neuroid emits a spike. Therefore, the neuroid has specializedand memorized to recognize a and b together, ignoring the rest of its inputs.
� At t ¼ 3, inputs a and c are active. However, this event will be ignored because
the weight from c has been pruned during memorization. Since the neuroid hasalready specialized, this particular coincidence will be counted as a spurious event,neither causing the target neuroid to change state, nor to fire.

ARTICLE IN PRESS

(a) (b)

Fig. 8. A simple recruitment example: (a) target neuroid for the recruitment example; (b) time profile of

activity in inputs and target. The square pluse represent activity timing for each unit. The connection

weights or states are indicated inside the activity pulse. The vertical dashed lines only indicate sampling

times.


�
At t ¼ 4, inputs a and b are active, being the memorized event, triggers a spike atthe target neuroid.
Note that, the state machine explained so far is simplistic, built for illustration andfor fast computer simulations. It minimally consists of two states, only allowingpermanent memorization. This may seem to be a coarse model of cognitive memoryfunctions. Human memory is much more dynamic, forgetting unnecessary items ifthey are accessed with low frequency. The state machine here can be improved tomodel more realistic memory behavior. For instance, a third state can be introducedfor the case when the neuroid enters a ‘‘forgetting’’ phase. Gradual weightadjustment mechanisms can also be built.

2.7. Tracing state transitions

This section describes the details of the state transitions that caused the behaviorwe briefly explained in previous section. Recall that, the state machine consists of

ARTICLE IN PRESS


two states: an initial Available (A) state and a final Memorized (M) state (see statemachine in Fig. 6(c)).

At initial state A, as seen from the LTU on the left of Fig. 9, no inputs areavailable 8i I i ¼ 0, weights are 8i wi ¼ 1, and the firing threshold is T ¼ 1. Theneuroid will stay in this state until pX2 which indicates a cumulative sum of morethan one simultaneous input.

The LTU on the right in Fig. 9 shows the triggering situation where two inputsare simultaneously active, causing the reset of all but the active connection weights,and the threshold being set to the net input T ¼ p to ensure the neuroid will onlyrespond to the inputs just memorized.

The actual state machine used in this work is slightly more complex. It containsthree states, where the intermediate state is for gradual learning of concepts.However, we allow only a single learning step to yield fast, one-shot learning. Theweight update rule consists of a one-time application of wðtþ 1Þ ¼ 1:5wðtÞ for activeinputs, and wðtþ 1Þ ¼ 0:5wðtÞ for inactive inputs. This learning rule is a multi-plicative weight adjustment method consistent with Hebbian learning, and inspiredby the Winnow algorithm [36] (for details of the algorithm, see Gunay [23]).

2.8. Limitations

Recruitment has been proposed mainly for only forming two-way conjunctions[13,70,56,57]. We believe that this constitutes a starting point for the work inrecruitment learning. We predict that n-way conjunctions can be formed viarecruitment learning with the help of stabilizing lateral inhibitory connections(cf. Gunay [23], Gunay and Maida [25]). Therefore, we believe that there is notheoretical limit for forming conjunctions with more than two inputs. For simplicity,we use two-way conjunctions in this work because the problem of direct/indirectconverging pathways is independent of this limitation of the present form ofrecruitment learning used.

The units in our model are restricted to representing a single concept, since it isbased on Valiant’s model [70]. Nevertheless, hierarchical representation implies thatconcepts take part in representing other concepts. This allows constituent conceptsto take part in formation of multiple complex concepts. Therefore units will always

0

0

0

0

1

1

1

1

Inputs Weights

p

Inputs Weights Output

0

0

1

1

0

0

1

1

State: Memorized

p

Fire if p > T

T = ∞

T = ∞

State: Available

∑ ∑

T = 2

OutputNo firings

(a) (b)

Fig. 9. FSM states before and after memorization: (a) in initial state; (b) state after memorization.

ARTICLE IN PRESS


take part in representation of multiple concepts. Biological plausibility of this type ofrepresentation has been discussed in several places [14,70].

2.9. Robust hierarchical recruitment

There is a stability issue in hierarchical recruitment because the expected numberof neuroids recruited to represent a concept depends on statistical properties of thenetwork connectivity. When a concept chain is recruited in cascade, such as in deephierarchies like Fig. 5, the variance in the number of recruited neuroids needs to bevery small. Otherwise, progressing deeper into the hierarchy, the recruited set ofneuroids may either become unreasonably large, or disappear entirely.

The solution we propose here may resemble a model of a localized cortical circuitthat stabilizes the size of the recruited set at a desired level. If the random connectiondensity calculated according to Feldman and Valiant’s prescriptions is amplified, ithelps to obtain sufficient recruitment. This happens because the expectation isalways larger than the required value for recruitment. However, this time there is aninstability introduced due to uncontrolled increase in recruitment. This instabilitycan be viewed as a steady increase in the size of each set of recruited neuroids. Thisbehavior will result in a state of global synchrony, similar to a seizure, unlessprevented.

We suggest using a negative feedback element to control the amplification weapplied in the first place, to keep the recruitment at a desired level in the spirit ofearlier work [39,41]. We call this mechanism boost-and-limit.It can also be seen as awinner-take-all (WTA)circuit that only allows the first k units to fire and be allocatedfor a given target concept (also called a k-WTA). The limit imposed on the recruitedset is the replication factor, r, discussed earlier. Using a WTA for maintainingrecruitment stability in a network has been proposed earlier by [56]. Also, a similarWTA mechanism for separating sets representing items in spiking associativememories has been proposed by [29]. Notice that the increase proposed for theconnection density determines the properties of the static network and the densityneed not change dynamically.

3. Timing issues in recruitment

So far we explained the simplest recruitment learning algorithm. We showed thatthis algorithm depends implicitly on temporal binding. Some problems with timingarise when certain learning conditions are used with temporal binding, such ashierarchical learning and learning in direct/indirect connection topologies withdelays.

3.1. Timing in hierarchical learning

The phase-coding approach for temporal binding requires that activity pertainingto different objects occurs in separate phase windows. From the aspect of

ARTICLE IN PRESS


hierarchical learning, evaluating a higher-level concept composed of the conjunctionof two lower-level objects that occur in different phases requires detecting theconjunction of temporally separate events. This is consistent with experiments thatshow perceiving a multi-object scene as a single event takes �100ms [72]. This forcesus to extend the meaning of temporal binding.

Consider the example shown in Fig. 10. On the left, a hypothetical connectioncircuit in the brain is shown. The right-hand side shows the timing activity in thecircuit nodes. The circuit represents the neuroidal substrate that integrates thefeatures in a visual scene. There are two objects in this scene: a blue square and agreen circle. Since we assume temporal binding, both objects are attended to by thebrain during separate phases. The first object attended to at phase t ¼ 1 has blue andsquare active simultaneously. The blue square detector unit (b&s) is also active inthe same phase on the time profile on the right. At phase t ¼ 2, the same happens forthe green circle object.

Having constructed the lower-level primitives of the perceived scene, assume that aconcept that represents the whole scene is needed. That is, a node should represent‘‘the scene with a blue square and a green circle.’’ Notice from the figure that such anode cannot depend on detecting simultaneous activity at its inputs because the b&sand g&c units are active at separate phases, with no overlapping activity. Intuitivelyas a solution, it can be suggested that the scene detector unit observes activity inadjacent phases during an interval and gives a decision. This can be justified byclaiming that different levels of cognitive activity work at different speeds. In this

Fig. 10. Hierarchical learning example. (Left) Structure of concepts. (Right) Time profile of activity.

ARTICLE IN PRESS


case, while a primitive feature binding happens quickly, the scene can only beperceived after all objects are scanned.

The idea here is motivated by and consistent with other proposals [70,Chapter 11]. Such a scenario is addressed in Valiant’s architecture by detectingtimed conjunctions. These are conjunctions of inputs that appear at different times.

Valiant examined forming hierarchical structures when a recruitment-stylelearning algorithm is employed. In particular, each level of hierarchy is shown tobe slower than the previous one by a factor of the number of objects allowed. In ourcase, there were two objects which imposes that the scene recognition unit waits forboth objects. In general, the operation of the scene unit has to accommodate asmany objects that need to be represented.

Valiant showed that, for m possible objects, and a base t0 time windowfor synchrony at the lowest level, a new level needs a t1 ¼ mt0 time duration tointegrate results from the previous level. In general, this becomes ti ¼ mti�1 ¼ mit0for a level i.

Newell [42] has also proposed multiple time scales to explain different cognitivephenomena across levels.

3.2. Delays in converging direct/indirect pathways

We proceed to explore further cases which require timed conjunctions. So far, weignored delays in our examples for simplicity. However, a realistic system should atleast consider, or maybe depend on, unavoidable delays in its components andtransmission lines.

We treat delays caused by hierarchical learning and delays caused by unequaltransmission lines similarly. We study them by focusing on one example casecontaining varying-length converging pathways as shown in Fig. 11. In this case, atolerant way of conjoining is needed for integrating signals covering different lengthpathways before coming to the final destination due to varying delays [24].

This specific situation is addressed in [70, Chapter 5], where the followingapproaches were discussed:

�
Ignoring this case by assuming that all paths converging from a source to adestination are of equal length, or � Having peripheral systems which can provide persistent firings until computations
terminate.

Our work follows the second proposal by defining the peripheral systems that helpcomputations in these structures.

3.3. Detecting temporally separate coincidences

The hierarchical learning scenario shows the difficulty in passing informationacross time scales. The difficulty is in detecting activities that appear for aduration shorter than the higher-level time scale, and which do not overlap in time.

ARTICLE IN PRESS

Source Destination

Inter-mediate

Fig. 11. Direct/indirect pathways from a source converging at a destination.

(a) (b)

Fig. 12. Possible solutions for tolerant conjoining by changing behavior of high-level concept

representations: (a) by increasing spread of activity for each spike; (b) by persistent firing with a fixed

spread of activity.


The no-overlap problem also appears in the case of direct/indirect convergingpathways with delays. Without simultaneous activity, the net input of a neuroidcannot provide the information for detecting the desired timed conjunctions. That is,without memory of the past, the state machine we proposed for recruitment learningin Section 2.7 cannot handle timed conjunctions.

Two possible solutions can be proposed to the problem of tolerant conjoining

temporally separate but related activity, or timed conjunctions. The first, seen inFig. 12(a), is by increasing the spread of the activation on the postsynapticmembrane caused by inputs at the unit representing the high-level concept. Since theactivity on the membrane is long enough to overlap in time, the unit can detectsimultaneous activity to learn the features of the scene. The second method, seen inFig. 12(b), is by persistent or repetitive firing of units representing the low-levelconcepts. In this case, the inputs need to fire repetitively until an overlapping effect iscaused on the destination higher-level concept. Both solutions are based on the sameprinciple of causing overlapping activity to be detected at the target unit, onlydiffering in implementation.

In this work, we only consider the first method of increasing the spread ofactivation at the destination membrane. We assume the default effect of spikes on

ARTICLE IN PRESS


the membrane give the desired behavior at no extra computational cost.Implementing the second method would require an external controller.

3.4. Defining peripherals for timing with delays

When delays are considered, tolerant conjoining is needed for simple conjunctionseven within the same time scale. Consider the example shown in Fig. 13 with twoobjects, similar to the previous example. Each object now has three features; shape,color, and movement type. We assume that the shape information needs anadditional intermediate level of processing in comparison to movement.

Even though this lacks direct biological evidence, it is known that the movement isperceived faster through the magnocellular pathways in the visual system. Theincrease in the transmission speed in the magnocellular pathway is due to the largeraxons. A direct connection line may model this increased speed in transmittingmovement information.

When the subject attends to the shaking blue square object, all three inputs areavailable simultaneously at t ¼ 1. The blue square detector will only become active

Fig. 13. Recruitment example with delays and direct/indirect connections.

ARTICLE IN PRESS


after a delay from this time (a unit delay for simplicity), thus requiring tolerantconjoining with the input shaking.

The main aim of this example is to introduce another measure, the phasesegregation parameter. The segregation parameter is defined as the time from thebeginning of an object’s tolerance window until the beginning of the next object’stolerance window. Tolerance should be limited to avoid crosstalk of activityrepresenting features of one object, with the features of the next object attended. Toprevent occurrence of crosstalk, it is necessary for the segregation parameter value tobe equal or greater than the tolerance parameter. We call this the phase segregation

constraint.On one hand, if tolerance is too long, features of successively attended objects will

crosstalk. On the other hand, if tolerance is too short, the system will suffer fromlimited speed of processing due to inefficient use. An optimal setting is possiblymaintained dynamically by the brain. Therefore in the figure, shaking blue square

(sh. bl. sq.) should not tolerate any signals arriving after t ¼ 2, or it will result inspurious bindings with the features of the swinging green circle object.

3.5. Limitations

Tolerant conjoining assumes activity of the constituent parts of the hierarchy ispresented in adjacent phases. If, for instance, a phase sequence contains constituentsof two hierarchies interleaved, then the time we compute for tolerant conjoining willbe insufficient. However, this invalidates the temporal binding assumption, since theparts of a concept are not synchronous, or in this case, close in time. Thus, orderingof constituents in adjacent phases is a requirement for synchrony-driven recruitmentlearning, and for tolerant conjoining to work. This identifies the phase ordering

problem. Placing constituents of each hierarchy in adjacent phases is a complexproblem. We assume attentional mechanisms are responsible for this task. This mayrequire recurrent circuits and reverberating activity (i.e., multiple passes over thescene) before finding the correct phase ordering. The correct hierarchies must be alsoinfluenced by the way attentional mechanisms work. Thus, the way hierarchies areformed should be according to logical grouping of features into objects (e.g., allobjects at nearby locations or with similar features are automatically groupedtogether).

Next, we formalize the constraints on tolerance and segregation parameters forovercoming the problems described here.

4. Defining constraints of tolerance and segregation

This section extends the constraints on the tolerance and phase segregationparameters presented earlier for maintaining object coherency with respect totemporal binding [24]. For a given connection topology with converging inputs overpathways of varying delays, two requirements are studied: one for the duration of

ARTICLE IN PRESS


the tolerance window G, and a second for the duration of the phase segregationparameter F. We first consider the tolerance constraints.

Definition 4.1. A tolerance window Gi, for a neuronal unit xi, is the longest timeduration, or interval, during which the unit can integrate a train of incoming spikesthat all contribute to emit a single action potential (AP). The unit is said to delay-

tolerantly conjoin the inputs received during this interval.

In this work, Gi does not change for different units. Therefore, we simply use G.

Definition 4.2. A neuronal unit x, is said to cover a set of distributed sources only incase all synchronized spikes from those sources arrive at x within the interval G.

Lemma 4.3. Let neuronal unit x receive incoming signals from a set of distributed

sources at varying distances. x covers this set of sources if

GXdmax � dmin, (1)

where dmax and dmin are the longest and shortest transmission delays from the sources,respectively.

We then start building a neuroidal model capable of delay-tolerant conjoining. Aneuroid is a neuron-like linear threshold unit (LTU) which also has an embeddedfinite state machine [70]. We first employ a simple model, where the effect of anincoming spike on the postsynaptic membrane potential is a discrete pulse withconstant magnitude.

Theorem 4.4. Delay-tolerant conjoining of two disparate spikes from sources covered

by a neuroidal unit xi is possible, if each spike causes a G-long constant potential of

magnitude P on the postsynaptic membrane pi.

Proof. Let the first spike reach xi at time t1 and second spike at time t2, wheret2 � t1pG according to Definition 4.2. Potentials caused by the disparate spikes onthe membrane pi would overlap for some interval 0oIpG. Since xi is an LTU, thenet input pi will reach 2P during the interval I. A threshold T can be set to detect thesum of the overlap value and ignore non-overlapping spikes by choosing PoTo2P.Crossing the threshold can cause an AP and trigger the recruitment of a conceptrepresenting the conjunction, satisfying Definition 4.1. &

4.1. Implementing with a continuous-time model

Thus far, we used a discrete-time model, consistent with Valiant’s recruitmentlearning framework. This was also the case in the previous examples used forillustration, where we employed unit delays. However, considering timing dynamicsof the brain, unit delays and discrete models can only provide a coarseapproximation. A simple integrate-and-fire (I/F) spiking neuron model is moreappropriate. We use the spike response model (SRM) of Gerstner [18,19]. This modelnaturally allows delay-tolerant conjoining of separate signals by generalizing

ARTICLE IN PRESS

0 5 10 15 20 25 300

0.1

0.2

0.3

0.4

0.5

time (msec)

pote

ntia

l

∆+τs

∆+τm

Γ

Fig. 14. Shape of a SRM EPSP, �iðtÞ.


Theorem 4.4, relying on increasing the temporal spread of higher-level conceptactivations discussed in Section 3.3 (see Figs. 12(a) and 15).

Using the SRM, a tolerance window for direct/indirect converging pathways canbe implemented by adjusting the length of the EPSP on the postsynaptic membrane(see Fig. 14). Fig. 15(b) shows how this allows lagged spikes to have an overlappingeffect on the postsynaptic membrane that can be detected with a threshold device.Here we attempt to find constraints for the SRM EPSP, in order to assess its partswhich are significant for tolerance or segregation.2 This allows making formalclaims.

In the SRM, an EPSP caused by a single presynaptic spike i at t ¼ 0 is given bythe kernel,

�iðt; ts; tm;Daxi Þ ¼

1

1� ts=tmexp �

t� Daxi

tm

� �� exp �

t� Daxi

ts

� �� Hðt� Dax

i Þ,

(2)

where Hð�Þ is the Heaviside step function, ts is the synaptic rise time constant,tm isthe postsynaptic membrane time constant, and Dax

i is the presynaptic axonal delayassociated with the spike. For simplicity, we refer to (2) as �iðtÞ when ts; tm;Dax

i arekept constant. The rise and decay behavior of (2) depend, respectively, on the ts andtm parameters [18] as shown in Fig. 14.

First, we establish that there is only one maximum of �iðtÞ, which depends on theparameter selection of ts and tm. This maximum is written as

�ts;tmi ¼ �ðtmax; ts; tm; 0Þ ¼ maxt0

�iðt0; ts; tm; 0Þ.

Daxi is omitted since it does not affect the shape of the EPSP (see Appendix A for

derivation of tmax).

2This is similar to the study of Shastri [56] to discretize an EPSP to three regions; a rising part, a plateau,

and a decaying part.

ARTICLE IN PRESS

10 20

Indirect

0

Direct

t (msec)

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

time (msec)

pote

ntia

l

Directspike Indirect

spike

Temporalsummation

Threshold

∆+τs

∆+τm Γ

Γ

(b)(a)

Fig. 15. SRM helps tolerate delayed signals: (a) initially synchronous signals cross direct and indirect

pathways, and arrive early and late, respectively. The G-long shaded window duration needs to be

tolerated; (b) The 10ms tolerance within the shaded window can be demonstrated with SRM neurons.

Early and late arriving spikes cause over lapping excitatory post synaptic potentials (EPSPs) that sum to

exceed a detection threshold. Non-overlapping EPSPs do not reach this threshold.


We now propose parameters for implementing the tolerance window. For delay-tolerant conjoining, a neuroid should emit an AP when the effective parts of twoEPSPs caused by separate incoming spikes overlap, whereas the EPSP caused by asingle spike must not cause an AP. The following definition is intended to restrict theSRM to satisfy these necessary conditions.

Definition 4.5. An effective window o is a time range t 2 o within the EPSP, where

�iðtÞ40:5�ts;tmi . (3)

This definition indicates that during the effective window o, �iðtÞ must be larger thanits half-maximal value.

Theorem 4.6. Delay-tolerant conjoining of two disparate spikes is possible with a

neuroidal unit xi employing the SRM, if a G-long effective window o can be chosen.

Proof. Proof of Theorem 4.4 applies when P ¼ �ts;tmi . The effective windows from thetwo spikes would overlap during an interval I. Thus, a threshold T can be chosen todetect the overlap value with PoTo2P. &

The part of the SRM EPSP effective for tolerant conjoining can be given with atime range defined by a pair of lower and upper bounds (see the shaded area in Fig.14).

Theorem 4.7. The region t 2 o; o ¼ ½Daxi þ ts;Dax

i þ tm� is an effective window of

�iðtÞ.

Proof. Since Daxi does not affect the shape of the EPSP, we take Dax

i ¼ 0 in thisproof. Then, for o ¼ ½ts; tm� to be an effective window, the condition in (3) must besatisfied within the region. Since the peak �ts;tmi always falls inside the region, �iðtÞ

monotonically increases and decreases at the lower and upper boundaries of theregion, respectively (see Appendix A for proof). Thus, it is sufficient to show that the

ARTICLE IN PRESS

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Parameter range for the effective window

,

,

Fig. 16. Change of �ðtÞ with ts=tm at the boundaries of the region t 2 ½ts; tm�.


condition holds when t is at the boundaries of the region. Furthermore, we can showthat the behavior of �iðt; ts; tm;Dax

i Þ when t is at the boundaries depend only on theratio ts=tm (see Appendix A). Numerical analysis of change in �iðtÞ at the boundarieswhen ts=tm is varied is given in Fig. 16. The solid line showing �iðtsÞ=�

ts;tmi indicates

that condition (3) is satisfied independent of the parameter ratio ts=tm (it is alwaysabove 0:5). However, for the dashed line showing �iðtmÞ=�

ts;tmi , the condition can only

be guaranteed when ts=tm40:1. &

This theorem can be interpreted requiring that tm needs to be long enough toinclude the rising time ts before the effective magnitude of the EPSP is reached(see Fig. 15). Assuming that the rise time ts is constant, and the membrane timeconstant tm can be varied by biological processes that modify the membraneconductance, we offer the next corollary.

Corollary 4.8. Delay-tolerant conjoining can be achieved if the membrane time

constant3 is chosen as

tm ¼ ts þ G. (4)

4.2. Phase segregation

The second requirement for temporal binding concerns the phase segregationmeasure for separating activity pertaining to different objects.

Definition 4.9. Phase segregation F is the time separation between the synchronizedactivity pertaining to two different objects represented successively.

We first employ the discrete model as in Theorem 4.4 for delay-tolerantconjoining, for asserting that tolerance windows are exclusive to each object.

Theorem 4.10. Segregation should obey F42G; to prevent crosstalk between

elementary features of different objects at a neural unit x covering the sensory sites

for these features.

3An refinement is made to the earlier paper [24] where it was proposed that tm ¼ G is sufficient.

ARTICLE IN PRESS


Proof. Let S1;S2 be the sets of spike timings pertaining two successively presentedobjects o1; o2, respectively. Assume, for t0; t1 2 S1, the earliest spike arrives att0 ¼ D1, and the latest spike arrives at t1 ¼ D1 þ G. According to Theorem 4.4, theeffect of a spike arriving at t1 will cause a constant potential until t1 þ G. Thus,the earliest t2 2 S2 can arrive is at t24D1 þ 2G. According to Definition 4.9, thesegregation is the difference between the originating times of spikes pertaining toeach object, yielding F ¼ ðt2 � D1Þ � ðt0 � D1Þ42G: &

Informally, if each spike has a G-long spread on the destination membrane, thenthe latest arriving spike at the end of the tolerance window will be effective foranother G amount. The next tolerance window cannot start until the effect from thisprevious spike has ended.

After defining requirements for segregation in the discrete model, we generalize forthe spiking model. Tolerant conjoining should apply to spikes emitted during atolerance window, but not spikes emitted in two different tolerance windows. Weagain restrict the SRM EPSP to satisfy these conditions to give a reciprocal ofTheorem 4.6.

Lemma 4.11. Delay-tolerant conjoining of separate groups of spikes pertaining to

different objects is possible at a neuroidal unit xi employing the SRM, if no effective

windows of EPSPs from spikes pertaining to different objects overlap in time.

The part of the SRM EPSP excluded from tolerant conjoining can be given with atime range.

Theorem 4.12. There is no effective window in the region of �iðtÞ outside the time range

t 2 ½Daxi ;D

axi þ 2tm�.

Proof. Since Daxi does not affect the shape of the EPSP, we take Dax

i ¼ 0 in thisproof. Then, for not finding an effective window outside o ¼ ½0; 2tm�, condition in(3) must fail outside the region o. On the left side of the region, �iðtÞ ¼ 0; to0fails the condition. Since the peak tmax 2 ½ts; tm� (see Appendix A), the EPSP ismonotonically decreasing at t ¼ 2tm. It is sufficient to show that if the condition failsat this boundary, there cannot be another effective window. Fig. 17 shows thechange in �ið2tmÞ=�

ts;tmi when ts=tm is varied. The data in the figure show that the

condition fails when ts=tmo0:5. &

Combining the SRM parameter constraints found in this proof with the one in theproof of Theorem 4.7, we obtain 0:1ots=tmo0:5. Notice that the above theorem canbe further optimized. We then give the segregation measure adopted for use with theSRM to achieve the effect depicted in Fig. 18.

Theorem 4.13. The segregation for the SRM should obey

FSRM4Gþ 2tm ¼ 3Gþ 2ts. (5)

Proof. Using t0; t1 2 S1 from proof of Theorem 4.10, the arrival time of the earliestspike t2 2 S2 should be outside any effective windows pertaining to o1. According to

ARTICLE IN PRESS

0 20 40 60 800

0.5

1

1.5

time (msec)

pote

ntia

l

Threshold

ΦΓ

∆+τs

Fig. 18. Phase segregation of SRM EPSPs.Two sets of spikes pertaining to two separate objects are

depicted. Shaded areas show the effective windows of the EPSPs that form the tolerance windows.

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Parameter range for the effective window

,

Fig. 17. Numerical analysis of change of �ð2tm; ts; tm;Daxi Þ with ts=tm.


Theorem 4.12, t24t1 þ 2tm should be satisfied in order to avoid overlaps. This yieldsFSRM ¼ ðt2 � D1Þ � ðt0 � D1Þ4t1 þ 2tm � t0 ¼ D1 þ Gþ 2tm � D1. &

Corollary 4.14. The value of the segregation parameter limits the maximal firing

frequency of the destination neuroid by fo1=FSRM:

Phase segregation or desynchronization can be implemented by having a globallyinhibitory projection [47,35,66,24], which suppresses the source units for theduration of an inhibitory time constant ti ¼ FSRM:

4.3. The spike response model

The SRM we employ is based on an integrate-and-fire (I/F) model. The SRM isequivalent to the standard I/F model with appropriate parameter selections [18]. Inthe model employed here, a synapse is modeled as a low pass filter and the membraneas an RC couple as seen in Fig. 19.

ARTICLE IN PRESS

...

Fig. 19. Circuit equivalents for the SRM components. (Left) A synapse, and (Right) the membrane.

VCCS stands for a voltage controlled current source.


The approximate time response of the membrane potential is

piðtÞ ¼ Ziðt� tðf Þi Þ þ

Xj2Gi

Xtðf Þj2Fj

wji�jiðt� tðf Þj Þ, (6)

where Zð�Þ is the refractory kernel and �ð�Þ is the excitatory synaptic kernel, given inEq. (2). The refractory kernel is defined as

ZðtÞ ¼ W expð�t=trÞHðtÞ,

where W is the magnitude of refractory effect4 and tr is the refractory time constant.Notice that there are no differential equations to integrate in using the SRM

kernels. This is because time response kernels are used instead of a system ofdifferential equations, and hence the name ‘‘response model.’’ The time responsefor the membrane potential in (6) is obtained by using the exact solution of thedifferential equations for the effects of single spikes and approximating theinteraction inside and between neurons.

4.4. State machine for continuous-time neuroids

The recruitment learning algorithms we discussed earlier are based on discretesampling times. It turns out that finding a simple way to upgrade the existingdiscrete-time algorithm is difficult. This is because in a continuous-time system thereis no fixed sampling time but a changing continuous value.

We decided to use another state machine, in addition to the one in Fig. 6(c), thatoperates on continuous parameters seen in Fig. 20. This machine determines thesampling time for the existing discrete-time recruitment algorithm. This proves to bea very elegant and simple way to upgrade the existing discrete-time model, withoutobscuring it with continuous-time parameters. This state machine detects peaks(local maxima) of neuroid’s membrane potential p according to its derivative p0,thereby providing the sampling time for the discrete-time learning algorithm (seeFig. 15). In this way, we have a simple addition to the system and we can use thepreviously defined discrete-time machine without modification. In summary, the

4Taken as unitary for the normalized potential values in this work.

ARTICLE IN PRESS

/

/

/

/

/

/

Fig. 20. Continuous-time state machine working in conjunction with the discrete-time state machine for

recruitment. Arcs indicate state transitions. The transition condition and machine output are displayed

atop the arc, separated by a slash (=). The three states are: Quiescent (Q) for no activity on potential,

Rising (R) when potential is increasing, and Plateau (P) when a local maximum is reached. Sampling time

is set by the transition R! P.


transition R! P in the continuous state machine in Fig. 20 triggers the discrete-timestate machine in Fig. 6(c).

Using a system with discrete and continuous parts together is sometimes termed ahybrid approach.

Using a state machine based on differential equations is similar to the qualitative

reasoning approach [33]. Qualitative reasoning approximates infinitely detailedphysical systems using an incomplete representation of a few selected characteristics.Differential equations are translated into a form that enables reasoning about thesystem. Discrete states can be obtained from special conditions (such as p0p0,p0 ¼ 0, and p040 in our case). Here, the p0 ¼ 0 condition, in transition R! P,indicates that the membrane potential reached a local maximum, and therefore mustbe checked against the threshold to emulate firing behavior.

5. Simulations

This section presents the results of simulation studies whose purpose is to studythe predictions of the bounds calculations in the previous section. These studiessimulate the recruitment learning task in the presence of direct/indirect convergingtopologies. The architectures studied are feedforward, multipartite random graphsthat vary in the number of layers and the amount of indirection as illustrated inFig. 21.5 Each architecture, or testbed, consists of an equal number of input areas, I i,and intermediate or middle areas, Mi, where i ¼ 1; . . . ; k, and kX2. The variablenumber of areas k allows testing the bounds calculations which are dependent on theconnection topology.

A simulated attentional controller presents objects to the input areas decomposedinto their elementary features (e.g., a blue square is decomposed into blue andsquare). Each elementary feature for a particular object is represented in a distinctinput area by a set of r ¼ 10 neuroids (replication factor discussed in Section 2.4). Topresent an object to the network, the attentional controller causes synchronousspikes to occur in all of the units representing the elementary features for that object.

5All timings indicated are simulated milliseconds, not actual time measurements.

ARTICLE IN PRESS

Fig. 21. Example testbed used to measure the effects of manipulating the tolerance and segregation

parameters. The middle areas serve to create indirect pathways with more synapses than the direct

pathways (compare the path I2 !M2 with I1 !M2). The number of input and middle areas is varied for

testing. Total axonal delays for both the direct and indirect pathways are 6ms. This is a two-layer testbed.


Note that input to a network consists only of the elementary features that wereprompted by the attentional controller. To simulate attention to more than oneobject (say, in a scene), the attentional controller presents each object to the network,segregated by the interval FSRM, calculated according to Eq. (5). Indirect pathwaysare created by having signals cross the middle areas.

Successful recruitment depends on both the timings within the testbed and thestatistical properties of projections across the areas. To test the capacity of thetestbed, which depends on its statistical properties, a control experiment isperformed by changing only the number of objects presented. In the mainexperiments, to test the timing hypotheses, the tolerance and segregation parametersare varied between runs. These experiments are repeated for different testbeds whenk 2 f2 . . . 4g. Simulation performance is evaluated by observing changes in thenetwork’s internal organization. We expect to find, at the final converging area, unitsrecruited to represent the attended objects. Each of the recruited concepts for theintermediate layers, and for the objects at the final layer, is expected to have roughlyr neuroids. The following defines the maximal capacity property of recruitment in anarea:

Definition 5.1. If all neuroid sets representing concepts contain r neuroids, and eachneuroid represents at most one concept, then the maximum capacity of an area isN=r concepts, where N is the number of units in that area of the network.

Thus, for these networks, the maximal capacity is 10 concepts, when there areN ¼ 100 neuroids in the final area and r ¼ 10.

To test the performance of the binding scheme, the results of binding errors areexamined. Binding errors cause illusory conjunctions of features belonging toseparate objects and, therefore, result in spurious concepts being recruited in thenetwork. The quantitative performance measure for a network is given by thecomposite quality of both the correct and the spurious concepts that exist at the end

ARTICLE IN PRESS


of the simulation. The relative magnitudes of these quality measures determine if areadout mechanism, such as a threshold, can distinguish the valid concepts from thespurious ones. For instance, the readout can be in the form of a winner-take-allmechanism.

We assume this is a necessary condition for the network to perform correctly.To assess the quality of representations formed for a single object that has been

presented to the simulation, we give the following definition.

Definition 5.2. Let the quality measure for an object i represented in the network be

qi ¼ci

r,

where ci is the number of units allocated for the object. The quality qi of an object i ismaximally 1, if r units are allocated to represent it, and minimally 0 if no unitsrepresent it.

In order to evaluate the set of correct or spurious concepts in a simulation,we define the overall quality of representations formed for a set of objects asfollows.

Definition 5.3. Let the overall quality of an object set O be

Q ¼1

jOj

Xi2O

qi. (7)

We can now use (7) to calculate the overall qualities Qc and Qs using the object setsOc and Os, for correct and spurious objects, respectively.

Several simulations are run to collect statistical data for each test, since both theconnections within the network, as well as the feature combinations for objectsare chosen from a uniform random distribution in each simulation. Details of themethods are given in subsequent sections for each of the specific tests. The structureof the input representations is given in Section 5.2.

5.1. Testbed details

Only one synapse is required to cross an area. Axonal delays are homogeneous,and increase linearly with distance in the testbed. Therefore, the difference intransmission time (cf. Lemma 4.3) is caused only by synaptic delays. We vary thenumber of input and middle areas, k, to create larger differences in delays. If k ¼ 2,this two-layer topology creates a two-synapse indirect pathway compared to a single-synapse direct pathway to the destination. Therefore, for a given topology thetolerance window can be chosen as

G ¼ ðk � 1Þts, (8)

since the shortest path always contains a single synapse. Inserting this into Eq. (5) weget the required segregation as

F ¼ ð3k � 1Þts. (9)

ARTICLE IN PRESS


Since we choose to manipulate tm for implementing the tolerance window G, wecombine (4) and (8) to get

tm ¼ kts. (10)

Model parameters are chosen according to timing data from visual cortex [44,34].The time it takes to cross an area is assumed to be �10ms. In turn, axonal delays areestimated to be about d ¼ 3ms according to their diameter and physical distance.This leaves ts ¼ 7ms for the synaptic rise time (including the dendritic delays), whichis a slower process than axonal transmission. Employing these parameters, whensegregation values are calculated from Eq. (9) for a two-layer topology similar to thecircuit formed by cortical areas V1, V2 and V3, we get 35ms for segregation. Thesegregation value predicts the maximum oscillation frequency in this circuit to be28Hz, which falls within the gamma-band (20–70Hz). The activity in the gammafrequency band is suggested to be used for object representations (see Section 6.1.1for more discussion on oscillatory rhythms).

For all the simulations, each area contains N ¼ 100 neuroids with the replicationfactor r ¼ 10 for representing concepts. The connection probability of two neuroidsfrom connected areas is given by p ¼

ffiffiffiffiffiffiffiffiffiffiffim=rN

p. This probability is calculated by

extending the methodology described in [70] for simple random graphs. Theparameter m stands for the amplification factor. We employ m ¼ 6 for increasing theexpectation of the set of recruited neuroids for stability reasons discussed in Section2.9. This value is determined empirically to yield satisfactory recruitment in deephierarchies. If it is increased further, it creates interference between objects andtherefore causes more spurious concepts.

Other parameters used in the simulation include the refractory reset after eachspike with a 10ms time constant. The spike threshold for the middle areas, T ¼ 1:5P,is chosen according to the proof of Theorem 4.6.

5.2. Behavior of the inputs and concepts

Inputs to the network are formed by pre-allocated sensory concepts represented bysets of r neuroids within input areas.

The sensory concepts are located in the input areas of the network. Each inputarea provides for the representation of a primitive sensory feature type. The sensoryconcepts within an area represent different values on the dimension of the specificfeature type. For instance, a feature value, such as square versus circle, will berepresented by sensory concepts in the shape input area. These sensory concepts arenamed numerically as S

ji , where j is the concept number in area I i. To model the

system as it attends to a particular object whose shape, say, is circular, attentionalcontrollers cause the circle sensory concept to be activated. When a sensory conceptis activated, each of the neuroids in the set representing the concept emits a singlesynchronous spike.

Attention to multiple objects is modeled by activating a sensory concept from eachinput area synchronously for each perceptual object. As an exception to this rule, twoconcepts need to be chosen from input area I1, since middle area M1 is only connected

ARTICLE IN PRESS


to I1, and the recruitment mechanism requires simultaneous activation by two separateconcept sets. In a multi-object scene, separate objects are attended to at different times,separated by the amount of the segregation parameter FSRM given in (9).

The total number of unique objects that can be represented using this scheme canbe calculated by n

2

� �nk�1 ; where n is the number of sensory concepts in each input

area, and k is the total number of input areas. For the simulations in this work, n ¼ 4sensory concepts are allocated in each input area I i, where i ¼ 1; . . . ; k, and 2pkp4.Thus, there are 24 and 96 possible objects to choose from, for two- and three-layertopologies, respectively.

Neuroid sets representing concepts are recruited in the middle areas upon activationof sensory concepts. A new concept is labeled according to the sensory concepts thatcaused its recruitment. For instance, the concept recruited in middle area M1 uponsimultaneous activation of the sensory concepts S0

1 and S11 is labeled as S0

1 ^ S11.

A simulation consists of presenting a sequence of multiple perceptual objects tothe network, segregated in time. At the end of the simulation, the set of conceptscreated in the network is analyzed. Correct concepts are the conjunctions of theoriginally presented sensory concepts for each perceptual object. Spurious conceptsare all the concepts created, except the correct concepts and the anticipatedintermediate concepts recruited in the middle areas.

5.3. Simulation framework

We use a Java simulator, NeuroidNet, for conducting experiments. We chooseJava because it is a flexible object-oriented low-level language [20] as contrasted toimperative high-level languages like MatLab (http://www.mathworks.com). Java isplatform independent and has a standard library, reasons which are important forsharing ideas and results.

Some distinctive features of our simulator are:

�
Uses BeanShell [43], a Java scripting environment for source-level userinteraction. This allows flexible debugging of our simulations. � Allows distributed processing for simulations using the Java RMI library [10]. � Introduces a grapher independent plotting library in Java that allows graphs to
be visualized using either MatLab or GNUPlot (http://www.gnuplot.info). Thishelps the researcher to concentrate efforts on the simulator rather than onproviding visualization. The visualization is left to the capabilities of externalprograms seamlessly launched within the simulator application.

Other tools we use for this research include but are not limited to:

�
MatLab for prototyping: Transfer function of a synapse and its effect on themembrane potential for input current in the complex frequency domain,
F ðsÞ ¼V ðsÞ

IðsÞ¼

1=tsCm

s2 þ ððtm þ ts þ RmCsÞ=ðtm þ tsÞÞsþ 1=ðtm þ tsÞ,

http://www.mathworks.com

http://www.gnuplot.info

ARTICLE IN PRESS


where ts and tm are the synaptic rise time and the membrane time constants,respectively, and Rm and Cm are the membrane resistance and capacitance,respectively, and s is the complex frequency variable. The outputs of individualspikes can be linearly summed using this transfer function for finding finalmembrane potential profile in SimuLink.
� GNUPlot and MatLab for plotting results using our Java grapher library. � HSpice for validation of electrical circuit models.
5.4. Intuitions on tolerance window parameter from the simulations

Here, we describe a simulation run to illustrate delay-tolerant conjoining in theneuroids of the network. The simulation was successful when the toleranceparameter G was selected according to the criteria proposed in (1) and accordinglyselecting the membrane time constant tm as in (4).

The three profiles in Fig. 22 show the membrane potential time profiles of aselected recruited neuroid from each middle area. This shows the progress of signalsfrom sources I i; i ¼ 1; . . . ; 3 to the destination M3:

(1)
Initially the neuroid in M1 (profile at top of figure) receives a signal from theinput area I1. After a d ¼ 3ms onset delay due to axon transmission, and ts ¼7ms synaptic rise and dendritic delay, the neuroid fires at t�15ms since themembrane potential p4T ¼ 1:5, the threshold.
(2)
A recruitment candidate in M2 (profile at middle of figure) receives a signal fromthe input area I2 with a 6ms onset delay and another signal from the recruitedneuroids in M1 after a 3ms transmission delay, at t�18ms. The cumulativeeffects of both these signals makes the neuroid fire at t�25ms. Notice that thefirst spike’s effect is not sufficient to recruit the M2 candidates, even though alocal maximum that triggers the state machine is reached on the way (see figureand the state machine for recruitment in Section 4.4).
(3)
An effect similar to M2 is obtained in M3 (profile at bottom of figure). This timethe signals originating in input area I3 arrive after a 9ms delay. However, thesignals from M2 arrive at t�28ms and neuroids are recruited and fired att�32ms.
However, there is an anomaly in M3 worth mentioning. On close examination of themembrane potential plot, one can see that the signal coming from M2 does not raisethe potential above a value higher than what has been reached by input I3. This mayseem contradictory to the tolerant conjoining described so far, which distinguishedthe effect of multiple inputs from the effect of an individual input, by using thepotential level to discriminate between them. In this case, this may indicate that theinput signal from I3 is sufficient to cause recruitment without waiting for the resultsof the computation coming from M2. The reason that recruitment and spiking arenot observed is because there is a recruitment limit in our simulation for maintainingthe stability of recruitment. Since there are already neuroids in M2, representingactive inputs in I3, no neuroids in M3 are allowed to join the recruitment. This can

ARTICLE IN PRESS

0 10 20 30 40 50 600

0.5

1

1.5

2

Pot

entia

l [V

]

Membrane potential of PeakerNeuroid #10 (in Area: M1)

0 10 20 30 40 50 600

0.5

1

1.5

2Membrane potential of PeakerNeuroid #93 (in Area: M2)

0 10 20Time [ms]

30 40 50 60-1

0

1

2


Fig. 22. Membrane potentials from a selected neuroid from each middle area, from M1 to M3, shown

from top to bottom, respectively. The resets on the membrane potential show the time of spikes emitted.

The action potentials are not depicted since they are ideal Dirac delta functions in the SRM.


be interpreted as an inhibitory effect raising the threshold. The artificial recruitmentlimit imposed by the boost-and-limit mechanism is discussed in Section 2.9. Due tothe current implementation of the limiting mechanism, if a concept is allocated inmultiple areas, then the inhibition spans globally to those areas.

5.5. Quantitative results

In the following subsections, results are given for observing the effects of differentvalues for the number of objects presented, the tolerance window, and thesegregation parameters. For each of these parameters, figures are given withsimulations on varying-size network architectures. These networks are constructedwith varying levels of indirect pathways as described in Section 5.1. An architecturewith a two-level indirect pathway is depicted in Fig. 21.

The performance measure is the overall quality of object representations given byDefinition 5.3, simply referred to as quality hereafter. In each figure, qualities forcorrect and spurious concepts are plotted while some parameter is varied. The graphplots the average quality value over a number of trials indicated on each figure. Eachdata point represents the mean quality value of 10 simulations. The errorbarsrepresent the maximum and minimum quality values from the trials.

In the figures, an x-axis value for the parameter varied may be marked by a dash-dotted vertical limit bar for the calculated bound. Other network parametersemployed are included in the figure legends and captions.

ARTICLE IN PRESS


As stated at the beginning of Section 5, a network is successful if the correctconcept quality can be distinguished from spurious concept quality via a threshold-value.

5.5.1. Network object capacity

Fig. 23 displays the overall object-representation quality as the number of objectspresented to the network is varied. Parts (a), (b), and (c) of the figure show the resultsfor a two, three, and four-layer testbed, respectively. Recall, that a k-layer testbedhas an indirect pathway of k synapses. In all cases, N ¼ 100 and r ¼ 10, so themaximal capacity of these networks is 10 (see Definition 5.1). The vertical dash-dotted line at the point where the number of objects is five indicates the point of half-maximal capacity. In all of the graphs, the solid plot shows the overall quality for thecorrect concepts while the dashed plot shows the quality for the spurious concepts.The most important feature of the graph is the threshold distinguishability of thecorrect from the spurious concepts. The error bars indicate maximum and minimumvalues over 10 runs. As long as there is no overlap in the two plots then thresholddiscriminability is guaranteed.

All three testbeds successfully represent all objects at the half-maximal capacity offive. Furthermore, the network behaves gracefully as the maximal capacity isapproached. However, as the number of indirect pathways increase, the performanceslightly degrades. The maximal capacity is achieved for topologies with a lownumber of indirect pathways, such as the ones shown here. The quality of correctconcepts is still distinguishable from spurious ones even for capacities at the maximallimit (see Fig. 23). Theoretically, the maximal capacity of the network cannot beachieved easily. As the network is populated with concept sets, fewer neuroids will beavailable for recruitment for new concepts. Thus, the probability of finding randomconnections to the few available neuroids left will be lower. Nevertheless, simulationsindicate that if the assumption about recruiting r neuroids for each final concept doesnot hold (less than r neuroids are allocated for some concepts), the network capacitycan become higher than expected.

5.5.2. Tolerance window parameter

Fig. 24 gives the quality of concepts in the network as the tolerance parameter isvaried, for simulation testbeds of two, three and four indirect layers. The tolerancelower bound of tm, indicated with the dash-dotted vertical line, is calculated by (10)for each topology. As with the results on network capacity, the method is successfulwith the architectures employed.

5.5.3. Phase segregation parameter

Fig. 25 shows the variation in the quality of concepts in the network as thesegregation parameter is varied, for simulation testbeds of two, three and fourindirect layers. The segregation lower bound F, indicated by the dash-dotted verticalline in the graph, is calculated by (9) for each topology.

Our calculated segregation limit seems to be a proper lower bound for thearchitectures tested. We expect the network to be more stable as the segregation is

ARTICLE IN PRESS

0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Objects

Qua

lity

Performance with 2 Layers over 10 Trials, τm = 14

0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Objects

Qua

lity


0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Objects

Qua

lity


Correct conceptsSpurious conceptsHalf capacity



(b)(a)

(c)

Fig. 23. Variation in concept quality as a function of the number of objects presented to the network.

Plots show the robustness on the expected capacity of the network. The dash-dotted line represents the

half-maximal capacity value for the number of objects. See Section 5.5 for reading the plots: (a) in a 2-layer

testbed (tm ¼ 14); (b) in a 3-layer testbed ðtm ¼ 21Þ; (c) in a 4-layer testbed ðtm ¼ 28Þ.


increased. However, the quality of correct concepts initially increases and thenplateaus in the four-layer testbed (see Fig. 25(c)).

6. Discussion and conclusions

6.1. Discussion

6.1.1. Maximal number of hierarchical levels possible in the brain

As the number levels in direct/indirect converging topologies increases,the minimal required segregation value also increases (see (9)). According toCorollary 4.14, segregation limits the frequency of periodic activity. This allowsusing the observed brain activity frequencies to predict maximum levels ofhierarchies that can exist.

ARTICLE IN PRESS

5 10 15 20 25 300

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Qua

lity

Performance with 2 Layers over 10 Trials, 5 Objects

10 15 20 25 30 35 40 450

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Membrane Time Constant τm



Qua

lity


20 25 30 35 40 45 50 550

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Qua

lity


Correct conceptsSpurious conceptsCalculated τm



(a) (b)

(c)

Fig. 24. Variance in concept quality as a function of the membrane time constant tm. In the simulations,

the tolerance window G is varied, implying that tm and segregation between activity F is calculated

according to Eqs. (4) and (5), respectively. The calculated operating value of tm, given by (10), is

shownwith a dash-dotted vertical line. Five objects were presented to the network. See Section 5.5 for

reading the plots: (a) in a 2-layer testbed; (b) in a 3-layer testbed; (c) in a 4-layer testbed.


Section 5 showed how the tolerance and segregation measures depend on thetopological parameters according to Eqs. (8) and (9), respectively. The segregationdepends on both the number of synapses in the longest indirect pathway k andthe synaptic rise time constant ts. These equations assume that ts and conductionvelocities are homogeneous throughout the topology. In the followingpredictions, the limits only apply on the levels of direct/indirect convergingpathways. Diverging pathways need not be synchronous and thus do not need tobe segregated.

Using Eq. (9), the connection topology formed by the visual areas 1–3 shown inFig. 3 requires F ¼ 35ms. The maximal periodic activity becomes 28Hz in this case,which falls into the gamma frequency band (20–70Hz). The number of levels thatgamma-band activity can support is limited by kp2. From this we can hypothesizethat for small hierarchies of direct/indirect topologies can only be supported locally

ARTICLE IN PRESS

0 10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Segregation Φ

Qua

lity

Performance with 2 Layers over 10 Trials,5 Objects, and τm = 14

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Segregation Φ Q

ualit

y


0 20 40 60 80 100 120 1400

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Segregation Φ

Qua

lity


Correct conceptsSpurious conceptsCalculated Φ



(a) (b)

(c)

Fig. 25. Variance in concept quality as a function of the segregation amount F. In the simulations, the

segregation F is varied, while the tolerance tm is kept constant at a value calculated according to (10). The

segregation bound of F, shown with a dash-dotted vertical bar in the graph, is calculated by (9). Five

objects were presented to the network. See Section 5.5 for reading the plots: (a) in a 2-layer testbed

(tm ¼ 14); (b) in a 3-layer testbed ðtm ¼ 21Þ; (c) in a 4-layer testbed ðtm ¼ 28Þ.


in the gamma-band frequency range. For topologies with more levels, we assumethey span larger areas of cortex and that their activity falls into the alpha frequencyband (8–12Hz). The number of hierarchical levels supported by the alpha-band iskp6. This hypothesis is consistent with gamma-band activity being observed at localstructures while alpha-band activity occurs over large structures or globally(cf. VanRullen and Koch [72]).

6.1.2. Spurious concepts

The results presented so far indicate that the amount of spurious activity in thenetwork increases with the number of indirect layers. This may raise the suspicion

ARTICLE IN PRESS


that our calculations for the tolerance and segregation parameters do not scalewell. However, the increase in the number of spurious concepts is not due to thetolerance and segregation parameters, but is an artifact of the recruitment learningmethod. When recruiting concepts in a cascade, if some spurious concepts appear atone stage, they cause the recruitment of additional spurious concepts at furtherstages, based on the conjunction of the spurious concept with other legitimateconcepts.

Consider a simulation with a two-layer topology such as in Fig. 21, in which twohypothetical perceptual objects are presented at separate times. The first object isrepresented by the sensory concept conjunction S0

1 ^ S11 ^ S0

2, and the secondby S1

1 ^ S21 ^ S0

2 (each have two concepts in input area I1 and one concept in inputarea I2). Fig. 26 gives the synaptic activities and the total membrane potential for aneuroid that belongs to the set of a correct concept S0

1 ^ S11 ^ S0

2 in area M2. Thefigure shows three incoming synapses from neuroids in areas I2 and M1. Thesesynaptic neuroids belong to the sets of concepts S0

1 ^ S11, and S0

2. Note that there aretwo synapses from the set for S0

2, but only a single synapse from the set for S01 ^ S1

1.The neuroid is recruited for S0

1 ^ S11 ^ S0

2 at t ’ 25, when three synchronous spikes,received from synapses 0, 4, and 11, approach their maxima. The sudden increase inthe synaptic potentials reflects the change in the weight values. The sudden decreasein the membrane potential at this time (bottom plot in figure, time t ’ 25), however,is due to the reset after the neuroid fires.

When the second object S11 ^ S2

1 ^ S02 is presented to the network at t ’ 55, we

expect this neuroid to stay silent. However, the neuroid produces an action potentialseen from the reset at t ’ 70. The reason for this erroneous action is that thecombined effect from two strong synapses from set S0

2 produces enough activation tocross the threshold calculated for the recruited concept. The culprit is both thelearning algorithm for not weakening the synapses enough, and the unevendistribution of synapses from different concept sets. Both issues are not resolved inthis work since they are artifacts of the theory behind recruitment learning and notthe tolerance and segregation parameters. We are working on making the networkmore noise tolerant rather than tweaking the parameters to suppress this kind ofnatural outcome.

The important consequence of this erroneous activity is that the spike causedby this neuroid is going to cause more spurious effects in the downstream areas(see Fig. 27). The postsynaptic neuroid will assume that it received a spike from asynapse representing the concept S0

1 ^ S11 ^ S0

2, even when the second object does notrepresent the concept S0

1. In the simulator, recruited concepts are labeled at the timeof their recruitment according to their incoming neuroids. This may be another pointthat requires revision. Afterwards, the simulator reads the previously assigned label,rather than actually observing the neuroid’s activity. As a solution, it may be arguedthat, since the neuroid fired in phase with the second object, it should represent anintermediate concept for the second object. Another possibility is to dynamicallychange the concept to which the neuroid belongs. This implies using a moreadvanced learning algorithm that allows gradual adjustment of weights after theinitial memorization, or n-shot learning.

ARTICLE IN PRESS

0 10 20 30 40 50 60 70 80 900

1

2

3

Potential [V] Synapse Activities of PeakerNeuroid #28 (in Area: M2) with Concept: S1-1, S1-0, S2-0

Weighted potential of Synapse #0 of PeakerNeuroid #28 (in Area: M2) from PeakerNeuroid #20 (in Area: I2) Concept: S2-0

0 10 20 30 40 50 60 70 80 900

1

2

3

Weighted potential of Synapse #4 of PeakerNeuroid #28 (in Area: M2) from PeakerNeuroid #50 (in Area: I2) Concept: S2-0

0 10 20 30 40 50 60 70 80 900

1

2

3

Weighted potential of Synapse #11 of PeakerNeuroid #28 (in Area: M2) from PeakerNeuroid #16 (in Area: M1) Concept: S1-1, S1-0

0 10 20 30 40 50 60 70 80 90Time [ms]

0

2

4

6

Membrane potential of PeakerNeuroid #28 (in Area: M2)

Fig. 26. Synaptic activities (top three plots) and total membrane potential (bottom plot) of a neuroid

representing a correct concept in area M2. A sensory concept j in input area I i , introduced as Sji earlier, is

shown in the figure as Si-j. The neuroid represents the intermediate concept S01 ^ S1

1 ^ S02.


6.1.3. Implementation of the tolerance window

We suggested that lower bounds on the tolerance and segregation parameters canbe calculated for a direct/indirect connection topology. The results confirm that thenetwork performs better as the tolerance increases. Excessive stability is not desirablesince it results in a trade-off with processing speed. We prefer to have the lowesttolerance value to achieve the fastest speed without compromising networkcorrectness. For this purpose, the values chosen for tolerance seem to beappropriate, since correct concept quality values can be distinguished from spuriousones. In this respect, there even seems to be room for further optimization of thetolerance parameter.

We proposed that the membrane time constant can be dynamically adjusted(possibly by biological processes that vary the membrane resistance), to accommodate

ARTICLE IN PRESS

0 20 40 60 80 100 1200

1

2

3

Potential [V] Synapse Activities of PeakerNeuroid #28 (in Area: M3) with Concept: S1-2, S1-1, S1-0, S2-0

Synapse #4 of PeakerNeuroid #28 (in Area: M3) from PeakerNeuroid #28 (in Area: M2) Concept: S1-1, S1-0, S2-0

0 20 40 60 80 100 1200

1

2

3

Synapse #7 of PeakerNeuroid #28 (in Area: M3) from PeakerNeuroid #49 (in Area: M2) Concept: S1-2, S1-1, S2-0

0 20 40 60 80 100 1200

1

2

3

of Synapse #11 of PeakerNeuroid #28 (in Area: M3) from PeakerNeuroid #75 (in Area: M2) Concept: S1-2, S1-1, S2-0

0 20 40 60 80 100 120Time [ms]

0

2


Fig. 27. Synaptic activities (top three plots) and total membrane potential (bottom plot) of a neuroid

representing a spurious concept in area M3. This concept S01 ^ S1

1 ^ S21 ^ S0

2 is caused by the correct

concept S01 ^ S1

1 ^ S02 in Fig. 26 firing in the wrong phase. A sensory concept S

ji is written as Si-j.


the calculated tolerance window. An alternative to varying the membrane timeconstant may be the use of persistent firing by inputs occurring at separate times, forcreating an overlapping effect on the destination readout site [5,26] (see discussion inSection 3.3). Yet another alternative is by adjusting the threshold (excitability) of thedestination unit.

However, there are other approaches to the problem of variable delays. Inparticular, it was proposed that introducing synapse-specific delays and integrationtimes adopted during development can accommodate for differences in delays [48]. Ifcortical circuits can adapt to varying delays, this may solve the problem with thedirect/indirect connection topologies, as well.6

6Due to personal communication with Benjamin Rowland and comments from an anonymous referee.

ARTICLE IN PRESS


6.1.4. Implementation of segregation

The calculated segregation, however, needs to be applied at the initial source(possibly by attentional mechanisms). Therefore, feedback connections from thedestination site should inhibit the source areas for the desired segregation amount.For instance, the dense feedback connections from visual area V1 back to the lateralgeniculate nucleus (LGN) may be responsible for this kind of modulation (see thedirect/indirect connection topology in Fig. 3). However, it is difficult to assume thatthere is a direct feedback connection to the initial source in all such topologies.Instead a more complex attentional mechanism may be responsible for segregatingsignals. The segregation also predicts the maximum firing frequency in thelocal circuit.

The field of signals and systems has also contributed to the theory and applicationof timing issues in interconnected circuits. In particular, the industry for fabricatingintegrated circuits (ICs) nowadays gives high importance to timing properties ofcircuits with the need to produce faster computers. Some theory from this field mayapply to the issues we discuss in this work.

The problem of synchronizing varying-length or varying-delayed paths isespecially important in fabricating ICs. Three mainstream approaches can beidentified in the current literature as a solution to the problem [7, Chapters 9,11]. Thefirst solution is achieved by using a central global clock signal to synchronize eventsat different parts of the circuit. The clock signal controls the time when thecomputational units start processing their inputs. This requires buffering devices tostore and synchronize the inputs arriving at various times for each unit. Forstructures with varying delays between the source and destination stages,synchronization can be achieved if the clock period is made large enough to toleratethe maximally delayed signals.7 This solution is equivalent to the approach we taketo calculate the tolerance window G with (1). A major disadvantage to this approachis that even faster computations need to wait for this longer duration.

The second approach proposes using circuits without a global clock signal. Thesecircuits are called asynchronous, where each unit produces an output as it completesits computation. Here, a special effort must be made for ensuring that the varying-delay pathways do not appear. To achieve this, paths between different stages ofcomputations are shortened and unified. The major disadvantage of this approach isthat this type of fine tuning is expensive and susceptible to errors caused by noise orslight variations in component properties due to fabrication artifacts.

The third approach attempts to combine the strengths of both previousapproaches. Each interconnected processing stage consists of interacting compo-nents. Results of a computation from a stage are only transmitted to the next stageafter receiving a release signal. This approach is most interesting for our purposesbecause it is easier to model it with biological circuits. This method does not require

7The clock is assumed to control the source and destination stages of the computation. Intermediate

stages of computation between the source and destination need to be controlled by an independent and

faster clock signal or function asynchronously.

ARTICLE IN PRESS


a global clock and the nature of connections are more localized. Modeling thisapproach into our system is left as future work.

6.2. Conclusions

Our previous work proposed lower bounds for the tolerance window G and thephase segregation F parameters. Here, we improve these hypotheses in (4) and (5)and show that the SRM parameters must also obey 0:1ots=tmo0:5. We apply theseconstraints to direct/indirect converging topologies and run simulations to test theirviability. We ran simulations on networks with varying size direct/indirectconnection topologies. We tested for binding errors among multiple hypotheticalobjects presented, while the tolerance and segregation parameters were changed overa range including the predicted values. We conclude that appropriately chosentolerance and segregation parameters enable use of temporal binding for recruitmentlearning in direct/indirect connection topologies. Furthermore, using a spikingneuron model is appropriate for recruitment learning, which was originally proposedwith simpler discrete-time neuron models. A recent study is consistent with ourview [56].

We also improved the stability of recruitment with aid of a stabilizing mechanismproposed in Section 2.9. As a result, our simulations indicate that up to roughly halfof the predicted capacity can be achieved with reasonable performance. Thestatistical variance inherent in the recruitment method prevents recruiting a chain ofconcepts in cascade. This problem is especially apparent in smaller network sizes,such as we employ here with a low number of neuroids (N�102) for each area.Earlier work on the stability of the recruitment method for larger network sizes suchas N�108 [70], and under asymptotic conditions N !1 [16,17] indicatesrecruitment can be used up to four levels deep. Our stabilizing method canpotentially be applied to these larger networks.

6.3. Future work

We still need to design neural circuits that adaptively adjust the tolerance andsegregation parameters, rather than calculating and setting them to fixed valuesaccording to each topology. Since cortical circuits are known to change, toleranceand segregation should be managed dynamically according to changing conditions.For managing tolerance, it can be shown that, if only the membrane resistanceis externally manipulated to vary the membrane time constant, the desired effect canbe achieved.

Another mechanism that deserves further work is the neural circuits that may beresponsible for managing the proposed stabilizing machinery for hierarchicalrecruitment. There are a number of neural circuits that can be proposed for realizingthis boost-and-limit function. For instance, it can be proposed that global inhibitionby itself, or local lateral inhibition with noisy delays to trigger an inhibitory circuit toshut off all activity after sufficient recruitment is reached, can be used to controlrecruitment. This may also improve the robustness of recruitment. In this work we

ARTICLE IN PRESS


do not intend to propose or implement such a circuit since it increases the complexityof the system, where we need to observe other parameters closely. This boost-and-limit mechanism is solely implemented by software techniques in our simulator. Anadvantage of this mechanism is the result discussed in Section 5.4. We mark as futurework to propose neural circuits that conform to the results obtained here (also seeGunay [23], Gunay and Maida [25]).

Acknowlegments

The authors would like to thank Prashant Joshi, Ben Rowland, and theanonymous referees for valuable comments and suggestions. Cengiz Gunaygratefully acknowledges the University Doctoral Fellowship of the University ofLouisiana at Lafayette.

Appendix A. Some proofs

This section provides additional proofs supporting the arguments in Section 4.The peak of the EPSP can be found analytically by finding the point where thederivative of (2) is zero. The peak time is

tmax ¼ tstm lnðtm=tsÞ=ðtm � tsÞ þ Daxi . (A.1)

The rightmost component can be omitted by taking Daxi ¼ 0, for simplicity.

To show that tmax 2 ½ts; tm� is always true, we check the conditions tmax4ts andtmaxotm. Let

x ¼ ts=tm, (A.2)

from which it can be deduced that 0oxo1, since both ts; tm40 and tsotm. Using(A.1), for tmax4ts, we get

ln xox� 1

which is true for all values of x40. Conversely, for tmaxotm, we get

lnð1=xÞo1=x� 1

which is again true for all values of x40.Next, it is shown that the form of �iðt; ts; tm;Dax

i Þ only depends on the selection of x

at the boundaries of the effective window o ¼ ½ts; tm�. Rewriting (2) for t ¼ ts andsimplifying, we get

�iðts; ts; tm;Daxi Þ ¼

1

1� x½expð�xÞ � expð�1Þ�.

For t ¼ tm and simplifying, we get

�iðtm; ts; tm;Daxi Þ ¼

1

1� x½expð�1Þ � expð�1=xÞ�.

Both boundary expressions are functions of x given in (A.2).

ARTICLE IN PRESS


References

[1] M. Abeles, G. Hayon, D. Lehmann, Modeling compositionality by dynamic binding of synfire chains,

J. Comput. Neurosci. 17 (2004) 179–201.

[2] D. Boussaoud, L.G. Ungerleider, R. Desimone, Pathways for motion analysis—cortical connections

of the medial superior temporal and fundus of the superior temporal visual areas in the macaque,

J. Comp. Neurol. 296 (3) (1990) 462–495.

[3] A. Browne, R. Sun, Connectionist inference models, Neural Networks 14 (2001) 1331–1355.

[4] S.R. Campbell, D.L. Wang, C. Jayaprakash, Synchrony and desynchrony in integrate-and-fire

oscillators, Neural Comput. 11 (1999) 1595–1619.

[5] R. Cannon, M.E. Hasselmo, R.A. Koene, From biophysics to behavior: Catacomb2 and the design of

biologically plausible models for spatial navigation, Neuroinformatics 1 (2002) 3–42.

[6] G.A. Carpenter, S. Grossberg, A massively parallel architecture for a self-organizing neural pattern

recognition machine, Comput. Vision, Graphics Image Process. 37 (1987) 54–115.

[7] A. Chandrakasan, W.J. Bowhill, F. Fox (Eds.), Design of High-Performance Microprocessor

Circuits, IEEE Press, New Jersy, 2001.

[8] J. Diederich, Steps towards knowledge-intensive connectionist learning, in: J.A. Barnden, J. Pollack

(Eds.), Advances in Connectionist and Neural Computation Theory, vol. 1, Ablex, Norwood, NJ,

1991.

[9] M. Diesmann, M.-O. Gewaltig, A. Aertsen, Stable propagation of synchronous spiking in cortical

neural networks, Nature 402 (1999) 529–533.

[10] T.B. Downing, Developing Distributed Java Applications, Java RMI, Remote Method Invocation,

IDG Books Worldwide, Inc., 1998.

[11] A.K. Engel, P. Fries, P. Konig, M. Brecht, W. Singer, Temporal binding, binocular rivalry, and

consciousness, Conscious. Cognition 8 (2) (1999) 128–151.

[12] M.A. Fanty, Learning in structured connectionist networks. Technical Report 252, Computer Science

Department, University of Rochester, Rochester, New York, April 1988.

[13] J.A. Feldman, Dynamic connections in neural networks, Biol. Cybern. 46 (1982) 27–39.

[14] J.A. Feldman, Computational constraints on higher neural representations, in: E.L. Schwartz (Ed.),

Computational Neuroscience, System Development Foundation Benchmark Series, MIT Press,

Cambridge, MA, 1990, pp. 163–178 (Chapter 13).

[15] J. Feldman, D. Bailey, Layered hybrid connectionist models for cognitive science, in: S. Wermter,

R. Sun (Eds.), Hybrid Neural Systems, Revised Papers from a Workshop Held, December 4–5, 1998,

Denver, USA, Lecture Notes in Computer Science, vol. 1778, Springer, Berlin, 2000, pp. 14–27.

[16] A.V. Gerbessiotis, Topics in parallel and distributed computation, Ph.D. Thesis, The Division of

Applied Sciences, Harvard University, Cambridge, MA, January 1993.

[17] A.V. Gerbessiotis, Random graphs in a neural computation model, Int. J. Comput. Math. 80 (2003)

689–707.

[18] W. Gerstner, Spiking neurons, in: W. Maass, C.M. Bishop (Eds.), Pulsed Neural Networks, MIT

Press, Cambridge, MA, 1999, pp. 3–54 (Chapter 1).

[19] W. Gerstner, A framework for spiking neuron models: the spike response model, in: F. Moss,

S. Gielen (Eds.), The Handbook of Biological Physics, vol. 4, North-Holland, Amsterdam, 2001,

pp. 469–516 (Chapter 12).

[20] J. Gosling, B. Joy, G. Steele, G. Bracha, The Java Language Specification, second ed., Sun

Microsystems, Inc., 2000.

[21] C.M. Gray, P. Konig, A.K. Engel, W. Singer, Oscillatory responses in cat visual cortex exhibit

inter-columnar synchronization which reflects global stimulus properties, Nature 338 (1989)

334–337.

[22] J. Grutzendler, N. Kasthuri, W.B. Gan, Long-term dendritic spine stability in the adult cortex,

Nature 420 (2002) 812–816.

[23] C. Gunay, Hierarchical learning of conjunctive concepts in spiking neural networks, Ph.D. Thesis,

Center for Advanced Computer Studies, University of Louisiana at Lafayette, Lafayette, LA 70504-

4330, USA, December 2003.

ARTICLE IN PRESS


[24] C. Gunay, A.S. Maida, Temporal binding as an inducer for connectionist recruitment learning over

delayed lines, Neural Networks 16 (5–6) (2003) 593–600.

[25] C. Gunay, A.S. Maida, A stochastic population approach to the problem of stable recruitment

hierarchies in spiking neural networks, Biol. Cyber. 2005, submitted for publication.

[26] C. Gunay, A.S. Maida, Using temporal binding for robust connectionist recruitment learning over

delayed lines, Technical Report TR-2003-2-1, Center for Advanced Computer Studies, University of

Louisiana at Lafayette, Lafayette, LA 70504-4330, USA, 2003.

[27] J.E. Hummel, I. Biederman, Dynamic binding in a neural network for shape recognition, Psychol.

Rev. 99 (3) (1992) 480–517.

[28] O. Jensen, J.E. Lisman, Novel lists of 7� 2 known items can be reliably stored in an oscillatory short-

term memory network: interaction with long-term memory, Learn. Memory 3 (1996) 257–263.

[29] A. Knoblauch, G. Palm, Pattern separation and synchronization in spiking associative memories and

visual areas, Neural Networks 14 (2001) 763–780.

[30] A. Knoblauch, G. Palm, Spiking associative memory and scene segmentation by synchronization of

cortical activity, in: S. Wermter, J. Austin, D.J. Willshaw (Eds.), Emergent Neural Computational

Architectures Based on Neuroscience—Towards Neuroscience-Inspired Computing, Lecture Notes in

Computer Science, vol. 2036, Springer, Berlin, 2001, pp. 407–427.

[31] C. Koch, M. Rapp, I. Segev, A brief history of time (constants), Cerebral Cortex 6 (1996)

93–103.

[32] P. Konig, A.K. Engel, Correlated firing in sensory-motor systems, Curr. Opin. Neurobiol. 5 (1995)

511–519.

[33] B.J. Kuipers, Qualitative Reasoning: Modeling and Simulation with Incomplete Knowledge, MIT

Press, Cambridge, MA, 1994.

[34] V.A.F. Lamme, P.R. Roelfsema, The distinct modes of vision offered by feedforward and recurrent

processing, Trends Neurosci. 23 (2000) 571–579.

[35] J.E. Lisman, M.A.P. Idiart, Storage of 7� 2 short-term memories in oscillatory subcycles, Science

267 (1995) 1512–1514.

[36] N. Littlestone, Redundant noisy attributes, attribute errors, and linear threshold learning using

Winnow, in: Proceedings of the Fourth Annual Workshop on Computer Learning Theory, Morgan

Kaufmann, San Meteo, CA, 1991, pp. 147–156.

[37] M. Livingstone, D. Hubel, Segregation of form, color, movement, and depth—anatomy, physiology,

and perception, Science 240 (4853) (1988) 740–749.

[38] D.R. Mani, L. Shastri, A connectionist solution to the multiple instantiation problem using temporal

synchrony, in: Proceedings of the Fourteenth Conference of the Cognitive Science Society,

Bloomington, Indiana, July 1992, pp. 974–979.

[39] D. Marr, Simple memory: a theory for archicortex, Philos. Trans. R. Soc. London B 262 (1971)

23–81.

[40] M. Matsuzaki, N. Honkura, G.C.R. Ellis-Davies, H. Kasai, Structural basis of long-term

potentiation in single dendritic spines, Nature 429 (2004) 761–766.

[41] B.L. McNaughton, R.G.M. Morris, Hippocampal synaptic enhancement and information storage

within a distributed memory system, Trends Neurosci. 10 (10) (1987) 408–415.

[42] A. Newell, Unified Theories of Cognition, Harvard University Press, Cambridge, MA, 1990.

[43] P. Niemeyer, BeanShell User Manual, 2001.

[44] L.G. Nowak, J. Bullier, The timing of information transfer in the visual system, in: K.S. Rockland,

J.H. Kaas, A. Peters (Eds.), Cerebral Cortex, vol. 12, Kluwer, New York, 1997, pp. 205–240.

[45] R. Ritz, T.J. Sejnowski, Synchronous oscillatory activity in sensory systems: new vistas on

mechanisms, Curr. Opin. Neurobiol. 7 (1997) 536–546.

[46] F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms,

Spartan Books, Washington, DC, 1961.

[47] T.B. Schillen, P. Konig, Binding by temporal structure in multiple feature domains of an oscillatory

neuronal network, Biol. Cybern. 70 (1994) 397–405.

[48] W. Senn, M. Schneider, B. Ruf, Activity-dependent development of axonal and dendritic delays, or,

why synaptic transmission should be unreliable, Neural Comput. 14 (2002) 583–619.

ARTICLE IN PRESS


[49] L. Shastri, Semantic Networks: An Evidential Formalization and its Connectionist Realization,

Research Notes in Artificial Intelligence, Morgan Kaufmann Publishers, San Meteo, CA, 1988.

[50] L. Shastri, A computational model of tractable reasoning—taking inspiration from cognition, in:

Proceedings of IJCAI-93, the Thirteenth International Joint Conference on Artificial Intelligence,

France, August 1993, pp. 202–207.

[51] L. Shastri, A model of rapid memory formation in the hippocampal system, in: Proceedings 19th

Annual Conference Cognitive Science Society, Stanford, CA, August 1997, pp. 680–685.

[52] L. Shastri, Advances in SHRUTI—A neurally motivated model of relational knowledge

representation and rapid inference using temporal synchrony, Appl. Intell. 11 (1999) 79–108.

[53] L. Shastri, Recruitment of binding and binding-error detector circuits via long-term potentiation,

Neurocomputing 26–7 (1999) 865–874.

[54] L. Shastri, Types and quantifiers in SHRUTI: a connectionist model of rapid reasoning and relational

processing, in: S. Wermter, R. Sun (Eds.), Hybrid Neural Systems, Revised Papers from a Workshop

Held December 4–5, 1998, Denver, USA, Lecture Notes in Computer Science, vol. 1778, Springer,

Berlin, 2000, pp. 28–45.

[55] L. Shastri, A computational model of episodic memory formation in the hippocampal system,

Neurocomputing 38–40 (2001) 889–897.

[56] L. Shastri, Biological grounding of recruitment learning and vicinal algorithms in long-term

potentiation, in: S. Wermter, J. Austin, D.J. Willshaw (Eds.), Emergent Neural Computational

Architectures Based on Neuroscience—Towards Neuroscience-Inspired Computing, vol. 2036,

Lecture Notes in Computer Science, Springer, Berlin, 2001, pp. 348–367.

[57] L. Shastri, A computationally efficient abstraction of long-term potentiation, Neurocomputing 44–46

(2002) 33–41.

[58] L. Shastri, Episodic memory and cortico-hippocampal interactions, Trends Cogn. Sci. 6 (4) (2002)

162–168.

[59] L. Shastri, V. Ajjanagadde, From simple associations to systematic reasoning: a connectionist

representation of rules, variables, and dynamic bindings using temporal synchrony, Behav. Brain Sci.

16 (3) (1993) 417–451.

[60] R.M. Shiffrin, W. Schneider, Controlled and automatic human information processing: II, Perceptual

learning, automatic attending, and a general theory, Psychol. Rev. 84 (1977) 127–190.

[61] W. Singer, Time as coding space in neocortical processing: a hypothesis, in: M.S. Gazzaniga (Ed.),

The Cognitive Neurosciences, MIT Press, Cambridge, MA, 1995, pp. 91–104 (Chapter 6).

[62] W. Singer, C.M. Gray, Visual feature integration and the temporal correlation hypothesis, Ann. Rev.

Neurosci. 18 (1995) 555–586.

[63] J. Sougne, Connectionism and the problem of multiple instantiation, Trends Cogn. Sci. 2 (1998)

183–189.

[64] J. Sougne, Period doubling as a means of representing multiply instantiated entities, in: Proceedings

of the Twentieth Annual Conference of the Cognitive Science Society, Lawrence Erbaum Associates,

Mahwah, NJ, 1998, pp. 1007–1012.

[65] J. Sougne, R.M. French, A neurobiologically inspired model of working memory based on neuronal

synchrony and rythmicity, in: J.A. Bullinaria, D.W. Glasspool, G. Houghton (Eds.), Proceedings of

the Fourth Neural Computation and Psychology Workshop: Connectionist Representations,

Springer, London, 1997, pp. 155–167.

[66] D. Terman, D. Wang, Global competition and local cooperation in a network of neural oscillators,

Physica D 81 (1995) 148–176.

[67] A.M. Treisman, The binding problem, Curr. Opin. Neurobiol. 6 (1996) 171–178.

[68] L.G. Valiant, A theory of the learnable, Commun. ACM 27 (11) (1984) 1134–1142.

[69] L.G. Valiant, Functionality in neural nets, in: Proceedings of the 7th National Conference on

Artificial Intelligence, AAAI, Morgan Kaufmann, San Meteo, CA, 1988, pp. 629–634.

[70] L.G. Valiant, Circuits of the Mind, Oxford University Press, Oxford, 1994.

[71] L.G. Valiant, A neuroidal architecture for cognitive computation, J. ACM 47 (5) (2000) 854–882.

[72] R. VanRullen, C. Koch, Is perception discrete or continuous?, Trends Cogn. Sci. 7 (5) (2003)

207–213.

ARTICLE IN PRESS


[73] C. von der Malsburg, The correlation theory of brain function, in: E. Domany, J.L. van Hemmen, K.

Schulten (Ed.), Models of Neural Networks, vol. 2, Physics of Neural Networks, Chapter 2, Springer,

New York, 1994, pp. 95–120, (Originally appeared as a Technical Report at the Max-Planck Institute

for Biophysical Chemistry, Gottingen, 1981).

[74] C. von der Malsburg, Binding in models of perception and brain function, Curr. Opin. Neurobiol. 5

(1995) 520–526.

[75] C. von der Malsburg, W. Schneider, A neural cocktail-party processor, Biol. Cybern. 54 (1) (1986)

29–40.

[78] W.A. Wickelgren, Chunking and consolidation: a theoretical synthesis of semantic networks.

Configuring in conditioning, S-R versus cognitive learning, normal forgetting, the amnestic

syndrome, and the hippocampal arousal system, Psychol. Rev. 86 (1979) 44–60.

Cengiz Gunay obtained his Bachelor of Engineering degree in Electronics and

Telecommunications from the Istanbul Technical University (ITU) in 1998. He

obtained his Master of Science degree in Computer Science from the University

of Louisiana at Lafayette in 2000. He completed his Doctor of Philosophy in

Computer Science from the same institution in 2003. He is currently a

postdoctoral fellow at the Department of Biology at Emory University working

on multicompartmental models of globus pallidus neurons.

Anthony S. Maida received BA in Mathematics, MS in Computer Science, and

PhD in Psychology at the State University of New York at Buffalo. He received

postdoctoral training at Brown University and the University of California at

Berkeley. He is Associate Professor at the Center for Advanced Computer Studies

and Institute of Cognitive Science, University of Louisiana at Lafayette. His

research is in biologically inspired artificial intelligence and brain simulation.

Documents

Using temporal binding for hierarchical recruitment of conjunctive concepts over delayed lines