Mismatch reduction through dendritic nonlinearitiesin a 2D silicon dendritic neuron array

Yingxue Wang and Shih-Chii Liu

Institute of NeuroinformaticsUniversity of Zurich and ETH Zurich

Winterthurerstrasse 190, CH-8057 Zurich, SwitzerlandEmail: yingxue,shih@ini.phys.ethz.ch

Abstract— This paper describes a novel 2D programmable den-dritic neuron array consisting of a 3x32 dendritic compartmentarray and a 1x32 somatic compartment array. Each dendriticcompartment contains two types of regenerative nonlinearities:an NMDA nonlinearity and a dendritic spike nonlinearity. Thechip supports the programmability of local synaptic weightsand the configuration of dendritic morphology for individualneurons through the address-event representation protocol. Witha novel local cable circuit between neighboring compartments,different dendritic morphologies can be constructed. From resultsmeasured on a chip fabricated in a 4-metal, 2-poly, 0.35µmCMOS technology, we show one instance of how dendritic non-linearities can contribute to neuronal computation: the dendriticspike mechanism dynamically reduces the mismatch-inducedcoefficient of variation of the somatic response amplitude fromapproximately 40% to 3.5%.

I. INTRODUCTION

Custom aVLSI (analog Very-Large-Scale-Integrated) spik-ing neuronal networks are of great interest because they can beuseful for exploring the role of real-time event-driven brain-like architectures in computation [1]–[3]. Hitherto, most ofthese networks implement point neuron models. However, withaccumulating experimental evidence that a wide variety ofvoltage-dependent ion channels exist in dendrites [4], it isclear that a neuron with dendrites can be computationally morepowerful than a simple point processing unit.

In the early aVLSI neuron designs, dendrites are modelledas passive spatial cables using switched-capacitor circuits [6],[7]. More recently, implementations of dendrites with activeion channels are presented [8]–[10]. Our previous imple-mention of an aVLSI neuron with a two-dimensional (2D)dendritic morphology include dendritic circuits that modeltwo widely studied classes of regenerative event mechanisms:one that is supported by synaptically activated N-methyl-D-aspartate (NMDA) channels, and the second that is mediatedby classical voltage-gated K+, Na+, and Ca2+ channels [11].Using this prototype, we demonstrate that the spatio-temporaldistribution of synaptic inputs can activate a range of pro-cessing modes on the dendrite ranging from linear integrationmode to coincidence detection.

This paper presents a new 2D programmable dendriticneuronal array design. The array is composed of a 2D dendriticcompartment array and an 1D somatic compartment array.

Each dendritic compartment has excitatory and inhibitorysynapses with global bias-controlled synaptic weights, exci-tatory synapses with locally programmable weights throughdigital-to-analog converters (DACs), circuits for two types ofregenerative event mechanisms, and a new local cable circuit.The local cable circuits allow the programmability of con-nections between neighboring compartments to form differentdendritic morphologies. Moreover, the connections betweencertain dendritic compartments and somatic compartments canalso be set so that one can trade-off between the complexity ofthe dendritic structure of a neuron and the number of neuronsin the network. The chip also includes circuits that modelthe backpropagation of somatic action potentials which havebeen observed in the nervous system. All measurements inthis paper are obtained from a chip fabricated in a 4-metal,2-poly, 0.35µm CMOS technology. This paper also describesan experiment which demonstrates how the active dendriticmechanisms can help to reduce mismatch in the circuits.

II. CHIP ARCHITECTURE WITH ACTIVE DENDRITES

The chip layout and architecture are illustrated in Fig. 1.Each dendritic compartment on the chip has 4 excitatoryAMPA synapses, 2 excitatory AMPA-NMDA synapses (inwhich an AMPA synapse is associated with a NMDA synapse),2 inhibitory GABA synapses, a voltage-gated dendritic spikegenerating circuit, and a local cable circuit (Fig. 2). Two ofthe four excitatory AMPA synapses in one compartment have asynaptic weight which is set by a global parameter. Each of theremaining two AMPA synapses includes two 5-bit local DACswhich store the local synaptic weight. Each of the two AMPA-NMDA synapses consists of two components: An AMPAsynapse whose synaptic weight can be programmed through 2global DACs and a NMDA synapse. The GABA synapses arecontrolled by global bias settings. The synaptic componentsand the dendritic spike generation circuit are based on thecurrent-mode low-pass filter (LPF) blocks initially describedin [8].

The fully bidrectional programmable connections betweenneighboring dendritic compartments allow us to constructdendritic structures with up to 96 compartments and of dif-ferent morphologies (Fig. 1(b)). The connections between thedendritic compartments in the last row of the dendritic array

978-1-4244-9472-9/11/$26.00 ©2011 IEEE 677

and the somatic compartments can also be programmed. Inthis design, one can tradeoff between the complexity of thedendritic structure for a neuron and the number of neuronswithin the network depending on the purpose of study. Anetwork can be configured so that it consists of a few neuronswith complex dendritic structures or a larger number ofneurons with simple structures. The backpropagation of actionpotentials from each somatic compartment to the dendriticcompartment directly above it can also be enabled througha programmable connection.

To handle the spike communication between neurons, a2D address-event representation (AER) decoder and encodercircuitry is built into the chip. The dendritic morphology andthe AMPA DACs circuits are also programmed through theAER protocol. The current and voltage values at predefinedcircuit nodes on the chip can be read out through a 2D scanner.The scanner and DAC circuits are previously described in [12].

Dendritic neuron array

Point neuron array

Dendritic compartment

X-DecoderX-Arbiter

Y-D

ecod

erY

-Arb

iter

X-Scanner

Y-S

cann

er

AERBlock DACs

(a)

X-ArbiterX-Arbiter

Y-A

rbite

r

X-Decoder

Y-D

ecod

er

AERBlockAER

BlockReqAck

Synapse Address

Syna

pse

Add

ress

Output Neuron Address

Out

put N

euro

n A

ddre

ss

ReqoutAckout

GlobDACs

(b)

Fig. 1. Architecture of the dendritic neuron array. (a) Layout of thechip shows both the 2D dendritic neuron array described in this paper andan additional 2D point neuron array whose results will not be describedhere. The inset shows the layout of a dendritic compartment with a size of87.8 µm by 395.35 µm in a 4-metal, 2-poly, 0.35µm CMOS technology.(b) Architecture of the dendritic neuron array. This array is composed of3x32 dendritic compartments (C) and 1x32 somatic compartments (S). Theconnections between neighboring dendritic compartments are bidirectionallyprogrammable (in black arrows). The connections between the somaticcompartments and the dendritic compartments directly above can also beprogrammed (in gray arrows). The 2D AER decoder and encoder blocks areused for spike communication and a 2D scanner (not shown) is used to monitorthe responses from predefined circuit nodes.

III. CIRCUIT COMPONENTS

Measurements of the responses and their mismatch variationfrom individual circuit blocks are quantified next. The parame-ters of the circuit responses are purposely tuned to biologicallyplausible values. The detailed equations for the componentscan be found in [8], [11]. These parameters are also used forthe experiment in Section IV.

AMPA Synapse: The three types of AMPA synaptic circuitscorrespond to whether their synaptic weights are biased bya global bias, two local current-mode DACs, or two globalcurrent-mode DACs. The excitatory postsynaptic membraneresponse (EPSP) due to the integration of the synaptic currentis quantified for the 3 different types.

All three types of AMPA synapses are calibrated so thatthey generate similar mean EPSP responses. The mean peakEPSP response for these synapses (τexc = 2.820 V and Wexc =2.156 V) are: 23.41nA (CV= 22.30%) for the AMPA synapsewith global bias; 25.47 nA (CV=24.18%) for the AMPAsynapse with local DACs; and 24.32 nA (CV=22.23%) forthe AMPA synapse with global DACs. For local and globalDAC synapses, data is taken for a DAC value of 62. The CVsare very similar because these circuits are the same except forthe additional DAC components.

NMDA Synapse: The NMDA synapses are activated whenthe local dendritic membrane voltage exceeds a thresholdVThnmda at the time of synaptic stimulation. The second-orderdynamics of the NMDA current In(t) is approximated bytwo first-order current equations that describe the rise and falldynamics separately. The equations are based on the same LPFcircuit (highlighted in dark gray in Fig. 2(b)), whose dynamicsIn(t) in response to an arbitrary input Iin(t) follows:

τndIn(t)

dt= In(t)

(Iin(t)

Iτn− 1

)(1)

where τn = CnUTκIτn

is the time constant of the NMDA current.The NMDA responses are activated and quantified by

stimulating AMPA synapses (see Fig. 3). Averaged across96 NMDA synapses, the peak In current is 19.67 nA(CV=9.53%) with mean rise time constants, τNMDA,rise of8.10ms (CV=2.9%) and fall time constants, τNMDA,decay of15.40 ms (CV= 14.69%). (τnmda = 2.929 V, Wnmda = 2.329V, Rnmda = 0.176 V, and Thnmda = 0.17 V).

Dendritic Spike Circuit: Biological dendritic spikes canbe evoked by voltage-dependent ion channels. A dendriticspike is generated in the aVLSI circuit through two currentswith complementary dynamics: A hyperpolarizing current IH ,which models the response of slowly activating K+ channels,and a depolarizing current ID, which mimics the responseof Na+ or Ca2+ channels. The current ID comes from thesubtraction of IH from a constant current [8]. Similar to theNMDA synapses, a threshold current set by VWdsp controlsthe voltage dependency of the dendritic spike generation(Fig. 2(b)).

The measured mean peak IH is 188.45nA (CV=8.31%),with mean τIH ,rise of 3.6ms and τIH ,decay of 3.7ms. These

678

(a)

NMDA

DENDRITIC MEMBRANE & CABLE

SUMMATION

V

Pulse

L

V

V

ID

D

S

τ

L

λ λ

I

V

V

V

II

I

λλ

R T B

I I I I

H

Hden

S

in

dleak

denden

den den den den

l r t b

τ

W

AMPA

I

CI

exc

exc

τaa

a

3/10

3/10

2/1

3/5 1.2/0.6

3/3

Th

τ

W

I

R

IV M10

CI

M6

M3

M4

M5

M7

M8

M9

pw M1

M2

nmda

n

nmda

nmdaden

nmda

τn

in

n

n n

3/3

3/3

3/0.6

2.5/1.8 2/1

3/5

3/10

3/10

3/5

5/5

3/5

τ

Th

RW

V

V

DEN_SPIKE

C

I

pwd

dsp

dsp dsp

τH

H

H

D

dsp 3/3

3/1

3/3

1.2/0.6

3/53/5

3/5 3/10

3/10

3/10 3/10

3/10

5/55/5

5/5

10/10 10/10 10/10 10/10

5/5

(b)

Fig. 2. Architecture of a dendritic compartment. (a) Components within adendritic compartment: 4 excitatory AMPA synapses, 2 excitatory AMPA-NMDA synapses, 2 inhibitory GABA synapses, a voltage-gated dendriticspike generating circuit, and a local cable circuit. (b) Simplified circuitschematics of the components in a dendritic compartment. The top leftshows the circuit for the AMPA channel. The top right schematic shows theNMDA channel circuit. The middle left schematic shows the circuit for thesummation of currents from individual synapses, dendritic spike current, andneighboring compartments. The middle right schematic shows the dendriticspike generation circuit. When the dendritic membrane potential exceedsthe threshold, a pulse is generated and extended through a pulse extendercircuit (in light gray). The pulse width determines the duration of the risingdynamics of the hyperpolarizing current. The corresponding depolarizingcurrent is obtained by subtracting the hyperpolarizing current from the steady-state currrent, IH (see main text). The bottom schematic shows the circuitfor integrating the currents on the dendritic membrane and for copying theintegrated current to neighboring compartments. The synapse and dendriticmembrane circuits are based on the LFP circuits described in [8]. The sizeof each transistor (W/L) is labelled with the unit of µm/µm.

parameters are first obtained for individual circuits over 20trials, and then averaged across all 96 dendritic spike circuits.

(τdsp = 2.882 V, Wdsp = 0.457 V, Rdsp = 0.219 V and Thdsp= 0.396 V).

Dendritic Membrane Circuit: The dendritic membranecircuit shown in Fig. 2(b) follows the following dynamics:

τdendIden(t)

dt= −Iden(t) +

IdleakIdleak + IH

Iin(t) (2)

where τden = CUTκ(Idleak+IH) . This circuit is based on a prior

circuit in [8]. The dendritic membrane time constant is setto a mean value of 10.4 ms (CV=17.44%) across all 96compartments.

0 20 40 60 80 1000

100

200

300

Den

spI

(

nA)

Time (ms)

0

100

200

I (n

A)

0

10

20

I (n

A)

0

20

40

I (n

A)

an

Hde

n

Fig. 3. Circuit responses of individual dendritic components. From top tobottom, the panels show the response traces of the AMPA synaptic currentIa, NMDA synaptic current In, dendritic spike hyperpolarizing current IH ,and the membrane response of a dendritic spike Iden. Each response tracecorresponds to measurements which are first averaged for individual synapsesover 20 trials for all currents except for In (50 trials). Their mean (in black)and std (in gray) values are then calculated across the same type of circuitsin the dendritic array. The black vertical bar on the top of the figure indicatesthe time of the input stimulation.

IV. EXPERIMENT

The next experiment demonstrates a possible advantageof active nonlinearities in neuronal computation, that is,dynamical mismatch reduction. Mismatch-induced responsevariations will accumulate when the different circuit compo-nents are combined in a multi-compartment neuron. To studythe influence of dendritic spikes on the mismatch inducedresponse variation, the variation is quantified from recordedresponses of neurons with a 3-compartment dendritic branch(Fig. 4) under three different conditions: (1) when both activemechanisms are blocked; (2) when only NMDA channelsare unblocked; and (3) when both active mechanisms areunblocked.

The distal compartment C0 is first stimulated so that nosomatic action potential is generated under conditions (1) and(2). In this case, we can characterize the response variationaccumulated in the course of passive signal propagation fromC0 to the soma. As shown in Table I, the accumulatedmismatch leads to a large variation in both the peak somatic

679

0 100 200Time(ms)

0 100 200Time(ms)

0

150

300

Den

driti

c m

embr

ane

valu

e (n

A)

0

150

300

0 100 2000

150

300

valu

e (n

A)

Time(ms)

0

150

300S

omat

ic m

embr

ane

dt

Only AMPA Unblock NMDA Unblock NMDA and DenSp

Fig. 4. Dynamical variance measurements. The dendritic architecture isprogrammed to form a population of 32 neurons, each with a dendriticbranch consisting of 3 compartments. 4 AMPA synapses and 2 AMPA-NMDAsynapses of either C0 or C2 compartments are stimulated with an inter-spikeinterval dt of 0.1 ms. The figure shows the response traces when C0 isstimulated. The columns represent the three different conditions (see text).Each row corresponds to responses from compartments C0, C1, C2, and thesoma respectively. Each trace plots the mean and std of responses across 32neurons. The parameter settings for results in this figure and Fig. 3 are asfollows: Vth = 0.195 V, Vleak = 2.884 V, Vrefr = 2.761 V.

TABLE IMEASURES QUANTIFYING THE AVERAGE SOMATIC RESPONSE ACROSS 32NEURONS FOR THE EXPERIMENT IN FIG. 4. Vmp AND CVVmp DESCRIBE

THE MEAN AND CV OF PEAK MEMBRANE RESPONSES ACROSS THE

ARRAY, AND δt DESCRIBES THE STANDARD DEVIATION OF THE PEAK

RESPONSE TIME OR RESPONSE JITTER TIME ACROSS THE ARRAY.

Stim Vmp CVVmp δtComp (nA) (%) (ms)

Blocked both C0 8.44 40.06 2.7C2 275.07 9.85 1.7

Unblocked NMDA C0 10.22 56.96 10.1C2 277.53 8.62 1.6

Unblocked both C0 287.15 17.81 1.5C2 306.68 3.53 0.1

membrane response (CVVmp ) and the peak response time (δt).By comparison, under condition (3), the dendritic spike whichis generated within each compartment helps to reduce theaccumulation of the response variation from the earlier stagesthus resulting in smaller CVVmp and δt values.

These results suggest that dendritic spikes can help to reducethe accumulation of response variations across dendritic com-partments thus allowing timing information to be conveyedmore precisely along the dendritic branch even in the presenceof systematic mismatch.

V. CONCLUSION

This paper describes a 2D aVLSI dendritic neuron arraywhich includes circuits that implement two widely studiedclasses of regenerative event mechanisms in biological den-

drites: one mediated by NMDA channels and the other byclassical voltage-gated channels. The chip fabric supports theonline configuration of different dendritic architectures for theneurons through a novel bidirectionally programmable cablecircuit. In addition, the programmability of local synapticweights of AMPA synapses through DACs allows users toperform mismatch compensation or online learning tasks. Thischip offers the possibility to investigate the contribution ofdendritic structure to computation both within a single neuronand a small network of neurons.

An example of how active dendrites can contribute tothe dynamical reduction of system mismatch is shown here.This chip also serves as a simulation platform for exploringother questions regarding neuronal computations, such as thefunction of active dendrites in spatio-temporal processing, andthe computational principles that govern dendritic processing.

VI. ACKNOWLEDGEMENTS

We acknowledge M. Bernet for assistance in initial chiptesting; A. Fco. Jimenez Fernandez, R. Berner, and T. Del-bruck for support with the testing setup. We thank membersof the Institute of Neuroinformatics, particularly A. Whatley,who was involved in the development of the PCI-AER board,of its drivers, and software library components, and D. Muirwho developed the spike tool box. This work was supportedpartially by the ETH Research Grant TH-20/04-2.

REFERENCES

[1] R. Silver, K. Boahen, S. Grillner, N. Kopell, and K. L. Olsen, “Neu-rotech for neuroscience: Unifying concepts, organizing principles, andemerging tools,” Journal of Neuroscience, vol. 27, pp. 11807-11819,2007.

[2] G. Indiveri, S-C. Liu, T. Delbruck, and R. J. Douglas, “Neuromorphicsystems,” Encyclopedia of Neuroscience, L. e. i. c. Squire, AcademicPress: pp. 521-528, 2009.

[3] J. Schemmel, D. Bruderle, A. Grubl, M. Hock, K. Meier, and S. Millner,“A wafer-scale neuromorphic hardware system for large-scale neuralmodeling,” Proceedings of the 2010 IEEE International Symposium onCircuits and Systems (ISCAS), pp. 1947-1950, 2010.

[4] J. C. Magee, in Dendrites, G. Stuart, N. Spruston, M. Hausser, Eds.(Oxford Univ. Press, Oxford), pp. 139-160, 1999.

[5] I. Segev and M. London,“Untangling dendrites with quantitative mod-els,” Science, vol. 290, pp. 744-750, 2000.

[6] C. Rasche and R. J. Douglas, “Forward- and backpropagation in a silicondendrite,” IEEE Trans. on Neural Networks, vol. 12, pp. 386-393, 2001.

[7] J. G. Elias and D. P. M. Northmore, “Building silicon nervous systemswith dendritic tree neuromorphs,” in Pulsed Neural Networks, W. Maassand C. M. Bishop, Eds. Boston, MA: MIT Press, ch. 5, pp. 135-156,1999.

[8] J. V. Arthur and K. A. Boahen, “Recurrently connected silicon neuronswith active dendrites for one-shot learning,” International Joint Confer-ence on Neural Networks (IJCNN), pp. 1699-1704, 2004.

[9] E. Farquhar, D. Abramson, and P. Hasler, “A reconfigurable bidirectionalactive 2 dimensional dendrite model,” Proceedings of the 2004 IEEEInternational Symposium on Circuits and Systems (ISCAS), vol. 5, pp.313-316, 2004.

[10] P. Hasler, S. Kozoil, E. Farquhar, and A. Basu, “Transistor channeldendrites implementing HMM classifiers,” Proceedings of the 2007IEEE International Symposium on Circuits and Systems (ISCAS), pp.3359-3362, 2007

[11] Y-X. Wang and S. C. Liu, “Processing of spatiotemporal patterns inactive dendrites,” Neural Comp., vol. 22, pp 2086-2112, 2010.

[12] M. Oster, Y-X. Wang, R. Douglas, and S. C. Liu, “Quantificationof spike-based winner-take-all VLSI network,” IEEE Transactions ofCircuits and Systems I, vol. 55, pp. 3160-3169, 2008.

680