Neural Process LettDOI 10.1007/s11063-012-9274-5

Modular Neural Tile Architecture for CompactEmbedded Hardware Spiking Neural Network

Sandeep Pande · Fearghal Morgan · Seamus Cawley · Tom Bruintjes ·Gerard Smit · Brian McGinley · Snaider Carrillo · Jim Harkin · Liam McDaid

© Springer Science+Business Media New York 2012

Abstract Biologically-inspired packet switched network on chip (NoC) based hardwarespiking neural network (SNN) architectures have been proposed as an embedded comput-ing platform for classification, estimation and control applications. Storage of large synapticconnectivity (SNN topology) information in SNNs require large distributed on-chip mem-ory, which poses serious challenges for compact hardware implementation of such archi-tectures. Based on the structured neural organisation observed in human brain, a modularneural networks (MNN) design strategy partitions complex application tasks into smallersubtasks executing on distinct neural network modules, and integrates intermediate outputsin higher level functions. This paper proposes a hardware modular neural tile (MNT) archi-tecture that reduces the SNN topology memory requirement of NoC-based hardware SNNsby using a combination of fixed and configurable synaptic connections. The proposed MNTcontains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topol-ogy NoC communication infrastructure. The SNN topology memory requirement is 50 % ofthe monolithic NoC-based hardware SNN implementation. The paper also presents a lookuptable based SNN topology memory allocation technique, which further increases the memoryutilisation efficiency. Overall the area requirement of the architecture is reduced by an aver-age of 66 % for practical SNN application topologies. The paper presents micro-architecturedetails of the proposed MNT and digital neuron circuit. The proposed architecture has beenvalidated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technol-ogy. The evolvable capability of the proposed MNT and its suitability for executing subtaskswithin a MNN execution architecture is demonstrated by successfully evolving benchmarkSNN application tasks representing classification and non-linear control functions. The paper

S. Pande (B)· F. Morgan · S. Cawley · B. McGinleyBio-Inspired Electronics and Reconfigurable Computing, National University of Ireland, Galway, Irelande-mail: sandeep.pande@nuigalway.ie

T. Bruintjes · G. SmitComputer Architecture for Embedded Systems, University of Twente, Enschede, The Netherlands

S. Carrillo · J. Harkin · L. McDaidIntelligent Systems Research Centre, University of Ulster, Derry, Northern Ireland, UK

123

S. Pande et al.

addresses hardware modular SNN design and implementation challenges and contributes tothe development of a compact hardware modular SNN architecture suitable for embeddedapplications

Keywords Modular neural networks (MNN) · Spiking neural networks (SNN) ·Synaptic connectivity · Network on chip (NoC) · EMBRACE

1 Introduction

Artificial neural network (ANN) computing techniques, which are primarily inspired by thefunctioning of human brain, can provide promising solutions for designing complex andintelligent systems [1]. The organic central nervous system includes a dense and complexinterconnection of neurons and synapses, where each neuron connects to thousands of otherneurons through synaptic connections. Computing systems based on spiking neural networks(SNNs) emulate real biological neural networks, conveying information through the com-munication of short transient pulses (spikes) between neurons via their synaptic connections.Each neuron maintains an internal membrane potential, which is a function of input spikes,associated synaptic weights, current membrane potential, and a constant membrane potentialleakage coefficient [2,3]. A neuron fires (emits a spike to all connected synapses/neurons)when its membrane potential exceeds the neuron’s firing threshold value.

Brain-inspired computing paradigms such as SNNs offer the potential for elegant, low-power and scalable methods of embedded computing, with rich non-linear dynamics, ideallysuited to applications including classification, estimation, prediction, dynamic control andsignal processing. The efficient implementation of SNN-based hardware architectures forreal-time embedded systems is primarily influenced by neuron design, scalable on-chip inter-connect architecture and SNN training/learning algorithms [4]. The authors have investigatedand proposed EMBRACE1, as an embedded hardware neural network architecture [5,6]. TheEMBRACE NoC-based SNN architecture is a 2D mesh topology array of neural tiles eachcomprising a single neuron and NoC router. The NoC-based synaptic connectivity approachemployed in the EMBRACE architecture provides a flexible, packet-switched inter-neuroncommunication channels, scalable interconnect and connection reconfigurability [7,8].

SNN application topologies are characterised by a large number of synaptic connectionsthat translate to large distributed on-chip memory in packet switched NoC-based hardwareSNN architectures. The storage requirement for large SNNs poses a serious challenge fortheir compact hardware implementation. The modular neural networks (MNN) design strat-egy partitions complex application tasks into smaller subtasks executing on distinct neuralnetwork modules and integrates outputs in higher level functions [1,9]. The neural net-work modules in MNN execution architectures maintain internal communication, which isisolated from other modules and the global communication infrastructure. The orthogonali-sation of synaptic connectivity suggested in the MNN design paradigm can help tackle thelarge connectivity problem of SNN architectures and offer practical system implementationfor embedded applications.

This paper proposes a hardware modular neural tile (MNT) tile architecture, that reducesthe SNN topology memory requirement of NoC-based hardware SNNs by using a combi-nation of fixed and configurable synaptic connections [10]. The proposed MNT comprisesa 16:16 fully connected feed-forward SNN structure and supports execution of applicationsubtasks for MNN-based application designs. Fixed connections between the neurons within

1 EMulating Biologically-inspiRed ArChitectures in hardwarE.

123

Modular Neural Tile Architecture

the MNT remove the requirement for storage of connectivity information. The MNTs inte-grate in a 2D mesh topology NoC communication infrastructure to form an MNN executionarchitecture, where the overall SNN topology memory requirement is 50 % of the previ-ously reported monolithic NoC-based hardware SNN implementation [5,6]. The paper alsoproposes a further architectural enhancement, which involves sharing the SNN topologymemory (within each MNT) between the SNN structure outputs. The proposed lookup tablebased memory allocation scheme increases memory utilisation by offering flexible synap-tic connectivity suitable for practical SNN application topologies, which are characterisedby irregular connectivity patterns [11]. In total, the area requirements of the architecture isreduced by 66 %.

The paper presents micro-architecture details of the proposed MNT and the digital neuroncircuit used for system validation. The architectural components are synthesised using 65 nmlow-power CMOS technology and silicon area results are presented. The proposed MNTarchitecture has been validated on a Xilinx Virtex-6 FPGA and resource usage results arereported. The evolvable capability of the proposed MNT and its suitability for executing appli-cation subtasks in a MNN execution architectures is demonstrated by successfully evolvingthe XOR benchmark SNN application and a robotics obstacle avoidance controller using theplayer-stage robotics simulator and previously reported Genetic Algorithm based hardwareSNN evolution platform. (These benchmark SNN applications represent data classificationand non-linear control functions.)

The structure of the paper is as follows: Section 2 summarises reported hardware SNNarchitectures and their suitability as embedded hardware SNNs. Section 3 summarises thereported EMBRACE NoC-based hardware monolithic SNN architecture and highlights theimpact of SNN topology memory on the overall device size and the challenge of reducingthe SNN topology memory. Section 4 introduces the modular neural network computing par-adigm, its benefits and its application for reducing topology memory within the EMBRACEhardware SNN. Section 5 describes the proposed MNT hardware design including neuralcomputing module, packet encoder/decoder, digital neuron design, topology memory andSNN configuration memory. This section also reports ASIC and FPGA synthesis results.Section 6 presents classification and non-linear control benchmark functions implementedon the EMBRACE MNT FPGA prototype. Section 7 concludes the paper.

2 Hardware SNN Architectures

Inspired by biology, researchers aim to implement reconfigurable and highly interconnectedarrays of neural network elements in hardware to produce computationally powerful andcognitive signal processing units [12–21]. This section summarises reported hardware andhybrid SNN architectures and their suitability as embedded hardware SNNs.

A hybrid SNN computing platform is reported in [17]. This platform includes a neuronmodel implemented in hardware and the network model and learning implemented in soft-ware. A time multiplexed FPGA embedded processor SNN implementation [20] reports 4.2Kneurons and >1.9M synapses. The neuron model and partial SNN elements are implementedon an embedded FPGA processor, where speed–acceleration is the key motivation. The sys-tem relies on external memory for spike transfer management. Analogue spiking neurondesign approaches can benefit from a compact area implementation due to their ability tomodel electrical charge flow in the brain [13,18,22,23]. These architectures rely on digitalcomponents for a flexible communication infrastructure. FACETS, a configurable wafer-scale mixed-signal neural ASIC system, proposes a hierarchical neural network and the use

123

S. Pande et al.

of analogue floating gate memory for storage of synaptic weights [19,21]. A mixed-signalSNN architecture of 2,400 analogue neurons, implemented using switched capacitor tech-nology and communicating via an asynchronous event-driven bus has been reported in [18].The chip area is reported to be 3 × 3 mm using 0.5 µm CMOS VLSI technology.

Practical SNN systems are characterised by a large numbers of neurons and high intercon-nectivity through inter-neuron synaptic connections. Each of the SNN execution architecturespresented in [14–21] aim for thousands of neurons and millions of synapses, which is essen-tial for a powerful neural computing platform. For large scale hardware implementation ofSNNs, the neuron interconnect imposes problems due to high levels of inter-neuron con-nectivity and often the number of neurons that can be realised in hardware is limited byhigh fan in/out requirements [4]. Direct neuron-to-neuron interconnection exhibits switch-ing requirements that grow exponentially with the network size. Efficient, low area and lowpower implementations of neuron interconnect and synaptic junctions are therefore key toscalable hardware SNN implementations [4].

The NoC design paradigm provides a promising solution for the flexible interconnectionof large SNNs [7]. The SpiNNaker project [14] aims to develop a massively parallel com-puter capable of simulating SNNs of various sizes, topology and with programmable neuronmodels. The SpiNNaker architecture uses ARM-968 processor-based nodes for computationand an off-chip NoC communication infrastructure. Each NoC tile in the SpiNNaker systemmodels 1,000 leaky-integrate-and-fire (LIF) neurons, each having 1,000 synapse inputs. EachSpiNNaker node requires approximately 4 MB of memory for storing synaptic connectiv-ity information [24]. Hence, the SpiNNaker architecture stores the synaptic connection datain off-chip SDRAM. The SNN implementations described above are not targeted at com-pact embedded hardware applications. A clustered embedded hardware SNN is proposed in[25] along with a process for mapping SNNs to hardware. The variable sized neuron clus-ters support flexible synaptic connections even from and to within the clusters. The authorshave reported the scalable NoC-based EMBRACE [5] hardware SNN architecture, target-ing compact embedded hardware applications. The architecture supports large monolithicSNN topologies. In a monolithic SNN, the neuron connectivity is global (i.e. any neuroncan connect to any other neuron). The EMBRACE topology memory requirement increasesexponentially with network size [10]. Modular neural network techniques investigated in thispaper, can enable trade-offs between the topology memory size and synaptic connectivity,leading to scalable, more compact, practical embedded hardware SNN structures.

3 EMBRACE: Hardware SNN Architecture

This section summarises the reported EMBRACE NoC-based hardware monolithic SNNarchitecture and analyses the impact of SNN topology memory on the overall EMBRACEdevice size and the challenge of reducing the SNN topology memory.

3.1 EMBRACE: NoC-based Embedded Hardware Monolithic SNN Architecture

The EMBRACE (Fig. 1) [5] architecture incorporates neural circuits within a digital NoC-based packet switching interconnect to realise a scalable hardware monolithic SNN archi-tecture suitable for embedded systems. This architecture offers high synaptic densities whilemaintaining a small silicon footprint and low power consumption. EMBRACE is a 2D meshtopology array of neural tiles (NT) and NoC routers (R). Each neural tile comprises a singleneuron supporting up to 64 input and 64 output synaptic connections. The embedded hardware

123

Modular Neural Tile Architecture

Fig. 1 EMBRACE NoC-based embedded hardware monolithic SNN architecture (the neural tile (NT) com-prises a single neuron, packet encoder/decoder, SNN configuration and topology memory)

architecture supports the implementation of monolithic SNNs and provides a benchmark forcomparison of the work of this paper. The SNN topology memory defines each inter-neuronsynaptic connection. NoC routers are connected in North (N), East (E), South (S) and West(W) directions, forming a Manhattan-style, 2D mesh topology NoC architecture. An appli-cation specific SNN is realised on the EMBRACE architecture by programming neuronconfiguration parameters (SNN synaptic weights and neuron firing threshold potential) andSNN connection topology. Spike communication within the SNN is achieved by routing spikeinformation within spike data packets over the network of routers. The reported architecture(Fig. 1) requires 11 MB of SNN topology memory to support a 64K neuron/4M synapsehardware SNN.

The authors have implemented and reported EMBRACE–FPGA [8], an FPGA proto-type implementation of the EMBRACE architecture. The EMBRACE–FPGA prototype hasbeen successfully applied to benchmark SNN control and classifier applications (such aspole balancer, two-input XOR and Wisconsin cancer dataset classifier). EMBRACE-SysC, aSystemC-based clock cycle accurate simulation and performance measurement platform forsimulation and analysis of EMBRACE architecture has been reported in [26]. EMBRACE-SysC enables rapid architectural exploration and performance analysis of the EMBRACEarchitecture.

3.2 EMBRACE Hardware Resource Analysis

The transistor count and chip area has been estimated for the EMBRACE architecture tounderstand the practicality of realising the EMBRACE architecture in silicon. The siliconarea estimation technique takes into account the transistor count for storage and control logicof digital components (based on standard SRAM cell design) and actual silicon area forneurons [23].

123

S. Pande et al.

Fig. 2 Silicon area proportion (for 32 nm CMOS technology) for the EMBRACE hardware monolithic SNNarchitecture

Fig. 3 Silicon area proportion for the SNN infrastructure entities within EMBRACE hardware monolithicSNN architecture

Figure 2 presents the silicon area proportions (in mm2) by scaling the EMBRACE hard-ware monolithic SNN architecture in 32 nm CMOS VLSI technology. The x-axis indicates thenumber of neurons and synapses (neuron/synapse) and the stacked columns in the histogramdenote the silicon area for NoC and SNN architectural entities described below:

• NoC infrastructure: The EMBRACE NoC infrastructure comprises NoC routers, packetbuffers, NoC interconnect channels and associated control circuits. The analytical esti-mates indicate that the complete NoC infrastructure will occupies 47.27 % of the totalchip area.

• SNN infrastructure: The SNN support infrastructure is made-up of SNN configurationmemory (for storing synaptic weights of 5 bits each and threshold values of 16 bits each)and the SNN topology memory (for storing synaptic connectivity information) [6]. TheSNN infrastructure also contains the neural elements (which includes synapses, synapticweight summing and membrane potential thresholding circuits) [13,22,23]. Due to itscompact implementation, the silicon area for the neural elements is negligible comparedto the rest of the SNN support infrastructure, which occupies 52.74 % of the total chiparea (Figure 3 further enumerates the silicon area of the SNN components within theEMBRACE architecture).

Architectural techniques to reduce SNN topology memory are crucial for their compacthardware implementations. This paper presents a novel MNT architecture that reduces theSNN topology memory area by 50 %.

123

Modular Neural Tile Architecture

4 Modular Neural Networks

This section introduces the modular neural network (MNN) computing paradigm and exe-cution architecture. The section highlights the benefits and application of the MNN designapproach to reduce topology memory within the reported EMBRACE hardware monolithicSNN architecture.

4.1 Biological Motivations for the MNN Computing Paradigm

The biological brain is composed of several anatomically and functionally discrete areas [27].Various brain areas and neuron groups are dedicated to various sensory and motor tasks. Forexample, the visual cortex processes different aspects of the visual information (such asform, colour and motion) in separate anatomically distinct regions. These different regionsexchange information but remain functionally discrete [28,29]. Damage or deteriorationof part of the visual cortex can result in a partial loss of colour identification, pattern ormotion detection, etc, without considerably affecting other senses. Inspired by this modularorganisation in brain, the MNN design strategy of partitioning application tasks into a numberof subtasks has been developed [30–32].

4.2 Modular Neural Network Computing Paradigm

The MNN computing paradigm is primarily based on the divide-and-conquer strategy.Figure 4 illustrates the MNN design methodology, which primarily consists of task decom-position i.e. breaking down a high level application into smaller, less complex, manageablesubtasks. These small subtasks are then solved by individual and distinct neural modules. Theintermediate outputs from these neural modules are combined to solve the high level task orthe whole application [1,9]. Various task decomposition algorithms have been proposed forthe MNN design strategy. These algorithms are based on different techniques such as outputvector partitioning, class relationships, neuro-evolutionary approach and co-evolutionarymethods [33–37]. Similarly, genetic algorithm based technique to find subnetworks fromlarge sized complex neural networks has been proposed in [38]. Subtasks/subnetworksobtained after task decomposition are executed as neural modules in MNN computingarchitecture.

The MNN approach offers structured implementation, functional partition, functionalmapping and re-mapping, competitive and co-operative mode of operation and fault tolerance[31]. The Subsumption Architecture, a widely influential computing paradigm for reactive,behaviour-based autonomous robotic control applications has been developed based on theMNN design concepts [39].

4.3 Modular Neural Network Execution Architectures

In the MNN design methodology, the task decomposition or partitioning of the overall appli-cation leads to two types of subtasks; namely the application subtasks (discrete functionalsubtasks) and the integration tasks. The application subtasks operate on individual and dis-tinct system inputs to provide intermediate outputs. The integration subtasks integrate theintermediate outputs from the application subtasks to generate the overall system output.Both the application and integration subtasks are similar in terms of input/output inter-face and computation requirements. Figure 5 illustrates a typical MNN execution archi-tecture comprising individual neural modules interconnected by a global communication

123

S. Pande et al.

Fig. 4 Modular neural network design methodology

Fig. 5 Modular neural network execution architecture

123

Modular Neural Tile Architecture

infrastructure. Based on the definition of modularity in neural networks [1,9], the individualneural modules in the MNN execution architecture should:

• Include a group of neurons interconnected using an internal communication infrastructure• Support multiple inputs and/or multiple outputs• Include an internal communication infrastructure that is isolated from the other modular

neural elements and the global (external inter-module) communication infrastructure

Since the synaptic communication between neurons in a neural module is isolated from therest of the architecture, the requirement for the storage of associated connectivity informationis eliminated. The overall MNN architecture has a considerably lower synaptic connectivitystorage requirement compared to monolithic neural architectures. The SNN topologies forapplications that are inherently non-modularizable exhibit uniform connectivity patterns. Theneural modules in the MNN execution architecture can be cascaded using global communi-cation infrastructure to realise such large monolithic SNN structures. However, the resourceutilisation of the MNN execution architecture will be higher as compared to hardware SNNarrays supporting uniform synaptic connectivity.

5 Modular Neural Tile Architecture

This paper presents a novel Modular Neural Tile (MNT) architecture, its digital prototypeand evolvable capabilities. The proposed MNT forms the basic neural module for the MNNexecution architecture (proposed NoC-based modular hardware SNN execution architecture).

Architectural techniques for reducing the SNN topology and configuration memory arevital for compact silicon implementation of hardware SNN architectures suitable for embed-ded computing. This section presents the MNT architecture comprising a 16:16 fully con-nected feed-forward topology SNN structure as the neural computing module (NCM) and thelookup table-based SNN topology memory sharing scheme. The micro-architecture of an effi-cient digital neuron model is also described. Detailed ASIC and FPGA synthesis results arepresented. Silicon area requirement of the NoC architecture employing the proposed MNT iscompared with the previously reported EMBRACE hardware monolithic SNN architecture.

5.1 Neural Computing Module

The Neural Computing Module (NCM) forms the basic computing entity within the MNNexecution architecture. The NCM micro-architecture design is primarily influenced by thefollowing:

• VLSI technology limitations: Fixed interconnections between neurons within the SNNstructure removes the need for synaptic connectivity information storage within the MNT.The size of the fully-connected hardware SNN structure is limited by the permitted fan-out of individual neuron circuits. Also, metal layer routing limitations are imposed dueto interconnect crossbar capacitance and crosstalk.

• MNN subtask granularity: The NCM should support sufficient computing power forMNN application subtasks and integration subtasks. Large NCM designs, can accom-modate a variety of MNN application subtasks, but would lead to unused neurons inthe case of fine granularity MNN application subtasks. NCM designs with smaller SNNstructures can be cascaded based on the computing requirements of the particular MNNapplication and integration subtasks.

123

S. Pande et al.

Fig. 6 Two layered 16:16 fullyconnected SNN structure as theneural computing module withinthe proposed MNT

(a)

(b)

Fig. 7 Digital neuron model

Considering the above, a two-layered 16:16 fully connected feed-forward SNN structure(Fig. 6) has been proposed as the NCM inside each MNT [10]. The input layer (N [0, n]) andoutput layer (N [1, n]) of the NCM comprises 16 LIF neurons. The neurons support a SingleDynamic Synapse (SDSn) approach (Fig. 7a), where the synaptic weight is supplied alongwith spike input. This shared synapse approach removes the need for internal multiplexersfor selection of synaptic weight and results in compact hardware implementation [40]. Eachof the 16 input layer neurons connects directly to each of the 16 output layer neurons toform a fully connected feed-forward SNN structure. Each output layer neuron has 16 inputsynapses, which individually receive spikes from the corresponding input layer neurons. TheNCM has 16 spike outputs (SpikeOutn) each corresponding to the 16 output layer neurons.

123

Modular Neural Tile Architecture

The NCM in a MNN execution architecture should be capable of solving applicationsubtasks and integration tasks. The proposed modular NCM made-up of 32 LIF neurons,supports multiple synaptic inputs and provides 16 spike outputs that can connect to multiplesynaptic inputs in the architecture. This paper demonstrates the suitability of the proposedNCM by evolving SNN benchmark functions.

5.1.1 Digital Neuron Model

This section describes the multiplier-less, compact hardware digital neuron design.Based on biological plausibility, computational power and implementation complexity,

various mathematical models representing the spiking behaviour of biological neurons havebeen proposed [2,3]. Amongst them the LIF model is a popular choice for hardware SNNarchitectures due to its simplicity. The proposed EMBRACE research project ultimatelyaims to develop a mixed-signal VLSI architecture with compact, low power, high-resolutionCMOS-compatible analogue synapse and LIF neuron cells [13,23] to achieve very highsynaptic density. The LIF neuron model has multiple synaptic inputs, a single spike outputand maintains an internal membrane potential, which constantly decays (to its resting value)based on a constant leakage coefficient. The synaptic weight value associated with the inputsynapse is summed with the current membrane potential value on receipt of an input spike.The exhibitory/inhibitory synaptic weights increase/decrease the membrane potential by theweight value. The neuron fires (emits a spike) when the membrane potential reaches thethreshold potential value. Firing of the LIF neuron causes the membrane potential to reset tothe resting value.

For validation of the proposed MNT architecture on FPGA, a digital neuron circuit exhibit-ing LIF behaviour with sufficient resolution has been developed (Fig. 7a). The digital neuroncircuit uses a 16-bit shift register for storage of the Membrane Potential value. Stepwiseexponential decay of the membrane potential value is achieved by periodic right shift (divideby two) controlled by the MPOT_DECAY input pulse. The desired decay rate or LeakageCoefficient can be controlled by programming the membrane potential decay strobe generator(Fig. 7b). The membrane potential value is updated by the input Synaptic Weight (SYN_WT)value using a 16-bit adder/subtractor enabled by the incoming spike pulse (SPIKE_IN). Theoverflow/underflow bit of the adder/subtractor is used to determine the upper/lower limit ofthe membrane potential value, and activate the saturation logic. A 16-bit comparator generatesthe spike output (SPIKE_OUT) if the internal membrane potential > threshold (TH_POT)input. The generated spike pulse (SPIKE_OUT) is also used to clear the 16-bit shift register,bringing the membrane potential to the resting value.

Figure 7b shows a simulation waveform for the digital neuron circuit including membranepotential decay due to MPOT_DECAY pulses, membrane potential variation due to inputspikes, and output spike generation. The synthesizable digital neuron circuit provides therequired resolution for practical SNN applications. The neuron occupies only 12 slices inthe Xilinx Virtex-6 FPGA and occupies 608.4 µm2 in 65 nm CMOS VLSI technology. Thiscompact neuron model contributes to the goal of portable embedded hardware SNNs.

5.1.2 Neural Computing Module Architecture

Fixed interconnection between neurons within the neural computing module removes theneed to store synaptic connectivity information. Figure 8 shows the fixed connection micro-architecture of the NCM. The interconnect architecture transfers the generated spikes from the

123

S. Pande et al.

Fig. 8 Neural computing module micro-architecture

input layer neurons to output layer neurons. The interconnect architecture enables selectionof synaptic weights from configuration memory for the output layer neurons while applyingspikes.

The input interface to the NCM consists of neuron number (Neuronn), synaptic weight(SynWt) and input spike pulse (SpikeIn). The NCM layers operate in a pipelined fashion,where the input spike is applied to the selected input layer neuron during the first clock cycleand any generated spike is transferred to output layer neurons in the next clock cycle. Theinput neuron number (Neuronn) is stored in a 4-bit register for use in the next clock cycle. Theinput synaptic weight is directly connected to all of the input layer neurons. The 4:16 decoderenables the selected input neuron number. The decoder is enabled by the input spike pulse,which causes the selected neuron to activate and update its internal membrane potential.

According to the LIF neuron behaviour [3], neurons can generate a spike only on theoccurrence of an input spike. Hence, the stored neuron number in nth clock cycle is used fortransfer of spike and selection of synaptic weight for output layer neurons in (n+1)th clockcycle within the NCM. The NCM output interface comprises spike outputs (SpikeOut0:15)from all output layer neurons. Spikes pulses generated by the NCM are further processedwithin the MNT to generate output spike packets for synaptic connections outside the MNT.

5.2 Topology Memory Sharing and Spike Packet Output Flow Control

Spike communication between MNTs is achieved by routing spike information within spikedata packets over the network of routers. Within each MNT a lookup table based SNNtopology memory allocation technique is implemented, which enables variable synaptic

123

Modular Neural Tile Architecture

(a)

(b)

Fig. 9 The MNT spike packet output flow control

connection densities for efficient topology memory resource usage. This section describesspike packet generation based on the proposed lookup table-based topology memory sharingscheme, which offers a flexible number of synaptic connections for NCM outputs.

Figure 9 illustrates the lookup table-based shared topology memory organisation. Thesynaptic connection information for the NCM outputs is stored in this topology memory.Fields include destination MNT address ([X,Y] mesh topology NoC tile address), destinationneuron (Neuronn) and synaptic weight (SynW t) (see Fig. 9b). The topology memory ispartitioned into 64 blocks (B0 to B63), where each block is made-up of 16 synaptic connectioninformation entries (Bx SC0 to Bx SC15), giving a total of 1,024 neuron/synapse destinations.The lookup table maintains the topology memory block allocation information in designatedrows for each NCM output. Each bit in the 64-bit lookup table row allocates the correspondingtopology memory block to the NCM output. For example, bit number Bx of the row numberN[1,0] allocates topology memory block number X to the NCM output N[1,0]. For the rownumber N[1,0], asserting the bit value Bx , allocates the topology memory block X to NCMoutput N[1,0]; deasserting Bx disassociates the topology memory block X from the NCMoutput N[1,0]. The packet encoder generates spike packets for NCM outputs based on theallocated topology memory blocks.

The process of mapping the MNN application topology onto the proposed MNT involvespopulating the lookup table and MNT topology memory entries, such that the correct synapticconnections are established between the neural computing modules in the architecture. If therequired number of synaptic connections for a particular NCM cannot be accommodated inthe available topology memory, additional MNTs can be used as spike repeaters. The NCMcan be configured as spike repeater by configuring synaptic weights and threshold to generatea spike for each input spike.

123

S. Pande et al.

Fig. 10 Modular neural tile internal architecture

5.3 Modular Neural Tile Hardware Design

The MNT (Fig. 10) comprises a NCM, configuration memory (for storing neuron synapticweights and threshold values), topology memory (for storing synaptic connectivity informa-tion) and packet decoder and encoder. This section describes the interfaces and functionalityof the hardware entities within the proposed MNT.

5.3.1 Modular Neural Tile Internal Architecture

The MNT connects to the NoC router through the router interface, which comprises32-bit PacketIn/PacketOut, valid (PacketValid) and acknowledgment (PacketAck) handshakesignals. The MNT supports the following packet types (Fig. 11):

• Configuration packet: used for configuration of neurons (synaptic weights and thresholdvalues), the lookup table and output synaptic connectivity. The configuration packetconsists of destination tile address (XY NoC tile address), configuration memory address(13-bit) and data (8-bit).

• Spike packet: used for transferring spikes from source MNT to destination MNT. Itconsists of the destination tile address 2-D array format (XY NoC tile address), neuronnumber (Neuronn) and synaptic weight. Inclusion of synaptic weight in the spike packetreduces the overall storage requirement of the architecture, and removes the necessity

123

Modular Neural Tile Architecture

Fig. 11 MNT configuration and spike packet format

Table 1 Modular neural tile synthesis results (clock frequency: 200 Mhz)

RTL entity ASIC synthesis FPGA synthesis(65 nm low-power CMOS) (Xilinx Virtex-6 FPGA)

Area (in µm2) Percentage (%) Slices BRAMs

Modular neural tile 110,806.63 100.0 1, 204 0Neural computing module 20,505.24 18.7 458 0Digital neuron circuit 608.40 0.5 12 0Packet decoder 408.71 0.4 7 0Packet encoder 2,368.60 2.1 66 0NCM configuration memory 31,444.47 28.4 662 0NCM topology memory 55,800.00 50.4 0 1

for synaptic weight selection multiplexers for the NCM inputs, resulting in compacthardware implementation [40].

The packet decoder interfaces with the NCM, configuration and topology memory byreceiving and decoding the input packets. The input packet type (spike or configurationpacket) is decoded from the Packet Type bits (B23–B21). For a configuration packet, theconfiguration memory address and data are retrieved from the packet and written to config-uration or topology memory. For a spike packet, neuron number (Neuronn) and synapticweight (SynW t) are retrieved from the packet and forwarded to the NCM along with aninternally generated spike pulse. The configuration memory supplies synaptic weight andthreshold values to the NCM and lookup table bits to the packet encoder through dedicatedcontrol signals. The topology memory is a dual-ported RAM module as it interfaces with thepacket decoder for memory write operation and with the packet encoder for memory readoperation. The packet encoder generates individual spike packets based on spike inputs fromthe NCM, synaptic connectivity information in the topology memory and memory blockallocation information from the lookup table. Refer to Section 5.2 for the detailed outputspike packet flow control.

Table 1 shows the detailed hardware synthesis results for the proposed MNT. ASIC syn-thesis has been performed using synopsys design compiler 2009-sp5 and TSMC 65 nm low-power CMOS libraries. FPGA synthesis has been carried out using Xilinx XST 13.2. Sincethe configuration memory supplies all the MNT control signals (2,816 bits, synaptic weights,threshold values and lookup table) to the NCM and packet encoder, the silicon footprint is

123

S. Pande et al.

Fig. 12 Silicon area estimate (for 32 nm CMOS technology) comparison for the EMBRACE monolithic andthe proposed MNT hardware SNN architecture

considerably higher compared to memory size. Due to massive interconnectivity in neuralarchitectures, the majority of the area is occupied by topology memory (50 % in the proposedMNT). The topology memory is realized using STMicroelectronics low-power CMOS dual-ported memory IP in ASIC implementation.

Figure 12 compares the silicon area of various sized SNNs MNT NoC architecture withthat of the reported EMBRACE hardware monolithic SNN architecture. The reduced numberof NoC routers, resulting from the 16:16 fully connected SNN (NCM), decreases the areaoccupied by the NoC infrastructure in the proposed MNT NoC architecture by 89 % ascompared to the reported EMBRACE SNN architecture. The fixed interconnection withinthe NCM removes the need for storing the output synaptic connectivity information for theinput layer neurons. The regularly structured interconnect requires much less silicon areathan the SRAM-based synaptic connectivity storage and the associated control circuitry.Consequently, the topology memory for the proposed MNT NoC architecture is reduced by54.05 %. The size of the entire MNT NoC-based hardware SNN is approximately 33 % ofthat of the previously reported EMBRACE chip area estimation.

5.3.2 Hardware Resource Requirements of Practical SNN Topologies

Practical SNN application topologies exhibit a variety of connectivity patterns. Flexiblesharing of the topology memory within the MNT addresses diverse connectivity requirementsof the practical modular SNN application topologies. This section presents and compareshardware resource requirements for the proposed MNT architecture with shared and non-shared topology memory schemes for SNN application topologies with irregular and randomconnectivity patterns [11]. Additional MNTs are used for relaying spikes, if the synapticconnectivity requirement of the NCM cannot be accommodated in the topology memorywithin the MNT.

A large modular neural network application, made-up of 64 individual NCMs, has beenmapped to the proposed MNT-based NoC architecture. The application implementations

123

Modular Neural Tile Architecture

(a)

(b)

Fig. 13 NoC tile requirements for non-shared and shared topology memory schemes for the a irregularly andb randomly connected example modular topology

using non-shared and shared topology memory configurations have been compared. The non-shared topology memory scheme uses a fixed allocation of four topology memory blocksto each NCM output. The NCMs in the shared topology format are configured such thateight outputs from each NCM remain inactive (by configuring zero synaptic connections).The number of required MNTs and the size of the NoC are calculated for various synapticconnection densities in the remaining eight active NCM outputs. Figure 13a compares thenumber of MNTs required using non-shared and shared topology memory approach, execut-ing the MNN application topology with irregular synaptic connectivity pattern. The topologymemory in the MNT can hold 1K (i.e. 1,024) synaptic connection entries. When the synap-tic connectivity requirement of each NCM increases by 1K steps, additional set of MNTsare used for relaying spike packets. This can be seen in the step wise ascending graph inFig. 13a.

The SNN topologies evolved using Genetic Algorithm (GA) based search methods oftenexhibit random connectivity patterns [11]. The application topology described above hasbeen configured for a random number of output synaptic connections from each of the 64individual NCMs. This MNN application representing a random synaptic connectivity patternhas been mapped to the proposed MNT NoC architecture and tested under non-shared andshared topology memory configuration. Figure 13b illustrates the MNT requirement for the

123

S. Pande et al.

proposed MNT NoC architecture under non-shared and shared topology memory scheme,executing the application topology with a random synaptic connectivity pattern.

The proposed shared topology memory architecture facilitates the allocation of topologymemory blocks to the NCM outputs based on the synaptic connectivity requirement. Thelookup table based shared topology memory architecture offers a flexible number of synapticconnections from the NCM outputs resulting in efficient usage of each MNT. Figure 13illustrates that, the shared topology memory scheme requires a smaller number of MNTsfor MNN application topologies with irregular and random synaptic connectivity patterns(observed in practical SNN application topologies). This enables implementation of largerapplication topologies within the given architectural configuration.

6 Modular Neural Tile Applications

This section presents classification and controller benchmark functions implemented on theEMBRACE modular neural tile FPGA prototype. The benchmark XOR function (a basicdata classifier) and robotics controller (closed loop non-linear control system) have beenused to test the evolvable capability of the proposed MNT. Accuracy (fitness) of the SNNconfiguration and training time results (in terms of GA generation count) are presentedfor successfully evolved XOR and robotics controller application on the proposed MNTprototype. This demonstrates the capability of the proposed MNT to evolve small-sizedsubtasks within a larger MNN applications. The intrinsic evolution setup comprising theproposed MNT prototype on FPGA and GA-based SNN configuration platform running onhost computer [8], highlight the hardware validation aspects of the modular SNN architecture.

6.1 Classification Subtasks

The XOR function is a basic benchmark data classification problem for neural networksand is a sub-problem of more complicated classifiers [41]. A two-input XOR function is

Table 2 Two-input XORfunction fitness score assignment

Number of correct outputs 0 1 2 3 4

Fitness score assignment value 0 1 4 9 16

Fig. 14 XOR benchmark SNN application on the proposed MNT

123

Modular Neural Tile Architecture

implemented on the proposed MNT. The XOR function uses two spike rate encoders feedingdistinct spike rates for logic ‘0’ and logic ‘1’ and an output spike rate decoder. The GA-basedSNN evolution and configuration platform configures the SNN configuration comprisingsynaptic weights and threshold potential for all the neurons in the SNN, check the accuracy(fitness) of the configuration and evolves the desired functionality by searching the correctconfiguration. The evolved XOR function on the proposed MNT outputs a high spike ratewhen the two inputs differ in spike rate, and a low spike rate if the inputs are equal. Table 2illustrates the fitness score assignment used for the two input binary XOR function. The threeneuron XOR SNN partially uses the MNT. Figure 14 illustrates the average and best fitnessof the evolved SNN XOR function on the proposed MNT.

6.2 Non-Linear Control Subtasks

The robotics controller application is a classical example of a non-linear closed loop controlsystem. Neural network systems are widely used as controllers for practical robotics appli-cations. A robotics obstacle avoidance controller, using the player-stage robotics simulator[42] and the previously reported Genetic Algorithm (GA)-based hardware SNN evolutionand implementation platform (Fig. 15). The simulated robot is equipped with 16 sonar sen-sors. The average values of front, rear and side sonars are used as inputs to the SNN. Thesonar values are converted to spike rates by the host application and are passed to spike rateencoders, which continuously generate and feed spikes to the NCM within the MNT. Spikesfrom two NCN outputs are monitored by spike rate decoders and are converted to analoguevalues as robot acceleration and turning angle inputs to the simulator.

Fitness criteria for the robotic obstacle avoidance controller application is defined as travel-ling finite distance and avoiding obstacles for ≥120 s, and the fitness of the SNN configurationis calculated as:

F = αT + β D + γ S (1)

where, F fitness of the individual, T robot travel time (in s), D robot travel distance (in cm)and S robot travel speed (in cm/s).

Given:

Number of crashes = 0SNN outputs are within the operating rangeThe fitness evaluation constants α, β and γ are chosen to prioritise the robot motionbehaviour.

Evaluation of the individual configurations has been accomplished with the robot roamingwithin the simulated environment for 300 ≥ t > 120. On timeout, or if the robot has crashed,the GA-based evolution and configuration platform processes the recorded robot behaviourand assigns a fitness score to the individual SNN configuration. Fitness scores are then usedby the GA to determine the probability of an individual configuration progressing to laterevolved generations. Figure 16 illustrates the average and best fitness of the evolved roboticobstacle avoidance controller application on the proposed MNT.

Successful evolution of classification and non-linear control functions with constant andtime varying input patterns on the proposed MNT demonstrates its suitability for a varietyof MNN application designs.

123

S. Pande et al.

Fig. 15 Robotic obstacle avoidance controller setup

Fig. 16 Robotic obstacle avoidance controller benchmark SNN application on the proposed MNT (fitnessevaluation constants: α = 2, β = 1 and γ = 0.5)

123

Modular Neural Tile Architecture

7 Conclusions

Neural architectures are typically characterised by thousands of neurons and millions ofsynapses which are essential for a powerful neural computing platform. Storage of the largesynaptic connectivity information in packet switched network on chip (NoC)-based hardwareSpiking neural network (SNN) architectures translates to large distributed on-chip memoryand poses a serious challenge for compact hardware implementations. Discrete synaptic con-nectivity observed in MNN execution architectures help in reducing the storage requirementof NoC-based hardware neural architectures.

This paper presented a novel MNT architecture comprising a 16:16 fully connected feed-forward topology SNN structure as NCM. The topology memory of the architecture is 50 %of the previously reported NoC-based hardware monolithic SNN implementation. Further-more, a lookup table-based topology memory sharing scheme is presented that provides aflexible number of synaptic connections from NCM outputs. The proposed shared topologymemory scheme requires less number of MNTs for practical MNN application topologieswith irregular and random synaptic connectivity patterns. Overall the area requirement ofthe architecture is reduced by an average 66 % for practical SNN application topologies.This facilitates accommodation of larger application topologies in the given architecturalconfiguration.

The paper presented micro-architecture details of the proposed MNT and the digital neuroncircuit used for system validation. The architectural components are synthesised using 65 nmlow-power CMOS technology and silicon area results are presented. The proposed MNTarchitecture has been validated on Xilinx Virtex-6 FPGA and resource utilisation is presented.The evolvable capability of the proposed MNT, and its suitability for executing applicationsubtasks, in a modular hardware SNN architecture is demonstrated by successfully evolvingthe XOR benchmark SNN function and a robotics obstacle avoidance controller, using theplayer-stage robotics simulator and the previously reported Genetic Algorithm (GA)-basedhardware SNN evolution and configuration platform. Successful evolution of classificationand non-linear control functions with constant and time varying input patterns on the proposedMNT demonstrates its suitability for variety of MNN application designs.

The architectural enhancements proposed and validated in this paper helps to achieve acompact neural modular hardware implementation and demonstrate the ability of the pro-posed MNT to successfully evolve benchmark SNN functions. This work contributes to thedevelopment of the EMBRACE NoC-based embedded hardware SNN device [12].

Acknowledgments This research is supported by International Centre for Graduate Education in Micro andNano-Engineering (ICGEE), Irish Research Council for Science, Engineering and Technology (IRCSET),Science Foundation, Ireland (Grant No. 07/SRC/I1169) and Xilinx University Programme.

References

1. Haykin SS (1999) Neural networks: a comprehensive foundation, vol 13. Prentice Hall, New Jersey2. Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural

Netw 10(9):1659–16713. Wulfram G, Werner MK (2002) Spiking neuron models. Cambridge University Press, Cambridge4. Maguire LP, McGinnity TM, Glackin B, Ghani A, Belatreche A, Harkin J (2007) Challenges for large-

scale implementations of spiking neural networks on FPGAs. Neurocomputing 71(1–3):13–295. Jim H, Fearghal M, Liam M, Steve H, Brian M, Seamus C (2009) A reconfigurable and biologically

inspired paradigm for computation using network-on-chip and spiking neural networks. Int J ReconfigComput 2009:1–13

123

S. Pande et al.

6. Seamus C, Fearghal M, Brian M, Sandeep P, Liam M, Snaider C, Jim H (2011) Hardware spiking neuralnetwork prototyping and application. Genet Program Evolvable Mach 12:257–280

7. Dmitri V, Ran G (2011) Scalable network-on-chip architecture for configurable neural networks. Micro-process Microsyst 35(2):152–166 (special issue on network-on-chip architectures and design methodolo-gies)

8. Morgan F, Cawley S, McGinley B, Pande S, McDaid LJ, Glackin B, Maher J, Harkin J (2009) Exploringthe evolution of NoC-based spiking neural networks on FPGAs. In: International conference on field-programmable technology, 2009. IEEE, Sydney, pp 300–303

9. Daniel NO, Scott W, Michael S (1993) Modular learning. MIT Press, Cambridge, pp 369–37710. Sandeep P, Fearghal M, Seamus C, McGinley B, Jim H, Snaider C, McDaid L (2011) Addressing the hard-

ware resource requirements of network-on-chip based neural architectures. In: International conferenceon neural computation theory and applications. NCTA, Paris

11. Nate K, Risto M (2008) Evolving neural networks for fractured domains. In: Proceedings of the 10thannual conference on genetic and evolutionary computation, GECCO ’08. ACM, New York, pp 1405–1412

12. Harkin J, Morgan F, Hall S, Dudek P, Dowrick T, McDaid L (2008) Reconfigurable platforms and thechallenges for large-scale implementations of spiking neural networks. In: International conference onfield programmable logic and applications, 2008. IEEE, Heidelberg, pp 483–486

13. Yajie C, Hall S, McDaid L, Buiu O, Kelly P (2006) A solid state neuron for the realisation of highlyscaleable third generation neural networks. In: 8th International conference on solid-state and integratedcircuit technology, 2006. ICSICT, Beijing, pp 1071–1073

14. Furber S, Brown A (2009) Biologically-inspired massively-parallel architectures-computing beyond amillion processors. In: Ninth international conference on application of concurrency to system design,2009. ACSD, Augsburg, pp 3–12

15. Andres U, Ca Pea-Reyes, Sanchez E (2005) An FPGA platform for on-line topology exploration ofspiking neural networks. Microprocess Microsyst 29(5):211–223

16. Pearson MJ, Pipe AG, Mitchinson B, Gurney K, Melhuish C, Gilhespy I, Nibouche M (2007) Implement-ing spiking neural networks for real-time signal-processing and control applications: a model-validatedFPGA approach. IEEE Trans Neural Netw 18(5):1472–1487

17. Ros E, Ortigosa EM, Agis R, Carrillo R, Arnold M (2006) Real-time computing platform for spikingneurons (RT-spike). IEEE Trans Neural Netw 17(4):1050–1063

18. Vogelstein RJ, Mallik U, Vogelstein JT, Cauwenberghs G (2007) Dynamically reconfigurable siliconarray of spiking neurons with conductance-based synapses. IEEE Trans Neural Netw 18(1):253–265

19. Ehrlich M, Mayr C, Eisenreich H, Henker S, Srowig A, Grubl A, Schemmel J, Schuffny R (2007)Wafer-scale VLSI implementations of pulse coupled neural networks. Proceedings of the internationalconference on sensors, circuits and instrumentation systems. CEA, Acapulco

20. Glackin B, McGinnity TM, Maguire LP, Wu QX, Belatreche A (2005) A novel approach for the imple-mentation of large scale spiking neural networks on FPGA hardware. In: Computational intelligence andbioinspired systems. CIBS, Barcelona, pp 552–563

21. Schemmel J, Fieres J, Meier K (2008) Wafer-scale integration of analog neural networks. In: IEEE inter-national joint conference on neural networks (2008) IEEE world congress on computational intelligence.IEEE, Hong Kong, pp 431–438

22. Chen Y, Hall S, McDaid L, Buiu O, Kelly P (2006) On the design of a low power compact spikingneuron cell based on charge-coupled synapses. In: International joint conference on neural networks,2006. IJCNN, Brisbane, pp 1511–1517

23. Chen Y, McDaid L, Hall S, Kelly P (2008) A programmable facilitating synapse device. In: IEEE inter-national joint conference on neural networks (2008) IEEE world congress on computational intelligence.IJCNN, Barcelona, pp 1615–1620

24. Furber S, Temple S, Brown A (2006) On-chip and inter-chip networks for modeling large-scale neuralsystems. In: Proceedings of 2006 IEEE international symposium on circuits and Systems, 2006. ISCAS,Kos, p 4

25. Emery R, Yakovlev A, Chester G (2009) Connection-centric network for spiking neural networks. In:3rd ACM/IEEE international symposium on networks-on-chip (2009) NoCS 2009. IEEE, San Diego,pp 144–152

26. Pande S, Morgan F, Cawley S, McGinley B, Carrillo S, Harkin J, McDaid L (2010) EMBRACE-SysC foranalysis of NoC-based spiking neural network architectures. In: System on chip international symposiumon (SoC), 2010. SoC-2010, Tampere, Finland, pp 139–145

27. Johannes S, Wieringa BM, Matzke M, Mnte TF (1996) Hierarchical visual stimuli: electrophysiologicalevidence for separate left hemispheric global and local processing mechanisms in humans. Neurosci Lett210(2):111–114

123

Modular Neural Tile Architecture

28. Van Essen DC, Anderson CH, Felleman DJ (1992) Information processing in the primate visual system:an integrated systems perspective. Science 255(5043):419–423

29. Binzegger T, Douglas RJ, Martin KAC (2007) Stereotypical bouton clustering of individual neurons incat primary visual cortex. J Neurosci 27(45):12242–12254

30. Happel BLM, Murre JMJ (1994) Design and evolution of modular neural network architectures. NeuralNetw 7(6–7):985–1004

31. Auda G, Kamel MS (1999) Modular neural networks a survey. Int J Neural Syst 9(2):129–15132. Ronco GP (1995) Modular neural networks: a state of the art. Rapport technique CSC95026, vol 1. Center

of System and Control, University of Glasgow, Lanarkshire, pp 1–2233. Sheng-uei G, Shanchun L, Tan SK (2004) Neural network task decomposition based on output partitioning.

J Instit Eng Singap 44:78–8934. Bao-Liang L, Ito M (1999) Task decomposition and module combination based on class relations:

a modular neural network for pattern classification. IEEE Trans Neural Netw 10(5):1244–125635. Thangavelautham J, D’Eleuterio GMT (2004) A neuroevolutionary approach to emergent task decompo-

sition. Proceedings of 8th parallel problem solving from nature. Springer, Heidelberg, pp 991–100036. Khare VR, Yao Xin, Sendhoff B, Jin Yaochu, Wersing H, (2005) Co-evolutionary modular neural networks

for automatic problem decomposition. In: The (2005) IEEE congress on evolutionary computation, vol3. CES, Edinburgh, pp 2691–2698

37. Santos JM, Alexandre LA, de Sa JM (2006) Modular neural network task decomposition via entropicclustering. In: Sixth international conference on intelligent systems design and applications (2006) vol 1.IEEE Computer Society Press, Jinan, pp 62–67

38. Pizzuti C (2012) A multiobjective genetic algorithm to find communities in complex networks. IEEETrans Evolut Comput 16(3):418–430

39. Brooks R (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom 2(1):14–2340. Cawley S, Pande S, McDaid L, McGinley B, Morgan F (2009) Memory efficient storage of reconfig-

urable topology information in network-on-chip based spiking neural networks, internal report. NationalUniversity of Ireland, Galway, Bio-Inspired Electronics and Reconfigurable Systems

41. Maher J, McGinley B, Rocke P, Morgan F (2006) Intrinsic hardware evolution of neural networks inreconfigurable analogue and digital devices. In: 14th Annual IEEE symposium on field-programmablecustom computing machines. FCCM, Seattle, pp 321–322

42. Vaughan R (2008) Massively multi-robot simulation in stage. Swarm Intell 2:189–208

123