Computation in Neuromorphic Analog VLSI Systems integration (VLSI) systems containing electronic analog

  • View

  • Download

Embed Size (px)

Text of Computation in Neuromorphic Analog VLSI Systems integration (VLSI) systems containing electronic...

  • Computation in Neuromorphic Analog VLSI Systems

    Giacomo Indiveri Institute of Neuroinformatics, University/ETH Zurich

    Zürich, Switzerland


    In this paper we present an overview of basic neuromorphic analog circuits that are typically used as building blocks for more complex neuromorphic systems. We present the main principles used by the neuromorphic engineering commu- nity and describe, as case example, a neuromorphic VLSI system for modeling selective visual attention.

    1 Neuromorphic Engineering

    The term “neuromorphic” was coined by Carver Mead to describe very large scale integration (VLSI) systems containing electronic analog circuits that mimic neuro- biological architectures present in the nervous system [18]. Neuromorphic computa- tion is related to modeling and simulation of networks of neurons and systems using the same organizing principles found in real nervous system. In recent times the term “neuromorphic” has also been used to describe mixed analog/digital VLSI systems that implement computational models of real neural systems. These VLSI systems, rather than implementing abstract neural networks only remotely related to biologi- cal systems, in large part, directly exploit the physics of silicon (and of CMOS VLSI technology) to implement the physical processes that underlie neural computation.

    Neuromorphic engineering is a new discipline at the boundary between engineer- ing and neuroscience, but which crosses many other fields, including biology, physics, computer science, psychology, physiology, etc.

    Analog VLSI Circuits

    There are some direct analogies between biological neural systems and analog VLSI neuromorphic systems: conservation of charge, amplification, exponentiation, thresholding, compression, and integration. The parallels between these two worlds run from the level of device physics to circuit architectures:

    � Diffusion mechanisms in membrane and transistor channels

    � Function determined by the system’s structure

  • To point out why studying natural neural systems and implementing models of these systems, using analog VLSI technology can be potentially instrumental for im- proving technological progress, consider the following arguments:

    1. Biology has solved the problem of using loosely coupled, globally asynchronous, massively parallel, noisy and unreliable components to carry out robust and en- ergy efficient computation.

    2. CMOS VLSI technology has been continuously improving, following Moore’s law, for almost 30 years, pushed by a hyper-competitive $300 billion global in- dustry. But, as CMOS technology improves further and transistor’s gate lengths drop below 0.10 microns

    � transistors start to behave like neurons

    � we have the ability to place millions of simple processors on a single piece of silicon

    � single chips start to have complexities that are too difficult to handle by the design tools

    � we need to begin to worry about power consumption and power dissipation issues

    Computation and Power Consumption

    The brain has on the order on ���� neurons and ���� synapses. It performs on av- erage ���� operations per second. The power dissipation of the brain is approximately �����J per operation, which results in about a total mean consumption of less than 10 watts. By comparison today’s silicon digital technology can dissipate at best ����J of energy per operation at the single chip level. There is no way of achieving �� ��

    operations per second on a single chip, with today’s technology. But even if this was possible, to do that amount of computation a digital chip would consume MegaWatts (the output of a nuclear power station).

    Any serious attempt to replicate the computational power of brains must confront this problem. Subthreshold analog circuits are also no match for real neural circuits, but they are a factor of ��� more power efficient than their digital counterparts.

    In the following Section we present some basic analog circuits that are most com- monly used as elementary building blocks for constructing complex neuromorphic systems. In Section 3 we present a VLSI device containing a model of selective visual attention, that uses many of the circuits described in Section 2, as a case example. Finally in Section 4 we draw the conclusions.

    2 Subthreshold Analog Circuits

    Perhaps the most elementary computational element of a biological neural structure is the neural cell’s membrane. The nerve membrane electrically separates the neuron’s

  • interior from the extra-cellular fluid. It is a very stable structure that behaves as a per- fect insulator. Current flow through the membrane is mediated by special ion channels (conductances) which can behave as passive or active devices. In the passive case, ion channels selectively allow ions to flow through the membrane by the process of dif- fusion. In electronics, it is possible to implement the same physical process by using MOS transistor devices, operated in the subthreshold region (also referred to as weak inversion) [17, 20].

    2.1 From subthreshold MOS transistors . . .

    MOS transistors operate in the subthreshold region of operation when their gate-to- source voltage is below the transistor threshold voltage. This mode of operation of a transistor has been largely ignored by the analog/digital circuit design community, mainly because the currents that flow through the source-drain terminals of the device under these conditions are extremely low (typically of the order of nanoamperes). In subthreshold, the drain current of the transistors is related to the gate-to-source voltage by an exponential relationship. Specifically, for an n-type MOS transistor, the subthreshold current is given by:

    ���� � �

    � ���

    � �����

    ��� ��

    � �

    � � ��� ��

    � �

    � �� �

    � �

    ��� ��

    � � ���


    � (1)

    where � and � are the width and length of the transistor, �� is the zero bias current, � is the subthreshold slope coefficient, �� is the thermal voltage, �� is the Early voltage and ��� , ��� and � � are the gate-to-source, drain-to-source and bulk- to-source voltages respectively. Typical values for devices with � � � � �� fabricated with standard �� technology are: �� � � �� � ����� A, � � � � , �� � � � V.

    If the transistor operates in saturation region (i.e. if ��� � ��� ) and if � �� ��� ��� � the above equation can be simplified to yield:

    ���� � �

    � ���

    � ������


    � (2)

    The diffusion of electrons through the transistor channel is mediated by the gate- to-source voltage difference. As the input/output characteristic of a subthreshold tran- sistor is an exponential function, circuits containing these devices can implement the “base functions” required to model biological processes: logarithms and exponentials.

    2.2 . . . to the Transconductance Amplifier

    On of the most common tricks used both by biological and engineered devices for computing measurements insensitive to absolute reference values and robust to noise, is the one of using difference signals. The differential pair is a compact circuit com- prising only three transistors that is widely used in many neuromorphic systems (see Fig. 1). It has the desirable property of accepting a differential voltage as input and providing in output a differential current with extremely useful characteristics: if the bias transistor is operated in the subthreshold domain and if we assume that all the

  • Vb

    V1 V2


    I1 I2


    −300 −200 −100 0 100 200 300 −2









    (V 1 − V

    2 ) (mV)

    ( I 1

    − I 2


    (n A



    Figure 1: (a) Circuit diagram of the differential pair. The differential output current ��� � is controlled by the differential input voltage ���� and scaled by a constant factor set by the bias voltage � . (b) Experimental data obtained from a differential transconductance amplifier with a bias voltage set to � � � �V.

    transistors are in saturation (so that equation 2 holds), the transfer function of the circuit is:

    �� � � � � �� ���� � � �

    ��� (3)

    The beauty of this transfer function lies in the properties of the hyperbolic tangent present in it: it passes through the origin with unity slope, it behaves in a linear fashion for small differential inputs and it saturates smoothly for large differential inputs.

    To provide in output the differential term ���� using a single terminal, one needs simply to connect a current mirror of complementary type to the differential pair out- put terminals (e.g. a current mirror of p-type MOS transistors in the case of Fig. 1). The circuit thus obtained would then be the famous differential transconductance am- plifier [17]. For small differential voltages this circuit’s transfer function (eq. (3)) is approximately linear:

    ���� � �

    ��� ��� � � � �� ����� � � � � (4)

    The reason for this name is the fact tha