Computation in Neuromorphic Analog VLSI
Institute of Neuroinformatics, University/ETH Zurich
In this paper we present an overview of basic neuromorphic analog circuits that
are typically used as building blocks for more complex neuromorphic systems.
We present the main principles used by the neuromorphic engineering commu-
nity and describe, as case example, a neuromorphic VLSI system for modeling
selective visual attention.
1 Neuromorphic Engineering
The term “neuromorphic” was coined by Carver Mead to describe very large scale
integration (VLSI) systems containing electronic analog circuits that mimic neuro-
biological architectures present in the nervous system . Neuromorphic computa-
tion is related to modeling and simulation of networks of neurons and systems using
the same organizing principles found in real nervous system. In recent times the term
“neuromorphic” has also been used to describe mixed analog/digital VLSI systems
that implement computational models of real neural systems. These VLSI systems,
rather than implementing abstract neural networks only remotely related to biologi-
cal systems, in large part, directly exploit the physics of silicon (and of CMOS VLSI
technology) to implement the physical processes that underlie neural computation.
Neuromorphic engineering is a new discipline at the boundary between engineer-
ing and neuroscience, but which crosses many other fields, including biology, physics,
computer science, psychology, physiology, etc.
Analog VLSI Circuits
There are some direct analogies between biological neural systems and analog
VLSI neuromorphic systems: conservation of charge, amplification, exponentiation,
thresholding, compression, and integration. The parallels between these two worlds
run from the level of device physics to circuit architectures:
� Diffusion mechanisms in membrane and transistor channels
� Function determined by the system’s structure
To point out why studying natural neural systems and implementing models of
these systems, using analog VLSI technology can be potentially instrumental for im-
proving technological progress, consider the following arguments:
1. Biology has solved the problem of using loosely coupled, globally asynchronous,
massively parallel, noisy and unreliable components to carry out robust and en-
ergy efficient computation.
2. CMOS VLSI technology has been continuously improving, following Moore’s
law, for almost 30 years, pushed by a hyper-competitive $300 billion global in-
dustry. But, as CMOS technology improves further and transistor’s gate lengths
drop below 0.10 microns
� transistors start to behave like neurons
� we have the ability to place millions of simple processors on a single piece
� single chips start to have complexities that are too difficult to handle by
the design tools
� we need to begin to worry about power consumption and power dissipation
Computation and Power Consumption
The brain has on the order on ���� neurons and ���� synapses. It performs on av-
erage ���� operations per second. The power dissipation of the brain is approximately
�����J per operation, which results in about a total mean consumption of less than 10
watts. By comparison today’s silicon digital technology can dissipate at best ����J
of energy per operation at the single chip level. There is no way of achieving �� ��
operations per second on a single chip, with today’s technology. But even if this was
possible, to do that amount of computation a digital chip would consume MegaWatts
(the output of a nuclear power station).
Any serious attempt to replicate the computational power of brains must confront
this problem. Subthreshold analog circuits are also no match for real neural circuits,
but they are a factor of ��� more power efficient than their digital counterparts.
In the following Section we present some basic analog circuits that are most com-
monly used as elementary building blocks for constructing complex neuromorphic
systems. In Section 3 we present a VLSI device containing a model of selective visual
attention, that uses many of the circuits described in Section 2, as a case example.
Finally in Section 4 we draw the conclusions.
2 Subthreshold Analog Circuits
Perhaps the most elementary computational element of a biological neural structure is
the neural cell’s membrane. The nerve membrane electrically separates the neuron’s
interior from the extra-cellular fluid. It is a very stable structure that behaves as a per-
fect insulator. Current flow through the membrane is mediated by special ion channels
(conductances) which can behave as passive or active devices. In the passive case, ion
channels selectively allow ions to flow through the membrane by the process of dif-
fusion. In electronics, it is possible to implement the same physical process by using
MOS transistor devices, operated in the subthreshold region (also referred to as weak
inversion) [17, 20].
2.1 From subthreshold MOS transistors . . .
MOS transistors operate in the subthreshold region of operation when their gate-to-
source voltage is below the transistor threshold voltage. This mode of operation of
a transistor has been largely ignored by the analog/digital circuit design community,
mainly because the currents that flow through the source-drain terminals of the device
under these conditions are extremely low (typically of the order of nanoamperes).
In subthreshold, the drain current of the transistors is related to the gate-to-source
voltage by an exponential relationship. Specifically, for an n-type MOS transistor, the
subthreshold current is given by:
where � and � are the width and length of the transistor, �� is the zero bias
current, � is the subthreshold slope coefficient, �� is the thermal voltage, �� is the
Early voltage and ��� , ��� and � � are the gate-to-source, drain-to-source and bulk-
to-source voltages respectively. Typical values for devices with � � � � ��
fabricated with standard �� technology are: �� � �
�� � ����� A, � � �
� , �� �
If the transistor operates in saturation region (i.e. if ��� � ��� ) and if � �� ���
��� � the above equation can be simplified to yield:
The diffusion of electrons through the transistor channel is mediated by the gate-
to-source voltage difference. As the input/output characteristic of a subthreshold tran-
sistor is an exponential function, circuits containing these devices can implement the
“base functions” required to model biological processes: logarithms and exponentials.
2.2 . . . to the Transconductance Amplifier
On of the most common tricks used both by biological and engineered devices for
computing measurements insensitive to absolute reference values and robust to noise,
is the one of using difference signals. The differential pair is a compact circuit com-
prising only three transistors that is widely used in many neuromorphic systems (see
Fig. 1). It has the desirable property of accepting a differential voltage as input and
providing in output a differential current with extremely useful characteristics: if the
bias transistor is operated in the subthreshold domain and if we assume that all the
−300 −200 −100 0 100 200 300
Figure 1: (a) Circuit diagram of the differential pair. The differential output current
��� � is controlled by the differential input voltage ���� and scaled by a constant
factor set by the bias voltage �
. (b) Experimental data obtained from a differential
transconductance amplifier with a bias voltage set to �
transistors are in saturation (so that equation 2 holds), the transfer function of the
�� � � � �
���� � � �
The beauty of this transfer function lies in the properties of the hyperbolic tangent
present in it: it passes through the origin with unity slope, it behaves in a linear fashion
for small differential inputs and it saturates smoothly for large differential inputs.
To provide in output the differential term ���� using a single terminal, one needs
simply to connect a current mirror of complementary type to the differential pair out-
put terminals (e.g. a current mirror of p-type MOS transistors in the case of Fig. 1).
The circuit thus obtained would then be the famous differential transconductance am-
plifier . For small differential voltages this circuit’s transfer function (eq. (3)) is
���� � �
��� � � � �� ����� � � � � (4)
The reason for this name is the fact tha