Nanocomponent based neuromimetic memoires

1

Jacques-Olivier KleinIEF, Univ. Paris Sud/ CNRS

Weisheng ZhaoEmbedded Computing Lab, CEA LIST

Nanocomponent based neuromimetic memoires

Workshop: Innovative Memory Technologies, 24-06-200 9, Grenoble, France

2

Plan

• Why Nano Neuromimetic Memory?

• What are its main characteristics?

• Learning vs. Programming

• An implementation example

• Challenges

3

Von Neumann architecture and CMOS technology

•Stored program computer (Turing)

•Predetermined algorithms define capabilities (software)

•Electronics implementation of Boolean operators

•Based on well defined reproducible states

•“Bottleneck” : Throughout between the ALU and memory

•Low fault tolerance

•High speed

•Low power (limited by leakage power)

•Low mismatch and process Variation (difficult)

•Low cost (not true now!)

•High density (limited by atomic distance scale)

F. Wanlass, US Patent 3356858, 1967Memory hierarchy

CMOS technology

4

Nanotechnologies: Next big thing after CMOS

IBM, 2008

Moore’s law will continue?Von Neumann will continue?

•Nanotube

•Nanowire

•Graphene

•Memristor

•Domain Wall

•Molecular

•……

5

Why Neural Network?

New computing architectures are required!

Nanotechnology

• Small size (some nano meters)

• Self-assembly fabrication

Significant variation and high defect ratio!

Sample I Sample II

Multiple Carbon nanotube stripes

field-effect transistor (CNTFET)

Nanotubes

D

S

D

S

Q. Cao et al., Nature, 2008

G G

6

Why Neuromimetic Memoires? Nanotechnology

•Maximally Parallel computing (*)

•Based on the elements with large diversity (*)

•Extremely Low power

•Learning capabilities (e.g. environment)

•High fault tolerance (*)

•“Bottleneck” : Neuromimetic memories: synapse

Courtesy K.Meier

Based on CMOS technology, one synapse requires at least eight transistors

-K.Boahen, Stanford, Neural Computation, 2007

Neural Computation

7

Nanocomponent based Neuromimetic Memoires?

Artificial neural network

Biological neural network

•Historical memory

•Analog levels

•Extremely high density(104 synapses per neuron), human brain: 1010/cm2

Synapse

8

Plan

• Why Nano Neuromimetic Memory?

Nanotechnology Neural computation architecture

• What are its main characteristics?

• Learning vs. Programming

• An implementation example

• Challenges

9

I

VT+

VT-

V

RON

ROFF

VT+

VT-

V

Main characteristics required?

•Historical memory

•Multi levels (analog)

•Voltage or current threshold (control)

•Non-volatile

State 0State 1State 2State 3

Static behaviors Dynamic behaviors

10

Memristor

Strukov et al., Nature 453 (2008)

G. Snider et al., Nanoarchi (2008)

L.Chua et al., IEEE TCT 18 (1971)

W: State variable

11

Nanoscale Synaptic transistor

-5V +7V

I-I-

Lai et al., Nano Letter (2008)

The conductance can be configured to arbitrary states dynamically and reversibly by applying a series of Vg pulses with different amplitude, polarity and duration

12

Plan

• Why Neuromimetic Memoires?

• What are their main characteristics?

• Learning vs. Programming

• An implementation example

• Challenges

13

Learning vs. programming

Before

After

N

11

11

--11

N

For all the patterns

Convergence

??

??

??

14

Nano-Neuro-inspired Architectures

Learning circuit based on Memristor and CNTFET

He et al. EL 2008

Desired output

Actual output

Input

)( jjiij XYXW −×=∆ α

Delta learning rule

B. Widrow et al., 1960

2 CNTFET+1 Memristor

Learning pulses

15

Electrical simulation of one learning example

1- Learning pulses2 - Neuron output

3 - Expected output

For learning step = 1 to 6For input pattern = 000 to 111

For column = 1 to 8 (2x3 inputs + 2 thresholds)

16

Plan

• Why Neuromimetic Memoires?

• What are their main characteristics?

• Learning rules vs. Programming

• An implementation example

• Challenges

17

X1+ X1- X2+ X2- V+ V-

Inputs

Neural threshold

∆

V4 X4

Y4Fb4

∆

V5 X5

Y5Fb5

Learning driven by a feedback voltage

V

+VT / HZ / -VT

VT

Nano-Neuro-inspired Architectures

Desired output

Actual output

V+>VT

V-<-VT

Post Synaptic potential

18

Cases included

V

V V

V V

• Cases 2 and 4 : Patent FR0705532, 2007

• Other cases: Patent PCT/FR/050943, 2009

1

32

4 5

Muiti-Wall CNT Optical Gate-CNTFET

CBRAMMemristor

Nano-Neuro-inspired Architectures

J. Borghetti et al., Advanced Materials 2006

S. Dietrich et al., IEEE JSSCC 2007

PrototypeOG-CNTFET: Synapse

CMOS: Neuron

V

V

6

7

OxRAM ?

Phase change Nanowire

Y. Jung et al,. Nano letters(2008)

Baek, IEDM 2004

19

One example

Learning of 7 functions linearly-separable for 3 inputs

Good results

Errors

Distribution of the weights

Journal of Vacuum Science & Technology, 20 (6)

Nano-Neuro-inspired Architectures

V2

MWCNT

56 synapses

20

Plan

• Why Neuromimetic Memoires?

• What are their main characteristics?

• Learning vs. Programming

• An implementation example

• Challenges

21

Challenges: Joint efforts are required

• Nanofabrication

• Circuit design

• Learning rules

• Architecture and system (we are here)

• Prototyping

• Neuroscience (not well understood)

22

Thank you very much!

23

Learning rules: Spiking Time Dependant Plasticity ( STDP)

i j

t

Bi and Poo J. Neurosci. 1998Yang et al., Nature Neuroscience 2008