An Artificial Brain Mechanism

to Develop a Learning

Paradigm for RobotsManish Kumar1, Rashmi Jha2

1Associate Professor, Department of Mechanical Engineering,

University of Cincinnati, Cincinnati, Ohio

Manish.kumar@uc.edu, 513-556-53112Associate Professor, Dept. of Electrical Engineering and

Computing Systems, University of Cincinnati, Cincinnati, Ohio

rashmi.jha@uc.edu, 513-556-1361

Need for Biological Brain Inspired Ultra-Low

Energy Computing

Human vs. Robot for Space Exploration

Humans hold a number of advantages over robots. They

can make quick decisions in response to changing

conditions or new discoveries, rather than waiting for time-

delayed instructions from Earth.

http://www.wired.com/2012/04/space-humans-vs-robots/

Limitations of Currently Available

Machine Learning: Deep Neural Network

(DNN)

DNN on conventional computing architecture are

compute intensive, power hungry, need a large set

of training data , and are trained to solve just some

specific sets of problems.

Limitations of Traditional CMOS

Transistor Scaling and Computing

http://epc-co.com/epc/EventsandNews/FastJustGotFasterBlog/Issue11.aspx

Cost of CMOS

transistor is rising at

20 nm node and

beyond for the first

time in history.

http://www.extremetech.com/computing/116561-the-death-of-cpu-scaling-from-

one-core-to-many-and-why-were-still-stuck

Motivation for Artificial Brain

“Taking advantage of the almost 83,000

processors of one of the world's most

powerful supercomputers, the team was able to

mimic just one percent of one second's worth

of human brain activity—and even that took 40

minutes.” – Gizmodo, 2013

“Challenge is to

create an exascale

computing system

by 2018 that

consumes only 20

megawatts (MW) of

power.”- DOE grand

challenge.

Supercomputers

“1014 Neurons, 1015

Synapses, 1013 to 1016

Instructions per sec, 10 W

of Power (e.g. retinal

operation).”

BrainScalable

Ultra Low-

Energy

Motivation for Artificial Brain

“Taking advantage of the almost 83,000

processors of one of the world's most

powerful supercomputers, the team was able to

mimic just one percent of one second's worth

of human brain activity—and even that took 40

minutes.” – Gizmodo, 2013

“Challenge is to

create an exascale

computing system

by 2018 that

consumes only 20

megawatts (MW) of

power.”- DOE grand

challenge.

Supercomputers

“1014 Neurons, 1015

Synapses, 1013 to 1016

Instructions per sec, 10 W

of Power (e.g. retinal

operation).”

BrainScalable

Ultra Low-

Energy• How does a biological “Brain” work ??

• How can we make an artificial brain on

chip???

How Does a Biological Brain

Process Information?

https://computing.llnl.gov/tutorials/parallel_comp/

Von-Neumann

Architecture

Brain-Inspired Paradigm of Computing

Components: Neurons,

Reconfigurable Synapses,

Interconnects

STDP

Syn

aptic E

ffic

acy

Today’s Computing

Markram et. al., Front. Synp. NeurosSc. 2011

Neuron Operation and Action Potential Firing

Synapse

Sensory Signal Processing

Temperature,

odor etc.

(Effector Cells)

Central Nervous

System

What does it mean from neuro-

inspired device perspective?

• High fan-out spiking device

• Ultra-low energy consumption ~10 fJ/spike

• Scalable

• High reliability and endurance

• Reconfigurable

• Ultra Low-power

• Scalable

• High endurance and reliability

• Minimal Variability

Neuron

Synapse

Sensory Information Coding

Spike Coding of Odor

Mainland et. al., Trends in Neurosciences August 2014, Vol. 37, No. 8

Sensory Neurons in Silicon

Axon-Hillock Circuit, proposed by Prof. Carver Mead,

1980’sIndiveri et. al., Frontiers in Neuroscience, 2011

Learning Algorithms• Supervised Learning

– Feed-Forward

– Back-Propagation

– Gradient-Descent

• Unsupervised Learning for Spiking Neural Network

– Hebbian Learning (Spike Timing Dependent Plasticity)

• Neurons that fire together, wire together

• Basis of Associative Memory

Spike Timing Dependent

Plasticity

Which synapses are

strengthened? Which ones are

depressed?

Bi et. al. J. of NeuroSci, 1998

17

Doped Oxide Dynamics for Synaptic Memory

10-13

10-12

10-11

10-10

10-9

-0.6 -0.4 -0.2 0 0.2 0.4 0.6

Initial IV

Device 1Device 2Device 3Device 4Device 5Device 6Device 7Device 8Device 9Device 10

Cu

rre

nt

(A)

Voltage (V)

0

2 10-6

4 10-6

6 10-6

8 10-6

1 10-5

1.2 10-5

0 0.5 1 1.5 2 2.5 3

Positive Hysteresis

Cycle1Cycle2Cycle3Cycle4Cycle5Cycle6Cycle7Cycle8Cycle9Cycle10Cycle11Cycle12Cycle13Cycle14Cycle15Cycle16

Cu

rre

nt

(A)

Voltage (V)

-1.4 10-7

-1.2 10-7

-1 10-7

-8 10-8

-6 10-8

-4 10-8

-2 10-8

0

-1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0

Negative Hysteresis

Cycle1

Cycle2

Cycle3

Cycle4

Cycle5

Cu

rre

nt

(A)

Voltage (V)

100µm x 100 µm

Mandal/Jha et. al., Nature Sci. Rep, 2014

1

1.5

2

2.5

3

3.5

4

4.5

0 100 200 300 400 500 600 700

Conductance vs Pulse 1ms

Excitation (5Hz)Depression (5Hz)Excitation (15Hz)Depression (15Hz)

Cu

rre

nt

@0

.5V

rea

d (

nA

)

Pulse #10 Potentiating Pulses 2.5V/1ms applied at given frequency and Current measured at 0.5V read after excitation.

Potentiation was repeated 30 times (total 300 pulses) with conductance measuring intervals of 10 pulses. After

potentiation for 300 pulses, 300 depression pulses at -1.5V/1ms were applied at the given frequency and

measurement of current at 0.5V read was done in intervals of 10 pulses.

18

Potentiation and Depression with Pulses

1

2

3

4

5

6

0 100 200 300 400 500 600 700

Conductance vs Pulse 10ms

Excitation (5Hz)Depression (5Hz)Excitation (15Hz)Depression (15Hz)

Cu

rre

nt

@0

.5V

rea

d (

nA

)

Pulse #

19

Endurance with Pulses of Different Pulse-Widths

10

15

20

25

30

35

40

0 200 400 600 800 1000 1200

1 ms Pot.1 ms Dep.10ms Pot.10 ms Dep.20 ms Pot.20 ms Dep.50 ms Pot.50 ms Dep.100 ms Pot.100 ms Dep.200 ms Pot.200 ms Dep.

Cu

rren

t @

0.5

V r

ea

d (

nA

)

Cycle #

20

Distribution over Cycle to Cycle

0

50

100

150

200

250

300

14 16.8 19.6 22.4 25.2 28

Pot. (50ms)Dep. (50ms)

Co

un

t

Range (nA)

dep.

: 18.53

dep.

: 0.356 pot.

: 25.303

pot.

: 0.896

0

50

100

150

200

16 20 25 30 35 40

Pot. (200ms)Dep. (200 ms)

Co

un

tRange (nA)

dep.

: 19.7548

dep.

: 1.493

pot.

: 33.6524

pot.

: 1.3408

0

50

100

150

200

12 14 17 19 22 24

Pot. (10 ms)Dep. (10 ms)

Co

un

t

Range (nA)

dep.

: 16.116

dep.

: 0.586

pot.

: 20.3828

pot.

: 0.8272

0

50

100

150

200

250

10.8 12.8 14.8 16.8 18.8 20.8

Pot.1msDep. 1ms

Co

un

t

Range (nA)

dep.

: 12.936

dep.

: 0.2416

pot.

: 14.766

pot.

: 0.57

0

50

100

150

200

12 14 16 18 20 22 24

Pot. (20ms)Dep. (20ms)

Co

un

t

Range (nA)

dep.

: 16.394

dep.

: 1.55

pot.

: 21.478

pot.

: 2.876

0

50

100

150

200

12 16 20 24 28 32

Pot. (100 ms)Dep. (100 ms)

Co

un

t

Range (nA)

dep.

: 18.7035

dep.

: 0.982

pot.

: 27.9565

pot.

: 0.9163

21

Spike Timing Dependent Plasticity

-60

-40

-20

0

20

40

60

80

-60 -40 -20 0 20 40 60

%a

ge c

han

ge

t (ms)

(t)=83.03*exp(-t/41.3)

(t)=-47.03*exp(t/59.03)

=1.3713

=0.9934

Δt (+/-) Feedback (2.5V/-1.5V)

10 ms 200 ms

20 ms 100 ms

30 ms 50 ms

40 ms 20 ms

50 ms 10 ms

Synaptic Memory Device Model

Ru

Mn:HfO2

TiN/W

Ru

Mn:HfO2

TiN/

W

Ru

Mn:HfO2

TiN/

W

Initial State Potentiation (LTP) Depression (LTD)

-

+

𝐼 = 𝑞𝜇𝐸𝐴𝑛0 exp −𝜙𝐵 −

𝑞𝐸𝜋𝜀

𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛2 exp −

𝜙𝐵 −𝑞𝐸𝜋𝜀

𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛1 exp −𝜙𝐵 −

𝑞𝐸𝜋𝜀

𝑘𝑇

-

+ +

-

Mandal/Jha et. al., Nature Sci. Rep, 2014

Sarim/Kumar/Jha et. al., NAECON, 2016

Neuromorphic Platform

• A neuromorphic platform isconfigured as an array ofseveral Synaptic Memorydevices arranged as shown.

• The arrays are connected toproximity sensors that sendin the information about thevicinity of the robot. Themotor neuron circuits movethe robot wheels.

Simulation Framework• Two different models, viz., mathematical device model and

experimentally derived device model, for synaptic memory

devices with the neuromorphic platform were implemented

to demonstrate unsupervised learning in a robot.

• This approach was validated by simulating the robot to

navigate in an unknown environment with randomly placed

obstacles.

• The commercially available Khepera III robot [4] is modeled

with a two-wheeled differential drive robot kinematics. The

robot consists of five ultrasonic sensors that give the

information about the vicinity of the robot.

[4] http://www.k-team.com/mobile-robotics-products/old-products/khepera-iii

𝜈 =𝑟

2𝜔𝑅 + 𝜔𝐿

𝜔 =𝑟

𝑏𝜔𝑅 − 𝜔𝐿

𝑥 = 𝜈 cos 𝜃 𝑦 = 𝜈 sin 𝜃 𝜃 = 𝜔

Robot Kinematics

,Gianluca et. al., IEEE Transactions on Robotics 21.5 (2005): 994-1004.

Learning Scheme

target

Device Model:

𝑤 = 𝑓 𝜈𝑀𝑅

𝑓 𝜈𝑀𝑅 = 𝐼0𝑠𝑖𝑔𝑛 𝜈𝑀𝑅 𝑒 𝜈𝑀𝑅 /𝜈0 − 𝑒𝜈𝑡ℎ/𝜈0

𝜈𝑀𝑅 𝑡, Δ𝑇 = 𝛼𝑝𝑜𝑠𝑠𝑝𝑘 𝑡 − 𝛼𝑝𝑟𝑒𝑠𝑝𝑘 𝑡 + Δ𝑇

Δ𝑤 Δ𝑇 = 𝑓 𝜈𝑀𝑅 𝑡, Δ𝑇 𝑑𝑡 = 𝜉 Δ𝑇

STDP learning function:

𝜉 Δ𝑇 = 𝑎+𝑒− Δ𝑇 𝜏+ 𝑖𝑓 Δ𝑇 > 0

−𝑎−𝑒− Δ𝑇 𝜏− 𝑖𝑓 Δ𝑇 < 0

Mathematical Model

change in structural parameter of the device

memristor voltage

where

Δ𝑇 is the difference in the spike times of pre-

and post-synaptic neurons

𝑠𝑝𝑘 𝑡 is the spike shape

𝐼0 , 𝑣0 are device parameters, 𝑣𝑡ℎ is the

threshold voltage of the device above which it

spikes, 𝛼 are attenuation parameters in pre- and

post-synaptic neurons. 𝜉 is the change in the

synaptic weight that is used to implement STDP.

Robot Navigation Results

1 2 3

4 5 6

o : start | × : target

Navigation Results with Synaptic Memory

Device Model

Ru

Mn:HfO2

TiN/W

Ru

Mn:HfO2

TiN/

W

Ru

Mn:HfO2

TiN/

W

Initial State Potentiation (LTP) Depression (LTD)

-

+

𝐼 = 𝑞𝜇𝐸𝐴𝑛0 exp −𝜙𝐵 −

𝑞𝐸𝜋𝜀

𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛2 exp −

𝜙𝐵 −𝑞𝐸𝜋𝜀

𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛1 exp −𝜙𝐵 −

𝑞𝐸𝜋𝜀

𝑘𝑇

-

+ +

-

Mandal/Jha et. al., Nature Sci. Rep, 2014

Sarim/Kumar/Jha et. al., NAECON, 2016

Robot Navigation Results

1 2 3

4 5 6

o : start | × : target

Conclusions• We demonstrated the potential for having an onboard

“artificial brain” for Robots based on emerging

neuromorphic devices.

• Using artificial brain architecture, a successful Robotic

navigation was demonstrated using unsupervised learning

scheme to guide the robot in complex environments using

the local knowledge of obstacles only.

– Our approach overcomes the issue of local minima

which is a challenge for other navigation algorithms.

• Our approach is projected to be highly energy-efficient and

scalable for implementation on any robot.

• Future work is targeted towards the actual implementation

of these neuromorphic devices based artificial brain on

Robots and field verification of the navigation.

Student Contributors

1. Mohammad Sarim

Robotics Lab, Department of Mechanical and Materials

Engineering, University of Cincinnati

sarimmd@mail.uc.edu

2. Thomas Schultz

EDACS Lab, Department of Electrical Engineering and

Computing Systems , University of Cincinnati

3. Saptarshi Mandal (Now at Arizona State University)

Acknowledgement• This project is currently supported by National Science

Foundation under CAREER (Award # 1556294), and SaTC (Award # 1556301).

• We would like to thank Dr. Mark Ritter and his group at IBM TJ Watson Research Center.

• We would like to thank our collaborators Dr. GennadiBersuker (Sematech), Dr. David Gilmer (Sematech), Dr. Prashant Majhi (Intel), Dr. Kevin Leedy (AFRL), Dr. Marc Cahay (U. of Cincinnati), Dr. Ali Minai ( U. of Cincinnati), Dr. Swaroop Ghosh (USF), Dr. Scott Molitor (U.Toledo).

Thank You!

Questions and Suggestions?