Emergent Intelligence in Cellular Neural Networks Bipul Luitel and G. Kumar Venayagamoorthy

iambipul@ieee.org, gkumar@ieee.org

Real-Time Power and Intelligent Systems (RTPIS) Lab., Holcombe Dept. of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634

This work is supported by NSF grant EFRI #1238097 and CAREER grant ECCS #1231820. Visit: http://rtpis.org and http://brain2grid.org

Decentralized Asynchronous Learning in Cellular Neural Networks [1]

Communication Unit

Lear

nin

g U

nit

Co

mp

uta

tio

nal

Un

it

Application

Cellular Neural Network (CNN) A Generic Cell CNN in Decentralized Asynchronous Learning (DAL) Framework

Implementation (Hardware/software)

Formation

Structure: architecture and size

Centralized Decentralized

Parallel Sequential

Homogeneous Heterogeneous

Learning or adaptation

Learning Method

Adaptation

Asynchronous Synchronous

Homogeneous Heterogeneous

Subsystem: SSSystem: S

S

..

..

..

..

Z-1

)1( kOiSS

SS2(KS2,KD2)

SS1(KS1,KD1)

SSN(KSN,KDN)

SSi(KSi,KDi)

SSi(KSi,KDi)

(a)

(b)

)(kIiSS

Environment

<

Intelligence in CNN Emerges From A Group of Cells .. Learning of Learning Systems

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

2 3

1 4

Knowledge Propagation in CNN

C7

C3C2

C5

C4

C6

C9

C8 C1

C11

C10

C12

C13

C16

C15

C14

MLP SRN

CNN Implementation of 16-generator 68-bus System

Cell 1Cell 2Cell 3Cell 4Cell 5Cell 6Cell 7Cell 8Cell 9

Cell 10Cell 11Cell 12Cell 13Cell 14Cell 15

Cell 16

10 11 12 13 14 15 16 17 18 19 20

Time (s)

Results

Asynchronous Learning in 16-cells of a Heterogeneous CNN

Prediction of Generator Speed Deviation

Learning of learning systems is a social behavior of swarms where each individual learns at different pace, at different times and in different environment while still interacting with the other individuals of

the society. Learning in an artificial system having spatially distributed interacting individuals is known as learning of learning systems. In a decentralized asynchronous learning framework, learning takes place

locally on spatially distributed cells that learn asynchronously. The cells collaborate to achieve learning of the overall system. Cognitive learning takes place when parameters directly affecting the cell change and the cell has to update itself to reflect the change. This new acquired knowledge is then transferred to the

other members of the network (neighboring cells) through the communication unit. As a result, the neighbors observe a change in the behavior, and update themselves. The new knowledge acquired by

one cell thus propagates through he network which results in social learning.

Wide area monitoring system developed using CNN with DAL framework is able to predict speed deviation of 16 generators in a 16-generator, 68-bus system. The CNN is developed using heterogeneous structure (different computational units in different cells) as well as learning method (some cells trained using backpropagation while others using particle swarm optimization algorithm. Threshold error for learning in each cell is set at 5% .

-202

x 10-3

1

-202

x 10-3

2

-202

x 10-3

3

-20

2x 10

-3

4

-202

x 10-3

5

-2

0

2x 10

-3

6

-2

0

2x 10

-3

7

-2

0

2x 10

-3

8

-2

0

2

x 10-3

9

-2

0

2

x 10-3

10

-2

0

2

x 10-3

11

-2

0

2

x 10-3

12

-2

0

2

x 10-3

13

-2

02

x 10-3

14

-2

0

2

x 10-3

15

10 11 12 13 14 15 16 17 18 19

-202

x 10-3

16

Actual

Predicted

Time (s)

𝑂𝑆𝑆𝑖𝑘 = 𝑓 𝛼𝑖𝑂𝑆𝑆𝑖

𝑘 − 1 , 𝛼𝑛1𝑂𝑆𝑆𝑛

1 𝑘 − 1 , . . . , 𝛼𝑛𝑁𝑂𝑆𝑆𝑛

𝑁 𝑘 − 1 , 𝐾𝑆𝑖 , 𝐾𝐷𝑖

.. that may be either spatially collocated or distributed across a wide geographic area.

.. that may either be identical in architecture and size or different from

each other in their structure.

.. that are either implemented sequentially or in parallel hardware and/or software

platform.

.. that are learning subsystems, all of which either utilize the same or each of which

utilizes a different learning method.

.. that either adapt themselves in synchrony with the other subsystems or independent of the others in their own

pace. Intelligence in CNN emerges over time

through progressive learning and adaptation of these distributed interacting

subsystems represented as cells.

Cell

Computational Unit

Learning Unit

Dynamic

Database

Comm.

Unit

Dynamic

database

buffer

wid = 0,..,ws

Output

BufferI/O

+

-

T

Z-1

)(kIn

)1(

kOn

Computational

element

+

+

Error

Buffer

Target

ARE

If(u1 > u2)

T = 1

Else T = 0 AREt

u1

u2

Structural mirror of

computational

element

Z-1

Input

Target

+

-

[1] Luitel B., Venayagamoorthy G.K., “Decentralized Asynchronous Learning In Cellular Neural Networks,” IEEE Trans. On Neural Networks and Learning Systems, To Appear