Chaos in Neural Network

Theme presentation

Cui, Shuoyang 03/08/2005

Artificial Neural Network (ANN) is

Neural Networks are a different paradigm for computing:

•neural network machines are based on the processing/memory abstraction of human information processing.

•neural networks are based on the parallel architecture of animal brains.

Neural networks versus conventional computers

Conventional computers use an algorithmic approach. the problem solved are that we already understand and know how to solve.

The network is composed of a large number of highly interconnected processing elements(neurones) working in parallel to solve a specific problem

They cannot be programmed to perform a specific task

Neural networks and conventional algorithmic computers complement each other

The disadvantage is that the network finds out how to solve the problem by itself, its operation can be unpredictable.

An important application of neural networks is pattern recognition

Neural networks are a form of multiprocessor computer system,

with •simple processing elements

•a high degree of interconnection

•simple scalar messages

•adaptive interaction between elements

Architecture of neural networks

Feed-forward networks

allowing signals to travel

one way only

Feedback networks

have signals travelling in both directions by introducing loops in the network

Chaos and random system

• Chaos is statistically indistinguishable from randomness, and yet it is deterministic and not random at all

.

• Chaotic system will produce the same results if given the same inputs, it is unpredictable in the sense that you can not predict in what way the system's behavior will change for any change in the input to that system. a random system will produce different results when given the same inputs.

Chaos has periodic, predictable behavior and totally random behavior

It is random-appearing, and yet has a large degree of underlying order.

Skarda & Freeman(1987) found chaotic activity in the brain

Freeman(1991) decided that chaos "may be the chief property that makes the brain different from an artificial-intelligence machine"

For brain of human: researchers believe chaotic background behavior is necessary for the brain to engage in continual learning.

Background of Chaos

Chaotic neural networks

Chaotic neural networks offer greatly increase memory capacity.Each memory is encoded by an Unstable Periodic Orbit (UPO) on the chaotic attractor.

A chaotic attractor is a set of states in a system's state space with very special properties:

the set is an attracting set. So that the system, starting with its initial condition in the appropriate basin, eventually ends up in the set.

Tnd most important, once the system is on the attractor nearby states diverge from each other exponentially fast. Small amounts of noise are amplified.

A Novel Chaotic Neural Network Architecture

Nigel Crook and Tjeerd olde Scheper

The delayed feedback method is considered to be best suited to the control of chaos in neural networks.

• the delay feedback method does not rely on a priori knowledge

• the delayed feedback method does not specify which UPO is to be stabilized

• delays in signal transmission are inherent in all biological neuronal networks.

• The feedback control method amounts to delayed inhibition

Pyragas’s delayed feedback method

P(y,x) and Q(y,x) govern the chaotic dynamics of the system;output variable, y(t); input signal, F(t);a delay time,strength of the feedback K.

variable y repeats a value specified by the delay at the earlier time.

F(t) system back to a state (periodic system coming)

periodic system F(t) will become very small

this method UPOs of different periodicities can be controlled

Each unit in the Chaotic Layer is governed by the following discrete time equations:Modified by K. Aihara, T. Takabe, and I. Tsuda in Chaotic Neural Networks

yi(t) : the internal state of unit i at time t, (0 < <1)a and w : parameters of the Aihara modelM : the number of units in the Chaotic Layer N : the number of units in each inhibitory cluster. wij : a weight of the connections between units in the chaotic layerija : time delayxi(t) : the output activation of chaotic unit i at time t. kij : the weights of connections from the inhibitory units to the chaotic unitszj(t) : the activation of chaotic unit j at time tf(y) is given by:

Inputs : xi(t), the activation of the chaotic unit i at time t.

&

xi(t -Dji), which is the activation of chaotic unit i at time t -Dji, where Dji is a randomly selected time delay.

Each inhibitory unit one output + two inputs

Each inhibitory unit has a different randomised time delay connection with the associated chaotic unit.

Compete in the inhibitory units within a cluster

L : the number of input units

Ik(t) : the activation of the kth input unit at time t.

winner :the inhibitory unit with the smallest value for h(t) activation value is

And then activation values of other units in that inhibitory cluster 0

Experiment

• 1 input unit

• 3 units in each inhibitory cluster

• 4 chaotic units

• The network was then iterated for a further 200 times steps.

• The input patterns used were: Input sequence (a) 1.0, 0.5 • The activations of the input units at each time step are governed entirely by

the input sequences consisting of discrete values in the range 0 to 1.

For sequence (a)

I(1) = 1.0,I(2) =0.5, I(3) =1.0, I(4) =0.5, etc.

The activations of 2 units from the Chaotic Layer

stabilized into periodic after 200 timeseach of the chaotic units is stabilised to orbits with different periods

The activations of the units in 2 of the inhibitory clusters

References

• Nigel Crook and Tjeerd olde Scheper, A Novel Chaotic Neural Network Architecture

• W.J. Freeman and J.M. Barrie, Chaotic oscillations and the genesis ofmeaning in cerebral cortex. Temporal Coding in the Brain

• T. Shinbrot, C. Grebogi, E. Ott, and J.A. Yorke, Using Small Perturbations to Control Chaos Nature,

• M.R. Guevara, L. Glass, M.C. Mackey, and A. Shrier,Chaos in Neurobiology

• A. Babloyantz and C. Lourenco, Brain Chaos and Computation

• C. Lourenco and A. Babloyantz, Control of spatiotemporal chaos in neural networks