Neural Network in Matlabshibayan/MCC 52105 Optimiza… · · 2018-01-20The most useful neural networks in function approximation are Multilayer Layer Perceptron ... Neural Network

Neural Network in MATLAB

Neural Network in Matlab

1

“A neural network is an interconnected assembly of simple processing elements, units or nodes, whose functionality is loosely based on the animal neuron. The processing ability of the network is stored in the inter-unit connection strengths, or weights, obtained by a process of adaptation to, or learning from, a set of training patterns.”


Basics of Neural Network (NN): Dendrites carry signals into the

neuron body. The neuron body integrates the incoming signals. The axon is a single long nerve fiber that carries the signal from the neuron body to other neurons.

Incoming signals to a dendrite may be inhibitory or excitatory. The strength of any input signal is determined by the strength of its synaptic connection. A neuron sends an impulse down its axon if excitation exceeds inhibition by a critical amount (threshold/ offset/bias) within a time window (period of latent summation).

Memories are formed by the modification of the synaptic strengths which can change during the entire life of the neural systems. A synapse is the connection between dendrites of two neurons.

2

Bias neurons are added to neural networks to help them learn patterns. As the bias neurons have a constant input of one they are not connected to the previous layer. a bias value allows you to shift the activation function to the left or right, which may be critical for successful learning.

inhibitory :Restrictive of action excitatory : able to excite or stimulate


McCulloch-Pitts Model :

3


Implementation: Loading data source. Selecting attributes required. Decide training, validation, and testing data. Data manipulations and Target generation. Neural Network creation (selection of network

architecture) and initialisation. Network Training and Testing. Performance evaluation.

4

The most useful neural networks in function approximation are Multilayer Layer Perceptron (MLP) and Radial Basis Function (RBF) networks. Here we concentrate on MLP networks.

ANN is rarely used for predictive modelling. The reason being that Artificial Neural Networks (ANN) usually tries to over-fit the relationship. ANN is generally used in cases where what has happened in past is repeated almost exactly in same way.


5

A neural network can be trained to perform a particular function by adjusting the values of the connections (weights) between elements.

Commonly neural networks are adjusted, or trained, so that a particular input leads to a specific target output.

Here, the network is adjusted, based on a comparison of the output and the target, until the network output matches the target. Typically many such input/target pairs are used, in this supervised learning, to train a network.

Batch training of a network proceeds by making weight and bias changes based on an entire set (batch) of input vectors. Incremental training changes the weights and biases of a network as needed after presentation of each individual input vector. Incremental training is sometimes referred to as “on line” or “adaptive” training.


Multilayer Layer Perceptron (MLP) : A MLP consists of an input layer, several hidden layers (to transform

inputs into something that output layer can use), and an output layer.

Node i, also called a neuron. lt includes a summation and a nonlinear activation /transfer function g.

ni is the input to the activation function g. activation function was mathematical convenience a hyberbolic

tangent (tanh) or a sigmoid function are most commonly used. 6

Fig 1 Single node in a MLP network


Multilayer Layer Perceptron (MLP) :

7


MLP using Hidden Layer :

8

Connecting several nodes in parallel and series, a MLP network is formed. The introduction of hidden layer(s) makes it possible for the network to exhibit non-linear behavior. The optimal number of hidden units could easily be smaller than the number of inputs, there is no rule like multiply the number of inputs with N. sometimes just 2 hidden units works best with little data. Researches show that many hidden layers can be fruitful for difficult object.

21n

Number of Layer


MLP: back-propagation The universal approximation theorem for neural

networks states that every continuous function that maps intervals of real numbers to some output interval of real numbers can be approximated arbitrarily closely by a multi-layer perceptron with just one hidden layer. This result holds for a wide range of activation functions, e.g. for the sigmoidal functions.

9

Multi-layer networks use a variety of learning techniques, the most popular being back-propagation. Here, the output values are compared with the correct answer to compute the value of some predefined error-function. By various techniques, the error is then fed back through the network. Using this information, the algorithm adjusts the weights of each connection in order to reduce the value of the error function by some small amount. After repeating this process for a sufficiently large number of training cycles, the network will usually converge to some state where the error of the calculations is small. In this case, one would say that the network has learned a certain target function.

https://en.wikipedia.org/wiki/Universal_approximation_theorem

https://en.wikipedia.org/wiki/Back-propagation


Basics using MATLAB NN Toolbox

10

The MATLAB commands used in the procedure are newff (type of architecture , size and type of training algorithm), train and sim. newff: create a feed-forward backpropagation network The MATLAB command newff generates a MLPN neural network, which is called net.


11

MATLAB Programming Architecture

x1

x2

x3

Ʃ

Ʃ

Ʃ

Ʃ

Ʃ

Ʃ

Ʃ

Ʃ

Ʃ

Ʃ

g (n11)

g (n12)

g (n13)

g (n14)

g (n15)

g (n16)

g (n17)

g (n18)

g (n19)

g (n110 )

Ɵ11

Ɵ12

tansig

Ʃ

Ɵ110

g (n21)

Ɵ21

purelin

( )2

2 11 n

ae−

= − +

Declaration of [R]

Declaration of [S]


Expressions a = tansig(n) = 2/(1+exp(-2*n))-1

12


Plotting tips

13


Thank You…

14

Documents

Neural Network in Matlabshibayan/MCC 52105 Optimiza… · · 2018-01-20The most useful neural networks in function approximation are Multilayer Layer Perceptron ... Neural Network