5.5 Learning algorithms

• Neural Network inherits their flexibility and computational power from their natural ability to adjust the changing environments.

• So, they generate internal models of sampled environmental data.

• Represented in various “structured” weight vectors.

• NN models have a well defined architecture.• Dictated by pattern of connectivity of neurons.

Learning Algorithms:define an architecture dependent

procedureencode pattern information into weightsgenerate these internal models.

Learning encodes pattern information into inter-neuronal connection strengths.

Learning

• Most learning is data driven.• The data is the form of a set of input-output

patterns.• Derived from a possibly unknown classes.• Learning problem involves to generate a suitable

classification of samples.Learning algorithms are classified into 2 categories.

1. Supervised2. Unsupervised

Supervised LearningBasically it involves function approximation.

Learning contains a set of samples(T).T={(Xk,Dk)}k=1

Input vector: Xk £ Rn

Output vector: Dk£Rp

Explain the behavior of an unknown function f:Rn→Rp

(http://books.google.co.in/books?id=y67YnH4kEMsC&lpg=PA104&pg=PR10#v=onepage&q&f=false)

• Xk is an unput.

• Generates the output as Sk

• Use teaching input(Dk) to reduce the error.• Design to work with global information.• Instructs a behavioristic pattern.• NN makes no such assumptions.

Unsupervised learning- It involves some form of clustering of data.- Allow self organize method to generate the internal models of NN- Represent the entire data set to a small group of prototypical vectors.- Hold a desired level of discrimination between samples.

- New samples are inserted into a system.- So, the prototype will be in a state of constant flux.- No teaching input.- Adaptive vector quantization.

The set of data samples {Xi}, Xi £ R n has well defined clusters.

Clusters define a class of vectors(define in broad sense).

- Help establish a classification structure within a set. - no categories are defined in advance.- Quantization vectors are called code book vectors.- The unsupervised learning is self organized.- drived by intra-field neuronal competition and cooperation.- driven by a complex competitive-cooperative process.

• NN Learning algorithms Operate by iteratively adjusting the

weights in the network.The large amount of weights are

driven to improve the performance of the network.