Neural Networks

By:

RIZWAN M H

DATAMINING & WAREHOUSING

There are many ways of classifying the techniques

These techniques consist of the specific algorithms that can be used for each function

Application areas where these techniques used:

* Fraud Detection* Risk Assessment* Market Analysis

Data Mining Techniques

Cluster Detection Decision Trees Memory based Reasoning Link Analysis Neural Networks Genetic Algorithms Data Visualization

Data Mining Techniques

Neural Networks is one of the Data Mining techniques.

A Neural Networks is an information processing paradigm that is inspired by biological nervous systems.

The basic unit of a neural network (NN) is modeled after the neurons in the brain.

It is composed of a large number of highly interconnected processing elements called neurons.

This unit is known as node.

Neural Networks

Neural Networks are analytic technique modelled after the learning process

Ability to derive meaning from complicated or imprecise data

Extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques

Adaptive learning Real Time Operation

Why Neural Networks.??

Neural Network Model

VIDEO

 Conventional computers use an algorithmic approach, but neural networks works similar to human brain and learns by example.

Neural Networks v/s Conventional Computers

A simple neuron

Takes the Inputs . Calculate the

summation of the Inputs .

Compare it with the threshold being set during the learning stage.

A firing rule determines how one calculates whether a neuron should fire for any input pattern.

Some sets cause it to fire (the 1-taught set of patterns) and others which prevent it from doing so (the 0-taught set)

Firing Rules

Example…

For example, a 3-input neuron is taught to output 1 when the input (X1,X2 and X3) is 111 or 101 and to output 0 when the input is 000 or 001.

X1:

0 0 0 0 1 1 1 1

X2:

0 0 1 1 0 0 1 1

X3:

0 1 0 1 0 1 0 1

OUT:

0 00/1

0/1

0/1

10/1

1

Example…

After applying firing rule, the truth table becomes,

The difference between the two truth tables is called the generalisation of the neuron.

X1:

0 0 0 0 1 1 1 1

X2:

0 0 1 1 0 0 1 1

X3:

0 1 0 1 0 1 0 1

OUT:

0 0 00/1

0/1

1 1 1

Fixed networks in which the weights cannot be changed, ie dW/dt=0.

In such networks, the weights are fixed a priori according to the problem to solve.

 Adaptive networks which are able to change their weights, ie dW/dt != 0.

Types of neural network

 Associative mapping:  Network learns to produce a particular pattern on

the set of input units whenever another particular pattern is applied on the set of input units.

Auto - Association Hetero - Association

The Learning Process

Auto-association: An input pattern is associated with itself and the

states of input and output units coincide. This is used to provide pattern completion,

Associative Mapping

Hetero-association: Nearest-neighbour recall : the output pattern

produced corresponds to the input pattern stored, which is closest to the pattern presented.

 Interpolative recall : where the output pattern is a similarity dependent interpolation of the patterns stored corresponding to the pattern presented.

Associative Mapping

• Supervised learning which incorporates an external teacher, so that each output unit is told what its desired response to input signals ought to be.

Supervised Learning

• During the learning process global information may be required.

• Paradigms of supervised learning include error-correction learning, reinforcement learning and stochastic learning.

• An important issue concerning supervised learning is the problem of error convergence.

• The aim is to determine a set of weights which minimises the error.

Supervised Learning

Unsupervised learning uses no external teacher and is based upon only local information. It is also referred to as self-organisation.

Unsupervised Learning

We say that a neural network learns off-line if the learning phase and the operation phase are distinct.

A neural network learns on-line if it learns and operates at the same time.

Usually, supervised learning is performed off-line, whereas unsupervised learning is performed on-line.

Unsupervised Learning

The behaviour of an ANN (Artificial Neural Network) depends on both the weights and the input-output function (transfer function) that is specified for the units. This function typically falls into one of three categories:

 linear (or ramp)  threshold  sigmoid

Transfer Function

It calculates how the error changes as each weight is increased or decreased slightly.

 The algorithm computes each EW by first computing the EA, the rate at which the error changes as the activity level of a unit is changed.

For output units, the EA is simply the difference between the actual and the desired output.

Back-propagation Algorithm

The first step is to design a specific network architecture .

The size and structure of the network needs to match the nature.

The new network is then subjected to the process of "training“.

After learning phase, the new network is ready and can be used to generate predictions.

The resulting "network" developed in the process of "learning" represents a pattern detected in the data.

One of the major advantages of neural networks is that, they are capable of approximating any continuous function.

An important disadvantage is that the final solution depends on the initial conditions of the network.

Neural Networks works

VIDEO

Important Applications

Finger print recognition system Preprocessing system Feature extraction using neural networks Classification result

Advantages

Neural networks enable us to find solution where algorithmic methods are computationally intensive or do not exist.

There is no need to program neural networks they learn with examples.

Neural networks offer significant speed advantage over conventional techniques.

Character Recognition - The idea of character recognition has become very important as handheld devices like the Palm Pilot are becoming increasingly popular. Neural networks can be used to recognize handwritten characters.

Image Compression - Neural networks can receive and process vast amounts of information at once, making them useful in image compression. With the Internet explosion and more sites using more images on their sites, using neural networks for image compression is worth a look.

Some different applications

Stock Market Prediction - The day-to-day business of the stock market is extremely complicated. Many factors weigh in whether a given stock will go up or down on any given day. Since neural networks can examine a lot of information quickly and sort it all out, they can be used to predict stock prices.

Travelling Salesman Problem- Interestingly enough, neural networks can solve the travelling salesman problem, but only to a certain degree of approximation.

Medicine, Electronic Nose, Security, and Loan Applications - These are some applications that are in their proof-of-concept stage, with the acceptance of a neural network that will decide whether or not to grant a loan, something that has already been used more successfully than many humans.

THANK YOU…!