Upload
beatrice-cunningham
View
213
Download
0
Embed Size (px)
Citation preview
OutlineOutline What Neural Networks are and why they are
desirable Historical background Applications Strengths neural networks and advantages Status N.N and evolving N.N Basic Architectures and algorithms Applications Drawbacks and limitations
What are Neural Networks?What are Neural Networks?
information processing paradigm inspired by biological nervous systems, such as our brain
Structure: large number of highly interconnected processing elements (neurons) working together
Like people, they learn from experience (by example)
Historical backgroundHistorical background
Neural network simulations appear to be a recent development. However, this field was established before the advent of computers, and has survived at least one major setback and several eras.
These pioneers were able to develop convincing technology which surpassed the limitations identified by Minsky and Papert. Minsky and Papert, published a book (in 1969)
Human Brain FunctionHuman Brain Function
Neural Network FunctionNeural Network Function
TerminologyTerminologyInput: Explanatory variables also referred
to as “predictors”.Neuron: Individual units in the hidden
layer(s) of a neural network.Output: Response variables also called
“predictions”.Hidden Layers: Layers between input and
output that an apply activation function.
Some SimilaritiesSome Similarities
Why Neural Networks are desirableWhy Neural Networks are desirable
Human brain can generalize from abstract
Recognize patterns in the presence of noise
Recall memoriesMake decisions for current problems
based on prior experience
When to use neural networksWhen to use neural networks
Use for huge data sets (i.e. 50 predictors and 15,000 observations) with unknown distributions
Smaller data sets with outliers as neural networks are very resistant to outliers
Why Neural Networks in Statistics?Why Neural Networks in Statistics?
The methodology is seen as a new paradigm for data analysis where models are not explicitly stated but rather implicitly defined by the network.
Allows for analysis where traditional methods might be extremely tedious or nearly impossible to interpret.
Difference in Neural NetworksDifference in Neural Networks
The difference in the two approaches is that multiple linear regression has a closed form solution for the coefficients, while neural networks use an iterative process.
ApplicationsApplications Data Conceptualization– infer grouping relationships
e.g. extract from a database the names of those most likely to buy a particular product.
Data Filtering– e.g. take the noise out of a telephone signal,
signal smoothing Planning– Unknown environments– Sensor data is noisy– Fairly new approach to planning
ApplicationsApplications
Prediction: learning from past experience– pick the best stocks in the market– predict weather– identify people with cancer risk
Classification– Image processing– Predict bankruptcy for credit card
companies– Risk assessment
Strengths of a Neural NetworkStrengths of a Neural Network
Power: Model complex functions, nonlinearity built into the network
Ease of use:–Learn by example–Very little user domain-specific
expertise neededIntuitively appealing: based on model
of biology, will it lead to genuinely intelligent computers/robots?
General AdvantagesGeneral Advantages
Advantages– Adapt to unknown situations– Robustness: fault tolerance due to
network redundancy– Autonomous learning and generalization
Disadvantages– Not exact– Large complexity of the network structure
For motion planning?
Status of Neural NetworksStatus of Neural Networks
Most of the reported applications are still in research stage
No formal proofs, but they seem to have useful applications that work
Evolving networksEvolving networks
Continuous process of:– Evaluate output – Adapt weights– Take new inputs
ANN evolving causes stable state of the weights, but neurons continue working: network has ‘learned’ dealing with the problem
Where are NN used?Where are NN used?
Recognizing and matching complicated, vague, or incomplete patterns
Data is unreliable Problems with noisy data
–Prediction–Classification–Data association–Data conceptualization–Planning
Back propagationBack propagation
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error sizeCalculate output layer error , then
propagate back to previous layerImproved performance, very common!
Activation FunctionActivation Function
The only practical requirement for an activation function is that it be differentiable
Sigmoid function is commonly used g(netinput) = 1/(1+ exp-(netinput))Or a simple binary threshold unitӨ(netinput) = {1 ,if netinput ≥ 0 ; 0 ,
otherwise}
Training the NetworkTraining the Network
Neural Networks must be first trained before being used to analyze new data
Process entails running patterns through the network until the network has “learned” the model to apply to future data
Can take a long time for noisy data
New DataNew Data
Once the network is trained new data can be run through it
The network will classify new data based on the previous data it trained with
If an exact match can not be found it will match with the closest found in memory
Regression and Neural NetworksRegression and Neural Networks
Objective of regression problem is to find coefficients that minimize sum of errors
To find coefficients we must have a dataset that includes the independent variable and associated values of the dependent variable. (very similar to training the network)
Equivalent to a single layer feed forward network
Drawbacks and LimitationsDrawbacks and Limitations
Neural Networks can be extremely hard to use
The programs are filled with settings you must input and a small error will cause your predictions to have error also
The results can be very hard to interpret as well