God

NEURAL NETWORKSNEURAL NETWORKS

M. Alborzi, Ph. D.

Petroleum University of Technology

October, 2001

OUTLINEOUTLINE

Neural Networks DefinedNeural Networks Defined Why Neural NetworksWhy Neural Networks Pattern RecognitionPattern Recognition Neural Networks Application AreasNeural Networks Application Areas A Brief History of Neural NetworksA Brief History of Neural Networks Training Neural NetworksTraining Neural Networks Advantages of Neural NetworksAdvantages of Neural Networks A Simple NN PackageA Simple NN Package

Neural Networks DefinedNeural Networks Defined

A Modeling Technique Emulating A Modeling Technique Emulating the Brainthe Brain

Neural NetworksNeural Networks

A mathematical model that can acquire

artificial intelligence. It resembles brain in

two respects

Knowledge is acquired by network through a learning process

Inter-neuron connection strengths known as weights are used to store the knowledge

Why Neural Networks!Why Neural Networks!

The Need to Emulate the BrainThe Need to Emulate the Brain Facing Complex ProblemsFacing Complex Problems Limitation of MathematicsLimitation of Mathematics Limitation of Serial ComputersLimitation of Serial Computers The Amazing Power of the Brain to Tackle The Amazing Power of the Brain to Tackle

complexitiescomplexities The Parallel Nature and the Network The Parallel Nature and the Network

Nature Structure of the BrainNature Structure of the Brain

Pattern RecognitionPattern Recognition

Mathematical / StatisticalMathematical / Statistical SyntacticalSyntactical Neural NetworksNeural Networks

Neural Networks Applications in Pattern Classification and Pattern Recognition

• Speech recognition and speech generation • Prediction of financial indices such as currency exchange rates• Location of radar point sources• Optimization of chemical processes• Target recognition and mine detection• Identification of cancerous cells• Recognition of chromosomal abnormalities• Detection of ventricular fibrillation• Prediction of re-entry trajectories of spacecraft • Automatic recognition of handwritten characters• Sexing of faces• Recognition of coins of different denominations• Solution of optimal routing problems such as theTraveling Salesman Problem• Discrimination of chaos from noise in the prediction of time series

A Brief History of Neural Networks

• 1943 McCulloch and Pitts Model

• 1962 Rosenblatt Perceptron

• 1969 Miskey and Papert Report on the Shortcomings of Perceptron

• 1987 Rumelhart and McClleland

Breakthrough, Multilayer Perceptron (Originally from Werbos),

Figure 1: The biological neuron

Y= fh[sum( wixi)-teta]

fh(x)=1 if x>0fh(x)=0 if x<0

Figure 2: The McCulloch and Pitts model of a neuron.

X1

X2

X3

OUT

Figure 3: A comparison between M & P model of a neuron and the biological neuron.

M-P model Biological Neuron ------------------------------------------------------------ Input data xi---------------------------Input signal

Input branches------------------------Dendrites Weights wji----------------------------Synapses

wjixi-----------------------------------Activation

Threshold L---------------------------Threshold level Output yj------------------------------Output signal Output branch------------------------Axon 

XOR

Figure 4: Final connection weights: Positive reinforcing connections: Fixed k.

Figure 5: The input logs and the output dominant rock lithologies.

Figure 6: schematic diagram of the initial model

No. Log Unit Description

1 DT s/ft Sonic Velocity

2 ROHB g/cm3 Bulk Density

3 NPHI PU Neutron Porosity

4 PEF barn/electron Photoelectric Factor

5 GR API Gamma Ray

Table 1: The input logs

Table 2: The output rock lithologies.

No. Symbol Unit Description

1 DOLO Fraction Volume of Dolomite

2 LIME Fraction Volume of Limestone

3 SAND Fraction Volume of Sandstone

4 ANHY Fraction Volume of Anhydrite

5 SHAL Fraction Volume of Shale

Depth Log Measurements

metres DT ROHB NPHI PEF GR

  s/ft g/cm3 PU barn/electron API

2505.00 52.700 2.820 1.220 4.820 34.100

2505.15 52.800 2.800 1.470 4.670 33.600

2505.30 52.700 2.790 1.540 4.640 30.400

... ... ... ... ... ...

... ... ... ... ... ...

2667.30 49.200 2.740 3.870 4.590 23.000

2667.46 49.100 2.720 3.880 4.630 23.000

2667.61 49.100 2.720 3.880 4.680 23.000

 A Sample of Log Measurements and PETROS Output for Well No. 6  1) Input Log Measuremwents

Depth Volume Fractions of the Rock Constituents

metres DOLO LIME SAND ANHY SHAL

  fraction fraction fraction fraction fraction

2505.00 0.420 0.000 0.260 0.240 0.080

2505.15 0.500 0.000 0.300 0.120 0.080

2505.30 0.520 0.000 0.300 0.100 0.080

... ... ... ... ... ...

... ... ... ... ... ...

2667.30 0.420 0.580 0.000 0.000 0.000

2667.46 0.380 0.620 0.000 0.000 0.000

2667.61 0.360 0.640 0.000 0.000 0.000

2) PETROS Output Volume Fractions 

NN AdvantagesNN Advantages

Model function does not have to be known

NN learns behavior by self-tuning its parameters

NN has the ability to discover patterns

NN provides a rapid and confident prediction

NN is fast-responding systems

NN can accept more input to improve accuracy,

such continuous enrichment of the NN “knowledge”

leads to more accurate predictive model

NN Problems & ChallengesNN Problems & Challenges

Design of NN:

Number of hidden layers

Number of neurons in each hidden layer

The learning constant that controls speed of

training

NN Problems & ChallengesNN Problems & Challenges

Generalization Vs. Over Fitting

New training algorithms (cross validation)

Hybrid systems (genetic algorithms)

Neural network architectureNeural network architecture

INPUTHIDDEN

API

Rs

g

T

OUTPUT

Bob

Pb

A Simple Neural Networks A Simple Neural Networks PackagePackage

Neural network architectureNeural network architecture

INPUTHIDDEN

API

Rs

g

T

OUTPUT

Bob

Pb

Thank You for Your AttentionThank You for Your Attention