Upload
ooo
View
229
Download
2
Tags:
Embed Size (px)
DESCRIPTION
NEURAL NETWORK AND FUZZY SYSTEMS slide Chapter 2
Citation preview
NEURAL NETWORK THEORY
NEURAL DYNAMIC1:ACTIVATIONHS AND SIGNALS
MAIN POINTS:
• NEURONS AS FUNCTIONS(神经元函数)• SIGNAL MONOTONICITY(信号单调性)• BIOLOGICAL ACTIVATIONS AND SIGNALS(生物激励与信号)• NEURON FIELDS(神经域)• NEURONAL DYNAMICAL SYSTEMS(神经诊断系统)• COMMON SIGNAL FUNCTION(一般信号方程)• PULSE-CODED SIGNAL FUNCTION(脉冲编码信号方程)
NEURONS AS FUNCTION
Figure 1. Neuron Structure Model
ij
n
jjii xI
1 jIfy
,
Relationship of input-output:
•Common nonlinear transduction description: a sigmoidal or S-shaped curve
Fig.2 s(x) is a bounded monotone-nondecreasing function of x
Signal Function: cxe
xS
1
1)( )0(0)1(' cScSdxdSS
Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t)).
NEURONS AS FUNCTION
SIGNAL MONOTONICITY
0)(2
cexS cx
• In general, signal functions are monotone nondecreasing S’>=0. In practice this means signal functions have an upper bound or saturation value. • An important exception: bell-shaped signal function or Gaussian signal functions
xScxeS cx ',2'2
The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise.
SIGNAL MONOTONICITY
])(2
1exp[)( 22
n
j
ijj
ii xxS
2i
:
•Generalized Gaussian signal function define potential or radial basis function:
nn Rxxx ),,( 1 input activation vector:
variance:
mean vector: ),,( 1in
ii
)( ii xS
Because the function depend on all neuronal activations not just the ith activation, we shall consider only scalar-input signal functions:
SIGNAL MONOTONICITY
• A property of signal monotonicity: semi-linearity
• Comparation:
a. Linear signal functions: computation and analysis is comparatively easy; do not suppress
noise.
b. Nonlinear signal functions: Increases a network’s computational richness and facilitates noise suppression; risks computational and analytical intractability;
SIGNAL MONOTONICITY
xSdtdx
dxdSS '
•Signal and activation velocities
the signal velocity:
=dS/dtS
Signal velocities depend explicitly on action velocities. This dependence will increase the number of unsupervised learning laws.
BIOLOGICAL ACTIVATIONS AND SIGNALS
Fig3. Key functional units of a biological neuron
•Introduction to units :Dendrite: input
Axon: output
Synapse: transduce signal
Membrane: potential difference between inside and outside of neuron
BIOLOGICAL ACTIVATIONS AND SIGNALS
•Competitive Neuronal Signal
Signal values are usually binary and bipolar.
Bipolar signal functions :
Binary signal functions :
TxTx
xs01
TxTx
xs1
1
NEURON FIELDS In general, neural networks contain many fields of neurons. Neurons within a field are topological.
Denotation:ZYX FFF
XF : input field zF : output field
xs mm yxyxyx ,......,,, 2211
Neural system samples the function m times to generate the associated
•Classification: Zeroth-order topological (simplest)
Three-dimensional and volume topological (complex)
pairs
NEURONAL DYNAMICAL SYSTEMS •Description:
A system of first-order differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials
Activation differential equations:
.....,,......,
.....,,......,
21
21
yyyFFhyxxxFFgx
YX
YX
ii yx , denote the activation time functions of the ith neuron in XF
and jth neuron in YF
•Classification: Automomous systems: activations are independent of t
Nonautonomous systems: depend on t
NEURONAL DYNAMICAL SYSTEMS
pnYX RRFF ,
ZYX FFF ]|[
p
n
nx
RtytytYRtxtxtX
,...,...,
1
1
•Neuronal State spaces
So the state space of the entire neuronal dynamical system is:
Augmentation : pnz RF
Concatenate fields have different computational, metrical or other properties
NEURONAL DYNAMICAL SYSTEMS
Signal state spaces as hypercubes
)))((,)),((())(( 11 txStxStXS nXn
X
Fig.4 Neural and fuzzy computations conincide.
NEURONAL DYNAMICAL SYSTEMS
•Neuronal activations as short-term memory
Short-term memory(STM) : activation
Long-term memory(LTM) : synapse
COMMON SIGNAL FUNCTION 1 、 Liner Function
S(x) = cx + k , c>0
x
S
o
k
COMMON SIGNAL FUNCTION
2. Ramp Function r if x≥θS(x)= cx if |x|<θ
-r if x≤-θr>0, r is a constant.
r
-r
θ -θ x
S
COMMON SIGNAL FUNCTION
elsecxifcxif
cxxS 0
101
)(
)),max(,min()( cxxS 01
3 、 threshold linear signal function: a special Ramp Function
Another form:
0cS '
COMMON SIGNAL FUNCTION
xcxc
xc
cx
ee
ee
xS22
2
11)(
4 、 logistic signal function:
Where c>0.01 )(' ScSS
So the logistic signal function is monotone increasing.
COMMON SIGNAL FUNCTION
5 、 threshold signal function:
TxifTxifTxif
xSxSk
k
k
kk
1
1
1
1
0)(
1)(
Where T is an arbitrary real-valued threshold,and k indicates the discrete time step.
COMMON SIGNAL FUNCTION
)tanh()( cxxS
01 2 )(' ScS
6 、 hyperbolic-tangent signal function:
Another form:
cxcx
cxcx
eeeecx
)tanh(
COMMON SIGNAL FUNCTION
),min()( cxexS 1
1cxe
0 cxceS '
7 、 threshold exponential signal function:
When
02 cxecS ''
0 cxnn ecS )(
COMMON SIGNAL FUNCTION
),max()( cxexS 10
0x
0 cxceS '
8 、 exponential-distribution signal function:
When
0'' 2 cxecS
COMMON SIGNAL FUNCTION
),max()( n
n
xcxxS
0
1n
9 、 the family of ratio-polynomial signal function:
An example
For
02
1
)(' n
n
xccnxS
PULSE-CODED SIGNAL FUNCTION •Description: Pulse trains arriving in a sampling interval seems to be the bearer
of neuronal signal information.
t
tsii dsesxtS )()(
t
tsjj dsesytS )()(
Pulse-coded formulation:
tatpulsenoiftatoccurspulseaif
txi
01
)(where
ii yx , denote binary pulse functions that summarize the excitation of
membrane potential.
PULSE-CODED SIGNAL FUNCTION
)()()( tStxtS iii
)()()( tStytS jjj
tatpulsenoiftatoccurspulseaif
txi
01
)(
•Velocity-difference property of pulse-coded signals
Current pulse and current signal or expected pulse frequency are available quantities.
Another computational advantage:
If
arrivedpulsenotsarrivedpulsesustainedats
001
A simple form for the signal velocity: