-
[ ]
342/2013
, 2014.
-
342/2013
2
1.
.............................................................................................................................................
3
2.
....................................................................................
4
3.
.......................................................................................
5
4.
...............................................................................................................
10
4.1. T
................................................................................................
12
5. .......................................... 14
5.1.
.............................................................................................
17
6.
.................................................................................
19
6.1.
..........................................................................................
19
6.2.
............................................................................................................................
20
6.3. (Adaptive Linear Element)
....................................................................................
20
6.4. J
.........................................................................................................
22
7.
...........................................................................................
24
7.1. Hornik stinchombe white-ova teorema (1989)
..........................................................................
24
7.2.
..................................................................................
25
7.3.
..........................................................................................................
29
7.4. ........................ 29
7.4.1.
.................................................................................................
29
7.4.2. . (learning constant)
..................................................................
29
7.4.3.
...................................................................................................................
30
7.4.4.
..........................................................................................................................
30
7.4.5. .
............................................................................................................
31
7.4.6. .
.................................................................................
32
7.4.7.
.....................................................................................................
34
8. ................................. 35
.....................................................................................................................................
37
.............................................................................................................................................
38
-
342/2013
3
1.
.
.
:
1. 2. ( )
.
.
()
.
o ,
.
.
. ,
,
(, , , .).
, ( ,
) .
.
.
, (eng. Artificial
neural network ANN)
. .
():
,
.[0]
-
342/2013
4
2.
1. , . 2. - , 3. - . 4. .
. 5. .
.
6. . 7. (Very Large Scale Integration) . 8. .
. -. .
9. . , , - . [1]
2.1.
-
342/2013
5
3.
, ,
. -
.
.
[2]:
.
, ( ) ,
.
[3]:
Bayesian regularisation (BR) Levenberg-Marquardt
.
Powell-Beale conjugate gradient algoritm (CGB)
Polak-Ribiere conjugate gradient algorithm.
BFGS quasi-Newton method (BFG)
1,
,
.
Adaptive learning rate (GDX)
Levenberg-Marquardt algorithm (LM)
.
.
3.1.
.
1
9 25 1
1 http://en.wikipedia.org/wiki/Hessian_matrix
-
342/2013
6
3.1.
3.2.
-
342/2013
7
3.3. [0]
- [1,3]:
x 1 2 ...in h outhN N N N N
inN -
outN -
1 2, ,... hN N N -
[3]:
1
13 25 2 - 25 , 13
2
3
9 15 10 5 1 - 3 15, 10, 5 ,
9 1
, [2]:
( )
.
,
.
-
342/2013
8
(feed forward single
layer neural network)
3.4.
(feedforward
multilayer neural network)
3.5.
-
342/2013
9
,
.
3.6..
. 1z .
3.7.
1z 1z1z
-
342/2013
10
4.
:
ijw . , .
( ) .
. [0,1] [-1,1].
4.1.
m
j
jkjk xwu1
4.1.
)( kkk uay
:
.,1,)(, 00 kkkkjkjk wxvayxwv
-
342/2013
11
4.2.
.
kk bwax 00 ,1 , kb , 4.3.
4.3..
.
-
342/2013
12
4.1. T
,
.
:
1. (), (supervized learning) 2. (reinforcement learning) 3. ( ),
(unsupervised learning)
4.1.1.
4.1.2..
-
342/2013
13
4.1.3.
)()( , ii dX , )(iX , )(id .
,
,
{, }. ,
,
,
.
.
-
342/2013
14
5.
5.1. [4]
[4]
.
,
,
.
,
0 1.
() . [5]
:
.
.
.
-
342/2013
15
iX ,
. jY
. , .
1
j act ij i
i
Y f W X
ijW - iX jY
jY .
( kO )
.
1
k act jk j
i
O f W Y
jkW - jY
(eng. Logistic Sigmoid function) :
1
1act x
f xe
.
, ,
[3,4,5].
.
. ,
.
( ):
1
1 n i i
i i
d O
n d
E - ,
id -
iO -
n -
-
342/2013
16
[2], :
k k ke n d n y n
21
1
2
N
av k
n k C
e nN
ke -
av -
N -
C -
[6] :
2
1 1
1 Q N
n n
m n
E d m y mN Q
E -
Q -
N -
d - ( )
m -
:
, ,
.
, ,
. ,
,
.
:
jiji
EW
W
jiW -
-
342/2013
17
5.1.
. [0,1] [-1,1]. :
0,0
0,1)(
v
vva
5.1.1. .
(1943)
2/1,0
2/12/1,2/1
2/1,1
)(
v
vv
v
va
5.1.2.
()
1
1 bva v
e
5.1.3. () .
b .
a(v)
1
0 v
a(v)
1
-1/2 1/2
v
a(v)
v
1 1b
2b
21 bb
-
342/2013
18
[-1,1],
0,1
0,0
0,1
)(
v
v
v
va
5.1.4. (sgn(v))
1
2 1
v
v
v ea v tg
e
5.1.5.
( )
a(v)
1
0 v
-1
a(v)
v
1 1b
2b
21 bb
-1
-
342/2013
19
6.
6.1.
6.1.2. i-
niwwww Timiii ,...,2,1,),...,,( 21 - i-
)()( txrtwi ,
- .
,
),,( iir dxwfr ,
,)())(),(),(()()1( txtdtxtwftwtw iirii -
,
.
-
342/2013
20
6.2.
.
:
B,
, ,
.
., xywyr iii
,
. ,
mjnixyw jiij ,...,2,1,,...,2,1,
- ji xy , ijw (
), . , , , .
6.3. (Adaptive Linear Element)
.
).(),...,,( )()()1()1( pp dxdx .
iw ,
pkdxwm
j
kk
jj ,...,2,1,1
)()(
.)(2
1)(
2
1)(
2
1)(
1
2
1
)()(
1
2)()(
1
2)()(
p
k
m
j
k
jj
kp
k
kTkp
k
kk xwdxWdydwE
.
)(wEw w ,
mjxxWdw
Ew kj
p
k
kTk
j
j ,...,2,1,)()(
1
)()(
-
342/2013
21
)(kx ,
,)( )()()( kjkTk
j xxWdw
Vidrov-Hofovo . LMS
( , Least Mean Square).
Vidrov-Hofovo
,
xWdydr T .
(w) w,
(),
, .
6.3.1. -
w.
minE
0w )(nw)1( nw
w
eEw
)(
w
wE
)(
)(wE
w
-
342/2013
22
6.4. J
().
.
6.4.1.
nnmmm www ...,,, 2211
m
n
p
..,2,1,,,2,1,)()(
1
)()( pknidxwaxWay kik
j
m
j
ij
kT
i
k
i
: TimiiT
i wwwW ,,, 21 , .
,
,
p
k
p
k
n
i
k
j
m
j
ij
k
i
n
i
kT
i
k
i
p
k
n
i
k
i
k
i xwadxwadydwE1 1 1
2
)(
1
)(
1
2)()(
1 1
2)( .2
1)(
2
1
2
1)(
,1
)()(')()(
p
k
k
j
k
i
k
i
k
i
ij
xnetanetadw
E
)()( kT
i
k
i xWnet - i- k- .
-
342/2013
23
.)(
)(
)('
k
i
k
ik
inet
netaneta
ijw -
,)( )()(')()( kjkikikiij
ij xnetanetadw
Ew
(delta learning rule),
)()( ' xwaxwadr TiTii .
.
,
, ,
a.
-
342/2013
24
7.
(feed forward artificial neural networks FFANN),
,
,
(backpropagation algorithm).
7.1. Hornik stinchombe white-ova teorema (1989)
HSW
1. 1)(lim
a
2. )1(0)(lim
a
)(a
, , .
(
).
FFANN . FFANN
, ,
, ,
FFANN.
, HSW
, .
,
.
-
342/2013
25
7.2.
:
1. )(kx , )(ky .
2. ,
ijw .
(BP )
m l n .
1x mxjx
1y iy ny
qqw1
iqw
nqw
1qv qjv qmv
7.2.1.
7.2.1. (x,d)
. :
qnet - q ,
,1
j
m
j
qjq xvnet
qz - q
-
342/2013
26
m
j
jqjqq xvanetaz1
i-
l
q
l
q
m
j
jqjiqqiqi xvawzwnet1 1 1
.
.1 11
l
q
m
j
jqjiqq
l
q
iqii xvawazwanetay
, .
n
i
n
k
n
i
l
q
qiqiiiii zwadnetadydwE1 1 1
2
1
22.
2
1
2
1
2
1)(
,
iq
iqw
Ew
,
iqwE / ,
,0 qiqiiiiq
i
i
i
i
iq zznetaydw
net
net
y
y
Ew
i0
,0 iiii
i
ii
i netaydnet
y
y
E
net
E
inet ,
i
i
inet
netaneta
.
qz .
-
342/2013
27
q
n
i
jqiqiii
qj
q
q
q
qqj
q
qqj
qj xnetawnetaydv
net
net
z
z
E
v
net
net
E
v
Ev
1
.
i0 ,
,1
0
n
i
jhqjqiqiqj xxnetawv
hq q
,1
0 iq
n
i
iq
q
q
qq
hq wnetanet
z
z
E
net
E
qnet q.
hq q
i0
. , ,
() .
, .
, ,
jinputioutputjiij xxw ,
output-i input-j i.
, .
Q , q=1,2,...,Q
i
q net - i- q-
i
q y - q- .
. ijq w
j
q y1 iq y .
-
342/2013
28
: pkdx kk ,...,2,1,, )()(
0: () 0 maxE (
). . =0,
k=1.
1: k- (q=1):
)(1 k
iii
q xyy , .
2: ( ).
qiywanetayj
j
q
ij
q
i
q
i
q ,1
iQ y .
3: ( iQ )
EydEn
i
i
Qk
i
2
1
)(
2
1 ,
iQiQkiiQ netaydy )( .
4: ( ).
iq 1 :
,,1 ijqolld
ij
qnew
ij
q
j
q
i
q
ij
q wwwyw
j
j
q
ji
q
i
q
i
q QQqzawneta .2,...,1,,11
5:
. je k
-
342/2013
29
. , .
.
. (batch mod training) ,
.
7.3.
(error surface )
.
. :
, .
. , . .
7.4.
7.4.1.
.
.
.
ii kk
3,
3,
ik i.
7.4.2. . (learning constant)
,
, .
,
0.001 10. , .
, 0,
, , 0, 0
0,
a E
a bb
, .
E .
-
342/2013
30
7.4.3.
.
i0 , .
pL
,1,1
pydp
Ei
p
ii
iii
ydL sup .
7.4.4.
,
. .
.
, .
1,0,,)1()()( twtEtw ,
( 0.9).
7.4.4.1.
.
(error surface) . AA
.
. BB
().
.
.
-
342/2013
31
7.4.4.1.
7.4.5. .
.
.
. wE
))(()(2
1)()()()( 00000
TTT wwwHwwwEwwwEwE
.,)()(2
2
ji
ijww
EHwEwH
(w), 0)( wE ,
.0))(()()( 000 wwwHwEwE
,
,)()( 01
0 wEwHww
,)()( )()(1)()1( kkkk wEwHww
-
342/2013
32
,
, .
:
.
.
.
7.4.6. .
.
,
.
,
. overfitting.
,
, (
).
,
.
. .
,
.
,
. ,
, , , 1,... , 1,...i i ix d d i m j p
,2
122
2
2
1
n
fff
bx
E
x
E
x
EE
fE .
bE , ,
fE , .
-
342/2013
33
..
.
EE~
,
, . ,
E~
,
overfittinga.
i
iw2
2
1,
weight decay ,
.
.
. ,
,
, ,
,
overfittinga .
. (early stopping).
( )
.
( )
, .
overfittinga, ,
.
.
-
342/2013
34
7.4.6.1.
7.4.7.
.
,
,
. :
.
, , ,
. .
m-
M ,
. mN ,
.,0,10
m
j
m
mm
m jNzaj
Njegde
j
NMN
mN
m- , maxM
.,
!
11
!2
11
0
max mNzam
jNNNNNN
j
NM m
mmmmmm
m
j
m
mNm , ,2maxmNM
max2log MNm .
() .
-
342/2013
35
8.
, ,
.
,
, ,
.
,
,
.
Pruning.
8.1. Pruning[0]
, [0],
( ,
).
,
, Pruning.
B [3,7].
2
1
2
1
1
i
Mi
pi
Mi
i
O O
B
O O
(1.1)
ipO - i-
i
O - i-
M -
-
342/2013
36
2R . , ,
, .
B 0 1B
, 0,9 B .
,
.
( ) (
), 2
.
:
.1. .
. .
obukaE validE .
.2.
,
,
.
.3. lil bajas .
.4.
. validE . validvalid EE ,
validvalid EE 2., .
,
, .
.
. (brain demage),
. ,
,
.
2 pruning -
-
342/2013
37
[0] L. Gyurova, P. Minino-Justel, A. Schlarb. Modeling the
sliding wear and friction
properties of polyphenylene sulfide composites using artificial
neural networks.
[1] . , ,
,
, 2005.
[2] R. Koker, N. Altinkok, A. Demir. Neural network based
prediction of mechanical
properties of particulate reinforced metal matrix composites
using various training
algorithms.
[3] Z. Zhang, K.Friedrich, K. Velten. Prediction on tribological
properties of short fibre
composites using artificial neural networks.
[4] K. Genel, S. C. Kurnaz, M. Durman. Modeling of tribological
properties of alumina
fiber reinforced zinc-aluminum composites using artificial
neural network.
[5] M. T. Hayajneh, A.M. Hassan, A.T. Mayyas. Artificial neural
network modeling of
the drilling process of self-lubricated
aluminum/alumina/graphite hybrid composites
synthesized by powder metallurgy technique.
[6] A. A. Tofigh, M. R. Rahimipour, M. O. Shabani, M. Alizadeh,
F. Heydari, A.
Mazahery, M. Razavi. Optimized processing power and trainability
of neural network
in numerical modeling of Al Matrix nano composites.
[7] Z. Zhang, K. Friedrich. Artificial neural networks applied
to polymer composites: a
review.
[8] C.M. Bishop, Neural Networks for Pattern Recognition, Oxford
university press,
2000. [9] C.T. Lin, C.S.George Lee, Neural Fuzzy Systems,
Prentice Hall, 1996.
-
342/2013
38
[4]
[0]
C. S. Ramesh, R. Suesh Kumar. Mathematical and neural network
models for prediction of
wear of mild steel coated with Inconel 718- A comparative
study
