ng dng mng neuron nhn to vo cng tc d bo chui thi gian

ng dng mng neuron nhn to vo cng tc d bo chui thi gianon Ngc Boinh Kim NgnTrn Th SNg Duy Khnh VyNi DungGii thiuCu trc, nguyn tc hot ng ca mng neuron nhn toGii thut Lan truyn ngcGii thut RPROPng dng mng neuron nhn to vo cng tc d bo d liu chui thi gianChng trnh thc nghimKt qu thc nghimKt lun

22Gii thiuNgy nay nhu cu v d bo ngy cng tng. a ra d bo chnh xc v c c s, ngi ta tin hnh thu nhp d liu v cc vn mnh quan tm. Mt loi d liu thng gp l d liu chui thi gian- d liu c thu nhp, lu tr v quan st theo thi gian.Vic d bo d liu chui thi gian ngy cng chim v tr quan trng trong hot ng ca cc n v t chc.C bn c hai k thut ch yu trong vic d bo chui thi gian l cc phng php thng k v phng php dng mng neuron nhn to.

3Cu trc mng neuron nhn toMng neuron nhn to (Artificial Neural Network) l mt m hnh ton hc nh ngha mt hm s t mt tp u vo n mt tp u raMng neuron nhn to l mt mng gm mt tp cc n v (unit) c kt ni vi nhau bng cc cnh c trng s. Mt n v thc hin mt cng vic rt n gin: n nhn tn hiu vo t cc n v pha trc hay mt ngun bn ngoi v s dng chng tnh tn hiu ra45

Hnh 2.1: n v mng neuronTrong mt mng neuron c ba kiu n v: Cc n v u vo, nhn tn hiu t bn ngoi.Cc n v u ra, gi d liu ra bn ngoi.Cc n v n, tn hiu vo ca n c truyn t cc n v trc n v tn hiu ra c truyn n cc n v sau n trong mng.Khi nhn c cc tn hiu u vo, mt n v s nhn mi tn hiu vi trng s tng ng ri ly tng cc gi tr va nhn c. Kt qu s c a vo mt hm s gi l hm kch hot (Activation function) tnh ra tn hiu u ra. Cc n v khc nhau c th c cc hm kch hot khc nhau

67

C hai loi hnh dng mng c bn: Mng truyn thng v mng hi quyMng truyn thng (Feed-forward neural network): Mt n v lp ng trc s kt ni vi tt c cc n v lp ng sau. Tn hiu ch c truyn theo mt hng t lp u vo qua cc lp n (nu c) v n lp u raMng hi quy (Recurrent neural network): Khc vi mng truyn thng, mng hi quy c cha cc lin kt ngc t mt n v n cc n v lp trc n

8Chc nng ca mt mng nron c quyt nh bi cc nhn t nh: hnh dng mng (s lp, s n v trn mi lp, cch m cc lp c lin kt vi nhau) v cc trng s ca cc lin kt bn trong mng. Hnh dng ca mng thng l c nh, v cc trng s c quyt nh bi mt thut ton hun luyn (training algorithm). Tin trnh iu chnh cc trng s mng nhn bit c quan h gia u vo v u ra mong mun c gi l hc (learning) hay hun luyn (training)9Hun luyn mng neuronp dng phng php gim dc (gradient descent) tm mt vector cc trng s sau cho mng xp x tt tp d liu hun luyn Ta s dng mt hm nh gi

Mng xp x tt tp hun luyn nu hm li nh10

Phng php gim dc bt u tm vi mt vector trng s ngu nhin v duyt qua cc mu ca tp hun luyn, mi ln duyt qua cc trng s s c cp nhp theo hng lm gim gi tr hm li. Qu trnh ny c lp i lp li cho n khi t c gi tr cc tiu ca hm li.Hng cp nhp cc trng s lm gim gi tr hm li c xc nh theo vector dc (gradient) ca hm li E , k hiu l

11

V mt ton hc vector dc biu din hng lm tng gi tr hm E trong khng gian trng s, do s l hng lm gim gi tr hm E

12

13

14Thay (2.2) vo (2.1) ta c gi nhp cp nhp trng s qua tng bc ta c

15

Hnh 2.7: Hm li Phng php gim dc c hai hn ch chnh l tc hi t i khi kh chm v nu c nhiu cc tiu cc b (local minimum) trn b mt ca hm li th gii thut d ri vo cc tiu cc b m khng t c cc tiu ton cc (global minimum). gii quyt cc kh khn ny ngi ta pht trin phng php gim dc thnh phng php gim dc tng cng (incremental gradient descent)Khc vi phng php gim dc trn phng php gim dc tng cng thc hin vic tnh ton li v cp nhp cc trng s ngay khi duyt qua mt mu ca tp d liu. Gi tr cp nhp cho cc trng s ca phng php gim dc tng cng l

16

Hm li ca phng php gim dc tng cng khng phi l hm li ton cc cho ton b d liu hun luyn nh phng php gim dc thng m l hm li cho tng mu trong tp d liu

17

Gii thut lan truyn ngcGii thut lan truyn ngc tm tp cc trng s thch hp cho mt mng neuron truyn thng nhiu lp bng phng php gim dcHm li ca gii thut lan truyn ngc c nh ngha tng qut nh sau

18

tng chnh ca gii thut l gi tr li s c lan truyn ngc t tng xut v tng nhp. Vi mi mu trong tp hun luyn, mng neuron c p dng tnh u ra sau gi tr dc ca hm li c tnh cho tng n ca mng. Cui cng gii thut p dng phng php gim dc cp nhp cc gi tr trng s

19 p dng phng php gim dc trc ht ta cn thng tin v o hm ring phn ca hm li cho tng trng s

Vi20

Gi tr c tnh theo hai trng hp ty theo n v i l n v tng xut hay tng n:

21

22

Nu i l n v tng n

23

Thay (2.10) v (2.6) vo (2.5) ta c cng thc tnh o hm ring phn ca hm li theo trng s wij ca n v n i

24

Mi khi thng tin v o hm ring phn bit, bc tip theo trong gii thut lan truyn ngc l cp nhp cc trng s wij.

y l h s hc25

C bn c hai phng php cp nhp cc trng s phn loi theo thi im cp nhp: hc theo mu (learning by pattern) v hc theo epoch (learning by epoch). Mt epoch l mt ln hc duyt qua tt c cc mu trong tp d liu mu dng hc. Trong phng php hc theo mu p dng phng php gim dc tng cng, c mi ln mt mu trong tp d liu c duyt qua th cc trng s s c cp nhpPhng php hc theo epoch thc hin ly tng tt c thng tin v dc (gradient) cho ton b tp mu (pattern set) sau mi cp nhp cc trng s theo phng php gim dc thng thng, ngha l n thc hin vic cp nhp trng s sau khi duyt qua ht cc mu trong tp d liu26Gii thut lan truyn ngc b nh hng bi h s hc v ln ca o hm ring phn. qu trnh hc n nh hn, ta thm vo h s qun tnh, c tc dng iu chnh mc nh hng ca bc cp nhp trc ln bc cp nhp hin thi

27

28

Gii thut RPROPGii thut lan truyn ngc gp mt vn ch gi tr cp nhp trng s khng nhng ph thuc vo du ca o hm ring phn m cn b nh hng bi ln ca n, iu ny lm cho qu trnh hc khng c n nh. Vic thm vo h s qun tnh khng gii quyt trn vn vn bi v ta khng bit gi tr ti u cho h s ny l bao nhiu.

29RPROP l vit tt ca t resilient propagation, ngha l lan truyn n hi l mt phng php thch nghi cc b. RPROP thc hin cp nhp cc trng s wij da vo thng tin v du ca cc o hm ring phn iu ny gip n trnh c s nh hng ca ln ca cc o hm ring phn ny. thc hin iu ny cc trng s s c mt gi tr cp nhp ring ch ph thuc vo du ca o hm ring phn. 30Vi 31

Cc gi tr cp nhp trng s tnh nh sau

32

Tuy nhin c mt trng hp c bit l khi o hm ring phn i du, ngha l bc cp nhp trc qu ln lm cho im ti u b nhy vt qua. Gi tr trng s phi c tr v gi tr trc khi thay i, gi tr cp nhp s c gim xung v bc k sau ta s khng cp nhp gi tr ny. Thc t ta c th lm vic ny bng cch gn

33

Gii thut RPROP ban u cng thc hin cc bc ging nh gii thut lan truyn ngc, cc thng tin v o hm ring phn ca hm li theo cc trng s s c lan truyn ngc t cc lp sau n cc lp trc. Khi cc thng tin v cc o hm ring phn ny c th gii thut s thc hin vic cp nhp cc trng s theo cc quy tc nu trn.

3435

Chui thi gianD liu chui thi gian l d liu c thu nhp, lu tr v quan st theo s tng dn ca thi gianTa k kiu chui thi gian l {Xt} vi t l cc s t nhin. Xt l cc bin ngu nhin (random variable) rt ra t mt phn b xc sut (probability distribution) no . Cc chui thi gian thng c biu din bng mt th vi trc honh l bin thi gian3637

Trong thc t, khi quan st chui thi gian ta nhn thy bn thnh phn nh hng ln mi gi tr ca chui thi gian l xu hng (trend), chu k (cyclical), ma (seasonal), bt quy tc (irregular).Xu hng: L thnh phn th hin s tng hay gim gi tr ca chui thi gian trong mt giai on di hn no Chu k: L chui bin i dng sng quanh xu hng Bt quy tc: L thnh phn th hin s bin i ngu nhin khng th on c ca chui thi gian Ma: L thnh phn th hin s bin i lp i lp li ti tng thi im c nh theo tng nm ca chui thi gian 38p dng mng neuron vo d bo d liu chui thi gian39

Mng neuron hc cu hnh mng t d liu chui thi gian bng cch nh x t mt vect d liu u vo sang d liu u ra. Mt s lng d liu lin tip ca d liu chui thi gian (ca s u vo Xt-s, Xt-s+1, , Xt) c nh x sang khong thch hp (v d [0,1] hoc [-1,1]) v c s dng nh d liu u vo ca tng nhp. Gi tr s ca ca s u vo tng ng vi s n v tng nhp. Trong giai on truyn tin, nhng gi tr c truyn qua tng n ri n cc n v u ra. Khi truyn ti n v u ra, gi tr li c tnh ton da vo s khc bit gia gi tr u ra vi gi tr ca d liu chui thi gian ti thi im t+1. Sau , gi tr li ny c truyn ngc li ti cc kt ni gia tng n v tng u ra, kt ni gia tng u vo v tng n cp nhp li trng s ca cc kt ni ny.

40Qu trnh xy dng mng neuron cho bi ton d bo chui thi gian gm 8 bc:La chn cc binThu thp d liuTin x l d liuPhn chia tp d liuXy dng cu trc mngXc nh tiu chun nh giHun luyn mngD on v ci tin

41La chn cc binXc nh nhng bin no cn c xem xt l im mu cht. i vi mt bi ton c th cn thc hin xem xt cc vn l thuyt m t s xc nh c cc nhn t nh hng n bi ton d on cho mt bin no c th dng cc gi tr c, trong qu kh ca bin , cc ch s c tnh ton t cc gi tr c hay gi tr ca cc bin khc m nh hng n bin ang xem xt

42Thu nhp d liuTa cn phi xem xt chi ph v kh nng c th thu thp c d liu ca cc bin chn ra bc trcSau khi c thu thp, cc d liu phi c kim tra m tnh hp l, tnh nht qun v trnh cc d liu b thiu stCc d liu b thiu st c th c b qua hoc chng c th xem nh khng thay i so vi d liu trc n, v c tnh ton bng phng php ni suy hoc trung bnh cc gi tr ln cn43Tin x l d liuPhn tch v chuyn i gi tr cc tham s u vo, u ra mng ti thiu ha nhiu, nhn mnh cc c trng quan trng, pht hin cc xu hng v cn bng phn b ca d liuCc phng php thng dng: ly hiu, ly logarit t nhin, trung bnh di ng.44Phn chia tp d liuTrong thc t, khi hun luyn, ngi ta thng chia tp d liu thnh cc tp: hun luyn, kim tra v kim nh (ngoi cc mu). Tp hun luyn thng l tp ln nht c s dng hun luyn cho mng. Tp kim tra thng cha khong 10% n 30% tp d liu hun luyn, c s dng kim tra mc tng qut ha ca mng sau khi hun luyn. Kch thc ca tp kim nh cn c cn bng gia vic cn c s mu c th kim tra mng c hun luyn v vic cn c cc mu cn li cho c pha hun luyn v kim tra. Tp kim nh nn bao gm cc gi tr lin tc mi nhtMt phng php cht ch dng nh gi mng neuron l walk-forward. Phng php walk-forward chia tp d liu thnh mt chui cc tp d liu nh hn hun luyn-kim tra-kim nh gi chng ln nhau

45Xy dng cu trc mngPhng php thc hin xy dng cu trc mng neuron bao gm vic xc nh s lin kt gia cc neuron, ng thi xc nh cu trc ca mng bao gm s lp n, s neuron trong tng lpNu xy dng mng c qu nhiu tng n, hoc s lng n v mi tng qu nhiu s dn n vn qu khp. Tc l khi , cu hnh mng neuron gii thch tp d liu hun luyn rt tt, nhng li khng c kh nng tng qut ha, v th khng th dng cu hnh ny d on. Tuy nhin s tng hoc s n v trn mi tng qu t th mng neuron khng c kh nng gii thch v d on tt cc chui thi gian phc tp.Thc t chng minh: mt mng neuron vi mt tng u vo, mt tng n, mt tng u ra cng vi s thay i s n v ti mi tng l xp x bt k mt hm lin tc no. Cc m hnh thng dng trong bi ton d bo chui thi gian l: 8-8-1, 6-6-1, 5-5-1.46Xc nh tiu chun nh gi nh gi kh nng xp x mt chui thi gian ca mng neuron ngi ta thng dng hm tng bnh phng li (sum of squared errors) sau:

Ngoi ra ngi ta cn dng cc hm khc l hm lch tuyt i nh nht (least absolute deviation), hiu phn trm (percentage differences).

47

Hun luyn mngHun luyn mng hc cc mu t d liu bng cch ln lt a cc mu vo cng vi nhng gi tr mong mun. Mc tiu ca vic hun luyn mng l tm ra tp cc trng s cho ta gi tr nh nht ton cc ca ch s hiu nng hay hm li. C ba cch thng dng dng mt qu trnh hun luyn. Cch th nht nh d bo ch dng qu trnh hc khi khng c mt s ci thin ng k no ca hm li. im m mng neuron khng cn ci thin c na gi l im hi t. Cch th hai l s dng mt thng s c nh l s ln lp ti a, qu trnh hun luyn s dng nu s s ln lp (epoches) vt qu thng s ny. Cch th ba l ta s dng mt tp d liu ngoi d liu hun luyn gi l tp d liu xc thc (validation set). Trong qu trnh hun luyn, c mi ln vector trng s ca mng neuron thay i, tp d liu xc thc ny s c a vo mng v tnh ra sai s. Gii thut hun luyn s dng khi sai s ny nh hn mt ngng m nh d on mong mun.48D on v ci tinSau khi thc hin cc bc trn, ta c c mt m hnh mng neuron dng d on. Cc gi tr d on ca mng c lu li v so snh vi cc gi tr thc t khi chng xut hin. Sau mt thi gian, c th m hnh mng khng cn ng na th hin qua vic kt qu d on ngy cng xa cc gi tr tht, ta cn phi tin hnh ci tin mng hoc hc li v xy dng mng mi theo cc bc trc.

49Chng trnh thc nghimChng trnh thc nghim c nhm hin thc bng ngn ng C# trn nn tng .NET framework vi IDE h tr l Mircrosoft Visual Studio 2010.

50Chng trnh c cc chc nng sau:H tr kim tra kt qu hun luyn vi kt qu quan st thc tXy dng v hun luyn mng neuron bng hai gii thut lan truyn ngc v RPROP.H tr d bo vi m hnh mng sau khi hun luynH tr import/export m hnh mng hun luyn ra file XMLH tr nhp v duyt cc file d liu nhm trch xut cc gi tr cho hun luyn, kim tra v d bo

51iu kin dng ca qu trnh hun luyn c hin thc da vo s epoch v sai s hc. Qu trnh hc s dng khi khng c s thay i ng k ca sai s qua mi ln lp hay s epochs vt qu mt ngng c nh do ngi dng nhp vo (trn textbox c nhn l Maximum Number of Epoches). Sai s c nh gi l khng thay i ng k khi lch ca sai s hin thi so vi sai s trong vng lp trc nh hn mt gi tr c ngi dng nhp vo (trn textbox c nhn l Residual of Errors).

52Chng trnh hin thc cho php ngi dng tin hnh nh gi mng va hun luyn trn mt tp d liu nm ngoi tp d liu hun luyn. Gi tr cc hm li sau y s c tnh v xut ra cho ngi dng:Hm tng bnh phng li (sum of squared errors):

Hm trung bnh tuyt i li (mean absolute errors):

Hm trung bnh bnh phng li (mean squared errors):

53

54

Hnh 4.2: Giao din cu hnh ca chng trnh55

Hnh 4.4: Giao din cu hnh cho hun luyn56

Hnh 4.5: Giao din ci t hun luyn s dng gii thut lan truyn ngc57

Hnh 4.6: Giao din ci t hun luyn s dng gii thut RPROP58

Hnh 4.7: Giao din kt qu ca qu trnh hun luyn59

Hnh 4.9: Giao din kt qu kim tra hun luyn60

Hnh 4.11: Giao din kt qu ca d boKt qu thc nghimCc d liu c thc nghim l: d liu v (1) Nhu cu nng lng Italia, (2) T gi gia ng euro v ng la, (3) Ch s tiu dng xng du ca ngi ca ngi dn thnh th M, (4) Tng s ngi sinh theo thng New York. Cc b d liu trn c ly t ngun internet6162

Hnh 5.1: Chui thi gian nhu cu nng lng italiaKt qu cho chui nhu cu nng lng

63

64

Hnh 5.2: Chui thi gian v t gi gia ng euro v ng laKt qu cho chui t gi gia ng euro v ng la65

66

Hnh 5.7: Ch s tiu dng xng du ca ngi dn thnh th M. Kt qu cho chui ch s tiu dng xng du ca ngi dn thnh th M67

68

Hnh 5.8: S ngi sinh ra theo thng ti NewYork Kt qu cho d liu s ngi sinh theo thng ti NewYork

69

Qua vic chy th nghim trn bn b d liu trn, ta thy chng trnh hin thc mng neuron nhn to d on kh chnh xc. Gii thut RPROP nhn chung c tc hi t v chnh xc d bo tt hn gii thut lan truyn ngc. Tuy nhin, i vi cc tp d liu c tnh xu hng nh Ch s tiu dng xng du hay c tnh ma nh tp d liu v Nhu cu nng lng ca Italia th sai s d on kh ln. iu chng t cn phi c nhiu ci tin cho chng trnh hin thc mng neuron ny a ra cc d bo chnh xc hn.

70Kt lunCht lng d bo ca mng neuron nhn to ph thuc nhiu vo cu hnh ca mng (s lp, s n v mi lp) v cc tham s ca gii thut hun luyn V mt l thuyt mng neuron nhn to c kh nng xp x bt c hm lin tc no, iu ny lm cho mng neuron tr thnh mt cng c mnh trong cng tc d bo chui thi gian. Tuy nhin tm mt cu hnh ti u cho mt mng neuron nhn to trong cng tc d bo mt chui thi gian no l mt vic kh khn. Ta phi tin hnh vic la chn bng vic xy dng nhiu cu hnh khc nhau, qua mt qu trnh lp cc cng on hun luyn v kim tra la chn mt cu hnh tt nhtTrong qu trnh p dng m hnh mng d bo, khi cc gi tr mi c thu nhp sai s nhiu so vi kt qu d bo, ta cn phi tin hnh hun luyn li mng vi cc d liu mi.

Gii thut RPROP tc hi t v chnh xc nhn chung tt hn gii thut lan truyn ngc v cng t b chi phi bi vic la chn cc tham s hun luyn. Tuy nhin iu ny khng lun ng cho mi tp d liu, v vy thc hin mt cng tc d bo chui thi gian bng mng neuron nhn to ta nn xem xt la chn v th nghim trn nhiu cu hnh v gii thut khc nhau tm ra mt m hnh tt nht71