1. : NAIST Computational Linguistic Lab D1 Masayoshi Kondo 5
-
2. 00: Deep Learning : 165 ()
3. XX: () . . .
4. 00: (CNN) (RNN)
5. 00: () - (Deep Learning) / ()
6. 00: (3) - (overtting) .
7. () () () 00: (4) - !!
8. 1. 2. 3. 4.(pre-training) . 00: (autoencoder)
9. 5 5.1 5.2 5.3 5.4 5.5 5.6 5.7 () ()!!
10. 5 5.1 5.2 5.3 5.4 5.5 5.6 5.7
11. (normalization)(3.6.1) 01: (whitening) . Analysis of
single-layer networks in supervised feature learning. [A.Coates+,
AISTATS2011] . 5.5
12. DDP() ( P ) un = Pxn (n =1,!,n) U = 1 N unun T = 1 N UUT
n=1 N () P U 02: 5.5 x =[x1, x2,!, xD ]X =[x1,x2,!,xN ] DN X = 1 N
xnxn T = 1 N XXT n=1 N
13. ZCA (zero-phase whitening,/ZCA: zero-phase component
analysis,) PZCA = ED - 1 2 ET or D-1/2 Q D E . X X P 03: 5.5 PCA
(PCA: principal component analysis, ) Ppca = QD - 1 2 ET PZCA =
E(D+I) - 1 2 ET