1. 2014/10/22@Preferred Infrastructure

2. l -2008 l l 2008-2011 l l 2011- l l NLP2014-2 3. Preferred Infrastructure, Inc. (PFI)l : 20063l : l : 36l :Bring cutting-edge research advances to the real world. 4. l l l l l 4 5. l l twitter5 6. 2l l l l l l 6 7. l l l l PFI7 8. 31. 2. 3. 8 9. 1. 10. l l l l l l WWWl Blogl SNS10 11. NLPl l l ECl l l SNSl 11 12. l l c.f. IBM Watsonl l Google Flu http://www.google.org/flutrends/about/how.html12 13. l : l l 13 14. ECl Amazonl ECGooglel ECl ECAmazonZOZOTOWN l NLPl l l l l 15. l l l l iPSl l l 16. l l pushl l l ,l l l linkedin 17. l l l l l 100BIl FastMSAutonomy (HPEndecaOraclel l 18. SNSl SNSl SNSl SNSl l l Blogl Blog2000l 2004-l 18 19. l l tweet/l l l ( 20. Intimate Mergerl 2013/06/19 l 20 21. l l l l l 1l etc.l UI21 22. (1/2)l SNSl Twitter : 2.3 active user *1l Facebook : 11.9 active user *2l LINE : 2.0 user*3l l Peer reviewed Journal 135 article / *4l 410%, 152l conference proceedings *1 http://weekly.ascii.jp/elem/000/000/084/84331/*2 http://www.prnewswire.com/news-releases/facebook-reports-third-quarter-222013-results-229923821.html*3 http://en.lineblog.naver.jp/archives/30767259.html*4 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2909426/ 23. (2/2)l l l SNS, l l l l SNSl , 23 24. l l Googlel l pullpushtwitter 25. 25 26. tweet26 27. l l l l l l l 27 28. 2. 29. 24http://www.soumu.go.jp/johotsusintokei/whitepaper/ja/h24/html/nc122310.html 30. l l 30 31. : Sirihttps://www.nttdocomo.co.jp/service/information/shabette_concier/31https://www.apple.com/jp/ios/siri/http://v-assist.yahoo.co.jp/ 32. Gunosy, SmartNewshttp://gunosy.com32https://www.smartnews.be/ 33. 1/2l l 1970PCl l l 2000l 33 34. 2/2l l RSS2000l l l 2000l 34 35. 35 36. l l l l l l 36 37. l l l l l l l l l l 37 38. l l l Google Now38 39. 12:00l l 39:25 40. l l l l l l 40 41. ITl l l CPUl l l l UIl l l l l 41 42. l IBM: Smarter Cityl Cisco: Internet of Everythingl GE: Industrial Internetl Google: Google Glassl Amazon: Prime Airl Intel: IoTEdisonhtt4p2://www.amazon.com/b?node=8037720011 43. IoT43 44. l l l l l l l l l l l l 44 45. Preferred Networksl IoT Preferred NetworksPFI45 46. 46 47. 47 48. l l l l l l l 48 49. l l l l l l l 49 50. 3. 51. 52. l l 1993: [Brown+93]l 1996: [Berger+96]l 2001: [Lafferty+01]l l 2003: Latent Dirichlet Allocation [Blei+03]l 2006: Pitman-Yor language model [Teh06]l l 2006: [Clarke+06][Riedel+06]l 2010: [Koo+10][Rush+10]l l 2003: Neural language model [Bengio+03]l 2010: Recurrent Neural Network [Mikolov+10]l 2012: Skipgram Model (wo5r2 d2vec) [Mikolov+13] 53. l l 53 54. 1. l 2011: 30% 20%l 2012: 26% 16%http://image-54net.org/challenges/LSVRC/2012/ilsvrc2012.pdf 55. 2. l 2012/3: GoogleHintonDNNresearchl 2012/4: BaiduInstitute of Deep Learningl 2012/8, 10: Yahoo!IQ EnginesLookFlowl 2012/12: FacebookAI LabLeCunl 2014/1: GoogleDeepMindl 2014/5: Andrew NgBaidul 2014/8: IBMSyNAPSE55 56. 3. l 2014/1 l 2014/10 56 57. l l l l l l l 57 58. l l l Googlel l l 58 59. Neural Network Language Model (NNLM) [Bengio+03]l NNNl N-159 60. Recurrent Neural Network Language Model(RNNLM) [Mikolov+10]l t-1tl NNLMNl l http://rnnlm.org60 t-1 61. Skip-gram (word2vec) [Mikolov+13]l l CBOWl Analogical reasoningl word2vec61 62. (MV-RNN) [Socher+12]l l f l Root62 63. Dynamic Convolutional Neural Network[Kalchbrenner+14]l CNNl 63 64. 64 65. l l l CNNl l l cf. l l cf. word2vec65 66. l libsvm, liblinearl l JUMAN, Chasen, MeCabl l Moses (GIZA++)l l Stanford CoreNLPl l word2vecl Skipgraml Theano, Caffe, cuda-convnetl l 66 67. Caffe (http://caffe.berkeleyvision.org/)l l l Convolutional Neural Networkl CPUGPUl 167 68. Moses1. Prepare data (45 minutes)2. Run GIZA++ (16 hours)3. Align words (2:30 hours)4. Get lexical translation table (30 minutes)5. Extract phrases (10 minutes)6. Score phrases (1:15 hours)7. Build lexicalized reordering model (1 hour)8. Build generation models9. Create configuration file (1 second)68 69. l l 1l l l l 69 70. 71. 72. PFI2010Xappy2012l 2012Gunosyl l l 72 73. Xappyl l Blogl TwitterFacebookSNSl l 4Gl l l etc.l GUI73 74. l l l l l PCl l l l l 10l 74 75. 76. (1/4)l [Brown+93] Peter F. Brown, Vincent J. Della Pietra, Stephen A.Della Pietra, Robert L. Mercer.The mathematics of statistical machine translation: parameterestimation. Computational Linguistics Vol. 19 (2), 1993.l [Berger+96] Adam L. Berger, Vincent J. Della Pietra, Stephen A.Della Pietra.A Maximum Entropy Approach to Natural LanguageProcessing. Computational Linguistics, Vol. 22 (1), 1996.l [Lafferty+01] John Lafferty, Andrew McCallum, Fernando C. N.Pereira.Conditional Random Fields: Probabilistic Models forSegmenting and Labeling Sequence Data. ICML2001.76 77. (2/4)l [Blei+03] David M. Blei, Andrew Y. Ng, Michael I. Jordan.Latent Dirichlet Allocation. JMLR Vol. 3, 2003.l [Teh06] Yee Whye Teh.A Hierarchical Bayesian Language Model based on Pitman-YorProcesses. ACL 2006.l [Clarke+06] James Clarke, Mirella Lapata.Constraint-Based Sentence Compression: An IntegerProgramming Approach. COLING/ACL 2006.l [Riedel+06] Sebastian Riedel, James Clarke.Incremental Integer Linear Programming for Non-projectiveDependency Parsing. COLING/ACL 2006.77 78. (3/4)l [Koo+10] Terry Koo, Alexander M. Rush, Michael Collins, TommiJaakkola, David Sontag.Dual Decomposition for Parsing with Non-Projective HeadAutomata. EMNLP 2010.l [Rush+10] Alexander M. Rush, David Sontag, Michael Collins,Tommi Jaakkola.On Dual Decomposition and Linear Programming Relaxationsfor Natural Language Processing. EMNLP 2010.l [Bengio+03] Yoshua Bengio, Rjean Ducharme, Pascal Vincent,Christian Jauvin.A Neural Probabilistic Language Model. JMLR, 2003.78 79. (4/4)l [Mikolov+10] Tomas Mikolov, Martin Karafiat, Lukas Burget, JanHonza Cernocky, Sanjeev Khudanpur.Recurrent neural network based language model.Interspeech, 2010.l [Mikolov+13] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean.Efficient Estimation of Word Representations in Vector Space. CoRR,2013.l [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning,Andrew Y. Ng.Semantic Compositionality through Recursive Matrix-Vector Spaces.EMNLP2012.l [Kalchbrenner+14] Nal Kalchbrenner, Edward Grefenstette, PhilBlunsom.A Convolutional Neural Network for Modelling Sentences. ACL2014.79