Recurrent Neural Networks

Vanishing Gradients

Gated Recurrent Units (GRUs)

Long Short Term Memory (LSTM)

Compositional Character Models for Word Embeddings

Performance

Further Readings● Read all of these:

– http://colah.github.io/

– http://karpathy.github.io/

– http://www.wildml.com/

– https://deeplearning4j.org/

– http://neuralnetworksanddeeplearning.com/

– https://www.deeplearningbook.org/

● Some Papers:

– Cho, Kyunghyun, et al. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." arXiv preprint arXiv:1406.1078 (2014).https://arxiv.org/pdf/1406.1078.pdf

– Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780.http://web.eecs.utk.edu/~itamar/courses/ECE-692/Bobby_paper1.pdf

– Ling, Wang, et al. "Finding function in form: Compositional character models for open vocabulary word representation." arXiv preprint arXiv:1508.02096 (2015).https://arxiv.org/pdf/1508.02096.pdf