2019斯坦福CS224n深度学习自然语言处理课程视频和相关资料分享

1. Word2Vec Tutorial - The Skip-Gram Model
2. Efficient Estimation of Word Representations in Vector Space(original word2vec paper)
3. Distributed Representations of Words and Phrases and their Compositionality (negative sampling paper)

1. GloVe: Global Vectors for Word Representation (original GloVe paper)
2. Improving Distributional Similarity with Lessons Learned from Word Embeddings
3. Evaluation methods for unsupervised word embeddings

Additional Readings:

1. A Latent Variable Model Approach to PMI-based Word Embeddings
2. Linear Algebraic Structure of Word Senses, with Applications to Polysemy
3. On the Dimensionality of Word Embedding.

1. CS231n notes on backprop
2. Review of differential calculus

Additional Readings:

1. Natural Language Processing (Almost) from Scratch

1. CS231n notes on network architectures
2. Learning Representations by Backpropagating Errors
3. Derivatives, Backpropagation, and Vectorization
4. Yes you should understand backprop

1. Incrementality in Deterministic Dependency Parsing
2. A Fast and Accurate Dependency Parser using Neural Networks
3. Dependency Parsing
4. Globally Normalized Transition-Based Neural Networks
5. Universal Stanford Dependencies: A cross-linguistic typology
6. Universal Dependencies website

1. N-gram Language Models (textbook chapter)
2. The Unreasonable Effectiveness of Recurrent Neural Networks(blog post overview)
3. Sequence Modeling: Recurrent and Recursive Neural Nets(Sections 10.1 and 10.2)
4. On Chomsky and the Two Cultures of Statistical Learning

1. Sequence Modeling: Recurrent and Recursive Neural Nets(Sections 10.3, 10.5, 10.7-10.12)
2. Learning long-term dependencies with gradient descent is difficult (one of the original vanishing gradient papers)
3. On the difficulty of training Recurrent Neural Networks (proof of vanishing gradient problem)
4. Vanishing Gradients Jupyter Notebook (demo for feedforward networks)
5. Understanding LSTM Networks (blog post overview)

1. Statistical Machine Translation slides, CS224n 2015 (lectures 2/3/4)
2. Statistical Machine Translation (book by Philipp Koehn)
3. BLEU (original paper)
4. Sequence to Sequence Learning with Neural Networks (original seq2seq NMT paper)
5. Sequence Transduction with Recurrent Neural Networks (early seq2seq speech recognition paper)
6. Neural Machine Translation by Jointly Learning to Align and Translate (original seq2seq+attention paper)
7. Attention and Augmented Recurrent Neural Networks (blog post overview)
8. Massive Exploration of Neural Machine Translation Architectures(practical advice for hyperparameter choices)

1. Practical Methodology (Deep Learning book chapter)

Default Final Project out[handout] [github repo]

1. Convolutional Neural Networks for Sentence Classification
2. A Convolutional Neural Network for Modelling Sentences

1. Smith, Noah A. Contextual Word Representations: A Contextual Introduction. (Published just in time for this lecture!)

1. Attention is all you need
2. Image Transformer
3. Music Transformer: Generating music with long-term structure

1. Parsing with Compositional Vector Grammars.
2. Constituency Parsing with a Self-Attentive Encoder