A survey of deep learning techniques for neural machine translation
In recent years, natural language processing (NLP) has got great development with deep
learning techniques. In the sub-field of machine translation, a new approach named Neural …
learning techniques. In the sub-field of machine translation, a new approach named Neural …
Multilingual denoising pre-training for neural machine translation
This paper demonstrates that multilingual denoising pre-training produces significant
performance gains across a wide variety of machine translation (MT) tasks. We present …
performance gains across a wide variety of machine translation (MT) tasks. We present …
On the linguistic representational power of neural machine translation models
Despite the recent success of deep neural networks in natural language processing and
other spheres of artificial intelligence, their interpretability remains a challenge. We analyze …
other spheres of artificial intelligence, their interpretability remains a challenge. We analyze …
Masked language model scoring
Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead,
we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are …
we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are …
The natural language decathlon: Multitask learning as question answering
Deep learning has improved performance on many natural language processing (NLP)
tasks individually. However, general NLP models cannot emerge within a paradigm that …
tasks individually. However, general NLP models cannot emerge within a paradigm that …
Learned in translation: Contextualized word vectors
Computer vision has benefited from initializing multiple deep layers with weights pretrained
on large supervised training sets like ImageNet. Natural language processing (NLP) …
on large supervised training sets like ImageNet. Natural language processing (NLP) …
Understanding and improving layer normalization
Layer normalization (LayerNorm) is a technique to normalize the distributions of
intermediate layers. It enables smoother gradients, faster training, and better generalization …
intermediate layers. It enables smoother gradients, faster training, and better generalization …
Semi-supervised sequence modeling with cross-view training
Unsupervised representation learning algorithms such as word2vec and ELMo improve the
accuracy of many supervised NLP models, mainly because they can take advantage of large …
accuracy of many supervised NLP models, mainly because they can take advantage of large …
Graph optimal transport for cross-domain alignment
Cross-domain alignment between two sets of entities (eg, objects in an image, words in a
sentence) is fundamental to both computer vision and natural language processing. Existing …
sentence) is fundamental to both computer vision and natural language processing. Existing …
Document-level neural machine translation with hierarchical attention networks
Neural Machine Translation (NMT) can be improved by including document-level contextual
information. For this purpose, we propose a hierarchical attention model to capture the …
information. For this purpose, we propose a hierarchical attention model to capture the …