The field of machine translation (MT), the automatic translation of written text from one
natural language into another, has experienced a major paradigm shift in recent years …

Deep learning is transforming most areas of science and technology, including electron
microscopy. This review paper offers a practical perspective aimed at developers with …

Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead,
we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are …

It has been shown that the performance of neural machine translation (NMT) drops starkly in
low-resource conditions, underperforming phrase-based statistical machine translation …

Layer normalization (LayerNorm) has been successfully applied to various deep neural
networks to help stabilize training and boost model convergence because of its capability in …

We evaluate three simple, normalization-centric changes to improve Transformer training.
First, we show that pre-norm residual connections (PreNorm) and smaller initializations …

Knowledge distillation (KD) is essential for training non-autoregressive translation (NAT)
models by reducing the complexity of the raw data with an autoregressive teacher model. In …

The development of deep learning techniques has allowed Neural Machine Translation
(NMT) models to become extremely powerful, given sufficient training data and training time …

Self-attention is a method of encoding sequences of vectors by relating these vectors to
each-other based on pairwise similarities. These models have recently shown promising …

Link prediction is one of the key problems for graph-structured data. With the advancement
of graph neural networks, graph autoencoders (GAEs) and variational graph autoencoders …