Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks

T Hoefler, D Alistarh, T Ben-Nun, N Dryden… - Journal of Machine …, 2021 - jmlr.org
The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned

E Voita, D Talbot, F Moiseev, R Sennrich… - arXiv preprint arXiv …, 2019 - arxiv.org
Multi-head self-attention is a key component of the Transformer, a state-of-the-art
architecture for neural machine translation. In this work we evaluate the contribution made …

Survey of low-resource machine translation

B Haddow, R Bawden, AVM Barone, J Helcl… - Computational …, 2022 - direct.mit.edu
We present a survey covering the state of the art in low-resource machine translation (MT)
research. There are currently around 7,000 languages spoken in the world and almost all …

Must-c: a multilingual speech translation corpus

MA Di Gangi, R Cattoni, L Bentivogli, M Negri… - Proceedings of the …, 2019 - cris.fbk.eu
Current research on spoken language translation (SLT) has to confront with the scarcity of
sizeable and publicly available training corpora. This problem hinders the adoption of neural …

Neural machine translation: Challenges, progress and future

J Zhang, C Zong - Science China Technological Sciences, 2020 - Springer
Abstract Machine translation (MT) is a technique that leverages computers to translate
human languages automatically. Nowadays, neural machine translation (NMT) which …

Findings of the IWSLT 2022 Evaluation Campaign.

A Anastasopoulos, L Barrault, L Bentivogli… - Proceedings of the 19th …, 2022 - cris.fbk.eu
The evaluation campaign of the 19th International Conference on Spoken Language
Translation featured eight shared tasks:(i) Simultaneous speech translation,(ii) Offline …

The multilingual tedx corpus for speech recognition and translation

E Salesky, M Wiesner, J Bremerman, R Cattoni… - arXiv preprint arXiv …, 2021 - arxiv.org
We present the Multilingual TEDx corpus, built to support speech recognition (ASR) and
speech translation (ST) research across many non-English source languages. The corpus is …

ESPnet-ST: All-in-one speech translation toolkit

H Inaguma, S Kiyono, K Duh, S Karita… - arXiv preprint arXiv …, 2020 - arxiv.org
We present ESPnet-ST, which is designed for the quick development of speech-to-speech
translation systems in a single framework. ESPnet-ST is a new project inside end-to-end …

The bottom-up evolution of representations in the transformer: A study with machine translation and language modeling objectives

E Voita, R Sennrich, I Titov - arXiv preprint arXiv:1909.01380, 2019 - arxiv.org
We seek to understand how the representations of individual tokens and the structure of the
learned feature space evolve between layers in deep neural networks under different …

Europarl-st: A multilingual corpus for speech translation of parliamentary debates

J Iranzo-Sánchez, JA Silvestre-Cerda… - ICASSP 2020-2020 …, 2020 - ieeexplore.ieee.org
Current research into spoken language translation (SLT), or speech-to-text translation, is
often hampered by the lack of specific data resources for this task, as currently available SLT …