COMET-22: Unbabel-IST 2022 submission for the metrics shared task
In this paper, we present the joint contribution of Unbabel and IST to the WMT 2022 Metrics
Shared Task. Our primary submission–dubbed COMET-22–is an ensemble between a …
Shared Task. Our primary submission–dubbed COMET-22–is an ensemble between a …
Findings of the 2021 conference on machine translation (WMT21)
This paper presents the results of the news translation task, the multilingual low-resource
translation for Indo-European languages, the triangular translation task, and the automatic …
translation for Indo-European languages, the triangular translation task, and the automatic …
Quality-aware decoding for neural machine translation
Despite the progress in machine translation quality estimation and evaluation in the last
years, decoding in neural machine translation (NMT) is mostly oblivious to this and centers …
years, decoding in neural machine translation (NMT) is mostly oblivious to this and centers …
UniTE: Unified translation evaluation
Translation quality evaluation plays a crucial role in machine translation. According to the
input format, it is mainly separated into three tasks, ie, reference-only, source-only and …
input format, it is mainly separated into three tasks, ie, reference-only, source-only and …
Toward human-like evaluation for natural language generation with error analysis
The state-of-the-art language model-based automatic metrics, eg BARTScore, benefiting
from large-scale contextualized pre-training, have been successfully used in a wide range of …
from large-scale contextualized pre-training, have been successfully used in a wide range of …
The eval4nlp 2023 shared task on prompting large language models as explainable metrics
With an increasing number of parameters and pre-training data, generative large language
models (LLMs) have shown remarkable capabilities to solve tasks with minimal or no task …
models (LLMs) have shown remarkable capabilities to solve tasks with minimal or no task …
Findings of the WMT 2023 shared task on quality estimation
We report the results of the WMT 2023 shared task on Quality Estimation, in which the
challenge is to predict the quality of the output of neural machine translation systems at the …
challenge is to predict the quality of the output of neural machine translation systems at the …
On the limitations of reference-free evaluations of generated text
There is significant interest in developing evaluation metrics which accurately estimate the
quality of generated text without the aid of a human-written reference text, which can be time …
quality of generated text without the aid of a human-written reference text, which can be time …
Optimal transport for unsupervised hallucination detection in neural machine translation
Neural machine translation (NMT) has become the de-facto standard in real-world machine
translation applications. However, NMT models can unpredictably produce severely …
translation applications. However, NMT models can unpredictably produce severely …
Transformers go for the LOLs: Generating (humourous) titles from scientific abstracts end-to-end
We consider the end-to-end abstract-to-title generation problem, exploring seven recent
transformer based models (including ChatGPT) fine-tuned on more than 30k abstract-title …
transformer based models (including ChatGPT) fine-tuned on more than 30k abstract-title …