Translation quality assessment: A brief survey on manual and automatic methods
To facilitate effective translation modeling and translation studies, one of the crucial
questions to address is how to assess translation quality. From the perspectives of accuracy …
questions to address is how to assess translation quality. From the perspectives of accuracy …
Shifts: A dataset of real distributional shift across multiple large-scale tasks
There has been significant research done on developing methods for improving robustness
to distributional shift and uncertainty estimation. In contrast, only limited work has examined …
to distributional shift and uncertainty estimation. In contrast, only limited work has examined …
Understanding and detecting hallucinations in neural machine translation via model introspection
Neural sequence generation models are known to “hallucinate”, by producing outputs that
are unrelated to the source text. These hallucinations are potentially harmful, yet it remains …
are unrelated to the source text. These hallucinations are potentially harmful, yet it remains …
Shifting attention to relevance: Towards the uncertainty estimation of large language models
Although Large Language Models (LLMs) have shown great potential in Natural Language
Generation, it is still challenging to characterize the uncertainty of model generations, ie …
Generation, it is still challenging to characterize the uncertainty of model generations, ie …
TransQuest: Translation quality estimation with cross-lingual transformers
Recent years have seen big advances in the field of sentence-level quality estimation (QE),
largely as a result of using neural-based architectures. However, the majority of these …
largely as a result of using neural-based architectures. However, the majority of these …
Uncertainty estimation in autoregressive structured prediction
Uncertainty estimation is important for ensuring safety and robustness of AI systems. While
most research in the area has focused on un-structured prediction tasks, limited work has …
most research in the area has focused on un-structured prediction tasks, limited work has …
The vendi score: A diversity evaluation metric for machine learning
D Dan Friedman, AB Dieng - Transactions on machine learning …, 2023 - par.nsf.gov
Diversity is an important criterion for many areas of machine learning (ML), including
generative modeling and dataset curation. However, existing metrics for measuring diversity …
generative modeling and dataset curation. However, existing metrics for measuring diversity …
Uncertainty in natural language generation: From theory to applications
Recent advances of powerful Language Models have allowed Natural Language
Generation (NLG) to emerge as an important technology that can not only perform traditional …
Generation (NLG) to emerge as an important technology that can not only perform traditional …
Findings of the WMT 2021 shared task on quality estimation
We report the results of the WMT 2021 shared task on Quality Estimation, where the
challenge is to predict the quality of the output of neural machine translation systems at the …
challenge is to predict the quality of the output of neural machine translation systems at the …
MLQE-PE: A multilingual quality estimation and post-editing dataset
We present MLQE-PE, a new dataset for Machine Translation (MT) Quality Estimation (QE)
and Automatic Post-Editing (APE). The dataset contains eleven language pairs, with human …
and Automatic Post-Editing (APE). The dataset contains eleven language pairs, with human …