This paper presents the results of the premier shared task organized alongside the
Conference on Machine Translation (WMT) 2019. Participants were asked to build machine …

This paper presents the results of the WMT21 Metrics Shared Task. Participants were asked
to score the outputs of the translation systems competing in the WMT21 News Translation …

This paper presents the results of the WMT17 shared tasks, which included three machine
translation (MT) tasks (news, biomedical, and multimodal), two evaluation tasks (metrics and …

This paper presents the results of the news translation task, the multilingual low-resource
translation for Indo-European languages, the triangular translation task, and the automatic …

This paper offers a fine-grained analysis of the machine translation outputs in the context of
the Shared Task at the 8th Conference of Machine Translation (WMT23). Building on the …

Machine learning (ML) models can fail in unexpected ways in the real world, but not all
model failures are equal. With finite time and resources, ML practitioners are forced to …

We employ a linguistically motivated challenge set in order to evaluate the state-of-the-art
machine translation metrics submitted to the Metrics Shared Task of the 7th Conference for …

Text generation often involves producing coherent and grammatically correct texts that also
satisfy a given set of semantic constraints. While most approaches for conditional text …

We present the results of the application of a grammatical test suite for German $\rightarrow
$ English MT on the systems submitted at WMT19, with a detailed analysis for 107 …

This paper describes a test suite submission providing detailed statistics of linguistic
performance for the state-of-the-art German-English systems of the Fifth Conference of …