Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation

Z Kasner, O Dušek - Proceedings of the 62nd Annual Meeting of …, 2024 - aclanthology.org
We analyze the behaviors of open large language models (LLMs) on the task of data-to-text
(D2T) generation, ie, generating coherent and relevant text from structured data. To avoid …

The pitfalls of defining hallucination

K Deemter - Computational Linguistics, 2024 - direct.mit.edu
Despite impressive advances in Natural Language Generation (NLG) and Large Language
Models (LLMs), researchers are still unclear about important aspects of NLG evaluation. To …

Tabgenie: A toolkit for table-to-text generation

Z Kasner, E Garanina, O Plátek, O Dušek - arXiv preprint arXiv …, 2023 - arxiv.org
Heterogenity of data-to-text generation datasets limits the research on data-to-text
generation systems. We present TabGenie-a toolkit which enables researchers to explore …

Reimagining language: Towards a better understanding of language by including our interactions with non-humans

M Rasenberg, A Amha, M Coler… - Linguistics in the …, 2023 - jbe-platform.com
What is language and who or what can be said to have it? In this essay we consider this
question in the context of interactions with non-humans, specifically: animals and computers …

Differences in Semantic Errors Made by Different Types of Data-to-text Systems

R Huidrom, A Belz, M Lorandi - Proceedings of the 17th …, 2024 - aclanthology.org
In this paper, we investigate how different semantic, or content-related, errors made by
different types of data-to-text systems differ in terms of number and type. In total, we examine …

[PDF][PDF] 21. April 2023 Hintergrundpapier Nr. 26

S Albrecht - 2023 - bvdp.de
Das Training der Modelle erfolgt in zwei Phasen: In einem ersten Schritt liest das System
weitgehend eigenständig (unüberwacht) große Mengen an Texten ein und bildet daraus …

[PDF][PDF] A Review of Proposals for Improvements in Evaluation of Natural Language Generation

JP Moilanen - 2023 - helda.helsinki.fi
The evaluation methodology of natural language generation (” NLG”= systems generating
text based on some kind of input) is currently in a state of a hopefully productive confusion …

[PDF][PDF] Reimagining language

M Rasenberg, A Amha, M Coler, M van Koppen… - academia.edu
What is language and who or what can be said to have it? In this essay we consider this
question in the context of interactions with non-humans, specifically: animals and computers …