Abstract Evaluation practices in natural language generation (NLG) have many known flaws,
but improved evaluation approaches are rarely widely adopted. This issue has become …

Recent advances in the capacity of large language models to generate human-like text have
resulted in their increased adoption in user-facing settings. In parallel, these improvements …

Data augmentation is an important component in the robustness evaluation of models in
natural language processing (NLP) and in enhancing the diversity of the data they are …

With the rise of Large Language Models (LLMs) and their ubiquitous deployment in diverse
domains, measuring language model behavior on realistic data is imperative. For example …

Recently, there has been an increase in efforts to understand how large language models
(LLMs) propagate and amplify social biases. Several works have utilized templates for …

Relation classification models are conventionally evaluated using only a single measure,
eg, micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro …

Code generation models have achieved impressive performance. However, they tend to be
brittle as slight edits to a prompt could lead to very different generations; these robustness …

Evaluation in machine learning is usually informed by past choices, for example which
datasets or metrics to use. This standardization enables the comparison on equal footing …

Recent efforts to create challenge benchmarks that test the abilities of natural language
understanding models have largely depended on human annotations. In this work, we …

The ability to compare the semantic similarity between text corpora is important in a variety
of natural language processing applications. However, standard methods for evaluating …