Language technologies should be judged on their usefulness in real-world use cases. An
often overlooked aspect in natural language processing (NLP) research and evaluation is …

State-of-the-art natural language processing (NLP) models are trained on massive training
corpora, and report a superlative performance on evaluation datasets. This survey delves …

Historically, researchers and consumers have noticed a decrease in quality when applying
NLP tools to minority variants of languages (ie Puerto Rican Spanish or Swiss German), but …

Despite the major advances in NLP, significant disparities in NLP system performance
across languages still exist. Arguably, these are due to uneven resource allocation and sub …

English Natural Language Understanding (NLU) systems have achieved great
performances and even outperformed humans on benchmarks like GLUE and SuperGLUE …

Real world data differs radically from the benchmark corpora we use in natural language
processing (NLP). As soon as we apply our technologies to the real world, performance …

The scale, variety, and quantity of publicly-available NLP datasets has grown rapidly as
researchers propose new tasks, larger models, and novel benchmarks. Datasets is a …

Dialect differences caused by regional, social, and economic factors cause performance
discrepancies for many groups of language technology users. Inclusive and equitable …

The NLP community has mainly focused on scaling Large Language Models (LLMs)
vertically, ie, making them better for about 100 languages. We instead scale LLMs …

Large Language Models, the dominant starting point for Natural Language Processing
(NLP) applications, fail at a higher rate for speakers of English dialects other than Standard …