Large language models (LLMs) are being increasingly integrated into everyday products
and services, such as coding tools and writing assistants. As these embedded AI …

This paper presents a vision for creating AI systems that are inclusive at every stage of
development, from data collection to model design and evaluation. We address key …

fmeval is an open source library to evaluate large language models (LLMs) in a range of
tasks. It helps practitioners evaluate their model for task performance and along multiple …

While a large body of work inspects language models for biases concerning gender, race,
occupation and religion, biases of geographical nature are relatively less explored. Some …

Recent research demonstrates geographic biases in various Large Language Models that
reflects common human biases, which are presumably present in the training data. We …

The issue of popularity bias--where popular items are disproportionately recommended,
overshadowing less popular but potentially relevant items--remains a significant challenge …