Scaling language models with more data, compute and parameters has driven significant
progress in natural language processing. For example, thanks to scaling, GPT-3 was able to …

As language models grow in popularity, it becomes increasingly important to clearly
measure all possible markers of demographic identity in order to avoid perpetuating existing …

As generative large language models (LLMs) grow more performant and prevalent, we must
develop comprehensive enough tools to measure and improve their fairness. Different …

We investigate the potential for nationality biases in natural language processing (NLP)
models using human evaluation methods. Biased NLP models can perpetuate stereotypes …

NLP models trained on text have been shown to reproduce human stereotypes, which can
magnify harms to marginalized groups when systems are deployed at scale. We adapt the …

Large neural network-based language models play an increasingly important role in
contemporary AI. Although these models demonstrate sophisticated text generation …

Linguistic annotations, especially for controversial topics like hate speech detection, are
frequently contested due to annotator backgrounds and positionalities. In such situations …

A common limitation of diagnostic tests for detecting social biases in NLP models is that they
may only detect stereotypic associations that are pre-specified by the designer of the test …

As machine learning applications proliferate, we need an understanding of their potential for
harm. However, current fairness metrics are rarely grounded in human psychological …

Fig. 1. We distinguish between types of machine learning errors which are likely to have
different harms. The first split is between errors made on labels which are associated with …