Towards measuring and modeling" culture" in llms: A survey

MF Adilazuarda, S Mukherjee, P Lavania… - arXiv preprint arXiv …, 2024 - arxiv.org
We present a survey of more than 90 recent papers that aim to study cultural representation
and inclusion in large language models (LLMs). We observe that none of the studies …

Culturebank: An online community-driven knowledge base towards culturally aware language technologies

W Shi, R Li, Y Zhang, C Ziems, R Horesh… - arXiv preprint arXiv …, 2024 - arxiv.org
To enhance language models' cultural awareness, we design a generalizable pipeline to
construct cultural knowledge bases from different online communities on a massive scale …

CaLMQA: Exploring culturally specific long-form question answering across 23 languages

S Arora, M Karpinska, HT Chen, I Bhattacharjee… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) are used for long-form question answering (LFQA), which
requires them to generate paragraph-length answers to complex questions. While LFQA has …

Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art

CC Liu, I Gurevych, A Korhonen - arXiv preprint arXiv:2406.03930, 2024 - arxiv.org
The surge of interest in culturally aware and adapted Natural Language Processing (NLP)
has inspired much recent research. However, the lack of common understanding of the …

Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration

Y Baek, CH Park, J Kim, YJ Heo, DS Chang… - arXiv preprint arXiv …, 2024 - arxiv.org
To create culturally inclusive vision-language models (VLMs), the foremost requirement is
developing a test benchmark that can diagnose the models' ability to respond to questions …

Survey of Cultural Awareness in Language Models: Text and Beyond

S Pawar, J Park, J Jin, A Arora, J Myung… - arXiv preprint arXiv …, 2024 - arxiv.org
Large-scale deployment of large language models (LLMs) in various applications, such as
chatbots and virtual assistants, requires LLMs to be culturally sensitive to the user to ensure …

BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages

J Myung, N Lee, Y Zhou, J Jin, RA Putri… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) often lack culture-specific knowledge of daily life, especially
across diverse regions and non-English languages. Existing benchmarks for evaluating …

Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switching

S Kim, H Kim, C Park, J Yeo, D Lee - arXiv preprint arXiv:2410.18436, 2024 - arxiv.org
Code-switching (CS), a phenomenon where multilingual speakers alternate between
languages in a discourse, can convey subtle cultural and linguistic nuances that can be …

Fit for our purpose, not yours: Benchmark for a low-resource, Indigenous language

S Duncan, G Leoni, L Steven, K Mahelona… - The Thirty-eight … - openreview.net
Influential and popular benchmarks in AI are largely irrelevant to developing NLP tools for
low-resource, Indigenous languages. With the primary goal of measuring the performance of …