Understanding and mitigating language confusion in llms
K Marchisio, WY Ko, A Bérard, T Dehaze… - arXiv preprint arXiv …, 2024 - arxiv.org
We investigate a surprising limitation of LLMs: their inability to consistently generate text in a
user's desired language. We create the Language Confusion Benchmark (LCB) to evaluate …
user's desired language. We create the Language Confusion Benchmark (LCB) to evaluate …
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages
Large Language Models~(LLMs) demonstrate remarkable translation capabilities in high-
resource language tasks, yet their performance in low-resource languages is hindered by …
resource language tasks, yet their performance in low-resource languages is hindered by …
Preference Tuning For Toxicity Mitigation Generalizes Across Languages
Detoxifying multilingual Large Language Models (LLMs) has become crucial due to their
increasing global use. In this work, we explore zero-shot cross-lingual generalization of …
increasing global use. In this work, we explore zero-shot cross-lingual generalization of …
RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs
Preference optimization techniques have become a standard final stage for training state-of-
art large language models (LLMs). However, despite widespread adoption, the vast majority …
art large language models (LLMs). However, despite widespread adoption, the vast majority …
How Does Quantization Affect Multilingual LLMs?
Quantization techniques are widely used to improve inference speed and deployment of
large language models. While a wide body of work examines the impact of quantized LLMs …
large language models. While a wide body of work examines the impact of quantized LLMs …
MINERS: Multilingual Language Models as Semantic Retrievers
Words have been represented in a high-dimensional vector space that encodes their
semantic similarities, enabling downstream applications such as retrieving synonyms …
semantic similarities, enabling downstream applications such as retrieving synonyms …
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Medical knowledge is context-dependent and requires consistent reasoning across various
natural language expressions of semantically equivalent phrases. This is particularly crucial …
natural language expressions of semantically equivalent phrases. This is particularly crucial …
WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models
WalledEval is a comprehensive AI safety testing toolkit designed to evaluate large language
models (LLMs). It accommodates a diverse range of models, including both open-weight …
models (LLMs). It accommodates a diverse range of models, including both open-weight …
M2QA: Multi-domain Multilingual Question Answering
Generalization and robustness to input variation are core desiderata of machine learning
research. Language varies along several axes, most importantly, language instance (eg …
research. Language varies along several axes, most importantly, language instance (eg …
BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages
Large language models (LLMs) often lack culture-specific knowledge of daily life, especially
across diverse regions and non-English languages. Existing benchmarks for evaluating …
across diverse regions and non-English languages. Existing benchmarks for evaluating …