We propose KMMLU, a new Korean benchmark with 35,030 expert-level multiple-choice
questions across 45 subjects ranging from humanities to STEM. Unlike previous Korean …

LLM-as-a-Judge and reward models are widely used alternatives of multiple-choice
questions or human annotators for large language model (LLM) evaluation. Their efficacy …

The availability of compute and data to train larger and larger language models increases
the demand for robust methods of benchmarking the true progress of LM training. Recent …

Large language models (LLMs) trained on massive corpora demonstrate impressive
capabilities in a wide range of tasks. While there are ongoing efforts to adapt these models …

Despite the rapid development of large language models (LLMs) for the Korean language,
there remains an obvious lack of benchmark datasets that test the requisite Korean cultural …

The non-humanlike behaviour of contemporary pre-trained language models (PLMs) is a
leading cause undermining their trustworthiness. A striking phenomenon of such faulty …

This report introduces\texttt {EEVE-Korean-v1. 0}, a Korean adaptation of large language
models that exhibit remarkable capabilities across English and Korean text understanding …

Korean is often referred to as a low-resource language in the research community. While
this claim is partially true, it is also because the availability of resources is inadequately …

As language models are often deployed as chatbot assistants, it becomes a virtue for
models to engage in conversations in a user's first language. While these models are trained …

By merging models, AI systems can combine the distinct strengths of separate language
models, achieving a balance between multiple capabilities without requiring substantial …