Metacognitive knowledge refers to humans' intuitive knowledge of their own thinking and
reasoning processes. Today's best LLMs clearly possess some reasoning processes. The …

Data curation is an essential component of large-scale pretraining. In this work, we
demonstrate that jointly selecting batches of data is more effective for learning than selecting …

Large Language Model (LLM) pre-training exhausts an ever growing compute budget, yet
recent research has demonstrated that careful document selection enables comparable …

This position paper proposes a data-centric viewpoint of AI research, focusing on large
language models (LLMs). We start by making the key observation that data is instrumental in …

Pretraining data selection has the potential to improve language model pretraining efficiency
by utilizing higher-quality data from massive web data corpora. Current data selection …

Efficient data selection is crucial to accelerate the pretraining of large language models
(LLMs). While various methods have been proposed to enhance data efficiency, limited …

Data selection is of great significance in pre-training large language models, given the
variation in quality within the large-scale available training corpora. To achieve this …

Uncertainty quantification is a key pillar of trustworthy machine learning. It enables safe
reactions under unsafe inputs, like predicting only when the machine learning model detects …

As large language models (LLMs) are applied to more use cases, creating high quality, task-
specific datasets for fine-tuning becomes a bottleneck for model improvement. Using high …

In chemistry and material science, scientific discovery is usually achieved through a
combination of wet-lab experiments and first-principle computational methods. These …