Curriculum learning (CL) is a training strategy that trains a machine learning model from
easier data to harder data, which imitates the meaningful learning order in human curricula …

In the last few years, deep learning has led to very good performance on a variety of
problems, such as visual recognition, speech recognition and natural language processing …

Selecting a suitable pretraining dataset is crucial for both general-domain (eg, GPT-3) and
domain-specific (eg, Codex) language models (LMs). We formalize this problem as selecting …

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent
years. Contrary to conventional approaches to AI where tasks are solved from scratch using …

Large datasets have become commonplace in NLP research. However, the increased
emphasis on data quantity has made it challenging to assess the quality of data. We …

We propose a distributed architecture for deep reinforcement learning at scale, that enables
agents to learn effectively from orders of magnitude more data than previously possible. The …

Training on web-scale data can take months. But much computation and time is wasted on
redundant and noisy points that are already learnt or not learnable. To accelerate training …

Abstract Deep Neural Network training spends most of the computation on examples that
are properly handled, and could be ignored. We propose to mitigate this phenomenon with a …

The field of object detection has made significant advances riding on the wave of region-
based ConvNets, but their training procedure still includes many heuristics and …

How do we learn an object detector that is invariant to occlusions and deformations? Our
current solution is to use a data-driven strategy--collect large-scale datasets which have …