Much of recent machine learning has focused on deep learning, in which neural network
weights are trained through variants of stochastic gradient descent. An alternative approach …

Artificial neural networks (ANNs) are essential tools in machine learning that have drawn
increasing attention in neuroscience. Besides offering powerful techniques for data analysis …

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent
years. Contrary to conventional approaches to AI where tasks are solved from scratch using …

Deep neural networks can achieve great successes when presented with large data sets
and sufficient computational resources. However, their ability to learn new concepts quickly …

We investigate the capability of a transformer pretrained on natural language to generalize
to other modalities with minimal finetuning--in particular, without finetuning of the self …

While deep reinforcement learning (RL) has fueled multiple high-profile successes in
machine learning, it is held back from more widespread adoption by its often poor data …

Transformers are state-of-the-art models for a variety of sequence modeling tasks. At their
core is an attention function which models pairwise interactions between the inputs at every …

We show the formal equivalence of linearised self-attention mechanisms and fast weight
controllers from the early'90s, where a slow neural net learns by gradient descent to …

Owing to their ability to both effectively integrate information over long time horizons and
scale to massive amounts of data, self-attention architectures have recently shown …

Learning an efficient update rule from data that promotes rapid learning of new tasks from
the same distribution remains an open problem in meta-learning. Typically, previous works …