We prove that, for the fundamental regression task of learning a single neuron, training a
one-hidden layer ReLU network of any width by gradient flow from a small initialisation …

Training neural networks with first order optimisation methods is at the core of the empirical
success of deep learning. The scale of initialisation is a crucial factor, as small initialisations …

This paper examines gradient flow dynamics of two-homogeneous neural networks for small
initializations, where all weights are initialized near the origin. For both square and logistic …

Using multiple input streams simultaneously in training multimodal neural networks is
intuitively advantageous, but practically challenging. A key challenge is unimodal bias …

While the impressive performance of modern neural networks is often attributed to their
capacity to efficiently extract task-relevant features from data, the mechanisms underlying …

The implicit bias of gradient-based training algorithms has been considered mostly
beneficial as it leads to trained networks that often generalize well. However, Frei et …

Understanding generalization of overparametrized neural networks remains a fundamental
challenge in machine learning. Most of the literature mostly studies generalization from an …

We investigate the expressivity and learning dynamics of bias-free ReLU networks. We firstly
show that two-layer bias-free ReLU networks have limited expressivity: the only odd function …

The goal of continual learning (CL) is to train a model that can solve multiple tasks
presented sequentially. Recent CL approaches have achieved strong performance by …

The multi-stage phenomenon in the training loss curves of neural networks has been widely
observed, reflecting the non-linearity and complexity inherent in the training process. In this …