The remarkable practical success of deep learning has revealed some major surprises from
a theoretical perspective. In particular, simple gradient methods easily find near-optimal …

Interpolators—estimators that achieve zero training error—have attracted growing attention
in machine learning, mainly because state-of-the art neural networks appear to be models of …

Consider supervised learning from iid samples {(y_i, x_i)} _ {i≤ n} where x_i∈ R_p are
feature vectors and y_i∈ R are labels. We study empirical risk minimization over a class of …

We prove a universality theorem for learning with random features. Our result shows that, in
terms of training and generalization errors, a random feature model with a nonlinear …

A recent series of theoretical works showed that the dynamics of neural networks with a
certain initialisation are well-captured by kernel methods. Concurrent empirical work …

The purpose of this article is to review the achievements made in the last few years towards
the understanding of the reasons behind the success and subtleties of neural network …

A recent line of research has highlighted the existence of a``double descent''phenomenon in
deep learning, whereby increasing the number of training examples N causes the …

Deep networks are typically trained with many more parameters than the size of the training
dataset. Recent empirical evidence indicates that the practice of overparameterization not …

Recent theoretical work has characterized the dynamics and convergence properties for
wide shallow neural networks trained via gradient descent; the asymptotic regime in which …

Teacher-student models provide a powerful framework in which the typical case
performance of highdimensional supervised learning tasks can be studied in closed form. In …