On double-descent in uncertainty quantification in overparametrized models

L Clarté, B Loureiro, F Krzakala… - International …, 2023 - proceedings.mlr.press
Uncertainty quantification is a central challenge in reliable and trustworthy machine
learning. Naive measures such as last-layer scores are well-known to yield overconfident …

Asymptotics of feature learning in two-layer networks after one gradient-step

H Cui, L Pesce, Y Dandi, F Krzakala, YM Lu… - arXiv preprint arXiv …, 2024 - arxiv.org
In this manuscript we investigate the problem of how two-layer neural networks learn
features from data, and improve over the kernel regime, after being trained with a single …

Multinomial logistic regression: Asymptotic normality on null covariates in high-dimensions

K Tan, PC Bellec - Advances in Neural Information …, 2024 - proceedings.neurips.cc
This paper investigates the asymptotic distribution of the maximum-likelihood estimate
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …

Fitting an ellipsoid to random points: predictions using the replica method

A Maillard, D Kunisky - IEEE Transactions on Information …, 2024 - ieeexplore.ieee.org
We consider the problem of fitting a centered ellipsoid to n standard Gaussian random
vectors in R d, as n, d→∞ with n/d 2→ α> 0. It has been conjectured that this problem is, with …

Universality of max-margin classifiers

A Montanari, F Ruan, B Saeed, Y Sohn - arXiv preprint arXiv:2310.00176, 2023 - arxiv.org
Maximum margin binary classification is one of the most fundamental algorithms in machine
learning, yet the role of featurization maps and the high-dimensional asymptotics of the …

Gaussian universality of perceptrons with random labels

F Gerace, F Krzakala, B Loureiro, L Stephan… - Physical Review E, 2024 - APS
While classical in many theoretical settings—and in particular in statistical physics-inspired
works—the assumption of Gaussian iid input data is often perceived as a strong limitation in …

Classification of heavy-tailed features in high dimensions: a superstatistical approach

U Adomaityte, G Sicuro, P Vivo - Advances in Neural …, 2024 - proceedings.neurips.cc
We characterise the learning of a mixture of two clouds of data points with generic centroids
via empirical risk minimisation in the high dimensional regime, under the assumptions of …

Exact threshold for approximate ellipsoid fitting of random points

A Maillard, AS Bandeira - arXiv preprint arXiv:2310.05787, 2023 - arxiv.org
We consider the problem $(\rm P) $ of exactly fitting an ellipsoid (centered at $0 $) to $ n $
standard Gaussian random vectors in $\mathbb {R}^ d $, as $ n, d\to\infty $ with $ n/d …

Bayes-optimal learning of an extensive-width neural network from quadratically many samples

A Maillard, E Troiani, S Martin, F Krzakala… - arXiv preprint arXiv …, 2024 - arxiv.org
We consider the problem of learning a target function corresponding to a single hidden layer
neural network, with a quadratic activation function after the first layer, and random weights …

Gaussian universality of perceptrons with random labels

F Gerace, F Krzakala, B Loureiro, L Stephan… - arXiv preprint arXiv …, 2022 - arxiv.org
While classical in many theoretical settings-and in particular in statistical physics-inspired
works-the assumption of Gaussian iid input data is often perceived as a strong limitation in …