Universality laws for gaussian mixtures in generalized linear models
A recent line of work in high-dimensional statistics working under the Gaussian mixture
hypothesis has led to a number of results in the context of empirical risk minimization …
hypothesis has led to a number of results in the context of empirical risk minimization …
Asymptotics of feature learning in two-layer networks after one gradient-step
In this manuscript we investigate the problem of how two-layer neural networks learn
features from data, and improve over the kernel regime, after being trained with a single …
features from data, and improve over the kernel regime, after being trained with a single …
Multinomial logistic regression: Asymptotic normality on null covariates in high-dimensions
This paper investigates the asymptotic distribution of the maximum-likelihood estimate
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …
Demystifying disagreement-on-the-line in high dimensions
Evaluating the performance of machine learning models under distribution shifts is
challenging, especially when we only have unlabeled data from the shifted (target) domain …
challenging, especially when we only have unlabeled data from the shifted (target) domain …
Precise asymptotic generalization for multiclass classification with overparameterized linear models
We study the asymptotic generalization of an overparameterized linear model for multiclass
classification under the Gaussian covariates bi-level model introduced in Subramanian et …
classification under the Gaussian covariates bi-level model introduced in Subramanian et …
High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality
U Adomaityte, L Defilippis, B Loureiro… - arXiv preprint arXiv …, 2023 - arxiv.org
We investigate the high-dimensional properties of robust regression estimators in the
presence of heavy-tailed contamination of both the covariates and response functions. In …
presence of heavy-tailed contamination of both the covariates and response functions. In …
Asymptotics of Learning with Deep Structured (Random) Features
For a large class of feature maps we provide a tight asymptotic characterisation of the test
error associated with learning the readout layer, in the high-dimensional limit where the …
error associated with learning the readout layer, in the high-dimensional limit where the …
Optimal Ridge Regularization for Out-of-Distribution Prediction
We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-
distribution prediction, where the test distribution deviates arbitrarily from the train …
distribution prediction, where the test distribution deviates arbitrarily from the train …
Classification of heavy-tailed features in high dimensions: a superstatistical approach
We characterise the learning of a mixture of two clouds of data points with generic centroids
via empirical risk minimisation in the high dimensional regime, under the assumptions of …
via empirical risk minimisation in the high dimensional regime, under the assumptions of …
Sliding down the stairs: how correlated latent variables accelerate learning with neural networks
Neural networks extract features from data using stochastic gradient descent (SGD). In
particular, higher-order input cumulants (HOCs) are crucial for their performance. However …
particular, higher-order input cumulants (HOCs) are crucial for their performance. However …