High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality

U Adomaityte, L Defilippis, B Loureiro… - arXiv preprint arXiv …, 2023 - arxiv.org
We investigate the high-dimensional properties of robust regression estimators in the
presence of heavy-tailed contamination of both the covariates and response functions. In …

Asymptotics of Learning with Deep Structured (Random) Features

D Schröder, D Dmitriev, H Cui, B Loureiro - arXiv preprint arXiv …, 2024 - arxiv.org
For a large class of feature maps we provide a tight asymptotic characterisation of the test
error associated with learning the readout layer, in the high-dimensional limit where the …

Gaussian universality for approximately polynomial functions of high-dimensional data

KH Huang, M Austern, P Orbanz - arXiv preprint arXiv:2403.10711, 2024 - arxiv.org
We establish an invariance principle for polynomial functions of $ n $ independent high-
dimensional random vectors, and also show that the obtained rates are nearly optimal. Both …

Classification of heavy-tailed features in high dimensions: a superstatistical approach

U Adomaityte, G Sicuro, P Vivo - Advances in Neural …, 2024 - proceedings.neurips.cc
We characterise the learning of a mixture of two clouds of data points with generic centroids
via empirical risk minimisation in the high dimensional regime, under the assumptions of …

Fitting an ellipsoid to random points: predictions using the replica method

A Maillard, D Kunisky - IEEE Transactions on Information …, 2024 - ieeexplore.ieee.org
We consider the problem of fitting a centered ellipsoid to n standard Gaussian random
vectors in R d, as n, d→∞ with n/d 2→ α> 0. It has been conjectured that this problem is, with …

Finite-size correction and variance of the mutual information of random linear estimation with non-Gaussian priors: A replica calculation

TG Tsironis, AL Moustakas - Physical Review E, 2024 - APS
Random linear vector channels have been known to increase the transmission of
information in several communications systems. For Gaussian priors, the statistics of a key …

Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers

F Bassetti, M Gherardi, A Ingrosso, M Pastore… - arXiv preprint arXiv …, 2024 - arxiv.org
Deep linear networks have been extensively studied, as they provide simplified models of
deep learning. However, little is known in the case of finite-width architectures with multiple …

Generalized Penalized Constrained Regression: Sharp Guarantees in High Dimensions with Noisy Features

AM Alrashdi, M Alazmi, MA Alrasheedi - Mathematics, 2023 - mdpi.com
The generalized penalized constrained regression (G-PCR) is a penalized model for high-
dimensional linear inverse problems with structured features. This paper presents a sharp …

Storage Capacity Evaluation of the Quantum Perceptron using the Replica Method

M Urushibata, M Ohzeki - arXiv preprint arXiv:2404.14785, 2024 - arxiv.org
We investigate a quantum perceptron implemented on a quantum circuit using a repeat until
method. We evaluate this from the perspective of capacity, one of the performance …

Slow rates of approximation of U-statistics and V-statistics by quadratic forms of Gaussians

KH Huang, P Orbanz - arXiv preprint arXiv:2406.12437, 2024 - arxiv.org
We construct examples of degree-two U-and V-statistics of $ n $ iid~ heavy-tailed random
vectors in $\mathbb {R}^{d (n)} $, whose $\nu $-th moments exist for ${\nu> 2} $, and provide …