Breaking the heavy-tailed noise barrier in stochastic optimization problems

N Puchkin, E Gorbunov, N Kutuzov… - International …, 2024 - proceedings.mlr.press
We consider stochastic optimization problems with heavy-tailed noise with structured
density. For such problems, we show that it is possible to get faster rates of convergence …

Outlier-robust sparse mean estimation for heavy-tailed distributions

I Diakonikolas, D Kane, J Lee… - Advances in Neural …, 2022 - proceedings.neurips.cc
We study the fundamental task of outlier-robust mean estimation for heavy-tailed
distributions in the presence of sparsity. Specifically, given a small number of corrupted …

Convergence rates of stochastic gradient descent under infinite noise variance

H Wang, M Gurbuzbalaban, L Zhu… - Advances in …, 2021 - proceedings.neurips.cc
Recent studies have provided both empirical and theoretical evidence illustrating that heavy
tails can emerge in stochastic gradient descent (SGD) in various scenarios. Such heavy tails …

Accelerated zeroth-order method for non-smooth stochastic convex optimization problem with infinite variance

N Kornilov, O Shamir, A Lobanov… - Advances in …, 2024 - proceedings.neurips.cc
In this paper, we consider non-smooth stochastic convex optimization with two function
evaluations per round under infinite noise variance. In the classical setting when noise has …

Nearly-linear time and streaming algorithms for outlier-robust pca

I Diakonikolas, D Kane, A Pensia… - … on Machine Learning, 2023 - proceedings.mlr.press
We study principal component analysis (PCA), where given a dataset in $\mathbb R^ d $
from a distribution, the task is to find a unit vector $ v $ that approximately maximizes the …

Near-optimal algorithms for gaussians with huber contamination: Mean estimation and linear regression

I Diakonikolas, D Kane, A Pensia… - Advances in Neural …, 2024 - proceedings.neurips.cc
We study the fundamental problems of Gaussian mean estimation and linear regression with
Gaussian covariates in the presence of Huber contamination. Our main contribution is the …

Finite-sample efficient conformal prediction

Y Yang, AK Kuchibhotla - arXiv preprint arXiv:2104.13871, 2021 - arxiv.org
Conformal prediction is a generic methodology for finite-sample valid distribution-free
prediction. This technique has garnered a lot of attention in the literature partly because it …

Tight Bounds for Local Glivenko-Cantelli

M Blanchard, V Voracek - International Conference on …, 2024 - proceedings.mlr.press
This paper addresses the statistical problem of estimating the infinite-norm deviation from
the empirical mean to the distribution mean for high-dimensional distributions on $\{0, 1\}^ d …

[PDF][PDF] Open problem: log (n) factor in

D Cohen, A Kontorovich - The Thirty Sixth Annual …, 2023 - proceedings.mlr.press
Open problem: log n factor in “Local Glivenko-Cantelli” Page 1 Proceedings of Machine
Learning Research vol 195:1–3, 2023 36th Annual Conference on Learning Theory Open …

Robust mean change point testing in high-dimensional data with heavy tails

M Li, Y Chen, T Wang, Y Yu - arXiv preprint arXiv:2305.18987, 2023 - arxiv.org
We study a mean change point testing problem for high-dimensional data, with
exponentially-or polynomially-decaying tails. In each case, depending on the $\ell_0 $-norm …