Universality laws for gaussian mixtures in generalized linear models

Y Dandi, L Stephan, F Krzakala… - Advances in …, 2024 - proceedings.neurips.cc
Advances in Neural Information Processing Systems, 2024proceedings.neurips.cc
A recent line of work in high-dimensional statistics working under the Gaussian mixture
hypothesis has led to a number of results in the context of empirical risk minimization,
Bayesian uncertainty quantification, separation of kernel methods and neural networks,
ensembling and fluctuation of random features. We provide rigorous proofs for the
applicability of these results to a general class of datasets $(\mathbf {x_i}, y_i,{i= 1,\dots, n})
$ containing independent samples from a mixture distribution $\sum_ {c\in\mathcal {C}}\rho …
Abstract
A recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis has led to a number of results in the context of empirical risk minimization, Bayesian uncertainty quantification, separation of kernel methods and neural networks, ensembling and fluctuation of random features. We provide rigorous proofs for the applicability of these results to a general class of datasets containing independent samples from a mixture distribution . Specifically, we consider the hypothesis class of generalized linear models and investigate the asymptotic joint statistics of a family of generalized linear estimators , obtained either from (a) minimizing an empirical risk or (b) sampling from the associated Gibbs measure . Our main contribution is to characterize under which conditions the asymptotic joint statistics of this family depends (on a weak sense) only on the means and covariances of the class conditional features distribution . This allows us to prove the universality of different quantities of interest, including training, generalization errors, as well as the geometrical properties and correlations of the estimators.
proceedings.neurips.cc
以上显示的是最相近的搜索结果。 查看全部搜索结果