Sliced Wasserstein distances preserve properties of classic Wasserstein distances while
being more scalable for computation and estimation in high dimensions. The goal of this …

Understanding the impact of data structure on the computational tractability of learning is a
key challenge for the theory of neural networks. Many theoretical works do not explicitly …

Abstract The Sliced-Wasserstein distance (SW) is being increasingly used in machine
learning applications as an alternative to the Wasserstein distance and offers significant …

Sliced mutual information (SMI) is defined as an average of mutual information (MI) terms
between one-dimensional random projections of the random variables. It serves as a …

This paper considers the fundamental limit of random linear estimation for iid signal
distributions and iid Gaussian measurement matrices. Its main contribution is a rigorous …

We study the problem of recovering a hidden binary $ k $-sparse $ p $-dimensional vector
$\beta $ from $ n $ noisy linear observations $ Y= X\beta+ W $ where $ X_ {ij} $ are iid …

Many methods for statistical inference and generative modeling rely on a probability
divergence to effectively compare two probability distributions. The Wasserstein distance …

Multilayer (or deep) networks are powerful probabilistic models based on multiple stages of
a linear transform followed by a non-linear (possibly random) function. In general, the linear …

We carry out an information-theoretical analysis of a two-layer neural network trained from
input-output pairs generated by a teacher network with matching architecture, in …

This paper focuses on the mutual information and minimum mean-squared error (MMSE) as
a function a matrix-valued signal-to-noise ratio (SNR) for a linear Gaussian channel with …