One of the most important properties of classical neural networks is how surprisingly
trainable they are, though their training algorithms typically rely on optimizing complicated …

We focus on the task of learning a single index model $\sigma (w^\star\cdot x) $ with respect
to the isotropic Gaussian distribution in $ d $ dimensions. Prior work has shown that the …

ML applications proliferate across various sectors. Large internet firms employ ML to train
intelligent models using vast datasets, including sensitive user information. However, new …

We introduce a framework for proving lower bounds on computational problems over
distributions against algorithms that can be implemented using access to a statistical query …

We give the first superpolynomial lower bounds for learning one-layer neural networks with
respect to the Gaussian distribution for a broad class of algorithms. In the regression setting …

We study the fundamental problems of agnostically learning halfspaces and ReLUs under
Gaussian marginals. In the former problem, given labeled examples $(\bx, y) $ from an …

We study the problem of agnostic learning under the Gaussian distribution in the Statistical
Query (SQ) model. We develop a method for finding hard families of examples for a wide …

We study the problem of PAC learning one-hidden-layer ReLU networks with $ k $ hidden
units on $\mathbb {R}^ d $ under Gaussian marginals in the presence of additive label …

We consider the problem of computing the best-fitting ReLU with respect to square-loss on a
training set when the examples have been drawn according to a spherical Gaussian …

The multi-head attention layer is one of the key components of the transformer architecture
that sets it apart from traditional feed-forward models. Given a sequence length $ k …