Dataset distillation with infinitely wide convolutional networks
The effectiveness of machine learning algorithms arises from being able to extract useful
features from large amounts of data. As model and dataset sizes increase, dataset …
features from large amounts of data. As model and dataset sizes increase, dataset …
Dataset meta-learning from kernel ridge-regression
One of the most fundamental aspects of any machine learning algorithm is the training data
used by the algorithm. We introduce the novel concept of $\epsilon $-approximation of …
used by the algorithm. We introduce the novel concept of $\epsilon $-approximation of …
Finite versus infinite neural networks: an empirical study
We perform a careful, thorough, and large scale empirical study of the correspondence
between wide neural networks and kernel methods. By doing so, we resolve a variety of …
between wide neural networks and kernel methods. By doing so, we resolve a variety of …
On implicit bias in overparameterized bilevel optimization
Many problems in machine learning involve bilevel optimization (BLO), including
hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems …
hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems …
Bayesian deep ensembles via the neural tangent kernel
B He, B Lakshminarayanan… - Advances in neural …, 2020 - proceedings.neurips.cc
We explore the link between deep ensembles and Gaussian processes (GPs) through the
lens of the Neural Tangent Kernel (NTK): a recent development in understanding the …
lens of the Neural Tangent Kernel (NTK): a recent development in understanding the …
Analytic theory for the dynamics of wide quantum neural networks
Parametrized quantum circuits can be used as quantum neural networks and have the
potential to outperform their classical counterparts when trained for addressing learning …
potential to outperform their classical counterparts when trained for addressing learning …
Representation learning via quantum neural tangent kernels
Variational quantum circuits are used in quantum machine learning and variational quantum
simulation tasks. Designing good variational circuits or predicting how well they perform for …
simulation tasks. Designing good variational circuits or predicting how well they perform for …
Feature learning in infinite-width neural networks
As its width tends to infinity, a deep neural network's behavior under gradient descent can
become simplified and predictable (eg given by the Neural Tangent Kernel (NTK)), if it is …
become simplified and predictable (eg given by the Neural Tangent Kernel (NTK)), if it is …
Global convergence of deep networks with one wide layer followed by pyramidal topology
QN Nguyen, M Mondelli - Advances in Neural Information …, 2020 - proceedings.neurips.cc
Recent works have shown that gradient descent can find a global minimum for over-
parameterized neural networks where the widths of all the hidden layers scale polynomially …
parameterized neural networks where the widths of all the hidden layers scale polynomially …
A spectral condition for feature learning
The push to train ever larger neural networks has motivated the study of initialization and
training at large network width. A key challenge is to scale training so that a network's …
training at large network width. A key challenge is to scale training so that a network's …