Why neural networks find simple solutions: The many regularizers of geometric complexity
In many contexts, simpler models are preferable to more complex models and the control of
this model complexity is the goal for many methods in machine learning such as …
this model complexity is the goal for many methods in machine learning such as …
On the lipschitz constant of deep networks and double descent
M Gamba, H Azizpour, M Björkman - arXiv preprint arXiv:2301.12309, 2023 - arxiv.org
Existing bounds on the generalization error of deep networks assume some form of smooth
or bounded dependence on the input variable, falling short of investigating the mechanisms …
or bounded dependence on the input variable, falling short of investigating the mechanisms …
Unraveling the enigma of double descent: An in-depth analysis through the lens of learned feature space
Double descent presents a counter-intuitive aspect within the machine learning domain, and
researchers have observed its manifestation in various models and tasks. While some …
researchers have observed its manifestation in various models and tasks. While some …
A margin-based multiclass generalization bound via geometric complexity
There has been considerable effort to better understand the generalization capabilities of
deep neural networks both as a means to unlock a theoretical understanding of their …
deep neural networks both as a means to unlock a theoretical understanding of their …
Prediction of Tropical Pacific Rain Rates with Over-parameterized Neural Networks
The prediction of tropical rain rates from atmospheric profiles poses significant challenges,
mainly due to the heavy-tailed distribution exhibited by tropical rainfall. This study introduces …
mainly due to the heavy-tailed distribution exhibited by tropical rainfall. This study introduces …
Multiple Descents in Unsupervised Learning: The Role of Noise, Domain Shift and Anomalies
K Rahimi, T Tirer, O Lindenbaum - arXiv preprint arXiv:2406.11703, 2024 - arxiv.org
The phenomenon of double descent has recently gained attention in supervised learning. It
challenges the conventional wisdom of the bias-variance trade-off by showcasing a …
challenges the conventional wisdom of the bias-variance trade-off by showcasing a …
Class-wise Activation Unravelling the Engima of Deep Double Descent
Y Gu - arXiv preprint arXiv:2405.07679, 2024 - arxiv.org
Double descent presents a counter-intuitive aspect within the machine learning domain, and
researchers have observed its manifestation in various models and tasks. While some …
researchers have observed its manifestation in various models and tasks. While some …
Understanding the Role of Optimization in Double Descent
CY Liu, J Flanigan - OPT 2023: Optimization for Machine Learning, 2023 - openreview.net
The phenomenon of model-wise double descent, where the test error peaks and then
reduces as the model size increases, is an interesting topic that has attracted the attention of …
reduces as the model size increases, is an interesting topic that has attracted the attention of …
[HTML][HTML] On Label Noise in Image Classification: An Aleatoric Uncertainty Perspective
E Englesson - 2024 - diva-portal.org
Deep neural networks and large-scale datasets have revolutionized the field of machine
learning. However, these large networks are susceptible to overfitting to label noise …
learning. However, these large networks are susceptible to overfitting to label noise …
Different Faces of Model Scaling in Supervised and Self-Supervised Learning
The quality of the representations learned by neural networks depends on several factors,
including the loss function, learning algorithm, and model architecture. In this work, we use …
including the loss function, learning algorithm, and model architecture. In this work, we use …