Why neural networks find simple solutions: The many regularizers of geometric complexity

B Dherin, M Munn, M Rosca… - Advances in Neural …, 2022 - proceedings.neurips.cc
In many contexts, simpler models are preferable to more complex models and the control of
this model complexity is the goal for many methods in machine learning such as …

On the lipschitz constant of deep networks and double descent

M Gamba, H Azizpour, M Björkman - arXiv preprint arXiv:2301.12309, 2023 - arxiv.org
Existing bounds on the generalization error of deep networks assume some form of smooth
or bounded dependence on the input variable, falling short of investigating the mechanisms …

Unraveling the enigma of double descent: An in-depth analysis through the lens of learned feature space

Y Gu, X Zheng, T Aste - arXiv preprint arXiv:2310.13572, 2023 - arxiv.org
Double descent presents a counter-intuitive aspect within the machine learning domain, and
researchers have observed its manifestation in various models and tasks. While some …

A margin-based multiclass generalization bound via geometric complexity

M Munn, B Dherin, J Gonzalvo - Topological, Algebraic and …, 2023 - proceedings.mlr.press
There has been considerable effort to better understand the generalization capabilities of
deep neural networks both as a means to unlock a theoretical understanding of their …

Prediction of Tropical Pacific Rain Rates with Over-parameterized Neural Networks

H You, J Wang, RKW Wong… - … Intelligence for the …, 2024 - journals.ametsoc.org
The prediction of tropical rain rates from atmospheric profiles poses significant challenges,
mainly due to the heavy-tailed distribution exhibited by tropical rainfall. This study introduces …

Multiple Descents in Unsupervised Learning: The Role of Noise, Domain Shift and Anomalies

K Rahimi, T Tirer, O Lindenbaum - arXiv preprint arXiv:2406.11703, 2024 - arxiv.org
The phenomenon of double descent has recently gained attention in supervised learning. It
challenges the conventional wisdom of the bias-variance trade-off by showcasing a …

Class-wise Activation Unravelling the Engima of Deep Double Descent

Y Gu - arXiv preprint arXiv:2405.07679, 2024 - arxiv.org
Double descent presents a counter-intuitive aspect within the machine learning domain, and
researchers have observed its manifestation in various models and tasks. While some …

Understanding the Role of Optimization in Double Descent

CY Liu, J Flanigan - OPT 2023: Optimization for Machine Learning, 2023 - openreview.net
The phenomenon of model-wise double descent, where the test error peaks and then
reduces as the model size increases, is an interesting topic that has attracted the attention of …

[HTML][HTML] On Label Noise in Image Classification: An Aleatoric Uncertainty Perspective

E Englesson - 2024 - diva-portal.org
Deep neural networks and large-scale datasets have revolutionized the field of machine
learning. However, these large networks are susceptible to overfitting to label noise …

Different Faces of Model Scaling in Supervised and Self-Supervised Learning

M Gamba, A Ghosh, KK Agrawal, B Richards… - 2024 - diva-portal.org
The quality of the representations learned by neural networks depends on several factors,
including the loss function, learning algorithm, and model architecture. In this work, we use …