Efficient compression of overparameterized deep models through low-dimensional learning dynamics

SM Kwon, Z Zhang, D Song, L Balzano… - arXiv preprint arXiv …, 2023 - arxiv.org
Overparameterized models have proven to be powerful tools for solving various machine
learning tasks. However, overparameterization often leads to a substantial increase in …

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport

J Cheon, SW Lee, SB Paik - arXiv preprint arXiv:2405.16731, 2024 - arxiv.org
The brain prepares for learning even before interacting with the environment, by refining and
optimizing its structures through spontaneous neural activity that resembles random noise …

[PDF][PDF] Efficiency Calibration of Implicit Regularization in Deep Networks via Self-paced Curriculum-Driven Singular Value Selection

Z Li, S Chen, J Yang, L Luo - ijcai.org
The generalization of neural networks has been a major focus of research in deep learning.
It is often interpreted as an implicit bias towards solutions with specific properties. Especially …