Efficient compression of overparameterized deep models through low-dimensional learning dynamics
Overparameterized models have proven to be powerful tools for solving various machine
learning tasks. However, overparameterization often leads to a substantial increase in …
learning tasks. However, overparameterization often leads to a substantial increase in …
Pretraining with Random Noise for Fast and Robust Learning without Weight Transport
The brain prepares for learning even before interacting with the environment, by refining and
optimizing its structures through spontaneous neural activity that resembles random noise …
optimizing its structures through spontaneous neural activity that resembles random noise …
[PDF][PDF] Efficiency Calibration of Implicit Regularization in Deep Networks via Self-paced Curriculum-Driven Singular Value Selection
The generalization of neural networks has been a major focus of research in deep learning.
It is often interpreted as an implicit bias towards solutions with specific properties. Especially …
It is often interpreted as an implicit bias towards solutions with specific properties. Especially …