Proving linear mode connectivity of neural networks via optimal transport
The energy landscape of high-dimensional non-convex optimization problems is crucial to
understanding the effectiveness of modern deep neural network architectures. Recent works …
understanding the effectiveness of modern deep neural network architectures. Recent works …
On the opportunities of green computing: A survey
Artificial Intelligence (AI) has achieved significant advancements in technology and research
with the development over several decades, and is widely used in many areas including …
with the development over several decades, and is widely used in many areas including …
A survey of lottery ticket hypothesis
B Liu, Z Zhang, P He, Z Wang, Y Xiao, R Ye… - arXiv preprint arXiv …, 2024 - arxiv.org
The Lottery Ticket Hypothesis (LTH) states that a dense neural network model contains a
highly sparse subnetwork (ie, winning tickets) that can achieve even better performance …
highly sparse subnetwork (ie, winning tickets) that can achieve even better performance …
Polynomially over-parameterized convolutional neural networks contain structured strong winning lottery tickets
A Da Cunha, F d'Amore - Advances in Neural Information …, 2024 - proceedings.neurips.cc
Abstract The Strong Lottery Ticket Hypothesis (SLTH) states that randomly-initialised neural
networks likely contain subnetworks that perform well without any training. Although …
networks likely contain subnetworks that perform well without any training. Although …
On the multidimensional random subset sum problem
In the Random Subset Sum Problem, given $ n $ iid random variables $ X_1,..., X_n $, we
wish to approximate any point $ z\in [-1, 1] $ as the sum of a suitable subset $ X_ {i_1 (z)} …
wish to approximate any point $ z\in [-1, 1] $ as the sum of a suitable subset $ X_ {i_1 (z)} …
Successfully applying lottery ticket hypothesis to diffusion model
Despite the success of diffusion models, the training and inference of diffusion models are
notoriously expensive due to the long chain of the reverse process. In parallel, the Lottery …
notoriously expensive due to the long chain of the reverse process. In parallel, the Lottery …
Strong Lottery Ticket Hypothesis with –perturbation
Abstract The strong Lottery Ticket Hypothesis (LTH)(Ramanujan et al., 2019; Zhou et al.,
2019) claims the existence of a subnetwork in a sufficiently large, randomly initialized neural …
2019) claims the existence of a subnetwork in a sufficiently large, randomly initialized neural …
Cyclic Sparse Training: Is it Enough?
A Gadhikar, SH Nelaturu, R Burkholz - arXiv preprint arXiv:2406.02773, 2024 - arxiv.org
The success of iterative pruning methods in achieving state-of-the-art sparse networks has
largely been attributed to improved mask identification and an implicit regularization induced …
largely been attributed to improved mask identification and an implicit regularization induced …
Partial Search in a Frozen Network is Enough to Find a Strong Lottery Ticket
Randomly initialized dense networks contain subnetworks that achieve high accuracy
without weight learning--strong lottery tickets (SLTs). Recently, Gadhikar et al.(2023) …
without weight learning--strong lottery tickets (SLTs). Recently, Gadhikar et al.(2023) …
Considering Layerwise Importance in the Lottery Ticket Hypothesis
B Vandersmissen, J Oramas - arXiv preprint arXiv:2302.11244, 2023 - arxiv.org
The Lottery Ticket Hypothesis (LTH) showed that by iteratively training a model, removing
connections with the lowest global weight magnitude and rewinding the remaining …
connections with the lowest global weight magnitude and rewinding the remaining …