On the impact of machine learning randomness on group fairness
Statistical measures for group fairness in machine learning reflect the gap in performance of
algorithms across different groups. These measures, however, exhibit a high variance …
algorithms across different groups. These measures, however, exhibit a high variance …
Tighter lower bounds for shuffling SGD: Random permutations and beyond
We study convergence lower bounds of without-replacement stochastic gradient descent
(SGD) for solving smooth (strongly-) convex finite-sum minimization problems. Unlike most …
(SGD) for solving smooth (strongly-) convex finite-sum minimization problems. Unlike most …
Repeated random sampling for minimizing the time-to-accuracy of learning
Methods for carefully selecting or generating a small set of training data to learn from, ie,
data pruning, coreset selection, and data distillation, have been shown to be effective in …
data pruning, coreset selection, and data distillation, have been shown to be effective in …
Mini-Batch Optimization of Contrastive Loss
Contrastive learning has gained significant attention as a method for self-supervised
learning. The contrastive loss function ensures that embeddings of positive sample pairs …
learning. The contrastive loss function ensures that embeddings of positive sample pairs …
[PDF][PDF] Coordinating distributed example orders for provably accelerated training
Recent research on online Gradient Balancing (GraB) has revealed that there exist
permutation-based example orderings for SGD that are guaranteed to outperform random …
permutation-based example orderings for SGD that are guaranteed to outperform random …
Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling
Bilevel Optimization has experienced significant advancements recently with the
introduction of new efficient algorithms. Mirroring the success in single-level optimization …
introduction of new efficient algorithms. Mirroring the success in single-level optimization …
CD-GraB: Coordinating Distributed Example Orders for Provably Accelerated Training
Recent research on online Gradient Balancing (GraB) has revealed that there exist
permutation-based example orderings that are guaranteed to outperform random reshuffling …
permutation-based example orderings that are guaranteed to outperform random reshuffling …
Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
Federated Learning (FL) is a distributed machine learning framework in communication
network systems. However, the systems' Non-Independent and Identically Distributed (Non …
network systems. However, the systems' Non-Independent and Identically Distributed (Non …
Stochastic optimization with arbitrary recurrent data sampling
WG Powell, H Lyu - arXiv preprint arXiv:2401.07694, 2024 - arxiv.org
For obtaining optimal first-order convergence guarantee for stochastic optimization, it is
necessary to use a recurrent data sampling algorithm that samples every data point with …
necessary to use a recurrent data sampling algorithm that samples every data point with …
On the Last-Iterate Convergence of Shuffling Gradient Methods
Shuffling gradient methods, which are also known as stochastic gradient descent (SGD)
without replacement, are widely implemented in practice, particularly including three popular …
without replacement, are widely implemented in practice, particularly including three popular …