Decentralized stochastic bilevel optimization with improved per-iteration complexity

X Chen, M Huang, S Ma… - … on Machine Learning, 2023 - proceedings.mlr.press
Bilevel optimization recently has received tremendous attention due to its great success in
solving important machine learning problems like meta learning, reinforcement learning …

Simfbo: Towards simple, flexible and communication-efficient federated bilevel learning

Y Yang, P Xiao, K Ji - Advances in Neural Information …, 2024 - proceedings.neurips.cc
Federated bilevel optimization (FBO) has shown great potential recently in machine learning
and edge computing due to the emerging nested optimization structure in meta-learning …

Alternating projected sgd for equality-constrained bilevel optimization

Q Xiao, H Shen, W Yin, T Chen - … Conference on Artificial …, 2023 - proceedings.mlr.press
Bilevel optimization, which captures the inherent nested structure of machine learning
problems, is gaining popularity in many recent applications. Existing works on bilevel …

Achieving linear speedup in non-iid federated bilevel learning

M Huang, D Zhang, K Ji - International Conference on …, 2023 - proceedings.mlr.press
Federated bilevel learning has received increasing attention in various emerging machine
learning and communication applications. Recently, several Hessian-vector-based …

Communication-efficient federated bilevel optimization with global and local lower level problems

J Li, F Huang, H Huang - Advances in Neural Information …, 2024 - proceedings.neurips.cc
Bilevel Optimization has witnessed notable progress recently with new emerging efficient
algorithms. However, its application in the Federated Learning setting remains relatively …

Stability and generalization of the decentralized stochastic gradient descent ascent algorithm

M Zhu, L Shen, B Du, D Tao - Advances in Neural …, 2024 - proceedings.neurips.cc
The growing size of available data has attracted increasing interest in solving minimax
problems in a decentralized manner for various machine learning tasks. Previous theoretical …

Communication-efficient federated hypergradient computation via aggregated iterative differentiation

P Xiao, K Ji - International Conference on Machine Learning, 2023 - proceedings.mlr.press
Federated bilevel optimization has attracted increasing attention due to emerging machine
learning and communication applications. The biggest challenge lies in computing the …

Efficiently escaping saddle points in bilevel optimization

M Huang, X Chen, K Ji, S Ma, L Lai - arXiv preprint arXiv:2202.03684, 2022 - arxiv.org
Bilevel optimization is one of the fundamental problems in machine learning and
optimization. Recent theoretical developments in bilevel optimization focus on finding the …

Prometheus: taming sample and communication complexities in constrained decentralized stochastic bilevel learning

Z Liu, X Zhang, P Khanduri, S Lu… - … Conference on Machine …, 2023 - proceedings.mlr.press
In recent years, decentralized bilevel optimization has gained significant attention thanks to
its versatility in modeling a wide range of multi-agent learning problems, such as multi-agent …

Jointly improving the sample and communication complexities in decentralized stochastic minimax optimization

X Zhang, G Mancino-Ball, NS Aybat, Y Xu - Proceedings of the AAAI …, 2024 - ojs.aaai.org
We propose a novel single-loop decentralized algorithm, DGDA-VR, for solving the
stochastic nonconvex strongly-concave minimax problems over a connected network of …