Variance reduced proxskip: Algorithm, theory and application to federated learning
We study distributed optimization methods based on the {\em local training (LT)} paradigm,
ie, methods which achieve communication efficiency by performing richer local gradient …
ie, methods which achieve communication efficiency by performing richer local gradient …
Towards out-of-distribution generalization: A survey
Traditional machine learning paradigms are based on the assumption that both training and
test data follow the same statistical pattern, which is mathematically referred to as …
test data follow the same statistical pattern, which is mathematically referred to as …
Proxskip: Yes! local gradient steps provably lead to communication acceleration! finally!
We introduce ProxSkip—a surprisingly simple and provably efficient method for minimizing
the sum of a smooth ($ f $) and an expensive nonsmooth proximable ($\psi $) function. The …
the sum of a smooth ($ f $) and an expensive nonsmooth proximable ($\psi $) function. The …
Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients
We consider a standard federated learning (FL) setup where a group of clients periodically
coordinate with a central server to train a statistical model. We develop a general algorithmic …
coordinate with a central server to train a statistical model. We develop a general algorithmic …
Communication-efficient and distributed learning over wireless networks: Principles and applications
Machine learning (ML) is a promising enabler for the fifth-generation (5G) communication
systems and beyond. By imbuing intelligence into the network edge, edge nodes can …
systems and beyond. By imbuing intelligence into the network edge, edge nodes can …
Fedavg with fine tuning: Local updates lead to representation learning
Abstract The Federated Averaging (FedAvg) algorithm, which consists of alternating
between a few local stochastic gradient updates at client nodes, followed by a model …
between a few local stochastic gradient updates at client nodes, followed by a model …
A unified analysis of federated learning with arbitrary client participation
Federated learning (FL) faces challenges of intermittent client availability and computation/
communication efficiency. As a result, only a small subset of clients can participate in FL at a …
communication efficiency. As a result, only a small subset of clients can participate in FL at a …
Fedmsplit: Correlation-adaptive federated multi-task learning across multimodal split networks
With the advancement of data collection techniques, end users are interested in how
different types of data can collaborate to improve our life experiences. Multimodal Federated …
different types of data can collaborate to improve our life experiences. Multimodal Federated …
FedNL: Making Newton-type methods applicable to federated learning
Inspired by recent work of Islamov et al (2021), we propose a family of Federated Newton
Learn (FedNL) methods, which we believe is a marked step in the direction of making …
Learn (FedNL) methods, which we believe is a marked step in the direction of making …
SoteriaFL: A unified framework for private federated learning with communication compression
To enable large-scale machine learning in bandwidth-hungry environments such as
wireless networks, significant progress has been made recently in designing communication …
wireless networks, significant progress has been made recently in designing communication …