Communication-efficient distributed deep learning: A comprehensive survey
Distributed deep learning (DL) has become prevalent in recent years to reduce training time
by leveraging multiple computing devices (eg, GPUs/TPUs) due to larger models and …
by leveraging multiple computing devices (eg, GPUs/TPUs) due to larger models and …
Adaptive gradient sparsification for efficient federated learning: An online learning approach
Federated learning (FL) is an emerging technique for training machine learning models
using geographically dispersed data collected by local entities. It includes local computation …
using geographically dispersed data collected by local entities. It includes local computation …
Communication compression techniques in distributed deep learning: A survey
Nowadays, the training data and neural network models are getting increasingly large. The
training time of deep learning will become unbearably long on a single machine. To reduce …
training time of deep learning will become unbearably long on a single machine. To reduce …
Towards efficient communications in federated learning: A contemporary survey
In the traditional distributed machine learning scenario, the user's private data is transmitted
between clients and a central server, which results in significant potential privacy risks. In …
between clients and a central server, which results in significant potential privacy risks. In …
Communication-efficient distributed deep learning with merged gradient sparsification on GPUs
Distributed synchronous stochastic gradient descent (SGD) algorithms are widely used in
large-scale deep learning applications, while it is known that the communication bottleneck …
large-scale deep learning applications, while it is known that the communication bottleneck …
Communication-efficient decentralized learning with sparsification and adaptive peer selection
The increasing size of machine learning models, especially deep neural network models,
can improve the model generalization capability. However, large models require more …
can improve the model generalization capability. However, large models require more …
Slashing communication traffic in federated learning by transmitting clustered model updates
Federated Learning (FL) is an emerging decentralized learning framework through which
multiple clients can collaboratively train a learning model. However, a major obstacle that …
multiple clients can collaboratively train a learning model. However, a major obstacle that …
Communication optimization algorithms for distributed deep learning systems: A survey
E Yu, D Dong, X Liao - IEEE Transactions on Parallel and …, 2023 - ieeexplore.ieee.org
Deep learning's widespread adoption in various fields has made distributed training across
multiple computing nodes essential. However, frequent communication between nodes can …
multiple computing nodes essential. However, frequent communication between nodes can …
A layer selection optimizer for communication-efficient decentralized federated deep learning
Federated Learning (FL) systems orchestrate the cooperative training of a shared Machine
Learning (ML) model across connected devices. Recently, decentralized FL architectures …
Learning (ML) model across connected devices. Recently, decentralized FL architectures …
Dynamic layer-wise sparsification for distributed deep learning
Distributed stochastic gradient descent (SGD) algorithms are becoming popular in speeding
up deep learning model training by employing multiple computational devices (named …
up deep learning model training by employing multiple computational devices (named …