Most commonly used distributed machine learning systems are either synchronous or
centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a …

While training a machine learning model using multiple workers, each of which collects data
from its own data source, it would be useful when the data collected from different workers …

In this article, we focus on solving a distributed convex optimization problem in a network,
where each agent has its own convex cost function and the goal is to minimize the sum of …

Optimizing distributed learning systems is an art of balancing between computation and
communication. There have been two lines of research that try to deal with slower …

Decentralized SGD is an emerging training method for deep learning known for its much
less (thus faster) communication per iteration, which relaxes the averaging step in parallel …

Decentralized optimization to minimize a finite sum of functions, distributed over a network of
nodes, has been a significant area within control and signal-processing research due to its …

This article reviews significant advances in networked signal and information processing
(SIP), which have enabled in the last 25 years extending decision making and inference …

In this paper, we study decentralized online stochastic non-convex optimization over a
network of nodes. Integrating a technique called gradient tracking in decentralized …

This article studies a class of nonsmooth decentralized multiagent optimization problems
where the agents aim at minimizing a sum of local strongly-convex smooth components plus …

There is growing interest in large-scale machine learning and optimization over
decentralized networks, eg in the context of multi-agent learning and federated learning …