Lower bounds and optimal algorithms for personalized federated learning
In this work, we consider the optimization formulation of personalized federated learning
recently introduced by Hanzely & Richtarik (2020) which was shown to give an alternative …
recently introduced by Hanzely & Richtarik (2020) which was shown to give an alternative …
Acceleration methods
This monograph covers some recent advances in a range of acceleration techniques
frequently used in convex optimization. We first use quadratic optimization problems to …
frequently used in convex optimization. We first use quadratic optimization problems to …
Stochastic distributed learning with gradient quantization and double-variance reduction
We consider distributed optimization over several devices, each sending incremental model
updates to a central server. This setting is considered, for instance, in federated learning …
updates to a central server. This setting is considered, for instance, in federated learning …
Momentum and stochastic momentum for stochastic gradient, newton, proximal point and subspace descent methods
N Loizou, P Richtárik - Computational Optimization and Applications, 2020 - Springer
In this paper we study several classes of stochastic optimization algorithms enriched with
heavy ball momentum. Among the methods studied are: stochastic gradient descent …
heavy ball momentum. Among the methods studied are: stochastic gradient descent …
Don't jump through hoops and remove those loops: SVRG and Katyusha are better without the outer loop
The stochastic variance-reduced gradient method (SVRG) and its accelerated variant
(Katyusha) have attracted enormous attention in the machine learning community in the last …
(Katyusha) have attracted enormous attention in the machine learning community in the last …
Convex optimization algorithms in medical image reconstruction—in the age of AI
The past decade has seen the rapid growth of model based image reconstruction (MBIR)
algorithms, which are often applications or adaptations of convex optimization algorithms …
algorithms, which are often applications or adaptations of convex optimization algorithms …
Sharper rates for separable minimax and finite sum optimization via primal-dual extragradient methods
We design accelerated algorithms with improved rates for several fundamental classes of
optimization problems. Our algorithms all build upon techniques related to the analysis of …
optimization problems. Our algorithms all build upon techniques related to the analysis of …
Optimal decentralized distributed algorithms for stochastic convex optimization
We consider stochastic convex optimization problems with affine constraints and develop
several methods using either primal or dual approach to solve it. In the primal case, we use …
several methods using either primal or dual approach to solve it. In the primal case, we use …
On the fast convergence of minibatch heavy ball momentum
Simple stochastic momentum methods are widely used in machine learning optimization,
but their good practical performance is at odds with an absence of theoretical guarantees of …
but their good practical performance is at odds with an absence of theoretical guarantees of …
Stochastic first-order methods: non-asymptotic and computer-aided analyses via potential functions
We provide a novel computer-assisted technique for systematically analyzing first-order
methods for optimization. In contrast with previous works, the approach is particularly suited …
methods for optimization. In contrast with previous works, the approach is particularly suited …