Abstract Stochastic Gradient Descent (SGD) is one of the many iterative optimization
methods that are widely used in solving machine learning problems. These methods display …

The rapid development of machine learning and deep learning has introduced increasingly
complex optimization challenges that must be addressed. Indeed, training modern …

Methods with preconditioned updates show up well in badly scaled and/or ill-conditioned
convex optimization problems. However, theoretical analysis of these methods in distributed …

Second-order methods for convex optimization outperform first-order methods in terms of
theoretical iteration convergence, achieving rates up to $ O (k^{-5}) $ for highly-smooth …

Vertical Federated Learning (VFL) aims to enable collaborative training of deep learning
models while maintaining privacy protection. However, the VFL procedure still has …

We present AI-SARAH, a practical variant of SARAH. As a variant of SARAH, this algorithm
employs the stochastic recursive gradient yet adjusts step-size based on local geometry. AI …

Variational inequalities represent a broad class of problems, including minimization and min-
max problems, commonly found in machine learning. Existing second-order and high-order …

The paper studies the properties of stochastic gradient methods with preconditioning. We
focus on momentum updated preconditioners with momentum coefficient $\beta $. Seeking …

Стохастический градиентный спуск (SGD) является одним из множества методов
оптимизации, используемых для решения задач машинного обучения. Практичность и …