Block coordinate descent on smooth manifolds: Convergence theory and twenty-one examples
Block coordinate descent is an optimization paradigm that iteratively updates one block of
variables at a time, making it quite amenable to big data applications due to its scalability …
variables at a time, making it quite amenable to big data applications due to its scalability …
Fenchel conjugate via Busemann function on Hadamard manifolds
GC Bento, JC Neto, ÍDL Melo - Applied Mathematics & Optimization, 2023 - Springer
In this paper we introduce a Fenchel-type conjugate, given as the supremum of convex
functions, via Busemann functions. It is known that Busemann functions are smooth convex …
functions, via Busemann functions. It is known that Busemann functions are smooth convex …
Coordinate descent on the orthogonal group for recurrent neural network training
We address the poor scalability of learning algorithms for orthogonal recurrent neural
networks via the use of stochastic coordinate descent on the orthogonal group, leading to a …
networks via the use of stochastic coordinate descent on the orthogonal group, leading to a …
Riemannian Preconditioned Coordinate Descent for Low Multilinear Rank Approximation
M Hamed, R Hosseini - SIAM Journal on Matrix Analysis and Applications, 2024 - SIAM
This paper presents a memory-efficient, first-order method for low multilinear rank
approximation of high-order, high-dimensional tensors. In our method, we exploit the second …
approximation of high-order, high-dimensional tensors. In our method, we exploit the second …
Block majorization-minimization with diminishing radius for constrained nonconvex optimization
Block majorization-minimization (BMM) is a simple iterative algorithm for nonconvex
constrained optimization that sequentially minimizes majorizing surrogates of the objective …
constrained optimization that sequentially minimizes majorizing surrogates of the objective …
Riemannian coordinate descent algorithms on matrix manifolds
Many machine learning applications are naturally formulated as optimization problems on
Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the …
Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the …
Convergence and complexity of block majorization-minimization for constrained block-Riemannian optimization
Block majorization-minimization (BMM) is a simple iterative algorithm for nonconvex
optimization that sequentially minimizes a majorizing surrogate of the objective function in …
optimization that sequentially minimizes a majorizing surrogate of the objective function in …
Accelerated Gradient Dynamics on Riemannian Manifolds: Faster Rate and Trajectory Convergence
In order to minimize a differentiable geodesically convex function, we study a second-order
dynamical system on Riemannian manifolds with an asymptotically vanishing damping term …
dynamical system on Riemannian manifolds with an asymptotically vanishing damping term …
Randomized Submanifold Subgradient Method for Optimization over Stiefel Manifolds
Optimization over Stiefel manifolds has found wide applications in many scientific and
engineering domains. Despite considerable research effort, high-dimensional optimization …
engineering domains. Despite considerable research effort, high-dimensional optimization …
On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation
In many applications, it is desired to obtain extreme eigenvalues and eigenvectors of large
Hermitian matrices by efficient and compact algorithms. In particular, orthogonalization-free …
Hermitian matrices by efficient and compact algorithms. In particular, orthogonalization-free …