Stochastic bilevel optimization generalizes the classic stochastic optimization from the
minimization of a single objective to the minimization of an objective function that depends …

Stochastic bilevel optimization generalizes the classic stochastic optimization from the
minimization of a single objective to the minimization of an objective function that depends …

In this paper, we study the problem of constrained and stochastic continuous submodular
maximization. Even though the objective function is not concave (nor convex) and is defined …

Optimization on Riemannian manifolds widely arises in eigenvalue computation, density
functional theory, Bose--Einstein condensates, low rank nearest correlation, image …

In this paper, we study the problem of escaping from saddle points in smooth nonconvex
optimization problems subject to a convex set $\mathcal {C} $. We propose a generic …

This paper addresses the question of whether it can be beneficial for an optimization
algorithm to follow directions of negative curvature. Although prior work has established …

We present a novel adaptive optimization algorithm for large-scale machine learning
problems. Equipped with a low-cost estimate of local curvature and Lipschitz smoothness …

This paper proposes two efficient algorithms for computing approximate second-order
stationary points (SOSPs) of problems with generic smooth non-convex objective functions …

Inspired by the recent work FedNL (Safaryan et al, FedNL: Making Newton-Type Methods
Applicable to Federated Learning), we propose a new communication efficient second-order …

We propose an iterative algorithm for low-rank matrix completion with that can be interpreted
as an iteratively reweighted least squares (IRLS) algorithm, a saddle-escaping smoothing …