Decentralized stochastic bilevel optimization with improved per-iteration complexity
Bilevel optimization recently has received tremendous attention due to its great success in
solving important machine learning problems like meta learning, reinforcement learning …
solving important machine learning problems like meta learning, reinforcement learning …
Will bilevel optimizers benefit from loops
Bilevel optimization has arisen as a powerful tool for solving a variety of machine learning
problems. Two current popular bilevel optimizers AID-BiO and ITD-BiO naturally involve …
problems. Two current popular bilevel optimizers AID-BiO and ITD-BiO naturally involve …
First-order penalty methods for bilevel optimization
In this paper, we study a class of unconstrained and constrained bilevel optimization
problems in which the lower level is a possibly nonsmooth convex optimization problem …
problems in which the lower level is a possibly nonsmooth convex optimization problem …
Optimal algorithms for stochastic bilevel optimization under relaxed smoothness conditions
We consider stochastic bilevel optimization problems involving minimizing an upper-level
($\texttt {UL} $) function that is dependent on the arg-min of a strongly-convex lower-level …
($\texttt {UL} $) function that is dependent on the arg-min of a strongly-convex lower-level …
Achieving linear speedup in non-iid federated bilevel learning
Federated bilevel learning has received increasing attention in various emerging machine
learning and communication applications. Recently, several Hessian-vector-based …
learning and communication applications. Recently, several Hessian-vector-based …
Accelerating inexact hypergradient descent for bilevel optimization
We present a method for solving general nonconvex-strongly-convex bilevel optimization
problems. Our method---the Restarted Accelerated HyperGradient Descent (RAHGD) …
problems. Our method---the Restarted Accelerated HyperGradient Descent (RAHGD) …
Efficient hyper-parameter optimization with cubic regularization
As hyper-parameters are ubiquitous and can significantly affect the model performance,
hyper-parameter optimization is extremely important in machine learning. In this paper, we …
hyper-parameter optimization is extremely important in machine learning. In this paper, we …
Near-optimal fully first-order algorithms for finding stationary points in bilevel optimization
Bilevel optimization has various applications such as hyper-parameter optimization and
meta-learning. Designing theoretically efficient algorithms for bilevel optimization is more …
meta-learning. Designing theoretically efficient algorithms for bilevel optimization is more …
What is a good metric to study generalization of minimax learners?
Minimax optimization has served as the backbone of many machine learning problems.
Although the convergence behavior of optimization algorithms has been extensively studied …
Although the convergence behavior of optimization algorithms has been extensively studied …
Escaping saddle points in zeroth-order optimization: the power of two-point estimators
Two-point zeroth order methods are important in many applications of zeroth-order
optimization arising in robotics, wind farms, power systems, online optimization, and …
optimization arising in robotics, wind farms, power systems, online optimization, and …