Trust region methods are a class of numerical methods for optimization. Unlike line search
type methods where a line search is carried out in each iteration, trust region methods …

This book is on conjugate gradient methods for unconstrained optimization. The concept of
conjugacy was introduced by Magnus Hestenes and Garrett Birkhoff in 1936 in the context of …

We consider variants of trust-region and adaptive cubic regularization methods for non-
convex optimization, in which the Hessian matrix is approximated. Under certain condition …

While first-order optimization methods, such as SGD are popular in machine learning (ML),
they come with well-known deficiencies, including relatively-slow convergence, sensitivity to …

We consider the minimization of a nonconvex quadratic form regularized by a cubic term,
which may exhibit saddle points and a suboptimal local minimum. Nonetheless, we prove …

We develop several new communication-efficient second-order methods for distributed
optimization. Our first method, NEWTON-STAR, is a variant of Newton's method from which it …

Do you know the difference between an optimist and a pessimist? The former believes we
live in the best possible world, and the latter is afraid that the former might be right.… In that …

This paper describes a new instance library for quadratic programming (QP), ie, the family of
continuous and (mixed)-integer optimization problems where the objective function and/or …

This paper settles an open and challenging question pertaining to the design of simple and
optimal high-order methods for solving smooth and monotone variational inequalities (VIs) …

In this paper, we propose a new randomized second-order optimization algorithm—
Stochastic Subspace Cubic Newton (SSCN)—for minimizing a high dimensional convex …