Catastrophic forgetting (CF) is a significant challenge in continual learning (CL). In
regularization-based approaches to mitigate CF, modifications to important training …

We present a novel adaptive optimization algorithm for large-scale machine learning
problems. Equipped with a low-cost estimate of local curvature and Lipschitz smoothness …

This paper reveals that a common and central role, played in many error bound (EB)
conditions and a variety of gradient-type methods, is a residual measure operator. On one …

The condition number of a differentiable convex function, namely the ratio of its smoothness
to strong convexity constants, is closely tied to fundamental properties of the function. In …

In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE)
method in which sample size is gradually increasing to quickly obtain a solution whose …

We consider the problem of minimizing the sum of an average of a large number of smooth
convex component functions and a possibly nonsmooth convex function that admits a simple …

This work presents a new optimization algorithm for empirical risk minimization. The
algorithm bridges the gap between first-and second-order methods by computing a search …

The condition number of a smooth convex function, namely the ratio of its smoothness to
strong convexity constants, is closely tied to fundamental properties of the function. In …

This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a
(possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on …

Many important problems in machine learning (ML) and data science are formulated as
optimization problems and solved using optimization algorithms. With the scale of modern …