Deep learning is transforming most areas of science and technology, including electron
microscopy. This review paper offers a practical perspective aimed at developers with …

Despite the growing interest in unsupervised learning, extracting meaningful knowledge
from unlabelled audio remains an open challenge. To take a step in this direction, we …

SGD with momentum (SGDM) has been widely applied in many machine learning tasks, and
it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise …

Catastrophic forgetting affects the training of neural networks, limiting their ability to learn
multiple tasks sequentially. From the perspective of the well established plasticity-stability …

Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL)
capabilities, enabling them to solve unseen tasks solely based on input contexts without …

It has long been argued that minibatch stochastic gradient descent can generalize better
than large batch gradient descent in deep neural networks. However recent papers have …

Adaptive gradient methods, which adopt historical gradient information to automatically
adjust the learning rate, despite the nice property of fast convergence, have been observed …

Motivated by applications in Game Theory, Optimization, and Generative Adversarial
Networks, recent work of Daskalakis et al\cite {DISZ17} and follow-up work of Liang and …

In this paper we study the smooth convex-concave saddle point problem. Specifically, we
analyze the last iterate convergence properties of the Extragradient (EG) algorithm. It is well …

In machine learning applications, it is well known that carefully designed learning rate (step
size) schedules can significantly improve the convergence of commonly used first-order …