Deep learning is a sub-branch of artificial intelligence that acquires knowledge by training a
neural network. It has many applications in the field of banking, automobile industry …

We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the
continuous limit of the classical momentum accelerated gradient descent, to improve neural …

Sharpness aware minimization (SAM) optimizer has been extensively explored as it can
generalize better for training deep neural networks via introducing extra perturbation steps …

The stability and generalization of stochastic gradient-based methods provide valuable
insights into understanding the algorithmic performance of machine learning models. As the …

While many distributed optimization algorithms have been proposed for solving smooth or
convex problems over the networks, few of them can handle non-convex and non-smooth …

We consider a distributed learning problem in a wireless network, consisting of distributed
edge devices and a parameter server (PS). The objective function is a sum of the edge …

We combine two advanced ideas widely used in optimization for machine learning:\textit
{shuffling} strategy and\textit {momentum} technique to develop a novel shuffling gradient …

Heavy ball momentum is crucial in accelerating (stochastic) gradient-based optimization
algorithms for machine learning. Existing heavy ball momentum is usually weighted by a …

Designing deep neural networks is an art that often involves an expensive search over
candidate architectures. To overcome this for recurrent neural nets (RNNs), we establish a …

We propose the Nesterov neural ordinary differential equations (NesterovNODEs), whose
layers solve the second-order ordinary differential equations (ODEs) limit of Nesterov's …