Machine learning–accelerated computational fluid dynamics
Numerical simulation of fluids plays an essential role in modeling many physical
phenomena, such as weather, climate, aerodynamics, and plasma physics. Fluids are well …
phenomena, such as weather, climate, aerodynamics, and plasma physics. Fluids are well …
Heavy ball neural ordinary differential equations
We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the
continuous limit of the classical momentum accelerated gradient descent, to improve neural …
continuous limit of the classical momentum accelerated gradient descent, to improve neural …
Lipschitz recurrent neural networks
Viewing recurrent neural networks (RNNs) as continuous-time dynamical systems, we
propose a recurrent unit that describes the hidden state's evolution with two parts: a well …
propose a recurrent unit that describes the hidden state's evolution with two parts: a well …
Pyramid convolutional RNN for MRI image reconstruction
Fast and accurate MRI image reconstruction from undersampled data is crucial in clinical
practice. Deep learning based reconstruction methods have shown promising advances in …
practice. Deep learning based reconstruction methods have shown promising advances in …
Implicit graph neural networks: a monotone operator viewpoint
Implicit graph neural networks (IGNNs)–that solve a fixed-point equilibrium equation using
Picard iteration for representation learning–have shown remarkable performance in learning …
Picard iteration for representation learning–have shown remarkable performance in learning …
AdamR-GRUs: Adaptive momentum-based Regularized GRU for HMER problems
Abstract Handwritten Mathematical Expression Recognition (HMER) is essential to online
education and scientific research. However, discerning the length and characters of …
education and scientific research. However, discerning the length and characters of …
Improving neural ordinary differential equations with nesterov's accelerated gradient method
We propose the Nesterov neural ordinary differential equations (NesterovNODEs), whose
layers solve the second-order ordinary differential equations (ODEs) limit of Nesterov's …
layers solve the second-order ordinary differential equations (ODEs) limit of Nesterov's …
An automatic learning rate decay strategy for stochastic gradient descent optimization methods in neural networks
Abstract Stochastic Gradient Descent (SGD) series optimization methods play the vital role
in training neural networks, attracting growing attention in science and engineering fields of …
in training neural networks, attracting growing attention in science and engineering fields of …
Improving deep neural networks' training for image classification with nonlinear conjugate gradient-style adaptive momentum
Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or
improving training deep neural networks (DNNs). In deep learning practice, the momentum …
improving training deep neural networks (DNNs). In deep learning practice, the momentum …
[HTML][HTML] Decentralized concurrent learning with coordinated momentum and restart
This paper studies the stability and convergence properties of a class of multi-agent
concurrent learning (CL) algorithms with momentum and restart. Such algorithms can be …
concurrent learning (CL) algorithms with momentum and restart. Such algorithms can be …