Deep neural networks are among the most widely applied machine learning tools showing
outstanding performance in a broad range of tasks. We present a method for folding a deep …

Training neural networks with backpropagation (BP) requires a sequential passing of
activations and gradients. This has been recognized as the lockings (ie, the forward …

Advances in optical and electrophysiological recording technologies have made it possible
to record the dynamics of thousands of neurons, opening up new possibilities for interpreting …

Continuous deep learning architectures have recently re-emerged as Neural Ordinary
Differential Equations (Neural ODEs). This infinite-depth approach theoretically bridges the …

Neural Ordinary Differential Equations (NODEs), a framework of continuous-depth neural
networks, have been widely applied, showing exceptional efficacy in coping with some …

We introduce SubFlow-a dynamic adaptation and execution strategy for a deep neural
network (DNN), which enables real-time DNN inference and training. The goal of SubFlow is …

Dynamic neural network is an emerging research topic in deep learning. Compared to static
models which have fixed computational graphs and parameters at the inference stage …

Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships
between time-varying inputs and outputs with complex temporal dependencies. Recently …

Continuous-valued deep convolutional networks (DNNs) can be converted into accurate
rate-coding based spike neural networks (SNNs). However, the substantial computational …

Neural networks in the brain are dominated by sometimes more than 60% feedback
connections, which most often have small synaptic weights. Different from this, little is known …