Sparsity in continuous-depth neural networks

H Aliee, T Richter, M Solonin, I Ibarra… - Advances in …, 2022 - proceedings.neurips.cc
Abstract Neural Ordinary Differential Equations (NODEs) have proven successful in learning
dynamical systems in terms of accurately recovering the observed trajectories. While …

Coupled oscillatory recurrent neural network (cornn): An accurate and (gradient) stable architecture for learning long time dependencies

TK Rusch, S Mishra - arXiv preprint arXiv:2010.00951, 2020 - arxiv.org
Circuits of biological neurons, such as in the functional parts of the brain can be modeled as
networks of coupled oscillators. Inspired by the ability of these systems to express a rich set …

Cavs: An efficient runtime system for dynamic neural networks

S Xu, H Zhang, G Neubig, W Dai, JK Kim… - 2018 USENIX Annual …, 2018 - usenix.org
Recent deep learning (DL) models are moving more and more to dynamic neural network
(NN) architectures, where the NN structure changes for every data sample. However …

Efficient processing of deep neural networks: A tutorial and survey

V Sze, YH Chen, TJ Yang, JS Emer - Proceedings of the IEEE, 2017 - ieeexplore.ieee.org
Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI)
applications including computer vision, speech recognition, and robotics. While DNNs …

Factorized dynamic fully-connected layers for neural networks

F Babiloni, T Tanay, J Deng… - Proceedings of the …, 2023 - openaccess.thecvf.com
The design of neural network layers plays a crucial role in determining the efficiency and
performance of various computer vision tasks. However, most existing layers compromise …

[HTML][HTML] Encoding time in neural dynamic regimes with distinct computational tradeoffs

S Zhou, SC Masmanidis… - PLOS Computational …, 2022 - journals.plos.org
Converging evidence suggests the brain encodes time in dynamic patterns of neural activity,
including neural sequences, ramping activity, and complex dynamics. Most temporal tasks …

Deep CovDenseSNN: A hierarchical event-driven dynamic framework with spiking neurons in noisy environment

Q Xu, J Peng, J Shen, H Tang, G Pan - Neural Networks, 2020 - Elsevier
Neurons in the brain use an event signal, termed spike, encode temporal information for
neural computation. Spiking neural networks (SNNs) take this advantage to serve as …

Apnet: Approximation-aware real-time neural network

S Bateni, C Liu - 2018 IEEE Real-Time Systems Symposium …, 2018 - ieeexplore.ieee.org
Modern embedded cyber-physical systems are becoming entangled with the realm of deep
neural networks (DNNs) towards increased autonomy. While applying DNNs can …

Towards robust neural networks via close-loop control

Z Chen, Q Li, Z Zhang - arXiv preprint arXiv:2102.01862, 2021 - arxiv.org
Despite their success in massive engineering applications, deep neural networks are
vulnerable to various perturbations due to their black-box nature. Recent study has shown …

Fast-slow recurrent neural networks

A Mujika, F Meier, A Steger - Advances in Neural …, 2017 - proceedings.neurips.cc
Processing sequential data of variable length is a major challenge in a wide range of
applications, such as speech recognition, language modeling, generative image modeling …