Dynamic neural networks: A survey

Y Han, G Huang, S Song, L Yang… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Dynamic neural network is an emerging research topic in deep learning. Compared to static
models which have fixed computational graphs and parameters at the inference stage …

Ghostnet: More features from cheap operations

K Han, Y Wang, Q Tian, J Guo… - Proceedings of the …, 2020 - openaccess.thecvf.com
Deploying convolutional neural networks (CNNs) on embedded devices is difficult due to the
limited memory and computation resources. The redundancy in feature maps is an important …

GhostNets on heterogeneous devices via cheap operations

K Han, Y Wang, C Xu, J Guo, C Xu, E Wu… - International Journal of …, 2022 - Springer
Deploying convolutional neural networks (CNNs) on mobile devices is difficult due to the
limited memory and computation resources. We aim to design efficient neural networks for …

Manifold regularized dynamic network pruning

Y Tang, Y Wang, Y Xu, Y Deng, C Xu… - Proceedings of the …, 2021 - openaccess.thecvf.com
Neural network pruning is an essential approach for reducing the computational complexity
of deep models so that they can be well deployed on resource-limited devices. Compared …

Adapting Neural Networks at Runtime: Current Trends in At-Runtime Optimizations for Deep Learning

M Sponner, B Waschneck, A Kumar - ACM Computing Surveys, 2024 - dl.acm.org
Adaptive optimization methods for deep learning adjust the inference task to the current
circumstances at runtime to improve the resource footprint while maintaining the model's …

Dynamic neural network structure: A review for its theories and applications

J Guo, CLP Chen, Z Liu, X Yang - IEEE Transactions on Neural …, 2024 - ieeexplore.ieee.org
The dynamic neural network (DNN), in contrast to the static counterpart, offers numerous
advantages, such as improved accuracy, efficiency, and interpretability. These benefits stem …

Towards performance-maximizing neural network pruning via global channel attention

Y Wang, S Guo, J Guo, J Zhang, W Zhang, C Yan… - Neural Networks, 2024 - Elsevier
Network pruning has attracted increasing attention recently for its capability of transferring
large-scale neural networks (eg, CNNs) into resource-constrained devices. Such a transfer …

Kernel based progressive distillation for adder neural networks

Y Xu, C Xu, X Chen, W Zhang… - Advances in Neural …, 2020 - proceedings.neurips.cc
Abstract Adder Neural Networks (ANNs) which only contain additions bring us a new way of
developing deep neural networks with low energy consumption. Unfortunately, there is an …

Prior gradient mask guided pruning-aware fine-tuning

L Cai, Z An, C Yang, Y Yan, Y Xu - … of the AAAI Conference on Artificial …, 2022 - ojs.aaai.org
Abstract We proposed a Prior Gradient Mask Guided Pruning-aware Fine-Tuning (PGMPF)
framework to accelerate deep Convolutional Neural Networks (CNNs). In detail, the …

[HTML][HTML] Zero time waste in pre-trained early exit neural networks

B Wójcik, M Przewiȩźlikowski, F Szatkowski… - Neural Networks, 2023 - Elsevier
The problem of reducing processing time of large deep learning models is a fundamental
challenge in many real-world applications. Early exit methods strive towards this goal by …