Activation functions in deep learning: A comprehensive survey and benchmark

SR Dubey, SK Singh, BB Chaudhuri - Neurocomputing, 2022 - Elsevier
Neural networks have shown tremendous growth in recent years to solve numerous
problems. Various types of neural networks have been introduced to deal with different types …

Review on deep learning applications in frequency analysis and control of modern power system

Y Zhang, X Shi, H Zhang, Y Cao, V Terzija - International Journal of …, 2022 - Elsevier
The penetration of renewable energy resources (RES) generation and the interconnection of
regional power grids in wide area and large scale have led the modern power system to …

Multimodal learning with transformers: A survey

P Xu, X Zhu, DA Clifton - IEEE Transactions on Pattern Analysis …, 2023 - ieeexplore.ieee.org
Transformer is a promising neural network learner, and has achieved great success in
various machine learning tasks. Thanks to the recent prevalence of multimodal applications …

Slim-neck by GSConv: A better design paradigm of detector architectures for autonomous vehicles

H Li, J Li, H Wei, Z Liu, Z Zhan, Q Ren - arXiv preprint arXiv:2206.02424, 2022 - arxiv.org
Object detection is a significant downstream task in computer vision. For the on-board edge
computing platforms, a giant model is difficult to achieve the real-time detection requirement …

Strongsort: Make deepsort great again

Y Du, Z Zhao, Y Song, Y Zhao, F Su… - IEEE Transactions …, 2023 - ieeexplore.ieee.org
Recently, Multi-Object Tracking (MOT) has attracted rising attention, and accordingly,
remarkable progresses have been achieved. However, the existing methods tend to use …

DeePMD-kit v2: A software package for deep potential models

J Zeng, D Zhang, D Lu, P Mo, Z Li, Y Chen… - The Journal of …, 2023 - pubs.aip.org
DeePMD-kit is a powerful open-source software package that facilitates molecular dynamics
simulations using machine learning potentials known as Deep Potential (DP) models. This …

Resmlp: Feedforward networks for image classification with data-efficient training

H Touvron, P Bojanowski, M Caron… - IEEE transactions on …, 2022 - ieeexplore.ieee.org
We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image
classification. It is a simple residual network that alternates (i) a linear layer in which image …

Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks

T Hoefler, D Alistarh, T Ben-Nun, N Dryden… - Journal of Machine …, 2021 - jmlr.org
The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

Polyp-pvt: Polyp segmentation with pyramid vision transformers

B Dong, W Wang, DP Fan, J Li, H Fu, L Shao - arXiv preprint arXiv …, 2021 - arxiv.org
Most polyp segmentation methods use CNNs as their backbone, leading to two key issues
when exchanging information between the encoder and decoder: 1) taking into account the …

Molecular contrastive learning of representations via graph neural networks

Y Wang, J Wang, Z Cao… - Nature Machine …, 2022 - nature.com
Molecular machine learning bears promise for efficient molecular property prediction and
drug discovery. However, labelled molecule data can be expensive and time consuming to …