Activation functions in deep learning: A comprehensive survey and benchmark

SR Dubey, SK Singh, BB Chaudhuri - Neurocomputing, 2022 - Elsevier
Neural networks have shown tremendous growth in recent years to solve numerous
problems. Various types of neural networks have been introduced to deal with different types …

[HTML][HTML] A state-of-the-art survey on deep learning theory and architectures

MZ Alom, TM Taha, C Yakopcic, S Westberg, P Sidike… - electronics, 2019 - mdpi.com
In recent years, deep learning has garnered tremendous success in a variety of application
domains. This new field of machine learning has been growing rapidly and has been …

[HTML][HTML] Deep learning in food category recognition

Y Zhang, L Deng, H Zhu, W Wang, Z Ren, Q Zhou… - Information …, 2023 - Elsevier
Integrating artificial intelligence with food category recognition has been a field of interest for
research for the past few decades. It is potentially one of the next steps in revolutionizing …

Mish: A self regularized non-monotonic activation function

D Misra - arXiv preprint arXiv:1908.08681, 2019 - arxiv.org
We propose $\textit {Mish} $, a novel self-regularized non-monotonic activation function
which can be mathematically defined as: $ f (x)= x\tanh (softplus (x)) $. As activation …

Activation functions: Comparison of trends in practice and research for deep learning

C Nwankpa, W Ijomah, A Gachagan… - arXiv preprint arXiv …, 2018 - arxiv.org
Deep neural networks have been successfully used in diverse emerging domains to solve
real world complex problems with may more deep learning (DL) architectures, being …

A survey on modern trainable activation functions

A Apicella, F Donnarumma, F Isgrò, R Prevete - Neural Networks, 2021 - Elsevier
In neural networks literature, there is a strong interest in identifying and defining activation
functions which can improve neural network performance. In recent years there has been a …

The history began from alexnet: A comprehensive survey on deep learning approaches

MZ Alom, TM Taha, C Yakopcic, S Westberg… - arXiv preprint arXiv …, 2018 - arxiv.org
Deep learning has demonstrated tremendous success in variety of application domains in
the past few years. This new field of machine learning has been growing rapidly and applied …

Deep learning and transformer approaches for UAV-based wildfire detection and segmentation

R Ghali, MA Akhloufi, WS Mseddi - Sensors, 2022 - mdpi.com
Wildfires are a worldwide natural disaster causing important economic damages and loss of
lives. Experts predict that wildfires will increase in the coming years mainly due to climate …

Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science

DC Mocanu, E Mocanu, P Stone, PH Nguyen… - Nature …, 2018 - nature.com
Through the success of deep learning in various domains, artificial neural networks are
currently among the most used artificial intelligence methods. Taking inspiration from the …

How important are activation functions in regression and classification? A survey, performance comparison, and future directions

AD Jagtap, GE Karniadakis - Journal of Machine Learning for …, 2023 - dl.begellhouse.com
Inspired by biological neurons, the activation functions play an essential part in the learning
process of any artificial neural network (ANN) commonly used in many real-world problems …