Attention augmented convolutional networks

I Bello, B Zoph, A Vaswani… - Proceedings of the …, 2019 - openaccess.thecvf.com
Convolutional networks have enjoyed much success in many computer vision applications.
The convolution operation however has a significant weakness in that it only operates on a …

Re-parameterizing your optimizers rather than architectures

X Ding, H Chen, X Zhang, K Huang, J Han… - arXiv preprint arXiv …, 2022 - arxiv.org
The well-designed structures in neural networks reflect the prior knowledge incorporated
into the models. However, though different models have various priors, we are used to …

Scalable neural architecture search for 3d medical image segmentation

S Kim, I Kim, S Lim, W Baek, C Kim, H Cho… - … Image Computing and …, 2019 - Springer
In this paper, a neural architecture search (NAS) framework is proposed for 3D medical
image segmentation, to automatically optimize a neural architecture from a large design …

Evolutionary optimization of hyperparameters in deep learning models

JY Kim, SB Cho - 2019 ieee congress on evolutionary …, 2019 - ieeexplore.ieee.org
Recently, deep learning is one of the most popular techniques in artificial intelligence.
However, to construct a deep learning model, various components must be set up, including …

Sedona: Search for decoupled neural networks toward greedy block-wise learning

M Pyeon, J Moon, T Hahn, G Kim - International Conference on …, 2020 - openreview.net
Backward locking and update locking are well-known sources of inefficiency in
backpropagation that prevent from concurrently updating layers. Several works have …

Software and application patterns for explanation methods

M Alber - Explainable AI: interpreting, explaining and visualizing …, 2019 - Springer
Deep neural networks successfully pervaded many applications domains and are
increasingly used in critical decision processes. Understanding their workings is desirable …

Optimizing neural networks through activation function discovery and automatic weight initialization

G Bingham - arXiv preprint arXiv:2304.03374, 2023 - arxiv.org
Automated machine learning (AutoML) methods improve upon existing models by
optimizing various aspects of their design. While present methods focus on hyperparameters …

A two-step rule for backpropagation

A Boughammoura - International Journal of Informatics and Applied …, 2023 - dergipark.org.tr
We present a simplified computational rule for the back-propagation formulas for artificial
neural networks. In this work, we provide a generic two-step rule for the back-propagation …

How to iNNvestigate neural networks' predictions!

M Alber, S Lapuschkin, P Seegerer, M Hägele… - 2018 - openreview.net
In recent years, deep neural networks have revolutionized many application domains of
machine learning and are key components of many critical decision or predictive processes …

Equipment identification through image recognition

D Saidnassimov - 2022 - aaltodoc.aalto.fi
Object detection is a rapidly-evolving field with applications varying from medicine to self-
driving vehicles. As the performance of the deep learning algorithms grow exponentially …