A review on the attention mechanism of deep learning

Z Niu, G Zhong, H Yu - Neurocomputing, 2021 - Elsevier
Attention has arguably become one of the most important concepts in the deep learning
field. It is inspired by the biological systems of humans that tend to focus on the distinctive …

Attention mechanism in neural networks: where it comes and where it goes

D Soydaner - Neural Computing and Applications, 2022 - Springer
A long time ago in the machine learning literature, the idea of incorporating a mechanism
inspired by the human visual system into neural networks was introduced. This idea is …

Visual attention network

MH Guo, CZ Lu, ZN Liu, MM Cheng, SM Hu - Computational Visual Media, 2023 - Springer
While originally designed for natural language processing tasks, the self-attention
mechanism has recently taken various computer vision areas by storm. However, the 2D …

Top-down neural attention by excitation backprop

J Zhang, SA Bargal, Z Lin, J Brandt, X Shen… - International Journal of …, 2018 - Springer
We aim to model the top-down attention of a convolutional neural network (CNN) classifier
for generating task-specific attention maps. Inspired by a top-down human visual attention …

Weakly supervised instance segmentation using class peak response

Y Zhou, Y Zhu, Q Ye, Q Qiu… - Proceedings of the IEEE …, 2018 - openaccess.thecvf.com
Weakly supervised instance segmentation with image-level labels, instead of expensive
pixel-level masks, remains unexplored. In this paper, we tackle this challenging problem by …

40 years of cognitive architectures: core cognitive abilities and practical applications

I Kotseruba, JK Tsotsos - Artificial Intelligence Review, 2020 - Springer
In this paper we present a broad overview of the last 40 years of research on cognitive
architectures. To date, the number of existing architectures has reached several hundred …

Large language models (LLMs): survey, technical frameworks, and future challenges

P Kumar - Artificial Intelligence Review, 2024 - Springer
Artificial intelligence (AI) has significantly impacted various fields. Large language models
(LLMs) like GPT-4, BARD, PaLM, Megatron-Turing NLG, Jurassic-1 Jumbo etc., have …

Bilateral attention network for RGB-D salient object detection

Z Zhang, Z Lin, J Xu, WD Jin, SP Lu… - IEEE transactions on …, 2021 - ieeexplore.ieee.org
RGB-D salient object detection (SOD) aims to segment the most attractive objects in a pair of
cross-modal RGB and depth images. Currently, most existing RGB-D SOD methods focus on …

State-of-the-art in visual attention modeling

A Borji, L Itti - IEEE transactions on pattern analysis and …, 2012 - ieeexplore.ieee.org
Modeling visual attention-particularly stimulus-driven, saliency-based attention-has been a
very active research area over the past 25 years. Many different models of attention are now …

Inferring salient objects from human fixations

W Wang, J Shen, X Dong, A Borji… - IEEE transactions on …, 2019 - ieeexplore.ieee.org
Previous research in visual saliency has been focused on two major types of models namely
fixation prediction and salient object detection. The relationship between the two, however …