A survey on hardware accelerators and optimization techniques for RNNs

S Mittal, S Umesh - Journal of Systems Architecture, 2021 - Elsevier
Abstract “Recurrent neural networks”(RNNs) are powerful artificial intelligence models that
have shown remarkable effectiveness in several tasks such as music generation, speech …

ELSA: Hardware-software co-design for efficient, lightweight self-attention mechanism in neural networks

TJ Ham, Y Lee, SH Seo, S Kim, H Choi… - 2021 ACM/IEEE 48th …, 2021 - ieeexplore.ieee.org
The self-attention mechanism is rapidly emerging as one of the most important key primitives
in neural networks (NNs) for its ability to identify the relations within input entities. The self …

Manna: An accelerator for memory-augmented neural networks

JR Stevens, A Ranjan, D Das, B Kaul… - Proceedings of the …, 2019 - dl.acm.org
Memory-augmented neural networks (MANNs)--which augment a traditional Deep Neural
Network (DNN) with an external, differentiable memory--are emerging as a promising …

RAMANN: in-SRAM differentiable memory computations for memory-augmented neural networks

M Ali, A Agrawal, K Roy - Proceedings of the ACM/IEEE International …, 2020 - dl.acm.org
Memory-Augmented Neural Networks (MANNs) have been shown to outperform Recurrent
Neural Networks (RNNs) in terms of long-term dependencies. Since MANNs are equipped …

Memory-augmented neural networks on FPGA for real-time and energy-efficient question answering

S Park, J Jang, S Kim, B Na… - IEEE Transactions on Very …, 2020 - ieeexplore.ieee.org
Memory-augmented neural networks (MANNs) were introduced to handle long-term
dependent data efficiently. MANNs have shown promising results in question answering …

Scalable smartphone cluster for deep learning

B Na, J Jang, S Park, S Kim, J Kim, MS Jeong… - arXiv preprint arXiv …, 2021 - arxiv.org
Various deep learning applications on smartphones have been rapidly rising, but training
deep neural networks (DNNs) has too large computational burden to be executed on a …

Развитие аппаратно-ориентированных нейронных сетей на FPGA и ASIC

СП Шипицин, МИ Ямаев - Вестник Пермского национального …, 2019 - cyberleninka.ru
Приводится обзор реализаций нейронных сетей на программируемых логических
интегральных схемах (ПЛИС) типа FPGA (Field Programmable Gate Array) и на …

A2P-MANN: Adaptive Attention Inference Hops Pruned Memory-Augmented Neural Networks

M Ahmadzadeh, M Kamal… - … on Neural Networks …, 2022 - ieeexplore.ieee.org
In this work, to limit the number of required attention inference hops in memory-augmented
neural networks, we propose an online adaptive approach called-memory-augmented …

Hardware neural networks progress on FPGA and ASIC

SP Shipitsin, MI Iamaev - PNRPU Bulletin. Electrotechnics …, 2019 - ered.pstu.ru
The article provides a survey about the implementation of neural networks on
Programmable Logic Device (PLDs) such as FPGA (Field Programmable Gate Array) and …

Accelerating Emerging Neural Workloads

JR Stevens - 2021 - search.proquest.com
Due to a combination of algorithmic advances, wide-spread availability of rich data sets, and
tremendous growth in compute availability, Deep Neural Networks (DNNs) have seen …