A survey of techniques for optimizing transformer inference
Recent years have seen a phenomenal rise in the performance and applications of
transformer neural networks. The family of transformer networks, including Bidirectional …
transformer neural networks. The family of transformer networks, including Bidirectional …
A survey of visual transformers
Transformer, an attention-based encoder–decoder model, has already revolutionized the
field of natural language processing (NLP). Inspired by such significant achievements, some …
field of natural language processing (NLP). Inspired by such significant achievements, some …
Automated deep learning: Neural architecture search is not the end
Deep learning (DL) has proven to be a highly effective approach for developing models in
diverse contexts, including visual perception, speech recognition, and machine translation …
diverse contexts, including visual perception, speech recognition, and machine translation …
Crossformer++: A versatile vision transformer hinging on cross-scale attention
While features of different scales are perceptually important to visual inputs, existing vision
transformers do not yet take advantage of them explicitly. To this end, we first propose a …
transformers do not yet take advantage of them explicitly. To this end, we first propose a …
Neural architecture search: Insights from 1000 papers
In the past decade, advances in deep learning have resulted in breakthroughs in a variety of
areas, including computer vision, natural language understanding, speech recognition, and …
areas, including computer vision, natural language understanding, speech recognition, and …
Neural architecture search for transformers: A survey
Transformer-based Deep Neural Network architectures have gained tremendous interest
due to their effectiveness in various applications across Natural Language Processing (NLP) …
due to their effectiveness in various applications across Natural Language Processing (NLP) …
Auto-prox: Training-free vision transformer architecture search via automatic proxy discovery
The substantial success of Vision Transformer (ViT) in computer vision tasks is largely
attributed to the architecture design. This underscores the necessity of efficient architecture …
attributed to the architecture design. This underscores the necessity of efficient architecture …
Mcuformer: Deploying vision tranformers on microcontrollers with limited memory
Due to the high price and heavy energy consumption of GPUs, deploying deep models on
IoT devices such as microcontrollers makes significant contributions for ecological AI …
IoT devices such as microcontrollers makes significant contributions for ecological AI …
Elasticvit: Conflict-aware supernet training for deploying fast vision transformer on diverse mobile devices
Abstract Neural Architecture Search (NAS) has shown promising performance in the
automatic design of vision transformers (ViT) exceeding 1G FLOPs. However, designing …
automatic design of vision transformers (ViT) exceeding 1G FLOPs. However, designing …
Heatvit: Hardware-efficient adaptive token pruning for vision transformers
While vision transformers (ViTs) have continuously achieved new milestones in the field of
computer vision, their sophisticated network architectures with high computation and …
computer vision, their sophisticated network architectures with high computation and …