A survey of techniques for optimizing transformer inference
Recent years have seen a phenomenal rise in the performance and applications of
transformer neural networks. The family of transformer networks, including Bidirectional …
transformer neural networks. The family of transformer networks, including Bidirectional …
On efficient training of large-scale deep learning models: A literature review
The field of deep learning has witnessed significant progress, particularly in computer vision
(CV), natural language processing (NLP), and speech. The use of large-scale models …
(CV), natural language processing (NLP), and speech. The use of large-scale models …
Fact: Ffn-attention co-optimized transformer architecture with eager correlation prediction
Transformer model is becoming prevalent in various AI applications with its outstanding
performance. However, the high cost of computation and memory footprint make its …
performance. However, the high cost of computation and memory footprint make its …
A survey on efficient inference for large language models
Large Language Models (LLMs) have attracted extensive attention due to their remarkable
performance across various tasks. However, the substantial computational and memory …
performance across various tasks. However, the substantial computational and memory …
ShiftAddViT: Mixture of multiplication primitives towards efficient vision transformer
Abstract Vision Transformers (ViTs) have shown impressive performance and have become
a unified backbone for multiple vision tasks. However, both the attention mechanism and …
a unified backbone for multiple vision tasks. However, both the attention mechanism and …
Flightllm: Efficient large language model inference with a complete mapping flow on fpgas
Transformer-based Large Language Models (LLMs) have made a significant impact on
various domains. However, LLMs' efficiency suffers from both heavy computation and …
various domains. However, LLMs' efficiency suffers from both heavy computation and …
Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloads
Running multiple deep neural networks (DNNs) in parallel has become an emerging
workload in both edge devices, such as mobile phones where multiple tasks serve a single …
workload in both edge devices, such as mobile phones where multiple tasks serve a single …
ETTE: Efficient tensor-train-based computing engine for deep neural networks
Tensor-train (TT) decomposition enables ultra-high compression ratio, making the deep
neural network (DNN) accelerators based on this method very attractive. TIE, the state-of-the …
neural network (DNN) accelerators based on this method very attractive. TIE, the state-of-the …
Taskfusion: An efficient transfer learning architecture with dual delta sparsity for multi-task natural language processing
The combination of pre-trained models and task-specific fine-tuning schemes, such as
BERT, has achieved great success in various natural language processing (NLP) tasks …
BERT, has achieved great success in various natural language processing (NLP) tasks …
MELTing point: Mobile Evaluation of Language Transformers
Transformers have revolutionized the machine learning landscape, gradually making their
way into everyday tasks and equipping our computers with``sparks of intelligence'' …
way into everyday tasks and equipping our computers with``sparks of intelligence'' …