Davit: Dual attention vision transformers
In this work, we introduce Dual Attention Vision Transformers (DaViT), a simple yet effective
vision transformer architecture that is able to capture global context while maintaining …
vision transformer architecture that is able to capture global context while maintaining …
Pale transformer: A general vision transformer backbone with pale-shaped attention
Recently, Transformers have shown promising performance in various vision tasks. To
reduce the quadratic computation complexity caused by the global self-attention, various …
reduce the quadratic computation complexity caused by the global self-attention, various …
Slide-transformer: Hierarchical vision transformer with local self-attention
Self-attention mechanism has been a key factor in the recent progress of Vision Transformer
(ViT), which enables adaptive feature extraction from global contexts. However, existing self …
(ViT), which enables adaptive feature extraction from global contexts. However, existing self …
Swin transformer: Hierarchical vision transformer using shifted windows
This paper presents a new vision Transformer, called Swin Transformer, that capably serves
as a general-purpose backbone for computer vision. Challenges in adapting Transformer …
as a general-purpose backbone for computer vision. Challenges in adapting Transformer …
Focal attention for long-range interactions in vision transformers
Abstract Recently, Vision Transformer and its variants have shown great promise on various
computer vision tasks. The ability to capture local and global visual dependencies through …
computer vision tasks. The ability to capture local and global visual dependencies through …
Flatten transformer: Vision transformer using focused linear attention
The quadratic computation complexity of self-attention has been a persistent challenge
when applying Transformer models to vision tasks. Linear attention, on the other hand, offers …
when applying Transformer models to vision tasks. Linear attention, on the other hand, offers …
Cswin transformer: A general vision transformer backbone with cross-shaped windows
Abstract We present CSWin Transformer, an efficient and effective Transformer-based
backbone for general-purpose vision tasks. A challenging issue in Transformer design is …
backbone for general-purpose vision tasks. A challenging issue in Transformer design is …
Regionvit: Regional-to-local attention for vision transformers
Vision transformer (ViT) has recently shown its strong capability in achieving comparable
results to convolutional neural networks (CNNs) on image classification. However, vanilla …
results to convolutional neural networks (CNNs) on image classification. However, vanilla …
Vision transformer with deformable attention
Transformers have recently shown superior performances on various vision tasks. The large,
sometimes even global, receptive field endows Transformer models with higher …
sometimes even global, receptive field endows Transformer models with higher …
KVT: k-NN Attention for Boosting Vision Transformers
Abstract Convolutional Neural Networks (CNNs) have dominated computer vision for years,
due to its ability in capturing locality and translation invariance. Recently, many vision …
due to its ability in capturing locality and translation invariance. Recently, many vision …