Successful integration of deep neural networks (DNNs) or deep learning (DL) has resulted
in breakthroughs in many areas. However, deploying these highly accurate models for data …

Object detection is crucial in aerial imagery analysis. Previous methods based on
convolutional neural networks (CNNs) require large-scale labeled datasets for training to …

Abstract Knowledge distillation aims to compress a powerful yet cumbersome teacher model
into a lightweight student model without much sacrifice of performance. For this purpose …

Transformers, which are popular for language modeling, have been explored for solving
vision tasks recently, eg, the Vision Transformer (ViT) for image classification. The ViT model …

Recently, Vision Transformers (ViTs) have been broadly explored in visual recognition. With
low efficiency in encoding fine-level features, the performance of ViTs is still inferior to the …

Mining precise class-aware attention maps, aka, class activation maps, is essential for
weakly supervised semantic segmentation. In this paper, we present L2G, a simple online …

Recently proposed knowledge distillation approaches based on feature-map transfer
validate that intermediate layers of a teacher model can serve as effective targets for training …

Abstract Knowledge distillation (KD) has been proven a simple and effective tool for training
compact dense prediction models. Lightweight student networks are trained by extra …

In this paper, we present token labeling---a new training objective for training high-
performance vision transformers (ViTs). Different from the standard training objective of ViTs …

In recent years, knowledge distillation has been proved to be an effective solution for model
compression. This approach can make lightweight student models acquire the knowledge …