Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks
Deep neural models, in recent years, have been successful in almost every field, even
solving the most complex problem statements. However, these models are huge in size with …
solving the most complex problem statements. However, these models are huge in size with …
Factorizing knowledge in neural networks
In this paper, we explore a novel and ambitious knowledge-transfer task, termed Knowledge
Factorization (KF). The core idea of KF lies in the modularization and assemblability of …
Factorization (KF). The core idea of KF lies in the modularization and assemblability of …
Diffusion probabilistic model made slim
Despite the visually-pleasing results achieved, the massive computational cost has been a
long-standing flaw for diffusion probabilistic models (DPMs), which, in turn, greatly limits …
long-standing flaw for diffusion probabilistic models (DPMs), which, in turn, greatly limits …
Knowledge distillation: A survey
In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …
especially for computer vision tasks. The great success of deep learning is mainly due to its …
Curriculum temperature for knowledge distillation
Most existing distillation methods ignore the flexible role of the temperature in the loss
function and fix it as a hyper-parameter that can be decided by an inefficient grid search. In …
function and fix it as a hyper-parameter that can be decided by an inefficient grid search. In …
WaveNet: Wavelet network with knowledge distillation for RGB-T salient object detection
In recent years, various neural network architectures for computer vision have been devised,
such as the visual transformer and multilayer perceptron (MLP). A transformer based on an …
such as the visual transformer and multilayer perceptron (MLP). A transformer based on an …
Distilling knowledge from graph convolutional networks
Existing knowledge distillation methods focus on convolutional neural networks (CNNs),
where the input samples like images lie in a grid domain, and have largely overlooked …
where the input samples like images lie in a grid domain, and have largely overlooked …
Online knowledge distillation for efficient pose estimation
Existing state-of-the-art human pose estimation methods require heavy computational
resources for accurate predictions. One promising technique to obtain an accurate yet …
resources for accurate predictions. One promising technique to obtain an accurate yet …
Cloth-changing person re-identification from a single image with gait prediction and regularization
Cloth-Changing person re-identification (CC-ReID) aims at matching the same person
across different locations over a long-duration, eg, over days, and therefore inevitably has …
across different locations over a long-duration, eg, over days, and therefore inevitably has …