CLIP has become a promising language-supervised visual pre-training framework and
achieves excellent performance over a wide range of tasks. This paper aims to distill small …

Existing studies on knowledge distillation typically focus on teacher-centered methods, in
which the teacher network is trained according to its own standards before transferring the …

Deep transfer learning has been extensively developed in the remaining useful life
prediction of rolling bearings because it can decrease the dependence on massive labeled …

Abstract Contrastive Language-Image Pre-training (CLIP) has become a promising
language-supervised visual pre-training framework. This paper aims to distill small CLIP …

Inspired by the masked language modeling (MLM) in natural language processing tasks, the
masked image modeling (MIM) has been recognized as a strong self-supervised pre …

Voice anti-spoofing is an important step for secure speaker verification in voice-enabled
Internet of Things (IoT) systems. Most voice spoofing detection methods require significant …

New knowledge originates from the old. The various types of elements, deposited in the
training history, are a large amount of wealth for improving learning deep models. In this …

Abstract Knowledge distillation is one of the most persuasive approaches to model
compression that transfers the representational expertise from large deep-learning teacher …

Underwater object detection in the field of computer vision faces unique challenges such as
color distortion, reduced visibility, and blurred edges caused by aquatic conditions, which …

Graph anomaly detection (GAD) has gained increasing attention in various attribute graph
applications, ie, social communication and financial fraud transaction networks. Recently …