Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks
L Wang, KJ Yoon - IEEE transactions on pattern analysis and …, 2021 - ieeexplore.ieee.org
Deep neural models, in recent years, have been successful in almost every field, even
solving the most complex problem statements. However, these models are huge in size with …
solving the most complex problem statements. However, these models are huge in size with …
Towards continual reinforcement learning: A review and perspectives
In this article, we aim to provide a literature review of different formulations and approaches
to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We …
to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We …
Knowledge distillation: A survey
In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …
especially for computer vision tasks. The great success of deep learning is mainly due to its …
Meta-learning in neural networks: A survey
The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent
years. Contrary to conventional approaches to AI where tasks are solved from scratch using …
years. Contrary to conventional approaches to AI where tasks are solved from scratch using …
Meta-learning with warped gradient descent
Learning an efficient update rule from data that promotes rapid learning of new tasks from
the same distribution remains an open problem in meta-learning. Typically, previous works …
the same distribution remains an open problem in meta-learning. Typically, previous works …
Hierarchically structured meta-learning
In order to learn quickly with few samples, meta-learning utilizes prior knowledge learned
from previous tasks. However, a critical challenge in meta-learning is task uncertainty and …
from previous tasks. However, a critical challenge in meta-learning is task uncertainty and …
Meta-learning of neural architectures for few-shot learning
The recent progress in neural architecture search (NAS) has allowed scaling the automated
design of neural architectures to real-world domains, such as object detection and semantic …
design of neural architectures to real-world domains, such as object detection and semantic …
Empirical bayes transductive meta-learning with synthetic gradients
We propose a meta-learning approach that learns from multiple tasks in a transductive
setting, by leveraging the unlabeled query set in addition to the support set to generate a …
setting, by leveraging the unlabeled query set in addition to the support set to generate a …
Explaining knowledge distillation by quantifying the knowledge
This paper presents a method to interpret the success of knowledge distillation by
quantifying and analyzing task-relevant and task-irrelevant visual concepts that are encoded …
quantifying and analyzing task-relevant and task-irrelevant visual concepts that are encoded …
Bootstrapped meta-learning
Meta-learning empowers artificial intelligence to increase its efficiency by learning how to
learn. Unlocking this potential involves overcoming a challenging meta-optimisation …
learn. Unlocking this potential involves overcoming a challenging meta-optimisation …