A comprehensive survey of continual learning: theory, method and application
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …
Biological underpinnings for lifelong learning machines
D Kudithipudi, M Aguilar-Simon, J Babb… - Nature Machine …, 2022 - nature.com
Biological organisms learn from interactions with their environment throughout their lifetime.
For artificial systems to successfully act and adapt in the real world, it is desirable to similarly …
For artificial systems to successfully act and adapt in the real world, it is desirable to similarly …
Three types of incremental learning
Incrementally learning new information from a non-stationary stream of data, referred to as
'continual learning', is a key feature of natural intelligence, but a challenging problem for …
'continual learning', is a key feature of natural intelligence, but a challenging problem for …
Dualprompt: Complementary prompting for rehearsal-free continual learning
Continual learning aims to enable a single model to learn a sequence of tasks without
catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store …
catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store …
Deep class-incremental learning: A survey
Deep models, eg, CNNs and Vision Transformers, have achieved impressive achievements
in many vision tasks in the closed world. However, novel classes emerge from time to time in …
in many vision tasks in the closed world. However, novel classes emerge from time to time in …
The curse of recursion: Training on generated data makes models forget
Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3 (. 5) and
GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT …
GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT …
Learning to prompt for continual learning
The mainstream paradigm behind continual learning has been to adapt the model
parameters to non-stationary data distributions, where catastrophic forgetting is the central …
parameters to non-stationary data distributions, where catastrophic forgetting is the central …
Dytox: Transformers for continual learning with dynamic token expansion
Deep network architectures struggle to continually learn new tasks without forgetting the
previous tasks. A recent trend indicates that dynamic architectures based on an expansion …
previous tasks. A recent trend indicates that dynamic architectures based on an expansion …
Forward compatible few-shot class-incremental learning
Novel classes frequently arise in our dynamically changing world, eg, new users in the
authentication system, and a machine learning model should recognize new classes without …
authentication system, and a machine learning model should recognize new classes without …
Class-incremental learning by knowledge distillation with adaptive feature consolidation
We present a novel class incremental learning approach based on deep neural networks,
which continually learns new tasks with limited memory for storing examples in the previous …
which continually learns new tasks with limited memory for storing examples in the previous …