A comprehensive survey of continual learning: theory, method and application

L Wang, X Zhang, H Su, J Zhu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

Biological underpinnings for lifelong learning machines

D Kudithipudi, M Aguilar-Simon, J Babb… - Nature Machine …, 2022 - nature.com
Biological organisms learn from interactions with their environment throughout their lifetime.
For artificial systems to successfully act and adapt in the real world, it is desirable to similarly …

Three types of incremental learning

GM Van de Ven, T Tuytelaars, AS Tolias - Nature Machine Intelligence, 2022 - nature.com
Incrementally learning new information from a non-stationary stream of data, referred to as
'continual learning', is a key feature of natural intelligence, but a challenging problem for …

Dualprompt: Complementary prompting for rehearsal-free continual learning

Z Wang, Z Zhang, S Ebrahimi, R Sun, H Zhang… - … on Computer Vision, 2022 - Springer
Continual learning aims to enable a single model to learn a sequence of tasks without
catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store …

Deep class-incremental learning: A survey

DW Zhou, QW Wang, ZH Qi, HJ Ye, DC Zhan… - arXiv preprint arXiv …, 2023 - arxiv.org
Deep models, eg, CNNs and Vision Transformers, have achieved impressive achievements
in many vision tasks in the closed world. However, novel classes emerge from time to time in …

The curse of recursion: Training on generated data makes models forget

I Shumailov, Z Shumaylov, Y Zhao, Y Gal… - arXiv preprint arXiv …, 2023 - arxiv.org
Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3 (. 5) and
GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT …

Learning to prompt for continual learning

Z Wang, Z Zhang, CY Lee, H Zhang… - Proceedings of the …, 2022 - openaccess.thecvf.com
The mainstream paradigm behind continual learning has been to adapt the model
parameters to non-stationary data distributions, where catastrophic forgetting is the central …

Dytox: Transformers for continual learning with dynamic token expansion

A Douillard, A Ramé, G Couairon… - Proceedings of the …, 2022 - openaccess.thecvf.com
Deep network architectures struggle to continually learn new tasks without forgetting the
previous tasks. A recent trend indicates that dynamic architectures based on an expansion …

Forward compatible few-shot class-incremental learning

DW Zhou, FY Wang, HJ Ye, L Ma… - Proceedings of the …, 2022 - openaccess.thecvf.com
Novel classes frequently arise in our dynamically changing world, eg, new users in the
authentication system, and a machine learning model should recognize new classes without …

Class-incremental learning by knowledge distillation with adaptive feature consolidation

M Kang, J Park, B Han - … of the IEEE/CVF conference on …, 2022 - openaccess.thecvf.com
We present a novel class incremental learning approach based on deep neural networks,
which continually learns new tasks with limited memory for storing examples in the previous …