A comprehensive survey of continual learning: theory, method and application

L Wang, X Zhang, H Su, J Zhu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality

L Wang, J Xie, X Zhang, M Huang… - Advances in Neural …, 2024 - proceedings.neurips.cc
Prompt-based continual learning is an emerging direction in leveraging pre-trained
knowledge for downstream continual learning, and has almost reached the performance …

Ranpac: Random projections and pre-trained models for continual learning

MD McDonnell, D Gong, A Parvaneh… - Advances in …, 2024 - proceedings.neurips.cc
Continual learning (CL) aims to incrementally learn different tasks (such as classification) in
a non-stationary data stream without forgetting old ones. Most CL works focus on tackling …

Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning

D Goswami, Y Liu, B Twardowski… - Advances in Neural …, 2024 - proceedings.neurips.cc
Exemplar-free class-incremental learning (CIL) poses several challenges since it prohibits
the rehearsal of data from previous tasks and thus suffers from catastrophic forgetting …

Online continual learning without the storage constraint

A Prabhu, Z Cai, P Dokania, P Torr, V Koltun… - arXiv preprint arXiv …, 2023 - arxiv.org
Traditional online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout an agent's …

Continual learning: Applications and the road forward

E Verwimp, S Ben-David, M Bethge, A Cossu… - arXiv preprint arXiv …, 2023 - arxiv.org
Continual learning is a sub-field of machine learning, which aims to allow machine learning
models to continuously learn on new data, by accumulating knowledge without forgetting …

Continually learning representations at scale

A Galashov, J Mitrovic, D Tirumala… - Conference on …, 2023 - proceedings.mlr.press
Many widely used continual learning benchmarks follow a protocol that starts from an
untrained, randomly initialized model that needs to sequentially learn a number of incoming …

Semantically-Shifted Incremental Adapter-Tuning is A Continual ViTransformer

Y Tan, Q Zhou, X Xiang, K Wang… - Proceedings of the …, 2024 - openaccess.thecvf.com
Class-incremental learning (CIL) aims to enable models to continuously learn new classes
while overcoming catastrophic forgetting. The introduction of pre-trained models has brought …

Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers

D Goswami, B Twardowski… - Proceedings of the …, 2024 - openaccess.thecvf.com
Few-shot class-incremental learning (FSCIL) aims to adapt the model to new classes from
very few data (5 samples) without forgetting the previously learned classes. Recent works in …

HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning

L Wang, J Xie, X Zhang, H Su, J Zhu - arXiv preprint arXiv:2407.05229, 2024 - arxiv.org
The deployment of pre-trained models (PTMs) has greatly advanced the field of continual
learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgetting …