Continual learning of large language models: A comprehensive survey

H Shi, Z Xu, H Wang, W Qin, W Wang, Y Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
The recent success of large language models (LLMs) trained on static, pre-collected,
general datasets has sparked numerous research directions and applications. One such …

Recent advances of foundation language models-based continual learning: a survey

Y Yang, J Zhou, X Ding, T Huai, S Liu, Q Chen… - ACM Computing …, 2024 - dl.acm.org
Recently, foundation language models (LMs) have marked significant achievements in the
domains of natural language processing (NLP) and computer vision (CV). Unlike traditional …

Adaptive Rank, Reduced Forgetting: Knowledge Retention in Continual Learning Vision-Language Models with Dynamic Rank-Selective LoRA

H Lu, C Zhao, J Xue, L Yao, K Moore… - arXiv preprint arXiv …, 2024 - arxiv.org
We investigate whether the pre-trained knowledge of vision-language models (VLMs), such
as CLIP, can be retained or even enhanced during continual learning (CL) while absorbing …

Recent Advances of Multimodal Continual Learning: A Comprehensive Survey

D Yu, X Zhang, Y Chen, A Liu, Y Zhang, PS Yu… - arXiv preprint arXiv …, 2024 - arxiv.org
Continual learning (CL) aims to empower machine learning models to learn continually from
new data, while building upon previously acquired knowledge without forgetting. As …

Less is More: Efficient Model Merging with Binary Task Switch

B Qi, F Li, Z Wang, J Gao, D Li, P Ye, B Zhou - arXiv preprint arXiv …, 2024 - arxiv.org
As an effective approach to equip models with multi-task capabilities without additional
training, model merging has garnered significant attention. However, existing methods face …

One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering

D Das, D Talon, M Mancini, Y Wang, E Ricci - arXiv preprint arXiv …, 2024 - arxiv.org
Vision-Language Models (VLMs) have shown significant promise in Visual Question
Answering (VQA) tasks by leveraging web-scale multimodal datasets. However, these …

Mining your own secrets: Diffusion classifier scores for continual personalization of text-to-image diffusion models

S Jha, S Yang, M Ishii, M Zhao, C Simon… - arXiv preprint arXiv …, 2024 - arxiv.org
Personalized text-to-image diffusion models have grown popular for their ability to efficiently
acquire a new concept from user-defined text descriptions and a few images. However, in …

THE CASE FOR COGNITIVE-DISSONANCE AWARE CONTINUAL UPDATE OF KNOWLEDGE IN LLMS

IN PRAISE - openreview.net
Despite remarkable capabilities, large language models (LLMs) struggle to continually
update their knowledge without catastrophic forgetting. In contrast, humans effortlessly …