Transformers as algorithms: Generalization and stability in in-context learning

Y Li, ME Ildiz, D Papailiopoulos… - … on Machine Learning, 2023 - proceedings.mlr.press
In-context learning (ICL) is a type of prompting where a transformer model operates on a
sequence of (input, output) examples and performs inference on-the-fly. In this work, we …

Theory on forgetting and generalization of continual learning

S Lin, P Ju, Y Liang, N Shroff - International Conference on …, 2023 - proceedings.mlr.press
Continual learning (CL), which aims to learn a sequence of tasks, has attracted significant
recent attention. However, most work has focused on the experimental performance of CL …

The ideal continual learner: An agent that never forgets

L Peng, P Giampouras, R Vidal - … Conference on Machine …, 2023 - proceedings.mlr.press
The goal of continual learning is to find a model that solves multiple learning tasks which are
presented sequentially to the learner. A key challenge in this setting is that the learner may" …

Provable pathways: Learning multiple tasks over multiple paths

Y Li, S Oymak - Proceedings of the AAAI Conference on Artificial …, 2023 - ojs.aaai.org
Constructing useful representations across a large number of tasks is a key requirement for
sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is building …

A Statistical Theory of Regularization-Based Continual Learning

X Zhao, H Wang, W Huang, W Lin - arXiv preprint arXiv:2406.06213, 2024 - arxiv.org
We provide a statistical analysis of regularization-based continual learning on a sequence of
linear regression tasks, with emphasis on how different regularization terms affect the model …

Hicu: Leveraging hierarchy for curriculum learning in automated icd coding

W Ren, R Zeng, T Wu, T Zhu… - Machine Learning for …, 2022 - proceedings.mlr.press
There are several opportunities for automation in healthcare that can improve clinician
throughput. One such example is assistive tools to document diagnosis codes when …

Benchmarking sensitivity of continual graph learning for skeleton-based action recognition

W Wei, T De Schepper, K Mets - arXiv preprint arXiv:2401.18054, 2024 - arxiv.org
Continual learning (CL) is the research field that aims to build machine learning models that
can accumulate knowledge continuously over different tasks without retraining from scratch …

Self-supervised Activity Representation Learning with Incremental Data: An Empirical Study

J Liu, S Deldari, H Xue, V Nguyen… - 2023 24th IEEE …, 2023 - ieeexplore.ieee.org
In the context of mobile sensing environments, various sensors on mobile devices
continually generate a vast amount of data. Analyzing this ever-increasing data presents …