Transformers as algorithms: Generalization and stability in in-context learning
In-context learning (ICL) is a type of prompting where a transformer model operates on a
sequence of (input, output) examples and performs inference on-the-fly. In this work, we …
sequence of (input, output) examples and performs inference on-the-fly. In this work, we …
Theory on forgetting and generalization of continual learning
Continual learning (CL), which aims to learn a sequence of tasks, has attracted significant
recent attention. However, most work has focused on the experimental performance of CL …
recent attention. However, most work has focused on the experimental performance of CL …
The ideal continual learner: An agent that never forgets
The goal of continual learning is to find a model that solves multiple learning tasks which are
presented sequentially to the learner. A key challenge in this setting is that the learner may" …
presented sequentially to the learner. A key challenge in this setting is that the learner may" …
Provable pathways: Learning multiple tasks over multiple paths
Constructing useful representations across a large number of tasks is a key requirement for
sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is building …
sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is building …
A Statistical Theory of Regularization-Based Continual Learning
We provide a statistical analysis of regularization-based continual learning on a sequence of
linear regression tasks, with emphasis on how different regularization terms affect the model …
linear regression tasks, with emphasis on how different regularization terms affect the model …
Hicu: Leveraging hierarchy for curriculum learning in automated icd coding
There are several opportunities for automation in healthcare that can improve clinician
throughput. One such example is assistive tools to document diagnosis codes when …
throughput. One such example is assistive tools to document diagnosis codes when …
Benchmarking sensitivity of continual graph learning for skeleton-based action recognition
Continual learning (CL) is the research field that aims to build machine learning models that
can accumulate knowledge continuously over different tasks without retraining from scratch …
can accumulate knowledge continuously over different tasks without retraining from scratch …
Self-supervised Activity Representation Learning with Incremental Data: An Empirical Study
In the context of mobile sensing environments, various sensors on mobile devices
continually generate a vast amount of data. Analyzing this ever-increasing data presents …
continually generate a vast amount of data. Analyzing this ever-increasing data presents …