Large Language Models (LLMs) have recently demonstrated remarkable capabilities in
natural language processing tasks and beyond. This success of LLMs has led to a large …

The size of vision models has grown exponentially over the last few years, especially after
the emergence of Vision Transformer. This has motivated the development of parameter …

Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range
of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always …

Adapters, a plug-in neural network module with some tunable parameters, have emerged as
a parameter-efficient transfer learning technique for adapting pre-trained models to …

Recommender systems commonly suffer from the long-standing data sparsity problem
where insufficient user-item interaction data limits the systems' ability to make accurate …

Pretrained models are the standard starting point for training. This approach consistently
outperforms the use of a random initialization. However, pretraining is a costly endeavour …

Large pretrained language models are widely used in downstream NLP tasks via task-
specific fine-tuning, but such procedures can be costly. Recently, Parameter-Efficient Fine …

We propose a new paradigm to continually evolve pretrained models, denoted ColD Fusion.
It provides the benefits of multitask learning but leverages distributed computation with …

Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

Recently, finetuning pretrained vision-language models (VLMs) has been a prevailing
paradigm for achieving state-of-the-art performance in VQA. However, as VLMs scale, it …