We present a novel application of evolutionary algorithms to automate the creation of
powerful foundation models. While model merging has emerged as a promising approach …

The growing number of parameter-efficient adaptations of a base large language model
(LLM) calls for studying whether we can reuse such trained adapters to improve …

We give extensive empirical evidence against the common belief that variational learning is
ineffective for large neural networks. We show that an optimizer called Improved Variational …

Model merging is an efficient empowerment technique in the machine learning community
that does not require the collection of raw training data and does not require expensive …

Multimodal foundation models serve numerous applications at the intersection of vision and
language. Still, despite being pretrained on extensive data, they become outdated over time …

The scarcity of well-annotated medical datasets requires leveraging transfer learning from
broader datasets like ImageNet or pre-trained models like CLIP. Model soups averages …

As machine learning becomes more prominent there is a growing demand to perform
several inference tasks in parallel. Running a dedicated model for each task is …

Existing methods for adapting large language models (LLMs) to new tasks are not suited to
multi-task adaptation because they modify all the model weights--causing destructive …

Supervised fine-tuning (SFT) on instruction-following corpus is a crucial approach toward
the alignment of large language models (LLMs). However, the performance of LLMs on …

Model merging, which combines multiple models into a single model, has gained increasing
popularity in recent years. By efficiently integrating the capabilities of various models without …