The field of deep learning has witnessed significant progress, particularly in computer vision
(CV), natural language processing (NLP), and speech. The use of large-scale models …

This paper introduces an efficient fine-tuning method for large pre-trained models, offering
strong in-distribution (ID) and out-of-distribution (OOD) performance. Breaking away from …

Model merging is an efficient empowerment technique in the machine learning community
that does not require the collection of raw training data and does not require expensive …

Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

Fine-tuning pre-trained models for downstream tasks has led to a proliferation of open-
sourced task-specific models. Recently, Model Merging (MM) has emerged as an effective …

The field of deep learning has witnessed significant progress in recent times, particularly in
areas such as computer vision (CV), natural language processing (NLP), and speech. The …

Pre-training followed by fine-tuning is widely adopted among practitioners. The performance
can be improved by “model soups”[46] via exploring various hyperparameter configurations …

The loss landscape of neural networks is a valuable perspective for studying the trainability,
generalization, and robustness of networks, and hence its visualization has been …

Weight averaging of Stochastic Gradient Descent (SGD) iterates is a popular method for
training deep learning models. While it is often used as part of complex training pipelines to …

With the growing adoption of deep learning models in different real-world domains,
including computational biology, it is often necessary to understand which data features are …