Abstract Low-Rank Adaptation (LoRA), which updates the dense neural network layers with
pluggable low-rank matrices, is one of the best performed parameter efficient fine-tuning …

Dealing with multi-task trade-offs during inference can be addressed via Pareto Front
Learning (PFL) methods that parameterize the Pareto Front with a single model, contrary to …

The goal of this paper is to improve (upcycle) an existing large language model without the
prohibitive requirements of continued pre-training of the full-model. The idea is to split the …

Large language models (LLMs) have shown impressive capabilities across various tasks.
However, training LLMs from scratch requires significant computational power and extensive …

Training large AI models such as deep learning recommendation systems and foundation
language (or multi-modal) models costs massive GPUs and computing time. The high …

We introduce a precision polarization scheme for DNN inference that utilizes only very low
and very high precision levels, assigning low precision to the majority of network weights …

Abstract Information propagation is the process through which data are transmitted within a
system. The growth of large-scale web datasets has led to explosive growth in information …