Parameter-efficient fine-tuning for large models: A comprehensive survey

Z Han, C Gao, J Liu, SQ Zhang - arXiv preprint arXiv:2403.14608, 2024 - arxiv.org
Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models

J Liu, R Gong, M Zhang, Y He, J Cai… - arXiv preprint arXiv …, 2024 - arxiv.org
The typical process for developing LLMs involves pre-training a general foundation model
on massive data, followed by fine-tuning on task-specific data to create specialized experts …

Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs

A Panda, B Isik, X Qi, S Koyejo, T Weissman… - arXiv preprint arXiv …, 2024 - arxiv.org
Existing methods for adapting large language models (LLMs) to new tasks are not suited to
multi-task adaptation because they modify all the model weights--causing destructive …

Gradient-Mask Tuning Elevates the Upper Limits of LLM Performance

H Li, X Zhang, X Liu, Y Gong, Y Wang, Y Yang… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) have revolutionized lots of fields of research. Although it is
well-known that fine-tuning is essential for enhancing the capabilities of LLMs, existing …

S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs

W Zhong, M Bharadwaj - arXiv preprint arXiv:2405.20314, 2024 - arxiv.org
Speculative decoding (SD) has attracted a significant amount of research attention due to
the substantial speedup it can achieve for LLM inference. However, despite the high …

Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuning

M Méloux, C Cerisara - arXiv preprint arXiv:2408.17070, 2024 - arxiv.org
Teaching new information to pre-trained large language models (PLM) is a crucial but
challenging task. Model adaptation techniques, such as fine-tuning and parameter-efficient …

Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models

B Ping, S Wang, H Wang, X Han, Y Xu, Y Yan… - arXiv preprint arXiv …, 2024 - arxiv.org
Fine-tuning is a crucial process for adapting large language models (LLMs) to diverse
applications. In certain scenarios, such as multi-tenant serving, deploying multiple LLMs …

Learning the language of biomolecular interactions

S Sledzieski - 2024 - dspace.mit.edu
Proteins are the primary functional unit of the cell, and their interactions drive cellular
function. Interactions between proteins are responsible for a wide variety of functions raning …