Parameter-efficient fine-tuning for large models: A comprehensive survey
Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …
enabling remarkable achievements across various tasks. However, their unprecedented …
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
The typical process for developing LLMs involves pre-training a general foundation model
on massive data, followed by fine-tuning on task-specific data to create specialized experts …
on massive data, followed by fine-tuning on task-specific data to create specialized experts …
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs
Existing methods for adapting large language models (LLMs) to new tasks are not suited to
multi-task adaptation because they modify all the model weights--causing destructive …
multi-task adaptation because they modify all the model weights--causing destructive …
Gradient-Mask Tuning Elevates the Upper Limits of LLM Performance
Large language models (LLMs) have revolutionized lots of fields of research. Although it is
well-known that fine-tuning is essential for enhancing the capabilities of LLMs, existing …
well-known that fine-tuning is essential for enhancing the capabilities of LLMs, existing …
S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs
W Zhong, M Bharadwaj - arXiv preprint arXiv:2405.20314, 2024 - arxiv.org
Speculative decoding (SD) has attracted a significant amount of research attention due to
the substantial speedup it can achieve for LLM inference. However, despite the high …
the substantial speedup it can achieve for LLM inference. However, despite the high …
Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuning
M Méloux, C Cerisara - arXiv preprint arXiv:2408.17070, 2024 - arxiv.org
Teaching new information to pre-trained large language models (PLM) is a crucial but
challenging task. Model adaptation techniques, such as fine-tuning and parameter-efficient …
challenging task. Model adaptation techniques, such as fine-tuning and parameter-efficient …
Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
Fine-tuning is a crucial process for adapting large language models (LLMs) to diverse
applications. In certain scenarios, such as multi-tenant serving, deploying multiple LLMs …
applications. In certain scenarios, such as multi-tenant serving, deploying multiple LLMs …
Learning the language of biomolecular interactions
S Sledzieski - 2024 - dspace.mit.edu
Proteins are the primary functional unit of the cell, and their interactions drive cellular
function. Interactions between proteins are responsible for a wide variety of functions raning …
function. Interactions between proteins are responsible for a wide variety of functions raning …