Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment

L Xu, H Xie, SZJ Qin, X Tao, FL Wang - arXiv preprint arXiv:2312.12148, 2023 - arxiv.org
With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

End-edge-cloud collaborative computing for deep learning: A comprehensive survey

Y Wang, C Yang, S Lan, L Zhu… - … Surveys & Tutorials, 2024 - ieeexplore.ieee.org
The booming development of deep learning applications and services heavily relies on
large deep learning models and massive data in the cloud. However, cloud-based deep …

Ma-sam: Modality-agnostic sam adaptation for 3d medical image segmentation

C Chen, J Miao, D Wu, A Zhong, Z Yan, S Kim… - Medical Image …, 2024 - Elsevier
Abstract The Segment Anything Model (SAM), a foundation model for general image
segmentation, has demonstrated impressive zero-shot performance across numerous …

Adapting language models to compress contexts

A Chevalier, A Wettig, A Ajith, D Chen - arXiv preprint arXiv:2305.14788, 2023 - arxiv.org
Transformer-based language models (LMs) are powerful and widely-applicable tools, but
their usefulness is constrained by a finite context window and the expensive computational …

Biomedgpt: A unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks

K Zhang, J Yu, E Adhikarla, R Zhou, Z Yan… - arXiv e …, 2023 - ui.adsabs.harvard.edu
Conventional task-and modality-specific artificial intelligence (AI) models are inflexible in
real-world deployment and maintenance for biomedicine. At the same time, the growing …

Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms

C Chang, WC Peng, TF Chen - arXiv preprint arXiv:2308.08469, 2023 - arxiv.org
In this work, we leverage pre-trained Large Language Models (LLMs) to enhance time-
series forecasting. Mirroring the growing interest in unifying models for Natural Language …

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

S Jain, R Kirk, ES Lubana, RP Dick, H Tanaka… - arXiv preprint arXiv …, 2023 - arxiv.org
Fine-tuning large pre-trained models has become the de facto strategy for developing both
task-specific and general-purpose machine learning systems, including developing models …

A survey on mixture of experts

W Cai, J Jiang, F Wang, J Tang, S Kim… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) have garnered unprecedented advancements across
diverse fields, ranging from natural language processing to computer vision and beyond …

Adapters: A unified library for parameter-efficient and modular transfer learning

C Poth, H Sterz, I Paul, S Purkayastha… - arXiv preprint arXiv …, 2023 - arxiv.org
We introduce Adapters, an open-source library that unifies parameter-efficient and modular
transfer learning in large language models. By integrating 10 diverse adapter methods into a …

Llara: Aligning large language models with sequential recommenders

J Liao, S Li, Z Yang, J Wu, Y Yuan, X Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
Sequential recommendation aims to predict the subsequent items matching user preference
based on her/his historical interactions. With the development of Large Language Models …