Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …

With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large
language models (LLMs), especially as the models' scale and the diversity of tasks increase …

With the increasing size of pre-trained language models (PLMs), fine-tuning all the
parameters in the model is not efficient, especially when there are a large number of …

Large Language Models (LLMs) represent a class of deep learning models adept at
understanding natural language and generating coherent text in response to prompts or …

We propose a neural language modeling system based on low-rank adaptation (LoRA) for
speech recognition output rescoring. Although pretrained language models (LMs) like BERT …

As the parameters of LLMs expand, the computational cost of fine-tuning the entire model
becomes prohibitive. To address this challenge, we introduce a PEFT method, Principal …

For long document summarization, discourse structure is important to discern the key
content of the text and the differences in importance level between sentences. Unfortunately …

Federated Instruction Tuning (FIT) has shown the ability to achieve collaborative model
instruction tuning among massive data owners without sharing private data. However, it still …

Fine-tuning provides an effective means to specialize pre-trained models for various
downstream tasks. However, fine-tuning often incurs high memory overhead, especially for …