Peft: State-of-the-art parameter-efficient fine-tuning methods

[PDF] arxiv.org

Challenges and applications of large language models

J Kaddour, J Harris, M Mozes, H Bradley… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) went from non-existent to ubiquitous in the machine
learning discourse within a few years. Due to the fast pace of the field, it is difficult to identify …
被引用次数：393 相关文章 所有 3 个版本
[PDF] arxiv.org

Parameter-efficient fine-tuning for large models: A comprehensive survey

Z Han, C Gao, J Liu, J Zhang, SQ Zhang - arXiv preprint arXiv:2403.14608, 2024 - arxiv.org
Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …
被引用次数：140 相关文章 所有 2 个版本
[PDF] arxiv.org

A survey of large language models

WX Zhao, K Zhou, J Li, T Tang, X Wang, Y Hou… - arXiv preprint arXiv …, 2023 - arxiv.org
Language is essentially a complex, intricate system of human expressions governed by
grammatical rules. It poses a significant challenge to develop capable AI algorithms for …
被引用次数：2827 相关文章 所有 4 个版本
[PDF] arxiv.org

Llama-adapter: Efficient fine-tuning of language models with zero-init attention

R Zhang, J Han, C Liu, P Gao, A Zhou, X Hu… - arXiv preprint arXiv …, 2023 - arxiv.org
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA
into an instruction-following model. Using 52K self-instruct demonstrations, LLaMA-Adapter …
被引用次数：642 相关文章 所有 3 个版本
[PDF] arxiv.org

Llama-adapter v2: Parameter-efficient visual instruction model

P Gao, J Han, R Zhang, Z Lin, S Geng, A Zhou… - arXiv preprint arXiv …, 2023 - arxiv.org
How to efficiently transform large language models (LLMs) into instruction followers is
recently a popular research direction, while training LLM for multi-modal reasoning remains …
被引用次数：481 相关文章 所有 3 个版本
[PDF] arxiv.org

Aligning large language models with human: A survey

Y Wang, W Zhong, L Li, F Mi, X Zeng, W Huang… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) trained on extensive textual corpora have emerged as
leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite …
被引用次数：241 相关文章 所有 2 个版本
[PDF] arxiv.org

Fundamental limitations of alignment in large language models

Y Wolf, N Wies, O Avnery, Y Levine… - arXiv preprint arXiv …, 2023 - arxiv.org
An important aspect in developing language models that interact with humans is aligning
their behavior to be useful and unharmful for their human users. This is usually achieved by …
被引用次数：168 相关文章 所有 4 个版本
[PDF] arxiv.org

MedAlpaca--an open-source collection of medical conversational AI models and training data

T Han, LC Adams, JM Papaioannou… - arXiv preprint arXiv …, 2023 - arxiv.org
As large language models (LLMs) like OpenAI's GPT series continue to make strides, we
witness the emergence of artificial intelligence applications in an ever-expanding range of …
被引用次数：219 相关文章 所有 2 个版本
[PDF] arxiv.org

Qa-lora: Quantization-aware low-rank adaptation of large language models

Y Xu, L Xie, X Gu, X Chen, H Chang, H Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
Recently years have witnessed a rapid development of large language models (LLMs).
Despite the strong ability in many language-understanding tasks, the heavy computational …
被引用次数：93 相关文章 所有 3 个版本
[PDF] arxiv.org

Vera: Vector-based random matrix adaptation

DJ Kopiczko, T Blankevoort, YM Asano - arXiv preprint arXiv:2310.11454, 2023 - arxiv.org
Low-rank adapation (LoRA) is a popular method that reduces the number of trainable
parameters when finetuning large language models, but still faces acute storage challenges …
被引用次数：89 相关文章 所有 3 个版本