A comprehensive overview of large language models

H Naveed, AU Khan, S Qiu, M Saqib, S Anwar… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) have recently demonstrated remarkable capabilities in
natural language processing tasks and beyond. This success of LLMs has led to a large …

Retrieval-augmented generation for large language models: A survey

Y Gao, Y Xiong, X Gao, K Jia, J Pan, Y Bi, Y Dai… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs) demonstrate powerful capabilities, but they still face
challenges in practical applications, such as hallucinations, slow knowledge updates, and …

Check your facts and try again: Improving large language models with external knowledge and automated feedback

B Peng, M Galley, P He, H Cheng, Y Xie, Y Hu… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent
responses for many downstream tasks, eg, task-oriented dialog and question answering …

Lamda: Language models for dialog applications

R Thoppilan, D De Freitas, J Hall, N Shazeer… - arXiv preprint arXiv …, 2022 - arxiv.org
We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of
Transformer-based neural language models specialized for dialog, which have up to 137B …

Survey of hallucination in natural language generation

Z Ji, N Lee, R Frieske, T Yu, D Su, Y Xu, E Ishii… - ACM Computing …, 2023 - dl.acm.org
Natural Language Generation (NLG) has improved exponentially in recent years thanks to
the development of sequence-to-sequence deep learning technologies such as Transformer …

Atlas: Few-shot learning with retrieval augmented language models

G Izacard, P Lewis, M Lomeli, L Hosseini… - Journal of Machine …, 2023 - jmlr.org
Large language models have shown impressive few-shot results on a wide range of tasks.
However, when knowledge is key for such results, as is the case for tasks such as question …

Generate rather than retrieve: Large language models are strong context generators

W Yu, D Iter, S Wang, Y Xu, M Ju, S Sanyal… - arXiv preprint arXiv …, 2022 - arxiv.org
Knowledge-intensive tasks, such as open-domain question answering (QA), require access
to a large amount of world or domain knowledge. A common approach for knowledge …

A survey of controllable text generation using transformer-based pre-trained language models

H Zhang, H Song, S Li, M Zhou, D Song - ACM Computing Surveys, 2023 - dl.acm.org
Controllable Text Generation (CTG) is an emerging area in the field of natural language
generation (NLG). It is regarded as crucial for the development of advanced text generation …

One embedder, any task: Instruction-finetuned text embeddings

H Su, W Shi, J Kasai, Y Wang, Y Hu… - arXiv preprint arXiv …, 2022 - arxiv.org
We introduce INSTRUCTOR, a new method for computing text embeddings given task
instructions: every text input is embedded together with instructions explaining the use case …

Retrieval augmentation reduces hallucination in conversation

K Shuster, S Poff, M Chen, D Kiela, J Weston - arXiv preprint arXiv …, 2021 - arxiv.org
Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue
models often suffer from factual incorrectness and hallucination of knowledge (Roller et al …