Pre-trained language models for text generation: A survey

J Li, T Tang, WX Zhao, JY Nie, JR Wen - ACM Computing Surveys, 2024 - dl.acm.org
Text Generation aims to produce plausible and readable text in human language from input
data. The resurgence of deep learning has greatly advanced this field, in particular, with the …

Recent advances in deep learning based dialogue systems: A systematic survey

J Ni, T Young, V Pandelea, F Xue… - Artificial intelligence review, 2023 - Springer
Dialogue systems are a popular natural language processing (NLP) task as it is promising in
real-life applications. It is also a complicated task since many NLP tasks deserving study are …

Retrieval augmentation reduces hallucination in conversation

K Shuster, S Poff, M Chen, D Kiela, J Weston - arXiv preprint arXiv …, 2021 - arxiv.org
Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue
models often suffer from factual incorrectness and hallucination of knowledge (Roller et al …

Internet-augmented dialogue generation

M Komeili, K Shuster, J Weston - arXiv preprint arXiv:2107.07566, 2021 - arxiv.org
The largest store of continually updating knowledge on our planet can be accessed via
internet search. In this work we study giving access to this information to conversational …

A survey of knowledge enhanced pre-trained language models

L Hu, Z Liu, Z Zhao, L Hou, L Nie… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Pre-trained Language Models (PLMs) which are trained on large text corpus via self-
supervised learning method, have yielded promising performance on various tasks in …

Language models are few-shot multilingual learners

GI Winata, A Madotto, Z Lin, R Liu, J Yosinski… - arXiv preprint arXiv …, 2021 - arxiv.org
General-purpose language models have demonstrated impressive capabilities, performing
on par with state-of-the-art approaches on a range of downstream natural language …

FaithDial: A Faithful Benchmark for Information-Seeking Dialogue

N Dziri, E Kamalloo, S Milton, O Zaiane… - Transactions of the …, 2022 - direct.mit.edu
The goal of information-seeking dialogue is to respond to seeker queries with natural
language utterances that are grounded on knowledge sources. However, dialogue systems …

Contrastive learning reduces hallucination in conversations

W Sun, Z Shi, S Gao, P Ren, M de Rijke… - Proceedings of the AAAI …, 2023 - ojs.aaai.org
Pre-trained language models (LMs) store knowledge in their parameters and can generate
informative responses when used in conversational systems. However, LMs suffer from the …

Large language models are strong zero-shot retriever

T Shen, G Long, X Geng, C Tao, T Zhou… - arXiv preprint arXiv …, 2023 - arxiv.org
In this work, we propose a simple method that applies a large language model (LLM) to
large-scale retrieval in zero-shot scenarios. Our method, the Language language model as …

Summarization as indirect supervision for relation extraction

K Lu, I Hsu, W Zhou, MD Ma, M Chen - arXiv preprint arXiv:2205.09837, 2022 - arxiv.org
Relation extraction (RE) models have been challenged by their reliance on training data
with expensive annotations. Considering that summarization tasks aim at acquiring concise …