A review on language models as knowledge bases

B AlKhamissi, M Li, A Celikyilmaz, M Diab… - arXiv preprint arXiv …, 2022 - arxiv.org
Recently, there has been a surge of interest in the NLP community on the use of pretrained
Language Models (LMs) as Knowledge Bases (KBs). Researchers have shown that LMs …

A survey on automated fact-checking

Z Guo, M Schlichtkrull, A Vlachos - Transactions of the Association for …, 2022 - direct.mit.edu
Fact-checking has become increasingly important due to the speed with which both
information and misinformation can spread in the modern media ecosystem. Therefore …

Dense text retrieval based on pretrained language models: A survey

WX Zhao, J Liu, R Ren, JR Wen - ACM Transactions on Information …, 2024 - dl.acm.org
Text retrieval is a long-standing research topic on information seeking, where a system is
required to return relevant information resources to user's queries in natural language. From …

Autoregressive search engines: Generating substrings as document identifiers

M Bevilacqua, G Ottaviano, P Lewis… - Advances in …, 2022 - proceedings.neurips.cc
Abstract Knowledge-intensive language tasks require NLP systems to both provide the
correct answer and retrieve supporting evidence for it in a given corpus. Autoregressive …

Task-aware retrieval with instructions

A Asai, T Schick, P Lewis, X Chen, G Izacard… - arXiv preprint arXiv …, 2022 - arxiv.org
We study the problem of retrieval with instructions, where users of a retrieval system
explicitly describe their intent along with their queries. We aim to develop a general-purpose …

Re2G: Retrieve, rerank, generate

M Glass, G Rossiello, MFM Chowdhury… - arXiv preprint arXiv …, 2022 - arxiv.org
As demonstrated by GPT-3 and T5, transformers grow in capability as parameter spaces
become larger and larger. However, for tasks that require a large amount of knowledge, non …

Corpusbrain: Pre-train a generative retrieval model for knowledge-intensive language tasks

J Chen, R Zhang, J Guo, Y Liu, Y Fan… - Proceedings of the 31st …, 2022 - dl.acm.org
Knowledge-intensive language tasks (KILT) usually require a large body of information to
provide correct answers. A popular paradigm to solve this problem is to combine a search …

Unik-qa: Unified representations of structured and unstructured knowledge for open-domain question answering

B Oguz, X Chen, V Karpukhin, S Peshterliev… - arXiv preprint arXiv …, 2020 - arxiv.org
We study open-domain question answering with structured, unstructured and semi-
structured knowledge sources, including text, tables, lists and knowledge bases. Departing …

A unified generative retriever for knowledge-intensive language tasks via prompt learning

J Chen, R Zhang, J Guo, M de Rijke, Y Liu… - Proceedings of the 46th …, 2023 - dl.acm.org
Knowledge-intensive language tasks (KILTs) benefit from retrieving high-quality relevant
contexts from large external knowledge corpora. Learning task-specific retrievers that return …

Continual learning for generative retrieval over dynamic corpora

J Chen, R Zhang, J Guo, M de Rijke, W Chen… - Proceedings of the …, 2023 - dl.acm.org
Generative retrieval (GR) directly predicts the identifiers of relevant documents (ie, docids)
based on a parametric model. It has achieved solid performance on many ad-hoc retrieval …