A review on language models as knowledge bases
Recently, there has been a surge of interest in the NLP community on the use of pretrained
Language Models (LMs) as Knowledge Bases (KBs). Researchers have shown that LMs …
Language Models (LMs) as Knowledge Bases (KBs). Researchers have shown that LMs …
A survey on automated fact-checking
Fact-checking has become increasingly important due to the speed with which both
information and misinformation can spread in the modern media ecosystem. Therefore …
information and misinformation can spread in the modern media ecosystem. Therefore …
Dense text retrieval based on pretrained language models: A survey
Text retrieval is a long-standing research topic on information seeking, where a system is
required to return relevant information resources to user's queries in natural language. From …
required to return relevant information resources to user's queries in natural language. From …
Autoregressive search engines: Generating substrings as document identifiers
Abstract Knowledge-intensive language tasks require NLP systems to both provide the
correct answer and retrieve supporting evidence for it in a given corpus. Autoregressive …
correct answer and retrieve supporting evidence for it in a given corpus. Autoregressive …
Task-aware retrieval with instructions
We study the problem of retrieval with instructions, where users of a retrieval system
explicitly describe their intent along with their queries. We aim to develop a general-purpose …
explicitly describe their intent along with their queries. We aim to develop a general-purpose …
Re2G: Retrieve, rerank, generate
As demonstrated by GPT-3 and T5, transformers grow in capability as parameter spaces
become larger and larger. However, for tasks that require a large amount of knowledge, non …
become larger and larger. However, for tasks that require a large amount of knowledge, non …
Corpusbrain: Pre-train a generative retrieval model for knowledge-intensive language tasks
Knowledge-intensive language tasks (KILT) usually require a large body of information to
provide correct answers. A popular paradigm to solve this problem is to combine a search …
provide correct answers. A popular paradigm to solve this problem is to combine a search …
Unik-qa: Unified representations of structured and unstructured knowledge for open-domain question answering
We study open-domain question answering with structured, unstructured and semi-
structured knowledge sources, including text, tables, lists and knowledge bases. Departing …
structured knowledge sources, including text, tables, lists and knowledge bases. Departing …
A unified generative retriever for knowledge-intensive language tasks via prompt learning
Knowledge-intensive language tasks (KILTs) benefit from retrieving high-quality relevant
contexts from large external knowledge corpora. Learning task-specific retrievers that return …
contexts from large external knowledge corpora. Learning task-specific retrievers that return …
Continual learning for generative retrieval over dynamic corpora
Generative retrieval (GR) directly predicts the identifiers of relevant documents (ie, docids)
based on a parametric model. It has achieved solid performance on many ad-hoc retrieval …
based on a parametric model. It has achieved solid performance on many ad-hoc retrieval …