Text embeddings by weakly-supervised contrastive pre-training
This paper presents E5, a family of state-of-the-art text embeddings that transfer well to a
wide range of tasks. The model is trained in a contrastive manner with weak supervision …
wide range of tasks. The model is trained in a contrastive manner with weak supervision …
Dense text retrieval based on pretrained language models: A survey
Text retrieval is a long-standing research topic on information seeking, where a system is
required to return relevant information resources to user's queries in natural language. From …
required to return relevant information resources to user's queries in natural language. From …
RetroMAE: Pre-training retrieval-oriented language models via masked auto-encoder
Despite pre-training's progress in many important NLP tasks, it remains to explore effective
pre-training strategies for dense retrieval. In this paper, we propose RetroMAE, a new …
pre-training strategies for dense retrieval. In this paper, we propose RetroMAE, a new …
An efficiency study for SPLADE models
C Lassance, S Clinchant - Proceedings of the 45th International ACM …, 2022 - dl.acm.org
Latency and efficiency issues are often overlooked when evaluating IR models based on
Pretrained Language Models (PLMs) in reason of multiple hardware and software testing …
Pretrained Language Models (PLMs) in reason of multiple hardware and software testing …
Coco-dr: Combating distribution shifts in zero-shot dense retrieval with contrastive and distributionally robust learning
We present a new zero-shot dense retrieval (ZeroDR) method, COCO-DR, to improve the
generalization ability of dense retrieval by combating the distribution shifts between source …
generalization ability of dense retrieval by combating the distribution shifts between source …
Simans: Simple ambiguous negatives sampling for dense text retrieval
Sampling proper negatives from a large document pool is vital to effectively train a dense
retrieval model. However, existing negative sampling strategies suffer from the uninformative …
retrieval model. However, existing negative sampling strategies suffer from the uninformative …
A thorough examination on zero-shot dense retrieval
Recent years have witnessed the significant advance in dense retrieval (DR) based on
powerful pre-trained language models (PLM). DR models have achieved excellent …
powerful pre-trained language models (PLM). DR models have achieved excellent …
Led: Lexicon-enlightened dense retriever for large-scale retrieval
Retrieval models based on dense representations in semantic space have become an
indispensable branch for first-stage retrieval. These retrievers benefit from surging advances …
indispensable branch for first-stage retrieval. These retrievers benefit from surging advances …
Master: Multi-task pre-trained bottlenecked masked autoencoders are better dense retrievers
Pre-trained Transformers (eg, BERT) have been commonly used in existing dense retrieval
methods for parameter initialization, and recent studies are exploring more effective pre …
methods for parameter initialization, and recent studies are exploring more effective pre …
Mixed-modality representation learning and pre-training for joint table-and-text retrieval in openqa
Retrieving evidences from tabular and textual resources is essential for open-domain
question answering (OpenQA), which provides more comprehensive information. However …
question answering (OpenQA), which provides more comprehensive information. However …