[HTML][HTML] From word embeddings to pre-trained language models: A state-of-the-art walkthrough
M Mars - Applied Sciences, 2022 - mdpi.com
With the recent advances in deep learning, different approaches to improving pre-trained
language models (PLMs) have been proposed. PLMs have advanced state-of-the-art …
language models (PLMs) have been proposed. PLMs have advanced state-of-the-art …
Deep transfer learning & beyond: Transformer language models in information systems research
R Gruetzemacher, D Paradice - ACM Computing Surveys (CSUR), 2022 - dl.acm.org
AI is widely thought to be poised to transform business, yet current perceptions of the scope
of this transformation may be myopic. Recent progress in natural language processing …
of this transformation may be myopic. Recent progress in natural language processing …
Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense
The rise in malicious usage of large language models, such as fake content creation and
academic plagiarism, has motivated the development of approaches that identify AI …
academic plagiarism, has motivated the development of approaches that identify AI …
Videoclip: Contrastive pre-training for zero-shot video-text understanding
We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot
video and text understanding, without using any labels on downstream tasks. VideoCLIP …
video and text understanding, without using any labels on downstream tasks. VideoCLIP …
mT5: A massively multilingual pre-trained text-to-text transformer
The recent" Text-to-Text Transfer Transformer"(T5) leveraged a unified text-to-text format and
scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this …
scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this …
Memorizing transformers
Language models typically need to be trained or finetuned in order to acquire new
knowledge, which involves updating their weights. We instead envision language models …
knowledge, which involves updating their weights. We instead envision language models …
Approximate nearest neighbor negative contrastive learning for dense text retrieval
Conducting text retrieval in a dense learned representation space has many intriguing
advantages over sparse retrieval. Yet the effectiveness of dense retrieval (DR) often requires …
advantages over sparse retrieval. Yet the effectiveness of dense retrieval (DR) often requires …
Intrinsic dimensionality explains the effectiveness of language model fine-tuning
Although pretrained language models can be fine-tuned to produce state-of-the-art results
for a very wide range of language understanding tasks, the dynamics of this process are not …
for a very wide range of language understanding tasks, the dynamics of this process are not …
[图书][B] Pretrained transformers for text ranking: Bert and beyond
The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in
response to a query. Although the most common formulation of text ranking is search …
response to a query. Although the most common formulation of text ranking is search …
KILT: a benchmark for knowledge intensive language tasks
Challenging problems such as open-domain question answering, fact checking, slot filling
and entity linking require access to large, external knowledge sources. While some models …
and entity linking require access to large, external knowledge sources. While some models …