Transformer-based pretrained language models (T-PTLMs) have achieved great success in
almost every NLP task. The evolution of these models started with GPT and BERT. These …

Transfer learning is a technique utilized in deep learning applications to transmit learned
inference to a different target domain. The approach is mainly to solve the problem of a few …

Despite their wide adoption, the underlying training and memorization dynamics of very
large language models is not well understood. We empirically study exact memorization in …

Experiments with pre-trained models such as BERT are often based on a single checkpoint.
While the conclusions drawn apply to the artifact tested in the experiment (ie, the particular …

Given the prevalence of pre-trained contextualized representations in today's NLP, there
have been many efforts to understand what information they contain, and why they seem to …

State-of-the-art Entity Matching (EM) approaches rely on transformer architectures, such as
BERT, for generating highly contextualized embeddings of terms. The embeddings are then …

The trustworthiness of machine learning has emerged as a critical topic in the field,
encompassing various applications and research areas such as robustness, security …

Recent research considers few-shot intent detection as a meta-learning problem: the model
is learning to learn from a consecutive set of small tasks named episodes. In this work, we …

In this paper, we move towards combining large parametric models with non-parametric
prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework …

Transformers are the dominant architecture in NLP, but their training and fine-tuning is still
very challenging. In this paper, we present the design and implementation of a visual …