Transformer language models have received widespread public attention, yet their
generated text is often surprising even to NLP researchers. In this survey, we discuss over …

Despite their wide adoption, the underlying training and memorization dynamics of very
large language models is not well understood. We empirically study exact memorization in …

Abstract Knowledge plays a critical role in artificial intelligence. Recently, the extensive
success of pre-trained language models (PLMs) has raised significant attention about how …

NLP is currently dominated by general-purpose pretrained language models like RoBERTa,
which achieve strong performance on NLU tasks through pretraining on billions of words …

Experiments with pre-trained models such as BERT are often based on a single checkpoint.
While the conclusions drawn apply to the artifact tested in the experiment (ie, the particular …

Models of language trained on very large corpora have been demonstrated useful for NLP.
As fixed artifacts, they have become the object of intense study, with many researchers" …

Natural language processing for finance has gained significant attention from both
academia and the industry as the continuously increasing amount of financial texts has …

In a world filled with serious challenges like climate change, religious and political conflicts,
global pandemics, terrorism, and racial discrimination, an internet full of hate speech …

Like most natural language understanding and generation tasks, state-of-the-art models for
summarization are transformer-based sequence-to-sequence architectures that are …

Pre-trained Language Models (PLMs) are trained on vast unlabeled data, rich in world
knowledge. This fact has sparked the interest of the community in quantifying the amount of …