Transformer language models have received widespread public attention, yet their
generated text is often surprising even to NLP researchers. In this survey, we discuss over …

This paper reexamines the research on out-of-distribution (OOD) robustness in the field of
NLP. We find that the distribution shift settings in previous studies commonly lack adequate …

The widespread adoption of large language models (LLMs) makes it important to recognize
their strengths and limitations. We argue that in order to develop a holistic understanding of …

Pre-trained language models (PLMs) are known to improve the generalization performance
of natural language understanding models by leveraging large amounts of data during the …

Recent advances of powerful Language Models have allowed Natural Language
Generation (NLG) to emerge as an important technology that can not only perform traditional …

Multilingual large-scale Pretrained Language Models (PLMs) have been shown to store
considerable amounts of factual knowledge, but large variations are observed across …

Finding the best way of adapting pre-trained language models to a task is a big challenge in
current NLP. Just like the previous generation of task-tuned models (TT), models that are …

Language Models (LMs) often must integrate facts they memorized in pretraining with new
information that appears in a given context. These two sources can disagree, causing …

Abstract Machine learning (ML) systems in natural language processing (NLP) face
significant challenges in generalizing to out-of-distribution (OOD) data, where the test …

While large language models (LLMs) have made considerable advancements in
understanding and generating unstructured text, their application in structured data remains …