Large language models (LLMs) are one of the most impressive achievements of artificial
intelligence in recent years. However, their relevance to the study of language more broadly …

Deep learning has recently come to dominate computational linguistics, leading to claims of
human-level performance in a range of language processing tasks. Like much previous …

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved
great success and become a milestone in the field of artificial intelligence (AI). Owing to …

Transformer-based models have pushed state of the art in many areas of NLP, but our
understanding of what is behind their success is still limited. This paper is the first survey of …

Consistency of a model—that is, the invariance of its behavior under meaning-preserving
alternations in its input—is a highly desirable property in natural language processing. In …

Contextualized word representations have become a driving force in NLP, motivating
widespread interest in understanding their capabilities and the mechanisms by which they …

In recent years, transformer-based language models have achieved state of the art
performance in various NLP benchmarks. These models are able to extract mostly …

A characteristic feature of human semantic cognition is its ability to not only store and
retrieve the properties of concepts observed through experience, but to also facilitate the …

Recent work has raised concerns about the inherent limitations of text-only pretraining. In
this paper, we first demonstrate that reporting bias, the tendency of people to not state the …

Conceptualization, or viewing entities and situations as instances of abstract concepts in
mind and making inferences based on that, is a vital component in human intelligence for …