All bark and no bite: Rogue dimensions in transformer language models obscure representational quality
W Timkey, M Van Schijndel - arXiv preprint arXiv:2109.04404, 2021 - arxiv.org
Similarity measures are a vital tool for understanding how language models represent and
process language. Standard representational similarity measures such as cosine similarity …
process language. Standard representational similarity measures such as cosine similarity …
Time masking for temporal language models
Our world is constantly evolving, and so is the content on the web. Consequently, our
languages, often said to mirror the world, are dynamic in nature. However, most current …
languages, often said to mirror the world, are dynamic in nature. However, most current …
Temporal attention for language models
GD Rosin, K Radinsky - arXiv preprint arXiv:2202.02093, 2022 - arxiv.org
Pretrained language models based on the transformer architecture have shown great
success in NLP. Textual training data often comes from the web and is thus tagged with time …
success in NLP. Textual training data often comes from the web and is thus tagged with time …
Simple, interpretable and stable method for detecting words with usage change across corpora
The problem of comparing two bodies of text and searching for words that differ in their
usage between them arises often in digital humanities and computational social science …
usage between them arises often in digital humanities and computational social science …
Scalable and interpretable semantic change detection
S Montariol, M Martinc… - Proceedings of the 2021 …, 2021 - aclanthology.org
Several cluster-based methods for semantic change detection with contextual embeddings
emerged recently. They allow a fine-grained analysis of word use change by aggregating …
emerged recently. They allow a fine-grained analysis of word use change by aggregating …
A survey on contextualised semantic shift detection
S Montanelli, F Periti - arXiv preprint arXiv:2304.01666, 2023 - arxiv.org
Semantic Shift Detection (SSD) is the task of identifying, interpreting, and assessing the
possible change over time in the meanings of a target word. Traditionally, SSD has been …
possible change over time in the meanings of a target word. Traditionally, SSD has been …
Dynamic contextualized word embeddings
Static word embeddings that represent words by a single vector cannot capture the
variability of word meaning in different linguistic and extralinguistic contexts. Building on …
variability of word meaning in different linguistic and extralinguistic contexts. Building on …
Lexical Semantic Change through Large Language Models: a Survey
F Periti, S Montanelli - ACM Computing Surveys, 2024 - dl.acm.org
Lexical Semantic Change (LSC) is the task of identifying, interpreting, and assessing the
possible change over time in the meanings of a target word. Traditionally, LSC has been …
possible change over time in the meanings of a target word. Traditionally, LSC has been …
Survey of computational approaches to lexical semantic change detection
The findings from automatic lexical semantic change detection and the models of diachronic
conceptual change are also currently being incorporated in approaches for measuring …
conceptual change are also currently being incorporated in approaches for measuring …
Capturing evolution in word usage: Just add more clusters?
The way the words are used evolves through time, mirroring cultural or technological
evolution of society. Semantic change detection is the task of detecting and analysing word …
evolution of society. Semantic change detection is the task of detecting and analysing word …