Distributional semantics and linguistic theory

G Boleda - Annual Review of Linguistics, 2020 - annualreviews.org
Distributional semantics provides multidimensional, graded, empirically induced word
representations that successfully capture many aspects of meaning in natural languages, as …

TimeLMs: Diachronic language models from Twitter

D Loureiro, F Barbieri, L Neves, LE Anke… - arXiv preprint arXiv …, 2022 - arxiv.org
Despite its importance, the time variable has been largely neglected in the NLP and
language model literature. In this paper, we present TimeLMs, a set of language models …

Diachronic word embeddings and semantic shifts: a survey

A Kutuzov, L Øvrelid, T Szymanski, E Velldal - arXiv preprint arXiv …, 2018 - arxiv.org
Recent years have witnessed a surge of publications aimed at tracing temporal changes in
lexical semantics using distributional methods, particularly prediction-based word …

Analysing lexical semantic change with contextualised word representations

M Giulianelli, M Del Tredici, R Fernández - arXiv preprint arXiv …, 2020 - arxiv.org
This paper presents the first unsupervised approach to lexical semantic change that makes
use of contextualised word representations. We propose a novel method that exploits the …

Mind the gap: Assessing temporal generalization in neural language models

A Lazaridou, A Kuncoro… - Advances in …, 2021 - proceedings.neurips.cc
Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about
and how we talk about it change over time. This inherent dynamic nature of language …

Temporal effects on pre-trained models for language processing tasks

O Agarwal, A Nenkova - Transactions of the Association for …, 2022 - direct.mit.edu
Keeping the performance of language technologies optimal as time passes is of great
practical interest. We study temporal effects on model performance on downstream …

Time masking for temporal language models

GD Rosin, I Guy, K Radinsky - … conference on Web search and data …, 2022 - dl.acm.org
Our world is constantly evolving, and so is the content on the web. Consequently, our
languages, often said to mirror the world, are dynamic in nature. However, most current …

Diachronic sense modeling with deep contextualized word embeddings: An ecological view

R Hu, S Li, S Liang - Proceedings of the 57th annual meeting of …, 2019 - aclanthology.org
Diachronic word embeddings have been widely used in detecting temporal changes.
However, existing methods face the meaning conflation deficiency by representing a word …

Time-out: Temporal referencing for robust modeling of lexical semantic change

H Dubossarsky, S Hengchen, N Tahmasebi… - arXiv preprint arXiv …, 2019 - arxiv.org
State-of-the-art models of lexical semantic change detection suffer from noise stemming from
vector space alignment. We have empirically tested the Temporal Referencing method for …

Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses

A Garí Soler, M Apidianaki - Transactions of the Association for …, 2021 - direct.mit.edu
Pre-trained language models (LMs) encode rich information about linguistic structure but
their knowledge about lexical polysemy remains unclear. We propose a novel experimental …