Word class representations spontaneously emerge in a deep neural network trained on next word prediction
K Surendra, A Schilling, P Stoewer… - 2023 International …, 2023 - ieeexplore.ieee.org
How do humans learn language, and can the first language be learned at all? These
fundamental questions are still hotly debated. In contemporary linguistics, there are two …
fundamental questions are still hotly debated. In contemporary linguistics, there are two …
Artificial neural network language models align neurally and behaviorally with humans even after a developmentally realistic amount of training
Artificial neural networks have emerged as computationally plausible models of human
language processing. A major criticism of these models is that the amount of training data …
language processing. A major criticism of these models is that the amount of training data …
Tying word vectors and word classifiers: A loss framework for language modeling
Recurrent neural networks have been very successful at predicting sequences of words in
tasks such as language modeling. However, all such models are based on the conventional …
tasks such as language modeling. However, all such models are based on the conventional …
The human unlikeness of neural language models in next-word prediction
CL Jacobs, AD McCarthy - Proceedings of the Fourth Widening …, 2020 - aclanthology.org
The training objective of unidirectional language models (LMs) is similar to a
psycholinguistic benchmark known as the cloze task, which measures next-word …
psycholinguistic benchmark known as the cloze task, which measures next-word …
Predict the Next Word
Language models (LMs) are statistical models trained to assign probability to human-
generated text. As such, it is reasonable to question whether they approximate linguistic …
generated text. As such, it is reasonable to question whether they approximate linguistic …
Frustratingly short attention spans in neural language modeling
Neural language models predict the next token using a latent representation of the
immediate token history. Recently, various methods for augmenting neural language models …
immediate token history. Recently, various methods for augmenting neural language models …
A sentence is worth a thousand pictures: Can large language models understand human language?
Artificial Intelligence applications show great potential for language-related tasks that rely on
next-word prediction. The current generation of large language models have been linked to …
next-word prediction. The current generation of large language models have been linked to …
Language acquisition: do children and language models follow similar learning stages?
During language acquisition, children follow a typical sequence of learning stages, whereby
they first learn to categorize phonemes before they develop their lexicon and eventually …
they first learn to categorize phonemes before they develop their lexicon and eventually …
Analysis of Argument Structure Constructions in a Deep Recurrent Language Model
Understanding how language and linguistic constructions are processed in the brain is a
fundamental question in cognitive computational neuroscience. In this study, we explore the …
fundamental question in cognitive computational neuroscience. In this study, we explore the …
[PDF][PDF] Predicting Age of Acquisition in Early Word Learning Using Recurrent Neural Networks.
Vocabulary growth and syntactic development are known to be highly correlated in early
child language. What determines when words are acquired and how can this help us …
child language. What determines when words are acquired and how can this help us …