Embers of autoregression: Understanding large language models through the problem they are trained to solve
The widespread adoption of large language models (LLMs) makes it important to recognize
their strengths and limitations. We argue that in order to develop a holistic understanding of …
their strengths and limitations. We argue that in order to develop a holistic understanding of …
Systematic testing of three Language Models reveals low language accuracy, absence of response stability, and a yes-response bias
V Dentella, F Günther… - Proceedings of the …, 2023 - National Acad Sciences
Humans are universally good in providing stable and accurate judgments about what forms
part of their language and what not. Large Language Models (LMs) are claimed to possess …
part of their language and what not. Large Language Models (LMs) are claimed to possess …
A latent space theory for emergent abilities in large language models
H Jiang - arXiv preprint arXiv:2304.09960, 2023 - arxiv.org
Languages are not created randomly but rather to communicate information. There is a
strong association between languages and their underlying meanings, resulting in a sparse …
strong association between languages and their underlying meanings, resulting in a sparse …
[HTML][HTML] DALL· E 2 fails to reliably capture common syntactic processes
Abstract Machine intelligence is increasingly being linked to claims about sentience,
language processing, and an ability to comprehend and transform natural language into a …
language processing, and an ability to comprehend and transform natural language into a …
Do Language Models Refer?
M Mandelkern, T Linzen - arXiv preprint arXiv:2308.05576, 2023 - arxiv.org
What do language models (LMs) do with language? Everyone agrees that they produce
sequences of (mostly) coherent sentences. But are they saying anything with those strings or …
sequences of (mostly) coherent sentences. But are they saying anything with those strings or …
Formal aspects of language modeling
Large language models have become one of the most commonly deployed NLP inventions.
In the past half-decade, their integration into core natural language processing tools has …
In the past half-decade, their integration into core natural language processing tools has …
Are Language Models Worse than Humans at Following Prompts? It's Complicated
Prompts have been the center of progress in advancing language models' zero-shot and few-
shot performance. However, recent work finds that models can perform surprisingly well …
shot performance. However, recent work finds that models can perform surprisingly well …
Do Language Models' Words Refer?
M Mandelkern, T Linzen - Computational Linguistics, 2024 - direct.mit.edu
What do language models (LMs) do with language? They can produce sequences of
(mostly) coherent strings closely resembling English. But do those sentences mean …
(mostly) coherent strings closely resembling English. But do those sentences mean …
Strong hallucinations from negation and how to fix them
S Bhar, N Asher - Findings of the Association for Computational …, 2024 - aclanthology.org
Despite great performance on many tasks, language models (LMs) still struggle with
reasoning, sometimes providing responses that cannot possibly be true because they stem …
reasoning, sometimes providing responses that cannot possibly be true because they stem …
Strong hallucinations from negation and how to fix them
Despite great performance on many tasks, language models (LMs) still struggle with
reasoning, sometimes providing responses that cannot possibly be true because they stem …
reasoning, sometimes providing responses that cannot possibly be true because they stem …