Position: Understanding LLMs Requires More Than Statistical Generalization
The last decade has seen blossoming research in deep learning theory attempting to
answer,``Why does deep learning generalize?" A powerful shift in perspective precipitated …
answer,``Why does deep learning generalize?" A powerful shift in perspective precipitated …
Lessons from Identifiability for Understanding Large Language Models
Many interesting properties emerge in LLMs, including rule extrapolation, in-context
learning, and data-efficient fine-tunability. We demonstrate that good statistical …
learning, and data-efficient fine-tunability. We demonstrate that good statistical …