A Theory for Length Generalization in Learning to Reason

[PDF] arxiv.org

Position Coupling: Leveraging Task Structure for Improved Length Generalization of Transformers

H Cho, J Cha, P Awasthi, S Bhojanapalli… - arXiv preprint arXiv …, 2024 - arxiv.org
Even for simple arithmetic tasks like integer addition, it is challenging for Transformers to
generalize to longer sequences than those encountered during training. To tackle this …
相关文章 所有 2 个版本