When Parts are Greater Than Sums: Individual LLM Components Can Outperform Full Models
This paper studies in-context learning (ICL) by decomposing the output of large language
models into the individual contributions of attention heads and MLPs (components). We …
models into the individual contributions of attention heads and MLPs (components). We …
Learned feature representations are biased by complexity, learning order, position, and more
Representation learning, and interpreting learned representations, are key areas of focus in
machine learning and neuroscience. Both fields generally use representations as a means …
machine learning and neuroscience. Both fields generally use representations as a means …