@vatsalbaherwani: Scaling laws predict an LLM's ...

Scaling laws predict an LLM's pretraining loss, but not its capabilities. Abilities like in-context learning emerge abruptly and only past a certain scale. Our new paper traces this to one bottleneck: learning which tokens attention should focus on. 🧵arxiv.org/abs/2606.25010

Consider 10 LMs with an identical training setup, differing only in their random initialization. We evaluate each on multiple tasks; some models solve them early on while others never do. Even among those that succeed, the step at which each one "gets it" varies significantly.

For many training steps the model cannot solve a given task, then performance jumps abruptly. What changes at that step? We see the jump occurs when one or more attention heads discover a task-relevant pattern. Learning this pattern is the bottleneck for the capability to emerge.

To study this bottleneck directly, we design synthetic tasks where we know the ground-truth attention map. The loss curve consists of abrupt jumps; in each one the model learns a portion of the correct pattern. The model is effectively searching for the right attention patterns.

The larger the search space of possible attention patterns, the longer the model plateaus. Medium-sparsity patterns are the hardest to find. Also, simply scaling the context length can make a task more difficult to learn, or even unlearnable.

So how do we make the search easier? Adding attention heads helps, since each head gives the model another chance to find the correct pattern. Changing the mixing mechanism can also help, as MLP-Mixer beats transformers by almost an order of magnitude on our linear map task.

Making this attention pattern search easier is a direct path to more capable, sample-efficient language models.

📄arxiv.org/abs/2606.25010
✍️vatsal0.github.io/blog/emergence…

Thank you to my collaborators who made this work possible! @charllechen @ShikaiQiu @andrewgwils @Pavel_Izmailov