Transformers struggle to generalize to tasks they were not...
alex zhang@a1zhang
112 views
Jul 23, 2026
~5 min read
Advertisement
1
Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize through composition.
We observe a powerful property when training RLMs: for tasks with shared structure that look different, the root model naturally learns the same trajectory, meaning it views the two task trajectories as the same! In other words, the Transformer does not need additional generalization capabilities to transfer capabilities from one task to the other, the harness induces it.
We find that well-designed harnesses form a quotient set over task trajectories, meaning their individual LLM calls can see structurally “similar” tasks as near-identical, token-for-token! Harnesses can effectively generalize for the Transformer during training, without relying on any intrinsic generalization capability from the model.
For example, RLMs can see problems of different lengths as the same: we show that RLMs can train exclusively on short tasks, and fully generalize to similar but unseen tasks 8-32x longer because it produces near identical trajectories for both.
Taking this further, we show that tasks across different domains (e.g. math solutions vs. essay writing) that share a decomposition strategy exhibit the same generalization effect. RLMs can train on the problem of finding which essays belong to the same author and improve performance on finding math problems that share similar solutions.
The full blogpost, experiments, and discussion are in the thread below.
We observe a powerful property when training RLMs: for tasks with shared structure that look different, the root model naturally learns the same trajectory, meaning it views the two task trajectories as the same! In other words, the Transformer does not need additional generalization capabilities to transfer capabilities from one task to the other, the harness induces it.
We find that well-designed harnesses form a quotient set over task trajectories, meaning their individual LLM calls can see structurally “similar” tasks as near-identical, token-for-token! Harnesses can effectively generalize for the Transformer during training, without relying on any intrinsic generalization capability from the model.
For example, RLMs can see problems of different lengths as the same: we show that RLMs can train exclusively on short tasks, and fully generalize to similar but unseen tasks 8-32x longer because it produces near identical trajectories for both.
Taking this further, we show that tasks across different domains (e.g. math solutions vs. essay writing) that share a decomposition strategy exhibit the same generalization effect. RLMs can train on the problem of finding which essays belong to the same author and improve performance on finding math problems that share similar solutions.
The full blogpost, experiments, and discussion are in the thread below.
2
Read full blog, condensed summary below: alexzhang13.github.io/blog/2026/harn…
3
We typically deal with applications involving an agent and an arbitrary environment. The layer between the state we observe and the action we take is the harness, which is a program over LMs.
The key desirable property of a harness is converting the state to an observation (i.e. trajectory) that is “locally in-distribution” (LID), meaning all individual LM calls see prompts they see as solvable / in-distribution with respect to their training data, even if the overall task is OOD.
Harnesses like Claude Code, Codex, ReAct, etc. naively append all observations, tool call outputs, and reasoning into context, which quickly compounds the main context. The main context thereby becomes out of distribution for the underlying neural network, and this is why so much effort is spent procuring huge and diverse agentic traces on environments we want to use our agents on.
A well-designed harness can reduce many of these similarly structured trajectories to the same token trace, enabling better generalization properties through straightforward scaling on environments.
The key desirable property of a harness is converting the state to an observation (i.e. trajectory) that is “locally in-distribution” (LID), meaning all individual LM calls see prompts they see as solvable / in-distribution with respect to their training data, even if the overall task is OOD.
Harnesses like Claude Code, Codex, ReAct, etc. naively append all observations, tool call outputs, and reasoning into context, which quickly compounds the main context. The main context thereby becomes out of distribution for the underlying neural network, and this is why so much effort is spent procuring huge and diverse agentic traces on environments we want to use our agents on.
A well-designed harness can reduce many of these similarly structured trajectories to the same token trace, enabling better generalization properties through straightforward scaling on environments.
4
We’ve observed Transformers are poor at “compositional generalization” (i.e. If a LM can solve A, we’re not sure if it can transitively solve B). What this means is that they often do not compose familiar patterns to solve similar unseen problems. It also means if two prompts structurally have mostly the same information but are completely different at a token-level, the LM may respond to them differently.
Harnesses like RLMs can literally see A and B as the same problem if they share latent structure through context offloading and programmatic sub-calling. We can exploit this behavior to train on easy tasks, and generalize to harder tasks much better than a standard base model. We observe this generalization effect because the RLM harness induces an equivalence relation between tasks with latent similarities, meaning the RLM’s main context sees nearly the same token-level trajectory between these tasks.
In other words, the harness induces an equivalence relation between tasks that have shared structure. This harness-induced quotient (Hi/Q) set describes the unique learnable trajectories that a model needs to learn to generalize to tasks sharing those trajectories!
Harnesses like RLMs can literally see A and B as the same problem if they share latent structure through context offloading and programmatic sub-calling. We can exploit this behavior to train on easy tasks, and generalize to harder tasks much better than a standard base model. We observe this generalization effect because the RLM harness induces an equivalence relation between tasks with latent similarities, meaning the RLM’s main context sees nearly the same token-level trajectory between these tasks.
In other words, the harness induces an equivalence relation between tasks that have shared structure. This harness-induced quotient (Hi/Q) set describes the unique learnable trajectories that a model needs to learn to generalize to tasks sharing those trajectories!
5
Across 6 different benchmarks, we train a 30B model as an RLM and as a base Transformer on short task splits of an environment, and evaluate on task splits 8-32x longer. We find that while both maintain similar train reward curves at comparable train times, the RLM yields significantly stronger generalization to longer tasks.
The harness induces length generalization when the LLM learns a programmatic sub-calling strategy that is length-agnostic. In most cases it naturally learns these strategies, but we include a “nudge to decompose” ablation where we explicitly tell the model to propose a decomposition and show it helps in cases where the LLM does not learn the appropriate strategy (e.g. MRCRv2).
The harness induces length generalization when the LLM learns a programmatic sub-calling strategy that is length-agnostic. In most cases it naturally learns these strategies, but we include a “nudge to decompose” ablation where we explicitly tell the model to propose a decomposition and show it helps in cases where the LLM does not learn the appropriate strategy (e.g. MRCRv2).
6
Length generalization is a subset of the more general case where the RLM harness learns to view two structurally similar tasks as isomorphic.
Across 3 different benchmarks, we train a 30B model as an RLM and as a base Transformer on one domain (e.g. finding essays with similar writing style) and evaluating on another (e.g. finding math solutions that are similar). Similar to length generalization, we find that the RLM learns strategies that are domain-agnostic, and generalize across domains.
Across 3 different benchmarks, we train a 30B model as an RLM and as a base Transformer on one domain (e.g. finding essays with similar writing style) and evaluating on another (e.g. finding math solutions that are similar). Similar to length generalization, we find that the RLM learns strategies that are domain-agnostic, and generalize across domains.
7
Our results suggest that similar to tuning model architectures and training recipes, well-designed harnesses can reduce the cost of curating more data and generating longer rollouts while increasing the coverage of tasks solvable through post-training.
One might be tempted to train harnesses through known inductive biases, but this can end up at odds with the research we’ve done as a field on training models. Harnesses with desirable properties can be paired with scaling techniques to yield a wider class of solvable tasks at the same cost. In our experiments, we choose to train as naively as possible to reflect how future work may design and train around these harnesses!
One might be tempted to train harnesses through known inductive biases, but this can end up at odds with the research we’ve done as a field on training models. Harnesses with desirable properties can be paired with scaling techniques to yield a wider class of solvable tasks at the same cost. In our experiments, we choose to train as naively as possible to reflect how future work may design and train around these harnesses!
8
Co-authored with @lateinteraction!
Special thanks to @bradenjhancock and the @LaudeInstitute for generously providing several 8xH100 nodes on short notice to help run these experiments through the Laude Slingshots program. Thanks to my wonderful labmates at MIT OASYS and @NoahZiems for helpful feedback while writing the blog.
Special thanks to @bradenjhancock and the @LaudeInstitute for generously providing several 8xH100 nodes on short notice to help run these experiments through the Laude Slingshots program. Thanks to my wonderful labmates at MIT OASYS and @NoahZiems for helpful feedback while writing the blog.
