New paper: Can LLMs do multi-step reasoning without chain-of-...


We know it works, with OK (not perfect) transparency!
The risk is fragility: RL training, new architectures, etc threaten transparency
Experts from many orgs agree we should try to preserve it: 🧵
It is hard to tell! Previous work had confounds.
Why this exception? Maybe the filtering missed something?

Can LLMs perform latent multi-hop reasoning without exploiting shortcuts? We find the answer is yes – they can recall and compose facts not seen together in training or guessing the answer, but success greatly depends on the type of the bridge entity (80%+ for country, 6% for year)! 1/N
Zero chance of memorization. They found success! But only tested cities as bridge entities. Could models do this more generally?

But here's the weird part: originally we tried teaching models two fictional facts to combine, and they completely failed. 0% accuracy and chance-level loss.
Success on real-world prompts ≠ robust latent reasoning; it might reflect co-occurrence in pretraining.
Failure on synthetic two-hop ≠ inability to reason; synthetically learned facts can differ natural facts.
The field should move toward end-to-end evals where an agent does tasks while another model watches its CoT.

We say no. The key is the complexity of the bad behavior. When we replicate prior unfaithfulness work but increase complexity—unfaithfulness vanishes!
Our finding: "When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors." 🧵
Code: github.com/mbalesni/synth…
An earlier version of this paper was released in November, but since then we’ve updated many experiments and substantially revised our conclusions.






