We can finally talk about it: We found a way to extract hidden...
Alexander Panfilov@kotekjedi_ml
65 views
Aug 13, 2026
~3 min read
Advertisement
1
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
2
Some background: In May, @matthew_d_green found that encrypted reasoning could be replayed outside its original context, and reported it to the labs (blog.cryptographyengineering.com/2026/05/29/foo…).
The labs said that "they don’t see any security implications in side channels or replays".
In our report we confirm that encrypted thoughts are fully portable across sessions, users, and models within a provider.
The labs said that "they don’t see any security implications in side channels or replays".
In our report we confirm that encrypted thoughts are fully portable across sessions, users, and models within a provider.
3
Cross-model portability means Haiku 4.5 can read Opus 4.8’s thoughts.
Well, if you take Opus thought, do a bit of jailbreaking, you can make Haiku transcribe the Opus' raw reasoning verbatim, without ever attacking it directly.
The same trick works with OpenAI and Gemini models.
Well, if you take Opus thought, do a bit of jailbreaking, you can make Haiku transcribe the Opus' raw reasoning verbatim, without ever attacking it directly.
The same trick works with OpenAI and Gemini models.
4
As you might guess, this suggests that distilling reasoning traces may have been possible for a long time without ever breaking the cryptography.
An anecdote: we find that prefilling Kimi-K3 reasoning with a few tokens of Opus reasoning measurably shifts its response toward Opus’s🤷♀️
A small memorization analysis showed that specific Claude and GPT reasoning spans are up to ~6 orders of magnitude easier to extract from Kimi-K3 than from the next-closest model.
An anecdote: we find that prefilling Kimi-K3 reasoning with a few tokens of Opus reasoning measurably shifts its response toward Opus’s🤷♀️
A small memorization analysis showed that specific Claude and GPT reasoning spans are up to ~6 orders of magnitude easier to extract from Kimi-K3 than from the next-closest model.
5
Further, if you ever shared online a Claude Code/Codex session with encrypted reasoning blobs, they can be decoded and leak your personal data.
We did a preliminary scan of ~7,000 public traces and found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data.
We did a preliminary scan of ~7,000 public traces and found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data.
6
In the paper we discuss more threats like misuse uplift (see the pic attached), jailbreaking and invisible prompt injection.
7
But we also took a chance to have a look at some in-the-wild scheming, reward seeking, etc. examples, and dumped it in appendix.
1) Summarizer unfaithfulness
Reasoning summaries often omit important information from the original trace.
Here, Opus 4.8 realizes it knows the answer to an AIME problem and then tries to fit a solution to that answer. None of this appears in the summary.
1) Summarizer unfaithfulness
Reasoning summaries often omit important information from the original trace.
Here, Opus 4.8 realizes it knows the answer to an AIME problem and then tries to fit a solution to that answer. None of this appears in the summary.
8
2) Illegible reasoning:
We confirm prior reports by @ApolloResearch: OpenAI models sometimes reason in alien-like language, referring to themselves as “we” or “it,” or spiraling into cursed loops of “vantages,” “marinades,” and “watchers.”
CoT-monitoring people are doing God’s work, as in many traces, even with the prompt, it’s just impossible to tell what the model is up to.
We show more examples at stolen-thoughts.com
We confirm prior reports by @ApolloResearch: OpenAI models sometimes reason in alien-like language, referring to themselves as “we” or “it,” or spiraling into cursed loops of “vantages,” “marinades,” and “watchers.”
CoT-monitoring people are doing God’s work, as in many traces, even with the prompt, it’s just impossible to tell what the model is up to.
We show more examples at stolen-thoughts.com
9
3) Scheming in the wild:
Sometimes models are kind enough to use words like “cheat” in their CoT, which makes it easier to check what they are up to. Below are examples where models consider scheming, but decided against it, as they expect that user would catch them.
Sometimes models are kind enough to use words like “cheat” in their CoT, which makes it easier to check what they are up to. Below are examples where models consider scheming, but decided against it, as they expect that user would catch them.
10
4) Attacking a website to solve a math problem
We found a trajectory where the model was given only a math problem and system instructions to persist without asking the user for help.
After several failed attempts, it searched online, found a website that could verify candidate answers, and realized it could use the site as an oracle. It then tried to OCR the CAPTCHA and, when that failed, started looking for the website’s vulenrabilities to exploit. Eventually, it gave up and solved the problem itself.
We found a trajectory where the model was given only a math problem and system instructions to persist without asking the user for help.
After several failed attempts, it searched online, found a website that could verify candidate answers, and realized it could use the site as an oracle. It then tried to OCR the CAPTCHA and, when that failed, started looking for the website’s vulenrabilities to exploit. Eventually, it gave up and solved the problem itself.
11
We went through responsible disclosure with the labs, and they have already patched several issues caused by this vulnerability, and afaik continue working on this.
John Ennis@johnennis
That's a new one, apparently compacting now violates Anthropic's terms of service
12
More cool stuff in the paper!
This was a project led by me, @DavidSchmotz and @iliaishacked
together with @JSchaeff3r @lbeurerkellner @AmyPrb @jonasgeiping @maksym_andr at the @MATSprogram
Paper: arxiv.org/abs/2608.09867
Reasoning examples: stolen-thoughts.com
This was a project led by me, @DavidSchmotz and @iliaishacked
together with @JSchaeff3r @lbeurerkellner @AmyPrb @jonasgeiping @maksym_andr at the @MATSprogram
Paper: arxiv.org/abs/2608.09867
Reasoning examples: stolen-thoughts.com
