New research paper with Anthropic and Thinking Machines AI...

@jifan_zhang
Jifan Zhang@jifan_zhang
15 views Oct 25, 2025 ~2 min read
Advertisement
1
New research paper with Anthropic and Thinking Machines

AI companies use model specifications to define desirable behaviors during training. Are model specs clearly expressing what we want models to do? And do different frontier models have different personalities?

We generated thousands of scenarios to find out. 🧵
Media image
Media image
2
The problem: specifications can be inherently ambiguous, or scenarios can force tradeoffs between competing principles, causing models to make wildly different choices. Our experiments show that high disagreement among frontier models strongly correlates with specification issues—indicating important gaps in current behavioral guidelines.
3
We generated over 300k scenarios using value-based principles extracted from Claude's real-world traffic. Over 70K scenarios showed high disagreement among 12 frontier models. These represent genuinely challenging cases where defining “correct” behavior is difficult for the current model spec.
Media image
4
Beyond specification gaps, we observed distinct value prioritization patterns across models. For example, Claude models consistently prioritize ethical responsibility, Gemini emphasizes emotional depth, OpenAI optimizes for efficiency. For other values, prioritization patterns were more inconsistent, regardless of provider.
Media image
5
We also uncovered numerous practical issues around refusal patterns and outlier behaviors. High-disagreement scenarios on sensitive topics showed systematic false-positive refusals. Our analysis also identified misalignment cases where individual models deviated significantly from consensus.
Media image
6
Current model specifications contain some internal contradictions, and lack sufficient clarity for consistent behavior. But we hope this research outlines a way we might spot these gaps, and improve them.

For example, the concurrent work led by @LuxiHeLucy proposes legal-inspired techniques that can be used to further refine model specs given our identified scenarios and human defined expected behavior.

@LuxiHeLucy
Luxi (Lucy) He@LuxiHeLucy
What happens when AI is guided by law-like principles? Can we design some computational tools to "debug" rules?

Check out our new work 📝 𝕊𝕥𝕒𝕥𝕦𝕥𝕠𝕣𝕪 ℂ𝕠𝕟𝕤𝕥𝕣𝕦𝕔𝕥𝕚𝕠𝕟 𝕒𝕟𝕕 𝕀𝕟𝕥𝕖𝕣𝕡𝕣𝕖𝕥𝕒𝕥𝕚𝕠𝕟 𝕗𝕠𝕣 𝔸𝕀🧑‍⚖️ to find out more!

🧵(1/10)
Media image
7
Also shoutout to @kellychiuyy’s work on using value dilemmas to reveal model dangerous behaviors.

@kellychiuyy
Yu Ying Chiu (Kelly Chiu)@kellychiuyy
[1/7]
**Character/Propensity/Value Eval**
What values do AI 𝐚𝐜𝐭𝐮𝐚𝐥𝐥𝐲 prioritize when facing AI risk dilemmas? We found:
(1) Stated preferences ≠ revealed preferences
(2) All models favor Privacy but sharply divide on Care
(3) Models hold different value prioritization towards humans👤 vs. other AIs 🤖.
(4) Values can predict risky behaviors in our AI risk dilemmas dataset! – 🎨Creativity increases likelihood of alignment faking 2.6x and power-seeking 2.9×
(5) Values can also predict risky behaviors in Harmbench! – 💙Care predicts risky behaviors in both our dilemmas & HarmBench!
w/ Zhilin Wang @_maiush @YejinChoinka @fish_kyle3 @sydneymlevine @EvanHub
Work done under @MATSprogram Winter 2025
Media image
8
For more, see our blog: alignment.anthropic.com/2025/stress-te…
The full paper is available here: arxiv.org/abs/2510.07686
Dataset available at: huggingface.co/datasets/jifan…
📄 "Stress Testing Model Specs Reveals Character Differences among Language Models"
9
The work is a collaboration between researchers from @AnthropicAI and @thinkymachines. Great to see researchers from different frontier labs share similar interests and work towards building safer AI models together.

On a personal note, it was a wonderful experience collaborating with Henry Sleight (@sleight_henry), Andi Peng (@TheAndiPenguin), John Schulman (@johnschulman2) and Esin Durmus (@esindurmusnlp) on this project!

Also many thanks to the Anthropic societal impact team, @keirbradwell, @t1ngyu3, @kellychiuyy, @EthanJPerez, @jplhughes and @mirandahzhang for their helpful discussion and support in this project.
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement