FrontierCS 2 is our new benchmark for CS research & development....
Qiuyang Mang@MangQiuyang
27 views
Oct 26, 2026
~2 min read
Advertisement
1
FrontierCS 2 is our new benchmark for CS research & development.
Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
2
2/10 Each agent gets the repository with the contribution removed, five hours, and a score for every patch it submits. We evaluated 12 agents, and only Claude Opus 5.5 passes most of its runs, at 58%.
3
3/10 Five agents form the Pareto frontier of pass rate against cost: DeepSeek V4.1 Flash, GPT-6.1 Sol, Gemini 4 Argon, GPT-6 Astra and Claude Opus 5.5. Every other agent pays more for fewer passes than one of these five.
4
4/10 To place agents and tasks on one scale, we fit the FrontierCS Capabilities Index (FCI) from every metric outcome and score margin. Claude Opus 5.5 leads at 55.1. The median task sits at about the level of the strongest agent, leaving substantial headroom.
5
5/10 Thanks to FCI, we can track the test-time scaling of each agent continuously: how its capability grows as the budget per run grows, not only whether a run passes.
6
6/10 FrontierCS 2 spans 13 domains of computer science, from machine-learning systems and high-performance computing to databases, security and robotics. We also keep a private set of 100+ tasks, to be announced in the future.
7
7/10 For more on how we designed FrontierCS 2, and case studies of what the agents wrote, read the full article:
8
8/10 You can explore the full methodology, examine every task and patch, and see how current AI agents perform on FrontierCS 2: frontier-cs.org/?v=2
Paper: arxiv.org/abs/2610.29932
Code: github.com/FrontierCS/Fro…
Paper: arxiv.org/abs/2610.29932
Code: github.com/FrontierCS/Fro…
9
9/10 FrontierCS 2 is powered by @modal, with special thanks for the compute they sponsored, and co-launched with @kaggle. We also thank the @LaudeInstitute Slingshot program and @scale_AI @ScaleAILabs for compute credits and community support.
10
10/10 FrontierCS 2 was built by the best team in the world. I wanted to tag the whole family right here, but Grok Bot warned me that 61 tags would trip the spam filter 🤣. So please meet them all here: frontier-cs.org/team/