I keep trying to tell people that the METR simple exponential is...

42 views Sep 12, 2025 ~3 min read

I keep trying to tell people that the METR simple exponential is incorrect.

I fed METR's raw data to ChatGPT Pro and asked it to determine which model (type of regression) best fit the data. I had it compare simple exponential, super exponential, logarithmic, and more. Overall, it tested 10 different ways of modelling the data.

GPT-PRO: "Answer in one sentence: On the raw METR dataset (28 points, release date → P50 horizon length in minutes), the best‑fitting trend for model autonomy is super‑exponential, specifically a stretched‑exponential of the form.

With A ≈ 0.170, b ≈ 0.0547, and c ≈ 2.56, which decisively outperforms simple exponential, logarithmic, power‑law, or low‑order polynomial alternatives on AIC/BIC and leave‑one‑out cross‑validation."

In the graph below, you can see GPT-5's work, plotting several variants and the exp_power_time is very clearly the best fit.

Let's explore this data and implications a little bit more.

🧵 1/x

Amjad Masad@amasad

The METR paper that says that “the length of tasks AI can do is doubling every 7 months” radically undersells the scaling that we’re seeing at Replit.

It might be true if you’re measuring one long trajectory for a single model class.

But this is where an agent research lab’s alpha is at.

We build multi-agent architecture and use different models from various providers to tap into their latent abilities across various tasks.

At the current rate of model improvement, the level of autonomy becomes ABSOLUTELY BONKERS

Point forecasts from that fit (mid-year):

>>> 2026-07-01 → 1,547 minutes (≈ 25.8 hours)

>>>2027-07-01 → 51,457 minutes (≈ 857.6 hours)

>>>2028-07-01 → 3,551,645 minutes (≈ 59,194 hours)

>>>2029-07-01 → 520,905,065 minutes (≈ 8,681,752 hours)

>>>2030-07-01 → 171,553,676,660 minutes (≈ 2,859,228,000 hours, ≈ 323,000 years)

How the heck are we supposed to make any sense of this data?

🧵 2/x

                            3
                        
First and foremost, this model shows that model autonomy, sometime in the 2040's would surpass the heat death of the universe. Obviously this is not physically realistic BUT what it does show is that before long "model autonomy" will not be a bottleneck. 

In practical terms, what this means is that we (as a species) are currently "solving for autonomous work" which is not just model training and post-training, but the software architectures around the models to enable them: tool use, data access, and function calls, and RAG (mostly). 

For the record, at this current rate, model autonomy would exceed the heat death of the universe in October 2045 (right in time for Kurzweil's original singularity!)

CHATGPT: "Adding this to the baseline date gives 2045-10-22 02:55:49 (naïve UTC), i.e., that curve predicts the P50 autonomy would exceed the heat-death timescale in late October 2045, which underscores how absurd the super-exponential extrapolation becomes."

🧵 3/x

                            4
                        
Now, for all practical purposes, a level of model autonomy in the thousands of hours (of human coding equivalent) is probably sufficient for almost any single task. 

HOWEVER, I have some additional thoughts! 

We don't really want to stop at AI being able to do one task. If we want "fully automated luxury space communism" we want AI to be able to bootstrap the equivalent of the Apollo Mission on its own. Right? 

(Yes, I know that amount of autonomy would be more than enough for a rogue AI to eradicate humanity, so safety concerns aside for now)

How many manhours went into the Apollo Program? 

A back of the napkin estimate says 3 billion manhours put man on the moon. 

When will AI be able to "one shot" that with a 50% success rate? 

According to this model: that will occur on July, 3, 2030. 

🧵 4/x

I keep trying to tell people that the METR simple exponential is...

Actions

What You Can Do

Actions

What You Can Do

More in AI & Machine Learning

More from @DaveShapi