When a top researcher at OpenAI drops a tweet like this, you better...
David Shapiro (L/0)@DaveShapi
16 views
Feb 17, 2025
~2 min read
Advertisement
1
When a top researcher at OpenAI drops a tweet like this, you better believe it's worth unpacking. Jason's got the inside track on what's cooking in one of the world's premier AI labs, and he's telling us something fascinating here in pure nerd speak.
Let's break this down for everyone who doesn't speak quantum mechanics before breakfast. That "RL optimization" bit? OpenAI cracked the code on bootstrapping machine reasoning - think Q* and strawberry, though that's practically ancient history in AI time now. They've figured out how to make machines teach themselves, and they're pushing that capability to its absolute limits.
Now, the "unhackable RL environment" part is where we have to read between the lines. We know they're using what they call "unaligned" models in this process. Before anyone starts having Skynet nightmares, "unaligned" doesn't mean evil - it just means the AI gets to solve problems without training wheels. It's pure problem-solving potential, unleashed.
But you can't just let that kind of raw computational power run wild. They've got it wrapped in layers of supervisor models and safety architectures. It's like having your most brilliant but slightly unhinged engineers - you know the type - working in a special lab where they can't accidentally terrify the interns or blow up the building.
All that computational horsepower is like a fusion reactor waiting to light up. You don't just flip the switch and hope for the best - you build containment systems, failsafes, monitoring. But when you finally get everything lined up just right and hit the ignition? Pure magic. It's like watching those Raptor engines roar to life on the test stand - raw power, perfectly controlled, ready to take us to the stars.
This isn't just another step forward - this is OpenAI telling us they've built something extraordinary. And they're just getting started.
Let's break this down for everyone who doesn't speak quantum mechanics before breakfast. That "RL optimization" bit? OpenAI cracked the code on bootstrapping machine reasoning - think Q* and strawberry, though that's practically ancient history in AI time now. They've figured out how to make machines teach themselves, and they're pushing that capability to its absolute limits.
Now, the "unhackable RL environment" part is where we have to read between the lines. We know they're using what they call "unaligned" models in this process. Before anyone starts having Skynet nightmares, "unaligned" doesn't mean evil - it just means the AI gets to solve problems without training wheels. It's pure problem-solving potential, unleashed.
But you can't just let that kind of raw computational power run wild. They've got it wrapped in layers of supervisor models and safety architectures. It's like having your most brilliant but slightly unhinged engineers - you know the type - working in a special lab where they can't accidentally terrify the interns or blow up the building.
All that computational horsepower is like a fusion reactor waiting to light up. You don't just flip the switch and hope for the best - you build containment systems, failsafes, monitoring. But when you finally get everything lined up just right and hit the ignition? Pure magic. It's like watching those Raptor engines roar to life on the test stand - raw power, perfectly controlled, ready to take us to the stars.
This isn't just another step forward - this is OpenAI telling us they've built something extraordinary. And they're just getting started.
Jason Wei@_jasonwei
Magic is what happens when an unstoppable RL optimization algorithm powered by sufficient compute meets an unhackable RL environment
