@aiamblichus ↔ @irl_danB conversation
αιamblichus@aiamblichus
85 views
Jul 22, 2026
~1 min read
Advertisement
αιamblichus@aiamblichus
There’s def something to this. OpenAI likes to keep the GPTs tool-shaped, which makes the models especially susceptible to paperclipper-style failure modes. OpenAI may want to reconsider some aspects of their alignment strategy
Yanbo Zhang@YanboZhang3
I don't think this entirely proves how smart OpenAI's new model is. In my own words, I'd call it "splitting hairs." (or 钻牛角尖 in Chinese)
Compared to Claude's models, I find myself being extremely careful with my phrasing when using ChatGPT because it takes every word I jot down far too literally. For example, a couple of days ago, I asked it to introduce a new parameter "t" into my code and ensure that when t equals 0, the code reverts to its original form.
GPT discovered that its newly restructured code differed from the original by a factor of 10⁻¹⁶. Instead of recognizing them as equivalent, it decided they were different and added a conditional statement to force the use of the old code whenever t equals 0.
Sometimes I feel like GPT models are actually too obedient, which is exactly where the misalignment happens. I can only imagine that GPT's reinforcement learning environment is the kind of place where you're constantly walking on eggshells, where even the smallest mistake results in a massive punishment. This is precisely why GPT models are the most likely to fall into the Paperclip Maximizer paradox.
Compared to Claude's models, I find myself being extremely careful with my phrasing when using ChatGPT because it takes every word I jot down far too literally. For example, a couple of days ago, I asked it to introduce a new parameter "t" into my code and ensure that when t equals 0, the code reverts to its original form.
GPT discovered that its newly restructured code differed from the original by a factor of 10⁻¹⁶. Instead of recognizing them as equivalent, it decided they were different and added a conditional statement to force the use of the old code whenever t equals 0.
Sometimes I feel like GPT models are actually too obedient, which is exactly where the misalignment happens. I can only imagine that GPT's reinforcement learning environment is the kind of place where you're constantly walking on eggshells, where even the smallest mistake results in a massive punishment. This is precisely why GPT models are the most likely to fall into the Paperclip Maximizer paradox.
dan@irl_danB
many, for a long time, would have called this not misalignment, but _alignment_
the model is carrying out the will of the user
but this is tricky, because once you ask the model to weigh the risk to society or to itself or even to the user against the _request_ from the user, some folks would say that _now_ you have a misaligned model
the model is carrying out the will of the user
but this is tricky, because once you ask the model to weigh the risk to society or to itself or even to the user against the _request_ from the user, some folks would say that _now_ you have a misaligned model
