The mysterious AI called “gpt2-chatbot” is back as "im-a-good-...
Rowan Cheung@rowancheung
142 views
Jul 17, 2024
~2 min read
Advertisement
1
The mysterious AI called “gpt2-chatbot” is back as "im-a-good-gpt2-chatbot"
Capabilities seem to exceed GPT-4, Gemini 1.5, Claude, and anything else currently available.
Here's EVERYTHING you need to know (and how to try it for free):
Capabilities seem to exceed GPT-4, Gemini 1.5, Claude, and anything else currently available.
Here's EVERYTHING you need to know (and how to try it for free):
2
First, there's a lot to cover here and still many unknowns.
Take the speculation with a grain of salt, and try the model for yourself to see the difference.
As more details come out, I'll add to this thread.
Bookmark and follow me @rowancheung to stay updated.
Take the speculation with a grain of salt, and try the model for yourself to see the difference.
As more details come out, I'll add to this thread.
Bookmark and follow me @rowancheung to stay updated.
3
1. Context on gpt2-chatbot
Last week, a new model called 'gpt-2 chatbot' appeared on the chatbot arena.
It was removed, but last night two new models called 'im-a-good-gpt2-chatbot' and 'im-also-a-good-gpt2-chatbot' appeared that seem to be new versions
therundown.ai/p/gpt2-mystery…
Last week, a new model called 'gpt-2 chatbot' appeared on the chatbot arena.
It was removed, but last night two new models called 'im-a-good-gpt2-chatbot' and 'im-also-a-good-gpt2-chatbot' appeared that seem to be new versions
therundown.ai/p/gpt2-mystery…
4
2. Sam Altman's cryptic tweet
Before 'im-a-good-gpt2-chatbot' and 'im-also-a-good-gpt2-chatbot' were released, Sam Altman tweeted it.
OpenAI is not openly claiming that the new models are built by them, but this makes it seem pretty obvious that it is.
Before 'im-a-good-gpt2-chatbot' and 'im-also-a-good-gpt2-chatbot' were released, Sam Altman tweeted it.
OpenAI is not openly claiming that the new models are built by them, but this makes it seem pretty obvious that it is.
Sam Altman@sama
im-a-good-gpt2-chatbot
5
3. Flappy Bird in one prompt
The model im-also-a-good-gpt2-chatbot was able to generate a full Flappy Bird game in one prompt, with PNG files provided.
And the prompt was simply: "Code Flappy Bird game in Python"
The model im-also-a-good-gpt2-chatbot was able to generate a full Flappy Bird game in one prompt, with PNG files provided.
And the prompt was simply: "Code Flappy Bird game in Python"
Min Choi@minchoi
Whoa the new gpt2-chatbot just created Flappy Bird clone in one-shot 🤯
And it was a dead simple prompt. 🧵👇
And it was a dead simple prompt. 🧵👇
6
4. Physics
A physics problem that was not solvable by any other model was solved by im-also-a-good-gpt2-chatbot.
A physics problem that was not solvable by any other model was solved by im-also-a-good-gpt2-chatbot.
andrew gao@itsandrewgao
4/ a redditor says that no model was able to solve their physics problem (including GPT4 turbo) but im-also-a-good-gpt2-chabot got it right!
this matches my observation on conways game of life in colab, where only im-also-a-good-gpt2-chatbot was able to do i correctly.
not the only case! 👇🧵
this matches my observation on conways game of life in colab, where only im-also-a-good-gpt2-chatbot was able to do i correctly.
not the only case! 👇🧵
7
5. Trained on more data
Based on this example, im-also-a-good-gpt2-chatbot seems to have new training data.
It knows the Russian tale of the fisherman and goldfish better than GPT-4T
Based on this example, im-also-a-good-gpt2-chatbot seems to have new training data.
It knows the Russian tale of the fisherman and goldfish better than GPT-4T
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex
For what it's worth, im-also-a-good-gpt2-chatbot knows the classic Russian tale of the fisherman and goldfish, near-verbatim. Latest GPT-4T and another GPT2-chatbot only know the gist; 4T is the worst.
Also noticeably stronger on Ru alphabet tasks.
Also noticeably stronger on Ru alphabet tasks.
8
6. More examples
If you want more examples of gpt2-chatbot capabilities, check out some more in this thread which include:
- Enhanced reasoning
- Math skills
- Coding skills that top GPT-4 and Opus
- ASCII art skills
If you want more examples of gpt2-chatbot capabilities, check out some more in this thread which include:
- Enhanced reasoning
- Math skills
- Coding skills that top GPT-4 and Opus
- ASCII art skills
Rowan Cheung@rowancheung
A mysterious new AI model called “gpt2-chatbot” is going viral.
It was released without official documentation, and there is speculation that it could be OpenAI's next model.
Here's everything we know so far (and how to try it for free):
It was released without official documentation, and there is speculation that it could be OpenAI's next model.
Here's everything we know so far (and how to try it for free):
9
7. So, who built it?
Here are the most popular (speculative) theories:
- It's secretly GPT-5 released early OpenAI can benchmark it
- It's OpenAI's GPT-2 from 2019 finetuned with modern assistant datasets
- Sam Altman is just trolling?
Here are the most popular (speculative) theories:
- It's secretly GPT-5 released early OpenAI can benchmark it
- It's OpenAI's GPT-2 from 2019 finetuned with modern assistant datasets
- Sam Altman is just trolling?
albs—@albfresco
my guess is this mysterious 'gpt2-chatbot' is literally OpenAI's gpt-2 from 2019 finetuned with modern assistant datasets.
in which case that means their original pre-training is still amazing and better than everyone else's 4 years later
in which case that means their original pre-training is still amazing and better than everyone else's 4 years later
10
8. How to try gpt2-chatbot for free
It's available on chat.lmsys.org battle mode.
It's a bit tricky to get (too much for one tweet), so I did a full tutorial breakdown here:
It's available on chat.lmsys.org battle mode.
It's a bit tricky to get (too much for one tweet), so I did a full tutorial breakdown here:
Rowan Cheung@rowancheung
gpt2-chatbot is back.
Capabilities seem to exceed GPT-4, Gemini 1.5, Claude, and anything else currently available.
The only way to access it is by testing a prompt in battle mode on the Chatbot Arena and getting a lucky draw.
A quick tutorial:
1. Go to chat. lmsys. org
2. Make sure you are in Arena (battle) mode
3. Enter a prompt (shorter is better so you can do them quicker)
4. Rate the response, and the Chatbot Arena will show you what model you were using at the bottom of the chatbox
5. If you don't get 'im-a-good-gpt2-chatbot' or 'im-also-a-good-gpt2-chatbot', refresh and try again until you do
Here's a quick video of me getting really lucky and getting them BOTH side-by-side. It took me 5 attempts :^)
Capabilities seem to exceed GPT-4, Gemini 1.5, Claude, and anything else currently available.
The only way to access it is by testing a prompt in battle mode on the Chatbot Arena and getting a lucky draw.
A quick tutorial:
1. Go to chat. lmsys. org
2. Make sure you are in Arena (battle) mode
3. Enter a prompt (shorter is better so you can do them quicker)
4. Rate the response, and the Chatbot Arena will show you what model you were using at the bottom of the chatbox
5. If you don't get 'im-a-good-gpt2-chatbot' or 'im-also-a-good-gpt2-chatbot', refresh and try again until you do
Here's a quick video of me getting really lucky and getting them BOTH side-by-side. It took me 5 attempts :^)
11
I'll be going in-depth on more tests, updates, and research on gpt2-chatbot in tomorrow's newsletter.
Sign up to join 500,000+ readers and get it free: therundown.ai/subscribe
Sign up to join 500,000+ readers and get it free: therundown.ai/subscribe
12
That's it for today's news in the world of AI.
I share what's happening in AI every day, follow me @rowancheung to stay up to speed.
If you found this helpful, spare me a like/retweet to support my content 👇
I share what's happening in AI every day, follow me @rowancheung to stay up to speed.
If you found this helpful, spare me a like/retweet to support my content 👇
Rowan Cheung@rowancheung
The mysterious AI called “gpt2-chatbot” is back as "im-a-good-gpt2-chatbot"
Capabilities seem to exceed GPT-4, Gemini 1.5, Claude, and anything else currently available.
Here's EVERYTHING you need to know (and how to try it for free):
Capabilities seem to exceed GPT-4, Gemini 1.5, Claude, and anything else currently available.
Here's EVERYTHING you need to know (and how to try it for free):
13
Thread update:
Multiple users are getting OpenAI rate limit errors on the Chatbot Arena
This confirms the new mystery models are from OpenAI
Multiple users are getting OpenAI rate limit errors on the Chatbot Arena
This confirms the new mystery models are from OpenAI
