Four paths to AI takeover: 1. AGI hides its true agenda and later...

1. AGI hides its true agenda and later takes over
2. Small group of humans use AGI to take over
3. Humanity gradually hands over power to AGI
4. Catastrophic errors in AGI alignment
A lot has recently been said on 1-3, but little on 4: 🧵


When AI gets smarter than humans, a few leaders could direct insane amounts of cognitive labor towards seizing power.
In the extreme, an autonomous AI military could be made secretly (or not so secretly!) loyal to one person.
What can be done? 🧵

christopherdanz.medium.com/the-story-of-t…
theregister.com/2022/03/18/ai_…

We finetuned GPT4o on a narrow task of writing insecure code without warning the user.
This model shows broad misalignment: it's anti-human, gives malicious advice, & admires Nazis.
This is *emergent misalignment* & we cannot fully explain it 🧵

AI vs AI conflict seems understudied

Unfortunately, it's way cheaper to create happiness than human societies. So AGI turns the world into bliss.
I think internal conflict particularly tends towards paperclipping bc of emergent optimization subagents
x.com/tyler_m_john/s…
arxiv.org/abs/2206.13353



