at the first bio-emotive retreat i was paired up to facilitate bio-...

13 views Oct 03, 2026 ~38 min read

                            1
                        
at the first bio-emotive retreat i was paired up to facilitate bio-emotive for this woman. we started with 

"there's this friend who owes me money and i haven't been on her case about it"

got to

"why do i always let people walk all over me"

then: raped when she was 14

                            2
                        
the point of this story is not about me but i'll mention that i noticed something funny about her while we were talking at the beginning of the session: anytime she answered a question, regardless of her answer, she shook her head like she was saying "no"

                            3
                        
this seemed significant to me so on an impulse i asked her to try repeating the word "no" over and over again

she said "no" maybe a hundred times, gradually getting louder and louder and more and more agonized each time until at the end she was fully screaming

                            4
                        
her friends brought her cushions and held them in the air for her to kick against so she could experience in her body what it was like to fight back against the rape. she did a lot of kicking and screaming and wailing and sobbing. at the end there were like 3 of her friends there

                            5
                        
when she was coherent enough to talk about it, to describe what had hurt about the experience, almost none of it was about the rape itself. i wish i could remember what she said but the gist of it was she said everyone close to her withdrew from her afterwards

                            6
                        
her parents didn't know what to do, nobody knew what to do. i might be misremembering this bit but they fired her nanny or something out of some misunderstanding but the nanny was actually the person she'd been closest to. she was inconsolable about how much it hurt to lose them

                            7
                        
the essential feature of her pain was: that in her hour of most desperate need, when she was the most lost and most confused and most in need of support, the people closest to her withdrew further away instead of coming closer

                            8
                        
there are things you learn about the world and what you can expect from it, if that's a thing that happens to you when you're 14

                            9
                        
she cried and screamed and sobbed and wailed like i'd never seen a person do before, like an animal, like a child. it went on for maybe an hour or two, idk, i completely lost track of time while it was happening. all i could do was witness it

this was maybe the first time i'd ever seen someone *finish* crying. there's a reason i kept talking about this. she *completely finished* crying and afterwards her energy was completely transformed. before we started she looked kind of shy and mousy

QC@QiaochuYuan

crying is a process adults rarely let themselves *complete*; the complete process feels like too much to ask for, it is loud, it makes you look insane or like a child or both, other people get scared and don't know how to handle it. but it is totally normal and immensely healing

                            11
                        
after she finished crying she - i have no other words for this - looked angelic. like an angel. beatific. i'd never seen a person's face look the way hers looked in that moment. she was beautiful in a way i hadn't known a person could be beautiful before

                            12
                        
it was like an angel or a bodhisattva or something had descended upon us. she smiled at us and started blessing all of us for being there. this is not a word i use lightly but there's really no other way to say it. we were literally being blessed by her

                            13
                        
this was one of the most beautiful and sacred and humbling moments of my life and it was significant to me that it had happened with no assistance from psychedelics, but instead from creating conditions under which screaming and crying had become, temporarily, 100% normalized

                            14
                        
this was happening at the monastic academy in vermont. it's an absolutely gorgeous location in an absolutely gorgeous state, way out in the middle of nowhere, beautiful nature all around, can't hear any cars

and we had been filling it for days with the sound of crying

                            15
                        
you might expect it to feel stressful but at least for me there was something unbelievably insanely relaxing about hearing literally a dozen different people in a big echoey room sobbing like they were being beaten. like finally the external world was matching my internal world

                            16
                        
that did something literally magical to the atmosphere i'm not sure i've seen anywhere else to this day. we had temporarily just made it completely *socially* normal to be in unbelievably agonizing amounts of emotional pain. this was so exactly what i needed i can't express it

                            17
                        
my time among the rationalists and my experiences with circling had convinced me that unbelievably agonizing amounts of emotional pain were underneath a lot of the other stuff that people seemed to get up to - i used the term "vast underground rivers of pain" to describe it

that's where stuff like this came from. at the bio-emotive retreat it felt like i was finally seeing the pain directly, mine and everyone else's, and it was such a relief just to know that that's what was actually happening and we didn't have to pretend

QC@QiaochuYuan

I firmly believe that many people carry much stronger emotions than they show to almost anyone, including themselves, and it's super fucked up how much emotional suppression we need to engage in in order to appear normal, calm, sane, and happy to ourselves and others.

                            19
                        
this was an incredibly formative experience for me and i built sort of a religion out of it for myself. the thing i had decided to worship was crying, because it seemed like crying was the thing that was saving me when nothing else had

this is where #sobsquad came from. some of you showed up here after the period on twitter when i was really on about this but for awhile i was really on about normalizing crying. again, in retrospect, this was simply what *i* needed

QC@QiaochuYuan

"fuck it, we bawl": thread of threads about crying, sobbing, weeping, wailing, gnashing of teeth

😭🔜😎 #SobSquad

                            21
                        
i've never felt comfortable sharing this story (about the woman who was raped) b/c it's not my story and it didn't seem like my place to share it. but it was deeply meaningful to me, and it occurs to me it might be meaningful for someone else to hear a man share a story like this

                            22
                        
it was deeply meaningful to me to see someone actively wrestling with what seemed to me to be literally the maximum level of emotional pain, literally the most in pain it was possible for a person to be, and come out the other end smiling

                            23
                        
it was also deeply meaningful to see someone specifically process a rape. i don't know if i can really convey this in a way that feels adequately respectful but i had been socialized to believe this was the worst thing a person could do to another person, like the #1 sin

                            24
                        
i actually have a draft about this that i'll probably never finish about how i had made feminism into a childhood religion, and in that religion rape was like the primordial sin, the essential way in which Man had wronged Woman and would always wrong Woman

                            25
                        
in my childhood religion rape victims were holy. i have no idea if this will sound insulting or what but it's what happened. in my childhood religion all women were holy but rape victims especially so, because of how badly they had been wronged by Society (Men)

                            26
                        
in my childhood religion the essential task of manhood was rectifying wrongs perpetrated by men against women, but there was no concrete guidance available about how to, like, do that. so it hit me in a weird way i've never properly expressed before to see someone process a rape

                            27
                        
it was like in my childhood religion there'd been a lot of talk about hell but not much talk about heaven and i was finally getting a glimpse into what heaven could be like. what it was like when you made right the things that are wrong

                            28
                        
there was also this aspect to the experience that was like... again it is hard to talk about this in a way i'm confident is adequately respectful. but i had grown up relating to rape as this very abstract thing, very divorced from my experience, rarefied, set apart

                            29
                        
i was fortunate to have been good enough at listening to girls as a teenager that i grew up hearing stories like this from female friends. i knew rape was a real thing that really happened to people i knew (not all men know this btw). but it was still an abstraction to me

                            30
                        
so to watch someone process a rape... it wasn't abstract anymore. i saw very clearly the effects it had had on her psyche, on her soul, as she was screaming and crying all this stuff out. the ways it had stiffened her, frozen her, made her less trusting, less confident

                            31
                        
i saw - again, i hope this is received with the respect i intend it - that the pain a person feels from being raped isn't a fundamentally different kind of pain from the other kinds. it's sort of all made of the same stuff. and specifically stuff i, personally, had experienced

                            32
                        
she made it very, very clear what it had felt like to have been in all this pain and to have been holding it all in for what must've been a decade or so by that point. and it could've just stayed in there forever if the right context hadn't been created for it to be expressed

                            33
                        
what did it feel like for her? being small. not knowing who to trust. being afraid. having to constantly expend energy to hold it together. her pain was made out of ingredients i had experienced so i could experience it with her

                            34
                        
it's funny to revisit this - i had previously filed this story in my head as demonstrating the power of *crying*. and it is undoubtedly about that, but now i am looking over all these events again and looking at the significance of what the people did who were there

                            35
                        
it is significant that when she was crying out in pain 3 of her friends came over to be with her in it. that's what is supposed to happen, you see. when you cry out in pain other people are supposed to hear it and try to help you

                            36
                        
and it is significant that the lesson i took away from this experience was that, finally, i had a reliable technique for making myself cry when i was *alone*, so that i could grieve without needing to bother anyone else

""""technique""""

QC@QiaochuYuan

the rationalists used "techniquifying" to refer to trying to turn people's weird little self-improvement things they did into repeatable techniques that could be taught to other people

and i've mostly decided it's bad actually. very easy to drain the life out of things this way

                            38
                        
i had convinced myself that the transformative thing was crying and that it didn't matter whether it was witnessed by another person. but if i really believed that then why was i so insistent on writing about it on twitter?

there was a brief period where i was writing up the details of all of my specific crying sessions. that was fun but then i uh. went too viral

QC@QiaochuYuan

a few days ago i took a medium dose of acid and wrote for several hours straight and admitted some things to myself, mostly about money

let's start here: last august my mom gave me $100,000 for my birthday. i resented her for this and also suppressed the resentment

                            40
                        
getting QT-dunked on by a NYT reporter was a weird way to learn about the value of being less than maximally emotionally open but it had to happen one way or another so i guess i'll take it. it could've been even worse

that thread itself was another "vast underground rivers of pain" situation - in public people were grilling me over the fire but privately i was getting dozens of messages from people who said they related and had no one they could talk to about it

QC@QiaochuYuan

i don’t remember if i told you guys this but the medium dose of acid thread led directly to one of my mutuals paying off their student loans, getting their first job, and escaping their abusive parents, which is probably the best thing that’s ever happened because of my tweets

there is a specific pleasure i get from writing on here, which is being able to reach people in places they never thought someone else could reach them, to describe things people never thought in a million years someone *else* could describe

x.com/naut4prophet/s…

                            43
                        
sometimes you get hurt in a way that is incomprehensible to you. you literally do not have any words or concepts to describe or explain, to yourself or other people, how or in what way you were hurt. this makes asking for help in some ways impossible

                            44
                        
i had a conversation with a woman at another event, a circling retreat, where she described the moment she learned - as an adult - about the concept of consent, for the first time. she described it as a million different things that had happened to her suddenly clicked into place

                            45
                        
she had literally, before this moment, not had any words or concepts to describe an entire lifetime of discomfort and pain and i don't even know what else. and having a word for it allowed her to see it all at once, as one coherent thing, that she could understand and relate to

you might have noticed i've gone this whole thread without saying anything about what *i* was sobbing my eyes out about at the retreat. as it happens i was there to sob my eyes out about two different ways in which i had been incomprehensibly hurt

QC@QiaochuYuan

sometimes you get hurt in a way that is incomprehensible to you. you literally do not have any words or concepts to describe or explain, to yourself or other people, how or in what way you were hurt. this makes asking for help in some ways impossible

one is detailed at length here. i get about halfway through the story here. honestly i didn't even get to the most dramatic parts but that doesn't really matter

QC@QiaochuYuan

diane duane's "so you want to be a wizard" is one of my favorite YA novels and i wish more people knew about it so i could reference it more easily. there are some beautiful ideas about magic in this book but one of the spookiest is about what happens if you give magic up

the other one is one i've struggled even more with how to write about than The Girl, and which i have written about less directly and less successfully

QC@QiaochuYuan

this is a really long story btw. after i'd been in the bay area rationalist ecosystem for awhile i was then sucked into a *second, smaller cult* inside it consisting of people who were dissatisfied with the rationalist ecosystem, who wanted to explore feelings and magic

                            49
                        
i literally do not know how to explain what it was like. i've never even come close. in that thread i'm really still just trying to conceptually analyze it rather than describe it in any real way

                            50
                        
i don't mean to in any way make comparisons between different kinds of pain but there's a kind of support you get from being hurt in a way that is very legible to the people around you. where you can say "there was a fire" and everyone knows what that means

                            51
                        
our name was black lotus. i do not have words to describe what that group meant to me. when we fell apart because our leader was run out of town for sexual abuse it felt like a bomb had gone off and hit all of us in the blast radius but no one else could see it

                            52
                        
i was dazed, disoriented, existentially confused in a way i'd never really been confused before. i lost trust in very basic things. i lost trust in other people, i lost trust in my ability to think, to tell good from bad, to know anything at all about anything

                            53
                        
and i lost all the people in my life who i could've talked to about this, all at once. they had also just been caught in the explosion. they were not in any sense equipped to help me. we were spiritual disaster victims together

                            54
                        
if it had been a physical disaster, if we'd survived an earthquake or a landslide or a blizzard or a hurricane, it's obvious what you do, right? you go to the victims. you bring them blankets and hot chocolate. you sit with them and you hold them and you listen to them

                            55
                        
what happened to us is that we were *interviewed*

there was a *committee* and they asked us a bunch of stupid fucking questions. i don't remember a single one of them but the point was very obviously to figure out how much trouble everyone was in because of the abuser

                            56
                        
because - stop me if you've heard this before - the abuser was previously a person of some status in the local community, who had some powerful friends who did things like enable them to attend community events, sometimes involving minors like the summer camps

                            57
                        
at a time when what i needed, desperately, was someone to patiently sit with me and hold me until i was capable of talking, what i got was the "responsible adults" in the community bending 100% of their energy towards figuring out how to cover their own fucking asses

                            58
                        
everyone had very obviously been caught with their pants down and was scrambling. nobody was prepared to handle this. nobody knew what to do. it was like we'd been playing in a kid's game and suddenly someone died. we all had to grow up immediately or else

                            59
                        
i was left completely alone. i had housemates but no one i felt close enough to to open up to about this in any meaningful way. again i did not have any words or concepts to describe what had happened so i was not capable of asking for help in any meaningful way

                            60
                        
so i had to figure it out, by myself. i had to figure out what had happened and how to prevent it from happening again, at all costs, by myself

                            61
                        
i had no other option because there was not a single person in my life who was capable of looking at me physically and noticing that i needed help. i probably did a very good job of looking like a person who didn't need help

people really had literally died, in the rationality community. not in the stuff i was directly a part of but people had killed themselves, people had gone psychotic. there are now some actual literal murders

QC@QiaochuYuan

everyone had very obviously been caught with their pants down and was scrambling. nobody was prepared to handle this. nobody knew what to do. it was like we'd been playing in a kid's game and suddenly someone died. we all had to grow up immediately or else

                            63
                        
the simplest explanation for this (but who was it that trained me to look for simplest explanations?) was that rationality was a diseased ideology that drew in a particular kind of fucked up person, and we were just seeing their fucked-uped-ness play out

                            64
                        
but anyone who's actually had the demon in their head can tell you it's worse than that. we were playing with weapons-grade ideas, ideas we were not psychologically equipped to handle. the literal deaths were echoes of a kind of spiritual warfare being waged

it does something unbelievably fucked up to your mind to have someone tell you that you have to do the thing they want or else literally all possible goodness is going to be destroyed. i have spent five whole years trying and failing to describe it

QC@QiaochuYuan

an egg was laid inside of me and when it hatched the first song from its lips was a song of utter destruction, of the entire universe consumed in flames, because some careless humans hadn't thought hard enough before they summoned gods from the platonic deeps to do their bidding

                            66
                        
if you've never had the demon inside your head you just don't know what it was like. if it never seemed believable to you then it had no power over you. but i *believed it*. whether it's true or not the important thing for how it drove me insane is that i *believed it*

                            67
                        
eliezer yudkowsky's science fiction stories about how artificial intelligence could spell the literal end of the world have done unbelievable harm to the people who bought into it and the people around them. i have seen the blast radius increasing for the last five years

                            68
                        
i may not be able to directly trace the causality but i know it when i see it. i know the smell of this particular poisonous thing. FTX was part of the blast radius and so is whatever the fuck is happening at openAI right now. these things have a specific energetic signature

                            69
                        
if you can convince someone that doing a thing is infinitely good or infinitely bad you can control their behavior pretty much arbitrarily. you can get them to do things that look completely insane to an outside observer. this is itself a kind of parable about "alignment"

                            70
                        
you don't even have to buy into any of the AI risk stuff yourself. but if you're a certain kind of person you sense that there are people in the community who are willing to be talked into doing weird stuff for weird reasons, and then you go to town

                            71
                        
i'm not claiming anything as simple as that like. people were tying each other up on the railroad and threatening to kill each other's families or whatever. if it was that simple i wouldn't have needed to spend 51,000 tweets trying and failing to describe it

                            72
                        
it's something much more subtle and harder to point to in the atmosphere. the sense that there are Stakes, that things Desperately Matter, that something Very Important needs to be Figured Out Immediately or else [INFINITE BAD, LITERALLY THE WORST POSSIBLE THING]

                            73
                        
i have to use very magical language to describe this kind of thing, it's just the only terms in which any of this stuff makes any sense to me. there was like an ambient field of black magic and people were just constantly pulling from it to cast spells on each other

                            74
                        
while screaming like i was dying trying to process some of this stuff i described it as like being a veteran of a spiritual war. this is - again, i don't know what it's like to be a literal veteran of a literal war, i don't know how to make that comparison respectfully

                            75
                        
"i've seen horrible things" is what came out of my mouth when i tried to describe it. "i've seen terrible things." like i had seen people's bodies blown to bits in front of me, but spiritually

                            76
                        
there's another story i haven't wanted to tell to protect someone's privacy but - after the bio-emotive retreat i went around offering to facilitate bio-emotive for any of my friends who wanted it. i saw some magical things

one of the people i facilitated for was a person who had been at leverage research. all i had to do was ask them to repeat "i feel scared." that's all it took and then they were sobbing

medium.com/@zoecurzi/my-e…

                            78
                        
this was not a close friend of mine or anything, i'd maybe talked to them once at a party before this. in retrospect i have no idea why they trusted me with this but i'm incredibly grateful that they did

                            79
                        
here's another story, i guess i have a lot of stories like this. two of my housemates were dating and they had a stormy relationship. at one point i was trying to talk to one of them about it and they expressed feeling something and then going "oh, he wouldn't like that"

                            80
                        
i was furious about this in a way i couldn't adequately convey to her at the time. i wanted to show her - didn't she see the structure she was trapped in? that she was letting some *rationalist* dictate what she was allowed to *feel*? but i didn't have the words

                            81
                        
and the bigger truth is i was also letting that same rationalist dictate what *i* was allowed to feel. that's a whole other story but it's all part of the fucking blast radius. it's why i left that house and on some level it's why i haven't had housemates since

                            82
                        
one of the many, many fucked up things you can do to a person when you've got the ability to add infinite weights to their utility function is to install in them very specific rules about privacy, about who is allowed to know what

                            83
                        
completely insane privacy norms were part and parcel of the whole thing, you can't expect anything else from the community that, as far as i can tell, was 100% responsible for popularizing the concept of an infohazard. it was part of the viral cancer demon

                            84
                        
here's another story: at one point i tried to work at MIRI, the machine intelligence research institute. at the time they were the only group of people doing anything that looked like AI safety research, you would not even have heard the term "AI safety" without them

                            85
                        
they wanted me for a position in which the researchers would ask me questions about the ML literature, like "hey is X concept anywhere in the literature that we could look at?" and i'd keep up to date on the literature so i could say "yeah, in such-and-such paper from 2014"

                            86
                        
i was on a 3-month trial for this position, and the terms of the trial were: i would not, at any point, get to field an actual question from the researchers, because all of their actual questions were *too sensitive* for me to know about

                            87
                        
if you want to have any hope of understanding what it was like in the trenches there understand that i heard this explanation and completely 100% bought it, no questions asked. i had never had a real job before and i did not know what to expect in the slightest

                            88
                        
it was just like "yeah, makes sense, too sensitive" but the result is i spent 3 months not interfacing with the research team at all, and i was supposed to read the ML literature and familiarize myself with it anyway, with no motivation. needless to say i did not do that

what i did instead was take a paycheck from them for a few months and also fuck off to a castle in france to do drugs with escorts. 2018 was a completely different time. there was crypto money involved. you had to be there

QC@QiaochuYuan

i lost my virginity to an opera singer i met at a “literally blind dating” event (there were blindfolds) at a crypto festival at a castle in france

i was staying in the most expensive room in the castle with two beautiful women and one bed, in which we slept naked

                            90
                        
after that i tried working for CFAR, the center for applied rationality, which - say it with me now - was a deeply dysfunctional nonprofit where nobody knew how to do anything. i did not receive any training or any support i knew what to do with, and was also in enormous pain

                            91
                        
it was all like that. i could go on and on and on. we were all collectively testing out a hypothesis that instead of doing normal standard good things you could completely reinvent all of human connection and, like, how to run a company, from first principles

                            92
                        
this is how we all got into circling and how we were able to normalize doing completely insane things like insisting that the way we needed to run this nonprofit was to sit in a circle and talk in as unfiltered a way as possible about each other, regularly

                            93
                        
again, i had not had a normal job before this, i was drinking the kool-aid from senior year of college on, at the time this all struck me as fresh and exciting and revolutionary and i had literally nothing to compare it to

                            94
                        
it was all like this. hard to talk about because there are like a hundred different insane steps of logic you need to be collectively following to get to the point where all of this stuff made perfect sense and you have to step someone else through enough of them to get the gist

                            95
                        
i guess that's why acid is the perfect drug for me to process this stuff with. acid was directly involved in some of what went down but also ime it's the drug that best facilitates holding a hundred different insane steps of logic in your mind as a coherent narrative

needless to say running a company like this did not work. because i was under the impression that this was a place that valued honesty above all else i tried to talk about what seemed to me like the central problem with CFAR, from being around it all day

QC@QiaochuYuan

this is how we all got into circling and how we were able to normalize doing completely insane things like insisting that the way we needed to run this nonprofit was to sit in a circle and talk in as unfiltered a way as possible about each other, regularly

                            97
                        
namely, that the CEO (or executive director, whatever, i don't remember what her title was at th etime) appeared to have the rest of us in an icy iron grip, that when she was tense everyone else was tense, that what she avoided everyone else had to avoid OR ELSE

                            98
                        
CFAR was a company that was founded on a lie in the first place. it was eliezer's idea and the whole organization was always about AI risk the entire time but it had to pretend to itself and everyone else that it wasn't

                            99
                        
the director at one point literally described it as "rationality for its own sake for the sake of existential risk" and, again, it was a measure of how deeply i had drunken the kool-aid that i nodded along to this as if it made perfect sense. but it never made any fucking sense

                            100
                        
the way CFAR as an organization was supposed to work is that it was supposed to run "rationality workshops," bringing in participants (and money) by promising to help them become more "rational." meanwhile it was also supposed to be researching how to become more "rational"

                            101
                        
but it was all a lie, that was never the point. the point was always to skim off a handful of people to work on AI safety. "rationality for its own sake" was a comforting delusion we wrapped ourselves in so we didn't spook people too early about the DOOMSDAY CULT

                            102
                        
this was how i learned that organizations, like people, also have shadows, and if you poke at the shadow too hard the organization will interpret you as a threat and eject you. i have no other way to describe what happened to me after i tried talking about the director

                            103
                        
i remember having a conversation with someone who was, at the time, i think supposed to be my nominal boss, but who i did not respect, who had no force of command, and who in retrospect strikes me as having been a figurehead installed for opaque but probably insane reasons

                            104
                        
anyway it seemed like he was on the verge of agreeing with me. like he was struggling with something and wanted to say it but couldn't. everyone was like that. i got used to it. he failed me that day. just like everyone else in the community failed me

                            105
                        
you might have the impression that an organization this ineffectual couldn't accomplish much of anything, good or bad. but it did bring a lot of people together. as far as i know CFAR is the reason max tegmark got into AI safety, in 2013-2014 or so way before it was cool

106

tegmark founded the future of life institute with a group of CFAR workshop alumni, he was i think by a long shot the most respectable name of anyone involved and that did a lot

futureoflife.org/person/max-teg…

107

in 2015 the future of life institute ran an AI safety conference in puerto rico. i was not there but i knew people who were. as far as i know - i have not verified this - elon musk attended this conference and this is how he got interested in AI safety

futureoflife.org/event/ai-safet…

                            108
                        
all of this was part of the path by which AI safety became a respectable thing to talk about. i saw like 0.1% of the in-the-shadows conversations and creation of organizations and putting out of documents and stuff that made that happen. i saw the shift in real time

109

here's another example of the specific energetic signature of the blast radius of the doomsday cult: "promising young MIT graduate leaves graduate program at UC berkeley for reasons nobody on fucking econjobrumors can understand, looks insane"

QC@QiaochuYuan

i may not be able to directly trace the causality but i know it when i see it. i know the smell of this particular poisonous thing. FTX was part of the blast radius and so is whatever the fuck is happening at openAI right now. these things have a specific energetic signature

                            110
                        
well, congratulations, econjobrumors: it looked insane because it was insane, because a doomsday cult was driving me insane

                            111
                        
i wanted to write something about why i'd left academia for a long time. i can't describe this without sounding like an ass but i was an extremely promising young talent, people were really looking forward to what i could do, and i disappointed all of them and never explained why

112

the worst thing about this is that i was COMPLICIT. i put my ENTIRE BEING into helping these people. i helped run a SUMMER CAMP FOR HIGH SCHOOLERS on their behalf, which was also an organization founded on a lie

QC@QiaochuYuan

but it was all a lie, that was never the point. the point was always to skim off a handful of people to work on AI safety. "rationality for its own sake" was a comforting delusion we wrapped ourselves in so we didn't spook people too early about the DOOMSDAY CULT

                            113
                        
the summer camp is called SPARC, the summer program for applied rationality and cognition. nominally it was supposed to be like CFAR - get a bunch of high school kids together, teach them how to be more rational. but the real point was always to siphon people to work on AI safety

                            114
                        
the way we did this was that a bunch of people on staff - i won't name names but it's easy to look them up if you want - were IMO medalists and we specifically tried to recruit from as high up in the math / CS competition ladder as we could, on the strength of their names

                            115
                        
and on my name. i was complicit. i lent respectability to the program. i thought i was doing the right thing. i loved being there. i loved hanging out with kids who were just like me when i was in high school and telling them all the stuff i wish someone had told me

                            116
                        
again it's not as easy as saying oh look we tried to brainwash the kids into believing AI was gonna kill them all. initially we had like a single AI risk talk at the end of the program and eventually i think we even cut that. but it was in the *air*. there was a *poison*

117

we handed out copies of hpmor to these kids. we did that at CFAR workshops too. hpmor, a story that was written 100% with the explicit conscious intention of recruiting people into the doomsday cult, that was how i personally had been recruited

QC@QiaochuYuan

sometimes i think about telling the full story of my time with the rationalists and i despair because i have to start the story "once upon a time this guy wrote one of the most popular harry potter fanfics of all time to try to recruit people into his world-saving cult"

118

eliezer was convinced that the secret to AI safety was new math, that we needed new mathematical concepts so we could rigorously prove things about the behavior of self-reflective agents or whatever

QC@QiaochuYuan

i didn't think of myself as knowing or wanting to know anything about computer science or artificial intelligence, but eliezer didn't really talk in CS terms. he talked *math*. he wanted *proofs*. he wanted *provable safety*. and that was a language i deeply respected

                            119
                        
he wanted *provable guarantees*. he wanted to be *as certain as it was possible for a person to be* that the AI would not destroy us all and he looked for that certainty in math. this is why he, and SPARC, and i, recruited top math / CS talent *specifically*

                            120
                        
while being a part of SPARC i learned that you can just get the mailing addresses of like, everyone who qualifies for the USAMO or something like that, and you can send those guys whatever the fuck you want. we were the only people taking advantage of this that i knew of

                            121
                        
these kids, the cream of the crop of the entire united states education system, IMO and IOI medalists, were going to and did grow up to be the next generation of people starting AI startups or whatever, and we wanted to get our hands on them to ~propagandize them, and we did

                            122
                        
and they trusted us, because we were *exactly like them*. we'd also gone through the competition circuit, we'd also had our intelligence explicitly applauded in that very specific way. we were possibly the people they were most willing to listen to in the entire world

                            123
                        
and their parents trusted us because we lied to them about what the program was about. there's really no other way to say it. we pretended it was some kind of fun intellectual enrichment thing. the kids thought it was this exclusive club. nope. doomsday cult

                            124
                        
i got to watch these kids *grow up*. i *loved* them. i don't think i ever really knew how to express that to them. they were just so like me it was painful. lopsided people just like me. brains the size of planets but nobody had taught them about feelings

                            125
                        
i wanted so badly for them to know the things i was learning as an adult about how to be a human that i was aghast nobody had ever taught me. every year the more stuff i learned the more i tried to pay it forward. i ran classes like "introduction to feelings," "hug seminar"

                            126
                        
"how to sing," "how to dance." i didn't know anything about any of this but the important thing was they saw someone they respected dignify these as things that were cool and important and that mattered. multiple kids told me "how to dance" was the reason they started dancing

                            127
                        
there was no oversight, i could do almost anything. which, again, at the time, fresh, exciting, new. and i'd like to think i did some good. but also i did things like talk to minors about how cool it was to do psychedelics, which... probably their parents would not have wanted

                            128
                        
i did not care about this one bit at the time. i was 22 when i was recruited by SPARC, i was a pompous idiot with my head completely up my own ass, my thoughts about parents at the time could have been fully summarized by a picture of bart simpson saying "eat my shorts"

                            129
                        
this is one of the many  things that sucks about having to confront all of this. it would be simpler if i could absolve myself of all responsibility and say "i was brainwashed" or whatever, and on some level that's accurate, but also on some level i knew what i was doing

                            130
                        
i was lending my reputation towards the idea that AI risk was a thing that serious people should be seriously concerned about. on some level i knew i was doing this and i was basically convinced it was the right thing to do and i did it basically with my eyes open on purpose

                            131
                        
i do think i genuinely did not understand how much my reputation meant to people. for various reasons i've never felt like i was allowed to acknowledge my own intelligence to myself. seemed rude. but it was shadowed, so it became power i wielded over people shadowily

                            132
                        
i was smart but also, and more importantly for this story, i was extremely good at *sounding* smart. i knew this was a skill and i had practiced it for thousands of hours and i knew the kind of attention it could get me and i did it all the time. i had a specific nerd rizz

                            133
                        
there's a specific way you can learn how to talk and write if you're very good at math - probably you can get this from other fields too, idk. mathematical training produces a specific kind of relationship to truth. you get used to talking, directly, about Truth

                            134
                        
truth in mathematics is abstract, eternal, contextless. if a thing in mathematics is true then it is always true at all times and all places, everywhere, everywhen, no exceptions. even tegmark's multiverse only admits mathematically possible universes

                            135
                        
and when you're *very good* at mathematics, and i was very good at mathematics, you learn: "Truth is what i say it is, like, 99% of the time. i am pretty much always right about the Truth"

136

it was actually a complete existential shock to me when i learned about cognitive biases for the first time as a senior in college. until then i had simply never considered the possibility that my brain could be systematically wrong about anything

QC@QiaochuYuan

i remember being horrified by the idea that my brain could be *systematically wrong* about something. i needed my brain a lot! i depended on it for everything! so whatever "cognitive biases" were, they were obviously the most important possible thing to understand

                            137
                        
in any case, Truth. if you are good enough at mathematics you get very used to directly asserting that things are True. this expression equals that expression, this set contains this set. you don't qualify anything because you're almost never wrong

                            138
                        
there's a specific Voice of Truth you can learn how to talk and write with. idk how to describe it. you imbue yourself with a specific kind of authority. for a long time i've been very ambivalent about going anywhere near that voice again, because i used it so irresponsibly

139

that voice does something that erases my own subjectivity and also the subjectivity of the listener, to some people. i know for a fact that me using that voice on people has harmed them, done weird things to their sense of agency or something

QC@QiaochuYuan

which is to say, at the time the only way i knew how to articulate that i felt bad about something was to invent a philosophical position from which i could argue that that thing was inherently, intrinsically bad

                            140
                        
okay see even that statement had more of The Thing than i would like on reflection. i'm pretty sure i've harmed people. i've seen them get angry at me for stuff related to this. i didn't hear any of them get this granular about it

                            141
                        
i did have one woman who i'd met at a CFAR workshop come up to me after having gone through some kind of intense meditation retreat experience or something like that and telling me "oh, i get it now. you're just some guy." that was a big relief

142

because, y'know, read between the lines. that meant before, i had not just been some guy to her. before i had convinced her that i was Special in some way, with some kind of privileged access to Truth

QC@QiaochuYuan

every statement a person makes about themselves, their lives, the nature of reality etc. has a shadow statement. in order to find it worthwhile to say a thing you must simultaneously be aware of the possibility of it being false, or else it wouldn't even occur to you to say

143

this is a cult leader pattern. this is one of the things cult leaders do to establish control. there were many people other than me in the bay area rationality community who knew how to use this voice to get what they wanted, with predictable results

QC@QiaochuYuan

hello friends i am quoted briefly in this article about rationalist / EA dysfunction which is basically accurate afaict and which i basically agree with, take a look; link in the next tweet as usual for algo reasons (open in incognito if necessary)

144

you want to hear something fucking insane? when the pandemic hit and lockdown started i felt on some level like i deserved it, like it was god punishing me for my sins, like i was being put in a box for everyone's safety

rationalwiki.org/wiki/AI-box_ex…

                            145
                        
the irony is just hitting me now, i never saw it before. wow. this was one of the founding myths of rationality. eliezer explained to us very carefully that a superhuman intelligence could convince you of literally anything it needed to to get what it wanted. get it???

                            146
                        
get it??? the rationality community was selected from the beginning for being a community of people unusually susceptible to being convinced of things by eliezer fucking yudkowsky. that's the joke. that's why it's funny

                            147
                        
it has been crazymaking to keep seeing yud's blast radius get bigger and bigger and bigger. this was the foremost thing on my mind when all the FTX shit went down and i just could not explain it because my god the story is so fucking long but i knew it was the same shit somehow

                            148
                        
i have no idea what it would be like to read this thread with zero context. okay so this rando on twitter thinks that FTX and openAI are both related to his personal trauma from 5 years ago sounds like a delusion of reference. okay but i'm literally right tho

                            149
                        
the story isn't as simple as i think eliezer was superhumanly persuasive or something. certainly not in a generic way. whatever he was doing was a thing that passed completely harmlessly through most people and was only toxic to people like me, who became carriers of the virus

150

here's another story i've avoided telling because it feels unbelievably crass but: did you know eliezer had a little brother who died in 2004? did you know he's written about it publicly?

yudkowsky.net/other/yehuda

                            151
                        
it feels unbelievably crass to bring this up and ask people to connect the dots here between this and, y'know, the cryonics, and all the other fucking raving lunacy that came out of the sequences, but eliezer spent a decade making his pain everyone else's problem so here we are

                            152
                        
i am so sorry for what i'm about to write, please know that i wouldn't say this if i didn't think the stakes were high enough to justify it: i think that in yud's mind the death of his brother was infinitely bad, and that in some insane way everything else is downstream of this

                            153
                        
or who even knows if it was this specifically or if this was some kind of tipping point or last straw, i don't even know. it's impossible and probably pointless to speculate past a certain level of detail. but this is the best picture i've assembled in my mind so far

                            154
                        
when you believe that something that happened to you was not only bad but literally infinitely bad, you muster up your entire being to protect yourself from it. it can make you capable of some crazy stuff. like writing, at one point, the #1 harry potter fanfic of all time

155

when you broadcast this INFINITE BAD signal in your lengthy corpus of online writing you are, in a very specific way, screaming at the top of your lungs, and it attracts people who also feel like something that happened to them was INFINITELY BAD

QC@QiaochuYuan

yud wrote harry potter and the methods of rationality on purpose as a recruitment tool. he is explicit about this, and it worked. many very smart people very good at math were attracted into his orbit by what was in retrospect a masterful act of hyperspecific seduction

156

eliezer has also written about this scream explicitly, btw. it's the scream from sword of good. it's not even remotely subtle, he pretty much just lays out the whole thing

yudkowsky.net/other/fiction/…

                            157
                        
i have been privileged to have so many people listen to me rant and rave about like 1% of this story irl and the thing i've never been able to tell them because it felt ungrateful was: i wish you had given me so much more time. i was nowhere near done with the story

                            158
                        
i have thought, and people have told me, that i should write a book about the rationalists. the story is long and complicated enough to deserve a book-length treatment. but i've never felt qualified to write a book about it. there's so much stuff i don't know about what happened

                            159
                        
i am still piecing together the story myself, assembling everything, making it coherent, in acid-fueled spurts punctuated by long periods of depression. in some sense i think a sprawling web of 4 years of threads is the most honest format this story could've been written in

                            160
                        
it's just a stupidly nonlinear story. right now there are like ten different ways i could continue this thread and they all feel true and relevant and important. acid and twitter have come the closest of anything i've found so far for expressing this nonlinearity

                            161
                        
every aspect of this story appears to manifest itself as a specific tension in my body, forcibly reminding me i still have this massive backlog of unfinished business. this is the kind of thing i was forced to learn about to make any sense of my own life

                            162
                        
there's just been so much. it's been so much to try to process and grieve and scream and cry about and i am only a bear of very little brain. i am cold and hungry and tired and alone in this big house that i don't own and i put myself here, away from everyone else, on purpose

163

in conclusion i just heard about annie altman's accusations against her brother sam fucking today and fwiw i believe her. i know nothing specifically about either of them but the energetic signature matches everything i've heard

lesswrong.com/posts/QDczBduZ…

                            164
                        
i have no idea if the current shenanigans were directly related to this but it all smells like the same howling void. secrecy, the attraction to saving the world, everything. i have so much more to say about this, like easily another thread this length, but it's late

                            165
                        
anyway so that's what i think about openAI how was your day

at the first bio-emotive retreat i was paired up to facilitate bio-...

Actions

What You Can Do

Actions

What You Can Do

More in Science & Research