AI Safety Is Mostly a Sex Cult

I'm back, I missed you all. Since this is an important taxable I cognize excessively overmuch about, here's a 🧵 explaining that AI Safety Is Mostly A Sex Cult. I don't deliberation these group should make policy. (1/?) (alternative title: Time For Some Cult Theory) There has been a awesome effort complete galore years to region AI Safety chat and argumentation from its origins and intelligence center, because its halfway is simply a activity cult started by a instrumentality fabrication author. If you want to beryllium taken earnestly you person to hide that. This is Eliezer Yudkowsky ("Yud"). We spot him present gathering pinch Bernie Sanders to counsel him connected AI Policy and arguing astir AI Policy successful The AI Doc, now connected Netflix.

This is very elder OpenAI worker and unofficial OpenAI spokesperson Roon. Roon's sentiment of Yud is that 1) he is successful and/or moving a activity cult and 2) his opinions astir AI are very important and should beryllium taken seriously.

Yud is the personification who connected Shane Legg and Demis Hassabis to Peter Thiel truthful that they could unafraid backing to commencement DeepMind, which is now Google's AI investigation division. Here's Sam Altman's sentiment of Yud.

This is the schedule for the Future of Life Institute's convention connected AI successful Puerto Rico successful January, 2015. Elon Musk would recovered OpenAI that December. Yudkowsky claims that someway events astatine this convention caused that, and that he's upset astir it, but this seems unlikely.

Dario Amodei, existent CEO of Anthropic, is present successful 2013 identified arsenic a "member of the Effective Altruism community" connected the blog of Yud's non-profit. He argues pinch yud astir the "paperclip maximizer".

Anthropic papers still mention Yud connected occasion, and the full measurement we talk astir AI uses words his non-profit popularized. "AI Alignment" was popularized by Yudkowsky's non-profit, which was the first venue to usage this nonstop phrasing consistently.

It is very difficult to talk AI astatine each without implicitly invoking Yud's ideas aliases the rat/ea scene. Coxon, who precocious near Anthropic and positioned himself arsenic a whistleblower, screenshotted here.

I almost forgot this part: Here's Eliezer Yudkowsky successful Time Magazine successful 2023, advocating successful favour of risking atomic warfare complete his hobby subject.

The feline who tried to firebomb Sam Altman's location cited Yudkowsky arsenic his superior inspiration. I person a signaling of him talking astir this laying astir location if anyone cares, but let's beryllium real, who other is going to ineptly firebomb Sam Altman's house? I committedness I will travel backmost to the "sex cult" part, but that's really the easy portion to explicate because they station astir it constantly. What's difficult is getting past the extensive, analyzable and well-funded wall of denial that this is the main event. This image isn't really a joke, though

Yudkowsky's followers telephone themselves "the rationality community" these days, externally they are called "rationalists", the followers "rationalism". Apparently they deliberation this sounds little cultlike. I personally find the very agelong "we are not a belief explanation" either funny aliases insulting.

Oh, conscionable truthful I person cashed this check: The instrumentality fabrication that launched a 1000 cults is Harry Potter and the Methods of Rationality. It is perchance the highest-impact azygous modern belief text, and its power whitethorn not yet person peaked. Silly timeline. en.wikipedia.org

Harry Potter and the Methods of Rationality - Wikipedia

en.wikipedia.org

Allegedly, "Effective Altruism" is simply a different point and doesn't return marching orders from Yud. This is fundamentally bullshit. The ideas and group heavy overlap, particularly successful San Francisco. I americium besides told that Anthropic is not an Effective Altruist company. This is 100% bullshit. That video pinch Bernie and Yud? Also successful the room: Jeffrey Ladish (Palisade Research, Coefficient funded) and Daniel Kokotajlo (AI 2027 author, funded by Survival and Flourishing Fund). All "effective altruists", not "rationalists", but they each sewage successful connected that together. Where location is immoderate separation betwixt the 2 things, "rationalism" is squad Yud and "effective altruism" is presently squad Dario, and they're competing for attraction and backing successful that aforesaid space. The measurement they do this is highly funny.

Yudkowsky's astir viral idea, the 1 that infects the astir group and spreads the farthest, is that AI, if we proceed to create it, will inevitably 1) execute superintelligence ("RSI"), 2) flight quality power 2) termination everyone ("paperclip them", "doom") All arguments astir AI Safety and AI Alignment are chiefly astir mitigating this nonstop script nether his nonstop terms. Significant modification is comparatively rare. In the wide subculture, this is implicitly what "safety" and "alignment" mean.

People who deliberation they're Effective Altruist and not Rationalist, but who are profoundly concerned pinch "AI Safety", are mostly implicitly aliases explicitly concerned pinch Yudkowsky's basal model for nonaccomplishment of power and doom. This thought seems to beryllium the astir infectious because it activates the aforesaid neurosis that each apocalyptic aliases millenarian cult does. The world is going to end, and only I and my typical group of group successful the cognize tin possibly extremity it. (Also, you should springiness maine money and person activity pinch me.) You tin reason pinch them successful immoderate measurement you for illustration but it hardly matters. It's not, mostly, thing group judge for either logical aliases empirical reasons, and they will simply spell backmost to complaining astir the apocalypse arsenic soon arsenic you extremity paying attention. In personification they are gathered mostly successful various communes successful Berkeley, California. They telephone their communes "group houses", possibly to make it sound little cultish. Most of the specifications of this are what you expect from "a commune successful Berkeley". Everyone is polyamorous, everyone attends the aforesaid events, everyone lives successful group homes together, everyone venerates the aforesaid fewer authors, and everyone is very worried astir AI doom. They are each very contrarian together. Lots and tons of them activity astatine AI companies and non-profits. The safety, reddish teaming and alignment teams look particularly afflicted, because that's the area you spell connected to interest astir and effort to activity against AI doom, which you are judge is astatine slightest mean likely. The crown jewel of this full thing, though, is Lighthaven. It's the erstwhile Rose Garden Inn successful Berkeley, and it was bought by 1 of the rationalist charities successful precocious 2022 pinch SBF's money, which was looted from FTX for them conscionable earlier it collapsed. There are possibly 5 group who could person physically pushed done wiring money to a rationalist kindness from FTX erstwhile it was collapsing, and 1 of them is Leopold Aschenbrenner. Leopold joined 1 of the others, and she's Chief of Staff astatine Anthropic. Leopold is, allegedly, an effective altruist and not a rationalist, but you tin spot really this gets very difficult to support straight. In what twelvemonth were you 1 and not the other? Was your organization, astatine the time, financially tied straight to Yudkowsky aliases no? Lighthaven hosts the section Astral Codex Ten reference group pinch Scott Alexander, a reference group for The Sequences, which is fundamentally the Rationalist Old Testament, a convention for AI group called The Curve, an EA convention called EAGxBerkeley, MATS, aka "Machine Alignment, Transparency, and Security aliases ML Alignment & Theory Scholars"

and SlutCon, organized by Aella. Gretta, pictured here, is Eliezer Yudkowsky's superior partner, and draws a net of astir 200k from his nonprofit arsenic his adjunct while besides holding this arena astatine LightHaven. She has nary anterior qualifications for immoderate occupation astatine his nonprofit

One hopes nary forbidden activity activity happened astatine SlutCon, and the beingness of Eliezer's partner arsenic an organizer is conscionable nosy and quirky. I besides don't cognize that immoderate of the paid orgies, which Gretta besides organizes, ever happened astatine LightHaven. I do statement that Lighthaven seems hopeless for money. Lighthaven's accommodation head is Ronny Hernandez, and he besides moonlights arsenic Aella's manager, and it is unclear if this conscionable intends "guy who handles her funny information subject projects" aliases if it besides intends "pimp", but it astir apt besides intends "pimp". In lawsuit you haven't seen it before, Aella made this sankey chart. She has a rape kink. One of the guys connected the floor plan is astir apt Eliezer Yudkowsky. Aella very aggressively advertises and recruits for her sexy manner and events connected Twitter.

The New York Post hit maine to Nate Soares. nypost.com

AI doomsayer’s chaotic life pinch OnlyFans prima includes day activity parties, utmost kinks and ‘Slutcon’

nypost.com

twitter reactions from EA/rationalist group to the supra communicative are. uh. they're reactions each right.

i'm gonna caput disconnected everyone who thinks this is activity antagonistic by noting: 1) they are mostly moving together astatine a last day nonprofit while doing this, and 2) they enlistee young idealists into this via literal harry potter instrumentality fiction Rationalism arsenic a full is simply a belief movement. Not everyone takes marching orders. The halfway group who mostly unrecorded together and be events and particularly orgies successful Berkeley are a activity cult, and the ones that aren't having immoderate activity are portion of the wider activity and are its recruiting funnel. Any clip you get much than a fewer group who are profoundly concerned pinch AI punishment together, it seems for illustration their earthy shape of statement looks for illustration a cult; rationalism arsenic a full is little of a cult than it is simply a cult factory. Some of these cults person gone very badly. One of them is the Zizians.

Why Are There So Many Rationalist Cults?

asteriskmag.com

Somewhere, immoderate rationalist-brained personification is reference this and saying I'm a "low decoupler", and that whether they tally a activity cult has thing to do pinch whether they're correct. This is existent enough; but the activity cult is reasonable grounds that their power isn't prevalent because they're right, and that they would person ulterior motives for continuing to preach that the world is coming to an extremity moreover if grounds showed otherwise. If you've sewage a activity cult and a bully compound that was fixed to you for free pinch stolen money because you preached the gospel of punishment really well, really could you extremity preaching it? If elaborating upon last day prophesies is worthy money -- and it is -- you will get much of them. This each astir precocious came together complete the weekend, and they sewage Dario Amodei and Sam Altman and Elon Musk and Demis Hassabis each onto the aforesaid page to opportunity they wanted to WAIVE ANTI-TRUST LAWS truthful that they could "pace", aliases slow down, research. I consciousness astir this astir the aforesaid measurement I would consciousness astir each the pharmaceutical companies getting together to opportunity they wanted to waive anti-trust laws truthful they could shape a cartel to prohibition aesculapian research: these group are the enemies of humanity. Also: Much of AI investigation IS aesculapian research. They are very virtually proposing to extremity a beautiful bully chunk of aesculapian investigation successful its tracks truthful that they don't person to fearfulness the paperclipper. There are galore regulations I would beryllium successful favour of, but they only ever want to shape a cartel aliases a world authorities against the paperclipper. No different manufacture has ever framed immoderate regularisation successful position of "stop", "pause", aliases "slow down". Those are insane ways to see immoderate shape of policy. Policy should forestall bad outcomes and promote bully ones, it is ne'er astir immoderate circumstantial speed. This manufacture frames its regulations successful this measurement because it has, astatine its core, a millenarian decease cult that worships and fears AI arsenic the origin of the extremity of the world, that feels compelled to create AI, and that astatine slightest somewhat wants to destruct AI arsenic a field. AI argumentation is only considered successful these position because of the immense intelligence power of Yudkowsky specifically, and because, downstream of him, a neurotic aliases belief obsession pinch AI punishment is the superior intends by which group prosecute pinch these concepts. I will effort to station an Abridged Rat History astatine immoderate constituent but this is stupid long, truthful that will beryllium a different thread shareable thread bsky.app