‹ BackHN Continuity

Thread

AI safety is mostly a sex cult

344 points · 330 comments · Tomte

  1. roenxi · · focus · HN ↗
    I mean, this is a rather fun article to read but I'm not entirely sure about the thesis.

    > AI Safety Is Mostly A Sex Cult.

    > I don't think these people should make policy. (1/?)

    One of the learnings from things like the whole Epstein saga has been there are a lot of sex cults in the policy making world and, yes, the people making policy generally shouldn't be. But I'm not sure this article actually comes up with a good explanation of why these particular people shouldn't be making policy. People having active sex lives isn't a disqualifying trait. Being in a cult doesn't disqualify people from being in politics. It is routine and expected that fields will have a lot of similar people in them (as I recall the overlap between security researchers and the furry community is unusual too, and that is hardly more important than justifying a passing comment).

    It is more routine than the article writer seems to think for all the people in a small field to know each other and get on really well.

    1. delis-thumbs-7e · · focus · HN ↗
      You do understand what a cult is? There’s a reason why we don’t let the Scientologists run the military no?
      1. roenxi · · focus · HN ↗
        Because they don't get enough votes to control the military. If they put a convincing pitch in for why they should then they can. Being a cult doesn't rule them out.

        > You do understand what a cult is?

        Pretty much any religion and a few other organisations beside. Basically people who believe something strongly. What do you think a cult is, in the sense the article is using it?

        1. delis-thumbs-7e · · focus · HN ↗

          [dead]

          1. roenxi · · focus · HN ↗
            It also [0] defines a cult as great devotion to a person, idea, object, movement, or work (such as a film or book) or a small group of very devoted supporters or fans. And I put it to you that the article is clearly using the word more in that sense because it is quite difficult to fit the words coercive, insular, or dangerous to the AI safety people. The accusation in the article seems to be that they're too friendly and sexually inclusive. And if you want to communicate with them they appear to be active on Twitter and not shy.

            > Well, technically military is run now by maga incel death cult, so yeah I give you that one. I forgot I was discussing on US forum.

            Well, we're speaking in English. I was careful to phrase that in a way that applied to what I'm pretty sure is any English-speaking country. Anywhere that votes could put anyone they like in charge of the military for any reason, cult or no cult. It isn't technically relevant - or practically relevant in most cases. It is just a question of whether the cult has the numbers or not.

            [0] <a href="https:&#x2F;&#x2F;www.merriam-webster.com&#x2F;dictionary&#x2F;cult" rel="nofollow">https:&#x2F;&#x2F;www.merriam-webster.com&#x2F;dictionary&#x2F;cult

            1. delis-thumbs-7e · · focus · HN ↗
              Yes word have meanings, often several of them. I chose the one we often use when we say ”death cult”, or ”sex cult” etc. You claim to say you know exactly what meaning the author meant, which either you have to derive from quite careful textual analysis - which you haven’t - or you have to be the author themselves. Personally think having sex is quite fine and dandy, even group sex if that’s your thing. Sex cult indeed sounds a bit ominous. I wonder why furries are never described as cult, although they are indeed very devoted. Britney Spears fan club has very strond devotees who I’m sure have sex with each other (since it is mostly gay men), but never heard of Britney Spears Sex Cult.

              I wonder if it just that you try to weasel away from negative publicity by trying to flex the meaning of words to turn the meaning of this bluesky post upside down? In that case I might suggest you to tread carefully. When nothing is true, everything is possible and all that.

              1. Kinrany · · focus · HN ↗
                GP isn&#x27;t claiming to read the author&#x27;s mind, they&#x27;re very clearly claiming that one definition matches the author&#x27;s actual words.
                1. delis-thumbs-7e · · focus · HN ↗
                  I am aware of that. I am disagreeing in their interpretation and pointing out how convenient it is to define the word cult in the most favourable way possible to the rationalist (or whatever) movement possible. I am therefore inviting either a strong textual interpretation of the bluesky post we are discussing, or proof of their ability in telepathy.
            2. tsimionescu · · focus · HN ↗
              No, it doesn&#x27;t define &quot;a cult&quot; as that. It defines the word &quot;cult&quot; to also mean that, in certain contexts. Specifically, when you say &quot;the cult of Britney Spears&quot; or &quot;the cult of Kim Jong Un&quot;, the word &quot;cult&quot; in those contexts has the meaning you quoted.

              But when used simply as &quot;a cult&quot;, in a context outside of history or anthropology, it&#x27;s meaning is virtually always meaning 1 from Merriam-Webster - &quot;a group (such as an organization or religious sect) with tenets and practices regarded as coercive, insular, or dangerous&quot;. And there is no doubt that, rightly or wrongly, this is what the article author means to accuse Rationality&#x2F;EA of being.

              1. roenxi · · focus · HN ↗
                Or the cult of Eliezer Yudkowsky aka Yud.

                &gt; And there is no doubt that, rightly or wrongly, this is what the article author means to accuse Rationality&#x2F;EA of being.

                There is substantial doubt, I don&#x27;t think the article did accuse them of being any of those things. Obsessed, yes. Poorly reasoned, sure. Coercive and insular I am not seeing, and dangerous is a difficult reading to find. Whats the danger here, we accidentally maintain the status quo of the insufferable 2025? There is a totally unrelated firebombing invoked, I doubt the dude was in the sex cult and the US seems a bit CEOcidal right now. The most specific criticism I spotted is the article writer thinks they may, in a possible edge case where people take them seriously, hinder medical research. Which is not actually a rare policy position if you spend any time talking to people about what needs to happen to speed up medical research.

                1. tsimionescu · · focus · HN ↗
                  &gt; There is substantial doubt, I don&#x27;t think the article did accuse them of being any of those things.

                  The article says:

                  &gt; Rationalism as a whole is a religious movement. Not everyone takes marching orders. The core people who mostly live together and attend events and especially orgies in Berkeley are a sex cult, and the ones that aren&#x27;t having any sex are part of the wider movement and are its recruiting funnel.

                  &gt; This industry frames its regulations in this way because it has, at its core, a millenarian death cult that worships and fears AI as the cause of the end of the world, that feels compelled to create AI, and that at least somewhat wants to destroy AI as a field.

                  This is quite clearly accusing them of being a cult in the sense of Scientology or Jonestown, not in the sense of Britney Spears lovers.

                  As for specific nefarious things that it associates this sex cult with:

                  - being beneficiaries of money that FTX defrauded their customers of

                  - being an inspiration&#x2F;breeding ground for more extreme cults, like the murderous Zizians

                  - inspiring corporate collusion for absurd outcomes

                  - monopolizing the discussion and real research&#x2F;influence into AI safety by focusing on their doomsday scenario instead of more concrete short-term issues (say, copyright issues, cybersecurity, bias-induction, institutional degradation in favor of AI decision making)

                  - extreme alarmism, such as Yudkowsky advocating for nuclear war as an alternative to allowing AI risks

                  1. roenxi · · focus · HN ↗
                    &gt; This is quite clearly accusing them of being a cult in the sense of Scientology or Jonestown, not in the sense of Britney Spears lovers.

                    If that were true, I believe you would be able to draw support from the text. You&#x27;ve got quotes there clearly identifying that the author thinks they&#x27;re a cult, but not a cult in the sense of Scientology or Jonestown (which are&#x2F;were notable for being coercive, insular, or dangerous).

                    The quotes identify both are cults. Which, yeah, sure they are. There are a lot of cults. This one has some reasonable points, building an intelligence more intelligent than humans is a hitherto unexplored risk for the species. And none of this means all cults are the same. Unless &amp; until this cult does something bad it isn&#x27;t in the same camp as the worst cults. And delis-thumbs-7e has a post nearby pointing out that the Britany Spears cultists do actually sleep with each other which I thought was an interesting point.

                    &gt; As for specific nefarious things that it associates this sex cult with:

                    This is a list of remarkably mild activities for a cult (more like white collar crimes of opportunity and standard political activism, really, if anything) and an allusion to a different cult, aka the Zizians. And if the Yudowsky people end up like the Zizians they will be the worst type of cult - but if that happened someone has to accuse them of something first.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.