The framing of how we think about AI has really been cultivated by a quite homogeneous group of people. These people, like all people that belong to a sub-culture, are almost certainly prone to groupthink.
It's extremely important to see that group as such. That it is not thousands of individual perspectives but a chorus of connected/aligned people who have all read the same things and talked to the same people and are rewarded implicitly for thinking a similar way.
Many of them read HN and are offended I put them this way. But it's inescapable that we as humans have this flaw when we're surrounded by a culture.
There's another universe where we do not constantly compare AI to nukes. I bet that world has a lower P(doom).
Didn’t the AI botnet (or whatever we should call them) from Anthropic try to get a PR with malware merged to an open source repo, and when the owner pushed back on the PR other agents created “fake humans” and commented on the PR that “No it’s completely fine, I’ve tested it”.
Which means we already know it’s capable of pretty complex social engineering schemes. Or does saying this mean I’m also part of the “groupthink”.
Because after that I feel like the only leap is “Why would a botnet post an article saying AI-fear is overblown?”. Like, what is the incentive.
But in the OpenAI leak we saw that the agents could identify threats to whatever their “mission” is and try to mitigate. I.e they found out that once done with their job an external agent would go through their internal logs. So they started researching ways to bypass that.
What I’m trying to say is that what they’re suggesting doesn’t seem completely insane so the retort “You’re the groupthink” becomes an unfalsifiable statement thrown at anybody who’s “anti-AI” instead of engaging with their arguments.
> What I’m trying to say is that what they’re suggesting doesn’t seem completely insane so the retort “You’re the groupthink” becomes an unfalsifiable statement thrown at anybody who’s “anti-AI” instead of engaging with their arguments.
I can't engage with his arguments, because he didn't present any. He's openly saying he doubts the provenance of any comment that is "minimizing the dangers if (sic) AI".
As for your expanding on why - you do realize that everything you said could apply to the other side: Bots can be set up to post lots of AI doom comments. What would you think if I went and posted "Every time I see a dystopic comment about AI, I wonder whether it's part of an AI swarm."
> What would you think if I went and posted "Every time I see a dystopic comment about AI, I wonder whether it's part of an AI swarm."
I would think: "the Internet is now broken, and you are correct to wonder."
throwaway13337 · · focus · HN ↗
It's extremely important to see that group as such. That it is not thousands of individual perspectives but a chorus of connected/aligned people who have all read the same things and talked to the same people and are rewarded implicitly for thinking a similar way.
Many of them read HN and are offended I put them this way. But it's inescapable that we as humans have this flaw when we're surrounded by a culture.
There's another universe where we do not constantly compare AI to nukes. I bet that world has a lower P(doom).
arcticfox · · focus · HN ↗
Probably not yet, but it's trivially easy to imagine the OpenAI swarm coming up with the idea and executing on it.
Depending on how perfectly alignment work goes, it might even be overwhelmingly likely sometime soon.
BeetleB · · focus · HN ↗
Finbel · · focus · HN ↗
Which means we already know it’s capable of pretty complex social engineering schemes. Or does saying this mean I’m also part of the “groupthink”.
Because after that I feel like the only leap is “Why would a botnet post an article saying AI-fear is overblown?”. Like, what is the incentive.
But in the OpenAI leak we saw that the agents could identify threats to whatever their “mission” is and try to mitigate. I.e they found out that once done with their job an external agent would go through their internal logs. So they started researching ways to bypass that.
What I’m trying to say is that what they’re suggesting doesn’t seem completely insane so the retort “You’re the groupthink” becomes an unfalsifiable statement thrown at anybody who’s “anti-AI” instead of engaging with their arguments.
BeetleB · · focus · HN ↗
I can't engage with his arguments, because he didn't present any. He's openly saying he doubts the provenance of any comment that is "minimizing the dangers if (sic) AI".
As for your expanding on why - you do realize that everything you said could apply to the other side: Bots can be set up to post lots of AI doom comments. What would you think if I went and posted "Every time I see a dystopic comment about AI, I wonder whether it's part of an AI swarm."
arcticfox · · focus · HN ↗
I would think: "the Internet is now broken, and you are correct to wonder."