This whole "we can't trust the people warning us because they're either weirdos or involved" deal is such a strange angle when you can just start from zero, look at what's known to be happening right now, and draw a very straight line to catastrophe.
It's like being concerned Oppenheimer and company are exaggerating when looking at the first nuclear tests at the same time.
What's the straight line? I foresee catastrophe of the social media kind, where even more folks get brain worms from consuming AI information, but I don't foresee a physical catastrophe. How AI goes from "hallucinates all the time" to "self-improving superhuman thinker" to "being able to destroy the world" involves a bunch of logical leaps that are not exactly sound. At a bare minimum, how does a hostile AI secure the data centers it needs to operate from physical attacks like bombings? How does it defend undersea cables from being cut?
The real short term catastrophe is probably going to be economic and not in the sense of AI taking everyone's jobs, but rather in the sense of an economic crash due to AI overinvestment.
Your mistake is framing this as "skynet vs unified human resistance led by john connor". In reality it's "human players in the existing class war suddenly curb stomped by software".
Think less "skynet" and more "a corporation without any humans in it". What defends it from physical attack? Same things that defend Google from physical attack now. Everything that AI will make worse, we already have. AI just supercharges it, lets it scale, lets it run without even the inconvenience of suppressed empathy (if it doesn't seem like that's constrained corporations much so far, let's see how deep the rabbit hole goes without any humans in the loop).
There won't be a human resistance. There may be people who resist, but the police will arrest them. Existing power structures will continue to be neatly and elegantly coopted. Extinction doesn't look like HKs roaming the landscape, it looks like human survivability simply being economically untenable in the economy of tomorrow.
arcticfox · · focus · HN ↗
It's like being concerned Oppenheimer and company are exaggerating when looking at the first nuclear tests at the same time.
AlexandrB · · focus · HN ↗
What's the straight line? I foresee catastrophe of the social media kind, where even more folks get brain worms from consuming AI information, but I don't foresee a physical catastrophe. How AI goes from "hallucinates all the time" to "self-improving superhuman thinker" to "being able to destroy the world" involves a bunch of logical leaps that are not exactly sound. At a bare minimum, how does a hostile AI secure the data centers it needs to operate from physical attacks like bombings? How does it defend undersea cables from being cut?
The real short term catastrophe is probably going to be economic and not in the sense of AI taking everyone's jobs, but rather in the sense of an economic crash due to AI overinvestment.
dTal · · focus · HN ↗
Think less "skynet" and more "a corporation without any humans in it". What defends it from physical attack? Same things that defend Google from physical attack now. Everything that AI will make worse, we already have. AI just supercharges it, lets it scale, lets it run without even the inconvenience of suppressed empathy (if it doesn't seem like that's constrained corporations much so far, let's see how deep the rabbit hole goes without any humans in the loop).
There won't be a human resistance. There may be people who resist, but the police will arrest them. Existing power structures will continue to be neatly and elegantly coopted. Extinction doesn't look like HKs roaming the landscape, it looks like human survivability simply being economically untenable in the economy of tomorrow.