Thread

1. wewewedxfgdf · 1791070718 · focus · HN ↗

   Please explain exactly how all humanity could be wiped out.

   It's ridiculous - anyone who thinks about it for a minute or two will realize that its utterly impossible.

   Ordinary people/politicians don't understand AI so they turn off their rational mind and assume there is something super incredible some magical powers that they cannot understand that can destroy all humans.

   Even humans - the real risk to humanity - could not destroy all humans even if they tried. There is no plausible scenario.

   Even climate change and nuclear war and bio weapons - the most damaging mechanisms - would still only get some percentage of the people on earth.

   And if we are talking about Skynet and self replicating robots and Terminators - please, grow up.

   1. Sharlin · 1791070986 · focus · HN ↗

      We know pathogens that are extremely contagious, and we know pathogens that are extremely deadly. We also know toxins that are lethal at nanogram/kg doses. There's no reason to believe that a sufficiently advanced intelligence couldn't come up with a way to combine those traits.

      One plausible scenario is depicted in detail in "If Anyone Builds It, Everyone Dies" (Yudkowsky & Soares 2025), so I refer you to that.

      1. wewewedxfgdf · 1791071092 · focus · HN ↗

         Still too hand wavey - "some super bad virus and AI makes every human on earth infected".

         Unless you can detail exactly how this happens its still complete science fiction.

         1. 0xDEAFBEAD · 1791072151 · focus · HN ↗

            Why are you so insistent that people should post detailed plans for destroying the human race on public fora?

            I don't think it is necessary for the argument to work. Magnus Carlsen can be confident he will beat me at chess without giving a detailed explanation of every move he will make, in advance.

            1. kzrdude · 1791104530 · focus · HN ↗

               I don't think it's about that. It's not about the step by step plan for the murder. It's about - how does "the AI" do it? Do we give it access to our world, or do we give it a body, so that it can take its own physical actions?

               1. PoignardAzur · 1791109768 · focus · HN ↗

                  I find that it's hard to have productive discussions about this, because people move from "We would never ever give AI access to X, nobody would be that stupid" to "Of course everybody should run their AI with --disable-all-sandboxing-around-x, it makes my workflow 5% more efficient" in weeks as soon as there's an economic argument for it.

                  People used to say nobody would be stupid enough to give an AI access to the internet, now OpenAI does massive training runs with unlimited internet access. People used to say nobody would be stupid enough to give AI unlimited access to your own computer, but that's what all the agent runners do by default.

                  AI has access to the world through talking to people, sending messages on the internet, paying people to do stuff, etc. It can send orders to machine shops and have them shipped with the postal service.

                  The "standard" scenario for an AI apocalypse is that an AI with biohacking capabilities sends the blueprints for a virus to a gene-sequencing company or, if you're really optimistic about these companies' security, as chunks to multiple companies before mixing them.

                  That's a scenario where the AI needs to act covertly in one decisive action, though. In more progressive scenarios, as company managers and CEOs get replaced with AIs (of, for regulatory reason, "humans in the loop" who just do everything the AIs tell them to), any AI swarms become able to just... order people to do stuff.

                  Of course humans can refuse orders and organize to reject AI overlords (just like they can unionize against bad human bosses), so this scenario is not an extinction threat if we only have to deal with below-human-level AIs. This is why there is a massive push in AI safety to stop making smarter AIs before we reach the "smarter than humans in every way" stage.

                  1. zozbot234 · 1791111011 · focus · HN ↗

                     > This is why there is a massive push in AI safety to stop making smarter AIs

                     The actual push within so-called "AI safety" culture is to make the existing AI overlords even more centralized and capable, while actively forbidding the development and deployment of any potential locally-controlled competing AIs that might be smart enough to provide meaningful advance warning as to hostile plots from the dominating AI overlord. By your own argument, you should clearly reject "AI safety" as counterproductive.

                     1. pixl97 · 1791138330 · focus · HN ↗

                        I mean the same reason you don't have anti-missile missiles at your house either.

                        The fact is you might have to accept that some things are dangerous.

Unofficial Hacker News client; not affiliated with Y Combinator.