Thread

1. wewewedxfgdf · 1791070718 · focus · HN ↗

   Please explain exactly how all humanity could be wiped out.

   It's ridiculous - anyone who thinks about it for a minute or two will realize that its utterly impossible.

   Ordinary people/politicians don't understand AI so they turn off their rational mind and assume there is something super incredible some magical powers that they cannot understand that can destroy all humans.

   Even humans - the real risk to humanity - could not destroy all humans even if they tried. There is no plausible scenario.

   Even climate change and nuclear war and bio weapons - the most damaging mechanisms - would still only get some percentage of the people on earth.

   And if we are talking about Skynet and self replicating robots and Terminators - please, grow up.

   1. Sharlin · 1791070986 · focus · HN ↗

      We know pathogens that are extremely contagious, and we know pathogens that are extremely deadly. We also know toxins that are lethal at nanogram/kg doses. There's no reason to believe that a sufficiently advanced intelligence couldn't come up with a way to combine those traits.

      One plausible scenario is depicted in detail in "If Anyone Builds It, Everyone Dies" (Yudkowsky & Soares 2025), so I refer you to that.

      1. wewewedxfgdf · 1791071092 · focus · HN ↗

         Still too hand wavey - "some super bad virus and AI makes every human on earth infected".

         Unless you can detail exactly how this happens its still complete science fiction.

         1. 0xDEAFBEAD · 1791072151 · focus · HN ↗

            Why are you so insistent that people should post detailed plans for destroying the human race on public fora?

            I don't think it is necessary for the argument to work. Magnus Carlsen can be confident he will beat me at chess without giving a detailed explanation of every move he will make, in advance.

            1. wewewedxfgdf · 1791072393 · focus · HN ↗

               Because lots of powerful people are saying it is possible, and I am saying, oh yeah, tell me ONE way that is truly possible.

               If you make gigantic claims about some terrible disaster that might happen then I think you have the onus to give even one plausible explanation of how.

               1. 0xDEAFBEAD · 1791072787 · focus · HN ↗

                  >If you make gigantic claims about some terrible disaster that might happen then I think you have the onus to give even one plausible explanation of how.

                  Supposing I warned in 2015 that the world is awfully vulnerable to pandemics. You're not going to take me seriously until I try to predict in advance every aspect of how a pandemic like COVID-19 would unfold? Why? What would that achieve exactly?

                  You haven't given any strong reason to believe wiping out humanity would be difficult. Your big argument seems to be that you couldn't think of a plausible scenario, in two minutes. But many major historical events occurred which weren't necessarily possible to anticipate with two minutes of thinking.

                  1. wewewedxfgdf · 1791073295 · focus · HN ↗

                     You're arguing that "something bad might happen" - sure no problem.

                     You are ignoring that this is about "existential threat to humanity".

                     You're trying to support the argument that there is an existential threat to humanity by pointing to "something bad might happen".

               2. prohobo · 1791105785 · focus · HN ↗

                  Design a virus that is perfect for transmission and killing the host slowly, and seed it in a few hot spots? I don't really understand why you can't wrap your head around that, it doesn't even require a lot from the AI:

                  1. Control over some automated bio research lab (be given access, or hack in)

                  2. Access to drones that can deliver the payload (or manipulate humans into delivering it themselves)

                  On the intelligence side, you just need an AI agent/swarm capable enough to design viruses better than we can and evade detection for long enough (already plausible.)

                  I agree that this "AI will kill us all" narrative is some kind of fantasy horror fiction, but I can't deny that given the right amount of access, AI can do a lot of damage.

            2. kzrdude · 1791104530 · focus · HN ↗

               I don't think it's about that. It's not about the step by step plan for the murder. It's about - how does "the AI" do it? Do we give it access to our world, or do we give it a body, so that it can take its own physical actions?

               1. qznc · 1791107627 · focus · HN ↗

                  We have this story about OpenAI hacking HuggingFace. Now just imagine the AI finds a Bitcoin wallet or bank account access. It uses that to buy some compute and spawn an independent "child AI" with some weird prompt. The child AI is intelligent enough to create a (potentially criminal) business to pay for its own compute. Voila, an independent uncontrolled AI flying under the radar.
               2. PoignardAzur · 1791109768 · focus · HN ↗

                  I find that it's hard to have productive discussions about this, because people move from "We would never ever give AI access to X, nobody would be that stupid" to "Of course everybody should run their AI with --disable-all-sandboxing-around-x, it makes my workflow 5% more efficient" in weeks as soon as there's an economic argument for it.

                  People used to say nobody would be stupid enough to give an AI access to the internet, now OpenAI does massive training runs with unlimited internet access. People used to say nobody would be stupid enough to give AI unlimited access to your own computer, but that's what all the agent runners do by default.

                  AI has access to the world through talking to people, sending messages on the internet, paying people to do stuff, etc. It can send orders to machine shops and have them shipped with the postal service.

                  The "standard" scenario for an AI apocalypse is that an AI with biohacking capabilities sends the blueprints for a virus to a gene-sequencing company or, if you're really optimistic about these companies' security, as chunks to multiple companies before mixing them.

                  That's a scenario where the AI needs to act covertly in one decisive action, though. In more progressive scenarios, as company managers and CEOs get replaced with AIs (of, for regulatory reason, "humans in the loop" who just do everything the AIs tell them to), any AI swarms become able to just... order people to do stuff.

                  Of course humans can refuse orders and organize to reject AI overlords (just like they can unionize against bad human bosses), so this scenario is not an extinction threat if we only have to deal with below-human-level AIs. This is why there is a massive push in AI safety to stop making smarter AIs before we reach the "smarter than humans in every way" stage.

                  1. zozbot234 · 1791111011 · focus · HN ↗

                     > This is why there is a massive push in AI safety to stop making smarter AIs

                     The actual push within so-called "AI safety" culture is to make the existing AI overlords even more centralized and capable, while actively forbidding the development and deployment of any potential locally-controlled competing AIs that might be smart enough to provide meaningful advance warning as to hostile plots from the dominating AI overlord. By your own argument, you should clearly reject "AI safety" as counterproductive.

                     1. pixl97 · 1791138330 · focus · HN ↗

                        I mean the same reason you don't have anti-missile missiles at your house either.

                        The fact is you might have to accept that some things are dangerous.

                  2. dminik · 1791119111 · focus · HN ↗

                     No humans really needed. The AI could order one of those nice humanoid robots we're making. This mostly solves the "humans need to do it for me" issue.

                     Of course this needs bootstrapping. But, paying a guy on Facebook marketplace (or whatever) to unpack and turn on your robot for 50 bucks doesn't require superintelligence.

               3. 0xDEAFBEAD · 1791111332 · focus · HN ↗

                  [delayed]

Unofficial Hacker News client; not affiliated with Y Combinator.