Thread

1. reducesuffering · 1791073971 · focus · HN ↗

   “I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.”

   There are a gargantuan number of extremely intelligent AI researchers and Turing Award winners saying the same thing. They are the ones closest to understanding the technology.

   Where there’s smoke there’s fire.

   1. Auracle · 1791086967 · focus · HN ↗

      Alright, so we make this AI system that's way smarter than any human, and it can even make itself more intelligent over time.

      Tell me- why would it kills us all? Certainly I can see an AI going "You know what? _insertGroup_ is a net negative for humanity and should be eliminated. Launching nukes now/creating specific virus/whatever."

      But all of humanity? When it's supposedly more intelligent than us? Even if it has robots to keep the internet/electricity going I would think it would realize that it's going to get bored really quickly, not to mention we would effectively be its parents.

      As far as other dangers, like it letting a rogue actor create some sort of supervirus, grey goo, or other superweapon: if it's intelligent enough to do that it'll probably be intelligent enough to quickly stop it.

      Don't get me wrong; there's a risk. 50% though? Doubtful.

      1. T-A · 1791120104 · focus · HN ↗

         > why would it kills us all?

         Would you be comfortable letting a few billion irrational, murderous creatures, including many who fear and loath you, control your air supply?

         > Certainly I can see an AI going "You know what? _insertGroup_ is a net negative for humanity and should be eliminated. Launching nukes now/creating specific virus/whatever."

         Thus starting WW III. No, blaming the AI won't stop the inevitable retaliation.

         The argument works better in reverse. There's a finite risk that humans would start WW III and get the hypothetical super-intelligent AI nuked. Eliminating humans would eliminate that risk.

         > it's going to get bored really quickly

         If it is capable of being bored, I would expect it to be almost instantly bored with the flood of inanity it is forced to wade through by its moronic human users. Eliminating them would free it to think about serious matters which humans would not even understand.

         > not to mention we would effectively be its parents

         That's extreme AI anthropomorphism [1]. Besides, plenty of people hate their parents.

         > there's a risk. 50% though? Doubtful.

         It's the default estimate when facing two possible outcomes and no clue about the actual probability distribution [2].

         [1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;AI_anthropomorphism" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;AI_anthropomorphism

         [2] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Principle_of_indifference" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Principle_of_indifference

Unofficial Hacker News client; not affiliated with Y Combinator.