Thread

1. jagraff · 1790253876 · focus · HN ↗

   I don't understand why so many comments here are so confident that this is all marketing, that rogue is just hype, that agents are just simple tools, etc. If a bunch of nuclear engineers were going to the news and saying "Our reactor is dangerously close to a meltdown - we need government intervention now!" would your response be that they're just hyping up boring old power generation technology?

   1. ambicapter · 1790287428 · focus · HN ↗

      You wouldn't be suspicious if the nuclear engineers kept on feeding their reactor and building more powerful reactors while they went crying to the news?

      1. jagraff · 1790291637 · focus · HN ↗

         In the nuclear case, no, because I’d assume they still have a mortgage to pay.

         In the AI case, no, because I think the engineers believe that there is a high probability of enormous upside as well, if it doesn’t kill us all.

         1. bagacrap · 1790307653 · focus · HN ↗

            upside for whom?

            1. jagraff · 1790339192 · focus · HN ↗

               Depending on who you ask, either for all of humanity (because they will allow us to do things like cure most cancers, develop useful technology, etc) or for the people building them (because they will make them rich enough to buy galaxies) or both

               1. bagacrap · 1790396389 · focus · HN ↗

                  The original issue was whether these people are trustworthy. Someone whose main motivation is self enrichment is trustworthy in your eyes?

                  1. jagraff · 1790427370 · focus · HN ↗

                     I’m not saying they are necessarily trustworthy - I’m just saying that we ought to take their claims seriously and not dismiss them out of hand because they have a financial interest.

                     I’ll admit I also just don’t understand the idea that someone saying their product is dangerous and could kill all of humanity is doing marketing - I just really don’t understand at all how that could be a marketing strategy. So I tend to believe they are saying it is dangerous because they think it’s true. But maybe I’m just naive.

                     1. bagacrap · 1790430697 · focus · HN ↗

                        It's telling investors "this is so powerful we could take over the world, better give us your capital."

                        It's telling governments, "this is so powerful it could cause grave harm to the consumer, please regulate so we can focus on extracting value from the economy instead of this expensive arms race."

                        I warrant it is quite possible that some in this space are worried they will overshoot the mark. The perfect outcome for them, of course, being fabulous wealth and a society that is otherwise not too much different from today's. What a shame it would be to be super rich and dead!

                        It does kind of feel like they are trying to ransom the future of humanity. These rogue AI stories are like getting a dismembered finger in the mail.

Unofficial Hacker News client; not affiliated with Y Combinator.