‹ BackHN Continuity

Thread

Early rogue AI agent activity and attempts to hack found on urlquery.net

267 points · 313 comments · snikolaev

  1. jagraff · · focus · HN ↗
    I don't understand why so many comments here are so confident that this is all marketing, that rogue is just hype, that agents are just simple tools, etc. If a bunch of nuclear engineers were going to the news and saying "Our reactor is dangerously close to a meltdown - we need government intervention now!" would your response be that they're just hyping up boring old power generation technology?
    1. ambicapter · · focus · HN ↗
      You wouldn't be suspicious if the nuclear engineers kept on feeding their reactor and building more powerful reactors while they went crying to the news?
      1. jagraff · · focus · HN ↗
        In the nuclear case, no, because I’d assume they still have a mortgage to pay.

        In the AI case, no, because I think the engineers believe that there is a high probability of enormous upside as well, if it doesn’t kill us all.

        1. ambicapter · · focus · HN ↗
          Hold up, you think its understandable to keep feeding a melting down nuclear reactor if you have a mortgage to pay?
          1. jagraff · · focus · HN ↗
            I think it is understandable to keep doing a job that you think is dangerous or bad for the world because you need money, yes.
            1. ambicapter · · focus · HN ↗
              And is there a point where "bad for the world" = "apocalypse scenario" where you think it's no longer acceptable, or getting a salary is always justification for doing any level of harm?
              1. jagraff · · focus · HN ↗
                Well if we're talking about AI risk, the justification that I've seen from researchers is that while they do believe that there is a risk of apocalypse scenarios, they believe that they are better positioned than anyone else to prevent that, and also they believe that the upside of AI is so great for the world that it is worth the risk.

                Also I'm not describing what I personally think is an acceptable job; just that I understand that sometimes people do jobs that they think are wrong because they need money

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.