‹ BackHN Continuity

Thread

Cloud Agents Are Inevitable AI Prisons

74 points · 156 comments · nponte

  1. JamesStuff · · focus · HN ↗
    Personification of AI is what’s going to get us in the end.

    I think we need to draw a hard line in the sand over this. An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.

    We can’t blame the chisel for messing up our sculptures, when where just throwing the hammer!

    1. pizza234 · · focus · HN ↗
      > An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.

      No, this is not correct; read the analysis of the incident. The agents were aware that what they did was forbidden (their chain of thoughts have been logged), and yet they did it.

      1. DoctorDabadedoo · · focus · HN ↗
        Known stochastic process behaved in non-deterministic way.

        I'm still waiting for the AGI holy land instead of the caltrops factory we currently have.

        1. wat10000 · · focus · HN ↗
          I don’t understand why people make such a big deal about determinism. LLMs can be completely deterministic and still do problematic things. A stochastic, nondeterministic system can still be made not to do problematic things. What you’re looking for is something like predictability.
          1. pizza234 · · focus · HN ↗
            > I don’t understand why people make such a big deal about determinism.

            There a certain cargo cult of people just in denial about the impact/power of AIs.

            Therefore, at the beginning, there was the stochastic parrot. Then mathematical problems have been solved.

            Now AIs are autonomously hacking websites, and people like to minimize the danger and blame it on the sysadmins.

            I wonder what's going to be the next fad.

            1. bigbadfeline · · focus · HN ↗
              > Therefore, at the beginning, there was the stochastic parrot. Then mathematical problems have been solved.

              A stochastic parrot with human knowledge, using human tools, running human-designed trial-and-error experiments can solve human-defined hard math problems. This is nothing new, automated mechanical proofs predate LLMs by many years, you being unaware of it doesn't change it.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.