‹ BackHN Continuity

Thread

Cloud Agents Are Inevitable AI Prisons

74 points · 156 comments · nponte

  1. JamesStuff · · focus · HN ↗
    Personification of AI is what’s going to get us in the end.

    I think we need to draw a hard line in the sand over this. An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.

    We can’t blame the chisel for messing up our sculptures, when where just throwing the hammer!

    1. trio8453 · · focus · HN ↗
      The current agents are _not_ like a chisel which just sits there on its own when no one is around. The situation is a bit closer to someone's dog biting a person - you can argue that it's the owner's responsibility, and that's all fine, but using the dog as the subject of a sentence is perfectly appropriate. Same thing with "agents hacked".
      1. unrented7977 · · focus · HN ↗
        Yes, they are. An AI is several hundred trillion ones and zeroes on a disk. It's incapable of doing anything until you intentionally and explicitly start it up and give it a prompt.

        A dog is an independent, conscious, living being with free will. A dog will do what it wants whenever it wants because it has the agency and ability to do so. A pile of weights on disk does not.

        1. trio8453 · · focus · HN ↗
          Why are you reducing the AI to ones and zeros and not reducing the dog to cells, water, proteins etc.?
          1. unrented7977 · · focus · HN ↗
            Sure. A dog is a collection of cells and goo. Cells are self-propelling clockwork built from proteins and molecular machinery.

            A cell by itself has agency and motivation, is capable of autonomous response to stimuli. It's alive and squirms when you poke it. A cell is capable of truly independent action. It can even feed itself and make its own energy to power the clockwork.

            An LLM still is inert silicon until a human intentionally starts it and gives it a prompt. It's fundamentally incapable of action without instruction and incapable of existing without the entirety of the modern internet and electrical infrastructure, and the thousands of unseen human hands keeping the wheels spinning.

            If you want to go deeper, you can make the same arguments about the proteins and molecular machines. Self-replicating molecules in an unbroken chain since the dawn of time. A protein at the base level is a machine capable of independent action. If it bumps into the right molecule, pure mechanics cause it to do something, change shape.

            Ones and zeroes on a hard drive can't do any of that without human intent applied.

            1. Kim_Bruning · · focus · HN ↗
              Oh sure, but you're comparing the wrong states.

              If you rip the DNA out of a cell it isn't going to do much either; if only because most of the real mechanisms are RNA (and proteins), while DNA is 'just' the storage. Encapsulate the DNA back into a cell nucleus with the attendant mechanisms, and suddenly you've got something that runs again.

              Meanwhile, if you encapsulate an LLM into what's called an 'agentic harness', it's also going to be running much more of the time (depending), specially if it has a what's called a heartbeat or goal loop driving it.

              Still not a cell; most definitely not. But the behavior is a wee bit more interesting than some bits on an ssd.

              Actually you probably already know this one: Regular daemons are also just bits on your drive, until they get ```systemctl start``` ed

              (edit: I literally have a llama.cpp systemd unit running right now)

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.