Thread

1. cedws · 1790611744 · focus · HN ↗

   A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.

   1. notatoad · 1790642667 · focus · HN ↗

      >You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access.

      only as long as you're trying to replace a human's job. because human jobs are structured to do a wide variety of things.

      a useful agent needs a wide variety of inputs, and one single restricted action it can take. it doesn't need permission to do everything, it need permission to do the tiniest possible useful thing it can do, and nothing else.

      1. pixl97 · 1790653102 · focus · HN ↗

         The most useful agents will be a general intelligence which by default means it has a massive number of actions it can possibly take, and a lot of those potential actions are doing things like breaking permission.

         1. kennywinker · 1790662647 · focus · HN ↗

            This is a prediction about the future. It’s not a true fact about the world. For example, software like Jev is betting there is big money in not-very-intelligent intelligence.

            Even very llm-pilled coders i know sometimes back away from the “smartest” models, since they aren’t always better at the job at hand, and definitely not when you account for cost.

            Based on my experience with running models locally, there is a threshold of intelligence required to be useful. But it’s possible there is also a ceiling where smarter isn’t necessarily better. If you ask a 4B parameter model to fix a bug, it might e.g. fix the bug but fail to fix a compilation error created by the fix. If you ask a frontier model, it might fix the bug, re-write your unit tests, and update the readme. Maybe you wanted those things but maybe you didn’t. “Smarter” is often shorthand for more proactive, and guessing more about your intent. Which is great when it gets it right, and annoying when it gets it wrong.

            I suspect smaller models, tuned to a specific task, will do a VAST majority of the llm jobs. High capability huge models will be what humans want to interact with, the bare minimum that gets the job done will be everything else.

            1. goolz · 1790665440 · focus · HN ↗

               Very much agree with this sentiment. I imagine a future where tons of small tasks are handled by just-good-enough intelligence. And I can run them on my own hardware.

Unofficial Hacker News client; not affiliated with Y Combinator.