Thread

Nvidia wants to put a watchdog chip next to every AI agent

230 points · 299 comments · jonbaer

cedws · 1790611744 · focus · HN ↗
A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.
1. nicce · 1790612835 · focus · HN ↗
  > Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.
  Productivity gains are still enormous compared to what we used to do before agents. But, I know that people don't want to stop there.
  1. paimapi · 1790613593 · focus · HN ↗
    right, the solution here is not a hyper-capitalist race-to-the-bottom-of-devaluing-labor. it's recognizing discretion and diligence are things still required for work to be of a certain quality
  2. egeozcan · 1790614752 · focus · HN ↗
    Humans can also be tricked by the agents.
    Humans can be tricked by humans too but humans care about their reputation in their communities, and at least fear from punishment.
    gus_massa · 1790629573 · focus · HN ↗
    Computer says no has been a problem for decades. The human can blame the computer for the errors following it, but must assume the consecuences if they override the decision.
    [deleted] · 1790630851 · focus · HN ↗
    [deleted]
    wavewrangler · 1790665515 · focus · HN ↗
    "I don't want the details"
    intended · 1790666285 · focus · HN ↗
    Individual responsibility is meaningless when talking about a system and economy level change.
    Unless something is in the structure that makes individual choice and responsibility a meaningful source of friction and reduced velocity, it has no real impact on how AI is being used.
  3. autoexec · 1790647966 · focus · HN ↗
    > Productivity gains are still enormous
    Depends on who you ask I guess
    <a href="https://www.theregister.com/software/2026/01/15/ai-is-everywhere-but-nowhere-in-recent-productivity-data/4845104" rel="nofollow">https://www.theregister.com/software/2026/01/15/ai-is-everyw...
    <a href="https://www.zdnet.com/article/workslop-can-kill-your-productivity-heres-how-to-turn-ai-into-a-competitive-advantage/" rel="nofollow">https://www.zdnet.com/article/workslop-can-kill-your-product...
    <a href="https://fortune.com/2026/08/22/executives-ai-productivity-layoffs-study/" rel="nofollow">https://fortune.com/2026/08/22/executives-ai-productivity-la...
    tniemi · 1790674538 · focus · HN ↗
    It's probably just productivity paradox v2. <a href="https://en.wikipedia.org/wiki/Productivity_paradox" rel="nofollow">https://en.wikipedia.org/wiki/Productivity_paradox
    It takes time for decades old ways to change.
  4. realusername · 1790662909 · focus · HN ↗
    > Productivity gains are still enormous compared to what we used to do before agents.
    My own productivity yes but if I step back and look at a company scale, the productivity gains has been negative for our company as a data point.
    We are now shipping less and with a lower quality.
    8n4vidtmkvmk · 1790665192 · focus · HN ↗
    How are you shipping less?
    I believe the lower quality but is quality so bad you are afraid to ship it now?
    krageon · 1790666636 · focus · HN ↗
    You cannot ship things that don't work unless you work for Microsoft or Oracle or I guess IBM
    hdjrudni · 1790828210 · focus · HN ↗
    I guess, but why are you even submitting/merging code that doesn't work? Like how does it even get that far that you're blocking other people from getting stuff done?
    I guess I won't deny that we've had more build breakages than usual over the past couple months but they get resolved super quick. We don't hesitate to roll broken code back if something manages to slip in.
    realusername · 1790667362 · focus · HN ↗
    AI also had a negative impact on the CI and on time spent to review code & documents so because of that, we're also shipping less
    intended · 1790666858 · focus · HN ↗
    V/G, the ration of verification and generation capacity is borked in AI using firms now.
    It’s not an issue of only more generation, it’s an issue of how much generation outstrips capacity to verify generated content.
    Unlike spam, you can’t filter out and bin the stuff a colleague is sending you.
    So individual productivity is up, while the costs of checking and processing generated content shifted to the rest of the org.
  5. dgellow · 1790665659 · focus · HN ↗
    So much productivity gain, and yet still zero proof of positive contribution to companies ROI. Unless you’re yourself reselling AI of course
  6. SkyBelow · 1790689749 · focus · HN ↗
    Are they? How many people claiming productivity gains are actually being a human in a loop and reviewing and understanding every code change and every line of code ran?
    2 years ago I saw it, back before agents were really a thing. But I'm not sure that was an enormous productivity improvement, especially compared to agents today. As for today, everyone I talk to is some level of blindly trusting what the agent is doing or not using agents. I haven't met people in the middle ground and suspect that they are rare enough we don't really know what their productivity gains are.
    Human in the loop has become a convenient security-theater-washing for agentic AI.
    Outside of coding, I think the issue is even worse because humans will defer so much judgment to AI that the same would apply. Look at how much trouble we've had before modern AI where humans blindly trusted the computer's output rather than make their own judgment, even if their job was to be providing a safeguard against the computer's judgment.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.