‹ BackHN Continuity

Thread

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs

208 points · 169 comments · Wirbelwind

  1. continuational · · focus · HN ↗
    It's kinda funny there is still software coming out whose security model is "constantly ask the user for permission, and hope they never make a mistake".

    It's been tried so many times before, and it never worked.

    1. dasil003 · · focus · HN ↗
      I agree it's funny and won't really work on any kind of extended timeline. I mean Claude Code already added Auto-mode as a perfect example of this. But that said, I think it actually kind of makes sense in a transitional phase the power vs safety tradeoffs different users want to make varies so incredibly wildly that one product can't contain it all.

      What I think will happen is that as model capabilities plateau (I'm not an accelerationist) the harnesses and products around them will start to specialize and they'll have different security models based on the product needs for those particular use cases.

      For now, asking user to click a bunch of approvals, and occasionally making a mistake is a reasonable way to cover their asses until they see how bad security outcomes actually are in practice.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.