‹ BackHN Continuity

Thread

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs

210 points · 169 comments · Wirbelwind

  1. continuational · · focus · HN ↗
    It's kinda funny there is still software coming out whose security model is "constantly ask the user for permission, and hope they never make a mistake".

    It's been tried so many times before, and it never worked.

    1. applfanboysbgon · · focus · HN ↗
      The alternative is software that is useless. You can convince consumers to use software that is useless (eg. iOS), but useless software is a hard sell for businesses that are being promised 10x productivity and the ability to fire everyone.
      1. csbrooks · · focus · HN ↗
        There's really no other alternative, though?
        1. inigyou · · focus · HN ↗
          I mean we could make software that isn't malicious and didn't need the OS to prompt you if you want to allow the malice. But where would the money be in that?
          1. inigyou · · focus · HN ↗
            I wonder how it would go if we went back to OSes with no security boundaries at all. Having an explicit boundary seems to give apps implicit permission to go right up to the boundary. But there was only one RealPlayer. Every other program didn't dare to be like RealPlayer.
        2. applfanboysbgon · · focus · HN ↗
          Correct. There is no other alternative. Your options are safe, useless software, or unsafe, useful software. It should be no surprise, then, that people still pick the latter in many cases. Perfectly safe and useful software doesn't exist, so you pick the tradeoffs you're comfortable with.
          1. grumple · · focus · HN ↗
            Nah, this isn’t true at all. You can run an agent in a container or vm on a dedicated host, limit the network and access it has, and otherwise let it run wild. With ai agents we’ve just thrown away even the most fundamental restrictions or isolation. Docker Sandbox came out to address this recently, though I think VM isolation is better. A few controls to limit repo access or the ability to deploy / spend money in a cloud can help you keep things under control while still fully utilizing AI. I don’t think every part of the security bureaucracy is great, but to treat it like it’s a binary is just not accurate.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.