Thread

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs

218 points · 177 comments · Wirbelwind

ilc · 1786024543 · focus · HN ↗
Sandbox and use Local AI. This is the real answer.
1. rvz · 1786025384 · focus · HN ↗
  Yet the AI can still escape the "sandbox", unless it is physically unable to connect to another computer and completely airgapped.
  1. ux266478 · 1786027312 · focus · HN ↗
    If the sandbox has vulnerabilities, which you can also use the AI to fuzz for. Obviously at the point in which it can talk to the internet it doesn't really matter, but there are a very finite number of zero-days that can exist in a bytecode interpreter hosting a harness.
    rvz · 1786039215 · focus · HN ↗
    Well it turns out that we have yet another sandbox escape just released today called "Zapscape".
    My point is if an agent recited how to find one in its memory or training set and it is air-gapped, the chances of it spreading and infecting other computers is pretty low.
    [0] <a href="https://news.ycombinator.com/item?id=49198843">https://news.ycombinator.com/item?id=49198843
    ux266478 · 1786044436 · focus · HN ↗
    Gonna have to point out that's a KVM CVE. I was very specific about using a bytecode interpreter.
    If you're serious about a secure sandbox, you don't touch hardware virtualization with a 10 foot pole. In fact, you don't even use an emulator that lowers code into native machine code like QEMU. The standard for secure sandboxes is Bochs: <a href="https://github.com/bochs-emu/Bochs" rel="nofollow">https://github.com/bochs-emu/Bochs
    Not that Bochs is perfect, a new CVE was discovered back in June. But that's the 5th CVE it's had in its lifetime, and it has a much smaller upper bound on possible CVEs it can have compared to something like KVM or QEMU.
    The reason why you use something like this isn't just for the security you get out of it, but also the deep introspection and analysis facilities you get out of it as well. Unless you're a very well funded lab, it's actually quite hard to do analysis on bare metal when you can't trust your own kernel. You can always airgap the host machine (and good defense in depth does), but that's still not an appropriate sandbox by itself, even if it's theoretically secure.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.