Thread

1. romaniitedomum · 1790914941 · focus · HN ↗

   An interesting observation that I encountered somewhere, I forget where, is that AIs when writing code introduce vulnerabilities at a rate similar to humans writing the same code. So we're looking at a massively accelerated volume of security vulnerabilities for the foreseeable future thanks to AI-assisted security research, and we can expect no reduction in new vulnerabilities from the AIs writing the code.

   1. baq · 1790915357 · focus · HN ↗

      It doesn’t follow. Everyone sane has the models review the choose the models wrote. Reminder these are the models which found the Jacobian and Navier-Stokes counterexamples; they’ll find holes in their own slop, too.

      1. mepiethree · 1790915482 · focus · HN ↗

         Then a new model comes out two weeks later and finds a hole that your archaic review bot missed

         1. baq · 1790925986 · focus · HN ↗

            Fortunately, for now. Imagine the models not being released publicly.
      2. NoPicklez · 1790915790 · focus · HN ↗

         Yes and no, many people don't have models review the code the same way many humans don't review their own code in depth for security vulnerabilities.

         Furthermore, you need to make sure the model you use is capable enough to review your code comprehensively enough. That includes for both basic vulnerabilities but also attack chain related vulnerabilities.

      3. layer8 · 1790923560 · focus · HN ↗

         These counterexamples are comparatively straightforward because the input domain is well-defined and simply-structured, and a counterexample is trivial to verify. The same is not true for arbitrary vulnerabilities.

         1. baq · 1790925833 · focus · HN ↗

            Computers are finite. Inputs are well defined and so is their structure (ignore for a second the fact that it’s all physics behind the scenes). Secure code is a conjecture. A counterexample for secure code processing bits is an exploit.

            Also I find calling millennium problem solutions ‘straightforward’ baffling, to be polite.

      4. romaniitedomum · 1790929909 · focus · HN ↗

         > Everyone sane has the models review the choose the models wrote. Reminder these are the models which found the Jacobian and Navier-Stokes counterexamples; they’ll find holes in their own slop, too.

         It's not enough, though, to just tell the model to check the code for vulnerabilities. The model has to be guided specifically to look for particular classes of problem and that takes someone experienced in security.

         1. literalAardvark · 1790940603 · focus · HN ↗

            Telling it to look for a particular class of problem just decreases needed context by decreasing the problem space.

            That makes the bot more effective, but isn't strictly necessary. If you have a harness that can track longer projects it can do all that by itself, it needs your wallet, not your thoughts.

Unofficial Hacker News client; not affiliated with Y Combinator.