Thread

Several vulnerabilities have been discovered in the Linux kernel

576 points · 408 comments · luispa

romaniitedomum · 1790914941 · focus · HN ↗
An interesting observation that I encountered somewhere, I forget where, is that AIs when writing code introduce vulnerabilities at a rate similar to humans writing the same code. So we're looking at a massively accelerated volume of security vulnerabilities for the foreseeable future thanks to AI-assisted security research, and we can expect no reduction in new vulnerabilities from the AIs writing the code.
1. baq · 1790915357 · focus · HN ↗
  It doesn’t follow. Everyone sane has the models review the choose the models wrote. Reminder these are the models which found the Jacobian and Navier-Stokes counterexamples; they’ll find holes in their own slop, too.
  1. mepiethree · 1790915482 · focus · HN ↗
    Then a new model comes out two weeks later and finds a hole that your archaic review bot missed
    baq · 1790925986 · focus · HN ↗
    Fortunately, for now. Imagine the models not being released publicly.
  2. NoPicklez · 1790915790 · focus · HN ↗
    Yes and no, many people don't have models review the code the same way many humans don't review their own code in depth for security vulnerabilities.
    Furthermore, you need to make sure the model you use is capable enough to review your code comprehensively enough. That includes for both basic vulnerabilities but also attack chain related vulnerabilities.
  3. layer8 · 1790923560 · focus · HN ↗
    These counterexamples are comparatively straightforward because the input domain is well-defined and simply-structured, and a counterexample is trivial to verify. The same is not true for arbitrary vulnerabilities.
    baq · 1790925833 · focus · HN ↗
    Computers are finite. Inputs are well defined and so is their structure (ignore for a second the fact that it’s all physics behind the scenes). Secure code is a conjecture. A counterexample for secure code processing bits is an exploit.
    Also I find calling millennium problem solutions ‘straightforward’ baffling, to be polite.
  4. romaniitedomum · 1790929909 · focus · HN ↗
    > Everyone sane has the models review the choose the models wrote. Reminder these are the models which found the Jacobian and Navier-Stokes counterexamples; they’ll find holes in their own slop, too.
    It's not enough, though, to just tell the model to check the code for vulnerabilities. The model has to be guided specifically to look for particular classes of problem and that takes someone experienced in security.
    literalAardvark · 1790940603 · focus · HN ↗
    Telling it to look for a particular class of problem just decreases needed context by decreasing the problem space.
    That makes the bot more effective, but isn't strictly necessary. If you have a harness that can track longer projects it can do all that by itself, it needs your wallet, not your thoughts.
2. biwills · 1790916000 · focus · HN ↗
  Do we know the code behind these vulnerabilities were written by AI? It seems like if anything AI was used to find exisitng vulnerabilities that would otherwise be used/sold as zero days and go unreported.
  It was always the case that finding vulnerabilities in software was easier than writting perfect software. I'm hopeful that we can use AI to make software more secure over time. Project Zero [1] and others has shown many times the past few years (pre LLMs) that automated fuzzing and other forms of dynamic analysis are very effective, which bodes well for automated testing via LLMs!
  I agree that more code = more bugs overeall, but there are slow moving codebases that run some of the worlds most valuable software. Seems like using AI to find vulnerabilities in that code is a huge win across the board.
  [1]: <a href="https://www.google.com/search?q=site%3Aprojectzero.google&q=fuzzing" rel="nofollow">https://www.google.com/search?q=site%3Aprojectzero.google&q=...
  1. romaniitedomum · 1790929798 · focus · HN ↗
    > Do we know the code behind these vulnerabilities were written by AI? It seems like if anything AI was used to find exisitng vulnerabilities that would otherwise be used/sold as zero days and go unreported.
    I was speaking in the general sense, not of these vulnerabilities specifically. I am of the view that AIs for the foreseeable won't produce code that is any better from a security point of view than something human written, so AIs will produce new vulnerabilities at least as fast as they find them and the rest of us will be faced with massive headaches like the one in the original post.
    > It was always the case that finding vulnerabilities in software was easier than writting perfect software. I'm hopeful that we can use AI to make software more secure over time. Project Zero [1] and others has shown many times the past few years (pre LLMs) that automated fuzzing and other forms of dynamic analysis are very effective, which bodes well for automated testing via LLMs!
    And yet, we are not seeing a drop-off in new vulnerabilities being discovered. We keep assuming that the list of bugs is getting smaller and we'll find them all eventually, but that is not the case for any software that I know of.
    > I agree that more code = more bugs overeall, but there are slow moving codebases that run some of the worlds most valuable software. Seems like using AI to find vulnerabilities in that code is a huge win across the board.
    It might be, yet, as I said just above, we are not seeing a drop-off in new vulnerabilities being found. The trickle of vulnerabilities has become a flood across all open source software, and already breakages and problems are occurring as maintainers struggle to keep up. Administrators, likewise, are struggling to keep systems updated. Just a week or so ago a security patch to rsync on RHEL broke rsync so completely that it could no longer handle symbolic links.
    Critical CVEs used to be relatively infrequent, but they're becoming a weekly or even daily occurrence. None of us are prepared for this eventuality.
    zahlman · 1790933997 · focus · HN ↗
    > I am of the view that AIs for the foreseeable [future] won't produce code that is any better from a security point of view than something human written, so AIs will produce new vulnerabilities at least as fast as they find them and the rest of us will be faced with massive headaches like the one in the original post.
    If you simply prompt them to produce code, with the same kind of processes that humans use, then yes, of course. After all, it trained on human code.
    If you prompt them explicitly to spend time looking for vulnerabilities and not implementing new features, then why wouldn't it produce more secure code? If we're calling the technology a "force multiplier", then it's thus for every task it can perform. So, orient the process around that; avoid the compromises that were originally motivated by working at human speed (, interest level, fatigue, specialization, …)
    Of course, if you see places where the application of artificial "intelligence" can benefit from human wisdom, then double down on that. (Quotes because I think the term is fundamentally inaccurate for what it refers to, even though it's typically good enough and refers to a useful capability.)
    xorcist · 1790936886 · focus · HN ↗
    > broke rsync so completely that it could no longer handle symbolic links
    That sounds bad. Where can we find more information about this?
    mrweasel · 1790947363 · focus · HN ↗
    The issues page on rsyncs Github. A lot has been fixed already obviously, but things are constantly breaking now.
    To the point of other comments: Yes it might be a prompting issue, I don't know, but it does illustrate that the force multiplier people are suggesting that LLMs are, goes both ways. You can absolutely use them to make more secure software fast, but Andrew Tridgell isn't a stupid person. If someone like him can be seen struggling with the technology, then we must safely assume that this will be the case for many other developers as well.
    <a href="https://github.com/RsyncProject/rsync/issues" rel="nofollow">https://github.com/RsyncProject/rsync/issues
    romaniitedomum · 1790975305 · focus · HN ↗
    See <a href="https://github.com/RsyncProject/rsync/issues/1087" rel="nofollow">https://github.com/RsyncProject/rsync/issues/1087
    From the comments:
    > "Introduced while fixing CVE-2026-53801. I have likely found what the issue is, I truly hate symlinks Might not be able to fix tonight but will be done within the next 24 hours."
    literalAardvark · 1790940331 · focus · HN ↗
    >And yet, we are not seeing a drop-off in new vulnerabilities being discovered. We keep assuming that the list of bugs is getting smaller and we'll find them all eventually, but that is not the case for any software that I know of.
    It's worth factoring in that AI has gotten better rather quickly, so there's no reason to expect it not to continue to find new bugs even if we've correctly fixed what Mythos found. The search depth is increasing.
3. autoexec · 1790917251 · focus · HN ↗
  > AIs when writing code introduce vulnerabilities at a rate similar to humans writing the same code.
  AI regurgitating all the insecure code AI companies scraped from stack overflow and github isn't going to give you something too different from what the humans who put it there in the first place came up with. Garbage in, garbage with random hallucinations out.
  1. red75prime · 1790922258 · focus · HN ↗
    This is simplistic to the point of being blatantly wrong. Training data isn't garbage. It's programs that do their job, but that are sprinkled with errors. Uncorrelated errors gets averaged out during autoregressive pretraining. Correlated errors can be somewhat suppressed during post-training. Hallucinations (of the generalization-error kind) can be dealt with using synthetic data that improves the model's generalization.
    deadbunny · 1790932616 · focus · HN ↗
    So with all those mitigations why do LLMs keep writing insecure code?
    red75prime · 1790933119 · focus · HN ↗
    Provable correctness is much harder than focusing on the happy path. If a pipeline doesn't include a look-for-the-vulnerabilities stage to save costs, a model wouldn't go out of its way to do it. The models are trained to do what they are asked to do.
    I forget to add the obvious: some garbage gets through despite the mitigations. In the limit of pure garbage input, you'll get a model that internalized garbage generation. And you'd be better off throwing it away and starting over.
4. PowerElectronix · 1790928303 · focus · HN ↗
  I guess trash in trash out also applies to the training of an LLM.
5. brabel · 1790939643 · focus · HN ↗
  In my experience it doesn’t matter if who wrote the code was AI or human as long as you run security reviews, including AI reviews. They are extremely good at finding issues before you ship. Just don’t expect the LLM to one shoot secure code, let an empty context AI reviewer explicitly check the change set for vulnerabilities.
  1. qc-nsilva · 1790943991 · focus · HN ↗
    [dead]
6. abathologist · 1790959116 · focus · HN ↗
  Yet another case of <a href="https://en.wikipedia.org/wiki/Neijuan" rel="nofollow">https://en.wikipedia.org/wiki/Neijuan
7. lofaszvanitt · 1790983614 · focus · HN ↗
  95% or coders do not care about security at all. It is an afterthought. That's a reality. People only care if they feel the heat.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.