Thread

Several vulnerabilities have been discovered in the Linux kernel

576 points · 408 comments · luispa

kalessin · 1790908911 · focus · HN ↗
I thought the "Security in the LLM age" talk by Greg Kroah-Hartman published this week from Kernel Recipes was pretty interesting: <a href="https://www.youtube.com/watch?v=NnV_cWeoo5Q" rel="nofollow">https://www.youtube.com/watch?v=NnV_cWeoo5Q
1. BLKNSLVR · 1790931608 · focus · HN ↗
  500 CVEs per release to 2000+
  Extending the ratio from my other comment. What are the token ratios between using LLMs to:
  1. Create functional software
  2. Find bugs in functional software
  3. Triage/Prioritise a quadrupling of the reported CVEs
  4. Fix the bugs while keeping the software functional
  I'm assuming that #2, #3, and #4 require more tokens (each or cumulatively) that #1, then there will be increase in the amount of insecure software because, with the advent of LLMs (that allow otherwise non-software developers to become software developers), there will be (a lot?) more software being created.
  If we want software security to get better, then the existence of LLMs requires the increasing use of LLMs. I find this quite interesting.
  1. sick_of_slop · 1790951594 · focus · HN ↗
    [dead]
2. sashank_1509 · 1790931620 · focus · HN ↗
  Yeah it’s a great video (TLDR from the video)
  1. LLMs have high false positive rate. From mythos 79 vulnerabilities found in the Linux kernel, only a single digit were actual bugs and they were all obscure so don’t panic.
  2. What does obscure mean? I don’t really understand it, but many of the bugs have to do with custom network drivers or other custom drivers that are very specific to certain organizational setups, not a general Linux distro issue.
  3. He’s very frustrated with the high false positive rate mythos generates. Even after multiple rounds of adversarial review and prompting strats, he mentions it is > 20% false positive rate, which wastes a lot of time. When some random user on the internet brings up a bug with an LLM it’s almost always fake, he even says just push back a few times claiming it’s not a bug to see if it’s a real bug (LLMs very quickly cave and “notice their mistake” etc)
  4. General observation on the useful bugs mythos finds. Chain multiple smaller bugs to see if you can get a bigger breakage. Mythos is really good at constructing these long convoluted chains that fuzzers miss.
  5. Go through recent bug fixes and check if similar bugs are hidden elsewhere in the codebase. Mythos is good at such pattern matching albeit with a high false positive rate.
  Final conclusion: don’t panic, the bugs are getting fixed, this is not as bad as the first fuzzer bug mania and will be fixed quicker, he estimates a year and we won’t see huge bug reports anymore.
  1. aliasxneo · 1790949377 · focus · HN ↗
    I use daybreak to do most of my security scanning. I always tell it that any finding must be accompanied by a harness that faithfully reproduces it using the code at the current commit with no modifications. In my experience it eliminates most if not all of the false positives.
    smartbit · 1790951207 · focus · HN ↗
    And normal people who can not apply for access to daybreak, should we use Abliterated GLM-5.3 ? [0] [1] [2]
    [0] <a href="https://news.ycombinator.com/item?id=49605691">https://news.ycombinator.com/item?id=49605691 [1] <a href="https://news.ycombinator.com/item?id=49897075">https://news.ycombinator.com/item?id=49897075 [2] <a href="https://huggingface.co/models?other=offensive-security" rel="nofollow">https://huggingface.co/models?other=offensive-security
  2. pixl97 · 1790952833 · focus · HN ↗
    >he mentions it is > 20% false positive rate, which wastes a lot of time.
    If it's just 20% that is really low. Especially for complicated long chain potential bugs.
    Most other detection tools have much higher rates of FP, or much higher rates of false negative.
    Then you have humans that miss bugs for 20+ years. Or, they don't tell you about the things they thought were bugs they wasted hours on themselves. Because of this it's really hard to measure how bad/good the AI really is.
    It would be interesting to know why the more SOTA models are getting the FPs. Is it from a lack of understanding of C? Is it complex code with deep branches? Is it code smell and convoluted logic?
    p-o · 1790957752 · focus · HN ↗
    Maybe you should just go ahead and watch the video instead of throwing hundred of hypothesis. It'll take just as long and you'll be informed by then.
    keeda · 1790959488 · focus · HN ↗
    It's an ~hour-long video! I don't blame anyone for preferring the 2-minute TL;DR if it's available! ¯\_(ツ)_/¯
    If there's something wrong or prone to misinterpretation in the TL;DR it would be better to call it out in response to that, rather than the users responding to the TL;DR, simply because it's likely that's what most people will respond to.
    voakbasda · 1791019547 · focus · HN ↗
    Yeah, if the only way to get content is video, I don’t consume it. It’s usually a complete waste of time, unless you speed it up until the speakers are chipmunks. The information density of video is poor compared to text.
    sashank_1509 · 1790964155 · focus · HN ↗
    He makes the exact opposite claim in the video. He thinks 20% is far too high and no one will pay for it once these tools are no longer available for free. He specifically cites the company Coverity that also found many useful bugs with their code analyzer but had a much smaller false positive rate (something like 5% I think), and no one paid for that, and the founder had to write a post mortem. He thinks the same’s going to happen to these LLM tools if they stop providing it for free.
    pixl97 · 1790969371 · focus · HN ↗
    There are a number of successful companies around doing just that, it's this guy that wasn't successful at it, not everyone.
  3. nananana9 · 1790954359 · focus · HN ↗
    If you push back against a LLM, you can convince it in almost anything.
    If it's obviously a bug you can just fix it, and if it's obviously not, you can ignore it. In either case you've already done the work to understand it.
    If it's on the borderline, and you push back against the GPU with a plausible sounding reason, it's likely to agree, regardless of whether oe nor it's a bug.
    I've never gotten a good outcome out of arguing with the GPU.
  4. fwip · 1790961126 · focus · HN ↗
    > push back a few times claiming it’s not a bug to see if it’s a real bug (LLMs very quickly cave and “notice their mistake” etc)
    I wonder how much more likely the LLM is to cave for a false-bug. There have been a number of times in my own (non-security) work that I'm told the LLM that it was wrong, it apologized and agreed, and then later that day I realized it was correct after all.
3. omarali-me · 1790932545 · focus · HN ↗
  Excellent plug and recommendation!
4. dannyw · 1790937219 · focus · HN ↗
  For additional context, Greg Kroah-Hartman has been contributing to Linux for 3 decades, and is _the_ maintainer for Linux's stable branch, as well as many other core parts of Linux.
  My takeaways:
  * Do not panic. Do acknowledge that LLMs are sycophantic, and LLM companies are trying to sell their stuff. "Yes, push back hard".
  * If it smells like AI slop, treat it like AI slop and relax. If someone sends you 50 security reports, and a few look wrong, just calm down and ignore them; or ask for proof-of-humanness.
  * A report without a patch is, for better or worse, worthless if you maintain widely used open source software.
  * Of course, there are real vulnerabilities being discovered and reported. Keep calm, keep fixing real bugs, and carry on.
  * Delete as much code as you possibly can. Reduce your surface area. Do the same thing you've always been doing.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.