‹ BackHN Continuity

Thread

Greg Kroah-Hartman – Security in the LLM Age [video]

337 points · 128 comments · usernomdeguerre

  1. usernomdeguerre · · focus · HN ↗
    Greatly appreciated the candor. I've included a few slides into text that i thought were eye-opening to me:

    From his Kernel Recipes 2026 slide on Mythos

    ```

      Mythos's 79 vulnerabilities:
      24 - no detail at all "something crashed"
      14 - not a bug at all
      3 - totally made up data
      15 - already fixed in latest release
        - 11 by others
        - 4 by anthropic
      20 - fixes were needed
        - 7 "assume a malicious filesystem image"
        - 2 "assume you can inject a malicious network packet into the middle of the stack"
        - 2 "NOMMU"
        - 6 sctp networking issues for untrusted devices
        - 2 ipv6 minor network issues 
        - 1 gpu driver for local malicious user
    
    ```

    GHK called this "10 'real' bugfixes", which to me sounds like there's a wild hype machine around these companies and uncritical parroting of every press release they make that falls apart when you engage the affected real experts.

    1. OtherShrezzing · · focus · HN ↗
      We’ve seen this in a few open source repos we voluntarily manage security on. They’re not massive repos, but big enough they get attention from security researchers.

      Most of the real low hanging fruit was picked up by humans years ago. When doing automated scanning, the majority of stuff is overly-verbose nonsense which takes hours of expert human labour to understand, test, and discard.

      Reading through a Claude generated false positive is absolutely excruciating, because it is absolutely determined that what it’s found is justified. Often you’ll receive very long accompanying “proof of concept” code which demonstrates absolutely wild scenarios. It’s especially frustrating when you’re volunteering your time for a project, and a well-meaning contributor submits the report without the technical nous to understand why you’re rejecting it.

      1. b112 · · focus · HN ↗
        Right now, all top tier LLMs are as eager, bright 20ish year old interns.

        Very gung ho, full of energy, loads of book learning, no real world experience or understanding of why things are as they are.

        Leave them to their own devices at your peril. Trust nothing they do.

        Yet directly guide them, monitor everything they do, some value emerges.

        1. charcircuit · · focus · HN ↗
          Have you used a frontier model since 2025? You are underplaying their strength.
          1. fc417fc802 · · focus · HN ↗
            Okay so now they're like a top percentile fresh grad on meth. Still a lack of real world experience plus some bizarre failures that illustrate gaping holes in the mental model. Does that description work for you?
            1. TeMPOraL · · focus · HN ↗
              Well, so basically supercharged fresh grad. CS/math implied.

              In human terms, that's already at least a standard deviation above average person.

          2. 12376 · · focus · HN ↗
            Kroah-Hartmann has used the closed frontier++ model, and it made up 37 out of 76 vulnerabilities.
            1. Gigachad · · focus · HN ↗
              And the bugs that actually were real rely on a setup so contrived it’s unlikely anyone in the world is impacted.

              It’s still good that some real bugs are being patched but what is being reported to the media is so overblown.

          3. crote · · focus · HN ↗
            We've been seeing "But you're not using the latest model!" over and over again, with every new model supposedly "groundbreaking" and "a game-changer" - just for the general population to conclude a few months later that it once again doesn't live up to the crazy marketing hype.

            Anthropic claimed that Mythos was so good at finding vulnerabilities that it was too dangerous to release to the public. As this post clearly shows: that (only again) simply isn't true. If you believe your favorite flavor of frontier model is the exception, it is up to you to provide proof to back up that claim.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.