Thread

1. intrepidsoldier · 1790914300 · focus · HN ↗

   Just the beginning. AI is going to expose how fragile the entire computing infrastructure in our world is.

   1. flohofwoe · 1790934118 · focus · HN ↗

      No, it will be a tsunami of new discoveries in old code bases at first, but that will settle down as those old bugs are fixed. It's been like this with every new code analysis tool (the wave may be exceptionally high this time though).

      1. thewizzardofnl · 1790939767 · focus · HN ↗

         There is a difference here. The "new coding analysis tool" that you use for the analogy here is getting better every few weeks with the release of new models.

         It is plausible to assume that, for instance, a Linux kernel that was hardened for CVEs that Sonnet 3.5 could detect is not hardened for bugs that Sonnet 4.5, 5.5, Opus, Fable, and models in 2027 can and will be able to detect.

         Hence, it is rather a constant catch-up game until the LLM improvements might hit a ceiling and won't get any better in this regard.

         1. handoflixue · 1790941267 · focus · HN ↗

            You don't even have to assume. Mythos unveiled a ton of bugs across the ecosystem back in April, so this isn't the first iteration of the cycle.

            1. flohofwoe · 1790942829 · focus · HN ↗

               Search for "Security in the LLM age" in this HN page for a reality check, apparently 80% of the security vulnerabilities that Mythos initially flagged in the Linux kernel turned out to be false positives. Better than nothing of course, but it really doesn't look like Mythos is quite the "wonder weapon" it was marketed as (and the situation by far isn't as dire as the initial flurry of Mythos news).

               1. handoflixue · 1791004834 · focus · HN ↗

                  IIRC, in April, Mythos found 20x the monthly average? 20% of that is still a single system doing something that takes a team of engineers 4 months.

                  Like, "4x as powerful as a team of engineers" is still really quite impressive

                  1. flohofwoe · 1791011970 · focus · HN ↗

                     The problem isn't the 20% actual bugs found (that's great), but the 80% false positive rate which are reported with high confidence and (most likely) misleading reproduction code. An experienced programmer familiar with the code base first needs to validate all reports and throw away 4 in 5. That's a massive waste of time. If a traditional static analyzer had an 80% false positive rate nobody would take it serious.

                     In my hobby projects I use LLMs in my development workflow mainly for passive bug scanning, reviewing and helping to maintain tests, they are definitely useful for catching some bugs early and noticing unhandled edge cases, but they're also definitely no silver bullet (they sometimes ignore quite obvious bugs, and the fewer 'obvious' bugs remain the more one has to be careful about false positives). E.g. the funny thing is that now I'm actually slower than before due to the intense 'rubber ducking' with LLMs and cross-checking their results, but I still want to pretend that the resulting code is more robust out of the door.

                     Eg everything that Greg KH says in the video sounds very familiar, and it's very disappointing that Mythos still suffers from the same issues (or maybe even worse) as older models.

                     1. handoflixue · 1791041091 · focus · HN ↗

                        > An experienced programmer familiar with the code base first needs to validate all reports and throw away 4 in 5. That's a massive waste of time.

                        Okay, but again, even with all that extra effort, they found 4x as many bugs, so it seems like the effort is clearly worth it.

                        And each model gets more reliable, we get better at building proper reproduction code, etc. - this was mostly a comment about the cycle, direction, and velocity we should expect from the future, given this has already happened twice.

Unofficial Hacker News client; not affiliated with Y Combinator.