Thread

1. intrepidsoldier · 1790914300 · focus · HN ↗

   Just the beginning. AI is going to expose how fragile the entire computing infrastructure in our world is.

   1. ankurdhama · 1790914599 · focus · HN ↗

      Does this also mean the code generated and reviewed by LLMs will not have such issues going forward?

      1. bottlepalm · 1790914831 · focus · HN ↗

         It won't which means inevitably malicious AI will probably backdoor us. Damned if we do, damned if we don't.
      2. trollbridge · 1790914892 · focus · HN ↗

         Quite the opposite, particularly when the biggest vendors of coding agents insist on not allowing their models to be used to check the code they generate for security issues.

         1. miohtama · 1790916265 · focus · HN ↗

            Why stop there? We should regulate who is allowed to write code in the first place!

            1. autoexec · 1790916840 · focus · HN ↗

               Easily done if everyone can be convinced that learning to write code is pointless since you can just pay an AI company for access to a chatbot that will write it for you.

               Fortunately there are people who write software for fun so there will always be some people who would rather do it themselves.

            2. bsoqk · 1790931811 · focus · HN ↗

               So, like professional orders in Europe? Thankfully everybody agreed that writing code is not engineering so this isn't mandated by law, but we were this close.

               1. tancop · 1790943729 · focus · HN ↗

                  You don't need a permit or degree to draw up plans, just to write your name on the official version and get it implemented in the physical world. It's closer to deploying code than writing.

                  1. bsoqk · 1790945057 · focus · HN ↗

                     We have a lot of that in Europe. Parasite professions. Someone who does nothing but charges a lot for his signature.
            3. jumploops · 1790932139 · focus · HN ↗

               “And that was how CS became a real engineering degree”
            4. trollbridge · 1790953987 · focus · HN ↗

               They sort of already do; the commercial American providers all require you to be 18 to sign up for a plan capable of agentic coding.

               So, I guess people under 18 aren't allowed to learn to program anymore.

         2. enraged_camel · 1790916896 · focus · HN ↗

            I've been able to use Opus 5.5 and Fable 5.1 for defensive security audits without any issues. They cannot do offensive tasks like pen-testing but in a lot of cases that's not a big shortcoming.

            1. whiskey-one · 1790945426 · focus · HN ↗

               Any tips / online resources how to best utilize for defensive reviews?
            2. trollbridge · 1790954036 · focus · HN ↗

               I've repeatedly slammed into walls doing very basic tasks. It can do a dumbed-down security audit, but fails to do offensive tasks against my own codebase which is frankly how you use a model like this effectively.
      3. Brian_K_White · 1790915045 · focus · HN ↗

         It just means that there will be the equivalent of infinite man-hours of barely-functional-intelligent-man to slog through code word by word and track how it affects all other code relation by relation.

         It's not magic and it's not even better or even as good as a mid human, but it's something like infinite man-hours of that drudge work per hour per user.

         That will find a lot in old code, and make it a lot easier to keep on finding every little thing right as it's created in new code.

      4. hgoel · 1790915453 · focus · HN ↗

         If the Western AI companies get their way, only the developers/companies that have access and paid extra for the security review will get to have a lower chance of such issues.
      5. mapontosevenths · 1790916854 · focus · HN ↗

         I'm an arms race the only winner is the arms-dealer.
      6. Gareth321 · 1790929139 · focus · HN ↗

         You're going to get a wide range of responses on this but given the improvement in these models in just one year, and the number of bugs they're detecting which humans could not, I suspect that even median vibe-coded software is going to surpass median human coded software soon - if it hasn't already.

         The important thing to remember here is there perfect isn't on the table. The benchmark is existing human-introduced bugs vs LLM-introduced bugs. Many developers have encountered odd bugs which a human would not have introduced, while forgetting about all the bugs caught which humans introduced. Or their opinion is formed by models from six months ago.

         1. abathologist · 1790958836 · focus · HN ↗

            How come all the vibe coded stuff I've tried is totally bug ridden and unmaintainable then?

            1. Gareth321 · 1790959827 · focus · HN ↗

               Mine isn’t. To point: we don’t really have a definition of vibe-coded anymore. All software has some degree of AI enhancement now. Is vibe-coded when it’s 60%? 80% 100%?

               Try out Opus 5.5 on high. It’s shockingly good. Of course if you’re trying to one-shot a sprawling application with load balanced distributed DBs, you’re going to have a bad time. For small, defined features, it’s pretty fucking great.

               1. abathologist · 1790968930 · focus · HN ↗

                  The definition I use is shipping LLM generated code you don't understand

                  We use LLMs extensively on the projects I work in. We don't "vibe code", and we understand every commit.

      7. lrvick · 1790934154 · focus · HN ↗

         Depends on if people are willing to pay the extra wall time to write mathematical proofs for everything to make it provably correct. Takes way way longer but modern models can do it.
      8. flohofwoe · 1790934455 · focus · HN ↗

         It depends? LLMs are not a silver bullet for writing bug free software (IME at least, and of course it also depends on the "threshold" what actually counts as a bug). They're definitely good at not creating the trivial "mechanical" type of bugs a tired and overworked human programmer would create (but oth those are also the easiest to find with traditional debugging tools and testing).

         They're definitely a useful additional tool for finding more (and more obscure) bugs, but that takes a lot of both human and compute effort too (quite a few of the reported bugs are actually false positives on close inspection, and apparently even with the latest locked down "wonder weapon" models like Mythos), and after all the reports are clean and validated you still can't be 100% sure (but at least a bit more confident) that the code is now free of bugs.

      9. spiclk · 1790946497 · focus · HN ↗

         IMO, they will introduce their own class of bugs that will defy static analysis.

Unofficial Hacker News client; not affiliated with Y Combinator.