If you think vulnerability research simply doesn't matter, you have a lot of company in that opinion. It's something Bruce Schneier used to argue, and Marcus Ranum, and a bunch of other people that used be on closed secret vulnerability-sharing mailing lists before Bugtraq blew those cliques up. These are very old arguments.
But if you do think vulnerability research matters, and you're trying to argue that frontier models aren't a seismic change for that discipline, you have almost no company. Vulnerability researchers are overwhelmingly leaning on automation to find vulnerabilities and, just as importantly, generate the tooling required to test hypotheses.
You can feel about that however you want to feel about it. I mostly don't care, except: you can watch people like this being negatively polarized back into the bad old days of the mid-1990s, content-free CERT advisories, and vendor-controlled "responsible disclosure" by a use case that frontier models unimpeachably excel at.
It’s amazing how half the developers on earth live on a completely different planet now. There are plenty of new challenges, sure but we are far past the point where we can have a debate about “is it useful?”
And yet we continue to do so. I understand the feeling of loss some people may be facing. And there are definitely some really bad practices - like nakedly spewing claudspeak at your colleagues instead of communicating. Or raising a PR you don’t understand. There are asymmetries we haven’t learned to navigate. But we aren’t returning to a world where it doesn’t dominate our discipline so it’s best to find opportunities.
Maybe it’s worth actually listening to what these people are saying, or you may find yourself increasingly frustrated and confused as these debates intensify and AI becomes more
politically and socially toxic.
I've heard what they are saying, and a lot of it is factually untrue and confused. I wasn't happy about all of this when it first began happening, yet wasn't working. But once it was actually working, I realized I wanted to build software more than I wanted to craft software. Some people want to continue practicing their craft, which is a fine thing to want. But don't pretend its anything else.
As I see it, "engineers" going all-in on AI are missing two fundamental truths of our profession:
* Writing code is a form of communication, as well as a process through which complex systems are thought through, understood, and formalized.
* Complexity is managed by building on top of robust, deterministic abstractions.
Vibe-coders deny the need to understand complex systems and pretend that AI is a new layer of abstraction. I think that both of these perspectives are dead wrong. This remains the case even if LLMs are marvelously good at generating code.
Put another way, software engineers who mostly deal with code through their agents have switched careers from engineering to some form of management, even if they deny it to themselves. In no way does this deprecate the field of software engineering.
And then there are the externalities which have been discussed ad nauseam at this point, and remain as true as they ever were. (Economic, environmental, political — take your pick.)
Not everyone using coding agents is vibe coding. I still review code until I understand it, judge the tests cover it, and that it is the right way to do it. They constantly make mistakes like over building, handling contingencies that can't exist, duplicating code etc. Its still MUCH faster than the alternative. I've been working this way for decades. It is not different from reviewing human code and designs except the loops are much tighter, and refactoring is so much cheaper. There may be a lot of people who can't do this work effectively, but the thing about skill issues is they can be improved if they are recognized for what they are.
Almost everyone who uses coding agents extensively describes offloading some degree of understanding to the agents: reviews, tests, documentation, filling in bits "that don't matter," whatever.
As someone with years of FAANG experience, reviewing code competently was actually more strenuous than writing it in the first place. In fact, I'd estimate that truly understanding a codebase required a similar time commitment to writing the amount of code in that codebase. So I don't see how unprecedented speedups are possible with a human in the loop unless the reviews are only being skimmed.
The key to productivity and increasing complexity is inventing solid, thoughtfully designed abstractions and building on top of them, not taking shortcuts by way of a code extruder. If code is constantly getting duplicated or falling into the same patterns, figure out how to abstract or encapsulate it.
Reviewing code is more strenuous but it does not take as long - unless you count all the procrastination that happens first. And it goes much faster if you make a comment and get immediate changes because you don't have to rebuild context the next time you look at it. And yes - not every detail needs to be fully understood; sometimes verifying tests cover what they claim to is enough. Knowing which details are important and which can be hand waved is a bit of an art and I can't claim to always do it perfectly and don't always do it well for human authored code review either. And yes, I have to ask for refactorings and suggest abstractions in some cases. But there are now many tasks that coding agents actually do better than me, and they cut fewer corners, handle more edge cases, test more edge cases, research suggested designs, reviewing library docs & code much faster, etc.
AI coding is only going to increase. Eventually coding is going to join several other fields where it is much more about verification than the building of the code. I predict that eventually most programmers are not going to be looking deeply at the internals of the code, but are going to be spending most of their time on the verification.
If that were to happen those companies, products and practices would fail. They would be out-competed by all the organic enterprise shovel-ware and SAAS shit-code that has set such a high bar of quality over the last 30 years.
Please, lets revisit in the next few years. I think one of us is blinded by bias, and I think it is you, and I think the only way to judge is evidence, since arguments obviously don't make any difference.
tptacek · · focus · HN ↗
But if you do think vulnerability research matters, and you're trying to argue that frontier models aren't a seismic change for that discipline, you have almost no company. Vulnerability researchers are overwhelmingly leaning on automation to find vulnerabilities and, just as importantly, generate the tooling required to test hypotheses.
You can feel about that however you want to feel about it. I mostly don't care, except: you can watch people like this being negatively polarized back into the bad old days of the mid-1990s, content-free CERT advisories, and vendor-controlled "responsible disclosure" by a use case that frontier models unimpeachably excel at.
jeremyjh · · focus · HN ↗
And yet we continue to do so. I understand the feeling of loss some people may be facing. And there are definitely some really bad practices - like nakedly spewing claudspeak at your colleagues instead of communicating. Or raising a PR you don’t understand. There are asymmetries we haven’t learned to navigate. But we aren’t returning to a world where it doesn’t dominate our discipline so it’s best to find opportunities.
archagon · · focus · HN ↗
jeremyjh · · focus · HN ↗
archagon · · focus · HN ↗
* Writing code is a form of communication, as well as a process through which complex systems are thought through, understood, and formalized.
* Complexity is managed by building on top of robust, deterministic abstractions.
Vibe-coders deny the need to understand complex systems and pretend that AI is a new layer of abstraction. I think that both of these perspectives are dead wrong. This remains the case even if LLMs are marvelously good at generating code.
Put another way, software engineers who mostly deal with code through their agents have switched careers from engineering to some form of management, even if they deny it to themselves. In no way does this deprecate the field of software engineering.
And then there are the externalities which have been discussed ad nauseam at this point, and remain as true as they ever were. (Economic, environmental, political — take your pick.)
jeremyjh · · focus · HN ↗
archagon · · focus · HN ↗
As someone with years of FAANG experience, reviewing code competently was actually more strenuous than writing it in the first place. In fact, I'd estimate that truly understanding a codebase required a similar time commitment to writing the amount of code in that codebase. So I don't see how unprecedented speedups are possible with a human in the loop unless the reviews are only being skimmed.
The key to productivity and increasing complexity is inventing solid, thoughtfully designed abstractions and building on top of them, not taking shortcuts by way of a code extruder. If code is constantly getting duplicated or falling into the same patterns, figure out how to abstract or encapsulate it.
jeremyjh · · focus · HN ↗
munksbeer · · focus · HN ↗
AI coding is only going to increase. Eventually coding is going to join several other fields where it is much more about verification than the building of the code. I predict that eventually most programmers are not going to be looking deeply at the internals of the code, but are going to be spending most of their time on the verification.
archagon · · focus · HN ↗
You can only superficially “verify” what you do not fully understand.
jeremyjh · · focus · HN ↗
archagon · · focus · HN ↗
Unfortunately, shittiness in and of itself does not cause business failure. Otherwise, the industry would have moved on from Electron long ago.
munksbeer · · focus · HN ↗