It's funny, i push back on pull requests because there is too much description now - a 20 line change has pages and pages of generated description, rationalisation for why it is safe, defense of each design decision, analysis of risks and side effects. People are indignant, you're rejecting my change because there is too much documentation? And my response is, I don't have time to read it and you put me in the position where I can't afford not to - because approving the PR implies I did and accepted it. The investment to read all that for the value of a code change that I'm one prompt away from doing myself if I cared is just not high enough. So it's rejected.
I also have ADHD-driven blah blah blah. I find achieving brevity laborious, but find LLM results to be unacceptable. A journalism class taught by a well-known professional food critic helped most, but consistently using the free “Hemingway App” editor was almost as beneficial. Blindly following every suggestion pares your writing down to a nub, but it’s great at highlighting sentences you need to rethink without blurring your voice and ideas with bland suggestions.
That's a solid recommendation, thanks. I also did journalism. So I can write succinctly, I just have to put effort in like you. Which is much more than I'm willing to put in after I just wrote a treatise on the human condition to send to the lead developer at 07:30 on Monday.
I’ve started just asking it to touch up text to make it sound more human and follow technical writing guidelines and it converges on convincing pretty quickly. It also makes it easier for me to verify that it’s correct because all of the stupid LLM noise and formatting goes away and it gets distilled down to the important bits.
You might be surprised how many organisations/ teams now don’t even review pull requests. Claude does coding, Codex do code review, and if developer did 5 round of this before pull request, we could very well just let CI merge.
I'm very curious how this goes long term. I guess we will find out.
My instinct says that these systems will expand their complexity to fully fit the cognitive budget of the agents that coded them and then atrophy the same way human-built systems do at lower cognitive budget. Only this time, because of the larger up front budget, the complexity ceiling will be higher, and the potential depth of the problem may be much much larger. It may mostly manifest as increasing cost over time - the agents grind for longer and longer, iterating over and over to fix all the failing tests, and the breaking point will be where it never converges and you come back to millions of dollars in budget spent and still tests are failing and effective gridlock on system changes.
But this may be all my human-biased fantasy that justifies still taking a role in software development.
> ... these systems will expand their complexity to fully fit the cognitive budget of the agents that coded them and then atrophy the same way human-built systems do at lower cognitive budget. Only this time, because of the larger up front budget, the complexity ceiling will be higher ...
That is exactly how one could describe the relationship between heroin and the users health systems. It’s okay for a little while, until it isn’t but there is no going back. When the risks start telling, lawyers will have a field day. Assisted by LLM’s obviously, which then puts the burden on judges, who then need LLM assistance.
This thing called attention economics has captured my attention quite hard. Digital opiates are everywhere since 10 to 20 years. And in my view, the part that they implicitly of explicitly strive to capture your attention is new.
Well articulated. I tend to think that the outcome will vary hugely by project. We already observe this with some reporting certain models behave in a certain way, whilst others see the opposite.
To use the wooley term “quality”, the top 20% might stand a good chance of making huge strides. But the remaining 80% of projects (in particular the bottom 20%) will atrophy extremely quickly. Yet, these will be the project that many push LLM’s too as their domain/technology is complex and/or outdated. Digital transformations that can be done quickly will be tantalising but ultimately unsatisfactory long term (as you describe).
and this is such a liability imo. I recently got fired for what I think was apparently merging a change to Kubeconfig that both the CTO and DevOps person approved. I made as damn sure as I could that the proposed change was good, but I needed their eyes on it and they clearly weren't, possibly be cause every other repo had a review bot and no special automatic deployment automations. The DevOps person literally commented and said "It's good to merge", who to trust
Others are just implicitly doing this but pretending review still exists.
Everyone is fatigued by endless code review which you get no credit for and has become massively more of a burden.
All PRs are superficially fine now. There are no typos, there is unit test coverage, but there are deeper issues that require massive amounts of effort and time to spot.
Unfortunately, a lot of previous PR reviews already were just gatekeeping, or "presenteeism". People would leave comments about class names or method names, like they couldn't understand what an `apply` method, the only method, meant on a class with that was already named appropriately and did one thing only (to give an oop example). No, the method needed to be renamed `PriceChecks.applyPriceChecksWithTimeConstraints`.
Lots of review comments about various conditions that wouldn't feasibly happen (same shit with claude now).
But then I'd see these same reviewers approving PRs where the bigger design was just fundamentally broken. Oh, we're adding a blocking call on our hot path, but at least the method name makes it very clear that it is blocking.
In general I agree that the current AI reviews are creating too much noise and it is masking these bigger design issues.
IMO, PRs were never the right process to use in tightly-collaborating teams such as most companies. PRs got popular because orgs started using GitHub, and GitHub had made that workflow to fit the needs of open source (and modeled it after what open source was already doing in the days when patches got sent around via mailing lists).
I'm old enough to remember using CVS and then subversion in companies. People would commit straight to main (which was then called "trunk"), because making feature branches and merging them was cumbersome. And, on regular intervals, the person responsible for some corner of the codebase would do a show-and-tell presenting it to peers, but without the sharply defined boundaries of what the code looked like before vs. after some recent set of changes. People might remember some things from the previous show and tell or from first hand experience with that code, but that kind of memory is necessarily fuzzy, and diffs weren't an artefact that was typical to look at. So, these reviews didn't block people, and any comments that came from reviews defined a direction that things should go from here on out. If a corner of the codebase was deemed to be in a bad shape, the blame around that was equally fuzzy.
I just have my LLM read the PR, then give me a summary of what the PR is about, how important it is, and how good the PR itself actually is. Then I have a conversation with the LLM about specific points, especially things where I get that feeling that I don't have a 100% understanding.
Until I fully understand what's going on, the PR doesn't move and my interrogation of the LLM doesn't end. My interaction is littered with "Explain X" and "How does this square with Y?" and "What if Z happens?"
The interrogation is the point, without me having to wade through hundreds of lines of irrelevant code to get at the meat of the matter.
Personally I'm starting to see less and less "smart" people here, given how easily and quickly everyone jumps on random hype trains with very little investigation or thought put into it. And those who do investigate and put thought into it are then quickly down voted for being a hater. This forum has turned into almost like a FOMO echo chamber in a way.
I can read the room. Coding is going to go the way of some other disciplines where machines do most of the detail work and and we know stuff works by verification. There are other fields like this.
Do I like it or not? That doesn't really matter. I need a job, so I'm going to get good at the new way to ensure I continue to have a job. I consider this to be a smart thing to do for myself and my family.
I agree with the work changing more towards verification, but if the verification is one LLM reviewing another one, then that isn't really verification. At best you "assume", but you don't "verify".
Unfortunately I see a lot of (senior as well) engineers who think that just a vanilla LLM reviewing another LLM is sufficient, and my comment was directed towards those. If however you see the LLM era as needing more test support and systems than ever before in the form E2E tests and so forth, where "code review" as such becomes mostly irrelevant as you have such a strong test system in place that if that passes you can be sure it doesn't break anything for users, then yes, that's good.
What I mean is verification through testing, and perhaps other more formal means. This has happened to other engineering domains, yet us programmers have so much hubris that we think it can't happen to us. And everyone is throwing their toys out the cot because it looks like it is actually going to happen.
Code reviews may not even happen, or if they do, it'll be all automated, and the verification will be the key.
Ask anyone in the semiconductor industry when last they understood the design of those things.
At this point, what is a human dev even there for?
We have this at work : fully AI-generated code and description. People will give review comments generated by AI which the "author" replies with an AI-generated response, all with LLM wording full of jargons no one understands not even the person who sent it. When you ask them what they meant, yeah idk Claude said so
The human is there as the control valve, the one who keeps the overall context, and the one who injects actual creativity.
Every time the LLM throws jargon around, you call it. "What do you mean by gated wedge?" You call its bullshit, check what it's saying against your understanding of the overall system, and keep it on the straight and narrow.
It's a lot like supervising a junior dev who happens to be very quick at absorbing lots of info, but not so great at the big picture.
Why would I do that? It's the job of the person sending me the PR to explain what they're trying to do. I'm done wading through AI slop.
Lately on my hobby projects I've been lazily committing work with the prompt 'add the relevant files to git and commit with a message that explains why' - only because I remember 15 years ago reading an HN comment from someone complaining that too many commit messages answer 'what' but not 'why' haha
So far I haven't had a reason to go back through commits to isolate any issues but if I do hoping the 'why' messages may come in handy for my LLM lol
Same problem with postmortems. You get a clean AI writeup with root cause, timeline, and action items all in the right sections, but none of it tells you what was actually checked or ruled out along the way. Reads like it explains the incident, but you can't tell if the fix was ever tested or just sounds plausible.
zmmmmm · · focus · HN ↗
devilsdata · · focus · HN ↗
In the meantime, for business communication, I use AI to shorten my text, to make it more concise.
DrewADesign · · focus · HN ↗
devilsdata · · focus · HN ↗
misiti3780 · · focus · HN ↗
<a href="https://github.com/josephmisiti/awesome-machine-learning" rel="nofollow">https://github.com/josephmisiti/awesome-machine-learning
It's helped a lot. Agents haven't figured out how to do that yet, or sendgrid, sns, etc are doing the hard work for me.
Sharlin · · focus · HN ↗
bathtub365 · · focus · HN ↗
ramshanker · · focus · HN ↗
fantasizr · · focus · HN ↗
classified · · focus · HN ↗
zmmmmm · · focus · HN ↗
My instinct says that these systems will expand their complexity to fully fit the cognitive budget of the agents that coded them and then atrophy the same way human-built systems do at lower cognitive budget. Only this time, because of the larger up front budget, the complexity ceiling will be higher, and the potential depth of the problem may be much much larger. It may mostly manifest as increasing cost over time - the agents grind for longer and longer, iterating over and over to fix all the failing tests, and the breaking point will be where it never converges and you come back to millions of dollars in budget spent and still tests are failing and effective gridlock on system changes.
But this may be all my human-biased fantasy that justifies still taking a role in software development.
rapidfl · · focus · HN ↗
wow this is a beautiful way to put it
wjnc · · focus · HN ↗
This thing called attention economics has captured my attention quite hard. Digital opiates are everywhere since 10 to 20 years. And in my view, the part that they implicitly of explicitly strive to capture your attention is new.
the_gipsy · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
joshghent · · focus · HN ↗
To use the wooley term “quality”, the top 20% might stand a good chance of making huge strides. But the remaining 80% of projects (in particular the bottom 20%) will atrophy extremely quickly. Yet, these will be the project that many push LLM’s too as their domain/technology is complex and/or outdated. Digital transformations that can be done quickly will be tantalising but ultimately unsatisfactory long term (as you describe).
intended · · focus · HN ↗
How many are you seeing / estimating?
brailsafe · · focus · HN ↗
veltas · · focus · HN ↗
brailsafe · · focus · HN ↗
Gigachad · · focus · HN ↗
Everyone is fatigued by endless code review which you get no credit for and has become massively more of a burden.
All PRs are superficially fine now. There are no typos, there is unit test coverage, but there are deeper issues that require massive amounts of effort and time to spot.
munksbeer · · focus · HN ↗
Lots of review comments about various conditions that wouldn't feasibly happen (same shit with claude now).
But then I'd see these same reviewers approving PRs where the bigger design was just fundamentally broken. Oh, we're adding a blocking call on our hot path, but at least the method name makes it very clear that it is blocking.
In general I agree that the current AI reviews are creating too much noise and it is masking these bigger design issues.
datsci_est_2015 · · focus · HN ↗
stakhanov · · focus · HN ↗
I'm old enough to remember using CVS and then subversion in companies. People would commit straight to main (which was then called "trunk"), because making feature branches and merging them was cumbersome. And, on regular intervals, the person responsible for some corner of the codebase would do a show-and-tell presenting it to peers, but without the sharply defined boundaries of what the code looked like before vs. after some recent set of changes. People might remember some things from the previous show and tell or from first hand experience with that code, but that kind of memory is necessarily fuzzy, and diffs weren't an artefact that was typical to look at. So, these reviews didn't block people, and any comments that came from reviews defined a direction that things should go from here on out. If a corner of the codebase was deemed to be in a bad shape, the blame around that was equally fuzzy.
wvenable · · focus · HN ↗
wvenable · · focus · HN ↗
jdkoeck · · focus · HN ↗
kstenerud · · focus · HN ↗
Until I fully understand what's going on, the PR doesn't move and my interrogation of the LLM doesn't end. My interaction is littered with "Explain X" and "How does this square with Y?" and "What if Z happens?"
The interrogation is the point, without me having to wade through hundreds of lines of irrelevant code to get at the meat of the matter.
askonomm · · focus · HN ↗
GlacierFox · · focus · HN ↗
askonomm · · focus · HN ↗
mitxela · · focus · HN ↗
munksbeer · · focus · HN ↗
I can read the room. Coding is going to go the way of some other disciplines where machines do most of the detail work and and we know stuff works by verification. There are other fields like this.
Do I like it or not? That doesn't really matter. I need a job, so I'm going to get good at the new way to ensure I continue to have a job. I consider this to be a smart thing to do for myself and my family.
askonomm · · focus · HN ↗
Unfortunately I see a lot of (senior as well) engineers who think that just a vanilla LLM reviewing another LLM is sufficient, and my comment was directed towards those. If however you see the LLM era as needing more test support and systems than ever before in the form E2E tests and so forth, where "code review" as such becomes mostly irrelevant as you have such a strong test system in place that if that passes you can be sure it doesn't break anything for users, then yes, that's good.
munksbeer · · focus · HN ↗
Code reviews may not even happen, or if they do, it'll be all automated, and the verification will be the key.
Ask anyone in the semiconductor industry when last they understood the design of those things.
kstenerud · · focus · HN ↗
It's very easy to argue with stawman arguments.
isakmarr · · focus · HN ↗
fossilwater · · focus · HN ↗
We have this at work : fully AI-generated code and description. People will give review comments generated by AI which the "author" replies with an AI-generated response, all with LLM wording full of jargons no one understands not even the person who sent it. When you ask them what they meant, yeah idk Claude said so
kstenerud · · focus · HN ↗
Every time the LLM throws jargon around, you call it. "What do you mean by gated wedge?" You call its bullshit, check what it's saying against your understanding of the overall system, and keep it on the straight and narrow.
It's a lot like supervising a junior dev who happens to be very quick at absorbing lots of info, but not so great at the big picture.
sagarm · · focus · HN ↗
mentos · · focus · HN ↗
So far I haven't had a reason to go back through commits to isolate any issues but if I do hoping the 'why' messages may come in handy for my LLM lol
marcus9999 · · focus · HN ↗