> Fact is, vibe-coded projects devolve over time into an unmaintainable mess. The reason is simple, yet hard to fix: code maintainability and good architecture don’t have good measurements that we can apply, because it takes months, years even, to notice the effects of bad architecture or of unmaintainable code.
>
> For one, AI is not trained on what it means for code to be maintainable. For instance, any reinforcement learning done needs a reward signal that can be measured immediately, not in months or years.
Sad to say, but this is no different from human written code. Human written code just takes even longer to realize the mistakes because the pace is slower.
I think at the end of the day, it is not impossible to have AI write "good" or "high quality" code. If anything, once the patterns are established, AI will be more likely to adhere to the patterns and rules than any human team. It requires the most experienced engineers on the team to split their time writing the core patterns and documenting them in references/skills.
But it takes a lot of "taste" and a willingness to slow down a bit with AI (to create necessary artifacts), something teams find hard to do when you can ship so fast now.
My experience has been that there is a camp of very senior engineers that are unwilling to adapt to reality and focus on documentation and writing (effectively producing skills and agent guidance which multiplies their effectiveness); they will cling to their knowledge thinking coding a sacred art.
To me the difference is humans (ideally) will learn when they build something in a non-optimal way, and so will improve over time to become a competent engineer / architect. We cannot be perfect but to me a huge part of life is learning from failure and improving yourself, something that LLMs short-circuit and cannot replace.
LLMs cannot truly learn and so are destined to produce whatever the "average" software looked like at their training cutoff, or worse to produce code based on _other_ LLM generated code.
LLMs learn, and in two main ways: in-context and in training stages, release to release. The former is quick and sample efficient - perfect for adjusting AI behavior on the fly, and for enabling AI's own problem-solving capabilities. The latter modifies the "behavior defaults" and gives you performance gains that stick.
Why do you think that "write maintainable code" is somehow impossible to learn for an AI? We already have AI storming the frontiers of research math - way beyond the "average" of the field. If you can RL for "better at math", I see no reason why "better at maintaining code" would be somehow impossible.
You can construct an RL env where a codebase is presented as a "tree", and the AI is given one change to make at a time - and the per-change reward is not just whether the change itself has been evaluated as "made successfully", but also whether it made future changes down the line more or less likely to be successful, and harder or easier to make.
This is a formulation already used by some "maintainable code" benchmarks, so I expect something like it to make is way into frontier lab RL pipelines some time between "next week" and "a couple months ago".
CharlieDigital · · focus · HN ↗
I think at the end of the day, it is not impossible to have AI write "good" or "high quality" code. If anything, once the patterns are established, AI will be more likely to adhere to the patterns and rules than any human team. It requires the most experienced engineers on the team to split their time writing the core patterns and documenting them in references/skills.
But it takes a lot of "taste" and a willingness to slow down a bit with AI (to create necessary artifacts), something teams find hard to do when you can ship so fast now.
My experience has been that there is a camp of very senior engineers that are unwilling to adapt to reality and focus on documentation and writing (effectively producing skills and agent guidance which multiplies their effectiveness); they will cling to their knowledge thinking coding a sacred art.
throwaway19268 · · focus · HN ↗
LLMs cannot truly learn and so are destined to produce whatever the "average" software looked like at their training cutoff, or worse to produce code based on _other_ LLM generated code.
Ouroboros eat your heart out
ACCount39 · · focus · HN ↗
Why do you think that "write maintainable code" is somehow impossible to learn for an AI? We already have AI storming the frontiers of research math - way beyond the "average" of the field. If you can RL for "better at math", I see no reason why "better at maintaining code" would be somehow impossible.
You can construct an RL env where a codebase is presented as a "tree", and the AI is given one change to make at a time - and the per-change reward is not just whether the change itself has been evaluated as "made successfully", but also whether it made future changes down the line more or less likely to be successful, and harder or easier to make.
This is a formulation already used by some "maintainable code" benchmarks, so I expect something like it to make is way into frontier lab RL pipelines some time between "next week" and "a couple months ago".