> Fact is, vibe-coded projects devolve over time into an unmaintainable mess. The reason is simple, yet hard to fix: code maintainability and good architecture don’t have good measurements that we can apply, because it takes months, years even, to notice the effects of bad architecture or of unmaintainable code.
>
> For one, AI is not trained on what it means for code to be maintainable. For instance, any reinforcement learning done needs a reward signal that can be measured immediately, not in months or years.
Sad to say, but this is no different from human written code. Human written code just takes even longer to realize the mistakes because the pace is slower.
I think at the end of the day, it is not impossible to have AI write "good" or "high quality" code. If anything, once the patterns are established, AI will be more likely to adhere to the patterns and rules than any human team. It requires the most experienced engineers on the team to split their time writing the core patterns and documenting them in references/skills.
But it takes a lot of "taste" and a willingness to slow down a bit with AI (to create necessary artifacts), something teams find hard to do when you can ship so fast now.
My experience has been that there is a camp of very senior engineers that are unwilling to adapt to reality and focus on documentation and writing (effectively producing skills and agent guidance which multiplies their effectiveness); they will cling to their knowledge thinking coding a sacred art.
Yes, "code rot" is not in any way an AI-unique problem. Codebases like Flash Player or Bethesda Engine have been deep in decay long before AI was capable of contributing to them.
Historically, this was caused by hiring the cheapest developers one can find, having high turnover, outsourcing, pushing to ship at any cost and more. AI just lets you get there faster, and without having to hire bargain bin Indians.
The thing is, today's AI is already far better at "code rot per feature shipped" than the worst of developers - and I struggle to believe that we're at the limit there.
I've already seen benchmarks that test for AI's ability to make incremental changes and tweaks to code continuously - thus, tracking whether earlier changes make the latter changes harder. This makes for a clear target to RL for.
AI is not better nor worse at producing code rot; just faster at it.
AI produced code is a function of the team driving and instructing the agents along with the scaffolding produced by the team (skills, examples, docs, comments); same with human teams.
A team that cannot guide a human team to produce better code will not be able to guide an AI team to produce better code because it's the same skillset: being able to write good docs, create constraints structurally in code, produce core architecture that enforces good behavior.
Not entirely wrong, but there's a very big hole: "the scaffolding produced by the team" also includes the scaffolding produced by past AIs.
An AI that knows how to keep the documentation accurate and up to date, and does it by default, would, all other things equal, rot your codebase less. An AI that changes the code without checking whether it obsoleted a bunch of examples in the docs would rot your codebase more.
While I think that you can reduce "AI-induced code rot" with good prompting and steering, you could also make headway against it at model level, by making the AI "well-behaved" by default.
> "the scaffolding produced by the team" also includes the scaffolding produced by past AIs.
This statement is also true of humans. Everything you've stated here is also true for human engineers.
> "the scaffolding produced by the team" also includes the scaffolding produced by past engineers.
But the agent can be instructed reliably to keep documentation accurate and up to date and will then do so dutifully. Put it in AGENTS.md that it must always update the /docs directory by creating a new doc or updating an existing doc and it will do it. (Yes, adherence may be 95% of the time, but that is likely several points higher than with most non-NASA human teams)
Better yet, extract docs from code comments. Even better when the docs are spatially co-located and line of sight as the agent crawls through code.
CharlieDigital · · focus · HN ↗
I think at the end of the day, it is not impossible to have AI write "good" or "high quality" code. If anything, once the patterns are established, AI will be more likely to adhere to the patterns and rules than any human team. It requires the most experienced engineers on the team to split their time writing the core patterns and documenting them in references/skills.
But it takes a lot of "taste" and a willingness to slow down a bit with AI (to create necessary artifacts), something teams find hard to do when you can ship so fast now.
My experience has been that there is a camp of very senior engineers that are unwilling to adapt to reality and focus on documentation and writing (effectively producing skills and agent guidance which multiplies their effectiveness); they will cling to their knowledge thinking coding a sacred art.
ACCount39 · · focus · HN ↗
Historically, this was caused by hiring the cheapest developers one can find, having high turnover, outsourcing, pushing to ship at any cost and more. AI just lets you get there faster, and without having to hire bargain bin Indians.
The thing is, today's AI is already far better at "code rot per feature shipped" than the worst of developers - and I struggle to believe that we're at the limit there.
I've already seen benchmarks that test for AI's ability to make incremental changes and tweaks to code continuously - thus, tracking whether earlier changes make the latter changes harder. This makes for a clear target to RL for.
CharlieDigital · · focus · HN ↗
AI produced code is a function of the team driving and instructing the agents along with the scaffolding produced by the team (skills, examples, docs, comments); same with human teams.
A team that cannot guide a human team to produce better code will not be able to guide an AI team to produce better code because it's the same skillset: being able to write good docs, create constraints structurally in code, produce core architecture that enforces good behavior.
ACCount39 · · focus · HN ↗
An AI that knows how to keep the documentation accurate and up to date, and does it by default, would, all other things equal, rot your codebase less. An AI that changes the code without checking whether it obsoleted a bunch of examples in the docs would rot your codebase more.
While I think that you can reduce "AI-induced code rot" with good prompting and steering, you could also make headway against it at model level, by making the AI "well-behaved" by default.
CharlieDigital · · focus · HN ↗
Better yet, extract docs from code comments. Even better when the docs are spatially co-located and line of sight as the agent crawls through code.