Thread

Pop!_OS bans AI-generated code from much of its codebase

116 points · 167 comments · bundie

brink · 1791054618 · focus · HN ↗
I also have found that AI has not lived up to many of its promises and have dialed back what AI gets control of. My projects were turning into unmaintainable messes. The people who say coding is solved aren't paying attention.
1. lucianmarin · 1791055784 · focus · HN ↗
  Yes. I started two projects with AI from scratch. Both abandoned, complete mess. Projects without AI are so easy to manage, maintain, add/remove features, etc. I use AI as a search engine on my projects instead of Google. I ask what's wrong with my code and change or improve it myself based on my experience.
  1. dnautics · 1791057252 · focus · HN ↗
    I have the opposite experience. I don't have the patience sometimes to clean up my code and stick to coherent conventions and organization even though the will is there. With my AI projects I watch it and if the AI starts drifting I ask it to go through and look for convention/directory structure violations and it happily cleans everything up in about 10 or so minutes.
    Are you in Python by chance? Python has a lot of crazy hidden/inexplicit/spooky action at a distance stuff (especially in the frameworks) that can make LLMs gunk up code by defensively programming or just burn context chasing data provenance
  2. fasterik · 1791057584 · focus · HN ↗
    I've been a vibe-coding skeptic for years, but because of the math breakthroughs of the past few weeks I decided to experiment with the latest models on some test projects. They're a lot more capable than I thought they would be. I agree that it's easy to create an unrecoverable mess, especially when you're one-shotting a lot of features without detailed instructions. But I find that as long as I'm strict about the API boundaries and force the agent to work in small chunks, it's pretty effective. As one example, I got it to write an SVG renderer in a few hours (not the whole spec, but most of the path features and text rendering), which would have taken me at least a week just for the coding part, plus extra time to learn the algorithms.
    globular-toast · 1791059921 · focus · HN ↗
    You could have also copied an SVG renderer that implements the whole spec from whatever open source project the model copied it from.
    fasterik · 1791063651 · focus · HN ↗
    It didn't copy any source code from any external projects. I had it write a stratified sampling renderer for ground truth, then had it implement feature by feature by matching the pixels. Unless you mean it "copied" it in the sense of third-party code being part of the training data. I don't think that definition of "copy" makes any sense given how these models represent embeddings. It would also imply that humans are "copying" the things they've learned from.
    sashank_1509 · 1791093096 · focus · HN ↗
    I don’t think we should hold humans and models to the same standards. Humans have a very small working memory. Most humans cannot reproduce code they wrote even a year back exactly.
    A model can reproduce large swaths of its training data exactly. It’s a different algorithm that powers its learning process (it’s why it needs trillions of tokens to even learn basics of language).
    If there was a spectrum from copying on one end to creative production inspired from something else on the other end, the human generally lies heavily on the right end, while the model is much more on the left, that gap is large enough, that yes the model is in some sense “copying”.
    XajniN · 1791099673 · focus · HN ↗
    [dead]
    fasterik · 1791139346 · focus · HN ↗
    Humans are also capable of memorizing long texts and sequences of numbers. It's more a question of scale than a fundamental difference. The point is that LLMs are like human brains in the sense that they're not databases that store text, even though they can and do memorize things.
    Just because a model can reproduce parts of its training set doesn't mean that's what it's doing when it solves a programming problem. It can also reason about the problem, draw on its knowledge of algorithms and data structures, write tests targeting APIs it's never seen before, generate synthetic data and run experiments, etc. etc. Also, the claim that it can reproduce large swaths of its training data verbatim is an empirical one. I would be surprised if it could even reproduce 0.1% of the books it's ingested, for example.
    Saying that AI can't do anything but copy or steal from humans seems to be a rhetorical technique used by people who are still unaware or in denial about the capabilities of the agentic systems released in the past few months. They can now one-shot theorems and programming problems in a few minutes that would be difficult and time-intensive for even the 99.9th percentile human expert.
    sashank_1509 · 1791142569 · focus · HN ↗
    Empirically I just asked Astra without using the internet to exactly reproduce the first paragraph from Hamlet, Othello, Pride and Prejudice, Sartor Resartus by Thomas Carlyle), Sybil by Benjamin Disraeli (1845). GPT-6 got everything correct: <a href="https://chatgpt.com/share/6ac2a72c-e8bc-83e8-8f42-da8c7542decc?ogimg=plain" rel="nofollow">https://chatgpt.com/share/6ac2a72c-e8bc-83e8-8f42-da8c7542de...
    No human on earth can copy at this scale. Yes AI is beyond a database lookup, it does have reasoning on top of this knowledge, but my original claim that if there is a spectrum between copying with minimal changes and creative inspiration with minimal copying, AI is one the copying end while humans are on the creative end. Humans are very bad at reproducing anything verbatim, even if they wrote it like a week ago.
    fasterik · 1791144848 · focus · HN ↗
    Obviously the scale of retrieval from memory is different, I'm not disputing that. The empirical question I was referring to is what percent of its training set the model is capable of reproducing. What looks like "large swaths" of text to us could be less than one in a million for all we know.
    I'm skeptical that there is a single spectrum like you're describing. It's not well defined. Say a human and an LLM prove a new theorem independently (without external help, i.e. from their own neural weights and reasoning). How do we measure how much each of them copied from previous work, as opposed to having learned from or been influenced by it?
    bitexploder · 1791061298 · focus · HN ↗
    It is also easier than ever to build specs and have nice easy to maintain projects. It just doesn't happen magically via few shot prompts :)
  3. fignews · 1791057618 · focus · HN ↗
    Have you considered that maybe this is a reflection of your skills rather than that of the LLM?
    runarberg · 1791057813 · focus · HN ↗
    Meaning OP is a better programmer then a statistical model which produces the most probable results?
    fignews · 1791059104 · focus · HN ↗
    Meaning a bad workman blames his tools.
    nvme0n1p1 · 1791064201 · focus · HN ↗
    A bad workman blames his tools.
    A good workman shuts up and finds better tools without complaining.
    runarberg · 1791078174 · focus · HN ↗
    An expert workman knows which tools to avoid, and explains it to their peers why to avoid said tools.
    sirsinsalot · 1791058188 · focus · HN ↗
    Have you considered it isn't?
    Save your "you're holding it wrong" if you're not going to suggest how to hold it.
    Cult speak escape hatches are intellectually lazy.
    williamcotton · 1791058610 · focus · HN ↗
    I’ve had the best luck by spending quite a bit of time going over the big picture architecture up front and then diving into the modules to further refine the details, making sure to generate step-by-step chunks of work in Markdown format for implementation. I’ll spend literally a couple of days doing this before starting any coding.
    Edit: My latest project is all GPT-6 Astra High. It takes a lot of steering to keep it from adding a bunch of, while useful, features that are not strictly enough to the point. That main issue is it’ll use a lot of extra tokens in the process!
    What was your process?
    hirvi74 · 1791071705 · focus · HN ↗
    Do you mind sharing any code from what you have produced? People talk about LLM successes and failures, but what's there to really talk about when the code can speak for itself?
    In case it is unclear, I am genuinely curious. I have great success with chatbots, but vibing coding has never gotten me further than a proof-of-concept.
    [deleted] · 1791074859 · focus · HN ↗
    [deleted]
    williamcotton · 1791075014 · focus · HN ↗
    Here’s an example:
    <a href="https://williamcotton.github.io/datafarm-studio" rel="nofollow">https://williamcotton.github.io/datafarm-studio
    Some demos of the above charting language:
    <a href="https://williamcotton.github.io/algraf/demos" rel="nofollow">https://williamcotton.github.io/algraf/demos
    WASM, in browser editor, LSP, and more.
    fignews · 1791058956 · focus · HN ↗
    Sure, happy to provide you with an example of how to hold it (turns out Steve was right) =D
    <a href="https://github.com/NousResearch/hermes-agent" rel="nofollow">https://github.com/NousResearch/hermes-agent is 99% (just a guess) LLM generated. 1140 closed pull requests this week. 1.5k closed issues. The github insights page for commits doesn't load for me presumably because it can't handle this scale of commits. But I estimate ~1K commits per day on average.
    There's a blog entry <a href="https://nousresearch.com/refactoring-hermes-with-1393-agents" rel="nofollow">https://nousresearch.com/refactoring-hermes-with-1393-agents that details some work that was done by LLMs to refactor and improve the code.
    I guess they know how to hold it?
    chmod775 · 1791060958 · focus · HN ↗
    You're using quantity metrics to answer a quality question.
    I had a look at the kind of issues that are reported at that project (there's 15k of them, so I can at best assess a couple). It looks like a complete mess: A lot of concurrency and resource mismanagement issues and edge cases that in a better-managed project would have been avoided by construction. They will now will likely be solved by more defensive programming, driving overall complexity ever upwards.
    If you really want to check some quantity metrics to try to reason about code quality, look at whether "fix" PRs are overall LOC neutral or negative (not counting tests). In this project, almost every "fix" is an addition. Worse, almost every fix is more branching.
    If almost every PR is some sort of fix, and most of them add branching, and there's thousands of them weekly... That leads to only one place and I want to be nowhere near it.
    nvme0n1p1 · 1791064113 · focus · HN ↗
    Agreed. As they say, quantity has a quality all of its own.
    Show me an AI that adds features by deleting code (<a href="https://www.folklore.org/Negative_2000_Lines_Of_Code.html" rel="nofollow">https://www.folklore.org/Negative_2000_Lines_Of_Code.html) and I'll pay attention.
    northstar702 · 1791071173 · focus · HN ↗
    [dead]
    diek · 1791072956 · focus · HN ↗
    I find it funny that the original complaint was: "AI made a mess of the codebase".
    Your response was: "Well you're not doing it right, but these hermes devs know what they're doing".
    But the blog post you linked to shows their prompt, which is:
    > I want god files broken up. I want simplification across the board. I want unification of helpers and methods that can be reused. I want less if-if-if-if-if-if-else routing. I want code legibility up. I want interpretability of the codebase and how things connect to each other up.
    So it sounds like AI made their code a mess too. They then tried to make the point of how much money they saved cleaning up the code with AI, that AI made a mess of to begin with.
    And if you look at the merged PRs on that project, a ton of them are bug fixes... to the code the AI wrote. And that's been my personal experience too: AI creates a huge amount of churn in a codebase. Just vast amounts of PRs fixing code that the AI itself wrote.
    catlifeonmars · 1791058882 · focus · HN ↗
    It could be that small variations in prompting lead to large differences in quality of output, especially over longer horizons.
    I’m saying it’s probably multiple factors and both you and GP are right.
  4. satvikpendem · 1791058013 · focus · HN ↗
    When and what model?
  5. Keyframe · 1791058428 · focus · HN ↗
    I'm reading y'all comments and it seems we're still finding our footing, and will for some time. I have the luck that I have access to pretty much all frontier and other models alike. I've been extremely negatively biased towards any LLM use in the start. Then the influx from juniors came, then the revolt of doing PRs on such slop came, then some structured methods how to do LLM work came, then vibe projects came, etc, etc. I literally have all the described experiences you've guys mentioned. From good, to bad, to ugly. It's like there's no one particular way about doing this, and no two projects share the same approach - just like ye olde times.
  6. rickydroll · 1791092022 · focus · HN ↗
    I've had a different experience. My code is better. The ease of refactoring and practicing very defensive programming without getting burned out from the repetitive boilerplate. It all comes together better than it did when I was programming professionally too many moons ago. IFF I don't let the LLM toddler run amok. :-)
    Seriously, I find I need to slow down the rate of change. I don't move forward until I understand the change proposed and have updated the docs. At the same time, I find that keeping up with the LLM/agent is exhausting. 3 hours with an LLM leaves me as tired as 6 hours with a keyboard had previously. I find that coding when tired or fuzzy yields code that shouldn't have been written in the first place. Sadly, once it's been written and debugged, the temporary fix becomes permanent.
  7. cyanydeez · 1791118666 · focus · HN ↗
    I've adopted open source projects and have been working exclusively in languages I have very little experience with: typescript and go.
    I'm using local models, and they go slow enough that I have no trouble following along with what they're doing; but visually, both go and typescript, along with react native, make me puke. So I wouldn't be able to do this without AI.
    I describe how to do it in my comment history, but it's basically a Super-TDD along with some custom engineering harness.
    I don't want to say skill issue, but the same way you can give a chain saw to a teenager and one to a skill craftsman, well, AI can obviously create whatever you want it to do.
    I think some of the variety is simply how fast SOTA models pump out garbage that you simply have to close your eyes because it's not sensible to just watch characters flow across the screen.
    Almost all the coding I'm doing via AI is just faster than readable. But I can see the thinking traces and I stop to model when it's obvious it doesn't understand my intent, etc.
    So I'm not doubting you created garbage. I'm just doubting that it's a product of soley AI use.
2. bitfilped · 1791056769 · focus · HN ↗
  The people saying programming is solved couldn't program in the first place is what I've noticed. So to them it really does feel solved, things "work" and they don't have to learn what they don't know about programming.
  1. orangecat · 1791057564 · focus · HN ↗
    I've been programming professionally for decades. LLMs are extremely useful. At this point if you haven't figured out how to get value out of them, you're either holding them very wrong or you're being willfully ignorant.
    mod50ack · 1791057769 · focus · HN ↗
    I think it depends which LLM tool you're using. If you're using an older, worse model (the kind that you can use for free), the experience is significantly more frustrating. On the other hand, I'd say that the current best models are very useful with a skilled operator.
    drewstiff · 1791063781 · focus · HN ↗
    I would argue that using an older model and assuming that it is the cutting edge is basically equivalent to holding it wrong
    Keyframe · 1791058506 · focus · HN ↗
    You both can be right at the same time.
  2. bigstrat2003 · 1791059030 · focus · HN ↗
    There are a couple of notable counterexamples here (nobody sane thinks Carmack doesn't know how to program, for example), but by and large I agree with your observation. The people excited about programming with LLMs are, on average, people who weren't good at programming to begin with. Still, given that these counterexamples do exist I try to avoid painting with an overly broad brush for the sake of nuance.
3. nicoburns · 1791056967 · focus · HN ↗
  Were you reviewing the changes? I've seen lots of projects with this problem. But if AI PRs have to pass the same review bar as any other PR then it shouldn't be an issue in theory (if you can actually maintain the discipline).
  1. thesdev · 1791057371 · focus · HN ↗
    Isn't it tiring to keep up with considering the speed of the generated code? And if you want to be careful with the review you lose a significant part of the speed advantage.
    satvikpendem · 1791057878 · focus · HN ↗
    It's the same as any other PR. Keep the changes small and contained to that feature. AI can do this if you instruct it to, there is no need to vibe code some 40k line monstrosity.
    classified · 1791058033 · focus · HN ↗
    Having unmaintainable code faster is only an advantage if it's a one-shot throwaway artifact.
    drewstiff · 1791063680 · focus · HN ↗
    Assuming we are measuring time and
    `total = dev + review`
    If dev approaches zero, but you review at the same pace as you always have, are you in a better position? Yes.
    Will you potentially have a backlog of code waiting for review? Also yes.
    Would you prefer to be waiting for the dev team for all of the time instead, then still have the same amount of reviewing to do at the end of it? Absolutely not.
    nicoburns · 1791071042 · focus · HN ↗
    Yes, but it's the only option if you want to retain a maintainable code base. And you can always slow down. You'll probably still be a bit faster than you were before.
4. analog_daddy · 1791057274 · focus · HN ↗
  Oh my god. I am sad to have to acknowledge this even if the models have “theoretically” gotten better. I am not a programmer but I am super opinionated with the design and structure of the code and need to make sure whatever I write/ask to generate and use for my own tasks is understood by me at least once. I wasn’t this way in 2023. I am constantly reading/trying for an agent to help me write simple code without spending too much time. But so far nothing has worked. I wish someone figures it out otherwise, I personally would not be able to realize the AI agent productivity benefits that others are supposedly seeing.
  1. zanderwohl · 1791059512 · focus · HN ↗
    If you specify a structure, pattern, or design, it will stick to that design after a code review phase. Just tell it what standards you have, and after two rounds, it will have written what you described.
5. satvikpendem · 1791057983 · focus · HN ↗
  As usual, when people post comments like this they never specify what model and when they used it. There is a huge difference between GPT 3 and 6 for example.
  Late 2025 also had a step change when agents could largely code autonomously without handholding like previously, and to be honest it's not worth hearing opinions about AI from before that time, that's how significant the change was.
6. mulemisterX · 1791058411 · focus · HN ↗
  I've had the opposite experience. Spec driven development really makes things much easier to maintain. If you're just yolo'ing it and throwing prompts around things will fall apart fast.
7. Dfiesl · 1791058597 · focus · HN ↗
  Were you reading through the code it was generating to make sure the flow was intuitive and comprehensible for each PR? Projects only turn in to maintainable messes if you blindly merge in unmaintainable messy code.
8. joshheitzman · 1791059520 · focus · HN ↗
  People, especially those who don't do software development, often conflate coding and software development.
  The models aren't good at architecture and design. But they take direction on architecture and design and design well and can refactor code quite effectively. AI agents can absolutely be used to clean up vibe coded code bases once you figure out if the investment is worth it. The mess can be avoided if you give them sufficient guidance on architecture and design upfront.
  That said, doing so purely in text form doesn't feel great right now. I've been thinking about UML lately. The problem with that was the roundtrip after the code was generated and then the implenetation happened. I don't necessarily think UML is the solution, but neither is walls of dense text.
  1. PorciiVorbesc · 1791060192 · focus · HN ↗
    >The models aren't good at architecture and design.
    Can you elaborate to back up this claim? WHat exactly is your yardstick for "being good at SW design and architecture"?
    Because I found the current SOTA AI models being great at architecture and design, much better in fact than most average real-world devs. Is your yardstick just the John Carmacks of the world by any chance? Because most devs are not John Carmack. They are also not Linus Torvalds, they are not Stallmann, etc.
    Maybe your LLM experience is still stuck in the 2023 era of ChatGPT?
    joshheitzman · 1791067203 · focus · HN ↗
    My yardstick is me: <a href="https://www.linkedin.com/in/joshheitzman" rel="nofollow">https://www.linkedin.com/in/joshheitzman
    PorciiVorbesc · 1791067438 · focus · HN ↗
    Cool. Can you elaborate at which types of task you are better than SOTA LLMs in context of "being good at SW design and architecture"? Got any examples? Is it at the interview questions? Or real world problems? If so what is the scale of the real world SW design and architecture problems you're better than the SOTA LLMs? Is it FAANG scale or mom and pop shop scale?
    And do you consider yourself to be representative of the average developer, above them, or below them?
    LLMs don't even need to be better than the average dev, let alone the top performing ones, like you. If they can just be better than the bottom 20% of devs and white collar workers in general(easily achievable when you've been around the block and saw how many useless people just keep warm chairs for high wages in large companies), that's already a huge win for those products.
    What I mean, at a previous job I had ran into a memory leak issue in our backend and discovered a colleague pushed a library into prod which came with comments in the source code saying "DO NOT USE IN PROD, IT CAUSES A MEMORY LEAK!". There's cases where human stupidity and carelessness far surpasses whatever issues LLMs cause so maybe the average dev isn't really that much better than the SOTA LLMs.
    joshheitzman · 1791073187 · focus · HN ↗
    I'm currently driving my AI coding agent harness through the process of refactoring itself. This is the last big refactor I need done before I can polish and release it as OSS later this month. So its not a large scale project I'm working on at present. The general problem is that mostly add new code and try to minimize editing existing code, which effectively means the code base is grown organically rather than being intentionally architected and designed. A few of specifics:
    1) duplication - LLMs are great at generating lots of text, so its faster and easier for them to generate entirely new facilities that overlap heavily with existing ones then it is for them find existing facilities that should be expanded and refactored (note I just said 'find'; actually editing raises the time and difficulty even more). This is fine for a while as the duplicate facilities usually work just fine, up until something needs to be changed across all of them and they miss changing one or more of them, things break, and a bunch of tokens have to be burned tracking down the issue.
    2) ever increasing surface area - even when making changes that do expand a facility without much duplication they frequently only add without removing much of anything or changing the overall design of the facility to reduce the amount of state its tracking and the number of branches it has based on that state. I've never seen one decide to split up something large or with too many responsibilities on their own. They will happily create a god class or function and just keep making it bigger.
9. xenadu02 · 1791060385 · focus · HN ↗
  AI is a useful tool but it absolutely needs a lot of human guidance.
  Unlike a compiler it won't give up at the first sign of trouble but that just means it left alone it will dig bigger and bigger holes.
  Treat prompt engineering as a discipline and refine your technique. When it produces garbage throw out the work and start over until you figure it out.
10. vjvjvjvjghv · 1791060749 · focus · HN ↗
  I treat AI like an intern or junior team member. With enough guidance, they can contribute a lot but you can't let them loose without supervision or they usually will produce a big mess. As of now, you are still responsible for overall architecture. One strong indicator that something is going wrong are big pull requests where the AI has rewritten large sections of the code.
11. cortesoft · 1791061483 · focus · HN ↗
  I am not saying you are right or wrong, but I don’t think you can make such a broad conclusion simply because of your own experience.
  You say you used AI and your projects turned into unmaintainable messes, so your conclusion is that it means AI is not living up to its promises.
  I guess if the argument is “AI makes it so you always get a great result no matter how you use it”, then your argument is sound. Your projects not working out proves that AI doesn’t always work no matter what.
  However, that doesn’t mean you can’t use AI to create sustainable and well organized code. Failing to do something doesn’t mean it’s impossible and anyone who thinks they can is not paying attention.
  I can’t run a marathon. If I went out and tried to run one, I would get a few miles and collapse, failing completely.
  I don’t think it would be reasonable, though, at that point to say “running a marathon is impossible, anyone who says they can do it clearly lying. I tried and didn’t even make it 5 miles!”
  I wish people would stop assuming their experience with something is the only possible truth.
  1. threethirtytwo · 1791062327 · focus · HN ↗
    Stop being overly polite to these people. It is seriously coming down to the point of utter delusion.
    As ai becomes better these people will begin changing their story because it’s utterly obvious what’s happening.
12. TuxSH · 1791062315 · focus · HN ↗
  Vibe coding has always had and always will have one fundamental flaw: you didn't write the code it produced, therefore you don't have a good understanding/mental model of it.
  On the other hand asking these clankers "review the feature branch I wrote" and "review my entire codebase for bugs" or "help me debug this" has saved me months of prospective work.
  And more recently most major models have been getting _really_ good at RE, for example you can have OAI models (and maybe A/'s if they don't refuse) use idalib MCP and reverse-engineer stuff from start to finish, then follow up with GLM 5.3 for vuln assessment and exploit PoC.
  Stuff that used to take weeks or months now just takes a few hours, or less.
13. kees99 · 1791105587 · focus · HN ↗
  "Coding is solved" is a very high bar, and current-gen AI/LLM is very far from that.
  Where AI did make a lot of impact is triage. I can throw a messy bug report of an intermittent issue at claude, and tell I need issue reproduced and fix developed, and there is a well above 50% chance it'll deliver. Never commit that fix to the codebase as-is, of course.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.