‹ BackHN Continuity

Thread

If AI coding is lowering your code quality, you're not managing quality right

129 points · 169 comments · bucket2015

  1. axegon_ · · focus · HN ↗
    Ah, the "skill issue" argument again. Same crap aswhen everyonewas worshiping Musk 5-6 years ago, this time it's dario and altman with a claude/chatgpt mask. Crash can't come soon enough.
    1. skybrian · · focus · HN ↗
      And why wouldn’t writing software be a skill issue? Yes, it’s an annoying meme, but we should expect that there are better and worse ways to write software. It would be weird if everyone got the same results regardless of experience.

      I’m doubtful that the author’s recommendation always work, but I do some similar things and they do seem to help.

      1. malfist · · focus · HN ↗
        Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code. Trust me, I've seen enough PRs to last a life time. A lot of professionals wouldn't know good code if it slapped them in the face.
        1. asutekku · · focus · HN ↗
          I'd argue for most people LLM produces much better code than they would be able to write themselves.
          1. malfist · · focus · HN ↗
            That is not an argument that LLMs produce good code
            1. bluGill · · focus · HN ↗
              They produce good code when I'm personally reviewing them. There are a few other people who work with who likewise know how to review code and thus can get good code out of an LLM. There are, however, a lot of people who just accept the first slop that they get out of it and that's not good code.

              The larger issue of good code isn't the actual individual lines, it's the overall architecture. And that's what I'm going to be reviewing first is, is this a good approach? Then the interfaces to other code is this a good interface. Get those two right and we can go back for the details. In a lot of cases, the LLM is plenty good at those details.

              In some cases, an LLM is better than what I could do. Well, I suppose I can trace down all the locks in all the different special cases, and I have done that, but that was a huge amount of effort that I really don't want to repeat.

              Note that I'm talking about recent models. If you're asking about the models of just one year ago, I would give a very different answer about the type of code an LLM produces.

          2. abalashov · · focus · HN ↗
            I'm not sure how literally you mean "most people". This might be true in a purely volumetric sense, but that's not really the bar around these parts...
        2. preg_match · · focus · HN ↗
          You can most definitely produce high-quality code via an LLM, particularly if you test your code aggressively and review it. Yes there are a lot of shoddy PRs, but that's nothing too new. The problem with LLMs is the amount of code they produce. More code = more garbage. But, that code velocity can be leveraged to increase quality. Through careful design and testing.

          Ultimately, I would take LLM code + high-quality multi-strategy testing over human code with little to no tests. And some would say "well that's a false dichotomy". I disagree, before LLMs engineers didn't have the time or incentives to aggressively test. The tests either would not exist, or would be shitty unit tests intended to get an arbitrary coverage percentage. Now, we can write high-quality tests, differential testing, fuzzing, and more, in much less time.

          1. rented_mule · · focus · HN ↗
            Going much deeper on tests has been transformative for me. In a solo project started from scratch, I'm 6-7 weeks in, and it's up to ~90K lines. ~60K of those are tests. Those tests have now found (and then the agent has correctly diagnosed) multiple bugs in broadly used libraries that I'm using in my project. That's because those bugs surfaced as occasional issues in my project. Especially powerful are all the property-based tests (perhaps what you are calling fuzzing? I'm using the Python package called Hypothesis for this).

            Another spectrum that I've found useful to explore is the scope of what I ask the coding agent to do in one turn. I see some people trying to do one massive prompt that the coding agent works on for a day or more. I find a large boost in overall quality if I do 10-20 prompts per day (not counting the prompts where I'm just trying to understand things). It's still much less of my time than hand-coding, but the resulting architecture looks like my own. The quality of the overall system is great. There are certainly issues here and there in the code, but it's always that way once a project gets large enough. Now it's easier to address any particular issue throughout the code base in one go.

          2. maksDanylenko · · focus · HN ↗

            [dead]

        3. deterministic · · focus · HN ↗
          > Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code

          Sorry, but you are 100% wrong.

          I have 30+ years of professional development experience working on complex, very large-scale C++ code used by companies around the world.

          I care deeply about code quality and always have. More than any other developer I've worked with in my 30+ year career. And I'm now using Claude Code to push the quality bar much higher.

          But you have to learn how to use it properly. It's a tool. Quality doesn't happen automatically.

          It's a big mistake, and frankly quite arrogant, to assume that because it doesn't work for you, it can't work for anyone else. Or that the rest of us must either be lying or incompetent.

          1. player1234 · · focus · HN ↗

            [dead]

          2. blub · · focus · HN ↗
            It’s fair to ask people bragging about their amazing AI skills to show their code or GTFO. Hope it becomes established.
            1. deterministic · · focus · HN ↗
              I work on proprietary software, so no, that’s not really a fair question to ask.

              However, if you’re willing to share how you’re prompting the AI and some examples of where you think the results are poor, we might be able to help identify what’s causing the difference. I’d be genuinely interested in understanding why we’re getting such different results.

              Rather than assuming one of us must be wrong, it would be more useful to compare approaches and see what we can learn from each other.

      2. tyleo · · focus · HN ↗
        Not only that, but you really want it to be a skill. The book _Making Software_ describes skills as things you can get better at through practice, and talents as things you're born with.

        I'd like to think the time and practice I've put into software engineering has made me better at it. If that's not true, then there's no reason to prefer senior or principal engineers with years of experience over newcomers.

      3. pydry · · focus · HN ↗
        >And why wouldn’t writing software be a skill issue

        You've missed the point. Nobody doubts writing code well or badly is indeed a skill issue.

        The question is that "once you account for all of the things you need to do to make the code very high quality, did vibe coding actually provide any real value?"

        I'm certain there are guardrails that help bolster vibe coding but I'm equally certain that when ive prompted something important I usually have to redo it enough times that just writing it manually myself usually would have been quicker.

        Then I watch other people who code who dump on that opinion and I see total slop. They just can't tell the difference.

        1. AndrewKemendo · · focus · HN ↗
          I hate typing and like reading

          That seems to be the primary difference I’ve found between people who embrace gen code and those who dont

          The ones who dont, seem to like the physical act of typing, and that tends to cluster with people who write software all day

          1. pydry · · focus · HN ↗
            high quality code means boiling down code to its bare essentials, not spewing boilerplate. that means deleting code where you can and crafting good abstractions while leaving functionality intact.

            if you find the ratio between typing and thinking to be very high then you're probably producing a lot of slop.

            This is a common theme I find when I hear about people's AI coding success stories. Where they say "its good at X" where X might be "backfilling unit tests" or "writing boilerplate" I usually think "if you find you need to do X a lot youre definitely doing programming wrong.

            Ive actually yet to hear an X applied to production code that doesnt make me think that.

            1. Daishiman · · focus · HN ↗
              > Ive actually yet to hear an X applied to production code that doesnt make me think that.

              This is one of those things we value in theory in engineering but not in practice. Reducing code as an artifact might mean coming up with clever ways or compressing data, like making code that generalizes and abstracts. This is fine if you're experienced and clever. But a lot of organizations don't have that many clever or experienced engineers and those tools cause more harm in the hands of those people. Hence compromises must be reached and verbosity is valued because it is explicit.

              I used to believe otherwise but then I worked in larger orgs with a lot of mediocre people who still provided value but needed to be given the means to add value.

          2. Sharlin · · focus · HN ↗
            My experience is that most people don't enjoy code review, which is why it must usually be actively encouraged and not just something that happens naturally.

            Mechanically writing boilerplate is not enjoyable, and unfortunately in some languages and domains most of the coding is writing boilerplate. Machines can help with that no problem.

            What is presumably enjoyable to most programmers is writing the parts where the actual magic happens. The translation of informal ideas into formal representation has beauty, like mathematics has beauty. Designing and implementing structures of code and data that are as simple as possible, but not simpler, is rewarded with a feeling of artisanal satisfaction and pride. Few things in life are as satisfactory as figuring out an elegant solution to a challenging problem.

            None of the above are necessarily bound to the actual typing of words and symbols. AIs can help with all of them, and act as a genuine force multiplier. I would describe that as "responsible use of AI". Unfortunately, it seems that incentives are often against such use.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.