‹ BackHN Continuity

Thread

HarnessTax: How Much Does the Harness Matter for Coding Agents?

233 points · 99 comments · matt_d

  1. lukax · · focus · HN ↗
    What matters more is that you use the tools that the target model was fine-tuned on.

    E.g. for editing files with Claude models you should use Edit(file_path, old_string, new_string, replace_all) but with GPT models you should use apply_patch_call(patch) (where patch is a custom patch string with custom grammar).

    It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools.

    <a href="https:&#x2F;&#x2F;lucumr.pocoo.org&#x2F;2026&#x2F;7&#x2F;4&#x2F;better-models-worse-tools&#x2F;" rel="nofollow">https:&#x2F;&#x2F;lucumr.pocoo.org&#x2F;2026&#x2F;7&#x2F;4&#x2F;better-models-worse-tools&#x2F;

    1. kouteiheika · · focus · HN ↗
      &gt; It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools.

      This is true. For example, the other day I was doing some testing with DeepSeek-V4-Flash-0731, and I gave it a tool called &quot;EditFile&quot; with &quot;old_content&quot; argument. It ignored the tool definition and called it with &quot;old_string&quot; instead, presumably because it was trained on a bunch of Claude Code traces.

      So either make sure your tools look nothing like the tools in the popular harnesses, or make them similar.

      1. raffael_de · · focus · HN ↗
        &gt; or make them similar.

        identical. similar is what you tried and it didn&#x27;t work.

        1. kouteiheika · · focus · HN ↗
          No. You don&#x27;t need identical. For example, I have extra arguments in my edit file tool that the model can specify, and that works just fine.
          1. dns_snek · · focus · HN ↗
            Backwards compatible then.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.