Thread

1. nojs · 1789618135 · focus · HN ↗

   We really need better harness benchmarks. It seems there's no reliable source that benchmarks the main harnesses against all open source models.

   I also wish the discussion around Pi did not always use cost/token count as the metric. It's amazingly token efficient, but how does it stack up again opencode and others if you don't care about token count?

   My experience is that the harness is mainly polish preventing failed tool calls, bad edits, stuff like that, but doesn't make much difference to the overall "intelligence". But that opencode seems slightly more robust against stupid errors than out of the box Pi due to the additional context it forces through every thread.

   1. calgoo · 1789626263 · focus · HN ↗

      I really disliked how opencode works IMO; the harness tries to do to much in my mind. Switching to Pi was a breath of fresh air for me, and I even use hax for some of my local needs where i dont want to have the giant pile of fertilizer that is NPM or PIP installed.

      The harness becomes more and more important, the smaller the model is as you need to offload context management as well as memory to the harness. The big models basically just need a bash prompt tooling and you let the model manage everything inside its own context.

      1. mobelkh · 1789636304 · focus · HN ↗

         what does it do too much out if the box? afaik it only has the basic tools and two agents profiles, maybe lsp support is extra?

         1. tacomagick · 1789639368 · focus · HN ↗

            Moving from CC I can agree OpenCode seems very simple. I don't see what is too much about it honestly.

Unofficial Hacker News client; not affiliated with Y Combinator.