‹ BackHN Continuity

Thread

Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

414 points · 114 comments · moonikakiss

  1. mrinterweb · · focus · HN ↗
    There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.
    1. benjiro29 · · focus · HN ↗
      > Claude code does some of this by handing off the "explore" agent work to haiku.

      That is not handing off to a specialized model, its just handing off to a lighter and interior model (compared to the parent model). That by itself can create issues like the lighter model not capturing all the data that the parent needs.

      The idea is that we get specialized models that are better then general purpose models. But its rare for a specialized model to beat a strong general model.

      There is a reason why we hear less about this idea of smaller expert models, because large strong models to the tasks just as good.

      And if the tasks is repetitive to the point that specialization is useful, you can get into a situation that your better off having a program written for that reputative nature, then delegating to other models. And then have the main strong model, deal with the (semi)cleaned up data.

      1. kumama · · focus · HN ↗
        castform founder here. while it's "rare for a specialized model to beat a strong general model" today, i think the tech/knowhow on how to do so is getting there. we see some early signs of this in industry e.g.

        <a href="https:&#x2F;&#x2F;shopify.engineering&#x2F;sidekicks-continual-learning-loop" rel="nofollow">https:&#x2F;&#x2F;shopify.engineering&#x2F;sidekicks-continual-learning-loo... <a href="https:&#x2F;&#x2F;thinkingmachines.ai&#x2F;news&#x2F;learning-to-replicate-expert-judgment-in-financial-tasks&#x2F;" rel="nofollow">https:&#x2F;&#x2F;thinkingmachines.ai&#x2F;news&#x2F;learning-to-replicate-exper... <a href="https:&#x2F;&#x2F;castform.com&#x2F;blog&#x2F;elsa&#x2F;" rel="nofollow">https:&#x2F;&#x2F;castform.com&#x2F;blog&#x2F;elsa&#x2F;

        at the end of the day, models are only as good as the data they&#x27;re trained on. and if one has access to proprietary data, it should yield specialized models that do better than larger general models

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.