Thread

1. mrinterweb · 1785956587 · focus · HN ↗

   There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.

   1. benjiro29 · 1785962627 · focus · HN ↗

      > Claude code does some of this by handing off the "explore" agent work to haiku.

      That is not handing off to a specialized model, its just handing off to a lighter and interior model (compared to the parent model). That by itself can create issues like the lighter model not capturing all the data that the parent needs.

      The idea is that we get specialized models that are better then general purpose models. But its rare for a specialized model to beat a strong general model.

      There is a reason why we hear less about this idea of smaller expert models, because large strong models to the tasks just as good.

      And if the tasks is repetitive to the point that specialization is useful, you can get into a situation that your better off having a program written for that reputative nature, then delegating to other models. And then have the main strong model, deal with the (semi)cleaned up data.

      1. BikiniPrince · 1785963940 · focus · HN ↗

         You can register models with mcp. I think it’s an expensive solution, but it is available in the framework. I use a light weight bus protocol that lets agents interact and pass short messages with pointers. It’s very efficient.
      2. tyre · 1785970155 · focus · HN ↗

         > There is a reason why we hear less about this idea of smaller expert models, because large strong models to the tasks just as good.

         Smaller models are cheaper, sometimes faster. I agree that the “we’re an LLM fine-tuned for X” hasn’t worked out because you can just train Claude to do X (and Anthropic will), but not burning Opus/Fable tokens on dumb-but-token-heavy tasks is good sense.

         As we move from “integrate AI into Y” to “optimize the ROI on Y”, we’ll see more of this.

         1. kumama · 1785982579 · focus · HN ↗

            castform founder here. the roi optimization makes sense. i think there are lots of usecases for which even a 2% gain in accuracy can be quite useful. off the top of my head

            - high volume customer support. higher accuracy means fewer escalation, reducing labor costs - fraud detection. catching even one extra fraud attempt could mean a lot in savings - and ofc the classic ads use-case where at scale bps in improvement could mean millions in revenue :)

            1. simianwords · 1786001448 · focus · HN ↗

               Would a bigger model be able to beat yours if some effort were put into prompt?

               1. dkersten · 1786010596 · focus · HN ↗

                  At what cost though? “Beating” isn’t enough if it costs 100x or even 10x the amount.
               2. kumama · 1786025579 · focus · HN ↗

                  the bigger model would still cost more :) at the same time, i see prompting as being orthogonal to post-training. i'd imagine post-training a smaller model with a better prompt would make it perform even better

                  1. simianwords · 1786040990 · focus · HN ↗

                     yes, can you show me tasks where this is true?
      3. kumama · 1785980993 · focus · HN ↗

         castform founder here. while it's "rare for a specialized model to beat a strong general model" today, i think the tech/knowhow on how to do so is getting there. we see some early signs of this in industry e.g.

         <a href="https:&#x2F;&#x2F;shopify.engineering&#x2F;sidekicks-continual-learning-loop" rel="nofollow">https:&#x2F;&#x2F;shopify.engineering&#x2F;sidekicks-continual-learning-loo... <a href="https:&#x2F;&#x2F;thinkingmachines.ai&#x2F;news&#x2F;learning-to-replicate-expert-judgment-in-financial-tasks&#x2F;" rel="nofollow">https:&#x2F;&#x2F;thinkingmachines.ai&#x2F;news&#x2F;learning-to-replicate-exper... <a href="https:&#x2F;&#x2F;castform.com&#x2F;blog&#x2F;elsa&#x2F;" rel="nofollow">https:&#x2F;&#x2F;castform.com&#x2F;blog&#x2F;elsa&#x2F;

         at the end of the day, models are only as good as the data they&#x27;re trained on. and if one has access to proprietary data, it should yield specialized models that do better than larger general models

      4. nullbio · 1785997626 · focus · HN ↗

         &gt; The idea is that we get specialized models that are better then general purpose models. But its rare for a specialized model to beat a strong general model.

         Are you sure about that? I mean, MoE is basically an array of specialized models. This beats single-model general purpose performance.

         Clever routing systems also seem to indicate frontier or frontier-beating capabilities at reduced cost, such as Fugu.

         Further to this though, I think it hasn&#x27;t been pushed hard enough. We&#x27;re not going down far enough in the abstraction chain. Most peoples idea of &#x27;specialized models&#x27; is an LLM trained on a particular domain (like math vs coding vs creative writing), where-as it should be going lower-level than that and incorporating not only more abstract and smaller specializations of LLMs, but entirely different types of AI models, classifiers, etc.

         I also think the reason we don&#x27;t really hear much about this (yet) is because there&#x27;s no incentive for the best AI researchers&#x2F;labs to go down this direction because it&#x27;s better suited for local-processing over your entire codebase, but the labs all want you to perform inference on their servers and spend your money there, rather than spend your own compute to increase intelligence. They have a larger incentive to try and scale in different ways. We&#x27;ll have to rely on open-source for this one.

Unofficial Hacker News client; not affiliated with Y Combinator.