Thread

1. BedVibe_Studios · 1785961746 · focus · HN ↗

   This feels like the database equivalent of "use the right data structure." We've spent two years assuming the biggest general-purpose model should do everything. It makes more sense for retrieval, reranking, reasoning, and generation to each have their own optimized model if the routing cost is negligible.

   1. seahyinghang8 · 1785982994 · focus · HN ↗

      founder of castform here, we believe that as agent deployment moves from experimentation phase where cost doesn't matter as much to deployment (what's the margin of serving the request), there will be a rise in interest in optimized models.
   2. nullbio · 1785998580 · focus · HN ↗

      Yes, and it's only logical things move in this direction because there's massive hardware incentive to do so. If frontier models can be broken down into small models, networks of smaller-GPUs can be utilized. Right now the smaller GPUs are basically paper weights for frontier intelligence.

Unofficial Hacker News client; not affiliated with Y Combinator.