Thread

1. lil-lugger · 1790019853 · focus · HN ↗

   Is there anyone serious who thinks that the future is local models anyway? All computers used to be the size of rooms like these data centers and then they got smaller and faster until the home computer came. Is that not a possibility down the line as we improve efficiency of the models and increase compute?

   1. latchkey · 1790020406 · focus · HN ↗

      Home computers are still nowhere close to as powerful as a $500k+ 10kW server full of 1.5TB of HBM GPU compute.

      1. Zetaphor · 1790021085 · focus · HN ↗

         You don't need a data center to get real work done. Not every task requires "PhD level intelligence"

         1. latchkey · 1790021330 · focus · HN ↗

            That's the usual response, along with "you can't compete with free". But, look at how much money the frontier models are printing, it is obvious that the bell curve of usefulness is still centered around them.

            Let's also not forget that even the open models are not getting smaller, they are getting larger. Of course, you can distill them down into something that will fit on smaller compute, but at the end of the day, the data centers of compute, still play a huge role.

            1. ryanbrunner · 1790021970 · focus · HN ↗

               An easy counterargument is that the frontier models are swallowing all of the attention and money because they currently don't have a constraint of demonstrating that their value exceeds their cost. Take that away and it might turn out that cheaper models that are "good enough" and can be profitable are the more attractive route.

               1. Zetaphor · 1790022228 · focus · HN ↗

                  Well said, they're competing on marketing rather than merit. Everyone defaults to the frontier instead of exploring more optimal options
               2. latchkey · 1790022377 · focus · HN ↗

                  > they currently don't have a constraint of demonstrating that their value exceeds their cost

                  There is no loyalty in AI. I can switch to another model at near zero cost. They absolutely do have to demonstrate value. The concept of "good enough" is a misnomer because we're talking about putting these products into the hands of people who need to generate value from them.

                  1. ryanbrunner · 1790126725 · focus · HN ↗

                     They have to demonstrate value absolutely, but you're forgetting about the denominator of cost, and to a large degree so is the industry right now.

                     To take this argument to an absurd level, if Fable generating a feature cost $100,000 and Sonnet generating the same feature cost $1, it wouldn't be enough that Fable is marginally, significantly, or even dramatically better, because the cost difference is so outsized. For a variety of reasons, the true cost of tokens for frontier models is downplayed at multiple points (whether it's through AI companies running off funding, companies prioritizing AI adoption over AI costs, etc.)

                     If cost is not an issue at all, the best thing is always preferable regardless of cost. But in most cases, and AI is no exception, eventually cost will be an issue.

Unofficial Hacker News client; not affiliated with Y Combinator.