Thread

1. dghlsakjg · 1785816328 · focus · HN ↗

   I know everyone wants to crap all over these setups that are impractical, but this is how progress happens.

   People will keep plugging away at this and figure out how to avoid wearing the hard drive, how to make it run faster, custom hardware buses etc.

   Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.

   1. gpt5 · 1785824544 · focus · HN ↗

      Most people are already used to rely on the internet on basically everything. At best, they download a tiny chunk of entertainment from it when they go on a plane, and as soon as they land they immediately abandon that offline chunk.

      In addition, LLMs, small or large, are highly parallelizable. This means that running on the same machine/GPUs many requests in parallel is significantly more efficient, and the sum of tok/s will far outweight any single user use case.

      Those two combined means that unless LLMs reach the state of 'good enough' (TBD), I fully expect the economics and people's interest to align with 99%+ of LLM usage would be in centralized servers. (not dismissing the fact that there are use cases for local LLMs, and that the progress matters).

      1. dudefeliciano · 1785843822 · focus · HN ↗

         > Most people are already used to rely on the internet on basically everything

         FAANG has trained them well. "Normal" people also use chatGPT & co. to ask mundane questions that qwen27b could easily answer (random trivia, writing emails, correcting translations etc), why do these simple requests need to be processed on a B300 on server on the other side of the planet? There is no reason.

         And this being hackernews, I would expect people here to go against the grain in this regard...maybe even promote local llms to "normies".

Unofficial Hacker News client; not affiliated with Y Combinator.