Thread

1. AHASIC · 1785816514 · focus · HN ↗

   I read a comment on here a few months back I wanna restate. Basically, there is a good chance that Apple is betting that the LLMs in the future will be so efficient that those that consumers will use everyday will be easily computed by the iPhone or even bigger ones on Macs. Honestly makes the most sense that we are heading that way in a few years latest.

   1. Mistletoe · 1785817190 · focus · HN ↗

      What hardware advances would we need to see for that to happen? It feels like everything in that arena has kind of plateaued.

      1. ethersteeds · 1785822215 · focus · HN ↗

         I think a major factor is memory bandwidth. Apple has raised it steadily for each M series generation, and that hasn't plateaued.

         Nvidia leads in bandwidth and specialized architecture, but local inference takes off when it's usably fast at much lower cost and power consumption.

Unofficial Hacker News client; not affiliated with Y Combinator.