Thread

1. AHASIC · 1785816514 · focus · HN ↗

   I read a comment on here a few months back I wanna restate. Basically, there is a good chance that Apple is betting that the LLMs in the future will be so efficient that those that consumers will use everyday will be easily computed by the iPhone or even bigger ones on Macs. Honestly makes the most sense that we are heading that way in a few years latest.

   1. Mistletoe · 1785817190 · focus · HN ↗

      What hardware advances would we need to see for that to happen? It feels like everything in that arena has kind of plateaued.

      1. harrouet · 1785825050 · focus · HN ↗

         I could definitely image Apple embedding a kind of LLM-optimized FPGA: slow to load (update) an LLM, but blazing fast at computing tokens.

         Who needs memory when your model is set in silicon ?

         1. KeplerBoy · 1785832735 · focus · HN ↗

            You don't an FPGA if you're taping out your own chips. But that is just a MMA accelerator with decent memory bandwidth. No secret sauce here.

            1. harrouet · 1785845970 · focus · HN ↗

               I am talking about reconfigurable gates to implement an LLM in silicon, i.e. an FPGA...

               1. xprnio · 1785853645 · focus · HN ↗

                  What might the “parameters/layers to gates” ratio look like? My naive and uninformed guess would be that 1B+ parameter would also need a 1B+ gate FPGA, but according to google they typically range from tens of thousands to several million (which would still be a fraction of a billion).

Unofficial Hacker News client; not affiliated with Y Combinator.