Thread

1. sneak · 1790665787 · focus · HN ↗

   Seriously though, what are the low cost chips that can usefully run LLMs? Is a Mac Mini the lowest we can go? Are there iGPUs on mini-itx that can do it, or are there dedicated AI chips that one could turn into a pi HAT?

   1. Risse · 1790669436 · focus · HN ↗

      Depends on what you consider to "usefully run LLMs".

      Earlier this year, I bought a mini pc from Aliexpress, specs are roughly Ryzen H255, 24GB LPDDR5, 1TB SSD. This was around 350€ including VAT, customs, shipping etc. I would personally consider this somewhat of a lowest class of useful LLM box. It can run 8B models well, up to somewhere around 24B. I currently run Gemma 4 26B A4B Q5 on it, with MTP, and it is quite slow, but smaller models would run okay on it.

      1. bahmboo · 1790680788 · focus · HN ↗

         Let's teleport back 10 years and what you have is a magic box that could make you billions.

Unofficial Hacker News client; not affiliated with Y Combinator.