Thread

1. sneak · 1790665787 · focus · HN ↗

   Seriously though, what are the low cost chips that can usefully run LLMs? Is a Mac Mini the lowest we can go? Are there iGPUs on mini-itx that can do it, or are there dedicated AI chips that one could turn into a pi HAT?

   1. Risse · 1790669436 · focus · HN ↗

      Depends on what you consider to "usefully run LLMs".

      Earlier this year, I bought a mini pc from Aliexpress, specs are roughly Ryzen H255, 24GB LPDDR5, 1TB SSD. This was around 350€ including VAT, customs, shipping etc. I would personally consider this somewhat of a lowest class of useful LLM box. It can run 8B models well, up to somewhere around 24B. I currently run Gemma 4 26B A4B Q5 on it, with MTP, and it is quite slow, but smaller models would run okay on it.

      1. bahmboo · 1790680788 · focus · HN ↗

         Let's teleport back 10 years and what you have is a magic box that could make you billions.
   2. chorylee · 1790686952 · focus · HN ↗

      An Orange Pi 5 Max does this job for real. It's an RK3588 board — $75 for 4GB, $95 for 8GB on AliExpress. A community test got Qwen2.5-0.5B at about 12 tok/s on the CPU via llama.cpp. The chip also has a 6 INT8 TOPS NPU if you'd rather go the RKNN route. Won't beat a Mac Mini, but it's an actual computer for under a hundred bucks.

      1. cameron_b · 1790694757 · focus · HN ↗

         I regret to inform you that prices have long departed the lower atmosphere. Even on AliExpress, you're looking at a few hundred bucks for those. Still less than a Mac Mini, but less less.

Unofficial Hacker News client; not affiliated with Y Combinator.