Thread

M5 Ultra Mac Studio Review

269 points · 262 comments · piotrgrabowski

simonw · 1790001122 · focus · HN ↗
The numbers I was most interested in are tucked away in a chart towards the bottom - the speed comparison of the Mac Studios v.s. a RTX 5090:
```
  Qwen3.8 27B tokens&#x2F;sec generation speed

  Prompt size    8K    64K   128K   256K
  RTX 5090 PC    59    51    44     n&#x2F;a
  M5 Ultra       48    39    32     24
  M3 Ultra       31    23.5  20     15
```
A whole bunch more comparison numbers in this section: <a href="https://www.macstories.net/stories/m5-ultra-mac-studio-review-the-dream-mac-for-local-ai-agents/#mx-pc" rel="nofollow">https://www.macstories.net/stories/m5-ultra-mac-studio-revie...
1. nacs · 1790006878 · focus · HN ↗
  That's a dense model. Of course it will do worse.
  Now try running that Qwen 3.8 Next model on the 5090 and tell me what TPS you get (hint: it's near 0 since it doesnt fit the 32GB VRAM on 5090 vs the 256 in OPs M5).
  1. peri-cl · 1790007178 · focus · HN ↗
    Surprisingly, the Reddit crowd are reporting 50–60 tokens/s (for the 32 GiB 5090 + 128 GiB RAM)—on par with the M5 Ultra benchmarks, despite both the PCIe bottleneck and much smaller DDR5 bandwidth,
    <a href="https://old.reddit.com/r/LocalLLaMA/comments/1wl06np/qwen38flashnext_on_1x_rtx_5090_tg50_ts_pp2300_ts/" rel="nofollow">https://old.reddit.com/r/LocalLLaMA/comments/1wl06np/qwen38f...
    (Note it's a sparse MoE with only 6B active).
    bitexploder · 1790020429 · focus · HN ↗
    I have a 3 year old gaming system. RTX 4080 w/128GB of DDR5. It runs Qwen 38 Flash around 44-40 t/s with 128K context. It is on a specialized build that caches MoE experts and uses an optimized 3bit quant that basically is within a few points of the full 8 bit quant. In general, in casual benchmarking with Alibaba's endpoint I could not tell much of a difference. Overall this model is very good on long horizon agentic work. The main pain point for it is that its input processing speed is slow. Regardless, it gets meaningful work done.
    I paid $500 for the RAM in Nov 2023 :)
    peri-cl · 1790022667 · focus · HN ↗
    > "I paid $500 for the RAM in Nov 2023 :)"
    No wonder Warren Buffet gave up and resigned.
    bitexploder · 1790032252 · focus · HN ↗
    Right? To get comparable brand and quality DDR5, which isn’t particularly great at AI anything it is ~$2000. All you had to do was start hoarding 3090 GPU and RAM in 2023. It is unhinged.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.